What does the optimization algorithm name 'Adam' stand for?
Show answer
Adaptive Moment Estimation
Adam combines the advantages of AdaGrad (which works well with sparse gradients) and RMSProp (which works well in online settings). It computes adaptive learning rates for each parameter by storing an exponentially decaying average of past squared gradients and past gradients.