adam optimizer

Tags: ml

• https://www.fast.ai/2018/07/02/adam-weight-decay/
• Notably keeps track of momentum, the pace at which we’re doing gradient descent, and RMSP, which tried to improve adagrad.