AMSGrad

AMSGrad

2019/12/02

-----

// Some State of the Art Optimizers in Neural Networks - By Aakash Bindal

-----



# AMSGrad

-----

# AMSGrad

-----

# AMSGrad

-----

References

# AMSGrad

Reddi, Sashank J., Satyen Kale, and Sanjiv Kumar. "On the convergence of adam and beyond." International Conference on Learning Representations. 2018.

http://www.satyenkale.com/papers/amsgrad.pdf 

-----

Some State of the Art Optimizers in Neural Networks - By Aakash Bindal
https://hackernoon.com/some-state-of-the-art-optimizers-in-neural-networks-a3c2ba5a5643

-----

深度学习中常见的优化方法（from SGD to AMSGRAD）和正则化技巧 - 知乎
https://zhuanlan.zhihu.com/p/36327151

最优化方法系列：Adam+SGD-AMSGrad - wishchinYang的专栏 - CSDN博客
https://blog.csdn.net/wishchin/article/details/80567558