確率的勾配降下法
↑ Ferguson, Thomas S. (1982). “An inconsistent maximum likelihood estimate”. Journal of the American Statistical Association 77 (380): 831–834. doi:10.1080/01621459.1982.10477894. JSTOR 2287314.
↑ Bottou, Léon; Bousquet, Olivier (2008). The Tradeoffs of Large Scale Learning. Advances in Neural Information Processing Systems. Vol. 20. pp. 161–168.
↑ Bottou, Léon (1998). “Online Algorithms and Stochastic Approximations”. Online Learning and Neural Networks. Cambridge University Press. ISBN 978-0-521-65263-6. http://leon.bottou.org/papers/bottou-98x
↑ Kiwiel, Krzysztof C. (2001年). “Convergence and efficiency of subgradient methods for quasiconvex minimization”. Mathematical Programming (Series A) (Berlin, Heidelberg: Springer) 90 (1): pp.
1983年に Yurii Nesterov が発表。
x 0 = w 0 x t = w t − 1 − η ∇ Q i ( w t − 1 ) w t = x t + t − 1 t + 2 ( x t − x t − 1 ) {\displaystyle {\begin{aligned}x_{0}&=w_{0}\\x_{t}&=w_{t-1}-\eta \nabla Q_{i}(w_{t-1})\\w_{t}&=x_{t}+{\frac {t-1}{t+2}}(x_{t}-x_{t-1})\end{aligned}}}