John C. Duchi, Elad Hazan, and Yoram Singer (2011). "Adaptive Subgradient Methods for Online Learning and Stochastic Optimization". In: Journal of Machine Learning Research 12, pp. 2121-2159.
Gary William Flake and Steve Lawrence (2002). "Efficient SVM Regression Training with SMO". In: Mach. Learn. 46.1-3, pp. 271-290.
Michel X. Goemans, Nicholas J. A. Harvey, Satoru Iwata, and Vahab Mirrokni (2009). "Approximating submodular functions everywhere". In: ACM-SIAM SODA 2009, pp. 535-544.
Luigi Grippo, Francesco Lampariello, and Stephano Lucidi (1986). "A Nonmonotone Line Search Technique for Newton's Method". In: SIAM Journal on Numerical Analysis 23.4, pp. 707-716.
Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky (2012). Neural Networks for Machine Learning. lecture 6.
Robert A. Jacobs (1988). "Increased rates of convergence through learning rate adaptation". In: Neural Networks 1.4, pp. 295-307.
Majid Jahani, Sergey Rusakov, Zheng Shi, Peter Richtárik, Michael W. Mahoney, and Martin Takac (2022). "Doubly Adaptive Scaled Algorithm for Machine Learning Using Second-Order Information". In: ICLR.
Arun Jambulapati, Jerry Li, Christopher Musco, Aaron Sidford, and Kevin Tian (2020). Fast and Near-Optimal Diagonal Preconditioning. arXiv/2008.01722.
Hamed Karimi, Julie Nutini, and Mark Schmidt (2016). "Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Condition". In: ECML, pp. 795-811.
S. Sathiya Keerthi and Dennis DeCoste (2005). "A Modified Finite Newton Method for Fast Solution of Large Scale Linear SVMs". In: Journal of Machine Learning Research 6.12, pp. 341-361.
R. Kelley Pace and Ronald Barry (1997). "Sparse spatial autoregressions". In: Statistics & Probability Letters 33.3, pp. 291-297.
Harry Kesten (1958). "Accelerated Stochastic Approximation". In: The Annals of Mathematical Statistics 29.1, pp. 41-59.
Diederik P. Kingma and Jimmy Ba (2015). "Adam: A Method for Stochastic Optimization". In: ICLR.
Etienne de Klerk, François Glineur, and Adrien B. Taylor (2017). "On the worst-case complexity of the gradient method with exact line search for smooth strongly convex functions". In: Optimization Letters 11.7, pp. 1185-1199.
Yann LeCun, Léon Bottou, Genevieve B. Orr, and Klaus-Robert Müller (2012). "Efficient BackProp". In: Neural Networks: Tricks of the Trade - Second Edition. Vol. 7700, pp. 9-48.
David D. Lewis, Yiming Yang, Tony G. Rose, and Fan Li (2004). "RCV1: A New Benchmark Collection for Text Categorization Research". In: Journal of Machine Learning Research 5, pp. 361-397.
Xiaoyu Li and Francesco Orabona (2019). "On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizes". In: AISTATS 2019, pp. 983-992.
Dong C. Liu and Jorge Nocedal (1989). "On the limited memory BFGS method for large scale optimization". In: Mathematical programming 45.1-3, pp. 503-528.
S. Łojasiewicz (1963). "Une propriété topologique des sous-ensembles analytiques réels". In: Les Équations aux Dérivées Partielles, pp. 87-89.
Dougal Maclaurin, David Duvenaud, and Ryan P. Adams (2015). "Gradient-based Hyperparameter Optimization through Reversible Learning". In: ICML, pp. 2113-2122.
Ashique Rupam Mahmood, Richard S. Sutton, Thomas Degris, and Patrick M. Pilarski (2012). "Tuning-free step-size adaptation". In: ICASSP, pp. 2121-2124.
Pierre-Yves Masse and Yann Ollivier (2015). Speed learning on the fly. arXiv/1511.02540.
H. Brendan McMahan and Matthew J. Streeter (2010). "Adaptive Bound Optimization for Online Convex Optimization". In:  $COLT$ , pp. 244-256.