[68] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. Human-level control through deep reinforcement learning. Nature, 2015. (Cited on pages 7 and 9.)
[69] Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International Conference on Machine Learning, 2016. (Cited on page 1.)
[70] Arkadi Nemirovski and David B. Yudin. Problem Complexity and Method Efficiency in Optimization. Wiley Interscience, 1983. (Cited on pages 1, 3, and 21.)
[71] Yurii E. Nesterov and Boris T. Polyak. Cubic regularization of Newton method and its global performance. Mathematical Programming, 2006. (Cited on page 23.)
[72] Gergely Neu, Anders Jonsson, and Vicenç Gomez. A unified view of entropy-regularized markov decision processes. arXiv preprint arXiv:1705.07798, 2017. (Cited on page 22.)
[73] Lam M. Nguyen, Jie Liu, Katya Scheinberg, and Martin Takáč. SARAH: A novel method for machine learning problems using stochastic recursive gradient. In International Conference on Machine Learning, 2017. (Cited on pages 22 and 23.)
[74] Francesco Orabona. A modern introduction to online learning, 2020. URL https://open.bu.edu/handle/2144/40900. (Cited on pages 6, 10, 30, and 31.)
[75] Jan Peters and Stefan Schaal. Natural actor-critic. Neurocomputing, 2008. (Cited on pages 1 and 22.)
[76] Boris T. Polyak. Gradient methods for the minimisation of functionals. USSR Computational Mathematics and Mathematical Physics, 1963. (Cited on page 22.)
[77] Bruno Scherrer. Approximate policy iteration schemes: A comparison. In International Conference on Machine Learning, 2014. (Cited on page 39.)
[78] John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In International Conference on Machine Learning, 2015. (Cited on pages 1, 19, and 22.)
[79] John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation. In International Conference on Learning Representations, 2016. (Cited on pages 7 and 10.)
[80] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. (Cited on pages 1, 19, and 22.)
[81] Shai Shalev-Shwartz, Shaked Shammah, and Amnon Shashua. Safe, multi-agent, reinforcement learning for autonomous driving. arXiv preprint arXiv:1610.03295, 2016. (Cited on page 1.)
[82] Lior Shani, Yonathan Efroni, and Shie Mannor. Adaptive trust region policy optimization: Global convergence and faster rates for regularized MDPs. In AAAI Conference on Artificial Intelligence, 2020. (Cited on pages 1, 6, 8, 21, 22, and 24.)
[83] David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin Riedmiller. Deterministic policy gradient algorithms. In International Conference on Machine Learning, 2014. (Cited on page 31.)
[84] David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, and Demis Hassabis. Mastering the game of Go without human knowledge. Nature, 2017. (Cited on page 9.)