[39] Chi Jin, Zhuoran Yang, Zhaoran Wang, and Michael I Jordan. Provably efficient reinforcement learning with linear function approximation. In Conference on Learning Theory, pages 2137-2143. PMLR, 2020. 24
[40] Sham Kakade and John Langford. Approximately optimal approximate reinforcement learning. In Proceedings of the Nineteenth International Conference on Machine Learning, pages 267-274, 2002. 26
[41] Sertac Karaman and Emilio Frazzoli. Sampling-based algorithms for optimal motion planning. The international journal of robotics research, 30(7):846-894, 2011. 8
[42] Ali Khanafer, Behrouz Touri, and Tamer Basar. Robust distributed averaging on networks with adversarial intervention. In 52nd IEEE Conference on Decision and Control, pages 7131-7136. IEEE, 2013. 8
[43] Karen Leung, Sushant Veer, Edward Schmerling, and Marco Pavone. Learning autonomous vehicle safety concepts from demonstrations. arXiv preprint arXiv:2210.02761, 2022. 2, 8
[44] J Lewin. The lion and man problem revisited. Journal of optimization theory and applications, 49:411-430, 1986. 8
[45] Tao Li and Suresh P Sethi. A review of dynamic stackelberg game models. Discrete and Continuous Dynamical Systems-B, 22(1):125, 2017. 16
[46] Michael L. Littman. Markov games as a framework for multi-agent reinforcement learning. In Proceedings of the Eleventh International Conference on International Conference on Machine Learning, ICML'94, page 157-163, San Francisco, CA, USA, 1994. Morgan Kaufmann Publishers Inc. ISBN 1558603352. 1
[47] Songtao Lu, Ioannis Tsaknakis, Mingyi Hong, and Yongxin Chen. Hybrid block successive approximation for one-sided non-convex min-max problems: Algorithms and applications. IEEE Transactions on Signal Processing, 68:3676–3691, 2020. doi: 10.1109/TSP.2020.2986363.4
[48] Kostas Margellos and John Lygeros. Hamilton-Jacobi Formulation for Reach-Avoid Differential Games. IEEE Transactions on Automatic Control, 56(8):1849-1861, August 2011. ISSN 1558-2523. doi: 10.1109/TAC.2011.2105730. 1
[49] Eric Maskin and Jean Tirole. Markov perfect equilibrium: I. observable actions. Journal of Economic Theory, 100(2):191-219, 2001. 2
[50] Francisco S Melo and M Isabel Ribeiro. Q-learning with linear function approximation. In International Conference on Computational Learning Theory, pages 308-322. Springer, 2007. 24
[51] Ashkan Mohammadi. Penalty methods to compute stationary solutions for constrained optimization problems. arXiv preprint arXiv:2206.04020, 2022. 19
[52] Ashkan Mohammadi, Boris S Mordukhovich, and M Ebrahim Sarabi. Variational analysis of composite models with applications to continuous optimization. Mathematics of Operations Research, 47(1):397-426, 2022. 19
[53] John F. Nash. Equilibrium points in  $\langle \mathrm{i} \rangle \mathrm{n} < \langle \mathrm{i} \rangle$  -person games. Proceedings of the National Academy of Sciences, 36(1):48-49, 1950. doi: 10.1073/pnas.36.1.48. URL https://www.pnas.org/doi/abs/10.1073/pnas.36.1.48.1
[54] Angela Nedic and Asuman Ozdaglar. Approximate primal solutions and rate analysis for dual subgradient methods. SIAM Journal on Optimization, 19(4):1757-1780, 2009. 19
[55] Angela Nedic and Asuman Ozdaglar. Subgradient methods for saddle-point problems. Journal of optimization theory and applications, 142(1):205-228, 2009. 19
[56] Arkadi Nemirovski, Anatoli Juditsky, Guanghui Lan, and Alexander Shapiro. Robust stochastic approximation approach to stochastic programming. SIAM Journal on optimization, 19(4): 1574-1609, 2009. 20
[57] Kazimierz Nikodem.  $K$ -convex and  $K$ -concave set-valued functions. Wydawnictwo Politechniki Lodzkiej, 1989. 3, 4
[58] Bernt Oksendal, Leif Sandal, and Jan Uboe. Stochastic stackelberg equilibria with applications to time-dependent newsvendor models. Journal of Economic Dynamics and Control, 37(7): 1284-1299, 2013. 16