[30] Miguel Calvo-Fullana, Santiago Paternain, Luiz FO Chamon, and Alejandro Ribeiro. State augmented constrained reinforcement learning: Overcoming the limitations of learning with rewards. IEEE Transactions on Automatic Control, 2023.
[31] Kenneth J. Arrow, Leonid Hurwicz, and Hirofumi Uzawa. Studies in linear and non-linear programming. Stanford University Press, 1958.
[32] Galina M Korpelevich. The extragradients method for finding saddle points and other problems. Matecon, 12:747-756, 1976.
[33] Angela Nedic and Asuman Ozdaglar. Subgradient methods for saddle-point problems. Journal of Optimization Theory and Applications, 142:205-228, 2009.
[34] Donghao Ying, Yuhao Ding, and Javad Lavaei. A dual approach to constrained Markov decision processes with entropy regularization. In Proceedings of the International Conference on Artificial Intelligence and Statistics, pages 1887-1909, 2022.
[35] Egor Gladin, Maksim Lavrik-Karmazin, Karina Zainullina, Varvara Rudenko, Alexander Gasnikov, and Martin Takac. Algorithm for constrained Markov decision process with linear convergence. In Proceedings of the International Conference on Artificial Intelligence and Statistics, pages 11506-11533. PMLR, 2023.
[36] Tianqi Zheng, Pengcheng You, and Enrique Mallada. Constrained reinforcement learning via dissipative saddle flow dynamics. arXiv preprint arXiv:2212.01505, 2022.
[37] Yu-Guan Hsieh, Franck Iutzeler, Jérôme Malick, and Panayotis Mertikopoulos. On the convergence of single-call stochastic extra-gradient methods. Proceedings of the Advances in Neural Information Processing Systems, 32, 2019.
[38] Chen-Yu Wei, Chung-Wei Lee, Mengxiao Zhang, and Haipeng Luo. Linear last-iterate convergence in constrained saddle-point optimization. In Proceedings of the International Conference on Learning Representations, 2020.
[39] Yang Cai, Argyris Oikonomou, and Weiqiang Zheng. Tight last-iterate convergence of the extragradient and the optimistic gradient descent-ascent algorithm for constrained monotone variational inequalities. arXiv preprint arXiv:2204.09228, 2022.
[40] Ted Moskovitz, Brendan O'Donoghue, Vivek Veeriah, Sebastian Flennerhag, Satinder Singh, and Tom Zahavy. ReLOAD: Reinforcement learning with optimistic ascent-descent for last-iterate convergence in constrained MDPs. arXiv preprint arXiv:2302.01275, 2023.
[41] Sharan Vaswani, Lin Yang, and Csaba Szepesvári. Near-optimal sample complexity bounds for constrained MDPs. In Proceedings of the Advances in Neural Information Processing Systems, volume 35, pages 3110-3122, 2022.
[42] Qinbo Bai, Amrit Singh Bedi, Mridul Agarwal, Alec Koppel, and Vaneet Aggarwal. Achieving zero constraint violation for constrained reinforcement learning via primal-dual approach. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 3682-3689, 2022.
[43] Chen-Yu Wei, Chung-Wei Lee, Mengxiao Zhang, and Haipeng Luo. Last-iterate convergence of decentralized optimistic gradient descent/ascient in infinite-horizon competitive Markov games. In Proceedings of the Conference on learning theory, pages 4259–4299, 2021.
[44] Zhuoqing Song, Jason D Lee, and Zhuoran Yang. Can we find Nash equilibria at a linear rate in Markov games? In Proceedings of the International Conference on Learning Representations, 2023.
[45] Dimitri P Bertsekas. Constrained optimization and Lagrange multiplier methods. Academic Press, 2014.