[46] Csaba Szepesvári. Constrained MDPs and the reward hypothesis. http://readingsml.blogspot.com/2020/03/constrained-mdps-and-reward-hypothesis.html, 2020. Access on January 21, 2023.
[47] Tom Zahavy, Brendan O'Donoghue, Guillaume Desjardins, and Satinder Singh. Reward is enough for convex MDPs. Proceedings of the Advances in Neural Information Processing Systems, 34:25746-25759, 2021.
[48] Dongsheng Ding, Kaiqing Zhang, Jiali Duan, Tamer Bašar, and Mihailo R Jovanović. Convergence and sample complexity of natural policy gradient primal-dual methods for constrained MDPs. arXiv preprint arXiv:2206.02346, 2022.
[49] Alekh Agarwal, Sham M Kakade, Jason D Lee, and Gaurav Mahajan. On the theory of policy gradient methods: Optimality, approximation, and distribution shift. The Journal of Machine Learning Research, 22(1):4431-4506, 2021.
[50] Constantinos Daskalakis, Stratis Skoulakis, and Manolis Zampetakis. The complexity of constrained min-max optimization. In Proceedings of the Annual ACM SIGACT Symposium on Theory of Computing, pages 1466-1478, 2021.
[51] Yujia Jin and Aaron Sidford. Efficiently solving MDPs with stochastic mirror descent. In Proceedings of the International Conference on Machine Learning, pages 4890-4900, 2020.
[52] Shicong Cen, Chen Cheng, Yuxin Chen, Yuting Wei, and Yuejie Chi. Fast global convergence of natural policy gradient methods with entropy regularization. Operations Research, 70(4):2563-2578, 2022.
[53] Sébastien Bubeck. Convex optimization: Algorithms and complexity. Foundations and Trends® in Machine Learning, 8(3-4):231-357, 2015.
[54] Dongsheng Ding and Mihailo R Jovanović. Policy gradient primal-dual mirror descent for constrained MDPs with large state spaces. In Proceedings of the IEEE 61st Conference on Decision and Control, pages 4892-4897, 2022.
[55] Tao Liu, Ruida Zhou, Dileep Kalathil, Panganamala Kumar, and Chao Tian. Learning policies with zero or bounded constraint violation for constrained MDPs. Proceedings of the Advances in Neural Information Processing Systems, 34:17183-17193, 2021.
[56] Honghao Wei, Xin Liu, and Lei Ying. Triple-Q: A model-free algorithm for constrained reinforcement learning with sublinear regret and zero constraint violation. In Proceedings of the International Conference on Artificial Intelligence and Statistics, pages 3274-3307, 2022.
[57] Shicong Cen, Yuting Wei, and Yuejie Chi. Fast policy extragradient methods for competitive games with entropy regularization. Proceedings of the Advances in Neural Information Processing Systems, 34:27952-27964, 2021.
[58] Jalaj Bhandari and Daniel Russo. On the linear convergence of policy gradient methods for finite mdps. In Proceedings of the International Conference on Artificial Intelligence and Statistics, pages 2386-2394, 2021.
[59] Rui Yuan, Simon Shaolei Du, Robert M Gower, Alessandro Lazaric, and Lin Xiao. Linear convergence of natural policy gradient methods with log-linear policies. In Proceedings of the International Conference on Learning Representations, 2022.
[60] Lin Yang and Mengdi Wang. Sample-optimal parametric Q-learning using linearly additive features. In Proceedings of the International Conference on Machine Learning, pages 6995-7004, 2019.