[39] T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, "Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor," in International conference on machine learning. PMLR, 2018, pp. 1861-1870.
[40] S. Cen, C. Cheng, Y. Chen, Y. Wei, and Y. Chi, "Fast global convergence of natural policy gradient methods with entropy regularization," Operations Research, 2021.
[41] M. Chen, Y. Wang, T. Liu, Z. Yang, X. Li, Z. Wang, and T. Zhao, “On computation and generalization of generative adversarial imitation learning,” in International Conference on Learning Representations, 2019.
[42] L. Shani, T. Zahavy, and S. Mannor, "Online apprenticeship learning," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 8, 2022, pp. 8240-8248.
[43] B. Zhu, J. Jiao, and M. I. Jordan, "Principled reinforcement learning with human feedback from pairwise or  $k$ -wise comparisons," arXiv preprint arXiv:2301.11270, 2023.
[44] Y. Liu, A. Swaminathan, A. Agarwal, and E. Brunskill, “Provably good batch off-policy reinforcement learning without great exploration,” Advances in neural information processing systems, vol. 33, pp. 1264–1274, 2020.
[45] M. Uehara and W. Sun, “Pessimistic model-based offline reinforcement learning under partial coverage,” in International Conference on Learning Representations, 2021.
[46] A. Ozdaglar, S. Pattathil, J. Zhang, and K. Zhang, “Revisiting the linear-programming framework for offline rl with general function approximation,” arXiv preprint arXiv:2212.13861, 2022.
[47] M. Rigter, B. Lacerda, and N. Hawes, “Rambo-rl: Robust adversarial model-based offline reinforcement learning,” arXiv preprint arXiv:2204.12581, 2022.
[48] J. Bhandari, D. Russo, and R. Singal, “A finite time analysis of temporal difference learning with linear function approximation,” in Conference on learning theory. PMLR, 2018, pp. 1691–1692.
[49] Y. F. Wu, W. Zhang, P. Xu, and Q. Gu, “A finite-time analysis of two time-scale actor-critic methods,” Advances in Neural Information Processing Systems, vol. 33, pp. 17617-17628, 2020.
[50] M. Hong, H.-T. Wai, Z. Wang, and Z. Yang, “A two-timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and application to actor-critic,” SIAM Journal on Optimization, vol. 33, no. 1, pp. 147–180, 2023.
[51] C. Jin, P. Netrapalli, and M. Jordan, “What is local optimality in nonconvex-nonconcave minimax optimization?” in International conference on machine learning. PMLR, 2020, pp. 4880–4889.
[52] Z. Guan, T. Xu, and Y. Liang, "When will generative adversarial imitation learning algorithms attain global convergence," in International Conference on Artificial Intelligence and Statistics. PMLR, 2021, pp. 1117-1125.
[53] E. Todorov, T. Erez, and Y. Tassa, "Mujoco: A physics engine for model-based control," in 2012 IEEE/RSJ international conference on intelligent robots and systems. IEEE, 2012, pp. 5026-5033.
[54] J. Fu, A. Kumar, O. Nachum, G. Tucker, and S. Levine, “D4rl: Datasets for deep data-driven reinforcement learning,” arXiv preprint arXiv:2004.07219, 2020.
[55] H. Cao, S. Cohen, and L. Szpruch, "Identifiability in inverse reinforcement learning," Advances in Neural Information Processing Systems, vol. 34, pp. 12362-12373, 2021.
[56] P. Rolland, L. Viano, N. Schürhoff, B. Nikolov, and V. Cevher, "Identifiability and generalizability from multiple experts in inverse reinforcement learning," arXiv preprint arXiv:2209.10974, 2022.
[57] C. Finn, S. Levine, and P. Abbeel, “Guided cost learning: Deep inverse optimal control via policy optimization,” in International conference on machine learning. PMLR, 2016, pp. 49–58.
[58] S. Liu and M. Zhu, "Distributed inverse constrained reinforcement learning for multi-agent systems," Advances in Neural Information Processing Systems, vol. 35, pp. 33444-33456, 2022.