[19] Vineet Goyal and Julien Grand-Clement. Robust Markov decision process: Beyond rectangularity. Mathematics of Operations Research, 48(1):203-226, 2023.
[20] Julien Grand-Clement and Christian Kroer. First-order methods for Wasserstein distributionally robust MDP. In International Conference on Machine Learning (ICML), pages 2010-2019, 2021.
[21] Vishal Gupta. Near-optimal Bayesian ambiguity sets for distributionally robust optimization. Management Science, 56(9), 2019.
[22] T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. In Proceedings of the 35th International Conference on Machine Learning (ICML), pages 1861-1870, 2018.
[23] C. A. R. Hoare. Algorithm 65: Find. Communications of the ACM, 4(7):321-322, 1961.
[24] Garud N. Iyengar. Robust dynamic programming. Mathematics of Operations Research, 30(2): 257-280, 2005.
[25] Marc Kery and Michael Schaub. Bayesian Population Analysis Using WinBUGS. Elsevier, 2012.
[26] Sasche Lange, Thomas Gabel, and Martin Riedmiller. Batch reinforcement learning. Springer, 2012.
[27] Elita A. Lobo, Mohammad Ghavamzadeh, and Marek Petrik. Soft-robust algorithms for batch reinforcement learning, 2021.
[28] Daniel J. Mankowitz, Nir Levine, Rae Ung Jeong, Abbas Abdelmaleki, Jost Tobias Springenberg, Timothy Mann, Todd Hester, and Martin A. Riedmiller. Robust reinforcement learning for continuous control with model misspecification. In Proceedings of the 2020 International Conference on Learning Representations (ICLR), 2020.
[29] Shie Mannor, Ofir Mebel, and Huan Xu. Lightning does not strike twice: Robust MDPs with coupled uncertainty. In Proceedings of the 29th International Conference on Machine Learning (ICML), pages 451-458, 2012.
[30] Shie Mannor, Ofir Mebel, and Huan Xu. Robust MDPs with K-rectangular uncertainty. Mathematics of Operations Research, 41(4):1484–1509, 2016.
[31] P. Massart. The tight constant in the Dvoretzky-Kiefer-Wolffowitz inequality. The Annals of Probability, 18:1269 - 1283, 1990.
[32] Ariel Neufeld and Julian Sester. Robust  $q$ -learning algorithm for Markov decision processes under Wasserstein uncertainty, 2023.
[33] Arnab Nilim and Laurent El Ghaoui. Robust control of Markov decision processes with uncertain transition matrices. Operations Research, 53(5):780-798, 2005.
[34] Marek Petrik and Ronny Luss. Interpretable Policies for Dynamic Product Recommendations. In Uncertainty in Artificial Intelligence (UAI), 2016.
[35] Marek Petrik, Mohammad Ghavamzadeh, and Yinlam Chow. Safe policy improvement by minimizing robust baseline regret. In International Conference on Neural Information Processing Systems, pages 2306-2314, 2016.
[36] L. A. Prashanth. Policy gradients for CVaR-constrained mdps. In Peter Auer, Alexander Clark, Thomas Zeugmann, and Sandra Zilles, editors, Algorithmic Learning Theory, pages 155-169, Cham, 2014. Springer International Publishing. ISBN 978-3-319-11662-4.
[37] Martin L. Puterman. Markov Decision Processes: Discrete stochastic dynamic programming. John Wiley and Sons, Inc., 2nd edition, 2005.
[38] R. Tyrrell Rockafellar. Convex analysis. Princeton University Press, 1970.