R. Y. Chitashvili. A controlled finite markov chain with an arbitrary set of decisions. Theory of Probability & Its Applications, 20(4):839-847, 1976.
E. Delage and S. Mannor. Percentile optimization for Markov decision processes with parameter uncertainty. Operations research, 58(1):203-213, 2010.
Y. Deng, F. Bao, Y. Kong, Z. Ren, and Q. Dai. Deep direct reinforcement learning for financial signal representation and trading. IEEE transactions on neural networks and learning systems, 28(3): 653-664, 2016.
V. Dewanto and M. Gallagher. Examining average and discounted reward optimality criteria in reinforcement learning. arXiv preprint arXiv:2107.01348, 2021.
V. Dewanto, G. Dunn, A. Eshragh, M. Gallagher, and F. Roosta. Average-reward model-free reinforcement learning: a systematic review and literature mapping. arXiv preprint arXiv:2010.08920, 2020.
K. Dong, Y. Wang, X. Chen, and L. Wang. Q-learning with UCB exploration is sample efficient for infinite-horizon MDP. arXiv preprint arXiv:1901.09311, 2019.
E. A. Feinberg and A. Shwartz. Handbook of Markov decision processes: methods and applications, volume 40. Springer Science & Business Media, 2012.
O. Friedmann. An exponential lower bound for the latest deterministic strategy iteration algorithms. Logical Methods in Computer Science, 7, 2011.
G. Garcia, L. Steimle, W. Marrero, and J. B. Sussman. Interpretable policies and the price of interpretability in hypertension treatment planning. *Optimization Online*, pages 1-46, 2021.
R. Givan, S. Leach, and T. Dean. Bounded parameter Markov decision processes. In European Conference on Planning, pages 234-246. Springer, 1997.
J. Goh, M. Bayati, S. A. Zenios, S. Singh, and D. Moore. Data uncertainty in Markov chains: Application to cost-effectiveness analyses of medical innovations. Operations Research, 66(3): 697-715, 2018.
O. Gottesman, F. Johansson, M. Komorowski, A. Faisal, D. Sontag, F. Doshi-Velez, and L. A. Celi. Guidelines for reinforcement learning in healthcare. Nature medicine, 25(1):16-18, 2019.
V. Goyal and J. Grand-Clément. A first-order approach to accelerated value iteration. Operations Research, 71(2):517-535, 2023a.
V. Goyal and J. Grand-Clément. Robust Markov decision processes: Beyond rectangularity. Mathematics of Operations Research, 48(1):203-226, 2023b.
J. Grand-Clément and M. Petrik. On the convex formulations of robust Markov decision processes. arXiv preprint arXiv:2209.10187, 2022.
J. Grand-Clément, C. W. Chan, V. Goyal, and G. Escobar. Robustness of proactive intensive care unit transfer policies. Operations Research, 71(5):1653-1688, 2022.
C. P. Ho, M. Petrik, and W. Wiesemann. Fast Bellman updates for robust MDPs. In International Conference on Machine Learning, pages 1979-1988. PMLR, 2018.
C. P. Ho, M. Petrik, and W. Wiesemann. Partial policy iteration for  $l_{1}$ -robust Markov decision processes. Journal of Machine Learning Research, 22(275):1-46, 2021.
A. Hordijk, R. Dekker, and L. C. M. Kallenberg. Sensitivity-analysis in discounted Markovian decision problems. Operations-Research-Spektrum, 7(3):143-151, 1985.
R. A. Horn and C. R. Johnson. Matrix analysis. Cambridge university press, 2012.
W. Hu, C. W. Chan, J. R. Zubizarreta, and G. J. Escobar. An examination of early transfers to the icu based on a physiologic risk score. Manufacturing & Service Operations Management, 20(3): 531-549, 2018.