[23] Ronen I. Brafman and Moshe Tennenholtz. R-MAX - A General Polynomial Time Algorithm for Near-Optimal Reinforcement Learning. Journal of Machine Learning Research, 3(Oct):213-231, 2002. ISSN ISSN 1533-7928. URL https://www.jmlr.org/papers/v3/brafman02a.html.
[24] Michael Kearns, Yishay Mansour, and Andrew Y. Ng. A Sparse Sampling Algorithm for Near-Optimal Planning in Large Markov Decision Processes. Machine Learning, 49(2):193–208, November 2002. ISSN 1573-0565. doi: 10.1023/A:1017932429737. URL https://doi.org/10.1023/A:1017932429737.
[25] Nan Jiang, Satinder Singh, and Ambuj Tewari. On structural properties of MDPs that bound loss due to shallow planning. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI'16, pages 1640-1647, New York, New York, USA, July 2016. AAAI Press. ISBN 978-1-57735-770-4.
[26] Ian Osband, Yotam Doron, Matteo Hessel, John Aslanides, Eren Sezener, Andre Saraiva, Katrina McKinney, Tor Lattimore, Csaba Szepesvari, Satinder Singh, Benjamin Van Roy, Richard Sutton, David Silver, and Hado Van Hasselt. Behaviour Suite for Reinforcement Learning, February 2020. URL http://arxiv.org/abs/1908.03568. arXiv:1908.03568 [cs, stat].
[27] Raghu Rajan, Jessica Lizeth Borja Diaz, Suresh Guttikonda, Fabio Ferreira, André Biedenkapp, Jan Ole von Hartz, and Frank Hutter. MDP Playground: A Design and Debug Testbed for Reinforcement Learning, June 2021. URL http://arxiv.org/abs/1909.07750.arXiv:1909.07750 [cs, stat].
[28] R. Devon Hjelm, Bogdan Mazoure, Florian Golemo, Felipe Frujeri, Mihai Jalobeanu, and Andrey Kolobov. The Sandbox Environment for Generalizable Agent Research (SEGAR), March 2022. URL http://arxiv.org/abs/2203.10351.arXiv:2203.10351 [cs].
[29] Michelangelo Conserva and Paulo Rauber. Hardness in Markov Decision Processes: Theory and Practice, October 2022. URL http://arxiv.org/abs/2210.13075. arXiv:2210.13075 [cs].
[30] B. Abramson. Expected-outcome: a general model of static evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(2):182-193, February 1990. ISSN 1939-3539. doi: 10.1109/34.44404. URL https://ieeexplore.ieee.org/abstract/document/44404?casa_token=uj3nSMaKLJcAAAAA: d7LSY9mmrlNL8qhkLN6WEKgrZpmv1pRnvwj0cAolitRmNtbtUTFegNNstXuXCEwBnrPayNKBO. Conference Name: IEEE Transactions on Pattern Analysis and Machine Intelligence.
[31] Bernd Brügmann. Monte Carlo Go. 1993. URL https:// wwwsemanticscholar.org/paper/Monte-Carlo-Go-Max-Planck/ 5f8b18be86be69077e66377e03198d21d06833f3.
[32] Gerald Tesauro and Gregory Galperin. On-line Policy Improvement using Monte-Carlo Search. In Advances in Neural Information Processing Systems, volume 9. MIT Press, 1996. URL https://proceedings.neurips.cc/paper/1996/bit/996009f2374006606f4c0b0fda878af1-Abstract.html.
[33] Dimitri P. Bertsekas, John N. Tsitsiklis, and Cynara Wu. Rollout Algorithms for Combinatorial Optimization. Journal of Heuristics, 3(3):245-262, December 1997. ISSN 1572-9397. doi: 10.1023/A:1009635226865. URL https://doi.org/10.1023/A:1009635226865.
[34] Hyeong Soo Chang, Michael C. Fu, Jiaqiao Hu, and Steven I. Marcus. An Adaptive Sampling Algorithm for Solving Markov Decision Processes. Operations Research, 53(1):126-139, February 2005. ISSN 0030-364X. doi: 10.1287/opre.1040.0145. URL https://pubsonline.informs.org/doi/10.1287/opre.1040.0145. Publisher: INFORMS.
[35] Levente Kocsis and Csaba Szepesvári. Bandit Based Monte-Carlo Planning. In Johannes Furnkranz, Tobias Scheffer, and Myra Spiliopoulou, editors, Machine Learning: ECML 2006, Lecture Notes in Computer Science, pages 282-293, Berlin, Heidelberg, 2006. Springer. ISBN 978-3-540-46056-5. doi: 10.1007/11871842_29.