Shipra Agrawal and Navin Goyal. Further optimal regret bounds for thompson sampling. In Carlos M. Carvalho and Pradeep Ravikumar, editors, Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics, volume 31 of Proceedings of Machine Learning Research, pages 99-107, Scottsdale, Arizona, USA, 29 Apr-01 May 2013. PMLR. URL https://proceedings.mlr.press/v31/agrawal13a.html.
Shipra Agrawal and Randy Jia. Optimistic posterior sampling for reinforcement learning: worst-case regret bounds. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper/2017/file/3621f1454cacf995530ea53652ddb8fb-Paper.pdf.
Duncan Alfers and Hermann Dinges. A normal approximation for beta and gamma tail probabilities. Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete, 65:399-420, 1984. URL https://link.springer.com/content/pdf/10.1007/BF00533744.pdf.
Mohammad Gheshlaghi Azar, Ian Osband, and Rémi Munos. Minimax regret bounds for reinforcement learning. In International Conference on Machine Learning, 2017. URL https://arxiv.org/pdf/1703.05449.pdf.
Kamyar Azizzadenesheli, Emma Brunskill, and Animashree Anandkumar. Efficient exploration through bayesian deep q-networks. In 2018 Information Theory and Applications Workshop, ITA 2018, San Diego, CA, USA, February 11-16, 2018, pages 1-9. IEEE, 2018. doi: 10.1109/ITA.2018.8503252. URL https://doi.org/10.1109/ITA.2018.8503252.
Yu Bai, Tengyang Xie, Nan Jiang, and Yu-Xiang Wang. Provably efficient q-learning with low switching cost, 2019. URL https://arxiv.org/abs/1905.12849.
Tongyi Cao and Akshay Krishnamurthy. Provably adaptive reinforcement learning in metric spaces. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 9736-9744. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/file/6ef1173b096aa200158bfbc8af3ae8e3-Paper.pdf.
Christoph Dann, Tor Lattimore, and Emma Brunskill. Unifying PAC and regret: Uniform PAC bounds for episodic reinforcement learning. In *Neural Information Processing Systems*, 2017. URL https://arxiv.org/pdf/1703.07710.pdf.
Christoph Dann, Mehryar Mohri, Tong Zhang, and Julian Zimmert. A provably efficient model-free posterior sampling method for episodic reinforcement learning. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 12040-12051. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/file/649d45bf179296e31731adfd4df25588-Paper.pdf.
Marc Peter Deisenroth and Carl Edward Rasmussen. *Pilco: A model-based and data-efficient approach to policy search*. In *Proceedings of the 28th International Conference on International Conference on Machine Learning*, ICML '11, page 465-472, Madison, WI, USA, 2011. Omnipress. ISBN 9781450306195.
Persi Diaconis and David Freedman. Iterated random functions. SIAM Review, 41(1):45-76, 1999. doi: 10.1137/S0036144598338446. URL https://doi.org/10.1137/S0036144598338446.
Omar Darwiche Domingues, Yannis Flet-Berliac, Edouard Leurent, Pierre Ménard, Xuedong Shang, and Michal Valko. rlberry - A Reinforcement Learning Library for Research and Education, 10 2021a. URL https://github.com/rlberry-py/rlberry.
Omar Darwiche Domingues, Pierre Ménard, Emilie Kaufmann, and Michal Valko. Episodic reinforcement learning in finite mdps: Minimax lower bounds revisited. In Vitaly Feldman, Katrina Ligett, and Sivan Sabato, editors, Proceedings of the 32nd International Conference on Algorithmic Learning Theory, volume 132 of Proceedings of Machine Learning Research, pages 578-598. PMLR, 16-19 Mar 2021b. URL https://proceedings.mlr.press/v132/domingues21a.html.