[2] S. Agrawal and N. Goyal. Analysis of Thompson Sampling for the Multi-armed Bandit Problem. In Proceedings of the Conference on Learning Theory (COLT), volume 23, pages 39.1–39.26, 2012.
[3] S. Agrawal and N. Goyal. Further optimal regret bounds for thompson sampling. In Artificial intelligence and statistics, pages 99-107, 2013.
[4] S. Agrawal and N. Goyal. Near-optimal regret bounds for thompson sampling. Journal of the ACM (JACM), 64(5):1-24, 2017.
[5] J.-Y. Audibert, S. Bubeck, and Others. Minimax Policies for Adversarial and Stochastic Bandits. In Proceedings of the Conference on Learning Theory (COLT), 2009.
[6] J.-Y. Audibert, S. Bubeck, et al. Minimax policies for adversarial and stochastic bandits. In  $COLT$ , volume 7, pages 1-122, 2009.
[7] J.-Y. Audibert, R. Munos, and C. Szepesvári. Exploration-exploitation tradeoff using variance estimates in multi-armed bandits. Theoretical Computer Science, 410(19):1876-1902, 2009.
[8] J.-Y. Audibert, R. Munos, and C. Szepesvári. Exploration-exploitation tradeoff using variance estimates in multi-armed bandits. Theoretical Computer Science, 410(19):1876–1902, 2009.
[9] P. Auer. Using Confidence Bounds for Exploitation-Exploration Trade-offs. Journal of Machine Learning Research, 3:397–422, 2002.
[10] P. Auer, N. Cesa-Bianchi, Y. Freund, and R. E. Schapire. The Nonstochastic Multiarmed Bandit Problem. SIAM J. Comput., 32(1):48-77, jan 2003. ISSN 0097-5397. doi: 10.1137/S0097539701398375.
[11] J. Bian and K.-S. Jun. Maillard Sampling: Boltzmann Exploration Done Optimally. In International Conference on Artificial Intelligence and Statistics (AISTATS), pages 54-72, 2022.
[12] A. N. Burnetas and M. N. Katehakis. Optimal adaptive policies for sequential allocation problems. Advances in Applied Mathematics, 17(2):122-142, 1996.
[13] O. Cappé, A. Garivier, O.-A. Maillard, R. Munos, and G. Stoltz. Kullback-leibler upper confidence bounds for optimal sequential allocation. The Annals of Statistics, pages 1516-1541, 2013.
[14] N. Cesa-Bianchi, C. Gentile, G. Lugosi, and G. Neu. Boltzmann exploration done right. Advances in Neural Information Processing Systems (NeurIPS), 2017.
[15] O. Chapelle and L. Li. An Empirical Evaluation of Thompson Sampling. In Advances in Neural Information Processing Systems (NIPS), pages 2249-2257, 2011.
[16] D. J. Foster, S. M. Kakade, J. Qian, and A. Rakhlin. The Statistical Complexity of Interactive Decision Making. CoRR, abs/2112.1, 2021.
[17] A. Garivier and O. Cappé. The kl-ucb algorithm for bounded stochastic bandits and beyond. In Proceedings of the 24th annual conference on learning theory, pages 359–376. JMLR Workshop and Conference Proceedings, 2011.
[18] A. Garivier, H. Hadiji, P. Menard, and G. Stoltz. Kl-ucb-switch: optimal regret bounds for stochastic bandits from both a distribution-dependent and a distribution-free viewpoints. The Journal of Machine Learning Research, 23(1):8049–8114, 2022.
[19] J. Honda and A. Takemura. An asymptotically optimal policy for finite support models in the multiarmed bandit problem. Machine Learning, 85(3), 2011.
[20] J. Honda and A. Takemura. Finite-time regret bound of a bandit algorithm for the semi-bounded support model. arXiv preprint arXiv:1202.2277, 2012.
[21] J. Honda and A. Takemura. Non-asymptotic analysis of a new bandit algorithm for semibounded rewards. J. Mach. Learn. Res., 16:3721-3756, 2015.