[39] Steven A Lippman. Applying a new device in the optimization of exponential queuing systems. Operations Research, 23(4):687-710, 1975.
[40] Ashok P. Maitra. Dynamic programming for countable state systems. PhD thesis, University of California, Berkeley, 1963.
[41] Armand M Makowski and Adam Shwartz. The Poisson equation for countable Markov chains: Probabilistic methods and interpretations. Handbook of Markov Decision Processes: Methods and Applications, pages 269-303, 2002.
[42] Akshay Mete, Rahul Singh, Xi Liu, and P R Kumar. Reward biased maximum likelihood estimation for reinforcement learning. In Learning for Dynamics and Control, pages 815-827. PMLR, 2021.
[43] Sean P Meyn and Richard L Tweedie. Markov chains and stochastic stability. Springer Science & Business Media, 2012.
[44] Michael J Neely, Scott T Rager, and Thomas F La Porta. Max-Weight learning algorithms for scheduling in unknown environments. IEEE Transactions on Automatic Control, 57(5):1179-1191, 2012.
[45] Pedro A Ortega and Daniel A Braun. A minimum relative entropy principle for learning and acting. Journal of Artificial Intelligence Research, 38:475-511, 2010.
[46] Ian Osband, Daniel Russo, and Benjamin Van Roy. (More) Efficient reinforcement learning via posterior sampling. Advances in Neural Information Processing Systems, 26, 2013.
[47] Ian Osband and Benjamin Van Roy. Why is posterior sampling better than optimism for reinforcement learning? In International Conference on Machine Learning, pages 2701-2710. PMLR, 2017.
[48] Reda Ouhamma, Debabrota Basu, and Odalric Maillard. Bilinear exponential family of MDPs: Frequentist regret bound with tractable exploration and planning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 9336-9344, 2023.
[49] Yi Ouyang, Mukul Gagrani, Ashutosh Nayyar, and Rahul Jain. Learning unknown Markov decision processes: A Thompson sampling approach. Advances in Neural Information Processing Systems, 30, 2017.
[50] Daniel J Russo, Benjamin Van Roy, Abbas Kazerouni, Ian Osband, and Zheng Wen. A tutorial on Thompson sampling. Foundations and Trends® in Machine Learning, 11(1):1-96, 2018.
[51] Devavrat Shah, Qiaomin Xie, and Zhi Xu. Stable reinforcement learning with unbounded state space. arXiv preprint arXiv:2006.04353, 2020.
[52] R. Srikant and Lei Ying. Communication networks: An optimization, control, and stochastic networks perspective. Cambridge University Press, 2013.
[53] Thomas Stahlbuhk, Brooke Shrader, and Eytan Modiano. Learning algorithms for minimizing queue length regret. IEEE Transactions on Information Theory, 67(3):1759-1781, 2021.
[54] Shaler Stidham and Richard Weber. A survey of Markov decision models for control of networks of queues. Queueing Systems, 13:291-314, 1993.
[55] Malcolm Strens. A Bayesian framework for reinforcement learning. In ICML, volume 2000, pages 943-950, 2000.
[56] Wojciech Szpankowski and Vernon Rego. Yet another application of a binomial recurrence order statistics. Computing, 43(4):401-410, 1990.
[57] Dengwang Tang, Rahul Jain, Botao Hao, and Zheng Wen. Efficient online learning with offline datasets for infinite horizon MDPs: A Bayesian approach. arXiv preprint arXiv:2310.11531, 2023.