[38] Harold Kushner and G George Yin. Stochastic approximation and recursive algorithms and applications, volume 35. Springer Science & Business Media, 2003.
[39] Jeongyeol Kwon, Yonathan Efroni, Constantine Caramanis, and Shie Mannor. RL for latent MDPs: Regret guarantees and a lower bound. Advances in Neural Information Processing Systems, 34:24523-24534, 2021.
[40] D. Lee and W. B. Powell. Bias-corrected Q-learning with multistate extension. IEEE Transactions on Automatic Control, 2019.
[41] Keqin Liu and Qing Zhao. Indexability of restless bandit problems and optimality of Whittle index for dynamic multichannel access. IEEE Transactions on Information Theory, 56(11): 5547-5567, 2010.
[42] Rahul Meshram, D Manjunath, and Aditya Gopalan. On the Whittle index for restless multiarmed hidden Markov bandits. IEEE Transactions on Automatic Control, 63(9):3046-3053, 2018.
[43] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013.
[44] Aditya Mohan, Amy Zhang, and Marius Lindauer. Structure in reinforcement learning: A survey and open problems. arXiv preprint arXiv:2306.16021, 2023.
[45] Selvaprabu Nadarajah and Andre Augusto Cire. Self-adapting network relaxations for weakly coupled Markov decision processes. Available at SSRN, 2021.
[46] Khaled Nakhleh, Santosh Ganji, Ping-Chun Hsieh, I Hou, Srinivas Shakkottai, et al. Neurwin: Neural Whittle index network for restless bandits via deep RL. Advances in Neural Information Processing Systems, 34:828-839, 2021.
[47] Ian Osband and Benjamin Van Roy. Near-optimal reinforcement learning in factored MDPs. Advances in Neural Information Processing Systems, 27, 2014.
[48] Ian Osband, Charles Blundell, Alexander Pritzel, and Benjamin Van Roy. Deep exploration via bootstrapped DQN. Advances in Neural Information Processing Systems, 29, 2016.
[49] Warren Powell, Andrzej Ruszczyński, and Huseyin Topaloglu. Learning algorithms for separable approximations of discrete stochastic optimization problems. Mathematics of Operations Research, 29(4):814-836, 2004.
[50] Francisco Robledo, Vivek Borkar, Urtzi Ayesta, and Konstantin Avrachenkov. QWI: Q-learning with whittle index. ACM SIGMETRICS Performance Evaluation Review, 49(2):47-50, 2022.
[51] Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. Prioritized experience replay. arXiv preprint arXiv:1511.05952, 2015.
[52] Lauren N Steimle, David L Kaufman, and Brian T Denton. Multi-model Markov decision processes. IISE Transactions, 53(10):1124-1139, 2021.
[53] Kalyan Talluri and Garrett Van Ryzin. An analysis of bid-price controls for network revenue management. Management Science, 44(11-part-1):1577–1593, 1998.
[54] Huseyin Topaloglu. Using Lagrangian relaxation to compute capacity-dependent bid prices in network revenue management. Operations Research, 57(3):637-649, 2009.
[55] J. N. Tsitsiklis. Asynchronous stochastic approximation and Q-learning. Machine Learning, 16 (3):185-202, 1994.
[56] Tom Van de Wiele, David Warde-Farley, Andriy Mnih, and Volodymyr Mnih. Q-learning in enormous action spaces via amortized approximate maximization. arXiv preprint arXiv:2001.08116, 2020.
[57] Hado Van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning with double Q-learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, 2016.