[19] Elad Hazan and C. Seshadhri. Efficient learning algorithms for changing environments. In Andrea Pohoreckiyj Danyluk, Léon Bottou, and Michael L. Littman, editors, Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, Montreal, Quebec, Canada, June 14-18, 2009, volume 382 of ACM International Conference Proceeding Series, pages 393-400. ACM, 2009.
[20] Wei-Kang Hsu, Jiaming Xu, Xiaojun Lin, and Mark R Bell. Integrated online learning and adaptive control in queueing systems with uncertain payoffs. Operations Research, 70(2):1166-1181, 2022.
[21] Jiatai Huang, Leana Golubchik, and Longbo Huang. Queue scheduling with adversarial bandit learning. arXiv preprint arXiv:2303.01745, 2023.
[22] Ramesh Johari, Vijay Kamble, and Yash Kanoria. Matching while learning. Operations Research, 69(2):655-681, 2021.
[23] Subhashini Krishnasamy, Ari Arapostathis, Ramesh Johari, and Sanjay Shakkottai. On learning the cµ rule in single and parallel server networks. arXiv preprint arXiv:1802.06723, 2018.
[24] Subhashini Krishnasamy, Rajat Sen, Ramesh Johari, and Sanjay Shakkottai. Learning unknown service rates in queues: A multiarmed bandit approach. *Oper. Res.*, 69(1):315-330, 2021.
[25] TL Lai and Herbert Robbins. Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6(1):4-22, 1985.
[26] Qingkai Liang and Eytan Modiano. Minimizing queue length regret under adversarial network models. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 2(1):1-32, 2018.
[27] John DC Little. A proof for the queuing formula:  $\mathrm{L} = \lambda$  w. Operations research, 9(3):383-387, 1961.
[28] Rahul Makhijani, Parikshit Shah, Vashist Avadhanula, Caner Gocmen, Nicolas E. Stier Moses, and Julián Mestre. QUEST: queue simulation for content moderation at scale. CoRR, abs/2103.16816, 2021.
[29] Laurent Massoulie and Kuang Xu. On the capacity of information processing systems. Operations Research, 66(2):568-586, 2018.
[30] Michael J. Neely, Scott Rager, and Thomas F. La Porta. Max weight learning algorithms for scheduling in unknown environments. IEEE Trans. Autom. Control., 57(5):1179-1191, 2012.
[31] Quang Minh Nguyen and Eytan Modiano. Learning to schedule in non-stationary wireless networks with unknown statistics. arXiv preprint arXiv:2308.02734, 2023.
[32] Daniel Russo and Benjamin Van Roy. Satisficing in time-sensitive bandit learning. Mathematics of Operations Research, 47(4):2815-2839, 2022.
[33] Flore Sentenac, Etienne Boursier, and Vianney Perchet. Decentralized learning in online queuing systems. pages 18501-18512, 2021.
[34] Virag Shah, Lennart Gulikers, Laurent Massoulie, and Milan Vojnovic. Adaptive matching for expert systems with uncertain task types. Oper. Res., 68(5):1403-1424, 2020.
[35] Simrita Singh, Itai Gurvich, and Jan A Van Mieghem. Feature-based priority queuing. Available at SSRN 3731865, 2022.
[36] R. Srikant and Lei Ying. Communication networks: an optimization, control, and stochastic networks perspective. Cambridge University Press, 2014.
[37] Thomas Stahlbuhk, Brooke Shrader, and Eytan H. Modiano. Learning algorithms for scheduling in wireless networks with unknown channel statistics. Ad Hoc Networks, 85:131-144, 2019.
[38] Thomas Stahlbuhk, Brooke Shrader, and Eytan H. Modiano. Learning algorithms for minimizing queue length regret. IEEE Trans. Inf. Theory, 67(3):1759-1781, 2021.