SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this paper, we proposed the first efficient batched algorithm for contextual linear bandits with large action spaces. Our algorithm achieves a high-probability regret upper bound of  $\tilde{O} (\sqrt{T})$ , uses  $O(\log \log T)$  batches, and has a computational complexity that is linear in  $T$  and polynomial in  $d$ .
This work was partially supported by the NSF grants #2007714 and #2221871, the DARPA grant #HR00112190130, and by the Army Research Laboratory under the Cooperative Agreement W911NF-17-2-0196.
[1] Y. Abbasi-Yadtori, D. Pál, and C. Szepesvári. Improved algorithms for linear stochastic bandits. Advances in neural information processing systems, 24, 2011.
[2] N. Abe and P. M. Long. Associative reinforcement learning using linear probabilistic concepts. In ICML, pages 3-11. CiteSeer, 1999.
[3] A. Agarwal, S. Agarwal, S. Assadi, and S. Khanna. Learning with limited rounds of adaptivity: Coin tossing, multi-armed bandits, and ranking from pairwise comparisons. In Conference on Learning Theory, pages 39–75. PMLR, 2017.
[4] A. Agarwal, S. Bird, M. Cozowicz, L. Hoang, J. Langford, S. Lee, J. Li, D. Melamed, G. Oshri, O. Ribas, et al. Making contextual decisions with low technical debt. arXiv preprint arXiv:1606.03966, 2016.
[5] D. Agarwal, B.-C. Chen, P. Elango, N. Motgi, S.-T. Park, R. Ramakrishnan, S. Roy, and J. Zachariah. Online models for content optimization. Advances in Neural Information Processing Systems, 21, 2008.
[6] S. Agrawal and N. Goyal. Thompson sampling for contextual bandits with linear payoffs. In International conference on machine learning, pages 127-135. PMLR, 2013.
[26] L. Li, W. Chu, J. Langford, and R. E. Schapire. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web, pages 661-670, 2010.
[27] Y. Li, Y. Wang, X. Chen, and Y. Zhou. Tight regret bounds for infinite-armed linear contextual bandits. In International Conference on Artificial Intelligence and Statistics, pages 370–378. PMLR, 2021.
[28] Y. Li, Y. Wang, and Y. Zhou. Nearly minimax-optimal regret for linearly parameterized bandits. In Conference on Learning Theory, pages 2173-2174. PMLR, 2019.
[29] C. D. Meyer. Matrix analysis and applied linear algebra, volume 71. Siam, 2000.
[30] V. Perchet, P. Rigollet, S. Chassang, and E. Snowberg. Batched bandit problems. The Annals of Statistics, pages 660–681, 2016.
[31] C. Pike-Burke, S. Agrawal, C. Szepesvari, and S. Grunewalder. Bandits with delayed, aggregated anonymous feedback. In International Conference on Machine Learning, pages 4105-4113. PMLR, 2018.
[32] Y. Ruan, J. Yang, and Y. Zhou. Linear bandits with limited adaptivity and learning distributional optimal design. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing, pages 74-87, 2021.
[33] S. Sahni. Computationally related problems. SIAM Journal on computing, 3(4):262-279, 1974.
[34] J. Sherman and W. J. Morrison. Adjustment of an inverse matrix corresponding to a change in one element of a given matrix. The Annals of Mathematical Statistics, 21(1):124-127, 1950.
[35] M. D. Summa, F. Eisenbrand, Y. Faenza, and C. Moldenhauer. On largest volume simplices and sub-determinants. In Proceedings of the twenty-sixth annual ACM-SIAM symposium on Discrete algorithms, pages 315-323. SIAM, 2014.
[36] K. T. Talluri, G. Van Ryzin, and G. Van Ryzin. The theory and practice of revenue management, volume 1. Springer, 2004.
[37] A. Tewari and S. A. Murphy. From ads to interventions: Contextual bandits in mobile health. Mobile Health: Sensors, Analytic Methods, and Applications, pages 495-517, 2017.
[38] M. J. Todd. Minimum-volume ellipsoids: Theory and algorithms. SIAM, 2016.
[39] S. Yang, T. Ren, S. Shakkottai, E. Price, I. S. Dhillon, and S. Sanghavi. Linear bandit algorithms with sublinear time complexity. In International Conference on Machine Learning, pages 25241-25260. PMLR, 2022.
[40] Z. Zhang, X. Ji, and Y. Zhou. Almost optimal batch-regret tradeoff for batch linear contextual bandits. arXiv preprint arXiv:2110.08057, 2021.
[41] Y. Zhu, D. J. Foster, J. Langford, and P. Mineiro. Contextual bandits with large action spaces: Made practical. In International Conference on Machine Learning, pages 27428-27453. PMLR, 2022.