[17] John C Gittins and David M Jones. A dynamic allocation index for the discounted multiarmed bandit problem. Biometrika, 66(3):561-565, 1979.
[18] Limor Gultchin, Virginia Aglietti, Alexis Bellot, and Silvia Chiappa. Functional causal bayesian optimization. arXiv preprint arXiv:2306.06409, 2023.
[19] Botao Hao, Tor Lattimore, and Mengdi Wang. High-dimensional sparse linear bandits. Advances in Neural Information Processing Systems, 33:10753-10763, 2020.
[20] Joey Hong, Branislav Kveton, Manzil Zaheer, and Mohammad Ghavamzadeh. Hierarchical bayesian bandits. In International Conference on Artificial Intelligence and Statistics, pages 7724-7741. PMLR, 2022.
[21] Nathan Kallus and Angela Zhou. Confounding-robust policy improvement. Advances in neural information processing systems, 31, 2018.
[22] Juha Karvanen. Study design in causal models. Scandinavian Journal of Statistics, 42(2):361-377, 2015.
[23] Finnian Lattimore, Tor Lattimore, and Mark D Reid. Causal bandits: Learning good interventions via causal inference. Advances in Neural Information Processing Systems, 29, 2016.
[24] Sanghack Lee and Elias Bareinboim. Structural causal bandits with non-manipulable variables. In AAAI Conference on Artificial Intelligence, volume 33, pages 4164-4172, 2019.
[25] Xiuyuan Lu and Benjamin Van Roy. Information-theoretic confidence bounds for reinforcement learning. Advances in Neural Information Processing Systems, 32, 2019.
[26] Yangyi Lu, Amirhossein Meisami, and Ambuj Tewari. Causal bandits with unknown graph structure. arXiv preprint arXiv:2106.02988, 2021.
[27] Yangyi Lu, Amirhossein Meisami, Ambuj Tewari, and William Yan. Regret analysis of bandit problems with causal background knowledge. In Conference on Uncertainty in Artificial Intelligence, pages 141-150. PMLR, 2020.
[28] Yangyi Lu, Amirhossein Meisami, Ambuj Tewari, and Zhenyu Yan. Regret analysis of causal bandit problems. arXiv preprint arXiv:1910.04938, 2019.
[29] Alan Malek and Silvia Chiappa. Asymptotically best causal effect identification with multiarmed bandits. Advances in Neural Information Processing Systems, 34:21960-21971, 2021.
[30] Hyejin Park, Seiyun Shin, Kwang-Sung Jun, and Jungseul Ok. Transfer learning in bandits with latent continuity. In 2021 IEEE International Symposium on Information Theory (ISIT), pages 1463-1468. IEEE, 2021.
[31] Judea Pearl. Causality. Cambridge University Press, 2009.
[32] Judea Pearl and Elias Bareinboim. Transportability of causal and statistical relations: A formal approach. In Twenty-fifth AAAI conference on artificial intelligence, 2011.
[33] Amit Peleg, Naama Pearl, and Ron Meir. Metalearning linear bandits by prior update. In International Conference on Artificial Intelligence and Statistics, pages 2885-2926. PMLR, 2022.
[34] Herbert Robbins. Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society, 58(5):527-535, 1952.
[35] Daniel Russo and Benjamin Van Roy. An information-theoretic analysis of Thompson sampling. Journal of Machine Learning Research, 17(1):2442-2471, 2016.
[36] Rajat Sen, Karthikeyan Shanmugam, Alexandros G Dimakis, and Sanjay Shakkottai. Identifying best interventions through online importance sampling. In International Conference on Machine Learning, pages 3057-3066. PMLR, 2017.