ICML 2025

Zero-Inflated Bandits
Haoyu Wei * 1 Runzhe Wan * 2 Lei Shi 2 Rui Song 2
Abstract
Many real-world bandit applications are charac-
terized by sparse rewards, which can significantly
hinder learning efficiency. Leveraging problem-
specific structures for careful distribution model-
ing is recognized as essential for improving esti-
mation efficiency in statistics. However, this ap-
proach remains under-explored in the context of
bandits. To address this gap, we initiate the study
of zero-inflated bandits, where the reward is mod-
eled using a classic semi-parametric distribution
known as the zero-inflated distribution. We de-
velop algorithms based on the Upper Confidence
Bound and Thompson Sampling frameworks for
this specific structure. The superior empirical
performance of these methods is demonstrated
through extensive numerical studies.
1. Introduction
Bandit algorithms have been widely applied in areas such
as clinical trials (Durand et al., 2018), finance (Shen et al.,
2015), recommendation systems (Zhou et al., 2017), among
others. Accurate uncertainty quantification is key to ad-
dressing the exploration-exploitation trade-off and typically
requires on certain reward distribution assumptions. Exist-
ing assumptions can be roughly classified into two groups:
• Parametric: the reward distribution is assumed to be-
long to a parameterized family, such as Gaussian or
Bernoulli distributions (Audibert et al., 2010; Krause
& Ong, 2011; Agrawal & Goyal, 2012). The strong
assumption typically ensures clean algorithmic design
and nice theoretical results. However, when misspeci-
fied, it may lead to over- or under-exploration.
*Equal contribution 1Department of Economics, University of
California San Diego, La Jolla, USA 2Amazon, Seattle, USA.
This work does not relate to the positions at Amazon.
Cor-
respondence to: Haoyu Wei <h8wei@ucsd.edu>, Rui Song
<songray@gmail.com>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 2...