NeurIPS 2025

Constrained Best Arm Identification
Tyron Lardy
CWI and Leiden University
Christina Katsimerou
Booking.com
christina.katsimerou@booking.com
Wouter M. Koolen
CWI and University of Twente
wmkoolen@cwi.nl
Abstract
In real-world decision-making problems, one needs to pick among multiple policies
the one that performs best while respecting economic constraints. This motivates
the problem of constrained best-arm identification for bandit problems where every
arm is a joint distribution of reward and cost. We investigate the general case where
reward and cost are dependent. The goal is to accurately identify the arm with the
highest mean reward among all arms whose mean cost is below a given threshold.
We prove information-theoretic lower bounds on the sample complexity for three
models: Gaussian with fixed covariance, Gaussian with unknown covariance, and
non-parametric distributions of rectangular support. We propose a combination of
a sampling and a stopping rule that correctly identifies the constrained best arm and
matches the optimal sample complexities for each of the three models. Simulations
demonstrate the performance of our algorithms.
1
Introduction
In real-world decision-making systems, identifying the best policy is rarely a matter of optimizing
a single performance metric in isolation. Effective policies often involve inherent trade-offs: they
achieve desirable outcomes but incur costs. For instance, online platforms routinely deploy pro-
motions to drive customer engagement [Zhao and Harinen, 2019, Zhang et al., 2024]. To ensure
sustainability, such incentives must not only be effective but also economically viable—typically
captured through feasibility constraints on budget or return on investment (ROI) [Goldenberg et al.,
2020]. Similarly, bidding policies in online advertising need to drive traffic or purchases at an
acceptable incremental ROI [Chen and Au, 2022]. In healthcare, a treatment needs to improve health
outcomes under safety constraints. In al...