Title: Best Arm Identification with Knapsacks: Minimax Policies
Primary area: reinforcement learning
Keywords: economics, best arm identification, pure exploration bandits, bayesian learning

Abstract: A resource-constrained decision maker (DM) designs a continuous-time sequential experiment to determine the best choice out of a set of treatments. The DM divides her attention between observing the treatments until one of the resources runs out. Under the minimax regret criterion, we characterize the optimal policy for two treatments when there is a fixed array of resources. Of additional interest is a prerequisite result in which we characterize the minimax regret optimal policy under a single infinite resource (money) and adaptive stopping. Our analysis relies on a reformulation of the typical optimal stopping problem in which we model diffusions with respect to the cumulative resource expenditure rather than the elapsed time.

TL;DR: Explicitly characterize the minimax optimal policy in a two-armed best arm identification problem with resource constraints.