Consider a setting in which a policy maker assigns subjects to treatments, observing each outcome before the next subject arrives. Initially, it is unknown which treatment is best . However, the sequential nature of the problem permits learning about the effectiveness of the treatments , which we measure by a functional of the associated outcome distributions, for example an inequality, welfare or poverty measure . In the present article, we evaluate assignment policies according to their regret, that is, the sum of all losses incurred due to assigning subjects to suboptimal treatments. We first study explore-then-commit (ETC) policies. These are policies, where one initially explores which treatment is of the highest quality, typically through a randomized controlled trial, and subsequently fully commits to the ``inferred superior'' (but potentially suboptimal) treatment . Then, we introduce and study the Functional Upper Confidence Bound (F-UCB) policy, which interweaves exploration and exploitation and is thus not of the ETC type. Our results show , in particular, that the F-UCB policy (i) performs much better than any ETC policy, and (ii) is near minimax optimal. We also show that a suitably adapted F-UCB policy is near minimax optimal under minimal assumptions when covariate information is available .
Consider a setting in which a policy maker assigns subjects to treatments, observing each outcome before the next subject arrives. Initially, it is unknown which treatment is best , but the sequential nature of the problem permits learning about the effectiveness of the treatments . While the multi-armed-bandit literature has shed much light on the situation when the policy maker compares the effectiveness of the treatments through their mean, economic decision making often requires targeting purpose specific characteristics of the outcome distribution, such as its inherent degree of inequality, welfare or poverty . In the present paper we introduce and study sequential learning algorithms when the distributional characteristic of interest is a general functional of the outcome distribution. In particular, it turns out that intuitively reasonable approaches, such as first conducting an experiment on an initial group of subjects followed by rolling out the inferred best treatment to the population, are dominated by the policies we develop and of which we show that they are optimal .
R
. However,
, but
R
, which we measure by a functional of the associated outcome distributions, for example an
. While the multi-armed-bandit literature has shed much light on the situation when the policy maker compares the effectiveness of the treatments through their mean, economic decision making often requires targeting purpose specific characteristics of the outcome distribution, such as its inherent degree of
D
measure
R
article, we evaluate assignment policies according to their regret, that is, the sum of all losses incurred due to assigning subjects to suboptimal treatments. We first study explore-then-commit (ETC) policies. These are policies, where one initially explores which treatment is
paper we introduce and study sequential learning algorithms when the distributional characteristic of interest is a general functional of the outcome distribution. In particular, it turns out that intuitively reasonable approaches, such as first conducting an experiment on an initial group of subjects followed by rolling out the inferred best treatment to the population, are dominated by the policies we develop and
R
the highest quality, typically through a randomized controlled trial, and subsequently fully commits to the ``inferred superior'' (but potentially suboptimal) treatment . Then, we introduce and study the Functional Upper Confidence Bound (F-UCB) policy, which interweaves exploration and exploitation and is thus not of the ETC type. Our results show , in particular, that the F-UCB policy (i) performs much better than any ETC policy, and (ii) is near minimax optimal. We also show that a suitably adapted F-UCB policy is near minimax optimal under minimal assumptions when covariate information is available
which we show that they are optimal