In this paper we study a treatment allocation problem with multiple treatments, in which the individuals to be treated arrive sequentially. The goal of the policy maker is to treat every individual as well as possible. Which treatment is "best" is allowed to depend on various characteristics (functionals) of the individual-specific outcome distribution of each treatment. For example measures of welfare, inequality, or poverty. We propose the Functional Sequential Allocation policy, and provide upper bounds on the regretit incurs compared to the oracle policy that knows the best treatment for each individual. These upper bounds increase sublinearly in the number of treatment assignments and we show that this regret guarantee is minimax optimal. In addition, the assumptions under which this regret guarantee is established are as weak as possible --- even a minimal weakening of them will imply non-existence of policies with regret increasing sub-linearly in the number of assignments. Furthermore, we provide an upper bound on the number of suboptimal assignments made by the FSA policy and show that every policy must make a number of suboptimal assignments at least of this order .
Consider a setting in which a policy maker assigns subjects to treatments, observing each outcome before the next subject arrives. The sequential nature of the problem permits learning about the effectiveness of the treatments, which we measure by a functional of the associated outcome distributions, for example an inequality, welfare or poverty measure. We evaluate different assignment policies according to their regret, that is, the sum of all losses incurred due to assigning subjects to suboptimal treatments. All our results are non-asymptotic, in that we provide finite sample lower and upper bounds on maximal expected regret. We first study explore-then-commit (ETC) policies. These are policies, where one initially explores which treatment is of the highest quality, typically through a randomized controlled trial, and subsequently fully commits to the inferred superior treatment. Then, we introduce and study the Functional Upper Confidence Bound (F-UCB) policy, which interweaves exploration and exploitation and is thus not of the ETC type. Our results show, in particular, that the F-UCB policy (i) performs much better than any ETC policy, and (ii) is near minimax optimal. We also show that a suitably adapted F-UCB policy is near minimax optimal under minimal assumptions when covariate information is available .
R
In this paper we study a treatment allocation problem with multiple treatments, in which the individuals to be treated arrive sequentially. The goal of the policy maker is to treat every individual as well as possible. Which treatment is "best" is allowed to depend on various characteristics (functionals) of the individual-specific outcome distribution of each treatment. For example measures of welfare, inequality, or poverty. We propose the Functional Sequential Allocation policy, and provide
Consider a setting in which a policy maker assigns subjects to treatments, observing each outcome before the next subject arrives. The sequential nature of the problem permits learning about the effectiveness of the treatments, which we measure by a functional of the associated outcome distributions, for example an inequality, welfare or poverty measure. We evaluate different assignment policies according to their regret, that is, the sum of all losses incurred due to assigning subjects to suboptimal treatments. All our results are non-asymptotic, in that we provide finite sample lower and
R
the regretit incurs compared to the oracle policy that knows the best treatment for each individual. These upper bounds increase sublinearly in the number of treatment assignments and we show that this regret guarantee is minimax optimal. In addition,
maximal expected regret. We first study explore-then-commit (ETC) policies. These are policies, where one initially explores which treatment is of the highest quality, typically through a randomized controlled trial, and subsequently fully commits to the inferred superior treatment. Then, we introduce and study the Functional Upper Confidence Bound (F-UCB) policy, which interweaves exploration and exploitation and is thus not of
R
assumptions under which this regret guarantee is established are as weak as possible --- even a minimal weakening of them will imply non-existence of policies with regret increasing sub-linearly in the number of assignments. Furthermore, we provide an upper bound on the number of suboptimal assignments made by the FSA policy and show that every policy must make a number of suboptimal assignments at least of this order
ETC type. Our results show, in particular, that the F-UCB policy (i) performs much better than any ETC policy, and (ii) is near minimax optimal. We also show that a suitably adapted F-UCB policy is near minimax optimal under minimal assumptions when covariate information is available