We consider the stochastic multi-armed bandit problem with a prior distribution on the reward distributions. We are interested in studying prior-free and prior dependent regret bounds, very much in the same spirit than the usual distribution free and distribution-dependent bounds for the non-Bayesian stochastic bandit. We first show that Thompson Sampling attains an optimal prior-free bound in the sense that for any prior distribution its Bayesian regret is bounded from above by 14 VnK. This result is unimprovable in the sense that there exists a prior dis tribution such that any algoritiun has a Bayesian regret bounded from below by 2 1 0 VnK. We also study the case of priors for the setting of Bubeck et al. [2013] (where the optimal mean is known as well as a lower bound on the smallest gap) and we show that in this case the regret of Thompson Sampling is in fact uniformly bounded over time, thus showing that Thompson Sampling can greatly take advan tage of the nice properties of these priors.
Bubeck and Liu REF showed that this second bound can be improved to one of order |A|T using more sophisticated confidence bound analysis, and also studied a problem setting where the regret of Thompson sampling is bounded uniformly over time.
Prior-free and prior-dependent regret bounds for Thompson Sampling
2014 48th Annual Conference on Information Sciences and Systems (CISS)
2014 48th Annual Conference on Information Sciences and Systems (CISS)
Computer Science Mathematics