We consider a linear stochastic bandit problem where the dimension K of the unknown parameter θ is larger than the sampling budget n. Since usual linear bandit algorithms have a regret of order O(K √ n), it is in general impossible to obtain a sub-linear regret without further assumption. In this paper we make the assumption that θ is S−sparse, i.e. has at most S−non-zero components, and that the set of arms is the unit ball for the ||.|| 2 norm. We combine ideas from Compressed Sensing and Bandit Theory to derive an algorithm with a regret bound in O(S √ n). We detail an application to the problem of optimizing a function that depends on many variables but among which only a small number of them (initially unknown) are relevant.
Similarly, REF also considered the high-dimensional stochastic linear bandits with sparsity, combining the ideas from compressed sensing and bandit theory.
Bandit Theory meets Compressed Sensing for high dimensional Stochastic Linear Bandit
AISTATS
Mathematics Computer Science