lil' UCB : An Optimal Exploration Algorithm for Multi-Armed Bandits
The Sample Complexity of Exploration in the Multi-Armed Bandit Problem