lil' UCB : An Optimal Exploration Algorithm for Multi-Armed Bandits
Almost optimal exploration in multi-armed bandits