Impact statement: This work is advertising a new way to do non-parametric exploration in bandit models, that enjoy good empirical performance and strong theoretical guarantees. First, bandit problems are at the heart of numerous applications to online content recommendation, hence the good performance of SDA algorithms may inspire new algorithms for more realistic models used for these applications, such as contextual bandits. Then, exploration is a central question in the broader field of reinforcement learning, hence new ideas for bandits may lead to new ideas for reinforcement learning.