K-armed Bandits for Autonomous Agents