LEC 14: Multi-Armed Bandits_A Deep Dive into Frequentist Exploration Schemes
Channel: Abhishek Gupta
Date: 26/02/2020

In this lecture on ECE 8851: Reinforcement Learning, we dive into the different types of frequentist exploration schemes in multi-armed bandits. We explore the concept of exploration and exploitation in the context of reinforcement learning and discuss the key moments when these ideas come into play.

We start by discussing the caution and probing approach, where the agent is cautious at the beginning of the exploration process and gradually becomes more confident in its decisions. We then move on to the signaling approach, which involves using signals to guide the exploration process.

Next, we discuss the differences between caution and exploitation and how they relate to each other. We also cover the Bayesian and frequentist approaches to exploration and discuss the notation used in these approaches.

We then dive into specific exploration schemes, including the epsilon-greedy, Upper Confidence Bound (UCB), KL-UCB, Boltzmann, Boltzmann-Gumbel, and MOSS exploration schemes. For each scheme, we discuss how it works and its strengths and weaknesses.