Lecture 10: Maximizing Long-Term Rewards:  Gittin's and Whittle's Index in Reinforcement Learning
Channel: Abhishek Gupta
Date: 2020-02-12

In this ECE 8851: Reinforcement Learning lecture, we continue our exploration of index policies by examining Gittin's index and Whittle's index. We focus on the key moments in these algorithms, including the discount rewards, stopping rule, optimality equation, reward function, tie-breaking rule, and Whittle's index.

First, we explore Gittin's index and examine the proof of Gittin's index theorem following Weber (1992). We discuss how Gittin's index can be used to determine the optimal time to stop a sequential decision-making process and maximize long-term discounted rewards.

Next, we delve into Whittle's index and a result from Weber and Weiss (1990). We examine how Whittle's index can be used to solve bandit problems and determine the optimal policy for a given set of rewards.

Throughout the lecture, we focus on the key moments in these algorithms, including the discount rewards, stopping rule, optimality equation, reward function, tie-breaking rule, and Whittle's index. We explore the advantages and disadvantages of each approach and discuss how they can be applied to real-world scenarios."