Deep RL Course documentation

Deep Q-Learning

Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Deep Q-Learning

Unit 3 thumbnail

In the last unit, we learned our first reinforcement learning algorithm: Q-Learning, implemented it from scratch, and trained it in two environments, FrozenLake-v1 ☃️ and Taxi-v3 🚕.

We got excellent results with this simple algorithm, but these environments were relatively simple because the state space was discrete and small (16 different states for FrozenLake-v1 and 500 for Taxi-v3). For comparison, the state space in Atari games can contain10910^{9} to101110^{11} states.

But as we’ll see, producing and updating a Q-table can become ineffective in large state space environments.

So in this unit, we’ll study our first Deep Reinforcement Learning agent: Deep Q-Learning. Instead of using a Q-table, Deep Q-Learning uses a Neural Network that takes a state and approximates Q-values for each action based on that state.

And we’ll train it to play Space Invaders and other Atari environments using RL-Zoo, a training framework for RL using Stable-Baselines that provides scripts for training, evaluating agents, tuning hyperparameters, plotting results, and recording videos.

Environments

So let’s get started! 🚀