8+ Reinforcement Learning Project Ideas
Tags: machinelearning, datascience, beginners, deeplearning
Published: 2021-09-30 07:00:29

This blog post is a compilation of reinforcement learning (RL) project ideas to check out. I've tried to select projects covering a range of different difficulties, concepts, and algorithms in RL.

## 1. Solve toy problems with [OpenAI Gym](https://gym.openai.com/) (beginner-friendly)

![Cartpole](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bh9xo8mqizlidg1l1172.JPG "Cartpole environment from OpenAI Gym")

[OpenAI Gym](https://gym.openai.com/) has become the de facto standard for reinforcement learning frameworks among researchers and practitioners. Solving toy problems from the gym library will help familiarize you with this popular framework. Good starting points include [Cartpole](https://gym.openai.com/envs/CartPole-v1/), [Lunar Lander](https://gym.openai.com/envs/LunarLander-v2/) and [Taxi](https://gym.openai.com/envs/Taxi-v3/).

If you're interested in a step-by-step walkthrough, check out our [introductory Q-learning tutorial with Taxi](https://www.gocoder.one/blog/rl-tutorial-with-openai-gym).

## 2. Play Atari games from pixel input with [OpenAI Gym](https://gym.openai.com/envs/#atari)

![Atari environments](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eoqwtm3ahn9vz1v9chop.jpg "Atari environments from OpenAI Gym")

OpenAI Gym also contains a suite of [Atari game environments](https://gym.openai.com/envs/#atari) as part of its Arcade Learning Environment (ALE) framework. Examples include [Breakout](https://gym.openai.com/envs/Breakout-v0/), [Montezuma Revenge](https://gym.openai.com/envs/MontezumaRevenge-v0/), and [Space Invaders](https://gym.openai.com/envs/SpaceInvaders-v0/). Environment observations are available in the form of screen input or RAM (direct observation of the Atari 2600's 1024 bits of memory).

*Additional resources:*

- [Jupyter notebook tutorial for Space Invaders](https://github.com/simoninithomas/Deep_reinforcement_learning_Course/blob/master/Deep%20Q%20Learning/Space%20Invaders/DQN%20Atari%20Space%20Invaders.ipynb) by Thomas Simonini

## 3. Simulate control tasks with [PyBullet](https://github.com/bulletphysics/bullet3)

![PyBullet](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7c2gzfd980o43q6qsddf.png "PyBullet environment examples")

Gym provides a library of continuous physics simulations in the form of its [MuJoCo](https://gym.openai.com/envs/#mujoco) environments. Since MuJoCo requires a paid license, I recommend checking out [PyBullet](https://github.com/bulletphysics/bullet3) as a free open-source alternative. Using PyBullet/MuJoCo, you can teach a variety of robots to walk, run, or swim.

## 4. Create your own reinforcement learning environment with [Unity ML-Agents](https://github.com/Unity-Technologies/ml-agents) (beginner-friendly)

![Unity ML-Agents](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8c0yn02hrgnf3iuxqsh1.png "Unity ML-Agents example environments")

[Unity ML-Agents](https://github.com/Unity-Technologies/ml-agents) is a relatively new add-on to the Unity game engine. It allows game developers to train intelligent NPCs for games and enables researchers to create graphics- and physics-rich RL environments. Project ideas to explore include:

- Experimenting with algorithms like PPO, SAC, GAIL, and Self-Play provided out-of-the-box
- Training agents in a library of 18+ environments including [Dodgeball](https://github.com/Unity-Technologies/ml-agents/tree/dodgeball-env), Soccer, and classic control problems
- Creating your own custom 3D RL environment

*Additional resources:*

- [Build a 3D Volleyball RL environment](https://www.gocoder.one/blog/hands-on-introduction-to-deep-reinforcement-learning)
- [Reinforcement Learning Penguins](https://www.immersivelimit.com/tutorials/reinforcement-learning-penguins-part-1-unity-ml-agents)
- [Unity Hummingbirds Course](https://learn.unity.com/course/ml-agents-hummingbirds)

## 5. Race self-driving cars with [AWS DeepRacer](https://aws.amazon.com/deepracer/) (beginner-friendly)

![AWS DeepRacer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3h9x75fnhqg9m3wfbz5t.png "AWS DeepRacer simulation")

[AWS DeepRacer](https://aws.amazon.com/deepracer/) is a 3D racing simulator designed to help developers get started with RL using Amazon SageMaker. You'll need to pay for training and evaluating your model on AWS. It features monthly competitive races as part of the [AWS DeepRacer league](https://aws.amazon.com/deepracer/league/), which awards prizes and the chance to compete at re:Invent.

DeepRacer also gives you the option of purchasing a physical 1/18th scale race car for USD399 that will allow you to deploy your model in the real-world.

Some other open-source projects relating to autonomous driving to check out:

- [AirSim](https://github.com/microsoft/AirSim)
- [CARLA](https://github.com/carla-simulator/carla)

## 6. Mine diamonds in Minecraft with [MineRL](https://minerl.io/)

![MineRL](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0f297v2xbl4kfj9ixmec.png "MineRL dataset example")

[MineRL](https://minerl.io/) contains an imitation learning dataset of over 60 million frames of recorded human player data in Minecraft. The goal is to train agents that can navigate an open world and overcome inherent challenges such as tasks with lots of hierarchy and sparse rewards.

MineRL is currently running two competition tracks as part of NeurIPS 2021:

1. [Diamond](https://minerl.io/diamond/): Obtain a diamond provided a fixed limit of raw pixel sample data and time training
2. [BASALT](https://minerl.io/basalt/):  Solve almost-lifelike tasks (e.g. build a house, search for a cave)

## 7. Join the community at [AIArena](https://aiarena.net/) building agents for StarCraft II

![AI Arena](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tyxd21t0ehjoj2gs619i.png "AI Arena StarCraft II stream")

If you're looking to train agents to play highly complex mainstream games, you should check out [AIArena](https://aiarena.net/). They run regular streams and ladders for a community of researchers, practitioners, and hobbyists building deep learning agents for StarCraft II.

Some other games with RL frameworks you might be interested in:

- [Rocket League](https://rlgym.github.io/)
- [League of Legends](https://leaguesandbox.github.io/)
- [Dota 2](https://games.mau.se/research/the-dota2-5v5-ai-competition/)

## 8. Build a Chess Bot with [OpenSpiel](https://github.com/deepmind/open_spiel)

![OpenSpiel](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5s6qop66ghgiwbiciqlm.jpg "Image credit: DeepMind")

[OpenSpiel](https://github.com/deepmind/open_spiel/) by DeepMind is worth taking a look at if you've been inspired by programs like [StockFish](https://stockfishchess.org/) or AlphaGo. It contains a collection of environments and algorithms for general RL and planning/search in a variety of games including Chess, Go, Backgammon, and more.

## Bonus ideas

Here are some additional project ideas that are also worth checking out:

- Predict stock prices with [TensorTrade](https://github.com/tensortrade-org/tensortrade)
- Train cooperative agents with [PettingZoo](https://www.pettingzoo.ml/)
- Build a Poker bot with [RLCard](https://github.com/datamllab/rlcard)
- Join an [AI Programming competition](https://www.gocoder.one/blog/ai-game-competitions-list)

## Closing remarks

There's a huge range of exciting projects to explore in reinforcement learning. This list is by no means comprehensive, but I hope it's given you some inspiration for your own RL project!