Package: dopamine-rl
DopamineGetting Started|Docs|Baseline Results|ChangelistDopamine is a research framework for fast prototyping of reinforcement learning
algorithms. It aims to fill the need for a small, easily grokked codebase in
which users can freely experiment with wild ideas (speculative research).Our design principles are:Easy experimentation: Make it easy for new users to run benchmark
experiments.Flexible development: Make it easy for new users to try out research ideas.Compact and reliable: Provide implementations for a few, battle-tested
algorithms.Reproducible: Facilitate reproducibility in results. In particular, our
setup follows the recommendations given byMachado et al. (2018).Dopamine supports the following agents, implemented with jax:DQN (Mnih et al., 2015)C51 (Bellemare et al., 2017)Rainbow (Hessel et al., 2018)IQN (Dabney et al., 2018)SAC (Haarnoja et al., 2018)For more information on the available agents, see thedocs.Many of these agents also have a tensorflow (legacy) implementation, though
newly added agents are likely to be jax-only.This is not an official Google product.Getting StartedWe provide docker containers for using Dopamine.
Instructions can be foundhere.Alternatively, Dopamine can be installed from source (preferred) or installed
with pip. For either of these methods, continue reading at prerequisites.PrerequisitesDopamine supports Atari environments and Mujoco environments. Install the
environments you intend to use before you install Dopamine:AtariInstall the atari roms following the instructions fromatari-py.pip install ale-py(we recommend using avirtual environment):unzip $ROM_DIR/ROMS.zip -d $ROM_DIR && ale-import-roms $ROM_DIR/ROMS(replace $ROM_DIR with the directory you extracted the ROMs to).MujocoInstall Mujoco and get a licensehere.Runpip install mujoco-py(we recommend using avirtual environment).Installing from SourceThe most common way to use Dopamine is to install it from source and modify
the source code directly:git clone https://github.com/google/dopamineAfter cloning, install dependencies:pip install -r dopamine/requirements.txtDopamine supports tensorflow (legacy) and jax (actively maintained) agents.
View theTensorflow documentationfor
more information on installing tensorflow.Note: We recommend using avirtual environmentwhen working with Dopamine.Installing with PipNote: We strongly recommend installing from source for most users.Installing with pip is simple, but Dopamine is designed to be modified
directly. We recommend installing from source for writing your own experiments.pip install dopamine-rlRunning testsYou can test whether the installation was successful by running the following
from the dopamine root directory.export PYTHONPATH=$PYTHONPATH:$PWD
python -m tests.dopamine.atari_init_testNext StepsView thedocsfor more information on training agents.We supplybaselinesfor each Dopamine agent.We also provide a set ofColaboratory notebookswhich demonstrate how to use Dopamine.ReferencesBellemare et al.,The Arcade Learning Environment: An evaluation platform for
general agents. Journal of Artificial Intelligence Research, 2013.Machado et al.,Revisiting the Arcade Learning Environment: Evaluation
Protocols and Open Problems for General Agents, Journal of Artificial
Intelligence Research, 2018.Hessel et al.,Rainbow: Combining Improvements in Deep Reinforcement Learning.
Proceedings of the AAAI Conference on Artificial Intelligence, 2018.Mnih et al.,Human-level Control through Deep Reinforcement Learning. Nature,
2015.Schaul et al.,Prioritized Experience Replay. Proceedings of the International
Conference on Learning Representations, 2016.Haarnoja et al.,Soft Actor-Critic Algorithms and Applications,
arXiv preprint arXiv:1812.05905, 2018.Giving creditIf you use Dopamine in your work, we ask that you cite ourwhite paper. Here is an example BibTeX entry:@article{castro18dopamine,
  author    = {Pablo Samuel Castro and
               Subhodeep Moitra and
               Carles Gelada and
               Saurabh Kumar and
               Marc G. Bellemare},
  title     = {Dopamine: {A} {R}esearch {F}ramework for {D}eep {R}einforcement {L}earning},
  year      = {2018},
  url       = {http://arxiv.org/abs/1812.06110},
  archivePrefix = {arXiv}
}