Package: tf-agents
TF-Agents: A reliable, scalable and easy to use TensorFlow library for Contextual Bandits and Reinforcement Learning.TF-Agentsmakes implementing, deploying,
and testing new Bandits and RL algorithms easier. It provides well tested and
modular components that can be modified and extended. It enables fast code
iteration, with good test integration and benchmarking.To get started, we recommend checking out one of our Colab tutorials. If you
need an intro to RL (or a quick recap),start here. Otherwise, check out ourDQN tutorialto get an agent up and
running in the Cartpole environment. API documentation for the current stable
release is ontensorflow.org.TF-Agents is under active development and interfaces may change at any time.
Feedback and comments are welcome.Table of contentsAgentsTutorialsMulti-Armed BanditsExamplesInstallationContributingReleasesPrinciplesContributorsCitationDisclaimerAgentsIn TF-Agents, the core elements of RL algorithms are implemented asAgents. An
agent encompasses two main responsibilities: defining a Policy to interact with
the Environment, and how to learn/train that Policy from collected experience.Currently the following algorithms are available under TF-Agents:DQN:Human level control through deep reinforcement learningMnih et
al., 2015DDQN:Deep Reinforcement Learning with Double Q-learningHasselt et
al., 2015DDPG:Continuous control with deep reinforcement learningLillicrap et
al., 2015TD3:Addressing Function Approximation Error in Actor-Critic MethodsFujimoto et al., 2018REINFORCE:Simple Statistical Gradient-Following Algorithms for
Connectionist Reinforcement LearningWilliams,
1992PPO:Proximal Policy Optimization AlgorithmsSchulman et al., 2017SAC:Soft Actor CriticHaarnoja et al., 2018TutorialsSeedocs/tutorials/for tutorials on the major components
provided.Multi-Armed BanditsThe TF-Agents library contains a comprehensive Multi-Armed Bandits suite,
including Bandits environments and agents. RL agents can also be used on Bandit
environments. There is a tutorial inbandits_tutorial.ipynb.
and ready-to-run examples intf_agents/bandits/agents/examples/v2.ExamplesEnd-to-end examples training agents can be found under each agent directory.
e.g.:DQN:tf_agents/agents/dqn/examples/v2/train_eval.pyInstallationTF-Agents publishes nightly and stable builds. For a list of releases read theReleasessection. The commands below cover installing
TF-Agents stable and nightly frompypi.orgas well as from a
GitHub clone.:warning: If using Reverb (replay buffer), which is very common,
TF-Agents will only work with Linux.Note: Python 3.11 requires pygame 2.1.3+.StableRun the commands below to install the most recent stable release. API
documentation for the release is ontensorflow.org.$pipinstall--usertf-agents[reverb]# Use this tag get the matching examples and colabs.$gitclonehttps://github.com/tensorflow/agents.git
$cdagents
$gitcheckoutv0.18.0If you want to install TF-Agents with versions of Tensorflow orReverbthat are flagged as not compatible
by the pip dependency check, use the following pattern below at your own risk.$pipinstall--usertensorflow
$pipinstall--userdm-reverb
$pipinstall--usertf-agentsIf you want to use TF-Agents with TensorFlow 1.15 or 2.0, install version 0.3.0:# Newer versions of tensorflow-probability require newer versions of TensorFlow.$pipinstalltensorflow-probability==0.8.0
$pipinstalltf-agents==0.3.0NightlyNightly builds include newer features, but may be less stable than the versioned
releases. The nightly build is pushed astf-agents-nightly. We suggest
installing nightly versions of TensorFlow (tf-nightly) and TensorFlow
Probability (tfp-nightly) as those are the versions TF-Agents nightly are
tested against.To install the nightly build version, run the following:# `--force-reinstall helps guarantee the right versions.$pipinstall--user--force-reinstalltf-nightly
$pipinstall--user--force-reinstalltfp-nightly
$pipinstall--user--force-reinstalldm-reverb-nightly# Installing with the `--upgrade` flag ensures you'll get the latest version.$pipinstall--user--upgradetf-agents-nightlyFrom GitHubAfter cloning the repository, the dependencies can be installed by runningpip install -e .[tests]. TensorFlow needs to be installed independently:pip install --user tf-nightly.ContributingWe're eager to collaborate with you! SeeCONTRIBUTING.mdfor a guide on how to contribute. This project adheres to TensorFlow'scode of conduct. By participating, you are expected to
uphold this code.ReleasesTF Agents has stable and nightly releases. The nightly releases are often fine
but can have issues due to upstream libraries being in flux. The table below
lists the version(s) of TensorFlow that align with each TF Agents' release.
Release versions of interest:0.18.0 dropped Python 3.8 support.0.16.0 is the first version to support Python 3.11.0.15.0 is the last release compatible with Python 3.7.If using numpy < 1.19, then use TF-Agents 0.15.0 or earlier.0.9.0 is the last release compatible with Python 3.6.0.3.0 is the last release compatible with Python 2.x.ReleaseBranch / TagTensorFlow Versiondm-reverb VersionNightlymastertf-nightlydm-reverb-nightly0.18.0v0.18.02.14.00.13.00.17.0v0.17.02.13.00.12.00.16.0v0.16.02.12.00.11.00.15.0v0.15.02.11.00.10.00.14.0v0.14.02.10.00.9.00.13.0v0.13.02.9.00.8.00.12.0v0.12.02.8.00.7.00.11.0v0.11.02.7.00.6.00.10.0v0.10.02.6.00.9.0v0.9.02.6.00.8.0v0.8.02.5.00.7.1v0.7.12.4.00.6.0v0.6.02.3.00.5.0v0.5.02.2.00.4.0v0.4.02.1.00.3.0v0.3.01.15.0 and 2.0.0.PrinciplesThis project adheres toGoogle's AI principles. By
participating, using or contributing to this project you are expected to adhere
to these principles.ContributorsWe would like to recognize the following individuals for their code
contributions, discussions, and other work to make the TF-Agents library.James DavidsonEthan HollyToby BoydSummer YueRobert OrmandiKuang-Huei LeeAlexa GreenbergAmir YazdanbakhshYao LuGaurav JainChristof AngermuellerMark DaoustAdam WoodCitationIf you use this code, please cite it as:@misc{TFAgents,
  title = {{TF-Agents}: A library for Reinforcement Learning in TensorFlow},
  author = {Sergio Guadarrama and Anoop Korattikara and Oscar Ramirez and
     Pablo Castro and Ethan Holly and Sam Fishman and Ke Wang and
     Ekaterina Gonina and Neal Wu and Efi Kokiopoulou and Luciano Sbaiz and
     Jamie Smith and Gábor Bartók and Jesse Berent and Chris Harris and
     Vincent Vanhoucke and Eugene Brevdo},
  howpublished = {\url{https://github.com/tensorflow/agents}},
  url = "https://github.com/tensorflow/agents",
  year = 2018,
  note = "[Online; accessed 25-June-2019]"
}DisclaimerThis is not an official Google product.