Package: sample-factory
Sample FactoryHigh-throughput reinforcement learning codebase. Version 2.0.0 is out! 🤗Resources:Documentation:https://samplefactory.devPaper:https://arxiv.org/abs/2006.11751Citation:BibTeXDiscord:https://discord.gg/BCfHWaSMkrTwitter (for updates):@petrenko_aiTalk (circa 2021):https://youtu.be/lLG17LKKSZcWhat is Sample Factory?Sample Factory is one of the fastest RL libraries.
We focused on very efficient synchronous and asynchronous implementations of policy gradients (PPO).Sample Factory is thoroughly tested, used by many researchers and practitioners, and is actively maintained.
Our implementation is known to reach SOTA performance in a variety of domains in a short amount of time.
Clips below demonstrate ViZDoom, IsaacGym, DMLab-30, Megaverse, Mujoco, and Atari agents trained with Sample Factory:Key features:Highly optimized algorithmarchitecturefor maximum learning throughputSynchronous and asynchronoustraining regimesSerial (single-process) modefor easy debuggingOptimal performance in both CPU-based andGPU-accelerated environmentsSingle- & multi-agent training, self-play, supportstraining multiple policiesat once on one or many GPUsPopulation-Based Training (PBT)Discrete, continuous, hybrid action spacesVector-based, image-based, dictionary observation spacesAutomatically creates a model architecture by parsing action/observation space specification. Supportscustom model architecturesLibrary is designed to be imported into other projects,custom environmentsare first-class citizensDetailedWandB and Tensorboard summaries,custom metricsHuggingFace 🤗 integration(upload trained models and metrics to the Hub)Multipleexampleenvironmentintegrationswith tuned parameters and trained modelsThis Readme provides only a brief overview of the library.
Visit full documentation athttps://samplefactory.devfor more details.InstallationJust install from PyPI:pip install sample-factorySF is known to work on Linux and macOS. There is no Windows support at this time.
Please refer to thedocumentationfor additional environment-specific installation notes.QuickstartUse command line to train an agent using one of the existing integrations, e.g. Mujoco (might need to runpip install sample-factory[mujoco]):python-msf_examples.mujoco.train_mujoco--env=mujoco_ant--experiment=Ant--train_dir=./train_dirStop the experiment (Ctrl+C) when the desired performance is reached and then evaluate the agent:python-msf_examples.mujoco.enjoy_mujoco--env=mujoco_ant--experiment=Ant--train_dir=./train_dirDo the same in a pixel-based VizDoom environment (might need to runpip install sample-factory[vizdoom], please also see docs for VizDoom-specific instructions):python-msf_examples.vizdoom.train_vizdoom--env=doom_basic--experiment=DoomBasic--train_dir=./train_dir--num_workers=16--num_envs_per_worker=10--train_for_env_steps=1000000python-msf_examples.vizdoom.enjoy_vizdoom--env=doom_basic--experiment=DoomBasic--train_dir=./train_dirMonitor any running or completed experiment with Tensorboard:tensorboard--logdir=./train_dir(or see the docs for WandB integration).To continue from here, copy and modify one of the existing env integrations to train agents in your own custom environment. We provide
examples for all kinds of supported environments, please refer to thedocumentationfor more details.AcknowledgementsThis project would not be possible without amazing contributions from many people. I would like to thank:Vladlen Koltunfor amazing guidance and support, especially in the early stages of the project, for
helping me solidify the ideas that eventually became this library.My academic advisorGaurav Sukhatmefor supporting this project
over the years of my PhD and for being overall an awesome mentor.Zhehui Huangfor his contributions to the original ICML submission, his diligent work on
testing and evaluating the library and for adopting it in his own research.Edward Beechingfor his numerous awesome contributions to the codebase, including
hybrid action distributions, new version of the custom model builder, multiple environment integrations, and also
for promoting the library through the HuggingFace integration!Andrew ZhangandMing Wangfor numerous contributions to the codebase and documentation during their HuggingFace internships!Thomas Wolfand others at HuggingFace for the incredible (and unexpected) support and for the amazing
work they are doing for the open-source community.Erik Wijmansfor feedback and insights and for his awesome implementation of RNN backprop using PyTorch'sPackedSequence, multi-layer RNNs, and other features!Tushar Kumarfor contributing to the original paper and for his help
with thefast queue implementation.Costa Huangfor developing CleanRL, for his work on benchmarking RL algorithms, and for awesome feedback
and insights!Denys Makoviichukfor developing rl_games, a very fast RL library, for inspiration and
feedback on numerous features of this library (such as return normalizations, adaptive learning rate, and others).Eugene Vinitskyfor adopting this library in his own research and for his valuable feedback.All my labmates at RESL who used Sample Factory in their projects and provided feedback and insights!Huge thanks to all the people who are not mentioned here for your code contributions, PRs, issues, and questions!
This project would not be possible without a community!CitationIf you use this repository in your work or otherwise wish to cite it, please make reference to our ICML2020 paper.@inproceedings{petrenko2020sf,
  author    = {Aleksei Petrenko and
               Zhehui Huang and
               Tushar Kumar and
               Gaurav S. Sukhatme and
               Vladlen Koltun},
  title     = {Sample Factory: Egocentric 3D Control from Pixels at 100000 {FPS}
               with Asynchronous Reinforcement Learning},
  booktitle = {Proceedings of the 37th International Conference on Machine Learning,
               {ICML} 2020, 13-18 July 2020, Virtual Event},
  series    = {Proceedings of Machine Learning Research},
  volume    = {119},
  pages     = {7652--7662},
  publisher = {{PMLR}},
  year      = {2020},
  url       = {http://proceedings.mlr.press/v119/petrenko20a.html},
  biburl    = {https://dblp.org/rec/conf/icml/PetrenkoHKSK20.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}For questions, issues, inquiries please join Discord.
Github issues and pull requests are welcome! Check out thecontribution guidelines.