Package: cerise
Cherry is a reinforcement learning framework for researchers built on top of PyTorch.Unlike other reinforcement learning implementations, cherry doesn't implement a single monolithic  interface to existing algorithms.
Instead, it provides you with low-level, common tools to write your own algorithms.
Drawing from the UNIX philosophy, each tool strives to be as independent from the rest of the framework as possible.
So if you don't like a specific tool, you don’t need to use it.FeaturesCherry extends PyTorch with only a handful of new core concepts.PyTorch Modules for reinforcement learning:cherry.nn.Policy:cherry.nn.ActionValue:cherry.nn.StateValue:Data structures for reinforcement learning:cherry.Transition:cherry.ExperienceReplay:Cherry also includes additional features, to help implement existing and new RL algorithms.Pythonic and low-level interfaceà laPytorch.Support for tabular (!) and function approximation algorithms.Various OpenAI Gym environment wrappers.Helper functions for popular algorithms. (e.g. A2C, DDPG, TRPO, PPO, SAC)Logging, visualization, and debugging tools.Painless and efficient distributed training on CPUs and GPUs.Unit, integration, and regression tested, continuously integrated.To learn more about the tools and philosophy behind cherry, check out ourGetting Started tutorial.ExampleThe following snippet showcases some of the tools offered by cherry.importcherryasch# Wrap environmentsenv=gym.make('CartPole-v0')env=ch.envs.Logger(env,interval=1000)env=ch.envs.Torch(env)policy=PolicyNet()optimizer=optim.Adam(policy.parameters(),lr=1e-2)replay=ch.ExperienceReplay()# Manage transitionsforstepinrange(1000):state=env.reset()whileTrue:mass=Categorical(policy(state))action=mass.sample()log_prob=mass.log_prob(action)next_state,reward,done,_=env.step(action)# Build the ExperienceReplayreplay.append(state,action,reward,next_state,done,log_prob=log_prob)ifdone:breakelse:state=next_state# Discounting and normalizing rewardsrewards=ch.td.discount(0.99,replay.reward(),replay.done())rewards=ch.normalize(rewards)loss=-th.sum(replay.log_prob()*rewards)optimizer.zero_grad()loss.backward()optimizer.step()replay.empty()Many more high-quality examples are available in theexamples/folder.InstallationNoteCherry is considered in early alpha release. Stuff might break.pip install cherry-rlChangelogA human-readable changelog is available in theCHANGELOG.mdfile.DocumentationDocumentation and tutorials are available on cherry’s website:http://cherry-rl.net.ContributingFirst, thanks for your consideration in contributing to cherry.
Here are a couple of guidelines we strive to follow.It's always a good idea to open an issue first, where we can discuss how to best proceed.If you want to contribute a new example using cherry, it would preferably stand in a single file.If you would like to contribute a new feature to the core library, we suggest to first implement an example showcasing your new functionality. Doing so is quite useful:it allows for automatic testing,it ensures that the functionality is correctly implemented,it shows users how to use your functionality, andit gives a concrete example when discussing the best way to merge your implementation.We don't have forums, but are happy to discuss with you on slack.
Make sure to send an email tosmr.arnold@gmail.comto get an invite.AcknowledgementsCherry draws inspiration from many reinforcement learning implementations, includingOpenAI Baselines,John Schulman'simplementationsIlya Kostrikov'simplementations,Shangtong Zhang'simplementations,Dave Abel'simplementations,Vitchyr Pong'simplementations,Kai Arulkumaran'simplementations,RLLab/Garage.Why 'cherry' ?Because it's the sweetest part ofthe cake.