Package: unstable-baselines
Unstable Baselines (Early Access)A Deep Reinforcement Learning codebase in TensorFlow 2.0 with an unified, flexible and highly customizable structure for fast prototyping.FeaturesUnstable BaselinesStable-Baselines3OpenAI BaselinesState of the art RL methods:heavy_minus_sign:(1):heavy_check_mark::heavy_check_mark:Documentation:x::heavy_check_mark::x:Custom callback(2):x::vomiting_face::heavy_minus_sign:TensorFlow 2.0 support:heavy_check_mark::x::x:Clean, elegant code:heavy_check_mark::x::x:Easy to trace, customize:heavy_check_mark::x:(3):x:(3)Standalone implementations:heavy_check_mark::heavy_minus_sign::x:(4)(1) Currently only support DQN, C51, PPO, TD3, ...etc. We are still working on other algorithms.(2) For example, in Stable-Baselines, you need towrite this disgusting custom callbackto save the best-performed model :vomiting_face:, while in Unstable Baselines, they are automatically saved.(3) If you have traced Stable-baselines or OpenAI/baselines once, you'll never do that again.(4) Many cross-dependencies across all algos make the code very hard to trace, for examplebaselines/common/policies.py,baselines/a2c/a2c.py.... Great job! OpenAI!:cat:DocumentationWe don't have any documentation yet.InstallationBasic requirements:Python >= 3.6TensorFlow (CPU/GPU) >= 2.3.0You can install from PyPI$ pip install unstable_baselinesOr you can also install the latest version from this repository$ pip install git+https://github.com/Ending2015a/unstable_baselines.git@masterDone! Now, you canGo through theQuick StartsectionOr run the example codes inexample folder.AlgorithmsModel-free RLAlgorithmBoxDiscreteMultiDiscreteMultiBinaryDQN:x::heavy_check_mark::x::x:PPO:heavy_check_mark::heavy_check_mark::x::x:TD3:heavy_check_mark::x::x::x:SD3:heavy_check_mark::x::x::x:2021.09.17:DQNsupportsMulti-step learningPrioritized experience replay:arXiv:1511.05952Dueling network:arXiv:1511.065812021.04.19: ImplementedDQNFrom paper:arXiv:1509.064612021.03.27:PPOsupport continuous (Box) action space2021.03.23: ImplementedSD3From paper:arXiv:2010.091772021.03.20: ImplementedTD3From paper:arXiv:1802.094772021.03.10: ImplementedPPOFrom paper:arXiv:1707.06347Distributional RLAlgorithmBoxDiscreteMultiDiscreteMultiBinaryC51:x::heavy_check_mark::x::x:QRDQN:x::heavy_check_mark::x::x:IQN:x::heavy_check_mark::x::x:2021.04.28: ImplementedIQNFrom paper:arXiv:1806.069232021.04.21: ImplementedQRDQNFrom paper:arXiv:1710.100442021.04.20: ImplementedC51From paper:arXiv:1707.06887Quick StartThis example shows how to train a PPO agent to playCartPole-v0. You can find the full scripts inexample/cartpole/train_ppo.py.First, import dependenciesimportgymimportunstable_baselinesasubfromunstable_baselines.algo.ppoimportPPOCreate environments for training and evaluation# create environmentsenv=ub.envs.VecEnv([gym.make('CartPole-v0')for_inrange(10)])eval_env=gym.make('CartPole-v0')Create a PPO model and train itmodel=PPO(env,learning_rate=1e-3,gamma=0.8,batch_size=128,n_steps=500).learn(# train for 20000 steps20000,verbose=1)Save and load the trained modelmodel.save('./my_ppo_model')model=PPO.load('./my_ppo_model')Evaluate the training resultsmodel.eval(eval_env,20,200,render=True)# don't forget to close the environments!env.close()eval_env.close()More examples:Playing with AtariSolving continuous control problemsUpdate Logs2021.05.22: Add benchmarks2021.04.27: Update to framework v2: supports saving/loading the best performed checkpoints.