Package: parl
English |简体中文PARL is a flexible and high-efficient reinforcement learning framework.About PARLFeaturesAbstractionsParallelizationInstallGetting StartedExamplesAbout PARLFeaturesReproducible. We provide algorithms that stably reproduce the result of many influential reinforcement learning algorithms.Large Scale. Ability to support high-performance parallelization of training with thousands of CPUs and multi-GPUs.Reusable.  Algorithms provided in the repository could be directly adapted to a new task by defining a forward network and training mechanism will be built automatically.Extensible. Build new algorithms quickly by inheriting the abstract class in the framework.AbstractionsPARL aims to build an agent for training algorithms to perform complex tasks.The main abstractions introduced by PARL that are used to build an agent recursively are the following:ModelModelis abstracted to construct the forward network which defines a policy network or critic network given state as input.AlgorithmAlgorithmdescribes the mechanism to update parameters inModeland often contains at least one model.AgentAgent, a data bridge between the environment and the algorithm, is responsible for data I/O with the outside environment and describes data preprocessing before feeding data into the training process.Note: For more information about base classes, please visit ourtutorialandAPI documentation.ParallelizationPARL provides a compact API for distributed training, allowing users to transfer the code into a parallelized version by simply adding a decorator. For more information about our APIs for parallel training, please visit ourdocumentation.Here is aHello Worldexample to demonstrate how easy it is to leverage outer computation resources.#============Agent.py=================@parl.remote_classclassAgent(object):defsay_hello(self):print("Hello World!")defsum(self,a,b):returna+bparl.connect('localhost:8037')agent=Agent()agent.say_hello()ans=agent.sum(1,5)# it runs remotely, without consuming any local computation resourcesTwo steps to use outer computation resources:use theparl.remote_classto decorate a class at first, after which it is transferred to be a new class that can run in other CPUs or machines.callparl.connectto initialize parallel communication before creating an object. Calling any function of the objectsdoes notconsume local computation resources since they are executed elsewhere.As shown in the above figure, real actors (orange circle) are running at the cpu cluster, while the learner (blue circle) is running at the local gpu with several remote actors (yellow circle with dotted edge).For users, they can write code in a simple way, just like writing multi-thread code, but with actors consuming remote resources. We have also provided examples of parallized algorithms likeIMPALA,A2C. For more details in usage please refer to these examples.Install:DependenciesPython 3.6+(Python 3.8+ is preferable for distributed training).paddlepaddle>=2.3.1(Optional, if you only want to use APIs related to parallelization alone)pip install parlGetting StartedSeveral-points to get you started:Tutorial: How to solve cartpole problem.Xparl Usage: How to set up a cluster withxparland compute in parallel.Advanced Tutorial: Create customized algorithms.API documentationFor beginners who know little about reinforcement learning, we also provide an introductory course: (Video|Code)ExamplesQuickStartDQNESDDPGA2CTD3SACQMIXMADDPGPPOCQLIMPALAWinning Solution for NIPS2018: AI for Prosthetics ChallengeWinning Solution for NIPS2019: Learn to Move ChallengeWinning Solution for NIPS2020: Learning to Run a Power Network Challenge