Package: raylab
Reinforcement learning algorithms inRLlibandPyTorch.InstallationpipinstallraylabQuickstartRaylab provides agents and environments to be used with a normal RLlib/Tune setup.
You can an agent’s name (from theAlgorithmssection) toraylab info listto list its top-level configurations:raylabinfolistSoftAClearning_starts: 0
    Hold this number of timesteps before first training operation.
policy: {}
    Sub-configurations for the policy class.
wandb: {}
    Configs for integration with Weights & Biases.

    Accepts arbitrary keyword arguments to pass to `wandb.init`.
    The defaults for `wandb.init` are:
    * name: `_name` property of the trainer.
    * config: full `config` attribute of the trainer
    * config_exclude_keys: `wandb` and `callbacks` configs
    * reinit: True

    Don't forget to:
      * install `wandb` via pip
      * login to W&B with the appropriate API key for your
        team/project.
      * set the `wandb/project` name in the config dict

    Check out the Quickstart for more information:
    `https://docs.wandb.com/quickstart`You can add the--rllibflag to get the descriptions for all the options common to RLlib agents
(orTrainers)Launching experiments can be done via the command line usingraylab experimentpassing a file path
with an agent’s configuration through the--configflag.
The following command uses the cartpoleexampleconfiguration file
to launch an experiment using the vanilla Policy Gradient agent from the RLlib library.raylabexperimentPG--namePG-straining_iteration10--configexamples/PG/cartpole_defaults.pyYou can also launch an experiment from a Python script normally using Ray and Tune.
The following shows how you may use Raylab to perform an experiment comparing different
types of exploration for the NAF agent.importrayfromrayimporttuneimportraylabdefmain():raylab.register_all_agents()raylab.register_all_environments()ray.init()tune.run("NAF",local_dir="data/NAF",stop={"timesteps_total":100000},config={"env":"CartPoleSwingUp-v0","exploration_config":{"type":tune.grid_search(["raylab.utils.exploration.GaussianNoise","raylab.utils.exploration.ParameterNoise"])}},num_samples=10,)if__name__=="__main__":main()One can then visualize the results usingraylab dashboard, passing thelocal_dirused in the
experiment. The dashboard lets you filter and group results in a quick way.raylabdashboarddata/NAF/You can find the best checkpoint according to a metric (episode_reward_meanby default)
usingraylab find-best.raylabfind-bestdata/NAF/Finally, you can pass a checkpoint toraylab rolloutto see the returns collected by the agent and
render it if the environment supports a visualrender()method. For example, you
can use the output of thefind-bestcommand to see the best agent in action.raylabrollout$(raylabfind-bestdata/NAF/)--agentNAFAlgorithmsPaperAgent NameActor Critic using Kronecker-factored Trust RegionACKTRTrust Region Policy OptimizationTRPONormalized Advantage FunctionNAFStochastic Value GradientsSVG(inf)/SVG(1)/SoftSVGSoft Actor-CriticSoftACStreamlined Off-Policy(DDPG)SOPModel-Based Policy OptimizationMBPOModel-based Action-Gradient-EstimatorMAGECommand-line interfaceFor a high-level description of the available utilities, runraylab--helpUsage:raylab[OPTIONS]COMMAND[ARGS]...RayLab:ReinforcementlearningalgorithmsinRLlib.Options:--helpShowthismessageandexit.Commands:dashboardLaunchtheexperimentdashboardtomonitortrainingprogress.episodesLaunchtheepisodedashboardtomonitorstateandaction...experimentLaunchaTuneexperimentfromaconfigfile.find-bestFindthebestexperimentcheckpointasmeasuredbyametric.infoViewinformationaboutanagent'sconfigparameters.rolloutWrap`rllibrollout`withcustomizedoptions.test-moduleLaunchdashboardtotestgenerativemodelsfromacheckpoint.PackagesThe project is structured as followsraylab
|-- agents            # Trainer and Policy classes
|-- cli               # Command line utilities
|-- envs              # Gym environment registry and utilities
|-- logger            # Tune loggers
|-- policy            # Extensions and customizations of RLlib's policy API
|   |-- losses        # RL loss functions
|   |-- modules       # PyTorch neural network modules for TorchPolicy
|-- pytorch           # PyTorch extensions
|-- utils             # miscellaneous utilities