Package: fastrl
fastrlWarning: This is in alpha, and so uses latest torch and torchdata,
very importantly torchdata. The base API, while at the point of
semi-stability, might be changed in future versions, and so there will
be no promises of backward compatiblity. For the time being, it is
best to hard-pin versions of the library.Warning: Even before fastrl==2.0.0, all Models should converge
reasonably fast, however HRL modelsDADSandDIAYNwill need
re-balancing and some extra features that the respective authors used.OverviewFastai for computer vision and tabular learning has been amazing. One
would wish that this would be the same for RL. The purpose of this repo
is to have a framework that is as easy as possible to start, but also
designed for testing new agents.This version fo fastrl is basically a wrapper aroundtorchdata.It is built around 4 pipeline concepts (half is from fastai):DataLoading/DataBlock pipelinesAgent pipelinesLearner pipelinesLogger pluginsDocumentation is being served athttps://josiahls.github.io/fastrl/from
documentation directly generated vianbdevin this repo.Basic DQN example:fromfastrl.loggers.coreimport*fromfastrl.loggers.vscode_visualizersimport*fromfastrl.agents.dqn.basicimport*fromfastrl.agents.dqn.targetimport*fromfastrl.data.blockimport*fromfastrl.envs.gymimport*importtorch# Setup Loggerslogger_base=ProgressBarLogger(epoch_on_pipe=EpocherCollector,batch_on_pipe=BatchCollector)# Setup up the core NNtorch.manual_seed(0)model=DQN(4,2)# Setup the Agentagent=DQNAgent(model,[logger_base],max_steps=10000)# Setup the DataBlockblock=DataBlock(GymTransformBlock(agent=agent,nsteps=2,nskips=2,firstlast=True),# We basically merge 2 steps into 1 and skip.(GymTransformBlock(agent=agent,nsteps=2,nskips=2,firstlast=True,n=100,include_images=True),VSCodeTransformBlock()))dls=L(block.dataloaders(['CartPole-v1']*1))# Setup the Learnerlearner=DQNLearner(model,dls,logger_bases=[logger_base],bs=128,max_sz=20_000,nsteps=2,lr=0.001,batches=1000,dp_augmentation_fns=[# Plugin TargetDQN codeTargetModelUpdater.insert_dp(),TargetModelQCalc.replace_dp()])learner.fit(10)#learner.validate()Whats new?As we have learned how to support as many RL agents as possible, we
found thatfastrl==1.*was vastly limited in the models that it can
support.fastrl==2.*will leverage thenbdevlibrary for better
documentation and more relevant testing, andtorchdatais the base
lib. We also will be building on the work of theptan1library as a close reference for pytorch based reinforcement learning
APIs.1“Shmuma/Ptan”. Github, 2020,https://github.com/Shmuma/ptan. Accessed 13 June 2020.InstallPyPIBelow will install the alpha build of fastrl.Cuda Installpip install fastrl==0.0.* --pre --extra-index-url https://download.pytorch.org/whl/nightly/cu113Cpu Installpip install fastrl==0.0.* --pre --extra-index-url https://download.pytorch.org/whl/nightly/cpuDocker (highly recommend)Install:Nvidia-DockerInstall:docker-composedocker-composepull&&docker-composeupContributingAfter you clone this repository, please runnbdev_install_hooksin
your terminal. This sets up git hooks, which clean up the notebooks to
remove the extraneous stuff stored in the notebooks (e.g. which cells
you ran) which causes unnecessary merge conflicts.Before submitting a PR, check that the local library and notebooks
match. The scriptnbdev_cleancan let you know if there is a
difference between the local library and the notebooks. * If you made a
change to the notebooks in one of the exported cells, you can export it
to the library withnbdev_build_libormake fastai2. * If you made
a change to the library, you can export it back to the notebooks withnbdev_update_lib.