Package: easyagents
Reinforcement Learning for Practitioners (v1.4.1, 20Q1)Status:
under active development, breaking changes may occur.Release notes.EasyAgents is a high level reinforcement learning api focusing on ease of use and simplicity.
Written in Python and running on top of established reinforcement learning libraries liketf-Agents,tensorforceorkeras-rl.
Environments are implemented inOpenAI gym.
For an example of an industrial application of reinforcement learning seehere.In collaboration withOliver Zeigermann.Featuresprovides thesame, simple api across all libraries. Thus you can easily switch between different implementations
and you don't have to learn for each of them a new api.to create and run any algorithm you only need2 lines of code, all the parameters are named
consistently across all algorithms.supports a broad set of different algorithmsruns insidejupyter notebooksas well as stand-alone, easy to install requiring only a single
'pip install easyagents'.easy to understand,ready-made plotsand logs to investigate the algorithms and environments behaviourNote: keras-rl backend is suspended until support for tensorflow 2.0 is available.Examplesfrom easyagents.agents import PpoAgent
from easyagents.callbacks import plot

ppoAgent = PpoAgent('CartPole-v0')
ppoAgent.train([plot.State(), plot.Loss(), plot.Rewards()])More Detailedfrom easyagents.agents import PpoAgent
from easyagents.callbacks import plot

ppoAgent = PpoAgent( 'Orso-v1',fc_layers=(500,500,500))
ppoAgent.train([plot.State(), plot.Loss(), plot.Rewards(), plot.Actions(), 
                plot.StepRewards(), plot.Steps(), plot.ToMovie()], 
                learning_rate = 0.0001, num_iterations = 500, max_steps_per_episode=50 )Tutorials1. Introduction (CartPole on colab):
training, plotting, switching algorithms & backends. Based on the classic reinforcement learning example
balancing a stick on a cart.2. Next steps & backend switching (Orso on colab):
custom training, creating a movie & switching backends. gym environment based on a routing problem.3. Controlling training & evaluation (on colab):
or 'what do all these agent.train(...) args mean ?'4. Creating your own environment (LineWorld on colab):
implement a gym environment from scratch, workshop example.5. Saving & loading (on colab):
Once a policy is trained, save it and reload it in a production environment.
You may also save intermediate policies as the training proceeds.6. Switching backends (on colab):
See how you can switch between backend implementations.7. Api logging, seeding & plot clearing (on colab):
Investigate how easyagent interacts with the backend api and the gym environment;
how to set seeds; controlling jupyter output cell clearingAvailable Algorithms and Backendsalgorithmtf-Agentstensorforcekeras-rl (suspended)easyagents class nameCEMnot availablenot availableyesCemAgentDqnyesyesyesDqnAgentDouble Dqnopennot availableyesDoubleDqnAgentDueling Dqnnot availableyesyesDuelingDqnAgentPpoyesyesnot availablePpoAgentRandomyesyesnot availableRandomAgentREINFORCEyesyesnot availableReinforceAgentSACpreviewnot availablenot availableSacAgent[191001]if you are interested in other algorithms, backends or hyperparameters let us know bycreating an issue.
We'll try our best to support you.for a documentation of the agents api seehere.starting with easyagents 1.3 (191102) the backend for keras-rl is suspended
until support for tensorflow 2.0 is available.Industrial ApplicationGeberit- a sanitary technology company with > 12'000 employees -
produces in particular pipes and other parts to get rain-water of flat roofs - so calledsyphonic roof drainage systems.
They warrant that large buildings like stadiums, airports or shopping malls do not collapse during
heavy rainfalls.
However it is surprisingly difficult to find the right dimensions for the pipes.
It is actually so difficult, that as of todayno feasable, deterministic algorithmis known.
Thus traditional heuristics and classic machine learning were used to support the users
in finding a suitable solution.Using reinforcement learning the failrate of the previous solution was reduced by 70%, resulting
in an end-to-end success-rate of > 98%.For more details take a look at thistalk.InstallationInstall from pypi using pip:pipinstalleasyagentsMoreDocumentationforrelease notes & class diagram,
foragents & api.Guiding Principleseasily train, evaluate & debug policies for (you own) gym environment over "designing new algorithms"simple & consistent over "flexible & powerful"inspired by keras:same api across all algorithmssupport different implementations of the same algorithmextensible (pluggable backends, plots & training schemes)EasyAgents may not be ideal ifyou would like to leverage implementation specific advantages of an algorithmyou want to do distributed or in parallel reinforcement learningNoteIf you have any difficulties in installing or using easyagents please let us know bycreating an issue.
We'll try our best to help you.Any ideas, help, suggestions, comments etc in python / open source development / reinforcement learning / whatever
are more than welcome.