Package: tella
tellatella stands for Training and Evaluating Lifelong Learning Agents.
It provides a standard API and tools for performing continual learning experiments.
Full documentation is available atreadthedocs.RequirementsPython 3.7 or greaterInstallCreate a conda or virtual environment and activate itWe recommend that you update pip and wheel in your environment:pip install -U pip wheelInstall tellapip install tellaThere are optional packages that can also be installed if you want
to use atari environments or run the unit tests.
They can be installed using pip's extras syntax:pip install tella[atari]orpip install tella[dev]APItella defines an event-based interface for agents.
tella calls methods of the agent to run through training and evaluation blocks.
The event handlers areblock_start() and block_end()task_start() and task_end()task_variant_start() and task_variant_end()choose_actions() and receive_transitions()A learning block or evaluation block consists of 1 or more tasks.
The agent is notified of the start of the block and the start of each task.
The task_start() method receives basic information about the task.
The agent is also notified of the end of the block and the end of each task.A task consists of multiple episodes.
The agent is notified of the start and end of the episode.
During the episode the agent is called through choose_actions() with an observation and must return an action.
After the environment is updated with the action, the reward is passed to the agent by calling receive_transitions().
The receive_transitions() method also received the previous observation and new observation.
These calls continue until the episode is complete.Here is a minimal agent that takes random agents:importtellaclassMinimalRandomAgent(tella.ContinualRLAgent):defchoose_actions(self,observations):"""Loop over the environments' observations and select action"""return[NoneifobsisNoneelseself.action_space.sample()forobsinobservations]defreceive_transitions(self,transitions):"""Do nothing here since we are not learning"""passif__name__=="__main__":# rl_cli() is tella's command line interface function.# It expects a constructor or factory function to create the agent.tella.rl_cli(MinimalRandomAgent)print("Done! Check logs for results.")RunAssuming your agent is defined in a file calledmy_agent.py,
run it through a curriculum like so:python my_agent.py --curriculum SimpleCartPoleTo see all the command line options, run:>pythonmy_agent.py--help
usage:my_agent.py[-h][--lifetime-idxLIFETIME_IDX][--num-lifetimesNUM_LIFETIMES][--num-parallel-envsNUM_PARALLEL_ENVS][--log-dirLOG_DIR][--render][--seedSEED][--agent-seedAGENT_SEED][--curriculum-seedCURRICULUM_SEED][--agent-configAGENT_CONFIG]--curriculum{...}optionalarguments:-h,--helpshowthishelpmessageandexit--lifetime-idxLIFETIME_IDXTheindex,startingatzero,ofthefirstlifetimetorun.Usethistoskiplifetimesorrunaspecificlifetimeotherthanthefirst.(default:0)--num-lifetimesNUM_LIFETIMESNumberoflifetimestoexecute.(default:1)--num-parallel-envsNUM_PARALLEL_ENVSNumberofenvironmentstoruninparallelinsideoftaskvariantblocks.ThisenablestheuseofmultipleCPUsatthesametimeforrunningenvironmentlogic,viavectorizedenvironments.(default:1)--log-dirLOG_DIRTherootdirectoryforthel2loggerlogsproduced.(default:./logs)--renderWhethertorendertheenvironment(default:False)--seedSEEDreplacedby--agent-seedand--curriculum-seed(default:None)--agent-seedAGENT_SEEDTheagentrngseedtouseforreproducibility.(default:None)--curriculum-seedCURRICULUM_SEEDThecurriculumrngseedtouseforreproducibility.(default:None)--agent-configAGENT_CONFIGOptionalpathtoagentconfigfile.(default:None)--curriculum{...}Curriculumnameforregistry.(default:None)All the curriculums registered with tella are listed in the help.The l2logger output by default is stored in your current directory inlogs.
This can be set with the--log-dirargument.For reproducing behavior, use the--agent-seedand--curriculum-seedarguments.
If a seed is not provided, a random seed is generated.
The seeds used will be logged using the python logging package.For utilizing multiple cores, use the--num-parallel-envsflag.
When using--num-parallel-envs> 1, you may need to configure
python multiprocessing's start method viamp.set_start_method("spawn")at the start of the program, depending on the underlying OS.To run an agent through multiple lifetimes of a curriculum, use the--num-lifetimesflag. If you want to run a specific lifetime (useful for running on a cluster),
use the--lifetime-idxflag. Note that the curriculum seed must be provided to use--lifetime-idx.To view a rendering of the agent learning, set the--renderflag.
This will render the first environment in the list.To pass a configuration file to the agent, set the--agent-configargument.
The format of the configuration file is determined by the specific agent.Bug Reports and Feature RequestsBug reports and feature requests should be made through issues on Github.A bug report should contain:descriptive titleenvironment (python version, operating system if install issue)expected behavioractual behaviorstack trace if applicableany input parameters need to reproduce the bugA feature request should describe what you want to do but cannot
and any recommendations for how this new feature should work.For DevelopersTo install tella in editable mode with our development requirements,
clone the git repo and run:pip install -e ".[dev]"To run unit tests:pytestFor running in conda environment:python -m pytestTo check for PEP8 compliance:black --check tellaTo autoformat for PEP8 compliance:black tellaLicenseSeeLICENSEfor license information.Acknowledgmentstella was created by the Johns Hopkins University Applied Physics Laboratory.This software was funded by the DARPA Lifelong Learning Machines (L2M) Program.The views, opinions, and/or findings expressed are those of the author(s) and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government.