Package: angorapy
Anthropomorphic Goal-Oriented Robotic Control for Neuroscientific ModelingAngoraPyis an open source modeling library forgoal-oriented researchinneuroscience. It provides a simple interface to train deep neural network models of the human brain on various, customizable, sensorimotor tasks, using reinforcement learning. It thereby empowers goal-driven modeling to surpass the sensory domain and enter that of sensori_motor_ control, closing the perception-action loop.AngoraPyis designed to require no deeper understanding of reinforcement learning. It employs state-of-the-art machine learning techniques, optimized for distributed computation scaling from local workstations to high-performance computing clusters. We aim to hide as much of this under the hood of an intuitive, high-level API but preserve the option for customizing most aspects of the pipeline.This library is developed as part of theHuman Brain ProjectatCCN Maastricht. It is an effort to build software by neuroscientists, for neuroscientists. If you have suggestions, requests or questions, feel free toopen an issue.:sparkles: FeaturesSupported Task SettingsDiscrete Action Spaces (Categorical, MultiCategorical)Continuous Action Spaces (Beta, Gaussian)Discrete State SpacesContinuous State SpacesSupported Model TypesRecurrent NetworksConvolutional NetworksRecurrent+Convolutional NetworksSupported Model TrainingLocal Distributed TrainingHPC Distributed TrainingTraining BackendProximal Policy OptimizationAsymmetric Policy/Value NetworksTruncated Backpropagation Through TimeEntrypoints & DeploymentPyPI PackageDocker filesSource code📥 InstallationAngoraPy is available on PyPI. First, install requirements:sudoaptinstallswiglibopenmpi-devThen install AngoraPy from pip.pipinstallangorapyFrom sourceAlternatively, you can download this repository or the source code of any previous release or branch and install from source, using pip.pipinstall-e.This way, if you make changes to the source code, these will be recognized in the installation (without the need to reinstall).DockerAlternatively, you can install AngoraPy and all its dependencies in a docker container using the Dockerfile provided in this repository (/docker/Dockerfile). To this end, download the repository and build the docker image from the /docker directory:sudodockerbuild-tangorapy:masterhttps://github.com/ccnmaastricht/angorapy.git#master-f-<DockerfileTo install different versions, replace#masterin the source by the tag/branch of the respective version you want to install.🚀 Getting StartedThe scriptstrain.py,evaluate.pyandobserve.pyprovide ready-made scripts for training and evaluating an agent in any environment. Withpretrain.py, it is possible to pretrain the visual component.benchmark.pyprovides functionality for training a batch of agents possibly using different configs for comparison of strategies.Training an AgentThetrain.pycommandline interface provides a convenient entry-point for running all sorts of experiments using the builtin models and environments in angorapy. You can train an agent on any environment with optional hyperparameters. Additionally, a monitor will be automatically linked to the training of the agent. For more detail consult theREADME on monitoring.Base usage oftrain.pyis as follows:python train.py ENV --architecture MODEL
For instance, trainingLunarLanderContinuous-v2using thedeeperarchitecture is possible by running:python train.py LunarLanderContinuous-v2 --architecture deeper
For more advanced options like custom hyperparameters, consultpython train.py -hEvaluating and Observing an AgentThere are two more entry points for evaluating and observing an agent:evaluate.pyandobserve.py. General usage is as followspython evaluate.py ID
Where ID is the agent's ID given when its created (train.pyprints this outt, in custom scripts get it withagent.agent_id).Writing a Training ScriptTo train agents with custom models, environments, etc. you write your own script. The following is a minimal example:fromangorapyimportmake_taskfromangorapy.modelsimportget_model_builderfromangorapy.agent.ppo_agentimportPPOAgentenv=make_task("LunarLanderContinuous-v2")model_builder=get_model_builder("simple","ffn")agent=PPOAgent(model_builder,env)agent.drill(100,10,512)For more details, consult theexamples.🎓 DocumentationDetailed documentation of AngoraPy is provided in the READMEs of most subpackages. Additionally, we provideexamples and tutorialsthat get you started with writing your own scripts using AngoraPy. For further readings on specific modules, consult the following READMEs:Agent[WIP]EnvironmentsModelsAnalysisMonitoringIf you are missing a documentation for a specific part of AngoraPy, feel free to open an issue and we will do our best to add it.🔀 Distributed ComputationPPO is an asynchronous algorithm, allowing multiple parallel workers to generate experience independently.
We allow parallel gathering and optimization through MPI. Agents will automatically distribute their workers evenly on
the available CPU cores, while optimization is distributed over all available GPUs. If no GPUs are available, all CPUs
share the task of optimizing.Distribution is possible locally on your workstation and on HPC sites.💻 Local Distributed Computing with MPITo use MPI locally, you need to have a running MPI implementation, e.g. Open MPI 4 on Ubuntu.
To executetrain.pyvia MPI, runmpirun-np12--use-hwthread-cpuspython3train.py...where, in this example, 12 is the number of locally available CPU threads and--use-hwthread-cpusmakes available threads (as opposed to only cores). Usage oftrain.pyis as described previously.:cloud: Distributed Training on SLURM-based HPC clustersPlease note that the following is optimized and tested on the specific cluster we use, but should extend to at least
any SLURM based setup.On any SLURM-based HPC cluster you may submit your job with sbatch usising the following script template:#!/bin/bash -l#SBATCH --job-name="angorapy"#SBATCH --account=xxx#SBATCH --time=24:00:00#SBATCH --nodes=32#SBATCH --ntasks-per-core=1#SBATCH --ntasks-per-node=12#SBATCH --cpus-per-task=1#SBATCH --partition=normal#SBATCH --constraint=gpu&startx#SBATCH --hint=nomultithreadexportOMP_NUM_THREADS=$SLURM_CPUS_PER_TASKexportCRAY_CUDA_MPS=1# load virtual environmentsource${HOME}/robovenv/bin/activateexportDISPLAY=:0
srunpython3-utrain.py...The number of parallel workers will equal the number of nodes times the number of CPUs per node
(32 x 12 = 384 in the template above).🔗 Citing AngoraPyIf you use AngoraPy for your research, please cite us as followsWeidler, T., & Senden, M. (2020). AngoraPy: Anthropomorphic Goal-Oriented Robotic Control for Neuroscientific Modeling [Computer software]
Or using bibtex@software{angorapy2020,
author = {Weidler, Tonio and Senden, Mario},
month = {3},
title = {{AngoraPy: Anthropomorphic Goal-Oriented Robotic Control for Neuroscientific Modeling}},
year = {2020}
}