Package: avalon-rl
AvalonWhat is Avalon?Avalon is a 3D video game environment and benchmark designed from scratch for reinforcement learning research. In Avalon, an embodied agent (human or computer) explores a procedurally generated 3D environment, attempting to solve a set of tasks that involve navigating terrain, hunting or gathering food, and avoiding hazards.Avalon is unique among existing RL benchmarks in that the reward function, world dynamics, and action space are the same for every task, with tasks differentiated solely by altering the environment: its 20 tasks, ranging in complexity from eat and throw to hunt and navigate, each create worlds in which the agent must perform specific skills in order to survive. This setup enables investigations of generalization within tasks, between tasks, and to compositional tasks that require combining skills learned from previous tasks.Avalon includes a highly efficient game engine, a library of baselines, and a benchmark with scoring metrics evaluated against hundreds of hours of human performance, all of which are open-source and publicly available. We find that standard RL baselines make progress on most tasks but are still far from human performance, suggesting Avalon is challenging enough to advance the quest for generalizable RL.Check outour research paperfor a deeper explanation of why we built Avalon.QuickstartUse Avalon just like you would any othergymenvironment.fromavalon.agent.godot.godot_gymimportGodotEnvironmentParamsfromavalon.agent.godot.godot_gymimportTrainingProtocolChoicefromavalon.agent.godot.godot_gymimportAvalonEnvfromavalon.common.log_utilsimportconfigure_local_loggerfromavalon.datagen.env_helperimportdisplay_videoconfigure_local_logger()env_params=GodotEnvironmentParams(resolution=256,training_protocol=TrainingProtocolChoice.SINGLE_TASK_FIGHT,initial_difficulty=1,)env=AvalonEnv(env_params)env.reset()defrandom_env_step():action=env.action_space.sample()obs,reward,done,info=env.step(action)ifdone:env.reset()returnobsobservations=[random_env_step()for_inrange(50)]display_video(observations,fps=10)For a full example on how to create random worlds, take actions as an agent, and display the resulting observations, seegym_interface_example.InstallingAs Avalon is designed as a high-performance RL environment, it's tailored to running in the cloud on headless linux servers with Nvidia GPUs. However, it should also work on macOS.Avalon relies ona custom Godot binaryoptimized for headless rendering and performance. If you intend to inspect, debug or build custom levels, you'll also want the accompanying editor:pipinstallavalon-rl==1.0.0# needed to actually run environmentspython-mavalon.install_godot_binaryNote: the binary will be installed in the package underavalon/bin/godotby default to avoid cluttering your system.
Pure-pip binary packaging is a work-in-progress.UbuntuOn Linux, a Nvidia GPU is required, as the linux builds are set up for headless GPU rendering.sudo apt install --no-install-recommends libegl-dev libglew-dev libglfw3-dev libnvidia-gl libopengl-dev libosmesa6 mesa-utils-extra
pip install avalon-rl
python -m avalon.install_godot_binary
python -m avalon.common.check_installIf you're looking to use our RL code, you'll need additionally:pytorch>=1.12.0with CUDAtheavalon-rl[train]extras package:pip install avalon-rl[train]MacOn Mac, a Nvidia GPU is not required, but the environment rendering is not headless - you'll see a godot window pop up for each environment you have open.brew install coreutils
pip install avalon-rl
python -m avalon.install_godot_binary
python -m avalon.common.check_installDockerWe also have Docker images set up to run Avalon and train our RL baselines. They require a Nvidia GPU on the host.Training imagedockerbuild-f./docker/Dockerfile.--targettrain--tag=avalon/train# start the container with an interactive bash terminal# to enable wandb, add `-e WANDB_API_KEY=<your wandb key>`dockerrun-it--gpus'all,"capabilities=compute,utility,graphics"'avalon/trainbash# in the container, try runningpython-mavalon.common.check_install# or launch eg a PPO training run withpython-mavalon.agent.train_ppo_avalonDev imageYou can use thedevimage to explore the bundled notebooks or to build on top of Avalondockerbuild-f./docker/Dockerfile.--targetdev--tag=avalon/dev# The default dev image command starts a jupyter notebook and exposes it on port 8888.# A typical dev setup is to expose that notebook and map the local repo to the project repo as a volume:dockerrun-it-p8888:8888-v$(pwd):/opt/projects/avalon--gpus'all,"capabilities=compute,utility,graphics"'avalon/devTutorialsUsing Avalon in your own RL code:Using Avalon via the OpenAI Gym interface:
an example of how to create random worlds, take actions as an agent, and display the resulting observations.Training a PPO Agent on Avalon using RLlib:
an example of how to use Avalon with an external RL library/codebase.Inspecting Test Worlds:
Generate and view Avalon worlds usingenv.debug_actand the godot editor.Using our RL library:Training PPO and Dreamer Agents using the Avalon RL library:
train RL agents using our own fast, tuned implementations!Reproducing the RL results from our paperBuilding on Avalon or creating new tasks:Generating and inspecting an Avalon worldRunning Avalon in VR: Generate and play worlds on an Oculus Quest.Adding custom procedurally-generated tasks to AvalonMaking an entirely custom non-Avalon RL environment using GodotResourcesHuman scoresHuman observationsHuman normalized actionsHuman raw inputsEvaluation worldsCustom Godot engine buildFinal baseline model weights can be found in theresults notebook.Citing Avalon@inproceedings{avalon,
    title={Avalon: A Benchmark for {RL} Generalization Using Procedurally Generated Worlds},
    author={Joshua Albrecht and Abraham J Fetterman and Bryden Fogelman and Ellie Kitanidis and Bartosz Wr{\'o}blewski and Nicole Seo and Michael Rosenthal and Maksis Knutins and Zachary Polizzi and James B Simon and Kanjun Qiu},
    booktitle={Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
    year={2022},
    url={https://openreview.net/forum?id=TzNuIdrHoU}
}LicenseHuman rollout dataset:CC-BY-SA 4.0Modificationsto the Godot engine:MITAll code in this repository and all other associated resources (model checkpoints, etc):GPLv3About Generally IntelligentAvalon was developed by Generally Intelligent, independent research company developing general-purpose AI agents with human-like intelligence that can be safely deployed in the real world. Check out ouraboutpage to learn more, or ourcareerspage if you're interested in working with us!