Package: nle
The NetHack Learning Environment (NLE) is a Reinforcement Learning environment presented atNeurIPS 2020.
NLE is based onNetHack 3.6.6and designed to provide a standard RL interface to the game, and comes with tasks that function as a first step to evaluate agents on this new environment.NetHack is one of the oldest and arguably most impactful videogames in history,
as well as being one of the hardest roguelikes currently being played by humans.
It is procedurally generated, rich in entities and dynamics, and overall an
extremely challenging environment for current state-of-the-art RL agents, while
being much cheaper to run compared to other challenging testbeds. Through NLE,
we wish to establish NetHack as one of the next challenges for research in
decision making and machine learning.You can read more about NLE in theNeurIPS 2020 paper, and about NetHack in itsoriginal
README, atnethack.org, and on theNetHack wiki.NLE Language WrapperWe thankngoodgerfor implementing theNLE Language Wrapperthat translates the non-language observations from NetHack tasks into similar language representations. Actions can also be optionally provided in text form which are converted to the Discrete actions of the NLE.NetHack Learning DatasetThe NetHack Learning Dataset (NLD) code now ships withNLE, allowing users to the load large-scale datasets featured inDungeons and Data: A Large-Scale NetHack Dataset, while also generating and loading their own datasets.importnle.datasetasnldifnotnld.db.exists():nld.db.create()# NB: Different methods are used for data based on NLE and data from NAO.nld.add_nledata_directory("/path/to/nld-aa","nld-aa-v0")nld.add_altorg_directory("/path/to/nld-nao","nld-nao-v0")dataset=nld.TtyrecDataset("nld-aa-v0",batch_size=128,...)fori,mbinenumerate(dataset):foo(mb)# etc...For information on how to download NLD-AA and NLD-NAO, see the dataset dochere.Otherwise checkout the tutorial Colab notebookhere.Papers using the NetHack Learning EnvironmentIzumiya and Simo-SerraInventory Management with Attention-Based Meta Actions(Waseda University, CoG 2021).Samvelyan et al.MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research(FAIR, UCL, Oxford, NeurIPS 2021).Zhang et al.BeBold: Exploration Beyond the Boundary of Explored Regions(Berkley, FAIR, Dec 2020).Küttler et al.The NetHack Learning Environment(FAIR, Oxford, NYU, Imperial, UCL, NeurIPS 2020).Open apull
requestto add papers.Getting startedStarting with NLE environments is extremely simple, provided one is familiar
with other gym / RL environments.InstallationNLE requirespython>=3.5,cmake>=3.15to be installed and available both when building the
package, and at runtime.OnMacOS, one can useHomebrewas follows:$brewinstallcmakeOn a plainUbuntu 18.04distribution,cmakeand other dependencies
can be installed by doing:# Python and most build deps$sudoapt-getinstall-ybuild-essentialautoconflibtoolpkg-config\python3-devpython3-pippython3-numpygitflexbisonlibbz2-dev# recent cmake version$wget-O-https://apt.kitware.com/keys/kitware-archive-latest.asc2>/dev/null|sudoapt-keyadd-
$sudoapt-add-repository'deb https://apt.kitware.com/ubuntu/ bionic main'$sudoapt-getupdate&&apt-get--allow-unauthenticatedinstall-y\cmake\kitware-archive-keyringAfterwards it's a matter of setting up your environment. We advise using a conda
environment for this:$condacreate-y-nnlepython=3.8
$condaactivatenle
$pipinstallnleNOTE: If you want to extend / develop NLE, please install the package as follows:$gitclonehttps://github.com/facebookresearch/nle--recursive
$pipinstall-e".[dev]"$pre-commitinstallDockerWe have provided some docker images. Please see therelevant
README.Trying it outAfter installation, one can try out any of the provided tasks as follows:>>>importgym>>>importnle>>>env=gym.make("NetHackScore-v0")>>>env.reset()# each reset generates a new dungeon>>>env.step(1)# move agent '@' north>>>env.render()NLE also comes with a few scripts that allow to get some environment rollouts,
and play with the action space:# Play NetHackStaircase-v0 as a human$python-mnle.scripts.play# Use a random agent$python-mnle.scripts.play--moderandom# Play the full game using directly the NetHack internal interface# (Useful for debugging outside of the gym environment)$python-mnle.scripts.play--envNetHackScore-v0# works with random agent too# See all the options$python-mnle.scripts.play--helpNote thatnle.scripts.playcan also be run withnle-play, if the package
has been properly installed.Additionally, aTorchBeastagent is bundled innle.agenttogether with a simple model to provide a
starting point for experiments:$pipinstall"nle[agent]"$python-mnle.agent.agent--num_actors80--batch_size32--unroll_length80--learning_rate0.0001--entropy_cost0.0001--use_lstm--total_steps1000000000Plot the mean return over the last 100 episodes:$python-mnle.scripts.plotaveraged episode return

  140 +---------------------------------------------------------------------+
      |             +             +            ++-+ ++++++++++++++++++++++++|
      |             :             :          ++++++++||||||||||||||||||||||||
  120 |-+...........:.............:...+-+.++++|||||||||||||||||||||||||||||||
      |             :        +++++++++++++++||||||||||AAAAAAAAAAAAAAAAAAAAAA|
      |            +++++++++++++||||||||||||||AAAAAAAAAAAA|||||||||||||||||||
  100 |-+......+++++|+|||||||||||||||||||||||AA||||||||||||||||||||||||||||||
      |       +++|||||||||||||||AAAAAAAAAAAAAA|||||||||||+++++++++++++++++++|
      |    ++++|||||AAAAAAAAAAAAAA||||||||||||++++++++++++++-+:             |
   80 |-++++|||||AAAAAA|||||||||||||||||||||+++++-+...........:...........+-|
      | ++|||||AAA|||||||||||||||++++++++++++-+ :             :             |
   60 |++||AAAAA|||||+++++++++++++-+............:.............:...........+-|
      |++|AA||||++++++-|-+        :             :             :             |
      |+|AA|||+++-+ :             :             :             :             |
   40 |+|A+++++-+...:.............:.............:.............:...........+-|
      |+AA+-+       :             :             :             :             |
      |AA-+         :             :             :             :             |
   20 |AA-+.........:.............:.............:.............:...........+-|
      |++-+         :             :             :             :             |
      |+-+          :             :             :             :             |
    0 |-+...........:.............:.............:.............:...........+-|
      |+            :             :             :             :             |
      |+            +             +             +             +             |
  -20 +---------------------------------------------------------------------+
      0           2e+08         4e+08         6e+08         8e+08         1e+09
                                       stepsContributingWe welcome contributions to NLE. If you are interested in contributing please
seethis document.ArchitectureNLE is direct fork ofNetHackand
therefore contains code that operates on many different levels of abstraction.
This ranges from low-level game logic, to the higher-level administration of
repeated nethack games, and finally to binding of these games to Pythongymenvironment.If you want to learn more about the architecture ofnleand how it works
under the hood, checkout thearchitecture document.
This may be a useful starting point for anyone looking to contribute to the
lower level elements of NLE.Related Environmentsgym_nethackrogueinaboxrogue-gymMiniGridCoinRunMineRLProject MalmoOpenAI Procgen BenchmarkObstacle TowerInterview about the environment with Weights&BiasesFacebook AI Research’s Tim & Heiner on democratizing reinforcement learning research.CitationIf you use NLE in any of your work, please cite:@inproceedings{kuettler2020nethack,
  author    = {Heinrich K{\"{u}}ttler and
               Nantas Nardelli and
               Alexander H. Miller and
               Roberta Raileanu and
               Marco Selvatici and
               Edward Grefenstette and
               Tim Rockt{\"{a}}schel},
  title     = {{The NetHack Learning Environment}},
  booktitle = {Proceedings of the Conference on Neural Information Processing Systems (NeurIPS)},
  year      = {2020},
}If you use NLD or the datasets in any of your work, please cite:@inproceedings{hambro2022dungeonsanddata,
  author    = {Eric Hambro and
               Roberta Raileanu and
               Danielle Rothermel and
               Vegard Mella and
               Tim Rockt{\"{a}}schel and
               Heinrich K{\"{u}}ttler and
               Naila Murray},
  title     = {{Dungeons and Data: A Large-Scale NetHack Dataset}},
  booktitle = {Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
  year      = {2022},
  url       = {https://openreview.net/forum?id=zHNNSzo10xN}
}