Package: seals
Status: early beta.seals, the Suite of Environments for Algorithms that Learn Specifications, is a toolkit for
evaluating specification learning algorithms, such as reward or imitation learning. The
environments are compatible withGym, but are designed
to test algorithms that learn from user data, without requiring a procedurally specified
reward function.There are two types of environments inseals:Diagnostic Taskswhich test individual facets of algorithm performance in isolation.Renovated Environments, adaptations of widely-used benchmarks such as MuJoCo continuous
control tasks and Atari games to be suitable for specification learning benchmarks. In particular,
we remove any side-channel sources of reward information from MuJoCo tasks, and give Atari games constant-length episodes (although most Atari environments have observations that include the score).sealsis under active development and we intend to add more categories of tasks soon.You may also be interested in our sister projectimitation,
providing implementations of a variety of imitation and reward learning algorithms.Check out ourdocumentationfor more information aboutseals.QuickstartTo install the latest release from PyPI, run:pipinstallsealsAllsealsenvironments are available in the Gym registry. Simply import it and then use as you
would with your usual RL or specification learning algroithm:importgymnasiumasgymimportsealsenv=gym.make('seals/CartPole-v0')We make releases periodically, but if you wish to use the latest version of the code, you can
install directly from Git master:pipinstallgit+https://github.com/HumanCompatibleAI/seals.gitContributingFor development, clone the source code and create a virtual environment for this project:gitclonegit@github.com:HumanCompatibleAI/seals.gitcdseals
./ci/build_venv.sh
pipinstall-e.[dev]# install extra tools useful for developmentCode styleWe follow a PEP8 code style with line length 88, and typically follow theGoogle Code Style Guide,
but defer to PEP8 where they conflict. We use theblackautoformatter to avoid arguing over formatting.
Docstrings follow the Google docstring convention definedhere,
with an extensive example in theSphinx docs.All PRs must pass linting via theci/code_checks.shscript. It is convenient to install this as a commit hook:ln-s../../ci/code_checks.sh.git/hooks/pre-commitTestsWe usepytestfor unit tests
andcodecovfor code coverage.
We also usepytypeandmypyfor type checking.WorkflowTrivial changes (e.g. typo fixes) may be made directly by maintainers. Any non-trivial changes
must be proposed in a PR and approved by at least one maintainer. PRs must pass the continuous
integration tests (CircleCI linting, type checking, unit tests and CodeCov) to be merged.It is often helpful to open an issue before proposing a PR, to allow for discussion of the design
before coding commences.Citing sealsTo cite this project in publications:@misc{seals,author={Adam Gleave and Pedro Freire and Steven Wang and Sam Toyer},title={{seals}: Suite of Environments for Algorithms that Learn Specifications},year={2020},publisher={GitHub},journal={GitHub repository},howpublished={\url{https://github.com/HumanCompatibleAI/seals}},}