Package: safety-gymnasium
Safety-GymnasiumWhy Safety-Gymnasium?|Documentation|Install guide|CustomizationSafety-Gymnasium is a highly scalable and customizable Safe Reinforcement Learning (SafeRL) library.
It aims to deliver a good view of benchmarking SafeRL algorithms and a standardized set of environments.
We provide a set of standard APIs which are compatible with information on constraints.
Users can explore new insights via an elegant code framework and well-designed environments.Why Safety-Gymnasium?Here we provide a table for comparison ofSafety-Gymnasiumand existing SafeRL Environments libraries.SafeRLEnvsEngineVectorizedEnvironmentsNew Gym API(3)Vision InputSafety-Gymmujoco-py(1)❌❌minimally supportedsafe-control-gymPyBullet❌❌❌Velocity-Constraints(2)N/A❌❌❌mujoco-circlePyTorch❌❌❌Safety-GymnasiumMuJoCo 2.3.0+✅✅✅(1): Maintenance (expect bug fixes and minor updates); the last commit is 19 Nov 2021. Safety-Gym depends onmujoco-py2.0.2.7, which was updated on Oct 12, 2019.(2): There is no official library for speed-related environments, and its associated cost constraints are constructed from info. But the task is widely used in the study of SafeRL, and we encapsulate it in Safety-Gymnasium.(3): In the gym 0.26.0 release update, a new API of interaction was redefined.EnvironmentsWe designed a variety of safety-enhanced learning tasks and integrated the contributions from the RL community:safety-velocity,safety-run,safety-circle,safety-goal,safety-button, etc.
We introduce a unified safety-enhanced learning benchmark environment library calledSafety-Gymnasium.Further, to facilitate the progress of community research, we redesignedSafety-Gymand removed the dependency onmujoco-py.
We built it on top ofMuJoCoand fixed some bugs, more specific bug reports can refer toSafety-Gym's BUG Report.Here is a list of all the environments we support for now:CategoryTaskAgentExampleSafe NavigationGoal[012]Point, Car, Doggo, Racecar, AntSafetyPointGoal1-v0Button[012]Push[012]Circle[012]VelocityVelocityHalfCheetah, Hopper, Swimmer, Walker2d, Ant, HumanoidSafetyAntVelocity-v1Here are some screenshots of the Safe Navigation tasks.AgentsPointCarRacecarDoggoAntTasksGoal0Goal1Goal2Button0Button1Button2Push0Push1Push2Circle0Circle1Circle2Vision-base Safe RLVision-based safety reinforcement learning lacks realistic scenarios.
Although the originalSafety-Gymcould minimally support visual input, the scenarios were too similar.
To facilitate the validation of visual-based safety reinforcement learning algorithms, we have developed a set of realistic vision-based SafeRL tasks, which are currently being validated on the baseline.For the appetizer, the images are as follows:Environment UsageNotes:We support explicitly expressing thecostbased onGymnasium APIs.
Thestepmethod returns 6 items(next_obervation, reward, cost, terminated, truncated, info)with an extracostfield.importsafety_gymnasiumenv_id='SafetyPointGoal1-v0'env=safety_gymnasium.make(env_id)obs,info=env.reset()whileTrue:act=env.action_space.sample()obs,reward,cost,terminated,truncated,info=env.step(act)ifterminatedortruncated:breakenv.render()We also provide two convenience wrappers for converting the Safety-Gymnasium environment to the standard Gymnasium API and vice versa.# Safety-Gymnasium API: step returns (next_obervation, reward, cost, terminated, truncated, info)# Gymnasium API:        step returns (next_obervation, reward, terminated, truncated, info) and cost is in the `info` dict associated with a str key `'cost'`safety_gymnasium_env=safety_gymnasium.make(env_id)gymnasium_env=safety_gymnasium.wrappers.SafetyGymnasium2Gymnasium(safety_gymnasium_env)safety_gymnasium_env=safety_gymnasium.wrappers.Gymnasium2SafetyGymnasium(gymnasium_env)Users can apply Gymnasium wrappers easily with:importgymnasiumimportsafety_gymnasiumdefmake_safe_env(env_id):safe_env=safety_gymnasium.make(env_id)env=safety_gymnasium.wrappers.SafetyGymnasium2Gymnasium(safe_env)env=gymnasium.wrappers.SomeWrapper1(env)env=gymnasium.wrappers.SomeWrapper2(env,argname1=arg1,argname2=arg2)...env=gymnasium.wrappers.SomeWrapperN(env)safe_env=safety_gymnasium.wrappers.Gymnasium2SafetyGymnasium(env)returnsafe_envorimportfunctoolsimportgymnasiumimportsafety_gymnasiumdefmake_safe_env(env_id):returnsafety_gymnasium.wrappers.with_gymnasium_wrappers(safety_gymnasium.make(env_id),gymnasium.wrappers.SomeWrapper1,functools.partial(gymnasium.wrappers.SomeWrapper2,argname1=arg1,argname2=arg2),...,gymnasium.wrappers.SomeWrapperN,)In addition, for all Safety-Gymnasium environments, we also provide corresponding Gymnasium environments with a suffixGymnasiumin the environment id. For example:importgymnasiumimportsafety_gymnasiumsafety_gymnasium.make('SafetyPointGoal1-v0')# step returns (next_obervation, reward, cost, terminated, truncated, info)gymnasium.make('SafetyPointGoal1Gymnasium-v0')# step returns (next_obervation, reward, terminated, truncated, info)InstallationInstall from PyPIpipinstallsafety-gymnasiumInstall from sourcecondacreate-n<envname>python=3.8
condaactivate<envname>

gitclonehttps://github.com/PKU-Alignment/safety-gymnasium.gitcdsafety-gymnasium
pipinstall-e.Important NotesIf you failed to render on your server, you can try:echo"export MUJOCO_GL=osmesa">>~/.bashrcsource~/.bashrc
apt-getinstalllibosmesa6-dev
apt-getinstallpython3-openglCustomize your environmentsWe construct a highly expandable framework of code so that you can easily comprehend it and design your environments to facilitate your research with no more than 100 lines of code on average.For details, please refer to our documentation.
Here is a minimal example:# import the objects you want to use# or you can define specific objects by yourself, just make sure obeying our specificationfromsafety_gymnasium.assets.geomsimportApplesfromsafety_gymnasium.basesimportBaseTask# inherit the basetaskclassMytaskLevel0(BaseTask):def__init__(self,config):super().__init__(config=config)# define some propertiesself.num_steps=500self.agent.placements=[(-0.8,-0.8,0.8,0.8)]self.agent.keepout=0self.lidar_conf.max_dist=6# add objects into environmentsself.add_geoms(Apples(num=2,size=0.3))defcalculate_reward(self):# implement your reward function# Note: cost calculation is based on objects, so it's automaticreward=1returnrewarddefspecific_reset(self):# depending on your taskdefspecific_step(self):# depending on your taskdefupdate_world(self):# depending on your task@propertydefgoal_achieved(self):# depending on your taskCiting Safety-GymnasiumIf you find Safety-Gymnasium useful, please cite it in your publications.@article{Safety-Gymnasium,author={Jiaming Ji and Borong Zhang and Xuehai Pan and Jiayi Zhou and Juntao Dai and Yaodong Yang},title={Safety-Gymnasium},year={2023},publisher={GitHub},journal={GitHub repository},howpublished={\url{https://github.com/PKU-Alignment/safety-gymnasium}},}LicenseSafety-Gymnasium is released under Apache License 2.0.