Package: gym-md
gym-md日本語のREADME.mdThe original Japanese README can be foundhere.ContentsOverviewInstallationInstalling from PyPIRunning build and testsPrerequisitesRunning the build and testsUsageThe MiniDungeons Gym EnvironmentOverviewActionsEnvironmentenv objectenv.step methodLevels and SettingsOverviewgym-md is a python reimplementation[^1] of the dungeon exploration gameMiniDungeons[^2] created as anOpenAI Gym[^3] environment.
MiniDungeons[^2] is a roguelike dungeon exploration game created as a benchmark research domain for modeling decision-making styles of human players[^4]. A Java implementation of MiniDungeons can be foundhere.[^1]: Y. Iwasaki. and K. Hasebe., “A framework for generating playstyles
of game ai with clustering of play logs,” in Proceedings of the 14th
International Conference on Agents and Artificial Intelligence - Volume
3: ICAART,, INSTICC. SciTePress, 2022, pp. 605–612.[^2]: C. Holmgård, A. Liapis, J. Togelius, and G. N. Yannakakis, “Evolving
personas for player decision modeling,” in 2014 IEEE Conference on
Computational Intelligence and Games, 2014, pp. 1–8.[^3]: G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” arXiv preprint
arXiv:1606.01540, 2016.[^4]: A. Liapis, “Minidungeons”, website, 2022. Accessed on: Mar. 27, 2022. [Online].
Available:http://antoniosliapis.com/projects/project_minidungeons.phpInstallationInstalling from PyPIThe gym-md python package can be found onpypi.
To install the latest gym-md package run:pipinstallgym-mdRunning build and testsPrerequisitesThe gym-md project makes use ofpipenvfor the overall project's package management.
In order to build the project's documentation and run the respective tests pipenv will need to be installed.
Please see the 'Installation' section on thepipenvPyPI page.
If you face any issues with the pipenv installation, you can also try installing pipenv using pip (seesource).Furthermore, several additional tests and code linting is orchestrated usingtox,
defined in thetox.inifile.
Please seetox installationfor more detail.Running the build and testsIf you would like to build and install gym-md from source, please run the following commands:gitclonehttps://github.com/Ganariya/gym-md.gitcdgym-md# create the pipenv gym-md build and testing environmentpipenvinstall# launch the pipenv environmentpipenvshell# build gym-md documentationpipenvrunbuild# run gym-md testspipenvruntest# start the tox testing orchestrationtox# to build and upload your own gym-md wheel (.whl) file, please see the upload.sh file.# your custom .whl can be locally installed using: pip install <path to .whl>rm-f-rgym_md.egg-info/*dist/*
pythonsetup.pybdist_wheel
twineuploaddist/*Usageimportgymimportgym_mdimportrandomenv=gym.make('md-test-v0')LOOP:int=100TRY_OUT:int=100for_inrange(TRY_OUT):observation=env.reset()reward_sum=0foriinrange(LOOP):env.render(mode='human')actions=[random.random()for_inrange(7)]observation,reward,done,info=env.step(actions)reward_sum+=rewardifdone:env.render()breakprint(reward_sum)The MiniDungeons Gym EnvironmentOverviewClickherefor aGetting Started with Gymoverview.ActionsAn action within the gym-md environment (env) is represented as a pythonlistcontaining seven (7) floating point values, for example:actions_eg=[0.7603953105618472,0.954037518265538,0.7224447519623062,0.35121023208759905,0.4878166326111911,0.6166020008598004,0.48734265188517545]Each index in theactionslist corresponds to a specific action available for the game agent to take:index 0: Head to the monsterindex 1: Head to the treasureindex 2: Head to the treasure (avoid monsters)index 3: Head to the potionindex 4: Head to the potion (avoid monsters)index 5: Head to the exitindex 6: Head to the exit (avoid monsters)The environment (env) selects the action, within the action float list, which has the highest value.In theactions_eglist example, the action with the highest value is 'Head to the treasure'
(which is index 1, with a value of 0.954037518265538).
However, if the selected highest action is not a valid action within the given state, then the next highest action value is taken.In theactions_eglist example, if the original highest action 'Head to the treasure' (index 1, with a value of 0.954037518265538) cannot be performed (e.g. there is no more treasure to collect) then the next highest action 'Head to the monster'
(which is index 0, with a value of 0.7603953105618472) is chosen. This action selection process is repeated until a valid action can be
performed within the given state. Furthermore, if the desired values are the same, an action is randomly selected.Environmentenv objectTheenvobject created usingenv = gym.make('md-test-v0')is based on theMdEnvBaseclass defined withinmd_env.py.
Theenvobject contains several objects and methods (only a subset is discussed here, please seemd_env.pyfor more).
Theenvobject contains:The current stage nameenv.stage_nameAn internal settings object of theSettingclass defined withinsetting.py. This object contains all the stage specific information and configuration.env.settinghelpful stage specific info can be accessed within the setting object:env.setting.PLAYER_MAX_HP: agent's max hit points.env.setting.REWARDS: reward values obtained.env.setting.ACTIONS: actions available in the stage.For the full list of setting values please seesetting.py.An internal grid object of theGridclass defined withingrid.py. This grid object is a grid world representation of the world stage.env.gridhelpful grid world info can be accessed within the grid object:env.grid.H: grid world's height.env.grid.W: grid world's width.env.grid.g: grid world represented as a 2D list.For the full list of attributes please seegrid.py.An internal agent object of theAgentclass defined withinagent.py. When an action is passed into the env.step method, this internal agent object is used to carry out the action.env.agenthelpful agent info can be accessed within the agent object:env.agent.hp: agent's hit points.env.agent.y: agent's y-position in grid world.env.agent.x: agent's x-position in grid world.For the full list of attributes please seeagent.py.TheOpenAI Gym environmentspecific methods are discussed as part of theenv.step methodsubsection.env.step methodThe gym-md environment's step method returns the following values:observation,reward,done,info=env.step(actions)observation (list): the resultant observation of the environment is represented as a list of integers of length eight (8).an example output observation is:[4,4,4,10,1000,8,8,30]each index in theobservationlist represents the following:index 0: Distance to the monsterindex 1: Distance to the treasureindex 2: Distance to treasure (avoid monsters)index 3: Distance to potionindex 4: Distance to potion (avoid monsters)index 5: Distance to the exitindex 6: Distance to the exit (avoid monsters)index 7: Agent's physical strength (i.e.Hit Points (HP))some of the observation values received is level specific and can be adjusted/set by modifying the respective level.jsonfile, found within thepropsfolder.for example, the 1000 value for index 4 (Distance to potion) corresponds to the "DISTANCE_INF" value set within thetest.jsonlevel file.reward (float): amount of reward achieved by the previous action.the reward received is level specific and can be adjusted/set by modifying the respective level.jsonfile, found within thepropsfolder.for example, the rewards set in the test level (test.json) are:"REWARDS":{"TURN":1,"EXIT":20,"KILL":4,"TREASURE":3,"POTION":1,"DEAD":-20}done (boolean): indicated whether the episode has terminated or not. In other words,Trueindicates 'yes, the level has ended', whilstFalseindicates 'no, the level has not yet ended'.info (dict): a dictionary containing play data, each action taken by the agent is tracked by this dictionary.an example info output is:defaultdict(<class'int'>, {'POTION': 7, '.': 38, 'MONSTER': 9, 'EXIT_SAFELY': 4, 'TREASURE': 3, 'TREASURE_SAFELY': 9, 'EXIT': 6})Levels and SettingsNew levels can be created by creating your own class which inherits from theMdEnvBaseclass. The below example, along with others, can be found within themd_env_list.pyscript.fromtypingimportFinalfromgym_md.envs.md_envimportMdEnvBaseclassTestMdEnv(MdEnvBase):"""TestMdEnv Class."""def__init__(self):stage_name:Final[str]="test"super(TestMdEnv,self).__init__(stage_name=stage_name)You will also need to set the respective level's json file and txt file, found respectively within thepropsandstagesfolders.
Furthermore, you will then need to add the additional levels to thegym_md.__init__.pyandgym_md.envs.__init__.pyfiles.For a list of the current available gym_md environmentlevels, please seeStages README.md. For a simpler level list view please seeenv_levels.txt.
Both theStages README.mdandenv_levels.txtfiles contains the env levels registered within thegym_md.__init__.py.