Package: gym-super-mario-bros
gym-super-mario-brosAnOpenAI Gymenvironment for
Super Mario Bros. & Super Mario Bros. 2 (Lost Levels) on The Nintendo
Entertainment System (NES) usingthe nes-py emulator.InstallationThe preferred installation ofgym-super-mario-brosis frompip:pipinstallgym-super-mario-brosUsagePythonYou must importgym_super_mario_brosbefore trying to make an environment.
This is because gym environments are registered at runtime. By default,gym_super_mario_brosenvironments use the full NES action space of 256
discrete actions. To contstrain this,gym_super_mario_bros.actionsprovides
three actions lists (RIGHT_ONLY,SIMPLE_MOVEMENT, andCOMPLEX_MOVEMENT)
for thenes_py.wrappers.JoypadSpacewrapper. Seegym_super_mario_bros/actions.pyfor a
breakdown of the legal actions in each of these three lists.fromnes_py.wrappersimportJoypadSpaceimportgym_super_mario_brosfromgym_super_mario_bros.actionsimportSIMPLE_MOVEMENTenv=gym_super_mario_bros.make('SuperMarioBros-v0')env=JoypadSpace(env,SIMPLE_MOVEMENT)done=Trueforstepinrange(5000):ifdone:state=env.reset()state,reward,done,info=env.step(env.action_space.sample())env.render()env.close()NOTE:gym_super_mario_bros.makeis just an alias togym.makefor
convenience.NOTE:remove calls torenderin training code for a nontrivial
speedup.Command Linegym_super_mario_brosfeatures a command line interface for playing
environments using either the keyboard, or uniform random movement.gym_super_mario_bros-e<theenvironmentIDtoplay>-m<`human`or`random`>NOTE:by default,-eis set toSuperMarioBros-v0and-mis set tohuman.NOTE:SuperMarioBrosRandomStages-*support the--stages/-Sflag for
supplying the set of stages to sample from like-S 1-4 2-4 3-4 4-4.EnvironmentsThese environments allow 3 attempts (lives) to make it through the 32 stages
in the game. The environments only send reward-able game-play frames to
agents; No cut-scenes, loading screens, etc. are sent from the NES emulator
to an agent nor can an agent perform actions during these instances. If a
cut-scene is not able to be skipped by hacking the NES's RAM, the environment
will lock the Python process until the emulator is ready for the next action.EnvironmentGameROMScreenshotSuperMarioBros-v0SMBstandardSuperMarioBros-v1SMBdownsampleSuperMarioBros-v2SMBpixelSuperMarioBros-v3SMBrectangleSuperMarioBros2-v0SMB2standardSuperMarioBros2-v1SMB2downsampleIndividual StagesThese environments allow a single attempt (life) to make it through a single
stage of the game.Use the templateSuperMarioBros-<world>-<stage>-v<version>where:<world>is a number in {1, 2, 3, 4, 5, 6, 7, 8} indicating the world<stage>is a number in {1, 2, 3, 4} indicating the stage within a world<version>is a number in {0, 1, 2, 3} specifying the ROM mode to use0: standard ROM1: downsampled ROM2: pixel ROM3: rectangle ROMFor example, to play 4-2 on the downsampled ROM, you would use the environment
idSuperMarioBros-4-2-v1.Random Stage SelectionThe random stage selection environment randomly selects a stage and allows a
single attempt to clear it. Upon a death and subsequent call toresetthe
environment randomly selects a new stage. This is only available for the
standard Super Mario Bros. game,notLost Levels (at the moment). To use
these environments, appendRandomStagesto theSuperMarioBrosid. For
example, to use the standard ROM with random stage selection useSuperMarioBrosRandomStages-v0. To seed the random stage selection use theseedmethod of the env, i.e.,env.seed(222), before any calls toreset.
Alternatively pass theseedkeyword argument to theresetmethod directly
likereset(seed=222).In addition to randomly selecting any of the 32 original stages, a subset of
user-defined stages can be specified to limit the random choice of stages to a
specific subset. For example, the stage selector could be limited to only
sample castle stages, water levels, underground, and more.To specify a subset of stages to randomly sample from, create a list of each
stage to allow to be sampled and pass that list to thegym.make()function.
For example:gym.make('SuperMarioBrosRandomStages-v0',stages=['1-4','2-4','3-4','4-4'])The example above will sample a random stage from 1-4, 2-4, 3-4, and 4-4 upon
every call toreset.StepInfo about the rewards and info returned by thestepmethod.Reward FunctionThe reward function assumes the objective of the game is to move as far right
as possible (increase the agent'sxvalue), as fast as possible, without
dying. To model this game, three separate variables compose the reward:v: the difference in agentxvalues between statesin this case this is instantaneous velocity for the given stepv = x1 - x0x0is the x position before the stepx1is the x position after the stepmoving right ⇔v > 0moving left ⇔v < 0not moving ⇔v = 0c: the difference in the game clock between framesthe penalty prevents the agent from standing stillc = c0 - c1c0is the clock reading before the stepc1is the clock reading after the stepno clock tick ⇔c = 0clock tick ⇔c < 0d: a death penalty that penalizes the agent for dying in a statethis penalty encourages the agent to avoid deathalive ⇔d = 0dead ⇔d = -15r = v + c + dThe reward is clipped into the range(-15, 15).infodictionaryTheinfodictionary returned by thestepmethod contains the following
keys:KeyTypeDescriptioncoinsintThe number of collected coinsflag_getboolTrue if Mario reached a flag or axlifeintThe number of lives left, i.e.,{3, 2, 1}scoreintThe cumulative in-game scorestageintThe current stage, i.e.,{1, ..., 4}statusstrMario's status, i.e.,{'small', 'tall', 'fireball'}timeintThe time left on the clockworldintThe current world, i.e.,{1, ..., 8}x_posintMario'sxposition in the stage (from the left)y_posintMario'syposition in the stage (from the bottom)CitationPlease citegym-super-mario-brosif you use it in your research.@misc{gym-super-mario-bros,
  author ={Christian Kauten},
  howpublished ={GitHub},
  title ={{S}uper{M}ario{B}ros for{O}pen{AI}{G}ym},
  URL ={https://github.com/Kautenja/gym-super-mario-bros},
  year ={2018},}