Package: gym-contra
Gym for ContraAnOpenAIGym environment for Contra.  on The Nintendo Entertainment System (NES) using thenes-py emulator.Project addressInstallationThe preferred installation of Contra is from pip:pipinstallgym-contraUsagePythonYou must import ContraEnv before trying to make an environment. This is because gym environments are registered at runtime. By default, ContraEnv use the full NES action space of 256 discrete actions. To contstrain this,ContraEnv.actions provides three actions lists (RIGHT_ONLY, SIMPLE_MOVEMENT, and COMPLEX_MOVEMENT) for the nes_py.wrappers.JoypadSpace wrapper. SeeContra/actions.pyfor a breakdown of the legal actions in each of these three lists.fromnes_py.wrappersimportJoypadSpaceimportgymfromContra.actionsimportSIMPLE_MOVEMENT,COMPLEX_MOVEMENT,RIGHT_ONLYenv=gym.make('Contra-v0')env=JoypadSpace(env,RIGHT_ONLY)print("actions",env.action_space)print("observation_space ",env.observation_space.shape[0])done=Falseenv.reset()forstepinrange(5000):ifdone:print("Over")breakstate,reward,done,info=env.step(env.action_space.sample())env.render()env.close()NOTE: ContraEnv.make is just an alias to gym.make for convenience.NOTE: remove calls to render in training code for a nontrivial speedup.Command LinePrepare to write please waitNOTE: by default,-m is set to human.EnvironmentsThese environments allow 3 attempts (lives) to play in the game. The environments only send reward-able game-play frames to agents; No cut-scenes, loading screens, etc. are sent from the NES emulator to an agent nor can an agent perform actions during these instances. If a cut-scene is not able to be skipped by hacking the NES's RAM, the environment will lock the Python process until the emulator is ready for the next action.StepInfo about the rewards and info returned by the step method.Reward FunctionThe reward function assumes the objective of the game is to move as far right as possible (increase the agent's x value), as fast as possible, without dying. To model this game, three separate variables compose the reward:v: the difference in agent x values between statesin this case this is instantaneous velocity for the given stepv = x1 - x0x0 is the x position before the stepx1 is the x position after the stepmoving right ⇔ v > 0moving left ⇔ v < 0not moving ⇔ v = 0d: a death penalty that penalizes the agent for dying in a statethis penalty encourages the agent to avoid deathalive ⇔ d = 0dead ⇔ d = -15b : if the agent defeated the bossthis reword will encourages the agent to defeat boss as possibleno defeated ⇔ 0defeated ⇔ 30So the reward function is:r = v + d + bNote:The reward is clipped into the range (-15, 15).info dictionaryThe info dictionary returned by the step method contains the following keys:life=self._life,dead=self._is_dead,done=self._get_done(),score=self._score(),status=self._player_state,x_pos=self._x_position,y_pos=self._y_position,defeated=self._get_boss_defeated,KeyTypeDescriptionlifeintThe number of lives left, i.e., {3, 2, 1}deadBoolGet The palyer is deaddoneBoolGet the game is game overscoreintGet the player's scorestatusBoolAlive Status (00 - Dead, 01 - Alive, 02 - Dying)x_posintPlayer's x position in the stage (from the left)y_posintPlayer's y position in the stage (from the bottom)defeatedBoolself._get_boss_defeated