Package: gym-notebook-wrapper
Gym-Notebook-WrapperGym-Notebook-Wrapper provides small wrappers for running and renderingOpenAI GymandBraxonJupyter
Notebookor similar (e.g.Google
Colab).1. RequirementLinuxXvfb(for Gym)On Ubuntu, you can installsudo apt update && sudo apt install xvfb.Open GL (for some environment)On Ubuntu, you can installsudo apt update && sudo apt install python3-opengl2. InstallationYou can install fromPyPIwithpip install gym-notebook-wrapper3. Rendering GymWarningGym has changed its API.
For example, until v0.25.2env.step(action)returns 4 values,
but from v0.26.0 it returns 5 values. (donewas divided toterminationandtruncation.)Three classes are implemented ingnwrappermodule in this
gym-notebook-wrapper package.3.1 Simple One Shot AnimationWrapgym.Envclass withgnwrapper.Animation. That's all! Therender()method shows the environment on its output. An example code
is following;3.1.1 Codeimportgnwrapperimportgymenv=gnwrapper.Animation(gym.make('CartPole-v1',render_mode="rgb_array"))obs=env.reset()for_inrange(1000):next_obs,reward,term,trunc,info=env.step(env.action_space.sample())env.render()obs=next_obsiftermortrunc:obs=env.reset()3.1.2 LimitationCallingrender()method delete the other output for the same cell.The output image is shown only once.3.2 Loop AnimationWrapgym.Envclass withgnwrapper.LoopAnimation. This wrapper
stores display image whenrender()methos is called and shows the
loop animation bydisplay(dpi=72,interval=50)methos.3.2.1 Codeimportgnwrapperimportgymenv=gnwrapper.LoopAnimation(gym.make('CartPole-v1',render_mode="rgb_array"))obs=env.reset()for_inrange(100):next_obs,reward,term,trunc,info=env.step(env.action_space.sample())env.render()obs=next_obsiftermortrunc:obs=env.reset()env.display()3.2.2 LimitationRequire a lot of memory to store and display large steps of displayCan raise memory error3.3 Movie AnimationWrapgym.Envclass withgnwrapper.Monitor. This wrapper inheritsgym.wrappers.Monitor(forgym<=0.19.0) orgym.wrapper.RecordVideo(forgym>=0.20.0), and implementsdisplay()method for embedding mp4
movie into Notebook.Note:gym.wrappers.Monitorwas deprecated atgym==0.20.0, which
was released on 14th
September 2021.See.If you calldisplay(reset=True), the video list is cleared and the
nextdisplay()method shows only new videos.3.3.1 Codeimportgnwrapperimportgymenv=gnwrapper.Monitor(gym.make('CartPole-v1',render_mode="rgb_array"),directory="./")o=env.reset()for_inrange(100):o,r,term,trunc,i=env.step(env.action_space.sample())iftermortrunc:env.reset()env.display()3.3.2 LimitationRequire disk space for save movie3.4 Notesgnwrapper.Animationandgnwrapper.LoopAnimationinherit fromgym.Wrapper, so that it can access any fields or mothods ofgym.Envandgym.Wrapper(e.g.action_space).4. Rendering BraxBrax has HTML rendering inbrax.io.html. We provide small wrapper
classes to record episodes automatically and to display on Jupyter
Notebook easily.Two classes are implemented ingnwrapper.braxmodule. Since this
module requiresbraxpackage, the statementimport gnwrapperdoesn't importgnwrapper.braxsubmodule. You must explicitly import
it byimport gnwrapper.braxorfrom gnwrapper import braxetc.4.1 HTML Viewer with Brax Native EnvironmentWrapbrax.envs.Envwithgnwrapper.brax.BraxHTML.step()method
automatically stores an episode, and saves it as html file at the
episode end. You can embeds HTML viewer by callingdisplay()method. Of cource, you can open the html file with your local browser
as long as you have internet access. (Data is saved in the html file,
however, the viewer is hosted on CDN.)Since this wrapper has Python side effect, you cannot wrapstep()/reset()methods withjax.jit. Insted, you can wrap internal
(original)step()/reset()methods by settingjit=Trueat the
wrapper constructor.4.1.1 Codefrombraximportenvsimportbrax.jumpyasjpfromgnwrapper.braximportBraxHTMLrng=jp.random_prngkey(seed=42)ant=BraxHTML(envs.create("ant",auto_reset=False),video_callable=lambdaep:True)foriinrange(2):rng,rng_use=jp.random_split(rng)state=ant.reset(rng_use)whileTrue:rng,rng_use=jp.random_split(rng)state=ant.step(state,jp.random_uniform(rng_use,(ant.action_size,)))ifstate.done:# When `state.done = True`, the episode is written at html file.break# We can get list of recorded episodes.episodes=ant.recorded_episodes()# `display()` method show all recorded episodes.# `display(1)` shows only episode 1, if it is recorded# `display([1, 2])` shows episode 1 & 2, if they are recorded, etc.ant.display()4.1.2 ParametersArgumentTypeDescriptionenvbrax.envs.EnvEnvironmentdirectory=NoneOptional[str]Directory to store html. IfNone(default), time stamp ("%Y%m%d-%H%M%S") is used.heght=480intViewer height in px. (There is a Brax bug (this issue), however, PR was merged.)video_callable=NoneOptional[Callable[[int], bool]]Function to determine whether each episode is recorded or not. IfNone(default), every 1000 and cubic number less than 1000 are recordedjit=TrueboolWhetherstep/resetmethods will be wapped byjax.jit4.2 HTML Viewer with Gym compatible Brax EnvironmentWrapbrax.wrappers.GymWrapperwithgnwrapper.brax.GymHTML.step()method automatically stores an
episode, and saves it as html file at the episode end. You can embeds
HTML viewer by callingdisplay()method. Of cource, you can open the
html file with your local browser as long as you have internet
access. (Data is saved in the html file, however, the viewer is hosted
on CDN.)Sincebrax.wrapper.GymWrapperalready wrapsstep()/reset()methods withjax.jit, we don't provide functionality to wrapjax.jitagain.4.2.1 Codefrombraximportenvsimportbrax.jumpyasjpfromgnwrapper.braximportGymHTMLrng=jp.random_prngkey(seed=42)rng,rng_use=jp.random_split(rng)ant=GymHTML(envs.create_gym_env("ant",auto_reset=False,seed=0),video_callable=lambdaep:True)foriinrange(2):obs=ant.reset()whileTrue:rng,rng_use=jp.random_split(rng)obs,rew,done,_=ant.step(jp.random_uniform(rng_use,ant.action_space.shape))ifdone:# When `done = True`, the episode is written at html file.break# We can get list of recorded episodes.episodes=ant.recorded_episodes()# `display()` method show all recorded episodes.# `display(1)` shows only episode 1, if it is recorded# `display([1, 2])` shows episode 1 & 2, if they are recorded, etc.ant.display()4.2.2 ParametersArgumentTypeDescriptionenvbrax.envs.EnvEnvironmentdirectory=NoneOptional[str]Directory to store html. IfNone(default), time stamp ("%Y%m%d-%H%M%S") is used.heght=480intViewer height in px. (There is a Brax bug (this issue), however, PR was merged.)video_callable=NoneOptional[Callable[[int], bool]]Function to determine whether each episode is recorded or not. IfNone(default), every 1000 and cubic number less than 1000 are recorded4.3 LimitationSincedoneis alwaysFalse, auto reset
(aka.brax.envs.wrappers.AutoResetWrapper) is not supported. You
must callbrax.envs.create()orbrax.envs.create_gym_env()withauto_reset=Falseargument.Vectorized (batched) environments
(aka.brax.envs.wrappers.VectorWrapper,brax.envs.wrappers.GymVectorWrapper) are not supported, too. You
should not specifybatch_sizeargument atbrax.envs.create()orbrax.envs.create_gym_env().5. LinksRepository at GitHub