Repository: openai/gym

Issue:
MuJoCo render incompatible with newer versions of MuJoCo Py
MujucoEnv's render method assumes the viewer has a 500x500 display. If it is larger than this, it will select pixels from the bottom-left corner when called with `mode='rgb_array'`. There's a comment which suggests this size used to be the default for mujoco-py, but with the latest mujoco-py it defaults to the screen resolution: see opengl_context.pyx:89 in _create_window.

This issue can be fixed by adding the line `self.viewer.opengl_context.set_buffer_size(W, H)` in `_get_viewer` to manually specify the desired frame size. (Oddly, I need to set both dimensions to twice the desired size, I think due to an overzealous workaround for Mac's on opengl_context.pyx:113.) But this is pretty hacky. 

I'm a little confused why we're rendering anything on the screen for `render(mode='rgb_array')`, which is typically used for video capture in a batch setting. In particular, MjViewer has the annoyance of showing an overlay menu by default, rarely desired for video capture. It might be better to use `mujoco_py.MjRenderContextOffscreen` instead for this purpose. If we allow render to be called with different modes, however, this would require making a breaking change to the interface for `viewer_setup` (or some hacking to make `self.viewer` point to different objects depending on the last call).

I'd be happy to work on a pull request to address these issues if we can agree on a good way forward for this.
MuJoCo render incompatible with newer versions of MuJoCo Py
MujucoEnv's render method assumes the viewer has a 500x500 display. If it is larger than this, it will select pixels from the bottom-left corner when called with `mode='rgb_array'`. There's a comment which suggests this size used to be the default for mujoco-py, but with the latest mujoco-py it defaults to the screen resolution: see opengl_context.pyx:89 in _create_window.

This issue can be fixed by adding the line `self.viewer.opengl_context.set_buffer_size(W, H)` in `_get_viewer` to manually specify the desired frame size. (Oddly, I need to set both dimensions to twice the desired size, I think due to an overzealous workaround for Mac's on opengl_context.pyx:113.) But this is pretty hacky. 

I'm a little confused why we're rendering anything on the screen for `render(mode='rgb_array')`, which is typically used for video capture in a batch setting. In particular, MjViewer has the annoyance of showing an overlay menu by default, rarely desired for video capture. It might be better to use `mujoco_py.MjRenderContextOffscreen` instead for this purpose. If we allow render to be called with different modes, however, this would require making a breaking change to the interface for `viewer_setup` (or some hacking to make `self.viewer` point to different objects depending on the last call).

I'd be happy to work on a pull request to address these issues if we can agree on a good way forward for this.


Discussion:
A more serious issue is that close() calls the method finish() on the viewer which no longer exists in the latest mujoco_py version. To reproduce:

1. Install latest version of mujoco-py (1.50.1.56 from PyPi, same as Git master)
2. Run:
```
import gym                                                                                                                                                                                            

env = gym.make('Reacher-v2')
ob = env.reset()
env.step([0.0, 0.0])
env.render()
env.close()
```
3. Output:
```
Creating window glfw                                                                                                                                                                                  
Traceback (most recent call last):                                                                                                                                                                    
  File "test_mjc.py", line 8, in <module>
    env.close()
  File "/home/adam/dev/gym/gym/core.py", line 288, in close
    return self.env.close()
  File "/home/adam/dev/gym/gym/envs/mujoco/mujoco_env.py", line 115, in close
    self.viewer.finish()
AttributeError: 'MjViewer' object has no attribute 'finish
```

I assume there's some version of mujoco-py where this all worked -- we could perhaps pin the Gym requirements to that version as a workaround, while a more permanent fix is developed?
A more serious issue is that close() calls the method finish() on the viewer which no longer exists in the latest mujoco_py version. To reproduce:

1. Install latest version of mujoco-py (1.50.1.56 from PyPi, same as Git master)
2. Run:
```
import gym                                                                                                                                                                                            

env = gym.make('Reacher-v2')
ob = env.reset()
env.step([0.0, 0.0])
env.render()
env.close()
```
3. Output:
```
Creating window glfw                                                                                                                                                                                  
Traceback (most recent call last):                                                                                                                                                                    
  File "test_mjc.py", line 8, in <module>
    env.close()
  File "/home/adam/dev/gym/gym/core.py", line 288, in close
    return self.env.close()
  File "/home/adam/dev/gym/gym/envs/mujoco/mujoco_env.py", line 115, in close
    self.viewer.finish()
AttributeError: 'MjViewer' object has no attribute 'finish
```

I assume there's some version of mujoco-py where this all worked -- we could perhaps pin the Gym requirements to that version as a workaround, while a more permanent fix is developed?

Resolving patch:
diff --git a/gym/envs/mujoco/mujoco_env.py b/gym/envs/mujoco/mujoco_env.py
--- a/gym/envs/mujoco/mujoco_env.py
+++ b/gym/envs/mujoco/mujoco_env.py
@@ -12,6 +12,8 @@
 except ImportError as e:
     raise error.DependencyNotInstalled("{}. (HINT: you need to install mujoco_py, and also perform the setup instructions here: https://github.com/openai/mujoco-py/.)".format(e))
 
+DEFAULT_SIZE = 500
+
 class MujocoEnv(gym.Env):
     """Superclass for all MuJoCo environments.
     """
@@ -28,6 +30,7 @@ def __init__(self, model_path, frame_skip):
         self.sim = mujoco_py.MjSim(self.model)
         self.data = self.sim.data
         self.viewer = None
+        self._viewers = {}
 
         self.metadata = {
             'render.modes': ['human', 'rgb_array'],
@@ -78,8 +81,11 @@ def viewer_setup(self):
     def reset(self):
         self.sim.reset()
         ob = self.reset_model()
-        if self.viewer is not None:
+        old_viewer = self.viewer
+        for v in self._viewers.values():
+            self.viewer = v
             self.viewer_setup()
+        self.viewer = old_viewer
         return ob
 
     def set_state(self, qpos, qvel):
@@ -99,26 +105,31 @@ def do_simulation(self, ctrl, n_frames):
         for _ in range(n_frames):
             self.sim.step()
 
-    def render(self, mode='human'):
+    def render(self, mode='human', width=DEFAULT_SIZE, height=DEFAULT_SIZE):
         if mode == 'rgb_array':
-            self._get_viewer().render()
+            self._get_viewer(mode).render(width, height)
             # window size used for old mujoco-py:
-            width, height = 500, 500
-            data = self._get_viewer().read_pixels(width, height, depth=False)
+            data = self._get_viewer(mode).read_pixels(width, height, depth=False)
             # original image is upside-down, so flip it
             return data[::-1, :, :]
         elif mode == 'human':
-            self._get_viewer().render()
+            self._get_viewer(mode).render()
 
     def close(self):
         if self.viewer is not None:
             # self.viewer.finish()
             self.viewer = None
+            self._viewers = {}
 
-    def _get_viewer(self):
+    def _get_viewer(self, mode):
+        self.viewer = self._viewers.get(mode)
         if self.viewer is None:
-            self.viewer = mujoco_py.MjViewer(self.sim)
+            if mode == 'human':
+                self.viewer = mujoco_py.MjViewer(self.sim)
+            elif mode == 'rgb_array':
+                self.viewer = mujoco_py.MjRenderContextOffscreen(self.sim, 0)
             self.viewer_setup()
+            self._viewers[mode] = self.viewer
         return self.viewer
 
     def get_body_com(self, body_name):


Test patch: