Repository: Farama-Foundation/Metaworld
Issue: Remote rendering issue with env.render(mode='rgb_array') and env.get_image()
Root cause: Environment Incompatibility

This is an issue I typically have with Mujoco-based simulations when running **remotely**. Basically rendering is problematic, even in `rgb_array` mode where we want to access the image frames.

Is there a workaround for this issue? The OpenAI gym envs have the same issue and I only know [DMSuite](https://github.com/deepmind/dm_control) somehow bypasses this - where I can render in `rgb_array` mode. 

So the errors you would get are the followings : 
- `env.render(mode='rgb_array')` 
```
GLFW error (code %d): %s 65542 b'EGL: Failed to get EGL display: Success'
Creating window glfw
X Error of failed request:  255
  Major opcode of failed request:  155 (GLX)
  Minor opcode of failed request:  5 (X_GLXMakeCurrent)
  Serial number of failed request:  136
  Current serial number in output stream:  137
```
- `env.get_image(width=84, height=84)`
```
ERROR: GLEW initalization error: Missing GL version
```

Are you rendering environments on a machine without a display (i.e. headless)?

If so, the way we get usually get around this limitation with gym (mujoco_py) is to introduce a fake display. This gives OpenGL an in-memory frame buffer to which it can render frames, so that they can later be read out when you call `env.render(mode='rgb_array')`

Our [Dockerfile](https://github.com/rlworkgroup/metaworld/tree/master/docker) does exactly this so that we can run tests with rendering on the CI servers.

Some options, you can...:
* ...use our Dockerfiles directly by calling `make run` from the root of the repository.
* ...create your own Dockerfile derived from ours, which is based on the [Dockerfiles from garage](https://github.com/rlworkgroup/garage/blob/master/docker/Dockerfile.headless)
* ...use headless rendering directly on your machine by installing `xvfb` and then calling your entrypoint with `xvfb-run` (see [this example](https://github.com/rlworkgroup/garage/blob/master/docker/entrypoint-headless.sh) from the Dockerfiles)

Note:
dm_control avoid this limitation by having a more sophisticated rendering backend, which choose a different rendering driver system if it detects the machine is headless.

@melfm let me know if this answered your question :)

@ryanjulian Thank you for your response. Yes I'm trying to get the image frames on a remote machine, so rendering headless.
`xvfb-run` didn't work in my case, so I'm trying with Dockerfiles. 
Btw I still had issues rendering on the PC directly with `env.render(mode='rgb_array')` although the human render mode works fine as well as the `get_image` API which I assume is a better way of accessing the frames.


So now with dockerfiles, starting from `rlworkgroup/garage-headless` and including the `entrypoint-headless.sh`, I can build the docker by running `sudo make build-headless` but when I try to run the script with the command `sudo make run-headless RUN_CMD="python scripts/demo_sawyer.py"`, I still get the `Missing GL version` error (This is the modified script that is calling `env.get_image(width=84, height=84)`). The entry point script should take care of the env setup, do you have any suggestions? 


@melfm as a quick check i edited `scripts/profile_memory_usage.py` with the following snippet:

```py
def build_and_step(env_cls):
    env = env_cls()
    env.reset()
    step_env(env, max_path_length=1000, iterations=10)
    print(env)
    img = env.get_image(width=84, height=84)
    assert img is not None
    return env
```

This runs fine when I run `make check-memory`, which runs this script using the Dockerfiles in `metaworld` (which are basically the same as the ones from `garage`). 

I recommend you write your own stripped-down script and try again using the `metaworld` Dockerfiles, since that demo script needs some updating. The garage Dockerfiles are really only designed to work from within the `garage` directory so I'm unsure what will happen if you are trying to run them from somewhere else.

Long-term for your project, if you are going to take this route, I recommend you basically copy the `docker/` directory and `Makefile` from garage and use it as a template for a project-specific Dockerfile. 

If you are still having trouble, it would be helpful if you sent us a link to a branch on Github to try and some instructions for running your experiment. 

@melfm I'm closing this issue, but please let us know if you need further help.

I have similar problem. But there is nothing to render in my case.
```
import metaworld
import random

print(metaworld.ML1.ENV_NAMES)  # Check out the available environments

ml1 = metaworld.ML1('pick-place-v2') # Construct the benchmark, sampling tasks

env = ml1.train_classes['pick-place-v2']()  # Create an environment with task `pick_place`
task = random.choice(ml1.train_tasks)
env.set_task(task)  # Set task

obs = env.reset()  # Reset environment
a = env.action_space.sample()  # Sample an action
obs, reward, done, info = env.step(a)  # Step the environoment with the sampled random action
```

`env.render(mode='rgb_array')`

-> `TypeError: render() got an unexpected keyword argument 'mode'`

`env.get_image`
-> `AttributeError: 'SawyerPickPlaceEnvV2' object has no attribute 'get_image'`




I found how to solve this problem
```
import matplotlib.pyplot as plt
plt.imshow(env.render(offscreen=True))
```