Repository: Farama-Foundation/Metaworld
Issue: Gymnasium AsyncVectorEnv for individual metaworld env
Root cause: Resource Mishandling

as title, when I try to create a async vec env wrapper for metaworld 2.0.0 version, it failed because `gymnasium.envs.mujoco.mujoco_rendering` involves 
```
def _import_egl(width, height):
    from mujoco.egl import GLContext
    return GLContext(width, height)
```
in which contains
```
class GLContext:
  """An EGL context for headless accelerated OpenGL rendering on GPU devices."""

  def __init__(self, max_width, max_height):
    del max_width, max_height  # unused
    num_configs = ctypes.c_long()           <---- ctypes
    config_size = 1
    config = EGL.EGLConfig()
    EGL.eglReleaseThread()
    EGL.eglChooseConfig(
        EGL_DISPLAY,
        EGL_ATTRIBUTES,
        ctypes.byref(config),
        config_size,
        num_configs)
    if num_configs.value < 1:
 
[skip]
```
Therefore, we got error from `multiprocessing`, saying that `ValueError: ctypes objects containing pointers cannot be pickled`, which then cause EGL / viewers / renderer breaks. When I use SyncVectorEnv as the wrapper, everything is fine. 

Can ppl provide me a way to async vectorize individual env in metaworld? Thanks.


Do you need to use async and not sync? If so please raise the error in gymnasium not meta world as we need to fix it there

yes, I need async. I'll raise a new issue in gymnasium. Thanks

Just FYI, so that maybe this is not a problem with gymnasium but is a mistake I made, here's the minimal code to reproduce the error I got:

```
import metaworld
import gymnasium
from gymnasium.envs.mujoco.mujoco_rendering import MujocoRenderer
from gymnasium.vector.async_vector_env import AsyncVectorEnv


task_name = 'shelf-place-v2-goal-observable'
seed = 42
n_env = 2

class MetaworldEnv(gymnasium.Env):
    metadata = {'render_modes': ['rgb_array', 'depth_array']}

    def __init__(self):
        self.env = metaworld.envs.ALL_V2_ENVIRONMENTS_GOAL_OBSERVABLE[task_name](seed=seed)
        self.observation_space = self.env.observation_space
        self.action_space = self.env.action_space
        self.corner_renderer = MujocoRenderer(
            self.env.model, self.env.data, None, 
            128, 128, 1000, None, 'corner', {}
        )
        self.render_mode = 'rgb_array'

    def _get_rgb(self):
        # NOTE: I'm using more than one camera in actual implementation,
        # here is just for error reproduction, so I'm using only one camera
        return {
            'default': self.env.mujoco_renderer.render('rgb_array'),    # <-- This is the default metaworld
            'corner': self.corner_renderer.render('rgb_array'),         #     renderer, and it also cannot
        }                                                               #     work. Mine too.

    def reset(self, **kwargs):
        self.env.reset()
        self.env.reset_model()
        state = self.env.reset()
        obs_dict = self._get_rgb()      # <-- This is the line that causes the error
        obs_dict['full_state'] = state
        return obs_dict

def env_fn():
    return MetaworldEnv()

env = AsyncVectorEnv([env_fn for _ in range(n_env)])
env.reset()
```

And the error message I got:

```
Exception ignored in: <function GLContext.__del__ at 0x7ec273eb8430>
Traceback (most recent call last):
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/site-packages/mujoco/egl/__init__.py", line 130, in __del__
    self.free()
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/site-packages/mujoco/egl/__init__.py", line 120, in free
    if self._context:
AttributeError: 'GLContext' object has no attribute '_context'
Exception ignored in: <function OffScreenViewer.__del__ at 0x7ec37d226790>
Traceback (most recent call last):
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/site-packages/gymnasium/envs/mujoco/mujoco_rendering.py", line 202, in __del__
    self.free()
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/site-packages/gymnasium/envs/mujoco/mujoco_rendering.py", line 199, in free
    self.opengl_context.free()
AttributeError: 'OffScreenViewer' object has no attribute 'opengl_context'
Exception ignored in: <function GLContext.__del__ at 0x7ec273eb8430>
Traceback (most recent call last):
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/site-packages/mujoco/egl/__init__.py", line 130, in __del__
    self.free()
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/site-packages/mujoco/egl/__init__.py", line 120, in free
    if self._context:
AttributeError: 'GLContext' object has no attribute '_context'
Exception ignored in: <function OffScreenViewer.__del__ at 0x7ec37d226790>
Traceback (most recent call last):
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/site-packages/gymnasium/envs/mujoco/mujoco_rendering.py", line 202, in __del__
    self.free()
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/site-packages/gymnasium/envs/mujoco/mujoco_rendering.py", line 199, in free
    self.opengl_context.free()
AttributeError: 'OffScreenViewer' object has no attribute 'opengl_context'
^CTraceback (most recent call last):
  File "/home/chaoqi/Desktop/gym_meta_bug/reproduce.py", line 40, in <module>
    env.reset()
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/site-packages/gymnasium/vector/async_vector_env.py", line 226, in reset
    return self.reset_wait()
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/site-packages/gymnasium/vector/async_vector_env.py", line 299, in reset_wait
    self._raise_if_errors(successes)
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/site-packages/gymnasium/vector/async_vector_env.py", line 623, in _raise_if_errors
    index, exctype, value = self.error_queue.get()
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/queues.py", line 103, in get
    res = self._recv_bytes()
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/connection.py", line 216, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/connection.py", line 414, in _recv_bytes
    buf = self._recv(4)
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/connection.py", line 379, in _recv
    chunk = read(handle, remaining)
```

Are you using a system (ie a Ubuntu server) that doesn't have a display output by default? If so, this isn't an issue with Metaworld or Gymnasium. It's an issue with your server not having a display driver. You can look into EGL or osmesa rendering options. 

Hi, I have display output, and actually with metaworld 0.1.0 + gym + mujoco-py, I can display properly with EGL, and for macOS, I also have display, the error "AttributeError: 'OffScreenViewer' object has no attribute 'opengl_context'", is caused by pickle not able to handle ctype, and it breaks the instantiation of GLContext(width, height). You can refer to my initial comment, where during the instantiation of GLContext, ctype appears and cannot be handled.

It can be helpful if ppl try to run the code I provided, and see if they can make it through.

What operating system are you using? Your error trace seems to be from a Linux based system. If it's Linux, are you running a Ubuntu variant? Would it be the desktop or server Ubuntu variant? 

https://github.com/google-deepmind/mujoco/issues/991#issuecomment-1875307602

This is ubunbu 22.04, macOS 14.4.1, gymnasium 1.0.0a2, metaworld 2.0.0, mujoco 2.3.7.

Thanks for redirect me to this issue. I'm currently busy with some ddl. I'll try this fix and report how macOS and ubuntu w/ and w/o this `multiprocessing.set_start_method('spawn')` behave tomorrow. Thank you again. 🙂

I'm a bit confused about where to add this line `multiprocessing.set_start_method('spawn')`. I tried two places, a) inside async vec env file, I put this line at the very beginning, and b) the very beginning of the program / main. The two places both generated the exact same error. Here's the error I got

1) with `mp.set_start_method('spawn')` I observe
```
MetaworldEnv: task=basketball-v2-goal-observable, seed=None
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/spawn.py", line 125, in _main
    prepare(preparation_data)
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/runpy.py", line 288, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/chaoqi/Desktop/mrp-dev/test.py", line 12, in <module>
    multiprocessing.set_start_method('spawn')
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/context.py", line 243, in set_start_method
    raise RuntimeError('context has already been set')
RuntimeError: context has already been set
Traceback (most recent call last):
  File "/home/chaoqi/Desktop/mrp-dev/test.py", line 30, in <module>
    env = AsyncVectorEnv(env_fns)
  File "/home/chaoqi/Desktop/mrp-dev/modular_policy/gymnasium_util/async_vector_env.py", line 177, in __init__
    self._check_spaces()
  File "/home/chaoqi/Desktop/mrp-dev/modular_policy/gymnasium_util/async_vector_env.py", line 640, in _check_spaces
    pipe.send(("_check_spaces", spaces))
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/connection.py", line 405, in _send_bytes
    self._send(buf)
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
```

1) with `mp.set_start_method('spawn', force=True)` I observe
```
MetaworldEnv: task=basketball-v2-goal-observable, seed=None
MetaworldEnv: task=basketball-v2-goal-observable, seed=None
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/spawn.py", line 125, in _main
    prepare(preparation_data)
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/runpy.py", line 288, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/chaoqi/Desktop/mrp-dev/test.py", line 30, in <module>
    env = AsyncVectorEnv(env_fns)
  File "/home/chaoqi/Desktop/mrp-dev/modular_policy/gymnasium_util/async_vector_env.py", line 173, in __init__
    process.start()
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError: 
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
Traceback (most recent call last):
  File "/home/chaoqi/Desktop/mrp-dev/test.py", line 30, in <module>
    env = AsyncVectorEnv(env_fns)
  File "/home/chaoqi/Desktop/mrp-dev/modular_policy/gymnasium_util/async_vector_env.py", line 177, in __init__
    self._check_spaces()
  File "/home/chaoqi/Desktop/mrp-dev/modular_policy/gymnasium_util/async_vector_env.py", line 640, in _check_spaces
    pipe.send(("_check_spaces", spaces))
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/connection.py", line 405, in _send_bytes
    self._send(buf)
  File "/home/chaoqi/anaconda3/envs/mrp-beta/lib/python3.9/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
```

I'm on ubuntu 22.04, I'm using the display, sitting right in front of the computer. You might notice that I'm calling my customized async vector env, but that's just direct copy&paste from gymnasium 1.0.0a2 release.

I think you can fix both issues with the same fix. Move all of your non class code into a `if __name__ == "__main__":` block. Multiprocessing needs to create a complete copy of the context for your code to work. Right now everytime a new process is made it calls all of your non-class code. 

```
import multiprocessing as mp 
import metaworld
import gymnasium
from gymnasium.envs.mujoco.mujoco_rendering import MujocoRenderer
from gymnasium.vector.async_vector_env import AsyncVectorEnv

# class and function definitions from above code

if __name__ == "__main__":
    task_name = 'shelf-place-v2-goal-observable'
    seed = 42
    n_env = 2
    env = AsyncVectorEnv([env_fn for _ in range(n_env)])
    env.reset()
```

yes you are right, I was in a rush and didn't realize this was causing problem, moving everything to __main__ is the fix. Thanks, I'll close that issue in Gymnasium as well.

Same code as @Chaoqi-LIU 😭

But I got the error:

ERROR: Received the following error from Worker-1: ValueError: too many values to unpack (expected 2)
ERROR: Shutting down Worker-1.
ERROR: Received the following error from Worker-0: ValueError: too many values to unpack (expected 2)
ERROR: Shutting down Worker-0.
ERROR: Raising the last exception back to the main process.
Traceback (most recent call last):
  File "/root/autodl-tmp/openpi/examples/metaworld/test.py", line 45, in <module>
    env.reset()
  File "/root/autodl-tmp/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/gymnasium/vector/async_vector_env.py", line 216, in reset
    return self.reset_wait()
  File "/root/autodl-tmp/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/gymnasium/vector/async_vector_env.py", line 287, in reset_wait
    self._raise_if_errors(successes)
  File "/root/autodl-tmp/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/gymnasium/vector/async_vector_env.py", line 623, in _raise_if_errors
    raise exctype(value)
ValueError: too many values to unpack (expected 2)

> Same code as [@Chaoqi-LIU](https://github.com/Chaoqi-LIU) 😭
> 
> But I got the error:
> 
> ERROR: Received the following error from Worker-1: ValueError: too many values to unpack (expected 2) ERROR: Shutting down Worker-1. ERROR: Received the following error from Worker-0: ValueError: too many values to unpack (expected 2) ERROR: Shutting down Worker-0. ERROR: Raising the last exception back to the main process. Traceback (most recent call last): File "/root/autodl-tmp/openpi/examples/metaworld/test.py", line 45, in env.reset() File "/root/autodl-tmp/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/gymnasium/vector/async_vector_env.py", line 216, in reset return self.reset_wait() File "/root/autodl-tmp/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/gymnasium/vector/async_vector_env.py", line 287, in reset_wait self._raise_if_errors(successes) File "/root/autodl-tmp/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/gymnasium/vector/async_vector_env.py", line 623, in _raise_if_errors raise exctype(value) ValueError: too many values to unpack (expected 2)

gymnasium==1.0.0a1
metaworld is lastest

If you update to v1.0.0 or v1.1.0, does this fix it?

@AoqunJin use a sync vector env for debugging as the error messages are a bit more descriptive in my experience. Are you using the most recent commit of Meta-World?

@pseudo-rnd-thoughts @reginald-mclean 

I tried this, but another error will occur:
```bash
Traceback (most recent call last):
  File "/home/jinaoqun/workspace/openpi/examples/metaworld/test_async.py", line 45, in <module>
    env = AsyncVectorEnv([env_fn for _ in range(n_env)])
  File "/home/jinaoqun/workspace/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/gymnasium/vector/async_vector_env.py", line 154, in __init__
    dummy_env = env_fns[0]()
  File "/home/jinaoqun/workspace/openpi/examples/metaworld/test_async.py", line 38, in env_fn
    return MetaworldEnv()
  File "/home/jinaoqun/workspace/openpi/examples/metaworld/test_async.py", line 12, in __init__
    self.env = metaworld.envs.ALL_V2_ENVIRONMENTS_GOAL_OBSERVABLE[task_name](seed=seed)
  File "/home/jinaoqun/workspace/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/metaworld/envs/mujoco/env_dict.py", line 186, in initialize
    super(type(env), env).__init__()
  File "/home/jinaoqun/workspace/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/metaworld/envs/mujoco/sawyer_xyz/v2/sawyer_shelf_place_v2.py", line 31, in __init__
    super().__init__(
  File "/home/jinaoqun/workspace/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py", line 208, in __init__
    super().__init__(
  File "/home/jinaoqun/workspace/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py", line 50, in __init__
    mjenv_gym.__init__(
  File "/home/jinaoqun/workspace/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/gymnasium/envs/mujoco/mujoco_env.py", line 85, in __init__
    assert self.metadata["render_modes"] == [
AssertionError: ['human', 'rgb_array', 'depth_array']
```

Gymnasium is 1.1.0 and metaworld is most recent commit.

> [@pseudo-rnd-thoughts](https://github.com/pseudo-rnd-thoughts) [@reginald-mclean](https://github.com/reginald-mclean)
> 
> I tried this, but another error will occur:
> 
> Traceback (most recent call last):
>   File "/home/jinaoqun/workspace/openpi/examples/metaworld/test_async.py", line 45, in <module>
>     env = AsyncVectorEnv([env_fn for _ in range(n_env)])
>   File "/home/jinaoqun/workspace/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/gymnasium/vector/async_vector_env.py", line 154, in __init__
>     dummy_env = env_fns[0]()
>   File "/home/jinaoqun/workspace/openpi/examples/metaworld/test_async.py", line 38, in env_fn
>     return MetaworldEnv()
>   File "/home/jinaoqun/workspace/openpi/examples/metaworld/test_async.py", line 12, in __init__
>     self.env = metaworld.envs.ALL_V2_ENVIRONMENTS_GOAL_OBSERVABLE[task_name](seed=seed)
>   File "/home/jinaoqun/workspace/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/metaworld/envs/mujoco/env_dict.py", line 186, in initialize
>     super(type(env), env).__init__()
>   File "/home/jinaoqun/workspace/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/metaworld/envs/mujoco/sawyer_xyz/v2/sawyer_shelf_place_v2.py", line 31, in __init__
>     super().__init__(
>   File "/home/jinaoqun/workspace/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py", line 208, in __init__
>     super().__init__(
>   File "/home/jinaoqun/workspace/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py", line 50, in __init__
>     mjenv_gym.__init__(
>   File "/home/jinaoqun/workspace/openpi/examples/metaworld/.venv/lib/python3.10/site-packages/gymnasium/envs/mujoco/mujoco_env.py", line 85, in __init__
>     assert self.metadata["render_modes"] == [
> AssertionError: ['human', 'rgb_array', 'depth_array']
> Gymnasium is 1.1.0 and metaworld is most recent commit.

It seems that the problem is not with AsyncVectorEnv, but with Gymnasium 1.1.0.

@AoqunJin Could you provide a minimal example to test with