Repository: rainx0r/metaworld-algorithms
Issue: EOF error
Root cause: Resource Mishandling

Thanks for releasing the code! When running the code, I had the following bug from time to time when running rl2_ml10:

Initially, I had a warning

```
/home/jy/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/multiprocessing/popen_fork.py:66: RuntimeWarning: os.fork() was called. os.fork() is incompatible with multithreaded code, and JAX is multithreaded, so this will likely lead to a deadlock.
  self.pid = os.fork()
```
Then after a random number of steps (If I'm lucky, I can reach around 2e6 env steps. Though sometimes it failed at the very beginning), I'll get the following error
```
Traceback (most recent call last):
  File "/home/jy/Desktop/metaworld-algorithms/examples/meta_learning/rl2_ml10.py", line 109, in <module>
    main()
  File "/home/jy/Desktop/metaworld-algorithms/examples/meta_learning/rl2_ml10.py", line 105, in main
    run.start()
  File "/home/jy/Desktop/metaworld-algorithms/metaworld_algorithms/run.py", line 179, in start
    agent = algorithm.train(
            ^^^^^^^^^^^^^^^^
  File "/home/jy/Desktop/metaworld-algorithms/metaworld_algorithms/rl/algorithms/base.py", line 418, in train
    next_obs, rewards, terminations, truncations, infos = envs.step(actions)
                                                          ^^^^^^^^^^^^^^^^^^
  File "/home/jy/Desktop/metaworld-algorithms/.venv/lib/python3.12/site-packages/gymnasium/vector/async_vector_env.py", line 388, in step
    return self.step_wait()
           ^^^^^^^^^^^^^^^^
  File "/home/jy/Desktop/metaworld-algorithms/.venv/lib/python3.12/site-packages/gymnasium/vector/async_vector_env.py", line 447, in step_wait
    env_step_return, success = pipe.recv()
                               ^^^^^^^^^^^
  File "/home/jy/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
          ^^^^^^^^^^^^^^^^^^
  File "/home/jy/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/multiprocessing/connection.py", line 430, in _recv_bytes
    buf = self._recv(4)
          ^^^^^^^^^^^^^
  File "/home/jy/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/multiprocessing/connection.py", line 399, in _recv
    raise EOFError
EOFError
/home/jy/Desktop/metaworld-algorithms/.venv/lib/python3.12/site-packages/gymnasium/vector/async_vector_env.py:608: UserWarning: WARN: Calling `close` while waiting for a pending call to `step` to complete.
  logger.warn(
Exception ignored in: <function AsyncVectorEnv.__del__ at 0x79885fa2ea20>
Traceback (most recent call last):
  File "/home/jy/Desktop/metaworld-algorithms/.venv/lib/python3.12/site-packages/gymnasium/vector/async_vector_env.py", line 716, in __del__
    self.close(terminate=True)
  File "/home/jy/Desktop/metaworld-algorithms/.venv/lib/python3.12/site-packages/gymnasium/vector/vector_env.py", line 234, in close
    self.close_extras(**kwargs)
  File "/home/jy/Desktop/metaworld-algorithms/.venv/lib/python3.12/site-packages/gymnasium/vector/async_vector_env.py", line 612, in close_extras
    function(timeout)
  File "/home/jy/Desktop/metaworld-algorithms/.venv/lib/python3.12/site-packages/gymnasium/vector/async_vector_env.py", line 447, in step_wait
    env_step_return, success = pipe.recv()
                               ^^^^^^^^^^^
  File "/home/jy/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/multiprocessing/connection.py", line 250, in recv
    buf = self._recv_bytes()
          ^^^^^^^^^^^^^^^^^^
  File "/home/jy/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/multiprocessing/connection.py", line 430, in _recv_bytes
    buf = self._recv(4)
          ^^^^^^^^^^^^^
  File "/home/jy/.local/share/uv/python/cpython-3.12.11-linux-x86_64-gnu/lib/python3.12/multiprocessing/connection.py", line 399, in _recv
    raise EOFError
EOFError: 
```

I tried it on both ubuntu 20.04 and 22.04, and I was following the instructions to use virtual environments created by `uv`. Would it be possible for you to provide me with any ideas on how to fix it? Thanks!

Hi!

The JAX multiprocessing warning ends up not mattering because the repository does not do any multi-gpu stuff so there is only one main process and multiprocessing is only used for gymnasium's AsyncVectorEnv. It just so happens the multiprocessing method AsyncVectorEnv uses is one JAX would not use for multi-GPU stuff so that's why it throws that warning. That's my understanding anyway. So you can safely ignore it.

Now as for the main error, sadly it's not very clear what the problem could be. The "EOFError" you're seeing is being thrown by the AsyncVectorEnv because the env processes it's communicating with are crashing, so it's kind of a generic error, and what we'd need to solve the issue is the reason for the actual env process crash (note: this also means that likely the issue is with MetaWorld and not this repository). Usually what happens is one env process crashes for some distinct reason, prints its actual stack trace that caused the crash, then you'll see the EOFError from VectorEnv after the process died, then the logs will get spammed by all the other processes being killed. So to find the real error you'd have to scroll past all the EOFErrors from all the other processes until you find the original env crash.

If your terminal does not scroll that far up, you can try piping the output into a log file that you can then open in a text editor for inspection (e.g. `python examples/meta_learning/rl2_ml10.py > out.txt`).

I'm sorry I can't be of any more use immediately, but I haven't actually seen the env randomly crash (even after a *lot* of experiments for the MetaWorld V3 publication and other related work) so I'm actually not really sure what could be happening in your case.

It's even more strange that it only happens randomly or towards the end of training. What sort of hardware are you using for your training runs?