zhiminy/EvalEng huggingface dataset:
Repository: Farama-Foundation/Metaworld
Issue: How to Load and Use a Trained Policy?
Root cause: Documentation Deficiency

Thanks for your great project and contribution! I'm  so sorry to bother you again, but can you tell me how to load and use the trained policy?  I have successfully run the ml1 task and got the training results.  

I am trying to use the trained policy and visualize the results, according to the instructions in the linked document.[https://garage.readthedocs.io/en/latest/user/reuse_garage_policy.html#applying-the-policy-to-an-environment](url)

but I faid. My code is here:
`from garage.experiment import Snapshotter`
`snapshotter = Snapshotter()`
`data = snapshotter.load('data/local/experiment/maml_ppo_metaworld_ml1_push/')`
`policy = data['algo'].policy`
`env = data['env']`
`steps, max_steps = 0, 150`
`done = False`
`obs = env.reset() `
`policy.reset()`
`while steps < max_steps and not done:`
    `obs, rew, done, _ = env.step(policy.get_action(obs))`
    `env.render() `
    `steps += 1`
`env.close()`

Error message is here:
`Traceback (most recent call last):`
`  File "read_policy.py", line 30, in <module>`
`    obs, rew, done, _ = env.step(policy.get_action(obs))`
`  File "/home/guojiaqi/.local/lib/python3.7/site-packages/garage/torch/policies/stochastic_policy.py", line 47, in get_action`
`    observation = as_torch(observation)`
`  File "/home/guojiaqi/.local/lib/python3.7/site-packages/garage/torch/_functions.py", line 146, in as_torch`
`    return torch.as_tensor(array).float().to(global_device())`
`TypeError: can't convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, complex64, complex128, int64, int32, int16, int8, uint8, and bool.`

I tried to change the format of obs, but I couldn't find the right way. Could you please provide a code file that can loads and uses the trained policy and can visualize the results? 

I would appreciate it if you can provide some help!

I tried the code in another issue,  #https://github.com/rlworkgroup/garage/issues/2314

but still had problems. :(

the code is here:
`from garage.experiment import Snapshotter`
`import metaworld`
`import random`
`snapshotter = Snapshotter()`
`data = snapshotter.load('data/local/experiment/maml_ppo_metaworld_ml1_push_10epoch/')`
`policy = data['algo'].policy`
`env = data['env']`
`print('env', env)`
`print("Data dictionary",data)`
`from garage import rollout`
`path = rollout(env[0], policy, animated=True)`
`print(path)`

Error message is here:
`Traceback (most recent call last):`
`  File "read_policy.py", line 57, in <module>`
`    path = rollout(env[0], policy, animated=True)`
`TypeError: 'TaskNameWrapper' object does not support indexing`

If you're using the `TaskNameWrapper` like this, you should use `env` instead of `env[0]`.
Repository: Farama-Foundation/Metaworld
Issue: Seeding not working for v2 environments
Root cause: Validation Gap

I am trying to get deterministic behavior for the same seed> However, running the following piece of code prints about different result every time.

```
import metaworld
ml10 = metaworld.ML10(seed=0)
cls_name = 'reach-v2'
env = ml10.train_classes[cls_name]()
task = [task for task in ml10.train_tasks if task.env_name == cls_name][0]
env.set_task(task)
print(env.obj_init_pos)
```

This is also the case for individual environments

```
from metaworld.envs import ALL_V2_ENVIRONMENTS_GOAL_HIDDEN
ReachEnv = ALL_V2_ENVIRONMENTS_GOAL_HIDDEN['reach-v2-goal-hidden']
env = ReachEnv(seed=0)
env.reset()   # doesn't matter if reset here. seeding not working in either case
print(env.obj_init_pos)
```

This could be an issue of this [line](https://github.com/rlworkgroup/metaworld/blob/29fe5d6d95cf9ad86f63eac38db8c0aef3837994/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py#L473)

However, changing it to self.np_random.uniform doesn't seem to fix the behaviour

I have reported the same issue some time ago here #350. The issue is indeed using the global RNG.

Unfortunately this isn't quite as simple as just changing that one line, but there is now a PR (#370) that should address this use case. It also has a test that shows how to use it and I believe verifies that everything works as expected.

However, your report also indicates that there's something wrong non-deterministic when using the `Benchmark` interface, which I will need to investigate.

I cannot replicate this issue, so I think PR (#370) has fixed this. As Meta-World ownership has been transferred from the RLWorkGroup to [Farama Foundation](https://farama.org/), I am closing this issue but it will be fixed in a future version of Meta-World. If there are any questions or requests for features please join our [Discord](https://discord.gg/PfR7a79FpQ)
Repository: Farama-Foundation/Metaworld
Issue: How to test the trained model in a new environment?
Root cause: Documentation Deficiency

I have used the file [https://github.com/rlworkgroup/garage/blob/08492007d6e2d9ead9beb83a8a4247e52019ac7d/metaworld_examples/sac_metaworld.py](url) 
to train a model based on a push-v2 environment. I want to test the trained model in another push-v2 environment with some environment parameters changed (to test if it can generalize),.
how should I do it?

As Meta-World ownership has been transferred from the RLWorkGroup to [Farama Foundation](https://farama.org/), I am closing this issue. If there are any questions or requests for features please join our [Discord](https://discord.gg/PfR7a79FpQ)
Repository: Farama-Foundation/Metaworld
Issue: Bug in push v2 environment when self.random_init = False
Root cause: Validation Gap

Hey,
I am using push v2 environment from ALL_V2_ENVIRONMENTS_GOAL_OBSERVABLE (single goal env). When random_init is set to False, the following error pops up:
```
File "debug_metaworld.py", line 50, in step
    obs, reward, done, _ = self._env.step(action)
  File "/mnt/beegfs/home/benedict/code/metaworld/metaworld/envs/mujoco/mujoco_env.py", line 25, in inner
    return func(*args, **kwargs)
  File "/mnt/beegfs/home/benedict/code/metaworld/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py", line 439, in step
    reward, info = self.evaluate_state(self._last_stable_obs, action)
  File "/mnt/beegfs/home/benedict/code/metaworld/metaworld/envs/mujoco/mujoco_env.py", line 25, in inner
    return func(*args, **kwargs)
  File "/mnt/beegfs/home/benedict/code/metaworld/metaworld/envs/mujoco/sawyer_xyz/v2/sawyer_push_v2.py", line 78, in evaluate_state
    ) = self.compute_reward(action, obs)
  File "/mnt/beegfs/home/benedict/code/metaworld/metaworld/envs/mujoco/sawyer_xyz/v2/sawyer_push_v2.py", line 153, in compute_reward
    object_grasped = self._gripper_caging_reward(
  File "/mnt/beegfs/home/benedict/code/metaworld/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py", line 564, in _gripper_caging_reward
    caging_xz_margin = np.linalg.norm(self.obj_init_pos[xz] - self.init_tcp[xz])
TypeError: list indices must be integers or slices, not list
```
This is because, when random_init is False, `self.obj_init_pos` is a list here in the below line. https://github.com/rlworkgroup/metaworld/blob/29fe5d6d95cf9ad86f63eac38db8c0aef3837994/metaworld/envs/mujoco/sawyer_xyz/v2/sawyer_push_v2.py#L122 This gets converted to a numpy array only when `random_init` is set to True. In the `gripper_caging_reward` method, therefore numpy-like slicing on a list fails. This doesn't happen in other environments (say pick-place, reach or hammer) because `gripper_caging_reward` method is not invoked there.

Have opened a PR here: https://github.com/rlworkgroup/metaworld/pull/368

looks like @krzentner closed this via #368
Repository: Farama-Foundation/Metaworld
Issue: mujoco_py.cymj.GlfwError: Failed to initialize GLFW
Root cause: Environment Incompatibility

When I use this repo on the server without interface, I Get an error : mujoco_py.cymj.GlfwError: Failed to initialize GLFW, I know it is because the MJViewer, but how can I fix it?

What mujoco version are you using? Also, is the DISPLAY environment variable set? I haven't experienced any errors like this on my headless machines.

As Meta-World ownership has been transferred from the RLWorkGroup to [Farama Foundation](https://farama.org/), I am closing this issue. If there are any questions or requests for features please join our [Discord](https://discord.gg/PfR7a79FpQ)
Repository: Farama-Foundation/Metaworld
Issue: can not render under Tesla-V100 GPU

Hi, it seems that the same bug always exists under V100-32G GPU environment:
```
Found 0 GPUs for rendering. Using device 0.
File "mjsim.pyx", line 156, in mujoco_py.cymj.MjSim.render
  File "mjsim.pyx", line 158, in mujoco_py.cymj.MjSim.render
  File "mjrendercontext.pyx", line 46, in mujoco_py.cymj.MjRenderContext.__init__
  File "mjrendercontext.pyx", line 114, in mujoco_py.cymj.MjRenderContext._setup_opengl_context
  File "opengl_context.pyx", line 130, in mujoco_py.cymj.OffscreenOpenGLContext.__init__
RuntimeError: Failed to initialize OpenGL
```
My code works well on P100 GPUs, does it mean that metaworld/mujoco_py do not support V100 rendering?
Have your ever encounter such problem using V100 GPUs?

To be honest, I never tried rendering the environments on a gpu instance. I would run them headless and then render on my local non gpu machine, as I didn't need any of the rendered images during training, but only for evaluation purposes. 

I have faced this issue before which was solved by `unset LD_PRELOAD` variable for offscreen `env.sim.render()`.  

As Meta-World ownership has been transferred from the RLWorkGroup to [Farama Foundation](https://farama.org/), I am closing this issue. If there are any questions or requests for features please join our [Discord](https://discord.gg/PfR7a79FpQ)
Repository: Farama-Foundation/Metaworld
Issue: Replacing mujoco-py with pybind11-based mujoco bindings
Root cause: External Dependency Breakage

Recently, other big benchmark suites, such as [OpenAI gym](https://github.com/openai/gym/blob/024b0f5160716b01effc27de45615bac7620e8b6/README.md?plain=1#L51-L52) and [DMC](https://github.com/deepmind/dm_control/blob/41d0c7383153f9ca6c12f8e865ef5e73a98759bd/README.md?plain=1#L61-L68), switched to the new [pybind11-based mujoco bindings](https://pypi.org/project/mujoco/). 
Since these bindings are made available in DeepMind's official mujoco repository, I expect that they will be properly maintained for quite a while. In addition, this makes installation easier for users as well as potentially allows for a larger user base and fewer issues related to the mujoco installation.

As MetaWorld uses its own [MujocoEnv](https://github.com/rlworkgroup/metaworld/blob/18118a28c06893da0f363786696cc792457b062b/metaworld/envs/mujoco/mujoco_env.py#L31) version,  I suggest including something similar to the [OpenAI gym repo](https://github.com/openai/gym/blob/51a6386a6bdaf3fd0a513133a1cd45ef27735090/gym/envs/mujoco/mujoco_env.py#L56-L74) to allow for a smooth transition and reproducibility of existing results.
While I don't think this change is urgent, I think it is in the long term interest of both users and maintainers.

Pros:
+ `mujoco-py` is unmaintained according to https://github.com/deepmind/mujoco#third-party-bindings
+ A copy of the MuJoCo library is provided as part of the package and does not need to be downloaded or installed separately.

Cons:
+ `mujoco` doesn't manage GLContext during rendering, it only provides GLContext API. metaworld need to manage GLContext by itself.

I think I speak for all the maintainers when saying we definitely would love to do this, but we don't have the time. 

If there is anyone who is inclined to do the update themselves that would be great! 

I can do the update, but I don't want to keep full compatibility with old API which requires a lot of boilerplate codes, especially given that `gym` recently removed the old render API.

Is there any update on doing this?

I made an attempt to update the bindings. Right now all scripted policy tests pass, but I am not entirely confident that the update may change the behavior for some environments.

I am also inclined to drop compatibility with the old gym API in favor of new ones, but I am not sure what the decision from the maintainer is. I personally think we should consider just dropping support for old mujoco_py support (i.e., deprecate v1 and migrate v2 in-place or create v3 that uses the new mujoco bindings) as it would bring a lot of maintenance burden both for now and future. If people want to use the old environments they can also check out an old version of the repo.

As Meta-World ownership has been transferred from the RLWorkGroup to [Farama Foundation](https://farama.org/), I am closing this issue. This task is on our road map for Meta-World. Please join our [Discord](https://discord.gg/PfR7a79FpQ) to stay up to date on this!
Repository: Farama-Foundation/Metaworld
Issue: The peg is somewhat gradually becoming "oblique" in the long run in SawyerPegInsertionSideEnvV2
Root cause: Algorithmic Error

Hi team, I am trying to play with [SawyerPegInsertionSideEnvV2](https://github.com/rlworkgroup/metaworld/blob/master/metaworld/envs/mujoco/sawyer_xyz/v2/sawyer_peg_insertion_side_v2.py). However, when I monitored the long-run performance of the environment, I found that the "peg" is gradually not horizontal like the figure shown below. Therefore, I am wondering if there is any solution or idea for this? Thank you so much!


![2022-07-04_21-44_1](https://user-images.githubusercontent.com/52727818/177239295-d8a3eef1-93f8-4693-b551-e13fb23ed254.png)
![2022-07-04_21-43](https://user-images.githubusercontent.com/52727818/177239278-638b0aa1-cc35-4e7c-8018-3fa23a89ef0e.png)
![2022-07-04_21-44](https://user-images.githubusercontent.com/52727818/177239288-7b778245-a8da-4fa0-a05e-d0169be844e4.png)
Repository: Farama-Foundation/Metaworld
Issue: Dtype of observation space does not match returned observation
Root cause: Interface Contract Mismatch

Currently, the observation space does not specify a dtype, which means it defaults to float32, however the returned numpy array is by default in float64. This will lead to exceptions with more recent versions of gym as they introduced explicit checks if the returned observations are contained in the observation space. 
Hence, I propose to either add the dtype to the Box space constructor https://github.com/rlworkgroup/metaworld/blob/18118a28c06893da0f363786696cc792457b062b/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py#L385-L403
and/or cast the returned values from `step()` and `reset()` to the appropriate dtype. 

The same thing should probably be done for the action space as well https://github.com/rlworkgroup/metaworld/blob/18118a28c06893da0f363786696cc792457b062b/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py#L128-L131

Thank you for opening this issue @ottofabian. This will be fixed in an upcoming release of Meta-World (alongside many other fixes). I am closing this issue for now and If there are any other questions or requests for features please join our [Discord](https://discord.gg/PfR7a79FpQ) and let us know! 

I'm actively working on fixing this, so re-opening.
Repository: Farama-Foundation/Metaworld
Issue: Incorrect reset space for object in disassemble
Root cause: Validation Gap

The lower bound for the random reset space of the object in disassemble is higher than the upper bound for the first index. 
This appears to be an issue for both v1 
https://github.com/rlworkgroup/metaworld/blob/18118a28c06893da0f363786696cc792457b062b/metaworld/envs/mujoco/sawyer_xyz/v1/sawyer_disassemble_peg.py#L14-L15
and v2 
https://github.com/rlworkgroup/metaworld/blob/18118a28c06893da0f363786696cc792457b062b/metaworld/envs/mujoco/sawyer_xyz/v2/sawyer_disassemble_peg_v2.py#L15-L16

`np.random.uniform` seems to invert the values in case `high > low`. However when using explicit seeding with `self.np_random, seed = seeding.np_random(seed)` `self.np_random.uniform` raises a `ValueError`. 

Hi @ottofabian  I cannot replicate this issue (it's possible PR #370 fixed this). As Meta-World ownership has been transferred from the RLWorkGroup to [Farama Foundation](https://farama.org/), I am closing this issue. If there are any questions/issues please join our [Discord](https://discord.gg/PfR7a79FpQ)

Hi @reginald-mclean. The referenced PR might have fixed this bug as it depends how you generate the uniform samples. 
Regardless, I would still propose to change the upper and lower bound of the first elements [here](https://github.com/Farama-Foundation/Metaworld/blob/a98086ababc81560772e27e7f63fe5d120c4cc50/metaworld/envs/mujoco/sawyer_xyz/v2/sawyer_disassemble_peg_v2.py#L15-L16) to avoid issues like this in the future

Oh sorry, I misread the original issue. Thanks for pointing out that issue!
Repository: Farama-Foundation/Metaworld
Issue: random_init flag is broken and should be removed
Root cause: Validation Gap

In the V2 push-wall, pick-place-wall, push-back, and shelf-place environments, setting `rand_init` to False and then resetting the environment would lead to the following error when calling `step()`:
```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/mnt/beegfs/home/zchuning/metaworld/metaworld/envs/mujoco/mujoco_env.py", line 25, in inner
    return func(*args, **kwargs)
  File "/mnt/beegfs/home/zchuning/metaworld/metaworld/envs/mujoco/sawyer_xyz/v2/sawyer_push_wall_v2.py", line 79, in evaluate_state
    ) = self.compute_reward(action, obs)
  File "/mnt/beegfs/home/zchuning/metaworld/metaworld/envs/mujoco/sawyer_xyz/v2/sawyer_push_wall_v2.py", line 157, in compute_reward
    object_grasped = self._gripper_caging_reward(
  File "/mnt/beegfs/home/zchuning/metaworld/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py", line 572, in _gripper_caging_reward
    caging_xz_margin = np.linalg.norm(self.obj_init_pos[xz] - self.init_tcp[xz])
TypeError: list indices must be integers or slices, not list
```
This is because the `reset()` function of these environments sets `self.obj_init_pos` to `self.adjust_initObjPos(self.init_config['obj_init_pos'])` when `self.random_init` is False (e.g. line 112 in `sawyer_push_wall_v2.py`). However, `self.adjust_initObjPos()` returns a python list instead of a numpy array. So line 572 of `swyer_xyz_env.py` triggers an error by accessing a python list with a list of indices.

A simple fix is to wrap a `np.array()` around `self.adjust_initObjPos(self.init_config['obj_init_pos'])`, similar to how some other environments (e.g. Push V2) handle calls to `fix_extreme_obs_pos()`. But there might be a more systematic way to fix this, hence the github issue instead of a pull request.

We should probably delete the random_init flag, since it should always be set to True. It no longer does what it says it does anyways, since `_freeze_rand_vec` (which defaults to true) prevents goal randomization from happening on reset anyways.

Copied from Discord:
> Basically, there's a flag called `random_init` that only works when set to True, but does not actually cause initialization to be randomized.
> So it should be deleted and all of the code that uses it should be simplified using the assumption that it's always set to True.
> (Actual random initialization is controlled by the _freeze_rand_vec flag here: https://github.com/Farama-Foundation/Metaworld/blob/master/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py#L470 )

Done! 

The flag is still there, so I'm re-opening this issue.

Currently we are trying to figure out the best way to maintain both the current V2 version of Meta-World while also having the ability to point users to the version of Meta-World that was used to create the Meta-World paper or other Meta-World results. For that reason, the changes for this issue have not been pushed to main. They can be found in the fixing-issues branch (which also contains some MuJoco updates).
Repository: Farama-Foundation/Metaworld
Issue: TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases
Root cause: External Dependency Breakage

Hi, I try to import metaworld in python3.8.5, but the typeError issue occurs:
`>>> import metaworld
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/data/yaoxt3/RL/garage/metaworld/metaworld/__init__.py", line 7, in <module>
    import metaworld.envs.mujoco.env_dict as _env_dict
  File "/data/yaoxt3/RL/garage/metaworld/metaworld/envs/mujoco/env_dict.py", line 6, in <module>
    from metaworld.envs.mujoco.sawyer_xyz.v1 import (
  File "/data/yaoxt3/RL/garage/metaworld/metaworld/envs/mujoco/sawyer_xyz/v1/__init__.py", line 1, in <module>
    from metaworld.envs.mujoco.sawyer_xyz.v1.sawyer_assembly_peg import SawyerNutAssemblyEnv
  File "/data/yaoxt3/RL/garage/metaworld/metaworld/envs/mujoco/sawyer_xyz/v1/sawyer_assembly_peg.py", line 5, in <module>
    from metaworld.envs.mujoco.sawyer_xyz.sawyer_xyz_env import SawyerXYZEnv, _assert_task_is_set
  File "/data/yaoxt3/RL/garage/metaworld/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py", line 11, in <module>
    from metaworld.envs.mujoco.mujoco_env import MujocoEnv, _assert_task_is_set
  File "/data/yaoxt3/RL/garage/metaworld/metaworld/envs/mujoco/mujoco_env.py", line 31, in <module>
    class MujocoEnv(gym.Env, abc.ABC):
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases`

Does anyone else encounter this problem?

OpenAI changed the Gym API recently, so using gym>=0.25.0 is probably breaking metaworld. Try downgrading gym to 0.24 and check?

Thank you very much! It works.
Repository: Farama-Foundation/Metaworld
Issue: Really High successful rate for RL2-PPO in ML45

Thanks for offering us this wonderful environments and benchmarks. When I checked the new benchmark results of ML10/ML45 on the updated paper https://arxiv.org/pdf/1910.10897.pdf, I found that the new RL2-PPO performance is really great and it reaches 70% successful ratio. It is kind of weird for me because RL2-PPO can achieve much great performance compared with all PPO based MT baselines in a large margin. 

As far as I know, the task id information is masked for RL2-PPO so it needs to interact with the environments for a few trajectories to infer the task information. It contains less information compared with MTPPO, but its performance is much better than MTPPO. Do you have any reasons or hypthesis for this phenmenon?

Yeah that's probably because there's a lot of information that a rnn like the one in rl2 captures dynamics between timesteps. 

We tried to overcome this by stacking consecutive timesteps in our mtppo experiments, but I suspect that more framestacking is required. 

Thanks for your reply. Btw I want to double check that, in MT10/50 the initialisation and goal are fixed while in ML10/45 the initialisation and goal are random generated and the corresponding goal state are masked in the observation space. It this right?

It is also quite weird because, rl2-ppo is using ppo while mt-ppo is really poor, mt-sac is good but pearl is bad.... completely make no sense for me.... @avnishn 

Yeah PEARL in general we found didn't scale well to complex environments and there are predecessors to the algorithm that were published. It was also very slow (the experiments when I ran them ran for v2 took 3 weeks to run on large single node gcp instances) 

I'm going to go ahead and close this for now since this isn't an issue with the metaworld.