zhiminy/EvalEng huggingface dataset:
Repository: Farama-Foundation/Metaworld
Issue: Is there any tricks to accelerate the running speed of the environment?

The running speed is slow for me. I don't know whether it is normal to cost 1.3 seconds to run per episode on single task, maybe it varies with the computational power of cpu, but the scale is still large. The agent inference time can be ignored compared to the environment step time. I use stable-baselines3 SubprocVecEnv to create one thread for each simulator.
I think it's slow because I am using the ML10 as inner loop. I wonder what the official running time is.

I found the old version meta-world, based on gym rather than gymnasium (I don't sure if it is the cause), it cost only a third of the time for each episode.

Here is the evaluation code
```
from metaworld.envs import (ALL_V2_ENVIRONMENTS_GOAL_OBSERVABLE,
                            ALL_V2_ENVIRONMENTS_GOAL_HIDDEN)
import time

door_open_goal_observable_cls = ALL_V2_ENVIRONMENTS_GOAL_OBSERVABLE["door-open-v2-goal-observable"]

env = door_open_goal_observable_cls(seed=5)

env.reset()
start = time.time()
for _ in range(500):
    a = env.action_space.sample()
    next_obs2, _, _, _ = env.step(a)

print("run time", time.time() - start)
```

This example code takes 1.1 seconds to run on a server I have access to. Can you share more about your use case? I'm not quite sure if 'agent inference time can be ignored.' If you load a GPU with many jobs then agent inference time could be the bottleneck. 

Just want to add for context the performance I am getting for the `Humanoid` environment is 1.7k steps/sec
```py
>>> import gymnasium
>>> from gymnasium.utils.performance import benchmark_step
>>> env = gymnasium.make("Humanoid-v5")
>>> benchmark_step(env, target_duration=60, seed=456)
1747.5079450369717
```

you are getting 455 steps/sec which is reasonable considering the environment is more complex than the humanoid, and you may have a slower CPU

@alexxchen what CPU does your computer have?

Can you `benchmark_step` for `"Humanoid-v5"` and all metaworld environments and share the results with us?

What environment runtime performance expectation (steps per second) do you have?


Thanks for your reply @reginald-mclean! According to your result, the difference is within the expected range. I am meta-learning the hyperparameters, taking the reinforcement leaning as inner loop. To train the 10 environment for 500 steps in inner loop during each update of the outter loop, I find the bottleneck is the inner loop training speed (the inner loop agent is small enough). I'm just afraid of making a mistake, since the paper did not mention the wall time. 

By the way, is it posssible to transplant meta-world to something like isaac gym?

 @Kallinteris-Andreas Thanks for your information! The benchmark result is almost the same as you. I'm just being greedy. The faster the simulation runs, the better. 



We are working on adding support for MJX But that is a while away.

> We are working on adding support for MJX But that is a while away.

That sounds very exciting. Thank you for your work!
Repository: Farama-Foundation/Metaworld
Issue: About camera size of observation

Hi~ I'm curious about the image size of the observation. When I create an env like below

```
mt1 = MT1(task_name, seed=seed) 
env = mt1.train_classes[task_name](render_mode='rgb_array')
env.set_task(mt1.train_tasks[task_id]) 
```

How can I change the img size of the observation in the env? Thanks a lot!

Hi! I haven't tried this so I am not 100% sure it will work, but you should be able to pass a desired height and width into the mujoco renderer. If you go to [this line](https://github.com/Farama-Foundation/Metaworld/blob/0745f6c905f7823747a8e6b9fb5580909bb5397c/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py#L37) that init function calls the gymnasium mujoco env. If you add height=X, width=Y, you should be able to manipulate the image size. 

Thanks a lot! How can I go into this function to change the size? Are there some external parameters that can be controlled? By the way, I find that in some codebases, we can use the following code to create an env with given img size:

```
mw_gym_make(name, cam_height=img_size, cam_width=img_size, cam_name=cam_name, train_distrib=train, seed=seed)
```

Unfortunately, I find that the function mw_gym_make is deleted in the lasted version of metaworld. Are there any similar functions? Thanks a lot!

No there aren't any similar functions in MW right now
Repository: Farama-Foundation/Metaworld
Issue: copy.deepcopy fails to correctly copy an environment
Root cause: Algorithmic Error

When using the following code:
```
import copy
import metaworld
import random

ml1 = metaworld.ML1('pick-place-v2')
env = ml1.train_classes['pick-place-v2']()
task = random.choice(ml1.train_tasks)
env.set_task(task)
env2 = copy.deepcopy(env)
print(env.action_space)
print(env2.action_space)
```

The console outputs:

```
Box(-1.0, 1.0, (4,), float32)
Box(-1.0, 1.0, (2,), float32)
```

It seems that the copy.deepcopy function is not correctly copying the environment. The action spaces of the original environment and the copied environment are different.

Environment Configuration:
> Python 3.8
> Mujoco 3.1.4
> Mujoco-py 2.1.2.14
> Ubuntu 20.04.4
> Gymnasium 0.29.1

In [Garage](https://github.com/rlworkgroup/garage), both LocalSampler and RaySampler cannot be used due to this issue.

I would suggest trying release 2.0.0 as well, but if I remember correctly this issue exists with 2.0.0 as well. 

Thank for your reply and help! Release 2.0.0 works well with copy.deepcopy
best wishes

Based on my experimentation, release 2.0.0 still exhibits this issue. After multiple attempts, I discovered the root cause: when using deepcopy to replicate environments, it triggers the self._set_action_space() function from the parent class MujocoEnv of SawyerXYZEnv, which incorrectly changes the action_space to Box(-1.0, 1.0, (2,), float32). A viable solution is to override the self._set_action_space() method in the SawyerXYZEnv class. The specific implementation steps are as follows:
1. Add the following code to the SawyerXYZEnv class in site-packages/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py:
`def _set_action_space(self):`
`    self.action_space = Box(  # type: ignore`
`        np.array([-1, -1, -1, -1]),`
`        np.array([+1, +1, +1, +1]),`
`        dtype=np.float32,`
`    )`
2. In the `self.__init__() `section of the SawyerXYZEnv class, replace the original code block:
`self.action_space = Box(  # type: ignore`
 `   np.array([-1, -1, -1, -1]),`
`    np.array([+1, +1, +1, +1]),`
 `   dtype=np.float32,`
`)`
with:
`self._set_action_space()`
This modification ensures the action_space is correctly maintained when using deepcopy replication.
Repository: Farama-Foundation/Metaworld
Issue: metaworld only works with gym 0.15.4
Root cause: External Dependency Breakage

gym version 0.15.5 upwards have deprecated the `tags` arguments in [environment registration](https://github.com/openai/gym/blob/a99e8d153e94f532e5728b01b02b42e076334bf7/gym/envs/registration.py#L36)

They have specified the change [here](https://github.com/openai/gym/blob/4ca60758b44b6ae9080ca47a72f52c2035a4133f/README.rst#L164)

Hi @iDurugkar 

Thanks for filing this issue.

The metaworld API doesn't make use of gym environment registration, so this should not affect functionality. Is it causing you to not be able to install metaworld?

We would happily accept a pull requests which simply deleted the contents of
https://github.com/rlworkgroup/metaworld/blob/4528f84dab357d562824dac3edf159bc68399f23/metaworld/envs/mujoco/__init__.py

It's not necessary for the benchmark to work and I think is the source of your problems.

I can confirm that deleting the __init__.py solved the problem for me.

Created a corresponding pull request: https://github.com/rlworkgroup/metaworld/pull/52

Oh you beat me to it. I created a pull request #53  that just removes the `tags` field from the `metaworld/envs/mujoco/__init__.py` file.

Hello, is the push from @iDurugkar merged already? Thanks!

No, I think it needs one more approval before being merged

closed by #69
Repository: Farama-Foundation/Metaworld
Issue: Reward of drawer open may lead algorithm stuck at sub optiaml policy.
Root cause: Algorithmic Error

The reward function currently assigns a higher reward when the gripper is merely against the handle compared to when it is actually hooking the handle. Additionally, the reward when the gripper is opening the drawer is similar to when it is against the handle. This can lead the algorithm to favor a policy where the gripper stays against the handle rather than correctly hooking and pulling it without applying tricks to the RL algorithm.
![hook](https://github.com/Farama-Foundation/Metaworld/assets/93953048/a2a11852-d488-4385-95f8-2fdd4d324f8e)
![open](https://github.com/Farama-Foundation/Metaworld/assets/93953048/7e0f12c1-7e78-471c-99cc-22a5c73af2d9)
![reach](https://github.com/Farama-Foundation/Metaworld/assets/93953048/1b18a2eb-a282-4701-924b-b1d8eb117a98)


Sure, but it's very unlikely that every reward function is going to be 100% perfect such that the agent always solves the task. I'm not quite sure if this is an issue, I am quite confident there are similar instances of this in many reward functions
Repository: Farama-Foundation/Metaworld
Issue: Button Press Topdown Wall Expert Policy gives 0 reward but solves task
Root cause: Algorithmic Error


<img width="1075" height="2393" alt="Image" src="https://github.com/user-attachments/assets/bb9d6bb8-c0d7-4529-955a-546426a2475d" />

fixed by #548
Repository: Farama-Foundation/Metaworld
Issue: env.render() returns None 
Root cause: Documentation Deficiency

Hi, 

I am trying to render image during rollout of my SAC policy on door-lock task using command:  frame = env.render(mode='rgb_array')

And frame turns out to be None rather than image.

How do I get the rendered image? Do I need to add any other additional arguments in this command:  frame = env.render(mode='rgb_array') ? 

Hi sorry, our documentation needs to be updated. Follow the examples here: https://metaworld.farama.org/rendering/rendering/
You just need to set a render mode upon creation of the environment and then call
`frame = env.render()`

Render mode should be set during environment initialization. Check the documentation for examples

Thanks , its working now
Repository: Farama-Foundation/Metaworld
Issue: Question regarding reward function and observation space

Hi!

I have a few questions about the reward function and observation spaces that I hope you can help with.

1) The reward function for some environments (e.g. reach) is not consistent in the paper and in the code. Should I modify the code so they are consistent, or I should just use the one in the code? The one in the code makes more sense for the reaching task.

2) What was the observation space used in the ML10 and ML45 evaluations? Earlier in the paper it mentioned that the observation is 9-dimensional consisting of hand position and two task-related positions. For ML1 it hides the goal position to be the task, so it has a 6-dimensional observation space. I'm wondering if ML10 and ML45 are the same (goal position hidden), or they can observe the true goal?

Thank you so much in advance!

Hi,

1. You could use the reward function in the code. The reward specified in the paper is a simplistic version, which basically demonstrates the idea of the design of the reward functions, while the reward function in the code is augmented with a few additional components and is the actual one used for training RL algorithms. We could revise the paper to make reward functions consistent in order to avoid further confusion.

2. ML10 and ML45 should also hide the goal positions as they are using `obs_type` = 'plain'` (see https://github.com/rlworkgroup/metaworld/blob/master/metaworld/benchmarks/ml10.py#L21).

Hope this helps!

To add more context to @tianheyu927 's answer about goal positions -- goal positions are hidden in metaworld because it's a meta-/multi-task RL benchmark, not a goal-conditioned benchmark. Providing the goal position makes task inference (one of the main challenges of meta-/MTRL) trivial.

We might yet create a goal-conditioned version.

@tianheyu927 Can you update the paper with the reward functions from the code?

I'm going to close this issue for now. Please feel free to re-open (or open a new one) if you have more questions.
Repository: Farama-Foundation/Metaworld
Issue: Issues with Rendering - no attribute 'glfwGetCurrentContext'
Root cause: Documentation Deficiency

I am new to this benchmark and tool.

I installed as per the documentation

```
pip install git+https://github.com/Farama-Foundation/Metaworld.git@master#egg=metaworld
```

And tried the started code,

```python
import metaworld
import random

print(metaworld.ML1.ENV_NAMES)  # Check out the available environments

env_name = 'basketball-v2' # Pick an environment name

render_mode = 'human' # set a render mode

ml1 = metaworld.ML1(env_name) # Construct the benchmark, sampling tasks

env = ml1.train_classes[env_name](render_mode=render_mode)
task = random.choice(ml1.train_tasks)
env.set_task(task)  # Set task

obs = env.reset()  # Reset environment
a = env.action_space.sample()  # Sample an action
obs, reward, terminate, truncate, info = env.step(a)
```

I run into this error,
```
Exception ignored in: <function WindowViewer.__del_ at 0x10724c310>
Traceback (most recent call last):
File "/opt/homebrew/Caskroom/miniconda/base/envs/meta/lib/python3.10/site-pack ages/gymnasium/envs/mujoco/mujoco_rendering.py", line 359, in _del_
File "/opt/homebrew/Caskroom/miniconda/base/envs/meta/lib/python3.10/site-pack
ages/gymnasium/envs/mujoco/mujoco_rendering.py", line 352, in free
File "/opt/homebrew/Caskroom/miniconda/base/envs/meta/lib/python3.10/site-pack
ages/glfw/_
-
init
_py", line 2366, in get_current_context
AttributeError: 'NoneType' object has no attribute 'glfwGetCurrentContext'
```

A similar error was observed on Ubuntu 20.04, with the same setup instruction and code.

Kindly help, 
Thanks.


That's not an error that causes problems, it happens when your script ends without explicitly calling `env.close()`

Thanks, figured that out ... rookie mistake 🫠
Repository: Farama-Foundation/Metaworld
Issue: Safely Using Customized Environment Without Goal/End Effector Sites for Visual-Based Task Completion
Root cause: Documentation Deficiency

## Question: Safely Using Customized Environment Without Goal/End Effector Sites for Visual-Based Task Completion

### Description

I have customized the Meta-World environments for several tasks (Button Press, Hammer, Box Close, Drawer Close) by modifying the visibility of the goal and end effector sites to make them invisible. This was done by setting the alpha value of the RGBA parameter for these sites to 0. My main interest is to solve these tasks purely based on visual input from the rendered images.

### Customization Details

Here is the commit with the changes I made to the Meta-World codebase:
[Commit: Make sites invisible](https://github.com/EwertzJN/Metaworld/commit/0fa8d295e3da433b82e89bc6ce335e421aa77457)

### Question:

Are these modifications safe to use for solving the tasks using only visual input?


I don’t know. I don’t know your use case is, or what you’re trying to accomplish. I really think that’s up to you to judge
Repository: Farama-Foundation/Metaworld
Issue:  new_mocap_pos = self.data.mocap_pos + pos_delta[None]
Root cause: Validation Gap

We are trying to pass the following action to env.step:
[[-0.15927427  1.9502463  -1.8231916  -0.9944631 ]]

And are seeing this error:
  File "/home/research/user/miniconda3/envs/myenv/lib/python3.7/site-packages/metaworld/envs/mujoco/sawyer_xyz/base.py", line 142, in set_xyz_action
    new_mocap_pos = self.data.mocap_pos + pos_delta[None]
ValueError: operands could not be broadcast together with shapes (1,3) (1,1,4) 

This was resolved by some support on discord. I needed to format the action differently (squeeze it)
Repository: Farama-Foundation/Metaworld
Issue: ImageEnv compatibility with benchmark classes
Root cause: Unimplemented Feature Gap

Currently, the benchmark tasks on support state observations / goals. However, a lot of RL researchers would benefit from the ability to get image observations / goals. There's an ImageEnv available but it's incompatible with the benchmark classes ML1, etc. It works with single-task environments, but currently there's no goal-setting enabled there.

Would be useful either (a) to have image support for ML1, etc benchmarks or (b) enable goal setting for single task environments.

Is this currently possible? Maybe I missed something in the codebase.

Hi, thanks for asking! It is possible to use image observations with ML1. Please see https://github.com/rlworkgroup/metaworld/blob/ad807f9d9b6dc577a95ec51795d2b2ed318914fe/tests/metaworld/envs/test_multitask_env.py#L121 as an example. We suggest only use our environment wrapper for meta-learning and multi-task learning. We are recently pushing for the camera ready of this work and will update our guidance after that. 

Thanks @zhanpenghe, indeed getting image observations is straightforward, but the issue is getting goal images. We want to train an agent that is given a goal image (e.g. closed door, stacked block, pushed puck) and achieves the goal using images as observations. 

So the question is - how do we render goal images? It seems that the goals are set as the object coordinates, and you want to move the object to those coordinates to set the goal image.



We don't currently have any specific support for retrieving a goal image. This would likely require manipulating the MuJoCo state (into the goal configuration), then rendering the image, then reseting to initial state.

We'd welcome a pull request with this feature, though!



@MishaLaskin we are unlikely to add this ourselves to the current incarnation of the benchmark, because a proper image-conditioned meta-RL or MTRL benchmark would be best if all tasks share a common initial workspace appearance.

Using the disjoint simulations we have today, in most cases the task is trivially-inferable from the initial workspace image.

I will leave this open in case you or someone else would like to contribute an enhancement to support this. 

As Meta-World ownership has been transferred from the RLWorkGroup to [Farama Foundation](https://farama.org/), I am closing this issue. If there are any questions or requests for features please join our [Discord](https://discord.gg/PfR7a79FpQ)
Repository: Farama-Foundation/Metaworld
Issue: Wrong camera view when using "gripper_pov" view to render images
Root cause: Algorithmic Error

I am using metaworld (pip show version = 2.0.0) try to get different camera views as shown : 
![image](https://github.com/user-attachments/assets/a08a630c-762f-406d-8317-a56f97fb01b7)
but is seems like the camera with camera_name='gripper_pov' or camera_id='5' renders in a wrong view

Should this view seems like this one (figure 1.a  in other papers) ?
![image](https://github.com/user-attachments/assets/db2d7dcd-b074-40d3-9180-885d942194e6)
How to solve this problem? I don't know the causes of this problem, thank you !

By the way, when I try to set 'caemra_id' = 4 , the rendering image like this:
![image](https://github.com/user-attachments/assets/f1f3df9f-0633-4d06-98f1-b4c03da18b7e)
It seems like the image is upside down, maybe this clue can be useful to figure out the question.

Hello? I wonder if there are some users or developers who have faced reletive experiences to help to fix this problem?
If this is relavant to the env config, Ican provide my setting details to together fix this bug.

The video being upside down is a known thing. I've tried to replicate this on my end and I can't seem to

the camera being upside down might be a windows thing @Aoyama7Hai what OS are you using

@Kallinteris-Andreas I'm using Ubuntu 22.04.4 LTS (GNU/Linux 6.5.0-44-generic x86_64)
BTW, I wonder the the render image of camera_name='gripper_pov' a.k.a camera_id='5' in the first conmmet is right? It seems like the position of the camera were set up incorrectly inside the gripper

Hello @Aoyama7Hai I proposed a pull request to fix the issue you proposed : https://github.com/Farama-Foundation/Metaworld/pull/525 tell me if it answers your question
Repository: Farama-Foundation/Metaworld
Issue: How to set a new camera view in metashape?

I wonder how to build a new camera view in metashape except of now exsiting six camera views?
Thank you !

Hi, I don't know what you are trying to accomplish. Can you provide some more info?

The only way I'm familiar with is adding a new camera inside the robot scene model file, You may be able to find an alternate way if you look at the MuJoCo.org documentation.

Thank you ! I have solved this question by the old issues to change the camera setting parameters in mujoco.

Please link the solution you found for future reference.

Please refer to:
https://github.com/Farama-Foundation/Metaworld/issues/35
Repository: Farama-Foundation/Metaworld
Issue: Goal Position Bug Sweep-V2
Root cause: Architectural Constraint

The goal position after stepping through the env gets moved back to coordinate (0, 0, 0). This is due to an origin conflict in the xml-file. Just change the position in that file to anywhere else, and it's not causing issues anymore.

**Coordinate 0 0 0 in sawyer_sweep_v2.xml**:
<img src="https://github.com/user-attachments/assets/a170f7b9-82ea-4cdc-b4e4-e1445e9119a2" alt="Screenshot 2024-10-03 103335" width="400"/>


**Coordinate 0.5 0.65 0.01 in sawyer_sweep_v2.xml**:
<img src="https://github.com/user-attachments/assets/ddd0f463-540a-4a37-8702-9e486d57f8cf" alt="Screenshot 2024-10-03 103735" width="400"/>


https://github.com/Farama-Foundation/Metaworld/blob/08489b46faffe3bfdeaa1287fa108260e74e194d/metaworld/envs/assets_v2/sawyer_xyz/sawyer_sweep_v2.xml#L18C30-L18C35
Repository: Farama-Foundation/Metaworld
Issue: Multiple cameras in render
Root cause: Unimplemented Feature Gap

Is there any way to render image from several cameras in current env?

```python
ml1 = metaworld.ML1(env_name) 
cameras_name = ['topview', 'corner'] 
env = ml1.train_classes[env_name](render_mode=render_mode, camera_name=cameras_name)
```


If you're using the most recent version of Meta-World, you should be able to specify the camera_id. The render function uses [this function from Gymnasium](https://github.com/Farama-Foundation/Gymnasium/blob/dc36e9654338ef4b9cc453ba64b2dee7473dc593/gymnasium/envs/mujoco/mujoco_rendering.py#L204). 

Currently we do not support multi-camera rendering., You could try to fake it by using the render function and then changing the internal `camera_id` Member variable of the MuJoCo renderer

Hi! I'm trying to get multiple camera views from a single environment, but am having a hard time. Did you get this to work? If so, would you mind sharing a tiny code snippet to help get started? Any help would be much appreciated!