zhiminy/EvalEng huggingface dataset:
Repository: Farama-Foundation/Metaworld
Issue: A recent commit introduced a bug in Metaworld v2 - env.step() doesn't return done when curr_path_length == max_path_length
Root cause: Algorithmic Error

Hi,

I can't exactly pinpoint where the bug was introduced but, I recently pulled a new version of Metaworld and a bug was introduced.
Specifically, while running an episode, env.step() would never return Done=True, thus the code would throw an error at step 501 (max_path_length = 500 by default).
It might have to do w/ a recent commit, e.g. [here ](https://github.com/Farama-Foundation/Metaworld/commit/82cecfb24eed51c0c4a79a5e06fee1cd36892b15#r115559039) you can see that self.curr_path_length is never reset when env.reset() is called. Now, that's not exactly the source of the issue but maybe the bug was introduced in the same commit. 
I'm guessing there is a bug in env.step() w.r.t. the logic that checks self.curr_path_length == self.max_path_length 


here's the part of the error that's relevant:

```
File "/Users/massimo/opt/anaconda3/envs/tacrl/lib/python3.8/site-packages/metaworld/envs/mujoco/mujoco_env.py", line 25, in inner
    return func(*args, **kwargs)
  File "/Users/massimo/opt/anaconda3/envs/tacrl/lib/python3.8/site-packages/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py", line 411, in step
    self.do_simulation([action[-1], -action[-1]])
  File "/Users/massimo/opt/anaconda3/envs/tacrl/lib/python3.8/site-packages/metaworld/envs/mujoco/mujoco_env.py", line 110, in do_simulation
    raise ValueError('Maximum path length allowed by the benchmark has been exceeded')
ValueError: Maximum path length allowed by the benchmark has been exceeded
```

Hi @optimass. Meta-World has never returned Done = True when the task has been completed. I don't know the exact reasoning behind that as I am not the original maintainer and can't find any great documentation of this. You can monitor the info['success'] field for completion. When info['success'] == 1.0 then the task is complete, 0.0 otherwise. 

The Maximum path length issue was introduced with PR #401
Repository: Farama-Foundation/Metaworld
Issue: Roadmap
GitHub labels: enhancement

This is a roadmap of the current plans for Meta-World. Further information can be found here [Monday Board](https://farama.monday.com/boards/3855618447). The Meta-World project is undergoing updates for Quality of Life, new package support, and ease of use improvements. 

January-May

- [x] Upgrade to Deepmind MuJoco from mujoco_py

June
- [x] Update Meta-World to Gymnasium from Gym
- [x] Update for Python 3.8-3.11 support (dropping 3.7)

July
- [x] #430
- [x] #431
- [ ] Release v0.3.0

Do we want to make a github release for the legacy metaworld project? 

@pseudo-rnd-thoughts I think that makes sense. I have no idea how to do that so I would need help. 

We have done it for miniwob++ project.
I believe you can make a release from a particular commit or branch.

If we know the point at which a legacy release makes sense, I.e. before farama took over then we should be able to do this
Repository: Farama-Foundation/Metaworld
Issue: seed function cannot guarantee identical reset results in environments
Root cause: Algorithmic Error

I ran the demo in "Accessing Single Goal Environments".
I noticed that two environments env1 and env2 were set with the same seed, but calling reset() on them resulted in different states.
```
env1 = door_open_goal_observable_cls(seed=5)
env2 = door_open_goal_observable_cls(seed=5)

s1 = env1.reset()
s2 = env2.reset()
```


Can you provide more code for replicating this? Running on our most recent commit of Meta-World results in the exact same states. 

@Cubism-star since I haven't heard back from you, I am going to assume this issue is fixed.
Repository: Farama-Foundation/Metaworld
Issue: Chaining the tasks
Root cause: Unimplemented Feature Gap

Hi, is it possible to stitch the tasks? For example, metaworld has `door-open` and `door-unlock`. I want to create a task to `open a locked door`. Do you have any tricks/suggestions to create this kind of task?

@reginald-mclean Might know but to my knowledge, I don't think this is easily possible without implementing a custom environment

That sounds like a custom environment (and also a really hard task). All the pieces are there for you to create that yourself if you wanted
Repository: Farama-Foundation/Metaworld
Issue: EZPickle of a Meta-World env does not produce the exact same env
Root cause: Architectural Constraint

The arm position of the pickled env is incorrect for some reason 

Could you provide a script to reproduce

```import metaworld
import pickle
import numpy as np

mt1 = metaworld.MT1('assembly-v2', seed=5)
env = mt1.train_classes['assembly-v2']()
task = [task for task in mt1.train_tasks if task.env_name == 'assembly-v2'][0]
env.set_task(task)

pickled_env = pickle.loads(pickle.dumps(env))
obs1, _ = env.reset()
obs2, _ = pickled_env.reset()

assert np.all(np.equal(obs1, obs2))

This may also be fixed by #458 

This seems to be an issue with where EZPickle is placed. Currently the init for EZPickle is placed in the Sawyer XYZ base class, but it may need to be moved into the individual classes for each environment.
Repository: Farama-Foundation/Metaworld
Issue: All metaworld environments report done=True at max time steps.
Root cause: Unimplemented Feature Gap

Although this can be convenient for on-policy algorithms, it technically makes the environment partially observable. In particular, this is known to significantly increase the sample requirements for off-policy algorithms in many cases.

Since the goal of this benchmark is to compare different meta-learning methods, it would be best to have a version of this benchmark that does not unfairly discriminate against off-policy meta-RL methods (such as PEARL).

This may be relevant to @katerakelly and @lywong92.

This would also be relevant to implementations of SAC and MT-SAC.

Just a comment: This can also be a problem for RL^2 as it would expect `done=True` when task is complete, not when the env reaches max time steps.

I also thought that might be the case. Surprisingly, no environments in metaworld report `done=True` when the task is complete / on task success. This means that as long as you aren't giving RL^2 the success signal, RL^2 *should* just learn to ignore the done signal (or learns it's an "end of episode" signal) for metaworld benchmarks.

Having said that, if you're training RL^2 by mixing together the done and success signals, then yes that should cause problems on these benchmarks. It's also not clear to me if giving access to the success signal is fair in the first place, so perhaps its best not to do that.
Repository: Farama-Foundation/Metaworld
Issue: Question of the reward of button press task.
Root cause: Validation Gap

Hi,

I use both the scripted policy in the Meta-World and the expert policy trained by SAC to rollout trajectories in the `Button Press` task. But the `near_object` flag in the returned `info` never turns `True`. When I try `Drawer Open` task, the `near_object` turns `True` when the arm approaches the drawer. I find that the `near_object` flag in the `Drawer Open` task is computed according the position of the `hand` of the arm, but the `near_object` in the `Button Press` task is computed based on the position of the center of `leftEndEffector` and `rightEndEffector`. What's the difference of these two positions? Is it more proper to use the position of the `hand` of the arm to compute `near_object` in the `Button Press` task?

Sorry for the delay. My thinking would be that for the drawer open task, the arm **needs** to grip the handle to open the drawer so near object is set up in a way that, in the reward computation, the agent is guided to grip the handle via the distance between the drawer handle and the hand. The button press top down task would not require gripping, and therefore the arm can keep the gripper closed and press the button with the tip(s) of the end effector "fingers." I hope this helps.  

Thanks for your response and it is clear. `near_object` flag of the button press task should be fixed since it never turns `True` using the scripted policy.
Repository: Farama-Foundation/Metaworld
Issue: Although seed is fixed, the randomness is still out of control.
Root cause: Documentation Deficiency

Hello.

Thanks for your wonderful project.

I found out that the randomness is out of control even if the seed is provided.

Here is snap-shot.

test_seed.py
```
import metaworld

seed = 777
task_id = "pick-place-v2"
task_idx = 0

benchmark = metaworld.ML1(task_id)
task = benchmark.train_tasks[task_idx]

env = benchmark.train_classes[task_id]()
env.set_task(task)
obs, _ = env.reset(seed=seed)
print(obs.mean())
```

Result
<img width="310" alt="image" src="https://github.com/Farama-Foundation/Metaworld/assets/61035345/9f2d4489-b4e4-4d17-a3c0-4472c4d9dcb0">



I suspect that the task generated is not the same, could you print that? 
@reginald-mclean Is there anyway that we could add a seed value when generating a task? This is probably a question when improving the API  

If you're using an environment within one of the benchmarks (ie MT10, or ML1) then you need to seed the creation of the benchmark not the individual environment. When you create the benchmark use seed, ie 
```env = metaworld.ML1(task_id, seed=seed)```

thanks !!
Repository: Farama-Foundation/Metaworld
Issue: Issues running MT50
Root cause: Architectural Constraint

Hi,

I really struggle to run MT50. In the beginning, it creates all the environments for the training and evaluation (100 environments so). It always crashes during any training.
I have quite a powerful computer and still, this does not work:
* Memory: 32GB
* CPU: Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz (18 cores, 36 threads)
* GPU: NVIDIA RTX A6000
* System: Ubuntu 20.04

Are there any tricks for running MT50? Thanks!

Hey, that is surprising that it is crashing. 
Can you confirm if it is a RAM issue thorough observing your system manager while running the script? 
Or identify what particular instruction seems to cause this issue?
Is there a minimal script to reproduce the issue that we can run?  

@reginald-mclean Any ideas on what the cause might be? 

These are multitask environments so you evaluate on the same environments you train on. So you might not need 2 copies of every env. Can you provide more info about your use case? What algorithms? 

Sorry for getting back to you so late. The repo I forked from was indeed creating both test and train environments while it was not necessary. This meant 100 environments which I think was too much for my hardware. Changing this to only 50 environments fixed the issue, thank you for you answer!
Repository: Farama-Foundation/Metaworld
Issue: No "self.sim" for v2 envs
Root cause: Architectural Constraint

I need to use sim to do the rendering things, but I found that for the latest version, `sim` is not there... Could anyone explain? 
I check other branches and find that in some branches `sim` is still there. I am just not sure why such changes are made and what should I do to get the rendering in this condition.
```
ipdb> env.sim
*** AttributeError: 'SawyerNutAssemblyEnvV2' object has no attribute 'sim'
```

I switch to an earlier commit temporarily, but maybe this issue still should be noted. 

It's because of a change in the Mujoco bindings. Instead of using env.sim, when you create an environment you can pass the render_type argument to the constructor. I am in the process of updating the single environment creation to handle that argument as well.
Repository: Farama-Foundation/Metaworld
Issue: Objects move slightly after reset
Root cause: External Dependency Breakage

Currently the CI fails on one of the tests where the observation and goal are compared after sequential resets. After changing to MuJoco from mujoco-py it seems that there are a few environments where issues arise because objects aren't in exactly the same places after resets. The current fix is to replace `len(np.unique(np.array(task_initial_pos), axis=0)) > 1` with `len(np.unique(np.array(task_initial_pos), axis=0)) > 1 and not np.allclose(
                task_initial_pos[0], task_initial_pos[1], rtol=1e-2, atol=1e-2
            )` but this masks the problem just to get the CI to pass

This may be because we need to add an extra ```mujoco.mj_forward(self.model, self.data)``` call to the reset/step functions. 

Fixed with PR #474
Repository: Farama-Foundation/Metaworld
Issue: Is the fixed env.reset() intended?
Root cause: Documentation Deficiency

Hello,

I wonder how to use single-goal environments refer to [Accessing Single Goal Environments](https://github.com/Farama-Foundation/Metaworld#accessing-single-goal-environments)
If I fix the seed of the environment,
```
from metaworld.envs import ALL_V2_ENVIRONMENTS_GOAL_OBSERVABLE
import numpy as np

door_open_goal_observable_cls = ALL_V2_ENVIRONMENTS_GOAL_OBSERVABLE["door-open-v2-goal-observable"]
env = door_open_goal_observable_cls(seed=0)

# Reset environment
obs = env.reset()  
obs = env.reset()
...
```
`obs` always returns the same observation including goal position.

My question is...
Is it possible to build an environment where the `obs` we get from env.reset() satisfies that **the goal is fixed and the rest (robotic arm's position) is randomized**?

Or is it intended that the initial state $s_0$ is always the same even when evaluating?

Sorry for the basic question, but I'm confused...

Thank you.

Hi! Yes if you use the environments individually, env.reset() returns the exact same values every time you reset. If you want to randomize the arm position you could randomly sample X actions and apply them in the environment before you start saving transitions. 

Oh! thank you for your reply and it was very helful!!
I will close the issue
thank you:)
Repository: Farama-Foundation/Metaworld
Issue: Add MJX support?
Root cause: Unimplemented Feature Gap


Currently blocked by Mujoco [issue](https://github.com/google-deepmind/mujoco/issues/1483)

We have isolated the Mujoco issue, we are now blocked on this by MJX not having support for all of the collisions that we need for Meta-World.
Repository: Farama-Foundation/Metaworld
Issue: Warnings in numpy >= 1.25
Root cause: External Dependency Breakage

Just FYI, as part of the migration / maintenance of `MetaWorld`, it might be important to take care of this `numpy` warning.

I've ran tests for all 50 metaworld envs for commit https://github.com/Farama-Foundation/Metaworld/commit/c822f28f582ba1ad49eb5dcf61016566f28003ba, and it occurs on `dial-turn-v2`, `door-open-v2`, `hammer-v2` and `lever-pull-v2`.

Here's an example stack trace from one of my tests:

```
self = <metaworld.envs.mujoco.sawyer_xyz.v2.sawyer_lever_pull_v2.SawyerLeverPullEnvV2 object at 0x7f330a01ebe0>
obs = array([ 0.0058129 ,  0.39943335,  0.19429615,  1.        ,  0.14983521,
        0.58119154,  0.24981888,  0.66279996, ...      ,  0.        ,  0.        ,  0.        ,  0.        ,
        0.        ,  0.        ,  0.        ,  0.        ])
action = array([ 0.2889182 , -0.93614006, -0.69611247, -0.55025635])

    @_assert_task_is_set
    def evaluate_state(self, obs, action):
        (
            reward,
            shoulder_to_lever,
            ready_to_lift,
            lever_error,
            lever_engagement,
        ) = self.compute_reward(action, obs)
    
        info = {
>           "success": float(lever_error <= np.pi / 24),
            "near_object": float(shoulder_to_lever < 0.03),
            "grasp_success": float(ready_to_lift > 0.9),
            "grasp_reward": ready_to_lift,
            "in_place_reward": lever_engagement,
            "obj_to_target": shoulder_to_lever,
            "unscaled_reward": reward,
        }
E       DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)

../../anaconda3/envs/rlcore/lib/python3.9/site-packages/metaworld/envs/mujoco/sawyer_xyz/v2/sawyer_lever_pull_v2.py:79: DeprecationWarning
```

It's harmless for now (I think), but it might break the code after a future version of `numpy`

This should be fixed by #475
Repository: Farama-Foundation/Metaworld
Issue: Camera view?
Root cause: Documentation Deficiency

Does anyone know how to change the camera view when rendering? 
It seems there are several camera view modes (topview, corner, etc.), but i can't find a way to set the camera_name attribute, or make it work.

 Hi @Hong753. I think there is an issue passing the selected camera from MW -> Gymnasium. I had to modify the Gymnasium mujoco_env.py render function to accept arguments then I was able to render from the desired camera. 

@reginald-mclean hey, can you please tell more about the change you mentioned?
I tried to follow the code trail, but couldn't quite figure it out.

I found camera_id to be passed in some places, and though that would work, but that too didn't change the camera view.

Thanks,
Repository: Farama-Foundation/Metaworld
Issue: Contributing.md setup isn't accurate
Root cause: Documentation Deficiency

Looking to set up for PRs helping with Metaworld. Tried the command 

`pip install -e .[dev]` 

but it says 

`zsh: no matches found: .[dev]`

I'm happy to put together a PR for this, but because its trivial i thought I would just report it

Specifically, we should highlight that you need quotes around it  for some shells like `zsh`:

`pip install -e '.[dev]' `

and even then there is a warning associated with `dev` not being an extra:
`WARNING: metaworld 2.0.0 does not provide the extra 'dev'`                                                                                   

Hi @balloch!
Sorry for the delay in getting back to you. We recently started using the pyproject.toml file to install dependencies, could you try ```pip install -e '.[testing]'``` after cloning the MW repo? That worked for me on my system.
Repository: Farama-Foundation/Metaworld
Issue: `SawyerXYZEnv` is not seeded correctly
Root cause: Algorithmic Error

`SawyerXYZEnv` has a method `.seed()`. Intuitively, this is supposed to ensure that seeded env creation is deterministic.

However, this is not the case at the moment. I wrote this simple test which fails on `main`:

```python
from metaworld.envs import ALL_V2_ENVIRONMENTS_GOAL_OBSERVABLE

MetaworldEnvName = str

def setup_metaworld_env(
    env_name: MetaworldEnvName, 
    seed: int = 0,
) -> GymEnv:
    env_cls = ALL_V2_ENVIRONMENTS_GOAL_OBSERVABLE[env_name]
    e: SawyerXYZEnv = env_cls(render_mode="rgb_array")

    # Hack: enable random reset
    e._freeze_rand_vec = False
    e.seed(seed)
    return e

def test_seeded_env_creation_is_deterministic():
    env1 = setup_metaworld_env("drawer-open-v2-goal-observable", "top_cap2", seed=0)
    obs1, _ = env1.reset()
    img1 = env1.render()
    
    env2 = setup_metaworld_env("drawer-open-v2-goal-observable", "top_cap2", seed=0)
    obs2, _ = env2.reset()
    img2 = env2.render()

    assert np.allclose(obs1, obs2)
    assert np.allclose(img1, img2)
```



I traced the error to this section: https://github.com/Farama-Foundation/Metaworld/blob/c822f28f582ba1ad49eb5dcf61016566f28003ba/metaworld/envs/mujoco/sawyer_xyz/sawyer_xyz_env.py#L547C1-L551C33

We should use `self.np_random` generator, which is seeded by `.seed()`, as opposed to using the unseeded `np.random` generator. After making this change the test passes. 

@dtch1997 thank you for pointing that out. If you want to make a PR to fix this, feel free
Repository: Farama-Foundation/Metaworld
Issue: Skipping tests for env gym.envs.atari:AtariEnv
Root cause: Environment Incompatibility

I'm trying to collect data from each environment in ML45, but I don't know why the following output appears when I run it:
<img width="410" alt="image" src="https://github.com/Farama-Foundation/Metaworld/assets/94092862/54aa9213-5fca-457a-8d03-6c2584a96d59">
Can anyone tell me what this is about? I was able to provide my source code. Huge thanks!
Repository: Farama-Foundation/Metaworld
Issue: ModuleNotFoundError: No module named 'metaworld.policies.sawyer_stick_push_policy'
Root cause: Environment Incompatibility

code: from metaworld.policies.sawyer_stick_push_policy import SawyerStickPushV1Policy
ModuleNotFoundError: No module named 'metaworld.policies.sawyer_stick_push_policy'
How can I solve this problem? My metaworld version is 2.0.0.


Is this a statement from a previous version？

Hi! I apologize for the delay, I was away for a bit. What code are you attempting to run? 

Closing this due to inactivity.
Repository: Farama-Foundation/Metaworld
Issue: benchmark environment for training
Root cause: Documentation Deficiency

I would like to know how should I set up my training environment and testing environment when I use ML45 or ML10 benchmark environment for agent training? For example, when I use metaworld's single_goal (ALL_V2_ENVIRONMENTS_GOAL_OBSERVABLE), my code is as follows:
```
env_cls = ALL_V2_ENVIRONMENTS_GOAL_OBSERVABLE["door-open-v2-goal-observable"]
train_env = env_cls(seed=SEED)
eval_env = env_cls(seed=SEED)
```
but when I use ML45 and ML10 benchmark to create the environment,
```
env = env_cls()  # render_mode="rgb_array"
task = random.choice([task for task in ml45.train_tasks if task.env_name == name])
train_env = env.set_task(task)
eval_env = env.set_task(task)
```
In this case, for each of the environment classes in ML45, I am taking an iterative approach:
```
ml45 = metaworld.ML45(seed=42)

    param_list = []
    for i, (name, env_cls) in enumerate(ml45.train_classes.items()):
```
there are all sorts of problems, such as: 
_AttributeError: 'NoneType' object has no attribute 'observation_space'_
very Thanks for your help!