Repository: Farama-Foundation/Metaworld
Issue: Possible BUG report
Root cause: Documentation Deficiency

Hi all, thanks for this amazing repo for meta RL and it's really helpful 👍. However, I have encountered some problems recently, and it seems there have few bugs in the codes.

I'm trying to modify the goal of tasks sampled from `pick-place-v1`, but it fails. And I try to look at the source codes to fix this problem: 

In [sawyer_xyz/base.py](https://github.com/rlworkgroup/metaworld/blob/9f613fe967349c00c6367af53ece499964253793/metaworld/envs/mujoco/sawyer_xyz/base.py#L180), the `set_goal_()` method deals with the attribute `self.goal`, while in [sawyer_pick_and_place.py](https://github.com/rlworkgroup/metaworld/blob/master/metaworld/envs/mujoco/sawyer_xyz/sawyer_pick_and_place.py), it seems the task computes rewards based on `self._state_goal`.

I'm so confused about this. Are they the same? Or it's just a bug of inconsistent names? I hope someone can help me with this. Thanks a lot!


BTW, it would be much better if there is a documentation about the repo in the future 😊!


Derek

@Derek-Wds thanks for the issue report. 

@zhanpenghe Can you try to address @Derek-Wds 's question?

**re: documentation**
I would also love more documentation, especially developer documentation. metaworld is an open source project, and we rely on the great work and kindness of people like you to stay alive. 

If you'd like to submit a PR with some documentation on metaworld internals, I'd love to review it and submit it!

Hi @Derek-Wds, thanks for the report! The state goal is actually always reset with the goal in the environment reset method so there should not be any problem. The reason why there are two representations of them is we would like to create an easy to use multitask wrapper. There was a time constraint for us to work on the software engineering side due to the paper deadline and we would like to make minimal changes so that we are confident about the results. 

Hi @zhanpenghe, thanks for the clarification. It seems I have looked into wrong file. The `pick-place-v1` environment is implemented in [sawyer_reach_push_pick_place.py](https://github.com/rlworkgroup/metaworld/blob/master/metaworld/envs/mujoco/sawyer_xyz/sawyer_reach_push_pick_place.py). Everything is clear to me now. 

I would love to make some documentations later if I have time after I explore more with metaworld. Thanks again for your help and contribution to this project. I will close this issue since I have got my answers.

Derek

Hi, I just have another quick question: why the goal is not changing when I use the `set_task` method in environment `pick-place-v1`? And how does the `ML1` is exactly working (since the README is a little bit abstract and unspecific)? Thanks in advance!

Derek

> why the goal is not changing when I use the set_task method in environment `pick-place-v1`?

I'm not sure what you mean by this. Can you provide a code snippet for what you are trying to do, and describe what happens and what you expected to happen?

> And how does the ML1 is exactly working (since the README is a little bit abstract and unspecific)?

I'm not sure how to address this question. Can you make it more specific? If you want to see how it works in detail, the source code is [here](https://github.com/rlworkgroup/metaworld/blob/4528f84dab357d562824dac3edf159bc68399f23/metaworld/benchmarks/ml1.py). You might also find the [technical report](https://arxiv.org/abs/1910.10897) valuable for understanding the precise motivation and formulation of the benchmarks.

Hi @ryanjulian, thanks for your reply and sorry for the unclear question. The code about what I want to do is here:
```python
from metaworld.benchmarks import ML1
env = ML1.get_train_tasks('pick-place-v1')
tasks = env.sample_tasks(2)
env.set_task(tasks[0])
goal1 = env.active_env.get_goal()
env.set_task(tasks[1])
goal2 = env.active_env.get_goal()
print(goal1)
print(goal2)
```
Since I sampled two tasks from the environment, I was expecting the goal I get would be different if I set different tasks to the environment. However, the `goal1` and `goal2` I get are the same.

Hope this clarifies things. As for the second question, I guess I should explore more about this benchmarks and hopefully would understand it better.

I was able to replicate the behavior you described.

I agree the API is confusing, though the behavior is not incorrect. New tasks only take effect once an environment is reset.

If you amend your snippet to:
```
from metaworld.benchmarks import ML1
env = ML1.get_train_tasks('pick-place-v1')
tasks = env.sample_tasks(2)
env.set_task(tasks[0])
env.reset()
goal1 = env.active_env.get_goal()
env.set_task(tasks[1])
env.reset()
goal2 = env.active_env.get_goal()
print(goal1)
print(goal2)
```

I believe you will get the expected behavior.