Repository: Farama-Foundation/Metaworld
Issue: Spec for metaworld envrionments
Root cause: Unimplemented Feature Gap

Currently metaworld environments inheriting from `gym.Env` don't have the `spec` attribute. This creates inconsistent behavior between metaworld envs and gym envs. It would be nice to populate this attribute, and put `max_episode_length` inside.

Maybe building upon this remark, it seems that all environments have a step() function returning unconditionally "None" as the third variable where it is supposed to be a "done" indicator on Gym environment definition. It seems to produce inconvenient behavior when using RL frameworks like stable-baselines which are assuming such behavior of the step function. Would be more than happy to help if necessary.

@perezjln metaworld environments never produce a termination signal (i.e. `done` is always `False`). The RL algorithm library needs to implement its own `max_episode_length` for sampling, and take care not to step past the maximum allowed episode length for each environment (or else metaworld will raise a `RuntimeError`).

If your RL algorithm library is assuming that all environments are finite-horizon OR it is assuming that infinite-horizon environments will corrupt the termination (`done`) signal in order to implement fixed-horizon sampling, I suggest you file a an issue with the library authors instead :). An MDP which never produces a termination signal is a perfectly-valid MDP, and there are many non-degenerate non-terminating MDPs.

Overloading the termination signal to enforce a maximum episode length (as is done in gym) corrupts the definition of an infinite-horizon MDP, and can cause off-policy algorithms to fail on these otherwise-solvable infinite-horizon problems. 

How? Q-value estimation for a terminal step (`done == True`) usually needs to be computed differently than a non-terminal step (i.e. off-policy algorithms usually use bootstrapping to compute the target Q-value of a non-terminal). Off-policy algorithms frequently have to reach into private fields of `gym` environments to fix this bug, and we at metaworld would prefer to simply get it right the first time instead.

Are you sure that the third return value is `None` rather than `False`? The `done` signal should be a boolean. If you are encountering `None` 

Dear @ryanjulian, thanks very much for your detailed answer. 

Maybe as a middle ground, letting the decision of an environment to be considered as a finite-horizon or infinite-horizon manner could be at the discretion of the end-user of the library, what do you think?

Maybe we will just fork and propose an update in that sense.
 
Regarding your point on Q-learning that I fully share, maybe it could be answered that it is more a problem of current off-policy learning algorithms than anything else :-) Furthermore, in practice, papers mainly seem to deal with finite-horizon task anyway while I fully agree that infinite-horizon MDP does exist and are worth considering.

Finally, "we at meta world would prefer to simply get it right the first time instead", once again assuming a contact-rich manipulation task can not be episodic seems more like a choice than the unilateral right thing to do, but I'm maybe wrong :-)

Once again, thanks for all your hard work with this project!

As Meta-World ownership has been transferred from the RLWorkGroup to [Farama Foundation](https://farama.org/), I am closing this issue. If there are any questions or requests for features please join our [Discord](https://discord.gg/PfR7a79FpQ)