Repository: pytorch/rl

Issue:
[BUG] `TransformedEnv` does not copy env properties
When wrapping an env in `TransofrmedEnv`, its properties are changed

```python
from torchrl.envs import VmasEnv, RewardSum, TransformedEnv

if __name__ == "__main__":
    env = VmasEnv("balance", num_envs=3)
    print(env._allow_done_after_reset)
    t_env = TransformedEnv(env, RewardSum())
    print(t_env._allow_done_after_reset)

```
```
True
False
```


Discussion:
Is this a generic bug or just specific to `_allow_done_after_reset`?

Not sure it's really a bug in general, there are instances where one would like to have different attributes in the transformed env and in the env.
For some (eg `_allow_done_after_reset`?) we can have fallbacks if we're confident that there is no reason for TransformedEnv to have a different behaviour. It's already the case for batch_size, device, run_type_checks and others.

For instance, I can imagine that if one is using the no-op transform she expects the behaviour of `_allow_done_after_reset` to change. 
We can default to the attribute value in the original env and the specific transforms can change it.

I think it is counterintuitive that, if the transform does not need to change the property, the wrapper changes it.

Not sure if it is just specific to this

Resolving patch:
diff --git a/torchrl/envs/transforms/transforms.py b/torchrl/envs/transforms/transforms.py
--- a/torchrl/envs/transforms/transforms.py
+++ b/torchrl/envs/transforms/transforms.py
@@ -572,7 +572,7 @@ def __init__(
             env = env.to(device)
         else:
             device = env.device
-        super().__init__(device=None, **kwargs)
+        super().__init__(device=None, allow_done_after_reset=None, **kwargs)
 
         if isinstance(env, TransformedEnv):
             self._set_env(env.base_env, device)
@@ -679,6 +679,18 @@ def run_type_checks(self, value):
             "run_type_checks is a read-only property for TransformedEnvs"
         )
 
+    @property
+    def _allow_done_after_reset(self) -> bool:
+        return self.base_env._allow_done_after_reset
+
+    @_allow_done_after_reset.setter
+    def _allow_done_after_reset(self, value):
+        if value is None:
+            return
+        raise RuntimeError(
+            "_allow_done_after_reset is a read-only property for TransformedEnvs"
+        )
+
     @property
     def _inplace_update(self):
         return self.base_env._inplace_update


Test patch:
diff --git a/test/mocking_classes.py b/test/mocking_classes.py
--- a/test/mocking_classes.py
+++ b/test/mocking_classes.py
@@ -111,6 +111,7 @@ def __init__(
         super().__init__(
             device=kwargs.pop("device", "cpu"),
             dtype=torch.get_default_dtype(),
+            allow_done_after_reset=kwargs.pop("allow_done_after_reset", False),
         )
         self.set_seed(seed)
         self.is_closed = False
diff --git a/test/test_transforms.py b/test/test_transforms.py
--- a/test/test_transforms.py
+++ b/test/test_transforms.py
@@ -7691,6 +7691,21 @@ def test_independent_reward_specs_from_shared_env(self):
         assert base_env.reward_spec.space.minimum == -np.inf
         assert base_env.reward_spec.space.maximum == np.inf
 
+    def test_allow_done_after_reset(self):
+        base_env = ContinuousActionVecMockEnv(allow_done_after_reset=True)
+        assert base_env._allow_done_after_reset
+        t1 = TransformedEnv(
+            base_env, transform=RewardClipping(clamp_min=0, clamp_max=4)
+        )
+        assert t1._allow_done_after_reset
+        with pytest.raises(
+            RuntimeError,
+            match="_allow_done_after_reset is a read-only property for TransformedEnvs",
+        ):
+            t1._allow_done_after_reset = False
+        base_env._allow_done_after_reset = False
+        assert not t1._allow_done_after_reset
+
 
 def test_nested_transformed_env():
     base_env = ContinuousActionVecMockEnv()