internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: OCA/social

Issue:
[10.0] mail_sendrid, Error: No module named simplejson
> File ".../addons/oca-social/mail_sendgrid/controllers/json_request.py", line 4, in <module>
>     import simplejson
> ImportError: No module named simplejson

I put `simplejson` into file requirements.txt and it solved this issue.


Discussion:
Version and module in the title, please.

Resolving patch:
diff --git a/mail_sendgrid/__manifest__.py b/mail_sendgrid/__manifest__.py
--- a/mail_sendgrid/__manifest__.py
+++ b/mail_sendgrid/__manifest__.py
@@ -1,9 +1,9 @@
 # -*- coding: utf-8 -*-
-# Copyright 2015-2017 Compassion CH (http://www.compassion.ch)
+# Copyright 2015-2018 Compassion CH (http://www.compassion.ch)
 # License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl).
 {
     'name': 'SendGrid',
-    'version': '10.0.1.0.0',
+    'version': '10.0.1.0.1',
     'category': 'Social Network',
     'author': 'Compassion CH, Odoo Community Association (OCA)',
     'license': 'AGPL-3',
@@ -20,6 +20,6 @@
     'installable': True,
     'auto_install': False,
     'external_dependencies': {
-        'python': ['sendgrid'],
+        'python': ['sendgrid', 'simplejson'],
     },
 }
diff --git a/mail_sendgrid/controllers/json_request.py b/mail_sendgrid/controllers/json_request.py
--- a/mail_sendgrid/controllers/json_request.py
+++ b/mail_sendgrid/controllers/json_request.py
@@ -1,12 +1,18 @@
 # -*- coding: utf-8 -*-
 # Copyright 2016-2017 Compassion CH (http://www.compassion.ch)
 # License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl).
-import simplejson
-
+import logging
 from odoo.http import JsonRequest, Root, Response
 
 # Monkeypatch type of request rooter to use RESTJsonRequest
 old_get_request = Root.get_request
+_logger = logging.getLogger(__name__)
+
+try:
+    import simplejson
+except ImportError:
+    _logger.error("Please install simplejson tu use mail_sendgrid module")
+    _logger.debug("ImportError details:", exc_info=True)
 
 
 def get_request(self, httprequest):


Test patch:
Repository: OCA/social

Issue:
[12.0][mail_tracking] many MailTracking email not found warnings
This code snippet is polluting my logs with warning messages.

https://github.com/OCA/social/blob/8d985d8da9fa864113f87cd59a2b3173f7f89193/mail_tracking/controllers/main.py#L69-L78

How can this work if state will be marked 'opened' the first time? Each successive open will trigger above warning, because state is now 'opened' and will not match the above domain `('state', 'in', ['sent', 'delivered'])`.

 Is it intended to generate warnings each time mail is opened? Or what am I missing?


Discussion:
Indeed it seems something weird. Please open a PR doing the search without the state and having the warning if not found, but comparing later if in the required state for doing the call o the event create.

Resolving patch:
diff --git a/mail_tracking/controllers/main.py b/mail_tracking/controllers/main.py
--- a/mail_tracking/controllers/main.py
+++ b/mail_tracking/controllers/main.py
@@ -68,14 +68,13 @@ def mail_tracking_open(self, db, tracking_email_id, token=False, **kw):
             try:
                 tracking_email = env['mail.tracking.email'].search([
                     ('id', '=', tracking_email_id),
-                    ('state', 'in', ['sent', 'delivered']),
                     ('token', '=', token),
                 ])
-                if tracking_email:
-                    tracking_email.event_create('open', metadata)
-                else:
+                if not tracking_email:
                     _logger.warning(
                         "MailTracking email '%s' not found", tracking_email_id)
+                elif tracking_email.state in ('sent', 'delivered'):
+                    tracking_email.event_create('open', metadata)
             except Exception:
                 pass
 


Test patch:
Repository: OCA/social

Issue:
[13.0] [BUG]"base_search_mail_content" module > Getting bug with "hr" (Employees) module
module: base_search_mail_content
version: 13.0

**Context :**
OCB 13.0 Odoo Server up to date [(08/30/2020),]
Virgin database , to reproduce issue faced on my test environnement.
Also !! >> Get same bug on runbot : http://3437172-13-0-56e0a2.runbot2-2.odoo-community.org

**Steps to reproduce**
- Install together  "base_search_mail_content" & "hr" (Employees) native odoo module, and try to access to : hr" (Employees)

**Current behavior**  (model=hr.employee&view_type=kanban, or tree)
When i try to access to menu "Employees"There is this following message : 
> Something went wrong !
Only types ['many2one'] are supported for category (found type text)

**Current resolution**
i uninstall "base_search_mail_content" to retreive access to hr" (Employees)

-----------------------------------------------------------------------------------------------------------------------
![Capture-Access _employees-1](https://user-images.githubusercontent.com/59052920/91721693-74c7db80-eb99-11ea-8871-c5a5b0b21eac.JPG)
![Capture-Access _employees](https://user-images.githubusercontent.com/59052920/91721766-932dd700-eb99-11ea-8c2b-12a280df4217.JPG)




Discussion:
Same here: http://3440272-13-0-7d4692.runbot2-2.odoo-community.org/web#action=&cids=1&menu_id=284
with Recruitment

Resolving patch:
diff --git a/base_search_mail_content/models/mail_thread.py b/base_search_mail_content/models/mail_thread.py
--- a/base_search_mail_content/models/mail_thread.py
+++ b/base_search_mail_content/models/mail_thread.py
@@ -50,8 +50,7 @@ def fields_view_get(
             res["fields"].update(
                 {"message_content": {"type": "char", "string": _("Message Content")}}
             )
-
-            for node in doc.xpath("//field[last()]"):
+            for node in doc.xpath("/search/field[last()]"):
                 # Add message_content in search view
                 elem = etree.Element("field", {"name": "message_content"})
                 node.addnext(elem)


Test patch:
Repository: OCA/social

Issue:
[15.0] mail_tracking: portal tests fail
<!-- Provide a general summary of the issue in the Title above.
Before opening the issue, please check that there isn't another issue addressing the same problem already created -->

## Module
mail_tracking

## Describe the bug
mail_tracking causes TestPortalWizard.test_portal_wizard_public_user and TestPortalWizard.test_portal_wizard_partner to fail

## To Reproduce
**Affected versions**: 15.0

Steps to reproduce the behavior:
1. init db with portal and mail_tracking
2. run portal tests



Discussion:

Resolving patch:
diff --git a/mail_tracking/models/ir_mail_server.py b/mail_tracking/models/ir_mail_server.py
--- a/mail_tracking/models/ir_mail_server.py
+++ b/mail_tracking/models/ir_mail_server.py
@@ -45,10 +45,10 @@ def build_email(
         if tracking_email_id:
             headers = self._tracking_headers_add(tracking_email_id, headers)
         msg = super(IrMailServer, self).build_email(
-            email_from,
-            email_to,
-            subject,
-            body,
+            email_from=email_from,
+            email_to=email_to,
+            subject=subject,
+            body=body,
             email_cc=email_cc,
             email_bcc=email_bcc,
             reply_to=reply_to,


Test patch:
Repository: pytorch/rl

Issue:
[BUG] Loggers registration
## Describe the bug

The [`__init__.py`](https://github.com/facebookresearch/rl/blob/main/torchrl/trainers/loggers/__init__.py) from loggers does not reference the implemented loggers. However, the [docstring from the trainer](https://github.com/facebookresearch/rl/blob/806733f27dfa9a878b75b079de9f18df83f54c2d/torchrl/trainers/helpers/trainers.py#L109) assumes that this kind of import can be run.


Discussion:

Resolving patch:
diff --git a/torchrl/trainers/loggers/__init__.py b/torchrl/trainers/loggers/__init__.py
--- a/torchrl/trainers/loggers/__init__.py
+++ b/torchrl/trainers/loggers/__init__.py
@@ -4,3 +4,6 @@
 # LICENSE file in the root directory of this source tree.
 
 from .common import Logger
+from .csv import CSVLogger
+from .tensorboard import TensorboardLogger
+from .wandb import WandbLogger


Test patch:
diff --git a/test/test_trainer.py b/test/test_trainer.py
--- a/test/test_trainer.py
+++ b/test/test_trainer.py
@@ -13,7 +13,7 @@
 
 try:
     from tensorboard.backend.event_processing import event_accumulator
-    from torchrl.trainers.loggers import TensorboardLogger
+    from torchrl.trainers.loggers.tensorboard import TensorboardLogger
 
     _has_tb = True
 except ImportError:
@@ -242,6 +242,7 @@ def test_recorder():
         args.record_frames = 24 // args.frame_skip
         args.record_interval = 2
         args.catframes = 4
+        args.collector_devices = ["cpu"]
 
         N = 8
Repository: pytorch/rl

Issue:
[BUG] FlattenObservation specs cannot be retrieved when first_dim!=0
## Describe the bug

`FlattenObservation` transformed features and specs conflicts when the environment has a batch size and the dimensions are greater than 0.

## To Reproduce

Steps to reproduce:

```python
>>> env = TransformedEnv(ParallelEnv(2, lambda:DMControlEnv("cheetah", "run", from_pixels=True)), FlattenObservation(1, -1, keys_in=["next_pixels"]))
>>> env.observation_spec["next_pixels"].shape
torch.Size([240, 960])
>>> env.reset()
TensorDict(
    fields={
        done: Tensor(torch.Size([2, 1]), dtype=torch.bool),
        pixels: Tensor(torch.Size([2, 230400]), dtype=torch.uint8),
        position: Tensor(torch.Size([2, 8]), dtype=torch.float64),
        velocity: Tensor(torch.Size([2, 9]), dtype=torch.float64)},
    batch_size=torch.Size([2]),
    device=cpu,
    is_shared=False)
 ```
Hence the flattened `pixels` don't have the expect shape.

## Expected behavior

The transformed feature shape and the transformed observation spec should match.


Discussion:

Resolving patch:
diff --git a/torchrl/envs/transforms/transforms.py b/torchrl/envs/transforms/transforms.py
--- a/torchrl/envs/transforms/transforms.py
+++ b/torchrl/envs/transforms/transforms.py
@@ -1010,10 +1010,24 @@ def _apply_transform(self, observation: torch.Tensor) -> torch.Tensor:
         observation = torch.flatten(observation, self.first_dim, self.last_dim)
         return observation
 
+    def set_parent(self, parent: Union[Transform, EnvBase]) -> None:
+        out = super().set_parent(parent)
+        observation_spec = self.parent.observation_spec
+        for key in self.keys_in:
+            if key in observation_spec:
+                observation_spec = observation_spec[key]
+                if self.first_dim >= 0:
+                    self.first_dim = self.first_dim - len(observation_spec.shape)
+                if self.last_dim >= 0:
+                    self.last_dim = self.last_dim - len(observation_spec.shape)
+                break
+        return out
+
     @_apply_to_composite
     def transform_observation_spec(self, observation_spec: TensorSpec) -> TensorSpec:
         observation_spec = deepcopy(observation_spec)
         space = observation_spec.space
+
         if isinstance(space, ContinuousBox):
             space.minimum = self._apply_transform(space.minimum)
             space.maximum = self._apply_transform(space.maximum)
diff --git a/torchrl/trainers/helpers/envs.py b/torchrl/trainers/helpers/envs.py
--- a/torchrl/trainers/helpers/envs.py
+++ b/torchrl/trainers/helpers/envs.py
@@ -129,7 +129,7 @@ def make_env_transforms(
         env.append_transform(Resize(cfg.image_size, cfg.image_size))
         if cfg.grayscale:
             env.append_transform(GrayScale())
-        env.append_transform(FlattenObservation(first_dim=batch_dims))
+        env.append_transform(FlattenObservation())
         env.append_transform(CatFrames(N=cfg.catframes, keys_in=["next_pixels"]))
         if stats is None:
             obs_stats = {"loc": 0.0, "scale": 1.0}


Test patch:
Repository: pytorch/rl

Issue:
[BUG] importing `dm_control` environments when the library is not installed does not display the right error message
## Describe the bug

We allow users to load the file `envs/libs/dm_control.py` even if `dm-control` is not installes. When creating an object in this situation, the error message now is that the environment configuration cannot be found. An `ImportError` should be raised instead (after a check for `_has_dmc` has been made).


Discussion:

Resolving patch:
diff --git a/torchrl/envs/libs/dm_control.py b/torchrl/envs/libs/dm_control.py
--- a/torchrl/envs/libs/dm_control.py
+++ b/torchrl/envs/libs/dm_control.py
@@ -272,6 +272,11 @@ class DMControlEnv(DMControlWrapper):
     """
 
     def __init__(self, env_name, task_name, **kwargs):
+        if not _has_dmc:
+            raise ImportError(
+                "dm_control python package was not found."
+                "Please install this dependency."
+            )
         kwargs["env_name"] = env_name
         kwargs["task_name"] = task_name
         super().__init__(**kwargs)


Test patch:
Repository: pytorch/rl

Issue:
[BUG] `inferece_mode` decorator is preventing `state_dict` loading
## Describe the bug

See title

## To Reproduce

Running example command `python sac.py` results in the following error:
```
Traceback (most recent call last):
  File "/Users/haoranpeng/Desktop/rl/examples/sac/sac.py", line 198, in main
    recorder_rm.load_state_dict(create_env_fn().state_dict())
  File "/Users/haoranpeng/Desktop/rl/torchrl/envs/transforms/transforms.py", line 418, in load_state_dict
    self.transform.load_state_dict(state_dict, **kwargs)
  File "/Users/haoranpeng/mambaforge/envs/torch_rl/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1667, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Compose:
        While copying the parameter named "transforms.2.loc", whose dimensions in the model are torch.Size([17]) and whose dimensions in the checkpoint are torch.Size([17]), an exception occurred : ('Inplace update to inference tensor outside InferenceMode is not allowed.
```

## Reason and Possible fixes

Remove `inference_mode` decorator from function `torchrl.trainers.helpers.envs.get_stats_random_rollout()`

## Checklist

- [x] I have checked that there is no similar issue in the repo (**required**)
- [x] I have read the [documentation](https://github.com/pytorch/rl/tree/main/docs/) (**required**)
- [x] I have provided a minimal working example to reproduce the bug (**required**)


Discussion:

Resolving patch:
diff --git a/torchrl/trainers/helpers/envs.py b/torchrl/trainers/helpers/envs.py
--- a/torchrl/trainers/helpers/envs.py
+++ b/torchrl/trainers/helpers/envs.py
@@ -351,7 +351,7 @@ def parallel_env_constructor(
     return parallel_env
 
 
-@torch.inference_mode()
+@torch.no_grad()
 def get_stats_random_rollout(
     cfg: "DictConfig",  # noqa: F821
     proof_environment: EnvBase = None,


Test patch:
Repository: pytorch/rl

Issue:
[BUG] Argument for `PPOLoss.get_entropy_bonus()` should probably not be optional
## Describe the bug

https://github.com/pytorch/rl/blob/86a1a139682edc8bc747f088423937db1773aa24/torchrl/objectives/costs/ppo.py#L88-L94

The code doesn't seem to be compatible with `None` argument, will throw `AttributeError`.

- [x] I have checked that there is no similar issue in the repo (**required**)
- [x] I have read the [documentation](https://github.com/pytorch/rl/tree/main/docs/) (**required**)
- [x] I have provided a minimal working example to reproduce the bug (**required**)



Discussion:
good point, do you want me to take care of it?
> good point, do you want me to take care of it?

please do thanks

Resolving patch:
diff --git a/torchrl/objectives/costs/ppo.py b/torchrl/objectives/costs/ppo.py
--- a/torchrl/objectives/costs/ppo.py
+++ b/torchrl/objectives/costs/ppo.py
@@ -85,7 +85,7 @@ def __init__(
     def reset(self) -> None:
         pass
 
-    def get_entropy_bonus(self, dist: Optional[d.Distribution] = None) -> torch.Tensor:
+    def get_entropy_bonus(self, dist: d.Distribution) -> torch.Tensor:
         try:
             entropy = dist.entropy()
         except NotImplementedError:


Test patch:
Repository: pytorch/rl

Issue:
[BUG] timeit profiling class does not correctly track how many times a function has been called.
## Describe the bug

In file **pytorch/rl/torchrl/_utils.py**, in the class **timeit**, starting at line 31, the code is the following

```
count = self._REG[self.name][1]
self._REG[self.name][0] = (self._REG[self.name][0] * count + t) / (count + 1)
self._REG[self.name][1] = self._REG[self.name][1] + t
self._REG[self.name][2] = count + 1
```

I understand self._REG[self.name][2] tracks the number of times a function has been called.
In that case, shouldn't the first line be changed to 

```
count = self._REG[self.name][2]
```

## Checklist

- [X] I have checked that there is no similar issue in the repo (**required**)
- [  ] I have read the [documentation](https://github.com/pytorch/rl/tree/main/docs/) (**required**)
- [  ] I have provided a minimal working example to reproduce the bug (**required**)



Discussion:

Resolving patch:
diff --git a/torchrl/_utils.py b/torchrl/_utils.py
--- a/torchrl/_utils.py
+++ b/torchrl/_utils.py
@@ -26,12 +26,13 @@ def __enter__(self):
 
     def __exit__(self, exc_type, exc_val, exc_tb):
         t = time.time() - self.t0
-        self._REG.setdefault(self.name, [0.0, 0.0, 0])
+        val = self._REG.setdefault(self.name, [0.0, 0.0, 0])
 
-        count = self._REG[self.name][1]
-        self._REG[self.name][0] = (self._REG[self.name][0] * count + t) / (count + 1)
-        self._REG[self.name][1] = self._REG[self.name][1] + t
-        self._REG[self.name][2] = count + 1
+        count = val[2]
+        N = count + 1
+        val[0] = val[0] * (count / N) + t / N
+        val[1] += t
+        val[2] = N
 
     @staticmethod
     def print(prefix=None):


Test patch:
diff --git a/test/test_helpers.py b/test/test_helpers.py
--- a/test/test_helpers.py
+++ b/test/test_helpers.py
@@ -5,10 +5,12 @@
 
 import argparse
 import dataclasses
+from time import sleep
 
 import pytest
 import torch
 from _utils_internal import generate_seeds, get_available_devices
+from torchrl._utils import timeit
 
 try:
     from hydra import compose, initialize
@@ -627,6 +629,32 @@ def test_seed_generator(initial_seed):
     assert seeds0 == seeds1
 
 
+def test_timeit():
+    n1 = 500
+    w1 = 1e-4
+    n2 = 200
+    w2 = 1e-4
+    w3 = 1e-4
+    # warmup
+    for _ in range(10):
+        sleep(w1)
+    for _ in range(n1):
+        with timeit("event1"):
+            sleep(w1)
+        sleep(w3)
+    for _ in range(n2):
+        with timeit("event2"):
+            sleep(w2)
+    val1 = timeit._REG["event1"]
+    val2 = timeit._REG["event2"]
+    assert abs(val1[0] - w1) < 1e-2
+    assert abs(val1[1] - n1 * w1) < 1
+    assert val1[2] == n1
+    assert abs(val2[0] - w2) < 1e-2
+    assert abs(val2[1] - n2 * w2) < 1
+    assert val2[2] == n2
+
+
 if __name__ == "__main__":
     args, unknown = argparse.ArgumentParser().parse_known_args()
     pytest.main([__file__, "--capture", "no", "--exitfirst"] + unknown)
Repository: pytorch/rl

Issue:
[BUG] `TensorDict.squeeze` does not support an null `dim` argument
## Describe the bug

While `torch.squeeze` can squeeze a tensor of all its singleton dimensions, the squeeze operation over a tensordict does not.
This discrepancy should be solved.


Discussion:

Resolving patch:
diff --git a/torchrl/data/tensordict/tensordict.py b/torchrl/data/tensordict/tensordict.py
--- a/torchrl/data/tensordict/tensordict.py
+++ b/torchrl/data/tensordict/tensordict.py
@@ -1159,13 +1159,28 @@ def unsqueeze(self, dim: int) -> TensorDictBase:
             inv_op_kwargs={"dim": dim},
         )
 
-    def squeeze(self, dim: int) -> TensorDictBase:
+    def squeeze(self, dim: Optional[int] = None) -> TensorDictBase:
         """Squeezes all tensors for a dimension comprised in between `-td.batch_dims+1` and `td.batch_dims-1` and returns them in a new tensordict.
 
         Args:
-            dim (int): dimension along which to squeeze
+            dim (Optional[int]): dimension along which to squeeze. If dim is None, all singleton dimensions will be squeezed. dim is None by default.
 
         """
+        if dim is None:
+            size = self.size()
+            if len(self.size()) == 1 or size.count(1) == 0:
+                return self
+            first_singleton_dim = size.index(1)
+
+            squeezed_dict = SqueezedTensorDict(
+                source=self,
+                custom_op="squeeze",
+                inv_op="unsqueeze",
+                custom_op_kwargs={"dim": first_singleton_dim},
+                inv_op_kwargs={"dim": first_singleton_dim},
+            )
+            return squeezed_dict.squeeze(dim=None)
+
         if dim < 0:
             dim = self.batch_dims + dim
 
@@ -4446,8 +4461,8 @@ class UnsqueezedTensorDict(_CustomOpTensorDict):
         True
     """
 
-    def squeeze(self, dim: int) -> TensorDictBase:
-        if dim < 0:
+    def squeeze(self, dim: Optional[int]) -> TensorDictBase:
+        if dim is not None and dim < 0:
             dim = self.batch_dims + dim
         if dim == self.custom_op_kwargs.get("dim"):
             return self._source


Test patch:
diff --git a/test/test_tensordict.py b/test/test_tensordict.py
--- a/test/test_tensordict.py
+++ b/test/test_tensordict.py
@@ -1093,6 +1093,18 @@ def test_squeeze(self, td_name, device, squeeze_dim=-1):
         assert (td_squeeze.get("a") == 1).all()
         assert (td.get("a") == 1).all()
 
+    def test_squeeze_with_none(self, td_name, device, squeeze_dim=None):
+        torch.manual_seed(1)
+        td = getattr(self, td_name)(device)
+        td_squeeze = torch.squeeze(td, dim=None)
+        tensor = torch.ones_like(td.get("a").squeeze())
+        td_squeeze.set_("a", tensor)
+        assert (td_squeeze.get("a") == tensor).all()
+        if td_name == "unsqueezed_td":
+            assert td_squeeze._source is td
+        assert (td_squeeze.get("a") == 1).all()
+        assert (td.get("a") == 1).all()
+
     def test_write_on_subtd(self, td_name, device):
         td = getattr(self, td_name)(device)
         sub_td = td.get_sub_tensordict(0)
Repository: pytorch/rl

Issue:
[BUG] step_tensordict is not properly used by data collectors
## Describe the bug

[Here](https://github.com/facebookresearch/rl/blob/6757d04199992dc0b15c5ebd35c387c8d347abd1/torchrl/collectors/collectors.py#L469) we use `step_tensordict` with a tensordict where we exclude keys, but this is extra work and can be done via `step_tensordict` itself.



Discussion:
Hello @vmoens! 
I've been studying RL lately and would like to contribute to TorchRL; can take this up if no one else is working on it.

Btw, the function `step_tensordict`  has been renamed to `step_mdp`.
Sure that would be wonderful. Also feel free to ping me if you need other ideas of contributions!

Resolving patch:
diff --git a/torchrl/collectors/collectors.py b/torchrl/collectors/collectors.py
--- a/torchrl/collectors/collectors.py
+++ b/torchrl/collectors/collectors.py
@@ -633,12 +633,7 @@ def rollout(self) -> TensorDictBase:
                 tensordict_out.append(self._tensordict.clone())
 
                 self._reset_if_necessary()
-                self._tensordict.update(
-                    step_mdp(
-                        self._tensordict.exclude("reward", "done"), keep_other=True
-                    ),
-                    inplace=True,
-                )
+                self._tensordict.update(step_mdp(self._tensordict), inplace=True)
             if self.return_in_place and len(self._tensordict_out.keys()) > 0:
                 tensordict_out = torch.stack(tensordict_out, len(self.env.batch_size))
                 tensordict_out = tensordict_out.select(*self._tensordict_out.keys())


Test patch: