Repository: pytorch/vision
Pull request: Remove in-place mutation in DefaultBoxGenerator
This replaces in-place mutation with more backend compiler-friendly one. In-place ops generally cause problems for TVM, so we want an input model to be as pure as possible. 

A related issue https://github.com/apache/tvm/issues/10050

Problem statement:
'NotImplementedError: The following operators are not implemented:' errors when converting PyTorch Object Detection models 
### 🐛 Describe the bug

I am trying to work with the SSD300_VGG16 object detection model and convert it to a serialized and optimized format to eventually run on an embedded system. The script that I use to create and save the traced model from TorchScript is the following:

```
import torch
import torchvision

def do_trace(model, in_size=500):
    model_trace = torch.jit.trace(model, torch.rand(1, 3, in_size, in_size))
    model_trace.eval()
    return model_trace


def dict_to_tuple(out_dict):
    if "masks" in out_dict.keys():
        return (out_dict["boxes"], out_dict["scores"], out_dict["labels"], out_dict["masks"])
    return (out_dict["boxes"], out_dict["scores"], out_dict["labels"])


class TraceWrapper(torch.nn.Module):
    def __init__(self, model):
        super().__init__()
        self.model = model

    def forward(self, inp):
        out = self.model(inp)
        return dict_to_tuple(out[0])


model_funcs = [torchvision.models.detection.ssd300_vgg16]

names = ["ssd300_vgg16"]

for name, model_func in zip(names, model_funcs):

    model = TraceWrapper(model_func(num_classes=50, pretrained_backbone=False))

    model.eval()
    in_size = 500
    inp = torch.rand(1, 3, in_size, in_size)

    with torch.no_grad():
        out = model(inp)
        
        script_module = do_trace(model)
        script_out = script_module(inp)

        assert len(out[0]) > 0 and len(script_out[0]) > 0
        torch._C._jit_pass_inline(script_module.graph)
        torch.jit.save(script_module, name + ".pt")
```

After that I would like to convert the saved model to another format that I am working with, but I am getting the error below:

  **File "/home/achalhoub/tvm/python/tvm/relay/frontend/pytorch.py", line 3091, in report_missing_conversion
    raise NotImplementedError(msg)
NotImplementedError: The following operators are not implemented: ['aten::clamp_min', 'aten::copy_']**

From the research that I've done on this, it seems like TVM doesn't support the conversion of some OPs in certain model architectures?

I would appreciate help with this issue!

Thanks.





### Versions

I am using the following versions:

 **torch 1.10.1+cu102**
**torchvision 0.11.2+cu102**
**tvm 0.9.dev0**

cc @datumbox

Hints from issue discussion:
The detection models are not traceable. They are JIT scriptable though, so I recommend using this instead.
I have tried scripting object detection models, but the issue is that the conversion of the saved scripted model to the end format that I need is not supported - I get similar errors to the one I posted here but with many more ops. Any recommendations? 

Also, is there any way I can access the source code for the different PyTorch model architectures? I am not sure if they're on GitHub anywhere but I am having a hard time finding them (if possible)

Commit message: device fix

Patch to review:
diff --git a/torchvision/models/detection/anchor_utils.py b/torchvision/models/detection/anchor_utils.py
index bac7cb6c742..b4791afe36f 100644
--- a/torchvision/models/detection/anchor_utils.py
+++ b/torchvision/models/detection/anchor_utils.py
@@ -255,16 +255,15 @@ def forward(self, image_list: ImageList, feature_maps: List[Tensor]) -> List[Ten
         default_boxes = default_boxes.to(device)
 
         dboxes = []
+        x_y_size = torch.tensor([image_size[1], image_size[0]]).to(default_boxes.device)
         for _ in image_list.image_sizes:
             dboxes_in_image = default_boxes
             dboxes_in_image = torch.cat(
                 [
-                    dboxes_in_image[:, :2] - 0.5 * dboxes_in_image[:, 2:],
-                    dboxes_in_image[:, :2] + 0.5 * dboxes_in_image[:, 2:],
+                    (dboxes_in_image[:, :2] - 0.5 * dboxes_in_image[:, 2:]) * x_y_size,
+                    (dboxes_in_image[:, :2] + 0.5 * dboxes_in_image[:, 2:]) * x_y_size,
                 ],
                 -1,
             )
-            dboxes_in_image[:, 0::2] *= image_size[1]
-            dboxes_in_image[:, 1::2] *= image_size[0]
             dboxes.append(dboxes_in_image)
         return dboxes


Code review comments:
diff_hunk: @@ -255,16 +255,15 @@ def forward(self, image_list: ImageList, feature_maps: List[Tensor]) -> List[Ten
         default_boxes = default_boxes.to(device)
 
         dboxes = []
+        x_y_size = torch.tensor([image_size[1], image_size[0]]).to(default_boxes.device)
path: torchvision/models/detection/anchor_utils.py
text: @user1:
Nit:
```suggestion
        x_y_size = torch.tensor([image_size[1], image_size[0]], device=default_boxes.device)
```