Repository: pytorch
Issue: [JIT] Not support for MaskRCNNPredictor
Labels: needs reproduction,oncall: jit,triaged

I am trying to product jit trace output of the MaskRCNNPredictor model for mobile deployment. The model was created using PyTorch's [official tutorial](https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html).
Model code is following.
```python
model = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=True)
in_features = model.roi_heads.box_predictor.cls_score.in_features
model.roi_heads.box_predictor = FastRCNNPredictor(in_features, 2)
in_features_mask = model.roi_heads.mask_predictor.conv5_mask.in_channels
hidden_layer = 256
model.roi_heads.mask_predictor = MaskRCNNPredictor(in_features_mask,
                                                                hidden_layer,
                                                                2)
device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')

model.load_state_dict(torch.load("saved.model", map_location=device))
model.eval()
example = torch.rand(1, 3, 224, 224)
traced_script_module = torch.jit.trace(model, example, strict=True)
traced_script_module.save("android_model.pt")```
```
using jit trace produces the following results. I have built the lastest pytorch(1.6.0a0+f41742f) from source following issue of [named tuple/dict in jit trace](https://github.com/pytorch/pytorch/issues/16453). 

```
/usr/local/lib/python3.7/site-packages/torch/tensor.py:459: RuntimeWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
  'incorrect results).', category=RuntimeWarning)
/usr/local/lib/python3.7/site-packages/torch/nn/functional.py:2892: UserWarning: The default behavior for interpolate/upsample with float scale_factor will change in 1.6.0 to align with other frameworks/libraries, and use scale_factor directly, instead of relying on the computed output size. If you wish to keep the old behavior, please set recompute_scale_factor=True. See the documentation of nn.Upsample for details. 
  warnings.warn("The default behavior for interpolate/upsample with float scale_factor will change "
/usr/local/lib/python3.7/site-packages/torch/nn/functional.py:2902: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  dtype=torch.float32)).float())) for i in range(dim)]
/usr/local/lib/python3.7/site-packages/torchvision/models/detection/rpn.py:164: UserWarning: Integer division of tensors using div or / is deprecated, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead. (Triggered internally at  ../aten/src/ATen/native/BinaryOps.cpp:81.)
  torch.tensor(image_size[1] / g[1], dtype=torch.int64, device=device)] for g in grid_sizes]
/usr/local/lib/python3.7/site-packages/torchvision/models/detection/rpn.py:164: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  torch.tensor(image_size[1] / g[1], dtype=torch.int64, device=device)] for g in grid_sizes]
/usr/local/lib/python3.7/site-packages/torchvision/ops/boxes.py:117: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  boxes_x = torch.min(boxes_x, torch.tensor(width, dtype=boxes.dtype, device=boxes.device))
/usr/local/lib/python3.7/site-packages/torchvision/ops/boxes.py:119: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  boxes_y = torch.min(boxes_y, torch.tensor(height, dtype=boxes.dtype, device=boxes.device))
/usr/local/lib/python3.7/site-packages/torchvision/ops/poolers.py:216: UserWarning: This overload of nonzero is deprecated:
	nonzero(Tensor input, *, Tensor out)
Consider using one of the following signatures instead:
	nonzero(Tensor input, *, bool as_tuple) (Triggered internally at  ../torch/csrc/utils/python_arg_parser.cpp:760.)
  idx_in_level = torch.nonzero(levels == level).squeeze(1)
/usr/local/lib/python3.7/site-packages/torchvision/models/detection/transform.py:254: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  for s, s_orig in zip(new_size, original_size)
/usr/local/lib/python3.7/site-packages/torchvision/models/detection/roi_heads.py:381: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  return torch.tensor(M + 2 * padding).to(torch.float32) / torch.tensor(M).to(torch.float32)
Traceback (most recent call last):
  File "android_model.py", line 20, in <module>
    traced_script_module = torch.jit.trace(model, example, strict=False)
  File "/usr/local/lib/python3.7/site-packages/torch/jit/__init__.py", line 883, in trace
    check_tolerance, strict, _force_outplace, _module_class)
  File "/usr/local/lib/python3.7/site-packages/torch/jit/__init__.py", line 1037, in trace_module
    module._c._create_method_from_trace(method_name, func, example_inputs, var_lookup_fn, strict, _force_outplace)
RuntimeError: Only tensors, lists, tuples of tensors, or dictionary of tensors can be output from traced functions
```

Looks like some output is not from mentioned supported types but i am unable to find one?



cc @suo