Repository: pytorch/benchmark
Issue: timm_efficientdet get_module() is broken
Root cause: Validation Gap

```
$ ./torchbench.py --no-skip -k timm_efficientdet --nothing
loading annotations into memory...
Done (t=0.01s)
creating index...
index created!
loading annotations into memory...
Done (t=0.01s)
creating index...
index created!
Traceback (most recent call last):
  File "./torchbench.py", line 1024, in <module>
    main()
  File "./torchbench.py", line 915, in main
    for device, name, model, example_inputs in iter_models(args):
  File "./torchbench.py", line 115, in iter_models
    yield load_model(device, model_name, args.training, args.check_accuracy)
  File "./torchbench.py", line 144, in load_model
    model, example_inputs = benchmark.get_module()
  File "/home/jansel/torchbenchmark/torchbenchmark/models/timm_efficientdet/__init__.py", line 126, in get_module
    for _, (input, target) in zip(range(self.num_batches), self.loader_eval):
  File "/home/jansel/conda/envs/torchdynamo/lib/python3.8/site-packages/effdet/data/loader.py", line 125, in __iter__
    for next_input, next_target in self.loader:
  File "/home/jansel/pytorch/torch/utils/data/dataloader.py", line 530, in __next__
    data = self._next_data()
  File "/home/jansel/pytorch/torch/utils/data/dataloader.py", line 570, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/jansel/pytorch/torch/utils/data/_utils/fetch.py", line 52, in fetch
    return self.collate_fn(data)
  File "/home/jansel/conda/envs/torchdynamo/lib/python3.8/site-packages/effdet/data/loader.py", line 83, in __call__
    cls_targets, box_targets, num_positives = self.anchor_labeler.label_anchors(
  File "/home/jansel/conda/envs/torchdynamo/lib/python3.8/site-packages/effdet/anchors.py", line 357, in label_anchors
    box_targets_out.append(box_targets[count:count + steps].view([feat_size[0], feat_size[1], -1]))
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.
```

After https://github.com/pytorch/benchmark/pull/842, I run the same command and get the following error:

```
  File "./torchbench.py", line 1023, in <module>
    main()
  File "./torchbench.py", line 917, in main
    run_one_model(
  File "./torchbench.py", line 973, in run_one_model
    correct_result = model_iter_fn(copy.deepcopy(model), example_inputs)
  File "./torchbench.py", line 472, in forward_pass
    return mod(*inputs)
  File "/data/home/xzhao9/cluster/miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1111, in _call_impl
    return forward_call(*input, **kwargs)
  File "/data/home/xzhao9/cluster/miniconda3/envs/py38/lib/python3.8/site-packages/effdet/bench.py", line 102, in forward
    class_out, box_out = self.model(x)
  File "/data/home/xzhao9/cluster/miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1111, in _call_impl
    return forward_call(*input, **kwargs)
  File "/data/home/xzhao9/cluster/miniconda3/envs/py38/lib/python3.8/site-packages/effdet/efficientdet.py", line 602, in forward
    x = self.backbone(x)
  File "/data/home/xzhao9/cluster/miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1111, in _call_impl
    return forward_call(*input, **kwargs)
  File "/data/home/xzhao9/cluster/miniconda3/envs/py38/lib/python3.8/site-packages/timm/models/efficientnet.py", line 525, in forward
    x = self.conv_stem(x)
  File "/data/home/xzhao9/cluster/miniconda3/envs/py38/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1111, in _call_impl
    return forward_call(*input, **kwargs)
  File "/data/home/xzhao9/cluster/miniconda3/envs/py38/lib/python3.8/site-packages/timm/models/layers/conv2d_same.py", line 30, in forward
    return conv2d_same(x, self.weight, self.bias, self.stride, self.padding, self.dilation, self.groups)
  File "/data/home/xzhao9/cluster/miniconda3/envs/py38/lib/python3.8/site-packages/timm/models/layers/conv2d_same.py", line 17, in conv2d_same
    return F.conv2d(x, weight, bias, stride, (0, 0), dilation, groups)
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same
```

Looks like it is because https://github.com/facebookresearch/torchdynamo/blob/main/torchbench.py#L970 only copies the model to CPU, not cuda. Adding one line `mod.to("cuda")` fixes this issue.

Please help confirm https://github.com/pytorch/benchmark/pull/842 fixes the problem, thanks!

copy.deepcopy() should leave devices alone.  There shouldn't be any device movement in torchbench.py.

Yeah confirmed.  I think the bug is in TorchBench.  It is ignoring `device="cpu"` and returning cuda tensors.

> Yeah confirmed. I think the bug is in TorchBench. It is ignoring `device="cpu"` and returning cuda tensors.

This is because the model itself is forcing the usage of CUDA:
https://github.com/rwightman/efficientdet-pytorch/blob/master/effdet/data/loader.py#L114

We have to disable CPU device in this case, similar to Moco.

Closed by https://github.com/pytorch/benchmark/pull/842