Repository: pytorch
Issue: Torchscript used to work, but now it fails with VariableTensorId error
Labels: oncall: jit,triaged,module: android,oncall: mobile

## 🐛 Bug
When I export torchvision's resnet18 to torchscript and run it on Android using the `speed_benchmark_torch` binary, it fails with the following error:

```
terminating with uncaught exception of type c10::Error: Could not run 'aten::empty.memory_format' with arguments from the 'VariableTensorId' backend. 'aten::empty.memory_format' is only available for these backends: [SparseCPUTensorId, CPUTensorId, MkldnnCPUTensorId]. (dispatch_ at ../aten/src/ATen/core/dispatch/Dispatcher.h:257)
(no backtrace available)
Aborted 
```

## History
For a while, I was using on hash b05d0fa67144d5af0545a333d5792c2e3946b8c5, which was committed to the master branch of pytorch in Oct 2019. I was working with Torchscript models on Android, and things were working fairly well.

Then, I updated to the latest code tag `v1.4.0`, specifically hash 7f73f1d591afba823daa4a99a939217fb54d7688. After that, the following issue emerged.

## To Reproduce

### Model
```
#python
import torch
import torchvision

model = torchvision.models.resnet18(pretrained=True)
model.eval()
example = torch.rand(1, 3, 224, 224)
traced_script_module = torch.jit.trace(model, example)
traced_script_module.save("resnet18.pt")
```

And, put the model onto the android phone (Pixel 3):
```
#!/bin/bash
adb push resnet18.pt /data/local/tmp/pt/ 

```

### Binary
```
cd pytorch

export ANDROID_DEBUG_SYMBOLS=1 ANDROID_ABI=arm64-v8a
export ANDROID_NDK=/path/to/Android/Sdk/ndk/21.0.6113669/
./scripts/build_android.sh \
-DBUILD_BINARY=ON \
-DBUILD_CAFFE2_MOBILE=OFF \
-DCMAKE_PREFIX_PATH=$(python -c 'from distutils.sysconfig import get_python_lib; print(get_python_lib())') \
-DPYTHON_EXECUTABLE=$(python -c 'import sys; print(sys.executable)')

# compiles build_android/bin/speed_benchmark_torch

adb push build_android/bin/speed_benchmark_torch /data/local/tmp/pt/ 
```

### Running the model
```
adb shell /data/local/tmp/pt/speed_benchmark_torch --model =/data/local/tmp/pt/resnet18.pt --input_dims="1,3,224,224" --input_type=float --warmup=5 --iter=20

```

## Expected behavior

On the older hash (b05d0fa67144d5af0545a333d5792c2e3946b8c5), the above would print after running the `adb shell ... speed_benchmark_torch ...` command:
```
Starting benchmark.
Running warmup runs.

Main runs.
Main run finished. Milliseconds per iter: 188.382. Iters per second: 5.30836
```

However, on the current `v1.4.0` tag's hash (7f73f1d591afba823daa4a99a939217fb54d7688), it fails with the following error:
```
terminating with uncaught exception of type c10::Error: Could not run 'aten::empty.memory_format' with arguments from the 'VariableTensorId' backend. 'aten::empty.memory_format' is only available for these backends: [SparseCPUTensorId, CPUTensorId, MkldnnCPUTensorId]. (dispatch_ at ../aten/src/ATen/core/dispatch/Dispatcher.h:257)
(no backtrace available)
Aborted 
```
I'm not too clear on what a `VariableTensorId` is, so I'm not quite sure what's going on. 

## Environment

Please copy and paste the output from our
[environment collection script](https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py)
(or fill out the checklist below manually).

You can get the script and run it with:
```
Collecting environment information...
PyTorch version: 1.4.0
Is debug build: No
CUDA used to build PyTorch: 10.1

OS: Ubuntu 18.04.3 LTS
GCC version: (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
CMake version: version 3.10.2

Python version: 3.8
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: TITAN V
GPU 1: TITAN V
GPU 2: TITAN V

Nvidia driver version: 430.50
cuDNN version: /usr/local/cuda-10.1/targets/x86_64-linux/lib/libcudnn.so.7

Versions of relevant libraries:
[pip] numpy==1.18.1
[pip] torch==1.4.0
[pip] torchvision==0.5.0
[conda] blas                      1.0                         mkl  
[conda] mkl                       2019.4                      243  
[conda] mkl-service               2.3.0            py38he904b0f_0  
[conda] mkl_fft                   1.0.15           py38ha843d7b_0  
[conda] mkl_random                1.1.0            py38h962f231_0  
[conda] pytorch                   1.4.0           py3.8_cuda10.1.243_cudnn7.6.3_0    pytorch
[conda] torchvision               0.5.0                py38_cu101    pytorch
```



cc @suo