Issue 1:
"type id is VariableTensorId" runtime error for some mobile models

guard torch::jit::load() with AutoGradMode & AutoNonVariableTypeMode

Summary:

Since PR #25597 landed some models started failing to run with mobile

build with the following error:

type id is VariableTensorId
terminate called after throwing an instance of 'std::runtime_error'
what(): false CHECK FAILED at ../c10/core/Backend.h

Seems we need AutoGradMode guard and AutoNonVariableTypeMode guard while

torch::jit::load().

Issue 2:
🐛 Bug

In pytorch 1.3, when doing inference with resnet34 on CPU with variable input shapes, much more memory is used compared to pytorch 1.1 (both CPU-only builds on one core): 6 GB for pytorch 1.3 vs. ~0.5 GB for pytorch 1.1

To Reproduce

Steps to reproduce the behavior:

Run the following script https://gist.github.com/lopuhin/0d100ef7df01fdfc91d9685f6e01ff64 - it performs inference with resnet34 on images with fixed width and variable height, and reports speed and memory growth over the course of the benchmark.

Running under pytorch 1.1:

$ python3 pytorch_high_mem.py --n 500
torch 1.1.0
heights: mean=1004, p50=278 p95=5100 max=7680
n=100 memory growth (kb): 477,952
n=200 memory growth (kb): 503,948
n=300 memory growth (kb): 503,948
n=400 memory growth (kb): 518,652
time: mean=0.924 s, p50=0.271 s, p95=4.626 s
memory (kb): 174,552 initial, 518,652 growth

Running under pytorch 1.3:

$ python3 pytorch_high_mem.py --n 500
torch 1.3.0+cpu
heights: mean=1004, p50=278 p95=5100 max=7680
n=100 memory growth (kb): 2,624,296
n=200 memory growth (kb): 4,480,012
n=300 memory growth (kb): 5,579,568
n=400 memory growth (kb): 5,600,888
time: mean=0.676 s, p50=0.196 s, p95=3.825 s
memory (kb): 187,840 initial, 6,200,664 growth

Expected behavior

Expected behavior is low memory usage as in pytorch 1.1. Alternatively, a way to control caching (e.g. something which disables caching or something like torch.cuda.clear_caches() but for CPU) - as I understand, high memory usage happens because allocations are cached, which makes sense for fixed shapes, but does not work well for variable shapes. Binning shapes is possible as a work-around but has a noticeable performance penalty and memory usage is still higher.

Environment

Environment under pytorch 1.1 (via collect_env.py script):

Collecting environment information...
PyTorch version: 1.1.0
Is debug build: No
CUDA used to build PyTorch: None

OS: Debian GNU/Linux 9 (stretch)
GCC version: Could not collect
CMake version: Could not collect

Python version: 3.6
Is CUDA available: No
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA

Versions of relevant libraries:
[pip3] numpy==1.16.3
[pip3] torch==1.1.0
[pip3] torchvision==0.3.0
[conda] Could not collect

pytorch installed with

pip install -U --no-cache-dir cython wheel pip http://download.pytorch.org/whl/cpu/torch-1.1.0-cp36-cp36m-linux_x86_64.whl http://download.pytorch.org/whl/cpu/torchvision-0.3.0-cp36-cp36m-linux_x86_64.whl

Environment under pytorch 1.3:

PyTorch version: 1.3.0+cpu
Is debug build: No
CUDA used to build PyTorch: None

OS: Debian GNU/Linux 9 (stretch)
GCC version: Could not collect
CMake version: Could not collect

Python version: 3.6
Is CUDA available: No
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA

Versions of relevant libraries:
[pip3] numpy==1.16.3
[pip3] torch==1.3.0+cpu
[pip3] torchvision==0.4.1+cpu
[conda] Could not collect

pytorch installed with

pip install torch==1.3.0+cpu torchvision==0.4.1+cpu -f https://download.pytorch.org/whl/torch_stable.html

Additional context

This may be similar to oneapi-src/oneDNN#489 but here mkldnn is not used explicitly.

cc @VitalyFedyunin @gujinghui @PenghuiCheng @XiaobingSuper @jianyuh @ezyang @gchanan @zou3519