Repository: pytorch
Issue: nn.Upsample result mismatch in 1.1.0a0+828a6a3 and 1.9.0
Labels: module: cudnn,module: cuda,triaged

## 🐛 Bug

I was using PyTorch 1.1.0a0+828a6a3 for some network training. Found that results of the bicubic mode in nn.Upsample are different in 1.1.0a0+828a6a3 and 1.9.0

## To Reproduce

I just ran the demo in PyTorch doc:

In PyTorch 1.1.0a0+828a6a3:
```
>>> import torch
>>> import torch.nn as nn
>>> input = torch.arange(1, 5, dtype=torch.float32).view(1, 1, 2, 2)
>>> input_3x3 = torch.zeros(3, 3).view(1, 1, 3, 3)
>>> input_3x3[:, :, :2, :2].copy_(input)
tensor([[[[1., 2.],
          [3., 4.]]]])
>>> m = nn.Upsample(scale_factor=2, mode='bicubic')
>>> m(input_3x3)
/opt/conda/lib/python3.6/site-packages/torch/nn/functional.py:2506: UserWarning: Default upsampling behavior when mode=bicubic is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  "See the documentation of nn.Upsample for details.".format(mode))
tensor([[[[ 1.0000,  1.6875,  2.0000,  1.0938,  0.0000, -0.1875],
          [ 2.2812,  3.1445,  3.3750,  1.7900,  0.0000, -0.3164],
          [ 3.0000,  3.8750,  4.0000,  2.0938,  0.0000, -0.3750],
          [ 1.6875,  2.1426,  2.1875,  1.1406,  0.0000, -0.2051],
          [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
          [-0.2812, -0.3633, -0.3750, -0.1963,  0.0000,  0.0352]]]])
```

In PyTorch 1.9.0:
```
>>> import torch
>>> import torch.nn as nn
>>> input = torch.arange(1, 5, dtype=torch.float32).view(1, 1, 2, 2)
>>> input_3x3 = torch.zeros(3, 3).view(1, 1, 3, 3)
>>> input_3x3[:, :, :2, :2].copy_(input)
tensor([[[[1., 2.],
          [3., 4.]]]])
>>> m = nn.Upsample(scale_factor=2, mode='bicubic')
>>> m(input_3x3)
/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py:3451: UserWarning: Default upsampling behavior when mode=bicubic is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  warnings.warn(
tensor([[[[ 0.6836,  1.0785,  1.7512,  1.4892,  0.4405, -0.1887],
          [ 1.4494,  1.8843,  2.6328,  2.1153,  0.6240, -0.2736],
          [ 2.7467,  3.2533,  4.1369,  3.1862,  0.9380, -0.4186],
          [ 2.4497,  2.8227,  3.4780,  2.6375,  0.7759, -0.3485],
          [ 0.7261,  0.8357,  1.0282,  0.7792,  0.2292, -0.1030],
          [-0.3053, -0.3551, -0.4425, -0.3374, -0.0993,  0.0445]]]])
```

## Expected behavior

The bicubic upsampling result should be the same across different versions.

## Environment

1.1.0a0+828a6a3 one:

Collecting environment information...
PyTorch version: 1.1.0a0+828a6a3
Is debug build: No
CUDA used to build PyTorch: 10.1.163

OS: Ubuntu 16.04.6 LTS
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609
CMake version: version 3.5.1

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.1.163
GPU models and configuration: GPU 0: Tesla V100-SXM2-32GB-LS
Nvidia driver version: 440.64.00
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.0

Versions of relevant libraries:
[pip] msgpack-numpy==0.4.3.2
[pip] numpy==1.16.3
[pip] torch==1.1.0a0+828a6a3
[pip] torchtext==0.4.0
[pip] torchvision==0.2.1
[conda] magma-cuda100             2.1.0                         5    local
[conda] mkl                       2019.1                      144
[conda] mkl-include               2019.1                      144
[conda] torch                     1.1.0a0+828a6a3          pypi_0    pypi
[conda] torchtext                 0.4.0                    pypi_0    pypi
[conda] torchvision               0.2.1                    pypi_0    pypi

1.9.0 one:

Collecting environment information...
PyTorch version: 1.9.0a0+df837d0
Is debug build: False
CUDA used to build PyTorch: 11.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.1 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: Could not collect
CMake version: version 3.19.6

Python version: 3.8 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 11.2.142
GPU models and configuration:
GPU 0: Tesla V100-SXM2-32GB-LS
GPU 1: Tesla V100-SXM2-32GB-LS
GPU 2: Tesla V100-SXM2-32GB-LS
GPU 3: Tesla V100-SXM2-32GB-LS

Nvidia driver version: 440.64.00
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.1.1
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.2
[pip3] nvidia-dlprof-pytorch-nvtx==1.0.0
[pip3] pytorch-quantization==2.1.0
[pip3] pytorch-transformers==1.1.0
[pip3] torch==1.9.0a0+df837d0
[pip3] torchtext==0.9.0a0
[pip3] torchvision==0.9.0a0
[conda] magma-cuda110             2.5.2                         5    local
[conda] mkl                       2019.4                      243
[conda] mkl-include               2019.4                      243
[conda] nomkl                     3.0                           0
[conda] numpy                     1.19.2           py38h6163131_0
[conda] numpy-base                1.19.2           py38h75fe3a5_0
[conda] nvidia-dlprof-pytorch-nvtx 1.0.0                    pypi_0    pypi
[conda] pytorch-quantization      2.1.0                    pypi_0    pypi
[conda] pytorch-transformers      1.1.0                    pypi_0    pypi
[conda] torch                     1.9.0a0+df837d0          pypi_0    pypi
[conda] torchtext                 0.9.0a0                  pypi_0    pypi
[conda] torchvision               0.9.0a0                  pypi_0    pypi

## Additional context

I used a standard container from NVIDIA, that's why I got the version 1.1.0a0+828a6a3, I also try to run it on my local PC, the result of official 1.1.0 and 1.9.0 are the same.


cc @ngimel @csarofeen @ptrblck @xwang233