Repository: pytorch
Issue: attribute and register_buffer are not the same on gpu 
Labels: oncall: jit,triaged

## 🐛 Bug

According to the [jit documentation](https://pytorch.org/docs/stable/jit.html), "2. register_buffer - Values wrapped in register_buffer will work as they do on nn.Modules. This is equivalent to an attribute (see 4) of type Tensor."

However, when using attribute, an error is raised, see below for an example. This does not occur when using `register_buffer` in pytorch/audio#369.

```
======================================================================                             
ERROR: test_scriptmodule_MelSpectrogram (__main__.Tester)                                          
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_transforms.py", line 109, in test_scriptmodule_MelSpectrogram
    _test_script_module(transforms.MelSpectrogram, tensor)
  File "test_transforms.py", line 41, in _test_script_module
    py_out = py_method(tensor)
  File "/private/home/vincentqb/anaconda3/envs/pytorch-from-source/lib/python3.7/site-packages/torc
h/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/private/home/vincentqb/anaconda3/envs/pytorch-from-source/lib/python3.7/site-packages/torc
haudio-0.4.0a0+774ebc7-py3.7-linux-x86_64.egg/torchaudio/transforms.py", line 215, in forward
    specgram = self.spectrogram(waveform)
  File "/private/home/vincentqb/anaconda3/envs/pytorch-from-source/lib/python3.7/site-packages/torc
h/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/private/home/vincentqb/anaconda3/envs/pytorch-from-source/lib/python3.7/site-packages/torc
haudio-0.4.0a0+774ebc7-py3.7-linux-x86_64.egg/torchaudio/transforms.py", line 70, in forward
    self.win_length, self.power, self.normalized)
  File "/private/home/vincentqb/anaconda3/envs/pytorch-from-source/lib/python3.7/site-packages/torc
haudio-0.4.0a0+774ebc7-py3.7-linux-x86_64.egg/torchaudio/functional.py", line 259, in spectrogram
    waveform, n_fft, hop_length, win_length, window, True, "reflect", False, True
  File "/private/home/vincentqb/anaconda3/envs/pytorch-from-source/lib/python3.7/site-packages/torc
haudio-0.4.0a0+774ebc7-py3.7-linux-x86_64.egg/torchaudio/functional.py", line 52, in _stft
    onesided,
  File "/private/home/vincentqb/anaconda3/envs/pytorch-from-source/lib/python3.7/site-packages/torc
h/functional.py", line 393, in stft
    return torch._C._VariableFunctions.stft(input, n_fft, hop_length, win_length, window, normalize
d, onesided)
RuntimeError: expected device cuda:0 but got device cpu
```

## To Reproduce

Steps to reproduce the behavior:

1. Compile torchaudio without pytorch/audio#369.
1. Run `test_transforms.py` and error occurs
1. Compile torchaudio with pytorch/audio#369.
1. Run `test_transforms.py` and error does not occur.

## Environment

Collecting environment information...
PyTorch version: 1.4.0a0+cd20ecb
Is debug build: No
CUDA used to build PyTorch: 10.0

OS: Ubuntu 18.04.1 LTS
GCC version: (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
CMake version: version 3.14.0

Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: 10.0.130
GPU models and configuration:
GPU 0: Quadro GP100
GPU 1: Quadro GP100

Nvidia driver version: 410.79
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] numpy==1.17.4
[pip3] torch==1.3.1
[conda] blas                      1.0                         mkl
[conda] magma-cuda100             2.5.1                         1    pytorch
[conda] mkl                       2019.4                      243
[conda] mkl-include               2019.4                      243
[conda] mkl-service               2.3.0            py37he904b0f_0
[conda] mkl_fft                   1.0.15           py37ha843d7b_0
[conda] mkl_random                1.1.0            py37hd6b4f25_0
[conda] torch                     1.4.0a0+cd20ecb          pypi_0    pypi
[conda] torchaudio                0.4.0a0+ed8e818          pypi_0    pypi


cc @suo