Repository: pytorch
Issue:  malloc(): memory corruption (fast)
Labels: triaged,module: data parallel

## 🐛 Bug
I use the official Docker Images of Pytorch. When I try train my model using the DataParallel module, and during the training it's randomly crash, by segfault. Without DataParallel I get no error.

## To Reproduce

Steps to reproduce the behavior:

1. Start to train a  model, using data DataParallel on 2 GPUs
Example of model (Sequential order) :
 - nn.Embedding()
 - nn.LSTM(bidirectionnal=True)
 - nn.Linear()
 
Loss used :
- BCEWithlogitsloss

Optimizer :
- Adam

```
*** Error in '/opt/conda/bin/python': malloc(): memory corruption (fast): 0x000055a9cabee518 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f6094b627e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x82651)[0x7f6094b6d651]
/lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7f6094b6f184]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_Znwm+0x18)[0x7f60848a5e78]
/opt/conda/lib/python3.6/site-packages/torch/lib/libc10_cuda.so(+0x1af81)[0x7f603c2b3f81]
/opt/conda/lib/python3.6/site-packages/torch/lib/libc10_cuda.so(+0x1cd5e)[0x7f603c2b5d5e]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(_ZN2at6native10empty_cudaEN3c108ArrayRefIlEERKNS1_13Tenso
rOptionsENS1_8optionalINS1_12MemoryFormatEEE+0x284)[0x7f6042151094]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(+0x455b8d8)[0x7f6040a218d8]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(+0x1eedc47)[0x7f603e3b3c47]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(+0x3ead8a5)[0x7f60403738a5]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(+0x1eedc47)[0x7f603e3b3c47]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(+0x1c5e1c0)[0x7f603e1241c0]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(_ZN2at6native5zerosEN3c108ArrayRefIlEERKNS1_13TensorOptio
nsE+0x25)[0x7f603e12dea5]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(+0x1fa1a83)[0x7f603e467a83]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(+0x3d912ce)[0x7f60402572ce]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(+0x1fe5819)[0x7f603e4ab819]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(+0x37c6c3b)[0x7f603fc8cc3b]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(_ZN5torch8autograd9generated13SliceBackward5applyEOSt6vec
torIN2at6TensorESaIS5_EE+0x111)[0x7f603fc8f6e1]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(+0x3fb3106)[0x7f6040479106]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(_ZN5torch8autograd6Engine17evaluate_functionERSt10shared_
ptrINS0_9GraphTaskEEPNS0_4NodeERNS0_11InputBufferE+0x1373)[0x7f6040475563]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(_ZN5torch8autograd6Engine11thread_mainERKSt10shared_ptrIN
S0_9GraphTaskEEb+0x4b2)[0x7f6040476192]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch.so(_ZN5torch8autograd6Engine11thread_initEi+0x39)[0x7f604046
fa89]
/opt/conda/lib/python3.6/site-packages/torch/lib/libtorch_python.so(_ZN5torch8autograd6python12PythonEngine11thread_in
itEi+0x2a)[0x7f60851f7dea]
/opt/conda/lib/python3.6/site-packages/torch/_C.cpython-36m-x86_64-linux-gnu.so(+0xedef)[0x7f6085e21def]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7f6094ebc6ba]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f6094bf241d]

```
## Expected behavior

Train normaly, and save the model

## Environment

PyTorch version: 1.4.0+cu100
Is debug build: No
CUDA used to build PyTorch: 10.0

OS: Ubuntu 16.04.5 LTS
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609
CMake version: Could not collect

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: Tesla V100-PCIE-32GB
GPU 1: Tesla V100-PCIE-32GB

Nvidia driver version: 410.79
cuDNN version: Could not collect

Versions of relevant libraries:
[pip] numpy==1.15.4
[pip] numpydoc==0.9.2
[pip] pytorch-pretrained-bert==0.6.2
[pip] pytorch-transformers==1.1.0
[pip] torch==1.4.0+cu100
[pip] torchcontrib==0.0.2
[pip] torchfile==0.1.0
[pip] torchtext==0.4.0
[pip] torchvision==0.5.0+cu100
[conda] blas                      1.0                         mkl  
[conda] mkl                       2019.1                      144  
[conda] mkl_fft                   1.0.10           py36ha843d7b_0  
[conda] mkl_random                1.0.2            py36hd81dba3_0  
[conda] pytorch-pretrained-bert   0.6.2                    pypi_0    pypi
[conda] pytorch-transformers      1.1.0                    pypi_0    pypi
[conda] torch                     1.4.0+cu100              pypi_0    pypi
[conda] torchcontrib              0.0.2                    pypi_0    pypi
[conda] torchfile                 0.1.0                    pypi_0    pypi
[conda] torchtext                 0.4.0                    pypi_0    pypi
[conda] torchvision               0.5.0+cu100              pypi_0    pypi


## Additional context

<!-- Add any other context about the problem here. -->