Repository: pytorch
Issue: DataLoader with cv2 and some numpy/cv2 import order causes workers to not work
Labels: high priority,needs reproduction,module: dataloader,triaged,module: multithreading,module: regression

## 🐛 Bug

Dataloader doesn't work when using cv2.resize on Dataset.

## To Reproduce

Steps to reproduce the behavior:

1. import torch before numpy
1. create some nn.module and load state_dict
1. resize a image to a larger size using `cv2.resize` in the Dataset Class
1. then DataLoader does not work (This worked in pytorch 1.3)

```python
import torch
import numpy as np
import cv2


class Network(torch.nn.Module):
    def __init__(self):
        super().__init__()
        self.mod = torch.nn.Sequential(
                        torch.nn.Conv2d(  3, 64, 3, 1, 1, bias=False),
                        torch.nn.Conv2d( 64, 64, 3, 1, 1, bias=False) )
        state_dict = {  '0.weight' : torch.randn((64, 3,3,3),dtype=torch.float32),
                        '1.weight' : torch.randn((64,64,3,3),dtype=torch.float32) }
        self.mod.load_state_dict(state_dict)
    def forward(self, x): return x


class MyDataset(torch.utils.data.Dataset):
    def __init__(self): super().__init__()
    def __len__(self): return 20
    def __getitem__(self,idx):
        img = np.zeros((512,512,3),dtype=np.uint8)
        
        CASE = 0
        print('stop here.')
        if CASE==0: resized = cv2.resize(img,(364,364)) # error
        if CASE==1: resized = cv2.resize(img,( 32, 32)) # ok
        if CASE==2: resized = cv2.resize(img,(300,300)) # ok
        if CASE==3: resized = cv2.resize(img,(350,350)) # error
        if CASE==4: resized = cv2.resize(img,(325,325)) # error
        if CASE==5: resized = cv2.resize(img,(312,312)) # ok
        if CASE==6: resized = cv2.resize(img,(320,320)) # error
        if CASE==7: resized = cv2.resize(img,(316,316)) # error
        if CASE==8: resized = cv2.resize(img,(314,314)) # error
        if CASE==9: resized = cv2.resize(img,(313,313)) # ok
        print('cannot see here.')

        return torch.from_numpy( resized.astype(np.float32) ).permute(2,0,1)


if __name__ == '__main__':
    print('pytorch :',torch.__version__)
    print('numpy :',np.__version__)
    print('opencv :',cv2.__version__)

    device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
    print('device :',device)
    net = Network().to(device)
    loader = torch.utils.data.DataLoader( MyDataset(), batch_size=4, num_workers=4, pin_memory=True )
    
    for batch_idx, imgs in enumerate(loader):
        print(batch_idx)
```

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

## Result

```
pytorch : 1.6.0
numpy : 1.19.1
opencv : 3.4.2
device : cuda:0
stop here.
stop here.
stop here.
stop here.
```
and this does not end.

- This happens whether you are using the GPU or the CPU.
- This reproduction code works fine if numpy is loaded before torch.
- If you reduce the number of network layers you create, it will work properly.
- It works when the image size is reduced.

## Environment
Collecting environment information...
PyTorch version: 1.6.0
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: Could not collect

Python version: 3.7 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: GeForce RTX 2080 Ti
GPU 1: GeForce RTX 2080 Ti
GPU 2: GeForce RTX 2080 Ti
GPU 3: GeForce RTX 2080 Ti
GPU 4: GeForce RTX 2080 Ti
GPU 5: GeForce RTX 2080 Ti
GPU 6: GeForce RTX 2080 Ti
GPU 7: GeForce RTX 2080 Ti

Nvidia driver version: 440.100
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.1
[pip3] torch==1.6.0
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               10.2.89              hfd86e86_1  
[conda] mkl                       2020.2                      256  
[conda] mkl-service               2.3.0            py37he904b0f_0  
[conda] mkl_fft                   1.2.0            py37h23d657b_0  
[conda] mkl_random                1.1.1            py37h0573a6f_0  
[conda] numpy                     1.19.1           py37hbc911f0_0  
[conda] numpy-base                1.19.1           py37hfa32c7d_0  
[conda] pytorch                   1.6.0           py3.7_cuda10.2.89_cudnn7.6.5_0    pytorch

 - PyTorch Version (e.g., 1.0):
 - OS (e.g., Linux):
 - How you installed PyTorch (`conda`, `pip`, source): ``conda create -n testenv numpy opencv pytorch -c pytorch`` in anaconda3-2020.02
```
conda list

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
blas                      1.0                         mkl  
bzip2                     1.0.8                h7b6447c_0  
ca-certificates           2020.7.22                     0  
cairo                     1.14.12              h8948797_3  
certifi                   2020.6.20                py37_0  
cudatoolkit               10.2.89              hfd86e86_1  
ffmpeg                    4.0                  hcdf2ecd_0  
fontconfig                2.13.0               h9420a91_0  
freeglut                  3.0.0                hf484d3e_5  
freetype                  2.10.2               h5ab3b9f_0  
glib                      2.65.0               h3eb4bd4_0  
graphite2                 1.3.14               h23475e2_0  
harfbuzz                  1.8.8                hffaf4a1_0  
hdf5                      1.10.2               hba1933b_1  
icu                       58.2                 he6710b0_3  
intel-openmp              2020.2                      254  
jasper                    2.0.14               h07fcdf6_1  
jpeg                      9b                   h024ee3a_2  
ld_impl_linux-64          2.33.1               h53a641e_7  
libedit                   3.1.20191231         h14c3975_1  
libffi                    3.3                  he6710b0_2  
libgcc-ng                 9.1.0                hdf63c60_0  
libgfortran-ng            7.3.0                hdf63c60_0  
libglu                    9.0.0                hf484d3e_1  
libopencv                 3.4.2                hb342d67_1  
libopus                   1.3.1                h7b6447c_0  
libpng                    1.6.37               hbc83047_0  
libstdcxx-ng              9.1.0                hdf63c60_0  
libtiff                   4.1.0                h2733197_1  
libuuid                   1.0.3                h1bed415_2  
libvpx                    1.7.0                h439df22_0  
libxcb                    1.14                 h7b6447c_0  
libxml2                   2.9.10               he19cac6_1  
lz4-c                     1.9.2                he6710b0_1  
mkl                       2020.2                      256  
mkl-service               2.3.0            py37he904b0f_0  
mkl_fft                   1.2.0            py37h23d657b_0  
mkl_random                1.1.1            py37h0573a6f_0  
ncurses                   6.2                  he6710b0_1  
ninja                     1.10.1           py37hfd86e86_0  
numpy                     1.19.1           py37hbc911f0_0  
numpy-base                1.19.1           py37hfa32c7d_0  
opencv                    3.4.2            py37h6fd60c2_1  
openssl                   1.1.1h               h7b6447c_0  
pcre                      8.44                 he6710b0_0  
pip                       20.2.2                   py37_0  
pixman                    0.40.0               h7b6447c_0  
py-opencv                 3.4.2            py37hb342d67_1  
python                    3.7.9                h7579374_0  
pytorch                   1.6.0           py3.7_cuda10.2.89_cudnn7.6.5_0    pytorch
readline                  8.0                  h7b6447c_0  
setuptools                49.6.0                   py37_0  
six                       1.15.0                     py_0  
sqlite                    3.33.0               h62c20be_0  
tk                        8.6.10               hbc83047_0  
wheel                     0.35.1                     py_0  
xz                        5.2.5                h7b6447c_0  
zlib                      1.2.11               h7b6447c_3  
zstd                      1.4.5                h9ceee32_0
```
 - Build command you used (if compiling from source):
 - Python version: 3.7.9
 - CUDA/cuDNN version: cuda10.2 / cudnn7605
 - GPU models and configuration: RTX2080ti
 - Any other relevant information:

## Additional context

I'm not sure what's going on.
When I run my full learning program, it still occupies GPU memory after the program is stopped.

cc @ezyang @gchanan @zou3519 @bdhirsh @heitorschueroff @SsnL @VitalyFedyunin @ejguan