Repository: pytorch
Issue: Pytorch model load failure in Gunicorn with Gevent workers
Labels: needs reproduction,module: nn,module: dataloader,triaged,module: numpy

## 🐛 Bug

If you run a `gunicorn` server with `gevent` workers, and try to load a model it after loading some data using a data loader, it fails in Ubuntu 18.04 .  Exact same code works in macOS (Catalina). The results were obtained using the Bottle (micro) web-framework for python.

<!-- A clear and concise description of what the bug is. -->

## To Reproduce

Create the following environment in python 3.7:

```
bottle==0.12.18
certifi==2021.5.30
chardet==4.0.0
gevent==20.6.2
greenlet==0.4.16
gunicorn==20.1.0
idna==2.10
numpy==1.18.1
Pillow==8.3.1
requests==2.25.1
six==1.16.0
torch==1.4.0
torchvision==0.5.0
typing-extensions==3.10.0.0
urllib3==1.26.6
zope.event==4.5.0
zope.interface==5.4.0
```

Run the following script to start a `gunicorn` server with `gevent` workers.

```python
"""
Example showing gunicorn/gevent and pytorch not working.
"""
from gevent import monkey; monkey.patch_all()
from bottle import Bottle, run, request, response, auth_basic, ServerAdapter
from collections import OrderedDict

import torch
from torch import nn
from torchvision import datasets, transforms
import numpy as np


def dict_from_model(model):
    dct = model.state_dict()
    return OrderedDict({key: torch.rand(dct[key].shape) for key in dct})


class SimpleNet(nn.Module):
    """
    A simple DNN.
    """
    def __init__(self):
        super(SimpleNet, self).__init__()
        self.conv1 = nn.Conv2d(1, 32, 3, 1)
        self.conv2 = nn.Conv2d(32, 64, 3, 1)
        self.dropout1 = nn.Dropout2d(0.25)
        self.dropout2 = nn.Dropout2d(0.5)
        self.fc1 = nn.Linear(9216, 128)
        self.fc2 = nn.Linear(128, 10)

    def forward(self, x):
        x = self.conv1(x)
        x = F.relu(x)
        x = self.conv2(x)
        x = F.relu(x)
        x = F.max_pool2d(x, 2)
        x = self.dropout1(x)
        x = torch.flatten(x, 1)
        x = self.fc1(x)
        x = F.relu(x)
        x = self.dropout2(x)
        x = self.fc2(x)
        output = F.log_softmax(x, dim=1)
        return x


class MNISTSubSet(torch.utils.data.Dataset):
    """
    Simple class to select a subset of the MNIST dataset.
    """
    def __init__(self, digits, is_train):

        input_transform = transforms.Compose([
            transforms.ToTensor(),
            transforms.Normalize((0.1307,), (0.3081,))
        ])

        mnist_ds = datasets.MNIST('./data', train=is_train, download=True)
        self.digits = digits
        mask = np.isin(mnist_ds.targets, digits)

        # --> These two lines work - uncomment these and comment out the
        # the next two lines to test.
        # self.data = torch.from_numpy(mnist_ds.data.numpy()[mask])
        # self.targets = torch.from_numpy(mnist_ds.targets.numpy()[mask])
        # -----------

        # --> These two lines do not work
        self.data = mnist_ds.data[mask].detach().clone()
        self.targets = mnist_ds.targets[mask].detach().clone()
        # -----------

        print("Just subselected")

        self.input_transform = input_transform

    def __getitem__(self, index):
        img, target = self.data[index], self.targets[index]
        img = Image.fromarray(img.numpy(), mode='L')

        if self.input_transform is not None: img = self.input_transform(img)

        return img, target

    def __len__(self):
        return len(self.data)

    def get_data_loader(self):
        return torch.utils.data.DataLoader(self, batch_size=64, shuffle=True)


def test_torch_gevent():
    print("Trying to load a model")
    model = SimpleNet()
    model.load_state_dict(dict_from_model(model))
    print("Successfully loaded a model")
    return "A worker id"

def enable_cors():
    """
    Enable the cross origin resource for the server.
    """
    response.add_header('Access-Control-Allow-Origin', '*')
    response.add_header('Access-Control-Allow-Methods',
                        'GET, POST, PUT, OPTIONS')
    response.add_header('Access-Control-Allow-Headers',
                        'Origin, Accept, Content-Type, X-Requested-With, X-CSRF-Token')

def start_server():
    application = Bottle()
    application.route(f"/test_torch_gevent", method='POST', callback=test_torch_gevent)
    application.add_hook('after_request', enable_cors)
    run(application, server='gunicorn', worker_class='gevent', debug=True, quiet=False)


def run_server():
    train_loader=MNISTSubSet(digits=(1, 2, 3), is_train=True).get_data_loader()
    test_loader=MNISTSubSet(digits=(1, 2, 3), is_train=False).get_data_loader()
    start_server()


if __name__ == '__main__':
    run_server()
```

Now from a different terminal send a POST request to the end point:

```
curl -X POST http://127.0.0.1:8080/test_torch_gevent
```

<!-- If you have a code sample, error messages, stack traces, please provide it here as well -->

I don't see any error messages on the server side, but after a while the worker just times out - e.g.:

```
[2021-07-14 16:30:11 +0100] [4916] [INFO] Starting gunicorn 20.1.0
[2021-07-14 16:30:11 +0100] [4916] [INFO] Listening at: http://127.0.0.1:8080 (4916)
[2021-07-14 16:30:11 +0100] [4916] [INFO] Using worker: gevent
[2021-07-14 16:30:11 +0100] [4935] [INFO] Booting worker with pid: 4935
Trying to load a model
[2021-07-14 16:30:44 +0100] [4916] [CRITICAL] WORKER TIMEOUT (pid:4935)
[2021-07-14 16:30:45 +0100] [4916] [WARNING] Worker with pid 4935 was terminated due to signal 9
[2021-07-14 16:30:45 +0100] [4958] [INFO] Booting worker with pid: 4958
```

## Expected behavior

Once the `curl` command is sent, I expect the server to print out 

```
Trying to load a model
Successfully loaded a model
```
compare this to the timeout in the previous section.

Also, in the terminal from which the POST request was made, I expect to see the server response:

```
A worker id
```

<!-- A clear and concise description of what you expected to happen. -->

## Environment

```
Collecting environment information...
PyTorch version: 1.4.0
Is debug build: False
CUDA used to build PyTorch: 10.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.5 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: version 3.10.2
Libc version: glibc-2.26

Python version: 3.7.5 (default, Feb 23 2021, 13:22:40)  [GCC 8.4.0] (64-bit runtime)
Python platform: Linux-5.4.0-74-generic-x86_64-with-Ubuntu-18.04-bionic
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.18.1
[pip3] torch==1.4.0
[pip3] torchvision==0.5.0
[conda] Could not collect
```

## Additional context

The exact same code produces the expected output in macOS (Catalina). Additionally, the code also works as expected in Ubuntu 18.04 if in `MNISTSubset.__init__()`, instead of using `detach().clone()' I convert to and from numpy (i.e. replace lines 65, 66 with 70, 71). Same behavior observed with latest versions of pytorch and torchvision.

<!-- Add any other context about the problem here. -->


cc @SsnL @VitalyFedyunin @ejguan @albanD @mruberry @jbschlosser @rgommers @heitorschueroff