Repository: pytorch
Issue: model.load_state_dict won't work in Child process if a sufficiently large tensor was padded in the Parent (even if empty padded)
Labels: high priority,module: multiprocessing,module: memory usage,triaged

### 🐛 Describe the bug

A `model.load_state_dict`on CPU failes when executing in child process (with `multiprocessing.Process`)  *if* you padded a sufficiently large tensor in the parent process (even with empty pad).
The child process blocks indefinitely on `model.load_state_dict` and ends up with a ghost process if you KeyInterrupt the parent.

Fun facts : 
- the issue occurs when the tensor defined in parent is sufficiently large, and only if padded
- tensor is not even passed to child process
- the critical size of the tensor seems to depend on the size of the model according to this relation : if the model size decreases, then no problem anymore

reproducible code :
```python
import multiprocessing
import torch
import torch.nn as nn
from torch.nn import functional as F

class TheModelClass(nn.Module):
    def __init__(self):
        super(TheModelClass, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        # no need for all this to have the error
        # x = self.pool(F.relu(self.conv1(x)))
        # x = self.pool(F.relu(self.conv2(x)))
        # x = x.view(-1, 16 * 5 * 5)
        # x = F.relu(self.fc1(x))
        # x = F.relu(self.fc2(x))
        # x = self.fc3(x)
        # return x
        pass

def init_model_and_load():
    model = TheModelClass()
    print("before load")
    model.load_state_dict(torch.load("./model.torch", map_location=torch.device('cpu')))
    print("after load") # never reaches it
    return

if __name__ == "__main__":
    # First, let's save a random init model
    model = TheModelClass()
    torch.save(model.state_dict(), "./model.torch")
    print("nb params", sum(p.numel() for p in model.parameters()))
    # it's probably related to the number of params : if we remove model.fc1, it works

    size = [33**3] # 32**3 works, but not 33**3 ... :scream:
    tensor = torch.rand(size) 
    pad = (0, 0)
    # whether you want the error or not...
    I_WANT_THIS_NOT_TO_WORK = True
    if I_WANT_THIS_NOT_TO_WORK:
        tensor = F.pad( # everything about this bug comes from padding, even with 0
            tensor,
            pad,
            mode="constant",
            value=0,
        )
    # and it doesn't work, even if you reinitialize the tensor!
    tensor = torch.rand([*size])
    p = multiprocessing.Process( # no need to pass tensor as argument, it still fails
        target=init_model_and_load,
    )
    p.start()
    p.join()
```

### Versions

```
PyTorch version: 1.12.1+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.31

Python version: 3.9.2 (default, May  8 2022, 20:05:14)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.15.0-46-generic-x86_64-with-glibc2.31
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.0
[pip3] torch==1.12.1+cpu
[pip3] torchaudio==0.12.1+cpu
[pip3] torchvision==0.13.1+cpu
[conda] Could not collect
```

cc @ezyang @gchanan @zou3519 @VitalyFedyunin