Repository: pytorch
Issue: _pickle.UnpicklingError: pickle data was truncated - Windows multiprocessing during training
Labels: module: windows,module: multiprocessing,module: dataloader,module: serialization,triaged

## 🐛 Bug
```
File "U:\m2\customizations.py", line 455, in runTraining      
    for xb, yb, weight, orig, segInfo, smallTorches in pb(train_dl):
  File "C:\anaconda3\envs\ai\lib\site-packages\progressbar\shortcuts.py", line 10, in progressbar
    for result in progressbar(iterator):
  File "C:\anaconda3\envs\ai\lib\site-packages\progressbar\bar.py", line 539, in __call__
    self._iterable = iter(iterable)
  File "C:\anaconda3\envs\ai\lib\site-packages\torch\utils\data\dataloader.py", line 354, in __iter__
    self._iterator = self._get_iterator()
  File "C:\anaconda3\envs\ai\lib\site-packages\torch\utils\data\dataloader.py", line 305, in _get_iterator
    return _MultiProcessingDataLoaderIter(self)
  File "C:\anaconda3\envs\ai\lib\site-packages\torch\utils\data\dataloader.py", line 918, in __init__
    w.start()
  File "C:\anaconda3\envs\ai\lib\multiprocessing\process.py", line 
121, in start
    self._popen = self._Popen(self)
  File "C:\anaconda3\envs\ai\lib\multiprocessing\context.py", line 
224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\anaconda3\envs\ai\lib\multiprocessing\context.py", line 
327, in _Popen
    return Popen(process_obj)
  File "C:\anaconda3\envs\ai\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\anaconda3\envs\ai\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
OSError: [Errno 22] Invalid argument
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\anaconda3\envs\ai\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\anaconda3\envs\ai\lib\multiprocessing\spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
_pickle.UnpicklingError: pickle data was truncated
```

This bug occurs on Windows only, Linux versions run fine. It happens intermittently after ~50 epochs and several hours of training when it's starting to train on a new batch (after a validation batch). Using multi-processing with  DataLoader parameters num_workers=3, persistent_workers=True. Similar errors reported here: https://discuss.pytorch.org/t/oserror-errno-22-invalid-argument-pickle-unpicklingerror-pickle-data-was-truncated/138469/2, https://github.com/ultralytics/yolov3/issues/296. I had the same error on previous versions of Pytorch and Python. They happen on both GPUs - usually if I'm training one model on one GPU and one on another. Plenty of available RAM.


PyTorch version: 1.10.0+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 Pro for Workstations
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.7 (default, Sep 16 2021, 16:59:28) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19043-SP0
Is CUDA available: True
CUDA runtime version: 11.3.58
GPU models and configuration:
GPU 0: NVIDIA TITAN V
GPU 1: NVIDIA GeForce RTX 3090

Nvidia driver version: 496.13
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.21.3
[pip3] torch==1.10.0+cu113
[pip3] torchaudio==0.9.1
[pip3] torchsummary==1.5.1
[pip3] torchvision==0.11.1+cu113
[conda] blas                      1.0                         mkl
[conda] cpuonly                   2.0                           0    pytorch
[conda] cudatoolkit               11.1.1               heb2d755_9    conda-forge
[conda] mkl                       2021.3.0           haa95532_524
[conda] mkl-service               2.4.0            py39h2bbff1b_0
[conda] mkl_fft                   1.3.1            py39h277e83a_0
[conda] mkl_random                1.2.2            py39hf11a4ad_0
[conda] numpy                     1.21.3                   pypi_0    pypi
[conda] pytorch-mutex             1.0                         cpu    pytorch
[conda] torch                     1.10.0+cu113             pypi_0    pypi
[conda] torchaudio                0.9.1                      py39    pytorch
[conda] torchsummary              1.5.1                    pypi_0    pypi
[conda] torchvision               0.11.1+cu113             pypi_0    pypi

cc @peterjc123 @mszhanyi @skyline75489 @nbcsm @VitalyFedyunin @SsnL @ejguan @NivekT @mruberry