Repository: pytorch
Issue: cannot load the image when num_workers > 0 in dataloader
Labels: module: dataloader,triaged

### 🐛 Describe the bug

as the problem described: It's OK when I set num_workers=0, but when I set num_workers>0 the data['img'] are all zero.
and then: I debug the collect_fn, its return is correct. 
go to:

 `for i, data in enumerate(dataloader):`

 the data is up to zero.

here is my collect_fn:
```
def detection_collate(batch): 
    new_batch = {}
    num_gpus = torch.cuda.device_count()
    device='cpu' if num_gpus <= 0 else 0
    keys = batch[0].keys()
    values = list(zip(*[list(b.values()) for b in batch]))
    batch_idx = [torch.full_like(cls, i) for i, cls in enumerate(values[0])]
    for i, k in enumerate(keys):
        value = values[i]
        if k == 'img':
            value = torch.stack(value, 0).permute(0, 3, 1, 2).to(device=device)
        if k in ['boxes', 'cls']:
            value = torch.cat(value, 0)
        new_batch[k] = value
    batch_idx = torch.cat(batch_idx,0)
    new_batch['batch_idx'] = batch_idx
    return new_batch
```
and here is my dataloader:
```
    for epoch in range(start_epochs,epochs):
       ...
        train_dataset, val_dataset = random_split(
            dataset=dataset,
            lengths=[int(0.7*len(dataset)), len(dataset)-int(0.7*len(dataset))],
            generator=torch.Generator().manual_seed(11)
        )
        train_dataloader = torch.utils.data.DataLoader(train_dataset,
                                                 batch_size=train_batchsize,
                                                 shuffle=True,
                                                 num_workers = 0, collate_fn=detection_collate)
        val_dataloader = torch.utils.data.DataLoader(val_dataset,batch_size=val_batchsize,
                                                 shuffle=True,
                                                 num_workers = 0, collate_fn=detection_collate)

        for i, data in enumerate(train_dataloader):
              print(data) 


```
### Versions

Collecting environment information...
PyTorch version: 2.1.0+cu118
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 家庭中文版
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.8.18 | packaged by conda-forge | (default, Oct 10 2023, 15:37:54) [MSC v.1929 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19045-SP0
Is CUDA available: True
CUDA runtime version: 11.4.120
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 2060
Nvidia driver version: 537.13
cuDNN version: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.4\bin\cudnn_ops_train64_8.dll
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture=9
CurrentClockSpeed=2592
DeviceID=CPU0
Family=198
L2CacheSize=1536
L2CacheSpeed=
Manufacturer=GenuineIntel
MaxClockSpeed=2592
Name=Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz
ProcessorType=3
Revision=

Versions of relevant libraries:
[pip3] numpy==1.24.4
[pip3] torch==2.1.0+cu118
[pip3] torchaudio==2.1.0+cu118
[pip3] torchvision==0.16.0+cu118
[conda] torch                     2.1.0+cu118              pypi_0    pypi
[conda] torchaudio                2.1.0+cu118              pypi_0    pypi
[conda] torchvision               0.16.0+cu118             pypi_0    pypi


cc @SsnL @VitalyFedyunin @ejguan @dzhulgakov