Repository: pytorch
Issue: Dataset  with Queue issue
Labels: module: dataloader,triaged,module: data

### 🐛 Describe the bug

In a classification task, due to the small number of certain categories, I added a queue in the function. This queue is used to cache images with a small number, but I found that each epoch will empty my queue. Why？

here is demo
```
import random

import cv2
import torch
import os
from torch.utils.data import Dataset, DataLoader
from queue import Queue


class DataSet_my(Dataset):
    def __init__(self, path_dir, is_train=True):
        self.path_dir = path_dir
        self.jpg_path_List = []
        self.label_2 = 2
        self.label_1 = 1
        self.out_h = 64
        self.out_w = 64
        self.T_max_obj = 10
        self.T_maxqueue_size = 1000
        self.queue_obj = Queue(maxsize = self.T_maxqueue_size)  # 创建一个队列对象


    def __getitem__(self, idx):
        path_img = self.jpg_path_List[idx]
        path_obscure_txt = path_img.replace(".jpg", ".txt")
        img_src = cv2.imread(path_img)
        with open(path_obscure_txt, "r") as fr:
            lines = fr.readlines()

        roi_img = cv2.imread(lines[0])
        label = lines[1]
        self.queue_obj.qsize() < self.T_maxqueue_size * 0.7:
            self.queue_obj.put(roi_img.copy())


        if self.queue_obj.qsize() > self.T_max_obj * 5 and (len(m_roi_obscure) < len(m_roi_normal)):
            while not self.queue_obj.empty():
                m_out.append(self.queue_obj.get())
       

        out_label = np.array(label)
        out_img = np.array(m_out)

        return out_img, out_label.astype(np.float32)
```



### Versions


```
Package              Version
-------------------- -----------
anyio                3.7.1
argon2-cffi          21.3.0
argon2-cffi-bindings 21.2.0
attrs                23.1.0
backcall             0.2.0
beautifulsoup4       4.12.2
bleach               6.0.0
certifi              2022.12.7
cffi                 1.15.1
charset-normalizer   3.1.0
cPython              0.0.6
cycler               0.11.0
debugpy              1.6.7
decorator            5.1.1
defusedxml           0.7.1
dnspython            2.3.0
easydict             1.10
entrypoints          0.4
exceptiongroup       1.1.2
fastjsonschema       2.17.1
fire                 0.5.0
fonttools            4.38.0
glog                 0.3.1
idna                 3.4
imageio              2.28.1
importlib-metadata   6.7.0
importlib-resources  5.12.0
ipykernel            6.16.2
ipython              7.34.0
ipython-genutils     0.2.0
ipywidgets           8.0.7
jedi                 0.18.2
Jinja2               3.1.2
joblib               1.1.0
jsonschema           4.17.3
jupyter_client       7.4.9
jupyter_core         4.12.0
jupyter-server       1.24.0
jupyterlab-pygments  0.2.2
jupyterlab-widgets   3.0.8
kiwisolver           1.4.4
lmdb                 1.4.0
MarkupSafe           2.1.3
matplotlib           3.5.3
matplotlib-inline    0.1.6
mistune              3.0.1
mkl-fft              1.3.1
mkl-random           1.2.2
mkl-service          2.4.0
nbclassic            1.0.0
nbclient             0.7.4
nbconvert            7.6.0
nbformat             5.8.0
nest-asyncio         1.5.6
networkx             2.6.3
notebook             6.5.4
notebook_shim        0.2.3
numpy                1.21.6
open3d-python        0.7.0.0
opencv-python        4.6.0.66
packaging            23.1
pandocfilters        1.5.0
parso                0.8.3
pexpect              4.8.0
pickleshare          0.7.5
Pillow               9.5.0
pip                  22.3.1
pkgutil_resolve_name 1.3.10
prefetch-generator   1.0.1
prettytable          3.7.0
prometheus-client    0.17.1
prompt-toolkit       3.0.39
protobuf             3.19.1
psutil               5.9.5
ptyprocess           0.7.0
pycparser            2.21
Pygments             2.15.1
pymongo              4.3.3
pyparsing            3.1.0
pyrsistent           0.19.3
python-dateutil      2.8.2
python-gflags        3.1.2
PyWavelets           1.4.0
PyYAML               3.13
pyzmq                25.1.0
requests             2.31.0
scikit-image         0.19.3
scikit-learn         1.0.2
scipy                1.7.3
Send2Trash           1.8.2
setuptools           65.6.3
six                  1.16.0
sklearn              0.0
sniffio              1.3.0
soupsieve            2.4.1
tensorboardX         2.1
termcolor            2.3.0
terminado            0.17.1
threadpoolctl        3.1.0
tifffile             2021.11.2
tinycss2             1.2.1
torch                1.7.1+cu110
torchvision          0.8.2+cu110
tornado              6.2
tqdm                 4.64.1
traitlets            5.9.0
typing_extensions    4.5.0
urllib3              2.0.2
wcwidth              0.2.6
webencodings         0.5.1
websocket-client     1.6.1
wheel                0.38.4
widgetsnbextension   4.0.8
zipp                 3.15.0

```

cc @SsnL @VitalyFedyunin @ejguan @dzhulgakov