Num_workers > 1 mixes samples and labels (h5py dataset)

username: rumo
cooked: <p>Hi everyone,<br>
I have a question regarding the understanding of num_workers, I couldn’t answer it myself.</p>
<p>I have a data set that contains image data and the corresponding masks (segmentation task). The file format is HDF5.<br>
I wrote a custom data set class that loads the data and use a data loader for loading the data in batches.<br>
To visualize the data, I wrote a function that grids the image data and the corresponding masks and overlays the two, so that I can see the masks on the image. For that, I directly pass a batch coming from the dataloader to  my “gridding function”.</p>
<p>As long as I use num_workers = 0 or 1, everything works as expected.<br>
For num_workers &gt; 1, it seems like my training samples and the corresponding masks are shuffled around (some images appear in several batches but with different masks, sometimes there’s no image at all, just the mask). I uploaded an example were you can see that the first two samples in the batch are identical but the mask (blueish  parts) are different (and both wrong). This never happens if I set num_workers to 0 or 1.</p>
<p>I read that there are some problems with HDF5 datasets and multitasking, but I’m not sure if my problem relates to this or if there is something else going on that I’m unaware of.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/2/92f86b4df28bef20f812835610fe6ca0d13cbb21.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/92f86b4df28bef20f812835610fe6ca0d13cbb21" title="example_image_1"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/2/92f86b4df28bef20f812835610fe6ca0d13cbb21_2_690x197.jpeg" alt="example_image_1" data-base62-sha1="kY9WFum8oo27ztqrNI4CegKoeyd" width="690" height="197" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/2/92f86b4df28bef20f812835610fe6ca0d13cbb21_2_690x197.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/9/2/92f86b4df28bef20f812835610fe6ca0d13cbb21_2_1035x295.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/9/2/92f86b4df28bef20f812835610fe6ca0d13cbb21_2_1380x394.jpeg 2x" data-dominant-color="C0C0C0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">example_image_1</span><span class="informations">1662×476 175 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Tech specs:</p>
<ul>
<li>Ubuntu 18.04</li>
<li>i7-4720HQ CPU</li>
<li>16 GB RAM</li>
<li>no CUDA, just CPU (for now)</li>
</ul>
<p>Thank you for your help!</p>

username: Kushaj
cooked: <p>If you are using a single dataloader that loads both the image and the mask <code>num_workers</code> won’t matter. When multiple workers are used, each worker works on a different <code>__get_item__</code>.</p>

username: rumo
cooked: <p>Hello Kushaj,<br>
thank you for your reply! I took another look and could actually solve my problem by following <a href="https://discuss.pytorch.org/t/dataloader-when-num-worker-0-there-is-bug/25643/14">piojanu’s workaround</a>. When I open the HFD5 file within <strong>getitem</strong> , and not from the <strong>init</strong> as I did before, everything works as expected. I had not realized, that my problem could be caused by that.</p>