akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
How does num_workers work under the hood?

Jeguarko: <ul>
<li>How does num_workers work under the hood?</li>
<li>Does each worker load a new batch in parallel while other workers transfer their batch to the device? Or are all the workers waiting in line until all batches are transferred to the device, and only then will they start loading a new batch?</li>
<li>Is this already implemented at the software level, or are there some homemade options for parallelizing the loading of batches?</li>
</ul>
Is there a way to specify gpu duty cycle while training?

Alexander_Soare: <p>I find that when I’m training models but still working on my machine I get slower responsiveness on other programs that make use of the GPU. One good example might be Xournal++ which I use to take notes with a stylus.</p>
<p>I’m guessing the way the GPU is accessed by programs is in a time-shared fashion like with a CPU. But is there any way I can specifically tell PyTorch to only use the GPU on a reduced duty cycle, thereby giving more space for my side applications to use it?</p>
<p>Put a sleep in the forward method of my model? <img src="https://discuss.pytorch.org/images/emoji/apple/stuck_out_tongue.png?v=9" title=":stuck_out_tongue:" class="emoji" alt=":stuck_out_tongue:"></p>

ptrblck: <p>I don’t think you can reduce the utilization in any other way then to artificially reduce the workload e.g. through sleeps. <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>

Alexander_Soare: <p>Yeah that makes sense. Thanks!</p>