Guidelines for assigning num_workers to DataLoader

abweiss: I realize that to some extent this comes down to experimentation, but are there any general guidelines on how to choose the num_workers for a DataLoader object?  Should num_workers be equal to the batch size?   Or the number of CPU cores in my machine?  Or to the number of GPUs in my data-parallelized model?  Is there a tradeoff with using more workers due to overhead?  Also, is there ever a reason to leave num_workers as 0 instead of setting it at least to 1?

apaszke: Having more workers will increase the memory usage and that’s the most serious overhead. I’d just experiment and launch approximately as many as are needed to saturate the training. It depends on the batch size, but I wouldn’t set it to the same number - each worker loads a single batch and returns it only once it’s ready. num_workers equal 0 means that it’s the main process that will do the data loading when needed, num_workers equal 1 is the same as any n, but you’ll only have a single worker, so it might be slow

mjchen611: And I set num_workers = 0,the (RAM, but not GPU) memory remains stable with the increase of epoch. Can you give me some suggestions or instructions about the problem? Thank you so much.

YossiB: I would love to get your advice about the recommended way to deal with my data - I feed my CNN with large batches (256/512/1024…) of small patches of size 50x50. I intend to use the ImageFolder DataLoader for that, but I’m afraid that it would be very uneffective to load from disk a lot of small images in high frequency.

harsv: 

taiky: 

Brando_Miranda: if the data set is small like cifar10, why doesn’t the whole data set stay in the GPU the whole time? Why would # workers do anything?

ptrblck: 

Eric_K: 

Brando_Miranda: 

ptrblck: If your dataset is really small and you don’t need batching, you can just push the data onto the GPU and simply apply your training procedure.

Brando_Miranda: 

singleroc: Correct me if you have a different opinion.

ptrblck: 

mnazaal: Cheers

isalirezag: 

niez01: Is there any one has met this situation that setting num_workers = 4 could make the train stop? Recently, I tested a RFBnet project, and find when I set num_workers= 4 will stop training at epoch = 2. However, num_workers=0 will be fine.

devforfu: It seems that during the training process the amount of free RAM continues to reduce. I am using a custom dataset that generates images from strokes (Quick Draw Doodles data), and probably the problem is that the dataset doesn’t work well in multitasking setting. Could somebody give an advice on how to implement a multithread ready dataset?

kuzand: What’s num_GPU? How to get it on google colab?

Junhao_Wen: Hi, I encountered the similar problem for DataLoader. Sorry to ask the similar question, after reading all your discussion, I am still confused about the relationship between the number_GPU, num_CPU and the num_works. Here is my situation: I have a convolutional Autoencoder (so it’s not a very big network), but I have a very big dataset: For one epoch, I have ` Always, it is very slow to load the data, the training time for each batch is very fast. To load each image, I found that with num_workers=8, it will take longer than num_workers=0. Every time, I will take a long time to load 74 batches and run the training, then reload another 74 batches (the case of num_workers=8). Do you have any suggestions to set the num_workers if the dataset is huge??? Thanks in advance