The best way to use neural network in Dataloader?

username: luvwinnie
cooked: <p>I have a neural network implemented in pytorch which i wished to used for augment the dataset within Dataloader. I loaded the neural network in a python generator and pass it to Dataloader. However it seems like it initialized multiple neural network on CPU, it there anyway or anybody know a way to use a neural network with a Dataloader? I keep search on google but seems like no people have done this?</p>

username: ptrblck
cooked: <p>The multiple initializations are expected in case you are using <code>num_workers&gt;0</code>, since each worker would create a <code>Dataset</code> copy and would thus also re-initialize the neural network to preprocess the data.<br>
Your approach is uncommon and I would guess that the more common use case would be to use the NN to preprocess the data into the <code>DataLoader</code> loop (in the same way the trainable model would be used).</p>

username: luvwinnie
cooked: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="129367">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>The multiple initializations are expected in case you are using <code>num_workers&gt;0</code>, since each worker would create a <code>Dataset</code> copy and would thus also re-initialize the neural network to preprocess the data.</p>
</blockquote>
</aside>
<p>Thank you for reply!<br>
Hmm… I have done this because my number of data is too large and it take too much time to use the neural network(network such as StyleGAN) to augment the dataset. Therefore, I use the neural network in the <code>Dataset</code> to augment the dataset on the fly. Wondering that someone have done this also.</p>

username: ptrblck
cooked: <aside class="quote no-group quote-modified" data-username="luvwinnie" data-post="3" data-topic="129367">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/luvwinnie/48/32894_2.png" class="avatar"> luvwinnie:</div>
<blockquote>
<p>I have done this because that my number of data is too large and it take too much time to use the neural network(network such as StyleGAN) to augment the dataset.</p>
</blockquote>
</aside>
<p>I’m unsure how to understand this in the context of the question.<br>
If the single neural network is too slow in its performance to transform the data fast enough, I would expect that model clones would be desired in each worker, so that the processing could be done in parallel (on the CPU). However, it seems you would like to use a single neural network inside the <code>Dataset</code> (and all workers), which would then yield the slow performance again, wouldn’t it?</p>

username: luvwinnie
cooked: <p>Hmm…maybe you are right that even if I use a single neural network inside the Dataset would cause it slower since it is not process in parallel CPUs.<br>
I was thinking to use a single neural in Dataset  and send the it to GPU to augment data, if the neural network is accessible on all worker. <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=10" title=":thinking:" class="emoji" alt=":thinking:"> Since GPU would process faster than CPU.</p>
<p>One way to do this is use the class object of collate_fn for the Dataloader, so that the collate_fn object initialize a neural network in GPU to augment in batch. However my dataset is a aiocr dataset which contains character level and sentence level data and I wish to augment in character level.</p>
<p>Maybe we should find a way to do this. Hmm…</p>

username: ptrblck
cooked: <p>If you want to use the model on the GPU, I would probably just use it inside the <code>DataLoader</code> loop, as it would process an entire batch (while in the <code>__getitem__</code> of the <code>Dataset</code> it would process a single sample by default). Re-initializing it in the <code>collate_fn</code> wouldn’t yield any gain (besides the initialization overhead of course).</p>

username: luvwinnie
cooked: <p>Thank you for advise, I would like to take a try with it!</p>