Dataset, DataLoader and its transformations: PIL vs Tensor, which one is faster?

eduardo4jesus: <p><strong>What I know</strong>, or at least what I think that I know</p>
<ol>
<li>When instantiating one of the <code>VisionDataset</code> from <code>torchvision.datasets</code>, such as the <code>MNIST</code>, a <code>transform</code> argument can be provided which can be built using  <code>torchvision.transforms.Compose</code> with a list of transformations.</li>
<li>If the argument <code>transformation</code> is provided to the particular chosen <code>VisionDataset</code>, the <code>transform</code> don’t seem to be used immediately by the object. But instead, it seems to be used only when we have a <code>torch.utils.data.DataLoader</code> iterating through the dataset.</li>
<li>I noticed from <a href="https://pytorch.org/vision/0.8/transforms.html" rel="noopener nofollow ugc">the documentation page on transforms</a> that there are transforms that work on top of PIL objects, Tensor objects and both.</li>
<li>It seems that only Tensor objects can be transferred to the <code>gpu</code>, not PIL objects.</li>
</ol>
<p><strong>Questions</strong> I have</p>
<ol>
<li>Is it the case that it is better to have the transforms working only on <code>Tensor</code>s instead of working on <code>PIL</code> objects in terms of execution time?</li>
<li>Would those transforms using <code>Tensor</code> be operated in the <code>GPU</code> and the ones using <code>PIL</code> only on <code>CPU</code>?</li>
<li>Is one type of input for the transform better than the other? If so, why having two types of transform?</li>
<li>Additionally, I am setting the the <code>pin_memory</code> to <code>True</code>. I am not sure what is a <code>batch</code> of a <code>custom type</code>, but I don’t think that this is my case, so I am assuming that the <code>pin_memory</code> is actually taking effect and making the transfer <code>memory -&gt; gpu_memory</code> faster. Is that the case? If most of the <code>out of the box</code> network examples fall in this categories, why <code>pin_memory</code> is default to <code>False</code>. Am I missing something on here?</li>
</ol>
<p>Thanks.</p>

JuanFMontesinos: <p>The trick here is that there is a drop-replacement library of PIL I don’t recall the name right now. It’s pillow sidm or ssdm or something similar. That library is compiled and it optimizes the performance depending on the instructions available for ur cpu.</p>
<p>One of the drawbacks of the pytorch’s dataloder is that it cannot run multiprocessing together with gpu processing. That makes the answer a bit hard. It’s a trade-off which depends on the dataset, which GPU you have, which cpu… I would say that a good practice (at least to me) is trying to split the dataloading in a cpu-intensive + a gpu-intensive stages.</p>
<p>If there is something which is really worth to run in the GPU, for example, stft or really heavy ops, I just apply them after the batch is loaded.</p>
<p>But once again, everything depends on your computer, the type of data ecetera</p>

eduardo4jesus: <p>Thanks for the reply.</p>
<p>Nice! Good to know.</p>
<aside class="quote no-group" data-username="JuanFMontesinos" data-post="2" data-topic="111696">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/juanfmontesinos/48/76115_2.png" class="avatar"> JuanFMontesinos:</div>
<blockquote>
<p>there is a drop-replacement library of PIL I don’t recall the name right now. It’s pillow sidm or ssdm or something similar. That library is compiled and it optimizes the performance depending on the instructions available for ur cpu.</p>
</blockquote>
</aside>
<aside class="quote no-group" data-username="JuanFMontesinos" data-post="2" data-topic="111696">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/juanfmontesinos/48/76115_2.png" class="avatar"> JuanFMontesinos:</div>
<blockquote>
<p>One of the drawbacks of the pytorch’s dataloder is that it cannot run multiprocessing together with gpu processing.</p>
</blockquote>
</aside>
<hr>
<p>Is there a way to easily determine this?</p>
<aside class="quote no-group" data-username="JuanFMontesinos" data-post="2" data-topic="111696">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/juanfmontesinos/48/76115_2.png" class="avatar"> JuanFMontesinos:</div>
<blockquote>
<p>I would say that a good practice (at least to me) is trying to split the dataloading in a cpu-intensive + a gpu-intensive stages.</p>
</blockquote>
</aside>

eduardo4jesus: <p>I have an additional question.</p>
<p>In what stage are the transforms applied?</p>
<ol>
<li>As we get the DataLoader object; or</li>
<li>As we get each batch, which would apply the transform every time that batch is selected.</li>
</ol>

eduardo4jesus: <p>I noticed the transforms are applied as we iterate over the data with DataLoader.</p>

JuanFMontesinos: <p>Im not really familirized with the prebuilt datasets.<br>
For a general dataset,<br>
the workload has to be allocated in the <code>__getitem__</code> func.<br>
This is the function which is called and parallelized by the dataloader.<br>
As I aforementioned, it works only with cpu computations. Therefore any other transformation should be applied after the batch is conformed.</p>