Storing images as tensor?

Naman-ntc: <p>I was thinking that would it be better to store and load images present as tensors?<br>
In general I load images with opencv or PIL and converting them to tensors, if I converted my data into tensor and dump them would it be faster while loading??</p>

Jk749: <p>You can try Dataloader, which doesn’t require to actually load all images in memory at any point.</p>
<p>You can read more about it here: <a href="https://pytorch.org/docs/stable/data.html" rel="nofollow noopener">https://pytorch.org/docs/stable/data.html</a><br>
tutorial: <a href="https://github.com/pytorch/tutorials/blob/master/beginner_source/data_loading_tutorial.py" rel="nofollow noopener">https://github.com/pytorch/tutorials/blob/master/beginner_source/data_loading_tutorial.py</a></p>

Naman-ntc: <p>Yeah I know about it, I was thibking about optimizing everytime <code>__geittem__</code> is called, you have to load images so what is better to load, image itself or image stored as a tensor</p>

ptrblck: <p>If you don’t want to transform your images, you could load them as <code>tensor</code>s.<br>
However, if you do need a transformation, you would have to transform the <code>tensor</code>s back to <code>PIL.Image</code>s, apply the augmentation/transformation, and transform them back to <code>tensor</code>s.<br>
This shouldn’t be too bad regarding the performance, but can be annoying.</p>
<p>On the other hand do you want to load single images/tensors?<br>
If so, I think you won’t get any performance improvement:</p>
<pre><code class="lang-python">x = torch.empty(3, 224, 224).uniform_(0, 1)
img = transforms.ToPILImage()(x)

# save
torch.save(x, 'tensor_image.pth')
img.save('pil_image.png')

%timeit torch.load('tensor_image.pth')
&gt; 1000 loops, best of 3: 206 µs per loop
%timeit Image.open('pil_image.png')
&gt; 10000 loops, best of 3: 86.6 µs per loop
</code></pre>
<p>I think you’ll be faster if you load a whole batch of tensors.</p>

Naman-ntc: <p>Oh <code> torch.load</code> is slower! I thought it would have been efficient!<br>
I guess due to pickle backend perhaps?!</p>
<blockquote>
<p>I think you’ll be faster if you load a whole batch of tensors.</p>
</blockquote>
<p>Yeah perhaps loading multiple images consecutively would (SHOULD!) be slower compared to loading tensor at once.</p>
<p>Thanks!</p>

ptrblck: <p>I think the tensor is just too small to get a significant performance boost.</p>
<aside class="quote no-group" data-username="Naman-ntc" data-post="5" data-topic="20096">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/naman-ntc/48/32044_2.png" class="avatar"> Naman-ntc:</div>
<blockquote>
<p>Yeah perhaps loading multiple images consecutively would (SHOULD!) be slower compared to loading tensor at once.</p>
</blockquote>
</aside>
<p>That’s what I think, too.</p>

Naman-ntc: <p>Just as a followup modified the script for loading and comparing multiple images<br>
Even for loading 19 images at once loading tensor greatly outperform other methods</p>
<pre><code class="lang-auto">from PIL import Image
import torch
import torchvision.transforms as transforms
import timeit
import cv2

K = 19

x = torch.zeros(3, 224, 224).uniform_(0, 1)
img = transforms.ToPILImage()(x)

for i in range(K):
	img.save(str(i) + ".png")


x = x.unsqueeze(0).expand(K,3,224,224)
torch.save(x, 'tensor_images.pth')

def loading1():
	torch.load('tensor_images.pth')

def loading2():
	for i in range(K):
		Image.open(str(i) + ".png")

def loading3():
	for i in range(K):
		cv2.imread(str(i) + ".png")

</code></pre>
<p>And the outputs</p>
<pre><code class="lang-auto">&gt;&gt;&gt; timeit.timeit(loading1, number=100000)
14.435413494997192
&gt;&gt;&gt; timeit.timeit(loading2, number=100000)
95.70677890698425

</code></pre>

justusschock: <p>That’s great. What about the filesize? Is there any significant pytorch-specific overhead?</p>

Naman-ntc: <p>Stored only 19 images so can’t say much but yeah file sizes are smaller by factor of about 2</p>

Daniel_Maier: <p>Hey I am facing the same consideration. I think in your performance test you should really compare loading image stored as tensors vs as .PNG + CONVERTING to tensor because you will have to make this conversion eventually.</p>
<p>Also keep in mind that if you want to use .PNG files in a traditional Dataset class, you will have to load the tensors one by one in the <strong>get_item</strong>() method, possibly negating your detected performance boost as stated by <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>Here maybe some other things to consider as well:<br>
-single image files are much easier to inspect and search for possible errors (e.g. by sorting images by size and looking at the extremes).<br>
-copying/moving single image files is very slow compared to a single .pt file. You will essentially have to zip/unzip to move your dataset around to avoid the system having to handle every single file.<br>
-single image files can be lazily loaded using Dataloaders, while a single .pt file needs to be loaded as a whole in a Datasets init method</p>