How to do image resizing with bilinear interpolation using torch tensor on CUDA?

username: tlim
cooked: <p>Hi all,</p>
<p>I was wondering whether has anyone done bilinear interpolation resizing with PyTorch Tensor under CUDA?</p>
<p>I tried this using <code>torch.nn.functional.F.interpolate(rgb_image,(size,size))</code> and it works to resize the RGB image tensor shape (batch,channel,size,size). However, the default mode is ‘nearest’ and when I change to ‘bilinear’, it comes with an error about the tensor type which I fixed by changing to <code>torch.FloatTensor</code> but then the output shown is grainy and pixelated with noise. My command:</p>
<pre><code class="lang-auto">resized = F.interpolate(rgb_image_tensor.type(torch.FloatTensor),(256,256),mode='bilinear')
</code></pre>
<p>I show the image by using</p>
<pre><code class="lang-auto">result =  torchvision.transform.functional.to_pil_image(resized[0])
result.show()
</code></pre>
<p>Default way works, shows the image is resized accordingly but ‘bilinear’ shows noisy image, any clues?</p>
<p>‘bilinear’ mode output:</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/2/a/2ab0e2be2266d264b9ae05befb35faaf14e4074a.jpeg" alt="Screenshot from 2021-06-24 09-15-04" data-base62-sha1="65F1PZ45aSqDISvENGLw9MlCoDo" width="276" height="311"></p>

username: ptrblck
cooked: <p>Your output might be clipped to <code>[0, 1]</code>, if you are trying to visualize floating point numbers or <code>[0, 255]</code> if you are using <code>uint8</code>.<br>
I cannot reproduce the issue using the latest stable releases and by making sure I’m casting to the expected type:</p>
<pre data-code-wrap="python"><code class="lang-python">img = PIL.Image.open('drums.png')
img_arr = np.array(img)[:, :, :3] # remove alpha channel
plt.imshow(img_arr)

x = torch.from_numpy(img_arr).permute(2, 0, 1).unsqueeze(0).float()

out1 = F.interpolate(x, size=(200, 200))
out2 = F.interpolate(x, size=(200, 200), mode='bilinear')

plt.imshow(out1[0].permute(1, 2, 0).byte().numpy())
plt.imshow(out2[0].permute(1, 2, 0).byte().numpy())
</code></pre>
<p>Original<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/e/2/e2869615ead54311cb40b2b7db6a94b9f695d9f6.jpeg" alt="image" data-base62-sha1="wjWg03vDrtDXI4DSpLVJMGXfxJk" width="368" height="252"></p>
<p><code>out1</code><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/3/1/3138fa4df6ae16d9b86c10757087a9a735d16702.png" alt="image" data-base62-sha1="71rt9V7OwEMewXszLOrmpMbTVbY" width="257" height="252"></p>
<p><code>out2</code><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/1/e/1e2ad91187d5fb781f55f207cd8ed1a98d228e25.png" alt="image" data-base62-sha1="4iS7LUoGYnOOPtQqsRoHJvIQAzr" width="257" height="252"></p>

username: tlim
cooked: <p>You are right! Thank you for your help, appreciate it!</p>
<p>I just needed to cast the bilinear output as <code>.byte()</code>!  I did not realised this as <code>mode='nearest'</code> is able to take in tensor as byte type and output given is already in byte. Meanwhile, <code>mode='bilinear</code> needs FloatTensor and outputs in float. Thus, to show the image, we have to put it back to byte type.</p>
<p>I am actually amazed that pytorch has implemented resizing for GPU. Now I can skip using <code>cv2.resize()</code> and remain the tensor on the GPU for resizing instead!</p>
<p>Any idea on whether are there major differences between <code>cv2.resize() -&gt; bilinear</code> to <code>F.interpolate(....,mode='bilinear')</code>? Your comments are appreciated.</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="tlim" data-post="3" data-topic="124894">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tlim/48/32364_2.png" class="avatar"> tlim:</div>
<blockquote>
<p>Any idea on whether are there major differences between <code>cv2.resize() -&gt; bilinear</code> to <code>F.interpolate(....,mode='bilinear')</code>? Your comments are appreciated.</p>
</blockquote>
</aside>
<p>I haven’t checked the latest updates in torchvision, but know that there was at least work in progress to avoid small numerical differences between different cv libraries.<br>
You could run a quick test with the latest <code>torchvision</code> release (<code>0.10.0</code>) or the nightly and see how large these differences would be.</p>

username: tlim
cooked: <p>thank you for the help and reply.</p>
<p>Actually, I realised that it matters more that the <code>torchvision.transform.resize()</code> is same as <code>torch.nn.functional.interpolate()</code> for my use case as the model is trained and tested under torchvision transformation for the DataLoader.</p>
<p>Just to complete this thread for anyone interested, I found that both functions is the same:</p>
<p><code>torchvision.transform.resize()</code> calls to this <code>resize()</code> function here <a href="https://github.com/pytorch/vision/blob/ab60e538961e4bb25e7d8db44a4d0d96155e3644/torchvision/transforms/functional_tensor.py#L480" class="inline-onebox" rel="noopener nofollow ugc">vision/functional_tensor.py at ab60e538961e4bb25e7d8db44a4d0d96155e3644 · pytorch/vision · GitHub</a> which imports <code>interpolate</code> from <code>torch.nn.functional</code></p>

username: edgarriba
cooked: <p>evnetually you can use from kornia which is more opencv oriented the api</p>
<ul>
<li><a href="https://kornia.readthedocs.io/en/latest/geometry.transform.html#kornia.geometry.transform.resize" rel="noopener nofollow ugc"><code>resize</code></a></li>
<li><a href="https://kornia.readthedocs.io/en/latest/geometry.transform.html#kornia.geometry.transform.rescale" rel="noopener nofollow ugc"><code>rescale</code></a></li>
</ul>