What we should use align_corners = False

lxtGH: <p>I am very confused with this parameter in pytroch document. According to wiki<br>
<a href="https://en.wikipedia.org/wiki/Bilinear_interpolation" rel="nofollow noopener">https://en.wikipedia.org/wiki/Bilinear_interpolation</a>, the bilinear interpolation formula result is consistent with<br>
align_corners =True. which is defatult before pytorch 0.4.0.<br>
I want to know when should use align_corners=False ??</p>

Mata_Fu: <p>same problem here.<br>
Someone can help?</p>

SimonW: <p>Have you seen the note and examples under <code>Upsample</code>? I think they do a great job in explaining why.</p>

hubutui: <p>Yeah, I check the example under <code>Upsample</code>. I don’t get it.</p>
<pre><code class="lang-auto">&gt;&gt;&gt; input_3x3 = torch.zeros(3, 3).view(1, 1, 3, 3)
&gt;&gt;&gt; input_3x3[:, :, :2, :2].copy_(input)
tensor([[[[ 1.,  2.],
          [ 3.,  4.]]]])
&gt;&gt;&gt; input_3x3
tensor([[[[ 1.,  2.,  0.],
          [ 3.,  4.,  0.],
          [ 0.,  0.,  0.]]]])

&gt;&gt;&gt; m = nn.Upsample(scale_factor=2, mode='bilinear')  # align_corners=False
&gt;&gt;&gt; # Notice that values in top left corner are the same with the small input (except at boundary)
&gt;&gt;&gt; m(input_3x3)
tensor([[[[ 1.0000,  1.2500,  1.7500,  1.5000,  0.5000,  0.0000],
          [ 1.5000,  1.7500,  2.2500,  1.8750,  0.6250,  0.0000],
          [ 2.5000,  2.7500,  3.2500,  2.6250,  0.8750,  0.0000],
          [ 2.2500,  2.4375,  2.8125,  2.2500,  0.7500,  0.0000],
          [ 0.7500,  0.8125,  0.9375,  0.7500,  0.2500,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000]]]])

&gt;&gt;&gt; m = nn.Upsample(scale_factor=2, mode='bilinear', align_corners=True)
&gt;&gt;&gt; # Notice that values in top left corner are now changed
&gt;&gt;&gt; m(input_3x3)
tensor([[[[ 1.0000,  1.4000,  1.8000,  1.6000,  0.8000,  0.0000],
          [ 1.8000,  2.2000,  2.6000,  2.2400,  1.1200,  0.0000],
          [ 2.6000,  3.0000,  3.4000,  2.8800,  1.4400,  0.0000],
          [ 2.4000,  2.7200,  3.0400,  2.5600,  1.2800,  0.0000],
          [ 1.2000,  1.3600,  1.5200,  1.2800,  0.6400,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000]]]])
</code></pre>
<p>Whether <code>align_corners</code> is <code>False</code> or <code>True</code>, the left top corner is always 1.</p>
<p>Oh, my mistake. Now I get it.</p>

ZeluDeng: <p>I will show you a 1-dimension example.</p>
<p>Suppose that you want to resize tensor [0, 1] to [?, ?, ?, ?], so the factor=2.<br>
Now we only care about coordinates.</p>
<p>For mode=‘bilinear’ and align_corners=False, the result is the same with opencv and other popular image processing libraries (I guess). Corresponding coordinates are [-0.25, 0.25, 0.75, 1.25] which are calculate by x_original = (x_upsamle + 0.5) / 2 - 0.5. Then you can these coordinates to interpolate.</p>
<p>For mode=‘bilinear’ and align_corners=True, corresponding coordinates are [0, 1/3, 2/3, 1]. From this, you can see why this is called align_corners=True.</p>
<p>I will be very happy if you find this answer useful.</p>

ZeluDeng: <p>Talk is cheap, show you the code!</p>
<pre><code class="lang-python"># align_corners = False
# x_ori is the coordinate in original image
# x_up is the coordinate in the upsampled image
x_ori = (x_up + 0.5) / factor - 0.5
</code></pre>
<pre><code class="lang-python"># align_corners = True
# h_ori is the height in original image
# h_up is the height in the upsampled image
stride = (h_ori - 1) / (h_up - 1)
x_ori_list = []
# append the first coordinate
x_ori_list.append(0)
for i in range(1, h_up - 1):
    x_ori_list.append(0 + i * stride)
# append the last coordinate
x_ori_list.append(h_ori - 1)
</code></pre>

Dhruvrnaik: <p>I have the same doubt. The corners are the same, what is the difference?</p>

bkkm16: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/6/6a242715685b8192f07c93a57a1d053b8add97bf.png" data-download-href="https://discuss.pytorch.org/uploads/default/6a242715685b8192f07c93a57a1d053b8add97bf" title="align_corners"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/6/6a242715685b8192f07c93a57a1d053b8add97bf_2_635x500.png" alt="align_corners" data-base62-sha1="f8Y4fITSfgsOSRSN53ayIJs91g3" width="635" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/6/6a242715685b8192f07c93a57a1d053b8add97bf_2_635x500.png, https://discuss.pytorch.org/uploads/default/original/2X/6/6a242715685b8192f07c93a57a1d053b8add97bf.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/6/6a242715685b8192f07c93a57a1d053b8add97bf.png 2x" data-dominant-color="F5F6F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">align_corners</span><span class="informations">852×670 66.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Here is a simple illustration I made showing how a 4x4 image is upsampled to 8x8.</p>
<p>When <code>align_corners=True</code>, pixels are regarded as a grid of points. Points at the corners are aligned.</p>
<p>When <code>align_corners=False</code>, pixels are regarded as 1x1 areas. Area boundaries, rather than their centers, are aligned.</p>

Sanster: <p>Thanks for your image! Do you know in semantic segmentation task should we use <code>align_corners=True</code> or <code>align_corners=False</code>?</p>

inamUllah538: <p>In semantic segmentation task should we use  <code>align_corners=True</code>  or  <code>align_corners=False</code> ?</p>

lxtGH: <p>we should set true for better performance.</p>

lxtGH: <p>set align_corners=True</p>

ducha-aiki: <p>Hi,</p>
<p>Thank you for the nice picture. Can we add it to the kornia documentation for explaining the difference?<br>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="16" height="16">
      <a href="https://github.com/kornia/kornia" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars2.githubusercontent.com/u/56968752?s=400&amp;v=4" class="thumbnail onebox-avatar" width="60" height="60">

<h3><a href="https://github.com/kornia/kornia" target="_blank" rel="nofollow noopener">kornia/kornia</a></h3>

<p>Open Source Differentiable Computer Vision Library for PyTorch - kornia/kornia</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>

bkkm16: <p>Sure. Feel free to use it.</p>

Sorench: <p>Awesome pic thanks!<br>
I had issue with my Unet not being equivariant to translations. Turns out align_corners=True was the culprit. As your figure clearly shows in the “True” case there is a shift between the input and output grids that depends on location. This add spatial bias in Unets. Your pic helped see this fast.</p>

csvance: <p>FYI the latest documentation is inconsistent about how bicubic works with <code>align_corners</code>: <a href="https://pytorch.org/docs/master/generated/torch.nn.Upsample.html#torch.nn.Upsample" class="inline-onebox" rel="noopener nofollow ugc">Upsample — PyTorch master documentation</a></p>
<p>For the <code>align_corners</code> argument documentation bicubic is not included after “This only has effect when <code>mode</code> is”. However in the warning section below that bicubic is mentioned and <code>align_corners</code> does seem to affect it when used:</p>
<pre><code class="lang-auto">import torch
import torch.nn.functional as F
import numpy as np

z = torch.from_numpy(np.array([
    [1, 2],
    [3, 4]
], dtype=np.float32)[np.newaxis, np.newaxis, ...])

print(F.interpolate(z, scale_factor=2, mode='bicubic', align_corners=True)[0, 0])
print(F.interpolate(z, scale_factor=2, mode='bicubic', align_corners=False)[0, 0])
</code></pre>
<pre><code class="lang-auto">tensor([[1.0000, 1.3148, 1.6852, 2.0000],
        [1.6296, 1.9444, 2.3148, 2.6296],
        [2.3704, 2.6852, 3.0556, 3.3704],
        [3.0000, 3.3148, 3.6852, 4.0000]])
tensor([[0.6836, 1.0156, 1.5625, 1.8945],
        [1.3477, 1.6797, 2.2266, 2.5586],
        [2.4414, 2.7734, 3.3203, 3.6523],
        [3.1055, 3.4375, 3.9844, 4.3164]])
</code></pre>

Bu_Xu: <p>So, you mean for the Unet we should set align_corners =False??</p>

Sorench: <p>I guess they fix the docs - it seems to mention that align_corners can be set for all IP types except NN now?</p>

Sorench: <p>Yes in my case that was the case if I wanted no bias based on location in the image.<br>
Soren</p>

bruceSz: <p>In my local experiment, this seems to be true: set align_corners = True will generally get results faster for torch.nn.grid_sample  comparing with setting align_corners = False.<br>
But , i am little confused why is that ， do you known any reason behind it ?</p>