Suggest to add more tips in F.grid_sample() function

username: WindVChen
cooked: <p>Hi,</p>
<p>I notice that in the newest version Pytorch, when query the coords in the right side and bottom side, it will assign the values of the padding part (align_corners=False). For example, using the example codes provided in <a href="https://discuss.pytorch.org/t/solved-torch-grid-sample/51662/2">https://discuss.pytorch.org/t/solved-torch-grid-sample/51662/2?u=windvchen</a>, the results will be as following when align_corners=False:</p>
<pre><code class="lang-auto">input = torch.arange(4*4).view(1, 1, 4, 4).float()
print(input)
&gt; tensor([[[[ 0.,  1.,  2.,  3.],
          [ 4.,  5.,  6.,  7.],
          [ 8.,  9., 10., 11.],
          [12., 13., 14., 15.]]]])

# Create grid to upsample input
d = torch.linspace(-1, 1, 8)
meshx, meshy = torch.meshgrid((d, d))
grid = torch.stack((meshy, meshx), 2)
grid = grid.unsqueeze(0) # add batch dim

output = torch.nn.functional.grid_sample(input, grid, mode='bilinear', align_corners=False)
print(output)
&gt; tensor([[[[ 0.0000,  0.0357,  0.3214,  0.6071,  0.8929,  1.1786,  1.4643,
            0.7500],
          [ 0.1429,  0.3571,  0.9286,  1.5000,  2.0714,  2.6429,  3.2143,
            1.6429],
          [ 1.2857,  2.6429,  3.2143,  3.7857,  4.3571,  4.9286,  5.5000,
            2.7857],
          [ 2.4286,  4.9286,  5.5000,  6.0714,  6.6429,  7.2143,  7.7857,
            3.9286],
          [ 3.5714,  7.2143,  7.7857,  8.3571,  8.9286,  9.5000, 10.0714,
            5.0714],
          [ 4.7143,  9.5000, 10.0714, 10.6429, 11.2143, 11.7857, 12.3571,
            6.2143],
          [ 5.8571, 11.7857, 12.3571, 12.9286, 13.5000, 14.0714, 14.6429,
            7.3571],
          [ 3.0000,  6.0357,  6.3214,  6.6071,  6.8929,  7.1786,  7.4643,
            3.7500]]]])
</code></pre>
<p>It can be seen that the results are totally not what we want! While if we add the <strong>padding_mode=‘border’</strong>, the results seem to be right.</p>
<pre><code class="lang-auto">&gt; tensor([[[[ 0.0000,  0.0714,  0.6429,  1.2143,  1.7857,  2.3571,  2.9286,
            3.0000],
          [ 0.2857,  0.3571,  0.9286,  1.5000,  2.0714,  2.6429,  3.2143,
            3.2857],
          [ 2.5714,  2.6429,  3.2143,  3.7857,  4.3571,  4.9286,  5.5000,
            5.5714],
          [ 4.8571,  4.9286,  5.5000,  6.0714,  6.6429,  7.2143,  7.7857,
            7.8571],
          [ 7.1429,  7.2143,  7.7857,  8.3571,  8.9286,  9.5000, 10.0714,
           10.1429],
          [ 9.4286,  9.5000, 10.0714, 10.6429, 11.2143, 11.7857, 12.3571,
           12.4286],
          [11.7143, 11.7857, 12.3571, 12.9286, 13.5000, 14.0714, 14.6429,
           14.7143],
          [12.0000, 12.0714, 12.6429, 13.2143, 13.7857, 14.3571, 14.9286,
           15.0000]]]])
</code></pre>
<p>This phenomenon can also be observed when change the mode to “nearest”. I guess what lead to this may be that when set align_corners=False, the current Pytorch tends to take the right and bottom padding part as boundary. Thus the padding_type “border” must be used together to ensure right results.</p>
<p>I think it is a nontrivial problem that users should pay attention. However, in the current Pytorch, there seem to miss the corresponding annotations to prompt users.</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="WindVChen" data-post="1" data-topic="162080">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/windvchen/48/36206_2.png" class="avatar"> WindVChen:</div>
<blockquote>
<p>However, in the current Pytorch, there seem to miss the corresponding annotations to prompt users.</p>
</blockquote>
</aside>
<p>I see a warning executing my code snippet from <a href="https://discuss.pytorch.org/t/solved-torch-grid-sample/51662/2">the older post</a> given as:</p>
<pre data-code-wrap="python"><code class="lang-python">UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
</code></pre>
<p>which results in a different output than seen in my old post.<br>
Setting <code>align_corners=True</code> to use the old behavior results in the same outputs again.</p>
<p>Your approach of using <code>align_corners=False</code> with <code>padding_mode="border"</code> does not yield the same result.</p>

username: WindVChen
cooked: <p>Hi there, thanks for your reply. Maybe I didn’t describe my questions clearly enough before.</p>
<p>I just doubt about the results when set the <strong>align_corners=False</strong>. Take a more simple example, the input is:</p>
<pre><code class="lang-auto">tensor([[[[ 0.,  1.,  2.,  3.],
          [ 4.,  5.,  6.,  7.],
          [ 8.,  9., 10., 11.],
          [12., 13., 14., 15.]]]])
</code></pre>
<p>When I set <strong>align_corners=False</strong> and <strong>mode=‘nearest’</strong>, after using a 8 X 8 coordinates grid map to query the input, the results should be expected to like this:</p>
<pre><code class="lang-auto">tensor([[[[ 0.,  0.,  1.,  1.,  2.,  2.,  3.,  3.],
          [ 0.,  0.,  1.,  1.,  2.,  2.,  3.,  3.],
          [ 4.,  4.,  5.,  5.,  6.,  6.,  7.,  7.],
          [ 4.,  4.,  5.,  5.,  6.,  6.,  7.,  7.],
          [ 8.,  8.,  9.,  9., 10., 10., 11., 11.],
          [ 8.,  8.,  9.,  9., 10., 10., 11., 11.],
          [12., 12., 13., 13., 14., 14., 15., 15.],
          [12., 12., 13., 13., 14., 14., 15., 15.]]]])
</code></pre>
<p>i.e., upsample 2x size. However, when execute the codeline:</p>
<pre><code class="lang-auto">torch.nn.functional.grid_sample(input, grid, mode='nearest', align_corners=False)
</code></pre>
<p>What we can get seems not we want, the bottom line and right column values are set to zero:</p>
<pre><code class="lang-auto">tensor([[[[ 0.,  0.,  1.,  1.,  2.,  2.,  3.,  0.],
          [ 0.,  0.,  1.,  1.,  2.,  2.,  3.,  0.],
          [ 4.,  4.,  5.,  5.,  6.,  6.,  7.,  0.],
          [ 4.,  4.,  5.,  5.,  6.,  6.,  7.,  0.],
          [ 8.,  8.,  9.,  9., 10., 10., 11.,  0.],
          [ 8.,  8.,  9.,  9., 10., 10., 11.,  0.],
          [12., 12., 13., 13., 14., 14., 15.,  0.],
          [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.]]]])
</code></pre>