Inconsistency between upsampling with F.interpolate() and F.grid_sample()

username: ang_shine
cooked: <p>Hi! I find upsampling with <code>F.interpolate</code> and <code>F.grid_sample</code> leads to inconsistent results.</p>
<p>First, I try to do the downsampling:</p>
<pre><code class="lang-python"># try downsampling
w_src, w_tgt = 8, 4
input = torch.arange(w_src * w_src).view(1, 1, w_src, w_src).float()

# Create grid to upsample input
d = torch.linspace(-1, 1, w_tgt)
meshx, meshy = torch.meshgrid((d, d))
grid = torch.stack((meshy, meshx), 2)
grid = grid.unsqueeze(0) # add batch dim

output_gs_true = torch.nn.functional.grid_sample(input, grid, mode='bilinear', align_corners=True)
# option-1: correct the wrong grid
grid_false = ((grid + 1) * (w_tgt - 1) + 1) / w_tgt - 1
# option-2: calculate in the correct way from ground-up
# d = torch.arange(w_tgt)
# meshx, meshy = torch.meshgrid((d, d))
# _grid = torch.stack((meshy, meshx), 2)[None]
# grid_false = ((_grid + 1/2) / w_tgt) * 2 - 1

output_gs_false = torch.nn.functional.grid_sample(input, grid, mode='bilinear', align_corners=False)
output_gs_false_c = torch.nn.functional.grid_sample(input, grid_false, mode='bilinear', align_corners=False)

output_itp_true = F.interpolate(input, size=w_tgt, mode='bilinear', align_corners=True)
output_itp_false = F.interpolate(input, size=w_tgt, mode='bilinear', align_corners=False)
</code></pre>
<p><code>output_gs_false_c</code> and <code>output_itp_false</code> are consistent.</p>
<p>Then, I try to do the upsampling:</p>
<pre><code class="lang-python"># try upsampling
w_src, w_tgt = 4, 8
input = torch.arange(w_src * w_src).view(1, 1, w_src, w_src).float()

# Create grid to upsample input
d = torch.linspace(-1, 1, w_tgt)
meshx, meshy = torch.meshgrid((d, d))
grid = torch.stack((meshy, meshx), 2)
grid = grid.unsqueeze(0) # add batch dim

output_gs_true = torch.nn.functional.grid_sample(input, grid, mode='bilinear', align_corners=True)

# option-1: correct the wrong grid
grid_false = ((grid + 1) * (w_tgt - 1) + 1) / w_tgt - 1
# option-2: calculate in the correct way from ground-up
# d = torch.arange(w_tgt)
# meshx, meshy = torch.meshgrid((d, d))
# _grid = torch.stack((meshy, meshx), 2)[None]
# grid_false = ((_grid + 1/2) / w_tgt) * 2 - 1

output_gs_false = torch.nn.functional.grid_sample(input, grid, mode='bilinear', align_corners=False)
output_gs_false_c = torch.nn.functional.grid_sample(input, grid_false, mode='bilinear', align_corners=False)

output_itp_true = F.interpolate(input, size=w_tgt, mode='bilinear', align_corners=True)
output_itp_false = F.interpolate(input, size=w_tgt, mode='bilinear', align_corners=False)
</code></pre>
<p><code>output_gs_false_c</code> and <code>output_itp_false</code> are different.</p>
<p>Is there any operation leading to inconsistency between <code>F.interpolate()</code> and <code>F.grid_sample()</code>, or I calculate the <code>grid</code> for sampling wrong?  <a class="mention" href="/u/bnehoran">@bnehoran</a></p>

username: ang_shine
cooked: <p>Oh, I think I get it. I should use <code>padding_mode="border"</code> in <code>F.grid_sample()</code> to be consistent with <code>F.interpolate</code>. Then, what are the cases making <code>interpolate()</code> and <code>grid_sample()</code>  different, as noted in this <a href="https://github.com/pytorch/pytorch/issues/21457" rel="noopener nofollow ugc">issue</a>?</p>