How to implement bilinear upsample (align_corners=True)?

nightmare4214: <p>I’m wandering how to implement bilinear upsample(align_corners=True).<br>
The code below is my attemption, and the mse is 0.0245.</p>
<pre><code class="lang-auto">#!/usr/bin/env python
# _*_ coding:utf-8 _*_
import torch
from PIL import Image
from torch import nn
from torchvision import transforms


def bilinear(img, upscale, align_corners=False):
    B, c, h, w = img.shape
    new_h, new_w = h * upscale, w * upscale
    if not align_corners:
        x = torch.clip((torch.arange(new_h, dtype=torch.double, device=img.device) + 0.5) * h / new_h - 0.5, 0, h - 1)
        y = torch.clip((torch.arange(new_w, dtype=torch.double, device=img.device) + 0.5) * w / new_w - 0.5, 0, w - 1)
        x = x[:, None].expand(-1, new_w)
        y = y[None, :].expand(new_h, -1)

        x1 = torch.clip(torch.floor(x), 0, h - 2).long()
        y1 = torch.clip(torch.floor(y), 0, w - 2).long()
        x2 = x1 + 1
        y2 = y1 + 1
    else:
        x = torch.clip(torch.arange(new_h, dtype=torch.double, device=img.device) * (h - 1) / (new_h - 1), 0, h - 1)
        y = torch.clip(torch.arange(new_w, dtype=torch.double, device=img.device) * (w - 1) / (new_w - 1), 0, w - 1)
        x = x[:, None].expand(-1, new_w)
        y = y[None, :].expand(new_h, -1)

        x1 = torch.floor(x).long()
        y1 = torch.floor(y).long()
        x2 = torch.ceil(x).long()
        y2 = torch.ceil(y).long()

    q11 = img[..., x1, y1]
    q12 = img[..., x1, y2]
    q21 = img[..., x2, y1]
    q22 = img[..., x2, y2]

    u = (x2 - x)[None, None, ...]
    v = (y2 - y)[None, None, ...]
    _1_u = 1 - u
    _1_v = 1 - v

    w11 = u * v
    w12 = u * _1_v
    w21 = _1_u * v
    w22 = _1_u * _1_v

    result = q11 * w11 + q12 * w12 + q21 * w21 + q22 * w22

    return result.float()


if __name__ == '__main__':
    img = Image.open('lena.png')
    transform = transforms.Compose([transforms.PILToTensor()])
    tensor = transform(img)[None, ...].float()  # (1,3,512,512)
    m1 = nn.Upsample(scale_factor=2, mode='bilinear', align_corners=True)
    x = m1(tensor)
    y = bilinear(tensor, 2, align_corners=True)
    print(torch.allclose(x, y))  # False
    print(torch.sum((x - y) ** 2))  # 0.0245

</code></pre>

tataganesh: <p>This might help to validate your implementation?</p><aside class="quote quote-modified" data-post="4" data-topic="15401">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-does-grid-sample-x-grid-work/15401/4">How does grid_sample(x, grid) work?</a> <a class="badge-category__wrapper " href="/c/vision/5"><span data-category-id="5" data-drop-close="true" class="badge-category " title="Topics related to either pytorch/vision or vision research related topics"><span class="badge-category__name">vision</span></span></a>
  </div>
  <blockquote>
    Ho, 
the implementation is in <a href="https://github.com/pytorch/pytorch/blob/f064c5aa33483061a48994608d890b968ae53fb5/aten/src/THNN/generic/SpatialGridSamplerBilinear.c" rel="noopener nofollow ugc">this</a> file. In particular, the forward method (that computed the output) is <a href="https://github.com/pytorch/pytorch/blob/f064c5aa33483061a48994608d890b968ae53fb5/aten/src/THNN/generic/SpatialGridSamplerBilinear.c#L41" rel="noopener nofollow ugc">this</a> one. 
Here is a quick description of what the C code is doing: 


<a href="https://github.com/pytorch/pytorch/blob/f064c5aa33483061a48994608d890b968ae53fb5/aten/src/THNN/generic/SpatialGridSamplerBilinear.c#L48" rel="noopener nofollow ugc">here</a> check that the input is valid.

<a href="https://github.com/pytorch/pytorch/blob/f064c5aa33483061a48994608d890b968ae53fb5/aten/src/THNN/generic/SpatialGridSamplerBilinear.c#L49-L60" rel="noopener nofollow ugc">here</a> initialize stuff and make the output have the right size.
Then it iterates over the <a href="https://github.com/pytorch/pytorch/blob/f064c5aa33483061a48994608d890b968ae53fb5/aten/src/THNN/generic/SpatialGridSamplerBilinear.c#L62" rel="noopener nofollow ugc">batch</a> then <a href="https://github.com/pytorch/pytorch/blob/f064c5aa33483061a48994608d890b968ae53fb5/aten/src/THNN/generic/SpatialGridSamplerBilinear.c#L63" rel="noopener nofollow ugc">height</a> then <a href="https://github.com/pytorch/pytorch/blob/f064c5aa33483061a48994608d890b968ae53fb5/aten/src/THNN/generic/SpatialGridSamplerBilinear.c#L64" rel="noopener nofollow ugc">width</a>.
For each of these, it gets the input coordinated ix and iy <a href="https://github.com/pytorch/pytorch/blob/f064c5aa33483061a48994608d890b968ae53fb5/aten/src/THNN/generic/SpatialGridSamplerBilinear.c#L66-L71" rel="noopener nofollow ugc">here</a>.

<a href="https://github.com/pytorch/pytorch/blob/f064c5aa33483061a48994608d890b968ae53fb5/aten/src/THNN/generic/SpatialGridSamplerBilinear.c#L73-L81" rel="noopener nofollow ugc">here</a> it computes the 4 points around it that will be used for the interpolation.

<a href="https://github.com/pytorch/pytorch/blob/f064c5aa33483061a48994608d890b968ae53fb5/aten/src/THNN/generic/SpatialGridSamplerBilinear.c#L83-L87" rel="noopener nofollow ugc">here</a> it computes the…
  </blockquote>
</aside>