Surprising behavior from grid_sample

username: erikbrntsn
cooked: <p>If I sample a square image using F.grid_sample(input, grid), where the grid has been created using F.affine_grid(theta, size), with a non zero rotation part of theta, I get the expected result. If I do the same for a non square image I get a very skewed image.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/2/f2121d9e70fe38aa6df43bc789d343f22e1050c2.png" data-download-href="https://discuss.pytorch.org/uploads/default/f2121d9e70fe38aa6df43bc789d343f22e1050c2" title="grid_sample_example"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/2/f2121d9e70fe38aa6df43bc789d343f22e1050c2_2_690x353.png" alt="grid_sample_example" data-base62-sha1="yxsmlYevUdHg5ux8VHtAYIl2HCy" width="690" height="353" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/2/f2121d9e70fe38aa6df43bc789d343f22e1050c2_2_690x353.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/2/f2121d9e70fe38aa6df43bc789d343f22e1050c2_2_1035x529.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/f/2/f2121d9e70fe38aa6df43bc789d343f22e1050c2_2_1380x706.png 2x" data-dominant-color="BBBBBB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">grid_sample_example</span><span class="informations">1920×983 25.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Is this the intended behavior? And if so is there another easy way to get a sampled image that is not skewed when sampling non square images?</p>
<p>Note: in the minimal examples here theta only has a rotation part. No translation. In the real task where I want to use affine_grid and grid_sample I am using both rotation and translation.</p>

username: erikbrntsn
cooked: <p>Code to produce these results:</p>
<pre><code class="lang-auto">import torch
import torch.nn.functional as F
import imageio


def fill_center(img, s):
    n, rows, cols = img.shape
    s //= 2
    c_row = rows // 2
    c_col = cols // 2
    img[:, c_row - s:c_row + s, c_col - s:c_col + s] = 1
    return img


def rot_mat(angles):
    coss = torch.cos(angles).unsqueeze(1)
    sins = torch.sin(angles).unsqueeze(1)
    rotations = torch.stack((torch.cat((coss, -sins), dim=1),
                             torch.cat((sins, coss), dim=1)), dim=2)  # n_images, 2, 2
    return rotations


shapes = [[300, 300], [300, 1000]]
names = ["square", "wide"]
n_images = 2
parameters = torch.zeros((n_images, 3))
parameters[1, 0] = 3.1415 / 4
for shape, name in zip(shapes, names):

    orig_image = torch.zeros((1, *shape)) + 0.1
    orig_image = fill_center(orig_image, 100)

    orig_image = orig_image.expand(n_images, *orig_image.shape)

    theta = torch.cat((rot_mat(parameters[:, 0]), parameters[:, 1:, None]), dim=-1)

    scale = 1
    n, c, rows, cols = orig_image.shape
    sample_grid = F.affine_grid(theta, (n_images, 1, int(rows * scale), int(cols * scale)), align_corners=False).to(orig_image.device)

    images = F.grid_sample(orig_image, sample_grid, align_corners=False)

    images *= 255
    images = images.permute(0, 2, 3, 1).to("cpu", torch.uint8)

    for i in range(n_images):
        imageio.imsave(f"{name}_{i:0&gt;3}.jpg", images[i, ...])

</code></pre>

username: erikbrntsn
cooked: <p>In case anybody is interested. I managed to obtain the desired results by adding scaling to theta according to the image size:</p>
<pre><code class="lang-auto">    theta = torch.cat((rot_mat(parameters[:, 0]), parameters[:, 1:, None]), dim=-1)
    theta[:, :, 0] *= shape[1] / 2
    theta[:, :, 1] *= shape[0] / 2

    scale = 1
    n, c, rows, cols = orig_image.shape
    sample_grid = F.affine_grid(theta, (n_images, 1, int(rows * scale), int(cols * scale)), align_corners=False).to(orig_image.device)
    sample_grid[..., 0] /= shape[1] / 2
    sample_grid[..., 1] /= shape[0] / 2

    images = F.grid_sample(orig_image, sample_grid, align_corners=False)
</code></pre>
<p>giving the following result<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/b/3/b3cf9884afe79179f6340293da617f46e2e2fbac.jpeg" alt="wide_001" data-base62-sha1="pEGeP7ITFpM7k5HFtzoE8Fb5kRu" width="300" height="90"></p>