Mapping of old pixels to new using torchvision.transforms.functional.rotate

M_S: <p>Hi,<br>
I have my own dataset of images, and labels of objects per image, each label is described as a set of (x,y) points forming a convex polygon.<br>
I want to implement my own augmentation.<br>
Specifically, I want to rotate using <code>torchvision.transforms.functional.rotate(..., expand=True)</code>.<br>
However I also need the transformation function, so I can apply it on my labels and get the new set of points defining the polygon of the object.<br>
Is there a way, after calling <code>rotate</code>, to have the transformation map from old pixels to new pixels?</p>

Amrit_Das: <p><a class="mention" href="/u/m_s">@M_S</a>  would suggest instead of using the torch.utils for transforms, you can design your own utils.py with self designed transformations. You can import it and use it accordingly and should be able to help you out with you problem, ill add a link as an example -</p>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/kuangliu/pytorch-retinanet/blob/master/transform.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/kuangliu/pytorch-retinanet/blob/master/transform.py" target="_blank" rel="nofollow noopener">kuangliu/pytorch-retinanet/blob/master/transform.py</a></h4>
<pre><code class="lang-py">'''Perform transforms on both PIL image and object boxes.'''
import math
import random

import torch
import torchvision.transforms as transforms

from PIL import Image, ImageDraw


def resize(img, boxes, size, max_size=1000):
    '''Resize the input PIL image to the given size.

    Args:
      img: (PIL.Image) image to be resized.
      boxes: (tensor) object boxes, sized [#ojb,4].
      size: (tuple or int)
        - if is tuple, resize image to the size.
        - if is int, resize the shorter side to the size while maintaining the aspect ratio.
      max_size: (int) when size is int, limit the image longer size to max_size.
</code></pre>

  This file has been truncated. <a href="https://github.com/kuangliu/pytorch-retinanet/blob/master/transform.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>Give it a look, Using your own file gives you more freedom to mess around with things.</p>

M_S: <p>Thanks for the answer,<br>
That will require me to calculate the mapping function of a rotation with expanding by myself - my question was if such implementation already exists in the library (doesn’t have to be PyTorch native code but preferred as the rest of my augmentations are such)</p>

Amrit_Das: <p>Its very simple you just have to create a rotation matrix :</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://www.mathworks.com/matlabcentral/answers/339432-how-to-rotate-an-image-using-rotation-matrix">
  <header class="source">
      <img src="https://www.mathworks.com/etc.clientlibs/mathworks/clientlibs/customer-ui/templates/common/resources/images/favicon.20231004195357773.ico" class="site-icon" width="256" height="256">

      <a href="https://www.mathworks.com/matlabcentral/answers/339432-how-to-rotate-an-image-using-rotation-matrix" target="_blank" rel="noopener nofollow ugc">mathworks.com</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/361;"><img src="https://www.mathworks.com/content/dam/mathworks/mathworks-dot-com/images/responsive/thumbnails/generic-og-thumbnail.jpg" class="thumbnail" width="690" height="361"></div>

<h3><a href="https://www.mathworks.com/matlabcentral/answers/339432-how-to-rotate-an-image-using-rotation-matrix" target="_blank" rel="noopener nofollow ugc">how to rotate an image using rotation matrix.</a></h3>

  <p>I want to rotate an image of 568*720*3 using the rotation matrix R=[cos(angle) -sin(angle);sin(angle) cos(angle)];
do anyone have any idea how to do it.
Thanks and Humble Regards</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>or you can check this post for suggested codes:</p><aside class="quote quote-modified" data-post="2" data-topic="19522">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/affine-transformation-matrix-paramters-conversion/19522/2">Affine transformation matrix paramters conversion</a> <a class="badge-category__wrapper " href="/c/vision/5"><span data-category-id="5" data-drop-close="true" class="badge-category " title="Topics related to either pytorch/vision or vision research related topics"><span class="badge-category__name">vision</span></span></a>
  </div>
  <blockquote>
    How are you comparing the results? 
It seems scipy.ndimage.affine_transform uses pixel values for the translation part, while F.affine_grid seems to want values in the range [-1, 1] (which you already provided). 
This code tries to rotate and translate a line: 
x = torch.eye(10).view(1, 1, 10, 10)
theta = torch.zeros(1, 2, 3)
angle = np.pi/2.
theta[:, :, :2] = torch.tensor([[np.cos(angle), -1.0*np.sin(angle)],
                                [np.sin(angle), np.cos(angle)]])
theta[:, :, 2] = 0.5
…
  </blockquote>
</aside>


M_S: <p>Yes, but as I mentioned that I require expansion, there is a translation followed after the rotation.<br>
A solution I had in mind is to first pad the original images alot, which will then allow me to drop out expanding, and then indeed an easy calculation with a rotation matrix.<br>
However I am still looking for an existing solution for this problem. If there’s no such, then I’ll implement…</p>

Marat: <p>There is a weird way (I soon will try it) to do so. Make a x,y grid transform it to PIL image and apply reverse transform for that grids and vuala you get mapping from old coordinates to new.</p>
<pre><code class="lang-auto">        w, h = img.width, img.height

        assert w == h

        grid_x = torch.arange(w).repeat(h, 1).view([h, w]).type(torch.FloatTensor) / w
        grid_y = torch.arange(h).repeat(w, 1).t().view([h, w]).type(torch.FloatTensor) / h
        dummy = torch.zeros((h, w))
        grid_xy = torch.stack([grid_x, dummy, grid_y])
        grid_xy = transforms.ToPILImage()(grid_xy)

        rot = 30
        img = img.rotate(rot, Image.NEAREST, expand=0)
        grid_xy = grid_xy.rotate(-rot, Image.NEAREST, expand=0)

        grid_xy.show()
        img.show()

        grid_xy = transforms.ToTensor()(grid_xy)
        x_orig = label[:, 2] * w
        y_orig = label[:, 3] * h

        label[:, 2] = grid_xy[0, y_orig, x_orig]
        label[:, 3] = grid_xy[2, y_orig, x_orig]
</code></pre>
<p>The most weird thing here is that you may apply ant number of transformations whats so ever without any geometry headache  the only thing you care about is keep transforming grid_xy in reverse order<br>
Enjoy!</p>