Different behavior between opencv and pytorch image transforms

username: kekday
cooked: <p>Some sample code from my notebook to start this off:</p>
<p>from skimage import io<br>
import cv2<br>
import torch<br>
import torch.nn.functional as F<br>
import matplotlib.pyplot as plt<br>
import numpy as np</p>
<p>test_image = io.imread(“<a href="https://upload.wikimedia.org/wikipedia/en/7/7d/Lenna_%28test_image%29.png" rel="noopener nofollow ugc">https://upload.wikimedia.org/wikipedia/en/7/7d/Lenna_(test_image).png</a>”)</p>
<h1><a name="create-transformation-matrix-1" class="anchor" href="#create-transformation-matrix-1"></a>Create transformation matrix</h1>
<p>M1 = np.array([[1,0,0.5],[0,1,0],[0,0,1]]).astype(np.float32)</p>
<h1><a name="apply-transformation-matrix-using-opencv-2" class="anchor" href="#apply-transformation-matrix-using-opencv-2"></a>Apply transformation matrix using OpenCV</h1>
<p>cv2im = cv2.warpAffine(test_image, M1[0:2], (test_image.shape[1], test_image.shape[0]))</p>
<h1><a name="reshape-image-for-pytorch-convention-and-apply-same-transformation-matrix-3" class="anchor" href="#reshape-image-for-pytorch-convention-and-apply-same-transformation-matrix-3"></a>Reshape image for PyTorch convention and apply same transformation matrix</h1>
<p>temp_test_image = torch.tensor(np.moveaxis(test_image.astype(np.float32), -1, 0))<br>
grid = F.affine_grid(torch.tensor(M1)[0:2].unsqueeze(0), temp_test_image.unsqueeze(0).shape)<br>
pytorchim = F.grid_sample(temp_test_image.unsqueeze(0), grid).squeeze(0)</p>
<h1><a name="display-results-from-both-4" class="anchor" href="#display-results-from-both-4"></a>Display results from both</h1>
<p>%matplotlib inline<br>
fig=plt.figure(figsize=(15,15))<br>
fig.add_subplot(1,2,1)<br>
plt.imshow(cv2im)<br>
plt.title(“With OpenCV”)<br>
fig.add_subplot(1,2,2)<br>
plt.imshow(np.transpose(pytorchim.numpy(), (1,2,0))/255)<br>
plt.title(“WithPytorch”)<br>
plt.show()</p>
<p>The results look like this:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/8/28e2949382b6b01b6b7c6eab202b338c53299d28.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/28e2949382b6b01b6b7c6eab202b338c53299d28" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/8/28e2949382b6b01b6b7c6eab202b338c53299d28_2_690x336.jpeg" alt="image" data-base62-sha1="5PGxN6WoTIYWLIiEHw1l361zluo" width="690" height="336" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/8/28e2949382b6b01b6b7c6eab202b338c53299d28_2_690x336.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/2/8/28e2949382b6b01b6b7c6eab202b338c53299d28.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/2/8/28e2949382b6b01b6b7c6eab202b338c53299d28.jpeg 2x" data-dominant-color="B57D82"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">888×433 166 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>So if I understand correctly, in case of the opencv transfrom, it correctly moves it right by barely half a pixel (barely noticeable, I know, but you can change the value and notice it’s effect). Whereas in case of the torch transform it seems to move it towards the left (?!) and also the scale is entirely different, since we see quite some movement even with 0.5. I think this is probably due to how we use the grid generated from affine_grid (range in [-1,1])? But could someone tell me how I can apply transforms with the same effects as in OpenCV and vice versa? Also I noticed for rotations, PyTorch rotates images about the center of the image, whereas the convention in OpenCV is to do it about the origin (which is top left corner of the image).</p>
<p>Thanks for your help!</p>

username: edgarriba
cooked: <p>use <a href="https://kornia.readthedocs.io/en/latest/geometry.transform.html#kornia.geometry.transform.warp_affine" rel="nofollow noopener">kornia.warp_affine</a></p>

username: kekday
cooked: <p>Thanks <a class="mention" href="/u/edgarriba">@edgarriba</a>. I’ll try it out. Could you tell me how it’s different?</p>