Image Processing with Python — Applying Homography for Image Warping | by Tonichi Edeza | Towards Data Science

Rotating images is nothing new, most individuals who have done even basic photo editing have had to do this at some point. In this article we shall tackle ways to adjust an image to make it seems that it had been taken at a different angle.
Let’s get started!
As always, begin by importing the required Python libraries.
import matplotlib.pyplot as pltimport numpy as npfrom matplotlib.patches import Circlefrom skimage import transformfrom skimage.io import imread, imshow
Excellent, now let us import the image we shall be working with.
sign = imread('words_of_wisdom.png')plt.figure(num=None, figsize=(8, 6), dpi=80)
As we can see, the sign above is slightly ajar. Though I intentionally took that photo from that perspective, let us see if we can “correct” it. A very basic way to do that is via the transform.rotate function in Skimage. The below code does precisely that.
sign_rotate = transform.rotate(sign, 330)plt.figure(num=None, figsize=(8, 6), dpi=80)imshow(sign_rotate);
As we can see the image has been rotated so that the sign is now parallel to the x-axis. However, we see two very glaring issues.
The first is that this is essentially just rotating the image (big surprise right?). The second is that there are now white spaces where the image used to be. Let us try to remedy this.
The function we used actually has a parameter call mode, this will help alleviate the white spaces. Let us experiment with the different modes we can use.
def rotate_fills(image):    modes = ['constant', 'edge','symmetric','reflect','wrap']    fig, ax = plt.subplots(3,2, figsize=(7, 10), dpi = 200)    for n, ax in enumerate(ax.flatten()):        n = n-1        if n == -1:            ax.set_title(f'original', fontsize = 12)            ax.imshow(image)            ax.set_axis_off()        else:             ax.set_title(f'{modes[n]}', fontsize = 12)            ax.imshow(transform.rotate(image, 330, mode = modes[n]))            ax.set_axis_off()            fig.tight_layout();rotate_fills(sign)
We can see that by experimenting with different filling methods, that reflect and symmetric both stand out as prime candidates. However they do come with a very subtle drawback.
def comparing_fills(image):    modes = ['symmetric','reflect']    fig, ax = plt.subplots(1,2, figsize=(7, 10), dpi = 200)    ax[0].imshow(transform.rotate(image, 330, mode = modes[0]))    ax[0].set_title(f'{modes[0]}', fontsize = 15)    ax[0].set_axis_off()        ax[1].imshow(transform.rotate(image, 330, mode = modes[1]))    ax[1].set_title(f'{modes[1]}', fontsize = 15)    ax[1].set_axis_off()            fig.tight_layout();    comparing_fills(sign)
Notice how the lower right corner of the image is being reflected. This makes it seem as if the tree has a second branch in that corner. Of course when we inspect the original image this is not the case.
So despite our best efforts, rotate does not seem to be the best function for our task. Fortunately, there is a function that we can use, the transform.warp function in Skimage.
However, it does require a little more work to implement when compared to rotate. For one we have to get the points we are concerned with and where we want to project them. Excuse the long code below, but I found it important to be explicit with the programming so that it would be easy to follow along.
points_of_interest =[[360, 110],                      [420, 270],                      [130, 400],                      [100, 280]]projection = [[500, 200],              [500, 390],              [100, 390],              [100, 200]]color = 'red'fig, ax = plt.subplots(1,2, figsize=(15, 10), dpi = 80)patch1 = Circle((points_of_interest[0][0],points_of_interest[0][1]),                 10, facecolor = color)patch2 = Circle((points_of_interest[1][0],points_of_interest[1][1]),                  10, facecolor = color)patch3 = Circle((points_of_interest[2][0],points_of_interest[2][1]),                 10, facecolor = color)patch4 = Circle((points_of_interest[3][0],points_of_interest[3][1]),                  10, facecolor = color)patch5 = Circle((projection[0][0],projection[0][1]), 10,                  facecolor = color)patch6 = Circle((projection[1][0],projection[1][1]), 10,                  facecolor = color)patch7 = Circle((projection[2][0],projection[2][1]), 10,                  facecolor = color)patch8 = Circle((projection[3][0],projection[3][1]), 10,                  facecolor = color)ax[0].add_patch(patch1)ax[0].add_patch(patch2)ax[0].add_patch(patch3)ax[0].add_patch(patch4)ax[0].imshow(sign);ax[1].add_patch(patch5)ax[1].add_patch(patch6)ax[1].add_patch(patch7)ax[1].add_patch(patch8)ax[1].imshow(np.ones((sign.shape[0], sign.shape[1])));
What we essentially did was identify the location of the current corners of the sign and plot out where we want them to go.
Note that this step is only to clarify the source location and the destination location. To actually implement the function we must use the below code.
points_of_interest = np.array([[360, 110],                                [420, 270],                                [130, 400],                                [100, 280]])projection = np.array([[500, 200],                       [500, 390],                       [100, 390],                       [100, 200]])tform = transform.estimate_transform('projective', points_of_interest, projection)tf_img_warp = transform.warp(sign, tform.inverse, mode = 'symmetric')plt.figure(num=None, figsize=(8, 6), dpi=80)fig, ax = plt.subplots(1,2, figsize=(15, 10), dpi = 80)ax[0].set_title(f'Original', fontsize = 15)ax[0].imshow(sign)ax[0].set_axis_off();ax[1].set_title(f'Transformed', fontsize = 15)ax[1].imshow(tf_img_warp)ax[1].set_axis_off();
We see that this is a tremendous improvement from our previous method. Not only has the angle of the sign been corrected, but we no longer have to deal with the pesky issue of white spaces.
However, note that though this method is impressive it does not always yield the best results. Take the below image for example.
house = imread('artsy_house.png')plt.figure(num=None, figsize=(8, 6), dpi=80)
Due to its noticeable thickness, the function will not provide the best results. To demonstrate let us plot out the points of interest. Additionally, I have provided a shorthand way to run the previous code
points_of_interest =[[105, 60],                      [260, 85],                      [275, 295],                      [110, 290]]projection = [[100, 75],              [275, 75],              [275, 290],              [100, 290]]color = 'green'patches = []fig, ax = plt.subplots(1,2, figsize=(15, 10), dpi = 80)for coordinates in (points_of_interest + projection):    patch = Circle((coordinates[0],coordinates[1]), 10,                     facecolor = color)    patches.append(patch)for p in patches[:4]:    ax[0].add_patch(p)ax[0].imshow(house);for p in patches[4:]:    ax[1].add_patch(p)ax[1].imshow(np.ones((house.shape[0], house.shape[1])));
Excellent, let us now project the image to the target points.
points_of_interest = np.array(points_of_interest)projection = np.array(projection)tform = transform.estimate_transform('projective', points_of_interest, projection)tf_img_warp = transform.warp(house, tform.inverse, mode = 'edge')plt.figure(num=None, figsize=(8, 6), dpi=80)fig, ax = plt.subplots(1,2, figsize=(15, 10), dpi = 80)ax[0].set_title(f'Original', fontsize = 15)ax[0].imshow(house)ax[0].set_axis_off();ax[1].set_title(f'Transformed', fontsize = 15)ax[1].imshow(tf_img_warp)ax[1].set_axis_off();
Though we were able to properly project the image onto the new points, note how it looks slightly unnatural. Looking at the left portion of the painting, we see that the raised section is still noticeable. That feature should not be seen if we had actually taken the shot from a from a front angle. Also notice how there is some slight distortion in the upper left section of the image, this was caused by the use of the edge fill.
In Conclusion
We have seen the power of using transform.warp to literally change the perspective of an image. There are a plethora of ways to apply this technique, ranging from terrain surveying to image recreation. I hope that you now have a better appreciation for this method and can now imagine how to use it in your projects.