Title: Rotate images (correctly) with OpenCV and Python - PyImageSearch

URL Source: https://pyimagesearch.com/2017/01/02/rotate-images-correctly-with-opencv-and-python/

Published Time: 2017-01-02T15:00:06+00:00

Markdown Content:
Let me tell you an embarrassing story of **how I _wasted_ three weeks of research time** during graduate school six years ago.

![Image 1: opencv_rotated_header](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20405%20218'%3E%3C/svg%3E)

It was the end of my second semester of coursework.

I had taken all of my exams early and all my projects for the semester had been submitted.

Since my school obligations were essentially nil, I started experimenting with [(automatically) identifying prescription pills in images](http://ieeexplore.ieee.org/document/6467032/?reload=true), something I know a [thing](http://www.prweb.com/releases/2013/8/prweb10995233.htm) or [two](http://my.umbc.edu/groups/entrepreneurship/posts/36976) about (but back then I was just getting started with my research).

At the time, my research goal was to find and identify methods to reliably quantify pills in a _rotation invariant_ manner. Regardless of how the pill was rotated, I wanted the output feature vector to be (approximately) the same (the feature vectors will never be to _completely identical_ in a real-world application due to lighting conditions, camera sensors, floating point errors, etc.).

After the first week I was making fantastic progress.

I was able to extract features from my dataset of pills, index them, and then _identify_ my test set of pills regardless of how they were oriented…

…however, there was a problem:

**My method was only working with _round, circular pills_ — I was getting completely nonsensical results for _oblong pills._**

How could that be?

I racked my brain for the explanation.

Was there a flaw in the logic of my feature extraction algorithm?

Was I not matching the features correctly?

Or was it something else entirely…**_like a problem with my image preprocessing._**

While I might have been ashamed to admit this as a graduate student, the problem was the latter:

**I goofed up.**

It turns out that during the image preprocessing phase, I was rotating my images incorrectly.

Since round pills have are approximately square in their aspect ratio, the rotation bug wasn’t a problem for them. Here you can see a round pill being rotated a full 360 degrees without an issue:

![Image 2: Figure 1: Rotating a circular pill doesn't reveal any obvious problems.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20268%20222'%3E%3C/svg%3E)

**Figure 1:** Rotating a circular pill doesn’t reveal any obvious problems.

But for oblong pills, they would be “cut off” in the rotation process, like this:

![Image 3: Figure 2: However, rotating oblong pills using the OpenCV's standard cv2.getRotationMatrix2D and cv2.warpAffine functions caused me some problems.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20268%20152'%3E%3C/svg%3E)

**Figure 2:** However, rotating oblong pills using the OpenCV’s standard _cv2.getRotationMatrix2D_ and _cv2.warpAffine_ functions caused me some problems that weren’t immediately obvious.

In essence, I was only quantifying _part_ of the rotated, oblong pills; hence my strange results.

I spent _three weeks_ and _part of my Christmas vacation_ banging my head against the wall trying to diagnose the bug — only to feel quite embarrassed when I realized it was due to me being negligent with the `cv2.rotate` function.

You see, the size of the output image needs to be _adjusted_, otherwise, the corners of my image would be cut off.

How did I accomplish this and squash the bug for good?

**To learn how to rotate images with OpenCV such that the _entire_ image is included and _none_ of the image is cut off, _just keep reading._**

![Image 4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

In the remainder of this blog post I’ll discuss common issues that you may run into when rotating images with OpenCV and Python.

Specifically, we’ll be examining the problem of what happens when the corners of an image are “cut off” during the rotation process.

To make sure we all understand this rotation issue with OpenCV and Python I will:

*   Start with a simple example demonstrating the rotation problem.
*   Provide a rotation function that ensures images are not cut off in the rotation process.
*   Discuss how I resolved my pill identification issue using this method.

### A simple rotation problem with OpenCV

Let’s get this blog post started with an example script.

Open up a new file, name it `rotate_simple.py` , and insert the following code:

\# import the necessary packages
import numpy as np
import argparse
import imutils
import cv2

# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add\_argument("-i", "--image", required=True,
	help="path to the image file")
args = vars(ap.parse\_args())

**Lines 2-5** start by importing our required Python packages.

If you don’t already have [imutils](https://github.com/jrosebr1/imutils), my series of OpenCV convenience functions installed, you’ll want to do that now:

$ pip install imutils

If you _already_ have `imutils` installed, make sure you have upgraded to the latest version:

$ pip install --upgrade imutils

From there, **Lines 8-10** parse our command line arguments. We only need a single switch here, `--image` , which is the path to where our image resides on disk.

Let’s move on to actually rotating our image:

\# load the image from disk
image = cv2.imread(args\["image"\])

# loop over the rotation angles
for angle in np.arange(0, 360, 15):
	rotated = imutils.rotate(image, angle)
	cv2.imshow("Rotated (Problematic)", rotated)
	cv2.waitKey(0)

# loop over the rotation angles again, this time ensuring
# no part of the image is cut off
for angle in np.arange(0, 360, 15):
	rotated = imutils.rotate\_bound(image, angle)
	cv2.imshow("Rotated (Correct)", rotated)
	cv2.waitKey(0)

**Line 14** loads the image we want to rotate from disk.

We then loop over various angles in the range _\[0, 360\]_ in 15 degree increments (**Line 17**).

For each of these angles we call `imutils.rotate` , which rotates our `image` the specified number of `angle` degrees about the center of the image. We then display the rotated image to our screen.

**Lines 24-27** perform an identical process, but this time we call `imutils.rotate_bound` (I’ll provide the implementation of this function in the next section).

As the name of this method suggests, we are going to ensure the _entire image_ is bound inside the window and none is cut off.

To see this script in action, be sure to download the source code using the **_“Downloads”_** section of this blog post, followed by executing the command below:

$ python rotate\_simple.py --image images/saratoga.jpg

The output of using the `imutils.rotate` function on a non-square image can be seen below:

![Image 5: Figure 3: An example of corners being cut off when rotating an image using OpenCV and Python.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20250%20200'%3E%3C/svg%3E)

**Figure 3:** An example of corners being cut off when rotating an image using OpenCV and Python.

As you can see, the image is “cut off” when it’s rotated — the entire image is not kept in the field of view.

But if we use `imutils.rotate_bound` we can resolve this issue:

![Image 6: Figure 4: We can ensure the entire image is kept in the field of view by modifying the matrix returned by cv2.getRotationMatrix2D.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20366%20288'%3E%3C/svg%3E)

**Figure 4:** We can ensure the entire image is kept in the field of view by modifying the matrix returned by _cv2.getRotationMatrix2D_.

Awesome, we fixed the problem!

So does this mean that we should _always_ use `.rotate_bound` over the `.rotate` method?

What makes it so special?

And what’s going on under the hood?

I’ll answer these questions in the next section.

### Implementing a rotation function that _doesn’t_ cut off your images

Let me start off by saying there is _nothing_ wrong with the `cv2.getRotationMatrix2D` and `cv2.warpAffine` functions that are used to rotate images inside OpenCV.

In reality, these functions give us more freedom than perhaps we are comfortable with (sort of like comparing manual memory management with C versus automatic garbage collection with Java).

The `cv2.getRotationMatrix2D` function **_doesn’t care_** if we would like the entire rotated image to kept.

It **_doesn’t care_** if the image is cut off.

And it **_won’t help you_** if you shoot yourself in the foot when using this function (I found this out the hard way and it took 3 weeks to stop the bleeding).

Instead, what you need to do is understand _what_ the rotation matrix is and _how_ it’s constructed.

You see, when you rotate an image with OpenCV you call `cv2.getRotationMatrix2D` which returns a matrix _M_ that looks something like this:

![Image 7: Figure 5: The structure of the matrix M returned by cv2.getRotationMatrix2D.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20265%2040'%3E%3C/svg%3E)

**Figure 5:** The structure of the matrix _M_ returned by _cv2.getRotationMatrix2D_.

This matrix looks scary, but I promise you: it’s not.

To understand it, let’s assume we want to rotate our image ![Image 8: \theta](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%2011'%3E%3C/svg%3E) degrees about some center ![Image 9: (c_{x}, c_{y})](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2046%2019'%3E%3C/svg%3E) coordinates at some _scale_ (i.e., smaller or larger).

We can then plug in values for ![Image 10: \alpha](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2010%207'%3E%3C/svg%3E) and ![Image 11: \beta](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2010%2014'%3E%3C/svg%3E):

![Image 12: \alpha = scale * cos \theta](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20115%2011'%3E%3C/svg%3E) and ![Image 13: \beta = scale * sin \theta](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20115%2015'%3E%3C/svg%3E)

That’s all fine and good for simple rotation — but it doesn’t take into account what happens if an image is cut off along the borders. How do we remedy this?

The answer is inside the `rotate_bound` function in [convenience.py of imutils](https://github.com/jrosebr1/imutils/blob/master/imutils/convenience.py#L41):

def rotate\_bound(image, angle):
    # grab the dimensions of the image and then determine the
    # center
    (h, w) = image.shape\[:2\]
    (cX, cY) = (w // 2, h // 2)

    # grab the rotation matrix (applying the negative of the
    # angle to rotate clockwise), then grab the sine and cosine
    # (i.e., the rotation components of the matrix)
    M = cv2.getRotationMatrix2D((cX, cY), -angle, 1.0)
    cos = np.abs(M\[0, 0\])
    sin = np.abs(M\[0, 1\])

    # compute the new bounding dimensions of the image
    nW = int((h \* sin) + (w \* cos))
    nH = int((h \* cos) + (w \* sin))

    # adjust the rotation matrix to take into account translation
    M\[0, 2\] += (nW / 2) - cX
    M\[1, 2\] += (nH / 2) - cY

    # perform the actual rotation and return the image
    return cv2.warpAffine(image, M, (nW, nH))

On **Line 41** we define our `rotate_bound` function.

This method accepts an input `image` and an `angle` to rotate it by.

We assume we’ll be rotating our image about its center _(x, y)_\-coordinates, so we determine these values on **lines 44 and 45**.

Given these coordinates, we can call `cv2.getRotationMatrix2D` to obtain our rotation matrix _M_ (**Line 50**).

However, to adjust for any image border cut off issues, we need to apply some manual calculations of our own.

We start by grabbing the cosine and sine values from our rotation matrix _M_ (**Lines 51 and 52**).

This enables us to compute the new width and height of the rotated image, ensuring no part of the image is cut off.

Once we know the new width and height, we can adjust for translation on **Lines 59 and 60** by modifying our rotation matrix once again.

Finally, `cv2.warpAffine` is called on **Line 63** to rotate the actual image using OpenCV while ensuring none of the image is cut off.

For some other interesting solutions (some better than others) to the rotation cut off problem when using OpenCV, be sure to refer to [this](http://stackoverflow.com/questions/11764575/python-2-7-3-opencv-2-4-after-rotation-window-doesnt-fit-image) StackOverflow thread and [this one](http://stackoverflow.com/questions/22041699/rotate-an-image-without-cropping-in-opencv-in-c) too.

### Fixing the rotated image “cut off” problem with OpenCV and Python

Let’s get back to my original problem of rotating oblong pills and how I used `.rotate_bound` to solve the issue (although back then I had not created the `imutils` Python package — it was simply a utility function in a helper file).

We’ll be using the following pill as our example image:

![Image 14: Figure 6: The example oblong pill we will be rotating with OpenCV.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20214%20214'%3E%3C/svg%3E)

**Figure 6:** The example oblong pill we will be rotating with OpenCV.

To start, open up a new file and name it `rotate_pills.py` . Then, insert the following code:

\# import the necessary packages
import numpy as np
import argparse
import imutils
import cv2

# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add\_argument("-i", "--image", required=True,
	help="path to the image file")
args = vars(ap.parse\_args())

**Lines 2-5** import our required Python packages. Again, make sure you have installed and/or upgraded the [imutils Python package](https://github.com/jrosebr1/imutils) before continuing.

We then parse our command line arguments on **Lines 8-11**. Just like in the example at the beginning of the blog post, we only need one switch: `--image` , the path to our input image.

Next, we load our pill image from disk and preprocess it by converting it to grayscale, blurring it, and detecting edges:

\# load the image from disk, convert it to grayscale, blur it,
# and apply edge detection to reveal the outline of the pill
image = cv2.imread(args\["image"\])
gray = cv2.cvtColor(image, cv2.COLOR\_BGR2GRAY)
gray = cv2.GaussianBlur(gray, (3, 3), 0)
edged = cv2.Canny(gray, 20, 100)

After executing these preprocessing functions our pill image now looks like this:

![Image 15: Figure 7: Detecting edges in the pill.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20400%20427'%3E%3C/svg%3E)

**Figure 7:** Detecting edges in the pill.

The outline of the pill is clearly visible, so let’s apply contour detection to find the outline of the pill:

\# find contours in the edge map
cnts = cv2.findContours(edged.copy(), cv2.RETR\_EXTERNAL,
	cv2.CHAIN\_APPROX\_SIMPLE)
cnts = imutils.grab\_contours(cnts)

We are now ready to extract the pill ROI from the image:

\# ensure at least one contour was found
if len(cnts) > 0:
	# grab the largest contour, then draw a mask for the pill
	c = max(cnts, key=cv2.contourArea)
	mask = np.zeros(gray.shape, dtype="uint8")
	cv2.drawContours(mask, \[c\], -1, 255, -1)

	# compute its bounding box of pill, then extract the ROI,
	# and apply the mask
	(x, y, w, h) = cv2.boundingRect(c)
	imageROI = image\[y:y + h, x:x + w\]
	maskROI = mask\[y:y + h, x:x + w\]
	imageROI = cv2.bitwise\_and(imageROI, imageROI,
		mask=maskROI)

First, we ensure that at least _one_ contour was found in the edge map (**Line 26**).

Provided we have at least one contour, we construct a `mask` for the largest contour region on **Lines 29 and 30**.

Our `mask` looks like this:

![Image 16: Figure 8: The mask representing the entire pill region in the image.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20400%20427'%3E%3C/svg%3E)

**Figure 8:** The mask representing the entire pill region in the image.

Given the contour region, we can compute the _(x, y)_\-coordinates of the bounding box of the region (**Line 34**).

Using both the bounding box and `mask` , we can extract the actual pill region ROI (**Lines 35-38**).

Now, let’s go ahead and apply both the `imutils.rotate` and `imutils.rotate_bound` functions to the `imageROI` , just like we did in the simple examples above:

	# loop over the rotation angles
	for angle in np.arange(0, 360, 15):
		rotated = imutils.rotate(imageROI, angle)
		cv2.imshow("Rotated (Problematic)", rotated)
		cv2.waitKey(0)

	# loop over the rotation angles again, this time ensure the
	# entire pill is still within the ROI after rotation
	for angle in np.arange(0, 360, 15):
		rotated = imutils.rotate\_bound(imageROI, angle)
		cv2.imshow("Rotated (Correct)", rotated)
		cv2.waitKey(0)

After downloading the source code to this tutorial using the **_“Downloads”_** section below, you can execute the following command to see the output:

$ python rotate\_pills.py --image images/pill\_01.png

The output of `imutils.rotate` will look like:

![Image 17: Figure 9: Incorrectly rotating an image with OpenCV causes parts of the image to be cut off.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20268%20152'%3E%3C/svg%3E)

**Figure 9:** Incorrectly rotating an image with OpenCV causes parts of the image to be cut off.

Notice how the pill is cut off during the rotation process — we need to _explicitly_ compute the new dimensions of the rotated image to ensure the borders are not cut off.

By using `imutils.rotate_bound` , we can ensure that no part of the image is cut off when using OpenCV:

![Image 18: Figure 10: By modifying OpenCV's rotation matrix we can resolve the issue and ensure the entire image is visible.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20525%20369'%3E%3C/svg%3E)

**Figure 10:** By modifying OpenCV’s rotation matrix we can resolve the issue and ensure the entire image is visible.

Using this function I was _finally_ able to finish my research for the winter break — but not before I felt quite embarrassed about my rookie mistake.

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

Summary
-------

In today’s blog post I discussed how image borders can be cut off when rotating images with OpenCV and `cv2.warpAffine` .

The fact that image borders can be cut off is _not_ a bug in OpenCV — in fact, it’s how `cv2.getRotationMatrix2D` and `cv2.warpAffine` are designed.

While it may seem frustrating and cumbersome to compute new image dimensions to ensure you don’t lose your borders, it’s actually a blessing in disguise.

OpenCV gives us _so much control_ that we can modify our rotation matrix to make it do _exactly_ what we want.

Of course, this requires us to know how our rotation matrix _M_ is formed and what each of its components represents (discussed earlier in this tutorial). Provided we understand this, the math falls out naturally.

To learn more about image processing and computer vision, be sure to take a look at the [PyImageSearch Gurus course](https://pyimagesearch.com/pyimagesearch-gurus/) where I discuss these topics in more detail.

_**Otherwise, I encourage you to enter your email address in the form below to be notified when future blog posts are published.**_

![Image 19](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!