Title: Are CNNs invariant to translation, rotation, and scaling? - PyImageSearch

URL Source: https://pyimagesearch.com/2021/05/14/are-cnns-invariant-to-translation-rotation-and-scaling/

Published Time: 2021-05-14T14:00:00+00:00

Markdown Content:
A common question I get asked is:

> _Are Convolutional Neural Networks invariant to changes in translation, rotation, and scaling? Is that why they are such powerful image classifiers?_

![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20251'%3E%3C/svg%3E)

To answer this question, we first need to discriminate between the _individual filters_ in the network along with the _final trained network_. Individual filters in a CNN are _not_ invariant to changes in how an image is rotated.

However, a CNN _as a whole_ can _learn_ filters that fire when a pattern is presented at a particular orientation. For example, consider **Figure 1**, adapted and inspired from [**_Deep Learning_ by Goodfellow et al. (2016)**](https://www.deeplearningbook.org/).

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20251'%3E%3C/svg%3E)

**Figure 1:** CNN as a whole learns filters that will fire when a pattern is presented at a particular orientation. On the _left_, the digit _9_ has been rotated _≈_ 10_◦_. This rotation is similar to node three, which has learned what the digit _9_ looks like when rotated in this manner. This node will have a higher activation than the other two nodes — the max pooling operation will detect this. On the _right_ we have a second example, only this time the _9_ has been rotated _≈ −_45_◦_, causing the first node to have the highest activation (Figure heavily inspired by [**Goodfellow et al. (2016)**](https://www.deeplearningbook.org/)).

Here, we see the digit _“9”_ (bottom) presented to the CNN along with a set of filters the CNN has learned (middle). Since there is a filter inside the CNN that has “learned” what a _“9”_ looks like, rotated by 10 degrees, it fires and emits a strong activation. This large activation is captured during the pooling stage and ultimately reported as the final classification.

The same is true for the second example (**Figure 1**, _right_). Here we see the _“9”_ rotated by _−_45 degrees, and since there is a filter in the CNN that has learned what a _“9”_ looks like when it is rotated by _−_45 degrees, the neuron activates and fires. Again, these filters themselves are _not_ rotation invariant — it’s just that the CNN has learned what a _“9”_ looks like under _small rotations_ that exist in the training set.

Unless your training data includes digits that are rotated across the full 360-degree spectrum, your CNN is _not_ truly rotation invariant.

The same can be said about scaling — the filters themselves are _not_ scale invariant, but it is highly likely that your CNN has learned a set of filters that _fire when patterns exist at varying scales_.

We can also “help” our CNNs to be scale invariant by presenting our example image to them at testing time under varying scales and crops, then averaging the results together.

Translation invariance; however, is something that a CNN excels at. Keep in mind that a filter slides from _left-to-right_ and _top-to-bottom_ across an input, and will activate when it comes across a particular edge-like region, corner, or color blob. During the pooling operation, this large response is found and thus “beats” all its neighbors by having a larger activation. Therefore, CNNs can be seen as “not caring” exactly where an activation fires, simply that it _does fire_ — and, in this way, we naturally handle translation inside a CNN.

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

**Summary**
-----------

In this tutorial, we answered the question, “are CCNs invariant to translation, rotation, and scaling?” We explored how CNNs recognize scaled and rotated objects through scaling and rotating training data and how CNNs robust to translation as they slide across an input.

![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Join the PyImageSearch Newsletter and Grab My FREE 17-page Resource Guide PDF

Enter your email address below to **join the PyImageSearch Newsletter** and **download my FREE 17-page Resource Guide PDF** on Computer Vision, OpenCV, and Deep Learning.

![Image 4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20240%20240'%3E%3C/svg%3E)

#### **About the Author**

Hi there, I’m Adrian Rosebrock, PhD. All too often I see developers, students, and researchers wasting their time, studying the wrong things, and generally struggling to get started with Computer Vision, Deep Learning, and OpenCV. I created this website to show you what I believe is the best possible way to get your start.