Title: Image Difference with OpenCV and Python - PyImageSearch

URL Source: https://pyimagesearch.com/2017/06/19/image-difference-with-opencv-and-python/

Published Time: 2017-06-19T14:00:29+00:00

Markdown Content:
Last updated on July 7, 2021.

![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20412'%3E%3C/svg%3E)

In a previous PyImageSearch blog post, I detailed [how to compare two images with Python](https://pyimagesearch.com/2014/09/15/python-compare-two-images/) using the Structural Similarity Index (SSIM).

Using this method, we were able to easily determine if two images were identical or had differences due to slight image manipulations, compression artifacts, or purposeful tampering.

Today we are going to extend the SSIM approach so that we can visualize the differences between images using OpenCV and Python. _Specifically, we’ll be drawing bounding boxes around regions in the two input images that differ._

**To learn more about computing and visualizing image differences with Python and OpenCV, _just keep reading_.**

*   **Update July 2021:** Added a section on the methods to compare images for differences and resources for additional reading about how to use siamese networks.

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

In order to compute the difference between two images we’ll be utilizing the Structural Similarity Index, first introduced by Wang et al. in their 2004 paper, [_Image Quality Assessment: From Error Visibility to Structural Similarity_](https://ece.uwaterloo.ca/~z70wang/publications/ssim.pdf). This method is already implemented in the [scikit-image](http://scikit-image.org/docs/stable/api/skimage.measure.html#skimage.measure.compare_ssim) library for image processing.

The trick is to learn how we can determine _exactly where_, in terms of _(x, y)_\-coordinate location, the image differences are.

To accomplish this, we’ll first need to make sure our system has Python, OpenCV, scikit-image, and imutils.

You can learn how to configure and install Python and OpenCV on your system using [one of my OpenCV install tutorials](https://pyimagesearch.com/opencv-tutorials-resources-guides/).

If you don’t already have `scikit-image` installed/upgraded, upgrade via:

$ pip install --upgrade scikit-image

While you’re at it, go ahead and install/upgrade `imutils` as well:

$ pip install --upgrade imutils

Now that our system is ready with the prerequisites, let’s continue.

### Computing image difference

Can you spot the difference between these two images?

![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20222'%3E%3C/svg%3E)

**Figure 1:** Manually inspecting the difference between two input images ([source](https://www.bedbathandbeyond.com/store/static/creditcard)).

If you take a second to study the two credit cards, you’ll notice that the MasterCard logo is present on the _left_ image but has been Photoshopped out from the _right_ image.

You may have noticed this difference immediately, or it may have taken you a few seconds. Either way, this demonstrates an important aspect of comparing image differences — _sometimes image differences are subtle —_ so subtle that the naked eye struggles to immediately comprehend the difference (we’ll see an example of such an image later in this blog post).

**So why is computing image differences so important?**

One example is _phishing_. Attackers can manipulate images ever-so-slightly to trick unsuspecting users who don’t validate the URL into thinking they are logging into their banking website — only to later find out that it was a scam.

Comparing logos and known User Interface (UI) elements on a webpage to an existing dataset could help reduce phishing attacks (a big thanks to Chris Cleveland for passing along _[PhishZoo: Detecting Phishing Websites By Looking at Them](http://www1.icsi.berkeley.edu/~sadia/papers/phishzoo-icsc_final.pdf)_ as an example of applying computer vision to prevent phishing).

Developing a phishing detection system is obviously much more complicated than simple image differences, but we can still apply these techniques to determine if a given image has been manipulated.

Now, let’s compute the difference between two images, and view the differences side by side using OpenCV, scikit-image, and Python.

Open up a new file and name it `image_diff.py` , and insert the following code:

\# import the necessary packages
from skimage.metrics import structural\_similarity as compare\_ssim
import argparse
import imutils
import cv2

# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add\_argument("-f", "--first", required=True,
	help="first input image")
ap.add\_argument("-s", "--second", required=True,
	help="second")
args = vars(ap.parse\_args())

**Lines 2-5** show our imports. We’ll be using `compare_ssim` (from scikit-image), `argparse` , `imutils` , and `cv2` (OpenCV).

We establish two command line arguments, `--first` and `--second` , which are the paths to the two respective input images we wish to compare (**Lines 8-13**).

Next we’ll load each image from disk and convert them to grayscale:

\# load the two input images
imageA = cv2.imread(args\["first"\])
imageB = cv2.imread(args\["second"\])

# convert the images to grayscale
grayA = cv2.cvtColor(imageA, cv2.COLOR\_BGR2GRAY)
grayB = cv2.cvtColor(imageB, cv2.COLOR\_BGR2GRAY)

We load our first and second images, `--first` and `--second` , on **Lines 16 and 17**, storing them as `imageA` and `imageB` , respectively.

![Image 4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20222'%3E%3C/svg%3E)

**Figure 2:** Our two input images that we are going to apply image difference to.

Then we convert each to grayscale on **Lines 20 and 21**.

![Image 5](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20226'%3E%3C/svg%3E)

**Figure 3:** Converting the two input images to grayscale.

Next, let’s compute the Structural Similarity Index (SSIM) between our two grayscale images.

\# compute the Structural Similarity Index (SSIM) between the two
# images, ensuring that the difference image is returned
(score, diff) = compare\_ssim(grayA, grayB, full=True)
diff = (diff \* 255).astype("uint8")
print("SSIM: {}".format(score))

Using the `compare_ssim` function from scikit-image, we calculate a `score` and difference image, `diff` (**Line 25**).

The `score` represents the structural similarity index between the two input images. This value can fall into the range _\[-1, 1\]_ with a value of one being a “perfect match”.

The `diff` image contains the actual _image differences_ between the two input images that we wish to visualize. The difference image is currently represented as a floating point data type in the range _\[0, 1\]_ so we first convert the array to 8-bit unsigned integers in the range _\[0, 255\]_ (**Line 26**) before we can further process it using OpenCV.

Now, let’s find the contours so that we can place rectangles around the regions identified as “different”:

\# threshold the difference image, followed by finding contours to
# obtain the regions of the two input images that differ
thresh = cv2.threshold(diff, 0, 255,
	cv2.THRESH\_BINARY\_INV | cv2.THRESH\_OTSU)\[1\]
cnts = cv2.findContours(thresh.copy(), cv2.RETR\_EXTERNAL,
	cv2.CHAIN\_APPROX\_SIMPLE)
cnts = imutils.grab\_contours(cnts)

On **Lines 31 and 32** we threshold our `diff` image using both `cv2.THRESH_BINARY_INV` and `cv2.THRESH_OTSU` — both of these settings are applied at the same time using the vertical bar ‘or’ symbol, `|` . For details on Otsu’s bimodal thresholding setting, see [this OpenCV documentation](http://docs.opencv.org/trunk/d7/d4d/tutorial_py_thresholding.html).

Subsequently we find the contours of `thresh` on **Lines 33-35**. The ternary operator on **Line 35** simply accommodates difference between the [cv2.findContours return signature](https://pyimagesearch.com/2015/08/10/checking-your-opencv-version-using-python/) in various versions of OpenCV.

The image in **Figure 4** below clearly reveals the ROIs of the image that have been manipulated:

![Image 6](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20373'%3E%3C/svg%3E)

**Figure 4:** Using thresholding to highlight the image differences using OpenCV and Python.

Now that we have the contours stored in a list, let’s draw rectangles around the different regions on each image:

\# loop over the contours
for c in cnts:
	# compute the bounding box of the contour and then draw the
	# bounding box on both input images to represent where the two
	# images differ
	(x, y, w, h) = cv2.boundingRect(c)
	cv2.rectangle(imageA, (x, y), (x + w, y + h), (0, 0, 255), 2)
	cv2.rectangle(imageB, (x, y), (x + w, y + h), (0, 0, 255), 2)

# show the output images
cv2.imshow("Original", imageA)
cv2.imshow("Modified", imageB)
cv2.imshow("Diff", diff)
cv2.imshow("Thresh", thresh)
cv2.waitKey(0)

Beginning on **Line 38**, we loop over our contours, `cnts` . First, we compute the bounding box around the contour using the `cv2.boundingRect` function. We store relevant _(x, y__)_\-coordinates as `x` and `y` as well as the width/height of the rectangle as `w` and `h` .

Then we use the values to draw a red rectangle on each image with `cv2.rectangle` (**Lines 43 and 44)**.

Finally, we show the comparison images with boxes around differences, the difference image, and the thresholded image (**Lines 47-50**).

We make a call to `cv2.waitKey` on **Line 50** which makes the program wait until a key is pressed (at which point the script will exit).

![Image 7](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20411'%3E%3C/svg%3E)

**Figure 5:** Visualizing image differences using Python and OpenCV.

Next, let’s run the script and visualize a few more image differences.

### Visualizing image differences

Using this script and the following command, we can quickly and easily highlight differences between two images:

$ python image\_diff.py --first images/original\_02.png 
	--second images/modified\_02.png

As you can see in **Figure 6**, the security chip and name of the account holder have both been removed:

![Image 8](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20412'%3E%3C/svg%3E)

**Figure 6:** Comparing and visualizing image differences using computer vision ([source](https://www.psecu.com/visa/)).

Let’s try another example of computing image differences, this time of a check written by President Gerald R. Ford ([source](https://commons.wikimedia.org/wiki/File:FORD,_Gerald_\(signed_check\).jpg)).

By running the command below and supplying the relevant images, we can see that the differences here are more subtle:

$ python image\_diff.py --first images/original\_03.png 
	--second images/modified\_03.png

![Image 9](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20305'%3E%3C/svg%3E)

**Figure 7:** Computing image differences and highlighting the regions that are different.

Notice the following changes in **Figure 7**:

*   Betty Ford’s name is removed.
*   The check number is removed.
*   The symbol next to the date is removed.
*   The last name is removed.

On a complex image like a check it is often difficult to find _all_ the differences with the naked eye. Luckily for us, we can now easily compute the differences and visualize the results with this handy script made with Python, OpenCV, and scikit-image.

### **Other methods to compare images for differences**

This tutorial covered how to use the Structural Similarity Index (SSIM) to compare two images and spot differences between the two.

SSIM is a traditional computer vision approach to image comparison; however, there are other image difference algorithms that can be utilized, specifically deep learning ones.

To start, one could apply [transfer learning](https://pyimagesearch.com/2019/05/20/transfer-learning-with-keras-and-deep-learning/) by:

1.  Selecting a pre-trained CNN (ResNet, VGGNet, etc.) on a large, diverse image dataset (e.g., ImageNet)
2.  Removing the fully connected layer head from the network
3.  Passing all images in your dataset through the CNN and extracting the features from the final layer
4.  Computing a [k-Nearest Neighbor](https://pyimagesearch.com/2016/08/08/k-nn-classifier-for-image-classification/) comparison between the features and then finding the most similar images to the input query

However, keep in mind that the k-NN approach does not actually “learn” any underlying patterns/similarity scores in images in your dataset.

To actually _learn_ image differences and make robust comparisons between them, you should utilize **siamese networks**.

The following tutorials will teach you about siamese networks:

1.  [_Building image pairs for siamese networks with Python_](https://pyimagesearch.com/2020/11/23/building-image-pairs-for-siamese-networks-with-python/)
2.  [_Siamese networks with Keras, TensorFlow, and Deep Learning_](https://pyimagesearch.com/2020/11/30/siamese-networks-with-keras-tensorflow-and-deep-learning/)
3.  [_Comparing images for similarity using siamese networks, Keras, and TensorFlow_](https://pyimagesearch.com/2020/12/07/comparing-images-for-similarity-using-siamese-networks-keras-and-tensorflow/)
4.  [_Contrastive Loss for Siamese Networks with Keras and TensorFlow_](https://pyimagesearch.com/2021/01/18/contrastive-loss-for-siamese-networks-with-keras-and-tensorflow/)

Additionally, siamese networks are covered in detail inside [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/).

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

Summary
-------

In today’s blog post, we learned how to compute image differences using OpenCV, Python, and scikit-image’s Structural Similarity Index (SSIM). Based on the image difference we also learned how to mark and visualize the different regions in two images.

To learn more about SSIM, be sure to [refer to this post](https://pyimagesearch.com/2014/09/15/python-compare-two-images/) and the [scikit-image documentation](http://scikit-image.org/docs/dev/api/skimage.measure.html#skimage.measure.compare_ssim).

I hope you enjoyed today’s blog post!

**And before you go, be sure to enter your email address in the form below to be notified when future PyImageSearch blog posts are published!**

![Image 10](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!