Title: Computing image “colorfulness” with OpenCV and Python - PyImageSearch

URL Source: https://pyimagesearch.com/2017/06/05/computing-image-colorfulness-with-opencv-and-python/

Published Time: 2017-06-05T14:00:59+00:00

Markdown Content:
![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20374'%3E%3C/svg%3E)

Today’s blog post is inspired by a question I received from a PyImageSearch reader on Twitter, [@makingyouthink](https://twitter.com/makinguthinkcom).

Paraphrasing the [tweets myself and @makingyouthink exchanged](https://twitter.com/makinguthinkcom/status/847979814472605698), the question was:

> Have you ever seen a Python implementation of Measuring colourfulness in natural images (Hasler and Süsstrunk, 2003)?
> 
> I would like to use it as an image/produce search engine. By giving each image a “colorfulness” amount, I can sort my images according to their color.

There are many practical uses for image colorfulness, including evaluating compression algorithms, assessing a given camera sensor module’s sensitivity to color, computing the “aesthetic qualities” of an image, or simply creating a bulk image visualization took to show a spectrum of images in a dataset arranged by colorfulness.

Today we are going to learn how to calculate the colorfulness of an image as described in Hasler and Süsstrunk’s 2003 paper, [_Measuring colorfulness in natural images_.](https://infoscience.epfl.ch/record/33994/files/HaslerS03.pdf) We will then implement our colorfulness metric using OpenCV and Python.

After implementing the colorfulness metric, we’ll sort a given dataset according to color and display the results using the [image montage tool](https://pyimagesearch.com/2017/05/29/montages-with-opencv/) that we created last week.

**To learn about computing image “colorfulness” with OpenCV, _just keep reading_.**

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

There are three core parts to today’s blog post.

First, we will walk through the colorfulness metric methodology described in in the [Hasler and Süsstrunk paper](https://infoscience.epfl.ch/record/33994/files/HaslerS03.pdf).

We’ll then implement the image colorfulness calculations in Python and OpenCV.

Finally, I’ll demonstrate how we can apply the colorfulness metric to a set of images and sort the images according to how “colorful” they are. We will make use of our handy [image montage routine](https://pyimagesearch.com/2017/05/29/montages-with-opencv/) for visualization.

To download the source code + example images to this blog post, be sure to use the _**“Downloads”**_ section below.

### Measuring colorfulness in an image

In their paper, Hasler and Süsstrunk first asked 20 non-expert participants to rate images on a 1-7 scale of colorfulness. This survey was conducted on a set of 84 images. The scale values were:

1.  Not colorful
2.  Slightly colorful
3.  Moderately colorful
4.  Averagely colorful
5.  Quite colorful
6.  Highly colorful
7.  Extremely colorful

In order to set a baseline, the authors provided the participants with 4 example images and their corresponding colorfulness value from 1-7.

Through a series of experimental calculations, they derived a simple metric that _correlated_ with the results of the viewers.

They found through these experiments that a simple [opponent color space representation](https://engineering.purdue.edu/~bouman/ece637/notes/pdf/Opponent.pdf) along with the mean and standard deviations of these values correlates to 95.3% of the survey data.

We now derive their image colorfulness metric:

![Image 3: rg = R - G](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2084%2014'%3E%3C/svg%3E)

![Image 4: yb = \frac{1}{2}(R + G) - B](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20138%2020'%3E%3C/svg%3E)

The above two equations show the opponent color space representation where R is Red, G is Green, and B is Blue. In the first equation, ![Image 5: rg](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2016%2010'%3E%3C/svg%3E) is the difference of the Red channel and the Green channel. In the second equation, ![Image 6: yb](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2015%2014'%3E%3C/svg%3E) is represents half of the sum of the Red and Green channels minus the Blue channel.

Next, the standard deviation (![Image 7: \sigma_{rgyb}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2032%2012'%3E%3C/svg%3E)) and mean (![Image 8: \mu_{rgyb}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2033%2012'%3E%3C/svg%3E)) are computed before calculating the final colorfulness metric, ![Image 9: C](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2011'%3E%3C/svg%3E).

![Image 10: \sigma_{rgyb} = \sqrt{\sigma_{rg}^2 + \sigma_{yb}^2}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20134%2031'%3E%3C/svg%3E)

![Image 11: \mu_{rgyb} = \sqrt{\mu_{rg}^2 + \mu_{yb}^2}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20136%2031'%3E%3C/svg%3E)

![Image 12: C = \sigma_{rgyb} + 0.3 * \mu_{rgyb}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20157%2017'%3E%3C/svg%3E)

As we’ll find out, this turns out to be an extremely efficient and practical way for computing image colorfulness.

In the next section, we will implement this algorithm with Python and OpenCV code.

### Implementing an image colorfulness metric in OpenCV

Now that we have a basic understanding of the colorfulness metric, let’s calculate it with OpenCV and NumPy.

In this section we will:

*   Import our necessary Python packages.
*   Parse our command line arguments.
*   Loop through all images in our dataset and compute the corresponding colorfulness metric.
*   Sort the images based on their colorfulness.
*   Display the “most colorful” and “least colorful” images in a [montage](https://pyimagesearch.com/2017/05/29/montages-with-opencv/).

To get started open up your favorite text editor or IDE, create a new file named `colorfulness.py` , and insert the following code:

\# import the necessary packages
from imutils import build\_montages
from imutils import paths
import numpy as np
import argparse
import imutils
import cv2

**Lines 2-7** import our required Python packages.

If you do not have `imutils` installed on your system (_v0.4.3_ as of this writing), then make sure you install/upgrade it via `pip`:

$ pip install --upgrade imutils

_**Note:** If you are using Python virtual environments (as all of my [OpenCV install tutorials](https://pyimagesearch.com/opencv-tutorials-resources-guides/) do), make sure you use the `workon` command to access your virtual environment first and then install/upgrade_ `imutils` _._

Next, we will define a new function, `image_colorfullness`:

def image\_colorfulness(image):
	# split the image into its respective RGB components
	(B, G, R) = cv2.split(image.astype("float"))

	# compute rg = R - G
	rg = np.absolute(R - G)

	# compute yb = 0.5 \* (R + G) - B
	yb = np.absolute(0.5 \* (R + G) - B)

	# compute the mean and standard deviation of both \`rg\` and \`yb\`
	(rbMean, rbStd) = (np.mean(rg), np.std(rg))
	(ybMean, ybStd) = (np.mean(yb), np.std(yb))

	# combine the mean and standard deviations
	stdRoot = np.sqrt((rbStd \*\* 2) + (ybStd \*\* 2))
	meanRoot = np.sqrt((rbMean \*\* 2) + (ybMean \*\* 2))

	# derive the "colorfulness" metric and return it
	return stdRoot + (0.3 \* meanRoot)

**Line 9** defines the `image_colorfulness` function, which takes an `image` as the only argument and returns the colorfulness metric as described in the section above.

_**Note: Line 11**, **Line 14**, and **Line 17** make use of color spaces which are beyond the scope of this blog post. If you are interested in learning more about color spaces, be sure to refer to_ [Practical Python and OpenCV](https://pyimagesearch.com/practical-python-opencv/) _and the [PyImageSearch Gurus course](https://pyimagesearch.com/pyimagesearch-gurus/)._

To break the image into it’s Red, Green, and Blue (RGB) channels we make a call to `cv2.split` on **Line 11**. The function returns a tuple in BGR order as this is how images are represented in OpenCV.

Next we use a very simple opponent color space.

As in the referenced paper, we compute the Red-Green opponent, `rg`, on **Line 14**. This is a simple difference of the Red channel minus the Blue channel.

Similarly, we compute the Yellow-Blue opponent on **Line 17**. In this calculation, we take half of the Red+Green channel sum and then subtract the Blue channel. This produces our desired opponent, `yb`.

From there, on **Lines 20 and 21** we compute the mean and standard deviation of both `rg` and `yb`, and store them in respective tuples.

Next, we combine the `rbStd` (Red-Blue standard deviation) with the `ybStd` (Yellow-Blue standard deviation) on **Line 24**. We add the square of each and then take the square root, storing it as `stdRoot`.

Similarly, we combine the `rbMean` with the `ybMean` by squaring each, adding them, and taking the square root on **Line 25**. We store this value as `meanRoot`.

The last step of computing image colorfulness is to add `stdRoot` and 1/3 `meanRoot` followed by returning the value to the calling function.

Now that our image `image_colorfulness` metric is defined, we can parse our command line arguments:

\# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add\_argument("-i", "--images", required=True,
	help="path to input directory of images")
args = vars(ap.parse\_args())

We only need one command line argument here, `--images` , which is the path to a directory of images residing on your machine.

Now let’s loop through each image in the dataset and compute the corresponding colorfulness metric:

\# initialize the results list
print("\[INFO\] computing colorfulness metric for dataset...")
results = \[\]

# loop over the image paths
for imagePath in paths.list\_images(args\["images"\]):
	# load the image, resize it (to speed up computation), and
	# compute the colorfulness metric for the image
	image = cv2.imread(imagePath)
	image = imutils.resize(image, width=250)
	C = image\_colorfulness(image)

	# display the colorfulness score on the image
	cv2.putText(image, "{:.2f}".format(C), (40, 40), 
		cv2.FONT\_HERSHEY\_SIMPLEX, 1.4, (0, 255, 0), 3)

	# add the image and colorfulness metric to the results list
	results.append((image, C))

**Line 38** initializes a list, `results` , which will hold a 2-tuple containing the image path and the corresponding colorfulness of the image.

We begin our loop through our images in our dataset specified by our command line argument, `--images` on **Line 41**.

In the loop, we first load the image on **Line 44**, then we resize the `image` to a `width=250` pixels on **Line 45**, _maintaining the aspect ratio_.

Our `image_colorfulness` function call is made on **Line 46** where we provide the only argument, `image`, storing the corresponding colorfulness metric in `C`.

On **Lines 49 and 50**, we draw the colorfulness metric on the image using `cv2.putText`. To read more about the parameters to this function, see the OpenCV Documentation ([2.4](http://docs.opencv.org/2.4/modules/core/doc/drawing_functions.html#puttext), [3.0](http://docs.opencv.org/3.0-beta/modules/imgproc/doc/drawing_functions.html#puttext)).

On the last line of the `for` loop, we append the tuple, `(imagePath, C)` to the `results` list (**Line 53**).

_**Note:** Typically, you would not want to store each image in memory for a large dataset. We do this here for convenience. In practice you would load the image, compute the colorfulness metric, and then maintain a list of the image ID/filename and corresponding colorfulness metric. This is a much more efficient approach; however, for the sake of this example we are going to store the images in memory so we can easily build our montage of “most colorful” and “least colorful” images later in the tutorial._

_**At this point, we have answered our PyImageSearch reader’s question. The colorfulness metric has been calculated for all images.**_

If you’re using this for an image search engine as @makingyouthinkcom is, you probably want to display your results.

And that is exactly what we will do next, where we will:

*   Sort the images according to their corresponding colorfulness metric.
*   Determine the 25 most colorful and 25 least colorful images.
*   Display our results in a montage.

Let’s go ahead and tackle these three tasks now:

\# sort the results with more colorful images at the front of the
# list, then build the lists of the \*most colorful\* and \*least
# colorful\* images
print("\[INFO\] displaying results...")
results = sorted(results, key=lambda x: x\[1\], reverse=True)
mostColor = \[r\[0\] for r in results\[:25\]\]
leastColor = \[r\[0\] for r in results\[-25:\]\]\[::-1\]

On **Line 59** we sort the `results` in reverse order (according to their colorfulness metric) making use of [Python Lambda Expressions](https://docs.python.org/3/tutorial/controlflow.html#lambda-expressions).

Then on **Line 60**, we store the 25 most colorful images into a list, `mostColor` .

Similarly, on **Line 61**, we load the least colorful images which are the last 25 images in our results list. We reverse this list so that the images are displayed in ascending order. We store these images as `leastColor` .

Now, we can visualize the `mostColor` and `leastColor` images using the `build_montages` function we learned about [last week](https://pyimagesearch.com/2017/05/29/montages-with-opencv/).

\# construct the montages for the two sets of images
mostColorMontage = build\_montages(mostColor, (128, 128), (5, 5))
leastColorMontage = build\_montages(leastColor, (128, 128), (5, 5))

A most-colorful and least-colorful montage are each built on **Lines 64 and 65**. Here we indicate that all images in the montage will be resized to 128 x 128 and there will be 5 columns by 5 rows of images.

Now that we have assembled the montages, we will display each on the screen.

\# display the images
cv2.imshow("Most Colorful", mostColorMontage\[0\])
cv2.imshow("Least Colorful", leastColorMontage\[0\])
cv2.waitKey(0)

On **Lines 68 and 69** we display each montage in a separate window.

The `cv2.waitKey` call on **Line 70** pauses execution of our script until we select a currently active window. When a key is pressed, the windows close and the script exits.

### Image colorfulness results

Now let’s put this script to work and see the results. Today we will use a sample (1,000 images) of the popular [UKBench dataset](http://vis.uky.edu/~stewe/ukbench/), a collection of images containing everyday objects.

Our goal is to sort the images by most colorful and least colorful.

To run the script, fire up a terminal and execute the following command:

$ python colorfulness.py --images ukbench\_sample

![Image 13](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20374'%3E%3C/svg%3E)

**Figure 1:** _(Left)_ Least colorful images. _(Right)_ Most colorful images.

Notice how our image colorfulness metric has done a good job separating non-colorful images _(left)_ that are essentially black and white from “colorful” images that are vibrant _(right)_.

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

Summary
-------

In today’s blog post we learned how to compute the “colorfulness” of an image using the approach detailed by Hasler and Süsstrunk’s in their 2003 paper, _Measuring colorfulness in nature images._

Their method is based on the mean and standard deviation of pixel intensities values in an opponent color space. This metric was derived by examining _correlations_ between experimental metrics and the colorfulness assigned to images by participants in their study.

We then implemented the image colorfulness metric and applied it to the UKBench dataset. As our results demonstrated, the Hasler and Süsstrunk method is a quick and easy way to quantify the colorfulness contents of an image.

Have fun using this method to experiment with the image colorfulness in your own datasets!

_**And before you go, be sure to enter your email address in the form below to be notified when new tutorials are published here on the PyImageSearch blog.**_

![Image 14](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!