Title: Color Quantization with OpenCV using K-Means Clustering - PyImageSearch

URL Source: https://pyimagesearch.com/2014/07/07/color-quantization-opencv-using-k-means-clustering/

Published Time: 2014-07-07T14:00:37+00:00

Markdown Content:
[![Image 1: Figure 1: Applying color quantization with OpenCV using k-means clustering.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20540'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2014/07/quant_kmeans_worldcup.jpg)

Ever see _[A Scanner Darkly](http://www.imdb.com/title/tt0405296/)?_

The movie was shot digitally, but then an animated feel was given to it in the post-processing steps — but it was a painstaking process. For each frame in the movie, animators traced over the original footage, frame by frame.

Just take a second to consider the number of frames in a movie…and that each one of those frames had to be traced by hand.

Quite the undertaking, indeed.

So what if there was a way to create _A Scanner Darkly_ animation effect using computer vision?

Is that possible?

You bet.

In this blog post I’ll show you how to use k-means clustering and color quantization to create _A Scanner Darkly_ type effect in images.

To find out how I do it, read on.

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

**OpenCV and Python versions:**  
This example will run on **Python 2.7/Python 3.4+** and **OpenCV 2.4.X/OpenCV 3.0+**.

So, what is color quantization?
-------------------------------

Color quantization is the process of reducing the number of distinct colors in an image.

Normally, the intent is to preserve the color appearance of the image as much as possible, while reducing the number of colors, whether for memory limitations or compression.

In my own work, I find that color quantization is best used when building Content-Based Image Retrieval (CBIR) systems.

If you’re unfamiliar with the term, CBIR is just a fancy academic way of saying “image search engine”.

Take a second to think about color quantization in the context of CBIR, though.

Any given 24-bit RGB image has _256 x 256 x 256_ possible colors. And sure, we can build standard [color histograms](https://pyimagesearch.com/2014/01/22/clever-girl-a-guide-to-utilizing-color-histograms-for-computer-vision-and-image-search-engines/ "Clever Girl: A Guide to Utilizing Color Histograms for Computer Vision and Image Search Engines") based on these intensity values.

But another approach is to explicitly quantize the image and _reduce_ the number of colors to say, 16 or 64. This creates a substantially smaller space and (ideally) less noise and variance.

In practice, you can use this technique to construct more rigid color histograms.

In fact, the famous [QBIC CBIR](http://courses.cs.washington.edu/courses/cse576/07sp/notes/CBIR_white.pdf) system (one of the original CBIR systems that demonstrated image search engines were possible) utilized quantized color histograms in the quadratic distance to compute similarity.

Now that we have an understanding of what color quantization is, let’s explore how we can utilize it to create _A Scanner Darkly_ type effect in images.

Let’s get our hands dirty.

Open up a new file, call it `quant.py`, and start coding:

\# import the necessary packages
from sklearn.cluster import MiniBatchKMeans
import numpy as np
import argparse
import cv2

# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add\_argument("-i", "--image", required = True, help = "Path to the image")
ap.add\_argument("-c", "--clusters", required = True, type = int,
	help = "# of clusters")
args = vars(ap.parse\_args())

The first thing we’ll do is import our necessary packages on **Lines 2-5**. We’ll use NumPy for numerical processing, `arparse` for parsing command line arguments, and `cv2` for our OpenCV bindings. Our k-means implementation will be handled by scikit-learn; specifically, the [`MiniBatchKMeans`](http://scikit-learn.org/stable/modules/generated/sklearn.cluster.MiniBatchKMeans.html) class.

You’ll find that `MiniBatchKMeans` is substantially faster than normal K-Means, although the centroids may not be as stable.

This is because `MiniBatchKMeans` operates on small “batches” of the dataset, whereas K-Means operates on the population of the dataset, thus making the mean calculation of each centroid, as well as the centroid update loop, much slower.

In general, I normally like to start with `MiniBatchKMeans` and if (and only if) my results are poor do I switch over to normal K-Means.

**Lines 7-12** then handle parsing our command line arguments. We’ll need two switches: `--image`, which is the path to the image we want to apply color quantization to, and `--clusters`, which is the number of colors that our output image is going to have.

Now the real interesting code starts:

\# load the image and grab its width and height
image = cv2.imread(args\["image"\])
(h, w) = image.shape\[:2\]

# convert the image from the RGB color space to the L\*a\*b\*
# color space -- since we will be clustering using k-means
# which is based on the euclidean distance, we'll use the
# L\*a\*b\* color space where the euclidean distance implies
# perceptual meaning
image = cv2.cvtColor(image, cv2.COLOR\_BGR2LAB)

# reshape the image into a feature vector so that k-means
# can be applied
image = image.reshape((image.shape\[0\] \* image.shape\[1\], 3))

# apply k-means using the specified number of clusters and
# then create the quantized image based on the predictions
clt = MiniBatchKMeans(n\_clusters = args\["clusters"\])
labels = clt.fit\_predict(image)
quant = clt.cluster\_centers\_.astype("uint8")\[labels\]

# reshape the feature vectors to images
quant = quant.reshape((h, w, 3))
image = image.reshape((h, w, 3))

# convert from L\*a\*b\* to RGB
quant = cv2.cvtColor(quant, cv2.COLOR\_LAB2BGR)
image = cv2.cvtColor(image, cv2.COLOR\_LAB2BGR)

# display the images and wait for a keypress
cv2.imshow("image", np.hstack(\[image, quant\]))
cv2.waitKey(0)

First, we load our image off disk on **Line 15** and grab its height and width, respectively, on **Line 16**.

**Line 23** handles converting our image from the RGB color space to the L\*a\*b\* color space.

Why are we bothering doing this conversion?

Because in the L\*a\*b\* color space the euclidean distance between colors has actual perceptual meaning — this is not the case for the RGB color space.

Given that k-means clustering also assumes a euclidean space, we’re better off using L\*a\*b\* rather than RGB.

In order to cluster our pixel intensities, we need to reshape our image on **Line 27**. This line of code simply takes a _(M, N, 3)_ image, (_M x N_ pixels, with three components per pixel) and reshapes it into a _(M x N, 3)_ feature vector. This reshaping is important since k-means assumes a two dimensional array, rather than a three dimensional image.

From there, we can apply our actual mini-batch K-Means clustering on **Lines 31-33**.

**Line 31** handles instantiating our `MiniBatchKMeans` class using the number of clusters we specified in command line argument, whereas **Line 32** performs the actual clustering.

Besides the actual clustering, **Line 32** handles something extremely important — “predicting” what _quantized color_ each pixel _in the original image_ is going to be. This prediction is handled by determining which centroid the input pixel is closest to.

From there, we can take these predicted labels and create our quantized image on **Line 33** using some fancy NumPy indexing. All this line is doing is using the predicted labels to lookup the L\*a\*b\* color in the centroids array.

**Lines 36 and 37** then handle reshaping our _(M x N, 3)_ feature vector back into a _(M, N, 3)_ dimensional image, followed by converting our images from the L\*a\*b\* color space back to RGB.

Finally, **Lines 44 and 45** display our original and quantized image.

Now that the coding is done, let’s take a look at the results.

Creating _A Scanner Darkly_ Effect using Computer Vision
--------------------------------------------------------

Since it’s World Cup season, let’s start with a soccer image.

[![Image 3: Figure 1: Applying color quantization with OpenCV using k-means clustering.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20540'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2014/07/quant_kmeans_worldcup.jpg)

**Figure 1:** Applying color quantization with OpenCV using k-means clustering.

Here we can see our original image on the _left_ and our quantized image on the _right_.

Clearly we can see that when using only _k=4_ colors that we have much lost of the color detail of the original image, although an attempt is made to model the original color space of the image — the grass is still green, the soccer ball still has white in it, and the sky still has a tinge of blue.

But as we increase the _k=8_ and _k=16_ colors we can definitely start to see improved results.

It’s really interesting to note that it only takes 16 colors to create a good representation of the original image, and thus our _A Scanner Darkly_ effect.

Let’s try another image:

[![Image 4: Figure 2: Apply color quantization using OpenCV to a nature scene.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20633'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2014/07/quant_kmeans_nature.jpg)

**Figure 2:** Apply color quantization using OpenCV to a nature scene.

Again, the original nature scene image is on the _left_ and the quantized output on the _right_.

Just like the World Cup image, notice that as the number of quantized clusters increase, we are able to better mimic the original color space.

[![Image 5: Color Quantization with OpenCV](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20170'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2014/07/quant_kmeans_jpcrew_24_colors.jpg)

**Figure 3:** Applying color quantization using OpenCV to Jurassic Park.

Finally, take a look at **Figure 3** to see a screen capture from _Jurassic Park_. Notice how the number of colors have been reduced — this is explicitly evident in Hammond’s white shirt.

There is an obvious tradeoff between the number of clusters and the quality of the quantized image.

The first tradeoff is that as the number of clusters increases, so does the amount of time it takes to perform the clustering.

The second tradeoff is that as the number of clusters increases, so does the amount of memory it takes to store the output image; however, in both cases, the memory footprint will still be smaller than the original image since you are working with a substantially smaller color palette.

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

Summary
-------

In this blog post I showed you how to perform color quantization using OpenCV and k-means clustering to create _A Scanner Darkly_ type of effect in images.

While color quantization does not perfectly mimic the movie effect, it does demonstrate that by reducing the number of colors in an image, you can create a more posterized, animated feel to the image.

Of course, color quantization has more practical applications than just visual appeal.

Color quantization is commonly used in systems where memory is limited or when compression is required.

In my own personal work, I find that color quantization is best used when building CBIR systems. In fact, [QBIC](http://courses.cs.washington.edu/courses/cse576/07sp/notes/CBIR_white.pdf), one of the seminal image search engines, demonstrated that by using quantized color histograms and the quadratic distance, that image search engines were indeed possible.

So take a second to use the form below to download the code. And then create some quantized images of your own and send them over to me. I’m looking forward to seeing your results!

![Image 6](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!