Title: How To Describe and Quantify an Image Using Feature Vectors

URL Source: https://pyimagesearch.com/2014/03/03/charizard-explains-describe-quantify-image-using-feature-vectors/

Published Time: 2014-03-03T15:00:59+00:00

Markdown Content:
If you haven’t noticed, the term “feature vector” is used quite often in this blog. And while we’ve seen it a lot, I wanted to dedicate an entire post to defining what exactly a feature vector is.

What is an Image Feature Vector?
--------------------------------

> **Image Feature Vector:** An abstraction of an image used to characterize and numerically quantify the contents of an image. Normally real, integer, or binary valued. Simply put, a feature vector is a list of numbers used to represent an image.

As you know, the first step of building any image search engine is to define what type of image descriptor you are going to use. Are you trying to characterize the color of an image and extracting color features? The texture? Or the shape of an object in an image?

Once you have selected an image descriptor, you need to apply your image descriptor to an image. This image descriptor handles the logic necessary to quantify an image and represent it as a list of numbers.

The output of your image descriptor is a _feature vector_: the list of numbers used to characterize your image. Make sense?

Two Questions to Ask Yourself
-----------------------------

Here is a general template you can follow when defining your image descriptors and expected output. This template will help ensure you always know what you are describing as well as what the output of your descriptor represents. In order to apply this template, you simply need to ask yourself two questions:

1.  **What image descriptor am I using?**
2.  **What is the expected output of my image descriptor?**

Let’s make this explanation a little more concrete and go through some examples.

If you’re a frequent reader of this blog, you know that I have an obsession with both Jurassic Park and Lord of the Rings. Let’s introduce my third obsession: Pokemon. Below is our example image that we will use throughout this blog post — a Charizard.

[![Image 1: Figure 1: Our example image - a Charizard.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20254%20198'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2014/02/charizard.png)

**Figure 1:** Our example image – a Charizard.

Now, fire up a Python shell and follow along:

\>>> import cv2
>>> image = cv2.imread("charizard.png")
>>> image.shape
(198, 254, 3)

Here we are just importing `cv2`, our Python package that interfaces with OpenCV. We then load our Charizard image off of disk and examine the dimensions of the image.

Looking at the dimensions of the image we see that it has a height of 198 pixels, a width of 254 pixels, and 3 channels — one for each of the Red, Green, and Blue channels, respectively.

Raw Pixel Feature Vectors
-------------------------

Arguably, the the most basic color feature vector you can use is the raw pixel intensities themselves. While we don’t normally use this representation in image search engines, it is sometimes used in machine learning and classification contexts, and is worth mentioning.

Let’s ask ourselves the two questions mentioned in the template above:

1.  **What image descriptor am I using?** I am using a raw pixel descriptor.
2.  **What is the excepted output of my descriptor?** A list of numbers corresponding to the raw RGB pixel intensities of my image.

Since an image is represented as NumPy array, it’s quite simple to compute the raw pixel representation of an image:

\>>> raw = image.flatten()
>>> raw.shape
(150876,)
>>> raw
array(\[255, 255, 255, ..., 255, 255, 255\], dtype=uint8)

We can now see that our image has been “flattened” via NumPy’s `flatten` method. The Red, Green, and Blue components of the image have been flattened into a single list (rather than a multi-dimensional array) to represent the image. Our flattened array has a shape of _150,876_ because there exists _198 x 254 = 50,292_ pixels in the image with 3 values per pixel, thus _50,292 x 3 = 150,876_.

Color Mean
----------

Our previous example wasn’t very interesting.

What if we wanted to quantify the color of our Charizard, without having to use the entire image of raw pixel intensities?

A simple method to quantify the color of an image is to compute the mean of each of the color channels.

Again, let’s fill out our template:

1.  **What image descriptor am I using?** A color mean descriptor.
2.  **What is the expected output of my image descriptor?** The mean value of each channel of the image.

And now let’s look at the code:

\>>> means = cv2.mean(image)
>>> means
(181.12238527002307, 199.18315040165433, 206.514296508391, 0.0)

We can compute the mean of each of the color channels by using the `cv2.mean` method. This method returns a tuple with four values, our color features. The first value is the mean of the blue channel, the second value the mean of the green channel, and the third value is the mean of red channel. Remember, OpenCV stores RGB images as a NumPy array, but in reverse order. We actually read them backwards in BGR order, hence the blue value comes first, then the green, and finally the red.

The fourth value can be ignored and exists only so that OpenCV’s built-in `Scalar` class can be used internally. This value can be ignored as such:

\>>> means = means\[:3\]
>>> means
(181.12238527002307, 199.18315040165433, 206.514296508391)

Now we can see that the output of our image descriptor (the `cv2.mean` function) is a feature vector with a list of three numbers: the means of the blue, green, and red channels, respectively.

Color Mean and Standard Deviation
---------------------------------

Let’s compute both the mean and standard deviation of each channel as well.

Again, here is our template:

1.  **What image descriptor am I using?** A color mean and standard deviation descriptor.
2.  **What is the expected output of my image descriptor?** The mean and standard deviation of each channel of the image.

And now the code:

\>>> (means, stds) = cv2.meanStdDev(image)
>>> means, stds
(array(\[\[ 181.12238527\],
       \[ 199.1831504 \],
       \[ 206.51429651\]\]), array(\[\[ 80.67819854\],
       \[ 65.41130384\],
       \[ 77.77899992\]\]))

In order to grab both the mean and standard deviation of each channel, we use the `cv2.meanStdDev` function, which not surprisingly, returns a tuple — one for the means and one for the standard deviations, respectively. Again, this list of numbers serves as our color features.

Let’s combine the means and standard deviations into a single color feature vector:

\>>> import numpy as np
>>> stats = np.concatenate(\[means, stds\]).flatten()
>>> stats
array(\[ 181.12238527,  199.1831504 ,  206.51429651,   80.67819854,
         65.41130384,   77.77899992\])

Now our feature vector `stats` has six entries rather than three. We are now representing the mean of each channel as well as the standard deviation of each channel in the image.

Color Histograms
----------------

Going back to the [Clever Girl: A Guide to Utilizing Color Histograms for Computer Vision and Image Search Engines](https://pyimagesearch.com/2014/01/22/clever-girl-a-guide-to-utilizing-color-histograms-for-computer-vision-and-image-search-engines/) and [Hobbits and Histograms](https://pyimagesearch.com/2014/01/27/hobbits-and-histograms-a-how-to-guide-to-building-your-first-image-search-engine-in-python/ "Hobbits and Histograms – A How-To Guide to Building Your First Image Search Engine in Python"), we could also use a 3D color histogram to describe our Charizard.

1.  **What image descriptor am I using?** A 3D color histogram.
2.  **What is the expected output of my image descriptor?** A list of numbers used to characterize the color distribution of the image.

\>>> hist = cv2.calcHist(\[image\], \[0, 1, 2\], None, \[8, 8, 8\], \[0, 256, 0, 256, 0, 256\])

Here we have a 3D histogram with 8 bins per channel. Let’s examine the shape of our histogram:

\>>> hist.shape
(8, 8, 8)

Our histogram has a shape of `(8, 8, 8)`. How can we use this as a feature vector if it’s multi-dimensional?

We simply flatten it:

\>>> hist = hist.flatten()
>>> hist.shape
(512,)

By defining our image descriptor as a 3D color histogram we can extract a list of numbers (i.e. our feature vector) to represent the distribution of colors in the image.

Summary
-------

In this blog post we have provided a formal definition for an image feature vector. A feature vector is an abstraction of the image itself and at the most basic level, is simply a list of numbers used to represent the image. We have also reviewed some examples on how to extract color features.

The first step of building any image search engine is to define your image descriptor. Once we have defined our image descriptor we can apply our descriptor to an image. The output of the image descriptor is our feature vector.

We then defined a two step template that you can use when defining your image descriptor. You simply need to ask yourself two questions:

1.  **What image descriptor am I using?**
2.  **What is the expected output of my image descriptor?**

The first question defines what aspect of the image you are describing, whether it’s color, shape, or texture. And the second question defines what the output of the descriptor is going to be after it has been applied to the image.

Using this template you can ensure you always know _what_ you are describing and _how_ you are describing it.

Finally, we provided three examples of simple image descriptors and feature vectors to make our discussion more concrete.

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Join the PyImageSearch Newsletter and Grab My FREE 17-page Resource Guide PDF

Enter your email address below to **join the PyImageSearch Newsletter** and **download my FREE 17-page Resource Guide PDF** on Computer Vision, OpenCV, and Deep Learning.