Title: k-NN classifier for image classification - PyImageSearch

URL Source: https://pyimagesearch.com/2016/08/08/k-nn-classifier-for-image-classification/

Published Time: 2016-08-08T14:00:29+00:00

Markdown Content:
![Image 1: knn_classifier_header](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20393'%3E%3C/svg%3E)

Now that we’ve had a taste of Deep Learning and Convolutional Neural Networks in [last week’s blog post on LeNet](https://pyimagesearch.com/2016/08/01/lenet-convolutional-neural-network-in-python/), we’re going to take a step back and start to study machine learning in the context of image classification in more depth.

To start, we’ll reviewing the k-Nearest Neighbor (k-NN) classifier, arguably the _most simple, easy to understand_ machine learning algorithm. **In fact, k-NN is _so simple_ that it doesn’t perform any “learning” at all!**

In the remainder of this blog post, I’ll detail how the k-NN classifier works. We’ll then apply k-NN to the [Kaggle Dogs vs. Cats dataset](https://www.kaggle.com/c/dogs-vs-cats/data), a subset of the [Asirra dataset from Microsoft](http://research.microsoft.com/en-us/um/redmond/projects/asirra/).

The goal of the Dogs vs. Cats dataset, as the name suggests, is to classify whether a given image contains a _dog_ or a _cat_. We’ll be using this dataset _a lot_ in future blog posts (for reasons I’ll explain later in this tutorial), so make sure you take the time now to read through this post and familiarize yourself with the dataset.

All that said, _**let’s get started implementing k-NN for image classification to recognize dogs vs. cats in images!**_

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

After getting your [first taste of Convolutional Neural Networks last week](https://pyimagesearch.com/2016/08/01/lenet-convolutional-neural-network-in-python/), you’re probably feeling like we’re taking a big step _backward_ by discussing k-NN today.

What gives?

Well, here’s the deal.

I once wrote a (controversial) blog post on [getting off the deep learning bandwagon and getting some perspective](https://pyimagesearch.com/2014/06/09/get-deep-learning-bandwagon-get-perspective/). Despite the antagonizing title, the overall theme of this post centered around various trends in machine learning history, such as Neural Networks (and how research in NNs almost died in the 70-80’s), Support Vector Machines, and Ensemble methods.

When each of these methods were introduced, researchers and practitioners were equipped with new, powerful techniques — in essence, _**they were given a hammer and every problem looked like a nail**_, when in reality, all they needed was a few simple turns of a phillips head to solve a particular the problem.

I have news for you: **Deep learning is no different.**

Go to the vast majority of popular machine learning and computer vision conferences and look at the recent list of publications. What is the overarching theme?

_Deep learning._

Then, hop on large LinkedIn groups related to computer vision and machine learning. What are many people asking about?

_How to apply deep learning to their datasets._

After that, go over to popular computer science sub-reddits such as [/r/machinelearning](https://www.reddit.com/r/machinelearning). What tutorials are the most consistently upvoted?

_You guessed it: deep learning._

Here’s the bottom line:

**Yes, I will teach you about Deep Learning and Convolutional Neural Networks on this blog — _but you’re damn-well going to understand that Deep Learning is just a TOOL, and like any other tool, there is a_ right _and_ wrong _time to use it._**

Because of this, it’s important for us to understand the basics of machine learning before we progress too far. Over the next few weeks, I’ll be discussing the basics of machine learning and Neural Networks, eventually building up to Deep Learning (where you’ll be able to appreciate the inner-workings of these algorithms more).

### Kaggle Dogs vs. Cats dataset

The Dogs vs. Cats dataset was actually part of a [Kaggle challenge](https://www.kaggle.com/c/dogs-vs-cats/data) a few years back. The challenge itself was simple: _given an image, predict whether it contained a dog or a cat:_

![Image 3: Figure 4: Examples from the Kaggle Dogs vs. Cats dataset.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20473'%3E%3C/svg%3E)

**Figure 1:** Examples from the Kaggle Dogs vs. Cats dataset.

Simple enough — but if you know anything about image classification, you’ll understand that given:

*   Viewpoint variation
*   Scale variation
*   Deformation
*   Occlusion
*   Background clutter
*   Intra-class variation

That the problem is _significantly harder_ than it might appear on the surface.

By simply randomly guessing, **you should be able to reach 50% accuracy** (since there are only two class labels). A machine learning algorithm will need to obtain > 50% accuracy in order to demonstrate that it has in fact “learned” something (or found an underlying pattern in the data).

**Personally, I love the Dogs vs. Cats challenge, _especially_ for teaching Deep Learning.**

Why?

The dataset is simple enough to wrap your head around — there are only two classes: _“dog”_ or _“cat”_.

However, the dataset is nicely sized, containing _25,000_ images in the training data. This means that you have enough data to train data-hungry Convolutional Neural Networks from scratch.

We’ll be using this dataset a lot in future blog posts. I’ve actually included it in the _**“Downloads”**_ section of this blog post for your convenience, so scroll down to grab the code + data before you follow along.

### Project structure

Once you’ve downloaded the archive for this blog post, unzip it to someplace convenient. From there let’s take a look at the project directory structure:

$ tree --filelimit 10
.
├── kaggle\_dogs\_vs\_cats
│   └── train \[25000 entries exceeds filelimit, not opening dir\]
└── knn\_classifier.py

2 directories, 1 file

The Kaggle dataset is included in the `kaggle_dogs_vs_cats/train` directory (it comes from `train.zip` available on the [Kaggle webpage](https://www.kaggle.com/c/dogs-vs-cats/data)).

We’ll be reviewing one Python script today — `knn_classifier.py` . This file will load the dataset, establish and run the K-NN classifier, and print out the evaluation metrics.

### How does the k-NN classifier work?

The [k-Nearest Neighbor](https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm) classifier is _by far_ the most simple machine learning/image classification algorithm. In fact, it’s _so simple_ that it doesn’t actually “learn” anything.

Inside, this algorithm simply relies on the distance between feature vectors, [much like building an image search engine](https://pyimagesearch.com/2014/12/01/complete-guide-building-image-search-engine-python-opencv/) — only this time, we have the _labels_ associated with each image so we can predict and return an actual _category_ for the image.

Simply put, the k-NN algorithm classifies unknown data points by finding the _most common class_ among the _**k-closest examples.**_ Each data point in the _k_ closest examples casts a vote and the category with the most votes wins!

Or, in plain english: _“Tell me who your neighbors are, and I’ll tell you who you are”_

To visualize this, take a look at the following toy example where I have plotted the “fluffiness” of animals along the _x-axis_ and the lightness of their coat on the _y-axis_:

![Image 4: Figure 2: Plotting the fluffiness of animals along the x-axis and the lightness of their coat on the y-axis.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20548'%3E%3C/svg%3E)

**Figure 2:** Plotting the fluffiness of animals along the _x-axis_ and the lightness of their coat on the _y-axis_.

Here we can see there are two categories of images and that each of the data points within each respective category are grouped relatively close together in an _n-_dimensional space. Our dogs tend to have dark coats which are not very fluffy while our cats have very light coats that are extremely fluffy.

This implies that the distance between two data points in the _red circle_ is _much smaller_ than the distance between a data point in the _red circle_ and a data point in the _blue circle_.

In order to apply the k-nearest Neighbor classification, we need to define a [distance metric or similarity function](https://pyimagesearch.com/2014/02/17/building-an-image-search-engine-defining-your-similarity-metric-step-3-of-4/). Common choices include the [Euclidean distance](https://en.wikipedia.org/wiki/Euclidean_distance):

![Image 5: Figure 2: The Euclidean distance.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20188%2031'%3E%3C/svg%3E)

**Figure 3:** The Euclidean distance.

And the [Manhattan/city block distance](https://en.wikipedia.org/wiki/Taxicab_geometry):

![Image 6: Figure 3: The Manhattan/city block distance.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20162%2022'%3E%3C/svg%3E)

**Figure 4:** The Manhattan/city block distance.

Other distance metrics/similarity functions can be used depending on your type of data (the chi-squared distance is often used for distributions \[i.e., histograms\]). In today’s blog post, for the sake of simplicity, we’ll be using the Euclidean distance to compare images for similarity.

### Implementing k-NN for image classification with Python

Now that we’ve discussed what the k-NN algorithm is, along with what dataset we’re going to apply it to, let’s write some code to actually perform image classification using k-NN.

Open up a new file, name it `knn_classifier.py` , and let’s get coding:

\# import the necessary packages
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model\_selection import train\_test\_split
from imutils import paths
import numpy as np
import argparse
import imutils
import cv2
import os

We start off on **Lines 2-9** by importing our required Python packages. If you haven’t already installed the [scikit-learn library](http://scikit-learn.org/stable/), then you’ll want to [follow these instructions](http://scikit-learn.org/stable/install.html) and install it now.

_**Note:** This blog post has been updated to be compatible with the future `scikit-learn==0.20` where `sklearn.cross_validation` has been replaced by `sklearn.model_selection` ._

Secondly, we’ll be using the [imutils library](https://github.com/jrosebr1/imutils), a package that I have created to store common computer vision processing functions. If you do not have `imutils` installed, you’ll want to do that now:

$ pip install imutils

Next, we are going to define two methods to take an input image and convert it to a [feature vector](https://pyimagesearch.com/2014/03/03/charizard-explains-describe-quantify-image-using-feature-vectors/), or a list of numbers that quantify the contents of an image. The first method can be seen below:

def image\_to\_feature\_vector(image, size=(32, 32)):
	# resize the image to a fixed size, then flatten the image into
	# a list of raw pixel intensities
	return cv2.resize(image, size).flatten()

The `image_to_feature_vector` method is an extremely naive function that simply takes an input `image` and resizes it to a fixed width and height (`size` ), and then flattens the RGB pixel intensities into a single list of numbers.

This means that our input `image` will be shrunk to _32 x 32_ pixels, and given three channels for each Red, Green, and Blue component respectively, our output “feature vector” will be a list of _32 x 32 x 3 = 3,072_ numbers.

Strictly speaking, the output of `image_to_feature_vector` is not a true “feature vector” since we tend to think of “features” and “descriptors” as _abstract quantifications of the image contents._

Furthermore, utilizing raw pixel intensities as inputs to machine learning algorithms tends to yield poor results as even small changes in rotation, translation, viewpoint, scale, etc., can _dramatically_ influence the image itself (and thus your output feature representation).

_**Note:** As we’ll find out in later tutorials, Convolutional Neural Networks obtain fantastic results using raw pixel intensities as inputs — but this is because they **learn** a robust set of discriminating filters during the training process._

We then define our second method, this one called `extract_color_histogram` :

def extract\_color\_histogram(image, bins=(8, 8, 8)):
	# extract a 3D color histogram from the HSV color space using
	# the supplied number of \`bins\` per channel
	hsv = cv2.cvtColor(image, cv2.COLOR\_BGR2HSV)
	hist = cv2.calcHist(\[hsv\], \[0, 1, 2\], None, bins,
		\[0, 180, 0, 256, 0, 256\])

	# handle normalizing the histogram if we are using OpenCV 2.4.X
	if imutils.is\_cv2():
		hist = cv2.normalize(hist)

	# otherwise, perform "in place" normalization in OpenCV 3 (I
	# personally hate the way this is done
	else:
		cv2.normalize(hist, hist)

	# return the flattened histogram as the feature vector
	return hist.flatten()

As the name suggests, this function accepts an input `image` and constructs a color histogram to characterize the color distribution of the image.

First, we convert our `image` to the HSV color space on **Line 19**. We then apply the `cv2.calcHist` function to compute a 3D color histogram for the `image` (**Lines 20 and 21**). You can read more about [computing color histograms in this post](https://pyimagesearch.com/2014/01/22/clever-girl-a-guide-to-utilizing-color-histograms-for-computer-vision-and-image-search-engines/). You might also be interested in [applying color histograms to image search engines](https://pyimagesearch.com/2014/12/01/complete-guide-building-image-search-engine-python-opencv/) and in general, [how to compate color histograms for similarity](https://pyimagesearch.com/2014/07/14/3-ways-compare-histograms-using-opencv-python/).

Given our computed `hist` , we then normalize it, taking care to use the appropriate `cv2.normalize` function signature based on our OpenCV version (**Lines 24-30**).

Given _8 bins_ for each of the Hue, Saturation, and Value channels respectively, our final feature vector is of size _8 x 8 x 8 = 512_, thus our image is characterized by a _512-d_ feature vector.

Next, let’s parse our command line arguments:

\# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add\_argument("-d", "--dataset", required=True,
	help="path to input dataset")
ap.add\_argument("-k", "--neighbors", type=int, default=1,
	help="# of nearest neighbors for classification")
ap.add\_argument("-j", "--jobs", type=int, default=-1,
	help="# of jobs for k-NN distance (-1 uses all available cores)")
args = vars(ap.parse\_args())

We require only one command line argument, followed by two optional ones, each of which are detailed below:

*   `--dataset` : This is the path to our input Kaggle Dogs vs. Cats dataset directory.
*   `--neighbors` : Here we can supply the number of nearest neighbors that are taken into account when classifying a given data point. We’ll default this value to _one_, meaning that an image will be classified by finding its closest neighbor in an _n_–_dimensional_ space and then taking the label of the closest image. In next week’s post, I’ll demonstrate how to automatically tune _k_ for optimal accuracy.
*   `--jobs` : Finding the nearest neighbor for a given image requires us to compute the distance from our _input image_ to _every other image in our dataset_. This is clearly a _O(N)_ operation that scales linearly. For larger datasets, this can become prohibitively slow. In order to speedup the process, we can distribute the computation of nearest neighbors across multiple processors/cores of our machine. Setting `--jobs` to `-1` ensures that _all_ processors/cores are used to help speedup the classification process.

_**Note:**_ _We can also speedup the k-NN classifier by utilizing specialized data structures such as [kd-trees](https://en.wikipedia.org/wiki/K-d_tree) or Approximate Nearest Neighbor algorithms such as [FLANN](http://www.cs.ubc.ca/research/flann/) or [Annoy](https://github.com/spotify/annoy). In practice, these algorithms can reduce nearest neighbor search to approximately_ O(log N)_; however, for the sake of simplicity in this post, we’ll perform an exhaustive nearest neighbor search._

We are now ready to prepare our images for feature extraction:

\# grab the list of images that we'll be describing
print("\[INFO\] describing images...")
imagePaths = list(paths.list\_images(args\["dataset"\]))

# initialize the raw pixel intensities matrix, the features matrix,
# and labels list
rawImages = \[\]
features = \[\]
labels = \[\]

**Line 47** grabs the paths to all 25,000 training images from disk.

We then initialize three lists (**Lines 51-53**) to store the raw image pixel intensities (the _3072-d_ feature vector), another to store the histogram features (the _512-d_ feature vector), and finally the class labels themselves (either “dog” or “cat”).

Let’s move on to extracting features from our dataset:

\# loop over the input images
for (i, imagePath) in enumerate(imagePaths):
	# load the image and extract the class label (assuming that our
	# path as the format: /path/to/dataset/{class}.{image\_num}.jpg
	image = cv2.imread(imagePath)
	label = imagePath.split(os.path.sep)\[-1\].split(".")\[0\]

	# extract raw pixel intensity "features", followed by a color
	# histogram to characterize the color distribution of the pixels
	# in the image
	pixels = image\_to\_feature\_vector(image)
	hist = extract\_color\_histogram(image)

	# update the raw images, features, and labels matricies,
	# respectively
	rawImages.append(pixels)
	features.append(hist)
	labels.append(label)

	# show an update every 1,000 images
	if i > 0 and i % 1000 == 0:
		print("\[INFO\] processed {}/{}".format(i, len(imagePaths)))

We start looping over our input images on **Line 56**. Each image is loaded from disk and the class label is extracted from the `imagePath` (**Lines 59 and 60**).

We apply the `image_to_feature_vector` and `extract_color_histogram` functions on **Lines 65 and 66** — these functions are used to extract our feature vectors from the input `image` .

Given our features labels, we then update the respective `rawImages` , `features` , and `labels` lists on **Lines 70-72**.

Finally, we display an update to our terminal to inform us on feature extraction progress every 1,000 images (**Lines 75 and 76**).

You might be curious how much memory our `rawImages` and `features` matrices take up — the following code block will tell us when executed:

\# show some information on the memory consumed by the raw images
# matrix and features matrix
rawImages = np.array(rawImages)
features = np.array(features)
labels = np.array(labels)
print("\[INFO\] pixels matrix: {:.2f}MB".format(
	rawImages.nbytes / (1024 \* 1000.0)))
print("\[INFO\] features matrix: {:.2f}MB".format(
	features.nbytes / (1024 \* 1000.0)))

We start by converting our lists to NumPy arrays. We then use the `.nbytes` attribute of the NumPy array to display the number of megabytes of memory the representations utilize (about 75MB for the raw pixel intensities and 50MB for the color histograms. This implies that we can _easily_ store our features in main memory).

Next, we need to take our data partition it into two splits — one for _training_ and another for _testing_:

\# partition the data into training and testing splits, using 75%
# of the data for training and the remaining 25% for testing
(trainRI, testRI, trainRL, testRL) = train\_test\_split(
	rawImages, labels, test\_size=0.25, random\_state=42)
(trainFeat, testFeat, trainLabels, testLabels) = train\_test\_split(
	features, labels, test\_size=0.25, random\_state=42)

Here we’ll be using 75% of our data for training and the remaining 25% for testing the k-NN algorithm.

Let’s apply the k-NN classifier to the raw pixel intensities:

\# train and evaluate a k-NN classifer on the raw pixel intensities
print("\[INFO\] evaluating raw pixel accuracy...")
model = KNeighborsClassifier(n\_neighbors=args\["neighbors"\],
	n\_jobs=args\["jobs"\])
model.fit(trainRI, trainRL)
acc = model.score(testRI, testRL)
print("\[INFO\] raw pixel accuracy: {:.2f}%".format(acc \* 100))

Here we instantiate the `KNeighborsClassifier` object from the [scikit-learn library](http://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html) using the supplied number of `--neighbors` and `--jobs` .

We then “train” our model by making a call to `.fit` on **Line 99**, followed by evaluation on the testing data on **Line 100**.

In a similar fashion, we can also train and evaluate a k-NN classifier on our histogram representations:

\# train and evaluate a k-NN classifer on the histogram
# representations
print("\[INFO\] evaluating histogram accuracy...")
model = KNeighborsClassifier(n\_neighbors=args\["neighbors"\],
	n\_jobs=args\["jobs"\])
model.fit(trainFeat, trainLabels)
acc = model.score(testFeat, testLabels)
print("\[INFO\] histogram accuracy: {:.2f}%".format(acc \* 100))

### k-NN image classification results

To test our k-NN image classifier, make sure you have downloaded the source code to this blog post using the _**“Downloads”**_ form found at the bottom of this tutorial.

The [Kaggle Dogs vs. Cats dataset](https://www.kaggle.com/c/dogs-vs-cats/data) is included with the download.

From there, just execute the following command:

$ python knn\_classifier.py --dataset kaggle\_dogs\_vs\_cats

At first, you’ll see that our images are being described and quantified via the `image_to_feature_vector`  and `extract_color_histogram`  functions:

![Image 7: Figure 5: Quantifying and extracting features from the Dogs vs. Cats dataset.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20547'%3E%3C/svg%3E)

**Figure 5:** Quantifying and extracting features from the Dogs vs. Cats dataset.

This process shouldn’t take longer than 1-3 minutes depending on the speed of your machine.

After the feature extraction process is complete, we can see some information on the size (in MB) of our feature representations:

![Image 8: Figure 6: Measuring the size of our feature matrices.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20550'%3E%3C/svg%3E)

**Figure 6:** Measuring the size of our feature matrices.

The raw pixel features take up 75MB while the color histograms require only 50MB of RAM.

Finally, the k-NN algorithm is trained and evaluated for both the raw pixel intensities and color histograms:

![Image 9: Figure 7: Evaluating our k-NN algorithm for image classification.](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20550'%3E%3C/svg%3E)

**Figure 7:** Evaluating our k-NN algorithm for image classification.

As the figure above demonstrates, by utilizing **_raw pixel intensities_** we were able to reach **54.42% accuracy.** On the other hand, applying k-NN to color histograms achieved a slightly better **57.58% accuracy**.

In both cases, we were able to obtain > 50% accuracy, demonstrating there is an underlying pattern to the images for both raw pixel intensities and color histograms.

_However, that 57% accuracy leaves much to be desired._

And as you might imagine, color histograms aren’t the best way to distinguish between a dog and a cat:

*   There are brown dogs. And there are brown cats.
*   There are black dogs. And there are black cats.
*   And certainly a dog and cat could appear in the same environment (such as a house, park, beach, etc.) where the _background color distributions_ are similar.

Because of this, utilizing strictly color is not a great choice for characterizing the difference between dogs and cats — but that’s okay. The purpose of this blog post was simply to introduce the concept of image classification using the k-NN algorithm.

We can easily apply methods to obtain higher accuracy. And as we’ll see, utilizing Convolutional Neural Networks we can **achieve > 95% accuracy without much effort** — but I’ll save that for a future discussion once we better understand image classification.

### Want to learn more about Convolutional Neural Networks _right now?_

If you enjoyed this tutorial on image classification, you’ll _definitely_ want to take a look at the _**[PyImageSearch Gurus course](https://pyimagesearch.com/pyimagesearch-gurus/?src=post-knn-classifier)**_ — the most _complete, comprehensive_ computer vision course online today.

Inside the course, you’ll find over **168 lessons** covering **2,161+ pages** of content on _Deep Learning_, _Convolutional Neural Networks_, _Image Classification,_ _F__ace Recognition_, _**and much more**._

To learn more about the PyImageSearch Gurus course (and grab **_10 FREE sample lessons + course syllabus_**), just click the button below:

[Click here to learn more about PyImageSearch Gurus!](https://pyimagesearch.com/pyimagesearch-gurus/?src=post-knn-classifier)

### Can we do better?

You might be wondering, can we do better than the 57% classification accuracy?

You’ll notice that I’ve used only _k=1_ in this example, implying that only _one_ nearest neighbor is considered when classifying each image. How would the results change if I used _k=3_ or _k=5_?

And how about the choice in distance metric — would the Manhattan/City block distance be a better choice?

How about changing _**both**_ the value of _k_ and distance _metric_ at the same time?

Would classification accuracy improve? Get worse? Stay the same?

The fact is that _nearly all_ machine learning algorithms require a bit of tuning to obtain optimal results. In order to determine the optimal set of values for these model variables, we apply a process called _hyperparameter tuning_, which is exactly the topic of next week’s blog post.

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

Summary
-------

In this blog post, we reviewed the basics of image classification using the k-NN algorithm. We then applied the k-NN classifier to the Kaggle Dogs vs. Cats dataset to identify whether a given image contained a dog or a cat.

Utilizing only the raw pixel intensities of the input image images, we obtained 54.42% accuracy. And by using color histograms, we achieved a slightly better 57.58% accuracy. Since both of these results are > 50% (we should expect to get 50% accuracy simply by random guessing), we can ascertain that there is an underlying pattern in the raw pixels/color histograms that can be used to discriminate dogs vs. cats (although 57% accuracy is quite poor).

That raises the question: _“Is it possible to obtain > 57% classification accuracy using k-NN? If so, how?”_

The answer is _**hyperparameter tuning**_ — which is exactly what we’ll be covering in next week’s blog post.

**Be sure to sign up for the PyImageSearch Newsletter using the form below to be notified when the next blog post goes live!**

![Image 10](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!