Title: Keras and deep learning on the Raspberry Pi - PyImageSearch

URL Source: https://pyimagesearch.com/2017/12/18/keras-deep-learning-raspberry-pi/

Published Time: 2017-12-18T15:00:16+00:00

Markdown Content:
![Image 1: Building a Not Santa Detector using deep learning, Keras, Python, and the Raspberry Pi](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%200'%3E%3C/svg%3E)

Today’s blog post is the _most fun_ I’ve _**EVER**_ had writing a PyImageSearch tutorial.

It has everything we have been discussing the past few weeks, including:

*   Deep learning
*   Raspberry Pis
*   3D Christmas trees
*   References to HBO’s Silicon Valley _“Not Hotdog”_ detector
*   **Me dressing up as Santa Clause!**

In keeping with the Christmas and Holiday season, I’ll be demonstrating how to take a deep learning model (trained with Keras) and then deploy it to the Raspberry Pi.

But this isn’t _any_ machine learning model…

This image classifier has been _specifically trained_ to detect if Santa Claus is in our video stream.

And if we do detect Santa Claus…

Well. I won’t spoil the surprise (but it does involve a 3D Christmas tree and a jolly tune).

Enjoy the tutorial. Download the code. Hack with it.

**And most of all, _have fun!_**

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

Today’s blog post is a _complete guide_ to running a deep neural network on the Raspberry Pi using Keras.

I’ve framed this project as a _Not Santa_ detector to give you a practical implementation (and have some fun along the way).

In the first part of this blog post, we’ll discuss what a _Not Santa_ detector is (just in case you’re unfamiliar with [HBO’s Silicon Valley _“Not Hotdog”_ detector](https://medium.com/@timanglade/how-hbos-silicon-valley-built-not-hotdog-with-mobile-tensorflow-keras-react-native-ef03260747f3) which has developed a cult following).

We’ll then configure our Raspberry Pi for deep learning by installing TensorFlow, Keras, and a number of other prerequisites.

Once our Raspberry Pi is configured for deep learning we’ll move on to building a Python script that can:

1.  Load our Keras model from disk
2.  Access our Raspberry Pi camera module/USB webcam
3.  Apply deep learning to detect if Santa Clause is in the frame
4.  Access our GPIO pins and play music if Santa is detected

These are my favorite types of blog posts to write here on PyImageSearch as they integrate a bunch of techniques we’ve discussed, including:

*   [Deep learning on the Raspberry Pi](https://pyimagesearch.com/2017/10/02/deep-learning-on-the-raspberry-pi-with-opencv/)
*   [Accessing your Raspberry Pi camera module/USB webcam](https://pyimagesearch.com/2016/01/04/unifying-picamera-and-cv2-videocapture-into-a-single-class-with-opencv/)
*   [Working with GPIO + computer on the Raspberry Pi](https://pyimagesearch.com/2016/05/02/accessing-rpi-gpio-and-gpio-zero-with-opencv-python/)

Let’s get started!

### What is a _Not Santa_ detector?

[![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20394'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2017/12/not_santa_detector_not_hotdog.jpg)

**Figure 1:** The Not Hotdog detector app from HBO’s Silicon Valley.

A _Not Santa_ detector is a play off [HBO’s Silicon Valley](https://www.hbo.com/silicon-valley) where the characters create a smartphone app that can determine if an input photo is a “hot dog” or is “not a hot dog”:

The show is clearly poking fun at the Silicon Valley startup culture in the United States by:

1.  Preying on the hype of machine learning/deep learning
2.  Satirically remaking on the abundance of smartphone applications that serve little purpose (but the creators are convinced their app will “change the world”)

I decided to have some fun myself.

Today we are creating a _Not Santa_ detector that will detect if Santa Claus is in an image/video frame.

For those unfamiliar with [Santa Claus](https://en.wikipedia.org/wiki/Santa_Claus) (or simply Santa” for short), he is a jolly, portly, white-bearded, fictional western culture figure who delivers presents to boys and girls while they sleep Christmas Eve

**However, this application is _not_ totally just for fun and satire!**

We’ll be learning some practical skills along the way, including how to:

1.  Configure your Raspberry Pi for deep learning
2.  Install Keras and TensorFlow on your Raspberry Pi
3.  Deploy a pre-trained Convolutional Neural Network (with Keras) to your Raspberry Pi
4.  Perform a given action once a positive detection has occurred

But before we can write a line of code, let’s first review the hardware we need.

### What hardware do I need?

[![Image 4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20450'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2017/12/not_santa_detector_setup03.jpg)

**Figure 2:** The _Not Santa_ detector setup includes the Raspberry Pi 3, speakers, 3D Christmas tree, and a webcam (not pictured). The Pi has LeNet implemented with Keras in a Python script in order to detect Santa.

In order to follow along _exactly_ with this tutorial (with no modifications) you’ll need:

*   A [Raspberry Pi 3](http://amzn.to/2BDZZx0) (or the [Raspberry Pi 3 Starter Kit](https://www.amazon.com/CanaKit-Raspberry-Complete-Starter-Kit/dp/B01C6Q2GSY/ref=as_li_ss_tl?s=pc&ie=UTF8&qid=1512999342&sr=1-4&keywords=raspberry+pi+3&linkCode=sl1&tag=trndingcom-20&linkId=155e01ef371aebb99db7cb8ea02c8a53) which I also highly recommend)
*   A [Raspberry Pi camera module](http://a.co/at6gpLa) or a USB camera. For the purposes of this tutorial, I went with the [Logitech C920](http://a.co/4xltunW) as it’s a great camera for the price (and having a USB cable gives you a bit of extra room to work with versus the short ribbon on the Pi camera)
*   The [3D Christmas Tree](https://thepihut.com/products/3d-xmas-tree-for-raspberry-pi) for the Raspberry Pi (designed by [Rachel Rayns](https://twitter.com/rachelrayns/))
*   A set of speakers — I recommend [these from Pi Hut](https://thepihut.com/products/mini-portable-speaker-for-the-raspberry-pi?variant=897680793), the [stereo speakers from Adafruit](https://www.adafruit.com/product/1363), or if you’re looking for something small that still packs a punch, you could get [this speaker from Amazon](http://amzn.to/2AF920r)

Of course, you do not need all these parts.

If you have just a Raspberry Pi + camera module/USB camera you’ll be all set (but you will have to modify the code so it doesn’t try to access the GPIO pins or play music via the speakers).

Your setup should look similar to mine in **Figure 2** above where I have connected my speakers, 3D Christmas Tree, and webcam (not pictured as it’s off camera).

I would also recommend hooking up an HDMI monitor + keyboard to test your scripts and debug them:

[![Image 5](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20450'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2017/12/not_santa_detector_setup01.jpg)

**Figure 3:** My deep learning setup includes the Raspberry Pi and components along with a keyboard, mouse, and small HDMI display. With this setup, we will surely catch Santa delivering presents in front of my tree.

In the image above, you can see my Raspberry Pi, HDMI, keyboard, and Christmas critter friends keeping me company while I put together today’s tutorial.

### How do I install TensorFlow and Keras on the Raspberry Pi?

[![Image 6](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20213'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2017/12/not_santa_detector_dl_logos.jpg)

**Figure 4:** We’re going to use Keras with the TensorFlow backend on the Raspberry Pi to implement a deep learning Not Santa detector.

Last week, we learned [how to train a Convolutional Neural Network using Keras](https://pyimagesearch.com/2017/12/11/image-classification-with-keras-and-deep-learning/) to determine if Santa was in an input image.

Today, we are going to take the pre-trained model and deploy it to the Raspberry Pi.

As I’ve mentioned before, the Raspberry Pi is not suitable for training a neural network (outside of “toy” examples). However, the Raspberry Pi can be used to _deploy_ a neural network [once it has already been trained](https://pyimagesearch.com/2017/12/11/image-classification-with-keras-and-deep-learning/) (provided the model can fit into a small enough memory footprint, of course).

I’m going to assume you have already installed OpenCV on your Raspberry Pi.

If you have not installed OpenCV on your Raspberry Pi, start by using [this tutorial](https://pyimagesearch.com/2017/10/09/optimizing-opencv-on-the-raspberry-pi/) where I demonstrate how to optimize your Raspberry Pi + OpenCV install for speed (leading to a 30%+ increase in performance).

_**Note:** This guide will not work with Python 3 — **you’ll instead need to use Python 2.7**. I’ll explain why later in this section. Take the time now to configure your Raspberry Pi with Python 2.7 and OpenCV bindings. In Step #4 of the [Raspberry Pi + OpenCV installation guide](https://pyimagesearch.com/2017/10/09/optimizing-opencv-on-the-raspberry-pi/), be sure to make a virtual environment with the `-p python2` switch._

From there, I recommend increasing the swap space on your Pi. Increasing the swap will enable you to use the Raspberry Pi SD card for additional memory (a critical step when trying to compile and install large libraries on the memory-limited Raspberry Pi).

To increase your swap space, open up `/etc/dphys-swapfile` and then edit the `CONF_SWAPSIZE` variable:

\# set size to absolute value, leaving empty (default) then uses computed value
#   you most likely don't want this, unless you have a special disk situation
# CONF\_SWAPSIZE=100
CONF\_SWAPSIZE=1024

Notice that I am increasing the swap from 100MB to 1024MB.

From there, restart the swap service:

$ sudo /etc/init.d/dphys-swapfile stop
$ sudo /etc/init.d/dphys-swapfile start

_**Note:** Increasing swap size is a great way to burn out your memory card, so **be sure to revert this change and restart the swap service when you’re done**. You can read more about large sizes corrupting memory cards [here](https://www.bitpi.co/2015/02/11/how-to-change-raspberry-pis-swapfile-size-on-rasbian/)._

Now that your swap size has been increased, let’s get started configuring our development environment.

To start, create a Python virtual environment named `not_santa` using **Python 2.7** (I’ll explain why Python 2.7 once we get to the TensorFlow install command):

$ mkvirtualenv not\_santa -p python2

Notice here how the `-p` switch points to `python2` , indicating that Python 2.7 will be used for the virtual environment.

If you are new to Python virtual environments, how they work, and why we use them, please refer to [this guide to help get you up to speed](https://pyimagesearch.com/2016/05/02/accessing-rpi-gpio-and-gpio-zero-with-opencv-python/) as well as [this excellent virtualenv primer from RealPython](https://realpython.com/blog/python/python-virtual-environments-a-primer/).

You’ll also want to make sure you have sym-linked your `cv2.so` bindings into your `not_santa` virtual environment (if you have not done so yet):

$ cd ~/.virtualenvs/not\_santa/lib/python2.7/site-packages
$ ln -s /usr/local/lib/python2.7/site-packages/cv2.so cv2.so

Again, make sure you have compiled OpenCV with **Python 2.7** bindings. You’ll want to double-check your path to your `cv2.so` file as well, just in case your install path is slightly different than mine.

If you compiled Python 3 + OpenCV bindings, created the sym-link, and then tried to `import cv2` into your Python shell, you will get a confusing traceback saying that the import failed.

_**Important:** For these next few pip commands, be sure that you’re in the `not_santa` environment (or your Python environment of choice), otherwise you’ll be installing the packages to your Raspberry Pi’s system Python._

To enter the environment simply use the `workon` command at the bash prompt:

$ workon not\_santa

From there, you’ll see “`(not_santa)`” at the beginning of your bash prompt.

Ensure you have NumPy installed in the `not_santa` environment using the following command:

$ pip install numpy

Since we’ll be accessing the GPIO pins for this project we’ll need to install both [RPi.GPIO](https://pypi.python.org/pypi/RPi.GPIO) and [gpiozero](https://pypi.python.org/pypi/gpiozero):

$ sudo pip install RPi.GPIO gpiozero

**We are now ready to install TensorFlow on your Raspberry Pi.**

The problem is that _there is not an official (Google released) TensorFlow distribution._

We can follow the long, arduous, painful process of [compiling TensorFlow from scratch](https://github.com/samjabrahams/tensorflow-on-raspberry-pi/blob/master/GUIDE.md) on the Raspberry Pi…

Or we can use the pre-compiled binaries created Sam Abrahams (published on [GitHub](https://github.com/samjabrahams/tensorflow-on-raspberry-pi)).

The problem is that there are only two types of pre-compiled TensorFlow binaries:

1.  One for Python 2.7
2.  And another for Python 3.4

The Raspbian Stretch distribution (the latest release of the Raspbian operating system at the time of this writing) ships with Python 3.5 — we, therefore, have a version mismatch.

To avoid any headaches between Python 3.4 and Python 3.5, I decided to stick with the Python 2.7 install.

While I would have liked to use Python 3 for this guide, the install process would have become more complicated (I could easily write multiple posts on installing TensorFlow + Keras on the Raspberry Pi, but since installation is not the main focus of this tutorial, I decided to keep it more straightforward).

Let’s go ahead and install TensorFlow for Python 2.7 using the following commands:

$ wget https://github.com/samjabrahams/tensorflow-on-raspberry-pi/releases/download/v1.1.0/tensorflow-1.1.0-cp27-none-linux\_armv7l.whl
$ pip install tensorflow-1.1.0-cp27-none-linux\_armv7l.whl

_**Note:** You will need to expand the code block above to copy the full URL. I recommend pressing the “<>” button before you copy the commands._

Once TensorFlow compiles and installs (which took around an hour on my Raspberry Pi) you’ll need to install HDF5 and h5py. These libraries will allow us to load our pre-trained model from disk:

$ sudo apt-get install libhdf5-serial-dev
$ pip install h5py

I Install HDF5 + h5py without the `time` command running so I can’t remember the exact amount of time it took to install, but I believe it was around 30-45 minutes.

And finally, let’s install Keras and the other prerequisites required for this project:

$ pip install pillow imutils
$ pip install scipy --no-cache-dir
$ pip install keras==2.1.5

The SciPy install in particular will take a few hours so make sure you let the install run overnight/while you’re at work.

It is _**very important**_ that you install Keras version 2.1.5 for compatibility with TensorFlow 1.1.0.

To test your configuration, open up a Python shell (while in the `not_santa` environment) and execute the following commands:

$ workon not\_santa
$ python
>>> import h5py
>>> from gpiozero import LEDBoard
>>> from gpiozero.tools import random\_values
>>> import cv2
>>> import imutils
>>> import keras
Using TesnsorFlow backend.
>>> print("OpenCV version: {}".format(cv2.\_\_version\_\_))
'OpenCV version: 3.3.1'
>>> print("Keras version: {}".format(keras.\_\_version\_\_))
'Keras version: 2.1.5'
>>> exit()

If all goes as planned you should see Keras imported using the TensorFlow backend.

As the output above demonstrates, you should also double-check that your OpenCV bindings (`cv2` ) can be imported as well.

Finally, _**don’t forget to reduce your swap size**_ from 1024MB back down to 100MB by:

1.  Opening up `/etc/dphys-swapfile` .
2.  Resetting `CONF_SWAPSIZE` to 100MB.
3.  Restarting the swap service (as we discussed earlier in this post).

As mentioned in the note above, setting your swap size back to 100MB is important for memory card longevity. If you skip this step, you may encounter memory corruption issues and a decreased lifespan of the card.

### Running a Keras + deep learning model on the Raspberry Pi

![Image 7](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%200'%3E%3C/svg%3E)

**Figure 5:** Running a deep learning model on the Raspberry Pi using Keras and Python.

We are now ready to code our _Not Santa_ detector using Keras, TensorFlow, and the Raspberry Pi.

Again, I’ll be assuming you have the same hardware setup as I do (i.e., 3D Christmas Tree and speakers) so if your setup is different you’ll need to hack up the code below.

To get started, open a new file, name it `not_santa_detector.py` , and insert the following code:

\# import the necessary packages
from keras.preprocessing.image import img\_to\_array
from keras.models import load\_model
from gpiozero import LEDBoard
from gpiozero.tools import random\_values
from imutils.video import VideoStream
from threading import Thread
import numpy as np
import imutils
import time
import cv2
import os

**Lines 2-12** handle our imports, notably:

*   `keras` is used to preprocess input frames for classification and to load our pre-trained model from disk.
*   `gpiozero` is used to access the 3D Christmas tree.
*   `imutils` is used to access the video stream (whether Raspberry Pi camera module or USB).
*   `threading` is used for non-blocking operations, in particular when we want to light the Christmas tree or play music while not blocking execution of the main thread.

From there, we’ll define a function to light our [3D Christmas tree](https://thepihut.com/products/3d-xmas-tree-for-raspberry-pi):

def light\_tree(tree, sleep=5):
	# loop over all LEDs in the tree and randomly blink them with
	# varying intensities
	for led in tree:
		led.source\_delay = 0.1
		led.source = random\_values()

	# sleep for a bit to let the tree show its Christmas spirit for
	# santa clause
	time.sleep(sleep)

	# loop voer the LEDs again, this time turning them off
	for led in tree:
		led.source = None
		led.value = 0

Our `light_tree` function accepts a `tree` argument (which is assumed be an `LEDBoard` object).

First, we loop over all LEDs in the `tree` and randomly light each of the LEDs to create a “twinkling” effect (**Lines 17-19**).

We leave the lights on for a period of time for some holiday spirit (**Line 23**) and then we loop over the LEDs again, this time turning them off (**Lines 26-28**).

An example of the 3D Christmas tree lights turned on can be seen below:

![Image 8](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20333'%3E%3C/svg%3E)

**Figure 6:** The 3D Christmas tree for the Raspberry Pi. You can get yours from [Pi Hut](https://thepihut.com/products/3d-xmas-tree-for-raspberry-pi) (photo credit: Pi Hut).

Our next function handles playing music when Santa is detected:

def play\_christmas\_music(p):
	# construct the command to play the music, then execute the
	# command
	command = "aplay -q {}".format(p)
	os.system(command)

In the `play_christmas_music` function, we make a system call to the `aplay` command which enables us to play a music file as we would from the command line.

Using the `os.system` call is a bit of a hack, but playing the audio file via pure Python (using a library such as Pygame) is overkill for this project.

From there, let’s hardcode the configurations we’ll use:

\# define the paths to the Not Santa Keras deep learning model and
# audio file
MODEL\_PATH = "santa\_not\_santa.model"
AUDIO\_PATH = "jolly\_laugh.wav"

# initialize the total number of frames that \*consecutively\* contain
# santa along with threshold required to trigger the santa alarm
TOTAL\_CONSEC = 0
TOTAL\_THRESH = 20

# initialize is the santa alarm has been triggered
SANTA = False

**Lines 38 and 39** hardcode paths to our [pre-trained Keras model](https://pyimagesearch.com/2017/12/11/image-classification-with-keras-and-deep-learning/) and audio file. Be sure to use the _**“Downloads”**_ section of this blog post to grab the files.

We also initialize parameters used for detection which include `TOTAL_CONSEC` and `TOTAL_THRESH` . These two values represent the number of frames containing Santa and the threshold at which we’ll both play music and turn on the tree respectively (**Lines 43 and 44**).

The last initialization is `SANTA = False` , a boolean (**Line 47**). We’ll use the `SANTA` variable later in the script as a status flag to aid in our logic.

Next, we’ll load our pre-trained Keras model and initialize our Christmas tree:

\# load the model
print("\[INFO\] loading model...")
model = load\_model(MODEL\_PATH)

# initialize the christmas tree
tree = LEDBoard(\*range(2, 28), pwm=True)

Keras allows us to save models to disk for future use. [Last week](https://pyimagesearch.com/2017/12/11/image-classification-with-keras-and-deep-learning/), we saved our _Not Santa_ model to disk and this week we’re going to load it up on our Raspberry Pi. We load the model on **Line 51** with the Keras `load_model` function.

Our `tree` object is instantiated on **Line 54**. As shown, `tree` is an `LEDBoard` object from the `gpiozero` package.

Now we’ll initialize our video stream:

\# initialize the video stream and allow the camera sensor to warm up
print("\[INFO\] starting video stream...")
vs = VideoStream(src=0).start()
# vs = VideoStream(usePiCamera=True).start()
time.sleep(2.0)

To access the camera, we’ll use `VideoStream` from my [imutils](https://github.com/jrosebr1/imutils) package (you can find the [documentation to VideoStream here](https://pyimagesearch.com/2016/01/04/unifying-picamera-and-cv2-videocapture-into-a-single-class-with-opencv/)) on either **Line 58 or 59**.

_**Important:** If you’d like to use the PiCamera module (instead of a USB camera) for this project, simply comment **Line 58** and uncomment **Line 59**._

We `sleep` for a brief 2 seconds so our camera can warm up (**Line 60**) before we begin looping over the frames:

\# loop over the frames from the video stream
while True:
	# grab the frame from the threaded video stream and resize it
	# to have a maximum width of 400 pixels
	frame = vs.read()
	frame = imutils.resize(frame, width=400)

On **Line 63** we start looping over video frames until the stop condition is met (shown later in the script).

First, we’ll grab a `frame` by calling `vs.read` (**Line 66**).

Then we resize `frame` to `width=400` , _maintaining the aspect ratio_ (**Line 67**). We’ll be preprocessing this `frame` prior to sending it through our neural network model. Later on we’ll be displaying the frame to the screen along with a text label.

From there let’s preprocess the image and pass it through our Keras + deep learning model for prediction:

	# prepare the image to be classified by our deep learning network
	image = cv2.resize(frame, (28, 28))
	image = image.astype("float") / 255.0
	image = img\_to\_array(image)
	image = np.expand\_dims(image, axis=0)

	# classify the input image and initialize the label and
	# probability of the prediction
	(notSanta, santa) = model.predict(image)\[0\]
	label = "Not Santa"
	proba = notSanta

**Lines 70-73** preprocess the `image` and prepare it for classification. To learn more about preprocessing for deep learning, be sure to check out the Starter Bundle of my latest book, [_Deep Learning for Computer Vision with Python_](https://pyimagesearch.com/deep-learning-computer-vision-python-book/).

From there, we query `model.predict` with our `image` as the argument. This sends the `image` through the neural network, returning a tuple containing class probabilities (**Line 77**).

We initialize the `label` to “Not Santa” (we’ll revisit `label` later) and the probability, `proba` , to the value of `notSanta` on **Lines 78 and 79**.

Let’s check to see if Santa is in the image:

	# check to see if santa was detected using our convolutional
	# neural network
	if santa > notSanta:
		# update the label and prediction probability
		label = "Santa"
		proba = santa

		# increment the total number of consecutive frames that
		# contain santa
		TOTAL\_CONSEC += 1

On **Line 83** we check if the probability of `santa` is greater than `notSanta` . If that is the case, we proceed to update the `label` and `proba` followed by incrementing `TOTAL_CONSEC` (**Lines 85-90**).

Provided enough consecutive “Santa” frames have passed, we need to trigger the Santa alarm:

		# check to see if we should raise the santa alarm
		if not SANTA and TOTAL\_CONSEC >= TOTAL\_THRESH:
			# indicate that santa has been found
			SANTA = True

			# light up the christmas tree
			treeThread = Thread(target=light\_tree, args=(tree,))
			treeThread.daemon = True
			treeThread.start()

			# play some christmas tunes
			musicThread = Thread(target=play\_christmas\_music,
				args=(AUDIO\_PATH,))
			musicThread.daemon = False
			musicThread.start()

We have two actions to perform if `SANTA` is `False` and if the `TOTAL_CONSEC` hits the `TOTAL_THRESH` threshold:

1.  Create and start a `treeThread` to twinkle the Christmas tree lights (**Lines 98-100**).
2.  Create and start a `musicThread` to play music in the background (**Lines 103-106**).

These threads will run independently without stopping the forward execution of the script (i.e., a _non-blocking_ operation).

You can also see that, on **Line 95**, we set our `SANTA` status flag to `True` , implying that we have found Santa in the input frame. In the next pass of the loop, we’ll be looking at this value as we did on **Line 93**.

Otherwise (`SANTA` is `True` or the `TOTAL_THRESH` is not met), we reset `TOTAL_CONSEC` to zero and `SANTA` to `False` :

	# otherwise, reset the total number of consecutive frames and the
	# santa alarm
	else:
		TOTAL\_CONSEC = 0
		SANTA = False

Finally, we display the frame to our screen with the generated text label:

	# build the label and draw it on the frame
	label = "{}: {:.2f}%".format(label, proba \* 100)
	frame = cv2.putText(frame, label, (10, 25),
		cv2.FONT\_HERSHEY\_SIMPLEX, 0.7, (0, 255, 0), 2)

	# show the output frame
	cv2.imshow("Frame", frame)
	key = cv2.waitKey(1) & 0xFF
 
	# if the \`q\` key was pressed, break from the loop
	if key == ord("q"):
		break

# do a bit of cleanup
print("\[INFO\] cleaning up...")
cv2.destroyAllWindows()
vs.stop()

The probability value is appended to the `label` containing either “Santa” or “Not Santa” (**Line 115**).

Then using OpenCV’s `cv2.putText` , we can write the label (in Christmas-themed green) on the top of the frame before we display the frame to the screen (**Lines 116-120**).

The exit condition of our infinite while loop is when the ‘q’ key is pressed on the keyboard (**Lines 121-125**).

If the loop’s exit condition is met, we `break` and perform some cleanup on **Lines 129 and 130** before the script itself exits.

That’s all there is to it. Take a look back at the 130 lines we reviewed together — this framework/template can easily be used for other deep learning projects on the Raspberry Pi as well.

Now let’s catch that fat, bearded, jolly man on camera!

### Deep learning + Keras + Raspberry Pi results

In [last week’s blog post](https://pyimagesearch.com/2017/12/11/image-classification-with-keras-and-deep-learning/) we tested our _Not Santa_ deep learning model using stock images gathered from the web.

**But that’s no fun — _and certainly not sufficient for this blog post._**

I’ve always wanted to dress up like good ole’ St. Nicholas, so I [ordered a cheap Santa Claus costume](https://www.target.com/p/men-s-pub-crawl-santa-suit-costume/-/A-14132120) last week:

[![Image 9](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20450'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2017/12/not_santa_detector_me.jpg)

**Figure 7:** Me, Adrian Rosebrock, dressed up as Santa. I’ll be personally testing our _Not Santa_ detector, built using deep learning, Keras, Python, and and OpenCV.

I’m a far cry from the real Santa, but the costume should do the trick.

I then pointed my camera attached to the Raspberry Pi at the Christmas tree in my apartment:

[![Image 10](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20450%20600'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2017/12/not_santa_detector_tree.jpg)

**Figure 8:** My very own Christmas tree will serve as the background for testing our _Not Santa_ detector deep learning model which has been deployed to the Raspberry Pi.

If Santa comes by to put out some presents for the good boys and girls I want to make sure he feels welcome by twinkling the 3D Christmas tree lights and playing some Christmas tunes.

I then started the _Not Santa_ deep learning + Keras detector using the following command:

$ python not\_santa\_detector.py

To follow along, make sure you use _**“Downloads”**_ section below to download the source code + pre-trained model + audio file used in this guide.

Once the _Not Santa_ detector was up and running, I slipped into action:

![Image 11](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%200'%3E%3C/svg%3E)

**Figure 9:** Successfully detecting Santa in a video stream using deep learning, Python, Keras, and a Raspberry Pi.

Whenever Santa is detected the 3D Christmas tree lights up and music starts playing! (which you cannot hear since this is a sample GIF animation).

To see the full _Not Santa_ detector (with sound), take a look at the video below:

Whenever Santa enters the scene you’ll see the 3D Christmas tree display turn on followed by a jolly laugh emitting from the Raspberry Pi speakers (audio credit to [SoundBible](http://soundbible.com/1257-Jolly-Laugh.html)).

Our deep learning model is surprisingly accurate and robust given the small network architecture.

I’ve been good this year, so I’m sure that Santa is stopping at my apartment.

I’m also more confident than I’ve ever been about seeing Santa bring some presents with my _Not Santa_ detector.

Before Christmas, I’ll probably hack this script (with a call to `cv2.imwrite` , or better yet, [save the video clip](https://pyimagesearch.com/2016/02/29/saving-key-event-video-clips-with-opencv/)) to make sure that I save some frames of Santa to disk, as proof. If it is someone else that puts presents under my tree, I’ll certainly know.

_**Dear Santa:** If you’re reading this, just know that I’ve got my Pi on you!_

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

Summary
-------

In today’s blog post you learned how to run a Keras deep learning model on the Raspberry Pi.

To accomplish this, we first [trained our Keras deep learning model](https://pyimagesearch.com/2017/12/11/image-classification-with-keras-and-deep-learning/) to detect if an image contains “Santa” or “Not Santa” on our laptop/desktop.

We then installed TensorFlow and Keras on our Raspberry Pi, enabling us to take our trained deep learning image classifier and then deploy it to our Raspberry Pi. While the Raspberry Pi isn’t suitable for training deep neural networks, it can be used for deploying them — and provided the network architecture is simplistic enough, we can even run our models in real-time.

To demonstrate this, we created a _Not Santa_ detector on our Raspberry Pi that classifies each individual input frame from a video stream.

If Santa is detected, we access our GPIO pins to light up a 3D Christmas tree and play holiday tunes.

What now?
---------

I hope you had fun learning how to build a _Not Santa_ app using deep learning!

If you want to continue studying deep learning and:

*   Master the fundamentals of machine learning and neural networks…
*   Study deep learning in more detail…
*   Train your own Convolutional Neural Networks from scratch…

…then you’ll want to take a look at my new book, _[Deep Learning for Computer Vision with Python](https://pyimagesearch.com/deep-learning-computer-vision-python-book/)_.

Inside my book you’ll find:

*   Super practical **walkthroughs**.
*   **Hands-on tutorials** (with lots of code).
*   Detailed, thorough guides to help you **replicate state-of-the-art results** from popular deep learning publications.

**To learn more about my new book (and start your journey to deep learning mastery), _just [click here](https://pyimagesearch.com/deep-learning-computer-vision-python-book/)_.**

Otherwise, be sure to enter your email address in the form below to be notified when new deep learning post are published here on PyImageSearch.

![Image 12](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!