Title: Optimizing OpenCV on the Raspberry Pi - PyImageSearch

URL Source: https://pyimagesearch.com/2017/10/09/optimizing-opencv-on-the-raspberry-pi/

Published Time: 2017-10-09T14:00:33+00:00

Markdown Content:
![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20435'%3E%3C/svg%3E)

This tutorial is meant for _**advanced Raspberry Pi users**_ who are looking to _milk every last bit of performance_ out of their Pi for computer vision and image processing using OpenCV.

I’ll be assuming:

1.  You have worked through my [previous Raspberry Pi + OpenCV install tutorials](https://pyimagesearch.com/opencv-tutorials-resources-guides/) (ideally _multiple_ times).
2.  You are comfortable with the command line and Unix environments.

Since this is an advanced guide, I’ll be doing less hand holding and instead focusing on the _optimizations themselves._ If you get stuck or run into an error, you’ll need to refer back to the [previous tutorials](https://pyimagesearch.com/opencv-tutorials-resources-guides/) I offer here on PyImageSearch.

By the time you finish this tutorial, your Raspberry Pi will enjoy a _**30% speed increase**_ when executing OpenCV and Python scripts.

**To learn more about optimizing OpenCV on your Raspberry Pi, _just keep reading._**

![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

A couple weeks ago I demonstrated [how to deploy a deep neural network to your Raspberry Pi](https://pyimagesearch.com/2017/10/02/deep-learning-on-the-raspberry-pi-with-opencv/).

The results were satisfactory, taking approximately 1.7 seconds to classify an image using GoogLeNet and 0.9 seconds for SqueezeNet, respectively.

However, I was left wondering if we could do better.

While we cannot _**train**_ neural networks on the Raspberry Pi, we can _**deploy**_ pre-trained networks to our Pi — provided we can optimize the Raspberry Pi sufficiently (and the network can fit into the limited memory of the Pi hardware).

In the remainder of this tutorial, we will discuss the optimizations we will leverage during our OpenCV installation, followed by walking through the seven installation steps.

After our optimized OpenCV compile is installed, we’ll run a few quick tests determine if our new OpenCV install is faster than the previous one.

My goal here to demonstrate that the optimizations are in fact **_much faster on the Raspberry Pi 3_** and you should not hesitate to use them in your own projects.

### NEON and FVPV3

In my research on how to optimize the Raspberry Pi for OpenCV I came across [this excellent article](https://www.theimpossiblecode.com/blog/build-faster-opencv-raspberry-pi3/) by Sagi Zeevi.

Inside the tutorial Sagi recommends using:

1.  NEON
2.  VFPV3
3.  And optionally Threading Building Blocks (TBB)

I’m not a big fan of TBB as (1) the performance gains are meager and (2) they are a royal pain in the ass to install on the Raspberry Pi.

The most bang for your buck is going to come from NEON and VFPV3.

[ARM NEON](https://developer.arm.com/technologies/neon) is an optimization architecture extension for ARM processors. It was designed by the ARM engineers specifically for faster video processing, image processing, speech recognition, and machine learning. This optimization supports [Single Instruction Multiple Data](https://en.wikipedia.org/wiki/SIMD) (SIMD) (as opposed to SISD, MISD, MIMD), which describes an architecture where multiple processing elements in the pipeline perform operations on multiple data points (hardware) all executed with a single instruction.

The ARM engineers also built [VFPV3](https://developer.arm.com/technologies/floating-point), a floating point optimization, into the chip our Raspberry Pi 3’s use. The ARM page linked here describes features included in this optimization such as configurable rounding modes and customizable default not a number (NaN) behavior.

What this means for us is that our neural network is likely to run faster because the ARM processor on the Raspberry Pi 3 has hardware optimizations that we can take advantage of with the 4× ARM Cortex-A53, 1.2GHz processor.

I think you’ll be really impressed with the results, so let’s go ahead and get your optimized OpenCV installed on the Raspberry Pi.

### Step #1: Expand filesystem and reclaim space

For the remainder of this tutorial I’ll be making the following assumptions:

1.  You are working with a brand new, fresh install of **Raspbian Stretch**.
2.  This is not the first time you have installed OpenCV on the Raspberry Pi using Python virtual environments. If it is, please get your feet wet using one of my [introductory OpenCV install guides](https://pyimagesearch.com/opencv-tutorials-resources-guides/).
3.  You are comfortable with the command line and Unix environments.
4.  You know how to debug CMake output for common errors (Python virtual environment not found, missing Python libraries, etc.).

Again, this tutorial is an **advanced guide** so I’ll presenting the commands and only providing an explanation if it is pertinent — by and large, you should know what these commands do before you execute them.

The first step is to run, `raspi-config` and expand your filesystem:

$ sudo raspi-config

And then reboot your Raspberry Pi:

$ sudo reboot

From there, delete both Wolfram Engine and LibreOffice to reclaim ~1GB of space on your Raspberry Pi:

$ sudo apt-get purge wolfram-engine
$ sudo apt-get purge libreoffice\*
$ sudo apt-get clean
$ sudo apt-get autoremove

### Step #2: Install dependencies

The following commands will update and upgrade any existing packages, followed by installing dependencies, I/O libraries, and optimization packages for OpenCV:

$ sudo apt-get update && sudo apt-get upgrade
$ sudo apt-get install build-essential cmake pkg-config
$ sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
$ sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
$ sudo apt-get install libxvidcore-dev libx264-dev
$ sudo apt-get install libgtk2.0-dev libgtk-3-dev
$ sudo apt-get install libcanberra-gtk\*
$ sudo apt-get install libatlas-base-dev gfortran
$ sudo apt-get install python2.7-dev python3-dev

This entire process should take about 5 minutes.

_**Note:** I added `libcanberra-gtk*` which grabs the ARM specific GTK to prevent GTK warnings (not errors; warnings) you may encounter when running Python + OpenCV scripts on the Raspberry Pi.  
_

### Step #3: Download the OpenCV source code

Next, download the OpenCV source code for both the [opencv](https://github.com/opencv/opencv) and [opencv\_contrib](https://github.com/opencv/opencv_contrib) repositories, followed by unarchiving them:

$ cd ~
$ wget -O opencv.zip https://github.com/opencv/opencv/archive/3.4.7.zip
$ unzip opencv.zip
$ wget -O opencv\_contrib.zip https://github.com/opencv/opencv\_contrib/archive/3.4.7.zip
$ unzip opencv\_contrib.zip

_**Note:** You will need to click the “<=>” button in the toolbar of the codeblock above to grab the full paths to the zip archives._

For this blog post, we’ll be using **OpenCV 3.4.7**; however, as newer versions of OpenCV are released you can update the corresponding version numbers (**Note:** some screenshots show 3.3.0 as they were collected at the time when that was the highest version available).

### Step #4: Create your Python virtual environment and install NumPy

We’ll be using Python virtual environments, a best practice when working with Python.

You can install pip, [virtualenv](https://virtualenv.pypa.io/en/latest/), and [virtualenvwrapper](https://virtualenvwrapper.readthedocs.org/en/latest/) using the following commands:

$ wget https://bootstrap.pypa.io/get-pip.py
$ sudo python get-pip.py
$ sudo python3 get-pip.py
$ sudo pip install virtualenv virtualenvwrapper
$ sudo rm -rf ~/.cache/pip

Once both `virtualenv` and `virtualenvwrapper` have been installed, open up your `~/.bashrc` and append the following lines to the _bottom_ of the file, using your favorite terminal-based text editor such as `vim` , `emacs` , or `nano` :

\# virtualenv and virtualenvwrapper
export WORKON\_HOME=$HOME/.virtualenvs
export VIRTUALENVWRAPPER\_PYTHON=/usr/bin/python3
source /usr/local/bin/virtualenvwrapper.sh

From there, reload your `~/.bashrc` file to apply the changes to your current bash session:

$ source ~/.bashrc

You’ll need to run `source ~/.bashrc` _**each time**_ you open a new terminal/SSH into your Pi to ensure your system variables have been set properly (it also loads this file on boot).

Next, create your Python 3 virtual environment:

$ mkvirtualenv cv -p python3

Here I am creating a Python virtual environment named `cv` using Python 3 (alternatively, you may also use Python 2.7 by changing the `-p` switch to `python2` ).

You can name the virtual environment whatever you want, but I use `cv` as the standard naming convention here on PyImageSearch.

Finally, install NumPy into the Python virtual environment:

$ pip install numpy

### Step #5: Compile and install the _optimized_ OpenCV library for Raspberry Pi

We’re now ready to compile and install the optimized version of Raspberry Pi.

Ensure you are in the `cv` virtual environment using the `workon` command:

$ workon cv

And from there configure your build:

$ cd ~/opencv-3.4.7/
$ mkdir build
$ cd build
$ cmake -D CMAKE\_BUILD\_TYPE=RELEASE \\
    -D CMAKE\_INSTALL\_PREFIX=/usr/local \\
    -D OPENCV\_EXTRA\_MODULES\_PATH=~/opencv\_contrib-3.4.7/modules \\
    -D ENABLE\_NEON=ON \\
    -D ENABLE\_VFPV3=ON \\
    -D BUILD\_TESTS=OFF \\
    -D INSTALL\_PYTHON\_EXAMPLES=OFF \\
    -D OPENCV\_ENABLE\_NONFREE=ON \\
    -D CMAKE\_SHARED\_LINKER\_FLAGS='-latomic' \\
    -D BUILD\_EXAMPLES=OFF ..

Notice how the NEON and VFPV3 flags have been enabled. These lines are highlighted.

Additionally, I’ve highlighted the NonFree algorithms flag (giving you the full install) as well as a special linker flag you need for OpenCV 3.4.7.

If you’re using Python 2.7, your _“Python 2”_ section should look like this:

[![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20550%20321'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2017/10/optimizing_opencv_cmake_cv33_python2_box.png)

**Figure 1:** Running CMake to generate the build files for OpenCV 3.3. OpenCV will correctly be built with Python 2.7 and NumPy from our \`cv\` virtualenv.

Otherwise, if you’re compiling OpenCV for Python 3, check the _“Python 3”_ output of CMake:

[![Image 4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20550%20321'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2017/10/optimizing-opencv_cmake_cv33_python3_box.png)

**Figure 2:** After running CMake, Python 3 + NumPy are correctly set from within our \`cv\` virtualenv on the Raspberry Pi.

Notice how the `Interpreter` , `Libraries` , `numpy` , and `packages path` variables have been properly set.

Before you start the compile I would suggest _**increasing your swap space**_. This will enable you to compile OpenCV with _**all four cores**_ of the Raspberry Pi without the compile hanging due to memory exhausting.

Open up your `/etc/dphys-swapfile` file and then edit the `CONF_SWAPSIZE` variable:

\# set size to absolute value, leaving empty (default) then uses computed value
#   you most likely don't want this, unless you have an special disk situation
# CONF\_SWAPSIZE=100
CONF\_SWAPSIZE=1024

Notice that I’m increasing the swap from 100MB to 1024MB. This is the secret sauce to compiling OpenCV with multiple cores on the Raspbian Stretch.

_**If you do not perform this step it’s very likely that your Pi will hang.**_

From there, restart the swap service:

$ sudo /etc/init.d/dphys-swapfile stop
$ sudo /etc/init.d/dphys-swapfile start

_**Note:** Increasing swap size is a great way to burn out your Raspberry Pi microSD card. Flash-based storage have limited number of writes you can perform until the card is essentially unable to hold the 1’s and 0’s anymore. We’ll only be enabling large swap for a short period of time, so it’s not a big deal. Regardless, be sure to backup your `.img` file after installing OpenCV + Python just in case your card dies unexpectedly early. You can read more about large swap sizes corrupting memory cards on [this page](https://www.bitpi.co/2015/02/11/how-to-change-raspberry-pis-swapfile-size-on-rasbian/)._

Now that we’ve updated the swap size, kick off the optimized OpenCV compile using all four cores:

$ make -j4

[![Image 5](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20550%20323'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2017/10/optimizing_opencv_make_complete.png)

**Figure 3:** Our optimized compile of OpenCV 3.3 for the Raspberry Pi 3 has been completed successfully.

Assuming OpenCV compiled without error (as in my screenshot above), you can install your optimized version of OpenCV on your Raspberry Pi:

$ sudo make install
$ sudo ldconfig

_**Don’t forget to go back**_ to your `/etc/dphys-swapfile` file and:

1.  Reset `CONF_SWAPSIZE` to 100MB.
2.  Restart the swap service.

### Step #6: Finish installing your _optimized_ OpenCV on the Raspberry Pi

If you compiled _**OpenCV for Python 3**_, you need to issue the following commands to sym-link the `cv2.so` bindings into your `cv` virtual environment:

$ cd /usr/local/lib/python3.5/site-packages/
$ sudo mv cv2.cpython-35m-arm-linux-gnueabihf.so cv2.so
$ cd ~/.virtualenvs/cv/lib/python3.5/site-packages/
$ ln -s /usr/local/lib/python3.5/site-packages/cv2.so cv2.so

Keep in mind that the exact paths will need to be updated depending if you are using Python 3.4, Python 3.5, Python 3.6, etc.

If you instead compiled **OpenCV for Python 2.7**, you can use these commands to sym-link your `cv2.so` file into the `cv` virtual environment:

$ cd ~/.virtualenvs/cv/lib/python2.7/site-packages/
$ ln -s /usr/local/lib/python2.7/site-packages/cv2.so cv2.so

### Step 7: Testing your _optimized_ OpenCV + Raspberry Pi install

As a quick sanity check, access the `cv` virtual environment, fire up a Python shell and try to import the OpenCV library:

$ source ~/.profile
$ workon cv
$ python
>>> import cv2
>>> cv2.\_\_version\_\_
'3.4.7'
>>>

**Congratulations!** You’ve just installed an optimized OpenCV 3.3 on your Raspberry Pi 3.

### So, how good are these optimizations?

After working through this tutorial you’re probably curious how good these OpenCV + Raspberry Pi optimizations are.

Given that we just optimized for floating point operations a great test would be to run a pre-trained deep neural network on the Raspberry Pi, [similar to what we did last week](https://pyimagesearch.com/2017/10/02/deep-learning-on-the-raspberry-pi-with-opencv/).

Go ahead and use the _**“Downloads”**_ section of this blog post to download our pre-trained Convolutional Neural Networks + example images + classification script.

From there, fire up a shell and execute the following command:

$ python pi\_deep\_learning.py --prototxt models/bvlc\_googlenet.prototxt \\
	--model models/bvlc\_googlenet.caffemodel --labels synset\_words.txt \\
	--image images/barbershop.png
\[INFO\] loading model...
\[INFO\] classification took 0.87173 seconds
\[INFO\] 1. label: barbershop, probability: 0.78055
\[INFO\] 2. label: barber chair, probability: 0.2194
\[INFO\] 3. label: rocking chair, probability: 3.4663e-05
\[INFO\] 4. label: restaurant, probability: 3.7258e-06
\[INFO\] 5. label: hair spray, probability: 1.4715e-06

[![Image 6](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20443'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2017/10/optimizing_opencv_googlenet_barbershop.png)

**Figure 4:** Running an image of a _“barbershop”_ through GoogLeNet on the Raspberry Pi 3 with an optimized install of OpenCV 3.3 achieves a 48.82% speedup.

Here you can see that GoogLeNet classified our image in _**0.87 seconds**_, which is a _massive **48.82%**_ improvement from last week’s 1.7 seconds.

Let’s give SqueezeNet a try:

$ python pi\_deep\_learning.py --prototxt models/squeezenet\_v1.0.prototxt \\
	--model models/squeezenet\_v1.0.caffemodel --labels synset\_words.txt \\
	--image images/barbershop.png
\[INFO\] loading model...
\[INFO\] classification took 0.4777 seconds
\[INFO\] 1. label: barbershop, probability: 0.80578
\[INFO\] 2. label: barber chair, probability: 0.15124
\[INFO\] 3. label: half track, probability: 0.0052872
\[INFO\] 4. label: restaurant, probability: 0.0040124
\[INFO\] 5. label: desktop computer, probability: 0.0033352

[![Image 7](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20433'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2017/10/optimizing_opencv_squeezenet_barbershop.png)

**Figure 5:** Squeezenet on the Raspberry Pi 3 also achieves performance gains using our optimized install of OpenCV 3.3.

Here we can see that SqueezeNet correctly classified the input image in **0.47 seconds**, another huge improvement from the 0.9 seconds from last week (47.78%).

**Based on our results, it’s _very_ clear that our OpenCV optimizations have made a _significant_ impact.**

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

Summary
-------

In today’s blog post, you learned how to optimize your OpenCV install on the Raspberry Pi.

These optimizations came from updating our CMake command to include NEON and VFPV3. When [benchmarking OpenCV](https://www.theimpossiblecode.com/blog/build-faster-opencv-raspberry-pi3/) this leads to an approximate _**30% increase**_ in speed. However, when applied strictly to the new `dnn`  module in OpenCV 3, we are seeing an increase of over _**48%**!_

I hope you enjoyed this tutorial and enjoy your optimized OpenCV + Raspberry Pi!

**But before you go…**
----------------------

Be sure to check out other [Raspberry Pi posts on my blog](https://pyimagesearch.com/tag/raspberry-pi/), and **_consider entering your email address in the form below_ to be notified when future deep learning/Raspberry Pi posts are published here on PyImageSearch.**

![Image 8](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!