Title: YOLO and Tiny-YOLO object detection on the Raspberry Pi and Movidius NCS - PyImageSearch

URL Source: https://pyimagesearch.com/2020/01/27/yolo-and-tiny-yolo-object-detection-on-the-raspberry-pi-and-movidius-ncs/

Published Time: 2020-01-27T15:00:56+00:00

Markdown Content:
In this tutorial, you will learn how to utilize YOLO and Tiny-YOLO for near real-time object detection on the Raspberry Pi with a Movidius NCS.

**The YOLO object detector is often cited as being one of the _fastest_ deep learning-based object detectors**, achieving a higher FPS rate than computationally expensive two-stage detectors (ex. Faster R-CNN) and some single-stage detectors (ex. RetinaNet and some, but not all, variations of SSDs).

However, even with all that speed, **YOLO is _still_ not fast enough to run on embedded devices such as the Raspberry Pi** — even with the aid of the Movidius NCS.

To help make YOLO _even faster_, Redmon et al. (the creators of YOLO), defined a variation of the YOLO architecture called **Tiny-YOLO.**

The Tiny-YOLO architecture is **approximately 442% _faster_** than it’s larger big brothers, achieving upwards of **244 FPS** on a single GPU.

The small model size (< 50MB) and fast inference speed make the Tiny-YOLO object detector naturally suited for embedded computer vision/deep learning devices such as the Raspberry Pi, Google Coral, and NVIDIA Jetson Nano.

Today you’ll learn how to take Tiny-YOLO and then deploy it to the Raspberry Pi using a Movidius NCS to obtain near real-time object detection.

**To learn how to utilize YOLO and TinyYOLO for object detection on the Raspberry Pi with the Movidius NCS, _just keep reading!_**

![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20128%20128'%3E%3C/svg%3E)

#### Looking for the source code to this post?

[Jump Right To The Downloads Section](#download-the-code)

In the first part of this tutorial, we’ll learn about the YOLO and Tiny-YOLO object detectors.

From there, I’ll show you how to configure your Raspberry Pi and OpenVINO development environment so that they can utilize Tiny-YOLO.

We’ll then review our directory structure for the project, including a shell script required to properly access your OpenVINO environment.

Once we understand our project structure, we’ll move on to implementing a Python script that:

1.  Accesses our OpenVINO environment.
2.  Reads frames from a video stream.
3.  Performs near real-time object detection using a Raspberry Pi, Movidius NCS, and Tiny-YOLO.

We’ll wrap up the tutorial by examining the results of our script.

### What are YOLO and Tiny-YOLO?

[![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20479'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2020/01/tiny_yolo_comparison.png)

**Figure 1:** Tiny-YOLO has a lower mAP score on the COCO dataset than most object detectors. That said, Tiny-YOLO may be a useful object detector to pair with your Raspberry Pi and Movidius NCS. ([image source](https://pjreddie.com/darknet/yolo/))

Tiny-YOLO is a variation of the “You Only Look Once” (YOLO) object detector proposed by Redmon et al. in their 2016 paper, [_You Only Look Once: Unified, Real-Time Object Detection_](https://arxiv.org/abs/1506.02640).

YOLO was created to help improve the speed of slower two-stage object detectors, such as Faster R-CNN.

While R-CNNs are _accurate_ they are _quite slow_, even when running on a GPU.

On the contrary, single-stage detectors such as YOLO are _quite fast_, obtaining super real-time performance on a GPU.

**The downside, of course, is that YOLO tends to be _less accurate_** (and in my experience, much harder to train than SSDs or RetinaNet).

Since Tiny-YOLO is a smaller version than its big brothers, this also means that Tiny-YOLO is unfortunately **_even less accurate_**.

For reference, Redmon et al. report ~51-57% mAP for YOLO on the COCO benchmark dataset while Tiny-YOLO is only 23.7% mAP — less than half of the accuracy of its bigger brothers.

That said, 23% mAP is _still_ reasonable enough for some applications.

**My general advice when using YOLO is to “simply give it a try”:**

*   In some cases, it may work perfectly fine for your project.
*   And in others, you may seek more accurate detectors (Faster R-CNN, SSDs, RetinaNet, etc.).

To learn more about YOLO, Tiny-YOLO, and other YOLO variants, [be sure to refer to Redmon et al.’s 2018 publication.](https://arxiv.org/abs/1506.02640)

### Configuring your Raspberry Pi + OpenVINO environment

[![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20450'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2019/04/openvino_install_header.png)

**Figure 2:** Configuring the OpenVINO toolkit for your Raspberry Pi and Movidius NCS to conduct TinyYOLO object detection.

This tutorial _requires_ a **Raspberry Pi 4B** _and_ **Movidius NCS2** (the NCS1 is _not_ supported) in order to replicate my results.

Configuring your Raspberry Pi with the Intel Movidius NCS for this project is admittedly challenging.

I suggest you (1) pick up a copy of **_[Raspberry Pi for Computer Vision](https://pyimagesearch.com/raspberry-pi-for-computer-vision/)_**, and (2) flash the **[included pre-configured .img](https://pyimagesearch.com/2016/11/21/raspbian-opencv-pre-configured-and-pre-installed/)** to your microSD. The .img that comes included with the book is worth its weight in gold as it will save you countless hours of toiling and frustration.

For the stubborn few who wish to configure their Raspberry Pi + [OpenVINO](https://software.intel.com/en-us/openvino-toolkit) on their own, here is a brief guide:

1.  Head to my **[BusterOS install guide](https://pyimagesearch.com/2019/09/16/install-opencv-4-on-raspberry-pi-4-and-raspbian-buster/)** and follow all instructions to create an environment named `cv`.
2.  Follow my **[OpenVINO installation guide](https://pyimagesearch.com/2019/04/08/openvino-opencv-and-movidius-ncs-on-the-raspberry-pi/)** and create a 2nd environment named `openvino`. Be sure to download OpenVINO 4.1.1 (4.1.2 has unresolved issues).

You will need a package called JSON-Minify to parse our JSON configuration. You may install it into your virtual environment:

$ pip install json\_minify

At this point, your RPi will have _both_ a normal OpenCV environment as well as an OpenVINO-OpenCV environment. You will use the `openvino` environment for this tutorial.

Now, simply plug in your NCS2 into a blue USB 3.0 port (the RPi 4B has USB 3.0 for maximum speed) and start your environment using either of the following methods:

**Option A:** Use the shell script on my _[Pre-configured Raspbian .img](https://pyimagesearch.com/2016/11/21/raspbian-opencv-pre-configured-and-pre-installed/)_ (the same shell script is described in the _“Recommended: Create a shell script for starting your OpenVINO environment”_ section of my [OpenVINO installation guide](https://pyimagesearch.com/2019/04/08/openvino-opencv-and-movidius-ncs-on-the-raspberry-pi/)).

From here on, you can _activate your OpenVINO_ environment with **one simple command** (as opposed to two commands like in the previous step:

$ source ~/start\_openvino.sh
Starting Python 3.7 with OpenCV-OpenVINO 4.1.1 bindings...

**Option B:** One-two punch method.

If you don’t mind executing two commands instead of one, you can open a terminal and perform the following:

$ workon openvino
$ source ~/openvino/bin/setupvars.sh

The first command activates our OpenVINO virtual environment. The second command sets up the Movidius NCS with OpenVINO (and is very important, otherwise your script will error out).

Both **Option A** and **Option B** assume that you either are using my _[Pre-configured Raspbian .img](https://pyimagesearch.com/2016/11/21/raspbian-opencv-pre-configured-and-pre-installed/)_ or that you followed my [OpenVINO installation guide](https://pyimagesearch.com/2019/04/08/openvino-opencv-and-movidius-ncs-on-the-raspberry-pi/) and installed OpenVINO 4.1.1 on your own.

**Caveats:**

*   Some versions of OpenVINO struggle to read .mp4 videos. This is a known bug that PyImageSearch has reported to the Intel team. Our [preconfigured .img](https://pyimagesearch.com/2016/11/21/raspbian-opencv-pre-configured-and-pre-installed/) includes a fix. Abhishek Thanki edited the source code and compiled OpenVINO from source. This blog post is long enough as is, so I cannot include the compile-from-source instructions. If you encounter this issue please encourage Intel to fix the problem, and either (A) [compile from source using our customer portal instructions](https://rpi4cv.pyimagesearch.com/how-to-compile-opencv-openvino/), or (B) pick up a copy of _[Raspberry Pi for Computer Vision](https://pyimagesearch.com/raspberry-pi-for-computer-vision/)_ and use the pre-configured .img.
*   The NCS1 does not support the TinyYOLO model provided with this tutorial. This is atypical — usually, the NCS2 and NCS1 are very compatible (with the NCS2 being faster).
*   _We will add to this list if we discover other caveats._

### Project Structure

Go ahead and grab today’s downloadable .zip from the **_“Downloads”_** section of today’s tutorial. Let’s inspect our project structure directly in the terminal with the `tree` command:

$ tree --dirsfirst
.
├── config
│   └── config.json
├── intel
│   ├── \_\_init\_\_.py
│   ├── tinyyolo.py
│   └── yoloparams.py
├── pyimagesearch
│   ├── utils
│   │   ├── \_\_init\_\_.py
│   │   └── conf.py
│   └── \_\_init\_\_.py
├── videos
│   └── test\_video.mp4
├── yolo
│   ├── coco.names
│   ├── frozen\_darknet\_tinyyolov3\_model.bin
│   ├── frozen\_darknet\_tinyyolov3\_model.mapping
│   └── frozen\_darknet\_tinyyolov3\_model.xml
└── detect\_realtime\_tinyyolo\_ncs.py

6 directories, 13 files

Our TinyYOLO model trained on the COCO dataset is provided via the `yolo/` directory.

The `intel/` directory contains two classes **provided by Intel Corporation:**

*   `TinyYOLOv3`: A class for parsing, scaling, and computing [Intersection over Union](https://pyimagesearch.com/2016/11/07/intersection-over-union-iou-for-object-detection/) for the TinyYOLO results.
*   `TinyYOLOV3Params`: A class for building a layer parameters object.

We will not review either of the Intel-provided scripts today. You are encouraged to review the files on your own.

Our `pyimagesearch` module contains our `Conf` class, a utility responsible for parsing `config.json`.

A testing video of people walking through a public place (grabbed from [Oxford University](http://www.robots.ox.ac.uk/ActiveVision/Research/Projects/2009bbenfold_headpose/project.html#datasets)‘s site) is provided for you to perform TinyYOLO object detection on. I encourage you to add your own `videos/` as well.

The heart of today’s tutorial lies in `detect_realtime_tinyyolo_ncs.py`. This script loads the TinyYOLOv3 model and performs inference on every frame of a realtime video stream. You may use your PiCamera, USB camera, or a video file residing on disk. The script will calculate the overall frames per second (FPS) benchmark for near real-time TinyYOLOv3 inference on your Raspberry Pi 4B and NCS2.

### Our Configuration File

[![Image 4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20240'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2020/01/tiny_yolo_openvino.png)

**Figure 3:** [Intel’s OpenVINO Toolkit](https://software.intel.com/en-us/openvino-toolkit) is combined with OpenCV allowing for optimized deep learning inference on Intel devices such as the Movidius Neural Compute Stick. We will use OpenVINO for TinyYOLO object detection on the Raspberry Pi and Movidius NCS.

Our configuration variables are housed in our `config.json` file. Go ahead and open it now and let’s inspect the contents:

{
	// path to YOLO architecture definition XML file
	"xml\_path": "yolo/frozen\_darknet\_tinyyolov3\_model.xml",

	// path to the YOLO weights
	"bin\_path": "yolo/frozen\_darknet\_tinyyolov3\_model.bin",

	// path to the file containing COCO labels
	"labels\_path": "yolo/coco.names",

**Line 3** defines our TinyYOLOv3 architecture definition file path while **Line 6** specifies the path to the pre-trained TinyYOLOv3 COCO weights.

We then provide the path to the COCO dataset label names on **Line 9**.

Let’s now look at variables used to filter detections:

	// probability threshold for detections filtering
	"prob\_threshold": 0.2,

	// intersection over union threshold for filtering overlapping
	// detections
	"iou\_threshold": 0.15
}

**Lines 12-16** define the probability and [Intersection over Union (IoU)](https://pyimagesearch.com/2016/11/07/intersection-over-union-iou-for-object-detection/) thresholds so that weak detections may be filtered by our driver script. If you are experiencing too many false positive object detections, you should increase these numbers. **As a general rule, I like to start my probability threshold at `0.5`.**

### Implementing the YOLO and Tiny-YOLO object detection script for the Movidius NCS

We are now ready to implement our Tiny-YOLO object detection script!

Open up the `detect_realtime_tinyyolo_ncs.py` file in your directory structure and insert the following code:

\# import the necessary packages
from openvino.inference\_engine import IENetwork
from openvino.inference\_engine import IEPlugin
from intel.yoloparams import TinyYOLOV3Params
from intel.tinyyolo import TinyYOLOv3
from imutils.video import VideoStream
from pyimagesearch.utils import Conf
from imutils.video import FPS
import numpy as np
import argparse
import imutils
import time
import cv2
import os

We begin on **Lines 2-14** by importing necessary packages; let’s review the most important ones:

*   `openvino`: The `IENetwork` and `IEPlugin` imports allow our Movidius NCS to takeover the TinyYOLOv3 inference.
*   `intel`: The `TinyYOLOv3` and `TinyYOLOV3Params` classes are **provided by Intel Corporation** (i.e., not developed by us) and assist with parsing the TinyYOLOv3 results.
*   `imutils`: The `VideoStream` class is threaded for speedy camera frame capture. The `FPS` class provides a framework for calculating frames per second benchmarks.
*   `Conf`: A class to parse commented JSON files.
*   `cv2`: OpenVINO’s modified OpenCV is optimized for Intel devices.

With our imports ready to go, now we’ll load our configuration file:

\# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add\_argument("-c", "--conf", required=True,
	help="Path to the input configuration file")
ap.add\_argument("-i", "--input", help="path to the input video file")
args = vars(ap.parse\_args())

# load the configuration file
conf = Conf(args\["conf"\])

The [command line arguments](https://pyimagesearch.com/2018/03/12/python-argparse-command-line-arguments/) for our Python script include:

*   `--conf`: The path to the input configuration file that we reviewed in the previous section.
*   `--input`: An _optional_ path to an input video file. If no input file is specified, the script will use a camera instead.

With our configuration path specified, **Line 24** loads our configuration file from disk.

Now that our configuration resides in memory, now we’ll proceed to load our COCO class labels:

\# load the COCO class labels our YOLO model was trained on and
# initialize a list of colors to represent each possible class
# label
LABELS = open(conf\["labels\_path"\]).read().strip().split("\\n")
np.random.seed(42)
COLORS = np.random.uniform(0, 255, size=(len(LABELS), 3))

**Lines 29-31** load our COCO dataset class labels and associate a random color with each label. We will use the colors when it comes to annotating our resulting bounding boxes and class labels.

Next, we’ll load our TinyYOLOv3 model onto our Movidius NCS:

\# initialize the plugin in for specified device
plugin = IEPlugin(device="MYRIAD")

# read the IR generated by the Model Optimizer (.xml and .bin files)
print("\[INFO\] loading models...")
net = IENetwork(model=conf\["xml\_path"\], weights=conf\["bin\_path"\])

# prepare inputs
print("\[INFO\] preparing inputs...")
inputBlob = next(iter(net.inputs))

# set the default batch size as 1 and get the number of input blobs,
# number of channels, the height, and width of the input blob
net.batch\_size = 1
(n, c, h, w) = net.inputs\[inputBlob\].shape

Our first interaction with the OpenVINO API is to initialize our NCS’s Myriad processor and loads the pre-trained TinyYOLOv3 from disk (**Lines 34-38**).

We then:

*   Prepare our `inputBlob` (**Line 42**).
*   Set the batch size to `1` as we will be processing a single frame at a time (**Line 46**).
*   Determine the input volume shape dimensions (**Line 47**).

Let’s go ahead and initialize our camera or file video stream:

\# if a video path was not supplied, grab a reference to the webcam
if args\["input"\] is None:
	print("\[INFO\] starting video stream...")
	# vs = VideoStream(src=0).start()
	vs = VideoStream(usePiCamera=True).start()
	time.sleep(2.0)

# otherwise, grab a reference to the video file
else:
	print("\[INFO\] opening video file...")
	vs = cv2.VideoCapture(os.path.abspath(args\["input"\]))

# loading model to the plugin and start the frames per second
# throughput estimator
print("\[INFO\] loading model to the plugin...")
execNet = plugin.load(network=net, num\_requests=1)
fps = FPS().start()

We query our `--input` argument to determine if we will process frames from a camera or video file and set up the appropriate video stream (**Lines 50-59**).

Due to a bug in Intel’s OpenCV-OpenVINO implementation, if you are using a video file **you must specify the _absolute path_ in the `cv2.VideoCapture` function.** If you do not, OpenCV-OpenVINO will not be able to process the file.

_**Note:** If the `--input` command line argument is not provided, a camera will be used instead. By default, your PiCamera (**Line 53**) is selected. If you prefer to use a USB camera, simply comment out **Line 53** and uncomment **Line 52**._

Our next interaction with the OpenVINO API is to `load` TinyYOLOv3 onto our Movidius NCS (**Line 64**) while **Line 65** starts measuring FPS throughput.

At this point, we’re done with the setup and we can now begin processing frames and performing TinyYOLOv3 detection:

\# loop over the frames from the video stream
while True:
	# grab the next frame and handle if we are reading from either
	# VideoCapture or VideoStream
	orig = vs.read()
	orig = orig\[1\] if args\["input"\] is not None else orig

	# if we are viewing a video and we did not grab a frame then we
	# have reached the end of the video
	if args\["input"\] is not None and orig is None:
		break

	# resize original frame to have a maximum width of 500 pixel and
	# input\_frame to network size
	orig = imutils.resize(orig, width=500)
	frame = cv2.resize(orig, (w, h))

	# change data layout from HxWxC to CxHxW
	frame = frame.transpose((2, 0, 1))
	frame = frame.reshape((n, c, h, w))

	# start inference and initialize list to collect object detection
	# results
	output = execNet.infer({inputBlob: frame})
	objects = \[\]

**Line 68** begins our realtime TinyYOLOv3 object detection loop.

First, we grab and preprocess our `frame` (**Lines 71-86**).

Then, we performs object detection inference (**Line 90**).

**Line 91** initializes an `objects` list which we’ll populate next:

	# loop over the output items
	for (layerName, outBlob) in output.items():
		# create a new object which contains the required tinyYOLOv3
		# parameters
		layerParams = TinyYOLOV3Params(net.layers\[layerName\].params,
			outBlob.shape\[2\])

		# parse the output region
		objects += TinyYOLOv3.parse\_yolo\_region(outBlob,
			frame.shape\[2:\], orig.shape\[:-1\], layerParams,
			conf\["prob\_threshold"\])

To populate our `objects` list, we loop over the `output` items, create our `layerParams`, and parse the output region (**Lines 94-103**). Take note that we are using **Intel-provided code** to assist with parsing our YOLO output.

YOLO and TinyYOLO tend to produce quite a few false-positives. To combat this, next, we’ll devise two weak detection filters:

	# loop over each of the objects
	for i in range(len(objects)):
		# check if the confidence of the detected object is zero, if
		# it is, then skip this iteration, indicating that the object
		# should be ignored
		if objects\[i\]\["confidence"\] == 0:
			continue

		# loop over remaining objects
		for j in range(i + 1, len(objects)):
			# check if the IoU of both the objects exceeds a
			# threshold, if it does, then set the confidence of that
			# object to zero
			if TinyYOLOv3.intersection\_over\_union(objects\[i\],
				objects\[j\]) > conf\["iou\_threshold"\]:
				objects\[j\]\["confidence"\] = 0

	# filter objects by using the probability threshold -- if a an
	# object is below the threshold, ignore it
	objects = \[obj for obj in objects if obj\['confidence'\] >= \\
		conf\["prob\_threshold"\]\]

**Line 106** begins a loop over our parsed `objects` for our **first filter:**

*   We allow only objects with confidence values not equal to zero (**Lines 110 and 111**).
*   Then we actually modify the confidence value (sets it to zero) for any object that does not pass our [Intersection over Union (IoU)](https://pyimagesearch.com/2016/11/07/intersection-over-union-iou-for-object-detection/) threshold (**Lines 114-120**).
*   Effectively, _**objects with a low IoU will be ignored.**_

**Lines 124 and 125** compactly account for our **second filter.** Inspecting the code carefully, these two lines:

*   Rebuild (overwrite) our `objects` list.
*   Effectively, we are _**filtering out objects that do not meet the probability threshold.**_

Now that our `objects` only contain those which we care about, we’ll annotate our output frame with bounding boxes and class labels:

	# store the height and width of the original frame
	(endY, endX) = orig.shape\[:-1\]

	# loop through all the remaining objects
	for obj in objects:
		# validate the bounding box of the detected object, ensuring
		# we don't have any invalid bounding boxes
		if obj\["xmax"\] > endX or obj\["ymax"\] > endY or obj\["xmin"\] \\
			< 0 or obj\["ymin"\] < 0:
			continue

		# build a label consisting of the predicted class and
		# associated probability
		label = "{}: {:.2f}%".format(LABELS\[obj\["class\_id"\]\],
			obj\["confidence"\] \* 100)

		# calculate the y-coordinate used to write the label on the
		# frame depending on the bounding box coordinate
		y = obj\["ymin"\] - 15 if obj\["ymin"\] - 15 > 15 else \\
			obj\["ymin"\] + 15

		# draw a bounding box rectangle and label on the frame
		cv2.rectangle(orig, (obj\["xmin"\], obj\["ymin"\]), (obj\["xmax"\],
			obj\["ymax"\]), COLORS\[obj\["class\_id"\]\], 2)
		cv2.putText(orig, label, (obj\["xmin"\], y),
			cv2.FONT\_HERSHEY\_SIMPLEX, 1, COLORS\[obj\["class\_id"\]\], 3)

**Line 128** extracts the height and width of our original frame. We’ll need these values for annotation.

We then loop over our filtered `objects`. Inside the loop beginning on **Line 131**, we:

*   Check to see if the detected _(x, y)_\-coordinates fall outside the bounds of the original image dimensions; if so, we discard the detection (**Lines 134-136**).
*   Build our bounding box `label` consisting of the object `"class_id"` and `"confidence"`.
*   Annotate the bounding box rectangle and label using the `COLORS` (from **Line 31**) on the output frame (**Lines 145-152**). If the top of the box is close to the top of the frame, **Lines 145 and 146** move the label down by `15` pixels.

Finally, we’ll display our frame, calculate statistics, and clean up:

	# display the current frame to the screen and record if a user
	# presses a key
	cv2.imshow("TinyYOLOv3", orig)
	key = cv2.waitKey(1) & 0xFF

	# if the \`q\` key was pressed, break from the loop
	if key == ord("q"):
		break

	# update the FPS counter
	fps.update()

# stop the timer and display FPS information
fps.stop()
print("\[INFO\] elapsed time: {:.2f}".format(fps.elapsed()))
print("\[INFO\] approx. FPS: {:.2f}".format(fps.fps()))

# stop the video stream and close any open windows1
vs.stop() if args\["input"\] is None else vs.release()
cv2.destroyAllWindows()

Wrapping up, we display the output frame and wait for the `q` key to be pressed at which point we’ll `break` out of the loop (**Lines 156-161**).

**Line 164** updates our FPS calculator.

When either (1) the video file has no more frames, or (2) the user presses the `q` key on either a video or camera stream, the loop exits. At that point, **Lines 167-169** print FPS statistics to your terminal. **Lines 172 and 173** stop the stream and destroy GUI windows.

### YOLO and Tiny-YOLO object detection results on the Raspberry Pi and Movidius NCS

To utilize Tiny-YOLO on the Raspberry Pi with the Movidius NCS, make sure you have:

1.  Followed the instructions in _“Configuring your Raspberry Pi + OpenVINO environment”_ to configure your development environment.
2.  Used the **_**_“Downloads”_**_** section of this tutorial to **download the source code and pre-trained model weights.**

After unarchiving the source code/model weights, you can open up a terminal and execute the following command:

$ python detect\_realtime\_tinyyolo\_ncs.py --conf config/config.json \\
	--input videos/test\_video.mp4
\[INFO\] loading models...
\[INFO\] preparing inputs...
\[INFO\] opening video file...
\[INFO\] loading model to the plugin...
\[INFO\] elapsed time: 199.86
\[INFO\] approx. FPS: 2.66

Here we have supplied the path to an input video file.

Our combination of Raspberry Pi, Movidius NCS, and Tiny-YOLO can apply object detection at the rate of **~2.66 FPS.**

**Video Credit:** _[Oxford University](http://www.robots.ox.ac.uk/ActiveVision/Research/Projects/2009bbenfold_headpose/project.html#datasets)._

Let’s now try using a **camera** rather than a video file, simply by omitting the `--input` [command line argument](https://pyimagesearch.com/2018/03/12/python-argparse-command-line-arguments/):

$ python detect\_realtime\_tinyyolo\_ncs.py --conf config/config.json
\[INFO\] loading models...
\[INFO\] preparing inputs...
\[INFO\] starting video stream...
\[INFO\] loading model to the plugin...
\[INFO\] elapsed time: 804.18
\[INFO\] approx. FPS: 4.28

Notice that processing a camera stream leads to a higher FPS (**~4.28 FPS** versus **2.66 FPS** respectively).

So, why is running object detection on a camera stream _faster_ than applying object detection to a video file?

The reason is quite simple — it takes the CPU more cycles to decode frames from a video file than it does to read a raw frame from a camera stream.

Video files typically apply some level of _compression_ to reduce the resulting video file size.

While the output file size is reduced, the frame still needs to be decompressed when read — the CPU is responsible for that operation.

On the contrary, the CPU has _significantly_ less work to do when a frame is read from a webcam, USB camera, or RPi camera module, hence why our script runs faster on a camera stream versus a video file.

It’s also worth noting that the _fastest_ speed can be obtained using a Raspberry Pi camera module. When using the RPi camera module the onboard _display and stream processing_ GPU (no, not a deep learning GPU) on the RPi handles reading and processing frames so the CPU doesn’t have to be involved.

I’ll leave it as an experiment to you, the reader, to compare USB camera vs. RPi camera module throughput rates.

_**Note:** All FPS statistics collected on RPi 4B 4GB, NCS2 (connected to USB 3.0) and serving an OpenCV GUI window on the Raspbian desktop which is being displayed over VNC. If you were to run the algorithm headless (i.e. no GUI), you may be able to achieve 0.5 or more FPS gains because displaying frames to the screen also takes precious CPU cycles. Please keep this in mind as you compare your results._

### Drawbacks and limitations of Tiny-YOLO

While Tiny-YOLO is **_fast_** and more than capable of running on the Raspberry Pi, **the biggest issue you’ll find with it is _accuracy_** — the smaller model size results in a _substantially_ less accurate model.

For reference, **Tiny-YOLO** achieves only **23.7% mAP** on the COCO dataset while the **larger YOLO models** achieve **51-57% mAP**, well over _double_ the accuracy of Tiny-YOLO.

When testing Tiny-YOLO I found that it worked well in some images/videos, and in others, it was _totally unusable._

Don’t be discouraged if Tiny-YOLO isn’t giving you the results that you want, it’s likely that the model just isn’t suited for your particular application.

Instead, consider trying a more accurate object detector, including:

*   Larger, more accurate YOLO models
*   Single Shot Detectors (SSDs)
*   Faster R-CNNs
*   RetinaNet

For embedded devices such as the Raspberry Pi, I typically always recommend Single Shot Detectors (SSDs) with a MobileNet base. These models are challenging to train (i.e. optimizing hyperparameters), but once you have a solid model, the speed and accuracy tradeoffs are well worth it.

If you’re interested in learning more about these object detectors, my book, **_[Deep Learning for Computer Vision with Python](https://pyimagesearch.com/deep-learning-computer-vision-python-book/)_**, shows you how to train each of these object detectors from scratch and then deploy them for object detection in images and video streams.

Inside of **[_Raspberry Pi for Computer Vision_](https://pyimagesearch.com/raspberry-pi-for-computer-vision/)** you’ll learn how to train MobileNet SSD and InceptionNet SSD object detectors and _deploy the models to embedded devices_ as well.

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

Summary
-------

In this tutorial, you learned how to utilize Tiny-YOLO for near real-time object detection on the Raspberry Pi using the Movidius NCS.

Due to Tiny-YOLO’s small size (< 50MB) and fast inference speed (~244 FPS on a GPU), the model is well suited for usage on embedded devices such as the Raspberry Pi, Google Coral, and NVIDIA Jetson Nano.

Using both a Raspberry Pi and Movidius NCS, we were capable of obtaining **~4.28 FPS**.

I would suggest using the code and pre-trained model provided in this tutorial as a template/starting point for your own projects — extend them to fit your own needs.

**To download the source code and pre-trained Tiny-YOLO model (and be notified when future tutorials are published here on PyImageSearch), _just enter your email address in the form below!_**

![Image 5](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Download the Source Code and FREE 17-page Resource Guide

Enter your email address below to get a .zip of the code and a **FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.** Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!