Raspberry Pi and OpenVino. With the Intel Compute Stick | by David Moore | Towards Data Science

If you have been reading my column here on Towards Data Science, you will know that I am on a mission. I wanted to count the number of cars passing my house using Computer Vision and Motion Detection. This article provides a small update on Computer Vision using the Intel Neural Compute Stick and the OpenVino library.
As a catch-up, I previously built a camera and then explained the need to tune the motion detection device(s). My earlier posts described the camera build and early scripts to review the data and make sense of passing traffic. We had even made a very early chart to plot motion events. Last time I sent 192 images forward through the Yolo model using the Raspberry Pi ARM CPU. There was a lot of heat, and things took a long time. In fact, it took 10 seconds per image. Worse still, I had to slow everything down to only do a shot in 15 seconds. The system over-heated getting to 85% degrees, and I closed, for XMAS, with wanting to run the Neural Compute Stick with that workload to see if things would be any better. My love for Raspberry Pi had started to wear off, and I was feeling a little sad. Let’s get on with updating my code to use the Intel co-processor and just see what happens!
I mentioned I had the Neural Compute Stick in my last article, I therefore went ahead and refactored my class to exploit OpenVino. That would allow me to transfer the CPU workload to a specialist Inference Engine. Since I had committed myself to report back on the effort here is my summary of the exercise.
There is an excellent tutorial on the whole topic over at pyimagesearch.com and plenty written about OpenCV and OpenVINO already such that I won’t need to get into the details here.
www.pyimagesearch.com
The value of using the Object Orientated approach is that the code gets nicely segregated out and is therefore much easier to maintain and update. I only really needed to add a single line of code to my __init__ function. After I load the ‘net’ from the disk files, I can update the Object to set my preferred device as the Intel stick. How cool is that!
self.net.setPreferableTarget(cv2.dnn.DNN_TARGET_MYRIAD)
Naturally, there are all sorts of problems when we take this simple approach. What if the Neural Compute Stick isn’t plugged in? Rather than default to the Raspberry Pi CPU I prefer the script to abend! Obviously, for production workloads, you’d really want to write more robust code! The last run, 192 photos, took 15 seconds per image, and produced a very hot Raspberry Pi, so how did we do?
I couldn’t be happier as it turns out. The co-processor cost me about 100.00 dollars, far less than an Nvidia Titan GPU and is much easier to set-up. I couldn’t be happier, really.
www.amazon.com
So I ran my code once again
my = myYolo()my.run()
This time I removed a 5-second delay to allow things to cool down. I can tell you that the compute stick is a Turbocharger (Supercharger) add-on to the Raspberry Pi for Deep Neural Network inference workloads. Let us examine the results.
Image 0 — took 29 seconds, and that might seem like a lot. But remember that we have the overhead of transferring our model to the Neural Compute stick on the first forward pass. Image 1 took a mere .58, so just half a second to perform a forward pass on the neural network. If we exclude image 0, as an outlier, and get the average of the remaining 191 images, the average inference time is .57 or half a second.
The Intel device performs object detection in .57 seconds whereas the Arm-based CPU for Raspberry Pi 4b took at least 9.5 seconds. Isn’t that incredible! But wait! there is more. Since we are offloading the compute-intensive workload to the Intel device, there is nothing much for the CPU to do. The CPU must mostly handle a bit of I/O between the SSD storage drive, cache memory, and the Neural stick. Therefore there is absolutely no heat produced. We went from >80degrees to a more or less constant 45degrees. That really is amazing. The Intel stick gets warm to the touch, but there is no hum or any noises at all.
My love of the Raspberry Pi is now restored, and I am delighted.
Now it makes no difference, to the accuracy, if you use the Intel Neural Compute stick or the Arm-based CPU. Things just move slower or faster, but the object detection is the system. I reported back on doing Computer Vision operations on the Intel Stick, but I still have my investigation into 100 pictures where Yolo didn’t find an object to complete! Is the data pointing to a need to train Yolo on an Irish data set? Exciting, please do come back! I can now run Yolo in a reasonable time and pass many more photos through the model.