Title: Introduction to OpenCV AI Kit (OAK) - PyImageSearch

URL Source: https://pyimagesearch.com/2022/11/28/introduction-to-opencv-ai-kit-oak/

Published Time: 2022-11-28T14:00:00+00:00

Markdown Content:
* * *

**Table of Contents**
---------------------

*   [Introduction to OpenCV AI Kit (OAK)](#h2BPTitle)

*   [Introduction](#h3Introduction)
*   [OAK Hardware](#h3Hardware)

*   [OAK-1](#h4OAK1)
*   [OAK-D](#h4OAKD)

*   [Limitation](#h5Limitation)

*   [OAK-FFC](#h4FFC)
*   [OAK USB Hardware Offerings](#h4USB)
*   [OAK PoE Hardware Offerings](#h4PoE)
*   [OAK Developer Kit](#h4Developer)
*   [OAK Modules Comparison](#h4Modules)

*   [Applications on OAK](#h3Applications)

*   [Image Classifier On-Device](#h4Classifier)
*   [Face Detection](#h4Detection)
*   [Face Mask Detection](#h4Mask)
*   [People Counting](#h4Counting)
*   [Age and Gender Recognition](#h4Age)
*   [Gaze Estimation](#h4Gaze)
*   [Automated Face-Blurring](#h4Blurring)
*   [Human Skeletal Pose Estimation](#h4Skeletal)
*   [3D Object Detection](#h4Object)
*   [3D Landmark Localization](#h4Landmark)
*   [Semantic Segmentation with Depth](#h4Semantic)

*   [Summary](#h2Summary)

*   [Citation Information](#h3Citation)

* * *

We are super excited to start a new series on the [Luxonis OpenCV AI Kit (OAK)](https://shop.luxonis.com/?aff=10), a one-of-a-kind embedded vision hardware. [OAK](https://shop.luxonis.com/?aff=10) is a one-stop shop solution for deploying computer vision and deep learning solutions since it has multiple cameras and neural network inference accelerators baked right into a coin-size device. It is powered by Intel Movidius Myriad X Visual Processing Unit (VPU) for neural network inference.

[![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%201024%20575'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2023/02/Intro-to-OAK-featured.png)

In today’s tutorial, we will introduce you to the [OAK hardware family](https://shop.luxonis.com/?aff=10) and discuss its capabilities and numerous applications that can run on [OAK](https://shop.luxonis.com/?aff=10). Furthermore, discuss how OAK with stereo depth can simultaneously run advanced neural networks and provide depth information, making it one-of-a-kind embedded hardware in the market. In short, we would like to give you a holistic view of the OAK device and showcase its true potential.

This lesson is the 1st in a 4-part series on **OAK 101**:

1.  [**_Introduction to OpenC_**](https://pyimg.co/dus4w)**_[V](https://pyimg.co/dus4w)_** [**_AI Kit (OAK)_**](https://pyimg.co/dus4w) **(today’s tutorial)**
2.  [_OAK-D: Understanding and Running Neural Network Inference with DepthAI API_](https://pyimg.co/8ynbk)
3.  [_Training a Custom Image Classification Network for OAK-D_](https://pyimg.co/ab1dj)
4.  [_Deploying a Custom Image Classifier on an OAK-D_](https://pyimg.co/tnx0u)

**To learn what [OAK-D](https://shop.luxonis.com/?aff=10) has to offer in computer vision and spatial AI and why it is one of the best embedded vision hardware in the market for hobbyists and enterprises, _just keep reading._**

* * *

[****Introduction to OpenCV AI Kit (OAK)****](#TOC)
---------------------------------------------------

* * *

### [**Introduction**](#TOC)

To celebrate the 20th anniversary of the OpenCV library, [Luxonis](https://www.luxonis.com/) partnered with the official [OpenCV.org](http://opencv.org/) organization to create the OpenCV AI Kit, an MIT-licensed open-source software API, and Myriad X-based embedded board/camera.

In July 2020, Luxonis and the OpenCV organization [launched a Kickstarter campaign](https://www.kickstarter.com/projects/opencv/opencv-ai-kit) to fund the creation of these fantastic embedded AI boards. The campaign was a mega success; it raised roughly $1.3M in funding from 6500+ Kickstarter backers, potentially making it one of the most successful embedded boards in crowdfunding history!

When they first made their product public via Kickstarter, OAK consisted of the OAK API software suite supporting both Python and OpenCV that allowed to program the device and two different types of hardware:

1.  **OAK-1:** The standard OpenCV AI board (shown in **Figure 1** (_left_)) that can perform neural network inference, object detection, object tracking, feature detection, and basic image processing operations.
2.  **OAK-D:** Everything in OAK-1, but with a stereo depth camera, 3D object localization, and tracking objects in a 3D space. OAK-D (shown in **Figure 1** (_right_)) was a flagship product that could simultaneously run advanced neural networks and provide object depth information.

[![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20394'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2022/11/opencv-ai-kit.png)

**Figure 1:** The OpenCV AI Kit (OAK). The OAK-D includes stereo depth (_left_), while the cheaper OAK-1 is a smaller form factor but has no depth capabilities (_right_).

OAK-D provides spatial AI leveraging two monocular cameras in addition to the 4K/30FPS 12MP camera that both models OAK-1 and OAK-D share. OAK-1 and OAK-D are super easy to use, even more so with the DepthAI API. Trust us, if you are a researcher, hobbyist, or professional, you won’t be disappointed, as OAK allows anyone to leverage its power to the fullest.

After a year of launching OAK-1 and OAK-D, in September 2021, Luxonis with OpenCV [launched a second Kickstarter campaign](https://www.kickstarter.com/projects/opencv/opencv-ai-kit-oak-depth-camera-4k-cv-edge-object-detection), and this time it was an OAK variant, OAK-D Lite (as shown in **Figure 2**). As the name suggests, it had a smaller weight and form factor but the same Spatial AI functionality as OAK-D. In addition, while OAK-1 and OAK-D were priced at $99 and $149, respectively, OAK-D Lite was priced at an even lower price than OAK-1 (i.e., just $89).

[![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20656%20380'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2022/11/oak-d-lite.png)

**Figure 2:** OAK-D Lite.

To reduce the cost and smaller form factor, the OAK-D Lite had lower resolution monocular cameras and removed the use of IMU (Inertial measurement unit) as they are costly and rarely used. With many such changes, OAK-D Lite required less power, and you could spin it up with just a USB-C delivering 900mA at 5V power.

The list of OAK variants is not limited to only OAK-1, OAK-D, and OAK-D Lite. Still, many more variants are offered by Luxonis, which targets almost all market segments like developers, researchers, and enterprises for production-grade deployment.

However, all the OAK modules have many standard features, including Neural Network Inference, Computer Vision Methods (Object Tracking, Feature Tracking, Crop, Warp/Dewarp), Onboard Python Scripting, OpenVINO compatibility for deploying deep learning models (as shown in **Figure 3**).

[![Image 4](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20193'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2022/11/oak-features.png)

**Figure 3:** Features offered on all OAK hardware.

* * *

### [**OAK Hardware**](#TOC)

Till now, we have discussed [OAK-1](https://shop.luxonis.com/?aff=10) and [OAK-D](https://shop.luxonis.com/?aff=10) from the family of [OAK hardware](https://shop.luxonis.com/?aff=10), and we also discussed a variant of OAK (i.e., OAK-D Lite). In this section, we will dive deeper into what all Luxonis has to offer apart from OAK-1, OAK-D, and OAK-D Lite, as they segregate all their hardware offerings into three categories: family, variant, and connectivity, as shown in **Figure 4**.

[![Image 5](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20239'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2022/11/oak-family-tree.png)

**Figure 4:** OAK Hardware Offerings.

As shown in the above figure, the OAK family of modules offers three types of hardware; all have the same onboard Myriad X VPU for Neural Network Inference and offer a set of standard features discussed in **Figure 3**.

If you want to learn more about Intel’s Myriad X VPU, check out [this product document](https://newsroom.intel.com/wp-content/uploads/sites/11/2017/08/movidius-myriad-xvpu-product-brief.pdf).

* * *

#### [**OAK-1**](#TOC)

It comes packaged with a 12MP RGB Camera supporting USB-based connectivity and Power-over-Ethernet (PoE) connectivity. The Myriad X VPU chip allows it to perform 4-trillion-operations-per-second onboard, keeping the host (computer) free. Since the device has a 12 MP camera and the neural inference chip onboard, the communication happens super fast over an onboard 2.1 Gbps MIPI interface. As a result, the overall inference speed is more than six times that compared to the standard setting (a Neural Compute Stick 2 connected to a computer with USB), where the host deals with the video stream. **Figure 5** shows an NCS2 connected to a laptop through a USB.

[![Image 6](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20394'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2022/11/NCS2-with-host.jpeg)

**Figure 5:** Intel’s Neural Compute Stick 2 connected to a Laptop through a USB ([source](https://jonamiki.com/tag/ncs2/)).

Following are the numbers achieved when an object detection network (MobileNet-SSD) is run on OAK-1 viz-a-viz NCS2 with host as Raspberry Pi:

*   OAK-1 + Raspberry Pi: 50+FPS, 0% RPi CPU Utilization
*   NCS2 + Raspberry Pi: 8FPS, 225% RPi CPU Utilization

* * *

#### [**OAK-D**](#TOC)

Like OAK-1, the OAK-D module comes with a 12 MP RGB color camera and supports USB and PoE connectivity. Along with the RGB camera, it has a pair of grayscale stereo cameras that help perform depth perception. All three onboard cameras implement stereo and RGB vision, piped directly into the OAK System on Module (SoM) for depth perception and AI inference.

One interesting fact that caught our attention was the distance between the left, and the right grayscale stereo camera is 7.5 cm, similar to the distance between the center of the pupils of the two eyes. This distance estimation works well for most applications. The minimum perceivable distance is 20 cm, and the maximum perceivable distance is 35 meters.

* * *

##### [**Limitation**](#TOC)

The stereo cameras in OAK-D have a baseline of 7.5 cm, denoted as `BL` in **Figure 6**. The grayscale stereo camera pair has an 89.5° Field of View (FOV), and since both the cameras are separated by 7.5 cm, they will have a specific blind spot zone. And only one of the two cameras would be able to see in this blind spot zone.

Since we need to compute the depth from disparity (depth is inversely proportional to disparity), which requires the pixels to overlap, there are blind spots. For example, areas on the left side of the left mono camera and on the right side of the right mono camera, where depth cannot be calculated since it is seen by only one of two cameras. Therefore, in **Figure 6**, that area is marked with `` `B` `` (i.e., the width of the band where depth cannot be calculated).

[![Image 7](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20580%20500'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2022/11/oak-d-limitation.png)

**Figure 6:** The image depicts the stereo camera’s baseline, a band where depth cannot be computed, the minimum distance at which depth can be calculated, and the amount of horizontal pixels covered by a mono camera.

* * *

#### [**OAK-FFC**](#TOC)

The OAK FFC modular lineup, as shown in **Figure 7**, is for developers and practitioners requiring flexibility in the hardware setup, and the standard OAK-D doesn’t fit well with their application. They are more for applications that require custom mounting, custom baseline (the distance between the left and right camera), or custom orientation of the cameras. The modularity allows you to place the stereo cameras at various baselines, unlike in OAK-D, which is set to 7.5 cm.

[![Image 8](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20498'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2022/11/oak-ffc.png)

**Figure 7:** OAK-FFC Modular Camera Designs.

As the saying goes, “Nothing worth having comes easy,” which means that with this flexibility comes a challenge that you need to figure out how and where you will mount the cameras, and once mounted, you will need to perform a [stereo calibration](https://docs.luxonis.com/en/latest/pages/calibration/). Of course, all of this groundwork is already done for you in OAK-D, so it’s essential to remember that these OAK-FFC modules aren’t “plug and play” like OAK-1, OAK-D, OAK-D Lite, etc. So, in short, you need to be a pro in working on these modular devices or have some experience setting up custom hardware to leverage the power of OAK-FFC.

* * *

#### [**OAK USB Hardware Offerings**](#TOC)

The OAK modules first brought to the market through a [Kickstarter](https://www.kickstarter.com/projects/opencv/opencv-ai-kit) campaign were the OAK with USB connectivity, which are also the most popular modules. OAK-1, OAK-D, and OAK-D Lite have solved real-world problems for over two years. USB connection is great for development – it’s easy to use and allows up to 10Gbps throughput.

[![Image 9](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20380'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2022/11/oak-usb-designs.png)

**Figure 8:** OAK-1 and OAK-D USB Hardware Designs.

The more recent variants of OAK, like the OAK-D Pro, have active stereo and night vision capabilities, making it an even more promising edge device in the market. It is often difficult to find a camera that supports night vision, let alone run the neural network inference in that scenario. However, what’s better than having all these capabilities baked into one device that perceives low-light environments and then performs neural network inference on those frames (as shown in **Figure 9**)?

[![Image 10](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20219'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2022/11/oak-night-vision.png)

**Figure 9:** Image captured in a low-light environment with an OAK-D (_left_) and an OAK-D Pro (_right_).

* * *

#### [**OAK PoE Hardware Offerings**](#TOC)

OAK PoE modules are very similar to USB modules, but instead of USB connectivity, they have PoE connectivity. The PoE cables have a much longer range than USB. Another critical difference is that they also feature onboard flash so you can run pipelines in [standalone mode](https://docs.luxonis.com/projects/api/en/latest/tutorials/standalone_mode/). This means you do not need to connect your OAK to a host, and the OAK PoE can run inference hostless or on-the-edge.

The PoE devices have IP67-rated enclosures, which means its waterproof and dustproof.

[![Image 11](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20473'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2022/11/oak-poe-designs.png)

**Figure 10:** OAK-1 and OAK-D PoE Hardware Designs.

* * *

#### [**OAK Developer Kit**](#TOC)

The OAK developer kit is our favorite of all the variants that Luxonis offers. In simple terms, the developer kit is like combining a Raspberry Pi (or any microprocessor like Nvidia’s Jetson Nano) with an OAK-D in a compact solution. The microprocessor acts as a host for the OAK-D and makes it fully standalone.

**Figure 11** shows three variants: the OAK-D-CM3, OAK-D-CM4, and OAK-D CM4 PoE. The latest Compute Module 4 variant integrates the Raspberry Pi Compute Module 4 (Wireless, 4 GB RAM) and all of its interfaces (USB, Ethernet, 3.5 mm Audio, etc.). In addition, the Raspberry Pi OS is also flashed into the 32 GB eMMC storage. Finally, to connect, configure, build pipelines, and communicate with the OAK device, it has a DepthAI Python interface.

[![Image 12](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20246'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2022/11/oak-devkit.png)

**Figure 11:** OAK-D Compute Module Variants.

* * *

#### [**OAK Modules Comparison**](#TOC)

**Table 1** shows all the OAK variants with the features and functionalities each offers. As you can observe from the below table, all the OAK models support AI and CV functionalities since all modules come with at least one camera and a Myriad X chip. In addition, all devices except OAK-1 variants support Spatial data (i.e., depth perception).

All OAK variants with PoE-based connectivity also have an IP67 rating, as evident from the below table. In addition, the OAK-D Pro USB and PoE have Active stereo and Night vision support, taking computer vision even one notch higher.

Recall the OAK with PoE has onboard flash so that it can run without a host. Similarly, the ones with Raspberry Pi Compute Modules already have the host. Hence, both PoE and CM variants support standalone mode. Additionally, the CM modules have a Coprocessor (i.e., the Raspberry Pi CM 4).

[![Image 13](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20568%20362'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2022/10/image.png)

**Table 1:** OAK Modules Comparison.

* * *

### [**Applications on OAK**](#TOC)

In this section, we will discuss a few of the applications that can be run out-of-the-box on OAK-1, while some require OAK-D.

* * *

#### [**Image Classifier On-Device**](#TOC)

Image Classification is like the “hello world” of Deep Learning for Computer Vision. Like most edge devices, you can also efficiently run image classification models on the OAK device. **Figure 12** shows an example of running an Efficientnet-b0 variant which classifies the animal as Ibex (Mountain Goat) with a probability of 34.46%. This Efficientnet classifier is trained on a 1000-class ImageNet dataset and can be loaded on-the-fly in DepthAI API.

[![Image 14](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20300%20300'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2022/11/image-classification.jpeg)

**Figure 12:** Image Classified as Ibex (Mountain Goat).

* * *

#### [**Face Detection**](#TOC)

**Figure 13** shows the face detection application demonstration on an OAK device based on the implementation of [libfacedetection](https://github.com/ShiqiYu/libfacedetection). It detects the faces in images and five facial landmarks: a pair of eyes, nose tip, and mouth corner points. The detector achieves 40 FPS on the OAK device.

[![Image 15](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)](https://lh4.googleusercontent.com/Irs66Kcv4rA_z3hpuXTxLcLMeT6OdZYZjypIW8-hOYwmbSijh-FHse-kKiPgJUxNb3IvIMMDY4ZqoR8h9d1NCOEUTgpDpAfuRr3OvYRpYiTbaFQFInFsQO3UzJ9DpbxxkaE-eVyTlXL3FJ6KSy-y3a4Y_0IE8OFLKiH6kV89CAb2SQdkVZLKUa1gUA)

**Figure 13:** Demonstration of face detection and facial landmark detection.

* * *

#### [**Face Mask Detection**](#TOC)

**Figure 14** shows a face mask/no-mask detector that uses the MobileNet SSD v2 object detection model to detect whether people are wearing a mask or not. As you can see from the below gif, the model does quite well. This was, of course, run on the OAK-D. You can read more about this [here](https://github.com/luxonis/depthai-experiments/tree/master/gen2-coronamask).

[![Image 16](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)](https://lh6.googleusercontent.com/nq4fo4hfd0If8jmZ0fPETnH2_pwM4lH3jH0hjcA4M-zpARYX76adAif3PVIWTl3fczDgF7OwtooYDawkab9URM4eY8iIS0Ri7Q7z48eCon3A9qAvGeGG8p-5E16p7XzdWbA2dFr0fXVqUz96mjdD58cLxItouEoAVxh92rYH1elIx-YJpRKJ0ovxRA)

**Figure 14:** Face Mask Detection Gif.

* * *

#### [**People Counting**](#TOC)

Object counting is a two-step process that involves detecting the objects and then counting the unique occurrences with the help of an object tracker. **Figure 15** shows a demonstration of counting people that are going left and right. It is inspired by the PyImageSearch blog post, [OpenCV People Counter](https://pyimagesearch.com/2018/08/13/opencv-people-counter/). People counting can be beneficial in scenarios when you need to count the number of people entering and exiting a mall or a department store. Refer to the [GitHub repository](https://github.com/luxonis/depthai-experiments/tree/master/gen2-cumulative-object-counting) to know more about object tracking for OAK.

[![Image 17](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20630%20394'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2022/11/cumulative-object-counting.png)

**Figure 15:** People Counting.

* * *

#### [**Age and Gender Recognition**](#TOC)

**Figure 16** shows a great example of running two-stage inference with the DepthAI library on OAK-D. It first detects the faces in a given frame, crops them, and then sends those faces to an age-gender classification model. The face detection and age gender models are available on OpenVINO’s GitHub Repository inside [Intel’s directory](https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/intel). You can also refer to [this](https://docs.openvino.ai/2021.4/omz_models_group_intel.html) official OpenVINO docs for pre-trained models.

[![Image 18](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)](https://lh3.googleusercontent.com/PEjc0ayTyFDQ-6oeS7Mcnzt6zCULfPDJXXNR7qZ6sXqoX72B79XiejmYUWNEHxySG6-61Lh4HWss76V7ntQ_tVKnlktPOVUloGNeax7AYwEhHrfl6U6VSztsYJRDaJBh6nsD9KsODf7l9ZEjS__JWeFp34Az_Qf3OHBmYxZSc3LNEx8r5KcZdhPxBg)

**Figure 16:** Two-stage pipeline for face detection and age and gender classification on OAK.

* * *

#### [**Gaze Estimation**](#TOC)

Gaze Estimation is a bit complex example but an interesting one. It predicts where a person looks, given the person’s entire face. As shown in **Figure 17**, the two gaze vectors move based on the person’s eye movement.

This example shows the capability of running more complex applications on the OAK that involves running not 1 but 4-5 different neural network models. Like the previous example was a two-stage inference pipeline, the gaze estimation is three-stage inference (3-stage, 2-parallel) with DepthAI. You can learn more about it on the [luxonis repository](https://github.com/luxonis/depthai-experiments/tree/master/gen2-gaze-estimation).

[![Image 19](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20350%20350'%3E%3C/svg%3E)](https://lh5.googleusercontent.com/dd7nlEaJ3wOXMdIxY6w5mVK4P6mvCGDllc5s9DGSI4BMu5r2bJidNgT72wK6SXd2kSck0FpgIQzSXBFZVOYFO2f10YoBdC-Y7aBJfBMh1qhNnjchj2eQwio0pjcNkLoUIfdIcP8cSgfHyrh-z3tbexhbzkaJZYxYp4w8EnZfCYc4K7xbwV7OkGxLxQ)

**Figure 17:** Gaze Estimation.

* * *

#### [**Automated Face-Blurring**](#TOC)

Another bit simpler example of running a pre-trained face detection neural network on the OAK device to extract the face region of interest from a given image. Then the face regions are blurred using standard OpenCV methods on the host. The end-to-end application shown in **Figure 18** runs in real-time.

![Image 20](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)

**Figure 18:** Automate Face-Blurring.

* * *

#### [**Human Skeletal Pose Estimation**](#TOC)

**Figure 19** shows an example of a Human Pose Estimation Network using Gen2 Pipeline Builder on an OAK device. This is a multi-person 2D pose estimation network (based on the [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose) approach). The network detects a human pose for every person in an image: a body skeleton consisting of key points and connections between them. For example, the pose may contain 18 key points like ears, eyes, nose, neck, shoulders, elbows, etc. You can load the human pose estimation model off-the-shelf with the `blobconverter` module that helps convert the OpenVINO to MyriadX blob format.

[![Image 21](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)](https://lh4.googleusercontent.com/Ud0hXQSbzlFNqTSxnCGdBuZ4YHgWlapQKMr-EzBZF4COWPg7uZBO-YrgqU2nwfCVA-Vs_eFFbNF2_ZmLQhxN7rxFR9EuGUOarqXVwKu3nk3VKYrDhYF7CKBy3Kn6n_QtQ4d5Vk3L8io9fBwNSuRkWkL910ZwuOQdZXAgVjcLyfAOT6GHIdhxRWo3KA)

**Figure 19:** Human Pose Estimation on OAK-D.

* * *

#### [**3D Object Detection**](#TOC)

Object detection with depth requires an OAK-D; it would not work on OAK-1 devices, and the following two examples you would see also require an OAK-D.

In simple terms, 3D object detection is about finding an object’s location/coordinates in physical space instead of pixel space (2D). Hence, it is useful when measuring or interacting with the physical world in real time. **Figure 20** shows an example of Neural inference fused with stereo depth where the DepthAI extends the 2D object detectors like SSD and YOLO with spatial information to give them 3D context.

[![Image 22](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%201024%20283'%3E%3C/svg%3E)](https://pyimagesearch.com/wp-content/uploads/2022/11/3d-object-detection.png)

**Figure 20:** Object Detection with Depth.

So in the below image, a MobileNet detects objects and fuses object detections with a depth map to provide spatial coordinates (XYZ) of objects it sees: person, monitor, potted plant, bottle, and chair. If you notice carefully, the `` `z` `` for a bottle is lesser than the person, and the `` `z` `` for the monitor is the largest. And `` `z` `` is the distance of the object from the camera.

* * *

#### [**3D Landmark Localization**](#TOC)

The 3D hand landmark detector is very similar to the previous example. With a regular camera, this network returns the 2D (XY) coordinates of all 21 hand landmarks (contours of all joints in fingers). Then, using this same concept of neural inference fused with stereo depth, these 21 hand landmarks are converted to 3D points in physical space, as shown in **Figure 21**. The application achieves roughly 20 FPS on the OAK-D.

![Image 23](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)

**Figure 21:** Hand Landmark Detection with Depth.

* * *

#### [**Semantic Segmentation with Depth**](#TOC)

Like image classification, object detection, and keypoint detection, you can also run semantic segmentation on OAK-D. **Figure 22** demonstrates an example that runs DeepLabv3+ at 27 FPS with DepthAI API. It further crops the depth image based on the semantic segmentation model output. The first window is the semantic segmentation output, the second is a cropped depth map based on the semantic segmentation output, and the third is the depth map.

![Image 24](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%200%200'%3E%3C/svg%3E)

**Figure 22:** Person Semantic Segmentation with Depth Image Cropping.

* * *

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

* * *

[**Summary**](#TOC)
-------------------

In this tutorial, we introduced you to the OAK hardware family and discussed its capabilities and numerous applications that can run on the OAK device.

Specifically, we discussed in detail the two flagship products of Luxonis, i.e., OAK-1 and OAK-D, and how they got traction from the market with the help of a Kickstarter campaign. We also discussed the limitation of OAK-D.

We further delved into the Luxonis hardware offerings, where we discussed the OAK-FFC, OAK USB offerings, OAK PoE hardware, the developer kit, and the OAK complete modules.

Finally, we discussed various computer vision applications that can be run on the OAK hardware.

* * *

### [**Citation Information**](#TOC)

**Sharma, A.** “Introduction to OpenCV AI Kit (OAK),” _PyImageSearch_, P. Chugh, A. R. Gosthipaty, S. Huot, K. Kidriavsteva, R. Raha, and A. Thanki, eds., 2022, [https://pyimg.co/dus4w](https://pyimg.co/dus4w)

@incollection{Sharma\_2022\_OAK1,
  author = {Aditya Sharma},
  title = {Introduction to {OpenCV AI} Kit {(OAK)}},
  booktitle = {PyImageSearch},
  editor = {Puneet Chugh and Aritra Roy Gosthipaty and Susan Huot and Kseniia Kidriavsteva and Ritwik Raha and Abhishek Thanki},
  year = {2022},
  note = {https://pyimg.co/dus4w},
}

* * *

![Image 25: Featured Image](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20276'%3E%3C/svg%3E)

### Unleash the potential of computer vision with Roboflow - Free!

*   Step into the realm of the future by [signing up or logging into your Roboflow account](https://roboflow.com/?ref=pyimagesearch). Unlock a wealth of innovative dataset libraries and revolutionize your computer vision operations.
*   Jumpstart your journey by choosing from our broad array of datasets, or benefit from PyimageSearch’s comprehensive library, crafted to cater to a wide range of requirements.
*   Transfer your data to Roboflow in any of the 40+ compatible formats. Leverage cutting-edge model architectures for training, and deploy seamlessly across diverse platforms, including API, NVIDIA, browser, iOS, and beyond. Integrate our platform effortlessly with your applications or your favorite third-party tools.
*   Equip yourself with the ability to train a potent computer vision model in a mere afternoon. With a few images, you can import data from any source via API, annotate images using our superior cloud-hosted tool, kickstart model training with a single click, and deploy the model via a hosted API endpoint. Tailor your process by opting for a code-centric approach, leveraging our intuitive, cloud-based UI, or combining both to fit your unique needs.
*   Embark on your journey today with absolutely no credit card required. Step into the future with Roboflow.

[Join Roboflow Now](https://roboflow.com/?ref=pyimagesearch)

* * *

![Image 26](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Join the PyImageSearch Newsletter and Grab My FREE 17-page Resource Guide PDF

Enter your email address below to **join the PyImageSearch Newsletter** and **download my FREE 17-page Resource Guide PDF** on Computer Vision, OpenCV, and Deep Learning.