Title: A Better, Faster, and Stronger Object Detector (YOLOv2) - PyImageSearch

URL Source: https://pyimagesearch.com/2022/04/18/a-better-faster-and-stronger-object-detector-yolov2/

Published Time: 2022-04-18T13:00:00+00:00

Markdown Content:
* * *

Table of Contents
-----------------

*   [A Better, Faster, and Stronger Object Detector (YOLOv2)](#h2Blog)

*   [Configuring Your Development Environment](#h3Config)
*   [Having Problems Configuring Your Development Environment?](#h3Having)
*   [Introduction to YOLOv2](#h3Intro)
*   [Better](#h3Better)

*   [Batch Normalization](#h4Batch)
*   [High-Resolution Classifier](#h4High)
*   [Convolution with Anchor Boxes](#h4Convolution)
*   [Dimension Clusters](#h4Dimension)
*   [Direct Location Prediction](#h4Direct)
*   [Fine-Grained Features](#h4Fine)
*   [Multi-Scale Training](#h4Multi)

*   [Faster](#h3Faster)

*   [Darknet-19](#h4Darknet-19)
*   [Training for Classification](#h4Classification)
*   [Training for Detection](#h4Detection)

*   [Stronger](#h3Stronger)

*   [Hierarchical Classification](#h4Hierarchical)

*   [Joint Classification and Detection](#h3Detection)

*   [Evaluating the YOLO9000](#h4YOLO9000)

*   [Configuring the Darknet Framework and Running Inference with the Pretrained YOLOv2 Model](#h3ConfigDarknet)

*   [Summary](#h2Summary)

*   [Citation Information](#h3Citation)

* * *

Detection frameworks have become increasingly fast and accurate, as seen in our last post on [YOLOv1](https://pyimg.co/3cpmz); however, most detection methods are still constrained to a small set of objects like 20 classes in PASCAL VOC and 80 classes in Microsoft COCO.

![Image 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20393'%3E%3C/svg%3E)

The image classification datasets have many object labels and are far easier and more inexpensive to annotate than detection. But the hope is to scale detection to the level of object classification.

And that’s partly the motivation of [Redmon and Farhadi (2017)](https://openaccess.thecvf.com/content_cvpr_2017/papers/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.pdf) behind publishing this paper.

The other primary motivation was to fix YOLOv1’s issues:

*   Detecting small objects in groups
*   The localization error
*   The use of anchor boxes (priors)
*   Introducing accurate and efficient novel Darknet-19 architecture

In this tutorial, you will learn all about the YOLOv2 Object Detection Model and why it is called Better, Faster, Stronger. Then, we will discuss all the essential concepts mentioned in the [paper](https://openaccess.thecvf.com/content_cvpr_2017/papers/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.pdf) in detail. Finally, run the pretrained YOLOv2 model on images and video and see it perform better than its predecessor, YOLOv1.

If you haven’t read our previous blog post on [YOLOv1](https://pyimg.co/3cpmz), we highly recommend you read that first, as understanding this post will become easier and enhance your learning experience.

This lesson is the third in our 7-part series on YOLO:

1.  [_Introduction to the YOLO Family_](https://pyimg.co/dgbvi)
2.  [_Understanding a Real-Time Object Detection Network: You Only Look Once (YOLOv1)_](https://pyimg.co/3cpmz)
3.  [_A Better, Faster, and Stronger Object Detector (YOLOv2)_](https://pyimg.co/ehaox) **(this tutorial)**
4.  [_Mean Average Precision (mAP) Using the COCO Evaluator_](https://pyimg.co/nwoka)
5.  [_An Incremental Improvement with Darknet-53 and Multi-Scale Predictions (YOLOv3)_](https://pyimg.co/8xfpg)
6.  [_Achieving Optimal Speed and Accuracy in Object Detection (YOLOv4)_](https://pyimg.co/c6kiu)
7.  [**_Training the YOLOv5 Object Detector on a Custom Dataset_**](https://pyimg.co/fq0a3)

Today’s post will discuss YOLO9000, more commonly known as YOLOv2, which detects objects much faster than the already fast YOLOv1 and achieves a 13-16% gain in mAP.

**To learn the theoretical concepts of the YOLOv2 object detector and see a demo of detecting objects in real-time, _just keep reading._**

* * *

[**A Better, Faster, and Stronger Object Detector (YOLOv2)**](#TOC)
-------------------------------------------------------------------

In this third part of the YOLO series, we will start with an introduction to YOLOv2.

We will discuss the **Better** part of YOLOv2:

*   High-resolution classifier
*   Batch Normalization
*   Use of anchor boxes
*   Dimension clusters
*   Direct location prediction
*   Fine-grained features
*   Multi-scale resolution training
*   Quantitative benchmarks, comparing YOLOv2 with YOLOv1, Faster R-CNN, and SSD

From there, we’ll discuss the **Faster** aspect of YOLOv2:

*   Darknet-19 network architecture
*   Training for Detection
*   Training for Classification

We will also discuss the **Stronger** aspect that formed the basis for YOLO9000:

*   Hierarchical Training
*   Combining ImageNet and MS COCO with word tree
*   Joint Training for classification and detection

Finally, we will wrap up this tutorial by installing the darknet framework on the Tesla V100 GPU and running inference on images and video with the YOLOv2 pretrained model.

* * *

### [**Configuring Your Development Environment**](#TOC)

To follow this guide, you need to have the Darknet Framework compiled and installed on your system. We will use [AlexeyAB’s Darknet Repository](https://github.com/AlexeyAB/darknet) for this tutorial.

We cover the step-by-step instructions on how to install the darknet framework on Google Colab. However, if you would like to configure your development environment now, consider heading to the [Configuring the Darknet Framework and Running Inference with the Pretrained YOLOv2 Model](#h3ConfigDarknet) section.

* * *

### [**Having Problems Configuring Your Development Environment?**](#TOC)

[![Image 2](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20334'%3E%3C/svg%3E)](https://pyimagesearch.com/pyimagesearch-university/)

**Figure 1:** Having trouble configuring your dev environment? Want access to pre-configured Jupyter Notebooks running on Google Colab? Be sure to join [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/) — you’ll be up and running with this tutorial in a matter of minutes.

All that said, are you:

*   Short on time?
*   Learning on your employer’s administratively locked system?
*   Wanting to skip the hassle of fighting with the command line, package managers, and virtual environments?
*   **Ready to run the code** **_right now_** **on your Windows, macOS, or Linux system?**

Then join [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/) today!

**Gain access to Jupyter Notebooks for this tutorial and other PyImageSearch guides that are** **_pre-configured_** **to run on Google Colab’s ecosystem right in your web browser!** No installation required.

And best of all, these Jupyter Notebooks will run on Windows, macOS, and Linux!

* * *

### [**Introduction to YOLOv2**](#TOC)

In 2017, Joseph Redmon (a Graduate Student at the University of Washington) and Ali Farhadi (a PRIOR team lead at the Allen Institute for AI) published the [YOLO9000: Better, Faster, Stronger](https://openaccess.thecvf.com/content_cvpr_2017/papers/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.pdf) paper at the CVPR conference. The authors proposed two _state-of-the-art_ YOLO variants in this paper: YOLOv2 and YOLO9000; both were identical but differed in training strategy.

YOLOv2 was trained on standard detection datasets like [PASCAL VOC](http://host.robots.ox.ac.uk/pascal/VOC/) and [MS COCO](https://cocodataset.org/#home). At the same time, the YOLO9000 was designed to predict more than 9000 different object categories by jointly training it on the MS COCO and [ImageNet](https://www.image-net.org/index.php) datasets.

**Figure 2** shows the output of the YOLO9000 model that can detect more than 9000 object classes in real-time. The output below shows that the model has learned to detect objects not in the MS COCO dataset.

![Image 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20230%20500'%3E%3C/svg%3E)

**Figure 2:** YOLO9000 model can detect a wide variety of object classes in real-time (image source: [Redmon and Farhadi](https://openaccess.thecvf.com/content_cvpr_2017/papers/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.pdf), p. 7263, fig. 1).

YOLOv2 is the second version in the YOLO family, significantly improving accuracy and making it even faster.

The improved YOLOv2 model used various novel techniques to outperform state-of-the-art methods like Faster-RCNN and SSD in both speed and accuracy. One such technique was multi-scale training that allowed the network to predict at varying input sizes, thus allowing a trade-off between speed and accuracy.

At ![Image 4: 416 \times 416](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2069%2013'%3E%3C/svg%3E) input resolution, YOLOv2 achieved 76.8 mAP on VOC 2007 dataset and 67 FPS on Titan X GPU. On the same dataset with ![Image 5: 544 \times 544](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2069%2014'%3E%3C/svg%3E) input, YOLOv2 attained 78.6 mAP and 40 FPS.

They also proposed the YOLO9000 model trained on the COCO detection dataset with the ImageNet classification dataset. The idea behind this type of training was to detect object classes that did not have ground truth for object detection but use supervision from object classes with ground truth. The domain for such training is referred to as **weakly supervised** learning. This approach helped them achieve 16 mAP on 156 classes that did not have detection ground truth. Isn’t that great!

* * *

### [**Better**](#TOC)

In our [previous post](https://pyimg.co/3cpmz), we learned that YOLOv1 has a few limitations. However, it primarily suffers from localization errors that result in low recall compared to two-stage object detectors. We also observed in the inference section that the model detected the objects with not very high confidence scores.

The goal of YOLOv2 was to reduce the localization errors, thereby increasing the recall while maintaining and outperforming classification accuracy. [Redmon and Farhadi](https://openaccess.thecvf.com/content_cvpr_2017/papers/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.pdf)’s idea was to develop an object detector that was more accurate than its predecessors and faster than them. Hence, building larger, deeper networks like ResNet or ensembling various networks did not fit the bill. Instead, with a simplified network architecture approach, they focussed on pooling many ideas from past work combined with their novel techniques. As a result, they improved YOLO’s performance in terms of both speed and accuracy.

Let’s now look at the components that made YOLO perform **Better**:

* * *

#### [**Batch Normalization**](#TOC)

*   Adding a batch normalization layer in all of the convolutional layers in YOLO improved the mAP by **2%**.
*   It helped improve the network training convergence and eliminated the need for other regularization techniques like dropout without the network getting overfitted on the training data.

* * *

#### [**High-Resolution Classifier**](#TOC)

![Image 6](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20324'%3E%3C/svg%3E)

**Figure 3:** High-Resolution Classifier Pipeline ([source](https://docs.google.com/presentation/d/14qBAiyhMOFl_wZW4dA1CkixgXwf0zKGbpw_0oHK8yEM/edit#slide=id.g23eff386b7_0_24)).

*   Finally, we fine-tuned the network for the detection task, and the high-resolution classifier approach increased the mAP by close to **4%**. And trust me, a gain of 4% in mAP is a considerable boost.

* * *

#### [**Convolution with Anchor Boxes**](#TOC)

*   YOLOv1 was an anchor-free model that predicted the coordinates of B-boxes directly using fully connected layers in each grid cell.
*   Inspired by Faster-RCNN that predicts B-boxes using hand-picked priors known as anchor boxes, YOLOv2 also works on the same principle.
*   YOLOv2 removes the fully connected layers and uses anchor boxes to predict bounding boxes. Hence, making it fully convolutional.
*   But what are Anchor Boxes? Anchor boxes are a set of predefined boxes with a specific height and width; they act as a guess or prior for the objects in the dataset that help optimize the network fast. They are multiple bounding boxes (priors) with different aspect ratios and sizes centered on each pixel. The goal of an object detection network is to predict a bounding box and its class label. Bounding boxes are usually represented in a normalized xmin, ymin, xmax, ymax format.For example, 0.5 xmin and 0.5 ymin mean the top-left corner of the box is in the middle of the image. So, intuitively, we are dealing with a regression problem to get a numeric value like 0.5.
    
    We may have the network predict four values and use Mean Square Error to compare the ground truth. However, due to the significant variance of scale and aspect ratio of boxes, researchers found that it’s tough for the network to converge if we use this “brute force” way to get a bounding box. Hence, in the Faster-RCNN paper, the idea of an anchor box was proposed.
    

*   Unlike YOLOv1, wherein each grid cell, the model predicted one set of class probabilities per grid cell, ignoring the number of boxes B, YOLOv2 predicted class and objectness for every anchor box.

*   Anchor boxes slightly decrease mAP, from 69.5mAP to 69.2mAP but increase the recall from 81% to 88%, meaning that the model has more room to improve.

*   YOLOv1 predicted 98 boxes per image, but YOLOv2 with anchor boxes can predict 845 boxes (![Image 7: 13 \times 13 \times 5](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2080%2014'%3E%3C/svg%3E)) per image and even more than a thousand based on the grid size.

* * *

#### [**Dimension Clusters**](#TOC)

*   The authors observed that the cluster centroids or anchors were significantly different compared to hand-picked ones. This new approach had more tall and thin boxes.
*   **Table 1** represents three types of prior generation strategy: Cluster Sum-Squared, Cluster IOU, Anchor Boxes (Hand-Picked Priors). We can see that the Cluster IOU with 5 anchors does a good job with an average IOU of 61.0 and performs similarly to hand-picked anchors. Further, if we increase the priors, we see a jump in the average IOU. This study concluded that using ![Image 8: k](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%209%2011'%3E%3C/svg%3E)\-means to generate the bounding box starts the model with a better representation and makes the network’s job easier.

![Image 9](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20600%20156'%3E%3C/svg%3E)

**Table 1:** Average IOU of objects on VOC 2007 to their closest anchor (prior) using different box generation methods (source: [Redmon and Farhadi (2017)](https://openaccess.thecvf.com/content_cvpr_2017/papers/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.pdf), p. 7265, table 1).

* * *

#### [**Direct Location Prediction**](#TOC)

In YOLOv1, we directly predicted the center ![Image 10: (x, y)](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2035%2018'%3E%3C/svg%3E) locations for the bounding box, which caused model instability, especially during the early iterations. Furthermore, since in YOLOv1, there was no concept of priors, directly predicting box locations led to a more significant loss as the model had no idea about the objects in the dataset.

However, in YOLOv2, with the concept of anchors, we still follow the approach of YOLOv1 and predict location coordinates relative to the location of the grid cell, but the model outputs the offsets. These offsets tell how far the priors are from the ground-truth bounding boxes. This formulation allows the model to not diverge from the center location too much. Thus, instead of predicting the direct coordinates ![Image 11: [\text{xmin}, \text{ymin}, \text{xmax}, \text{ymax}]](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20177%2018'%3E%3C/svg%3E), we predict offsets to these bounding boxes during the training. This works because our ground-truth box should look like the anchor box we pick with ![Image 12: k](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%209%2011'%3E%3C/svg%3E)\-means clustering, and only subtle adjustment is needed, which gives us a good head start in training.

Since they constrain the prediction coordinates relative to the grid cell location, this bounds the ground truth to fall between `0` and `1`. They use a logistic (sigmoid) activation function to constrain the network’s prediction to fall in `[0, 1]` to match the ground-truth range.

**Figure 5** shows the predicted bounding box (in blue) and the anchor box (in dotted black lines). The cell/grid responsible for predicting this blue box is the cell in the second row since the center of the bounding box falls in this particular cell.

The model predicts five bounding boxes in the output feature map at each cell (five anchors per cell). For each bounding box, the network predicts five values (shown in **Figure 6**): the offset, and the scale ![Image 13: (t_{x}, t_{y}, t_{w}, t_{h})](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2087%2019'%3E%3C/svg%3E) and the confidence score ![Image 14: t_{o}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2012%2013'%3E%3C/svg%3E).

The corresponding predicted bounding box ![Image 15: b](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%207%2011'%3E%3C/svg%3E) has center ![Image 16: (b_{x}, b_{y})](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2046%2019'%3E%3C/svg%3E) and width and height ![Image 17: (b_{w}, b_{h})](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2048%2018'%3E%3C/svg%3E). At each grid cell, the anchor/prior box has size ![Image 18: (p_{w}, p_{h})](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2051%2018'%3E%3C/svg%3E) with its top-left corner at ![Image 19: (c_{x}, c_{y})](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2046%2019'%3E%3C/svg%3E). The confidence score is the sigmoid of the predicted output ![Image 20: t_{o}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2012%2013'%3E%3C/svg%3E).

![Image 21](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20465'%3E%3C/svg%3E)

**Figure 6:** Network Coordinate Predictions (source: [Musib, 2020](https://medium.com/analytics-vidhya/deep-dive-on-yolov2-and-yolo9000-2eba212dcf8a)).

More specifically, ![Image 22: t_{x}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2013'%3E%3C/svg%3E) and ![Image 23: t_{y}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2012%2015'%3E%3C/svg%3E) are the ![Image 24: x](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%209%207'%3E%3C/svg%3E)\-coordinate and ![Image 25: y](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%208%2010'%3E%3C/svg%3E)\-coordinate of the centroid relative to the top-left corner of that cell. ![Image 26: (p_{w}, p_{h})](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2051%2018'%3E%3C/svg%3E) are the width and height of the anchor box, ![Image 27: t_{w}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2015%2013'%3E%3C/svg%3E) and ![Image 28: t_{h}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2013'%3E%3C/svg%3E) are the offsets for anchor adjustment predicted by the network.

Taking the exponent of ![Image 29: t_{w}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2015%2013'%3E%3C/svg%3E) and ![Image 30: t_{h}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2013%2013'%3E%3C/svg%3E) helps make them positive if the network predicts them as negative since width and height cannot be negative. To get the real width and height of the bounding box, we multiply the offsets with the anchor box width ![Image 31: p_{w}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2018%2010'%3E%3C/svg%3E) and height ![Image 32: p_{h}](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2016%2010'%3E%3C/svg%3E).

Like in YOLOv1, here, too, the box confidence score should be equal to the IOU between the predicted box and the ground-truth box.

By constraining the location prediction, the network’s learning becomes easy, making the network more stable.

Combining dimension clusters with directly predicting the bounding box center location improves YOLOv2 by `~5%` over the version with anchor boxes.

* * *

#### [**Fine-Grained Features**](#TOC)

*   YOLOv2 predicts detections over a ![Image 33: 13 \times 13](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2052%2013'%3E%3C/svg%3E) feature map, which works well for large objects, but detecting smaller objects can benefit from fine-grained features. Fine-grained features refer to feature maps from the earlier layers of the network.
*   While both Faster R-CNN and SSD (Single-Shot Detector) run the region proposal network at various layers (feature maps) in the network for multiple resolutions, YOLOv2 adds a **passthrough** layer.
*   The passthrough layer was partly inspired by the [U-Net paper](https://arxiv.org/pdf/1505.04597.pdf) in which skip-connections were used to concatenate features between the encoder and decoder layers.
*   Similarly, YOLOv2 concatenates the high-resolution features with the low-resolution ones by stacking adjacent features into different channels (**Figure 7**). This could also be thought of as identity mappings in [ResNet](https://arxiv.org/pdf/1512.03385.pdf) architecture.

![Image 34](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20150'%3E%3C/svg%3E)

**Figure 7:** Concatenating the high-resolution features with the low-resolution features ([source](https://miro.medium.com/max/875/1*RuW-SCIML8SHc5_PrIE9-g.jpeg)).

* * *

#### [**Multi-Scale Training**](#TOC)

![Image 35](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20586%20500'%3E%3C/svg%3E)

**Figure 8:** Multi-Scale Training ([source](https://docs.google.com/presentation/d/14qBAiyhMOFl_wZW4dA1CkixgXwf0zKGbpw_0oHK8yEM/edit#slide=id.g23eff386b7_0_62)).

![Image 36](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20352'%3E%3C/svg%3E)

****Table 2:**** Quantitative Evaluation of Detection Frameworks on PASCAL VOC 2007 ([Redmon and Farhadi](https://openaccess.thecvf.com/content_cvpr_2017/papers/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.pdf), p. 7267, table **3).**

* * *

### [**Faster**](#TOC)

Till now, we discussed how YOLOv2 makes object detection accurate by using various techniques like anchor boxes, batch normalization, high-resolution classifiers, dimension clusters, etc. But what about the speed? That’s what we will discuss in this section of the post.

Many applications that involve object detection like autonomous driving, robotics, video surveillance require real-time performance and rely on low latency predictions. To maximize the speed, YOLOv2 was designed to be fast from the ground level.

[Redmon and Farhadi](https://openaccess.thecvf.com/content_cvpr_2017/papers/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.pdf) do a detailed comparative study between Darknet-19 (used by YOLOv2) and other classification architectures in terms of speed, accuracy, and FLOPs (floating-point operations) shown in **Table 3** and show their architecture does the best in terms of both speed and accuracy.

![Image 37](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20172'%3E%3C/svg%3E)

**Table 3:** Darknet-19 and other classifiers in terms of accuracy, speed, and FLOPs ([source](https://pjreddie.com/darknet/imagenet/)).

VGG-16 has been the go-to feature extractor for most detection frameworks; it is a robust and accurate image classification network but requires a massive amount of floating-point operations (30.95 billion FLOPs) for a single forward pass at ![Image 38: 224 \times 224](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2069%2013'%3E%3C/svg%3E) image resolution. The YOLOv1 architecture used a custom architecture inspired by the GoogLeNet network. As we can see from **Table 3**, the YOLOv1 extraction module is much faster and more accurate than VGG-16 and only uses 8.52 billion operations for a single pass.

ResNet-50 is the slowest among all but achieves the best Top-1 and Top-5 accuracy; people love the family of ResNets. One important point to note is that ResNet uses less than one-third of the FLOPs compared to VGG-16 but still achieves lower FPS which means FLOPs are not directly proportional to FPS. Moreover, ResNet-50 has much more layers than VGG-16, it has fewer filters per layer, not saturating the GPU, and you end up doing a lot more work in transferring data between layers that adds to the time making ResNet slower. Hence, speed is not just FLOPs.

* * *

#### [**Darknet-19**](#TOC)

Darknet-19 is a new classification architecture proposed as the base of object detection, the summary of which is shown in **Figure 10**.

* * *

#### [**Training for Classification**](#TOC)

As discussed previously, we trained the Darknet-19 network in a High-Resolution Classifier fashion. The network was initially trained on the ImageNet 1000 class dataset for 160 epochs at ![Image 39: 224 \times 224](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2069%2013'%3E%3C/svg%3E) image resolution using stochastic gradient descent with a learning rate of 0.1. Then, various data augmentation techniques like random crops, exposure shifts, hue, and saturation were applied. As a result, the network achieved Top-1 accuracy of 72.9% and Top-5 accuracy of 91.2% with only 5.58 billion FLOPs.

The same network was fine-tuned at a larger resolution (![Image 40: 448 \times 448](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2069%2013'%3E%3C/svg%3E)) with the same parameters for only ten epochs with a learning rate of 0.001. Again, the model performed better than the 224 resolution achieving Top-1 accuracy of 76.5% and Top-5 accuracy of 93.3%.

* * *

#### [**Training for Detection**](#TOC)

[Redmon and Farhadi](https://openaccess.thecvf.com/content_cvpr_2017/papers/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.pdf) modified the Darknet-19 network for the object detection task. They removed the last convolutional layer along with average pooling and softmax and replaced the convolutional layer with three ![Image 41: 3 \times 3](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2036%2012'%3E%3C/svg%3E) convolutional layers with 1024 filters. Followed by a ![Image 42: 1 \times 1](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2034%2013'%3E%3C/svg%3E) convolutional layer to convert ![Image 43: 13 \times 13 \times 1024](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20104%2013'%3E%3C/svg%3E) (input resolution of ![Image 44: 416 \times 416](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2069%2013'%3E%3C/svg%3E) downsampled to ![Image 45: 13 \times 13](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2052%2013'%3E%3C/svg%3E)) with the number of outputs required for detection, that is, ![Image 46: 13 \times 13 \times 125](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2096%2014'%3E%3C/svg%3E) (five boxes predicted at each grid with each box having four box coordinates, one objectness score, and 20 conditional class probabilities per box so 125 filters).

The network was trained for 160 epochs with a learning rate of 0.001. This decayed the learning rate by a factor of 10 at 60 and 90 epochs. The same training strategy is used for training on both MS COCO and PASCAL VOC datasets.

**Figure 11** shows the object detection architecture with the base network shown in dotted lines, pretrained with imagenet weights. As learned before, we add a passthrough layer from block 5 of Darknet-19 to the second to last convolutional layer so that the model learns fine-grained features and performs well on smaller objects.

![Image 47](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%2089'%3E%3C/svg%3E)

**Figure 11:** YOLOv2 with Darknet-19 Network ([source](https://jonathan-hui.medium.com/real-time-object-detection-with-yolo-yolov2-28b1b93e2088)).

Congratulations on making it here! You learned all the significant contributions this paper made to build a state-of-the-art real-time object detector YOLOv2. The next and the final theoretical section is optional since it doesn’t discuss much object detection per se, so feel free to skip it and go directly to the inference section.

* * *

### [**Stronger**](#TOC)

The motivation of this final section comes from the fact that there are far more labels in the image classification datasets than object detection datasets since drawing boxes on images for object detection is a lot more expensive than labeling images for classification.

**Figure 12** shows that the MS COCO object detection dataset has only 100K images with 80 classes, while the ImageNet dataset has 14 million images with over 22K classes. That’s a huge difference. And this is more than a reason to leverage the classification dataset to solve object detection problems.

![Image 48](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20441'%3E%3C/svg%3E)

**Figure 12:** Comparison of Object Detection and Image Classification Datasets ([source](https://docs.google.com/presentation/d/14qBAiyhMOFl_wZW4dA1CkixgXwf0zKGbpw_0oHK8yEM/edit#slide=id.g23eff386b7_0_80)).

The COCO dataset has common and general objects with coarse-grained labels like a cat, dog, car, bicycle, etc. On the other hand, the ImageNet dataset is vast and has fine-grained labels like a Persian cat, airbus, and American twinflower.

Hence, the paper proposed an approach to combine object detection and classification datasets and jointly learn a much larger spectrum of classes than just 80 (in MS COCO). [Redmon and Farhadi](https://openaccess.thecvf.com/content_cvpr_2017/papers/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.pdf) called this model YOLO9000 because it was trained to detect over 9000 object categories. The idea was that the object detection labels would learn detection-specific features like bounding box coordinates, objectness score, and classifying common objects (in MS COCO). In contrast, the images with only class labels (from ImageNet) would help expand the number of categories it can detect.

**Figure 13** shows the MS COCO and ImageNet class representation in a tree format where we usually apply a softmax function over all these classes to compute the final probability. The softmax function assumes that the datasets’ classes are mutually exclusive; for example, you cannot apply softmax to output probabilities for classes with both cat and a Persian cat since the Persian cat belongs to the parent class cat. A Persian cat image could also be classified as a cat.

![Image 49](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20341'%3E%3C/svg%3E)

**Figure 13:** MS COCO and ImageNet Class Representation ([source](https://docs.google.com/presentation/d/14qBAiyhMOFl_wZW4dA1CkixgXwf0zKGbpw_0oHK8yEM/edit#slide=id.g23f389e290_1_46)).

Initially, the authors thought they could merge both the datasets (shown in **Figure 14**) and do joint training. When the network is fed images labeled for detection, it backpropagates with the complete YOLOv2 loss function. When it is fed images labeled for classification, only the loss from classification-specific parts is backpropagated.

![Image 50](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20288'%3E%3C/svg%3E)

**Figure 14:** Merging the two datasets in a flat hierarchy ([source](https://docs.google.com/presentation/d/14qBAiyhMOFl_wZW4dA1CkixgXwf0zKGbpw_0oHK8yEM/edit#slide=id.g23f389e290_1_78)).

However, [Redmon and Farhadi](https://openaccess.thecvf.com/content_cvpr_2017/papers/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.pdf) believe that the above approach poses a few challenges: simply merging the two datasets would defy the purpose of using a softmax layer for classification as the classes would no longer remain mutually exclusive. For example, given an image of a bear, the network would be confused whether to assign the image a COCO label of “bear” or the ImageNet label of “American black bear.”

* * *

#### [**Hierarchical Classification**](#TOC)

To solve the merging problem of ImageNet labels with MS COCO, [Redmon and Farhadi](https://openaccess.thecvf.com/content_cvpr_2017/papers/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.pdf) leverage the hierarchical tree structure with reference to [WordNet](https://wordnet.princeton.edu/) from which ImageNet labels were formed (**Figure 15**). For example, in WordNet, “Norfolk terrier” and “Yorkshire terrier” are both hyponyms of “terrier,” which is a type of “hunting dog,” which is a type of “dog,” which is a “canine,” etc.

![Image 51](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20463'%3E%3C/svg%3E)

**Figure 15:** WordNet’s graph structure ([source](https://docs.google.com/presentation/d/14qBAiyhMOFl_wZW4dA1CkixgXwf0zKGbpw_0oHK8yEM/edit#slide=id.g23f389e290_1_122)).

WordNet is a database of semantic relations between words. It links words into semantic relations, including synonyms, hyponyms, and meronyms. Nouns, verbs, adjectives, and adverbs are grouped into synonyms, called **synsets** (also known as a set of synonyms), each expressing a distinct concept. A hyponym is a word or phrase whose semantic field is more specific than its hypernym. The semantic field of a hypernym, also known as a superordinate, is broader than that of a hyponym. For example, _pigeons_, _crow_, _eagle_, and _seagull_ are all hyponyms of _bird_, their hypernym.

Most approaches to classification assume a flat structure to the labels; however, for combining datasets, the structure is precisely what we need. But WordNet is structured as a directed graph, not a tree, and graphs tend to be more connected and complex than trees. For example, a “dog” is both a type of “canine” and a type of “domestic animal.” So instead of using a complete graph structure, a simplified hierarchical tree is built from the concepts in ImageNet.

The WordNet was cleaned up and converted into a tree structure by considering the words for which images were present in the dataset, which resulted in a word tree that described physical objects.

More specifically, the following steps are performed to build the tree:

*   Visual nouns in ImageNet are observed, and we follow their paths through the WordNet graph to the root node. Here the root node is the “physical object.”
*   Synsets that just have one path through the graph are added first to the tree.
*   Then we iteratively observe the concepts we are left with and add the paths that grow the tree by as minute as possible (if a concept has two paths to the root, one having five edges and the other having three edges, we’ll choose the latter).

This results in a WordTree, a hierarchical model of visual concepts. The classification is performed in the WordTree hierarchy. We predict conditional probabilities at every node of each hyponym given that synset. So, for example, at the “terrier” node, we predict:

Pr(Norfolk terrier | terrier)  
Pr(Airedale terrier | terrier)  
Pr(Sealyham terrier | terrier)  
Pr(Lakeland terrier | terrier)  
…

A Darknet-19 model is trained on WordTree built using the 1000 classes from ImageNet. The WordTree has 1000 leaf nodes corresponding to the original 1K labels plus 369 nodes for their parent classes.

The model predicts a vector of 1369 values and calculates the softmax over all synsets that are hyponyms of the same concept (see **Figure 16**). Most ImageNet models use one large softmax layer to predict a probability distribution over 1000 classes. However, in WordTree, multiple softmax operations are performed over co-hyponyms like head, hair, vein, mouth, so all the hyponyms of synset body are one softmax calculation.

![Image 52](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20612%20500'%3E%3C/svg%3E)

**Figure 16:** Prediction on ImageNet vs. WordTree ([source](https://docs.google.com/presentation/d/14qBAiyhMOFl_wZW4dA1CkixgXwf0zKGbpw_0oHK8yEM/edit#slide=id.g1f9fb98e4b_0_103)).

Using the same Darknet-19 model with the same parameters but with hierarchical training, the model achieves 71.9% Top-1 accuracy and 90.4% Top-5 accuracy. Now the same idea is applied for object detection. The detector predicts the bounding box and tree of probabilities, using the highest confidence synset at every split until we reach a threshold and predict that node as an object class.

**Figure 17** shows an example of using WordTree to combine the labels from ImageNet (specific concepts: lower nodes and leaves) and COCO (general concepts: higher nodes). WordNet is highly diverse, so we can use this technique with most datasets. Blue nodes are COCO labels, and red nodes are ImageNet labels.

![Image 53](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20374'%3E%3C/svg%3E)

**Figure 17:** Combined MS COCO and ImageNet Labels using WordTree hierarchy ([source](https://docs.google.com/presentation/d/14qBAiyhMOFl_wZW4dA1CkixgXwf0zKGbpw_0oHK8yEM/edit#slide=id.g23f389e290_1_116)).

* * *

### [**Joint Classification and Detection**](#TOC)

Now that we have combined both datasets (containing 9418 classes), we can jointly train the YOLO9000 model on classification and detection.

For the joint training, only 3 priors instead of 5 were considered to limit the output size. In addition, the ImageNet detection challenge dataset was added for evaluating the YOLO9000 model.

The training of YOLO9000 is performed in two ways:

*   When the network sees the **detection image**, the complete YOLOv2 loss is backpropagated (i.e., bounding box coordinates, objectness score, classification error) (**Figure 18**).

![Image 54](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20199'%3E%3C/svg%3E)

**Figure 18:** Backpropagating the loss for detection image ([source](https://docs.google.com/presentation/d/14qBAiyhMOFl_wZW4dA1CkixgXwf0zKGbpw_0oHK8yEM/edit#slide=id.g23f389e290_1_128), slide 39).

*   For **[classification images](https://pyimagesearch.com/2021/04/17/image-classification-basics/)**, the network backpropagates classification and objectness loss (**Figure 19)**. The objectness loss is computed by assuming that the predicted bounding box overlaps with the ground-truth box by at least 0.3 IOU. To backpropagate the classification loss, the bounding box with the highest probability for that class is found, and the loss is computed on just its predicted tree.

![Image 55](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20205'%3E%3C/svg%3E)

**Figure 19:** Backpropagating the loss for classification image ([source](https://docs.google.com/presentation/d/14qBAiyhMOFl_wZW4dA1CkixgXwf0zKGbpw_0oHK8yEM/edit#slide=id.g1f9fb98e4b_0_40), slide 40).

* * *

#### [**Evaluating the YOLO9000**](#TOC)

The model is evaluated on the ImageNet detection task. The evaluation dataset has 44 object categories in common with MS COCO, meaning it has seen very few object categories for detection since the majority of the test images are for image classification. Despite that, the YOLO9000 model achieves 19.7 mAP overall with 16.0 mAP on the 156 objects classes the network has never seen the labeled detection data for during the training.

[Redmon and Farhadi](https://openaccess.thecvf.com/content_cvpr_2017/papers/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.pdf) found that the joint YOLO9000 model did well on the new species of animals but struggled with learning categories like clothing and equipment. And the reason is COCO has a lot of labeled data for animals compared to clothing and equipment; in fact, it does not have bounding box labels for any clothing category, so this performance drop is apparent.

Now that we have covered a lot of ground on YOLOv2 and YOLO9000, let’s move on to running the inference on the YOLOv2 model in the darknet framework pre-trained on the PASCAL VOC dataset.

* * *

### [**Configuring the Darknet Framework and Running Inference with the Pretrained YOLOv2 Model**](#TOC)

In our previous post on [YOLOv1](https://pyimagesearch.com/2022/04/11/understanding-a-real-time-object-detection-network-you-only-look-once-yolov1/), we had learned to configure the darknet framework and ran inference with the pretrained YOLOv1 model; we would follow the same steps as before to configure the darknet framework. Then, finally, run the inference with the YOLOv2 pretrained model and see it perform better than YOLOv1.

Configuring the darknet framework and running the inference with YOLOv2 on images and video is divided into seven easy-to-follow steps. So, let’s get started!

**_Note_**_: Please ensure you have the matching CUDA, CUDNN, and NVIDIA Driver Installed on your machine. For this experiment, we use CUDA-10.2, CUDNN-8.0.3. But if you plan to run this experiment on Google Colab, do not worry, as all these libraries come pre-installed with it._

**Step #1:** We will use the GPU for this experiment, so make sure the GPU is up and running.

\# Sanity check for GPU as runtime
$ nvidia-smi

**Figure 20** shows the GPUs available in the machine (i.e., V100) and the driver and CUDA versions.

![Image 56](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20260'%3E%3C/svg%3E)

**Figure 20:** Output showing the GPU availability and specifications (image by the author).

**Step #2:** We will install a few libraries like OpenCV, FFmpeg, etc., that would be required before compiling and installing Darknet.

\# Install OpenCV, ffmpeg modules
$ apt install libopencv-dev python-opencv ffmpeg

**Step #3:** Next, we clone the modified version of the darknet framework from the AlexyAB repository. As learned earlier, Darknet is an open-source neural network written by Joseph Redmon. It is written in C and CUDA, supporting both CPU and GPU computation. The official implementation of the Darknet is available at: [https://pjreddie.com/darknet/](https://pjreddie.com/darknet/); we will download the YOLOv2 weights provided by the official website.

\# Clone AlexeyAB darknet repository
$ git clone https://github.com/AlexeyAB/darknet/
$ cd darknet/

Make sure to change the directory to darknet since, in the next step, we will configure the `Makefile` and compile it. Also, do a sanity check using `!pwd`; we should be in the `/content/darknet` directory.

**Step #4:** Using stream editor (`sed`), we will edit the make files and enable flags: `GPU`, `CUDNN`, `OPENCV`, and `LIBSO`.

**Figure 21** shows a snippet of the `Makefile` contents, which are discussed later:

![Image 57](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20700%20445'%3E%3C/svg%3E)

**Figure 21:** Contents of Darknet Makefile (image by the author).

*   We enable the `GPU=1` and `CUDNN=1` to build darknet with `CUDA` to perform and accelerate the inference on the `GPU`. Note `CUDA` should be in `/usr/local/cuda`; otherwise, the compilation will result in an error, but don’t worry if you are compiling it on Google Colab.
*   If your `GPU` has Tensor Cores, enable `CUDNN_HALF=1` to gain up to `3X` inference and `2X` training speedup. Since we use a Tesla V100 GPU with tensor cores, we will enable this flag.
*   We enable `OPENCV=1` to build darknet with `OpenCV`. This will allow us to detect video files, IP cameras, and other OpenCV off-the-shelf functionalities like reading, writing, and drawing bounding boxes over the frames.
*   Finally, we enable `LIBSO=1` to build the `darknet.so` library and binary runnable file `uselib` that uses this library. Enabling this flag would allow us to use Python scripts for inference on images and videos, and we will be able to import `darknet` inside it.

Now let’s edit the `Makefile` and compile it.

\# Enable the OpenCV, CUDA, CUDNN, CUDNN\_HALF & LIBSO Flags and Compile Darknet
$ sed -i 's/OPENCV=0/OPENCV=1/g' Makefile
$ sed -i 's/GPU=0/GPU=1/g' Makefile
$ sed -i 's/CUDNN=0/CUDNN=1/g' Makefile
$ sed -i 's/CUDNN\_HALF=0/CUDNN\_HALF=1/g' Makefile
$ sed -i 's/LIBSO=0/LIBSO=1/g' Makefile
$ make

The `make` command will take around 90 seconds to finish the execution. Now that the compilation is complete, we are all set to download the YOLOv2 weights and run the inference.

Aren’t you excited?

**Step #4:** We will now download the YOLOv2-VOC weights from the official [YOLOv2 documentation](https://pjreddie.com/darknet/yolov2/).

\# Download YOLOv2 Weights
$ wget https://pjreddie.com/media/files/yolov2-voc.weights

**Step #5:** Now, we will run the `darknet_images.py` script to infer the images.

\# Run the darknet image inference script
$ python3 darknet\_images.py --input data --weights \\ 
yolov2-voc.weights --config\_file cfg/yolov2-voc.cfg \\
--data\_file cfg/voc.data --dont\_show

Let’s put some light on the command line arguments we pass to `darknet_images.py`:

*   `--input`: Path to the images directory or text file with the path to the images or a single image name. Supports `jpg`, `jpeg`, and `png` image formats. In this case, we pass the path to the image folder called `data`.
*   `--weights`: YOLOv2 weights path.
*   `--config_file`: Configuration file path of YOLOv2. On an abstract level, this file stores the neural network model architecture and a few other parameters like `batch_size`, `classes`, `input_size`, etc. We recommend you give a quick read of this file by opening it in a text editor.
*   `--data_file`: Here, we pass the PASCAL VOC labels file.
*   `--dont_show`: This will disable OpenCV from displaying the inference results, and we use this since we are working with Google Colab.

After running YOLOv2 PASCAL VOC pretrained model on the below images, we learn that the model detects objects with much higher confidence and has almost close to zero false negatives compared to YOLOv1. However, you would see a couple of false positives.

We can see from **Figure 22**, the model detects a dog, bicycle, and car with a very high confidence score.

![Image 58](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20416%20416'%3E%3C/svg%3E)

**Figure 22:** Model predicting a dog, car, and bicycle (image by the author).

In **Figure 23**, the model detects all three objects correctly (i.e., a dog, person, and a horse); however, it detects the horse as a sheep but with a much lower confidence score. Recall in YOLOv1, the network made the same mistake of predicting the horse as a sheep, but at least YOLOv2 predicts it as a horse too! And since the confidence score for the sheep is less, you can filter that detection by increasing the `--thresh` argument!

![Image 59](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20416%20416'%3E%3C/svg%3E)

**Figure 23:** YOLOv2 prediction on a dog, person, and a horse image (image by the author).

In **Figure 24**, the model does a good job detecting all the horses. However, it also predicts one horse on the right as a cow. But unlike YOLOv1, it does not miss detecting the object.

![Image 60](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20416%20416'%3E%3C/svg%3E)

**Figure 24:** Prediction on a group of horses (image by the author).

**Figure 25** is an image of an Eagle, similar to the YOLOv1 model that predicted a bird with 75%. However, YOLOv2 also detects a bird with a much higher confidence score of 95%.

![Image 61](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20416%20416'%3E%3C/svg%3E)

**Figure 25:** Prediction on an image of an Eagle (image by the author).

**Step #6:** Now, we will run the pretrained YOLOv2 model on a video from the movie Skyfall; it is the same video that the Authors had used in one of their experiments.

Before running the `darknet_video.py` demo script, we will first download the video from YouTube using the Pytube library and crop the video with the moviepy library. So let’s quickly install these modules and download the video.

\# Install pytube and moviepy for downloading and cropping the video
$ pip install git+https://github.com/rishabh3354/pytube@master
$ pip install moviepy

\# Import the necessary packages
$ from pytube import YouTube
$ from moviepy.video.io.ffmpeg\_tools import ffmpeg\_extract\_subclip

# Download the video in 720p and Extract a subclip
$ YouTube('https://www.youtube.com/watch?v=tHRLX8jRjq8'). \\ streams.filter(res="720p").first().download()
$ ffmpeg\_extract\_subclip("/content/darknet/Skyfall.mp4", \\ 
0, 30, targetname="/content/darknet/Skyfall-Sample.mp4")

**Step #7:** Finally, we will run the `darknet_video.py` script to generate predictions for the Skyfall video. We print the FPS information over each frame of the output video.

Be sure to change the video codec in the `set_saved_video` function from `MJPG` to `mp4v` at **Line 57** in `darknet_video.py` if using an mp4 video file; otherwise, you will get a decoding error while playing the inference video.

\# Change the VideoWriter Codec
fourcc = cv2.VideoWriter\_fourcc(\*"mp4v")

Now that all the necessary installations and modifications are complete, we will run the `darknet_video.py` script:

\# Run the darknet video inference script
$ python darknet\_video.py --input \\ 
/content/darknet/Skyfall-Sample.mp4 \\ 
--weights yolov2-voc.weights --config\_file \\ 
cfg/yolov2-voc.cfg --data\_file ./cfg/voc.data \\
--dont\_show --out\_filename pred-skyfall.mp4

Let’s look at the command line arguments we pass to `darknet_video.py`:

*   `--input`: Path to the video file or `0` if using a webcam
*   `--weights`: YOLOv2 weights path
*   `--config_file`: The configuration file path
*   `--data_file`: Here, we pass the PASCAL VOC labels file
*   `--dont_show`: This will disable OpenCV from displaying the inference results
*   `--out_filename`: Inference results output video name, if empty output video not saved

Below are the inference results on the Skyfall Action Scene Video, and the predictions seem to be a lot better than YOLOv1. The YOLOv2 network achieves an average of **84 FPS** on the Tesla V100 GPU with mixed precision.

### What's next? We recommend [PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend).

**Course information:**  
84 total classes • 114+ hours of on-demand code walkthrough videos • Last updated: February 2024  
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

**I strongly believe that if you had the right teacher you could _master_ computer vision and deep learning.**

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s _not_ the case.

All you need to master computer vision and deep learning is for someone to explain things to you in _simple, intuitive_ terms. _And that’s exactly what I do_. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to _successfully_ and _confidently_ apply computer vision to your work, research, and projects. Join me in computer vision mastery.

**Inside PyImageSearch University you'll find:**

*   ✓ **84 courses** on essential computer vision, deep learning, and OpenCV topics
*   ✓ **84 Certificates** of Completion
*   ✓ **114+ hours** of on-demand video
*   ✓ **Brand new courses released _regularly_**, ensuring you can keep up with state-of-the-art techniques
*   ✓ **Pre-configured Jupyter Notebooks in Google Colab**
*   ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
*   ✓ Access to **centralized code repos for _all_ 536+ tutorials** on PyImageSearch
*   ✓ **Easy one-click downloads** for code, datasets, pre-trained models, etc.
*   ✓ **Access** on mobile, laptop, desktop, etc.

[Click here to join PyImageSearch University](https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend)

* * *

[**Summary**](#TOC)
-------------------

Congratulations on making it this far! It was a lot to grasp as we discussed almost everything in detail about YOLOv2 along with YOLO9000, so let’s quickly summarize:

1.  You were introduced to the second version of YOLO, where we discussed [Redmon and Farhadi’s (2017)](https://openaccess.thecvf.com/content_cvpr_2017/papers/Redmon_YOLO9000_Better_Faster_CVPR_2017_paper.pdf) essential contributions to creating a state-of-the-art object detector. We also briefly touched upon the speed and accuracy benchmarks of YOLOv2 and YOLO9000.
2.  We then looked at various technical components that made YOLOv2 perform Better like batch normalization, high-resolution classifier, convolution with anchor boxes, dimension clusters, etc., each progressively improving the object detector.
3.  You then learned the improvements made to run both training and testing of YOLOv2 Faster than before, especially the use of Darknet-19 architecture that consumed fewer FLOPs.
4.  We then also discussed the Stronger aspect of YOLO called YOLO9000. We covered the hierarchical training approach, merging the MS COCO and ImageNet dataset, and jointly training for classification and detection.
5.  Finally, we ran the Inference with the pretrained YOLOv2-VOC model on images and video on a Tesla V100 and compared the detection results with YOLOv1.

With learning YOLOv2, you have come one more step closer to mastering the YOLO series, as the initial posts act as a foundation for later YOLO versions. Keep that momentum going as we bring you more exciting posts on YOLO in the coming weeks.

* * *

### [**Citation Information**](#TOC)

**Sharma, A.** “A Better, Faster, and Stronger Object Detector (YOLOv2),” _PyImageSearch_, D. Chakraborty, P. Chugh, A. R. Gosthipaty, S. Huot, K. Kidriavsteva, R. Raha, and A. Thanki, eds., 2022, [https://pyimg.co/ehaox](https://pyimg.co/ehaox)

@incollection{Sharma\_2022\_YOLOv2,
  author = {Aditya Sharma},
  title = {A Better, Faster, and Stronger Object Detector ({YOLO}v2)},
  booktitle = {PyImageSearch},
  editor = {Devjyoti Chakraborty and Puneet Chugh and Aritra Roy Gosthipaty and Susan Huot and Kseniia Kidriavsteva and Ritwik Raha and Abhishek Thanki},
  year = {2022},
  note = {https://pyimg.co/ehaox},
}

* * *

![Image 62: Featured Image](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20500%20276'%3E%3C/svg%3E)

### Unleash the potential of computer vision with Roboflow - Free!

*   Step into the realm of the future by [signing up or logging into your Roboflow account](https://roboflow.com/?ref=pyimagesearch). Unlock a wealth of innovative dataset libraries and revolutionize your computer vision operations.
*   Jumpstart your journey by choosing from our broad array of datasets, or benefit from PyimageSearch’s comprehensive library, crafted to cater to a wide range of requirements.
*   Transfer your data to Roboflow in any of the 40+ compatible formats. Leverage cutting-edge model architectures for training, and deploy seamlessly across diverse platforms, including API, NVIDIA, browser, iOS, and beyond. Integrate our platform effortlessly with your applications or your favorite third-party tools.
*   Equip yourself with the ability to train a potent computer vision model in a mere afternoon. With a few images, you can import data from any source via API, annotate images using our superior cloud-hosted tool, kickstart model training with a single click, and deploy the model via a hosted API endpoint. Tailor your process by opting for a code-centric approach, leveraging our intuitive, cloud-based UI, or combining both to fit your unique needs.
*   Embark on your journey today with absolutely no credit card required. Step into the future with Roboflow.

[Join Roboflow Now](https://roboflow.com/?ref=pyimagesearch)

![Image 63](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%20410%20567'%3E%3C/svg%3E)

#### Join the PyImageSearch Newsletter and Grab My FREE 17-page Resource Guide PDF

Enter your email address below to **join the PyImageSearch Newsletter** and **download my FREE 17-page Resource Guide PDF** on Computer Vision, OpenCV, and Deep Learning.