Custom object training and detection with YOLOv3, Darknet and OpenCV
By: Vino Mahendran
Tags: Opencv, Object Detection, Darknet, Yolov3, Deep Learning

Photo by Jessica Ruscello on Unsplash

YOLO is a state-of-the-art, real-time object detection system. It looks at the whole image at test time so its predictions are informed by global context in the image. It also makes predictions with a single network evaluation which makes it extremely fast when compared to R-CNN and Fast R-CNN.

This paper gives more details about how YOLO achieves the performance improvement.

Darknet

We will use Darknet, an open source neural network framework to train the detector. Download and build darknet



cd darknet

make git clone https://github.com/pjreddie/darknet cd darknetmake

Once that’s successful, To test the build we can download pre trained YOLO weights and perform detection with the test image.

./darknet detector test cfg/coco.data cfg/yolov3.cfg yolov3.weights data/dog.jpg

For training with custom objects, let us create the following required files and directories

custom_data/custom.names

custom_data/images

custom_data/train.txt

custom_data/test.txt

In this example we will train to detect Tesla cars for which we should collect the images and place it in custom_data/images directory.

Labels of our objects should be saved in custom_data/custom.names file, each line in the file corresponds to an object. In our case since we have only one object class, the file should contain the following

Annotation

After we collect the images containing our custom object, we will need to annotate them. For YOLOv3, each image should have a corresponding text file with the same file name as that of the image in the same directory.

In our case text files should be saved in custom_data/images directory. For e.g. image1.jpg should have a text file image1.txt .

Each row in the text file corresponds to a single bounding box of the object and should have the following information

<object-class-id> <x-centre> <y-centre> <width> <height>

where

<object-class-id> an integer from 0 to (classes - 1) corresponding to the classes in the custom_data/custom.names file height, width - Actual height and width of the image

x, y - centre coordinates of the bounding box

h, w - height and width of the bounding box <x-centre> : x / width

<y-centre> : y / height

<width> : w / width

<height> : h / height

This can also be done with LabelImg, a graphical image annotation tool which creates .txt files for the images in YOLO format.

Train and Test sets

We can then randomly split the annotated images into train and test sets in the ratio of 80:20

custom_data/train.txt Each row in the file should have the location of train dataset.

custom_data/test.txt Each row in the file should have the location of test dataset.

Pre-trained weights

To train our object detector we can use the existing pre trained weights that are already trained on huge data sets. From here we can download the pre trained weights to the root directory.

YOLO data file

Create a detector.data file in the custom_data directory which should contain information regarding the train and test data sets

classes=1

train=custom_data/train.txt

valid=custom_data/test.txt

names=custom_data/custom.names

backup=backup/

backup is the location where newly trained weights would be saved.

Configurations

Based on the required performance we can select the YOLOv3 configuration file. For this example we will be using yolov3.cfg . We can duplicate the file from cfg/yolov3.cfg to custom_data/cfg/yolov3-custom.cfg

While training the images, weights of the neural networks are updated iteratively. We may use huge training sets which makes it resource consuming to update the weights for the entire training set in a single iteration. To use a small set of images to iteratively update the weights, batch param is set. By default it is set to 64.

The maximum number of iterations for which our network should be trained is set with the param max_batches=4000 . Also update steps=3200,3600 which is 80%, 90% of max_batches .

We will need to update the classes and filters params of [yolo] and [convolutional] layers that are just before the [yolo] layers.

In this example since we have a single class (tesla) we will update the classes param in the [yolo] layers to 1 at line numbers: 610, 696, 783

Similarly we will need to update the filters param based on the classes count filters=(classes + 5) * 3 . For a single class we should set filters=18 at line numbers: 603, 689, 776

All the configuration changes are made to custom_data/cfg/yolov3-custom.cfg

Training

With all the required files and annotated images we can start our training

./darknet detector train custom_data/detector.data custom_data/cfg/yolov3-custom.cfg darknet53.conv.74

We can continue training until the loss reaches a certain threshold. By default, weights for the custom detector is saved for every 100 iterations until 1000 iterations and then continues to save for every 10000 iterations. This behaviour can be modified by updating the condition at line 138 of examples/detector.c file.

Once the training is complete we can use the generated weights to perform detection.

Detection with OpenCV

We can perform detection with OpenCV DNN as it is a fast DNN implementation for CPU.