Object Detection Using YOLOv3 on Colab with questions for interview preparation | by Manish Sharma | Towards Data Science

This blog will help you to train a model which can identify different objects in an image. It’s fun and exciting to observe how the model can behave better than humans in certain conditions. It can differentiate between car and pickup truck even if the rear of the truck is not clear in an image, personally, I wasn’t able to make that difference. In case you have already used YOLO as a black box this blog will help you to understand the model and its nuances. Finally, there are some questions that can help you to gauge your understanding of the model. Try to answer the questions in the comment section and don’t hesitate to ask me if you need an answer.
Requirements: A pc with an internet connection and a Google account.
Learnings: An hands-on experience of object detection using YOLOv3 to deepen the understanding of YOLO algorithm.
Setup: Set up a Colab notebook account through your google drive (My Drive > New > More > Connect More apps > Colab). To perform the object detection on images in your pc install ‘Drive Backup and Sync’. Allow one folder on your pc to sync on google drive. The files (images or videos) from this folder will be accessed by Colab (via google drive).
Object detection
The object detection part is divided into 9 easy steps. It will allow you to apply object detection on the images clicked by you. So let’s begin the object detection first and later on I will explain the algorithm (YOLO) behind it.
STEP1: Connect your Colab notebook with google drive. Once you import and mount the drive, you need to click on the link which appears below your code. You need to allow the Colab to access the drive by permitting it.
from google.colab import drivedrive.mount('/content.gdrive')
Step2: Change the hardware accelerator to GPU (Runtime > Change Runtime type > Hardware accelerator = GPU). To make sure you are connected to GPU, type !nvidia-smi, if you are connected you should get the details of GPU you are connected (as shown below).
!nvidia-smi
Step3: Darknet is an open-source neural network written by Joseph Redmon. It is written in C and CUDA. It supports both CPU and GPU computation. The official implementation of the darknet is available at: https://pjreddie.com/darknet/ . We will use a slightly modified version of the darknet available at AlexyAB/darknet. This neural network framework can be used for object detection using YOLO.
#clone darknet repositoryimport osos.environ['PATH'] += ':/usr/local/cuda/bin'!rm -fr darknet!git clone https://github.com/AlexeyAB/darknet
Step4: Check the current directory using !pwd. We should be in /content/darknet folder. Or else change to darknet folder (%cd /darknet). Once in this folder, we using stream editor (sed) to edit the make files of GPU and OpenCV (in, in-place mode, i.e, sed -i). We change all the instances of GPU=0 to GPU =1, to enable GPU and OpenCV.
#go to the darknet folder, edit and remake Makefiles of GPU and OPENCV!sed -i 's/GPU=0/GPU=1/g' Makefile!sed -i 's/OPENCV=0/OPENCV=1/g' Makefile!make
Step5: Loading the pre-trained weights for YOLO object detection. We get the pre-trained weights for YOLOv3 from pjreddie.com. This website belongs to Joseph Redmon the man behind YOLO and darknet.
# get yolov3 weights!wget https://pjreddie.com/media/files/yolov3.weights!chmod a+x ./darknet
Step6: Make sure that you are in the correct directory (/content/darknet) using !pwd. If yes, then install the required packages. For the complete list, I will encourage you to have a look at my jupyter file on github.
!apt install ffmpeg libopencv-dev libgtk-3-dev python-numpy python3-numpy libdc1394-22 libdc1394-22-dev libjpeg-dev libtiff5-dev libavcodec-dev libavformat-dev libswscale-dev libxine2-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libv4l-dev libtbb-dev qtbase5-dev libfaac-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev x264 v4l-utils unzip
Step7: Loading the image from your drive to Colab and running YOLO on it. You can transfer any image from your pc to the folder you shared with google drive in step 1. That image will appear in the same folder in the drive. In my case, I named the folder ‘darknet’ and the name of the image is ‘test2.jpg’. The address of the folder will be: /content.gdrive/My Drive/darknet/test2.jpg but since space is not allowed in the address path you can use: /content.gdrive/My\ Drive/darknet/test2.jpg.
!./darknet detect cfg/yolov3.cfg yolov3.weights /content.gdrive/My\ Drive/darknet/test2.jpg
Step8: You need OpenCV and matplotlib to view your result. If you are on this step first of all congratulations that you have run your very first YOLO object detection using your image.
import cv2import matplotlib.pyplot as pltimport os.pathfig,ax = plt.subplots()ax.tick_params(labelbottom="off",bottom="off")ax.tick_params(labelleft="off",left="off")ax.set_xticklabels([])ax.axis('off')file = './predictions.jpg'if os.path.exists(file):img = cv2.imread(file)show_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)#show_img(show_img)plt.imshow(show_img)plt.show()#cv2.imshow(img)
Step9: Result analysis. Here you can see that algorithm can correctly detect a baby as a person, cell phone, and hairbrush (as a toothbrush). While several other items (hairdryer, small purse, makeup items)can’t be detected. Can you guess the reason? Maybe you should check the weights we used were trained on which all objects.
Here I have presented an exercise to help new users to start object detection by just using their pc. Moreover, users can use their images so it will further add to their fun. Once you are done with this fun part it will be intriguing to know how YOLO can detect the objects.
Let’s try to learn how does the algorithm works.
YOLO (link to original paper ): You Only Look Once is an object detection network. It localizes and classifies an object. It does both these tasks in a single step. The backbone of YOLO in the darknet-53 neural network, there are 53 convolutional layers in the network for feature extraction.
YOLO divides the input image in m x m grid. Each cell of the grid contains some (B) anchor boxes to localize the object. For an object, the cell in which the center of the object lies is responsible for detecting the object.
Let’s try to understand the output vector which will give insights into different aspects of this algorithm. The output vector will consist of B * (5 + C) elements. B is the number of anchor boxes that are present in each cell. Each box will give a probability element showing what’s the probability of an object being present in the cell, 4 elements describing the bounding box (2 for center coordinates bx, by and other two to describe height and width of the box bh and bw). C is the number of classes. If there are 6 classes then there will be 11 elements per box, and if there are 3 boxes then in totality there will be 33 elements in the output.
YOLOv3 is among the fastest object detection algorithm currently present. The speed comes at the cost of accuracy. It makes a high error in object localization compared to fast R-CNN but still, it makes less background error compared to later.
To detect multiple objects per cell, it makes use of multiple anchor boxes per cell. Of course, these anchors would be of a different dimension. Let’s say one is a wider rectangle (lengthwise) other being a longer rectangle (width-wise). One other aspect to mention is to get rid of multiple bounding boxes per object it uses standard non-max suppression.
You may comment on your answers in the comment section or ask me in case you want me to answer any of these. I will highly appreciate it if you could answer the questions for which you know the answers, other readers will definitely get benefitted because of your comments.
Q1. An n x n image convolved with a filter of size f x f, padding of p and stride of s, what will be the size of output? Do check what happens if there is no padding and f & s both are equal to 2.
Q2. Why are fully connected layers required in an object detection neural network?
Q3. List down the strategy/methods for data augmentation.
Q4. What is the difference between the anchor box and the bounding box in YOLO?
Q5. How is mean average precision calculated?
Q6. Explain the concept of NMS or non-max suppression?
Q7. What are the different components of the loss function in YOLO?
To access the complete jupyter file: click here.
I hope this blog gives you comprehensive hands-on practice on YOLO and would help you start with your journey in object detection. In case you wish to discuss more feel free to comment.