Object Detection: Find me if you can!
By: Abhimanyu Banerjee
Tags: Computer Vision, Object Detection, Artificail Intelligence, Machine Learning

Let’s try performing object detection through sliding window approach…

NAIVE WAY— “Sliding Window For Object Classification”.

First, we take an image as input. We decide a window size and slide it across an image to obtain corresponding regions.

3. Pass all these regions(images) to the CNN and classify them into various classes

4. Once we have divided each region into its corresponding class, we can combine all these regions to get the original image with the detected objects.

Problem with this approach: COMPUTATIONAL COMPLEXITY!

A huge number of regions are obtained as the image gets larger. Results in large amount of computational time.

Problem with Scale — What should be the window size?

A fixed window size cannot capture all objects in an image. Objects in an image can have different scales.

In the picture below, the bird is detected but the fish is not due to its smaller size. All sizes of objects must be captured by the object detector.

Objects in an image can have different aspect ratios and spatial locations. Here, the person is tall and thin whereas the horse is broad.

Problem with Object Coverage in Image —

Solution!

In order to solve all these problems and reduce the number of regions, we use “Region-based CNN”.

What is R-CNN? — Region based CNN or RCNN proposes a set of regions as bounding boxes and sees if any of them actually correspond to an object.

Who proposed R-CNN? — Inspired by the research of Hinston’s lab at the University of Toronto, a small team at UC Berkeley, led by Professor Jitendra Malik, comprised of RRoss Girshick, Jeff Donahue, and Trevor Darrel proposed R-CNN by testing on the PASCAL VOX challenge.

What is “Selective Search” and how it identifies different regions?

Selective search identifies patterns in the image and combines similar patterns based on the following characteristics, to propose various regions:

How do we find where these Bounding boxes are?

Solution — Region Proposal Method

RCNN uses selective search to extract boxes from an image.

It tries to find boxes of different sizes where it feels there might be an object.

RCNN alogorithm proposes a bunch of 2000 boxes called regions in an image.

ARCHITECTURE OF R-CNN —

Complete network of RCNN

How does R-CNN work?

An image is input to the network. Generates 2000 regions of interest using Selective Search. Convert regions into fixed sized images. Each region proposal is passed into a CNN for feature extraction. Linear SVM classifier helps classify objects Regression model finds bounding box coordinates.

LIMITATIONS of R-CNN

Training an RCNN model is expensive and slow Takes a huge amount of time to train the network as there are 2000 region proposals per image. Cannot be implemented real time as it takes around 47seconds for each test image.

Here’s the “good news “— we have another object detection technique which fixes most of the limitations we saw in RCNN

INTRODUCING — Fast RCNN

Fast RCNN inputs an image and a group of region proposals. The convolutional network is trained on the whole image instead of individual regions.

“Hence the computation time is reduced from 2000 convolutions to 1 convolution.”

FAST RCNN NETWORK

First Section: CONVOLUTIONAL LAYER

Use of Convolutional layers —

CNNs extract high level features from an image like lines, edges, shapes. The Fast R-CNN detector processes the entire image. Hence we did the work of 2000 convolutions in a single step. Fast R-CNN chooses CNN features corresponding to each region proposal.

Second Section: ROI POOLING

Use of Region Proposal layer and ROI Pool —

ROI pool layer convert these regions to a smaller feature map of fixed size before feeding to fully connected neural network.

This is done using max pooling or average pooling.

Third Section: FULLY CONNECTED LAYER

Fully Connected layers are used to produce 2 outputs-

Predict the class label of the object in ROI. Regression to find (x,y,w,h) coordinates of Bounding Box. (where x,y = X and Y coordinates of the bounding box ; w= width of bounding box; h= height of bounding box)

Performance of fast RCNN in training —

Fast RCNN is 10x faster compared to RCNN because:

Only a single image is processed by a CNN, instead of 2000 CNNs on images obtained from selective search in RCNN algorithm.

Performance of fast RCNN in test —

“Fast RCNN is really fast in prediction.”

The only bottleneck is region proposal which takes almost 2 seconds which is a lot of time.

LIMITATIONS of fast RCNN —

Region proposal takes a lot of time (2 seconds).

Selective Search is a slow and time consuming process which is unsuitable for real world, large datasets.

This issue is resolved with “Faster RCNN”.

UNDERSTANDING FASTER RCNN

What is faster RCNN

Faster RCNN uses an inbuilt region proposal network (RPN) to generate region proposals directly in the network without using an external algorithm like Selective Search.

Outcome: We just saved 2 seconds.

Faster RCNN Network —

This is a complete network of faster RCNN. This is similar to fast RCNN except addition of region proposal network.

Region Proposal Network

Region Proposal Network creates boxes called anchors which are most likely to contain object.

Anchors in faster RCNN

Anchors are fixed boxes placed across the images to find location of object

Faster RCNN has 9 anchors of fixed size like 128x128, 256x256 and 512x512 (red, green and blue)

For each size, the width to height ratio is 1:1, 1:2 and 2:1

Step 1: Convolutional Layer

First picture goes through convolutional layer and feature maps are extracted.

Step 2: Region Proposal Network

A sliding window is drawn over each point in feature map.

At every point, 9 anchor boxes are drawn for generating region proposals

Step 3: Detection Network

Performance of faster RCNN —

Faster RCNN is much faster than its predecessors

Summary

The achievements displayed through RCNN and its successor’s are truly amazing, yet none of these architectures manage to create a real-time object detector.

Hence, new architectures were created to address the bottlenecks of RCNN and its successors, enabling real-time object detection.

One of the most famous ones is “SSD MultiBox (Single Shot Detector)”.