Simple Image Segmentation and Object Detection(Google Colab)
By: Sridhar P
Tags: Image Segmentation, AI, Data Science, Object Detection

Hey there,

OMG, It’s Monday,

Segmentation:

Image credit: AAIC

In digital image processing and computer vision, image segmentation is the process of partitioning a digital image into multiple segments (sets of pixels, also known as image objects). (Wikipedia)

It is a pixel-level segmentation. Based on pixels it will be located

Image credit: AAIC

Problem Definition:

Each object needs to be indifferent Colour

Instance Segmentation

Finding the multiple of an instance of the same object

Semantic segmentation

It is also known as Meaningful segmentation

Difference between semantic and instance segmentation

Performance metric:

Average precision and recall for each class Jaccard similarity()

The Jaccard index, also known as Intersection over Union and the Jaccard similarity coefficient (originally given the French name coefficient de communauté by Paul Jaccard), is a statistic used for gauging the similarity and diversity of sample sets. The Jaccard coefficient measures similarity between finite sample sets, and is defined as the size of the intersection divided by the size of the union of the sample sets:

The group truth bounding box is the existing output.

The Predicted bounding box is an algorithm-generated output

Here, You can see the accuracy measures based on the correlation between two outputs.

Image credit: Wikipedia

Classical Approaches

Clustering Edge detection Graph-theoretic(spectral based algorithm)

Deep learning-based approaches

This research paper came from one of my favorite university in Germany(University of Freiburg)

Fully Convolutional neural networks Masked RCNN SegNet

Object detection

Image Credit: Wikipedia

Object detection is a computer technology related to computer vision and image processing that deals with detecting instances of semantic objects of a certain class (such as humans, buildings, or cars) in digital images and videos.

Well-researched domains of object detection include face detection and pedestrian detection. Object detection has applications in many areas of computer vision, including image retrieval and video surveillance. (Wikipedia)

Problem Definition :

Given an input image, find the bounding boxes along with a class label like a cup, laptop, etc….

Performance metric

1.For each class compute average precision

2.Mean average precision(mAP)

Segmentation Vs Object Detection

You may have a question why do we need segmentation if I already have a bounding box?

I will explain it with the simple example

Let’s say if we would like to detect traffic signals using an object detection technique

Image credit: Google

Including Traffic light, It is also capturing other unnecessary background images which reduce the accuracy of the model

Instead, if we use both segmentation and object detection to increase the performance of the model

Image credit: Google

It exactly predicts traffic light itself

Machine learning approaches:

Deep learning approaches:

Region Proposals (R-CNN, Fast R-CNN, Faster R-CNN, cascade R-CNN)

Single Shot MultiBox Detector (SSD)

You Only Look Once (YOLO)

Single-Shot Refinement Neural Network for Object Detection (RefineDet)

Retina-Net

Deformable convolutional networks

Let's start coding….

First You should change google colab notebook settings to GPU for better performance

Go to Edit option

Click Notebook settings

Select GPU and Save it

To verify if GPU is connected. Let's try this code

If it is not found, Print an error like “GPU device not found”

Necessary Libraries

Uploading necessary images

Plotting input image

You need to download Mask_rcnn_coco.h5 Pretrained model to perform image segmentation

Detecting objects along with class label

Trying for another image:

Here, I did image segmentation and object detection in a video

Thank you ……

Happy learning….