Weakly Supervised Learning for Object Localization
By: Manmohan Dogra
Tags: Object Localization, Artificial Intelligence, Class Activation Map, Object Detection, Weakly Supervised

Supervised learning techniques require large quantities of high-quality annotations as labels. For tasks like segmentation, the time and cost of creating annotations are often higher than tasks like classification. To overcome these limitations, one low-cost option is to make use of lower quality annotations collected in large quantities, i.e using weakly supervised learning.

In this article, we’re going to focus on applying WSL techniques to image-based data, to be precise, WSL for object Localization.

Object Localization

Object localization refers to identifying the location of one or more objects in an image and drawing a bounding box around their extent. Object detection combines the localization of an image with the classification of one or more objects in an image.

Object localization locates the presence of objects in an image and highlights their location with a bounding box.

Input: An image with one or more than one object classes in it. Output: One or more than one bounding box highlighting the object classes.

Some top-performing DL models for object detection:

R-CNN

Fast R-CNN

Faster R-CNN

YOLO

YOLOv2 and YOLOv3

With Weakly Supervised Learning

While traditional supervised methods require bounding box annotations to learn such a task, Weakly Supervised Object Localization (WSOL) methods can learn with just image-level labels (such as the list of objects in the image).

Standard Supervised object detection Vs. Weakly-Supervised Approach

Standard object detection method

Weakly Supervised object detection

In a weakly-supervised approach, instead of providing with the ground truth labels, we do a lower degree annotation at train time than the required output at test time.

When the image is positive, at least one object instance from the target category is present.

When the image is negative, no object instance from the target category is present.

A DL approach to solving this problem would be by finding the Class Activation Map (CAM) of the objects of interest and highlight it with a bounding box. Class Activation Map was introduced in the paper Learning Deep Features for Discriminative Localization by using the Global Average Pooling in CNNs. A CAM indicates the discriminative region used by CNN to identify the category.

The architecture of CNN with CAM | Source

The Network primarily consists of numerous convolutional layers followed by Global Average Pooling right before the final output layer. In simple terms, we can say that CAM basically determines the region which has the primary components of the image determining the class of that image.

Now, we can use a simple thresholding method to segment the CAM and obtain a bounding box. Firstly, regions in the CAM whose value is above 20% of the max-value of the CAM are used to create a segmentation map. Then, a bounding box is drawn to tightly encompass the largest connected component in the segmentation map.

Localization results by Zhou et al. | Source

For implementing CAM (using transfer learning):

Follow this article: Demystifying Convolutional Neural Networks using GradCam

Stay tuned for custom model implementation. ;)

To read more on WSOL, the following are some papers based on it: