Intro to Segmentation
By: Johar M. Ashfaque
Tags: Artificial Intelligence, Machine Learning, Image Segmentation, Deep Learning

Image Segmentation is the process by which a digital image is partitioned into various subgroups (of pixels) called Image Objects, which can reduce the complexity of the image, and thus analysing the image becomes simpler.

We use various image segmentation algorithms to split and group a certain set of pixels together from the image. By doing so, we are actually assigning labels to pixels and the pixels with the same label fall under a category where they have some or the other thing common in them.

Using these labels, we can specify boundaries, draw lines, and separate the most required objects in an image from the rest of the not-so-important ones.

Need for Image Segmentation

The concept of partitioning, dividing, fetching, and then labelling and later using that information to train various ML models have indeed addressed numerous problems.

Segmentation in Image Processing is being used in the medical industry for efficient and faster diagnosis, detecting diseases, tumors, and cell and tissue patterns from various medical imagery generated from radiography, MRI, endoscopy, etc.

This is a basic, but a pivotal and significant application of Image Classification, where the algorithm was able to capture only the required components from an image, and those pixels were later being classified as the good, the bad, and the ugly by the system. A rather simple looking system was making a colossal impact on that business — eradicating human effort, human error and increasing efficiency.

The Approach

Similarity Detection (Region Approach)

This fundamental approach relies on detecting similar pixels in an image — based on a threshold, region growing, region spreading, and region merging. Machine learning algorithms like clustering relies on this approach of similarity detection on an unknown set of features, so does classification, which detects similarity based on a pre-defined (known) set of features.

Discontinuity Detection (Boundary Approach)

This is a stark opposite of similarity detection approach where the algorithm rather searches for discontinuity. Image Segmentation Algorithms like Edge Detection, Point Detection, Line Detection follows this approach — where edges get detected based on various metrics of discontinuity like intensity etc.

The Types

Based on the two approaches, there are various forms of techniques that are applied in the design of the Image Segmentation Algorithms. These techniques are employed based on the type of image that needs to be processed and analysed and they can be classified into three broader categories as below:

Structural Segmentation Techniques

These sets of algorithms require us to firstly, know the structural information about the image under the scanner. This can include the pixels, pixel density, distributions, histograms, colour distribution etc. Second, we need to have the structural information about the region that we are about to fetch from the image — this section deals with identifying our target area, which is highly specific to the business problem that we are trying to solve. Similarity based approach will be followed in these sets of algorithms.

Stochastic Segmentation Techniques

In these group of algorithms, the primary information that is required for them is to know the discrete pixel values of the full image, rather than pointing out the structure of the required portion of the image. This proves to be advantageous in the case of a larger group of images, where a high degree of uncertainty exists in terms of the required object within an object. ANN and Machine Learning based algorithms that use k-means etc. make use of this approach.

Hybrid Techniques

As the name suggests, these algorithms for image segmentation make use of a combination of structural method and stochastic methods i.e., use both the structural information of a region as well as the discrete pixel information of the image.

Image segmentation Techniques

Based on the image segmentation approaches and the type of processing that is needed to be incorporated to attain a goal, we have the following techniques for image segmentation.

Threshold Method:

Focuses on finding peak values based on the histogram of the image to find similar pixels.

Edge Based Segmentation:

Based on discontinuity detection unlike similarity detection.

Region Based Segmentation:

Based on partitioning an image into homogeneous regions.

Clustering Based Segmentation:

Divides image into k number of homogeneous, mutually exclusive clusters — hence obtaining objects.

Watershed Based Method:

Based on topological interpretation of image boundaries.

Artificial Neural Network Based Segmentation:

Based on deep learning algorithms especially Convolutional Neural Networks.

Deep Dive

Images are considered as one of the most important

medium of conveying information, in the field of computer

vision, by understanding images the information extracted

from them can be used for other tasks. An image is a word derived from Latin word ‘imago’, which is a representation of visual perception in a two-dimension or three-dimension picture that has a similar appearance to some subject.

A digital image is a numeric representation of a

two-dimensional image. A digital image is composed of a

finite number of elements, each of which has a particular

location and value, are called picture elements, image

elements called pixels. Pixels are the smallest

individual element in an image, holding finite, discrete,

quantized values that represent the brightness, intensity or

gray level at any specific point.

There are generally two types of image- raster type and

vector type. Raster images are images having a finite set of

digital values which are represented in a fixed number of rows

and columns of pixels where these pixels are stored in

memory as a two-dimensional array. Digital images are

usually referred as raster images. Vector images are images

generated from mathematical geometry known as vector

which have points having both magnitude and direction.

Image segmentation is the foundation of object recognition

and computer vision. Image segmentation is the process of

subdividing a digital image into multiple regions or objects

consisting of sets of pixels sharing same properties or

characteristics which are assigned different labels for

representing different regions or objects. The goal of

segmentation is to simplify and/or change the representation

of an image into something that is more meaningful and easier

to analyse. Image segmentation is used to locate objects

and boundaries in images. Segmentation is done on basis of

similarity and discontinuity of the pixel values.

There are two types of segmentations — soft segmentations

and hard segmentations. Segmentations that allow regions or

classes to overlap are called soft segmentations whereas a

hard segmentation forces a decision of whether a pixel is

inside or outside the object.

Image segmentation is practically implemented in many

applications such as medical imaging, content based image

retrieval, object detection, feature recognition (such as face

recognition, fingerprint recognition, iris recognition, object

recognition, etc) and real-time object tracking in video.

The following computational steps have to applied for image

segmentation process on the image taken as input to get the

required segmented data:

1)Preprocessing: The main aim of the preprocessing step

is to determine the area of focus in the image. As the input

image may have a certain amount of noise in the images, it is

necessary to reduce or remove the noise.

2) Image Segmentation: In this step, the preprocessed

image is segmented in its constituent sub-regions.

3) Post Processing: To improve the segmented image,

further processing may be required which is performed in post

processing step.

4) Feature Extraction: Feature extraction is the method in

which unique features of an image are extracted. This method

helps in reducing the complexity in classification problems

and the classification can be made more efficient.

Different kind of features present in an image can be

intensity-based, textural, fractal, topological, morphological,

etc.

5) Classification: The aim of the classification step is to

classify the segmented image by making use of extracted

features. This step uses statistical analysis of the features and machine learning algorithms to reach a decision.