Image Segmentation: Part 1. Mathematical and practical... | by Mrinal Tyagi | Towards Data Science

Image segmentation is a method in which a digital image is broken down into various subgroups called Image segments which helps in reducing the complexity of the image to make further processing or analysis of the image simpler. Segmentation in easy words is assigning labels to pixels. All picture elements or pixels belonging to the same category have a common label assigned to them. For example: Let’s take a problem where the picture has to be provided as input for object detection. Rather than processing the whole image, the detector can be inputted with a region selected by a segmentation algorithm. This will prevent the detector from processing the whole image thereby reducing inference time.
Similarity approach: This approach is based on detecting similarity between image pixels to form a segment, based on a threshold. ML algorithms like clustering are based on this type of approach to segment an image.Discontinuity approach: This approach relies on the discontinuity of pixel intensity values of the image. Line, Point, and Edge Detection techniques use this type of approach for obtaining intermediate segmentation results which can be later processed to obtain the final segmented image.
Similarity approach: This approach is based on detecting similarity between image pixels to form a segment, based on a threshold. ML algorithms like clustering are based on this type of approach to segment an image.
Discontinuity approach: This approach relies on the discontinuity of pixel intensity values of the image. Line, Point, and Edge Detection techniques use this type of approach for obtaining intermediate segmentation results which can be later processed to obtain the final segmented image.
Threshold Based SegmentationEdge Based SegmentationRegion-Based SegmentationClustering Based SegmentationArtificial Neural Network Based Segmentation
Threshold Based Segmentation
Edge Based Segmentation
Region-Based Segmentation
Clustering Based Segmentation
Artificial Neural Network Based Segmentation
In this article, we will cover Threshold Based and Edge-based Segmentation. Other segmentation techniques will be discussed in later parts.
Image thresholding segmentation is a simple form of image segmentation. It is a way to create a binary or multi-color image based on setting a threshold value on the pixel intensity of the original image.
In this thresholding process, we will consider the intensity histogram of all the pixels in the image. Then we will set a threshold to divide the image into sections. For example, considering image pixels ranging from 0 to 255, we set a threshold of 60. So all the pixels with values less than or equal to 60 will be provided with a value of 0(black) and all the pixels with a value greater than 60 will be provided with a value of 255(white).
Considering an image with a background and an object, we can divide an image into regions based on the intensity of the object and the background. But this threshold has to be perfectly set to segment an image into an object and a background.
Various thresholding techniques are :
Global thresholding: In this method, we use a bimodal image. A bimodal image is an image with 2 peaks of intensities in the intensity distribution plot. One for the object and one for the background. Then we deduce the threshold value for the entire image and use that global threshold for the whole image. A disadvantage of this type of threshold is that it performs really poorly during poor illumination in the image.Manual thresholding: The following process goes as follows -
Global thresholding: In this method, we use a bimodal image. A bimodal image is an image with 2 peaks of intensities in the intensity distribution plot. One for the object and one for the background. Then we deduce the threshold value for the entire image and use that global threshold for the whole image. A disadvantage of this type of threshold is that it performs really poorly during poor illumination in the image.
Manual thresholding: The following process goes as follows -
3. Adaptive Thresholding: To overcome the effect of illumination, the image is divided into various subregions, and all these regions are segmented using the threshold value calculated for all these regions. Then these subregions are combined to image the complete segmented image. This helps in reducing the effect of illumination to a certain extent.
4. Optimal Thresholding: Optimal thresholding technique can be used to minimize the misclassification of pixels performed by segmentation.
Calculation of an optimal threshold using an iterative method by minimizing the misclassification loss of a pixel.
The probability of a pixel value is given by the following formulae :
We consider a threshold value T as the initial threshold value. If the pixel value of the pixel to be determined is less or equal to T then it belongs to the background, else it belongs to the object.
E1: Error if a background pixel is misclassified as an object pixel.
E2: Error if an object pixel is misclassified as a background pixel.
E(T): Total Error in the classification of pixels as background or object. To obtain great segmentation, we have to minimize this error. We do so by taking the derivative of the following equation and equating it to zero.
Consider a gaussian pixel density, the value of P(Z) can be calculated as :
After this the new value of T can be calculated by inputting it into the following equation:
5. Local Adaptive Thresholding: Due to variation in the illumination of pixels in the image, global thresholding might have difficulty in segmenting the image. Hense the image is divided into smaller subgroups and then adaptive thresholding of those individual groups is done. After individual segmentation of these subgroups, all of them are combined to form the completed segmented image of the original image. Hence, the histogram of subgroups helps in providing better segmentation of the image.
import numpy as np
import cv2
import matplotlib.pyplot as plt
import seaborn as sns
import warnings

warnings.filterwarnings("ignore")

image = cv2.imread("/content/1.jpg", 0)

plt.figure(figsize=(8, 8))
plt.imshow(image, cmap="gray")
plt.axis("off")
plt.show()

flattened_image = image.reshape((image.shape[0] * image.shape[1],))

flattened_image.shape

(6553600,)
plt.figure()
sns.distplot(flattened_image, kde=True)
plt.show()

ret, thresh1 = cv2.threshold(image, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)

plt.figure(figsize=(8, 8))
plt.imshow(thresh1, cmap="binary")
plt.axis("off")
plt.show()

Edge-based segmentation relies on edges found in an image using various edge detection operators. These edges mark image locations of discontinuity in gray levels, color, texture, etc. When we move from one region to another, the gray level may change. So if we can find that discontinuity, we can find that edge. A variety of edge detection operators are available but the resulting image is an intermediate segmentation result and should not be confused with the final segmented image. We have to perform further processing on the image to the segment it. Additional steps include combining edges segments obtained into one segment in order to reduce the number of segments rather than chunks of small borders which might hinder the process of region filling. This is done to obtain a seamless border of the object. The goal of edge segmentation is to get an intermediate segmentation result to which we can apply region-based or any other type of segmentation to get the final segmented image.
Types of edges
Edges are usually associated with “Magnitude” and “Direction”. Some edge detectors give both directions and magnitude. We can use various edge detectors like Sobel edge operator, canny edge detector, Kirsch edge operator, Prewitt edge operator, Robert’s edge operator, etc.
Any of the three formulae can be used to calculate the value of g. After calculation of g and theta, we obtain the edge vector with both magnitudes as well as direction.
Region based Segmentation Operation. Thresholding Techniques, https://www.youtube.com/watch?v=4S7BezwSAmQEdge Based Segmentation, https://www.youtube.com/watch?v=ygVCjURa7ZEUniversity Resource, https://www.cse.unr.edu/~bebis/CS791E/Notes/Thresholding.pdf
Region based Segmentation Operation. Thresholding Techniques, https://www.youtube.com/watch?v=4S7BezwSAmQ
Edge Based Segmentation, https://www.youtube.com/watch?v=ygVCjURa7ZE
University Resource, https://www.cse.unr.edu/~bebis/CS791E/Notes/Thresholding.pdf
I hope that you find this article and explanation useful. Stay tuned for other segmentation techniques in upcoming parts.
Feel free to connect and give ur suggestions: https://www.linkedin.com/in/mrinal-tyagi-02a1351b1/