Object detection via color-based image segmentation using python | by Salma Ghoneim | Towards Data Science

If you already have jupyter notebook or an IDE with which you can run python & OpenCV installed, just skip to Execution.
Our hero today is Anaconda. a free open-source distribution that helps with installing different packages & sorts out their messes into isolated environments.
What Wikipedia’s telling us about Anaconda
Anaconda is a free and open-source distribution of the Python and R programming languages for scientific computing (data science, machine learning applications, large-scale data processing, predictive analytics, etc.), that aims to simplify package management and deployment. Package versions are managed by the package management system conda. The Anaconda distribution is used by over 12 million users and includes more than 1400 popular data-science packages suitable for Windows, Linux, and MacOS.
Here are detailed tutorials on how to download Anaconda.anaconda for Windows & anaconda for Linux.
Open the bash (cmd) and type this
$ conda create -n myEnv python=3
Type y (for yes) when prompted to download the packages.
$ source activate myEnv$ conda install anaconda$ conda activate myEnv$ conda install opencv$ jupyter notebook
This will open jupyter notebook in the browser for you.
Contours can be explained simply as a curve joining all the continuous points (along with the boundary), having the same color or intensity. The contours are a useful tool for shape analysis and object detection and recognition.
Applying thresholding on a grayscale image makes it a binary image. You set a threshold value, in which all values below this threshold are turned to black and all values above go white.
Now you have all you need to start.We’re gonna start with a simple example just to show you how color-based segmentation works.
Bear with me till we get to the good stuff.
If you want to try this with me, you can get this image for free from here.In the following code, I’m gonna segment this image into 17 gray levels. Then measure each level’s area using contouring.
import cv2import numpy as npdef viewImage(image):    cv2.namedWindow('Display', cv2.WINDOW_NORMAL)    cv2.imshow('Display', image)    cv2.waitKey(0)    cv2.destroyAllWindows()def grayscale_17_levels (image):    high = 255    while(1):          low = high - 15        col_to_be_changed_low = np.array([low])        col_to_be_changed_high = np.array([high])        curr_mask = cv2.inRange(gray, col_to_be_changed_low,col_to_be_changed_high)        gray[curr_mask > 0] = (high)        high -= 15        if(low == 0 ):            breakimage = cv2.imread('./path/to/image')viewImage(image)gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)grayscale_17_levels(gray)viewImage(gray)
def get_area_of_each_gray_level(im):## convert image to gray scale (must br done before contouring)    image = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)    output = []    high = 255    first = True    while(1):low = high - 15        if(first == False):# making values that are of a greater gray level black             ## so it won't get detected              to_be_black_again_low = np.array([high])            to_be_black_again_high = np.array([255])            curr_mask = cv2.inRange(image, to_be_black_again_low,             to_be_black_again_high)            image[curr_mask > 0] = (0)                    # making values of this gray level white so we can calculate        # it's area        ret, threshold = cv2.threshold(image, low, 255, 0)        contours, hirerchy = cv2.findContours(threshold,         cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)if(len(contours) > 0):output.append([cv2.contourArea(contours[0])])            cv2.drawContours(im, contours, -1, (0,0,255), 3)high -= 15        first = False        if(low == 0 ):breakreturn output
In this function, I simply convert the range (of intensities) of gray that I want to contour (highlight) in this iteration by unifying all of those which lie within this range to one intensity. I turn every other intensity but this range to black (including greater & smaller intensities).The second step I threshold the image so that only the color that I want to contour right now appears white and all others convert to black. This step doesn’t change much here but it must be done because contouring works best with black and white (thresholds) images.Before applying this step (thresholding) the image below would be the same except the white ring would’ve been gray (the gray intensity of the 10th gray level (255–15*10 ) )
image = cv2.imread('./path/to/image')print(get_area_of_each_gray_level(image))viewImage(image)
This way we obtain the area of each gray level.
Before we move on, I want to stress the importance of this topic.I’m a Computer Engineering student and I’m working on a project called Machine learning for intelligent tumor detection and identification.Color-based image segmentation is used in this project to help the computer learn how to detect the tumor. When dealing with an MRI scan, the program has to detect the cancer level of said MRI scan. It does that by segmenting the scan into different grayscale levels in which the darkest is the most filled with cancerous cells and the closest to white is the healthier parts. Then it calculates the degree of membership of the tumor to each grayscale level. With this information, the program is able to identify the tumor and its stage.This project is based on soft computing, fuzzy logic & machine learning, you can learn more about it on Fuzzy logic and how it is curing cancer.
You can get this image for free on Pexels from here. You just need to crop it.
In this image, we want to contour the leaf only. Since the texture of this image is very irregular and uneven, meaning that although there aren’t many colors. The intensity of the green color in this image changes, also, its brightness. So, the best thing to do here is to unify all these different shades of green into one shade. This way when we apply contouring, it will deal with the leaf as one whole object.
Note: This is the result if you apply contouring on the image without any pre-processing. I just wanted you to see how the uneven nature of the leaf makes OpenCV not understand that this is just one object.
import cv2import numpy as npdef viewImage(image):    cv2.namedWindow('Display', cv2.WINDOW_NORMAL)    cv2.imshow('Display', image)    cv2.waitKey(0)    cv2.destroyAllWindows()
First, you have to know the HSV representation of your color, you can know it by converting its RGB to HSV just like the following.
## getting green HSV color representationgreen = np.uint8([[[0, 255, 0 ]]])green_hsv = cv2.cvtColor(green,cv2.COLOR_BGR2HSV)print( green_hsv)
Converting the image to HSV: It’s easier with HSV to get the complete range of one color. HSV, H stands for Hue, S for Saturation, V for value. We already know that the green color is [60, 255, 255]. All the greens in the world lie within [45, 100, 50] to [75, 255, 255] that is [60–15, 100, 50] to [60+15, 255, 255]. 15 is just an approximation value.We take this range and convert it to [75, 255, 200] or any other light color (3rd value must be relatively large) cause that’s the brightness of the color, that’s the value that will make this part be white when we threshold the image.
image = cv2.imread('./path/to/image.jpg')hsv_img = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)viewImage(hsv_img) ## 1green_low = np.array([45 , 100, 50] )green_high = np.array([75, 255, 255])curr_mask = cv2.inRange(hsv_img, green_low, green_high)hsv_img[curr_mask > 0] = ([75,255,200])viewImage(hsv_img) ## 2## converting the HSV image to Gray inorder to be able to apply ## contouringRGB_again = cv2.cvtColor(hsv_img, cv2.COLOR_HSV2RGB)gray = cv2.cvtColor(RGB_again, cv2.COLOR_RGB2GRAY)viewImage(gray) ## 3ret, threshold = cv2.threshold(gray, 90, 255, 0)viewImage(threshold) ## 4contours, hierarchy =  cv2.findContours(threshold,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)cv2.drawContours(image, contours, -1, (0, 0, 255), 3)viewImage(image) ## 5
Since there seem to be irregularities in the background as well, We can get the largest contour using this method, The largest contour is, of course, the leaf.We can get the index of the leaf contour in the contours array, from that we get the area and center of the leaf. Contours have many other features that you can make use of such as the contour perimeter, convex hull, bounding rectangle, and many others. You can learn more about it from here.
def findGreatesContour(contours):    largest_area = 0    largest_contour_index = -1    i = 0    total_contours = len(contours)    while (i < total_contours ):        area = cv2.contourArea(contours[i])        if(area > largest_area):            largest_area = area            largest_contour_index = i        i+=1                return largest_area, largest_contour_index# to get the center of the contourcnt = contours[13]M = cv2.moments(cnt)cX = int(M["m10"] / M["m00"])cY = int(M["m01"] / M["m00"])largest_area, largest_contour_index = findGreatesContour(contours)print(largest_area)print(largest_contour_index)print(len(contours))print(cX)print(cY)