Question: I am currently working on developing an algorithm to determine centroid positions from (Brightfield) microscopy images of bacterial clusters. This is currently a major open problem in image processing. This question is a follow-up to: Python/OpenCV — Matching Centroid Points of Bacteria in Two Images. Currently, the algorithm is effective for sparse, spaced-out bacteria. However, it becomes totally ineffective when the bacteria become clustered together. In these images, notice how the bacterial centroids are located effectively. Bright-Field Image #1  Bright-Field Image #2  Bright-Field Image #3  However, the algorithm fails when the bacteria cluster at varying levels. Bright-Field Image #4  Bright-Field Image #5  Bright-Field Image #6  Bright-Field Image #7  Bright-Field Image #8  Original Images Bright-Field Image #1 Bright-Field Image #2 Bright-Field Image #3 Bright-Field Image #4 Bright-Field Image #5 Bright-Field Image #6 Bright-Field Image #7 Bright-Field Image #8 I'd like to optimize my current algorithm so it's more robust for these type of images. This is the program I'm running. import cv2 import numpy as np import os kernel = np.array([[0, 0, 1, 0, 0], [0, 1, 1, 1, 0], [1, 1, 1, 1, 1], [0, 1, 1, 1, 0], [0, 0, 1, 0, 0]], dtype=np.uint8) def e_d(image, it): image = cv2.erode(image, kernel, iterations=it) image = cv2.dilate(image, kernel, iterations=it) return image path = r"(INSERT IMAGE DIRECTORY HERE)" img_files = [file for file in os.listdir(path)] def segment_index(index: int): segment_file(img_files[index]) def segment_file(img_file: str): img_path = path + "\\" + img_file print(img_path) img = cv2.imread(img_path) img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # Applying adaptive mean thresholding th = cv2.adaptiveThreshold(img, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY_INV, 11, 2) # Removing small noise th = e_d(th.copy(), 1) # Finding contours with RETR_EXTERNAL flag and removing undesired contours and # drawing them on a new image. cnt, hie = cv2.findContours(th, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) cntImg = th.copy() for contour in cnt: x, y, w, h = cv2.boundingRect(contour) # Eliminating the contour if its width is more than half of image width # (bacteria will not be that big). if w > img.shape[1] / 2: continue cntImg = cv2.drawContours(cntImg, [cv2.convexHull(contour)], -1, 255, -1) # Removing almost all the remaining noise. # (Some big circular noise will remain along with bacteria contours) cntImg = e_d(cntImg, 3) # Finding new filtered contours again cnt2, hie2 = cv2.findContours(cntImg, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) # Now eliminating circular type noise contours by comparing each contour's # extent of overlap with its enclosing circle. finalContours = [] # This will contain the final bacteria contours for contour in cnt2: # Finding minimum enclosing circle (x, y), radius = cv2.minEnclosingCircle(contour) center = (int(x), int(y)) radius = int(radius) # creating a image with only this circle drawn on it(filled with white colour) circleImg = np.zeros(img.shape, dtype=np.uint8) circleImg = cv2.circle(circleImg, center, radius, 255, -1) # creating a image with only the contour drawn on it(filled with white colour) contourImg = np.zeros(img.shape, dtype=np.uint8) contourImg = cv2.drawContours(contourImg, [contour], -1, 255, -1) # White pixels not common in both contour and circle will remain white # else will become black. union_inter = cv2.bitwise_xor(circleImg, contourImg) # Finding ratio of the extent of overlap of contour to its enclosing circle. # Smaller the ratio, more circular the contour. ratio = np.sum(union_inter == 255) / np.sum(circleImg == 255) # Storing only non circular contours(bacteria) if ratio > 0.55: finalContours.append(contour) finalContours = np.asarray(finalContours) # Finding center of bacteria and showing it. bacteriaImg = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR) for bacteria in finalContours: M = cv2.moments(bacteria) cx = int(M['m10'] / M['m00']) cy = int(M['m01'] / M['m00']) bacteriaImg = cv2.circle(bacteriaImg, (cx, cy), 5, (0, 0, 255), -1) cv2.imshow("bacteriaImg", bacteriaImg) cv2.waitKey(0) # Segment Each Image for i in range(len(img_files)): segment_index(i)  Ideally I would like at least to improve on a couple of the posted images.

Answer: The mask is always the weak point in identifying objects, and the most important step. This will improve identifying images with high numbers of bacteria. I have modified your e_d function by adding an OPEN and another ERODE pass with the kernal, and changed the it (number of iterations) variable (to 1, 2 instead of 1,3) for your code to do this. This is by no means a finished effort, but I hope it will give you an idea of what you might try to enhance it further. I used the images you provided, and since they already have a red dot, this may be interfering with my result images... but you can see it is able to identify more bacteria on most. Some of my results show two dots, and the image with only one bacteria, I missed it, each quite possibly because it was already marked. Try it with the raw images and see how it does. Also, since the bacteria are relatively uniform in both size and shape, I think you could work with the ratio and/or average of height to width of each bacteria to filter out the extreme shapes (small or large) and the skinny, long shapes too. You can measure enough bacteria to see what is the average contour length, or height and width, or height/width ratio, etc., to find reasonable tolerances rather than the proportion to the image size itself. Another suggestion, would be to rethink how you are masking the images all together, possibly to try it in two steps. One to find the boundary of the long shape containing the bacteria, and then to find the bacteria within it. This assumes all of your images will be similar to these, and if that is so, it may help to eliminate the stray hits outside of this boundary, that are never bacteria.         #!usr/bin/env python # https://stackoverflow.com/questions/63182075/python-opencv-centroid-determination-in-bacterial-clusters import cv2 import numpy as np import os kernel = np.array([[0, 0, 1, 0, 0], [0, 1, 1, 1, 0], [1, 1, 1, 1, 1], [0, 1, 1, 1, 0], [0, 0, 1, 0, 0]], dtype=np.uint8) def e_d(image, it): print(it) image = cv2.erode(image, kernel, iterations=it) image = cv2.dilate(image, kernel, iterations=it) image = cv2.morphologyEx(image, cv2.MORPH_OPEN, kernel, iterations = 1) image = cv2.morphologyEx(image, cv2.MORPH_ERODE, kernel, iterations = 1) return image #path = r"(INSERT IMAGE DIRECTORY HERE)" path = r"E:\stackimages" img_files = [file for file in os.listdir(path)] def segment_index(index: int): segment_file(img_files[index]) def segment_file(img_file: str): img_path = path + "\\" + img_file print(img_path) head, tail = os.path.split(img_path) img = cv2.imread(img_path) img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) cv2.imshow("bacteriaImg-1", img) cv2.waitKey(0) # Applying adaptive mean thresholding th = cv2.adaptiveThreshold(img, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY_INV, 11, 2) # Removing small noise th = e_d(th.copy(), 1) # Finding contours with RETR_EXTERNAL flag and removing undesired contours and # drawing them on a new image. cnt, hie = cv2.findContours(th, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) cntImg = th.copy() for contour in cnt: x, y, w, h = cv2.boundingRect(contour) # Eliminating the contour if its width is more than half of image width # (bacteria will not be that big). if w > img.shape[1] / 2: continue else: cntImg = cv2.drawContours(cntImg, [cv2.convexHull(contour)], -1, 255, -1) # Removing almost all the remaining noise. # (Some big circular noise will remain along with bacteria contours) cntImg = e_d(cntImg, 2) cv2.imshow("bacteriaImg-2", cntImg) cv2.waitKey(0) # Finding new filtered contours again cnt2, hie2 = cv2.findContours(cntImg, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE) # Now eliminating circular type noise contours by comparing each contour's # extent of overlap with its enclosing circle. finalContours = [] # This will contain the final bacteria contours for contour in cnt2: # Finding minimum enclosing circle (x, y), radius = cv2.minEnclosingCircle(contour) center = (int(x), int(y)) radius = int(radius) # creating a image with only this circle drawn on it(filled with white colour) circleImg = np.zeros(img.shape, dtype=np.uint8) circleImg = cv2.circle(circleImg, center, radius, 255, -1) # creating a image with only the contour drawn on it(filled with white colour) contourImg = np.zeros(img.shape, dtype=np.uint8) contourImg = cv2.drawContours(contourImg, [contour], -1, 255, -1) # White pixels not common in both contour and circle will remain white # else will become black. union_inter = cv2.bitwise_xor(circleImg, contourImg) # Finding ratio of the extent of overlap of contour to its enclosing circle. # Smaller the ratio, more circular the contour. ratio = np.sum(union_inter == 255) / np.sum(circleImg == 255) # Storing only non circular contours(bacteria) if ratio > 0.55: finalContours.append(contour) finalContours = np.asarray(finalContours) # Finding center of bacteria and showing it. bacteriaImg = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR) for bacteria in finalContours: M = cv2.moments(bacteria) cx = int(M['m10'] / M['m00']) cy = int(M['m01'] / M['m00']) bacteriaImg = cv2.circle(bacteriaImg, (cx, cy), 5, (0, 0, 255), -1) cv2.imshow("bacteriaImg", bacteriaImg) cv2.waitKey(0) # Segment Each Image for i in range(len(img_files)): segment_index(i)