# Improved Mask R-CNN Combined with Otsu Preprocessing for Rice Panicle Detection and Segmentation
## 2 Dataset and Method
### Method
#### 2.2.1 Otsu Preprocessing  
Otsu is a traditional adaptive thresholding algorithm for grayscale image segmentation [35]. In this case, rice panicles can be segmented based on the grayscale pixel distribution in an image. The rice panicle in a rice image shows a clear contrast with the background. First, the excess green index (ExG) is calculated based on the dataset [38]. Then, the collected image is converted to grayscale using the ExG, and the Otsu algorithm is used to calculate the threshold to transform the grayscale image into a binary image that allows segmentation without leakage. For the collected rice color images, the values of the rice panicle and background in the R, G, and B color components have different characteristics. In creating the ExG grayscale images, the original image is first separated into three independent primary color planes, and different color feature combinations are selected. Then, each pixel in the image is transformed to enhance the contrast between the target and the background. The ExG is obtained by the system after automatically exhausting the color components and adding human supervised judgement. Equation (1) is the RGB linear combination calculation formula, and the best combination is calculated by the dataset, that is, the ExG. This combination is robust to light and color changes and can more completely extract the features of the part of the image corresponding to the ears of rice. Compared to the original images, the ExG-based extraction of green plant images is better; shadows, withered grass, and soil elements can be substantially suppressed, and areas representing the plant are more prominent. For crop or weed recognition, the most commonly used gray method is the ExG method. Equation (2) represents a common ExG expression. For segmentation based on the Otsu algorithm, the ExG is calculated based on the dataset and then used to perform feature extraction to segment the rice panicles. Equation (3) shows the ExG expression in this paper, where R and G are the red and green color components of the image, respectively. \[\text{T(i,j)}=\text{rR(i,j)}+\text{gG(i,j)}+\text{bB(i,j)} \tag{1}\]  
\[\text{ExG}=2\text{G(i,j)}-\text{R(i,j)}-\text{B(i,j)} \tag{2}\]  
Figure 3: Panicle-Mask rice panicle detection and segmentation algorithm flowchart. \[\text{Ex}G^{\prime}=255-\text{G(i,j)}+\text{R(i,j)} \tag{3}\]  
where T(i,j) is the resultant feature after the linear operation, R(i,j), G(i,j), and B(i,j) are the grayscale values of the red, green and blue color components, respectively, of the image at (i,j), r, g, and b are the linear coefficients of the color components R(i,j), G(i,j), and B(i,j), and (i,j) denotes the two-dimensional array variables of the color components. Equation (4) is the Otsu algorithm formula:  
\[ \begin{array}{l}\text{I}=1,\text{I}\geq\text{threshold}\\ \text{I}=0,\text{I}<\text{threshold}\end{array}  \tag{4}\]  
where I is the gray value of image pixels, and threshold is the threshold that maximizes the variance between all pixel classes in the gray map. When the pixel gray value in the image is greater than the threshold, the pixel corresponds to a rice panic; otherwise, it belongs to the background area. According to the underlying principle of the Otsu algorithm, to obtain a better segmentation effect, the variance between the classes needs to be maximized while minimizing the within-class variances in the target and background. Thus, the variance is a measure of the uniformity of the grey scale distribution; the larger the value is, the greater the difference between the target and the background after segmentation at a given threshold, and the more favorable the feature extraction [35]. When part of the target is misclassified into the background or the background is misclassified into the target, the difference between the two parts of the image decreases; thus, by achieving the largest variance between classes, the probability of misalignment will be minimal. This method can segment out the ears of rice and will not mistake them as background.