# A Novel Method for Wheat Spike Phenotyping Based on Instance Segmentation and Classification
## 2 Materials and Methods
### Spikelet Instance Segmentation Method  
#### 2.2.1 Image Perspective Transformation  
As the wheat spike images were collected by a hand-held smartphone camera at different times in the field, the collected images could not be kept uniform in the collection distance and angle [21]. Therefore, varying degrees of perspective distortion appeared, as shown in Figure 3a. To improve the unity of the input images and the accuracy of the segmentation results, this study conducted the perspective transformation on the wheat spike image. The original image was first converted from RGB to grayscale, and the white box of the black board was extracted by the OTSU threshold algorithm. The OTSU algorithm is a method used for automatic image thresholding to separate pixels into two classes based on maximizing the between-class variance. Still, the wheat rachis occluded the bottom of the white box, causing a gap in the white box. A morphological closing operation is used to process the white box to close it, as shown in Figure 3b.  
Then, the internal contour of the white box was fitted by the polygonal approximation method. The maximum distance threshold between the contour and the fitting polygon was 0.1 times the contour perimeter. The internal contour of the white box with perspective deformation was fitted as a quadrilateral by polygonal approximation, and the four vertices of the fitted quadrilateral were regarded as the four corner points of the internal contour, as shown in Figure 3c. According to Equation (1), the perspective transformation matrix \(M\) was obtained through four pairs of coordinates: the four corner points and their perspective coordinates. Since the perspective transformation deleted some pixels of the image, the size  
Figure 3: Perspective transformation of wheat spike image. (**a**) Original image; (**b**) white box; (**c**) corner points of internal contour; (**d**) perspective image.  
Figure 2: Wheat spike image collection.  
of the perspective image was set as 2400 \(\times\) 2400 pixels uniformly, where the perspective coordinates of the four corner points were (0, 0), (2400, 0), (0, 2400), and (2400, 2400), respectively. Therefore, the perspective coordinates of each pixel in the spike image were obtained through Equations (1) and (2), and the original spike image was projected to a new unified plane to realize the perspective transformation, as shown in Figure 3d.  
\[\begin{bmatrix}x^{\prime}\\ y^{\prime}\\ z^{\prime}\end{bmatrix}=M\begin{bmatrix}x\\ y\\ z\end{bmatrix} \tag{1}\]  
\[M=\begin{bmatrix}T_{1}&T_{1}\\ T_{1}&a_{33}\end{bmatrix} \tag{2}\]  
where (\(x\),\(y\)) are the coordinates of the pixel in the original spike image, \(z\) defaults to 1, \(M\) is the perspective transformation matrix, and (\(x\)',\(y\)') are the perspective coordinates. \(T_{1}=\begin{bmatrix}a_{11}&a_{12}\\ a_{21}&a_{22}\end{bmatrix}\) is used for the linear transformation, \(T_{2}=\begin{bmatrix}a_{13}&a_{23}\end{bmatrix}^{T}\) is for the perspective transformation, and \(T_{3}=\begin{bmatrix}a_{31}&a_{32}\end{bmatrix}\) is for the image translation.