<figure><img src="image_1.png" /></figure>

HAL
open science

# Feature Extraction and Recognition Based on Machine Vision Application in Lotus Picking Robot

Shuping Tang, Dean Zhao, Weikuan Jia, Yu Chen, Wei Ji, Chengzhi Ruan

► To cite this version:

Shuping Tang, Dean Zhao, Weikuan Jia, Yu Chen, Wei Ji, et al.. Feature Extraction and Recognition Based on Machine Vision Application in Lotus Picking Robot. 9th International Conference on Computer and Computing Technologies in Agriculture (CCTA), Sep 2015, Beijing, China. pp.485-501, 10.1007/978-3-319-48357-3_46 . hal-01557795

HAL Id: hal-01557795

https://inria.hal.science/hal-01557795

Submitted on 6 Jul 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Feature Extraction and Recognition Based on
Machine Vision Application in Lotus Picking Robot

Shuping Tang<sup>1,a,*</sup>, Dean Zhao<sup>1,2,b</sup>, Weikuan Jia<sup>1,c</sup>, Yu Chen<sup>1,2,d</sup>, Wei Ji<sup>1,2,e</sup>, Chengzhi

Ruan<sup>1,f</sup>

<sup>1</sup>School of Electrical and Information Engineering, Jiangsu University, Zhenjiang 212013, China; <sup>2</sup>Key Laboratory of Facility Agriculture Measurement and Control Technology and Equipment of Machinery Industry, Jiangsu University, Zhenjiang 212013, China

<sup>a</sup>1015412838@qq.com, <sup>b</sup>dazhao@ujs.edu.cn, <sup>c</sup>jwk_1982@163.com, <sup>d</sup>chenyu@ujs.edu.cn, <sup>e</sup>jiwei@ujs.edu.cn, <sup>f</sup>ruanczhi@163.com

**Abstract.** Recently the picking technology of high value crops has become a new research hot spot, and the image segmentation and recognition are still the key link of fruit picking robot. In order to realize the lotus image recognition, this paper proposes a new feature extraction method combined with shape and color, and uses the K-Means clustering algorithm to get lotus recognition model. Before the feature extraction, the existing pulse coupled neural network segmentation algorithm, combined with morphological operation, is used to achieve nice segmentation image, including lotus, lotus flower, lotus leaf and stems. Then in the feature extraction processing, the chromatic aberration method and the moment invariant algorithm are selected to extract the color and shape features of the segmented images, in which principal component analysis algorithm is selected to reduce the dimension of the color and shape features to achieve principal components of lotus, lotus flower, lotus leaf and stems. In the experiment, K-Means clustering algorithm is used to get lotus recognition model and four clustering centers according to above principal components of training samples about lotus, lotus flower, lotus leaf and stems; then the testing experiment is applied to validate the recognition model. Experimental results shows that the correct recognition rate is 90.57% about 53 testing samples of lotus, and the average recognition time is 0.0473s, which further indicates that the feature extraction algorithm is applicable to lotus feature extraction, and
K-Means algorithm is simple, reliable and feasible, providing a theoretical basis for positioning and picking of lotus harvest robot.

**Keywords:** Picking Robot of lotus, Feature Extraction, Invariant moment, Principal component analysis, K-Means clustering

# 1 Introduction

Recently, the picking technology of high value crops has become a new research hot spot, and lotus belongs to high value crops because of its great medical value, high risk of picking and high labor input required[1~3]. Fresh lotus seeds have great medicinal value in calming nerves, having healthy brain, developing intelligence and removing fatigue, but the high risk and low efficiency of manual picking restrict the lotus market development. Therefore the study of lotus picking robot has broad research prospect.

The image segmentation and recognition algorithms are key link of lotus harvesting robot vision system, and the ability to accurately and quickly identify the lotus directly affects the reliability and timeliness of lotus harvesting robot[4~6]. Then many factors increase the difficulty of lotus recognition, for example, the growth of lotus is changeable and mature lotus, lotus leaf, lotus flower and lotus stem often densely distribute in an area of the image, usual recognition algorithm based on RGB components[7,8] is difficult to distinguish the lotus from the complex background because the colors of lotus leaf and lotus stem are closer than lotus.

However it is obvious about color difference between lotus and lotus flower, and shape difference between lotus and lotus flower and lotus stem, so in this study, the chromatic aberration method and the moment invariant algorithm are selected to extract the color and shape features from the divided images[9~13], in which there are seven moment invariant shape characteristics and one color feature (r-g) extracted from each connected domain in divided image. The moment invariant algorithm is simple and reliable, but has too many extracted characteristic variables, that causes the calculation is too big, and the time of recognition is too long, then the principal component analysis (PCA) algorithm is introduced to reduce the dimensionality of color and shape information in order to accelerate the speed of image recognition[14~17].
K-Means clustering algorithm is simple, reliable and suitable for high dimensional data clustering[18~21]. Due to the higher dimension of sample characteristics and the rapid identification requirement, we choose the most classical K-Means algorithm to cluster the extracted training feature set in the case of ensuring the recognition accuracy and to get four clustering centers of lotus, lotus leaf, lotus flower and lotus stem in order to identify lotus goal.

This paper describes a machine vision recognition system of lotus picking robot, and proposes a new shape and color feature extraction method combined with K-Means clustering algorithm to get lotus recognition model. Before the feature extraction, the existing pulse coupled neural network segmentation algorithm(PCNN), combined with morphological operation[22~25], is used to achieve nice segmentation image, including lotus, lotus flower, lotus leaf and stems. Then the chromatic aberration method and the moment invariant algorithm are selected to extract the color and shape features of the segmented images, in which principal component analysis algorithm is selected to reduce the dimension of the color and shape features to achieve principal components of lotus, lotus flower, lotus leaf and stems.

# 2 Image Acquisition and Segmentation

## 2.1 Image Acquisition

Lotus pictures researched in this paper are taken from the lotus pond of Jiangsu university in Jiangsu province. In here, we choose by CCD color light source with 1024x960 pixels resolution to get lotus color images. Then 120 images, used for training and testing experiments, are closely shot with the method of elevation shooting on the side of lotus live action, in which we adopt 15 to 30 degrees elevation through the experience summary, and each picture include lotus and lotus stem with lotus flower or lotus leaf. Moreover all lotus pictures are shot under natural light conditions, and saved by JPG format with the resolution of 320 x 240 pixels in order to improve the seed of image process. Figure 1 is a lotus real picture.
<figure><img src="image_3.png" /><figcaption>Fig.1 Lotus color picture</figcaption></figure>

## 2.2 Image Segmentation Algorithm Introduction

This paper chooses the existing PCNN segmentation algorithm proposed by Ma Yide professor from Lanzhou University to get the segmentation images. The method uses PCNN segmentation algorithm and morphological image processing algorithm to divide the acquired image[22,23]. Neuron model of simplified PCNN is shown in Figure 2.

<figure><img src="image_4.png" /><figcaption>Fig.2 Neuron model of simplified PCNN</figcaption></figure>

Mathematical model of simplified PCNN neurons describes as follows:

$$F_{ij}(n) = S_{ij} \qquad (1)$$

$$L_{ij}(n) = V_L \cdot \sum W_{ijkl} Y_{kl}(n-1) \qquad (2)$$

$$U_{ij}(n) = F_{ij}(n)(1 + \beta L_{ij}(n)) \qquad (3)$$

$$E_{ij}(n) = e^{-\alpha_E} \cdot E_{ij}(n-1) + V_E \cdot Y_{ij}(n-1) \qquad (4)$$
$$Y_{ij}(n) = \begin{cases} 1 & (U_{ij}(n) > E_{ij}(n-1)) \\ 0 & (U_{ij}(n) \le E_{ij}(n-1)) \end{cases} \qquad (5)$$

In these equations, $F_{ij}$ is neural input item, $S_{ij}$ is external neural input excitation, which is corresponding to the eigenvalues of pixel $(i, j)$, $L_{ij}$ is connection input item, $Y_{kl}$ is pulse output item, $W_{ijkl}$ is connecting factor of coupling connection field, $V_L$ is the connection power amplification factor of $L_{ij}$, $U_{ij}$ is internal activity item, $\beta$ is the connecting factor of internal activity item, $E_{ij}$ is dynamic threshold item, $\alpha_E$ is the iterative damping time constant of $E_{ij}$, and $V_E$ is amplification factor of dynamic threshold.

In this paper, we set up $V_L \cdot \beta = D$, and $W_{ijkl} = [0.5/D \ 1/D \ 0.5/D \ 1/D \ 1/D \ 0.5/D \ 1/D \ 0.5/D]$. We select figure 1 as an example, and after PCNN segmentation and morphological noise reduction, we can get the initial segmentation figure of figure 1 as figure 3.

<figure><img src="image_5.png" /><figcaption>Fig.3 Segmentation Image of Figure 1 by PCNN</figcaption></figure>

Then we use the corrosion and expansion morphology method to remove the lotus stem connected with lotus and lotus flower in figure 3.
# 3 Image Feature Extraction

Considering the color differences between lotus and lotus flower and the shape differences between lotus, lotus leaf and lotus stem, this paper takes the color and shape feature extraction algorithm to extract the color and shape features of target image.

## 3.1 Color Feature Extraction

This paper uses color difference method to extract the target color feature based on the difference between lotus and lotus flower.

In the RGB color space, colors can be compared directly, and the color characteristic information can be controlled easily, so we can choose RGB color difference method for feature extraction[26,27]. The classic color feature extraction method is selected to extract the color features of target image, in which R component and G component are regarded as color feature according the color difference between lotus and lotus flower. We choose $(r-g)$ on behalf of color difference, and $(x, y)$ on behalf of arbitrary pixel values in the image, then the formula of color difference is

$$r - g = \bar{R}(x, y) - \bar{G}(x, y) \qquad (6)$$

Where $\bar{R}(x, y)$ is average pixels of red component, $\bar{G}(x, y)$ is average pixels of green component.

## 3.2 Shape Feature Extraction

Classic Hu invariant moments algorithm combined with PCA algorithm is selected to extract the shape features of segmented image considering about the significant shape differences of lotus, lotus leaf, lotus stem, and changeable growth posture of lotus. Moment invariant algorithm processes the area of the target, and has simple concept, invariant translation, scaling and rotation, and stable and reliable recognition rate, which can avoid the shape deviation caused by changeable growth posture of lotus.
PCA algorithm is used to get the invariant feature extraction to obtain comprehensive
shape characteristic descriptions of the target image, which avoids the low recognition
rate caused by invariant moment data redundancy.

3.2.1 Hu Invariant Moment Feature Extraction

Hu invariant moments is put into use for extracting the shape features of each
connected area in target image, the seven invariant moments are as follows:

$$
\begin{align*}
a_1 &= \eta_{02} + \eta_{20} \\
a_2 &= (\eta_{20} - \eta_{02})^2 + 4\eta_{11}^2 \\
a_3 &= (\eta_{30} - 3\eta_{12})^2 + (3\eta_{21} - \eta_{03})^2 \\
a_4 &= (\eta_{30} + \eta_{12})^2 + (\eta_{21} + \eta_{03})^2 \\
a_5 &= (\eta_{30} - 3\eta_{12})(\eta_{30} + \eta_{12})[(\eta_{30} + \eta_{12})^2 - 3(\eta_{21} + \eta_{03})^2] + (3\eta_{21} - \eta_{03}) \\
    &\qquad (\eta_{21} + \eta_{03})[3(\eta_{30} + \eta_{12})^2 - (\eta_{21} + \eta_{03})^2] \\
a_6 &= (\eta_{20} - \eta_{02})[(\eta_{30} + \eta_{12})^2 - (\eta_{21} + \eta_{03})^2] + 4\eta_{11}(\eta_{30} + \eta_{12})(\eta_{21} + \eta_{03}) \\
a_7 &= (3\eta_{21} - \eta_{03})(\eta_{30} + \eta_{12})[(\eta_{30} + \eta_{12})^2 - 3(\eta_{21} + \eta_{03})^2] + (3\eta_{12} - \eta_{03}) \\
    &\qquad (\eta_{21} + \eta_{03})[3(\eta_{30} + \eta_{12})^2 - (\eta_{21} + \eta_{03})^2]
\end{align*}
$$

Where, $\eta_{pq}$ is normalized central moment, which can ensure the rotation,
translation and scaling invariance of Hu invariant moments. We get the normalized
center moment through the following steps.

Firstly, we set (x, y) as one pixel point on the connected domain, f(x, y) is
the corresponding gray value of (x, y), then the (p + q) moment of the entire
connected domain is

$$
m_{pq} = \sum_{x} \sum_{y} f(x, y) x^p y^q \quad (8)
$$

From the above formula, we can get the image barycentric coordinates is

$$
\begin{align}
x_0 &= m_{10}/m_{00} \\
y_0 &= m_{01}/m_{00}
\end{align}
$$
Where, $m_{00}$ and $m_{01}, m_{10}$ are zero-order and first-order moments of the target image respectively.

Then, we use barycentric coordinates to get the center distance which is as follows

$$ \mu_{pq} = \sum_{x} \sum_{y} (x - x_0)^p (y - y_0)^q f(x, y) \quad (10) $$

From the above formula, we know that central distance has translation invariance.

Then we need normalize the central distance in order to get the normalization central distance which has rotation, translation and scaling invariance. The formula of normalization central distance is as follows:

$$ \eta_{pq} = \frac{\mu_{pq}}{\mu_{00}^{1+(p+q)/2}} \quad (p+q=2,3,4,\dots) \quad (11) $$

By the above principle formula, we can get the seven Hu invariant moment shape feature quantities of the lotus, lotus leaf, lotus flower, lotus stem.

### 3.2.2 PCA Feature Reduction

We can get seven Hu invariant moments which are used to describe the shape feature amounts of the lotus, lotus leaf, lotus flower, lotus stem from the section 3.2.1 feature extraction principle. But the calculated amount of Hu invariant moments is bigger and the dimension of feature amount is too high, PCA algorithm is introduced, to get on the feature extraction, in order to reduce data redundancy and avoid curse of dimensionality. The PCA algorithm gets the linear combination for the obtained seven invariant moments, and extracts the principal components which can able to represent the different shape features of lotus, lotus leaf, lotus flower and lotus stem.

Suppose, the sample size is n, each sample has p invariant moments, so there is a $n \times p$ matrix,

$$ A = \begin{bmatrix} a_{11} & a_{12} & \Lambda & a_{1p} \\ a_{21} & a_{22} & \Lambda & a_{2p} \\ \mathbf{M} & \mathbf{M} & \mathbf{M} & \mathbf{M} \\ a_{n1} & a_{n2} & \Lambda & a_{np} \end{bmatrix} \quad (12) $$

To calculate the correlation coefficient matrix $r_{ij}$,
$$r_{ij} = \frac{\sum_{k=1}^{n} (a_{ki} - \bar{a}_i)(a_{kj} - \bar{a}_j)}{\sqrt{\sum_{k=1}^{n} (a_{ki} - \bar{a}_i)^2 \sum_{k=1}^{n} (a_{kj} - \bar{a}_j)^2}} \quad (13)$$

According to the correlation coefficient matrix R to construct characteristic equation $|\lambda I - R| = 0$, and to calculate the eigenvalue, and sorted them by their value, that is, $\lambda_1 \ge \lambda_2 \ge \dots \ge \lambda_p$.

By the eigenvalue, to calculate the principal component contribution rate $\alpha$ and the accumulative contribution rate $\eta$,

$$\alpha = \lambda_i / \sum_{i=1}^{p} \lambda_i \quad (i = 1, 2, \dots, p) \quad (14)$$

$$\eta = \sum_{i=1}^{m} \lambda_i / \sum_{i=1}^{p} \lambda_i \quad (i = 1, 2, \dots, p) \quad (15)$$

The front $m$ eigenvalues make $\eta \ge 85\%$, to calculate their corresponding eigenvectors, and to calculate the loading $l_{ij}$ of invariant moments $a_1, a_2, \dots, a_7$ load on the principal components $z_1, z_2, \dots, z_m$

$$l_{ij} = p(z_i, a_j) = \sqrt{\lambda_i} e_{ij}^{\omega} \quad (i, j = 1, 2, \dots, p) \quad (16)$$

And extract the front $m$ principal components

$$Z = \begin{bmatrix} z_1 \\ z_2 \\ K \\ z_m \end{bmatrix} = \begin{bmatrix} l_{11} & l_{12} & K & l_{17} \\ l_{21} & l_{22} & K & l_{27} \\ K & K & K & K \\ l_{m1} & l_{m2} & K & l_{m7} \end{bmatrix} \begin{bmatrix} a_1 \\ a_2 \\ K \\ a_7 \end{bmatrix} = \begin{cases} l_{11}a_1 + l_{12}a_2 + l_{13}a_3 + K + l_{17}a_7 \\ l_{21}a_1 + l_{22}a_2 + l_{23}a_3 + K + l_{27}a_7 \\ K \\ l_{m1}a_1 + l_{m2}a_2 + l_{m3}a_3 + K + l_{m7}a_7 \end{cases} \quad (17)$$

The basic steps of optimized Hu invariant moments algorithm based on PCA are as follows:
Step 1 to extract the $p+q$ moments of each segmented image's connected domain, according to equation (8) calculate the barycentric coordinate $(x_0, y_0)$;

Step 2 according to equation (9) calculates the center distance, and normalized by equation (10);

Step 3 by equation (11) to calculate the each eigenvalue of invariant moments corresponding with connected domain, and extract the shape features, then obtained invariant moments' information of all samples;

Step 4 aim at the data sample matrix, to calculate its the correlation coefficient matrix $R$ by equation (13);

Step 5 to calculate the eigenvalue of matrix $R$, and sorted them by their value;

Step 6 according to equation (15) determine the front $m$ principal components;

Step 7 by equation (16) to calculate the load of principal components, and obtained the principal features $Z$.

# 4 K-means Cluster Analysis

K-means cluster algorithm is one of classic cluster algorithm, it has the advantages of description easy, algorithm simple, operating efficient high, and suitable for large-scale data processing. In this study, using the K-means cluster algorithm can be accurately obtain the lotus clustering center, to achieve the purpose of fast recognition for lotus.

In the clustering process, the selection of parameters is directly influence the reliability and recognition efficiency of the experiment. The number of clusters is $K$, the $K$ initial clustering centers and the distance parameter $d$ plays a decisive role for K-Means cluster algorithm. The selection of $K$ has a certain human factors, and the selection of $K$ initial clustering centers has a certain randomness, all of these have a directly impact on the result of clustering.

In this study, the city distance is adopted,

$$d = |x - x'| + |y - y'| + |z - z'| + \dots \quad (18)$$

where, $(x, y, z, \Lambda)$ and $(x', y', z'\Lambda)$ respectively represent the cluster center and the coordinate point of sample.
Suppose the number of samples is *n*, using the color and shape features to cluster by K-means algorithm, the basic steps are as follow.

Step 1 the color and shape features of *n* samples as inputs;

Step 2 to determine *K* initial clustering centers $c = \{c_1, c_2, \dots, c_k\}$ randomly, the samples is divided into *K* classes;

Step 3 to calculate the city distance between eigenvalue of each sample to the center of clustering, according the principle of the shortest distance, each sample is divided into the sub-classes of corresponding cluster center;

Step 4 to calculate the average value of each sub-class, and regard as the new cluster center;

Step 5 repeat the step 3 to 4, until the cluster center no longer changes, the algorithm stop, and output the cluster graph and cluster center *C*.

# 5 Image Recognition

The lotus image recognition algorithm based on machine vision is roughly divided into four steps, that is, image segmentation, feature extraction, recognition model training, and sample recognition. The flow chart of feature extraction is shown in figure 4, the flow chart of model training is shown in figure 5, the flow chart of sample recognition is shown in figure 6.

<figure><img src="image_6.png" /><figcaption>Fig. 4 the flow chart of feature extraction</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 5 the flow chart of K-means cluster training</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>Fig. 6 the flow chart of testing sample recognition</figcaption></figure>

The basic steps of lotus image recognition algorithm are as follow.

Step 1 the captured image is segmented by PCNN algorithm, get the segmented images;

Step 2 to extract the color feature information of segmented images by color difference method;

Step 3 to extract the Hu invariant moments of segmented images, using PCA algorithm to reduce dimension, get the principle component data of sample;

Step 4 the color feature and principle components as the inputs, for the training samples, by K-means clustering algorithm, to determine the four cluster centers, that is, lotus, lotus leaf, lotus flower and lotus stem, and then get the trained recognition model;

Step 5 to calculate the distance between test samples to each cluster center;

Step 6 to judge the distance to lotus cluster center whether is closest, if yes, the sample is the lotus target.

# 6 Experiments

## 6.1 Experiment Design

This experiment's operation platform, CPU: Intel(R) Core(TM) 2 Duo E7300 @ 2.66 GHz, RAM: 1.99GB, operating environment: windows XP and matlab R2012a, graphic card: Intel® G33/G31 ECF.

In the experiment, the captured 120 images, all of these images are captured under natural light. In the image process, the pictures are converted to 320×240 pixels to improve the speed of image processing. After the image segmentation, the segmented images are divided into training samples and testing samples.

This experiment firstly uses combination method of PCNN and morphological processing to segment the captured images, obtained segmented images; respectively uses color difference method to extract the color feature of segmented images, uses Hu invariant moments method based on PCA to extract the shape feature of segmented images; and then uses K-means algorithm to cluster the training samples,
obtained *K* cluster center; finally, classifies the test sample according to the
distance to cluster center in order to verify the reliability, recognition precision and
operating efficiency of the lotus recognition model.

## 6.2 Lotus Image Segmentation

All images are segmented by PCNN method described in section 2, and then do some
morphological processing, obtained the segmented images. Segmentation images are
pictured above figure 3. Then we select 3×3 circular structure element to etch figure 3
and use 4×4 structural elements to expand corrosion image, and then make ‘&’
operation between expansion image and original image. The final segmentation image
as shown in figure 7.

<figure><img src="image_9.png" /><figcaption>Fig.7 Final segmentation image</figcaption></figure>

From figure 7, most lotus stem has been removed, only leaving a small part connected with lotus and lotus flower for the positioning of the fruit and picking, and the outline of segmentation target is clear, the connected area is obvious. In other words, it is easy to extract the target's features. So the result shows that PCNN used to segment lotus image is feasible and reliable.

The captured 120 images, after segmentation, obtained 200 segmented images, among 73 lotus images, 39 lotus flower images, 44 lotus leaf images and 46 lotus stem images. Respectively, selected 20 images from each type segmented images as training sample, the rest of the images as the test sample.

## 6.3 Lotus Image Feature Extraction

### 6.3.1 Hu Invariant Moments
For all segmented images, using the invariant moment algorithm to extract the shape feature, and get the invariant moment data of all samples, each sample has seven feature variables $a_1, a_2, \Lambda, a_7$. Invariant moments feature of 8 examples are listed in table 1.

<table><caption>**Table 1** invariant moment data</caption><thead><tr><th>sample</th><th>a<sub>1</sub></th><th>a<sub>2</sub></th><th>a<sub>3</sub></th><th>a<sub>4</sub></th><th>a<sub>5</sub></th><th>a<sub>6</sub></th><th>a<sub>7</sub></th></tr></thead><tbody><tr><td>a</td><td>1.67E-01</td><td>6.15E-04</td><td>5.78E-04</td><td>5.81E-06</td><td>-3.10E-10</td><td>-1.30E-07</td><td>1.31E-10</td></tr><tr><td>b</td><td>1.70E-01</td><td>5.98E-05</td><td>1.12E-03</td><td>6.29E-07</td><td>-1.62E-12</td><td>-5.62E-10</td><td>1.66E-11</td></tr><tr><td>c</td><td>1.03E-01</td><td>2.84E-03</td><td>1.22E-05</td><td>6.61E-07</td><td>-1.83E-12</td><td>-3.22E-08</td><td>-4.06E-13</td></tr><tr><td>d</td><td>1.99E-01</td><td>6.42E-03</td><td>1.44E-04</td><td>9.72E-06</td><td>-3.08E-10</td><td>-5.46E-07</td><td>-1.93E-10</td></tr><tr><td>e</td><td>2.33E-01</td><td>2.68E-02</td><td>1.83E-04</td><td>4.98E-05</td><td>4.57E-09</td><td>6.87E-06</td><td>1.31E-09</td></tr><tr><td>f</td><td>2.15E-01</td><td>1.81E-02</td><td>2.89E-04</td><td>9.39E-05</td><td>1.55E-08</td><td>1.26E-05</td><td>-4.82E-10</td></tr><tr><td>g</td><td>3.30E+00</td><td>1.07E+01</td><td>5.46E-01</td><td>8.09E-02</td><td>-1.82E-03</td><td>-1.21E-01</td><td>-1.69E-02</td></tr><tr><td>h</td><td>1.45E+00</td><td>2.06E+00</td><td>2.87E-02</td><td>3.20E-03</td><td>-3.03E-05</td><td>-4.57E-03</td><td>4.36E-06</td></tr></tbody></table>

From table 1, the results show that the value of invariant moments has a descending trend, except lotus stem, this is caused by the particularity of the shape of lotus stem. The scope of invariant moment of lotus is part overlap with lotus flower and lotus leaf, this will bring some interference for lotus recognition. Thus it can be seen, single from the shape feature to recognize lotus, lotus flower, lotus leaf, is very difficult, so in this study, need to combine with the color feature.

### 6.3.2 PCA Feature Reduction

Invariant moment information is not directly used to recognize the lotus, lotus flower, lotus leaf and lotus stem. The data information is too redundant, is not convenient to cluster analysis. So in this study, in order to using these invariant moment information, need to dimension reduction. PCA algorithm is introduced, and the invariant moment information is integrated into a few principal components.

We do principle component analysis for 200 segmented sample, and the eigenvalue, contribution rate (CR) and accumulative contribution rate (ACR) are listed in table 2.

<table><caption>**Table 2** eigenvalue, contribution rates and accumulative contribution rates</caption><thead><tr><th>principal component</th><th>eigenvalue</th><th>CR, %</th><th>ACR, %</th></tr></thead><tbody><tr><td>z<sub>1</sub></td><td>4.99E+00</td><td>71.286</td><td>71.286</td></tr><tr><td>z<sub>2</sub></td><td>1.15E+00</td><td>16.429</td><td>87.714</td></tr></tbody></table>
<table><tr><td>z<sub>3</sub></td><td>8.36E-01</td><td>11.943</td><td>99.657</td></tr><tr><td>z<sub>4</sub></td><td>2.06E-02</td><td>0.294</td><td>99.951</td></tr><tr><td>z<sub>5</sub></td><td>2.81E-03</td><td>0.040</td><td>99.992</td></tr><tr><td>z<sub>6</sub></td><td>5.09E-04</td><td>0.007</td><td>99.999</td></tr><tr><td>z<sub>7</sub></td><td>8.09E-05</td><td>0.001</td><td>100.000</td></tr></table>

Table 2 shows that, if the amount of information can reach 85%, the solution will not be influenced, in this study need to extract 3 principal components. That is, the dimension of the shape features is reduced from 7 to 3. These 3 principal components represent the shape feature of lotus, lotus flower, lotus leaf and lotus stem.

### 6.3.3 Combination of Color and Shape Feature

Combination of one color feature and three shape features, using these four variables to describe the characteristics of the sample. Take table 1 as an example, the samples corresponding feature are listed in table 3.

<table><caption>Table 3 feature variables of example samples</caption><thead><tr><th>sample</th><th>z<sub>1</sub></th><th>z<sub>2</sub></th><th>z<sub>3</sub></th><th>r-g</th></tr></thead><tbody><tr><td>a</td><td>2.98E-01</td><td>-5.36E-02</td><td>-6.15E-02</td><td>-1.04E+01</td></tr><tr><td>b</td><td>3.04E-01</td><td>-5.41E-02</td><td>-6.26E-02</td><td>-1.62E+01</td></tr><tr><td>c</td><td>1.83E-01</td><td>-3.43E-02</td><td>-3.86E-02</td><td>6.70E+01</td></tr><tr><td>d</td><td>3.53E-01</td><td>-6.67E-02</td><td>-7.50E-02</td><td>2.84E+01</td></tr><tr><td>e</td><td>4.07E-01</td><td>-8.68E-02</td><td>-9.42E-02</td><td>-5.24E+01</td></tr><tr><td>f</td><td>3.78E-01</td><td>-7.70E-02</td><td>-8.48E-02</td><td>-4.31E+01</td></tr><tr><td>g</td><td>2.53E+00</td><td>-5.63E+00</td><td>-4.92E+00</td><td>-1.84E+01</td></tr><tr><td>h</td><td>1.90E+00</td><td>-1.38E+00</td><td>-1.22E+00</td><td>-1.93E+01</td></tr></tbody></table>

Table 3 shows that, these four variables regard as the inputs of the training and recognition model.

## 6.4 Training

In the process of lotus recognition, the interference factors mainly include lotus flower, lotus leaf, lotus stem, so in the training, set $K = 4$. The determination of cluster center is randomly, selected the city distance as the criteria of similarity and discriminated. From 200 samples, respectively, choose 20 lotus, 20 lotus flowers, 20 lotus leaves and 20 lotus stems, so total 80 samples as training sample. The 80
training samples can be divided into four classes by K-means cluster algorithm, four cluster centers {$c_1, c_2, c_3, c_4$} are obtained. In order to the clustering figure show in 2 dimension coordinate, for 4 extracted features ($z_1, z_2, z_3, (r-g)$) to do linear combination, get a new feature $(z', (r-g))$. The transfer formula is

$$ \begin{cases} z' = \lg |z_1.*z_2.*z_3| \\ (r-g)' = 0.01(r-g) \end{cases} \qquad (20) $$

The clustering figure is show as figure 6.

<figure><img src="image_10.png" /><figcaption>Fig.8 Clustering chart of K-Means</figcaption></figure>

Note: Abscissa represents the linear combination of principal components ($\lg |z_1.*z_2.*z_3|$), ordinate represents color feature value after the linear operation ($0.01(r-g)$); peach 'o' area represents lotus, red '*' area represents lotus flower, green '+' area represents lotus leaf, blue '△' area represents lotus stem; '□' represents the cluster center of each cluster area.

The cluster centers' coordinate is

$$ C = \begin{Bmatrix} c_1 & c_2 & c_3 & c_4 \\ -4.1389 & -3.4894 & -3.3214 & 0.0637 \\ -0.1536 & +0.6433 & -0.5244 & -0.1928 \end{Bmatrix} \qquad (21) $$

Figure 8 shows that, the 80 samples are clustered into four classes, the cluster area is obvious. It is illustrate that K-means cluster algorithm is suitable for distinguish lotus, lotus flower, lotus leaf, lotus stem. Lotus flower area locate at above coordinate
axis (x,0), it well to distinguish the lotus flower and the lotus, lotus leaf, and to avoid the lotus recognition interference caused by lotus flower. Lotus flower area locate at right coordinate axis (−1, y), the cluster area is fragmented, it is verify that the difference of lotus stem shape is bigger, the recognition interference is small.

The cluster area of lotus is relatively concentrated, it has obvious boundary with lotus leaf, noninterference with each other. It is further illustrating that K-Mean clustering algorithm for lotus recognition is simple and reliable.

## 6.5 Testing

For 120 test samples, to calculate the city distance of each sample to each cluster center, according the shortest distance to judge the sample belong to which class. The results of recognition are listed in label 4.

<table><caption>**Table 4** Recognition result of testing images</caption><thead><tr><th>indexes</th><th>Lotus</th><th>Lotus flower</th><th>Lotus lesf</th><th>Lotus stem</th></tr></thead><tbody><tr><td>Number of images</td><td>53</td><td>19</td><td>22</td><td>26</td></tr><tr><td>Correct recognition</td><td>48</td><td>16</td><td>20</td><td>26</td></tr><tr><td>Misrecognized to Lotus</td><td>—</td><td>3</td><td>2</td><td>0</td></tr><tr><td>Misrecognition rate, %</td><td>9.43</td><td>0</td><td>0</td><td>0</td></tr><tr><td>Correct recognition rate, %</td><td>90.57</td><td>84.21</td><td>90.91</td><td>100</td></tr><tr><td>Recognition time, s</td><td colspan="4">0.0473</td></tr></tbody></table>

Note: Recognition time is the average of total time, include the color feature extraction, shape feature extraction, and test running time of each image.

From table 4, the results show that, using for testing 53 lotus images, the number of correct recognition image is 48, 2 images are misrecognized to lotus flower, 3 images are misrecognized to lotus leaf, the recognition precision of lotus is 90.57%.

For 19 lotus flowers, the recognition precision is 84.21%, 3 images are misrecognized to lotus, because they are un-blooming lotus flower, their shape is very similar with lotus, and color also is very close.

For 22 lotus leaves, the recognition precision is 90.91%, 2 images are misrecognized to lotus, due to the different shooting angle of the camera, and the growth posture of the lotus is variety.
For 26 lotus stems, the recognition precision is 100%, because the shape of lotus is
difference with other target.

To test 120 images, the average recognition time of each image is 0.0473s,
recognition time include feature extraction and testing time.

7 Conclusion

In order to achieve the automatic harvesting lotus, this paper carries on some related
preliminary study of lotus harvesting robot, and a lotus recognition method for vision
system of harvesting robot is designed.

Lotus target image recognition based on machine vision, the target image
segmentation by PCNN, and obtained the segmented lotus image, lotus flower image,
lotus leaf image, and lotus stem image. Due to the lotus has color difference with
lotus flower, has the shape difference with lotus leaf and lotus stem, so the feature
extraction method of lotus target adopts the combination of color and shape feature.
In the color feature extraction, color difference method is used to extract the red and
green feature difference, to avoid the interference from lotus flower to the lotus and
lotus leaf recognition. In the shape feature extraction, Hu invariant moment algorithm
is adopted, combine with PCA algorithm, to reduce the dimension of shape features.
The extracted principal components represent the shape feature of lotus, lotus flower,
lotus leaf and lotus stem. It reduces the data redundancy, improve the computing
speed.

All samples are divided into training sample and testing sample. For training
sample, by K-means cluster algorithm to determine four cluster center of lotus, lotus
flower, lotus leaf and lotus stem, obtained the lotus recognition model. In the
K-means cluster algorithm, city distance is adopted as the criteria of similarity and
discriminated. In order to easily observe the cluster figure, make its display in 2d
coordinate system, the extracted four features need to be done linear combination.
Finally, the city distance of each testing sample to four cluster center has been
calculated, according the shortest distance to discriminate sample of belonging.

By the experiment analysis, from image segmentation and feature extraction, to
recognition model training and testing sample recognition, this study adopts the
method of lotus image processing, it gets the better effect, and the results are practical
and reliable. From the lotus correct recognition rate, this study achieves the desired
effect. This paper implements the preliminary discussion on the lotus auto-harvesting, next step of study, we will do some improvement on the recognition accuracy and efficiency.

## Acknowledgments

This work is supported by the National Natural Science Foundation of China (No. 31571571); Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD); The Specialized Research Fund for the Doctoral Program of Higher Education of China (No. 20133227110024); Ordinary University Graduate Student Research Innovation Projects of Jiangsu Province (No. KYLX 14_1062).

## References

1. Bac C W, Henten E J, Hemming J, et al. Harvesting robots for high-value crops: state-of-the-art review and challenges ahead[J]. Journal of Field Robotics, 2014, 31(6): 888-911.

2. Li P, Lee S H, Hsu H Y. Review on fruit harvesting method for potential use of automatic fruit harvesting systems[J]. Procedia Engineering, 2011, 23(5): 351-366.

3. Sánchez-González M J, Sánchez-Guerrero M C, Medrano E, et al. Influence of pre-harvest factors on quality of a winter cycle, high commercial value, tomato cultivar [J]. Scientia Horticulturae, 2015, 189: 104-111.

4. Xu L, Zhang T. Present situation of fruit and vegetable harvesting robot and its key problems and measures in application[J]. Transactions of the Chinese Society of Agricultural Engineering, 2004, 20(5): 38-42.

5. Zhao D A, Lv J, Ji W, et al. Design and control of an apple harvesting robot[J]. Biosystems Engineering, 2011, 110(2): 112-122.

6. Xiang R, Ying Y B, Jiang H Y. Development of real-time recognition and localization methods for fruits and vegetables in field[J]. Transactions of the Chinese Society for Agricultural Machinery, 2013, 44(11): 208-223.

7. Qian J P, Yang X T, Wu X M, Chen M X, Wu B G. Mature apple recognition based on hybrid color space in natural scene[J]. Transactions of the Chinese Society of Agricultural Engineering, 2012, 28(17): 137-142.
8. Shen B G, Chen S R, Yin J J, Mao H P. Image recognition of green weeds in cotton fields based on color feature[J]. Transactions of the CSAE, 2009, 25(6): 163-167.

9. Dai J B. Algorithms of image feature extraction based on visual information[D]. Jilin University, 2013.

10. Chaki J, Parekh R, Bhattacharya S. Plant leaf recognition using texture and shape features with neural classifiers[J]. Pattern Recognition Letters, 2015, 58:61-68.

11. Banerji S, Sinha A, Liu C. New image descriptors based on color, texture, shape, and wavelets for object and scene image classification[J]. Neurocomputing, 2013, 117(14): 173-185.

12. Yang M, Kpalma K, Ronsin J. Shape-based invariant feature extraction for object recognition[M]. Advances in Reasoning-Based Image Processing Intelligent Systems. Springer Berlin Heidelberg, 2012, 29: 255-314.

13. Yang M, Kpalma K, Ronsin J. Shape-based invariant feature extraction for object recognition[J]. Intelligent Systems Reference Library, 2012, 29: 255-314.

14. Diao Z, Zheng A, Wu Y. Shape feature extraction of wheat leaf disease based on invariant moment theory[M]. Computer and Computing Technologies in Agriculture V. Springer Berlin Heidelberg, 2012,369: 168-173.

15. Vijayalakshmi B. A new shape feature extraction method for leaf image retrieval[C]. Proceedings of the Fourth International Conference on Signal and Image Processing 2012 (ICSIP 2012). Springer India, 2013: 235-245.

16. Wang M, Wang G H, Fang P Y, Sun S J. Method for vehicle-logo location and recognition based on PCA and invariant moment[J]. Geomatics and Information Science of Wuhan University, 2008, 33(1): 36-40.

17. Stéphane Dray, Josse J. Principal component analysis with missing values: a comparative survey of methods[J]. Plant Ecology, 2014, 216(5): 657-667.

18. Chandrasekhar A M, Raghuveer K. Intrusion detection technique by using k-means, fuzzy neural network and SVM classifiers[C]. 2013 International Conference on Computer Communication and Informatics (ICCCI), IEEE, 2013, 33: 1-7.

19. George A. Efficient high dimension data clustering using constraint-partitioning k-means algorithm[J]. International Arab Journal of Information Technology, 2013, 10(5): 467-476.

20. Li Z, Hong T S, Zeng X Y, Zheng J B. Citrus red mite image target identification based on k-means clustering[J]. Transactions of the Chinese Society of Agricultural Engineering, 2012, 28(23): 147-153.

21. Wang L S, Zhang G Y, Sha Y. Feature extraction and clustering of complex ore image[J]. Journal of Beijing Institute of Petro-chemical Technology, 2010, 18(4): 36-42.
22. Wang Z, Ma Y, Cheng F, et al. Review of pulse-coupled neural networks[J]. Image & Vision Computing, 2010, 28(1): 5-13.

23. Wang H Q, Ji C Y, Gu B X, An Q. In-greenhouse cucumber recognition based on machine vision and least squares support vector machine[J]. Transactions of Chinese Society for Agricultural Machinery, 2012, 43(3): 163-167.

24. Wen C, Yu H, He S. An image segmentation algorithm of corn disease based on the modified bionic pulse coupled neural network[C]. 2013 Fourth Global Congress on Intelligent Systems (GCIS). IEEE, 2013: 97 - 101.

25. Sarkar S, Das S, Chaudhuri S S. A multilevel color image thresholding scheme based on minimum cross entropy and differential evolution[J]. Pattern Recognition Letters, 2015, 54(1): 27-35.

26. Dun S K, Wei H P, Sun M Z. A new distance color difference formula in RGB color space[J]. Science Technology and Engineering, 2011, 11(8): 1833-1836.

27. Zhang C L, Zhang J, Zhang J X, Li W. Recognition of green apple in similar background [J]. Transactions of Chinese Society for Agricultural Machinery, 2014 (10): 277-281.

28. Wang J J, Zhao D A, Ji W, Zahng C. Apple fruit recognition based on support vector machine using in harvesting robot[J]. Transactions of Chinese Society for Agricultural Machinery, 2009, 40(1): 148-151.

29. Jiang L, Chen D F, Pan C, et al. Recognition of difformity mature fruits in natural environment[J]. Science Technology and Engineering, 2012, 20(21): 5135-5138.

30. Zhang H T, Mao H P, Qiu D J. Feature extraction for the stored-grain insect detection system based on image recognition technology[J]. Transactions of the Chinese Society of Agricultural Engineering, 2009, 25(2): 126-130.

31. Deng J Z, Li M, Yuan Z B, Jin J, Huang H S. Feature extraction and classification of Tilletia diseases based on image[J]. Transactions of the Chinese Society of Agricultural Engineering, 2012, 28(3): 172-176.

32. Xiong J T, Zou X J, Liu N, Peng H X, Lin G C. Fruit quality detection based on machine vision technology when picking litchi[J]. Transactions of Chinese Society for Agricultural Machinery, 2014, 45(7): 54-60.

33. Wang X P, He Z J. Accurate recognition of hairs in canned mushroom under different kinds of lighting conditions[J]. Transactions of the Chinese Society of Agricultural Engineering, 2014, 30(4): 264-271.