# The Research of the Strawberry Disease Identification Based on Image Processing and Pattern Recognition

Changqi Ouyang, Daoliang Li, Jianlun Wang, Shuting Wang, Yu Han

► To cite this version:

Changqi Ouyang, Daoliang Li, Jianlun Wang, Shuting Wang, Yu Han. The Research of the Strawberry Disease Identification Based on Image Processing and Pattern Recognition. 6th Computer and Computing Technologies in Agriculture (CCTA), Oct 2012, Zhangjiajie, China. pp.69-77, 10.1007/978-3-642-36124-1_9. hal-01348083

HAL Id: hal-01348083

https://inria.hal.science/hal-01348083

Submitted on 22 Jul 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# The Research of the Strawberry Disease Identification Based on Image Processing and Pattern Recognition

Changqi Ouyang<fnref n="1" />,<fnref n="1" />*, Daoliang Li<fnref n="1" />, Jianlun Wang <fnref n="1" />,<fnref n="2" />, Shuting Wang<fnref n="1" />, Yu Han<fnref n="1" />

¹College of Information and Electrical Engineering, China Agricultural University,
Beijing 100083, P. R. China

**Abstract.** In this paper, a synthesis segmentation algorithm is designed for the real-time online diseased strawberry images in greenhouse. First, preprocess images to eliminate the impact of uneven illumination through the “top-hat” transform, and remove noise interference by median filtering. After comprehensively applying the methods of gray morphology, logical operation, OTSU and mean shift segmentation, we can obtain the complete strawberry fruit area of the image. Normalize the extracted eigenvalues, and use eigenvectors of part of the samples for training the BP neural network and support vector machine, the remaining samples were tested in two kinds of disease strawberry recognition model. Results show that support vector machines have a higher recognition rate than the BP neural network.

**Keywords:** strawberry fruit; gray morphology; OTSU; mean shift segmentation; Pattern recognition

## 1 Introduction

Strawberry fruit image segmentation is used widely in strawberry picking and fruit grading. In strawberry picking, the strawberry fruit is mature normal strawberry, it's easy to segment the fruit image because the color of the fruit area is almost the same. In strawberry fruit grading, the strawberry fruit has already been picked, and it's not difficult for image segmentation because the background is relatively consistent. The strawberry image segmentation algorithm discussed in this paper aims to offer the objects to the real-time online identification of the three common strawberry diseases.

<footnote n="1">¹※ Ouyang Changqi, the first author</footnote>

<footnote n="2">² Corresponding author, Email: ouyangchangqi@126.com</footnote>
Therefore, this algorithm is for the image segmentation of the complete strawberry fruit without picking it. As the color distribution of diseased strawberry fruit is inconsistent, and fruit background is quite complicated, it needs to combine kinds of image processing methods to obtain the complete segmentation of the strawberry fruit according to the characteristics of the diseased fruit area.

Pattern recognition is an intelligent activity, including analysis and judgment. The analysis process is to determine the division of pattern class characteristics and expression method; judgment is reflected in the characteristics of the unknown object, and its judgment belongs to a certain class. It aims to use a computer to achieve the ability to identify the class, which is the contrast of two different levels of identification ability. Pattern recognition system includes the sample acquisition, preprocessing, feature extraction, classification decisions and classifier design.

## 2 Material and equipment

The objects studied in this paper are the strawberries of the three common diseases in greenhouse, as shown in Fig2-1. These three diseases are powdery mildew, shrinkage and uneven ripening. The characteristic of powdery mildew is that the fruit surface is coarse and with a layer of white powder. The characteristic of shrinkage is that the fruit atrophies and is deformed. The characteristic of uneven ripening is that the fruit's color is uneven, but its surface is smooth.

<figure><img src="image_2.png" /><figcaption>Fig 2-1 Three common diseases of strawberry</figcaption></figure>

## 3 Image Preprocessing

As a result of noise, uneven illumination, and the transform of analog and digital signals of the CCD camera, the online images captured in the greenhouse under
natural light are blurred in details, such as its contours. Therefore, before image segmentation, the image preprocessing must be taken to eliminate the impact.

## 3.1 Image denoising

To ensure that the edge of the strawberry fruit is not vague while image denoising, the median filtering is applied to remove image noise, in detail, each channel of the RGB image is removed noise through median filtering respectively, and then the denoised image can be obtained through the integration of the three denoised channels.

## 3.2 Uneven illumination effect elimination

Strawberry images captured under natural light are greatly affected by uneven illumination. If this impact is not eliminated, the object may be segmented into disperse areas. In this paper, the “top-hat” transform is applied to eliminate the impact.

# 4 Image Segmentation

## 4.1 The extraction and transform of the red and green area templates

Figure 4-1 (a) shows the preprocessed strawberry image. The red area template can be got by 2R-G-B, as shown in Figure 4-1 (b), where R, G and B were the three channels of the color image in RGB space. Due to the uneven surface of the strawberry fruit, the fruit area contains many holes, which needs to be filled in order to get the closed template. Then, execute the OTSU operation, the initial fruit template can be obtained as shown in Figure 4-1 (c).

Conduct the logical AND operation between the template complementation and Figure 4-1 (a) to remove the fruit area in Figure 4-1(a) while keeping the image outside of the template unchanged. The result is shown in Figure 4-2 (a). The green area template can be obtained by 2G-R-B, as shown in Figure 4-2 (b). Fig 4-2(a) can be removed the green zone in similar way, and backfilled with the red template, then the result can be got as shown in Fig4-2(c). Now, the green area around the fruit is all black, and the fruit region is almost all white. The purpose of this series of operations
is to separate strawberry fruit region from other objects, and create a favorable condition for the subsequent image segmentation.

<figure><img src="image_3.png" /><figcaption>Fig 4-2 The separation between the fruit region and other objects</figcaption></figure>

## 4.2 Strawberry fruit area color clustering

Because K-means cluster and fuzzy C-means cluster both need to set the cluster number initially, which is unknown for the image containing complicated background, it is infeasible to apply these two methods to segment the fruit image.

The mean shift algorithm is an adaptive probability density gradient ascent method of searching for the peak. It is as follows: 1) Select the window size and initial position; 2) Calculate the gravity center of the window; 3) Move the center of the window to the center of gravity; 4) Repeat Step 2 and Step 3 until the center of the window "convergence", that is, the moving distance of the window must be less than a certain threshold. Each pixel in the image can be set as the initial point, after performed the mean shift algorithm respectively, the initial points converged to the same point are considered as the same class. Therefore, the mean shift algorithm can be applied to image cluster segmentation.

Based on the above discussion, the mean shift segmentation algorithm is used in image segmentation, which make pixels of the strawberry fruit area cluster into the
same class in color space, namely strawberry fruit regional pixels are all of the same value.

## 4.3 Target area extraction

After the operation of converting the image into grayscale and filling holes, execute the “open” operation on the image with circular structural element of a radius of 2 to remove the adhesion between the target and background. The experiments showed that: when the split threshold is 0.8, the grayscale threshold segmentation can obtain good results as shown in Figure 4-3; and then extract the largest connected area of the image to obtain strawberry fruit template. Combine this template and the preprocessed image by logical AND operation, then the image of the complete strawberry fruit area can be obtained.

<figure><img src="image_4.png" /><figcaption>Fig 4-3 Threshold segmentation image</figcaption></figure>
# 5 Strawberry fruit image segmentation algorithm

After the above discussion on the image preprocessing and image segmentation method, here are the strawberry fruit image segmentation algorithm steps, including image reading, image preprocessing, image segmentation (shown in Fig5-1)

<figure><img src="image_5.png" /><figcaption>Fig5-1 The flow chart of the segmentation algorithm</figcaption></figure>
## 6 Segmentation results

Figure 6-1, Figure 6-2, Figure 6-3, and Figure 6-4 are the segmentation results of the above strawberry image segmentation algorithm, which corresponds to normal strawberry, powdery mildew of strawberry, shrinkage of fruit disease strawberries and uneven ripening disease strawberries respectively. Each subgraph corresponds to the result after the corresponding step of the above segmentation algorithm. The algorithm can get effective segmentation results under natural illumination.

<figure><img src="image_6.png" /><figcaption>Fig6-1 Segmentation results of
the normal strawberry</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig6-3 Segmentation results of the strawberry
fruit with the shrink disease</figcaption></figure>

<caption>Fig 6-4 Segmentation results of the strawberry
fruit with the uneven ripening disease</caption>
# 7 Experimental schemes

## 7.1 Data selection

### a) Data access

The color, texture and shape feature were extracted from four kinds of strawberry respectively, including 12 kinds of color characteristic parameters, 8 kinds of texture parameters, nine kinds of shape characteristic parameters, so each sample corresponds to a set of characteristic data of a total of 12 + 8 + 9 = 29 kinds of characteristic parameters. The sample size of each type of disease strawberry and normal strawberry is 20. Set category labels are as follows: normal strawberry labels 1, powdery mildew strawberry labels 2, fruit shrink disease strawberry labels 3, and the uneven ripening disease strawberry labels 4. The two main equations are:

$$ \mu = \frac{1}{N} \sum_{i=1}^{N} x_i $$

$$ \text{standard deviation} \quad \sigma = \sqrt{\frac{1}{N} \sum_{i=1}^{N} (x_i - \mu)^2} $$

### b) Division of the data

Establishing training and testing are two essential processes while using BP neural network and SVM to build the classifier. So data should be divided into two parts, one for training and one for the test. 3/4 of the data are used for training, 1/4 of the data are used for testing. Select 60 sets of data randomly as a training data set, the remaining 20 sets of data as a test data set from the data set.

### c) Data preprocessing

The physical meaning of the color, texture and shape characteristic value is different, not comparable, and their range changes, so there is a big numerical difference among them. Data normalization is necessary, or it will lead to the classifier learning slows down or even does not converge. In this study, each dimensional feature data were normalized preprocessed, used in a normalized mapping shown as follows:

$$ x' = \frac{x - x_{min}}{x_{max} - x_{min}} $$

In which x is the corresponding eigenvalue of a certain characteristic properties, $x_{max}$, $x_{min}$ shows the largest and the smallest eigenvalue of the properties.
While classifier training is completed, the input data for the test needs to be
normalized in order to enter the classifier.

7.2 BP neural network training and prediction

According to Kolmogorov theorem, using a three-layer BP neural network( 29 × 59 × 4)for the classifier, in which the hidden layer activation function is the non-symmetric Sigmoid function: $f(x) = \frac{1}{1+e^{-x}}$, the output layer activation function is a linear function $f(x) =x$, and set a target error e = 0.01. Use the training data to train the BP neural network, to make the BP neural network can predict the nonlinear function output.

Use trained BP neural network to predict the test data, and use the predicted output and actual output of the BP neural network to analyze the classification ability of BP neural network.

7.3 SVM training and prediction

Implementation of SVM in this experiment uses the libsvm toolbox, the kernel function uses the radial basis kernel function. The training data is used to train SVM classifiers to get the SVM classification model.

Use the trained SVM classifier to make the label prediction of test data, and
analyze the Classification ability of SVM with predicted SVM output and actual
output.

8 Conclusion

Image segmentation of the strawberry fruit of the three common diseases consists of
two main parts: the first part is image pre-processing, including the application of
median filtering to remove the noise from image acquisition, and the application of
top-hat transform to eliminate the effects of uneven illumination; the second part is
image segmentation, using gray morphology, OTSU algorithm, the mean shift
segmentation algorithm to segment the fruit from the image. These experimental
results indicate that image processing algorithm can obtain effective segmentation
results to the online images of the three common kinds of diseased strawberries and
nomal ones under natural illumination.

This paper uses digital image processing, pattern recognition technology to extract
the integrated fruit of disease strawberries. Based on parameters of color
characteristic, texture features and shape features, use BP neural network and support
vector machine to make classification and recognition for disease strawberry. The
experimental results show that support vector machine has a higher recognition rate
than the BP neural network so that support vector machine is used as the classifier for
disease strawberry.

9 References

1. Zhou Tianjuan, Zhang Tiezhong, Yang Li, et al. Comparison of two algorithms based on mathematical morphology for segmentaion of touching strawberry fruits [J]. Transactions of the CSAE, 2007, 23 (9) : 164 - 168 (in Chinese with English abstract).

2. Xie Zhiyong, Zhang Tiezhong. A new method of segmentation of strawberry image[J]. Journal of China Agricultural University, 2006, 11(1) : 84-86 (in Chinese with English abstract).

3. Zhang Tiezhong, Zhou Tianjuan. Strawberry harvesting robot[J]. Journal of China Agricultural University, 2004, 9(4) : 65-68.

4. Xie Zhiyong, Zhang Tiezhong, Zhao Jinying. Ripened Strawberry Recognition Based on Hough Transform[J]. Transactions of the Chinese Sociey for Agricultural Machinery, 2007, 38(3) : 106-109.

5. Yu Shiqi, Liu Ruizhen. OpenCV course[M]. Beijing: Beijing University of Aeronautics and Astronautics Press, 2009.

6. Zhang Yujin. Image Processing[M]. Beijing: Tsinghua University Press, 2008.

7. Zhang Yujin. Image Analysis[M]. Beijing: Tsinghua University Press, 2008.

8. Liu Haibo, Shen Jing, Guo Song, et al. Digital Image Processing Using Visual C++[M].
Beijing: China Machine Press, 2010.