# Measurement Method of Plant Phenotypic Parameters Based on Image Deep Learning

Authors: Mo Dong; Haiye Yu; Lei Zhang; Mingzhi Wu; Zhipeng Sun; Dequan Zeng; Ruohan Zhao
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

This article applies deep learning and electromechanical technology to plant phenotype measurement. First, an electromechanical device is designed to collect plant phenotype images, which solves the difficulty of collecting deep learning training data. The data set required for deep learning model training for plant phenotype detection is made by an automated method. This paper takes the Lactuca sativa plant image as an example and uses the ASM-based data enhancement method to solve the problem of insufficient image data of Lactuca sativa leaf pests and effectively avoid the phenomenon of overfitting. The plant image recognition method based on deep learning proposed breaks through the limitations of plant local feature recognition, gets rid of the limitation of highly specialized data collection, lowers the threshold of plant image recognition, and has advantages in recognition speed and accuracy. This method requires a large amount of training data. In the future, we can explore the collection of massive plant pictures from the Internet as a training set to achieve rapid iteration and optimization of the model.


---

## Body

## 1. Introduction

Plants are the most abundant and widely distributed life forms on earth. It is the most important part of ecological resources, and plants play a very important role in human life. Leaves are the organs for plants to produce nutrients and are the basis for their growth and fruit. Plant leaves used to classify and identify plants are the most effective and economical method. The prerequisite for the effective identification of plant leaves is to separate the plant leaves from the background, which is a key link in the image processing of plant leaves. Only through the reasonable segmentation of plant leaf images can the analysis of plants become possible. Many traditional image segmentation techniques, such as histogram threshold method, edge detection method, region-based method, and random field-based method, all use the sharp changes in the gray value of the edge pixels of the plant leaf image, and their essence is still only the image gray-scale feature information. This type of method can achieve better segmentation results in single blade and simple scenes [1]. The system is more robust, which is unable to give effective semantic information. Phenotype, phenome, and phenomics are three important concepts in the field of life sciences. Phenotype refers to the observable morphological characteristics of biological individuals or groups under the interaction of genes and environment. Phenotypic group refers to all the traits of a certain organism. Phenotypics is the study of all the phenotypes of a certain organism or cell at the genome level. With the development of genes, gene expression products, and various genetic association analysis studies, phenotypics has gradually attracted people’s attention and has developed into an important branch of biology. Especially in recent years, it has been booming. These characteristics and traits should be possible to distinguish the determinants or influences of different plant genotypes and environments. In general, plant phenotypes not only reflect expression regulation at the molecular level but also reflect the physiological, biochemical, and biological characteristics of plants [2]. With the development of modern high-performance computing, in the field of artificial intelligence, deep learning technology is increasingly applied to scientific research and industrial production. Among them, convolutional neural network (CNN) is the most applied deep learning network. Aiming at the unsatisfactory effect of blade segmentation in complex backgrounds, which are greatly affected by background noise, environmental scenes, and poor system robustness, a deep learning blade image segmentation method using active shape model (ASM) algorithm is proposed. The supervised learning method trains ASM for plant leaf detection.Traditional plant phenotype research obtains the relationship between genes and phenotypes. Common challenges include low efficiency, analyzing multiple plant traits and measurement errors.. This method often relies on manual detection of individual traits from small samples of plants; so, the amount of data that can be obtained is very limited and very low efficiency. It is difficult to analyze a variety of plant traits comprehensively, and it is easy to introduce measurement data errors caused by human factors. With the rapid and in-depth development of plant functional genomics and crop molecular breeding, researchers urgently need high precision, high-throughput, and low-cost technologies to obtain phenotypic data related to plant growth, yield, quality, and tolerance to biological and abiotic stresses, which require biological science, computer science, mathematics, and engineering. In 2020, Liu et al. extracted the leaf shape, area, perimeter, length and width, and other characteristics of 40 types of plants [3] and then converted these low-level features into high-level features for qualitative descriptions, thereby classifying plant images. Li et al. proposed to combine the color, shape, leaf edge gears, and other characteristics of plant pictures [4] and input them into the artificial neural network (ANN) to classify plant images. This method is applied to 33 different plants. In the image library, the average recognition rate is 94.3%. Wang et al. proposed to combine the vein and shape features of leaves in plant images [5] and use fast Fourier transform to obtain frequency domain data as features. The average recognition accuracy in 32 plant image databases is 96.9%. Ma et al. used leaf-related features for multifeature fusion comparison based on the leaf images of 7 species of Salix [6], and the comprehensive discrimination accuracy reached 90.8%, achieving high-accuracy recognition of specific species leaf image analysis. Yang et al. integrated the multifeatures of foliage plants [7] and extracted the color, shape, and texture of foliage images of foliage plants by analyzing the morphological features of foliage plants. A total of 26 features were extracted based on support vector machine (SVM) algorithm that completes the classification and recognition of foliage plant leaves, and the recognition rate reaches 91.41%.Plants are indispensable organisms on the earth, and the rapid identification and classification of plant images have high academic value and scientific and technological application value. To sum up, in the past plant image classification algorithms, they tend to preextract a variety of leaf features from a single plant leaf image as the basis for classification and recognition. The above methods are used to identify and classify plants, from low-level image features to high-level concepts. It is difficult to resolve the semantics that exist between them, the application cost is high, the plant species is single, and the recognition accuracy is low. The contributions of this paper are as follows. This paper proposes a plant image processing method based on the active shape model (ASM). Through the analysis of related work, it is found that in the existing research in the field of leaf segmentation, the segmentation effect mostly depends on the quality of the data set and the complexity of environmental factors such as background and lighting, and the processing methods are mostly based on supervision. Learning or traditional manual segmentation methods are not robust and cannot achieve high-throughput blade segmentation. In response to these shortcomings, this paper proposes a deep learning method based on the all active shape model (ASM) algorithm for leaf segmentation of plant images. The significance of various attributes for the clear execution of the work is that the model trained with a small sample size can achieve high-throughput segmentation of plant leaf images, and the model has good robustness.


## 2. Key Technology and Theoretical Knowledge

### 2.1. Plant Growth Model

The energy required for the growth of plant tissues is provided by photosynthesis that promotes the combination of obtaining carbon from the atmosphere and obtaining nitrogen from the soil. The model we built mainly considers these two elements and does not consider other chemical substances. In each cell of the plant, the ratio of carbon and nitrogen is fixed. The carbon and nitrogen in newly produced cells also maintain the same ratio. Carbon and nitrogen move between other parts of the plant. Usually, plants are divided into roots, stems, and leaves, but we simplify them into two parts, the roots that grow underground and the leaves that grow above the ground. Now, we are divided into three stages, gradually establishing and perfecting the model from the shallower to the deeper, and each stage establishes an independent model [8]. If the roots and leaves of plants are not distinguished, carbon and nitrogen are not distinguished, and the growth process is generally regarded as the growth of plants absorbing nutrients; a simple mathematical model can be obtained.It is assumed that the mass of the plant isW, the volume is V, and the nutrients absorbed by the plant are proportional to the volume, namely,
(1)vs=xs,ys.The solution is as follows.(2)vs=xs,ys.W0 is the initial plant mass.(3)Wt=Wm1−1−WmW0e−kt.Obviously,Wt is a monotonically increasing function of t, and when t⟶∞, Wt⟶∞, that is, the actual meaning of it is the maximum mass of the plant.It is supposed that there is a set of curves to be selected for the fitting target, and the energy function is defined to be associated with each curve in the set to be selected. The design principle of the energy function is as follows: the favorable attributes should lead to the reduction of energy [9]. Favorable attributes include the following: the curve is continuous and smooth, and the curve is close to the high gradient area and some other specific prior knowledge. When the active contour moves within the range of values, it can converge to the local boundary under the guidance of the energy function and can maintain the continuity and smoothness of the curve.
(4)vs=xs,ys.S is the normalized curve length, and the range of variation is (0, 1).The total energy function of the snake model is as follows.(5)Esnake=∫Eintvs+Eextvsds.They are the first and second derivatives ofv to s. The coefficient α and the coefficient β represent the elasticity and rigidity of the snake model. These parameters control the physical behavior and local continuity of the model. The external energy Eext decides to move the snake model to a certain fixed feature, attracting the snake model to a prominent image feature. Because these characteristics can only be defined according to specific problems, the general external energy function is not easy to determine. Therefore, Eext does not have a unified mathematical expression and must proceed from the characteristics of the problem itself and deal with it according to the actual situation. It is defined by the function Eimage as x,y, which reflects the degree of interest in image features (such as boundaries).
(6)Eext=∫sEimagevsds.The definition of theEimagex,y function is a key issue.


### 2.2. Deep Learning

In recent years, deep learning has become a popular technology in various fields such as image processing, speech recognition, and machine translation. The name of its depth is relative to traditional machine learning methods (such as support vector machines). Traditional machine learning methods are mostly neural networks that do not contain or only have one or two hidden layers, while deep learning networks contain more. There are many hidden layers, and many of them can even reach hundreds of layers [10]. The essence of deep learning is a spontaneous feature learning method, which is trained to learn parameters. After the optimization process, the deep learning network performs multilayer nonlinear transformation operations on the input data, continuously coupling low-level features, layer by layer upwards, and finally high-level semantic expression [11]. In traditional machine learning methods, pipelines and multisteps are common operations, such as manual design and extraction of features, and are often required, and then machine-learning models are used to obtain the results. Alternatively, preprocess the input data, filter out unnecessary redundant interference data, and postprocess the output of the machine-learning model to obtain the target output. The emergence of deep learning can simplify the process and pass the input data through deeper nerves. The concept and approach of the deep learning procedures are as follows. The network model directly obtains the result, and this kind of “end-to-end” system is easier to realize [12]. Since the processing of plant phenotypes is often linked to work such as image processing and image recognition, the convolutional neural network CNN inspired by the visual structure of the human brain is very useful in the field of plant phenotype processing. In fact, in the field of plant phenotype processing, the most commonly used is CNN (Figure 1). The essence of deep learning procedures is to learn more useful features by building machine learning models with many hidden layers and massive training data [13]. This ultimately improves the accuracy of classification or prediction. Therefore, “deep models” are the means, and “feature learning” is the end.Figure 1
The architecture of CNN.A typical CNN is shown in Figure1. It usually consists of a convolutional layer, a pooling layer, a fully connected layer, etc. In the convolutional layer, neurons are arranged in a matrix to form a multichannel feature map. Each channel’s neuronA is connected to part of the feature map of the upper layer. After convolution operation is performed through a convolution kernel, the activation function is used to obtain the final input of the neuron. This design conforms to the principle of “receptive field” and uses image. The local correlation in the information reduces the number of parameters, which greatly reduces the space and time costs required for training. In addition, weight sharing is an important feature of CNN.
(7)neti=b+∑j=1nwijxj,yi=fneti.x1~xn is the input signal from other neurons, wij represents the connection weight from neuron j to neuron i, and b represents a threshold or bias.Different from traditional shallow learning, deep learning is different in the following tips: (1) DP emphasizes the depth of the model structure, usually with 5, 6, or even 10 layers of hidden layer nodes. (2) DP clearly highlights the importance of feature learning; that is, through layer-by-layer feature transformation, the feature representation of the sample in the original space is transformed into a new feature space, thereby making classification or prediction easier. Compared with the method of constructing features by artificial rules, using big data to learn features can better describe the rich intrinsic information of the data.Neurals located on the same channel feature map in the same layer of convolutional layer. Elements are all obtained by processing the feature map of the previous layer by the same convolution. The convolution kernel corresponding to the feature map is different. This bionic design simplifies the model again, which not only reduces the number of parameters but also reduces the possibility of overfitting. The role of the convolutional layer is to detect the upper layer features. The contact information is presented by the local features of the graph, the pooling layer is the contact information presented by the local features of the phase, and the pooling layer is the fusion of information with the same meaning [14]. The information may be slightly jittery, and the pooling operation can extract the main information from the upper layer feature map. The more common pooling operation is maximum pooling or average pooling. Through pooling sampling, the resolution of the feature map can be reduced. After the alternation of convolution and pooling, it is often that the fully connected layer and each neuron in the fully connected layer are connected with each other. All neurons in the upper layer are connected, all the information is integrated, and the multidimensional features are transformed into one-dimensional features, which are handed over to the final regressor, classifier, etc. to obtain the result.


## 2.1. Plant Growth Model

The energy required for the growth of plant tissues is provided by photosynthesis that promotes the combination of obtaining carbon from the atmosphere and obtaining nitrogen from the soil. The model we built mainly considers these two elements and does not consider other chemical substances. In each cell of the plant, the ratio of carbon and nitrogen is fixed. The carbon and nitrogen in newly produced cells also maintain the same ratio. Carbon and nitrogen move between other parts of the plant. Usually, plants are divided into roots, stems, and leaves, but we simplify them into two parts, the roots that grow underground and the leaves that grow above the ground. Now, we are divided into three stages, gradually establishing and perfecting the model from the shallower to the deeper, and each stage establishes an independent model [8]. If the roots and leaves of plants are not distinguished, carbon and nitrogen are not distinguished, and the growth process is generally regarded as the growth of plants absorbing nutrients; a simple mathematical model can be obtained.It is assumed that the mass of the plant isW, the volume is V, and the nutrients absorbed by the plant are proportional to the volume, namely,
(1)vs=xs,ys.The solution is as follows.(2)vs=xs,ys.W0 is the initial plant mass.(3)Wt=Wm1−1−WmW0e−kt.Obviously,Wt is a monotonically increasing function of t, and when t⟶∞, Wt⟶∞, that is, the actual meaning of it is the maximum mass of the plant.It is supposed that there is a set of curves to be selected for the fitting target, and the energy function is defined to be associated with each curve in the set to be selected. The design principle of the energy function is as follows: the favorable attributes should lead to the reduction of energy [9]. Favorable attributes include the following: the curve is continuous and smooth, and the curve is close to the high gradient area and some other specific prior knowledge. When the active contour moves within the range of values, it can converge to the local boundary under the guidance of the energy function and can maintain the continuity and smoothness of the curve.
(4)vs=xs,ys.S is the normalized curve length, and the range of variation is (0, 1).The total energy function of the snake model is as follows.(5)Esnake=∫Eintvs+Eextvsds.They are the first and second derivatives ofv to s. The coefficient α and the coefficient β represent the elasticity and rigidity of the snake model. These parameters control the physical behavior and local continuity of the model. The external energy Eext decides to move the snake model to a certain fixed feature, attracting the snake model to a prominent image feature. Because these characteristics can only be defined according to specific problems, the general external energy function is not easy to determine. Therefore, Eext does not have a unified mathematical expression and must proceed from the characteristics of the problem itself and deal with it according to the actual situation. It is defined by the function Eimage as x,y, which reflects the degree of interest in image features (such as boundaries).
(6)Eext=∫sEimagevsds.The definition of theEimagex,y function is a key issue.


## 2.2. Deep Learning

In recent years, deep learning has become a popular technology in various fields such as image processing, speech recognition, and machine translation. The name of its depth is relative to traditional machine learning methods (such as support vector machines). Traditional machine learning methods are mostly neural networks that do not contain or only have one or two hidden layers, while deep learning networks contain more. There are many hidden layers, and many of them can even reach hundreds of layers [10]. The essence of deep learning is a spontaneous feature learning method, which is trained to learn parameters. After the optimization process, the deep learning network performs multilayer nonlinear transformation operations on the input data, continuously coupling low-level features, layer by layer upwards, and finally high-level semantic expression [11]. In traditional machine learning methods, pipelines and multisteps are common operations, such as manual design and extraction of features, and are often required, and then machine-learning models are used to obtain the results. Alternatively, preprocess the input data, filter out unnecessary redundant interference data, and postprocess the output of the machine-learning model to obtain the target output. The emergence of deep learning can simplify the process and pass the input data through deeper nerves. The concept and approach of the deep learning procedures are as follows. The network model directly obtains the result, and this kind of “end-to-end” system is easier to realize [12]. Since the processing of plant phenotypes is often linked to work such as image processing and image recognition, the convolutional neural network CNN inspired by the visual structure of the human brain is very useful in the field of plant phenotype processing. In fact, in the field of plant phenotype processing, the most commonly used is CNN (Figure 1). The essence of deep learning procedures is to learn more useful features by building machine learning models with many hidden layers and massive training data [13]. This ultimately improves the accuracy of classification or prediction. Therefore, “deep models” are the means, and “feature learning” is the end.Figure 1
The architecture of CNN.A typical CNN is shown in Figure1. It usually consists of a convolutional layer, a pooling layer, a fully connected layer, etc. In the convolutional layer, neurons are arranged in a matrix to form a multichannel feature map. Each channel’s neuronA is connected to part of the feature map of the upper layer. After convolution operation is performed through a convolution kernel, the activation function is used to obtain the final input of the neuron. This design conforms to the principle of “receptive field” and uses image. The local correlation in the information reduces the number of parameters, which greatly reduces the space and time costs required for training. In addition, weight sharing is an important feature of CNN.
(7)neti=b+∑j=1nwijxj,yi=fneti.x1~xn is the input signal from other neurons, wij represents the connection weight from neuron j to neuron i, and b represents a threshold or bias.Different from traditional shallow learning, deep learning is different in the following tips: (1) DP emphasizes the depth of the model structure, usually with 5, 6, or even 10 layers of hidden layer nodes. (2) DP clearly highlights the importance of feature learning; that is, through layer-by-layer feature transformation, the feature representation of the sample in the original space is transformed into a new feature space, thereby making classification or prediction easier. Compared with the method of constructing features by artificial rules, using big data to learn features can better describe the rich intrinsic information of the data.Neurals located on the same channel feature map in the same layer of convolutional layer. Elements are all obtained by processing the feature map of the previous layer by the same convolution. The convolution kernel corresponding to the feature map is different. This bionic design simplifies the model again, which not only reduces the number of parameters but also reduces the possibility of overfitting. The role of the convolutional layer is to detect the upper layer features. The contact information is presented by the local features of the graph, the pooling layer is the contact information presented by the local features of the phase, and the pooling layer is the fusion of information with the same meaning [14]. The information may be slightly jittery, and the pooling operation can extract the main information from the upper layer feature map. The more common pooling operation is maximum pooling or average pooling. Through pooling sampling, the resolution of the feature map can be reduced. After the alternation of convolution and pooling, it is often that the fully connected layer and each neuron in the fully connected layer are connected with each other. All neurons in the upper layer are connected, all the information is integrated, and the multidimensional features are transformed into one-dimensional features, which are handed over to the final regressor, classifier, etc. to obtain the result.


## 3. Research Methods

### 3.1. Collecting Device and Measuring Principle

Place the collected plant leaves on a white carrier and flatten them with a transparent object. At this time, the transparent object should have selected to avoid greater light reflection and then use a digital camera or CCD camera with a certain resolution to shoot the leaves vertically. That is, the plant leaf information can be converted into computer-recognizable image information, as shown in Figure1. The image input to the computer is represented by a matrix, that is, a group of pixels arranged in a mesh. The number of pixels per unit length is the resolution of the image, usually expressed in pixels per inch (DPI) [15]. Therefore, as long as you know the size of each pixel and know the number of pixels that make up each blade, you can calculate the area of the blade. At the same time, as long as the number of pixels composing the contour of the leaf is known, the perimeter of the leaf can be as follows.


### 3.2. Dataset Source

The plant images used in this article come from the CVPPP data set, which consists of four folders A1, A2, A3, and A4. This is a data set published by the CVPR conference for the challenge of leaf segmentation. The images were taken with 7 mP Canon AD1000 Power-Shot camera. The plant images are composed of plant top images placed in flowerpots and trays. The images of the A1 and A2 datasets are Lactuca sativa plants. The A1 dataset is acquired every 6 hours during the day and lasts for 3 weeks. The light intensity and background environment of the images are more complicated. The A2 dataset is captured every 20 minutes during the day and lasts for 7 hours. The background environment is relatively simple, but the plant changes are quite different. The A3 data set is composed of Lactuca sativa images [16]. The A4 data set is acquired every other day after emergence, until the growth and development of the plants are fully mature, and there are 20 pseudonans in the data set: top view image of the complete growth period of mustard. In summary, the data set contains images of different types of plants, images of different growth stages of the same plant, and images of plant groups and individual plants. The images in the data set are different in resolution, scene complexity, and light intensity. They have complexity and diversity of plant objects. Therefore, the data set has certain challenges in image segmentation, and the research results are very convincing. Some images in the data set are shown in Figure 2.Figure 2
Dataset source of Lactuca sativa.


### 3.3. Algorithm Implementation

#### 3.3.1. Blade Segmentation Network Model

Convolutional neural network (CNN) is a multilayer network architecture with autonomous learning, which is the most widely used in image classification. However, CNN networks tend to lose the detailed information of the image during the feature abstraction process, and the object outline is unclear. Therefore, CNN network has limitations in the field of image segmentation, which has fewer applications. In response to the problems of CNN in image detail processing, Jonathan Long of the University of California at Berkeley and others proposed an improved network architecture, the end-to-end fully convolutional networks (FCN) [17].The biggest difference between FCN and CNN is that it converts all fully connected layers in CNN into convolutional layers. The image enters the FCN network as input data. The first 5 convolutional layers are the same as the CNN network, but the fully connected layers of the last three layers are replaced with convolutional layers. Unlike the one-dimensional output after the CNN fully connected layer, the output after the FCN convolution layer is still a two-dimensional matrix. The main purpose of this article is to use deep learning technology to achieve end-to-end image segmentation. Therefore, a fully convolutional neural network is selected to achieve pixel-level segmentation of plant images. In order to better realize image segmentation and reduce training costs, the network model structure in this paper refers to the U-net network model of Olf Ronneberger [18] and is improved and optimized. The model framework is adjusted, the data reading method in the training process is modified, and the activation function and loss function are optimized. The improved network model not only shortens the training time and saves training costs but also has higher segmentation accuracy and better results. The specific network structure is shown in Figure 3.Figure 3
The overall structure of the model.The first 4 layers on the left side of the network are the typical structure of traditional convolutional layers. The spatial dimension of the input data has gradually reduced to extract high-dimensional features. The maximum pooling layer (maxi on) is composed of 32, 64, and 128 convolution kernels. The fourth layer at the bottom needs to be upsampled input; so, only two3×3 convolutions are performed, and the number of convolution kernels is 256. The right half of the network is centrally symmetrical with the left half. The 5 to 7 layers on the right are deconvolution layers, that is, upward convolution. Each layer is used for upsampling in addition to the normal convolution layer [19]. In addition to the local features of the output of the corresponding downsampling layer, the input of each layer of the deconvolution process also has the deep abstract features obtained by the upsampling of the previous layer. These features are fused through the method of linking, thereby ensuring the spatial dimension of the image. The feature map details are restored without change. The last layer uses 1×1 convolution to map each component feature vector to the required classification. Except for the last layer, all the previous layers have added a BN (batch normalizatoin) layer, which normalizes the features of each layer of the network layer, so that the feature distribution of each layer is more uniform, which can improve the convergence speed of the model while at the same time. The constructed network parameters have reduced to a certain extent, mainly to relieve the operating pressure of hardware equipment and reduce training time while ensuring the effect of segmentation. Because the number of training samples is small, the dripout layer is added to prevent network overfitting appropriately. The network uses the largest pooling layer for the next sampling, mainly considering the use of the largest recognition of edge features. In addition, an optimizer is added to the top of the compiled model to allow the model to support a random gradient drop, to support momentum parameters, etc., and to change the learning rate through the attenuation rate, which has a certain auxiliary optimization effect for the model.


#### 3.3.2. Blade Shape Restoration Based on ASM

Damage to the shape of the leaves will affect the assessment of the degree of crop diseases and insect pests. The automatic detection of a single leaf is the basic task of realizing agricultural precision operations. For example, in the evaluation of the damage degree of crop diseases and insect pests, it is generally necessary to calculate the damaged area statistically. Compared with the internal damage, the damaged area formed at the edge of the leaf is more difficult to evaluate; so, it is necessary to restore the shape of the leaf. The above algorithm is to judge the edge damage through indirect algorithm, and it is quite different from the manual processing result. Based on this, this section will take rape leaves as the object and introduce a leaf shape restoration algorithm based on ASM (Active Shape Model). ASM (Active Shape Model) is a variability model proposed by Zedde et al. [18] that use statistical methods to learn on correctly labeled training samples. It uses a set of discrete key feature points to describe the object. On the one hand, the shape of the model can be changed flexibly to adapt to the change of the target shape. On the other hand, the shape change is limited to the allowable range of the model, and the sample does not deviate excessively from the average shape and cause the shape when the sample changes. The traditional ASM algorithm is generally divided into two stages: model construction and shape matching. The algorithm flow is shown in Figure 4.Figure 4
The ASM shape matching flowchart.The algorithm first establishes a shape model for a specific research object and uses a series of feature points to describe it, which is called the PDM (Principle Distribute Model). In PDM, the geometric description of objects with similar contours, such as human faces and organs, can be represented by a shape vector formed by the series of the coordinates of several key feature points (landmarks), and then the average shape of the sample and the control change are calculated through the shape vector Parameters. When marking the key feature points of the sample, generally select some shape center points, turning points, the largest coordinate point of the contour in a certain direction, or the arc point with the largest curvature. The number of feature points is arbitrary\ and generally depends on the contour complexity of the target shape. Since the feature points are a specific part that characterizes the contour shape of the sample, the number and order of the marked feature points must be strictly consistent when performing the training sample labeling work.


## 3.1. Collecting Device and Measuring Principle

Place the collected plant leaves on a white carrier and flatten them with a transparent object. At this time, the transparent object should have selected to avoid greater light reflection and then use a digital camera or CCD camera with a certain resolution to shoot the leaves vertically. That is, the plant leaf information can be converted into computer-recognizable image information, as shown in Figure1. The image input to the computer is represented by a matrix, that is, a group of pixels arranged in a mesh. The number of pixels per unit length is the resolution of the image, usually expressed in pixels per inch (DPI) [15]. Therefore, as long as you know the size of each pixel and know the number of pixels that make up each blade, you can calculate the area of the blade. At the same time, as long as the number of pixels composing the contour of the leaf is known, the perimeter of the leaf can be as follows.


## 3.2. Dataset Source

The plant images used in this article come from the CVPPP data set, which consists of four folders A1, A2, A3, and A4. This is a data set published by the CVPR conference for the challenge of leaf segmentation. The images were taken with 7 mP Canon AD1000 Power-Shot camera. The plant images are composed of plant top images placed in flowerpots and trays. The images of the A1 and A2 datasets are Lactuca sativa plants. The A1 dataset is acquired every 6 hours during the day and lasts for 3 weeks. The light intensity and background environment of the images are more complicated. The A2 dataset is captured every 20 minutes during the day and lasts for 7 hours. The background environment is relatively simple, but the plant changes are quite different. The A3 data set is composed of Lactuca sativa images [16]. The A4 data set is acquired every other day after emergence, until the growth and development of the plants are fully mature, and there are 20 pseudonans in the data set: top view image of the complete growth period of mustard. In summary, the data set contains images of different types of plants, images of different growth stages of the same plant, and images of plant groups and individual plants. The images in the data set are different in resolution, scene complexity, and light intensity. They have complexity and diversity of plant objects. Therefore, the data set has certain challenges in image segmentation, and the research results are very convincing. Some images in the data set are shown in Figure 2.Figure 2
Dataset source of Lactuca sativa.


## 3.3. Algorithm Implementation

### 3.3.1. Blade Segmentation Network Model

Convolutional neural network (CNN) is a multilayer network architecture with autonomous learning, which is the most widely used in image classification. However, CNN networks tend to lose the detailed information of the image during the feature abstraction process, and the object outline is unclear. Therefore, CNN network has limitations in the field of image segmentation, which has fewer applications. In response to the problems of CNN in image detail processing, Jonathan Long of the University of California at Berkeley and others proposed an improved network architecture, the end-to-end fully convolutional networks (FCN) [17].The biggest difference between FCN and CNN is that it converts all fully connected layers in CNN into convolutional layers. The image enters the FCN network as input data. The first 5 convolutional layers are the same as the CNN network, but the fully connected layers of the last three layers are replaced with convolutional layers. Unlike the one-dimensional output after the CNN fully connected layer, the output after the FCN convolution layer is still a two-dimensional matrix. The main purpose of this article is to use deep learning technology to achieve end-to-end image segmentation. Therefore, a fully convolutional neural network is selected to achieve pixel-level segmentation of plant images. In order to better realize image segmentation and reduce training costs, the network model structure in this paper refers to the U-net network model of Olf Ronneberger [18] and is improved and optimized. The model framework is adjusted, the data reading method in the training process is modified, and the activation function and loss function are optimized. The improved network model not only shortens the training time and saves training costs but also has higher segmentation accuracy and better results. The specific network structure is shown in Figure 3.Figure 3
The overall structure of the model.The first 4 layers on the left side of the network are the typical structure of traditional convolutional layers. The spatial dimension of the input data has gradually reduced to extract high-dimensional features. The maximum pooling layer (maxi on) is composed of 32, 64, and 128 convolution kernels. The fourth layer at the bottom needs to be upsampled input; so, only two3×3 convolutions are performed, and the number of convolution kernels is 256. The right half of the network is centrally symmetrical with the left half. The 5 to 7 layers on the right are deconvolution layers, that is, upward convolution. Each layer is used for upsampling in addition to the normal convolution layer [19]. In addition to the local features of the output of the corresponding downsampling layer, the input of each layer of the deconvolution process also has the deep abstract features obtained by the upsampling of the previous layer. These features are fused through the method of linking, thereby ensuring the spatial dimension of the image. The feature map details are restored without change. The last layer uses 1×1 convolution to map each component feature vector to the required classification. Except for the last layer, all the previous layers have added a BN (batch normalizatoin) layer, which normalizes the features of each layer of the network layer, so that the feature distribution of each layer is more uniform, which can improve the convergence speed of the model while at the same time. The constructed network parameters have reduced to a certain extent, mainly to relieve the operating pressure of hardware equipment and reduce training time while ensuring the effect of segmentation. Because the number of training samples is small, the dripout layer is added to prevent network overfitting appropriately. The network uses the largest pooling layer for the next sampling, mainly considering the use of the largest recognition of edge features. In addition, an optimizer is added to the top of the compiled model to allow the model to support a random gradient drop, to support momentum parameters, etc., and to change the learning rate through the attenuation rate, which has a certain auxiliary optimization effect for the model.


### 3.3.2. Blade Shape Restoration Based on ASM

Damage to the shape of the leaves will affect the assessment of the degree of crop diseases and insect pests. The automatic detection of a single leaf is the basic task of realizing agricultural precision operations. For example, in the evaluation of the damage degree of crop diseases and insect pests, it is generally necessary to calculate the damaged area statistically. Compared with the internal damage, the damaged area formed at the edge of the leaf is more difficult to evaluate; so, it is necessary to restore the shape of the leaf. The above algorithm is to judge the edge damage through indirect algorithm, and it is quite different from the manual processing result. Based on this, this section will take rape leaves as the object and introduce a leaf shape restoration algorithm based on ASM (Active Shape Model). ASM (Active Shape Model) is a variability model proposed by Zedde et al. [18] that use statistical methods to learn on correctly labeled training samples. It uses a set of discrete key feature points to describe the object. On the one hand, the shape of the model can be changed flexibly to adapt to the change of the target shape. On the other hand, the shape change is limited to the allowable range of the model, and the sample does not deviate excessively from the average shape and cause the shape when the sample changes. The traditional ASM algorithm is generally divided into two stages: model construction and shape matching. The algorithm flow is shown in Figure 4.Figure 4
The ASM shape matching flowchart.The algorithm first establishes a shape model for a specific research object and uses a series of feature points to describe it, which is called the PDM (Principle Distribute Model). In PDM, the geometric description of objects with similar contours, such as human faces and organs, can be represented by a shape vector formed by the series of the coordinates of several key feature points (landmarks), and then the average shape of the sample and the control change are calculated through the shape vector Parameters. When marking the key feature points of the sample, generally select some shape center points, turning points, the largest coordinate point of the contour in a certain direction, or the arc point with the largest curvature. The number of feature points is arbitrary\ and generally depends on the contour complexity of the target shape. Since the feature points are a specific part that characterizes the contour shape of the sample, the number and order of the marked feature points must be strictly consistent when performing the training sample labeling work.


## 3.3.1. Blade Segmentation Network Model

Convolutional neural network (CNN) is a multilayer network architecture with autonomous learning, which is the most widely used in image classification. However, CNN networks tend to lose the detailed information of the image during the feature abstraction process, and the object outline is unclear. Therefore, CNN network has limitations in the field of image segmentation, which has fewer applications. In response to the problems of CNN in image detail processing, Jonathan Long of the University of California at Berkeley and others proposed an improved network architecture, the end-to-end fully convolutional networks (FCN) [17].The biggest difference between FCN and CNN is that it converts all fully connected layers in CNN into convolutional layers. The image enters the FCN network as input data. The first 5 convolutional layers are the same as the CNN network, but the fully connected layers of the last three layers are replaced with convolutional layers. Unlike the one-dimensional output after the CNN fully connected layer, the output after the FCN convolution layer is still a two-dimensional matrix. The main purpose of this article is to use deep learning technology to achieve end-to-end image segmentation. Therefore, a fully convolutional neural network is selected to achieve pixel-level segmentation of plant images. In order to better realize image segmentation and reduce training costs, the network model structure in this paper refers to the U-net network model of Olf Ronneberger [18] and is improved and optimized. The model framework is adjusted, the data reading method in the training process is modified, and the activation function and loss function are optimized. The improved network model not only shortens the training time and saves training costs but also has higher segmentation accuracy and better results. The specific network structure is shown in Figure 3.Figure 3
The overall structure of the model.The first 4 layers on the left side of the network are the typical structure of traditional convolutional layers. The spatial dimension of the input data has gradually reduced to extract high-dimensional features. The maximum pooling layer (maxi on) is composed of 32, 64, and 128 convolution kernels. The fourth layer at the bottom needs to be upsampled input; so, only two3×3 convolutions are performed, and the number of convolution kernels is 256. The right half of the network is centrally symmetrical with the left half. The 5 to 7 layers on the right are deconvolution layers, that is, upward convolution. Each layer is used for upsampling in addition to the normal convolution layer [19]. In addition to the local features of the output of the corresponding downsampling layer, the input of each layer of the deconvolution process also has the deep abstract features obtained by the upsampling of the previous layer. These features are fused through the method of linking, thereby ensuring the spatial dimension of the image. The feature map details are restored without change. The last layer uses 1×1 convolution to map each component feature vector to the required classification. Except for the last layer, all the previous layers have added a BN (batch normalizatoin) layer, which normalizes the features of each layer of the network layer, so that the feature distribution of each layer is more uniform, which can improve the convergence speed of the model while at the same time. The constructed network parameters have reduced to a certain extent, mainly to relieve the operating pressure of hardware equipment and reduce training time while ensuring the effect of segmentation. Because the number of training samples is small, the dripout layer is added to prevent network overfitting appropriately. The network uses the largest pooling layer for the next sampling, mainly considering the use of the largest recognition of edge features. In addition, an optimizer is added to the top of the compiled model to allow the model to support a random gradient drop, to support momentum parameters, etc., and to change the learning rate through the attenuation rate, which has a certain auxiliary optimization effect for the model.


## 3.3.2. Blade Shape Restoration Based on ASM

Damage to the shape of the leaves will affect the assessment of the degree of crop diseases and insect pests. The automatic detection of a single leaf is the basic task of realizing agricultural precision operations. For example, in the evaluation of the damage degree of crop diseases and insect pests, it is generally necessary to calculate the damaged area statistically. Compared with the internal damage, the damaged area formed at the edge of the leaf is more difficult to evaluate; so, it is necessary to restore the shape of the leaf. The above algorithm is to judge the edge damage through indirect algorithm, and it is quite different from the manual processing result. Based on this, this section will take rape leaves as the object and introduce a leaf shape restoration algorithm based on ASM (Active Shape Model). ASM (Active Shape Model) is a variability model proposed by Zedde et al. [18] that use statistical methods to learn on correctly labeled training samples. It uses a set of discrete key feature points to describe the object. On the one hand, the shape of the model can be changed flexibly to adapt to the change of the target shape. On the other hand, the shape change is limited to the allowable range of the model, and the sample does not deviate excessively from the average shape and cause the shape when the sample changes. The traditional ASM algorithm is generally divided into two stages: model construction and shape matching. The algorithm flow is shown in Figure 4.Figure 4
The ASM shape matching flowchart.The algorithm first establishes a shape model for a specific research object and uses a series of feature points to describe it, which is called the PDM (Principle Distribute Model). In PDM, the geometric description of objects with similar contours, such as human faces and organs, can be represented by a shape vector formed by the series of the coordinates of several key feature points (landmarks), and then the average shape of the sample and the control change are calculated through the shape vector Parameters. When marking the key feature points of the sample, generally select some shape center points, turning points, the largest coordinate point of the contour in a certain direction, or the arc point with the largest curvature. The number of feature points is arbitrary\ and generally depends on the contour complexity of the target shape. Since the feature points are a specific part that characterizes the contour shape of the sample, the number and order of the marked feature points must be strictly consistent when performing the training sample labeling work.


## 4. Experimental Design and Result Analysis

### 4.1. Tool Selection

Based on the basic elements of the previous in-depth learning, use Caffe to build the model. Caffe is an in-depth learning framework developed by C++. It has high computational efficiency. It includes Python and MATLAB interfaces. It can realize automatic derivation and back propagation training after reading the network model definition, and it can switch between CPU and GPU calculations. In this study, the FCN-based taproot segmentation model of rapeseed at seedling stage is implemented on the PyTorch deep learning framework. PyTorch is the Python version of Torch [20]. Torch is a deep learning framework released by Facebook. Table 1 shows the configuration and tools. Because it supports dynamic definition of calculation graphs, it is more flexible and convenient to use and is especially suitable for small and medium machine learning projects. But because Torch’s development language is Lua, its usage is relatively niche. PyTorch inherits Torch’s flexible features and uses the widely popular Python as a development language; so, it is very popular once it is launched. PyTorch provides an abstract method similar to NumPy to represent tensors (or multidimensional arrays), supports single or multiple CPU or GPU operations, and can be switched freely, which improves the training speed of neural networks. Compared with the existing mainstream deep learning architecture, PyTorch has two outstanding features: it can not only support GPU acceleration but also support dynamic neural networks.Table 1
The configuration and tools.
HardwareModel parameterSoftwareVersionHardware configurationProcessorIntel i7-8700Software configurationCUDA9.3RAM32GUbuntu16.1MotherboardDellOpenCV2.4.2Graphics cardGeForce GTX 1080TiPython3.6.6


### 4.2. Image Preprocessing

The area of the blade area is a basic characteristic of the blade, which describes the size of the blade. For the regionR, suppose the side length of the square pixel is the unit length, then the area is calculated as follows. It can be seen that calculating the area of the area is to count the pixels in the leaf area. Figure 5 shows the results obtained by using different area calculation methods for the same area (suppose the side length of the pixel is 1).6(a) (1), 6(b)6(c)) The two methods shown are intuitively okay, but the errors are relatively large. It can be proved that using the method of pixel counting to find the area of the leaf area is not only the simplest but also the most unbiased and consistent area of the original simulation area. The above unbiased estimate corresponds to the measurement accuracy and unanimously estimates the accuracy of the corresponding measurement.Figure 5
Collected Lactuca sativa leaves.Figure 6
Comparison of several calculation methods.
(a)(b)(c)The extracting effective points of leaves used to calculate the leaf area, and the leaf image needs to be binarized. Because some parts of the leaf area are affected by diseases, the color of the leaf area is quite different from that of the healthy part. Therefore, the result of binarization may cause the diseased spot area to appear white (see Figure6), and the white pixels cannot participate. The calculation of the effective point affects the accuracy of the calculation.


### 4.3. Perimeter Nondestructive Measurement

#### 4.3.1. Selection of the Calculation Method

The boundary length of the blade is a simple global feature of the boundary, which is the perimeter of the contour of the area enclosed by the boundary. Consider the case where the blade area is composed of internal points plus boundary points [21]. The boundary B of the area R is composed of all the boundary points of R connected in the 4-connected direction or 8-connected direction, and the other points of the area become the internal points of the area. For the region R, every boundary point P should meet the following conditions: one is that P itself belongs to the region R; the other is that there are pixels in the neighborhood of P that do not belong to the region R. If the internal points of the region R are determined by the 8-way connectivity, the obtained boundary is 4-connected. If the internal points of the region R are determined by the 4-directional connectivity, the obtained boundary is 8-connected. The boundary points and internal points of the area should be defined by different connectivity; otherwise, there will be ambiguity. Figure 3 can be used to understand the above concept.CONV is the convolution calculation layer, linear product summation. It can be seen from Figure7 that the left is the image input, and the middle part is the ASM (with a set of fixed-weight neurons called convolution kernel). Different filter filters will get different output data, such as color depth and contour. Equivalent to if you want to extract different features of the image, use different filters to extract the desired specific information about the image, such as color depth or contour.Figure 7
Comparison results by ASM.


#### 4.3.2. Morphological Feature Extraction Accuracy and Growth Analysis

In order to evaluate the segmentation performance of the model and the accuracy of phenotypic parameter extraction, 30 images are selected in the test set, and these 30 images are segmented using the model in this paper. Ferdinand et al. [22] used image processing methods to measure the defects of cotton leaf holes and edges to assess the severity of pests. Xu et al. [23] designed an algorithm for locating unclosed leaf wormholes based on polar coordinates, which can accurately locate and reconstruct a rough edge. Phenotypic parameters such as coverage, circumscribed circle radius, bounding box area, and perimeter were extracted from the segmentation results, and the correlations were compared with the values extracted from the manual segmentation results. The comparison results are shown in Figure 8.Figure 8
Vegetation coverage density in different data sets.Vegetation coverage refers to the variation ratio of the vertical projection area of all plants including crops, shrubs, trees and weeds, branches, and leaves on the ground of their growth area to the area of the study statistical area [24, 25]. During the processing of the data set, standardized cropping has been carried out to ensure the same size of the background area. Therefore, the ratio of leaf pixels to the total pixel points in the image segmented by the segmentation network is the ratio of the vertical projection area of the plant to the total background area. Multiply the above ratio with the total area, and the result is the coverage of different products in the same area. From the analysis results in Figure 8, it can be seen that the rosettes such as Lactuca sativa and tobacco have similar growth trends. There is a certain slow development period in the early stage, and they begin to grow rapidly after reaching a certain number of days. The growth and development of Lactuca sativa thaliana in the first five days were relatively slow. After the fifth day, it began to grow rapidly, and the coverage was significantly increased. However, tobacco plants entered a period of rapid growth starting on the third day. After a series of processing and training, the trained neural network model and parameters are obtained, and the test set is used to test and finally get an average of 90% accuracy. The plant identification model is packaged as a callable program module, which can be actually applied to web services and mobile. The application program realizes real-time online identification of plant varieties through cloud services.


## 4.1. Tool Selection

Based on the basic elements of the previous in-depth learning, use Caffe to build the model. Caffe is an in-depth learning framework developed by C++. It has high computational efficiency. It includes Python and MATLAB interfaces. It can realize automatic derivation and back propagation training after reading the network model definition, and it can switch between CPU and GPU calculations. In this study, the FCN-based taproot segmentation model of rapeseed at seedling stage is implemented on the PyTorch deep learning framework. PyTorch is the Python version of Torch [20]. Torch is a deep learning framework released by Facebook. Table 1 shows the configuration and tools. Because it supports dynamic definition of calculation graphs, it is more flexible and convenient to use and is especially suitable for small and medium machine learning projects. But because Torch’s development language is Lua, its usage is relatively niche. PyTorch inherits Torch’s flexible features and uses the widely popular Python as a development language; so, it is very popular once it is launched. PyTorch provides an abstract method similar to NumPy to represent tensors (or multidimensional arrays), supports single or multiple CPU or GPU operations, and can be switched freely, which improves the training speed of neural networks. Compared with the existing mainstream deep learning architecture, PyTorch has two outstanding features: it can not only support GPU acceleration but also support dynamic neural networks.Table 1
The configuration and tools.
HardwareModel parameterSoftwareVersionHardware configurationProcessorIntel i7-8700Software configurationCUDA9.3RAM32GUbuntu16.1MotherboardDellOpenCV2.4.2Graphics cardGeForce GTX 1080TiPython3.6.6


## 4.2. Image Preprocessing

The area of the blade area is a basic characteristic of the blade, which describes the size of the blade. For the regionR, suppose the side length of the square pixel is the unit length, then the area is calculated as follows. It can be seen that calculating the area of the area is to count the pixels in the leaf area. Figure 5 shows the results obtained by using different area calculation methods for the same area (suppose the side length of the pixel is 1).6(a) (1), 6(b)6(c)) The two methods shown are intuitively okay, but the errors are relatively large. It can be proved that using the method of pixel counting to find the area of the leaf area is not only the simplest but also the most unbiased and consistent area of the original simulation area. The above unbiased estimate corresponds to the measurement accuracy and unanimously estimates the accuracy of the corresponding measurement.Figure 5
Collected Lactuca sativa leaves.Figure 6
Comparison of several calculation methods.
(a)(b)(c)The extracting effective points of leaves used to calculate the leaf area, and the leaf image needs to be binarized. Because some parts of the leaf area are affected by diseases, the color of the leaf area is quite different from that of the healthy part. Therefore, the result of binarization may cause the diseased spot area to appear white (see Figure6), and the white pixels cannot participate. The calculation of the effective point affects the accuracy of the calculation.


## 4.3. Perimeter Nondestructive Measurement

### 4.3.1. Selection of the Calculation Method

The boundary length of the blade is a simple global feature of the boundary, which is the perimeter of the contour of the area enclosed by the boundary. Consider the case where the blade area is composed of internal points plus boundary points [21]. The boundary B of the area R is composed of all the boundary points of R connected in the 4-connected direction or 8-connected direction, and the other points of the area become the internal points of the area. For the region R, every boundary point P should meet the following conditions: one is that P itself belongs to the region R; the other is that there are pixels in the neighborhood of P that do not belong to the region R. If the internal points of the region R are determined by the 8-way connectivity, the obtained boundary is 4-connected. If the internal points of the region R are determined by the 4-directional connectivity, the obtained boundary is 8-connected. The boundary points and internal points of the area should be defined by different connectivity; otherwise, there will be ambiguity. Figure 3 can be used to understand the above concept.CONV is the convolution calculation layer, linear product summation. It can be seen from Figure7 that the left is the image input, and the middle part is the ASM (with a set of fixed-weight neurons called convolution kernel). Different filter filters will get different output data, such as color depth and contour. Equivalent to if you want to extract different features of the image, use different filters to extract the desired specific information about the image, such as color depth or contour.Figure 7
Comparison results by ASM.


### 4.3.2. Morphological Feature Extraction Accuracy and Growth Analysis

In order to evaluate the segmentation performance of the model and the accuracy of phenotypic parameter extraction, 30 images are selected in the test set, and these 30 images are segmented using the model in this paper. Ferdinand et al. [22] used image processing methods to measure the defects of cotton leaf holes and edges to assess the severity of pests. Xu et al. [23] designed an algorithm for locating unclosed leaf wormholes based on polar coordinates, which can accurately locate and reconstruct a rough edge. Phenotypic parameters such as coverage, circumscribed circle radius, bounding box area, and perimeter were extracted from the segmentation results, and the correlations were compared with the values extracted from the manual segmentation results. The comparison results are shown in Figure 8.Figure 8
Vegetation coverage density in different data sets.Vegetation coverage refers to the variation ratio of the vertical projection area of all plants including crops, shrubs, trees and weeds, branches, and leaves on the ground of their growth area to the area of the study statistical area [24, 25]. During the processing of the data set, standardized cropping has been carried out to ensure the same size of the background area. Therefore, the ratio of leaf pixels to the total pixel points in the image segmented by the segmentation network is the ratio of the vertical projection area of the plant to the total background area. Multiply the above ratio with the total area, and the result is the coverage of different products in the same area. From the analysis results in Figure 8, it can be seen that the rosettes such as Lactuca sativa and tobacco have similar growth trends. There is a certain slow development period in the early stage, and they begin to grow rapidly after reaching a certain number of days. The growth and development of Lactuca sativa thaliana in the first five days were relatively slow. After the fifth day, it began to grow rapidly, and the coverage was significantly increased. However, tobacco plants entered a period of rapid growth starting on the third day. After a series of processing and training, the trained neural network model and parameters are obtained, and the test set is used to test and finally get an average of 90% accuracy. The plant identification model is packaged as a callable program module, which can be actually applied to web services and mobile. The application program realizes real-time online identification of plant varieties through cloud services.


## 4.3.1. Selection of the Calculation Method

The boundary length of the blade is a simple global feature of the boundary, which is the perimeter of the contour of the area enclosed by the boundary. Consider the case where the blade area is composed of internal points plus boundary points [21]. The boundary B of the area R is composed of all the boundary points of R connected in the 4-connected direction or 8-connected direction, and the other points of the area become the internal points of the area. For the region R, every boundary point P should meet the following conditions: one is that P itself belongs to the region R; the other is that there are pixels in the neighborhood of P that do not belong to the region R. If the internal points of the region R are determined by the 8-way connectivity, the obtained boundary is 4-connected. If the internal points of the region R are determined by the 4-directional connectivity, the obtained boundary is 8-connected. The boundary points and internal points of the area should be defined by different connectivity; otherwise, there will be ambiguity. Figure 3 can be used to understand the above concept.CONV is the convolution calculation layer, linear product summation. It can be seen from Figure7 that the left is the image input, and the middle part is the ASM (with a set of fixed-weight neurons called convolution kernel). Different filter filters will get different output data, such as color depth and contour. Equivalent to if you want to extract different features of the image, use different filters to extract the desired specific information about the image, such as color depth or contour.Figure 7
Comparison results by ASM.


## 4.3.2. Morphological Feature Extraction Accuracy and Growth Analysis

In order to evaluate the segmentation performance of the model and the accuracy of phenotypic parameter extraction, 30 images are selected in the test set, and these 30 images are segmented using the model in this paper. Ferdinand et al. [22] used image processing methods to measure the defects of cotton leaf holes and edges to assess the severity of pests. Xu et al. [23] designed an algorithm for locating unclosed leaf wormholes based on polar coordinates, which can accurately locate and reconstruct a rough edge. Phenotypic parameters such as coverage, circumscribed circle radius, bounding box area, and perimeter were extracted from the segmentation results, and the correlations were compared with the values extracted from the manual segmentation results. The comparison results are shown in Figure 8.Figure 8
Vegetation coverage density in different data sets.Vegetation coverage refers to the variation ratio of the vertical projection area of all plants including crops, shrubs, trees and weeds, branches, and leaves on the ground of their growth area to the area of the study statistical area [24, 25]. During the processing of the data set, standardized cropping has been carried out to ensure the same size of the background area. Therefore, the ratio of leaf pixels to the total pixel points in the image segmented by the segmentation network is the ratio of the vertical projection area of the plant to the total background area. Multiply the above ratio with the total area, and the result is the coverage of different products in the same area. From the analysis results in Figure 8, it can be seen that the rosettes such as Lactuca sativa and tobacco have similar growth trends. There is a certain slow development period in the early stage, and they begin to grow rapidly after reaching a certain number of days. The growth and development of Lactuca sativa thaliana in the first five days were relatively slow. After the fifth day, it began to grow rapidly, and the coverage was significantly increased. However, tobacco plants entered a period of rapid growth starting on the third day. After a series of processing and training, the trained neural network model and parameters are obtained, and the test set is used to test and finally get an average of 90% accuracy. The plant identification model is packaged as a callable program module, which can be actually applied to web services and mobile. The application program realizes real-time online identification of plant varieties through cloud services.


## 5. Conclusion

Phenotypics and measurement methods are developing rapidly, but large-scale automatic collection of field data needs to be improved. Therefore, fast, high-throughput measurement methods are necessary. Many current researches integrate related sensors and realize automatic data collection. Many of the abovementioned sensors can measure multiple parameters at the same time; so, it is very important to optimize the installation position and angle of the sensor toy develop its performance full. Fusion of various data (such as color data, depth data, and spectral data) provided by one or more sensors is helpful for data processing and phenotypic parameter extraction. On the other hand, some current researches are facing a variety of problems, which seriously affect the measurement accuracy of phenotypic parameters. It is especially important to improve the applicability and robustness of sensors in complex field environments. In order to improve the accuracy of plant image classification, this paper proposes to combine the ASM method to construct a new model and then train it in the collected plant image data set. The experimental results show that the average classification accuracy of this method is 94.3%, which is higher than the traditional method of plant image classification. The research contributes to apply this model to various data sets, which improves the fault tolerance of this model. It is also necessary to monitor the growing environment of crops. Parameters such as soil nutrients, external environment temperature and humidity, and light irradiance should have recorded and related databases should be established to analyze the relationship between crop phenotypes and genomics. In addition, roots are a vital part of crop growth, and rapid and nondestructive root measurement methods should have developed in the future. The leaf counting, age regression, and mutant classification parameters can be included in the future.



---

*Source: 7664045-2022-04-15.xml*