# Research Progress of Crop Disease Image Recognition Based on Wireless Network Communication and Deep Learning

Authors: Yuanhui Yu
Journal: Wireless Communications and Mobile Computing
 (2021)

---

## Abstract

The traditional digital image processing technology has its limitations. It requires manual design features, which consumes manpower and material resources, and identifies crops with a single type, and the results are bad. Therefore, to find an efficient and fast real-time disease image recognition method is very meaningful. Deep learning is a machine learning algorithm that can automatically learn representative features to achieve better results in areas of image recognition. Therefore, the purpose of this paper is to use deep learning methods to identify crop pests and diseases and to find efficient and fast real-time image recognition methods of disease. Deep learning is a newly developed discipline in recent years. Its purpose is to study how to actively obtain a variety of feature representation methods from data samples and rely on data-driven methods, a series of nonlinear transformations are applied to finally collect the original data from specific to abstract, from general to specified semantics, and from low-level to high-level characteristic forms. This paper analyzes the classical and the latest neural network structure based on the theory of deep learning. For the problem that the network based on natural image classification is not suitable for crop pest and disease identification tasks, this paper has improved the network structure that can take care of both recognition speed and recognition accuracy. We discussed the influence of the crop pest and disease feature extraction layer on recognition performance. Finally, we used the inner layer as the main structure to be the pest and disease feature extraction layer by comparing the advantages and disadvantages of the inner and global average pooling layers. We analyze various loss functions such as Softmax Loss, Center Loss, and Angular Softmax Loss for pest identification. In view of the shortcomings of difficulty in loss function training, convergence, and operation, making the distance between pests and diseases smaller and the distance between classes more greater improved the loss function and introduced techniques such as feature normalization and weight normalization. The experimental results show that the method can effectively enhance the characteristic expression ability of pests and diseases and thus improve the recognition rate of pests and diseases. Moreover, the method makes the pest identification network training simpler and can improve the pest and disease recognition rate better.


---

## Body

## 1. Introduction

Crop diseases can lead to the abuse of pesticides while causing a decline in the yield and quality of agricultural products [1, 2]. This not only increases the cost of agricultural production but also brings food safety and environmental pollution problems [3]. The proposal of deep learning comes from artificial neural networks. In the middle of the 20th century, as humans made major breakthroughs in the domain of neuroscience, researchers began to imitate the structure of the human brain and proposed the idea of artificial neural networks. Subsequently, the research field was gradually expanded and scholars began to pay attention to how to make machines as intelligent as humans. The area of cultivated land currently polluted by pesticides in China has exceeded 16 million hectares. Practice has shown that rational application of pesticides is the most effective means of controlling crop diseases. Rational application of pesticides can not only effectively control the occurrence of crop diseases but also reduce the pollution of pesticides to the environment and agricultural products. It is well known that the rational application of pesticides requires accurate access to crop growth status information, the most critical of which is the rapid and accurate identification of the types of crop diseases. The identification of traditional crop diseases mainly relies on the experience of farmers or professional technicians in farmland. Due to the limitations of professional knowledge, farmers often cannot judge or misjudge some crop diseases [4]. In addition, because some crop diseases are not obvious or the symptoms are similar at the beginning of the disease, even professional technicians are difficult to identify them accurately [5]. When the symptoms are clearly identifiable, they miss the best prevention time, which makes it difficult to obtain accurate and timely control of crop diseases. Therefore, traditional crop disease identification is not only time consuming and laborious but also inefficient [6].The deep learning is derived from artificial neural networks. In the mid-20th century, as humans made major breakthroughs in the field of neuroscience, researchers began to imitate the structure of the human brain and proposed the idea of artificial neural networks. Then, they gradually expand the field of research; scholars began to pay attention to how to make the machine as intelligent as human beings. As a special neural network that was originally constructed and implemented, the perceptron has gradually attracted more attention to the research of neural networks. However, further research by humans has found that a perceptron with a two-layer structure is only applicable to prebuilt feature learning linear functions, but not linear indivisible problems [7]. The phenomenon of crop diseases and insect pests is obvious in the early stage. If appropriate control measures are not taken in the early stage, it will cause irreversible losses to yield and crop survival, which will affect the yield, quality, and productivity of agricultural products. Limited by the theoretical and technical levels at that time, the research on artificial neural networks is becoming less and less optimistic. Until the end of the 20th century, the back propagation algorithm was ridiculed, making the artificial neural network attract attention. Compared with the perceptron, the BP neural network has increased the hidden layer unit, which has the advantage of being able to construct a complicated data model and flexible feature expression ability, but it is because of the increased level that the learning difficulty becomes large and it is easy to fall into the local minimum. However, when calculating the gradient of each layer node, the gradient weakening will occur in the lower layer direction of the network. It is precisely because these shortcomings affect the performance of the BP neural network, so that it can only build a shallow learning model, and then, its scalability is affected [8].Considering that the depth of the multilayer network in deep learning will easily lead to the increase of training complexity and the disappearance of gradients, the ResNet convolutional neural network model is used to train the crop disease dataset. The residual learning unit in the ResNet network can simplify the convolutional layer. The learning process only learns the difference between the input and output, deepening the network layer and accelerating the model. Alvaro et al. proposed a tomato pest detection method based on deep learning, which uses images captured by different resolution camera devices for detection. Their goal is to find a more appropriate deep learning framework for the task. Therefore, they considered three main detectors: a faster region-based convolutional neural network, a region-based complete convolutional network, and a single-point multibox detector. These three detectors are called “deep learning elements.” They combined these meta-architectures with the “deep feature extractor.” They demonstrate the performance of deep meta-architectures and feature extractors and propose a method for local and global class annotation and data expansion to improve accuracy during training and reduce the number of false positives [9]. Using convolutional neural network models and deep learning methods to detect and diagnose simple leaf images of healthy and diseased plants. [10, 11]. The model was trained using an open database of 87,848 images containing 25 different plants and 58 different (plant, disease) combinations, including healthy plants. Several model architectures were trained to achieve the best performance of 99.53% in identifying the corresponding (plant, disease) combination (or healthy plant) [12]. Zhang et al. proposed a plant disease identification method based on plant leaf images. Firstly, the spots are segmented to extract the disease feature vector. Then, the K-nearest neighbor classifier is used to identify the plant diseases to provide extraction features [13]. Kyosuke et al. applied the super-resolution method to low-resolution images of tomato diseases to restore the detailed performance of plant organ damage. They also used high-resolution, low-resolution, and super-resolution images for disease classification to assess the effectiveness of super-resolution methods in disease classification [14]. The phenomena of crop diseases and insect pests are obvious in the early stage. If appropriate control measures are not taken in the early stage, it will cause irreversible losses to yield and crop survival, which will affect the yield, quality, and productivity of agricultural products.Based on the theory of deep learning, this paper analyzes the classical and the latest neural network structure [15]. The network based on natural image classification is not suitable for crop pest identification tasks. This paper improves the network structure that can simultaneously recognize the recognition speed and recognition accuracy. The research objects of this article are wheat powdery mildew and leaf rust, peanut black and brown spots, tobacco wildfire, brown spot, and mosaic disease collected in a field environment. Use deep convolutional neural networks to identify diseases in complex backgrounds. The influence of crop pest and disease feature extraction layer on recognition performance is discussed. The advantages and disadvantages of the layered and global average pooling layer are finally adopted as the structure of the pest and disease feature extraction layer. Analyze various loss functions such as Softmax Loss, Center Loss, and Angular Softmax Loss for pest identification. Plant diseases and insect pests will cause national economic losses. Pests and diseases in farming are a complex issue of timeliness. During the process of crops from sowing to growth and harvest, the pathogens are different at different stages, which in turn affects the final agricultural products and quality. It is difficult to train, difficult to converge, and difficult to operate, so that the distance between pests and diseases is smaller and the distance between classes is larger, improved the loss function, and introduced techniques such as feature normalization and weight normalization. The experimental results show that the method can effectively enhance the characteristic expression ability of pests and diseases and thus improve the recognition rate of pests and diseases. Moreover, the method makes the pest identification network training simpler and can better improve the pest and disease recognition rate.


## 2. Proposed Method

### 2.1. Deep Learning

#### 2.1.1. Overview of Deep Learning

With the development of big data and general parallel computing units, applications based on deep learning have also developed rapidly. Deep learning is a neural network with multiple hidden layers. Similar to shallow neural networks, deep neural networks can model complex nonlinear systems. Many layers hidden layer networks provide a higher level of abstraction for the model, thus improving the capabilities of the model. The biggest difference between deep learning and other machine learning is how to find features. The extraction process of features is an abstract process. Deep learning abstraction simulates the way human neurons communicate information. In theory, many classification and prediction problems can be solved. Convolutional neural networks are a typical deep neural network. Great success has been achieved in the many categories and recognition tasks of images.


#### 2.1.2. Artificial Neural Network

There are many diseases and insect pests of crops. There are external factors, including their own factors, their own plant attributes, external weather factors, and the influence of other plants such as weeds in the growing environment and pests in the natural environment, which will affect the crops. The output and quality have a huge impact.The research of the artificial neural network is largely inspired by the bionics. It consists of a series of simple artificial neurons connected to each other. Each neuron has three parts: input, artificial nerve cells, and output. As shown in Figure1, the input signal of one neuron can come from an externally given initial value or it can be the output of another neuron. Artificial nerve cells integrate these input signals and perform threshold operations. If the integrated stimulus value exceeds a certain threshold, the neuron enters an active state; otherwise, the neuron is in a suppressed state.Figure 1
Schematic diagram of artificial neurons.Just as the brain’s neurons can constantly adjust the connections between neurons to achieve continuous learning and progress. The ANN network can also adapt the network to the training set by adjusting the weight values on the continuous input.(1) PerceptronA perceptron is a simple artificial neuron with only two outputs. Inputx1,x2,⋯,xn into the perceptron in turn, and each input value corresponds to a weight wi, in addition to an offset w0 as shown in equation (1):
(1)net=∑1nwi⋅xi+w0.Ifx0=1, then, equation (1) can be expressed as
(2)net=∑0nwi.xi=w⟶⋅x⟶.The output of the perceptron is as shown in equation (2) after the thresholding process:
(3)Ox⟶=1,∑0nwi⋅xi≥0,−1,other.The sensor works in much the same way as a neuron, and it can classify two types of samples. For the input sample output 1 on one side and the sample output −1 on the other side, the training process is to continuously adjust the weightw1,w2,⋯,wn.(2) Linear UnitsSensors with only 1 and −1 outputs limit the classification of processing. A simple generalization is to cancel the threshold module. One input is a linear unit ofx1,x2,⋯,xn, and its output is a linear combination of n inputs based on weights, plus offset w0. The core task of training a linear unit is to adjust the weight w1,w2,⋯,wn so that the output is as similar as possible to the target output of the training sample.(3) Error CriteriaA metric is defined to measure the training error of the ANN relative to the training example under the current weight vectorw⟶. A common training error criterion is the squared error:
(4)Ew⟶=12∑d∈Dtd−Od2,where D is the trained sample set, td is the target output of the training sample d, and od is the actual output Oxd of the linear unit pair training sample d. It is necessary to calculate the direction of the fastest fall of the error, so the maximum direction of the directional derivative can be obtained by calculating the partial derivative of the E with respect to each component of the vector w⟶, denoted as ∇Ew⟶.
(5)∇Ew⟶=∂E∂w1,∂E∂w2,⋯,∂E∂wn.Since∇Ew⟶ itself is a vector that represents the largest directional derivative, it corresponds to the fastest rising direction of the E and the direction of the fastest falling of the error is only required to require −∇Ew⟶. So, the law of training should be
(6)w⟶⟵w⟶+Δw⟶,Δw⟶=−η∇Ew⟶.η in equation (6) is that the learning rate is a positive super parameter and is the step size of the gradient descent search. w⟶ is the current search point in the solution space, Δw⟶ represents a displacement to the current fastest descent direction, and w⟶⟵w⟶+Δw⟶ means to move a short distance from the current point along the fastest descent direction and update the current position to this position.All neurons are of Boolean type, that is, state 0 means inhibition or 1 means activation, and then, the energy of RBM is definedp as(7)Ev,hθ=−∑n=1naivi−∑j=1nbjhj−∑i=1mviWijhj.The probability distributionPvθ of the visible layer is as follows, where Zθ is the normalization factor:
(8)Pvθ=1Zθ∑he−Ev,hθ,Zθ=∑v∑he−Ev,hθ.The cost (loss) function of the Softmax regression algorithm is(9)Jθ=−1M∑i=1m∑j=1k1yi=jlog∑j=1keθJT.Prove that the current loss function is strictly convex and its local minimum is not unique, so the optimal parameter to be solved is not unique. The proof process is as follows:(10)Pyi=jxi;θ=eθj−ωTxi∑l=1keθj−ωTxi.Usually, add regular items to modify the cost function, as follows:(11)Jθ=−1m∑i=1m∑j=1k1yi=jlog∑l=1keθlT.λ is the weight of the regular term. At this time, the cost function can also be updated with the gradient descent method. The gradient after derivation is
(12)∇θjJθ=−1m∑i=1mxi1yi=j−pyi=j;θ+λθ.Update the parameters with the help of partial derivatives:(13)θj=θj−∂∇θJθ.The parameter update of Softmax is completed through repeated introduction.(4) BackpropagationThe weighting parameters of the neural network need to be able to be adjusted like human nerve cells, so the ability to automatically adjust parameters is required, and the backpropagation algorithm can be used to adjust the error according to the error.(5) Many Layer Artificial Neural NetworksIn a single neuron or linear unit, only linear problems can be solved. However, there are great deficiencies in solving nonlinear problems, but real-life problems are often nonlinear. The cells in the brain and other cells are interconnected, mimicking the cellular connections of the human brain, and the artificial neural network can also form a many-layered neural network in a similar manner. The characteristics of pests and diseases in specific areas are periodicity, seasonality, and repetitiveness. That is, the diseases and insect pests in the same area show different diseases and insect pests in different seasons, multiple diseases and insect pests occur repeatedly in the same season throughout the year, and several fixed diseases and insect pests appear in turn. The emergence of many layers of neural networks can solve nonlinear problems. In the layered feedforward network, as shown in Figure2, generally, there are at least three layers: one input layer, one output layer, and hidden layer, and the hidden layer is generally multiple. Adjacent neurons are interconnected.Figure 2
Schematic diagram of multilayer artificial neurons.


#### 2.1.3. Convolutional Neural Network

Before studying the subject of crop disease image recognition, you need to be familiar with relevant basic theoretical knowledge, such as the recognition model used in deep learning, image recognition process, convolutional neural network structure, deep learning framework, experimental dataset, preprocessing and labeling categories, and many more. As the most representative deep learning network model, the convolutional neural network has benefited from the development of computer networks and computing power. The two-dimensional image is stored in the form of a matrix in a computer, and the relative position between each pixel is fixed. The convolution operation of the fixed convolution kernel size can extract the features of the two-dimensional image. It is this feature of two-dimensional images that enables convolutional neural networks to be successfully applied to identify images and target detection. The CNN network is an important network model for deep learning. CNN’s connection method and network structure are very suitable for dealing with image problems, as shown in Figure3 (picture from the network, http://www.baidu.com/).Figure 3
General model of convolutional neural networks.In the convolutional neural network, the convolution kernel represents the local features of the picture. The image is processed by multilayer convolution and pooling to convert the image information into features with higher information. At the same time, it is necessary to constantly adjust the convolution kernel. The parameters enable the convolution kernel to complete the work of a certain image feature. In image recognition or classification tasks, images are extracted by a convolution operation. Then, the size of the feature map is reduced through the pooling layer and the classification is output through the full connection layer and finally through some activation functions. The main process of accurate crop management is shown in Figure4.Figure 4
Accurate management process of crops.(1) Convolutional LayerThe function of the convolutional layer is to extract features of the original image. There may be many convolution kernels in the same layer, each of which extracts a feature. The difference between convolution and full join is that convolution uses local joins and feature extraction in order to reduce computation. The local connection is the image area of the convolution kernel size only after the input image is multiplied, and the convolution kernel image is smoothly moved to obtain the feature map, so that the pixel that originally needs to connect the entire image is reduced to a convolution kernel size. Although the operation is reduced after the partial connection, the parameters are still many and each convolution kernel slip has to learn different parameters. Reducing the amount of computation by using the weight parameters of the same convolution kernel to be equal is called weight sharing. The weight sharing is mainly based on the information learned from a local area, applied to other positions of the image, that is, the same convolution kernel is used to convolve the entire image, that is, a full filtering. The local statistical properties of the image are repetitive (position independent) across the image. If there is every basic graphic in the image, the graphic may appear anywhere, and sharing the same weight at different locations enables the same mode to be detected at different locations of the data. For example, if the feature obtained after convolution in a window is an edge, then, the convolution kernel corresponds to the edge feature extraction method, and then, the convolution kernel can be used to extract edge features of other regions at other locations.In the input layer, some preprocessing is performed on the original data, so that the original data can be more adapted to the neural network for training and feature extraction. The quality of the original dataset varies due to different collection methods. Therefore, it is necessary to perform a series of processing on the collected and labeled raw data.(2) Pooling LayerIn the CNN network, a special convolution operation is usually called pooling after the convolution operation. Through the pooling operation, the input feature dimension can be reduced, the number of training parameters can be reduced, and overfitting can be prevented. The more commonly used pooling methods are average pooling and maximum pooling. The specific operation is to obtain the mean and maximum values of the elements in the pooled area as the output. Pooling can be seen as a fusion of local information, and a combination of multiple operations can be combined to a larger extent to obtain more complete features.(3) Regularization LayerFor convolutional neural networks, network training is a complex process. Due to the migration of internal variables, the changes in the previous training parameters are amplified as the training progresses. The probability distribution of the current feature is inconsistent with the probability distribution of the initial layer feature, and the previous training parameters are not applicable. There is a hypothesis in machine learning and deep learning that the distribution of training data should be the same as the distribution of features. Here, enter the data of the network using batch normalization (BN) as shown in equation (7):
(14)μB⟵1m∑i=1mxi,δB2⟵1m∑i=1mxi−μB2,x^i⟵xi−μBδB2+ε,yi⟵γx^i+β≡BNγ,βxi.The corresponding equation is(15)−div∇u∇u−λu0−u=0.An optimization problem that can be transformed into a function, let the error function be(16)Ex,y=div∇u∇u−λu−u0.Assuming that the final output is an ideal model, we can get(17)ux,y=Nu0x,y,w,ts=exp−∫0sκtdt.From this, we can see(18)∂=1−ts=1−exp−∫0sκtdt.At this time, each individual in the offspring population corresponds to the amount of change in the initial solution(19)ΔEk=fitnessx′−avgfitness.Make the input data of each layer have a mean of 0 and a variance of 1. However, the output limit is 0 and the variance is 1. This will make the generalization ability of the model worse. It is therefore necessary to add the parametersβ and γ that can be learned to scale and translate the data. The values of β and γ are constantly updated during model training. BN is an optimized data model after the model is trained in batches. That is, when the batch input data is guaranteed, the distribution of the generated and confronted data is limited to a controllable range.(4) Activation FunctionWhen the network is running, it activates a certain part of the neurons in the neural network and transmits the activation information back to the next layer of the neural network. The reason why neural networks can solve nonlinear problems is that the activation function adds nonlinear factors to make up for the expressive ability of the linear model and retains the characteristics of the activated neurons through the function to the next layer. Because the mathematical foundation of the neural network is differentiated everywhere, the activation function is chosen to ensure that the data input and output are also differentiable.


### 2.2. Pest Detection

With the successful application of deep learning in the fields of target detection, recognition, and segmentation, many different depth-based learning-based pest detection algorithms have been proposed. Among them, the MTCNN algorithm and the single-step detection algorithm SSH of the cascaded multitasking network are the most the representative algorithm, the former not only works well in detection but also has an outstanding effect on the alignment of pests and diseases; the most prominent feature of the latter is the scale invariance, which occupies less memory and is faster.


#### 2.2.1. MTCNN Algorithm

The MTCNN pest detection algorithm turns the pest detection task into a coarse-to-fine process through the cascade convolutional neural network. The specific process can be divided into three stages: in the first stage, the shallow convolutional neural network quickly generates candidate windows, removing a large number of negative samples; in the second stage, further refining the candidate window through a more complex convolutional neural network is made, using nonmaximum suppression to discard a large number of repeated windows; in the third stage, using a more powerful convolutional neural network further decides the retention of the candidate window and finally completes the detection and outputs the key points of the pest image (the type of pests and diseases, the degree of pests and diseases, and the crops of pests and diseases).(1)
Pnet is a full convolutional network structure. The input of the network is12×12. After three convolutional layers with a convolution kernel size of 3 and a pooling layer, the 1×1 convolution is used to generate classification, border regression, and key points. Pnet classifies the samples and returns the four candidate regions to the frame parameters of the target region in each candidate region. After that, the candidate regions are corrected by four parameters to obtain a more accurate target frame and the nonmaximum suppression filter is selected(2)
Rnet has an inner layer (full connection layer) compared to Pnet. The network input is that we can get more detailed features through the inner layer and eliminate a large number of candidate pest and disease areas that do not meet the requirements. The pest and disease areas were calibrated and finally combined by nonmaximum suppression(3)
Onet’s network input is48×48, and the network structure has a layer of convolution layer more than Rnet, which can get more complicated and accurate image characteristics of pests and diseases, which plays an important role in frame regression and key point return of pests and diseases


#### 2.2.2. Nonmaximum Suppression

Nonmaximum value suppression (NMS) refers to the suppression of elements that are not maximal values. In essence, it is a local search. In target detection, it is generally used for filtering and removing the late target-repeating bounding box.Target detection generates a number of bounding boxes in a region when positioning. Each bounding box has a classification score, and most of them are redundant. NMS is to perform the local maximum search for all detected bounding boxes. Search for the maximum value within the domain to filter out part of the bounding box. The main process is as follows:(1)
First, get the bounding box of the object; the same object may have many bounding boxes and its classification score(2)
Sort the scores of all bounding boxes and select the highest score and its corresponding bounding box(3)
Traverse all the remaining boxes; if the IOU value of the current highest score box is greater than the preset threshold, delete it(4)
Continue to select the box with the highest score in the unprocessed bounding box and repeat the 3rd process until all reserved bounding boxes are found


#### 2.2.3. Difficult Sample Mining

Difficult sample mining is an important strategy to improve the quality of models. It has a wide range of applications in image classification and general target detection. The main idea is to find some difficult positive samples and difficult negative samples in each step and then further training these difficult samples to enhance the performance of the model.In recent years, with the continuous improvement of the performance of target detection frameworks, online difficult sample mining and Focal Loss have been widely used to screen difficult samples. OHEM selects the most difficult and difficult samples in each mini batch to calculate the neural network gradient, while Foal Loss is aimed at focusing more on the difficult and wrongly classified samples, adding a factor to control the standard cross-entropy.(1) OHEMWhen optimizing SVM, some samples in the algorithm that cannot be distinguished due to concentration are removed, samples that cannot be correctly judged are added to the model, and then a new training is started. The OHEM forwards all samples, then sorts the losses generated by each sample, and selects the former part of the sample for back propagation. For example, only 1 : 3 positive and negative samples are taken to calculate the loss and the remaining negative sample weights are all set to zero. A simple sample with a small loss value has a small contribution to network training, while a difficult sample with a large loss value enhances the generalization ability of the network. Unlike Focal Loss, OHEM takes only 1 : 3 positive and negative samples and the remaining negative sample weights are reset to zero, while Focal Loss trains all negative samples, giving the positive and negative samples different weights based on the loss value.(2) Focal LossIn the one-stage-based target detection algorithm, the positive and negative samples can reach 1 : 1000, which leads to the problem that the positive and negative samples are extremely unbalanced and the negative sample dominates the loss function. Although the negative sample has a high accuracy and the loss value is small, a large number of negative samples eventually produce a loss value that is sometimes greater than the loss value generated by the positive sample. Focal Loss first tries to solve the problem of sample imbalance and introduces a weight coefficient in the cross entropy loss function. When the sample is detected as foreground, the cross entropy loss function is multiplied by the weightα. When it is the background, the cross entropy loss function is multiplied. For weight 1−α, the loss function can be changed to
(20)CEpt=−αtlogpt.After solving the problem of sample imbalance, the author also multiplies the1−ptγ by the cross-entropy loss function, which means that if the accuracy of the sample prediction is higher, the smaller the 1−ptγ, the smaller the overall loss value. The higher the accuracy of the sample, the more the loss will be attenuated, and the lower the accuracy of the sample with less attenuation, then the overall loss function will be dominated by the lower accuracy of the sample, then the final loss function is
(21)FLpt=−αt1−ptγlogpt.


## 2.1. Deep Learning

### 2.1.1. Overview of Deep Learning

With the development of big data and general parallel computing units, applications based on deep learning have also developed rapidly. Deep learning is a neural network with multiple hidden layers. Similar to shallow neural networks, deep neural networks can model complex nonlinear systems. Many layers hidden layer networks provide a higher level of abstraction for the model, thus improving the capabilities of the model. The biggest difference between deep learning and other machine learning is how to find features. The extraction process of features is an abstract process. Deep learning abstraction simulates the way human neurons communicate information. In theory, many classification and prediction problems can be solved. Convolutional neural networks are a typical deep neural network. Great success has been achieved in the many categories and recognition tasks of images.


### 2.1.2. Artificial Neural Network

There are many diseases and insect pests of crops. There are external factors, including their own factors, their own plant attributes, external weather factors, and the influence of other plants such as weeds in the growing environment and pests in the natural environment, which will affect the crops. The output and quality have a huge impact.The research of the artificial neural network is largely inspired by the bionics. It consists of a series of simple artificial neurons connected to each other. Each neuron has three parts: input, artificial nerve cells, and output. As shown in Figure1, the input signal of one neuron can come from an externally given initial value or it can be the output of another neuron. Artificial nerve cells integrate these input signals and perform threshold operations. If the integrated stimulus value exceeds a certain threshold, the neuron enters an active state; otherwise, the neuron is in a suppressed state.Figure 1
Schematic diagram of artificial neurons.Just as the brain’s neurons can constantly adjust the connections between neurons to achieve continuous learning and progress. The ANN network can also adapt the network to the training set by adjusting the weight values on the continuous input.(1) PerceptronA perceptron is a simple artificial neuron with only two outputs. Inputx1,x2,⋯,xn into the perceptron in turn, and each input value corresponds to a weight wi, in addition to an offset w0 as shown in equation (1):
(1)net=∑1nwi⋅xi+w0.Ifx0=1, then, equation (1) can be expressed as
(2)net=∑0nwi.xi=w⟶⋅x⟶.The output of the perceptron is as shown in equation (2) after the thresholding process:
(3)Ox⟶=1,∑0nwi⋅xi≥0,−1,other.The sensor works in much the same way as a neuron, and it can classify two types of samples. For the input sample output 1 on one side and the sample output −1 on the other side, the training process is to continuously adjust the weightw1,w2,⋯,wn.(2) Linear UnitsSensors with only 1 and −1 outputs limit the classification of processing. A simple generalization is to cancel the threshold module. One input is a linear unit ofx1,x2,⋯,xn, and its output is a linear combination of n inputs based on weights, plus offset w0. The core task of training a linear unit is to adjust the weight w1,w2,⋯,wn so that the output is as similar as possible to the target output of the training sample.(3) Error CriteriaA metric is defined to measure the training error of the ANN relative to the training example under the current weight vectorw⟶. A common training error criterion is the squared error:
(4)Ew⟶=12∑d∈Dtd−Od2,where D is the trained sample set, td is the target output of the training sample d, and od is the actual output Oxd of the linear unit pair training sample d. It is necessary to calculate the direction of the fastest fall of the error, so the maximum direction of the directional derivative can be obtained by calculating the partial derivative of the E with respect to each component of the vector w⟶, denoted as ∇Ew⟶.
(5)∇Ew⟶=∂E∂w1,∂E∂w2,⋯,∂E∂wn.Since∇Ew⟶ itself is a vector that represents the largest directional derivative, it corresponds to the fastest rising direction of the E and the direction of the fastest falling of the error is only required to require −∇Ew⟶. So, the law of training should be
(6)w⟶⟵w⟶+Δw⟶,Δw⟶=−η∇Ew⟶.η in equation (6) is that the learning rate is a positive super parameter and is the step size of the gradient descent search. w⟶ is the current search point in the solution space, Δw⟶ represents a displacement to the current fastest descent direction, and w⟶⟵w⟶+Δw⟶ means to move a short distance from the current point along the fastest descent direction and update the current position to this position.All neurons are of Boolean type, that is, state 0 means inhibition or 1 means activation, and then, the energy of RBM is definedp as(7)Ev,hθ=−∑n=1naivi−∑j=1nbjhj−∑i=1mviWijhj.The probability distributionPvθ of the visible layer is as follows, where Zθ is the normalization factor:
(8)Pvθ=1Zθ∑he−Ev,hθ,Zθ=∑v∑he−Ev,hθ.The cost (loss) function of the Softmax regression algorithm is(9)Jθ=−1M∑i=1m∑j=1k1yi=jlog∑j=1keθJT.Prove that the current loss function is strictly convex and its local minimum is not unique, so the optimal parameter to be solved is not unique. The proof process is as follows:(10)Pyi=jxi;θ=eθj−ωTxi∑l=1keθj−ωTxi.Usually, add regular items to modify the cost function, as follows:(11)Jθ=−1m∑i=1m∑j=1k1yi=jlog∑l=1keθlT.λ is the weight of the regular term. At this time, the cost function can also be updated with the gradient descent method. The gradient after derivation is
(12)∇θjJθ=−1m∑i=1mxi1yi=j−pyi=j;θ+λθ.Update the parameters with the help of partial derivatives:(13)θj=θj−∂∇θJθ.The parameter update of Softmax is completed through repeated introduction.(4) BackpropagationThe weighting parameters of the neural network need to be able to be adjusted like human nerve cells, so the ability to automatically adjust parameters is required, and the backpropagation algorithm can be used to adjust the error according to the error.(5) Many Layer Artificial Neural NetworksIn a single neuron or linear unit, only linear problems can be solved. However, there are great deficiencies in solving nonlinear problems, but real-life problems are often nonlinear. The cells in the brain and other cells are interconnected, mimicking the cellular connections of the human brain, and the artificial neural network can also form a many-layered neural network in a similar manner. The characteristics of pests and diseases in specific areas are periodicity, seasonality, and repetitiveness. That is, the diseases and insect pests in the same area show different diseases and insect pests in different seasons, multiple diseases and insect pests occur repeatedly in the same season throughout the year, and several fixed diseases and insect pests appear in turn. The emergence of many layers of neural networks can solve nonlinear problems. In the layered feedforward network, as shown in Figure2, generally, there are at least three layers: one input layer, one output layer, and hidden layer, and the hidden layer is generally multiple. Adjacent neurons are interconnected.Figure 2
Schematic diagram of multilayer artificial neurons.


### 2.1.3. Convolutional Neural Network

Before studying the subject of crop disease image recognition, you need to be familiar with relevant basic theoretical knowledge, such as the recognition model used in deep learning, image recognition process, convolutional neural network structure, deep learning framework, experimental dataset, preprocessing and labeling categories, and many more. As the most representative deep learning network model, the convolutional neural network has benefited from the development of computer networks and computing power. The two-dimensional image is stored in the form of a matrix in a computer, and the relative position between each pixel is fixed. The convolution operation of the fixed convolution kernel size can extract the features of the two-dimensional image. It is this feature of two-dimensional images that enables convolutional neural networks to be successfully applied to identify images and target detection. The CNN network is an important network model for deep learning. CNN’s connection method and network structure are very suitable for dealing with image problems, as shown in Figure3 (picture from the network, http://www.baidu.com/).Figure 3
General model of convolutional neural networks.In the convolutional neural network, the convolution kernel represents the local features of the picture. The image is processed by multilayer convolution and pooling to convert the image information into features with higher information. At the same time, it is necessary to constantly adjust the convolution kernel. The parameters enable the convolution kernel to complete the work of a certain image feature. In image recognition or classification tasks, images are extracted by a convolution operation. Then, the size of the feature map is reduced through the pooling layer and the classification is output through the full connection layer and finally through some activation functions. The main process of accurate crop management is shown in Figure4.Figure 4
Accurate management process of crops.(1) Convolutional LayerThe function of the convolutional layer is to extract features of the original image. There may be many convolution kernels in the same layer, each of which extracts a feature. The difference between convolution and full join is that convolution uses local joins and feature extraction in order to reduce computation. The local connection is the image area of the convolution kernel size only after the input image is multiplied, and the convolution kernel image is smoothly moved to obtain the feature map, so that the pixel that originally needs to connect the entire image is reduced to a convolution kernel size. Although the operation is reduced after the partial connection, the parameters are still many and each convolution kernel slip has to learn different parameters. Reducing the amount of computation by using the weight parameters of the same convolution kernel to be equal is called weight sharing. The weight sharing is mainly based on the information learned from a local area, applied to other positions of the image, that is, the same convolution kernel is used to convolve the entire image, that is, a full filtering. The local statistical properties of the image are repetitive (position independent) across the image. If there is every basic graphic in the image, the graphic may appear anywhere, and sharing the same weight at different locations enables the same mode to be detected at different locations of the data. For example, if the feature obtained after convolution in a window is an edge, then, the convolution kernel corresponds to the edge feature extraction method, and then, the convolution kernel can be used to extract edge features of other regions at other locations.In the input layer, some preprocessing is performed on the original data, so that the original data can be more adapted to the neural network for training and feature extraction. The quality of the original dataset varies due to different collection methods. Therefore, it is necessary to perform a series of processing on the collected and labeled raw data.(2) Pooling LayerIn the CNN network, a special convolution operation is usually called pooling after the convolution operation. Through the pooling operation, the input feature dimension can be reduced, the number of training parameters can be reduced, and overfitting can be prevented. The more commonly used pooling methods are average pooling and maximum pooling. The specific operation is to obtain the mean and maximum values of the elements in the pooled area as the output. Pooling can be seen as a fusion of local information, and a combination of multiple operations can be combined to a larger extent to obtain more complete features.(3) Regularization LayerFor convolutional neural networks, network training is a complex process. Due to the migration of internal variables, the changes in the previous training parameters are amplified as the training progresses. The probability distribution of the current feature is inconsistent with the probability distribution of the initial layer feature, and the previous training parameters are not applicable. There is a hypothesis in machine learning and deep learning that the distribution of training data should be the same as the distribution of features. Here, enter the data of the network using batch normalization (BN) as shown in equation (7):
(14)μB⟵1m∑i=1mxi,δB2⟵1m∑i=1mxi−μB2,x^i⟵xi−μBδB2+ε,yi⟵γx^i+β≡BNγ,βxi.The corresponding equation is(15)−div∇u∇u−λu0−u=0.An optimization problem that can be transformed into a function, let the error function be(16)Ex,y=div∇u∇u−λu−u0.Assuming that the final output is an ideal model, we can get(17)ux,y=Nu0x,y,w,ts=exp−∫0sκtdt.From this, we can see(18)∂=1−ts=1−exp−∫0sκtdt.At this time, each individual in the offspring population corresponds to the amount of change in the initial solution(19)ΔEk=fitnessx′−avgfitness.Make the input data of each layer have a mean of 0 and a variance of 1. However, the output limit is 0 and the variance is 1. This will make the generalization ability of the model worse. It is therefore necessary to add the parametersβ and γ that can be learned to scale and translate the data. The values of β and γ are constantly updated during model training. BN is an optimized data model after the model is trained in batches. That is, when the batch input data is guaranteed, the distribution of the generated and confronted data is limited to a controllable range.(4) Activation FunctionWhen the network is running, it activates a certain part of the neurons in the neural network and transmits the activation information back to the next layer of the neural network. The reason why neural networks can solve nonlinear problems is that the activation function adds nonlinear factors to make up for the expressive ability of the linear model and retains the characteristics of the activated neurons through the function to the next layer. Because the mathematical foundation of the neural network is differentiated everywhere, the activation function is chosen to ensure that the data input and output are also differentiable.


## 2.1.1. Overview of Deep Learning

With the development of big data and general parallel computing units, applications based on deep learning have also developed rapidly. Deep learning is a neural network with multiple hidden layers. Similar to shallow neural networks, deep neural networks can model complex nonlinear systems. Many layers hidden layer networks provide a higher level of abstraction for the model, thus improving the capabilities of the model. The biggest difference between deep learning and other machine learning is how to find features. The extraction process of features is an abstract process. Deep learning abstraction simulates the way human neurons communicate information. In theory, many classification and prediction problems can be solved. Convolutional neural networks are a typical deep neural network. Great success has been achieved in the many categories and recognition tasks of images.


## 2.1.2. Artificial Neural Network

There are many diseases and insect pests of crops. There are external factors, including their own factors, their own plant attributes, external weather factors, and the influence of other plants such as weeds in the growing environment and pests in the natural environment, which will affect the crops. The output and quality have a huge impact.The research of the artificial neural network is largely inspired by the bionics. It consists of a series of simple artificial neurons connected to each other. Each neuron has three parts: input, artificial nerve cells, and output. As shown in Figure1, the input signal of one neuron can come from an externally given initial value or it can be the output of another neuron. Artificial nerve cells integrate these input signals and perform threshold operations. If the integrated stimulus value exceeds a certain threshold, the neuron enters an active state; otherwise, the neuron is in a suppressed state.Figure 1
Schematic diagram of artificial neurons.Just as the brain’s neurons can constantly adjust the connections between neurons to achieve continuous learning and progress. The ANN network can also adapt the network to the training set by adjusting the weight values on the continuous input.(1) PerceptronA perceptron is a simple artificial neuron with only two outputs. Inputx1,x2,⋯,xn into the perceptron in turn, and each input value corresponds to a weight wi, in addition to an offset w0 as shown in equation (1):
(1)net=∑1nwi⋅xi+w0.Ifx0=1, then, equation (1) can be expressed as
(2)net=∑0nwi.xi=w⟶⋅x⟶.The output of the perceptron is as shown in equation (2) after the thresholding process:
(3)Ox⟶=1,∑0nwi⋅xi≥0,−1,other.The sensor works in much the same way as a neuron, and it can classify two types of samples. For the input sample output 1 on one side and the sample output −1 on the other side, the training process is to continuously adjust the weightw1,w2,⋯,wn.(2) Linear UnitsSensors with only 1 and −1 outputs limit the classification of processing. A simple generalization is to cancel the threshold module. One input is a linear unit ofx1,x2,⋯,xn, and its output is a linear combination of n inputs based on weights, plus offset w0. The core task of training a linear unit is to adjust the weight w1,w2,⋯,wn so that the output is as similar as possible to the target output of the training sample.(3) Error CriteriaA metric is defined to measure the training error of the ANN relative to the training example under the current weight vectorw⟶. A common training error criterion is the squared error:
(4)Ew⟶=12∑d∈Dtd−Od2,where D is the trained sample set, td is the target output of the training sample d, and od is the actual output Oxd of the linear unit pair training sample d. It is necessary to calculate the direction of the fastest fall of the error, so the maximum direction of the directional derivative can be obtained by calculating the partial derivative of the E with respect to each component of the vector w⟶, denoted as ∇Ew⟶.
(5)∇Ew⟶=∂E∂w1,∂E∂w2,⋯,∂E∂wn.Since∇Ew⟶ itself is a vector that represents the largest directional derivative, it corresponds to the fastest rising direction of the E and the direction of the fastest falling of the error is only required to require −∇Ew⟶. So, the law of training should be
(6)w⟶⟵w⟶+Δw⟶,Δw⟶=−η∇Ew⟶.η in equation (6) is that the learning rate is a positive super parameter and is the step size of the gradient descent search. w⟶ is the current search point in the solution space, Δw⟶ represents a displacement to the current fastest descent direction, and w⟶⟵w⟶+Δw⟶ means to move a short distance from the current point along the fastest descent direction and update the current position to this position.All neurons are of Boolean type, that is, state 0 means inhibition or 1 means activation, and then, the energy of RBM is definedp as(7)Ev,hθ=−∑n=1naivi−∑j=1nbjhj−∑i=1mviWijhj.The probability distributionPvθ of the visible layer is as follows, where Zθ is the normalization factor:
(8)Pvθ=1Zθ∑he−Ev,hθ,Zθ=∑v∑he−Ev,hθ.The cost (loss) function of the Softmax regression algorithm is(9)Jθ=−1M∑i=1m∑j=1k1yi=jlog∑j=1keθJT.Prove that the current loss function is strictly convex and its local minimum is not unique, so the optimal parameter to be solved is not unique. The proof process is as follows:(10)Pyi=jxi;θ=eθj−ωTxi∑l=1keθj−ωTxi.Usually, add regular items to modify the cost function, as follows:(11)Jθ=−1m∑i=1m∑j=1k1yi=jlog∑l=1keθlT.λ is the weight of the regular term. At this time, the cost function can also be updated with the gradient descent method. The gradient after derivation is
(12)∇θjJθ=−1m∑i=1mxi1yi=j−pyi=j;θ+λθ.Update the parameters with the help of partial derivatives:(13)θj=θj−∂∇θJθ.The parameter update of Softmax is completed through repeated introduction.(4) BackpropagationThe weighting parameters of the neural network need to be able to be adjusted like human nerve cells, so the ability to automatically adjust parameters is required, and the backpropagation algorithm can be used to adjust the error according to the error.(5) Many Layer Artificial Neural NetworksIn a single neuron or linear unit, only linear problems can be solved. However, there are great deficiencies in solving nonlinear problems, but real-life problems are often nonlinear. The cells in the brain and other cells are interconnected, mimicking the cellular connections of the human brain, and the artificial neural network can also form a many-layered neural network in a similar manner. The characteristics of pests and diseases in specific areas are periodicity, seasonality, and repetitiveness. That is, the diseases and insect pests in the same area show different diseases and insect pests in different seasons, multiple diseases and insect pests occur repeatedly in the same season throughout the year, and several fixed diseases and insect pests appear in turn. The emergence of many layers of neural networks can solve nonlinear problems. In the layered feedforward network, as shown in Figure2, generally, there are at least three layers: one input layer, one output layer, and hidden layer, and the hidden layer is generally multiple. Adjacent neurons are interconnected.Figure 2
Schematic diagram of multilayer artificial neurons.


## 2.1.3. Convolutional Neural Network

Before studying the subject of crop disease image recognition, you need to be familiar with relevant basic theoretical knowledge, such as the recognition model used in deep learning, image recognition process, convolutional neural network structure, deep learning framework, experimental dataset, preprocessing and labeling categories, and many more. As the most representative deep learning network model, the convolutional neural network has benefited from the development of computer networks and computing power. The two-dimensional image is stored in the form of a matrix in a computer, and the relative position between each pixel is fixed. The convolution operation of the fixed convolution kernel size can extract the features of the two-dimensional image. It is this feature of two-dimensional images that enables convolutional neural networks to be successfully applied to identify images and target detection. The CNN network is an important network model for deep learning. CNN’s connection method and network structure are very suitable for dealing with image problems, as shown in Figure3 (picture from the network, http://www.baidu.com/).Figure 3
General model of convolutional neural networks.In the convolutional neural network, the convolution kernel represents the local features of the picture. The image is processed by multilayer convolution and pooling to convert the image information into features with higher information. At the same time, it is necessary to constantly adjust the convolution kernel. The parameters enable the convolution kernel to complete the work of a certain image feature. In image recognition or classification tasks, images are extracted by a convolution operation. Then, the size of the feature map is reduced through the pooling layer and the classification is output through the full connection layer and finally through some activation functions. The main process of accurate crop management is shown in Figure4.Figure 4
Accurate management process of crops.(1) Convolutional LayerThe function of the convolutional layer is to extract features of the original image. There may be many convolution kernels in the same layer, each of which extracts a feature. The difference between convolution and full join is that convolution uses local joins and feature extraction in order to reduce computation. The local connection is the image area of the convolution kernel size only after the input image is multiplied, and the convolution kernel image is smoothly moved to obtain the feature map, so that the pixel that originally needs to connect the entire image is reduced to a convolution kernel size. Although the operation is reduced after the partial connection, the parameters are still many and each convolution kernel slip has to learn different parameters. Reducing the amount of computation by using the weight parameters of the same convolution kernel to be equal is called weight sharing. The weight sharing is mainly based on the information learned from a local area, applied to other positions of the image, that is, the same convolution kernel is used to convolve the entire image, that is, a full filtering. The local statistical properties of the image are repetitive (position independent) across the image. If there is every basic graphic in the image, the graphic may appear anywhere, and sharing the same weight at different locations enables the same mode to be detected at different locations of the data. For example, if the feature obtained after convolution in a window is an edge, then, the convolution kernel corresponds to the edge feature extraction method, and then, the convolution kernel can be used to extract edge features of other regions at other locations.In the input layer, some preprocessing is performed on the original data, so that the original data can be more adapted to the neural network for training and feature extraction. The quality of the original dataset varies due to different collection methods. Therefore, it is necessary to perform a series of processing on the collected and labeled raw data.(2) Pooling LayerIn the CNN network, a special convolution operation is usually called pooling after the convolution operation. Through the pooling operation, the input feature dimension can be reduced, the number of training parameters can be reduced, and overfitting can be prevented. The more commonly used pooling methods are average pooling and maximum pooling. The specific operation is to obtain the mean and maximum values of the elements in the pooled area as the output. Pooling can be seen as a fusion of local information, and a combination of multiple operations can be combined to a larger extent to obtain more complete features.(3) Regularization LayerFor convolutional neural networks, network training is a complex process. Due to the migration of internal variables, the changes in the previous training parameters are amplified as the training progresses. The probability distribution of the current feature is inconsistent with the probability distribution of the initial layer feature, and the previous training parameters are not applicable. There is a hypothesis in machine learning and deep learning that the distribution of training data should be the same as the distribution of features. Here, enter the data of the network using batch normalization (BN) as shown in equation (7):
(14)μB⟵1m∑i=1mxi,δB2⟵1m∑i=1mxi−μB2,x^i⟵xi−μBδB2+ε,yi⟵γx^i+β≡BNγ,βxi.The corresponding equation is(15)−div∇u∇u−λu0−u=0.An optimization problem that can be transformed into a function, let the error function be(16)Ex,y=div∇u∇u−λu−u0.Assuming that the final output is an ideal model, we can get(17)ux,y=Nu0x,y,w,ts=exp−∫0sκtdt.From this, we can see(18)∂=1−ts=1−exp−∫0sκtdt.At this time, each individual in the offspring population corresponds to the amount of change in the initial solution(19)ΔEk=fitnessx′−avgfitness.Make the input data of each layer have a mean of 0 and a variance of 1. However, the output limit is 0 and the variance is 1. This will make the generalization ability of the model worse. It is therefore necessary to add the parametersβ and γ that can be learned to scale and translate the data. The values of β and γ are constantly updated during model training. BN is an optimized data model after the model is trained in batches. That is, when the batch input data is guaranteed, the distribution of the generated and confronted data is limited to a controllable range.(4) Activation FunctionWhen the network is running, it activates a certain part of the neurons in the neural network and transmits the activation information back to the next layer of the neural network. The reason why neural networks can solve nonlinear problems is that the activation function adds nonlinear factors to make up for the expressive ability of the linear model and retains the characteristics of the activated neurons through the function to the next layer. Because the mathematical foundation of the neural network is differentiated everywhere, the activation function is chosen to ensure that the data input and output are also differentiable.


## 2.2. Pest Detection

With the successful application of deep learning in the fields of target detection, recognition, and segmentation, many different depth-based learning-based pest detection algorithms have been proposed. Among them, the MTCNN algorithm and the single-step detection algorithm SSH of the cascaded multitasking network are the most the representative algorithm, the former not only works well in detection but also has an outstanding effect on the alignment of pests and diseases; the most prominent feature of the latter is the scale invariance, which occupies less memory and is faster.


### 2.2.1. MTCNN Algorithm

The MTCNN pest detection algorithm turns the pest detection task into a coarse-to-fine process through the cascade convolutional neural network. The specific process can be divided into three stages: in the first stage, the shallow convolutional neural network quickly generates candidate windows, removing a large number of negative samples; in the second stage, further refining the candidate window through a more complex convolutional neural network is made, using nonmaximum suppression to discard a large number of repeated windows; in the third stage, using a more powerful convolutional neural network further decides the retention of the candidate window and finally completes the detection and outputs the key points of the pest image (the type of pests and diseases, the degree of pests and diseases, and the crops of pests and diseases).(1)
Pnet is a full convolutional network structure. The input of the network is12×12. After three convolutional layers with a convolution kernel size of 3 and a pooling layer, the 1×1 convolution is used to generate classification, border regression, and key points. Pnet classifies the samples and returns the four candidate regions to the frame parameters of the target region in each candidate region. After that, the candidate regions are corrected by four parameters to obtain a more accurate target frame and the nonmaximum suppression filter is selected(2)
Rnet has an inner layer (full connection layer) compared to Pnet. The network input is that we can get more detailed features through the inner layer and eliminate a large number of candidate pest and disease areas that do not meet the requirements. The pest and disease areas were calibrated and finally combined by nonmaximum suppression(3)
Onet’s network input is48×48, and the network structure has a layer of convolution layer more than Rnet, which can get more complicated and accurate image characteristics of pests and diseases, which plays an important role in frame regression and key point return of pests and diseases


### 2.2.2. Nonmaximum Suppression

Nonmaximum value suppression (NMS) refers to the suppression of elements that are not maximal values. In essence, it is a local search. In target detection, it is generally used for filtering and removing the late target-repeating bounding box.Target detection generates a number of bounding boxes in a region when positioning. Each bounding box has a classification score, and most of them are redundant. NMS is to perform the local maximum search for all detected bounding boxes. Search for the maximum value within the domain to filter out part of the bounding box. The main process is as follows:(1)
First, get the bounding box of the object; the same object may have many bounding boxes and its classification score(2)
Sort the scores of all bounding boxes and select the highest score and its corresponding bounding box(3)
Traverse all the remaining boxes; if the IOU value of the current highest score box is greater than the preset threshold, delete it(4)
Continue to select the box with the highest score in the unprocessed bounding box and repeat the 3rd process until all reserved bounding boxes are found


### 2.2.3. Difficult Sample Mining

Difficult sample mining is an important strategy to improve the quality of models. It has a wide range of applications in image classification and general target detection. The main idea is to find some difficult positive samples and difficult negative samples in each step and then further training these difficult samples to enhance the performance of the model.In recent years, with the continuous improvement of the performance of target detection frameworks, online difficult sample mining and Focal Loss have been widely used to screen difficult samples. OHEM selects the most difficult and difficult samples in each mini batch to calculate the neural network gradient, while Foal Loss is aimed at focusing more on the difficult and wrongly classified samples, adding a factor to control the standard cross-entropy.(1) OHEMWhen optimizing SVM, some samples in the algorithm that cannot be distinguished due to concentration are removed, samples that cannot be correctly judged are added to the model, and then a new training is started. The OHEM forwards all samples, then sorts the losses generated by each sample, and selects the former part of the sample for back propagation. For example, only 1 : 3 positive and negative samples are taken to calculate the loss and the remaining negative sample weights are all set to zero. A simple sample with a small loss value has a small contribution to network training, while a difficult sample with a large loss value enhances the generalization ability of the network. Unlike Focal Loss, OHEM takes only 1 : 3 positive and negative samples and the remaining negative sample weights are reset to zero, while Focal Loss trains all negative samples, giving the positive and negative samples different weights based on the loss value.(2) Focal LossIn the one-stage-based target detection algorithm, the positive and negative samples can reach 1 : 1000, which leads to the problem that the positive and negative samples are extremely unbalanced and the negative sample dominates the loss function. Although the negative sample has a high accuracy and the loss value is small, a large number of negative samples eventually produce a loss value that is sometimes greater than the loss value generated by the positive sample. Focal Loss first tries to solve the problem of sample imbalance and introduces a weight coefficient in the cross entropy loss function. When the sample is detected as foreground, the cross entropy loss function is multiplied by the weightα. When it is the background, the cross entropy loss function is multiplied. For weight 1−α, the loss function can be changed to
(20)CEpt=−αtlogpt.After solving the problem of sample imbalance, the author also multiplies the1−ptγ by the cross-entropy loss function, which means that if the accuracy of the sample prediction is higher, the smaller the 1−ptγ, the smaller the overall loss value. The higher the accuracy of the sample, the more the loss will be attenuated, and the lower the accuracy of the sample with less attenuation, then the overall loss function will be dominated by the lower accuracy of the sample, then the final loss function is
(21)FLpt=−αt1−ptγlogpt.


## 2.2.1. MTCNN Algorithm

The MTCNN pest detection algorithm turns the pest detection task into a coarse-to-fine process through the cascade convolutional neural network. The specific process can be divided into three stages: in the first stage, the shallow convolutional neural network quickly generates candidate windows, removing a large number of negative samples; in the second stage, further refining the candidate window through a more complex convolutional neural network is made, using nonmaximum suppression to discard a large number of repeated windows; in the third stage, using a more powerful convolutional neural network further decides the retention of the candidate window and finally completes the detection and outputs the key points of the pest image (the type of pests and diseases, the degree of pests and diseases, and the crops of pests and diseases).(1)
Pnet is a full convolutional network structure. The input of the network is12×12. After three convolutional layers with a convolution kernel size of 3 and a pooling layer, the 1×1 convolution is used to generate classification, border regression, and key points. Pnet classifies the samples and returns the four candidate regions to the frame parameters of the target region in each candidate region. After that, the candidate regions are corrected by four parameters to obtain a more accurate target frame and the nonmaximum suppression filter is selected(2)
Rnet has an inner layer (full connection layer) compared to Pnet. The network input is that we can get more detailed features through the inner layer and eliminate a large number of candidate pest and disease areas that do not meet the requirements. The pest and disease areas were calibrated and finally combined by nonmaximum suppression(3)
Onet’s network input is48×48, and the network structure has a layer of convolution layer more than Rnet, which can get more complicated and accurate image characteristics of pests and diseases, which plays an important role in frame regression and key point return of pests and diseases


## 2.2.2. Nonmaximum Suppression

Nonmaximum value suppression (NMS) refers to the suppression of elements that are not maximal values. In essence, it is a local search. In target detection, it is generally used for filtering and removing the late target-repeating bounding box.Target detection generates a number of bounding boxes in a region when positioning. Each bounding box has a classification score, and most of them are redundant. NMS is to perform the local maximum search for all detected bounding boxes. Search for the maximum value within the domain to filter out part of the bounding box. The main process is as follows:(1)
First, get the bounding box of the object; the same object may have many bounding boxes and its classification score(2)
Sort the scores of all bounding boxes and select the highest score and its corresponding bounding box(3)
Traverse all the remaining boxes; if the IOU value of the current highest score box is greater than the preset threshold, delete it(4)
Continue to select the box with the highest score in the unprocessed bounding box and repeat the 3rd process until all reserved bounding boxes are found


## 2.2.3. Difficult Sample Mining

Difficult sample mining is an important strategy to improve the quality of models. It has a wide range of applications in image classification and general target detection. The main idea is to find some difficult positive samples and difficult negative samples in each step and then further training these difficult samples to enhance the performance of the model.In recent years, with the continuous improvement of the performance of target detection frameworks, online difficult sample mining and Focal Loss have been widely used to screen difficult samples. OHEM selects the most difficult and difficult samples in each mini batch to calculate the neural network gradient, while Foal Loss is aimed at focusing more on the difficult and wrongly classified samples, adding a factor to control the standard cross-entropy.(1) OHEMWhen optimizing SVM, some samples in the algorithm that cannot be distinguished due to concentration are removed, samples that cannot be correctly judged are added to the model, and then a new training is started. The OHEM forwards all samples, then sorts the losses generated by each sample, and selects the former part of the sample for back propagation. For example, only 1 : 3 positive and negative samples are taken to calculate the loss and the remaining negative sample weights are all set to zero. A simple sample with a small loss value has a small contribution to network training, while a difficult sample with a large loss value enhances the generalization ability of the network. Unlike Focal Loss, OHEM takes only 1 : 3 positive and negative samples and the remaining negative sample weights are reset to zero, while Focal Loss trains all negative samples, giving the positive and negative samples different weights based on the loss value.(2) Focal LossIn the one-stage-based target detection algorithm, the positive and negative samples can reach 1 : 1000, which leads to the problem that the positive and negative samples are extremely unbalanced and the negative sample dominates the loss function. Although the negative sample has a high accuracy and the loss value is small, a large number of negative samples eventually produce a loss value that is sometimes greater than the loss value generated by the positive sample. Focal Loss first tries to solve the problem of sample imbalance and introduces a weight coefficient in the cross entropy loss function. When the sample is detected as foreground, the cross entropy loss function is multiplied by the weightα. When it is the background, the cross entropy loss function is multiplied. For weight 1−α, the loss function can be changed to
(20)CEpt=−αtlogpt.After solving the problem of sample imbalance, the author also multiplies the1−ptγ by the cross-entropy loss function, which means that if the accuracy of the sample prediction is higher, the smaller the 1−ptγ, the smaller the overall loss value. The higher the accuracy of the sample, the more the loss will be attenuated, and the lower the accuracy of the sample with less attenuation, then the overall loss function will be dominated by the lower accuracy of the sample, then the final loss function is
(21)FLpt=−αt1−ptγlogpt.


## 3. Experiments

### 3.1. Experimental Platform

At present, convolutional neural networks are all three-dimensional input data, which reflects two-dimensional pixels and RGB channels on a flat image. The input is information such as image pixels in matrix form. The length and width of the image and the depth of the color channel constitute a three-dimensional matrix. The depth of the black and white image is 1, and the depth of the color image is 3.Due to the batch processing of image data, a large amount of memory is required. The computation of deep learning relies on a generic GPU acceleration module. Therefore, the experimental hardware platform used in this paper is the DGX-1 deep learning server of NVIDIA. The experimental platform configuration is shown in Table1.Table 1
Experimental platform configuration.
CPUIntel XeonE5-2698 v4 2.2 GHz processorMemory256 GHard disk4T SSD solid stateGPUTesla P4 256bit∗ 8Operating systemUbuntu16.04Deep learning frameworkTensorFlow1.8.0-gpuDeep learning accelerationCUDA 9.0, CUDNN 7.14Programming languagePython 3.6


### 3.2. Training Sample Generation

Considering that the depth of the multilayer network in deep learning can easily lead to the increase in training complexity and the disappearance of gradients; the ResNet convolutional neural network model is used to train the crop disease dataset. The residual learning unit in the ResNet network can simplify the convolutional layer. In the learning process, only the difference between input and output is learned, deepening the network layer and accelerating model training.For the training samples, an offset value can be randomly taken at the real pest area and the candidate pest and disease window can be randomly selected in the original map by the offset value and the real pest area, and then, the candidate pest and insect window can be obtained and the size of the IOU through the real pest area is determined. Negative samples, suspect samples, and positive samples can be generated. The IOU is specified to be greater than 0.7 as a positive sample. If the maximum IOU value is less than 0.7, the highest threshold is considered as a positive sample, the IOU is less than 0.3 as a negative sample, and between 0.3 and 0.7 is a suspect sample. The interpolation is scaled to the specified scale, and the IOU is the ratio of the real sample to the candidate window area, as shown in Figure5 (picture from the network, http://www.baidu.com/):Figure 5
IOU schematic.The calculation formula is(22)IOU=areaC∩areaGareaC∪areaG.The ratio of positive samples, suspect samples, and negative samples is about 1 : 1.5 : 3. The negative samples are only responsible for detecting whether there are pests and diseases, which is a two-class problem. The suspected samples and positive samples are subject to the pest return frame and the key points of pest and disease. In order to enhance the robustness of the network, the samples obtained by random cropping can be sent to the training network to obtain an initial network model to further obtain training samples. After training Pnet, the original image can be input into the Pnet training model and the detected pest area and the real pest area IOU can be obtained, and finally, the Rnet training sample is generated. As described above, when the IOU is less than 0.3, it is regarded as a negative sample. An IOU greater than 0.7 is considered a positive sample and an IOU is considered a suspect sample between 0.3 and 0.7. Repeat this step to send the original image to the trained Rnet model to generate the training samples of Onet, so that the samples sent to Rnet and Onet have both samples generated by offset values and samples predicted by the network, which can enhance the generalization of the model.


## 3.1. Experimental Platform

At present, convolutional neural networks are all three-dimensional input data, which reflects two-dimensional pixels and RGB channels on a flat image. The input is information such as image pixels in matrix form. The length and width of the image and the depth of the color channel constitute a three-dimensional matrix. The depth of the black and white image is 1, and the depth of the color image is 3.Due to the batch processing of image data, a large amount of memory is required. The computation of deep learning relies on a generic GPU acceleration module. Therefore, the experimental hardware platform used in this paper is the DGX-1 deep learning server of NVIDIA. The experimental platform configuration is shown in Table1.Table 1
Experimental platform configuration.
CPUIntel XeonE5-2698 v4 2.2 GHz processorMemory256 GHard disk4T SSD solid stateGPUTesla P4 256bit∗ 8Operating systemUbuntu16.04Deep learning frameworkTensorFlow1.8.0-gpuDeep learning accelerationCUDA 9.0, CUDNN 7.14Programming languagePython 3.6


## 3.2. Training Sample Generation

Considering that the depth of the multilayer network in deep learning can easily lead to the increase in training complexity and the disappearance of gradients; the ResNet convolutional neural network model is used to train the crop disease dataset. The residual learning unit in the ResNet network can simplify the convolutional layer. In the learning process, only the difference between input and output is learned, deepening the network layer and accelerating model training.For the training samples, an offset value can be randomly taken at the real pest area and the candidate pest and disease window can be randomly selected in the original map by the offset value and the real pest area, and then, the candidate pest and insect window can be obtained and the size of the IOU through the real pest area is determined. Negative samples, suspect samples, and positive samples can be generated. The IOU is specified to be greater than 0.7 as a positive sample. If the maximum IOU value is less than 0.7, the highest threshold is considered as a positive sample, the IOU is less than 0.3 as a negative sample, and between 0.3 and 0.7 is a suspect sample. The interpolation is scaled to the specified scale, and the IOU is the ratio of the real sample to the candidate window area, as shown in Figure5 (picture from the network, http://www.baidu.com/):Figure 5
IOU schematic.The calculation formula is(22)IOU=areaC∩areaGareaC∪areaG.The ratio of positive samples, suspect samples, and negative samples is about 1 : 1.5 : 3. The negative samples are only responsible for detecting whether there are pests and diseases, which is a two-class problem. The suspected samples and positive samples are subject to the pest return frame and the key points of pest and disease. In order to enhance the robustness of the network, the samples obtained by random cropping can be sent to the training network to obtain an initial network model to further obtain training samples. After training Pnet, the original image can be input into the Pnet training model and the detected pest area and the real pest area IOU can be obtained, and finally, the Rnet training sample is generated. As described above, when the IOU is less than 0.3, it is regarded as a negative sample. An IOU greater than 0.7 is considered a positive sample and an IOU is considered a suspect sample between 0.3 and 0.7. Repeat this step to send the original image to the trained Rnet model to generate the training samples of Onet, so that the samples sent to Rnet and Onet have both samples generated by offset values and samples predicted by the network, which can enhance the generalization of the model.


## 4. Discussion

### 4.1. Pest Image Recognition

#### 4.1.1. Pretreatment of Crop Disease Images

The training process of using the ResNet network to recognize crop disease images is initializing the weight parameters of the network model, the convolutional layer in the ResNet network extracts the characteristics of the training dataset image, the characteristics are pooled to obtain the output, and the output value and the difference between the target values use the error to update the weight parameters of the network to obtain a trained crop disease image recognition model.The purpose of preprocessing is to remove the noise and redundant information in the sample picture, which can simplify the image and improve the recognition rate of valuable information, thereby improving the performance of model identification. And the use of deep learning method for classification and identification requires a high amount of data. This paper only collects about 5,000 images, so the data enhancement process needs to be done later. There may be content in the image that has nothing to do with the target recognition. They not only are useful for extracting image features but also sometimes may reduce the effect of feature extraction. Therefore, the image should be extracted before image recognition. In this paper, ACDSee9.0 software is used to manually cut the picture. For the disease image, the cutting leaves a leaf and the main cutting leaves the diseased part. The cutting effect is shown in Figure6.Figure 6
Crop disease image after cropping.
(a)
Soy(b)
Fruit tree(c)
Rice(d)
Cucumber


#### 4.1.2. Pathological Leaf Lesion Extraction

By multiplying the background part and the preprocessed image, the green leaf part can be removed and the leaf lesions can be extracted and the treatment effect is shown in Figure7. By processing the image of the diseased leaf collected, a clear image of the lesion is obtained, but the color, shape, and texture characteristic parameters of the lesion image need to be separately extracted to identify the disease.Figure 7
Extraction of diseased leaf lesions.
(a)
Preprocessed image(b)
Color lesion image


### 4.2. Simulation Experiment of Disease Image Fusion

Under the complex background of the field, in the same scene, the collection equipment shoots diseased leaves under different focal lengths and different lighting conditions. Some images are clear and blurred in the foreground, and some images are blurred in the foreground and clear. The disease image fusion method based on wavelet transform is adopted. The disease image is decomposed by a wavelet, and the schematic diagram of wavelet decomposition is shown in Figure8.Figure 8
Schematic diagram of wavelet decomposition of the disease image.After that, the disease image fusion is performed on the two fuzzy disease images of wolfberry leaves and the result of disease fusion is shown in Figure9.Figure 9
Disease image fusion result.It can be seen from the figure above that there are fuzzy parts in the first two disease images. After the disease images are fused, the fused image obtained is clearer. Based on the wireless network communication and deep learning image fusion method, the coefficient with a large absolute value is selected as the wavelet coefficient to retain the details, which not only corresponds to the significant brightness change in the image, but also retains the salient features in the image, making the disease image clearer.For the diseased images ofLycium barbarum leaves collected under the complex background of the field, the Sobel operator, Robert operator, Prewitt operator, Cannon operator, Gauss-Laplacian operator, Canny operator, and phase-based edge detection method were used to detect the diseased leaves. For edge detection, the detection result is shown in Figure 10.Figure 10
Disease image monitoring results.
(a)
Original disease image(b)
The edge detection result of the image by the Sobel operator(c)
Robert operator performs edge detection results on the image(d)
Prewitt operator performs edge detection results on the image(e)
The edge detection result of the image by the Cannon operator(f)
Gauss-Laplacian operator performs edge detection results on the image(g)
Canny operator performs edge detection results on the image(h)
Phase-based image edge detection resultsFor fuzzy disease images of the same diseased leaf, six objective evaluation methods, including contrast, sharpness, average fuzzy degree, fuzzy degree, color aggregation vector, and hue count, are used to evaluate the image quality. The value of the color aggregation vector is the histogram of the aggregation pixel. For graph feature values, perform quality evaluation on one of the blurred images and the evaluation results are shown in Table2.Table 2
Image quality evaluation results of blurred diseases.
DiseaseContrastClarityAverage ambiguityAmbiguityColor aggregation vectorHuePowdery mildew59.152752.19350.19970.68970.573657.9287Grey spot67.027260.98130.21580.89711.463859.7571Gall mite disease59.267244.73160.14020.29830.499152.3104Anthrax51.951440.58790.10210.47910.294741.7839The objective quality evaluation of the regional image after fusion of the diseased image is carried out by six indicators such as contrast, sharpness, average fuzzy degree, fuzzy degree, color aggregation vector, and hue count. The evaluation results are shown in Table3.Table 3
Image quality after image fusion.
DiseaseContrastClarityAverage ambiguityAmbiguityColor aggregation vectorHuePowdery mildew61.957963.91590.38940.99780.690759.9135Grey spot71.910865.87960.57091.19871.509261.9083Gall mite disease68.439164.09570.27940.39970.591362.0304Anthrax55.794255.08760.39350.67920.679247.7189In order to prove that the segmentation method based on wireless network communication and deep learning can effectively segment disease images under complex background, this paper compares it with traditional segmentation algorithms. When the segmentation results are basically the same, the segmentation algorithm based on wireless network communication and deep learning has obvious advantages. The comparison between the number of iterations and the split time is shown in Table4.Table 4
Comparison of the number of iterations and split time.
Deep learningTraditional algorithmNumber of iterations (times)Split time (seconds)Number of iterations (times)Split time (seconds)Standard image30014.012609.98Disease image25011.642208.09When the segmentation results are basically the same, the number of iterations of the segmentation experiment on the standard image set is reduced by about 13% and the segmentation time is reduced by about 27%; the number of iterations of the segmentation experiment on the diseased image set is reduced by about 12%, and the segmentation time is reduced by about 30%, and as the number of iterations increases, the time for algorithm segmentation also continues to increase.Aiming at five different disease types (no disease, gray spot, powdery mildew, gall mites, and anthracnose) of wolfberry disease images in small sample data, the segmentation method based on wireless network communication and deep learning proposed in this paper is compared with traditional segmentation and comparison of fuzzy C-means clustering segmentation algorithms. The average error rate of the method is shown in Table5.Table 5
Comparison of average error rates of different segmentation methods.
Disease typeFuzzy C-means clustering algorithmTraditional algorithmDeep learning algorithmDisease free19.814.74.5Grey spot41.9135.735.27Powdery mildew32.8531.663.08Gall mite disease33.0332.182.27Anthrax27.5926.872.86As can be seen in Table5, the average error rate of the fuzzy C-means clustering segmentation algorithm is large, which will cause a large number of redundant backgrounds; the average error rate of the traditional algorithm segmentation is lower than the average error rate of the fuzzy C-means clustering algorithm. The average error rate of the methods exceeds 10%, and the average error rate of the deep learning algorithm used in this paper is below 6%.In order to verify the segmentation effectiveness of the deep learning algorithm on the fine-grained category dataset, compared with the GrabCut algorithm segmentation method based on the saliency map, the average accuracy of the two segmentation methods is shown in Table6.Table 6
Comparison of the average accuracy of the two segmentation methods.
Disease typeDeep learning algorithmGrabCut algorithm based on saliency mapDisease free94.490.2Grey spot93.5271.27Powdery mildew95.7176.14Gall mite disease96.5275.64Anthrax96.0381.09Finally, in order to prove the segmentation effect based on the deep learning algorithm in the light-affected environment, for the diseased images in the normal field environment and the light-affected environment, the segmentation algorithm proposed in this paper is mainly based on the average error rate and the average recall rate, as shown in Table7.Table 7
Performance evaluation of the segmentation algorithm.
Environment typeAverage error rate (%)Average recall rate (%)Normal field environment0.0890.913Light affects the environment0.1520.856As can be seen in Table7, for the disease images in the normal field environment and the light-affected environment, the average recall rate under normal lighting conditions based on the deep learning algorithm proposed in this paper can reach 90%, while in a strong lighting environment, the following will affect the performance of the segmentation algorithm, so the lighting conditions are an important factor in the performance evaluation of the segmentation algorithm.


### 4.3. Analysis of the Accuracy of Image Recognition of Pests and Diseases

#### 4.3.1. Accuracy of the Test Set under Different Settings of the First Layer of Convolution

The image of crop pests and diseases can be well satisfied with the crop pest identification task after the background is removed at a resolution of about100×100. Therefore, all relevant experiments are to align the image detection of crop pests and diseases to 112×112 and the pixels of the image of three channels. Normalize by subtracting 127.5 and dividing by 128. Like most convolutional neural networks, the experiments in this article are based on the ImageNet dataset, so the network input size is generally large (such as 224×225). The first convolutional layer is generally 7×7, and the size of the convolution kernel in step A is a sample image of length 2, but for crop pest images, greater perception also means smaller feature maps. Therefore, the first layer of convolution kernel is set to 7×7 convolution kernel, the step size is 2, and it is named A. The following is a ResNet-50 network. The first layer of convolutional layers is set to A and B, namely, AResNet-50 and BResNet-50, the loss function is the standard Softmax loss function, and the training dataset is CASIA-Web, in LFW, and the YTF test set was tested, and the experimental results are shown in Table 8 and Figure 11.Table 8
Accuracy of the test set under different settings of the first layer of the network convolution layer.
NETLFWYTFAResNet-50-(4)97.70%92.11%BResNet-50-(4)98.08%92.59%Figure 11
Network first-layer convolution layer different settings in the test set accuracy.We can find that the use of setting B is better than A. The reason may be that the large convolution kernel will make the feature size drop too fast, and the network cannot effectively learn the features with strong expressive ability. Using setting B, the first-layer output feature map size is56×56, and the setting A first-layer output feature map size is 112×112, so that the latter feature image size is twice that of the former, improving the ability and speed of pest identification.The experiment uses BResNet-50 as the baseline model, and the experiment is carried out separately. The loss function is the standard Softmax Loss, the training dataset is CASIA-Web, the test set is LFW and YTF, the parameter in Dropout is set to 0.4, and the Dropout has a slight regularization. The effect can alleviate the problem that the network is overfitting. The results of the six groups of experiments are shown in Figure12. We can see that the 5th setting has the best recognition effect, that is, Dropout, inner layer, and bulk lie can effectively enhance the expression of pests and diseases. Improve the ability and speed of pest identification.Figure 12
Accuracy of image recognition of pests and diseases under different feature extraction layers.In the feature extraction layer setting, the final output of the pest characteristic dimension has multiple choices and which dimension is most suitable for expressing pest and disease characteristics still depends on the experience of the researchers. If you experiment with each dimension, the cost of the experiment can be very high. In the past, the characteristics of pests and diseases in the study of pest and disease identification were also different. Generally speaking, the higher the characteristic dimension of pests and diseases, the stronger the expression ability, but the increase of the number of feature channels will also increase the computational burden of the network, resulting in inefficiency. Therefore, it is particularly important to find a suitable pest and disease feature dimension, not only to reduce network parameters but also to accelerate the network training process.The 7 Hu moment features of each lesion image are shown in Table9.Table 9
Seven Hu moment eigenvalues of crop images.
DiseaseHu1Hu2Hu3Hu4Hu5Hu6Hu7Grey spot0.2430.0962.4612.4934.9714.3415.751Powdery mildew0.6471.5695.2585.85411.4326.71512.815Gall mite disease0.0030.0940.3810.66330.3910.7430.779Anthrax0.3711.2071.6541.9353.8122.5643.626It can be seen in Table9 that the parameter values of the Hu moment feature between different lesions have obvious differences, which can effectively distinguish 4 different diseases.Through each kind of lesion image (choose one image for each), the size is unchanged, and when the rotation degree is different, such as −30 degrees, −15 degrees, 0 degrees, 15 degrees, and 30 degrees, the calculated value of the Zernike moment is 4 kinds.Lycium barbarum diseases are analyzed, as shown in Table 10.Table 10
Image feature values at different degrees of rotation.
Disease0 degree15 degrees30 degrees-5 degrees-30 degreesGrey spot0.0570.0510.0530.0460.058Powdery mildew0.640.6430.6480.6450.637Gall mite disease0.0030.0360.0340.0370.038Anthrax0.3710.370.3730.3710.373It can be seen in Table10 that for each type of lesion image, when the image size is unchanged, the feature parameters are relatively stable and the shape characteristics of the lesion can be well represented by rotating at different angles.In order to verify the stability of the characteristics, for a kind of lesion image, under different rotation degrees and different zoom magnifications, the calculated values are compared and analyzed for the two diseases, as shown in Table11.Table 11
Different zoom multiples and different rotation angles.
Zoom factorDegree of rotationGrey spotAnthrax0.8−150.3130.74400.3020.750300.3270.7651−150.3300.76300.3140.743300.3200.761It can be known that the combination of feature and Hu moment feature extraction of lesion features can be better applied to the recognition of disease images and the recognition effect of the feature is better than the single use of the Hu moment feature.The average accuracy, precision, recall, and F-1 value are used to evaluate the three models as a whole. The average accuracy is the average of the last five results of the model. The specific indicators are shown in Table12. The data in the table shows that DCNN has a better performance. With strong representation ability, excellent results have been achieved on the crop disease dataset in this article.Table 12
Evaluation indicators of different algorithms.
ModelAccuracyPrecisionRecallF-1 valueAlexNet0.830.860.820.84DCNN0.990.920.910.91GoogLeNet0.850.820.820.85


## 4.1. Pest Image Recognition

### 4.1.1. Pretreatment of Crop Disease Images

The training process of using the ResNet network to recognize crop disease images is initializing the weight parameters of the network model, the convolutional layer in the ResNet network extracts the characteristics of the training dataset image, the characteristics are pooled to obtain the output, and the output value and the difference between the target values use the error to update the weight parameters of the network to obtain a trained crop disease image recognition model.The purpose of preprocessing is to remove the noise and redundant information in the sample picture, which can simplify the image and improve the recognition rate of valuable information, thereby improving the performance of model identification. And the use of deep learning method for classification and identification requires a high amount of data. This paper only collects about 5,000 images, so the data enhancement process needs to be done later. There may be content in the image that has nothing to do with the target recognition. They not only are useful for extracting image features but also sometimes may reduce the effect of feature extraction. Therefore, the image should be extracted before image recognition. In this paper, ACDSee9.0 software is used to manually cut the picture. For the disease image, the cutting leaves a leaf and the main cutting leaves the diseased part. The cutting effect is shown in Figure6.Figure 6
Crop disease image after cropping.
(a)
Soy(b)
Fruit tree(c)
Rice(d)
Cucumber


### 4.1.2. Pathological Leaf Lesion Extraction

By multiplying the background part and the preprocessed image, the green leaf part can be removed and the leaf lesions can be extracted and the treatment effect is shown in Figure7. By processing the image of the diseased leaf collected, a clear image of the lesion is obtained, but the color, shape, and texture characteristic parameters of the lesion image need to be separately extracted to identify the disease.Figure 7
Extraction of diseased leaf lesions.
(a)
Preprocessed image(b)
Color lesion image


## 4.1.1. Pretreatment of Crop Disease Images

The training process of using the ResNet network to recognize crop disease images is initializing the weight parameters of the network model, the convolutional layer in the ResNet network extracts the characteristics of the training dataset image, the characteristics are pooled to obtain the output, and the output value and the difference between the target values use the error to update the weight parameters of the network to obtain a trained crop disease image recognition model.The purpose of preprocessing is to remove the noise and redundant information in the sample picture, which can simplify the image and improve the recognition rate of valuable information, thereby improving the performance of model identification. And the use of deep learning method for classification and identification requires a high amount of data. This paper only collects about 5,000 images, so the data enhancement process needs to be done later. There may be content in the image that has nothing to do with the target recognition. They not only are useful for extracting image features but also sometimes may reduce the effect of feature extraction. Therefore, the image should be extracted before image recognition. In this paper, ACDSee9.0 software is used to manually cut the picture. For the disease image, the cutting leaves a leaf and the main cutting leaves the diseased part. The cutting effect is shown in Figure6.Figure 6
Crop disease image after cropping.
(a)
Soy(b)
Fruit tree(c)
Rice(d)
Cucumber


## 4.1.2. Pathological Leaf Lesion Extraction

By multiplying the background part and the preprocessed image, the green leaf part can be removed and the leaf lesions can be extracted and the treatment effect is shown in Figure7. By processing the image of the diseased leaf collected, a clear image of the lesion is obtained, but the color, shape, and texture characteristic parameters of the lesion image need to be separately extracted to identify the disease.Figure 7
Extraction of diseased leaf lesions.
(a)
Preprocessed image(b)
Color lesion image


## 4.2. Simulation Experiment of Disease Image Fusion

Under the complex background of the field, in the same scene, the collection equipment shoots diseased leaves under different focal lengths and different lighting conditions. Some images are clear and blurred in the foreground, and some images are blurred in the foreground and clear. The disease image fusion method based on wavelet transform is adopted. The disease image is decomposed by a wavelet, and the schematic diagram of wavelet decomposition is shown in Figure8.Figure 8
Schematic diagram of wavelet decomposition of the disease image.After that, the disease image fusion is performed on the two fuzzy disease images of wolfberry leaves and the result of disease fusion is shown in Figure9.Figure 9
Disease image fusion result.It can be seen from the figure above that there are fuzzy parts in the first two disease images. After the disease images are fused, the fused image obtained is clearer. Based on the wireless network communication and deep learning image fusion method, the coefficient with a large absolute value is selected as the wavelet coefficient to retain the details, which not only corresponds to the significant brightness change in the image, but also retains the salient features in the image, making the disease image clearer.For the diseased images ofLycium barbarum leaves collected under the complex background of the field, the Sobel operator, Robert operator, Prewitt operator, Cannon operator, Gauss-Laplacian operator, Canny operator, and phase-based edge detection method were used to detect the diseased leaves. For edge detection, the detection result is shown in Figure 10.Figure 10
Disease image monitoring results.
(a)
Original disease image(b)
The edge detection result of the image by the Sobel operator(c)
Robert operator performs edge detection results on the image(d)
Prewitt operator performs edge detection results on the image(e)
The edge detection result of the image by the Cannon operator(f)
Gauss-Laplacian operator performs edge detection results on the image(g)
Canny operator performs edge detection results on the image(h)
Phase-based image edge detection resultsFor fuzzy disease images of the same diseased leaf, six objective evaluation methods, including contrast, sharpness, average fuzzy degree, fuzzy degree, color aggregation vector, and hue count, are used to evaluate the image quality. The value of the color aggregation vector is the histogram of the aggregation pixel. For graph feature values, perform quality evaluation on one of the blurred images and the evaluation results are shown in Table2.Table 2
Image quality evaluation results of blurred diseases.
DiseaseContrastClarityAverage ambiguityAmbiguityColor aggregation vectorHuePowdery mildew59.152752.19350.19970.68970.573657.9287Grey spot67.027260.98130.21580.89711.463859.7571Gall mite disease59.267244.73160.14020.29830.499152.3104Anthrax51.951440.58790.10210.47910.294741.7839The objective quality evaluation of the regional image after fusion of the diseased image is carried out by six indicators such as contrast, sharpness, average fuzzy degree, fuzzy degree, color aggregation vector, and hue count. The evaluation results are shown in Table3.Table 3
Image quality after image fusion.
DiseaseContrastClarityAverage ambiguityAmbiguityColor aggregation vectorHuePowdery mildew61.957963.91590.38940.99780.690759.9135Grey spot71.910865.87960.57091.19871.509261.9083Gall mite disease68.439164.09570.27940.39970.591362.0304Anthrax55.794255.08760.39350.67920.679247.7189In order to prove that the segmentation method based on wireless network communication and deep learning can effectively segment disease images under complex background, this paper compares it with traditional segmentation algorithms. When the segmentation results are basically the same, the segmentation algorithm based on wireless network communication and deep learning has obvious advantages. The comparison between the number of iterations and the split time is shown in Table4.Table 4
Comparison of the number of iterations and split time.
Deep learningTraditional algorithmNumber of iterations (times)Split time (seconds)Number of iterations (times)Split time (seconds)Standard image30014.012609.98Disease image25011.642208.09When the segmentation results are basically the same, the number of iterations of the segmentation experiment on the standard image set is reduced by about 13% and the segmentation time is reduced by about 27%; the number of iterations of the segmentation experiment on the diseased image set is reduced by about 12%, and the segmentation time is reduced by about 30%, and as the number of iterations increases, the time for algorithm segmentation also continues to increase.Aiming at five different disease types (no disease, gray spot, powdery mildew, gall mites, and anthracnose) of wolfberry disease images in small sample data, the segmentation method based on wireless network communication and deep learning proposed in this paper is compared with traditional segmentation and comparison of fuzzy C-means clustering segmentation algorithms. The average error rate of the method is shown in Table5.Table 5
Comparison of average error rates of different segmentation methods.
Disease typeFuzzy C-means clustering algorithmTraditional algorithmDeep learning algorithmDisease free19.814.74.5Grey spot41.9135.735.27Powdery mildew32.8531.663.08Gall mite disease33.0332.182.27Anthrax27.5926.872.86As can be seen in Table5, the average error rate of the fuzzy C-means clustering segmentation algorithm is large, which will cause a large number of redundant backgrounds; the average error rate of the traditional algorithm segmentation is lower than the average error rate of the fuzzy C-means clustering algorithm. The average error rate of the methods exceeds 10%, and the average error rate of the deep learning algorithm used in this paper is below 6%.In order to verify the segmentation effectiveness of the deep learning algorithm on the fine-grained category dataset, compared with the GrabCut algorithm segmentation method based on the saliency map, the average accuracy of the two segmentation methods is shown in Table6.Table 6
Comparison of the average accuracy of the two segmentation methods.
Disease typeDeep learning algorithmGrabCut algorithm based on saliency mapDisease free94.490.2Grey spot93.5271.27Powdery mildew95.7176.14Gall mite disease96.5275.64Anthrax96.0381.09Finally, in order to prove the segmentation effect based on the deep learning algorithm in the light-affected environment, for the diseased images in the normal field environment and the light-affected environment, the segmentation algorithm proposed in this paper is mainly based on the average error rate and the average recall rate, as shown in Table7.Table 7
Performance evaluation of the segmentation algorithm.
Environment typeAverage error rate (%)Average recall rate (%)Normal field environment0.0890.913Light affects the environment0.1520.856As can be seen in Table7, for the disease images in the normal field environment and the light-affected environment, the average recall rate under normal lighting conditions based on the deep learning algorithm proposed in this paper can reach 90%, while in a strong lighting environment, the following will affect the performance of the segmentation algorithm, so the lighting conditions are an important factor in the performance evaluation of the segmentation algorithm.


## 4.3. Analysis of the Accuracy of Image Recognition of Pests and Diseases

### 4.3.1. Accuracy of the Test Set under Different Settings of the First Layer of Convolution

The image of crop pests and diseases can be well satisfied with the crop pest identification task after the background is removed at a resolution of about100×100. Therefore, all relevant experiments are to align the image detection of crop pests and diseases to 112×112 and the pixels of the image of three channels. Normalize by subtracting 127.5 and dividing by 128. Like most convolutional neural networks, the experiments in this article are based on the ImageNet dataset, so the network input size is generally large (such as 224×225). The first convolutional layer is generally 7×7, and the size of the convolution kernel in step A is a sample image of length 2, but for crop pest images, greater perception also means smaller feature maps. Therefore, the first layer of convolution kernel is set to 7×7 convolution kernel, the step size is 2, and it is named A. The following is a ResNet-50 network. The first layer of convolutional layers is set to A and B, namely, AResNet-50 and BResNet-50, the loss function is the standard Softmax loss function, and the training dataset is CASIA-Web, in LFW, and the YTF test set was tested, and the experimental results are shown in Table 8 and Figure 11.Table 8
Accuracy of the test set under different settings of the first layer of the network convolution layer.
NETLFWYTFAResNet-50-(4)97.70%92.11%BResNet-50-(4)98.08%92.59%Figure 11
Network first-layer convolution layer different settings in the test set accuracy.We can find that the use of setting B is better than A. The reason may be that the large convolution kernel will make the feature size drop too fast, and the network cannot effectively learn the features with strong expressive ability. Using setting B, the first-layer output feature map size is56×56, and the setting A first-layer output feature map size is 112×112, so that the latter feature image size is twice that of the former, improving the ability and speed of pest identification.The experiment uses BResNet-50 as the baseline model, and the experiment is carried out separately. The loss function is the standard Softmax Loss, the training dataset is CASIA-Web, the test set is LFW and YTF, the parameter in Dropout is set to 0.4, and the Dropout has a slight regularization. The effect can alleviate the problem that the network is overfitting. The results of the six groups of experiments are shown in Figure12. We can see that the 5th setting has the best recognition effect, that is, Dropout, inner layer, and bulk lie can effectively enhance the expression of pests and diseases. Improve the ability and speed of pest identification.Figure 12
Accuracy of image recognition of pests and diseases under different feature extraction layers.In the feature extraction layer setting, the final output of the pest characteristic dimension has multiple choices and which dimension is most suitable for expressing pest and disease characteristics still depends on the experience of the researchers. If you experiment with each dimension, the cost of the experiment can be very high. In the past, the characteristics of pests and diseases in the study of pest and disease identification were also different. Generally speaking, the higher the characteristic dimension of pests and diseases, the stronger the expression ability, but the increase of the number of feature channels will also increase the computational burden of the network, resulting in inefficiency. Therefore, it is particularly important to find a suitable pest and disease feature dimension, not only to reduce network parameters but also to accelerate the network training process.The 7 Hu moment features of each lesion image are shown in Table9.Table 9
Seven Hu moment eigenvalues of crop images.
DiseaseHu1Hu2Hu3Hu4Hu5Hu6Hu7Grey spot0.2430.0962.4612.4934.9714.3415.751Powdery mildew0.6471.5695.2585.85411.4326.71512.815Gall mite disease0.0030.0940.3810.66330.3910.7430.779Anthrax0.3711.2071.6541.9353.8122.5643.626It can be seen in Table9 that the parameter values of the Hu moment feature between different lesions have obvious differences, which can effectively distinguish 4 different diseases.Through each kind of lesion image (choose one image for each), the size is unchanged, and when the rotation degree is different, such as −30 degrees, −15 degrees, 0 degrees, 15 degrees, and 30 degrees, the calculated value of the Zernike moment is 4 kinds.Lycium barbarum diseases are analyzed, as shown in Table 10.Table 10
Image feature values at different degrees of rotation.
Disease0 degree15 degrees30 degrees-5 degrees-30 degreesGrey spot0.0570.0510.0530.0460.058Powdery mildew0.640.6430.6480.6450.637Gall mite disease0.0030.0360.0340.0370.038Anthrax0.3710.370.3730.3710.373It can be seen in Table10 that for each type of lesion image, when the image size is unchanged, the feature parameters are relatively stable and the shape characteristics of the lesion can be well represented by rotating at different angles.In order to verify the stability of the characteristics, for a kind of lesion image, under different rotation degrees and different zoom magnifications, the calculated values are compared and analyzed for the two diseases, as shown in Table11.Table 11
Different zoom multiples and different rotation angles.
Zoom factorDegree of rotationGrey spotAnthrax0.8−150.3130.74400.3020.750300.3270.7651−150.3300.76300.3140.743300.3200.761It can be known that the combination of feature and Hu moment feature extraction of lesion features can be better applied to the recognition of disease images and the recognition effect of the feature is better than the single use of the Hu moment feature.The average accuracy, precision, recall, and F-1 value are used to evaluate the three models as a whole. The average accuracy is the average of the last five results of the model. The specific indicators are shown in Table12. The data in the table shows that DCNN has a better performance. With strong representation ability, excellent results have been achieved on the crop disease dataset in this article.Table 12
Evaluation indicators of different algorithms.
ModelAccuracyPrecisionRecallF-1 valueAlexNet0.830.860.820.84DCNN0.990.920.910.91GoogLeNet0.850.820.820.85


## 4.3.1. Accuracy of the Test Set under Different Settings of the First Layer of Convolution

The image of crop pests and diseases can be well satisfied with the crop pest identification task after the background is removed at a resolution of about100×100. Therefore, all relevant experiments are to align the image detection of crop pests and diseases to 112×112 and the pixels of the image of three channels. Normalize by subtracting 127.5 and dividing by 128. Like most convolutional neural networks, the experiments in this article are based on the ImageNet dataset, so the network input size is generally large (such as 224×225). The first convolutional layer is generally 7×7, and the size of the convolution kernel in step A is a sample image of length 2, but for crop pest images, greater perception also means smaller feature maps. Therefore, the first layer of convolution kernel is set to 7×7 convolution kernel, the step size is 2, and it is named A. The following is a ResNet-50 network. The first layer of convolutional layers is set to A and B, namely, AResNet-50 and BResNet-50, the loss function is the standard Softmax loss function, and the training dataset is CASIA-Web, in LFW, and the YTF test set was tested, and the experimental results are shown in Table 8 and Figure 11.Table 8
Accuracy of the test set under different settings of the first layer of the network convolution layer.
NETLFWYTFAResNet-50-(4)97.70%92.11%BResNet-50-(4)98.08%92.59%Figure 11
Network first-layer convolution layer different settings in the test set accuracy.We can find that the use of setting B is better than A. The reason may be that the large convolution kernel will make the feature size drop too fast, and the network cannot effectively learn the features with strong expressive ability. Using setting B, the first-layer output feature map size is56×56, and the setting A first-layer output feature map size is 112×112, so that the latter feature image size is twice that of the former, improving the ability and speed of pest identification.The experiment uses BResNet-50 as the baseline model, and the experiment is carried out separately. The loss function is the standard Softmax Loss, the training dataset is CASIA-Web, the test set is LFW and YTF, the parameter in Dropout is set to 0.4, and the Dropout has a slight regularization. The effect can alleviate the problem that the network is overfitting. The results of the six groups of experiments are shown in Figure12. We can see that the 5th setting has the best recognition effect, that is, Dropout, inner layer, and bulk lie can effectively enhance the expression of pests and diseases. Improve the ability and speed of pest identification.Figure 12
Accuracy of image recognition of pests and diseases under different feature extraction layers.In the feature extraction layer setting, the final output of the pest characteristic dimension has multiple choices and which dimension is most suitable for expressing pest and disease characteristics still depends on the experience of the researchers. If you experiment with each dimension, the cost of the experiment can be very high. In the past, the characteristics of pests and diseases in the study of pest and disease identification were also different. Generally speaking, the higher the characteristic dimension of pests and diseases, the stronger the expression ability, but the increase of the number of feature channels will also increase the computational burden of the network, resulting in inefficiency. Therefore, it is particularly important to find a suitable pest and disease feature dimension, not only to reduce network parameters but also to accelerate the network training process.The 7 Hu moment features of each lesion image are shown in Table9.Table 9
Seven Hu moment eigenvalues of crop images.
DiseaseHu1Hu2Hu3Hu4Hu5Hu6Hu7Grey spot0.2430.0962.4612.4934.9714.3415.751Powdery mildew0.6471.5695.2585.85411.4326.71512.815Gall mite disease0.0030.0940.3810.66330.3910.7430.779Anthrax0.3711.2071.6541.9353.8122.5643.626It can be seen in Table9 that the parameter values of the Hu moment feature between different lesions have obvious differences, which can effectively distinguish 4 different diseases.Through each kind of lesion image (choose one image for each), the size is unchanged, and when the rotation degree is different, such as −30 degrees, −15 degrees, 0 degrees, 15 degrees, and 30 degrees, the calculated value of the Zernike moment is 4 kinds.Lycium barbarum diseases are analyzed, as shown in Table 10.Table 10
Image feature values at different degrees of rotation.
Disease0 degree15 degrees30 degrees-5 degrees-30 degreesGrey spot0.0570.0510.0530.0460.058Powdery mildew0.640.6430.6480.6450.637Gall mite disease0.0030.0360.0340.0370.038Anthrax0.3710.370.3730.3710.373It can be seen in Table10 that for each type of lesion image, when the image size is unchanged, the feature parameters are relatively stable and the shape characteristics of the lesion can be well represented by rotating at different angles.In order to verify the stability of the characteristics, for a kind of lesion image, under different rotation degrees and different zoom magnifications, the calculated values are compared and analyzed for the two diseases, as shown in Table11.Table 11
Different zoom multiples and different rotation angles.
Zoom factorDegree of rotationGrey spotAnthrax0.8−150.3130.74400.3020.750300.3270.7651−150.3300.76300.3140.743300.3200.761It can be known that the combination of feature and Hu moment feature extraction of lesion features can be better applied to the recognition of disease images and the recognition effect of the feature is better than the single use of the Hu moment feature.The average accuracy, precision, recall, and F-1 value are used to evaluate the three models as a whole. The average accuracy is the average of the last five results of the model. The specific indicators are shown in Table12. The data in the table shows that DCNN has a better performance. With strong representation ability, excellent results have been achieved on the crop disease dataset in this article.Table 12
Evaluation indicators of different algorithms.
ModelAccuracyPrecisionRecallF-1 valueAlexNet0.830.860.820.84DCNN0.990.920.910.91GoogLeNet0.850.820.820.85


## 5. Conclusions

(1)
Based on the theory of deep learning, this paper analyzes the classical and the latest neural network structure. The network based on natural image classification is not suitable for crop pest identification tasks. This paper improves the network structure that can simultaneously recognize the recognition speed and recognition accuracy. The influence of the crop pest and disease feature extraction layer on recognition performance is discussed. The advantages and disadvantages of the layered and global average pooling layer are finally adopted as the structure of the pest and disease feature extraction layer(2)
For the Softmax cross entropy loss function, the problem of within-the-class austerity and interclass increase cannot be satisfied at the same time. The distance between classes is increased by adding intervals from the metric learning perspective; the Softmax cross entropy loss function is changed to the cosine distance loss function. Forcing the loss function to shrink the inner distance of the class, so that the interface becomes a boundary area, the distance between classes increases, which significantly improves the recognition rate of pests and diseases. This paper is different from the shortcomings of multiplication interval, which is difficult to train and difficult to adjust. The additive interval loss function method is proposed. The experimental results show that not only the training becomes simple but also the recognition rate of pests and diseases can be improved(3)
There may be overlapping problems between the pest and disease datasets, which is not conducive to the generalization of the final model. This paper carefully screens the commonly used pest and disease datasets. After removing the overlapping data, the final accuracy of the test set will be slightly reduced but can improve the generalization ability of the model. The pest and disease detection system, the improved pest network, and the loss function were used to develop the interface of the pest identification system. The tensor flow deep learning framework was used as the model forward reasoning tool to verify the effectiveness of the system under attitude and occlusion. In the later work, the diseased leaves and the diseased parts were identified in crop disease samples with complex environments. At the same time, the recognition of the pictures taken directly in the natural environment makes the recognition accuracy of the model higher, so that the article as a whole has more practical guiding significance



---

*Source: 7577349-2021-10-14.xml*