aGROdet: A Novel Framework for Plant Disease
Detection and Leaf Damage Estimation

Alakananda Mitra, Saraju P. Mohanty, Elias Kougianos

► To cite this version:

Alakananda Mitra, Saraju P. Mohanty, Elias Kougianos. aGROdet: A Novel Framework for Plant Disease Detection and Leaf Damage Estimation. 5th IFIPIoT, Oct 2022, Amsterdam, Netherlands. pp.3-22, 10.1007/978-3-031-18872-5_1 . hal-04704222

HAL Id: hal-04704222

https://inria.hal.science/hal-04704222v1

Submitted on 25 Sep 2024

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire **HAL**, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
aGROdet: A Novel Framework
for Plant Disease Detection and Leaf Damage Estimation

Alakananda Mitra¹[0000-0002-8796-4819],
Saraju P. Mohanty¹[0000-0003-2959-6541], Elias Kougianos²[0000-0002-1616-7628]

¹ Department of Computer Science and Engineering, University of North Texas, USA.
alakanandamitra@my.unt.edu, saraju.mohanty@unt.edu

² Department of Electrical Engineering, University of North Texas, USA.
elias.kougianos@unt.edu

**Abstract.** By 2050, 60% more food will be required to feed a world population of 9.7 billion. Producing more food with traditional agriculture will stress the earth's limited natural resources. To avoid such a scenario, greener, sustainable, and modern agricultural practices should be followed. More efficient food production along with a reduction of food wastage at different levels of the food supply chain will ease our ecosystem. Plant disease outbreaks are one of the major causes of crop damage, which is essentially one of the causes of food wastage. Hence, plant disease detection and damage estimation are important to prevent crop loss. However, until now, not much work has been done to estimate the damage caused by the disease. In this paper, we propose a novel method, *aGROdet*, to detect plant disease and to estimate the leaf damage severity. *aGROdet* is aimed at being implemented at the edge platform of IoT systems in the proposed Agriculture Cyber Physical System. A convolutional neural network-based model has been proposed to detect different plant diseases. The model has been trained with large publicly available datasets. More than 97% accuracy has been achieved in the initial phase of the experiment. A pixel-based thresholding method has been used for estimating the severity of the damage. Damage estimation limiting factors, such as on the leaf and around the leaf shadows, have also been addressed.

**Keywords:** Smart Agriculture · Smart Villages · Internet of Agro Things (IoAT) · Agriculture Cyber Physical System (A-CPS) · Plant Health · Plant Disease · Crop Damage · Convolutional Neural Network

# 1 Introduction

Agriculture is one of the major industries of today's society. It is complex and is affected by various unpredictable factors such as climate change, population explosion, natural resource limitation, and plant diseases. Due to the recent advancements in information and communication technology (ICT), breakthrough hardware innovations, and the computing paradigm shift from cloud-based computing to more edge-oriented computing, various issues in agriculture are being addressed. The inclusion of automation in agriculture through Artificial Intelligence (AI)/Machine Learning (ML)/Deep Learning Technologies (DLT) has welcomed *Agriculture 4.0* [19], and *Agriculture 5.0* is knocking at the door. The need for initiatives for agriculture cyber physical systems (A-CPS)-based solutions is greater than ever. Fig. 1 shows some of the agricultural problems which can be solved using A-CPS concepts.
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Agricultural Problems solvable using Agriculture Cyber Physical Systems</figcaption></figure>

Plants, like all living things, are prone to diseases. Disease inhibits a plant from reaching its full capacity [2]. It varies with seasons and plant types. External conditions or living organisms can cause diseases. Nutritional deficit, heat, flooding, and freezing are some examples of external agents that cause non-infectious or abiotic diseases, whereas plant pathogens like fungi, bacteria, viruses, and algae cause biotic diseases. The occurrence of a biotic disease is illuminated by the “*Disease Triangle*” [32] shown in Fig. 2. Disease occurs when all three factors-favorable environment, vulnerable host, and harmful pathogens-are present concurrently. The red region in the Venn diagram of Fig. 2 represents the occurrence of the disease. However, certain factors, such as pathogen genetic variation, local micro-climate, and host plant immunity at a specific stage of its life cycle, may alter this fact [2]. To develop a disease in a plant, the pathogen needs to complete its life cycle in the host.

<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Disease Triangle [32]</figcaption></figure>
## 1.1 Research Problem

Diseases prevent the growth of plants. They affect the quality of the crop and reduce the final yield. Billions of dollars in crop losses happen per year. The food supply chain is also gravely impacted [3]. Hence, farmers need to:

- Detect the disease early.

- Identify the disease.

- Know about the severity of the disease.

- Determine the extent of damage.

Regular monitoring of plants is necessary for a successful disease management system. According to [25], speedy detection of plant disease at early stages of outbreak and its prevention will become the two major goals of agricultural research by 2030. In this paper, three of the four points are addressed.

## 1.2 Proposed Solution

We propose a novel automatic method, aGROdet, to detect plant disease and estimate corresponding leaf damage. However, the damage due to diseases can be present in different stages of plant growth and at different parts of a plant. A convolutional neural network-based method for the identification of the disease and a novel pixel-based thresholding method for estimating the leaf damage severity are proposed. Regular monitoring of fields and checking the conditions of the plants through aGROdet can detect the disease early.

The paper is organized in the following way: Section 2 discusses the significance of the work in the context of a smart village. Section 3 reviews recent work on plant disease detection. The proposed A-CPS is described in Section 4. Section 5 provides an overview of aGROdet, detailed methodology, and experimental details. Section 6 evaluates the performance of aGROdet and compares our work with existing work. Finally, the paper concludes with future work direction in Section 7.

# 2 Significance of aGROdet in a Smart Village Context

Today, close to 3.4 billion people live in rural areas. The majority of villages lack technology, innovation, energy, and industry even today. However, the modernization of villages with Internet connectivity, smart agriculture, smart healthcare, smart grid, and education is required. A holistic approach is needed for rural areas to ensure the sustainable development of society. To implement that goal, various smart village movements have recently emerged across the globe in various sectors. For example, Fig. 3 shows the smart energy project sites of IEEE Smart Village initiatives [1].

The application of heterogeneous technologies centered on the Internet-of-Things (IoT) can shape rural areas as smart villages [8]. As the financial backbone of the smart village is agriculture industry, it is one of the most important areas of research for smart villages. To transform the traditional agriculture to an efficient, sustainable, and green agriculture, digital transformation is the key. In this context, our proposed method aGROdet is appropriate.

- Plant disease is a major challenge for sustainable agriculture. It is a nightmare for farmers as disease can destroy the plants and cause huge losses. The common method of plant disease detection in developing countries even today is manual observation. It is an arduous process. It needs expertise, and the service is so expensive that it is not always affordable for farmers [31].
<figure><img src="image_5.png" /><figcaption>Fig. 3. IEEE Smart Village Map for Smart Energy Projects [1]</figcaption></figure>

In such a scenario, the farmer can have an overall idea of the disease and its severity through the proposed method, aGROdet.

- It automatically and accurately detects plant diseases and estimates damage. Significantly less effort is needed from the farmers’ perspective to use aGROdet. It is accessible through a mobile app. To get the results, farmers only need to take a photo of the diseased leaf. The rest of the process is automatic.

- It is an edge-based Internet of Agro Things (IoAT) method that can detect plant disease and estimate the damage even without an Internet connection. If an Internet connection is not available for any reason, the damage estimation procedure will not be affected. An Internet connection is used to store data in the cloud. This stored data is used for future training of the model.

- This is a very useful tool for farmers who can detect plant diseases with an estimation of plant damage on their own. No expert knowledge is required.

- We hope that aGROdet will help farmers take proper control measures and save time, money, and secondary plant losses.

## 3 Related Works

During a review of the literature, two types of papers addressing plant or crop diseases stand out: the first addresses multi-crop disease solutions, while the second focuses on a specific crop or plant type. In the last decade, mostly traditional image processing algorithms and hand-picked features with machine learning (ML) classifiers have been used to detect plant and crop diseases. Those approaches have their own difficulties, along with not so great accuracy [18]. In recent studies, mostly computer vision-based methods with deep learning networks are being proposed for this purpose. The use of deep learning networks, mostly convolutional neural network (CNN)-based approaches, makes the disease identification automatic, reduces manual intervention, and performs better in detecting plant diseases.

Complex features are obtained automatically in deep learning network-based solutions via various layers and types of neural networks, particularly CNN. Different CNNs have been used for different purposes, such as feature extractor [10], classification network [35], and disease localization network [38].
## 3.1 Single Plant/Crop Diseases Detection:

Non-parametric ML classifiers are used in various works, along with the recent trend of deep learning networks for detecting plant/crop diseases. For example, the K-means algorithm is used in [24] for paddy leaf diseases. Several studies have been conducted on cotton diseases. The K-nearest neighbors (KNN) algorithm has been used in [28] for cotton leaf diseases. Ramularia leaf blight cotton disease has been identified using non-parametric classifiers from multi-spectral imagery of an UAV in [39]. A decision tree classifier has been used for detecting cotton crop diseases [7]. Cotton leaf spot disease has been detected in [5] using Support Vector Machines (SVM). Cucumber's powdery mildew has been segmented using U-Net at pixel level with high accuracy in [17].

A combination of InceptionV3 and ResNet50 networks has been used to identify grape leaf diseases with 98.57% testing accuracy [13]. A shallow 3D CNN structure has been used on hyperspectral images to identify a soil-borne fungal disease, charcoal rot, for soybean [23]. An improved AlexNet model has been used to identify fragrant pear diseases and insect pests [37]. A typical accuracy of 96.26% has been achieved. In [26], a Faster RCNN has been used to detect sugar beet leaf spot disease with 95.48% accuracy. Northern maize leaf blight detection has been done in [34] using multi-scale feature fusion method with improved SSD. Mask R-CNN has also been used to segment UAV images in [33] for northern maize leaf blight detection. In [4], a YOLOv3 network was used to detect pests and diseases in tea leaves. Using SegNet, four categories of grape vine diseases have been identified in [14] from UAV images.

## 3.2 Multi Plants/Crops Diseases Detection:

Deep learning techniques are popular in the research community for multi-plant detection. A convolution neural network-based Teacher-Student network has been utilized to detect plant diseases [6]. A sharper visualization of the diseased leaf has been achieved with the PlantVillage dataset [11]. Another deep convolution neural network-based on GoogleNet and AlexNet has been used to detect crop diseases with 99.35% accuracy [22] using the earlier mentioned dataset. In [30], Single Shot MultiBox Detector (SSD) model has been chosen among three different deep learning models for plant disease detection. It shows 73.07% mean average precision (mAP) with the Adam optimizer on the Plant Village dataset. In [12] severity of crop leaf disease has been estimated along with crop type and crop disease prediction with an 86.70% accuracy using binary relevance (BR) multi-label learning algorithm and Convolutional Neural Network. Another CNN-based structure, built from a ResNet50 network with shuffle units, has been used to detect plant disease and estimate the severity of the disease in [16] with an accuracy of 91%, 99%, and 98% for disease severity, plant type, and plant disease classification, respectively. In [9] several networks have been tested and finally an accuracy of 99.53% in identifying plant disease has been achieved. Disease prediction has also been done along with crop selection and irrigation [36]. In this work, a CNN-based plant disease detection network has achieved an accuracy of 99.25%.

From the above discussion, it is clear that the majority of papers address various diseases for different plants or crops. However, it is highly important to estimate the disease-related damage. Without that knowledge, plant disease management and prevention is not possible.

# 4 Proposed A-CPS

Fig. 4 shows the agriculture cyber physical system (A-CPS) [21] for plant disease detection and damage severity estimation. It is developed through the proposed IoAT-based method aGROdet.
The A-CPS consists of two systems - physical systems and cyber systems. Physical systems consist of “things”, stakeholders, and computing devices. In our case, the “things” are UAV cameras and phone cameras, the computing devices are single board computers and mobile phones, and the “stakeholders” are microbiologists, plant pathologists, agriculture companies, farmers, and the Agriculture Research Service. Cyber systems comprise deep learning models, software, efficient data storage, and blockchain for data security. It is distributed in two different platforms. Deep learning models and software are present both at the edge and in the cloud, whereas the rest are mainly in the cloud. Physical systems and cyber systems are connected through the network fabric. Depending on the location and range, the network fabric can be Sigfox, ZigBee, LoRa, Wi-Fi, 4G, or 5G.

<figure><img src="image_6.png" /><figcaption>Fig. 4. Agriculture Cyber Physical System</figcaption></figure>

As aGROdet performs two jobs - plant disease identification and damage severity estimation- we divide the work into two parts. The methods have been described in the following Section 5.

# 5 aGROdet: Proposed Method

## 5.1 Detection of Plant Disease

**Methodology** This section describes the proposed deep learning-based method for identifying plant diseases from images of leaves. It is a multi-class image classification problem wherein the model learns to label images through supervised learning techniques and predicts the label of an unknown image. The model learns the features of the labeled images during training and classifies the unknown and unlabeled images with a confidence score. The success of accurate prediction depends on the classifying skill of the model, which in turn depends on how well the model has learned.

**Network Architecture:** Convolutional neural networks (CNN) are state-of-the-art architectures for image classification. Various CNN structures are being used for image classification in the literature. Here, a custom CNN has been used for plant disease detection purposes, as shown
in Fig. 5. It has 5 convolutional blocks. Each block comprises a *Convolutional* layer with *ReLU* activation followed by a *BatchNormalization* layer and a *MaxPooling* layer. There are 32 filters in the first Conv2D layer, 64 filters in the Conv2D layers of the next three blocks, and the final block Conv2D layer consists of 128 filters. The kernel sizes of the convolutional layers are kept the same as (3×3) with stride 1 and no zero padding. *BatchNormalization* layers only normalize the previous layer output during inference after being trained on a similar type of images as testing data. A *MaxPooling* layer has been used to reduce the spatial dimensions. The kernel size of the *MaxPooling* layer is 2×2 with stride 2. The final block is followed by a *Flatten* layer which is succeeded by two *Dense* layers. The first *Dense* layer uses *ReLU* activation and 1280 nodes, whereas the last one has 39 nodes and a *Softmax* activation function. 6,117,287 of the 6,117,991 parameters are trained. Table 1 describes the output shapes of the layers in detail.

<figure><img src="image_7.png" /><figcaption>Fig. 5. Plant Disease Detection Network</figcaption></figure>

## Experimental Validation

**Dataset Details:** In this section, experimental validation of disease detection is presented. Publicly available plant leaf data has been used for training and evaluating purposes. The PlantVillage dataset [11] has been used for training the system. The dataset has 55,448 images of 39 different classes. 38 classes are related to plants' leaves, and 1 class is for images with no leaves. 49,886 images were used for training and validation whereas 5,562 images to test the method. Fig. 6 shows some sample images from the dataset. 80% - 20% distribution has been used for the training and validation.

**Dataset Processing:** RGB images of size 256×256 have been used for training. The images have been normalized before sending them to the network to avoid slowing down during training by limiting computation with large numbers. Data augmentation has been performed on training and validation data for better and more accurate performance. Fig. 7 shows samples of augmented data. Image processing techniques, e.g., rotation, zoom, brightness, horizontal and vertical flip, have been used to generate augmented data on the go.

**Experiment:** Fig. 8 shows the plant disease detection workflow. The augmented and preprocessed data is used for training the network. The Adam optimizer [15] has been used with an initial learning rate of 0.001. The model has been trained for 75 epochs, meaning 75 times the network iterated through the total dataset during training. The model has been trained with
<table><caption>**Table 1.** *aGROdet* CNN Architecture for Plant Disease Detection</caption><thead><tr><th>Layers</th><th>Output Shape</th></tr></thead><tbody><tr><td>Conv2D (f=32, k=3, s=1, p=0)<br>Activation: ReLU</td><td>(254, 254, 32)</td></tr><tr><td>BatchNormalization</td><td></td></tr><tr><td>Maxpooling2D (k=2, s=2)</td><td>(127, 127, 32)</td></tr><tr><td>Conv2D (f=64, k=3, s=1, p=0)<br>Activation: ReLU</td><td>(125, 125, 64)</td></tr><tr><td>BatchNormalization</td><td></td></tr><tr><td>Maxpooling2D (k=2, s=2)</td><td>(62, 62, 64)</td></tr><tr><td>Conv2D (f=64, k=3, s=1, p=0)<br>Activation: ReLU</td><td>(60, 60, 64)</td></tr><tr><td>BatchNormalization</td><td></td></tr><tr><td>Maxpooling2D (k=2, s=2)</td><td>(30, 30, 64)</td></tr><tr><td>Conv2D (f=64, k=3, s=1, p=0)<br>Activation: ReLU</td><td>(28, 28, 64)</td></tr><tr><td>BatchNormalization</td><td></td></tr><tr><td>Maxpooling2D (k=2, s=2)</td><td>(14, 14, 64)</td></tr><tr><td>Conv2D (f=128, k=3, s=1, p=0)<br>Activation: ReLU</td><td>(12, 12, 128)</td></tr><tr><td>BatchNormalization</td><td></td></tr><tr><td>Maxpooling2D (k=2, s=2)</td><td>(6, 6, 128)</td></tr><tr><td>Flatten</td><td>(4,608)</td></tr><tr><td>Dense (u=1280)</td><td>(1280,)</td></tr><tr><td>Dense (u=39)</td><td>(39,)</td></tr></tbody></table>

<figure><img src="image_8.png" /><figcaption>**Fig. 6.** Sample Images from PlantVillage Dataset [11]</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>**Fig. 7.** Sample Augmented Data. Data is augmented on the fly for different rotation, zoom, brightness, horizontal and vertical flip.</figcaption></figure>

and without a reduced learning rate of factor 0.1. Then the trained model is saved for future inference. The model is evaluated using the 5,562 images that were kept aside. The disease detection network in *aGROdet* has been implemented in Keras with TensorFlow back end.

<figure><img src="image_10.png" /><figcaption>**Fig. 8.** Plant Disease Detection Workflow</figcaption></figure>

## 5.2 Estimation of Leaf Damage Severity

This section describes the leaf damage severity estimation process. To estimate damage severity, leaf area and damage area are calculated. The ratio of these two areas gives the percentage of leaf damage. Finally, a rule-based system predicts the damage severity. Fig. 9 shows the pipeline of the method.
<figure><img src="image_11.png" /><figcaption>**Fig. 9.** Leaf Damage Estimation Workflow</figcaption></figure>

**Leaf Area Detection** This is the first step to estimating leaf damage severity. First, the leaf area is detected and a mask is created for the leaf. Background segmentation and thresholding have been used to create the mask. Finally, the area of the mask is calculated to obtain the leaf area.

*Background Segmentation:* The leaf image consists of two parts- foreground object and the background. Our object of interest is the foreground object, or leaf. To segment the background from the leaf, the GrabCut [29] algorithm has been used. When different foreground objects are present, the number of iterations and the parameters of the algorithm need to be changed manually. But as in our case, only a specific type of object, i.e., leaf, is detected, no manual adjustment is necessary.

In this method, an initial rectangle is drawn over the foreground object. The outside of the rectangle is considered the confirmed background. The inside of the rectangle consists of the foreground and some parts of the background. In our work, we kept the image size of 256x256 as in Fig. 10(a) and chose to draw a large rectangle of size 226x226. A large rectangle is drawn to ensure that the whole foreground object or leaf stays within the Region of Interest (ROI).

Once the ROI is defined, the GrabCut algorithm applies a Gaussian Mixture Model (GMM) to the ROI. The pixels are grouped based on their similarity in color. A graph is created based on the pixel distribution where each pixel forms a node. Two additional nodes work as the references. The pixels attached to the *Source* node are considered foreground pixels. However, background pixels are connected to the *Sink* node. The probabilities of connecting to *Source* or *Sink* nodes decide the weights of the edges of the graph. Similar pixel nodes are connected by edges with higher weight values. Finally, the foreground pixels are segmented from the background pixels by minimizing a cost function, which is the summation of the weights of the cut edges. We iterated the process 5 times to segment the leaf from its background. After segmentation, the background pixels are turned black for the next step of processing, as shown in Fig. 10(b).

*Thresholding and Leaf Area Detection:* Shadows can be present on and around the leaves. They have an impact on accurate leaf detection. The outer shadow increases the leaf area, whereas the on-leaf shadows hinder the creation of a perfect mask for the leaf. The large red ovals in Fig. 10 show around the leaf shadows, and smaller circles denote on the leaf shadows.

As HSV color space separates image color (hue) from the color intensity (value), we transform the leaf images from RGB color space to HSV color space. The thresholding is then performed over black color, as in Fig. 10(c). As the foreground object, a leaf, is our object of interest, the mask is inverted. But several masks have noise due to specular reflection and shadows on the
<figure><img src="image_12.png" /><figcaption>**Fig. 10.** Leaf Area Detection by Creating Leaf Mask. a. Input Image b. Background Segmentation c. Mask Creation for the Leaf d. Noise Reduction from the Mask. Red large ovals show the shadow around the foreground object and small circles highlight the shadows on the foreground object.</figcaption></figure>

leaf. This noise has been shown in small red circles in Fig. 10(c). To get a noise-free mask, we selected the largest contour of the foreground object. The healthy leaf consists of a large contour, whereas a damaged leaf has a larger contour and several smaller contours depending on the damage. Hence, the largest contour, selected from the foreground image, is drawn over the mask as in Fig. 10(d). It gives a perfect noise-free mask for the leaf.

***Around the Leaf Shadow Removal:*** Around the leaf shadows have been removed before background segmentation. As shown in Fig. 11(b), pixel-based thresholding is performed to select the shadow. The area around the leaf shadow part is then segmented from the foreground leaf during background segmentation, as in Fig. 11(c). It is removed through contour selection during final mask generation as in Fig. 11(d). Finally, the final mask is made noise free in Fig. 11(e).

<figure><img src="image_13.png" /><figcaption>**Fig. 11.** Removal of Shadow around the Leaf. a. Input Image b. Detection of Shadow around the Leaf c. Shadow Removal d. Leaf Mask Creation e. Noise Reduction from the Mask. Red large ovals show shadow around the leaf and brown ovals highlight the shadow on the leaf.</figcaption></figure>

**Damage Area Detection** Leaf damage area calculation is also necessary to estimate the leaf damage severity. The process is shown in Fig. 12. First, around the leaf shadow is detected and removed, as in Fig. 12(b) and Fig. 12(c). As shown in Fig. 12(d), a mask is generated for the green portion of the leaf and is bit-wise merged with the input image, as shown in Fig. 12(e). Next, the black background of the image is segmented from the merged image and recolored with any other color to differentiate it from the damage, as in Fig. 12(f). Next, pixel-based thresholding is performed on the black color to generate the mask for the damage, as in Fig. 12(g).
<figure><img src="image_14.png" /><figcaption>**Fig. 12.** Leaf Damage Area Detection a. Input Image b. Detection of Shadow around the Leaf c. Shadow Removal d. Leaf Mask Creation e. Merging of Mask and Input Image f. Recoloration of the Black Background to Differentiate them from the Damage g. Damage Mask Creation.</figcaption></figure>

**Leaf Damage Estimation** For estimating leaf damage, the areas of the leaf mask and damage mask are calculated. Pixels, present in the masks, are counted to calculate the area. Fig. 13 shows a sample area calculation and the estimated percentage damage of a leaf.

<figure><img src="image_15.png" /><figcaption>**Fig. 13.** Leaf Damage Estimation</figcaption></figure>

Then, a rule-based system decides the severity of the damage to the leaf. The damage severity grade scale is suggested in Table 2. If there is no damage detected, the system predicts the leaf as healthy. However, if the percentage of damage is greater than 0, it grades the damage severity into different tiers depending on the values. According to Table 2, the damage severity grade of the damaged leaf in Fig. 13 is Gr-1 as the damage is between 0 and 5%.

<table><caption>**Table 2.** Damage Severity Grade Scale</caption><thead><tr><td>Estimated Damage (%)</td><td>Damage Severity Grade</td></tr></thead><tbody><tr><td>0</td><td>Healthy</td></tr><tr><td>&gt;0 and &lt;=5</td><td>1</td></tr><tr><td>&gt;5 and &lt;=10</td><td>2</td></tr><tr><td>&gt;10 and &lt;=25</td><td>3</td></tr><tr><td>&gt;25 and &lt;=50</td><td>4</td></tr><tr><td>&gt; 50</td><td>5</td></tr></tbody></table>
# 6 Performance Evaluation of aGROdet

This section describes the performance of *aGROdet* for disease detection and disease severity estimation. Unseen images from the PlantVillage Dataset [11] have been used for evaluation purposes.

## 6.1 Disease Detection

The performance of the model has been evaluated through various metrics. 5,562 unseen images of the [11] dataset have been used for validating the model. Fig. 14 shows the *confusion matrix* for this multi-class problem. Different performance metrics [20] have been calculated as in Eqns. 1, 2, 3, and 4.

<figure><img src="image_16.png" /><figcaption>**Fig. 14.** Confusion Matrix for Disease Detection Network (Trained without reduced learning rate). Classes are denoted by numbers instead of the class names to fit into the figure space.</figcaption></figure>
$$ Accuracy = \frac{TP+TN}{TP+TN+FP+FN} \qquad (1) $$

$$ Precision = \frac{TP}{TP+FP} \qquad (2) $$

$$ Recall = \frac{TP}{TP+FN} \qquad (3) $$

$$ F1-score = \frac{2}{\frac{1}{Precision} + \frac{1}{Recall}} \qquad (4) $$

TP is True Positive, TN is True Negative, FP is False Positive, and FN is False Negative. Two other diagnostic curves-ROC curves and Precision-Recall curves-are drawn too. Fig. 15(a) and Fig. 15(b) show such curves for only 8 classes. These evaluating tools are originally defined for binary class problems. However, for multi-class problems, these metrics and curves have been obtained by utilizing the *one vs. all* method. A weighted average precision of 98% has been achieved.

<figure><img src="image_17.png" /><figcaption>**Fig. 15.** Performance Evaluation Curves for Disease Detection (Trained without reduced learning rate)</figcaption></figure>

Table 3 shows the accuracy of the model for two different training scenarios. When the model is trained with a reduced learning rate of factor 0.1, better accuracy is obtained.

<table><caption>**Table 3.** Accuracy for Disease Detection Network</caption><thead><tr><th rowspan="2">Training Type</th><th colspan="3">Accuracy (%)</th></tr><tr><th>Training</th><th>Validation</th><th>Testing</th></tr></thead><tbody><tr><td>Without reduced learning rate</td><td>97.62</td><td>97.42</td><td>97.68</td></tr><tr><td>With reduced learning rate</td><td>98.89</td><td>98.41</td><td>98.58</td></tr></tbody></table>

## 6.2 Leaf Damage Severity Estimation

The method has been validated with part of the PlantVillage dataset [11]. No experiment has been done with corn leaf images in the dataset. Estimation of damage will not be correct in those cases, as the whole leaf is not visible in the image. Table 4 shows some sample results. The first
column shows the tested images, whereas the second and third columns present leaf and damage masks, respectively. The results are stated in columns four and five. The estimated leaf damage presented in the fourth column of the table matches with the leaf and mask damage images of columns two and three. The shadows on and around the leaves impact the damage estimation negatively. However, damage estimation by aGROdet is not affected as damage masks in column three of Table 4 are accurately generated even in the presence of shadows. Even if there is some specular reflection in the image, aGROdet can still correctly estimates the damage of leaves.

There are certain scenarios when aGROdet will not estimate leaf damage correctly, e.g., for variegated plants. In those plants, the healthy leaves have other colors, e.g., yellow or white, along with green. Abelia, Azalia, Boxwood, Cape Jasmine, Hydrangea, and Lilac are such variegated plants. However, our area of interest is mainly crops, fruits, and vegetable plants or trees where the color of the leaves is usually green. They may turn yellow if they are under abiotic stress due to lack of nutrients in the soil, over or under watering, over use of fertilizers, extreme cold, and absence of enough light. However, aGROdet can detect those yellow parts as damage.

<table><caption>**Table 4.** Damage Severity Prediction through aGROdet</caption><thead><tr><th>Image</th><th>Leaf Mask</th><th>Damage Mask</th><th>Estimated Damage (%)</th><th>Damage Severity Grade</th></tr></thead><tbody><tr><td></td><td></td><td></td><td>3.95</td><td>1</td></tr><tr><td></td><td></td><td></td><td>2.97</td><td>1</td></tr><tr><td></td><td></td><td></td><td>53.49</td><td>5</td></tr><tr><td></td><td></td><td></td><td>10.69</td><td>3</td></tr><tr><td></td><td></td><td></td><td>9.49</td><td>2</td></tr></tbody></table>

<figure><img src="image_18.png" /></figure>

<figure><img src="image_19.png" /></figure>

<figure><img src="image_20.png" /></figure>

<figure><img src="image_21.png" /></figure>

<figure><img src="image_22.png" /></figure>

<figure><img src="image_23.png" /></figure>

<figure><img src="image_24.png" /></figure>

<figure><img src="image_25.png" /></figure>

<figure><img src="image_26.png" /></figure>

<figure><img src="image_27.png" /></figure>

<figure><img src="image_28.png" /></figure>

<figure><img src="image_29.png" /></figure>

<figure><img src="image_30.png" /></figure>

<figure><img src="image_31.png" /></figure>

<figure><img src="image_32.png" /></figure>
## 6.3 Comparative Analysis

Table 5 shows a comparative analysis between *aGROdet* and other existing works. The majority of the papers did not address the disease severity issue. [12] has addressed the disease severity issue, but lower accuracy has been obtained. However, an accurate leaf damage percentage has been achieved in our work along with the disease type. *aGROdet* gives a better perspective of leaf damage.

<table><caption>**Table 5.** A Quantitative Analysis of the Current Paper with Existing Works</caption><thead><tr><th>Works</th><th>Disease Type</th><th>Accuracy (%)</th><th>Damage Estimation</th></tr></thead><tbody><tr><td>Ji et al. [12]</td><td>Multi Disease</td><td>86.70</td><td>Yes</td></tr><tr><td>Mohanty et al. [22]</td><td>Multi Disease</td><td>99.35</td><td>No</td></tr><tr><td>Ji et al. [13]</td><td>Single</td><td>98.57</td><td>No</td></tr><tr><td>Wang [37]</td><td>Single</td><td>96.26</td><td>No</td></tr><tr><td>Ozguven et al. [26]</td><td>Single</td><td>95.48</td><td>No</td></tr><tr><td>Pallagani et al. [27]</td><td>Multi Disease</td><td>99.24</td><td>No</td></tr><tr><td><strong>Current paper</strong></td><td><strong>Multi Disease</strong></td><td><strong>98.58</strong></td><td><strong>Yes</strong></td></tr></tbody></table>

## 7 Conclusion and Future Work

Plant disease is one of the major causes of crop damage. It stalls a plant's growth and prevents plants from reaching their full potential. Hence, plant disease detection is important. However, to prevent the disease, farmers need to know the severity of the disease. Hence estimation of the damage is another important area of research to know the severity of the disease. Our proposed *aGROdet* could be a useful component to smart village initiatives. In this paper:

- We proposed a plant disease detection system, *aGROdet*, for plant disease detection and leaf damage estimation.

- We evaluated our system through various performance metrics. *aGROdet* has a very high success rate in detecting disease and estimating leaf damage.

- Even when there are shadows in the image, *aGROdet* accurately calculates the damage.

- *aGROdet* accurately estimates damage, even in the presence of some specular reflection. However, there are limitations to *aGROdet* which need further experimentation. In future work, these limitations are required to be addressed.

- [11] has images of single leaves. In reality, when the images are taken with a mobile phone camera or UAV, there will be several leaves in the same image. Hence, a single leaf image needs to be detected from the shot image before applying *aGROdet*.

- As previously stated, *aGROdet* does not estimate damage in variegated leaves. Inclusion of these plants' damage estimates would be a good addition.

- Extent of damage is another area that needs attention.

- Disease can appear in any part of the plant. Here, only the top of the leaves are considered. In the future, other parts of the plants affected by disease need to be considered too.

- More work on the removal of shadows and specular reflections is needed. This will increase the accuracy of damage estimation.

- The presence of pests on the leaf has not been considered. Inclusion of damage estimation in the presence of the pest would be an interesting task too.
- Finally, more publicly available datasets will be an important addition to this research. Clean and more informational datasets will orchestrate the progress of data-centric AI initiatives.

## References

1. IEEE Smart Village Map. https://smartvillage.ieee.org/our-projects/, Accessed April 6, 2022

2. Plant Disease: Pathogens and Cycles. https://cropwatch.unl.edu/soybean-management/plant-disease, Accessed March 31, 2022

3. Plant Diseases. https://www.ars.usda.gov/crop-production-and-protection/plant-diseases/docs/action-plan-2022-2026/, Accessed April 4, 2022

4. Bhatt, P.V., Sarangi, S., Pappula, S.: Detection of diseases and pests on images captured in uncontrolled conditions from tea plantations. In: Autonomous Air and Ground Sensing Systems for Agricultural Optimization and Phenotyping IV. vol. 11008, p. 1100808. International Society for Optics and Photonics (2019)

5. Bhimte, N.R., Thool, V.: Diseases detection of cotton leaf spot using image processing and SVM classifier. In: Proc. Second International Conference on Intelligent Computing and Control Systems (ICICCS). pp. 340–344 (2018)

6. Brahimi, M., Mahmoudi, S., Boukhalfa, K., Moussaoui, A.: Deep interpretable architecture for plant diseases classification. In: Proc. Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA). pp. 111–116 (2019)

7. Chopda, J., Raveshiya, H., Nakum, S., Nakrani, V.: Cotton crop disease detection using decision tree classifier. In: Proc. International Conference on Smart City and Emerging Technology (ICSCET). pp. 1–5 (2018)

8. Degada, A., Thapliyal, H., Mohanty, S.P.: Smart Village: An IoT Based Digital Transformation. In: Proc. IEEE 7th World Forum on Internet of Things (WF-IoT). pp. 459–463 (2021)

9. Ferentinos, K.P.: Deep learning models for plant disease detection and diagnosis. Computers and Electronics in Agriculture 145, 311–318 (2018). https://doi.org/https://doi.org/10.1016/j.compag.2018.01.009, https://www.sciencedirect.com/science/article/pii/S0168169917311742

10. Hasan, M.J., Mahbub, S., Alom, M.S., Nasim, M.A.: Rice disease identification and classification by integrating support vector machine with deep convolutional neural network. In: Proc. 1st international conference on advances in science, engineering and robotics technology (ICASERT). pp. 1–6 (2019)

11. Hughes, D.P., Salathé, M.: An open access repository of images on plant health to enable the development of mobile disease diagnostics through machine learning and crowdsourcing. CoRR abs/1511.08060 (2015), http://arxiv.org/abs/1511.08060

12. Ji, M., Zhang, K., Wu, Q., Deng, Z.: Multi-label learning for crop leaf diseases recognition and severity estimation based on convolutional neural networks. Soft Computing 24(20), 15327–15340 (2020)

13. Ji, M., Zhang, L., Wu, Q.: Automatic grape leaf diseases identification via unitedmodel based on multiple convolutional neural networks. Information Processing in Agriculture 7(3), 418–426 (2020). https://doi.org/https://doi.org/10.1016/j.inpa.2019.10.003, https://www.sciencedirect.com/science/article/pii/S2214317319301003

14. Kerkech, M., Hafiane, A., Canals, R.: Vine disease detection in UAV multispectral images using optimized image registration and deep learning segmentation approach. Computers and Electronics in Agriculture 174, 105446 (2020)

15. Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization (2014). https://doi.org/10.48550/ARXIV.1412.6980, https://arxiv.org/abs/1412.6980

16. Liang, Q., Xiang, S., Hu, Y., Coppola, G., Zhang, D., Sun, W.: PD2SE-Net: Computer-assisted plant disease diagnosis and severity estimation network. Computers and Electronics in Agriculture 157, 518–529 (2019). https://doi.org/https://doi.org/10.1016/j.compag.2019.01.034, https://www.sciencedirect.com/science/article/pii/S0168169918318313
17. Lin, K., Gong, L., Huang, Y., Liu, C., Pan, J.: Deep Learning-Based Segmentation and Quantification of Cucumber Powdery Mildew Using Convolutional Neural Network. Frontiers in Plant Science **10** (2019). https://doi.org/10.3389/fpls.2019.00155, https://www.frontiersin.org/article/10.3389/fpls.2019.00155

18. Liu, J., Wang, X.: Plant diseases and pests detection based on deep learning: a review. Plant Methods **17**(22) (2021). https://doi.org/10.1186/s13007-021-00722-9, https://doi.org/10.1186/s13007-021-00722-9

19. Mitra, A., Vangipuram, S.L.T., Bapatla, A.K., Bathalapalli, V.K.V.V., Mohanty, S.P., Kougianos, E., Ray, C.: Everything You wanted to Know about Smart Agriculture. arXiv Computer Science arXiv:2201.04754 p. 45 pages (Jan 2022)

20. Mitra, A., Mohanty, S.P., Corcoran, P., Kougianos, E.: A Machine Learning Based Approach for Deepfake Detection in Social Media Through Key Video Frame Extraction. SN Computer Science **2**(2), 98 (2021). https://doi.org/10.1007/s42979-021-00495-x

21. Mitra, A., Singhal, A., Mohanty, S.P., Kougianos, E., Ray, C.: eCrop: A Novel Framework for Automatic Crop Damage Estimation in Smart Agriculture. SN Computer Science **3**(4), 1–16 (2022)

22. Mohanty, S.P., Hughes, D.P., Salathé, M.: Using Deep Learning for Image-Based Plant Disease Detection. Frontiers in Plant Science **7** (2016). https://doi.org/10.3389/fpls.2016.01419, https://www.frontiersin.org/article/10.3389/fpls.2016.01419

23. Nagasubramanian, K., Jones, S., Singh, A.K.e.a.: Plant disease identification using ex-
plainable 3D deep learning on hyperspectral images. Plant Methods **15**, 98 (2019).
https://doi.org/10.1186/s13007-019-0479-8

24. Narmadha, R., Arulvadivu, G.: Detection and measurement of paddy leaf disease symptoms using image processing. In: Proc. International Conference on Computer Communication and Informatics (ICCCI). pp. 1–4 (2017)

25. National Academies of Sciences, Engineering and Medicine and others: Science breakthroughs to advance food and agricultural research by 2030. National Academies Press (2019)

26. Ozguven, M.M., Adem, K.: Automatic detection and classification of leaf spot disease in sugar beet using deep learning algorithms. Physica A: statistical mechanics and its applications **535**, 122537 (2019)

27. Pallagani, V., Khandelwal, V., Chandra, B., Udutalapally, V., Das, D., Mohanty, S.P.: DCrop: A deep-learning based framework for accurate prediction of diseases of crops in smart agriculture. In: Proc. IEEE International Symposium on Smart Electronic Systems (iSES). pp. 29–33 (2019)

28. Parikh, A., Raval, M.S., Parmar, C., Chaudhary, S.: Disease detection and severity estimation in cotton plant from unconstrained images. In: Proc. IEEE international conference on data science and advanced analytics (DSAA). pp. 594–601 (2016)

29. Rother, C., Kolmogorov, V., Blake, A.: "GrabCut": Interactive Foreground Extraction Using Iterated Graph Cuts. ACM Trans. Graph. **23**(3), 309–314 (aug 2004). https://doi.org/10.1145/1015706.1015720, https://doi.org/10.1145/1015706.1015720

30. Saleem, M.H., Khanchi, S., Potgieter, J., Arif, K.M.: Image-Based Plant Disease Identification by Deep Learning Meta-Architectures. Plants **9**(11) (2020). https://doi.org/10.3390/plants9111451, https://www.mdpi.com/2223-7747/9/11/1451

31. Singh, V., Misra, A.: Detection of plant leaf diseases using image segmentation and soft computing techniques. Information Processing in Agriculture **4**(1), 41–49 (2017). https://doi.org/https://doi.org/10.1016/j.inpa.2016.10.005, https://www.sciencedirect.com/science/article/pii/S2214317316300154

32. Stevenson, J.A.: Plant Pathology, an Advanced Treatise, vol. 3, J. G. Horsfall and A. E. Dimond, Eds. Academic Press, New York, 1960

33. Stewart, E.L., Wiesner-Hanks, T., Kaczmar, N., DeChant, C., Wu, H., Lipson, H., Nelson, R.J., Gore, M.A.: Quantitative phenotyping of Northern Leaf Blight in UAV images using deep learning. Remote Sensing **11**(19), 2209 (2019)

34. Sun, J., Yang, Y., He, X., Wu, X.: Northern maize leaf blight detection under complex field environment based on deep learning. IEEE Access **8**, 33679–33688 (2020)

35. Thenmozhi, K., Reddy, U.S.: Crop pest classification based on deep convolutional neural network and transfer learning. Computers and Electronics in Agriculture **164**, 104906 (2019)
36. Udutalapally, V., Mohanty, S.P., Pallagani, V., Khandelwal, V.: sCrop: A novel device for sustainable automatic disease prediction, crop selection, and irrigation in internet-of-agro-things for smart agriculture. IEEE Sensors Journal (2020)

37. Wang, B.: Identification of Crop Diseases and Insect Pests Based on Deep Learning. Scientific Programming **2022**, Article ID **9179998**, 10 pages (2022). https://doi.org/10.1155/2022/9179998

38. Wiesner-Hanks, T., Wu, H., Stewart, E., DeChant, C., Kaczmar, N., Lipson, H., Gore, M.A., Nelson, R.J.: Millimeter-level plant disease detection from aerial photographs via deep learning and crowdsourced data. Frontiers in Plant Science **10**, 1550 (2019)

39. Xavier, T.W.F., Souto, R.N.V., Statella, T., Galbieri, R., Santos, E.S., S. Suli, G., Zeilhofer, P.: Identification of Ramularia Leaf Blight Cotton Disease Infection Levels by Multispectral, Multiscale UAV Imagery. Drones **3**(2) (2019). https://doi.org/10.3390/drones3020033, https://www.mdpi.com/2504-446X/3/2/33