[image] [image] Ressource protégée - Inria - Institut national de recherche en sciences et technologies du numérique **Accéder directement au contenu**

[image] [image]

* Documentation

* FR
  Français (FR)

* Anglais (EN)
  Se connecter

* Portail HAL Inria

* Recherche
  Loading...

## Recherche avancée

Information de documents

Titres

○ Titres

○ Sous-titre

○ Titre de l'ouvrage

○ Titre du volume (Série)

○ Champ de recherche par défaut (multicritères) + texte intégral des PDF

○ Résumé

○ Texte intégral indexé des documents PDF

○ Mots-clés

○ Type de document

○ Sous-type de document

○ Tous les identifiants du document

○ Identifiant HAL du dépôt

○ Langue du document (texte)

○ Pays (Texte)

○ Ville

○ À paraître (true ou false)

Ajouter
# Disease Identification in Tomato Leaf Using Pre-trained ResNet and Deformable Inception

Arnav Ahuja, Aditya Tulsyan, J. Jennifer Ranjani

<figure><img src="image_1.png" /></figure>

## To cite this version:

Arnav Ahuja, Aditya Tulsyan, J. Jennifer Ranjani. Disease Identification in Tomato Leaf Using Pre-trained ResNet and Deformable Inception. 5th International Conference on Computational Intelligence in Data Science (ICCIDS), Mar 2022, Virtual, India. pp.221-234, 10.1007/978-3-031-16364-7_17. hal-04381303

HAL Id: hal-04381303

https://inria.hal.science/hal-04381303v1

Submitted on 9 Jan 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
## Auteur

### Auteur (multicritères)

○ Auteur (multicritères)

○ Auteur : Nom complet

○ Auteur : Nom de famille

○ Auteur : Prénom

○ Auteur : Complément de nom, deuxième prénom

○ Auteur : Organisme payeur

○ Auteur : IdHal (chaîne de caractères)

○ Auteur : Fonction

○ Auteur : personID (entier)

○ Auteur : Identifiant de l'organisme payeur

○ Auteur : Identifiant de la structure

○ Directeur de thèse

○ Éditeur

○ Éditeur scientifique

○ Éditeur de la série

## Ajouter

### Structure

#### Structure (multicritères)

○ Structure (multicritères)

○ Structure : Acronyme

○ Structure : Nom

○ Structure : Code

○ Structure : Pays

○ Structure : Type

○ Structure : État dans le référentiel

○ Structure : Identifiant HAL de la structure

○ Structure/équipe de recherche : Pays

○ Structure/regroupement d'équipes : Pays

○ Structure/laboratoire : Pays

○ Structure/regroupement de laboratoires : Pays

○ Structure/institution : Pays

○ Structure/regroupement d'institutions : Pays

## Ajouter
<figure><img src="image_3.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
Autres

Champ de recherche par défaut (multicritères)

○ <u>Champ de recherche par défaut (multicritères)</u>

○

○ <u>Revues (multicritères)</u>

○ <u>Revue : Éditeur</u>

○ <u>Revue : Titre abrégé</u>

○ <u>Revue : Titre</u>

○ <u>Revue : Identifiant interne</u>

○ <u>Revue : Date de début de publication</u>

○ <u>Revue : ISSN électronique</u>

○ <u>Revue : ISSN</u>

○ <u>Revue : Éditeur</u>

○ <u>Revue : Couleur dans SHERPA/RoMEO</u>

○ <u>Revue : État dans le référentiel</u>

○

○ <u>Colloque (multicritères)</u>

○ <u>Colloque : Titre</u>

○ <u>Colloque : Organisateur</u>

○ <u>Colloque : date de début (Année)</u>

○ <u>Colloque : date de fin (Année)</u>

○ <u>Conférence invitée (oui/non)</u>

○

○ <u>Projets ANR (multicritères)</u>

○ <u>Projet ANR : Acronyme</u>

○ <u>Projet ANR : Acronyme du programme</u>

○ <u>Projet ANR : Code décision (référence)</u>

○ <u>Projet ANR : Nom</u>

○ <u>Projet ANR : Identifiant interne</u>

○ <u>Projet ANR : État dans le référentiel</u>

○

○ <u>Projets européens (multicritères)</u>

○ <u>Projet européen : Acronyme</u>

○ <u>Projet européen : Identifiant de l'appel à projet</u>

○ <u>Projet européen : Référence</u>

○ <u>Projet européen : Nom</u>

○ <u>Projet européen : Date de fin</u>

○ <u>Projet européen : Financement</u>

○ <u>Projet européen : Date de début</u>
# Disease Identification in Tomato Leaf using pre-trained ResNet and Deformable Inception

Arnav Ahuja¹, Aditya Tulsyan¹, and J. Jennifer Ranjani²

¹ Department of Computer Science and Information Systems and Department of Mathematics, Birla Institute of Technology and Science, Pilani, India - 333031

² Virufy, United States j.jenniferranjani@yahoo.co.in

**Abstract.** Disease in crops is a growing concern for the food and farm-
ing industry. In the last couple of decades, lack of immunity in plants and
extreme climate conditions due to drastic climate changes have caused a
significant increase in the growth of diseases among crops. These cause
large-scale demolition of crops, decrease in cultivation, and eventually
financial loss of farmers. Fast and early detection of the disease remains
difficult in most parts of the world because of the lack of proper research
infrastructure. Automated techniques for disease detection are high in
demand with a Worldwide increase in image capturing and video record-
ing devices, together with the continuous evolution in computer vision
and machine learning. In this paper, we propose a deep learning model
trained using a publicly available dataset containing images of healthy
as well as diseased crop leaves. The model adopts a new approach com-
bining the ResNet and the InceptionNet architectures. Skip connections
and the 1 × 1 convolutions in both the architectures are put to good use
here. We achieve an accuracy of 99.08% in the PlantVillage dataset [17]
and reasonable accuracy of 66.06% on the PlantDoc dataset [16] which
is an increment of more than 25% from the approaches in the previous
works. This paper also suggests a method to improve the detection of
diseases in crops in the real world by augmenting the number of data
points. We have discussed the use of deformable convolution, which is
capable of learning various geometric transformations and can improve
the performance of the architecture. The model serves its objective by
classifying images of leaves into diseased categories based on the pattern
of the defect.

**Keywords:** Crop disease detection and classification, Tomato leaf, Convolu-
tional Neural Network, Deep Learning, Computer Vision, Residual Network,
ResNet, Deformable Convolution.

# 1 Introduction

Agriculture is the key source of food, raw material, and fuel which contributes to the economic development of any nation. More than two-thirds of the population all over the globe depend on agriculture directly or indirectly. Food security remains threatened by various circumstances including, lack of irrigation, climate
○ Projet européen : État dans le référentiel

○ Projet européen : Identifiant interne

○

○ Date de production : année

○ Date de mise en ligne : année

○ Date de publication : année

○ Date d'écriture : année

○ Date de modification du dépôt : année

○ Date de dépôt : année

○ Date de publication électronique : année

○

○ Collection HAL (multicritères)

○ Collection HAL : catégorie

○ Collection HAL : Code

○ Collection HAL : Nom

○ Collection HAL : Identifiant interne

○

○ Identifiant interne du contributeur/déposant

○ Nom complet du contributeur/déposant

○

○ Domaines

○ Domaine primaire

○ Domaine racine

○ Sous-domaine niveau 1

○ Sous-domaine niveau 2

○ Sous-domaine niveau 3

○

○ Statut du document

○ Version du document

○ Type de dépôt

○ Type de document

○ ISBN

○ Numéro - référence

○ Identifiant DOI

○

○ Classification

○ Audience

○ Vulgarisation

○ Comité de lecture - texte (oui ou non)

○ Actes de colloque
change, the decline in pollinators, crop diseases, etc. Crop disease alleviates the production and also quality of food. Plant diseases and pests have become a vital challenge for the agriculture sector. Through this work, we focus on recognizing diseases that affect tomato plants. Tomato is economically one of the most vital crops not only in India but across the globe, and production of tomato has significantly increased through the years [1]. The worldwide cultivation of tomatoes exposes the crop to a wide range of new pathogens instilling new diseases in crops of tomato. Many pathogens have found this crop to be highly susceptible and essentially defenseless. Moreover, new viral diseases keep emerging [2] which drives a need to have a robust crop disease identification system.

## 2 Related Works

In the last few decades, multiple works were done by researchers in crop disease identification to identify the presence of pathogens accurately in different crops. Most models were able to attain quite a high accuracy but, a key concern is the ability of the models to work in non-laboratory i.e., in an environment like actual farmland. Among the previous work done by various researchers in this field, most publishers have used the PlantVillage dataset containing 54000 images from 14 different crops having over 25 plant diseases and taken in laboratory setup and in a particular time interval. In [3], transfer learning with the standard architectures was used, with a focus on the InceptionNet and MobileNet. Many approaches like [4] and [13] use the PlantVillage dataset in detecting the disease and have achieved good results. In [5], five different CNN architectures were used, namely, GoogLeNet, AlexNet, AlexNetOWTbn, VGG, and Overfeat architectures, in which VGG showcased the best results among all the architectures. Similarly in [18] the authors have compared architectures like ResNet, MobileNet, DenseNet and Inception (v3) for the PlantVillage dataset. They have done three types of classifications- binary classification (diseased or not diseased leaf), six class classification and 10 class classification. We use some of these models later for bench marking the performance of our architecture in comparison with the other architectures. For these papers, apart from the above-stated drawback of using images of plants clicked in laboratory conditions, the authors also used only one single architecture in the model and mainly did a hit and trial method to find the best architecture for various plants without giving out proper rationale. Each architecture comes with a set of disadvantages giving rise to the need for developing a model that works the best in most scenarios. In [6], seven different plant diseases were classified and recognized by three classifiers, Extreme Learning Machine, Support Vector Machines, and K-Nearest Neighbor, used with the state-of-the-art Deep Learning models (DL) like ResNet (50 and 101), Inception (v3), GoogLeNet, SqueezeNet, and InceptionResNetv2. A comparison was made between these models and ResNet-50 with the Support Vector Machines, which got the best results in terms of various performance metrics like specificity, sensitivity, and F1-score. In [7], plant diseases in cumber were classified by the two basic versions of CNN and got the highest
○ Référence interne

○ Financement

○ Collaborations

Ajouter

Lancer la recherche
Recherche experte (SolR)

Recherche experte (SolR)

Lancer la recherche
Vers la recherche avancée

• Déposer

Portail INRIA.HAL.SCIENCE
Les publications, logiciels... des scientifiques Inria

• Accueil

• Publications Inria

○ Consulter les derniers dépôts

○ Par équipe de recherche

○ Par auteur

○ Rapports de recherche et techniques

○ Livres blancs

○ Rapports d'activité des équipes

○ Statistiques

• Boite à Outils

○ X2Hal : importer par lot

○ Logiciels : créer le Codemeta.json

○ Créer mon IDHal

○ Créer mon CVHal

○ Haltools : créer ma page de publications

○ Haltools : extraire des données (réservé Inria)

○ Les bonnes pratiques

■ Comment bien s'affilier ?
accuracy. According to [8], a new Deep Learning (DL) model Inception (v3) was used, for the detection of cassava disease. The traditional plant disease recognition and classification method were replaced by Super-Resolution Convolutional Neural Network (SRCNN) in [9]. For the classification of tomato plant disease, SqueezeNet v1.1, and AlexNet models were used in which AlexNet was found to be the better among the two DL models in terms of accuracy in the work referenced in [10]. A comparative analysis was presented in [11] to select the best DL architecture for the detection of plant diseases. And in [12], six different tomato plant diseases were classified by using VGG-16 and AlexNet DL architectures, and a detailed comparison was provided with the help of classification accuracy. Most researchers in all work mentioned above have either used the PlantVillage dataset or any other dataset containing images taken under laboratory conditions. Also, most publishers restrict their model to a particular architecture along with the conventional convolution technique that uses the kernel of fixed size as a filter. Other approaches like the deformable convolutional networks can improve the results of the models as shown in [14]. We have summarized the network used, dataset and metrics from various related works in table 1.

<table><caption>**Table 1.** Summary on Related Works</caption><thead><tr><th>Paper</th><th>Network Used</th><th>Dataset</th><th>Metrics</th></tr></thead><tbody><tr><td rowspan="2">Crop Disease Detection Using Deep Learning [3]</td><td>MobileNet</td><td>Plant Village</td><td>Accuracy = 99.6%</td></tr><tr><td>InceptionNet</td><td>Plant Village</td><td>Accuracy = 99.7%</td></tr><tr><td>ResNet based approach for Classification and Detection of Plant Leaf Diseases [4]</td><td>ResNet</td><td>Plant Village</td><td>Accuracy = 99.4%<br>Precision = 0.965</td></tr><tr><td rowspan="2">Tomato Crop Disease Classification using pre-trained deep learning algorithm [12]</td><td>AlexNet</td><td>Plant Village</td><td>Accuracy = 97.49%</td></tr><tr><td>VGG-16</td><td>Plant Village</td><td>Accuracy = 97.23%</td></tr><tr><td rowspan="4">Tomato Leaf Diseases Detection Using Deep Learning Technique [18]</td><td>ResNet</td><td>Plant Village</td><td>Accuracy = 96.75%<br>Precision = 0.968</td></tr><tr><td>MobileNet</td><td>Plant Village</td><td>Accuracy = 97.2%<br>Precision = 0.972</td></tr><tr><td>DenseNet</td><td>Plant Village</td><td>Accuracy = 98.05%<br>Precision = 0.980</td></tr><tr><td>Inception (v3)</td><td>Plant Village</td><td>Accuracy = 97.35%<br>Precision = 0.973</td></tr><tr><td>Deep Learning for Image Based Cassava Disease Detection [8]</td><td>Inception (v3)</td><td>Custom Cassava Leaves Dataset</td><td>Accuracy = 92.9%</td></tr><tr><td rowspan="2">Disease Detection on the leaves of Tomato plants by using deep learning [10]</td><td>AlexNet</td><td>Plant Village</td><td>Accuracy = 95.65%</td></tr><tr><td>SqueezeNet</td><td>Plant Village</td><td>Accuracy = 94.3%</td></tr><tr><td rowspan="4">A comparative study of fine tuning learning models for plant disease identification [11]</td><td>VGGNet</td><td>Plant Village</td><td>Accuracy = 81.83%</td></tr><tr><td>ResNet</td><td>Plant Village</td><td>Accuracy = 99.5%</td></tr><tr><td>Inception</td><td>Plant Village</td><td>Accuracy = 98.08%</td></tr><tr><td>DenseNet</td><td>Plant Village</td><td>Accuracy = 99.6%</td></tr></tbody></table>
■ Quelle version puis-je déposer ?

■ Choisir le bon « type de document »

■ Mettre en valeur ses publications

○ Les tutos IES Inria

■ Lier les notices dans HAL

■ Umbrhal Inria pour l'accessibilité

■ Personnaliser son CVHAL

■ Créer et Personnaliser sa collection

• Documentation

○ Aide en ligne HAL

○ Aide en ligne Haltools

○ Aide en ligne X2hal

○ Tutoriels video HAL

○ Documentation API HAL

○ Ajouter des vignettes

○ Identifiant auteur IdHAL et CV

○ Compléter son dossier ORCID avec ses dépôts HAL

○ Inria et la Science Ouverte

X

X

X

Loading...

X

Le document que vous demandez est en access restreint
jusqu'au 2025-01-01 !

Revenir a la notice

Retourner à l'accueil

Contact

•

Ressources
The proposed model aims to address the issues and drawbacks mentioned above. Our model is trained on plant images from real farmland setup with varying angles, resolution, and plants in different stages of growth. Secondly, we tried merging various network architectures according to their strength and developed a model to classify tomato leaves accurately. The organization of the paper is as follows: Section 3 describes the proposed architecture, section 4 summarizes the two popular datasets along with the new mast dataset used in this work. Results from the various experiments conducted are summarized in section 5. Section 5 and 6 are on conclusion and future work, respectively.

## 3 Proposed Model

In this paper, we propose an architecture based on the Residual network, ResNet-50, and Inception network architecture along with deformable convolution [14] for accurate prediction of disease in any tomato plant. ResNet architecture comprises core building blocks, known as residual blocks. Skip connections in the residual block address the decreasing gradient problem in deep neural networks. In the process of training our tomato plant images over ResNet-50, the residual shortcuts guarantee us the network integrity in our architecture if the regular connection's coefficient converges to zero. These alternative connections improve the network by providing the option of choosing these shortcuts when required. Also, skip connection ensures that our model will learn an identity function which ensures that the higher layer will perform at least as good as the lower layer and not worse. By referring to Inception, we mean Inception (v1), also known as GoogLeNet. Since, salient feature in a plant image is critical in identifying the type of diseases like bacterial spot, early or late blight will vary largely in size, shape, and position for every plant leaf image. It might be possible that the entire leaf inside the image has the spatial feature required to identify and classify the disease, or there can be a very tiny spot of infection in the whole leaf. Hence, we need filters of varying sizes to accurately extract the desired feature from the tomato plant and classify it into the type of disease. Multiple filters for a single input map demand a wide network rather than a deep network. Inception (v1) networks offer three different sizes of filters (1 × 1, 3 × 3, 5 × 5), and hence it fits the best as per our needs. The output from every module is further concatenated and sent to the next Inception module. There are in total four such Inception modules in our architecture.

The major drawback of a typical Convolution Neural Network (CNN) is the fixed size of the kernel used to sample the feature map. The CNN kernel uses a fixed size, rectangular window to sample, the input feature map at fixed locations, and the pooling layer of the Inception module uses the same fixed size rectangular kernel to reduce the spatial resolution by a fixed ratio. This introduces various problems, such as all the activation units in a given CNN layer having the same receptive field even though there might be objects of different scales present at different spatial positions. Having different receptive field sizes for each object is often desirable for visual recognition tasks requiring fine localization such as
* Documentation

* FAQ

* API

* OAI-PMH

* AuréHAL

## Informations

* À propos

* Données personnelles

* Mentions légales

* Accessibilité

* Conformité RGAA

## Questions juridiques

* Je publie, quels sont mes droits ?

* Loi pour une République numérique

* Stratégie de non-cession des droits

* SHERPA/RoMEO

## Portails

* Portails institutionnels

* HAL

* HAL SHS

* HAL Thèses

* MédiHAL

## CCSD

* CCSD

* Episciences

* Sciencesconf
object detection, segmentation, etc, for adapting to the scale of the object. So along with the Inception network having kernels of various sizes, we have used deformable convolution over our architecture for more accurate extraction of features. Deformable convolutions add 2-dimensional offsets to the regular grid sampling locations in the standard convolution. It enables free-form deformation of the sampling grid. Offsets are learned from the preceding feature maps via additional convolutional layers. Thus the deformation is conditioned on the input features in a local, dense, and adaptive manner.

The ResNet-50 network comprises five stages, each with convolution and an iden-
tity block. Each convolution block has three convolution layers, and each iden-
tity block also has three convolution layers. The complete ResNet-50 has over
23 million trainable parameters. For our purpose, we have used a pre-trained
ResNet-50 trained on the ImageNet dataset [15] and have replaced the last stage
of ResNet-50 with three Inception (v1) blocks to extract salient features from
the image accurately. We trained our network using a deformable convolutional
technique for complete stage five i.e., Inception block with a varying number of
convolution layers for the fourth stage of the network. We have done an ablation
study in section 4 to decide the number of trainable parameters, which give the
most accurate and optimal result. A block diagram of the architecture is shown
in Fig 1. The first two layers of the architecture are the same as the first two
layers of the ResNet-50 model. The third layer of the architecture consists of 4
bottleneck layers of the ResNet-50 architecture as well as 2 Inception modules.
The fourth layer consists of 2 Inception modules and one bottleneck layer of the
ResNet-50 architecture. Finally there are avg pooling and fully connected layers.

4 Dataset

4.1 PlantVillage Dataset

The most widely used dataset for plant disease identification is the PlantVillage Dataset [17], which consists of a large number of, high-quality images taken under laboratory conditions. The dataset curators created an automated system using GoogleNet and AlexNet architecture for disease detection, achieving an accuracy of up to 99.35%. The dataset consists of 16,011 images in the tomato class, and these are the images we will use for classification in our model. Images in the PlantVillage dataset are taken in the laboratory setup and have a uniform, white background. These images are not captured under natural conditions like cultivation fields, due to which their efficacy in the real world is likely to be poor.

4.2 PlantDoc Dataset

In contrast, we have used the PlantDoc Dataset [16] containing images of healthy
and diseased plants in real farmland scenarios. PlantDoc is comparatively close
to the application that can help us use our model and identify diseases in crops
and solve degradation of quality in plants. PlantDoc dataset originally contains
<figure><img src="image_4.png" /><figcaption>Fig. 1. Proposed InceptionResNet Architecture for Crop Disease Identification</figcaption></figure>

<table><caption>Table 2. Number of Training and Testing Images</caption><thead><tr><th rowspan="2">Serial No.</th><th rowspan="2">Class Name</th><th colspan="2">PlantVillage</th><th colspan="2">PlantDoc</th></tr><tr><th>Train Data</th><th>Test Data</th><th>Train Data</th><th>Test Data</th></tr></thead><tbody><tr><td>1</td><td>Leaf Mold</td><td>809</td><td>143</td><td>85</td><td>6</td></tr><tr><td>2</td><td>Bacterial Spot</td><td>1808</td><td>319</td><td>101</td><td>9</td></tr><tr><td>3</td><td>Septoria Spot</td><td>1505</td><td>266</td><td>140</td><td>11</td></tr><tr><td>4</td><td>Early Blight</td><td>850</td><td>150</td><td>78</td><td>9</td></tr><tr><td>5</td><td>Late Blight</td><td>1623</td><td>286</td><td>101</td><td>10</td></tr><tr><td>6</td><td>Yellow Virus</td><td>2728</td><td>481</td><td>69</td><td>6</td></tr><tr><td>7</td><td>Mosaic Virus</td><td>317</td><td>56</td><td>44</td><td>10</td></tr><tr><td>8</td><td>Healthy</td><td>1352</td><td>239</td><td>54</td><td>8</td></tr></tbody></table>

2,598 data points, in total, across 13 plant species and up to 17 classes of diseases. For this paper, we restricted our study to one plant species i.e., tomato leaf, with eight classes containing one class for healthy leaves while the other seven with the diseased leaves. Out of 741 data points for tomato leaves, we segregated 672 data points for training our model and the remaining for testing the model i.e., 90%-10% split. Table 2 contains the exact distribution of tomato leaf for testing and training across different diseases.
Fig 2 contains samples from various classes in both the PlantVillage (PVD) and PlantDoc dataset.

<figure><img src="image_5.png" /><figcaption>**Fig. 2.** Samples from various classes to show gap between lab-controlled and real-life images</figcaption></figure>

## 4.3 New Mask Dataset using Data Augmentation

The PlantDoc dataset is a small dataset with about 700 images, hence, training a large model becomes difficult. Training only the PlantDoc dataset has led to lower accuracy in our models, as is evident in tables 3 and 4, because of over-fitting in the images due to the small dataset size. However, the PlantDoc dataset mimics the images taken in real conditions, and hence it is necessary to get good accuracy in this dataset. To solve these two approaches were used—

*   **Approach 1** - Basic augmentation like a random rotation of the image, flipping of the image, etc, was used. The torch transform library was used in performing the random rotation and image flipping. Then, these images were used in random batches to train the models. However, this did not drive the accuracy significantly up, as the number of images originally was comparatively less for the parameters to learn any general trend. The need for a larger dataset arose.
*   **Approach 2** - To solve this problem the PlantVillage dataset, which had an abundance of images, was put to use. Images from the PlantVillage dataset are superimposed on the PlantDoc dataset. Images of the same class are superimposed on top of each other, i.e., images of the yellow virus class of the PlantVillage dataset are superimposed on images with the yellow virus in the PlantDoc dataset. This led to the formation of the augmented dataset consisting of 14,000 images. These images were now used to train the ResNet and the InceptionResNet models. Samples from the augmented dataset are given in Fig 3
<figure><img src="image_6.png" /><figcaption>Fig. 3. Samples from various classes in the New Mask Dataset</figcaption></figure>

5 Experimental Results

In this section, we verify the performance of our model in identifying diseases
in tomato leaf and summarize the experiments conducted together with the
results. We have provided the code for further experiment and research <fnref n="1" />. The
experimental analysis is summarized in several sections. In the first few sections,
we have done an ablation study on the number of layers needed to be trained and
the hyperparameters that give the most optimal result. Next, we have showcased
the final result obtained from the conclusion of the ablation study.

<figure><img src="image_7.png" /><figcaption>Fig. 4. Loss vs Iteration for the InceptionResNet model</figcaption></figure>

<footnote n="1">¹ github.com/arnavahuja/CropDiseaseIdentification</footnote>
## 5.1 Training

We have trained the model in random batches of a specific size for all three datasets. This training process takes place for the number of epochs passed as a parameter in the train function. The loss vs. iterations graph in fig 4 gives a snapshot of the neural network after every ten iterations, and it gives an idea of the performance of the network during training. Since the loss vs. iterations graph smoothly decreases during the iterations, it suggests the correctness of the model.

<table><caption>**Table 3.** Results for ResNet-50 Architecture</caption><thead><tr><th>Architecture Name</th><th>Description</th><th>Dataset</th><th>Accuracy</th></tr></thead><tbody><tr><td>ResNet</td><td>Fully connected layer unfreezed</td><td>PlantVillage</td><td>0.726741</td></tr><tr><td>ResNet</td><td>Fully connected and one convolutional layer unfreezed</td><td>PlantVillage</td><td>0.925436</td></tr><tr><td>ResNet</td><td>Fully Connected Layer and Two Conv Layer Unfreezed</td><td>PlantVillage</td><td>0.975007</td></tr><tr><td>ResNet</td><td>Fully Connected Layer unfreezed</td><td>PlantDoc</td><td>0.261029</td></tr><tr><td>ResNet</td><td>Fully connected and one convolutional layer unfreezed</td><td>PlantDoc</td><td>0.295189</td></tr><tr><td>ResNet</td><td>Fully Connected Layer and Two Conv Layer Unfreezed</td><td>PlantDoc</td><td>0.272518</td></tr><tr><td>ResNet</td><td>Fully connected layer unfreezed</td><td>New Mask Dataset</td><td>0.74499(NewData) 0.330968(PD)</td></tr><tr><td>ResNet</td><td>Fully connected and one convolutional layer unfreezed</td><td>New Mask Dataset</td><td>0.83155(NewData) 0.440379(PD)</td></tr><tr><td>ResNet</td><td>Fully Connected Layer and Two Conv Layer Unfreezed</td><td>New Mask Dataset</td><td>0.96711(NewData) 0.624149(PD)</td></tr><tr><td>ResNet</td><td>All layers unfreezed</td><td>PlantVillage</td><td>0.979115</td></tr><tr><td>ResNet</td><td>All layers unfreezed</td><td>PlantDoc</td><td>0.265165</td></tr><tr><td>ResNet</td><td>All layers unfreezed</td><td>New Mask Dataset</td><td>0.99179(NewData) 0.652945(PD)</td></tr></tbody></table>
## 5.2 Performance Evaluation

The calculate accuracy function takes the batch size as a parameter and gets a random batch from the validation data. The model then predicts the label of the validation data and then the number of correct labels is calculated. This accuracy is averaged over the entire validation dataset.

<table><caption>**Table 4.** Results for the InceptionResNet Architecture</caption><thead><tr><td>Architecture Name</td><td>Description</td><td>Dataset</td><td>Accuracy</td></tr></thead><tbody><tr><td>InceptionResNet</td><td>Inception Blocks in the layer i.e layer 4</td><td>PlantVillage</td><td>0.981685</td></tr><tr><td>InceptionResNet</td><td>Inception Blocks in the layer i.e layer 4</td><td>PlantDoc</td><td>0.424019</td></tr><tr><td>InceptionResNet</td><td>Inception Blocks in the layer i.e layer 4</td><td>New Mask Dataset</td><td>0.95889(NewData) 0.533121(PD)</td></tr><tr><td>InceptionResNet</td><td>All layers unfreezed, along with bottleneck 0,1,2 of layer 4 changed to inception block</td><td>PlantVillage</td><td>0.990011</td></tr><tr><td>InceptionResNet</td><td>All layers unfreezed, along with bottleneck 0,1,2 of layer 4 changed to inception block</td><td>PlantDoc</td><td>0.589996</td></tr><tr><td>InceptionResNet</td><td>All layers unfreezed, along with bottleneck 4,5 of layer 3 and bottleneck 0,1 of layer 4 changed to inception block</td><td>New Mask Dataset</td><td>0.99086(NewData) 0.660636(PD)</td></tr></tbody></table>

## 5.3 Ablation Study on the Neural Network Architecture

The impact of various hyper-parameters on the architectures was experimented with, and we have tabulated their results in table 3 and table 4.

In table 4, we have summarized the results for the InceptionResNet architecture, which has produced promising results. It has given an accuracy of 66.036%, which is greater than the previously achieved accuracy. Thus, it testifies the success of the InceptionResNet architecture and the new augmented dataset.

The success of this architecture is due to the unique 1 × 1 convolutions in the Inception block along with the skip connections in the ResNet. The 1 × 1 convolutions ensure that the number of trainable parameters does not significantly increase while increasing the number of kernels.
## 5.4 Benchmark Results

Benchmark results are a test of how our model performs in comparison with others.

The benchmark used here is the accuracy for the standard architectures in the same situations. The standard architectures like the VGG, ResNet, Alexnet, XceptionNet all give lower accuracies than our proposed architecture Inception-ResNet.

<table><caption>**Table 5.** Benchmark Results of Standard Architectures</caption><thead><tr><th>Architecture Name</th><th>Description</th><th>Dataset</th><th>Accuracy</th></tr></thead><tbody><tr><td>AlexNet</td><td>Trained entire network due to its small size</td><td>PlantVillage</td><td>0.983786</td></tr><tr><td>AlexNet</td><td>Trained entire network due to its small size</td><td>PlantDoc</td><td>0.508655</td></tr><tr><td>AlexNet</td><td>Trained entire network due to its small size</td><td>New Mask Dataset</td><td>0.96529(NewData)<br>0.504191(PD)</td></tr><tr><td>VGG</td><td>Trained the entire classifier of the VGG which consists of all fully connected layers</td><td>PlantVillage</td><td>0.800348</td></tr><tr><td>VGG</td><td>Trained the entire classifier of the VGG which consists of all fully connected layers</td><td>PlantDoc</td><td>0.413526</td></tr><tr><td>VGG</td><td>Trained the entire classifier of the VGG which consists of all fully connected layers</td><td>New Mask Dataset</td><td>0.73766(NewData)<br>0.281480(PD)</td></tr><tr><td>Xception</td><td>Trained last block (no. 12) of XceptionNet)<br>-68,01,706 params</td><td>PlantVillage</td><td>0.980904</td></tr><tr><td>Xception</td><td>Trained last block (no. 12) of XceptionNet)<br>-68,01,706 params</td><td>PlantDoc</td><td>0.477558</td></tr><tr><td>Xception</td><td>Trained last block (no. 12) of XceptionNet)<br>-68,01,706 params</td><td>New Mask Dataset</td><td>0.84175(NewData)<br>0.460062(PD)</td></tr></tbody></table>

## 5.5 Hyperparameter Study

Table 6 illustrates the ablation study done on different hyper-parameters like learning and batch size for both the PlantVillage and the new mask dataset.
We have experimented with these hyperparameters for both the ResNet and the InceptionResNet architectures. The number of epochs used was 1500 for all the experiments. Only the results of the PlantVillage dataset and the new mask dataset are relevant here as the PlantDoc dataset has low accuracy in itself and does not require a hyperparameter study.

<table><caption>**Table 6.** Hyperparameter Study</caption><thead><tr><th>Model</th><th>Dataset</th><th>Learning Rate</th><th>Batch Size</th><th>Accuracy</th></tr></thead><tbody><tr><td rowspan="10">ResNet</td><td rowspan="4">PlantVillage</td><td>0.0001</td><td>64</td><td>0.958135</td></tr><tr><td>0.0001</td><td>128</td><td>0.964829</td></tr><tr><td>0.00001</td><td>64</td><td>0.971923</td></tr><tr><td>0.00001</td><td>128</td><td>0.979115</td></tr><tr><td rowspan="4">New Mask Dataset</td><td>0.0001</td><td>64</td><td>0.964921</td></tr><tr><td>0.0001</td><td>128</td><td>0.991799</td></tr><tr><td>0.00001</td><td>64</td><td>0.984289</td></tr><tr><td>0.00001</td><td>128</td><td>0.974923</td></tr><tr><td rowspan="10">Inception ResNet</td><td rowspan="4">PlantVillage</td><td>0.0001</td><td>64</td><td>0.990011</td></tr><tr><td>0.0001</td><td>128</td><td>0.989294</td></tr><tr><td>0.00001</td><td>64</td><td>0.969211</td></tr><tr><td>0.00001</td><td>128</td><td>0.975479</td></tr><tr><td rowspan="4">New Mask Dataset</td><td>0.0001</td><td>64</td><td>0.980857</td></tr><tr><td>0.0001</td><td>128</td><td>0.968935</td></tr><tr><td>0.00001</td><td>64</td><td>0.971294</td></tr><tr><td>0.00001</td><td>128</td><td>0.990868</td></tr></tbody></table>

# 6 Conclusion

In this paper, we propose a novel neural network model based on standard convolution and deformable convolution techniques to identify disease in the Tomato plant in a real-time manner. The model we developed was trained and tested over images taken in actual farmland conditions and its augmented form to make our model more robust.

We expect that our proposed system will make a significant contribution to the agriculture research area and crucially help farmers across the globe to solve the problem of infections in plants, especially fruits and vegetable bearing plants. As informed to our readers, at the beginning of this paper, more than 30 percent of the farm produce becomes trash because of various diseases. Hence, through this paper, we are hopeful that our work will help solve the problem of food scarcity in several underdeveloped nations that eventually leads to malnutrition among children.
# 7 Future Work

In future work, these models can be trained entirely using deformable convolutions. Deformable convolutions offer an advantage over traditional convolutions as they use learnable parameters, and the shape of the convolution kernels is not fixed. However, due to our limitations in terms of computational capacity, we were able to train only the last layers in the deformable models. Thus in the future, we can train the entire network with deformable convolutions from scratch, as the initial results look promising. Also, our work can be extended to various other groups of plant species apart from tomatoes. Moreover, numerous diseases exist, which we could not cover due to dataset limitations. Hence, our model can be made more robust and much more exhaustive.

# Acknowledgement

This work was carried out when Jennifer Ranjani J was affiliated to Birla Institute of Technology and Science, Pilani Campus, Pilani, Rajasthan - 333031.

# References

1. Food and Agriculture Organization of the United Nations. Value of Agricultural Production-Tomatoes. Food and Agriculture Data. 2015. Available online: http://www.fao.org/faostat/en/#data/QV/visualize

2. Hanssen, I.; Lapidot, M.; Thomma, B. Emerging Viral Diseases of Tomato Crops. Mol. Plant Microbe Interact. 2010, 23, 539–548.

3. O. Kulkarni, "Crop Disease Detection Using Deep Learning," 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA), Pune, India, 2018, pp. 1-4, doi: 10.1109/ICCUBEA.2018.8697390.

4. V. Kumar, H. Arora, Harsh and J. Sisodia, "ResNet-based approach for Detection and Classification of Plant Leaf Diseases," 2020 International Conference on Electronics and Sustainable Communication Systems (ICESC), Coimbatore, India, 2020, pp. 495-502, doi: 10.1109/ICESC48915.2020.9155585.

5. Ferentinos, Konstantinos. (2018). Deep learning models for plant disease detection and diagnosis. Computers and Electronics in Agriculture. 145. 311-318. 10.1016/j.compag.2018.01.009.

6. türkoğlu, Muammer and Hanbay, Davut. (2019). Plant disease and pest detection using deep learning-based features. TURKISH JOURNAL OF ELECTRICAL ENGINEERING and COMPUTER SCIENCES. 27. 1636-1651. 10.3906/elk-1809-181.

7. Fujita, E.; Kawasaki, Y.; Uga, H.; Kagiwada, S.; Iyatomi, H. Basic investigation on a robust and practical plant diagnostic system. In Proceedings of the 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), Anaheim, CA, USA, 18–20 December 2016; pp. 989–992.

8. Ramcharan, A.; Baranowski, K.; McCloskey, P.; Ahmed, B.; Legg, J.; Hughes, D.P. Deep learning for image-based cassava disease detection. Front. Plant Sci. 2017, 8, 1852.

9. Yamamoto, K.; Togami, T.; Yamaguchi, N. Super-resolution of plant disease images for the acceleration of image-based phenotyping and vigor diagnosis in agriculture. Sensors 2017, 17, 2557.
10. Durmuş, H.; Güneş, E.O.; Kırcı, M. Disease detection on the leaves of the tomato plants by using deep learning. In Proceedings of the 2017 6th International Conference on Agro-Geoinformatics, Fairfax, VA, USA, 7–10 August 2017; pp. 1–5.

11. Too, E.C.; Yujian, L.; Njuki, S.; Yingchun, L. A comparative study of fine-tuning deep learning models for plant disease identification. Comput. Electron. Agric. 2019, 161, 272–279.

12. Aravind Krishnaswamy Rangarajan, Raja Purushothaman, Anirudh Ramesh, Tomato crop disease classification using pre-trained deep learning algorithm, Procedia Computer Science, Volume 133, 2018, Pages 1040-1047, ISSN 1877-0509, https://doi.org/10.1016/j.procs.2018.07.070.

13. Saleem MH, Potgieter J, Arif KM. Plant Disease Detection and Classification by Deep Learning. Plants. 2019; 8(11):468. https://doi.org/10.3390/plants8110468

14. J. Dai et al., "Deformable Convolutional Networks," 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017, pp. 764-773, doi: 10.1109/ICCV.2017.89.

15. J. Deng, W. Dong, R. Socher, L. J. Li, Kai Li and Li Fei-Fei, "ImageNet: A large-scale hierarchical image database," 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248-255, doi: 10.1109/CVPR.2009.5206848.

16. Singh, D., Jain, N., Jain, P., Kayal, P., Kumawat, S., and; Batra, N. (2020). PlantDoc. Proceedings of the 7th ACM IKDD CoDS and 25th COMAD. https://doi.org/10.1145/3371158.3371196

17. Hughes DP, Salath'e M. An open access repository of images on plant health to enable the development of mobile disease diagnostics through machine learning and crowdsourcing. CoRR.2015;abs/1511.08060. Available from: http://arxiv.org/abs/1511.08060.

18. M. E. Chowdhury, T. Rahman, A. Khandakar, N. Ibtehaz, A. U. Khan, M. S. Khan, N. Al-Emadi, M. B. Ibne, M. T. Islam, and S. H. Md., "Tomato Leaf Diseases Detection Using Deep Learning Technique", in Technology in Agriculture. London, United Kingdom: IntechOpen, 2021 [Online]. Available: https://www.intechopen.com/chapters/76494 doi: 10.5772/intechopen.97319