# Crop Disease Diagnosis with Deep Learning-Based Image Captioning and Object Detection
## 2 Related Work
### Model Application Case  
There were only a few number of studies related to Korean context on Image Captioning. Those studies translate the English captions of the MS COCO dataset into Korean using RNNs and LSTMs [10; 11]. Moreover, most of them employ old-fashioned models with poor accuracy in the field of Natural Langue Processing. Although several models have started to conduct research using Attention, we have rarely found research that handles Korean context using Transformer.  
In the non-agricultural field, several studies using Object Detection models have been conducted. There are examples of implementing dirt and garbage detection models using SSDs and VGG16, and there are also studies that have constructed a total of three CNN models\(-\)InceptionV3, VGG16, MobileNet\(-\)to determine whether people are wearing a mask at the time when COVID-19 was spread worldwide [12].  
There were several cases in which an Object Detection model was constructed using a deep learning model in the agricultural field. Leveraging Plant village data, AlexNet and GoogleNet-based crop disease detection cases exist [13]. In addition, there are Fast R-CNN, R-FCN, SDD-based tomato disease, and pest detection models [14]. Meanwhile, using the In Yolo County California, 2014 dataset, the crop classification model combining the enhanced LSTM model and the 1D CNN model yielded higher accuracy of 85.54% and f1_score results of 0.73 compared to existing machine learning models such as XGBoost, RF, and SVM [15]. In addition, there is also a case of improving the apple leaf disease detection model through deep residual learning [16]. There was also a study of recognizing plant leaf diseases by segmenting disease symptoms through a hue, saturation and intensity-based, and LAB-based hybrid segmentation algorithm, as well as by classifying the name of diseases through CNN network [17]. Moreover, we find transfer-learning-based plant disease detection studies that detect rice plant diseases based on the DenseNet pre-trained on ImageNet and the Inception module [18]. There is also a case of identifying plant diseases based on the VGGNet pre-trained on ImageNet and the Inception module [19]. Meanwhile, we also find a case of identifying plant diseases based on ensemble  
Figure 1: Structure of the YOLO.  
learning, aggregating three lightweight CNNs, including SE-MobileNet, Mobile-DANet, and MobileNet V2 [20].  
There were also cases in which Object Detection research was conducted using the YOLO model. There existed a YOLOv3-based apple growth phase prediction model utilizing the Author Collection Dataset, which led to performance above the Fast R-CNN model and the VGG16 model. There was also a study in which the YOLOv5 model used in this work is applied in conjunction with the optical flow algorithm to develop a tram-to-pedestrian collision prediction prevention model [21].  
Therefore, through this study, we used a Transformer model for Image Captioning technology to apply it to Korean, which has not yet been studied, to help agricultural fields and diagnose crop diseases. We also wanted to study it because there were few cases of crop disease diagnostic studies using Image Captioning models.  
In addition, we used YOLOv5 as our Object Detection model to visualize the affected area of the crop's disease with bounding boxes so that the damaged part can be seen at a glance.