# Crop Disease Diagnosis with Deep Learning-Based Image Captioning and Object Detection
## 2 Related Work  
### History of Deep Learning-Based Image Captioning and Object Detection  
Research using deep learning has been conducted in various fields. Among them, Image Captioning and Object Detection model research using deep learning are also being actively conducted.  
Image Captioning is a technique that captures various features of images and generates sentences describing them in detail. It should accurately grasp the various features contained in the image, such as people, objects, and actions, and generate high-quality sentences that explain these features well. A representative Image Captioning technique using deep learning is the \"End-to-End\" method. It is a method built on the Encoder-Decoder structure, which originated with the field of machine translation [2]. The End-to-End method typically consists of a CNN-based encoder that processes features of the image and a RNN-based decoder that generates captions which describe features of the image. For example, there is a model which was designed to extract the features of the image from the CNN model of the encoder, compress the extracted features into a 'global visual feature vector', and pass it on to the RNN model of the decoder to generate sentences [3]. In addition, a model with \"Attention\", a technique for finding parts to focus on among various features, was devised to enhance the accuracy of image feature description [4]. Recently, models developed based on Attention, such as Transformer, have also been applied to the decoder of the End-to-End method [5]. The Image Captioning model used in this study was also applied with the End-to-End technique that applied the Transformer model to the decoder.  
Object Detection is a technique that detects a specific object, such as a person, from an image to classify the class of the object and generates a bounding box that distinguishes the object from others [6]. With the advent of the deep CNN algorithm, the performance of feature extraction of images has increased dramatically. Research on deep learning-based Object Detection models has become active as well [7]. Deep learning-based Object Detection algorithms are largely divided into two types. First, there is a type that separates the process of detecting and classifying an object. For example, The RCNN model proposed by [PERSON] et al. (2014) finds multiple candidate regions that are expected to be objects, converts the sizes of each candidate region to the same, inputs candidate regions into the CNN model to extract features, and passes each feature to the SVM model to classify the object category [8]. Second, there is a type that processes detecting and classifying an object at once. This type of model has the advantage of relatively fast image processing since it detects and classifies an object simultaneously. For Instance, The YOLO model proposed by [PERSON] et al. (2015) is one of the representatives of this type. As shown in Figure 1, it selects candidate regions separately from images, inputs the entire image into the neural network instead of inputting each candidate region, divides the image into multiple grid cells, and performs detecting and classifying objects simultaneously using each cell [9].  
Even after the YOLO was released, other YOLO models with improved performance were steadily released. YOLOv5, the fifth-updated YOLO model, was applied to the Object Detection model used in this study.