# Crop Disease Diagnosis with Deep Learning-Based Image Captioning and Object Detection
## 3 Materials and Methods
### Data Preprocessing
#### 3.2.2 Preprocessing for Object Detection Model  
We also collected crop images from AI-hub's \"Outdoor Crop Disease Diagnostic Image\" and \"Facility Crop Disease Diagnostic Image\" datasets to train the Object Detection model. We classified seven predicted classes, which was different from the Image Captioning model that defines nine predicted classes since Anthracnose and Leaf Blight were shown in two types of plants.  
We labelled images after the data collecting and class defining process. We used 'Labellmg', a python-based program that can create bounding boxes of the images, to box the damaged area. Bounding boxes were based on metadata in AI-hub, which describes the name and types of diseases. We generated boxes where crops were affected with diseases. There were also a number of cases that box the entire part when the infected areas were widely spread on the crop.  
The total number of the original dataset from AI-hub is 5970, and among them, we split the entire dataset into a training set and a validation set given the ratio 8:2. The specific figure is as follows: the red pepper data with 1507 images for training and 377 images for validation, the tomato data with 850 images for training and 213 images for validation, the spring onion data with 928 images for training and 112 images for validation, the bean data with 928 images for training and 201 images for testing, and zucchini data with 784 images for training and 196 images for validation. We collected the total 4871 images for training and 1099 images for validation.  
We decided to augment image data since the original dataset is not enough to train the proper model with each label not uniformly distributed. What our team had to do before the augmentation process is make a module to resize the bounding boxes based on the augmentation, since the existing tools offer augmentation only for images. The process includes transferring the coordinates from PascalVOC type to Yolo type for model training.  
The augmenting options are as follows: Vertical Flip, Horizontal Flip, Linear Contrast, Grayscale, Superpixel, Affine transformation, Embossing, and Sharpening. The Grayscale option was triggered with a 20% chance while other options were with a 50% chance as a way to guarantee the randomness and maximize the diversity of augmented images. We also resized the image to 640 \(\times\) 640 to reduce the training time of the model.  
The total number of image datasets after augmentation were 31,394 with 25,458 images for training and 5936 images for validation. The specific figures were as follows: the red pepper data with 4521 images for training and 1131 images for validation, the tomato data with 3196 images for training and 952 images for validation, the spring onion data with 5950 images for training and 1278 images for validation, the bean data with 3208 images for training and 928 images for validation, and the zucchini data with 3712 images for training and 804 images for validation.  
Figure 4: The Overall Preprocessing Step of Image Captioning.