# A Study on Pine Larva Detection System Using Swin Transformer and Cascade R-CNN Hybrid Model
## 4 Implementation
### Training of the Pine Pest Detection Model  
In the training of the YOLOv5s model for optimization, the input image size was 640 * 640, the batch size was 32, and 1000 epochs were performed. During the training process, changes in training loss were recorded and stored in the wandb environment and were then used. The YOLOv5s model in Table 3 compares 0 steps with 400 steps, and it can be seen that the training loss quickly goes down from 0.07266 to 0.03762, and the training loss from 400 to 800 steps decreased from 0.03762 to 0.034. Herein, the training was terminated because there was no change in the value when learning 800-1000 steps. For the training of the Faster R-CNN model, the input image size was 640 * 640, the batch size was 64, 1000 epochs were performed, and the stochastic gradient descent (SGD) method was applied to optimize it. In addition, comparing 0 steps with 0.4 (\(10^{-6}\)) steps, training loss quickly decreases from 2.7628 to 0.3276. Herein, the train loss from 0.4 (\(10^{-6}\)) to 0.8 (\(10^{-6}\)) steps changes from 0.3276 to 0.160, where there is little change in value from 0.8 (\(10^{-6}\)) to 1.4 (\(10^{-6}\)) steps. Because of this, training was terminated. For training the Cascade R-CNN model, the input image size was 640 * 640, the batch size was 48, 1000 epochs were performed, and the SGD algorithm was applied for optimization. Herein, comparing 0 and 0.4 (\(10^{-6}\)) steps, it can be seen that the training loss goes down from 5.0919 to 0.0831, and the training loss from 0.4 (\(10^{-6}\)) to 0.6 (\(10^{-6}\)) \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Division** & **Original** & **Gaussian** & **Flip** & **Rotation** & **Multiply** & **Cutout** & **Total** & **Image** \\ \hline Rhagium & 162 & 648 & 648 & 648 & 648 & 648 & 648 & \\ Dendrolimus & 107 & 428 & 428 & 428 & 428 & 428 & 428 & \\ Rhyacionia & 163 & 652 & 652 & 652 & 652 & 652 & 652 & \\ Duplana & 163 & 652 & 652 & 652 & 652 & 652 & 652 & \\ \hline Total & 432 & 1728 & 1728 & 1728 & 1728 & 1728 & 9072 & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Data quantity results using model of data augmentation technique. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multicolumn{2}{c}{**YOLOv5s Model**} & \multicolumn{2}{c}{**Faster R-CNN Model**} & \multicolumn{2}{c}{**Cascade R-CNN Model**} & \multicolumn{2}{c}{**Mixed Model of Swin**} \\  & & & & & & & **Transformer and Cascade** \\ \hline Step & Training loss & Steps & Training loss & Steps & Training loss & Steps & Training loss \\
0 & 0.07266 & 0 & 2.7628 & 0 & 5,0919 & 0 & 8.1189 \\
200 & 0.03912 & 0.2 (\(10^{-6}\)) & 0.4615 & 0.2 (\(10^{-6}\)) & 0.4591 & 100,000 & 0.1094 \\
400 & 0.03762 & 0.4 (\(10^{-6}\)) & 0.3276 & 0.4 (\(10^{-6}\)) & 0.0831 & 200,000 & 0.1161 \\
600 & 0.03671 & 0.6 (\(10^{-6}\)) & 0.2704 & 0.6 (\(10^{-6}\)) & 0.0166 & 300,000 & 0.0852 \\
800 & 0.03400 & 0.8 (\(10^{-6}\)) & 0.0163 & 0.8 (\(10^{-6}\)) & 0.0108 & 400,000 & 0.0725 \\
1000 & 0.03285 & 1.0 (\(10^{-6}\)) & 0.0158 & 1.0 (\(10^{-6}\)) & 0.0083 & 500,000 & 0.0623 \\  & & 1.2 (\(10^{-6}\)) & 0.0137 & 1.2 (\(10^{-6}\)) & 0.0079 & 600,000 & 0.0543 \\  & & 1.4 (\(10^{-6}\)) & 0.0119 & 1.4 (\(10^{-6}\)) & 0.0065 & 700,000 & 0.0431 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Training the detection model for pine pests. steps decreases from 0.0831 to 0.0166. The training was terminated because there was no difference in value from 0.6 (10\({}^{-6}\)) to 1.4 (10\({}^{-6}\)) steps when learning and training. Lastly, to train the mixed model composed of the Swin Transformer model and Cascade R-CNN model proposed in this study, the input image size was 640 * 640, the batch size was 48, 1000 epochs were performed, and the SGD algorithm was optimized. Comparing 0 steps and 100,000 steps in the mixed model, training loss fell rapidly from 8.1189 to 0.1094. In addition, the training loss from 200,000 to 300,000 steps dropped from 0.1161 to 0.0852, and the training was terminated because there was no difference in the changed values from 300,000 to 700,000 steps.