# AI-Based Smart Monitoring Framework for Livestock Farms
## 5 Analysis of Experimental Results
### Comparison of RT-DETR and YOLO Models  
In this study, the performance of the proposed RT-DETR method was evaluated by comparing it with other object detection techniques from the YOLO series. The methods used for comparison were YOLOv5 and YOLOv8. The left graph in Figure 17 shows the precision curves for the RT-DETR, YOLOv5, and YOLOv8 methods. Precision refers to the proportion of true positive predictions among all objects predicted as positive. All three methods demonstrate similar results with high precision, indicating accurate detection.  
The right graph in Figure 17 presents the recall curves for RT-DETR, YOLOv5, and YOLOv8. Recall is defined as the proportion of actual positive objects that were correctly predicted as positive. The three methods show excellent recall performance, highlighting their effectiveness in detecting relevant objects. The left graph in Figure 18 compares the mAP@50 (mean Average Precision at an IoU threshold 0.5) for the RT-DETR, YOLOv5, and YOLOv8 models. AP@50 refers to the Average Precision when the Intersection over Union (IoU) is 0.5 or higher, and mAP@50 is the mean of AP@50 across all object classes. In this context, predictions with an IoU \(\geq\) 0.5 are classified as true positives (TPs). Predictions with an IoU \(<\) 0.5 are classified as false positives (FPs). False negatives (FNs) refer to objects that should have been detected but were not. Precision and recall are calculated using TP, FP, and FN values, and they typically have a trade-off relationship. To improve both precision and recall, the confidence threshold needs to be adjusted: Lowering the threshold  
Figure 16: Validation results for RT-DETR.  
increases FPs and decreases FNs, resulting in lower precision but higher recall. Raising the threshold decreases FPs and increases FNs, leading to higher precision but lower recall.  
The AP (Average Precision) is calculated as the area under the precision-recall curve, and the mAP is the average of APs across multiple object classes. The right graph in Figure 18 illustrates the evaluation method used in MS COCO, where the AP is calculated at IoU thresholds ranging from 0.5 to 0.95 in 0.05 increments and then averaged. From this evaluation, RT-DETR shows superior performance compared to YOLOv5 and YOLOv8.  
To validate the performance of the proposed model, a comparative analysis was conducted by examining the class loss of YOLOv5 and YOLOv8. The results of the comparison test are presented in Table 3, which clearly demonstrate that the RT-DETR model achieved the highest performance among the evaluated models. Table 3 presents the final loss values for the validation dataset, where the RT-DETR method achieves the lowest loss, further demonstrating its superior detection performance.  
Figure 17: Results of precision curves and recall curves.  
Figure 18: Results of mAP50 and mAP50-95 curves.  
As shown in Figure 19, which presents the classification loss curves for both training and validation sets, the RT-DETR method outperforms both YOLOv5 and YOLOv8. Classification loss in this context is calculated as the sum of squared errors of the predicted class probabilities, and lower values indicate better performance. The consistently lower loss observed with RT-DETR demonstrates its superior accuracy and generalization in classifying lesion-suspected behaviors in Hanwoo cattle. The right-hand graph in Figure 19 shows the class loss for the validation dataset. The RT-DETR method exhibits a lower loss compared to YOLOv5 and YOLOv8. A lower loss indicates that the model is able to detect objects more accurately.