# EnsemblePigDet: Ensemble Deep Learning for Accurate Pig Detection
## 4 Experimental Results
### Evaluation of Detection Performance
Therefore, a visual test is performed on the key frame and hard frame, instead of the total frame. If the ACCmax_pigs value and ACCmanual_inspection value on the keyframe and hard frame are not significantly \begin{table}
\begin{tabular}{c c c c} \hline \hline \multicolumn{2}{c}{**Model**} & \multicolumn{3}{c}{**\# Error Frames with max_pigs**} \\ \cline{3-4} \multicolumn{2}{c}{} & \multicolumn{1}{c}{**13,997 Key Frames**} & \multicolumn{1}{c}{**4193 Hard Frames**} \\ \hline \multirow{3}{*}{Single Model} & Baseline YOLOv4 & 3976 (71.59\%) & 1906 (54.54\%) \\  & Model A (proposed) & 1619 (88.43\%) & 848 (79.78\%) \\  & Model B (proposed) & 1844 (86.83\%) & 778 (81.45\%) \\ \hline \multirow{2}{*}{Ensemble Model} & EnsemblePigDet (proposed) & 621 (95.56\%) & 279 (93.34\%) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of accuracy ACCmax_pigs for 13,997 key frames and 4193 hard frames with overexposed regions (obtained during 8:30 a.m. to 10:30 a.m.). \begin{table}
\begin{tabular}{c c c c} \hline \hline \multicolumn{2}{c}{**Model**} & \multicolumn{3}{c}{**\# Error Frames with max_pigs**} \\ \cline{3-4} \multicolumn{2}{c}{} & \multicolumn{1}{c}{**(ACCmax_pigs)**} \\ \cline{3-4} \multicolumn{2}{c}{} & \multicolumn{1}{c}{**216,000 Raw Video Frames**} & \multicolumn{1}{c}{**13,997 Key Frames**} \\ \hline \multirow{3}{*}{Single Model} & Baseline YOLOv4 & 43,344 (79.93\%) & 3976 (71.59\%) \\  & Model A (proposed) & 20,248 (90.63\%) & 1619 (88.43\%) \\  & Model B (proposed) & 21,092 (90.24\%) & 1844 (86.83\%) \\ \hline Ensemble Model & EnsemblePigDet (proposed) & 12,244 (94.33\%) & 621 (95.56\%) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of accuracy ACCmax_pigs for 216,000 raw video frames and 13,997 key frames with overexposed regions (obtained during 8:30 a.m. to 10:30 a.m.). different, it can be inferred that the ACC\({}_{\text{max\_pigs}}\) value and ACC\({}_{\text{manual\_inspection}}\) value on 216,000 total test frames also does not have a significant difference. Table 5 shows the result of actual counting with visual checking (hence, ACC\({}_{\text{manual\_inspection}}\)), which is considerably similar to the counting result using the number of boxes (ACC\({}_{\text{max\_pigs}}\) shown in Table 5). As explained in Figure 6, there are cases where ACC\({}_{\text{max\_pigs}}\) does not consider a frame to be an error frame when it actually is and deems a frame to be an error frame when it is not. Overall, ACC\({}_{\text{max\_pigs}}\) and ACC\({}_{\text{manual\_inspection}}\) have similar results. Hence, if the test data are too large to waive box annotation for each individual pig or pose difficulties in checking the frames visually, ACC\({}_{\text{max\_pigs}}\) is identified as an actual performance metric to represent the detection accuracy of the deep learning model when determining the number of pigs within a pig pen (max\({}_{\text{pigs}}\)). In addition, unlike the general detection accuracy metric, which is based on box annotation of individual pigs, AP (average precision)/AR (average recall), ACC\({}_{\text{max\_pigs}}\) and ACC\({}_{\text{manual\_inspection}}\) consider a frame even with one error as an error frame (for example, eight pigs are correctly detected, but if a pig is omitted, then the frame is considered to have an error). Thus, the metrics display fewer numbers overall when compared to AP/AR. To test the effect on image preprocessing and the model ensemble, a comparison experiment was performed, and Table 6 shows the results. In a single model, as shown in  
Figure 6: Limitations of ACC\({}_{\text{max\_pigs}}\) when max\({}_{\text{pigs}}\) = 9. (**a**) An “error” frame with one false-positive error and one false-negative error based on ground-truth (although it is regarded as a “correct” frame based on ACC\({}_{\text{max\_pigs}}\)); (**b**) a “correct” frame with no false positive/negative error based on ground-truth (although it is regarded as an “error” frame based on ACC\({}_{\text{max\_pigs}}\)).