# EnsemblePigDet: Ensemble Deep Learning for Accurate Pig Detection
## 3 Proposed Method
### Key Frame Extraction and Accuracy Metrics  
To measure the detection accuracy of the test dataset, an expensive \"box-level\" annotation to create ground truth is usually required. Therefore, a method to measure the detection accuracy at an inexpensive cost is necessary for continuous pig monitoring applications. Raw video frames have a lot of redundant information for pig monitoring because pigs sleep frequently for a long time. In this study, key frames that show significant changes in movement (hence, pigs that show movement) were chosen to reduce the test dataset. After reducing the test dataset through key frame extraction, CorLoc [34] used in weakly supervised object detection (WSOD) was modified to measure the detection accuracy of key frames, under the assumption of knowing the number of pigs to be detected (max_pigs). This study proposes a method to measure the detection accuracy without the box-level annotation cost. To extract key frames, the number of pigs that show movement in a current video frame is estimated by comparing the previous and current frames. YOLOv4 is then applied to calculate average size of bounding boxes. Note that YOLOv4 is applied only for a sample frame from training dataset, not input video data. The pixel difference between the previous and current frames is computed to decide whether a pig is moving. Finally, the number of pigs that are moving is estimated, and the current frame is set as the key frame, if at least one moving pig exists. Initially, an object detector YOLOv4 is applied to the sample frame, and bounding boxes are acquired. Bounding boxes that have less than a certain confidence value (set to 0.7 in this study) are removed to reduce false positives, and boxes that can be trusted are chosen. The average detection box size S for the sample frame is calculated. Following this, the number of pixels \(D\) for which the difference in pixel value is higher than a certain threshold (i.e., TH\({}_{\text{pixeldiff}}\)) is calculated for regions in individual detection boxes of the current frame. Subsequently, if the divided value of D to S \(\times\) N is higher than a certain threshold (TH\({}_{\text{keyframe}}\)), where the N is number of pigs in a pig pen, then the current frame is designated as the key frame. For this study, threshold values that are needed to extract key frames were set to TH\({}_{\text{pixeldiff}}\) = 1, TH\({}_{\text{keyframe}}\) = 1, and the algorithm to extract the key frame is shown as Algorithm 2. After extracting the key frames, the detection accuracy is evaluated without box-level annotation. As explained, the maximum number of pigs that can be detected within an enclosed pig pen (i.e., max_pigs) is known. With max_pigs, the accuracy \(\text{ACC}_{\text{max\_pigs}}\) is defined for n test frames. For each test frame i, the number of detection boxes \(\text{C}_{\text{i}}\) is compared with max_pigs. If they are equal, the frame is considered as the correct frame (i.e., \(\text{F}_{\text{i}}=1\)). Then, \(\text{ACC}_{\text{max\_pigs}}\) is computed as the ratio of the total number of test frames to the total number of correct frames w.r.t max_pigs. However, even though the number of detection boxes is equal to max_pigs, there is a possibility of false-positive and/or false-negative errors. As explained, the number of pigs by manual inspection (\(\text{GT}_{\text{i}}\)) for each test frame i can be less than max_pigs because of possible occlusion. Therefore, with \(\text{GT}_{\text{i}}\), an accuracy \(\text{ACC}_{\text{manual\_inspection}}\) for n test frames is defined. For each test frame i, the number of detection boxes \(\text{C}_{\text{i}}\) is compared to the value of \(\text{GT}_{\text{i}}\). If they are equal, the frame is considered as the correct frame (i.e., \(\text{M}_{\text{i}}=1\)). Then, \(\text{ACC}_{\text{manual\_inspection}}\) is computed as the ratio of the total number of test frames to the total number of correct frames for manual inspection.