# Deep Learning-Based Bird's Nest Detection on Transmission Lines Using UAV Imagery
## 5 Simulation Results and Analysis
### Comparative Experiment
#### 5.3.2 Performance Evaluation  
We present the comparison of the ROI mining faster RCNN model with faster RCNN, faster RCNN with focal loss, and cascade RCNN in terms of bird's nest detection using UAV images. TP, TN, FP and FN are determined by the pixel-wise intersection-over-union. This work sets the intersection-over-union (IoU) threshold to 0.5. When the IoU of the output detection box and the \begin{table}
\begin{tabular}{c c} \hline \hline
**Computer Configuration** & **Specific Parameters** \\ \hline CPU & Intel Core i7-8700k \\ GPU & NVIDIA GeForce GTX 1080 Ti \\ Operating system & Ubuntu16.04, Canonical, London, United Kingdom \\ Random Access Memory & 16 GB \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Computer specifications. annotation box is greater than 0.5, we believe that the model correctly detects the object (P). Otherwise, it is considered that the model performs a wrong detection (F). The significance of TP, FP, FN and TN is to reflect the accuracy of detector classification and location, which directly affects recall and precision. Similarly, the accuracy of location is intuitively reflected by AP. In the detection, the real situations of TP, TN, FP and FN are shown in Figure 8.  
In practical applications, in order to ensure the safe operation of transmission lines, it is necessary to accurately detect all birds' nests on the tower. Recall is used to represent the ability of the detector to detect birds' nests and suspected bird's nest objects. Excessive false detections result in the waste of computing resources, precision represents the accuracy of the detector to detect the birds' nests. F1 score is synthesized by precision and recall, which is used to balance the detection ability of the bird's nest and computing resources. AP is used to indicate the detection accuracy for bird's nest category. It measures whether the category and position of the bounding box predicted by the model are accurate. According to the analyses above, this work uses these four indicators to measure the detection ability of the detectors for the birds' nests. This work uses 5-fold cross-validation to test the ROI mining faster RCNN. We evaluate the performance and stability of the model according to the mean and standard deviation of the validation mAP in the five validation sets. The results of the five tests are shown in Table 2.  
As shown in Table 2, the mean mAP of ROI mining faster RCNN is 82.51%. After 5-fold cross-validation, the standard deviation of mAP is 0.0041. It can be seen from the validation standard \begin{table}
\begin{tabular}{c c c} \hline \hline
**K-Fold** & **Mean Average Precision (mAP)** & **Loss Value** \\ \hline
1-fold & 0.8249 & 0.2037 \\
2-fold & 0.8311 & 0.1756 \\
3-fold & 0.8243 & 0.2023 \\
4-fold & 0.8198 & 0.2103 \\
5-fold & 0.8256 & 0.1989 \\ Mean & 0.8251 & 0.1982 \\ Standard deviation & 0.0041 & 0.0133 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Five-fold cross-validation results. Figure 8: Practical examples of (**a**) true positive (TP), (**b**) false positive (FP), (**c**) false negative (FN) and (**d**) true negative (TN). Please note that the green box represents the label box and the red box represents the detection box. deviation and validation loss that ROI mining faster RCNN has performed better generalization ability and robustness. In order to contrast and validate the functionality of our method in industrial applications, we use the images of high-voltage electric poles in the urban area as an input to detect foreign objects that are also small objects, e.g., honeycombs and water bottles. By detecting objects in different backgrounds, this can measure the effectiveness and stability of this method in a wide range of industrial applications. The results of the comparative experiments are shown in Table 3. It can be seen from the results that our method achieves high accuracy under different backgrounds and has good generalization.