## 3 Foreign Objects Detection Algorithm
### Multi-Network Feature Fusion  
The accuracy and robustness of the deep learning methods rely on a large number of training samples, a small dataset is insufficient to train a deep learning model with high precision and strong robustness [32]. The sizes of the input images will be cut or compressed to meet the requirements of different networks, which may cause the loss or distortion of some image information [33]. Therefore, the features extracted by a single convolutional neural network cannot fully characterize the image information, which may affect the detection accuracy. The fusion of features extracted by different networks is a strategy to improve the accuracy of foreign object detection. The features extracted by different networks express different information of images, so superimposing useful feature vectors of the images can increase the diversity of the features.  
Four types of foreign object images collected from UAV aerial images and video surveillance were used to establish the dataset. Due to different pixels of UAV devices, the image resolutions in the dataset are not the same, which are in the range of 640 \(\times\) 435 to 2279 \(\times\) 2181. Therefore, the target region in the foreign object image was extracted by the Otsu adaptive threshold segmentation algorithm and morphological processing, thus removing the complex background in the image and retaining the region with the foreign object. The size of the input images was transformed into 224 \(\times\) 224 \(\times\) 3 to meet the requirements of the employed networks, and they were input to five types of CNN, including GoogLeNet, EfficentNet-B0, DenseNet-201, ResNet-101, and AlexNet, thus extracting the features.  
The concatenation fusion strategy is adopted to fuse the features extracted by different networks. The features extracted by GoogLeNet, EfficientNet-B0, DenseNet-201, ResNet-101 and AlexNet are denoted as \(F_{G}\), \(F_{E}\), \(F_{D}\), \(F_{R}\), \(F_{A}\), respectively, and then the fused features \(F\) are concatenated according to Equation (4), which is shown as follows:  
\[F=\text{concatenate}(F_{G},F_{E},F_{D},F_{R},F_{A}) \tag{4}\]  
In order to verify the effectiveness of feature fusion and the complementarity of features extracted by different networks, the gradient-weighted class activation mapping (Grad-CAM) [34] was applied to visualize the decision region of each network, which is a technique for producing visual explanations for the decision region of CNN models.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Layer Name** & **Output Size** & **Operation** \\ \hline Conv3\_x & 28 \(\times\) 28 & 3 \(\times\) 3, 128 \\ Conv4\_x & 14 \(\times\) 14 & \(\left[\begin{array}{c}1\ \times\ 1,\ 128\\ 3\ \times\ 3,\ 128\\ 1\ \times\ 1,\ 512\\ 3\ \times\ 3,\ 256\\ 1\ \times\ 1,\ 1024\\ \end{array}\right]\) & \(\times\) 4 \\ Conv5\_x & 7 \(\times\) 7 & \(\left[\begin{array}{c}1\ \times\ 1,\ 512\\ 3\ \times\ 3,\ 512\\ 1\ \times\ 1,\ 2048\\ \end{array}\right]\) & \(\times\) 4 \\  & 1 \(\times\) 1 & average pool, 1000-d \(\text{fc}\), softmax \\ \hline \hline \end{tabular}
\end{table}
Table 3: _Cont._Grad-CAM uses the gradient of classification and the last convolution layer to produce a coarse localization map that highlights the decision region of the image.  
In order to verify the complementarity of features extracted by different networks, the saliency maps of five networks obtained by Grad-CAM were used to visualize the decision regions of each network. Taking an image of a kite hanging on transmission lines as an example, the visualization results are shown in Figure 6, where the deeper color corresponds to the area that the selected network pays more attention to. Different CNN has diverse decision region, it can be seen from Figure 6 that the coverage of the decision region of the proposed method is larger than that of a single convolutional neural network, which means that the extracted features contain more image information.