# Multispectral Target Detection Based on Deep Feature Fusion of Visible and Infrared Modalities
## 5 Experimental Results
### Ablation Study  
In this section, we conducted a series of ablation experiments to thoroughly evaluate the performance of the proposed network. For comparison, we constructed two parallel CSPDarknet53 networks using element-wise summation as the fusion operation to build the baseline backbone network, as shown in Figure 6. Subsequently, we introduced various improvements for experimentation, primarily including the AEFF and the C3-PSA module. In this study, we compared the performance of different model configurations based on the key evaluation metric, mean Average Precision (mAP), which covers all categories in the dataset. Specifically, M1, M2, and M3 represent the following: a YOLOv5 trained exclusively for visible light image detection, a YOLOv5 trained exclusively for infrared image detection, and a dual-stream YOLOv5 with element-wise summation for modality fusion. M4 and M5 correspond to the dual-stream YOLOv5 with AEFF and C3-PSA improvements, respectively, while M6 represents the fully enhanced dual-stream YOLOv5 with both AEFF and C3-PSA improvements. Through ablation experiments, we performed a quantitative analysis of the mean values for the three fusion metrics: [EMAIL_ADDRESS], [EMAIL_ADDRESS], and [EMAIL_ADDRESS]:0.95. For simplicity in this section, the experiments focus on the FLIR aligned dataset, the KAIST dataset, and the M3 FD dataset, with the relevant results presented in Tables 7-9. The results of the ablation experiments performed on the FLIR dataset show that the dual-stream YOLOv5 architecture (M3) demonstrated significant improvements across several performance metrics compared to the unimodal models (M1 and M2). In terms of the [EMAIL_ADDRESS] metric, M3 achieved a performance of 78.4, surpassing M1's 67.8 and M2's 73.9. This highlights the dual-stream architecture's ability to effectively fuse visible and infrared image data, thereby enhancing overall detection performance. Upon the introduction of the AEFF module, the [EMAIL_ADDRESS] value of M4 improved to 80.8, further validating the effectiveness of the AEFF in facilitating information interaction between different modalities. The inclusion of the AEFF enhances the network's ability to fuse cross-modal features, thus improving the model's robustness. In contrast, while the introduction of the C3-PSA in M5 led to a slight reduction in the [EMAIL_ADDRESS] metric (79.8), it significantly enhanced feature representation by improving detail. Finally, when both the AEFF and \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Method** & **Data** & **[EMAIL_ADDRESS]** & **[EMAIL_ADDRESS]:0.95** \\ \hline DdcGAN [50] + YOLOv5 & RGB + IR & 80.9 & 50 \\ MFEIF [51] + YOLOv5 & RGB + IR & 82 & 50.7 \\ U2 Fusion [52] & RGB + IR & 81.9 & 51.5 \\ TarDAL [53] & RGB + IR & 82.7 & 50.8 \\ Ours & RGB + IR & 84.2 & 52.4 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: _Cont._ \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Model** & **M1 (Vis)** & **M2 (Inf)** & **M3** & **M4** & **M5** & **M6** \\ \hline AEFF & ✗ & ✗ & ✗ & ✓ & ✗ & ✓ \\ C3-PSA & ✗ & ✗ & ✗ & ✗ & ✓ & ✓ \\ \hline mAP0.5 & 67.8 & 73.9 & 78.4 & 80.8 & 79.8 & 79.9 \\ mAP0.75 & 25.9 & 35.7 & 29.2 & 32.8 & 32.5 & 31.6 \\ mAP0.5:0.95 & 39.5 & 37.4 & 37.1 & 39 & 39.6 & 38.9 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Ablation findings for the FLIR dataset. ✓ indicates the component is enabled; ✗ indicates it is disabled. C3-PSA were applied to M6, the model's performance for [EMAIL_ADDRESS] stabilized at 79.9 and maintained high values for the [EMAIL_ADDRESS] and [EMAIL_ADDRESS]:0.95 metrics. This demonstrates that the synergy of the two modules is essential for optimizing the model's performance in multimodal target detection. Table 8 presents the results of ablation experiments conducted on the KAIST dataset, comparing the two-stream YOLOv5 baseline model (M3) with the models incorporating the AEFF (M4), the C3-PSA module (M5), and a combination of both the AEFF and C3-PSA (M6).