# Multispectral Target Detection Based on Deep Feature Fusion of Visible and Infrared Modalities
## 1 Introduction  
The integration of multispectral sensors with deep learning algorithms has proven to be a highly effective approach across various applications [1, 2]. These sensors can capture data across both visible and invisible spectral ranges, thereby providing extensive environmental information [3, 4, 5]. Moreover, when integrated with deep learning techniques such as computer vision and target detection models, these sensors facilitate the automation of tasks in the invisible spectrum [6]. As depicted in Figure 1, under optimal lighting conditions, RGB images accurately capture objects with distinct colors, textures, and contours. However, in thermal images, these visual details are often harder to distinguish. In contrast, under low-light or nighttime conditions, object details and the boundaries between edges and the background in RGB images may become blurred and challenging to discern. In such scenarios, thermal imaging, leveraging its unique energy-radiation-based imaging mechanism, can reveal distinguishable object contours, making it a viable alternative. The integration of additional modalities significantly enhances detection robustnesscompared to conventional unimodal RGB images, which exhibit inherent limitations.The samples in Figure 1 are taken from the public FLIR dataset. Multimodal information fusion techniques have recently made significant strides in target detection, particularly in improving robustness under diverse environmental conditions. Traditional fusion methods, such as splicing or element-wise summation [7; 8], provide basic multimodal integration but often struggle in complex scenarios. To address these challenges, attention-based approaches have been explored, leveraging adaptive weighting mechanisms to selectively enhance or suppress features. For instance, integrating attention modules into twinned convolutional neural networks has been shown to improve feature extraction efficiency [9]. Similarly, recurrent fusion strategies refine feature representation post-fusion, enabling better feature discrimination within detection frameworks [10]. Beyond attention-based enhancements, novel fusion architectures have been introduced to tackle specific challenges in multispectral detection. The Gated Fusion Single-Stage Multi-Frame Detector (GFD-SSD) incorporates specialized fusion units that dynamically integrate feature maps across intermediate layers, optimizing cross-modal feature learning [11]. Meanwhile, addressing the cross-modal disparity issue, regional feature alignment techniques have been proposed to reduce misalignment errors and improve multimodal consistency [12]. Further refinements at the pixel level utilize spatial and channel-wise attention mechanisms to enhance the granularity of feature fusion, facilitating more precise target representation [13]. Recent advancements have also been driven by the adoption of Transformer-based architectures, which introduce global attention mechanisms to capture long-range dependencies and improve multimodal fusion effectiveness [14]. At the same time, the increasing complexity of multispectral datasets, such as DroneVehicle [15], VEDAI [16], and LLVIP [17], has pushed the boundaries of detection algorithms, necessitating more sophisticated fusion strategies. Despite these advancements, challenges remain in achieving optimal cross-modal feature integration, highlighting the need for further innovation in multispectral target detection. Figure 1: Separate images for different scenarios. The figure on the left shows visible light images captured under varying lighting conditions. The figure on the right presents the aligned visible light and infrared images. To overcome the constrained receptive field issue in multispectral image feature fusion and the limitations of convolution-based methods in modeling long-range dependencies, this paper proposes a Transformer-based modality fusion network called YOLO-MEDet.