# Multispectral Target Detection Based on Deep Feature Fusion of Visible and Infrared Modalities
## 3 Proposed Methodology  
### YOLOv5-Based Dual-Stream Architecture  
To effectively extract and fuse multispectral features, we propose a dual-stream extension of the YOLOv5 architecture. Unlike single-stream detectors, this design processes visible and infrared modalities in parallel to retain spectral uniqueness. This subsection outlines the overall network architecture and details how each component--including the backbone, neck, and head--has been adapted to improve feature learning and fusion for multispectral object detection.  
This study extends the YOLOv5 framework for multispectral target detection and introduces a dual-stream backbone network architecture. The proposed architecture processes both RGB and thermal images, incorporating improvements in three key components: the backbone network, neck network, and detection head.  
The backbone network consists of Focus, CSPNet, and SPP modules. The Focus module reorganizes the original image information through slicing operations, enriching the input features without significantly increasing computational complexity. The CSPNet module employs cross-stage local connectivity to enhance the network's learning ability while reducing computational redundancy. Additionally, the SPP module performs multi-scale pooling operations on feature maps, enabling the model to better adapt to different object scales.  
The neck network utilizes the FPN-PAN structure, which effectively transfers and fuses multiscale feature outputs from the backbone network. The Feature Pyramid Network (FPN) fuses high-resolution and low-resolution features in a top-down pathway, while the Path Aggregation Network (PAN) further enhances feature transfer in a bottom-up process. Finally, the detection head receives the fused multiscale feature maps and predicts the class, location, and size of the target through a series of convolution and prediction operations.  
In this study, multimodal midway fusion is applied to key layers (e.g., C3-C5) of the backbone network, leveraging the characteristics of multispectral data. This fusion strategy is informed by the existing research, which demonstrates that midway fusion outperforms other approaches, and it establishes a robust foundation for subsequent multimodal feature interactions. The architecture of our proposed YOLO-MEDet model, as shown in Figure 2, illustrates the integration of this fusion strategy within the model's dual-stream backbone, which is designed to extract and fuse features from both RGB and thermal images.  
The proposed dual-stream design enables the network to preserve modality-specific representations before fusion, which is essential for maintaining spectral integrity and leveraging complementary information. This design is motivated by recent advances in multispectral learning, where modality-specific encoders have shown better adaptability to spectral discrepancies compared to early fusion strategies [18; 21]. Furthermore, the structure retains compatibility with YOLOv5's modular efficiency while enabling more flexible intermediate feature interactions. These characteristics contribute to the robustness and generalization ability of our model in complex environments.This dual-stream foundation sets the stage for advanced fusion strategies, which are introduced in the next subsection.