# RMFNet: Redetection Multimodal Fusion Network for RGBT Tracking
## 1 Introduction  
Object tracking is a computer vision task that involves continuously estimating and predicting the position of an object in each subsequent frame of a video sequence, based on the annotation of the object in the first frame, to track and predict the object's motion trajectory and position. Video tracking technology involves theories from many fields and has great practical value in areas such as security monitoring, human-computer interaction, and intelligent life. While single-modality target tracking technology has made significant progress, it often performs poorly in complex scenes, such as those involving background clutter, occluded targets, low light, rain, and snow. In contrast, thermal infrared cameras, which are based on thermal radiation imaging, can effectively compensate for the shortcomings of visible light imaging and are better suited to dealing with such challenges. Furthermore, the visual light image can provide the detailed texture and color information that may be lost in the thermal infrared image. Therefore, the two modalities have their respective strengths and weaknesses, and exhibit strong complementarity. Combining the information from both modalities in target tracking can provide more comprehensive target information and improve the accuracy and robustness of tracking. Hence, the research on target tracking using both visible light and thermal infrared modalities has substantial significance. The current RGBT tracking algorithms based on deep networks can be broadly classified into two types of research. The first type investigates how to use deep networksto adaptively fuse the features of visible and thermal infrared modality. For example, [PERSON] et al. [1] proposed a recursive strategy to densely aggregate hierarchical features of the network and reduced redundant features and noise through channel pruning. [PERSON] et al. [2] performed bilinear pooling on any two layers using cross-product (a second-order computation effectively aggregating deep semantic and shallow texture information of the target) and aggregated bilinear pooling features of different modalities using a quality-aware network. [PERSON] et al. [3] proposed multi-scale adapters and multi-modality fusion modules to adaptively aggregate features of different scales and modalities. The second direction aims to explore the commonalities and characteristics of the two modalities and fully leverage their potential value. For example, [PERSON] et al. [4] proposed an enhanced transformer structure for feature fusion, which can further strengthen the fused feature and modality-specific features. [PERSON] et al. [5] proposed a hierarchical structure with a parallel combination of general adapters and modality adapters and integrated hierarchical divergence losses, which enabled single-stage joint learning of modality-sharing and modality-specific features. Additionally, a dynamic fusion module was designed in the instance adapter, which can perform quality-aware fusion of different source data. Although these tracking networks demonstrate reliable performance and great potential in various environments and challenges, two issues still exist. Firstly, these networks only use feature-level fusion and, subsequently, only use the fused feature for classification tasks. However, in certain scenarios (e.g., extreme lighting or thermal crossover), fused features may be easily influenced by low-quality interference when one modality image fails or has poor quality, making them not always superior to single-modality features. Decision-level fusion can address this issue and to some extent alleviate the impact of image mismatch. Unfortunately, the importance of decision-level fusion has been overlooked by the aforementioned networks. Secondly, these networks belong to the MDNet network architecture-based algorithms, and the MDNet network has a cumulative failure issue in targeting re-detection algorithms.