# RMFNet: Redetection Multimodal Fusion Network for RGBT Tracking
## 1 Introduction
To overcome these two problems, we propose a network for RGBT tracking: a redetection multimodal fusion network that improves the redetection algorithm and combines feature-level and decision-level fusion. We extracted features using the same network configuration as the first three convolutional layers of VGG-M [6] with the modification of expanding the receptive field. To fully exploit the complementarity of multimodal information and explore the differences between two modalities, we divide the network into three branches, the visible light branch, the thermal infrared branch, and the fusion branch. The two specific modality branches each use three convolutional layers to extract features, with no weight sharing between the convolutional layers of the two modalities. A fusion branch is introduced in parallel between the two modalities, where the two features output by the 7 \(\times\) 7 convolution are, respectively, input into the second 5 \(\times\) 5 convolution and third 3 \(\times\) 3 convolution of the fusion branch to extract shared information between the two modalities. The ECA [7] attention-based feature fusion module designed in this paper can adaptively compute the weights of two modalities and perform a weighted fusion of multimodal features to obtain more robust fused features. Three fully-connected classification layers are added after each of the three obtained features to obtain the sample scores output using each of the three features, and the three scores are summed to obtain the final sample score. We further improved the re-detection mechanism of existing algorithms. Specifically, when the tracker's pre-location is unreliable, we directly expand the search area and perform a global target search to obtain target re-location. We then determine whether the re-location is reliable, and, if so, use the re-location as the target location for the frame, or, if not, use the one with the higher score of pre-location and re-location as the target position for this frame. Finally, the Alpha-Refine [8] bounding box refinement module is added in this paper to adjust the coarse positioning of the tracker precisely. This paper offers several key contributions, summarised as follows:  
1. We propose an RGBT tracking network: the Redetection Multimodal Fusion Network (RMFNet), which contains two types of fusion, medium-term feature-level and late term decision-level fusion, and can be divided into three branches, visible, thermal infrared, and fusion, and can fully utilize the complementarity and correlation of multimodal information to achieve robust RGBT tracking. 2. We have designed an ECA attention-based multimodal feature fusion module that adaptively computes two modalities' reliability and performs a weighted fusion of multimodal features to obtain a more favourable feature representation. 3. We improve the re-detection algorithm of the base tracker by adding a re-detection step that performs a global target search at the current frame, which mitigates the accumulation of failures and can increase the robustness and precision of the tracking algorithm while decreasing the amount of computation and making the tracking algorithm more efficient. 4. Extensive experimental results on the GTOT [9] and RGBT234 [10] datasets show that RMFNet outperforms other advanced RGBT tracking methods and obtains good performance.