# FusionTrack: Multiple Object Tracking with Enhanced Information Utilization  
[PERSON]  
1 Institute of Artificial Intelligence, Beihang University, Beijing 100191, China; [EMAIL_ADDRESS] (Y.Y. ); martin_z_2_  
in Figure 1b. They directly classify and localize targets in the current frame using object and track queries to represent the states of newborn targets and existing targets, respectively. Both models can continuously track targets under complex occlusion and similar appearance conditions well because of the expressive ability brought by the transformer decoder architecture. However, there are still some areas to be improved in the existing TBA methods for scenes with similar appearance and complex motion. Firstly, the tracking and detection tasks are not properly connected. In single-stage models, detection and tracking share the same network weights, making it difficult for models to accomplish two tasks simultaneously under similar appearance. Since the features required for detection and tracking are not consistent and sometimes even contradictory, this creates a trade-off between the two tasks of detection and tracking, reducing the accuracy of both. On the contrary, detection and tracking are completely independent in the two-stage models. While this avoids the coupling problem caused by inconsistent gradients, it also makes the tracking branch lack guidance from the detection branch. Secondly, in single-stage models, object queries are only responsible for matching with new targets rather than all targets. This prevents the model from fully utilizing the knowledge brought by state-of-the-art pretrained models to handle complex motion. Moreover, existing methods do not utilize the multi-layer design of the transformer decoder. For example, in TransCenter [10] and MOTR [11], only the output from the last layer of the decoder, is used as the feature for tracking targets, and the information from other layers is discarded. This paper proposes a transformer-based model to address these issues. It achieves efficient multi-object tracking in complex scenes by improving the feature fusion within and between frames; hence, we call it FusionTrack. To better transfer and aggregate information within the frame between the track and detection tasks, we propose a joint track-detection decoder to simultaneously identify all detection and tracking targets by separating the object query and track query tasks on top of MOTR [11]. The optimization goal of the detection branch is reset to detect all targets. This allows it to fully utilize the knowledge in the pretrained weights of existing DETR detectors [7,12] to achieve more robust detection results. Masked self-attention is adopted to transfer in-frame information from object queries to track queries only, maintaining a relatively independent detector to ensure that the model is adaptable to different detectors. To improve the feature fusion between the frame for better continuous tracking, a score-guided multi-level query fuser is proposed to align and transfer information between frames for track query propagation. This module aggregates queries from different decoder layers and fuses them with the guide of prediction scores for better appearance representation. Figure 1: Comparison between three types of model structures under track-by-attention paradigm. (**a**) Two-stage methods like TransTrack use separate decoders to process different queries and require a matching step to get results from detections. (**b**) Single-stage methods like MOTR process different queries using the same decoder and directly predict all tracking targets. (**c**) We adopt a new decoder structure that allows different queries to process relatively independently while allowing the tracking branch to take information from the detection branch. The model is evaluated on the DanceTrack [13] dataset, a MOT dataset featuring similar appearance and complex motions, where association is the main challenge for tracking.