# FusionTrack: Multiple Object Tracking with Enhanced Information Utilization
## 5 Analysis and Discussion
### Score-Based Multi-Level Query Fuser  
We analyze how the score-based multi-level query fuser works by recording and visualizing the information sharing through attention maps. This module consists of two consecutive attention modules in the implementation. Given a input query \(q\) containing \(N_{L}+2\) tokens, the appearance feature \(a\) is calculated as follows:  
\[a=(\text{FFN}\circ\text{Attn}\circ\text{FFN}\circ\text{Attn})(q)[-1]\]  
\[\text{Attn}(q)=W_{attn}q\]  
where \(W_{attn}\) represents the product of the attention map and query in-projection weights. We ignore the influence of the intermediate feed-forward network and normalization  
Figure 7: Visualization of information flow in an example frame containing 7 instances. **Left**: the attention map of a decoder layer. **Right**: detection and tracking results, boxes are colored in blue and red respectively. Notice that the information flow from the tracking branch to the detection branch is blocked so that the detection branch is identical to the base detector, allowing the full utilization of pretrained knowledge. This leads to the empty space in the bottom-right corner.  
modules and approximate the relationship between the final appearance feature and each intermediate query as the matrix product \(W^{0}_{attn}W^{1}_{attn}\) of two attention maps. The last row of the influence matrix is taken to represent the weight of each token in the output appearance feature. For a continuously tracked target, we visualized the change of weights as shown in Figure 8.  
The upper segment of the figure illustrates how the approximated weights of each layer occupying the final feature change across frames. Each row represents the influence of the input query generated by the previous appearance, the query output by each of the six decoder layers, and the score embeddings in a top-down order, forming an old-to-new transition. A noticeable trend is that the score embeddings hardly contribute any information to the final appearance, while other weights fluctuate within a small range around a fixed value. We believe that this can be explained as follows: when most predicted scores are greater than 0.9, the difference between the score embedding queries used for querying across frames is not significant, which leads to this relatively fixed weight for each layer.  
Meanwhile, for frames where targets overlap with each other, the weights of tokens closer to the previous frame are higher than those in frames without occlusion. An example is shown at the bottom-right corner of the figure, where we demonstrate the relative difference between the weights of single frames and the average weights across the video for a fixed instance. In frame 161, where object 2 and object 4 are severely occluded, the weights for the old query and the first decoder output are more than 50% higher than average. On the contrary, in frame 166 when all objects are separated, the weights for relatively new queries are higher. This is consistent with our intuition that \"under complex overlapping conditions, old appearance features are more credible than current ones\".