Dual-Path Temporal Decoder for End-to-End Multi-Object Tracking
Authors: Hyunseop Kim
Juheon Jeong
Hanul Kim
Yeong Jun Koh
Venue: NeurIPS 2025 poster ()
Keywords: Multi-Object Tracking
Transformer
Temporal Modeling
Attention Mechanism
Query Update
Primary area: applications
TL;DR: A dual-temporal decoder framework for end-to-end multi-object tracking that preserves identity and models temporal variation through offset-guided query updates.

Abstract: We present a novel end-to-end transformer-based framework for Multiple Object Tracking (MOT) that advances temporal modeling and identity preservation. Despite recent progress in transformer-based MOT, existing methods still struggle to maintain consistent object identities across frames, especially under occlusions, appearance changes, or detection failures. We propose a dual-path temporal decoder that explicitly separates appearance adaptation and identity preservation. The appearance-adaptive decoder dynamically updates query features using current frame information, while the identity-preserving decoder freezes query features and reuses historical sampling offsets to maintain long-term temporal consistency. To further enhance stability, we introduce a confidence-guided update suppression strategy that retains previously reliable features when predictions are unreliable. Extensive experiments on MOT benchmarks demonstrate that our approach achieves state-of-the-art performance across major tracking metrics, with significant gains in association accuracy and identity consistency. Our results demonstrate the importance of decoupling dynamic appearance modeling from static identity cues, and provide a scalable foundation for robust tracking in complex scenarios.

Dual-Path Temporal Decoder for End-to-End Multi-Object Tracking
Hyunseop Kim I+
Juheon Jeong
Hanul Kim
Yeong Jun Koh
+Chungnam National University
"Souul National University of Science and Technology
hyunseop350gmail. com, jjh9904276gmail. com, tecim ac. kr, yjkohōcnu ac. kr
Abstract
We present novel end-to-end transformer- based framework for Multiple Object Tracking (MOT) that advances temporal modeling and identity preservation. De- spite recent progress in transformer-basee MOT, existing methods still struggle maintain consistent object identities across frames, especially under occlusions, appearance changes, or detection failures. We propose dual-path temporal de- coder that explicitly separates appearance adaptation and identity preservation. The appearance-adune adaptive decoder dynamically updates query features using current preservingg decoder freezes query features
frame information, while the and reuses historical sampling offsets to maintain long-term temporal consistency. To further enhance stability, we introduce strategy that retains previously reliable features when predictions are unreliable. Ex- tensive experiments MOT benchmarks demonstrate that approach achieves state-of-the-ar performance across major tracking metrics, with significant gains in association accuracy and identity consistency. Our results demonstrate the im- portance of decoupling dynamic appearance modeling from static identity cues, and provide scalable foundation for robust tracking in complex scenarios. Code is available at
seffidenne-guided update suppression
Introduction
Multi-object tracking (MOT) aims consistently estimate the spatial locations and identities of multiple objects across a video sequence. As a fundamental task in computer vision, MOT is essential for broad range of real-world scenarios, including autonomous driving [35], surveillance [29, 43], sports analytics [9, 32], and crowd analysis [23], where consistent -patio-temporal tracking is required. Early MOT studies typically have followed the backing-by-detectin
paradigm [1, 3, 17.
31, 42]. These methods first detect objects in each frame and then perform association steps across frames. These methods perform association steps based on the spatial proximity between objects in consecutive frames, measured by Interaction of Unions (loU) [3, 6] appearance similarity using RelD embeddings [1, 24, 30, 39]. These methods have benefited from rapid advances in object detection, but they remain limited in complex scenarios involving occlusion, similar appearance, and non-linear motions.
To address these limitations, recent approaches [12, 18, 36, 40] have adopted transformer-based MOT. These methods unify the DETR baseb detector and tracker in an end-to-end manner. In these models, track queries are propagated from the previous frame and used to associate and track objects over time. However, transformer-bas based MOT still faces key challenge in maintaining the
"quual contributions. Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

temporal consisteney of track query features. Although these methods refine query features over multiple decoder layers, erroneous updates can accumulate and degrade feature consistency over time, ultimately resulting in identity switches.
In this work, we propose transformer- MOT framework with novel dual-path temporal decoder that explicitly addresses this issue. Each decoder layer consists of two parallel components: an appearance-adaative decoder layer that refines query features using current frame information and an adentity-preserving decoder layer that maintains temporal consistency by reusing fixed query features and historical sampling offsets from the previous frame. The layer reuses fixed query features and historical sampling offsets to enhance temporal stability and reduce sensitivity to abrupt changes in feature representation. To further enhance robustness, we introduce a confidence-guii
decoder
update suppression strategy during inference, which retains the states of low-confidence track queries instead of updating them. This mechanism alleviates identity drift and improves tracking reliability in challenging scenarios such as occlusion and detection failure. Together, these designs enable the model to balance adaptability and stability, leading to more accurate and consistent multi-object tracking across long temporal spans.
The proposed MOT achieves the state-of-the-art performance on the DanceTrrac [27] and SportsMOT [9] benchmarks, demonstrating strong association ability in challenging scenarios involv- ing diverse objects with similar appearances.
In summary, the contributions are as follows:
We propose dual-path temporal decoder that disentangles appearance adaptation and identity preservation. The appearance-adaptivy decoder layer dynamically refines query features using current-frame information, while the peresrty- decoder layer keeps track queries fixed and reuses historical sampling offsets to maintain temporal consistency.
We introduce a confidence-guided update suppression strategy that prevents unreliable updates under low-confidence conditions, thereby stabilizing identity association in the presence of occlusions and detection noise.
Our method achieves new state-of-the-art performance the DanceTrack [27] and SportsMOT [9] benchmarks, demonstrating strong improvements in both tracking accuracy and identity preservation.
2
Related Work
Baecking-by-dteettie remains a dominant paradigm in MOT, where per-frame object detections are temporally associated to form object trajectories. SORT [3] em- ploys the Kalman filter and the Hungarian algorithm to associate bounding boxes based on IoU. DeepSORT [31] enhances identity stability by combining the Kalman filter with appearance-batte based ReID embeddings. JDE [30], FairMOT [39], and Unicorn [33] jointly optimize detection and RelD features to learn discriminative representations and achieve consistent identity preservation. Re- cent efforts have extended this framework by improving detection quality and robustness against association errors. BoT-SORT [1] introduces appearance fusion, motion compensation, and the improved Kalman filter to strengthen real-time performance. tranformmer-assee approaches such as TransMOT [8] and GTR [42] model long-range spatiotemporal dependencies enable more structured data association. OC-SORT [6] replaces heuristic motion models with learnable predictors for improved motion estimation. Extensions such as GHOST [26] and StrongSORT [10] focus on practical deployment by addressing domain shift, embedding refinement, and inference efficiency. More recently, DeconfuseTres Track [16] formulates association as multi-stage decision problem to reduce ID switches. DifTMOT [20] formulates data association as the denoising diffusion process to jointly predict object trajectories over time.
Transformer-based MOT. Recently, the transformer-basee MOT has emerged as promising direction in end-to-end MOT. Trackformer [22] and MOTR [36], both built upon on the DETR [7] architecture, perform joint detection and tracking by propagating track queries across frames within the decoder. MeMOT [5] extends this idea by incorporating both short- and long-term memories into track embeddings, improving robustness to occlusions and appearance changes. Subsequent works have explored improvements in temporal consistency, identity preservation, and scalability. MOTRv2 [40] enhances detection recalls by integrating external YOLOX [13] detections into the
2

---

Transformer Encoder
Trunsformer Enooder D
p
D
Dual-path
Dual-path
Dtt
Temporal decoder
Confidence-guide
Temporal decoder
((-1)
g(t)
update suppression
appurracceaddpiiie decoder layer
Feature fusion
Add
Cross-
MLP
(1-1)
Linear
Lincar
morm
attention
Add norm
(3+1))
F()
Q,
Decoder layer
(0)
Solf- attention
Linear
i
adı
Feature sampling
Padn
Q
Lincar
FFN
Q2-1)
0
(1+1)
(2+1)
docoder layer
Feature sampling
b
p(0
Add &
Cross- attention
(
a4-1)
Q:
Decoder layer
norm
b
T.M
CD
[+1 (1t1l
2t (+1)
0 Qr
Self-
Linear
Feuture sampling
p(
0,
Decoder layer (
Qp
Figure I: An overview of the proposed MOT framework. The dual-path temporal decoder consists of an appearance-adaptiv decoder layer that refines query features using the current frame, and an decoder layer that maintains temporal consistency by freezing track query features and reusing historical sampling offsets. During inference, a confidence-guil guided update suppression strategy is applied to prevent unreliable feature updates. C denotes concatenation and ( denotes feature splitting for track and candidate queries.
MOTR framework. MeMOTR [12] further strengthens identity stability under occlusion by directly injecting long-term memory into the transformer backbone. ColTrack [18] maintains identity consis- tency by applying self-attention between the current query and past queries along the same object's trajectory. MOTIP [11] explicitly decouples detection and association into two learnable modules, offering more interpretable and flexible query interaction framework. However, these methods often suffer from inconsistent query updates, where inaccurate feature refinement across frames can degrade identity representations and lead to frequent identity switches.
3 Method
We aim to compose the set of tracked objects (++) (0 .....,
(t) ....,,(() in each video frame 7(1), where
Tin E R a confidence score T E R²,, query feature 47.m E R², and sampling offsets (A)
N is the number of tracked objects. Each tracked object o includes a bounding box b (8)
Tix E Rx2
where C denotes the feature dimension and K is the number of sampling points. For each frame I(t), we extract feature map (tt) H xC where H and W denote height and width, and set of d(t) using a DINO [37]-based detector. Each detection candidate d(') contains an anchor box b D.m R2 and query feature q Rº We aim to predict updated bounding boxes and confidence scores for tracked objects and detection candidates
M detection candidates (()) = (d
(t)
the current frame é based on thier previous states (t--1) and current observations (tt),
Figure/ 1 illustrates the structure of the proposed MOT framework, which integrates a dual-path temporal decoder consisting of two complementary decoding layers. An aopearance-adaptivv decoder layer updates query features using the current frame, enhancing localization accuracy and adaptability to appearance changes. In parallel, an identity-preserving decoder layer maintains the original track queries and reuses historical sampling offsets, thereby preserving temporal consistency and alleviating identity drift. In addition, we introduce a confidence-gun guided update suppression strategy, which preserves previously reliable query features by preventing updates when current predictions are unreliable.
3

---

3.1 Dual-Path temporal decoder
The dual-path temporal decoder consists of stacked layers, each including two parallel branches: an addappiiv decoder layer and an identity preesering decoder layer. The appearance-adapee the
decoder layer dynamically updates query features through deformable attention and sampling, while
path maintains temporal identity consistency by keeping track query features fixed and reusing their historical sampling offsets. For each decoder layer I, outputs from both branches are fused to refine the object representation and estimate bounding boxes and confidence scores.
Appearance-adaptivy decoder layer. The appearance-adapiin decoder layers iteratively refine query features across L decoder layers. For the initial decoder layer (1 = 0), the track query features (0) E N are initialized from the propagated queries of the previous frame, where n-th row corresponds to a track query feature (t-1), while candidate query features Q E IR MxC are initialized from the detection candidates, where m-th row corresponds to candidate query feature (t)
4p. For subsequent decoder layers, both track and candidate query features, denoted as Q7 and ( respectively, are obtained as outputs from the previous decoder layer.
For each layer L, we concatenate these query features to form the combined query matrix Q [ ) We then apply multi-head self-attention, followed by residual connection and layer normalization to compute refined features
Q Qu
LN (0 + SelfAttn (Q.).)
(1)
We then predict the sampling offsets by applying a linear projection over the features Q i.e. (I) ada Linear( ( E R(N+M)xXX2 where K is the number of sampling points. These offsets guide the deformable attention to extract features adu E RRN++1))KXX from the image feature map F(O), inspired by the deformable attention mechanism in deformable DETR [44]. For the feature sampling process, the bounding box of the track query at the initial decoder layer (0) is initialized from the estimated bounding box at the previous frame, b Similarly, the bounding box of each candidate query b D.m is initialized from the detection candidate's bounding box D.m at the current frame. Unlike the previous transformer-baat
MOT methods [18, 22, 40] that compute attention weights only from the query features, our approach aggregates the sampled image features ada based on affinity between query and sampled image features. Thus, cross-attention layer adopted to obtain the final enhanced query features d at layer l:
F(!)
CrossAttn (Q..)
F(()
ada
auda
where Q serves as query, while p(l) serves key and value in the cross-attention.
Srentity-preserving decoder layer. The identity-preeeeee preserving decoder layer shares the same structure as the appearance-adaptiv decoder, but handles track queries differently. By keeping track queries fixed across decoder layers and reusing historical sampling offsets, it prevents the injection of unstable evidence from the current frame, thereby preserving -dentity-specific features and ensuring temporal consistency across frames. Specifically, the track queries QT NxC are initialized from the propagated queries of the previous frame as in the adaptive decoder layer, but remain fixed across decoder layers to preserve temporal identity. For each layer I, the static track queries and candidate
queries Q self-attention, followed by residual connection and layer normalization to compute refined features
(1) E MxC are concatenated form 9 [Qr (0)
We then apply multi head
(!) Qai
I.N (Q.") SelfAttn Q
We split these refined features Q0 into track and candidate components, denoted as Q (1) respectively.
Q and D

---

where queries are Q and keys and values sampled features F(!) id Sampling offsets for candidate queries are obtained by applying a linear projection to Q i.e. A (1) Linear ( RMxXx2 In contrast, the offsets for track queries are reused from Id.D
their historical offsets A (t-1) in the previous frame / - based the observation that the same object typically exhibits similar sampling offset patterns across adjacent frames. Reusing historical offsets promotes spatial stability in attention, maintains alignment with persistent object regions, and suppresses transient noise arising from frame-specific variations. Although the track query features Qr themselves do not change across decoder layers, their corresponding bounding boxes {b.. (
T,N } are iteratively refined at cach layer. Deformable attention uses the updated bound-
ing box centers as reference positions, sampling image features F(U) id E R(N-M)x Kx from the
image feature map F(t) using A (t-1) and (1) D. A cross-attention layer is then applied to obtain enhanced query features layer l:
Q (1) CrossAttn (Q. F..)
Feature fusion and state prediction. For each decoder layer I, we fuse the outputs from the aopearrance-daattin and incorporate both dynamic appearance changes and stable identity information. Specifically, given the enhanced query features from the appearance-adaptive path Qu and from the identity-peseervng path Q we combine the corresponding track and candidate features from each path separately using linear projections, resulting in the fused track and candidate query features follows:
decoder layers to obtain refined representations that
Q (1+1) Linear (Linear(
Q + Linear (,0))
The fused features are further refined using feed-forward network (FFN) with residual connection and layer normalization. Then, Q (1+1) split into @(+1)) and Qf++2), which are used for the next layer / ++1 1. Here, Q is used for the appearance-adaptivn and Seentityprreeeringg decoder layers, while (((+1) is used for the appearance-adaptive layer only.
The refined fused query features Q Q(++1) are further used to predict bounding boxes and corresponding confidence scores for each object. Specifically, the bounding boxes b(++1) E R(N-M)x4 and confidence scores clI+1) E R R(N+M)xx1
obtained by passing Q Q(++1) through two separate multi- layer perceptrons (MLPs), each consisting of two linear layers with an activation function in between. These predictions serve the updated object states at decoder layer 1 + 1, facilitating accurate tracking and identification of objects across frames. The predicted bounding boxes and confidence scores are subsequently split into track and candidate components (b (1+1) ((+1), and (bf+!) C respectively, for further sampling processing in subsequent decoding layers.
(b(+1)) cr
(1+1) (1+1))
At the last layer T., the final tracking decisions for frame I are determined based confidence thresholding. Tracked objects whose predicted confidence scores c//1) are larger than a threshold a are regarded as active tracks, otherwise, they are marked as lost. Similarly, candidate objects with confidence scores ( > a are determined as new tracks. Tracks that remain lost for more than consecutive frames are terminated. Finally, sampling offsets A are extracted from the entries in A (L) corresponding to both the track queries and the selected candidate queries, and are reused in uda
the Deeettty-preserving decoder at frame 1.
3.2 Confidence-guided update suppression
While the confidence score determines whether an object considered tracked or lost, conventional transformer-basee tracking frameworks [18, 22, 36] continue to update query features of all track objects at every frame, regardless of their confidence. That is, even after an object marked as lost, its query feature continues to be updated through self-attention and cross-attention. As result, noisy observations from unreliable predictions are incorporated into the query representation. This can be problematic, since the updated query feature is subsequently used to predict the object's bounding box
5

---

and confidence score in the next frame. As result, this unconditional update strategy accumulates noisy query features over time, which can cause identity drift or misassociation with nearby objects.
To address this issue, we propose a confidence-guided update suppression strategy. For objects with low confidence, we do not update their query features unless their predicted confidence exceeds predefined threshold 3. Instead, we preserve their previously reliable representations from earlier frames where their confidence was sufficiently high. This selective update mechanism suppresses the accumulation of noisy predictions from uncertain objects, thereby preserving clean identity embeddings and improving long-tern tracking stability.
3.3 Training
We train our model in an end-to-end manner using a bipartite matching objective [7], as in previous transformer-based MOT frameworks [18, 36, 40). Given the predicted tracked objects O and ground-truth set, we first compute the optimal assignment using Hungarian matching. The training loss is then computed over matched pairs using a weighted sum of a classification loss, a bounding box regression loss, and a generalized IoU loss, following standard practice.
Experiments
4.1 Datasets & Metrics
Dance [27]. It is a multi-human tracking dataset in dancing scenes with similar uniform appear- ance and diverse motion, requiring strong association under occlusion and ambiguity. Dance'Trach contains 40, 25, and 35 videos for training, validation, and test sets.
SportsMOT [9]. It is recently released multi-object tracking dataset that focuses athlete tracking in fast-paced sports such as soccer, basketball, and volleyball. The SportsMOT dataset presents significant challenges for motion modeling due to frequent acceleration and abrupt direction changes in these scenes. The dataset consists of 45, 45, and 150 sports sequences for training, validation, and test sets, respectively.
widely used pedestrian tracking dataset. MOT17 mainly contains massive pedestrians with simple and linear motions. It contains 7 training sequences and 7 test sequences. The sequences contaín 500-1500 1500 frames, recorded and annotated at 25-30 FPS.
MOT17 [23].
Metries. We assess the performance of the proposed method based on diverse MOT metrics for comparisons with other methods. Higher order tracking accuracy (HOTA) [19] is used as the primary metric, as it provides a balanced assessment of detection and association performance. To further dissect this trade-off, decompose HOTA into its constituent factors. We also include ID F1 score (IDF1) [25], which measures the alignment between predicted and ground identities, and multi-object tracking accuracy (MOTA) [2]. conventional metric that emphasizes detection errors, including false positives, false negatives, and ID switches.
report detection accuracy (DetA) and association accuracy (AssA), which
4.2 Implementation Details
The proposed framework is built on DINO [37] that uses ResNet-50 [14] backbone and transformer-based encoder. We select the top-M 300 detection candidates from the encoder in
MOT Network.
DINO anchor boxes and extract a candidate query feature q0.m. for each candidate by combining its learnable query embedding and positional embedding, following [37]. We set the number of dual-path temporal decoder layers L = 6, a feature dimension C 256, and sampling points
K = 256. The confidence threshold a, the suppression threshold B, and 7 are set to 0.6, 0.4, and 60, respectively.
Training. As in the prior works [18, 22, 36), we perform two-stage training strategy. In the first stage, the object detector is trained for epochs. the second stage, the backbone and encoder are frozen, and only the dual-path temporal decoder is trained. The input images are resized to resolution of 1440 x 800. The proposed MOT framework employs multi-scale training, Mosaic [4]. and MixUp [15] for data augmentation. We use the Adam optimizer with a learning rate of 1 x 10-1

---

Table 1: Quantitative comparison on the DanceTrack [27] test set. The performance with validation
data presents that the validation set is also included during training. The best results are boldfaced.
Methods
HOTA
DetA
AssA
MOTA
IDF1
w/o valid data:
CenterTrack [41]
41.8
78.1
22.6
86.8
35.7
TransTrack [28]
45.5
75.9
27.5
88.4
45.2
Byte'Track [38]
47.7
71.0
32.1
89.6
53,9
QDTrack [24]
54.2
80.1
36.8
87.7
50.4
MOTR [36]
54.2
73.5
40.2
79.7
51.5
OC-SORT [6]
55.1
80.3
38.3
92.0
54.6
DifiMOT [20]
62.3
82.5
47.2
92.8
63.0
MeMOTR [12]
68.5
80.5
58.4
89.9
71.2
CO-MOT [34]
69.4
82.1
58.9
91.2
71.9
MOTRv2 [40]
69.9
83.0
59.0
91.9
71.7
MOTIP [11]
72.0
81.8
63.5
91.9
76.8
ColTrack [18]
72.6
62.3
92.1
74.0
Ours
74.1
83.9
65.6
92.5
78.6
with valid data:
MOTRv2 [40]
73.4
83.7
64.4
92.1
76.0
ColTrack [18]
75.3
66.9
92.2
77.3
Ours
76.2
85.0
68.3
92.5
79.9
Table 2: Quantitative comparison on the SportMOT [9] test set. The best results are boldfaced.
Methods
HOTA
DetA
AssA
IDF1 MOTA
QDTrack [24]
60.4
77.5
47.2
90.1
62.3
CenterTrack [41]
62.3
82.1
48.0
90.8
60.0
ByteTrack [38]
62.8
77.1
51.2
94.1
69.8
TrackFormer [22]
63.3
66.0
61.1
74.1
72.4
B-T-SORT [1]
68.7
84.4
55.9
94.5
70.0
MeMOTR [12]
68.8
82.0
57.8
90.2
69.9
TransTrack [28]
68.9
82.7
57.5
92.6
71.5
ColTrack [18]
71.5
80.5
63.6
89.4
74.6
OC-SORT [6]
71.9
86.4
59.8
94.5
72.2
DiffMOT [20]
72.1
86.0
60.5
94.5
72.8
MOTIP [11]
72.6
83.5
63.2
92.4
77.1
Ours
73.9
82.2
66.6
91.5
78.7
and a weight decay of 14-4 The learning rate is decayed by a factor 0.1 during the final 15
training epochs. The model is trained for 45, 45 and 65 epochs on DanceTrack [27], SportsMOT [9] and MOTI7 [23], respectively. All experiments are conducted on 8 NVIDIA RTX 4090 Ti GPUs with batch size of 1, where each batch consists of 4-frame video clip.
4.3 Benchmark Evaluation
DanceTrack. Table shows the comparison of the proposed method with the existing methods on the test set in DanceTrack [27]. The proposed MOT achieves HOTA score of 74.1 1 and achieves tfate-of-the-an performance across all metrics. In particular, compared to the previous best method ColTrack [18]. it exhibits significant improvements in association accuracy, with AssA increasing from 62.3 to 65.6 and IDF1 increasing from 74.0 78.6. These results demonstrate the effectiveness of our temporal modeling in maintaining consistent object identities over time. Even when compared to prior works [18, 40) trained on both training and validation sets, our model consistently outperforms all competitors, demonstrating the effectiveness and robustness of our approach.

---

Table 3: Quantitative comparison on MOTI7 [23] test set. The best results are boldfaced.
Methods
HOTA
DetA
AssA
MOTA
IDF1
Heuristic:
OC-SORT [6]
63.2
63.2
78.0
77.5
ByteTTrack [38]
63.1
64.5
62.0
80.3
77.3
BoT-SORT [1]
64.6
80.6
79.5
MixSort-OC [9]
63.4
63.8
63.2
78.9
77.8
MixSort-Byte [9]
64.0
64.1
64.2
79.3
78.7
Deep OC-SORT [21]
64.9
65.9
79.4
80.6
Deconfuse Track [16]
64.9
65.0
65.1
80.4
80.6
End-to-ends
MOTR [36]
57.8
60.3
55.7
73.4
68.6
MeMOTR [36]
56.9
58.9
55.8
72.5
69.0
TransTrack [28]
54.1
61.6
47.9
74.5
63.9
MOTRv2 [40]
62.0
63.8
60.6
78.6
75.0
TrackFormer [22]
74.1
68.0
ColTrack [18]
61.0
78.8
73.9
MOTIP [11]
59.3
62.0
57.0
75.3
71.3
Ours
61.5
60.8
62.5
73.8
75.1
Table 4: Ablation studies for the
decoder layer (IDL) on the DanceTrack [27]
validation set. The best results are boldfaced.
Method
HOTA
DetA
AssA
MOTA
IDFI
without IDL
66.7
76.1
56.3
87.0
69.5
IDL with varying offsets A
(1)
67.5
77.5
59.0
87.1
73.2
id.
IDL with static historical offsets A (t-1)
69.1
77.8
61.6
87.5
74.9
MOOrtsMOT. Table 2 lists the performance on the SportMOT [32] test set. The proposed MOT achieves 73.9 HOTA, surpassing the previous state-of-the-an MOTIP [11] by margins of 1.3. Also, ours significantly improves the IDFI score by 5.9 over DifIMOT [20]. demonstrating superior association accuracy. DifIMOT, a -beeeettn method reliant on pretrained detectors [13], excels on detection-centric metrics (e.g. MOTA, DetA) but underperforms on association-focused metrics such IDFI.
MOT17. Table presents the results on the MOT17 [23] test set. The proposed method achieves the best AssA and IDFI performance among end-to-ens -o-etod approaches. It indicates that the proposed method maintains stable associations and robust against ID switches. Despite our lower detection accuracy (MOTA 73.8) than ColTrack (MOTA 78.8) and MOTIP (MOTA 75.3), our stronger association capability enables higher HOTA and lower IDF1 than them, narrowing the gap to heuristic-augmented pipelines. Compared to MOTRv2, which uses heuristic post-processing, our method achieves superior AssA and comparable IDF1 while remaining fully end-to-end.
4.4 Ablation Study
We conduct ablation studies on the validation set of DanceTrace [27] to evaluate the effectiveness of the proposed components, including the identity-preserving decoder layer and the seciidenne-giided update suppression. In addition, We analyze the performance under various detection settings to further validate the robustness of the proposed framework.
decoder layer. Table reports the ablation study on pdentity-preseervii decoder layer (IDL). We evaluate three model variants to analyze its impact. First, we remove the IDI. from the dual- dual-path temporal decoder, reducing it to a single-path structure that consists appearance-adaptive decoder only. Second, include the IDL but replace the historical sampling

---

Table 5: Ablation studies for the confidence-guided update suppression on Dance Track [27] validation set. The best results are boldfaced.
Method
HOTA
DetA
AssA
MOTA
IDF1
without cefiddene-guiides
67.9
78.1
59.2
87.6
72.9
update suppression
0.2
68.2
77.6
60.2
87.3
73.8
with confidence- guided
0.4
69.1
77.8
61.6
87.5
74.9
update suppression
0.6
68.7
77.9
60.7
87.6
74.2
Table 6: Comparison of the proposed method with other methods using various detectors on the DanceTrack [27] validation set. The best results are boldfaced.
Detector
mAP
Tracker
HOTA
DetA
AssA
MOTA
IDFI
MOTRv2 [40]
64.5
78.7
53,0
YOLOX
72.1
Ours
67.9
77.2
60.0
87.2
73.3
MOTIP [11]
62.2
75.3
51.5
85.2
64.8
Deformable DETR
63.7
Ours
66.4
77.1
57.3
85.9
70.6
ColTrack [18]
61.9
86.5
61.6
DINO
73.1
Ours
69.1
77.8
61.6
87.5
74.9
offsets AT (t-1) with predicted offsets A (I) id obtained by applying linear projection Q Finally, the full model includes the IDL with historical sampling offsets.
We observe that incorporating the improvements over using only the appearance-adaptive decoder. Specifically, the variant with fixed track query features Qr achieves notable gains, improving HOTA from 66.7 to 67.5 and IDFI from 69.5 to 73.2. These results indicate that maintaining stable query features across decoder layers strengthens identity association and reduces drift. Furthermore, augmenting this with historical sampling offsets A r (t-1) yields the best performance, achieving 69.1 HOTA and 74.9 IDF1. This underscores the importance of both feature consistency and temporally coherent attention for robust identity preservation over time.
decoder layer leads consistent performance
Confidence-guiie update suppression. Table shows ablation study to evaluate the impact of the confidence-guidee update suppression strategy. Without this suppression, the model achieves 67.9 HOTA and 72.9 IDF1. contrast, enabling the strategy consistently improves performance. Setting the threshold to 3 0.4 yields the best results, improving HOTA by 1.2 and IDF1 by 2.0, while also increasing AssA from 59.2 61.6. These results indicate that selectively retaining previously reliable query features under con-coniie conditions stabilizes identity association and reduces identity drift.
Analysis under various object detectors. To validate the generalization capability of the proposed dual-path decoder, reproduced experiments the DrnceTrack validation using the same detectors adopted by prior methods. Table shows the comparison our model with previous transformer- based MOT under three detectors: YOLOX [13], Deformable DETR [44], and DINO [37]. which are used in MOTRv2 [40], MOTIP [11], and ColTrack [18], respectively. The proposed method outperforms other transformer-bee based MOT methods for all detectors with significant HOTA improve- ments. These results demonstrate that the proposed dual-path decoder and stable query propagation consistently enhance association accuracy across all detector settings.
Tracking results and sampling locations according to IDL. Figure/ illustrates a layer-wise comparison of tracking results and sampling locations from the appearance-adaptig decoder layer decoder layer (IDL). For each setting, we visualize
(ADL), with and without the dentityy-pesservv the predicted bounding boxes and the top 50 sampling points (based on attention weights) for multiple decoder layers 1,2,5, 5, at the current frame Without IDI., the model fails to preserve the identity of the target object (green box) from frame 1, resulting in misaligned bounding box

---

1(-1)
Target object
Incorrect track
Correct track
without IDL
rolo
with IDI.
without IDL
with IDL
ADL (1=
ADL (1= 1)
ADL (L (1=2) 2)
ADL (1 5)
ADL (1= 6)
Visualization of tracking results and sampling locations in the appearance-adaptive decoder layer (ADL), with and without the adennttt--reserviigg decoder layer (IDL). The green box indicates the target object from frame ( 1. The cyan box represents an incorrectly tracked result, corresponding to an identity switch (IDSW), while the magenta box denotes correctly tracked object. Green dots indicate the top 50 sampling locations with the highest attention weights.
Figure 2:
at frame / that corresponds to different object (cyan box). The associated sampling points also shift toward this incorrect object, yielding drift in query attention. contrast, with IDL., the model consistently tracks the correct target object (magenta box) across layers, and the sampling points remain localized around the intended target. These observations demonstrate that IDL, by reusing static queries from the previous frame, enhances temporal stability and guides the decoder to maintain focus on the correct object during the iterative processes.
5 Conclusions
We introduced a transformer-based MOT framework that explicitly addresses the challenges of maintaining temporal consistency and robust identity association in complex scenes. At the core of our approach is dual-path temporal decoder that decouples appearance adaptation from identity preservation, enabling the model to refine object representations while safeguarding identity-specifie information from frame-specific noise. Additionally, we proposed a confidence-guidee update suppression mechanism that further stabilizes tracking by selectively retaining reliable features under unreliable predictions. Through extensive experiments on DanceTrace and SportsMOT, our method consistently outperforms existing approaches both detection and association metries, establishing new state-of- -o---of-thh of-thh-art results on two benchmarks. The significant improvements in IDFI and HOTA demonstrate the effectiveness of our temporal modeling strategy. We believe this work provides strong foundation for further advancements in end-to-end multi-object tracking and highlights the importance of disentangling temporal dynamics and identity stability in transformer-based architectures.
10

---

Acknowledgements. This work was partly by the National Research Foundation of Korea (NRF) grants funded by the Korea government (MSIT) (No. RS-2024-003997293, No. RS-2024-00352566, No. RSS2025-005551133 and partly by Institute of Information & communications Technology
Planning Evaluation (IIPP) grant funded by the Korea government (MSIT) (No.RS-2022-001. Artificial Intelligence Convergence Innovation Human Resources Development (Chungnam National
University))
References
[1] N. Aharon, Orfaig, and B.-Z. Bobrovsky. Bot-sort: Robust associations multi melti-pedestrian tracking. arXiv preprint arXiv:2206. 14651, 2022.
[2] K. Bernardin and R. Stiefelhagen. Evaluating multiple object tracking performance: the clear mot metrics. EURASIP Journal on Image and Video Processing, 2008: 1-10, 2008.
[3] A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Uperoft. Simple online and realtime tracking. In 2016 IEEE international conference on image processing (ICIP), pages 3464-3468. Icee, 2016.
[4] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao. Yolov4: Optimal speed and accuracy of object detection. arXiv preprini arXiv:2004. 10934, 2020.
[5] J. Cai, M. Xu, W. Li, Y. Xiong, W. Xia, Z. Tu, and S. Soatto. Memot: Multi-object tracking with memory. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8090-8100, 2022.
[6] Cao, J. Pang, X. Weng, R. Khirodkar, and Kitani. Observation-centric sort: Rethinking sort for robust multi-object tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9686-9696, 2023.
[7] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko. End-toend object detection with transformers. in ecev. Springer, 1(2),4, 2020.
[8] Chu, J. Wang. Q You, H. Ling, and Z. Liu. Transmot: Spatial-temporal graph transformer for multiple object tracking. In Proceedings of the IEEE/CVF Winter Conference on applications of computer vision, pages 47004880, 2023.
[9] Y. Cui, C. Zeng. X. Zhao, Y. Yang, G. Wu, and L. Wang. Sportsmot: A large multi-object tracking dataset in multiple sports scenes. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9921-9931, 2023.
[10] Y. Du, Z. Zhao, Y. Song, Y. Zhao, F. Su, T. Gong, and H. Meng. Strongsort: Make deepsort great again. IEEE Transactions on Multimedia, 25:8725-8737, 2023.
[11] R. Gao, Qi, and L. Wang. Multiple object tracking as id prediction. Proceedings of the Computer Vision and Patlern Recognition Conference, pages 27883-27893, 2025.
[12] R. Gao and L. Wang. Memotr: Long-term memory-augment transformer for multi-object tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9901-9910 10, 2023.
[13] Z. Ge, Liu, Wang, Z. Li, and J. Sun. Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.088 2021.
[14] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770- 778. 2016.
[15] Y. N. D. D. L.-P. Hongyi Zhang. Moustapha Cisse. mixup: Beyond empirical risk minimization. International Conference on Learning Representations, 2018.
[16] C. Huang, S. Han, M. He, W. Zheng, and Y. Wei. Deconfusetrack: Dealing with confusion for multi-object tracking. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 19290-19299, 2024.

---

[17] H. Kim, H.-J. Lee. Y. Lee, J. Lee, H. Kim, and Y. J. Koh. Grae-3dmot: Geometry aware encoder for online 3d multi-object tracking. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages I 11697-11706, 2025.
[18] Y. Liu, J. Wu, and Y. Fu. Collaborative tracking learning for frame-rate-insensitvv multi-object tracking. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9964-9973, 2023.
[19] J. Luiten, A. Osep, P. Dendorfer, P. Torr, A. Geiger, L. Leal-Taixé, and B. Leibe. Hota: A higher order metrie for evaluating multi-object tracking. International journal of computer vision, 129:548-574 2021.
[20] W. Lv, Y. Huang, N. Zhang, R.-S. Lin, M. Han, and D. Zeng. Diffmot: A real-time diffusion- based multiple object tracker with non-linear prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19321-19330, 2024.
[21] G. Maggiolino, A. Ahmad, J. Cao, and K. Kitani. Deep oc-sort: Multi-pedestrian tracking by adaptive re-identification. In 2023 IEEE International conference on image processing (ICIP), pages 3025-3029. IEEE, 2023.
[22] T. Meinhardt, A. Kirillov, L. Leal-Taixe, and C. Feichtenhofer. Trackformer: Multi-object tracking with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8854, 2022.
[23] A. Milan, L. Leal-Taixé, Reid, S. Roth, and K. Schindler. Mot16: A benchmark for multi- object tracking. arXiv preprint arXiv: 1603300831 2016.
[24] J. Pang, L. Qiu, X. Li, H. Chen, Q. Li, T. Darrell, and F. Yu. Quasi-dense similarity learning for multiple object tracking. Proceedings of the IEEE/CVF conference on computer vision and pattem recognition, pages 164-173 2021.
[25] E. Ristani, F. Solera, R. Zou, R. Cucchiara, and C. Tomasi. Performance measures and data set for multi-target, multi-camera tracking. In European conference on computer vision, pages 17-35. Springer, 2016.
[26] Seidenschwarz. G. Brasó, V. C. Serrano, I. Elezi, and L. Leal-Taixé. Simple cues lead to strong multi-object tracker. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13813-13823, 2023.
[27] Sun, Cao, Jiang, Z. Yuan, Bai, K. Kitani, and Luo. Dancetrack: Multi-object tracking in uniform appearance and diverse motion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20993-21002, 2022.
[28] Sun, J. Cao, Y. Jiang, R. Zhang, Xie, Z. Yuan, C. Wang, and P. Luo. Transtrack: Multiple object tracking with transformer. arxiv 2020. arXiv preprint arXiv:2012. 15460, 2, 2012.
[29] Z. Tang, M. Naphade, M.-Y. Liu, X. Yang. S. Birchfield, S. Wang, R. Kumar, D. Anastasiu. and J.-N. Hwang. Cityflow: A sctle benchmark for multi-target multi-camera vehicle tracking and re-identification. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages -899,8806, 2019.
[30] Z. Wang, L. Zheng, Y. Liu, Y. Li, and S. Wang. Towards real-time multi-object tracking. In European conference on computer vision, pages 107-122. Springer, 2020.
[31] N. Wojke, A. Bewley, and D. Paulus. Simple online and realtime tracking with deep association metric. In 2017 IEEE international conference on image processing (ICIP), pages 3645-3649, IEEE, 2017.
[32] Wu, R. He, G. Wu, and I., Wang. Sportshhi: A dataset for human-human interaction detection in sports videos. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18537-18546. 2024.
[33] B. Yan, Jiang, Sun, D. Wang, Z. Yuan, Luo, and Lu. Towards grand unification of object tracking. In European conference on computer vision, pages 733-751. Springer, 2022.
12

---

[34] F. Yan, W. Luo, Y. Zhong. Y. Gan, and L. Ma. Bridging the gap between end-to-end and non-end-to-end multi-object tracking. arXiv preprint arXiv:2305. 2023.
[35] Yu, H. Chen, Wang, W. Xian, Chen, F. Liu, Madhavan, and Darrell. Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2636 2636-2645, 2020.
[36] Zeng, B. Dong, Y. Zhang, T. Wang, X. Zhang, and Y. Wei. Motr: End-to-end multiple-object tracking with transformer. In European conference on computer vision, pages 659-675. Springer, 2022.
[37] H. Zhang, F. Li, S. Liu, L. Zhang, H. Su, J. Zhu, L. M. Ni, and H.-Y. Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022.
[38] Y. Zhang. P. Sun, Y. Jiang, D. Yu, F. Weng. Z. Yuan, P. Luo, W. Liu, and X. Wang. Bytetrack: Multi-object tracking by associating every detection box. European conference on computer vision, pages 1-21. Springer, 2022.
[39] Y. Zhang, C. Wang, X. Wang. W. Zeng, and W. Liu. Fairmot: On the faireess of detection and re-identification multiple object tracking. International journal of computer vision, 129:3069-300 -0877, 2021.
[40] Y. Zhang, T. Wang, and X. Zhang. Motrv2: Bootstrapping end-to-end multi-object tracking by pretrained object detectors. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 22056-22065, 2023.
[41] X. Zhou, V. Koltun, and P. Krähenbühl. Tracking objects as points. In European conference on computer vision, pages 474-490. Springer, 2020.
[42] X. Zhou, T. Yin, V. Koltun, and P. Krähenbühl. Global tracking transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8771-8780, 2022.
[43] Zhu, L. Wen, X. Bian, H. Ling, and Q. Hu. Vision meets drones: A challenge. arXiv preprint 2018.
[44] X. Zhu, W. Su, L.. Lu, B. Li, X. Wang, and J. Dai. Deformable detr: Deformable transformers for end-to-end object detection. arxiv 2020. arXiv preprint arXiv:2010.04159. 3, 2010.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction of this paper clearly identify the challenges in existing MOT methods and appropriately present the proposed solutions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are explicitly stated in Section Conclusion and Limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
We conducted both quantitative and qualitative ablation studies for all assump-
tions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
The experimental environment and training configurations are described in Section 4.2 Implementation Details of the main text.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
15

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The code and result files have been made publicly available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ /nwpp...c////////o https/ccppblcccguieeeeeooo Stupppuiiiiguuuieeeeeettttutaaia
/CooeSuboeeeeeeeee wetsSSubiissiiiiii wwwiissssoooiiiieeieoeeooo onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The experimental environment and training configurations are described in Section 4.2 Implementation Details of the main text.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We conducted experiments on multiple datasets and reported the results from the official evaluation servers.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
is specified Section 4.2 Implementation Details.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that NeurlPS Code of Ethics atttp:://eeurie httpp/rwwpp.pppppppieeeeeeeeeeee hwwbbli//Ehiccccuideennee
didn't It make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the
Answer: [Yes]
Justification: The research was conducted in accordance with the Code of Ethics, using publicly available datasets.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
It is appropriately described in the Introduction section.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do not use any data or models that pose a high risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The assets used in the paper are properly cited with appropriate references.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswithcode. thcode. com/ datasets
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
18

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: All newly introduced assets, including the model code and checkpoints, are released with accompanying documentation covering training details, usage instructions, license information, and limitations. This ensures clarity and reproducibility.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve human subjects or crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This work does not involve human subjects or crowdsourcing.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
19

---

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
20


Reviews:
[{"review_id": "NKdIx9XNc5", "reviewer": "Reviewer_N3nk", "summary": "This paper proposes a novel dual-path temporal decoder to enhance temporal object consistency in transformer-based track-by-query MOT frameworks (e.g., MOTR, TrackFormer). The decoder is integrated into the DINO detector, differing from the Deformable DETR detector used in MOTR/TrackFormer. Specifically, the proposed decoder comprises two parallel branches: an appearance-adaptive branch that refines query features using current frame information, and an identity-preserving branch that maintains temporal consistency by reusing fixed query features and historical sampling offsets (from the deformable attention in DINO) from the previous frame. Additionally, a query feature update strategy is introduced where features are not updated in frames with low confidence scores.\nQuantitative experiments demonstrate SOTA performance of 76.2 HOTA on the DanceTrack dataset, and competitive performance on the SportsMOT dataset and MOT17 validation set. Ablation studies on the DanceTrack validation set validate the effectiveness of the main designs.", "questions": "See the weaknesses section. Each point corresponds to a suggestion.", "limitations": "no", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The paper is clearly-written.\n2. The designed dual-path temporal decoder presents a well-motivated and novel approach to enhancing temporal consistency.\n3. The proposed method achieves excellent performance on the DanceTrack and SportsMOT datasets.  Specifically, it achieves 76.2 HOTA on DanceTrack, establishing a new SOTA result on this benchmark. Furthermore, it demonstrates excellent association performance, highlighting its effectiveness in improving temporal consistency and reducing ID switches.\n\nWeaknesses\n1. The paper reports results for MOT17 on the validation set but omits test set results. The reason for not reporting test set results is not provided.\n2. While MOTRv2 identifies limited detection performance as a key weakness in MOTR and addresses it with an additional detector, and MOTRv3 attributes this to optimization imbalance between detection and track queries, the proposed method achieves better detection performance than MOTRv2. It remains unclear whether this improvement stems primarily from DINO's stronger detection capability and the query denoising technique. Analyzing DINO's contribution in the ablation study would offer valuable insights.\n3. The main text lacks qualitative results (e.g., visualizations of tracking outputs, failure cases) demonstrating the method's effectiveness in improving association performance and reducing identity switches, particularly on challenging sequences. Additionally, the method section is overly detailed. Streamlining this section could save space to incorporate visualizations currently in the supplementary material \n4. The computational overhead introduced by the dual-path decoder and the overall inference speed of the framework are not reported in the main text.\n5. The modification to attention weight computation in the deformable attention (L137-139) is not validated in the ablation study.", "was_revised": true, "final_justification": "The authors' rebuttal addressed my primary concerns:\n* Lack of evaluation on the MOT17 test set: The newly reported performance (Table in A1) on the MOT17 test set demonstrates significantly better association performance in terms of IDF1 and AssA metrics than that of other end-to-end methods.\n* Impacts of base detector: The newly added experiments (Table in A2) with YOLOX and Deformable-DETR offer a fair comparison with MOTRv2 and MOTIP. These results show the performance superiority stems from the proposed method's ability to achieve obviously stronger association performance than the counterparts, which well supports the central claim of the paper.\n* Lack of validation of affinity-based attention: The authors have added an ablation study explicitly analyzing this component.\n* Minor presentational issues: The authors will streamline the method section (which currently contains redundant notations), and move visualizations/efficiency comparisons from supplementary material to the main text.\n\nFurthermore, considering the stronger association performance of their proposed method compared to other end-to-end MOT approaches and the new SOTA results achieved on the challenging DanceTrack dataset, I have raised my rating to Accept.", "rebuttal": "We sincerely appreciate the reviewer’s time and constructive feedback. Your insightful comments have helped us clarify and improve the manuscript, and we address each point in detail below.\n\n**A1. Results on the MOT17 test set:**\n\nWe were unable to include MOT17 test set results in the submission due to delays in account approval for test server access, preventing evaluation before the deadline. We have now completed this evaluation and report the results in the table below (with private detections). MOTRv2 &dagger; denotes the result of MOTRv2 without heuristic post-processing algorithms, as reported in the experimental results in the paper of MOTIP. Our method achieves HOTA 61.5, AssA 62.5, and IDF1 75.1, outperforming all non-heuristic end-to-end trackers, including MOTRv2† (HOTA 57.6, AssA 57.5, IDF1 70.3) and MOTIP (59.3 / 57.0 / 71.3). Despite lower detection accuracy (MOTA 73.8) than ColTrack (MOTA 78.8) and MOTIP (MOTA 75.3), our stronger association capability enables higher HOTA than all heuristic-free approaches, narrowing the gap to heuristic-augmented pipelines. Compared to the full MOTRv2 with heuristic post-processing, our method achieves superior AssA (62.5 vs. 60.6) and comparable IDF1 (75.1 vs. 75.0) while remaining fully end-to-end. We will include this table and explanation in the revised manuscript to clarify that our dual-path design achieves state-of-the-art association accuracy and competitive overall HOTA among purely end-to-end trackers, demonstrating its effectiveness in maintaining identities even with lower raw detection quality.\n| **Method**        | **HOTA** | **DetA** | **AssA** | **MOTA** | **IDF1** |\n|--------------------|----------|----------|----------|----------|----------|\n| **heuristic**      |          |          |          |          |          |\n| MixSort-OC         | 63.4     | 63.8     | 63.2     | 78.9     | 77.8     |\n| MixSort-Byte       | 64.0     | 64.1     | 64.2     | 79.3     | 78.7     |\n| Deep OC-SORT       | **64.9**     | -        | **65.9**     | 79.4     | **80.6**     |\n| DeconfuseTrack     | **64.9**     | **65.0**     | 65.1     | **80.4**     | **80.6**     |\n| **end-to-end** |       |          |          |          |          |\n| MOTR              | 57.8     | 60.3     | 55.7     | 73.4     | 68.6     |\n| TransTrack        | 54.1     | 61.6     | 47.9     | 74.5     | 63.9     |\n| MOTRv2&dagger;            | 57.6     | 58.1    | 57.5     | 70.1    | 70.3     |\n| MOTRv2            | **62.0**     | **63.8**     | 60.6     | 78.6     | 75.0     |\n| TrackFormer       | -        | -        | -        | 74.1     | 68.0     |\n| MOTIP             | 59.3     | 62.0     | 57.0     | 75.3     | 71.3     |\n| ColTrack          | 61.0     | -        | -        | **78.8**     | 73.9     |\n| **Ours**          | 61.5| 60.8 | **62.5** | 73.8| **75.1** |\n\n\n**A2. Detector contribution analysis:**\n\nThe improvement of our framework over MOTRv2 is not solely due to using a stronger detector (DINO) or its query denoising mechanism, but primarily to our dual-path temporal modeling. To ensure a fair comparison, we reproduced experiments on the DanceTrack validation using the same detectors adopted by prior methods as in the table below:\n\n- YOLOX detector (mAP 72.1, used in MOTRv2): Our method achieves HOTA 67.9 vs. 64.5 (MOTRv2), despite identical detection inputs.\n\n\n- Deformable DETR (mAP 63.7, used in MOTIP): Our method improves HOTA 66.4 vs. 62.2 and IDF1 70.6 vs. 64.8 over MOTIP under the same detection quality.\n\n\n- DINO (mAP 73.1, used in ColTrack): While providing a stronger baseline, our framework still achieves HOTA 69.1 vs. 61.9  and IDF1 74.9 vs. 61.6 over ColTrack, showing consistent gains independent of the detector choice.\n\nThese results demonstrate that detection quality alone does not explain our improvements. The dual-path decoder and stable query propagation consistently enhance association accuracy across all detector settings. We will clarify this experimental setup and add an explicit detector-wise ablation in the revised manuscript.\n\n\n| Detector        | mAP  | Tracker    | HOTA | DetA | AssA | MOTA | IDF1 |\n|-----------------|------|------------|------|------|------|------|------|\n| YOLOX           | 72.1 | MOTRv2     | 64.5 | **78.7** | 53.0 |   -  |   -  |\n|                 |      | Ours       | **67.9** | 77.2 | **60.0** | 87.2 | 73.3 |\n| Deformable DETR | 63.7 | MOTIP      | 62.2 | 75.3 | 51.5 | 85.2 | 64.8 |\n|                 |      | Ours       | **66.4** | **77.1** | **57.3** | **85.9** | **70.6** |\n| DINO            | 73.1 | ColTrack   | 61.9 |   -  |   -  | 86.5 | 61.6 |\n|                 |      | Ours       | **69.1** | 77.8 | 61.6 | **87.5** | **74.9** |\n\n\n\n**A3. Qualitative results in main paper:**\n\nWe thank the reviewer for the valuable suggestion. Our supplementary material (Figs S-3 to S-5) provides strong qualitative evidence showing that our method significantly reduces identity switches and improves association stability, particularly under challenging occlusions. We will move these key visualizations into the main text to better complement the quantitative gains reported. Additionally, we will streamline Section 3 by condensing redundant derivations and shifting technical details to the supplementary, freeing space for qualitative results. These revisions will make the manuscript more concise, accessible, and visually convincing while preserving technical rigor.\n\n\n**A4. Computational complexity:**\n\nThe supplementary material (Table S-1) reports detailed efficiency metrics: our method achieves 12.7 FPS on NVIDIA 4090Ti while using fewer parameters (11.3M) than other end-to-end baselines (e.g., ColTrack 15.8M, 11.3 FPS), with only a minor FLOPs increase. We will incorporate these results into the main text and clarify that the additional branch adds negligible computational cost relative to single-path transformers while delivering substantial association improvements.\n\n**A5. Ablation on deformable attention weight computation:**\n\nWe thank the reviewer for this insightful comment. We conducted an additional ablation study validating  the effectiveness of the proposed affinity-based attention weight computation (lines 137–139). As shown in the table below, replacing standard query-based weights with our affinity formulation yields consistent gains across all metrics on DanceTrack: HOTA 69.1 vs. 67.6, AssA 61.6 vs. 59.1, and IDF1 74.9 vs. 72.7, while maintaining similar DetA and MOTA. This demonstrates that computing attention weights from affinity between query and sampled image features improves association accuracy and identity preservation. We will include these results in the revised manuscript to explicitly validate this design choice.\n\n| **Attention weight computation method** | **HOTA** | **DetA** | **AssA** | **MOTA** | **IDF1** |\n|-----------------------------------------|----------|----------|----------|----------|----------|\n| Query-based                             | 67.6     | 77.6     | 59.1     | **87.7** | 72.7     |\n| Affinity-based (ours)                   | **69.1** | **77.8** | **61.6** | 87.5     | **74.9** |"}, {"review_id": "D4VItdIXHB", "reviewer": "Reviewer_7Vx5", "summary": "The paper introduces a dual-path temporal decoder in transformer-based MOT, which separates appearance adaptation and identity preservation using two sets of decoders. The evaluation on the Dancetrack and SportMOT datasets demonstrates the effectiveness of the proposed dual-path temporal decoder.", "questions": "1. The performance of the tracker adopting standard DETR should be ablated.\n2. Introduce the loss functions for each decoder.\n3. Provide results on the test sets of the MOT17 and MOT20 datasets.\n4. Provide the computational complexity analysis.", "limitations": "Yes.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength\n1. The concept of separating appearance and motion is commonly employed in tracking by detection methods. It is interesting to propose such thoughts in the transformer-based methods.\n2. The presentation is good. The writing is easy to follow, and the figures are clear.\n\nWeakness\n1. The comparisons are not fair.  Other transformer-based methods adopt standard DTER as the object detector, while the proposed tracker adopts a DINO-based detector.\n2. The loss functions are missing, which makes the learning objective of each decoder unclear.\n3. The benchmark results on the MOT17 and MOT20 datasets are based on the validation sets, which share the same video sequences as the training sets. Such settings make the results not convincing enough.\n4. Since the paper introduces two decoders compared to one decoder in existing methods, the computational complexity should be included and compared with existing methods.", "was_revised": true, "final_justification": "I maintain the negative rating as the performance is not even as good as MOTRv2 in HOTA on the MOT17 dataset, and it requires a larger computation cost. Although this justification is very strict, the performance of HOTA and computation is key for the MOT task.", "rebuttal": "We sincerely appreciate the reviewer’s time and constructive feedback. Your insightful comments have helped us clarify and improve the manuscript, and we address each point in detail below.\n\n**A1. Comparison with different object detectors:**\n\nAs shown in Table 1 (DanceTrack) and Table 2 (SportsMOT) of our paper, our method consistently outperforms ColTrack even when both trackers use the same DINO detector, demonstrating that the observed improvements are not simply due to adopting a stronger detector. To further ensure fairness, we also reproduced experiments on the DanceTrack validation using the detectors employed by other transformer-based MOT methods  (YOLOX for MOTRv2 and Deformable DETR for MOTIP). As in the table below, our tracker achieves superior results under identical detection inputs. These results validate that our gains mainly originate from the proposed dual-path temporal design and stable query propagation, rather than from detector choice. We will clarify this evaluation protocol and explicitly report detector-wise ablation results in the revised manuscript.\n| Detector        | mAP  | Tracker    | HOTA | DetA | AssA | MOTA | IDF1 |\n|-----------------|------|------------|------|------|------|------|------|\n| YOLOX           | 72.1 | MOTRv2     | 64.5 | **78.7** | 53.0 |   -  |   -  |\n|                 |      | Ours       | **67.9** | 77.2 | **60.0** | 87.2 | 73.3 |\n| Deformable DETR | 63.7 | MOTIP      | 62.2 | 75.3 | 51.5 | 85.2 | 64.8 |\n|                 |      | Ours       | **66.4** | **77.1** | **57.3** | **85.9** | **70.6** |\n| DINO            | 73.1 | ColTrack   | 61.9 |   -  |   -  | 86.5 | 61.6 |\n|                 |      | Ours       | **69.1** | 77.8 | 61.6 | **87.5** | **74.9** |\n\n**A2. Loss function:**\n\nIn lines 206–210, we clarify that our framework uses the standard bipartite matching loss [7] adopted in transformer-based MOT methods [16, 35, 40]. Specifically, the overall loss is a weighted sum of focal classification loss, L1 box regression loss, and generalized IoU loss, computed after the outputs of the appearance-adaptive and identity-preserving decoders are fused. Since both paths jointly contribute to the final predictions, the loss is applied only once to the fused output. We will explicitly include this formulation in the revised manuscript to clarify the learning objective.\n\n**A3. Results on the MOT17 test set:**\n\nWe were unable to include MOT17 test set results in the submission due to delays in account approval for test server access, preventing evaluation before the deadline. We have now completed this evaluation and report the results in the table below (with private detections). MOTRv2 &dagger; denotes the result of MOTRv2 without heuristic post-processing algorithms, as reported in the experimental results in the paper of MOTIP. Our method achieves HOTA 61.5, AssA 62.5, and IDF1 75.1, outperforming all non-heuristic end-to-end trackers, including MOTRv2† (HOTA 57.6, AssA 57.5, IDF1 70.3) and MOTIP (59.3 / 57.0 / 71.3). Despite lower detection accuracy (MOTA 73.8) than ColTrack (MOTA 78.8) and MOTIP (MOTA 75.3), our stronger association capability enables higher HOTA than all heuristic-free approaches, narrowing the gap to heuristic-augmented pipelines. Compared to the full MOTRv2 with heuristic post-processing, our method achieves superior AssA (62.5 vs. 60.6) and comparable IDF1 (75.1 vs. 75.0) while remaining fully end-to-end. We will include this table and explanation in the revised manuscript to clarify that our dual-path design achieves state-of-the-art association accuracy and competitive overall HOTA among purely end-to-end trackers, demonstrating its effectiveness in maintaining identities even with lower raw detection quality.\n\n| **Method**        | **HOTA** | **DetA** | **AssA** | **MOTA** | **IDF1** |\n|--------------------|----------|----------|----------|----------|----------|\n| **heuristic**      |          |          |          |          |          |\n| MixSort-OC         | 63.4     | 63.8     | 63.2     | 78.9     | 77.8     |\n| MixSort-Byte       | 64.0     | 64.1     | 64.2     | 79.3     | 78.7     |\n| Deep OC-SORT       | **64.9**     | -        | **65.9**     | 79.4     | **80.6**     |\n| DeconfuseTrack     | **64.9**     | **65.0**     | 65.1     | **80.4**     | **80.6**     |\n| **end-to-end** |       |          |          |          |          |\n| MOTR              | 57.8     | 60.3     | 55.7     | 73.4     | 68.6     |\n| TransTrack        | 54.1     | 61.6     | 47.9     | 74.5     | 63.9     |\n| MOTRv2&dagger;            | 57.6     | 58.1    | 57.5     | 70.1    | 70.3     |\n| MOTRv2            | **62.0**     | **63.8**     | 60.6     | 78.6     | 75.0     |\n| TrackFormer       | -        | -        | -        | 74.1     | 68.0     |\n| MOTIP             | 59.3     | 62.0     | 57.0     | 75.3     | 71.3     |\n| ColTrack          | 61.0     | -        | -        | **78.8**     | 73.9     |\n| **Ours**          | 61.5| 60.8 | **62.5** | 73.8| **75.1** |\n\n\n**A4. Computational complexity of the proposed dual-decoder:**\n\nComputational complexity comparisons are already included in our supplementary material (Table S-1), reporting FLOPs, parameter counts, and FPS against representative transformer-based MOT baselines (MOTRv2, MOTIP, ColTrack). These results show that our model remains parameter-efficient (11.3M vs. 41.7M for MOTRv2, 58.9M for MOTIP) and achieves comparable or better runtime speeds.\n\nTo further address this point, we have added an additional experiment (the table below) directly comparing our dual-path decoder to a single-path variant without IDL. The results show only a negligible overhead (620.8G → 623.0G FLOPs, 13.0 → 12.7 FPS). This is because the two decoders partially share parameters, and their attention computations are executed in parallel, minimizing extra cost despite the dual-path architecture. We will make these details explicit in the main paper to clarify the computational trade-offs of the dual-path design.\n\n| Method                       | FLOPs (G) | Params (M) | FPS  |\n|------------------------------|-----------|------------|------|\n| MOTRv2                       | 431.8     | 41.7       | 25.7 |\n| MOTIP                        | -         | 58.9       | 12.0 |\n| ColTrack                     | 600.9     | 15.8       | 11.3 |\n| Ours without IDL (single-path decoder) | 620.8     | 10.9       | 13.0 |\n| Ours (dual-path decoder)     | 623.0     | 11.3       | 12.7 |"}, {"review_id": "8owJRFQSVM", "reviewer": "Reviewer_LCuU", "summary": "The paper introduces a new multi-object tracking (MOT) architecture. The method is built upon DINO-DETR, a variant of the detection transformer (DETR). \n\nThe authors design **a new *dual-path* transformer decoder**. This decoder takes as input:\n1. *track queries*: which come from objects detected at the previous frame\n2. *candidate query features*: which are object candidates from the transformer encoder module (top-k scoring objects).\n\nEvery decoder layer is composed of two parallel decoder layers (i.e., the *appearance-adaptive* and the *identity-preserving* decoder layers) followed by a *feature fusion* module. \n- The *appearance-adaptative* module iteratively refines the object queries using the image features, following DINO-DETR/Deformable-DETR.\n- The *identity-preserving* module follows a similar design, but keeps track queries fixed (from one layer to the other) and reuses historical sampling offsets. This helps safeguard identity-specific information from the past. \n\nThe authors **additionally introduce a selective update mechanism** which only updates queries from previous frames if the object is predicted with high (classification) confidence, to help preserve clean identity embeddings and thus improve long-term tracking.\n\nExperimentally, **the method outperforms previous works on the test sets from DanceTrack and SportsMOT**. It also achieves relatively high accuracy on MOT17's validation split. \nFurthermore, the **ablation studies validate** the utility of the new dual path decoder and the confidence-guided update rule.", "questions": "1. Could the authors evaluate their method on the official test set from MOT17 instead of the validation set?\n2. Could the authors discuss more in-depth the DETR-like related works (most importantly TransTrack, as it shares many similarities with this method)?\n3. Do you observe any tracking performance degradation under fast motion? Indeed, one would expect that re-using historical offsets in the identity-preserving decoder layers may be detrimental in this case. \n\nThe reviewer is willing to raise the rating if the requests and questions are addressed; however, they will lower the rating if the two main concerns (1 and 2) remain unaddressed. \n\n\n*[Remark: the reviewer is familiar with DETR-based MOT methods but is NOT an author of TransTrack]*", "limitations": "No, the authors only briefly discuss limitations. The reviewer suggests discussing further the method's limitations and future research directions. \n\nFurthermore, the paper does not discuss the potential negative societal impact of MOT (e.g., for autonomous killing robots, surveillance, etc.).", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n- The method outperforms state-of-the-art methods on both DanceTrack and SportsMOT.\n- The adaptive update rule is novel and could be beneficial in other methods as well.\n\n### Weaknesses\n\n- The Method section is not easy to read. It would benefit from being improved (e.g., by starting with an overview/intuition instead of notations, etc.). \n- The related work section does not discuss the most similar work, TransTrack [1]. Discussing it would help readers understand how it is similar and how it differs from similar DETR-like approaches: The TransTrack paper, from 2020, also uses two types of queries, a dual transformer decoder, and confidence scores. However, this submission brings a new decoder design and introduces an interesting selective update mechanism.\n- The paper is evaluated on the validation split from MOT17, rather than the official test set. This practice should be avoided, especially when the validation split comes from the same videos as the training split. \n\n\n[1] Sun, Peize, et al. \"Transtrack: Multiple object tracking with transformer.\" arXiv preprint arXiv:2012.15460 (2020).", "was_revised": true, "final_justification": "The other reviewers primarily raised similar concerns to mine, such as evaluating on the validation set rather than the test set (of MOT17) and potential challenges in scenarios involving fast motion. These issues have been adequately addressed. Additionally, the updated related works section now offers a clear comparison with the most relevant similar approaches.\n\nAlthough the paper remains somewhat challenging to follow, the authors' clarifications and updates have significantly improved its quality. I thus believe the paper meets the required standard and accordingly raise my score to \"accept\".", "rebuttal": "We sincerely appreciate the reviewer’s time and constructive feedback. Your insightful comments have helped us clarify and improve the manuscript, and we address each point in detail below.\n\n**A1. Enhancing clarity of the Method section:**\n\nWe will revise the Method section by first providing an intuitive overview of the proposed model along, clearly illustrating the overall model architecture and tracking pipeline as shown in the example below.. \n\nFigure 1 provides an overview of our proposed framework. Similar to prior DETR-based trackers, object queries are propagated across frames and updated by a transformer decoder. However, existing single-path designs struggle to maintain temporal consistency: noisy or low-confidence observations can corrupt query features, causing errors that accumulate over time and lead to identity switches. To address this, we introduce a dual-path temporal decoder that explicitly considers both adaptability and stability in query evolution. The appearance-adaptive path focuses on refining query embeddings with current-frame information for accurate localization and adaptation to visual changes. In parallel, the identity-preserving path retains stable features and reuses historical offsets, preventing feature drift when observations are unreliable. The outputs from both paths are fused, providing a balanced representation that is both flexible to genuine motion and robust to noise. A confidence-guided suppression mechanism further filters low-confidence updates, ensuring that only reliable information is propagated to subsequent frames. This architecture directly tackles the temporal stability problem that prior DETR-like methods leave unresolved, leading to improved long-term identity preservation and stronger association performance.\n\n**A2. Result on the MOT17 test set:**\n\nWe were unable to include MOT17 test set results in the submission due to delays in account approval for test server access, preventing evaluation before the deadline. We have now completed this evaluation and report the results in the table below (with private detections). MOTRv2 &dagger; denotes the result of MOTRv2 without heuristic post-processing algorithms, as reported in the experimental results in the paper of MOTIP. Our method achieves HOTA 61.5, AssA 62.5, and IDF1 75.1, outperforming all non-heuristic end-to-end trackers, including MOTRv2† (HOTA 57.6, AssA 57.5, IDF1 70.3) and MOTIP (59.3 / 57.0 / 71.3). Despite lower detection accuracy (MOTA 73.8) than ColTrack (MOTA 78.8) and MOTIP (MOTA 75.3), our stronger association capability enables higher HOTA than all heuristic-free approaches, narrowing the gap to heuristic-augmented pipelines. Compared to the full MOTRv2 with heuristic post-processing, our method achieves superior AssA (62.5 vs. 60.6) and comparable IDF1 (75.1 vs. 75.0) while remaining fully end-to-end. We will include this table and explanation in the revised manuscript to clarify that our dual-path design achieves state-of-the-art association accuracy and competitive overall HOTA among purely end-to-end trackers, demonstrating its effectiveness in maintaining identities even with lower raw detection quality.\n\n| **Method**        | **HOTA** | **DetA** | **AssA** | **MOTA** | **IDF1** |\n|--------------------|----------|----------|----------|----------|----------|\n| **heuristic**      |          |          |          |          |          |\n| MixSort-OC         | 63.4     | 63.8     | 63.2     | 78.9     | 77.8     |\n| MixSort-Byte       | 64.0     | 64.1     | 64.2     | 79.3     | 78.7     |\n| Deep OC-SORT       | **64.9**     | -        | **65.9**     | 79.4     | **80.6**     |\n| DeconfuseTrack     | **64.9**     | **65.0**     | 65.1     | **80.4**     | **80.6**     |\n| **end-to-end** |       |          |          |          |          |\n| MOTR              | 57.8     | 60.3     | 55.7     | 73.4     | 68.6     |\n| TransTrack        | 54.1     | 61.6     | 47.9     | 74.5     | 63.9     |\n| MOTRv2&dagger;            | 57.6     | 58.1    | 57.5     | 70.1    | 70.3     |\n| MOTRv2            | **62.0**     | **63.8**     | 60.6     | 78.6     | 75.0     |\n| TrackFormer       | -        | -        | -        | 74.1     | 68.0     |\n| MOTIP             | 59.3     | 62.0     | 57.0     | 75.3     | 71.3     |\n| ColTrack          | 61.0     | -        | -        | **78.8**     | 73.9     |\n| **Ours**          | 61.5| 60.8 | **62.5** | 73.8| **75.1** |\n\n\n**A3.Comparison and discussion of DETR-like related works:**\n\nWe agree that clarifying the relationship to DETR-like trackers, particularly TransTrack, will strengthen the paper. In the revised manuscript, we will expand Section 2 to highlight the architectural differences:\n\n- TransTrack: Employs the dual-decoder design, where one decoder handles detection queries and the other handles tracking queries. The two branches operate largely independently, extracting features separately for detection and tracking, without explicit mechanisms for cross-path feature refinement or identity stability.\n\n\n- MOTRv2, ColTrack: Use the single-path decoder that processes detection and tracking jointly. While this design unifies the tasks, it lacks any temporal stability control. Features for active tracks are continuously updated in the same pathway, making them prone to drift and identity switches under noisy observations.\n\n\n- Our method: Introduces the dual-path temporal decoder, where both paths jointly handle detection and tracking, but each is specialized for complementary roles: 1) the appearance-adaptive path dynamically refines query features to improve localization and adapt to visual changes and 2) the identity-preserving path maintains stable historical features and reuses past sampling offsets to prevent drift.\n\nThe two paths are fused, combining adaptability and stability in a principled manner that is absent in prior DETR-like approaches. This formulation makes our approach conceptually distinct from both TransTrack and one-path designs (MOTRv2, ColTrack). We explicitly introduce temporal consistency control as a missing component in prior DETR-based trackers, leading to stronger association accuracy and fewer ID switches. We will revise the related work section to make these distinctions clear.\n\n\n**A4. Analysis of tracking performance under fast motion:**\n\nWe conducted an additional analysis on the DanceTrack validation set, grouping trajectories by frame-to-frame object displacement to evaluate whether re-using historical offsets impacts tracking under fast motion. As shown in the table below, our method achieves consistently higher accuracy scores than the baseline without historical offsets across all displacement ranges. The improvement remains clear even for fast motions (>40 pixel), indicating that the dual-path decoder with historical offset reuse maintains robust tracking accuracy under fast motion. This stability arises because the appearance-adaptive path dynamically refines features for localization, while the identity-preserving path exploits historical offsets to preserve identity consistency.\n| Distance (pixel) | without historical offset | with historical offset (Ours) |\n|------------------|---------------------------|--------------------------------|\n| 0 ~ 20          | 71.1                      | 73.4 (+2.3)                   |\n| 20 ~ 40         | 71.1                      | 73.1 (+2.0)                   |\n| 40 ~ ∞          | 63.0                      | 65.2 (+2.2)                   |\n\n\n\n**A5. Discussion of method limitations, future directions, and societal impact:**\n\nWe will add a dedicated limitations and future directions section discussing handling long-term occlusions and scalability to dense scenes, outlining paths for improvement. Furthermore, we acknowledge the potential negative societal impacts of MOT, such as surveillance misuse or autonomous weapon deployment, and will include a broader impact statement addressing dual-use risks and the importance of ethical safeguards. These additions will strengthen the paper’s transparency and societal awareness."}, {"review_id": "aHgspik2KL", "reviewer": "Reviewer_sKej", "summary": "This paper presents a transformer-based Multi-Object Tracking (MOT) framework with a dual-path temporal decoder that separates appearance adaptation and identity preservation. The key contribution is decomposing the temporal modeling into two parallel branches: an appearance-adaptive decoder that dynamically updates query features using current frame information, and an identity-preserving decoder that maintains temporal consistency by freezing track query features and reusing historical sampling offsets. Additionally, the paper introduces a confidence-guided update suppression strategy that prevents unreliable feature updates during inference. Experimental results demonstrate state-of-the-art performance on DanceTrack and SportsMOT benchmarks.", "questions": "1. Identity-Preserving Mechanism Validity: The reuse of sampling offsets $\\Delta^{(t-1)}_T$ assumes similar attention patterns across consecutive frames. How does this approach handle scenarios with rapid object motion or camera movement where the optimal sampling locations should change significantly? Have you considered learning a dynamic offset adjustment rather than direct reuse?\n2. Confidence-Guided Update Recovery: When an object has low confidence due to temporary occlusion or scene changes, the current mechanism freezes its features indefinitely until confidence recovers. What happens if the confidence threshold is never met again? Do you have mechanisms to prevent permanent feature degradation, and how do you handle re-emerging objects after extended periods?\n3. Long-term vs. Short-term Trade-offs: The identity-preserving decoder only maintains consistency with the immediate previous frame ($t-1$). How does this limited temporal window perform in scenarios requiring longer-term memory? Have you experimented with incorporating features from multiple previous frames or implementing a more sophisticated temporal memory mechanism?\n4. Notation Clarity and Experimental Fairness: The notation $Q_T$ and $Q_T^{(l)}$ can be confusing. Additionally, are the detector backbones consistent across all compared methods?", "limitations": "1. Temporal Window Limitations: The paper should acknowledge that the identity-preserving mechanism only considers $t-1$ frame information, which may be insufficient for complex long-term tracking scenarios.\n2. Recovery Mechanisms: Lack of discussion about failure recovery when confidence-guided suppression leads to permanent feature freezing.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. Clear Technical Approach: The dual-path temporal decoder design is well-motivated and addresses a key challenge in transformer-based MOT, which maintains temporal consistency while adapting to appearance changes.\n\n2. Strong Experimental Results: The method achieves state-of-the-art performance on challenging benchmarks (DanceTrack HOTA: 74.1, SportsMOT HOTA: 73.9).\n\n3. Comprehensive Ablation Studies: The paper provides thorough ablation studies validating both the identity-preserving decoder layer and confidence-guided update suppression components.\n\nWeaknesses\n1. Limited Novelty in Individual Components: While the combination is effective, the individual components (deformable attention, confidence thresholding) are relatively standard. The core novelty lies primarily in the dual-path architecture design.\n2. Questionable Identity-Preserving Strategy: The identity-preserving approach using $t-1$ frame offsets $\\Delta^{(t-1)}_T$ seems overly simplistic for dynamic tracking scenarios. This only considers adjacent frame consistency rather than long-term temporal stability, which limits its effectiveness for long-range tracking.\n3. Insufficient Confidence-Guided Update Suppression Justification: The strategy of completely discarding features below threshold $\\beta$ may be too aggressive. In scenarios with sudden scene changes, objects might have consistently low confidence, leading to permanent feature freezing. The paper lacks a discussion of failure cases and recovery mechanisms.\n4. Limited Long-term Tracking Analysis: The approach primarily addresses frame-to-frame stability but doesn't convincingly demonstrate improvements in long-term tracking scenarios with extended occlusions or re-identification.", "was_revised": true, "final_justification": "Overall, I still feel that the innovation aspect is lacking. However, the final experimental results are quite good. During the rebuttal stage, the discussion on innovation merely reiterates the main text without providing strong convincing power. Nonetheless, the related experiments do possess a certain degree of persuasiveness; therefore, I maintain my original score.", "rebuttal": "We sincerely appreciate the reviewer’s time and constructive feedback. Your insightful comments have helped us clarify and improve the manuscript, and we address each point in detail below.\n\n**A1.  Core novelty in the dual-path architecture design:**\n\nThe comment notes that deformable attention and confidence thresholding are standard tools, but our contribution is not a simple combination of existing techniques. The dual-path temporal design is a novel mechanism that explicitly decouples adaptability and stability, a capability not addressed by prior transformer-based MOT approaches. We also introduce a modified affinity-based attention computation (lines 137–139) that enhances the dual-path decoder’s ability to align queries with relevant visual features, further improving association accuracy and identity preservation. Ablation results in Table 4 and the table below show that each of these contributions provides measurable gains, and their combination establishes a principled framework for temporal consistency control rather than a simple aggregation of standard components. We will include the new ablation study in the revised manuscript. Finally, our confidence-guided update suppression strategy selectively filters out unreliable updates under low-confidence predictions, ensuring that only stable and reliable query features are propagated to the next frame’s decoder.\n| **Attention weight computation method** | **HOTA** | **DetA** | **AssA** | **MOTA** | **IDF1** |\n|-----------------------------------------|----------|----------|----------|----------|----------|\n| Query-based                             | 67.6     | 77.6     | 59.1     | **87.7** | 72.7     |\n| Affinity-based (ours)                   | **69.1** | **77.8** | **61.6** | 87.5     | **74.9** |\n\n\n**A2. Validity of the identity-preserving mechanism under rapid motion:**\n\nThe motivation behind re-using historical offsets is to mitigate feature drift caused by noisy updates, a known limitation of existing transformer-based MOT frameworks. This approach is designed based on the empirical observation (see Figs. S-1 and S-2 in the supplementary) that attention sampling patterns for the same object remain largely stable between consecutive frames. Our expectation is that by maintaining reliable associations between adjacent frames, we can achieve long-term temporal stability without requiring explicit long-horizon modeling, as temporal identity consistency is naturally preserved when consecutive links are robust.\n\nRegarding rapid object or camera motion, both paths in our dual-path decoder contribute to maintaining accuracy. The appearance-adaptive path dynamically updates queries to capture large displacements and sudden motion changes, while the identity-preserving path reuses stable offsets to prevent identity switches when motion patterns remain partially consistent. The table below shows the matching accuracy on the DanceTrack validation according to the frame-to-frame object displacement  for fast-motion analysis. The proposed combination with historical offset consistently improves the matching accuracy scores across all displacement ranges, including >40 px, indicating that our approach remains robust even in challenging fast-motion scenarios.\n| Distance (pixel) | without historical offset | with historical offset (Ours) |\n|------------------|---------------------------|--------------------------------|\n| 0 ~ 20          | 71.1                      | 73.4 (+2.3)                   |\n| 20 ~ 40         | 71.1                      | 73.1 (+2.0)                   |\n| 40 ~ ∞          | 63.0                      | 65.2 (+2.2)                   |\n\nWe acknowledge that extreme conditions such as prolonged occlusions or abrupt camera changes may limit the effectiveness of direct offset reuse. Incorporating a learnable offset adjustment module, or combining offset memory with global motion cues, is a promising future direction we plan to explore. Nonetheless, current results demonstrate that our effective identity-preserving strategy already provides substantial gains in temporal stability without introducing additional complexity. \n\n\n\n**A3.  Effect of confidence-guided update suppression for consistent low confidence due to long-term occlusions:**\n\nThe table below reports tracking accuracy on trajectories in DanceTrack validation that disappear from the ground truth for more than 30 frames due to occlusions and later reappear, evaluating how well each method recovers the correct identity after long-term absence, which yields consistent low confidence. Our method with confidence-guided update suppression achieves the highest score (0.192), outperforming that without suppression (0.188) and other transformer-based trackers such as MOTRv2 (0.139) and ColTrack (0.160). This demonstrates that discarding low-confidence updates does not harm re-identification and in fact improves robustness, as noisy updates are a stronger source of long-term identity drift.\n\nWe acknowledge that in rare cases where confidence never recovers, a track may eventually fail due to outdated features not being refreshed. However, our analysis shows that the benefit of avoiding error propagation outweighs this risk, leading to higher overall recovery accuracy than competing methods. Long-term occlusion remains a challenging open problem for all trackers, as indicated by the generally low scores in the table below. We plan to address this limitation in future work by incorporating global memory mechanisms and learnable recovery strategies to better maintain identity information across long temporal gaps.\n| Method                                      | Long-term occlusion (> 30 frames) |\n|---------------------------------------------|------------------------------------|\n| MOTRv2                                      | 0.139                              |\n| ColTrack                                    | 0.160                              |\n| Ours without confidence-guided update suppression | 0.188                       |\n| Ours with confidence-guided update suppression    | **0.192**                      |\n\n\n**A4.  Long-term vs. short-term trade offs:**\n\nTo further analyze long-term tracking, we divided trajectories into two groups based on their temporal span: short-term (≤1500 frames) and long-term (>1500 frames). For each trajectory, we computed the mean accuracy as the fraction of frames where the predicted bounding box overlaps the ground-truth with IoU ≥ 0.5, and report the average across all trajectories. As shown in the table below, identity-preserving decoder (IDL) improves accuracy from 68.7 to 70.3 (+1.6) in the short-term regime and from 58.3 to 60.1 (+1.8) in the long-term regime, demonstrating that IDL consistently enhances performance across both temporal scales by preventing error accumulation. \n\nCompared to other transformer-based trackers, our method achieves the highest accuracy in both regimes (70.3 / 60.1 vs. MOTRv2 65.2 / 58.1 and ColTrack 63.6 / 58.9), showing that the proposed dual-path design not only stabilizes adjacent-frame associations but also improves long-range identity preservation beyond existing approaches. We see that all trackers, including ours, exhibit reduced accuracy under long trajectories, highlighting that long-term tracking under extreme temporal gaps remains an open challenge. Future work will explore multi-frame memory modules or advanced temporal fusion mechanisms to further enhance re-identification robustness under extended occlusions or large spatial displacements.\n\n| Method              | Short-term (≤1500) | Long-term (>1500) |\n|--------------------- |------------------- |------------------ |\n| MOTRv2              | 65.2              | 58.1             |\n| ColTrack            | 63.6              | 58.9             |\n| Ours without IDL    | 68.7              | 58.3             |\n| Ours               | **70.3**          | **60.1**         |\n\n\n**A5.  Notation clarity and experimental fairness:**\n\nWe acknowledge that the notations used in our paper may cause confusion. In the revised manuscript, we will clearly define each notation and improve readability. Regarding the question on experimental fairness, we confirm that all compared methods such as MOTR, TrackFormer, MOTRv2, ColTrack, and MOTIP use the same ResNet-50 detector backbone. We will explicitly state this in the revised manuscript to avoid any potential confusion."}]

Author rebuttal:

Meta-review: The paper presents a novel dual-path temporal decoder for multi-object tracking, integrating appearance and motion cues with confidence-guided updates. Reviewers unanimously acknowledge the technical soundness of the approach and its strong empirical performance, particularly on DanceTrack and SportsMOT datasets, with additional MOT17 test set results validating association accuracy. The authors have effectively addressed all major reviewer concerns in the rebuttal, including method clarity, ablation studies, detector-specific contributions, qualitative visualisations, related work comparisons (TransTrack, MOTRv2, ColTrack), and discussion of limitations and societal impacts. While one reviewer remains cautious regarding computational overhead and relative HOTA improvements on MOT17, the consensus is that the paper meets the standards for publication. All critical experiments and results from the rebuttal should be included in the final version. Based on these updates, the paper is recommended for acceptance by the AC and has been approved by the SAC.
Decision: Accept (poster)