UniTrack: Differentiable Graph Representation Learning for Multi-Object Tracking
Authors: Bishoy Galoaa
Xiangyu Bai
Utsav Nandi
Sai Siddhartha Vivek Dhir Rangoju
Somaieh Amraee
Sarah Ostadabbas
Venue: ICLR 2026 Poster (2026)
Keywords: Multi-object tracking
graph representation learning
differentiable optimization
end-to-end learning
identity preservation
spatio-temporal modeling
flow networks
unified loss functions
video understanding
deep learning
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: UniTrack is a differentiable graph representation learning framework that unifies detection, identity preservation, and temporal consistency for improved multi-object tracking.

Abstract: We present UniTrack, a plug-and-play graph-theoretic loss function designed to significantly enhance multi-object tracking (MOT) performance by directly optimizing tracking-specific objectives through unified differentiable learning. Unlike prior graph-based MOT methods that redesign tracking architectures, UniTrack provides a universal training objective that integrates detection accuracy, identity preservation, and spatiotemporal consistency into a single end-to-end trainable loss function, enabling seamless integration with existing MOT systems without architectural modifications. Through differentiable graph representation learning, UniTrack enables networks to learn holistic representations of motion continuity and identity relationships across frames. We validate UniTrack across diverse tracking models and multiple challenging benchmarks, demonstrating consistent improvements across all tested architectures and datasets including Trackformer, MOTR, FairMOT, ByteTrack, GTR,  and MOTE. Extensive evaluations show up to 53\% reduction in identity switches and 12\% IDF1 improvements across challenging benchmarks, with GTR achieving peak performance gains of 9.7\% MOTA on SportsMOT. Code and additional resources are available at https://github.com/ostadabbas/UniTrack and https://ostadabbas.github.io/unitrack.github.io/.

Published as a conference paper at ICLR 2026
UNITRACK: DIFFERENTIABLE GRAPH REPRESENTA- TION LEARNING FOR MULTI-OBJECT TRACKING
Bishoy Galoaa", Xiangyu Bai", Utsav Nandi,
Sai Siddhartha Vivek Dhir Rangoju, Somaieh Amraee, Sarah Ostadabbas Augmented Cognition Lab (ACLab)) Electrical and Computer Engineering Department, Northeastern University "Equal contribution
ABSTRACT
We present UniTrack, plug-and-play graph-theored loss function designed to significantly enhance multi-object tracking (MOT) performance by directly opti- mizing tracking-specific objectives through unified differentiable learning. Un- like prior graph-based MOT methods that redesign tracking architectures, Uni- Track provides universal training objective that integrates detection accuracy, identity preservation, and spatiotemporal consistency into single end-to-end trainable loss function, enabling seamless integration with existing MOT sys- tems without architectural modifications. Through differentiable graph represen- tation learning, UniTrack enables networks to learn holistic representations of motion continuity and identity relationships across frames. We validate UniTraak across diverse tracking models and multiple challenging benchmarks, demonstrat- ing consistent improvements across all tested architectures and datasets including Trackformer, MOTR, FairMOT, Byte'Track, GTR, and MOTE. Extensive evalu- ations show up to 53% reduction identity switches and 12% IDFI improve- ments across challenging benchmarks, with GTR achieving peak performance gains of 9.7% MOTA on SportsMOT. Code and additional resources are avail- able htthtts:: inttps://gittute wtt. wwt.w///ww..caaaaa.ooooao Un Track and https: // Eetaaabbaa..githue htw.://baa..tta.b.....uuaaaaaaaa http://www.raac.i.uuuuuuuuuuuu
INTRODUCTION
(A)
(1)
Maintained [
Ground Truth
ID:
Prediction
Temporal Edge
Goshosin
coundorryy
Spatial Edge (ieter (lete olyooryyeerre rolutions
ID:2
IDJ
1DK
[D:3
DD-2
Component
pronorsstion
Frunse-)
Figure 1: Comparison of UniTrack's 's graph based approach and classical multi-object tracking. (A) A detection- based tracking handles trajectories independently, resulting in ID switches at occlusion: person 1 reassigned to ID 3 and person 2 to ID 1 (crossing arrows and hedhhighl boxes). (B) Our traph-baser approach maintains correct identities through the same occlusion via three inte- grated components: temporal edges (red arrows) for motion consistency, spatial edges (green lines) for inter-object relationships, and flow components (blue dashed ellipses) for identity preservation. Green-highlighte ID boxes show successful identity maintenance throughout the sequence. Ground truth (blue circles) and predictions (dark centers) demonstrate how unified optimization prevents ID switches challenging scenarios.
graph-based
bojee- tracking (MOT) is as a critical element of video- learning, aiding extraction of detailed information about movement, interaction patterns, and spatial relationships in downstream

---

Published as a conference paper at ICLR 2026
applications such as activity recognition, behavior analysis, and anomaly detection. A central ob- jective of MOT is to accurately identify and consistently follow individuals across video frames, assigning unique identifiers to subjects throughout the sequence Zeng et al. MMinnhard et al. (202T]: Zhang et al. ) Wojke et al.] (2017); Galoaa et al. (2025a). While the last decade has seen significant advancements in MOT methodologies, effectively tracking multiple interacting and occluded objects in diverse and complex environments continues to pose substantial challenges. Despite the advances from recent Transformrr-baee approaches such as MOTR Zeng et al. (2021), TrackFormer Meinhardt et al. (02T), and GTR GTR/Zhou et al. (2022) and recent work on multi-camera tracking Galoaa et al. (2025c), inefficiencies stemming from the separation of detection and tracking in traditional approaches partis--paticcularyy under challenging conditions like occlusions, crowded scenes, and complex motion dynamics.
Existing MOT techniques Bewley et al. (006b); Wojke et al. 2017); Zhang et al. (021a)) uti- lize a mix of detection metrics such as IoU (intersection over union) Rezatofighi et al. (0119 and classification metrics like cross-entropy Meinhardt et al. (2021) Zeng et al. (2021) as their train- ing objectives. These metrics do not adequately evaluate the complex interplay between temporal stability, spatial awareness, and identity preservation-elements vital for effective tracking (Zhang jet al.] (202Íb); [Sun et al.] (2020). Consequently, high detection accuracy models falter in maintain- ing consistent object identities, particularly through occlusion episodes (Zhang et al.) (2021a); Zeng et al.] (2021). These limitations become especially pronounced in complex scenarios involving dense crowds, rapid motion, variable object scales, and congested environments Dendorfer et al. (2021), demanding robust mechanisms for preserving object identities. Our research reveals that there are three primary categories of errors that existing methods fail address holistically: post-occlusion ID switches, where identity tracking fails when direct line-of-sight is obstructed (error Type 1), tem- poral inconsistency, where the tracker struggles maintain stable ID assignments when subjects change posture (error Type 2), and crosy-subject ID switches, where IDs are incorrectly exchanged when subjects cross paths and later separate, leading to tracking instability (error Type 3).
To overcome the noted limitations in MOT, we introduce UniTrack, a novel differentiable graph rep- resentation learning framework with an embedded loss function. This innovative approach leverages hierarchical graph structure to model the spatial and temporal dynamics among tracked objects, integrating detection accuracy, spatial consistency, and temporal continuity into a singular, compre- hensive learning objective. By enabling direct optimization of tracking-speciff metrics through dif- ferentiable graph computations, UniTrack offers a robust and adaptable framework that seamlessly integrates with existing end-to-end MOT systems. Compared tory approaches, UniTrack addresses three key error types through a joint optimization enabled by unified graph structure: flow components reduce Type I errors by maintaining identity consistency, temporal edges prevent Type 2 errors by enforcing motion consistency, and spatial edges mitigate Type errors by modeling inter-object relationships, significantly reducing identity switches and enhancing overall stability (see Figure I.
to traditional independent trajec-
The versatility of UniTrack lies in its capability to dynamically re-weight tracking components, enabling automatic model tuning for particular environmental conditions without architectural changes. In crowded scenes, spatial consistency is prioritized to handle occlusions, while fast- motion settings will benefit from stronger temporal coherence. Our evaluations underscore the ef- ficacy of UniTrack, showcasing enhancements over traditional methods on well-established bench- marks like MOTI7 Milan et al. (2017). M0T20 et al. (2021). SportsMOT (uiiooCCCiii et al. (2023) and DanceTrrakk Sun et al. (2022). Notably, our models excel in maintaining object identities and tracking precision, especially in occluded and crowded environments, highlighting the significant benefits of our unified approach. The key contributions of our are as follows:
We introduce UniTrack, a novel graph-based representation learning framework unifying detection accuracy, identity preservation, and spatial-temporal consistency to directly ad- dress three key tracking error types.
We develop an adaptive weighting mechanism using graph Laplacian analysis that auto- matically balances components based on scene characteristics without manual tuning.
We integrate UniTrack with existing approaches (MOTR Zeng et al. (2021). TrackFormer Meinhardt et al. (202T), FairMOT Zhang et al. (2021b), Byte Zhang et al.] (2022). ETRRZoo et al. (022). MOTE Galoaa et al. (2025b) without architectural modifications.
2

---

Published as a conference paper at ICLR 2026
We conduct extensive experiments MOT17 Milan al. (2017). MOT20 Dendorfer al. (202I), SportsMOT Cui et al. (2023), and Dance' Track Sun et al. (2022), demonstrate con- sistent reductions in ID switches, temporal inconsistency, and tracking failures, with GTR achieving up to 9.7% MOTA and 12.3% IDFI improvements on the SportsMOT dataset.
RELATED WORK
Classical Approaches in MOT.
The training of MOT systems has traditionally
relied on a combination of separate tracking and detection modules. Most tracking frame- works optimize detection using standard object detection models such as YOLO Daang al. (2022) with loU or generalized loU (GloU) as loss metrics, while handling tracking associ- ation through separate optimization objectives Bewley et al. (2016a); Wojke et al. (2017). This disjoint formulation often fails to cap- ture the intricate relationships between detec- tion and tracking performance, particularly dur- ing training phase, where end-to-end optimiza- tion is most beneficial. As shown in Figure traditional approaches primarily focus on de- tection accuracy (partially addressing aspects of Type errors) but struggle with preserving identities through occlusions |Bergmann 2019); Luo et al.] (2021). Furthermore, they lack explicit mechanisms for enforcing tempo- ral consistency (Type 2 errors) and modeling inter-object relationships (Type 3 errors), lead- ing to unstable ID assignments when subjects interact or change appearance Ristani & Tomasi 2018): et al. (2014). Recent work has in- troduced more sophisticated approaches to en- hance MOT training and address these chal- lenges: Byte Byte'Track/ Zhang et al. (2021a) utilizes detection confidence as part of the loss func- tion, though primarily focusing on inference- time association rather than training optimiza- tion. TransTrack [Sun et al. (2020) introduces memory-based temporal modeling, yet lacks unified framework that could optimize all tracking components simultaneously. Joint Detection-Tracking Approaches. Recent al- gorithms explored integrated training for end- to-end MOT systems. SroTRZengg et (202)) incorporates track query matching function alongside detection supervision in its training objective. Similarly, TrackFormer Meinhardt et al. (2021) proposes losses for track initialization and propagation during training. However, these tracking methods still largely treat spatial and temporal consistency as separate components, lim- iting the potential for truly integrated joint optimization during the training phase. While these approaches advance the handling of Type errors such as post-occlusion identity switches via im- proved detection-to-trackin associations, their reliance separate formulations continues to fall short in addressing Type 2 (temporal inconsistency) and Type 3 (cross-subject identity switches) errors, especially in complex scenarios.
.113
1.0
3.A
3.B
3.C
Figure 2: Illustaaioon of tracking errors in MOTR (022I) on MOT17 sequences and The first row (sequence 9) highlights post- occlusion ID switches (error Type 1): subject loses tracking when occluded behind subject 4 in frame 1.B, with IDs 8, 3. and subsequently re- assigned as IDs 15, 14, and 12 in frame 1.C. The second row (sequence 8) demonstrates temporal inconsistency (error Type 2), where the tracker fails to maintain IDs when subjects change pos- tures: ID 15 changes 27 (2.B), and in 2.C, MOTR erroneously assigns two bounding boxes subject 6. 13 while ID 14 changes to 10 and ID 15 is reassigned to 29, illustrating instability in temporal association. The third row (sequence 9) demonstrates cross-subject ID switches (error Type 3): 22 and 3 are correctly assigned in 3.A, but when subject 42 occludes subject 3 in 3.B, it triggers a cascade of errors-ID 22 gets incorrectly swapped with ID 80, followed by ID 3 being er- roneously reassigned as in 3.C, showcasing how occlusions propagate tracking failures.
Zeng
Graph-based MOT Methods. Prior works have explored graph representations in MOT algorithms. Neural MOT solver Brasó & Teal-Taixé 2002) message passing networks for data association,
3

---

Published as a conference paper at ICLR 2026
treating tracking as edge classification. Recent approaches including SUSHI Cetintas et al. 2023) Global Tracking Transformers (GTR) Zhou et al. (2222) and DiffMOT Luo et al. (2024) incorporate graph structures into their tracking pipelines. Crucially, our proposed UniTrack framework differs fundamentally from these architectural approaches. While prior graph-based methods redesign the tracking algorithm risuuriruguirin modifications network architecture, forward pass logic, and inference procedures-UniTeene training objectíve. This distinction is central to understanding our contribution: UniTrack is not an architectural innovation but rather plug-and-pln any existing MOT system by simply adding our loss term during training, with zero modifications to model architecture inference code. As demonstrated in our experiments across 7 diverse architec- macking-by-detese
provides graph-theoretic loss function that serves as a universal
training enhancement that can be integrated into
joint ceteetiionttackingg and memory-augmentedt this universal applicability distinguishes UniTrack as training-time enhancement rather than spe- cialized architectural design.
tures (transformer-basee
3 UNITRACK: A GRAPH-THEORETIC LEARNING FRAMEWORK
Problem Formulation a video sequence of T frames, we aim to learn a tracking model that optimizes both detection accuracy and tracking consistency through unified differentiable represen- tation and objective. Our approach specifically addresses three key error types: post-occlusion ID switches (Type 1), temporal inconsistency (Type 2), and cross-subject ID switches (Type 3). Tra- ditional tracking approaches handle trajectories independently. leading to identity switches when objects intersect, while UniTrack leverages unified graph structures with spatial-temporal connec- tions for consistent tracking. We formulate this as a graph-based optimization problem where the loss function L is defined over temporal sequence of weighted directed graphs:
G [GG (Vz, E,, Wo)
where = {ufli E Ie} the set of all vertices at time with 2i representing tracked object at time 1. The set I, contains all object identities present at time t. Edges F, capture spatial-temporal relationships between objects across consecutive frames, and weights W. [w (i,j) Eg encode confidence and association strengths for each potential object correspondence. The graph construction enables joint optimization across all objects and time steps, forming structured flow optimization framework that maintains both local coherence and global consistency.
3.1 GRAPH FLOW NETWORK ARCHITECTURE
We model the tracking problem as a flow network, where the graph structure encodes the complex relationships inherent in multi-object tracking. The key insight is that object tracking can be viewed as a flow conservation problem: objects cannot arbitrarily appear or disappear, and each detection should correspond to at most one physical object across time. To capture the life-cycle of tracked objects, we introduce balance variables bj E (-1,0,1) 13 for each object at time t, where 6f = indicates track initialization (new object appearing), by 0 indicates track continuation (existing object persisting), and ij = -1 indicates track termination (object disappearing from view). These balance variables encode the net flow change for each object each time step, ensuring that the flow conservation constraints maintain physically consistent tracking states throughout the sequence.
We introduce flow variables fed representing the association strength between object i at time and object j at time i 1. To maintain physical consistency in tracking, we enforce flow conservation constraints at each node:
S JEN+(0
Σ
Σ
Vi V2,
REN- (8)
where N is the set of nodes that can receive flow from node i (outgoing neighbors), and W- (i) is the set of nodes that send flow to node i (incoming neighbors). Here, j indexes over all possible destination objects for flow leaving object while k indexes over all possible source objects sending flow to object i. This constraint ensures that the total outgoing flow minus incoming flow for each object equals its balance variable, naturally encoding object appearance, persistence, and disappear- ance. We construct the graph using a sliding window of W = 5 frames to balance computational efficiency with temporal context. The differentiable graph loss assembly follows three steps: (1)
4

---

Published as a conference paper at ICLR 2026
construct node embeddings from detection features, (2) compute pairwise similarities to form edge weights and flow variables, and (3) apply flow conservation constraints while optimizing the unified loss. This process integrates seamlessly with backpropagation, enabling end-to-end training.
3.2 UNIFIED Loss COMPONENTS
Our loss function unifies three complementary components differentiable framework:
Laow + Xg + Crempoal
where A, and A are adaptive weights automatically determined by scene characteristics (detailed in Section 3.3).
The flow-based loss Lnow primarily addresses Type errors by encouraging confident object associ- ations while adapting to detection quality:
FP PP]
FM CT
Lnow
Σ (E,)EE,
w
S exp
where FP and FN represent false positive and false negative detections, (Pl and [9T denote total counts of predictions and ground truth objects, and a controls the influence of detection errors on the loss.
Differentiability of Detection Quality Terms: The FP and FN counts are computed from cur- rent predictions and ground truth at each training step. During backpropagation, these counts are treated as constants (top-ggaddeen), serving as adaptive coefficients that scale the loss mag- nitude based on detection performance. This approach maintains full differentiability with respect to flow variables ft -wich directly receive gradients- while avoiding complications from differen- tiating discrete counting operations. The gradient with respect to flow variables is well-defined: a
[FN] since fi E [0, 1] and the exponential function is smooth. When detection quality high (low FP/EN rates), the exponential term approaches 1, fully trusting learned associations; when quality degrades, the term decreases, reducing commitment uncertain associations.
0Cas ww exp a
-a [FP) [P[
[9T]
The spatial coherence loss Lspatial targets Type errors by enforcing that objects with similar spatial relationships should maintain consistent associations. This prevents identity switches when objects cross paths by considering the spatial context of nearby objects:
Espatial
Σ (.,)),,
w Pi
(5)
where pf = (xf, vε) represents the spatial coordinates object i time 4, d(:,)) computes ge- ometric distance between positions across consecutive frames, and wij are learned spatial attention weights. This term penalizes spatially distant associations to promote local coherence.
The temporal coherence loss Lumporal addresses Type 2 errors by ensuring smooth motion patterns and penalizing abrupt velocity changes that indicate tracking inconsistencies:
Cimporal
lv2 iEV,
v
Σ
(6)
JEN+(()
where v denotes the velocity vector of object at time f, computed from frame-frame position differ- ences. The term At represents the interval between consecutive frames and normalizes the temporal loss with respect to frame rate. This formulation penalizes sudden acceleration changes weighted by the confidence of the object's continued existence (sum of outgoing flows).
To account for varying problem scales, we apply logarithmic normalization to the final loss:
Limal = L log((E) 1),
where |8] represents the total number of valid edges, ensuring that the loss magnitude scales appro- priately with scene complexity.
5

---

Published as a conference paper at ICLR 2026
3.3 ADAPTIVE WEIGHT LEARNING
The relative importance of spatial and temporal components varies across different tracking scenar- ios. In crowded scenes, spatial consistency becomes crucial for handling occlusions, while fast- moving objects require greater emphasis on temporal coherence. To address this, we introduce an adaptive weighting mechanism that automatically adjusts X, and X based on scene characteristics.
The key insight is that when object interactions are complex (indicating high spatial coupling), we should emphasize spatial consistency, while when motion patterns are erratic (indicating temporal challenges), we should emphasize temporal smoothness. We measure this complexity through the connectivity structure our tracking graph using Laplacian analysis.
The adaptive weights are computed through graph Laplacian eigenvalue analysis, where the second smallest eigenvalue (algebraic connectivity) (2( measures the connectivity of the components:
02 (L,) 02 (L.) )- (L,)- d o2 +22
(2(L,)
A
(8)
where L2 and L, are the Laplacian matrices constructed from spatial and temporal edges respec- tively. Lower connectivity (smaller 02) indicates more fragmented relationships, requiring higher weighting to improve coherence. To provide mathematical insight into how the weights influence the system, we examine the partial derivatives of the total loss with respect to the weights (noting that the loss components Espatial and Lemporal are already defined in Equations and
dc
Oc
Lspatial
= Liemperal-
a^,
a%.
However, A and A not learnable parameters and cannot be updated via gradient descent (i.e., not: ( x oT) Instead, they are recomputed at each training step from the current graph structure using Equation 8 This ensures the weights always reflect scene-specific graph con- nectivity rather than accumulating gradient-hased updates. The model parameters 0 are updated via standard backpropagation:
ac
0(k+1) ((())
(10)
30
where the gradients flow through the loss weighted by the current X2 and X values. As 0 evolves, the resulting embeddings change the graph structure, which in turn updates the Laplacian matrices and recomputes new weight values. This mechanism enables automatic adaptation: when spatial rela- tionships are fragmented (low o2(L.)), As increases to emphasize spatial consistency; when tempo- ral flows disrupted (low 02 (L.)), A, increases to enforce smoother motion patterns. We validate this recomputation approach against fixed and learned weight alternatives in Section 4.5 (Table 4
3.4 THFORETICAL ANALYSTS OF CONVERGENCE
Our unified loss formulation provides theoretical guarantees for both optimization convergence and tracking consistency. We establish these properties to ensure UniTrack can be reliably integrated into existing MOT systems.
Theorem (Unified Convergence and Consistency) The UniTrack loss function C = -Siiow +
X₂ Xyrrpaial + Seemoorral satisfies differentiability, local convergence under standard regularity condi- tions, and ensures physically plausible tracking solutions via flow conservation constraints.
Proof and Analysis: The full theoretical analysis, including convergence conditions, differentiability proofs, and tracking consistency properties, provided in Appendix [.. These theoretical prop- erties ensure that UniTrack not only improves tracking performance empirically but also provides principled framework that maintains mathematical consistency throughout the optimization process.
4 EXPERIMENTAL RESULTS
We conduct extensive experiments to evaluate the effectiveness of the loss function provided in UniTrack for existing MOT frameworks. Our approach is designed enhance identity preservation

---

Published as a conference paper at ICLR 2026
MOTR
UT-MOTR
MOTR
UT-OTT
AIRSES C
1.B
I.C LB I.C 2.B
2.C
2.B 2.C
Figure 3: Comparative performance of MOTR Zeng et al. 2021) and UT-MOTR revisiting the same challenging scenarios from Figure 2 Red bounding boxes indicate tracking errors, green boxes show successful tracking. and dotted red boxes highlight missed detections. UT-MOTR successfully addresses the three error types demonstrated in Figure maintaining consistent IDs through occlu- sions (frames 1.B-1.C). preserving temporal consistency during posture changes (frames 2.B-2.C), and preventing cross-subject ID switches in crowded scenes. The unified traph-theoretic approach enables robust tracking where the baseline MOTR fails. Additional qualitative results with Track- former are provided in Section A.3
while maintaining the core strengths of each baseline architecture, as demonstrated through compre- hensive evaluations on MOTI7 Milan al. (2017), MOT20/Dendorie, et al. (2021), SportsMOTTCui et al. (2023), and Dance Track (Sun et al. (2022) benchmarks (as seen in Figure 3). Additionally, we analyzed the impact of each component on UniTrack's performance (see Figure and Table
4.1 IMPLEMENTATION DETAILS
Table 1:
Performance comparison challeng- ing sequences that highlight the three error types. UT variants generally outperform their respective baselines across most metrics, with improvements varying by tracking architecture.
UniTrack is implemented as an additional loss component that addresses three key MOT er- ror types through its unified graph structure. For all baselines, we maintain their original hyperparameters and training protocols, with UniTrack's components weighted through our adaptive mechanism. The weighting parame- ter a in Equation 4 was set to 0.9, and we use temporal window of 5 frames for graph con- struction. The adaptive weight learning rates start at 77 = 0.01 and decay following the base- line models' schedules. Comprehensive sen- sitivity analysis of all hyperparameters (r, W, At) with computational cost evaluation is pro- vided in Appendix A.4 Memory requirements increase by approximately 5% due to graph construction, with computational complexity of O(n²t) for objects over f. frames only for training. We follow standard evaluation protocols using HOTA Luiten et al. (202I), IDFI Ristani
Sequenee
Mahhod
MOTAT
IDFIT
HOTA
FN]
1[x]
Tracffemeer
S7.42
63.16
66.29
1742
204
UT-Trecfommer
62.86
67.38
68.45
1435
186
MOTI7 13
MOTR UT-MOTR
65.18
1695
164
67.21
FiiM UT FaiMOT
Post occiuscnn
68.43
64.38
66.92
1678
185
59.
113
UT Trackfommer
68.24
65.93
63.18
2087
164
MOT17-10
MOTR UT-MOTR
64.59
63.12
60.04
2427
$4
67.31
66.25
62.38
2156
102
Temporal
Trackformer
UT-Tackfonner
5103
MUTR
50.46
39.25
43.48
3967
143
UT-MOTR
53,7
45.36
47.95
5287
132
ParMOT
40.85
38.76
42.15
6054
160
Cross subjecc
FaiMOT
$1.94
43.89
46.83
5465
149
ct al. (2016), MOTA Keni Bernardin (2008), and identity switches (IDS) metries.
4.2 UNITRACK ADDED EXISTING MOT
We selected diverse baseline architectures to demonstrate UniTrack's architectural agnosticism: Trackformer Meinhardt et al. (021). MOTR Zeng et al. (2021). FairMOT Zhang et al. 20021b). ByteTrack Zhang et al. (2022 MOTE Galoaa et al. (2025b), and GTR Zhou et al. (2022). This di- versity spans transformer based end-to-end tracking (Trackformer, MOTR), joint detection detection-tracking (FairMOT), tracking- beeling--y-deteer detection with association strategies (Byte Brack,rck), and decection-embedee paradigms (GTR, MOTE). This selection allows us to validate UniTrack's effectiveness across dif- ferent tracking approaches and demonstrate its plug-and-play nature. Each architecture represents

---

Published as a conference paper at ICLR 2026
Table 2: Performance evaluation MOT17 Milan Milan al. (0117) and Dendorfer et al. (202T) test sets. Results compare standard tracking methods with and without UniTrack (UT) loss, or- dered by publication year. Models trained with UT show significant improvements in detection and identity preservation, especially in MOTA and IDFI scores. UniTrack significantly reduces false positives (FP) and false negatives (FN) while maintaining tracking stability. Performance con- sistent between MOT17 and MOT20 results, demonstrating UniTrack's 's robustness. Bold values highlight performance gains achieved by integrating UniTrack, while shaded values denote the top- performing tracker among all contestants. We use this system for all tables unless stated otherwise.
MOT17
MOT20
Method
MODA?
IDFI1
HOEAS
FN]
[13]]
MCTEA
IDFIT
HUTAT
HP.
PN]
[0s]
BairMr et al. 2021h
61.7
61.5
52.9
1902
20456
388
53.5
58.3
52.4
1902
25456
488
UT- UT-FaiMoT
64,5
64.2
55.3
1623
19234
482
55.2
61.5
55.8
1723
23234
402
MOTR Zong et al. 2021 UT UO-M
62.1
61.3
53.2
1843
21034
289
53.2
57.9
51.8
1843
25034
389
64.8
63,9
55.7
1562
19845
356
55.8
60.4
54.2
1562
23845
356
Trackformer (Meinhurtt et UT TTT-krccfoeeme
57.6
21893
54.1
50.9
1965
25893
65.9
66.4
56.2
1039
16667
705
56.2
64.1
57.7
1374
22004
314
ByteTrack Zhang al.
2022
80.3
77.3
63.1
25491
83721
2196
77.8
75.2
61.3
26249
87594
1223
UT-Bytc Flak
82.1
79.8
65.4
22145
79350
1865
79.5
77.8
63.7
23520
83145
1045
GIR Zhou al
2023
75.3
71.5
59.1
1250
15800
1445
63.6
52.3
42.6
9916
205166
X604
UT-GTK
74.8
14200
951
63.8
43.0
9885
204530
8570
MOTE/Galoom clal. 3025b
82.0
80.3
66.3
1100
8500
620
81.7
79.8
65.8
7800
12000
685
UT-MOTE
84,5
83.5
68.2
850
7200
542
83.2
81.4
67.1
7200
10500
578
distinct philosophy in MOT: MOTR and Trackformer leverage transformer architectures for end- to-end tracking, FairMOT balances detection and re-identification in a single network, while GTR employs global tracking transformers and MOTE introduces enhanced multi-object tracking capabil- ities. By showing consistent improvements across these varied approaches, we establish UniTrack's ss generalizability universal enhancement for MOT systems.
4.3 PERFORMANCE ON EXISTING BENCHMARKS
The integration of UniTrack shows consistent and substantial improvements across different archi- tectural designs when tested on multiple challenging datasets. Table [ presents results on MOT17 and MOT20, while Table shows performance on SportsMOT and DanceTrrack MOT17 and MOT20. Most notably, when combined with any of these SOTA algorithms, Uni-
Table 3: Performance evaluation on SportsMOT Cui et al. 2023) and DanceTrrckk [Sun et al. (2022) test sets. UniTrack demonstrates significant improvements on both datasets, with notable gains in identity preservation (IDFI) and ID switch reduction. SportsMOT presents challenges with rapid movements and complex interactions in sports scenarios, while Dance features frequent oc- clusions, similar appearances, and complex dance movements. The consistent improvements across both datasets highlight UniTrack's 's effectiveness in handling diverse tracking scenarios. Bold value indicates performance gains from UniTrack integration; shading marks the best overall tracker.
SporsMOO
DanceTrack
Method
MOTAT
IDFIt
HOTAT
EP]
FN,
IDsL
MOTA
IDFIt
HOTAT
FPJ
FN.
IDs]
FairMOT Zhang ot al.
2021b
90.8
53.5
49.3
8765
7234
2845
82.2
40.8
39.7
11234
16890
2987
UT-FairNUT
92.5
56.2
52.1
7890
6456
2234
84,8
43,5
42.3
10456
15234
2456
Trackformer MMinnardt et al.
2021
88.1
50.0
60.0
13983
16778
4250
48.2
19.4
48500
30200
37800
UT Trackformer
90.3
51.5
60.8
12252
15769
3264
50.4
13.6
20.5
46825
28967
35876
MOTR Zeng cl. (2021)
76.2
58.4
55.8
11545
12890
2890
79.7
51.5
54.2
15234
28967
4567
MOTR
79.5
62.1
58.4
10234
11267
2156
$2.1
54.8
57.3
13456
262.44
3892
Byи:Taa Zhang ctal. 2022
94.1
69.8
62.8
8934
5827
3267
88.2
51.9
47.1
12456
18934
3456
WTT-Byelioe Tack
96.2
71.1
64.3
7545
4412
2234
91.3
56.5
49.1
10234
16782
2134
GTR Zhou et al. UT-GTR
2022
74.8
61.3
54.4
12176
11578
2364
80.6
45.9
43.7
9683
25191
4338
84.5
73.6
66.1
8212
6628
1092
82.6
48,5
50.2
8521
15234
3456
MOTE MOTE
Galoza et al.
2025b
93.8
68.2
61.5
7892
5234
2987
87.4
53.2
46.8
10892
17234
3124
UT-MOTE
95.1
70.5
63,2
7123
4789
2456
89.8
56.1
48.9
9567
15892
2567
Track significantly improves MOTA, IDF1, and HOTA metrics, boosting both detection accuracy and identity preservation. Notably, we observe reduction in false positives and false negatives UniTrack helps Trackformer reduce FP by approximately 47% (from 1965 to 1039) and FN by about 24% (from 21,893 to 16,667). This indicates that our approach helps maintain more stable
8

---

Published as a conference paper at ICLR 2026
tracks through challenging scenarios of occlusions and crowded scenes, GTR demonstrates excep- tional gains with UniTrack, where UT-GTR achieves 79.1% MOTA (+3.8%), 74.8% IDFI (+3,3%), and 67.9% HOTA (+8.8%), establishing new performance benchmarks. The dramatic reduction in identity switches from 1445 to 951 (34.2% reduction) underscores UniTTrack's effectiveness in main- taining stable identities.
SportsMOT poses unique challenges due to rapid motion, complex interactions, and frequent oc- clusions. UniTrack shows its strongest improvements on this dataset (see Table 3). UT-GTR achieves exceptional results with 84.5% MOTA (+9.7%), 73.6% IDFI (+12.3%), and 66. 1% HOTA (+11.7%) The reduction of identity switches from 2364 to 1092 (a 53.8% improvement) high- lights UniTrack's effectiveness in handling fast-paced scenarios where traditional trackers struggle with motion blur and rapid direction changes. ETT-ByteTrack also achieves notable gains: 96.2% MOTA (+2.1%) %), 71.1% % IDF1 (+1,3%), reducing ID switches from 3267 to 2234 (31.6% reduction).
DanceTrack introduces different challenges with its focus on dance scenarios featuring sim- ilar appearances, synchronized movements, and frequent occlusions. UniTrack shows consis- tent improvements compared to SportsMOT. UT-GTR improves IDF1 by 2.6% (from 45.9% to 48.5%) and HOTA by 6.5% (from 43.7% 50.2%), while reducing ID switches by 20.3% (from 4338 to 3456). ByteTTrack demonstrates substantial gains: UT-ByteTracck achieves 91.3% MOTA (+3,1%), 56.5% IDFI (+4.6%), and reduces ID switches from 3456 to 2134 (38.2% reduction). Dance'rrac''s 's lower baseline (80.6% MOTA for GTR) underscores its challenge, driven by high appearance simi- larity among dancers.
Table 4: Ablation study of UniTrack on MOT17 after 15 epochs. Top: Component ablation on Trackformer. Bottom: Weighting strategy com- parison on GTR demonstrates Laplacian- weighting outperforms alternatives. Underlined values indicate second-best performance.
Configuration
MOTA IDFI HOTA IDs)
Component Ablation (Trackformer)
Laow + Lya + Lume
$6.2
64.1
57.7
288
Wio Laww
529
6C3
35.5
312
wio Cys
54.3
62.9
56.3
213
w/o Lump
$8.3
62.1
51.5
380
Weighting Strategy (GTR)
Fixed (Am0.5)
76.8
72.1
65.4
1087
Learned (rand)
77.5
73.2
66.2
1023
Learned (Lap.)
78.3
73.9
66.8
978
Laplacian (ours)
79_1
74.8
67.9
951
4.4 ERROR TYPE
ANALYSIS ON CHALLENGING SEQUENCES
Seen in Table T to better understand how UniTrack addresses specific tracking challenges, we an- alyze its performance on three representative MOT17 sequences. Error Type 1: Post-occlusion ID switches (MOT17-13-FRCNNY This sequence features occlusion events that challenge identity consistency, All models improve with UniTrack: UT-Trackformer gains 5.44% MOTA, UT-MOTR 3.12%, and UT-FairMOT 3,45%, demonstrating effective identity preservation through occlusions. Error Type 2: Temporal inconsistency MOT177-1-FRRCNT When subjects change appearance, UniTrack's 's temporal edge modeling consistently improves temporal stability in all architectures. UT-Trackformer reduces fragmentations by 32%, UT-MOTR by 23%, and UT-FairMOT by 26%. Error Type 3: Cross-subject ID switches (MOT17-02-FRCNNN)
In this crowded sequence, UT- Trackformer, UT-MOTR, and UT-FairMOT improve IDF1 by 6.77%, 6.11%, and 5.13% respec- tively, highlighting UniTrack's effectiveness in handling complex object interactions.
Overall, all architectures benefit from UniTrack in addressing specific error types across diverse challenging scenarios. Additional analysis of alternative spatial relationship formulations is provided in Appendix A.2
4.5 ABLATION STUDIES
To understand each component's contribution, we conduct ablation experiments at the 15th epoch of training. Table 4 shows component ablation using Trackformer and weighting strategy com- parison using GTR. The spatial component proves critical for identity consistency-withee MOTA drops 1.9% and IDFI falls 1.2%. The temporal component reveals a critical trade- off: removing it increases MOTA 2.1% despite raising identity switches 32% (288-3380 as FP/EN reductions outweigh IDSW penalties in MOTA's linear summation. However, HOTA drops 6.2%, better reflecting the severe tracking consistency loss. We compare our Laplacian- based adaptive weighting (continuously recomputed from graph connectivity) against fixed and
it,

---

Published as a conference paper at ICLR 2026
learned alternatives, demonstrating that dynamic adaptation (79.1% MOTA) outperforms fixed weights (76.8%) and learned parameters with Laplacian initialization (78.3%), validating our de- sign choice. See full hyperparameter ablation in A.4 and spatial formulation analysis in A.2 Frame-Rate Resilience, [ Uni- Track's adaptive behavior across frame rates. While both methods plateau in absolute per- formance between 5-15 FPS (Fig. [A), the performance gaps tell a more nuanced story (Fig. ДВ). The HOTA gap decreases mono- tonically from 12% at I FPS to 7% at 30 FPS, demonstrating UniTrack's consistent ad- vantage. Interestingly, MOTA and IDFI gaps show initial sharp increases from 3% to 6-7% between 1-5 FPS, suggesting our spatial-temporal graph becomes effective once minimal temporal information is available. The adaptive weighting mechanism automat- ically adjusts A based on frame rate FPS, spatial components dominate (A.=0.78, A==0.22). while 30 FPS, weights balance (A,=0.52, =x=0.88). This dynamic adaptation ensures optimal performance whether tracking in surveillance scenarios (low FPS) or sports broadcasts (high FPS).
Figure 4: Frame-rate resilience analysis of Uni- Track. (A) HOTA scores show UT-GTR main- tains superior performance, with both methods plateauing around 5-15 FPS. (B) Performance im- provements of UT-GTR over GTR: HOTA gap de- creases from 12% to 7% as frame rate increases,
while MOTA and IDF1 gaps increase sharply from 1-5 FPS before converging. UniTrack main- tains consistent advantages across all frame rates.
Hyperparameter Sensitivity and Scalability Table 5 validates our hyperparameter choices on GTR (MOT17). Optimal settings (r 0.1, W = frames, linear normalization) achieve 79.1% MOTA with practical computational costs (4.s 6.7GB V100s). Suboptimal choices degrade performance: low temperature (r 0.05) increases ID switches
Table 5: Hyperparameter sensitivity analysis on GTR
(MOT17). Optimal settings achieve best performance with practical costs. Full analysis in Appendix A.4
Param Value
MOTA
IDF1
HOTA"
IDs]
Time
Mem
0.05
78.3
73.1
66.2
1047
4.2h
6.8G
0.1
79.1
74.8
67.9
951
4.1h
6.7G
0.5
77.9
72.8
65.8
1134
4.0h
6.6G
3
78.2
73.9
66.8
1078
3.8h
5.9G
W
5
79.1
74.8
67.9
951
4.1h
6.7G
1047, small windows (W = 3) lose temporal context reducing MOTA 78.2%, and removing normalization
10
78.6
74.1
66.9
1012
5.3h
9.2G
Norm.
Nome
77.3
72.4
65.1
1234
4.0h
6.7G
Linear
79.1
74.8
67.9
951
4.1h
6.76
drops MOTA to 77.3%. Memory scales approximately linearly with window size (5.9GB at W 3 to 9.2GB W 10), with training time increasing super-linearly for W > 10 due to increased graph edge complexity. Our MOT20 results demonstrate effective scaling to dense crowds (170 objects/firame). Critically, all computational overhead is vaining-only-UniiTeee
adds
zero inference cost. Full sensitivity analysis with additional parameter values is in Appendix A.4 (Table 8
5 CONCLUSION
We introduced UniTrack, novel graph-based loss function that unifies detection accuracy, iden- tity preservation, and spatial-temporal consistency in multi-object tracking. Unlike traditional ap- proaches that separate detection and tracking, UniTrack optimizes tracking-specific metrics within unified graph structure, significantly reducing ID switches, temporal inconsistencies, and track- ing failures. Extensive evaluations on MOT17, MOT20. SportsMOT, and Dance demonstrate consistent improvements up to 9.7% MOTA and 12.3% IDF1 across diverse architectures. While UniTrack seamlessly integrates with existing MOT systems, it introduces 5% memory overhead and Onn²t) computational complexity during training only, without affecting inference performance. This training overhead may limit scalability in extremely dense scenarios. Additionally, formulation focuses on single-camera tracking. Despite these limitations, UniTrack provides a prac- tical enhancement for current MOT systems. Future work will leverage the scalable graph structure reasoning across
current
to extend to multi multi-camera tracking scenarios, enabling enhanced multiple viewpoints while maintaining computational efficiency inference time.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Philipp Bergmann, Tim Meinhardt, and Laura Leal-Taixe. Tracking without bells and whistles. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 941-951 2019.
Dimitri P. Bertsekas. Nonlinear Programming. Athena Scientific, Belmont, MA, 2nd edition, 1999.
Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, and Ben Uperoft. Simple online and realtime tracking. JEEE International Conference on Image Processing, 2016a.
Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, and Ben Upcroft. Simple online and realtime tracking. In IEEE International Conference Image Processing, 2016b.
Guillem Brasó and Laura Leal-Taixe Learning neural solver for multiple object tracking. In IEEE Conference on Computer Vision and Puttern Recognition, 2020.
Orcun Cetintas, Guillem Brasó, and Laura Leal-Taixé. Unifying short and long-term tracking with graph hierarchies. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22877-22887, 2023.
Yutao Cui, Chenkai Zeng, Xiaoyu Zhao, Yichun Yang, Gangshan Wu, and Limin Wang. Sportsmot:
A large multi-object tracking dataset in multiple sports scenes. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 5830-5840, 2023.
Patrick Dendorfer, Hamid Rezatofighi, Anton Milan, Javen Shi, Daniel Cremers, Ian Reid, Stefan Roth, Konrad Schindler, and Laura Leal-Taixé. Mot20: A benchmark for multi object tracking in crowded scenes. International Journal of Computer Vision, 129:1-17, 2021.
Bishoy Galoaa, Somaich Amraee, and Sarah Ostadabbas. Dragontrack: Transformer-enhange person tracking in complex scenarios. Proceedings of the Winter Conference on Applications of Computer Vision (WACV), pp. 6372-6382, February 2025a.
graphical multi-person
Bishoy Galoaa, Somaieh Amraee, and Sarah Ostadabbas. More than meets the eye: Enhancing multi-object tracking even with prolonged occlusions. In (ed.), Proceedings of the 42nd Inter- national Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. -, 2025b.
Bishoy Galoaa, Xiangyu Bai, Shayda Moezzi, Utsav Nandi, Sai Siddhartha Vivek Dhir Rangoju, Somaieh Amraee, and Sarah Ostadabbas. Look around and pay attention: Multi-camera point tracking reimagined with transformers. urXiv preprint arXiv:2512.0421:. 2025c.
Peiyuan Jiang, Daji Ergu, Fangyao Liu, Ying Cai, and Bo Ma. review of yolo algorithm develop- ments. Procedia computer science, 199:1066-107,, 2022. 2022.
Rainer Stiefelhagen Keni Bernardin. Evaluating multiple object tracking performance: The clear mot metrics. EURASIP Journal on Image and Video Processing, 2008:246309,, 2008.
Harold W. Kuhn and Albert W. Tucker. Nonlinear programming. In Proceedings of the Second Berkeley Symposium Mathematical Statistics and Probability, pp. 481-492. University of Cal- ifornia Press, 1951.
Jonathon Luiten, Aljosa Osep, Patrick Dendorfer, Philip Torr, Andreas Geiger, Laura Leal-Taixé, and Bastian Leibe. Hota: A higher order metric for evaluating multi-object tracking. International journal of computer vision, 129:548-578 2021.
Weiyi Luo, Guangcong Yang. Zheng Zhong, et al. Diffmot: A real-time diffusion-basel multi- ple object tracker with non-linear prediction. Proceedings of the AAAI Conference Artificial Intelligence, 2024.
Wenhan Luo. Junliang Xing, Anton Milan, Xiaoqin Zhang, Wei Liu, and Tac-Kyun Kim. Multiple object tracking: A literature review. Artificial Intelligence, 293:103448 2021.
Tim Meinhardt, Alexander Kirillov, Laura Leal-Taixe, and Christoph Feichtenhofer. Trackformer: Multi-object tracking with transformers. In IEEE Conference on Computer Vision and Pattenn Recognition, 2021.
11

---

Published as a conference paper at ICLR 2026
Anton Milan, Stefan Roth, and Konrad Schindler. Continuous energy minimization for multitarget tracking. IEEE transactions on pattem analysis and machine intelligence, 36:58-72, 2014.
Anton Milan, Laura Leal-Taixé, Konrad Schindler, and Ian Reid. Mot17: A benchmark for multi- object tracking. In (0X3vv6603300833 2017.
Jorge Nocedal and Stephen J. Wright. Numerical Optimization. Springer, New York, 2nd edition, 2006.
Hamid Rezatofighi, Nathan Tsoi, Jun Young Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. Generalized intersection over union: A metric and a loss for bounding box regression. In IEEE Conference Computer Vision and Pattern Recognition, 2019.
Ergys Ristani and Carlo Tomasi. Features for multi-target multi-camera tracking and identification, In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6036-6046, 2018.
Ergys Ristani, Francesco Solera, Roger Zou, Rita Cucchiara, and Carlo Tomasi. Performance mea- sures and a data set for multi-target, multi-camera tracking. In Gang Hua and Hervé Jégou (eds.), Computer Vision ECCV 2016 Workshops, pp. 17-35, Cham, 2016. Springer International Pub- lishing. ISBN 978-3-319-48881-5
Peize Sun, Yi Jiang, Rufeng Zhang, Enze Xie, Jinkun Cao, Xinting Hu, Tao Kong, Zehuan Yuan, Changhu Wang, and Ping Luo. Transtrack: Multiple object tracking with transformer. In arXiv preprint ar 2xxiv:20/2.1546/ 2020.
Peize Sun, Jinkun Cao, Yi Jiang, Zehuan Yuan, Song Bai, Kris Kitani, and Ping Luo. Dancetrack: Multi-object tracking in uniform appearance and diverse motion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 20961-20970, 2022.
Nicolai Wojke, Alex Bewley, and Dietrich Paulus. Simple online and realtime tracking with a deep association metric. In IEEE International Conference on Image Processing, 2017.
Fangao Zeng, Bin Dong, Tiancai Wang, Xiangyu Zhang, and Yichen Wei. Motr: End-to-end multiple-object tracking with transformer. In European Conference on Computer Vision, 2021.
Yifu Zhang, Peize Sun, Yi Jiang, Dongdong Yu, Zehuan Yuan, Ping Luo, Wenyu Liu, and Xinggang Wang. Bytetrack: Multi-object tracking by associating every detection box. In International Conference on Computer Vision, 2021a.
Yifu Zhang, Chunyu Wang, Xinggang Wang, Wenjun Zeng, and Wenyu Liu. Fairmot: On the fairness of detection and re-identification in multiple object tracking. International Journal of Computer Vision, 129:3069-3088 2021b.
Yifu Zhang, Peize Sun, Yi Jiang, Dongdong Yu, Fucheng Weng, Zehuan Yuan, Ping Luo, Wenyu Liu, and Xinggang Wang. Bytetrack: Multi-object tracking by associating every detection box. In European Conference on Computer Vision (ECCV), 2022.
Xingyi Zhou, Tianwei Yin, Vladlen Koltun, and Philipp Krähenbühl. Global tracking transformers. In CVPR, 2022.
12

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 UNITRACK FULL THEORETICAL ANALYSIS
Theorem (Unified Differentiability and Local Convergence Properties) The UniTrack loss function from Equation 3 C = Lyow + Xa Axpopaalal + Ctemppora satisfies the following properties:
Differentiability: All loss components are differentiable with respect to flow variables f.², spatial positions pi, and temporal velocities vi. enabling end-to-end training through standard backprop- agation.
Under standard regularity conditions for non-convex optimization (bounded gradients, Lipschitz continuity). gradient descent with appropriate learning rates will converge to stationary points of the loss function.
Local Convergence:
Flow Conservation: The optimization respects flow conservation constraints from Equation suring physically plausible tracking solutions.
Graph-Theoretic Properties: The loss formulation leverages graph connectivity promote tracking consistency across spatial and temporal domains.
Proof:
Part I: Differentiability Analysis
We analyze the differentiability of each loss component defined in our method section:
Flow Loss Differentiability (Equation 4) The flow loss is:
FP
(FN
Boow
Σ
w S2 exp
-
[P]
[T
(
The gradient with respect flow variables is:
0Lnow
FP - P
JF
w exp
0f
GT
This is well-defined since the exponential function is smooth everywhere and J [0, 1).
Spatial Loss Differentiability (Equation 5): The spatial loss is:
Lapatial
w d(p 1.
(i,))E,
For Euclidean distance d(p), p(-1) = |lp; - P/+illa. the gradients are:
@Lrpatial
Σ witt sod
p p/+1
Öp,
lp 1/+1ll2
OL
w p(.))
Both are well-defined for pł * P/+1. which holds almost surely in practice.
Temporal Loss Differentiability (Equation 6): The temporal loss is:
Ciemporal
1
vv
vin
Σ f jEN+(()
At
VEV.
The gradients are:
@Leeppont
(v)
v(_))
Σ
Ov:
At
JENL (1)
13

---

Published as a conference paper at ICLR 2026
b Cromprall
1 11v AI
1/2
05e
Both smooth and well-defined.
Part H: Local Convergence Analysis
For convergence analysis, we establish that the loss function from Equation regularity conditions for non-convex optimization (Bertsekas)
7 satisfies standard
1999):
Bounded Gradients: Each gradient component is bounded: Flow gradients:
@C af.2
S w (since
exponential term < 1) Spatial gradients:
&C dpp
Σ, wij (unit direction vectors) Temporal
2
gradients: Bounded by velocity magnitude constraints in practical tracking scenarios
Lipschitz Continuity: The loss function is locally Lipschitz continuous (Nocedal & Wright 2006) For any compact domain f containing feasible tracking parameters:
[V(())) VL((2)I < L 02l
where L depends on the maximum values of weights w velocity differences, and spatial distances within f2
ner 800
ter B030
By standard results in non-convex optimization, gradient descent with learning rate < con- verges to stationary points:
-00
lim
V
00000
Loss Landscape Analysis: The theoretical guarantees are empirically validated through loss surface visualization in F The Uni- Track loss formulation creates broader, more stable convergence basins compared to base- line approaches. This improved loss geome- try arises from the multi-component structure that balances flow, spatial, and temporal con- straints. The smoother gradients observed in the UniTrack loss landscape (top row) facili- tate more robust optimization compared to the fragmented basins in standard training (bot- tom row), providing empirical evidence for the theoretical convergence properties established above.
Figure 5: Loss surface evolution during training with and without UniTrack loss the MOTI7 dataset. Contour plots show normalized loss land- scapes iteration 800 (left) and 8000 (right) for models trained with UniTrack loss (top row, viridis colormap) and baseline training (bottom row, plasma colormap). The UniTrack loss cre- ates broader, more stable convergence basins with smoother gradients, while the baseline approach results narrower, more fragmented loss land- scapes. Parameters and 3 represent perturba- tions around trained model weights. All surfaces normalized to [0,1] for visual comparison. sKi bi: Vi Vi
Part III: Flow Conservation Properties
The constrained optimization problem incor- porates the flow conservation constraints from Equation?? 2
min subject to (
Σ jENT (6)
ft
KEA
(6)
Using Lagrange multipliers μ for each constraint:
Cang = C+ Σ pi
Σ JEN+(()
p bi
REN
The Karush-Kuhn Kubn-Tucker conditions (Kuhn & Tucker 195I) ensure that any stationary point satis- fies the flow conservation constraints. The constraint qualification satisfied since the constraints
14

---

Published as a conference paper at ICLR 2026
linear in f Thus, achieving Equation 2 flow conservation:
Σ
=
(E)+(()
KEV
outgoing incoming Bow
where the left side represents the net flow change for object i (outgoing minus incoming), and by captures whether object appears, continues, disappears at time t.
Part IV: Graph-Theored Properties
Spectral Analysis of Tracking Graphs:
From the graph construction in Equation | the tracking
graph G2 (V,, E,, Wi) has Laplacian matrix L, with eigenvalues A1
S X2
S .... S Aprl-
The algebraic connectivity (L)) measures graph coherence:
High d2: Well-connected components (stable tracking)
Low Xz: Fragmented components (challenging tracking scenarios)
Adaptive Weight Analysis: The adaptive weights from Equation respond to graph connectivity:
02 (L.) A o2( + (L,)
The partial derivatives from Equation reveal the gradient structure, though X and X are not directly updated. Instead, this creates an implicit feedback mechanism through model parameter updates (Equation 10::
When spatial graph is fragmented (Ag(L.) small), A increases to strengthen spatial coher- ence
When temporal graph is fragmented (A2(L,) small), d increases to enforce temporal con- sistency
Convergence of Adaptive Weights: The adaptive mechanism from Equation I0 has the fixed point property. At equilibrium:
Vx,C = Vx,C=
This occurs when the spatial and temporal loss contributions are balanced relative to their graph connectivity, providing principled adaptation scene complexity.
Part V: Tracking Consistency Analysis
Identity Preservation: The flow conservation constraints from Equation 2 with Σ ensure that the total "tracking mass" is conserved. This prevents:
Object multiplication: En < (one object cannot become multiple)
Object merging: Σe fh1 < (multiple objects cannot become one)
The spatial loss from Equation 5 creates an energy landscape that penal- izes long-distance associations. For objects i.j with d(p). p(+1) > 5, the loss contribution wiyd(pi, Pl+ grows linearly, making such associations energetically unfavorable.
Spatial Coherence:
Temporal Smoothness: The temporal loss from Equation 6 penalizes sudden velocity changes. For smooth motion, vž vi-1. minimizing the temporal penalty. Abrupt changes incur quadratic penalties, encouraging physically plausible trajectories.
This completes the theoretical analysis, establishing that UniTrack provides a mathematically princi- pled framework for multi-object tracking with guaranteed differentiability. local convergence prop- erties, and built-in mechanisms for maintaining tracking consistency through graph-theoreee theoretic prin- ciples. D
15

---

Published as a conference paper at ICLR 2026
Table 6: Sigmoid-based formulation performance challenging sequences highlighting three error types. The sigmoid approach shows strong identity preservation but with detection detection-accuracy trade- offs.
Sequence
Method
MOTAЛ
IDF1
HOTA
FNL
IDs]
FML
MOT17-13
Trackformer Meinhardi al.
202T
57.42
63.16
66.29
1742
224
147
Post-occlusion
UT-Tracctormert (Sigmord)
58.20
67.86
66.98
781
138
99
MOT17-10
Trackfomerr Meinhardi et al.
(ecoomaarMeiihaddl UT- "T-Tracklormer (Sigmord)
2021
65.86
62.84
59.72
2356
113
131
Temporal inconsis.
68.30
63.96
62.27
1576
53
80
MOT17-02
Trackformer Meinhardt al. UT- Tracktetrrmrr (Sigmoid)
2021
51.23
37.51
42,63
5895
156
128
Cross-subject
37.32
38.76
37,66
6011
66
66
Table 7: Ablation study of sigmoid- -based UniTrrack formulation MOT17 after 15 epochs. Com- ponent analysis shows different trade-offs compared to the standard threshold-based approach.
Model
Components
MOTA
IDFI1
HOTA
IDs
Caow
Ligun
Lump
/
56.2
62.0
56.7
297
Trackformer
X
/
/
53.1
60.8
55.1
325
(Sigmoid)
v
55.1
62.0
56.5
287
x
55.3
62.9
57.5
273
A.2 SLEMOID-BASER SPATIAL ADIACENCY ANALYSIS
To evaluate different spatial relationship formulations, we analyze an alternative dynamic weighing and thresholding mechanism for computing spatial adjacency matrices. The standard approach uses thresholding, while the sigmoid variant implements dynamic weighing: A a o<k(0.1 d with & = 50.0, creating soft spatial relationships that enable more nuanced modeling of object interactions.
Tables 6 and present results for this dynamic weighing approach. The mechanism shows partic- ularly strong identity preservation, often achieving the lowest identity switches and fragmentations due to smoother spatial relationships that are less sensitive to positional variations. However, the softer spatial boundaries can reduce discriminative power in complex scenarios, leading lower MOTA scores. This analysis demonstrates the trade-off between spatial stability and tracking ac- curacy, confirming that our standard kard-threshold formulation provides optimal balance across diverse tracking conditions.
A.3 ADDITIONAL QUALITATIVE RESULTS WITH TRACKFORMER
In Figure 6 and we demonstrate UniTrack's effectiveness across different architectures, pro- vide additional qualitative comparisons between Trackformer and UT-Trackformer on challenging MOT17 sequences. These examples complement our MOTR analysis by showcasing how the unified graph-theoretic loss function consistently improves identity preservation across transformer-based tracking architectures.
Also, we refer readers to our supplementary material and the accompanying HTMML interactive page for additional visual results and interactive demonstrations that showcase the versatility and con- sistent performance improvements achieved by integrating the UniTrack loss across diverse MOT architectures.
A.4 KEY OBSERVATIONS FROM HYPERPARAMETER ANALYSIS
We conduct an extensive hyperparameter ablation study as shown in Table g All experiments were conducted on 4 V100 GPUs
Temperature Sensitivity: = 0.1 provides the optimal balance between soft assignments and decision confidence. Lower values 0.05) create overly confident assignments leading more
16

---

Published as a conference paper at ICLR 2026
Track former
UT-Track former
(A)
Frams 447
Foctie 464
Frome
Figure Additional qualitative comparison of Trackformer and UT-Trackformer on MOT17 challenging sequences. Red bounding boxes indicate tracking failures, green boxes show cessful tracking, and dotted boxes highlight missed detections. (A) Post-occlusion ID switches: Trackformer incorrectly reassigns subject ID from 90 94 between frames 445-490, while UT- Trackformer maintains consistent ID 132. (B) Temporal inconsistency: Trackformer loses subject ID in frame 540 (dotted red box) and assigns new ID
frame 560, while UTTTTrackformer successfully maintains ID) 60 throughout the sequence. These results demonstrate UniTrack'ss 's con- sistent improvements in identity preservation across different transformer-based architectures.
Table 8: Hyperparameter Sensitivity Analysis (GTR on MOT17)
Parameter
Value
MOTA!
IDFI
HOTA
IDSI
FPI
FNI
Time (hrs)
Memory (GB)
Temperature Parameter (r)
78.3
73.1
66.2
1047
8234
47891
4.2
6.8
79.1
74.8
67.9
951
7892
46123
4.1
6.7
0.2
78.8
74.2
67.1
978
8156
46734
4.1
6.7
0.5
77.9
72.8
65.8
1134
8743
48562
4.0
6.6
1.0
76.5
71.2
64.3
1289
9456
50123
4.0
6.5
Window Size (W frames)
W
78.2
73.9
66.8
1078
8134
47234
3.8
5.9
79.1
74.8
67.9
951
7892
46123
4.1
6.7
74.5
8023
7.8
W
10
78.6
74.1
66.9
1012
8267
46891
5.3
9.2
W
15
78.1
73.6
66.2
1089
8634
47567
6.8
12.1
Frame-rate Norma lization
(Ar)
No normalization
77.3
72.4
65.1
1234
8956
48234
4,0
6.7
Linear (current)
Ar
79.1
74.8
67.9
951
7892
46123
4.1
6.7
Logarithmic
logí f(pps)
78.7
74.3
67.2
987
8089
46567
4.1
6.8
Adaptive
78.9
74.6
67.6
963
7934
46289
4.3
7.1
identity switches, while higher values (r 2 0.3) result in ambiguous flow distributions that reduce tracking precision.
Window Size Trade-off: W = 5 frames offers the best performance-cfffcciennyy balance. Smaller windows (W 3) lose crucial temporal context for motion prediction, while larger windows (W > 10) increase computational overhead without proportional performance gains. Memory usage scales approximately linearly with window size due graph construction complexity.
Normalization Impact: Linear frame-rate normalization consistently outperforms alternatives across all metrics. The adaptive approach shows promise with competitive HOTA scores but intro-
17

---

Published as a conference paper at ICLR 2026
Track former
UT-Trackformer
(A)
BB)
(C)
Frame 120
Frame 1.4
Frame 120
Frome 140
Figure 7: Additional qualitative comparison of Trackformer and UT-Trackformer MOT17 chal- lenging sequences. Red bounding boxes indicate tracking failures, green boxes show successful tracking, and dotted boxes highlight missed detections.
duces additional computational complexity. No normalization significantly degrades performance, confirming the importance of frame-rate-awant temporal modeling.
Computational Scalability: Training time increases super-linearly for W 10 due to increased edge complexity in the graph structure. The O(n²t) complexity becomes evident memory quirements grow from 5.9GB to 12. GB when increasing window size from 3 to 15 frames.
18


Reviews:
[{"review_id": "Dx1AxEMGU1", "reviewer": "Reviewer_fpzX", "summary": "UniTrack introduces a training-only loss that couples three differentiable terms: flow, spatial, and temporal inside a graph, over a sliding-window graph with flow conservation and adaptive 𝜆. λ chosen by Laplacian connectivity; it is added to TrackFormer/MOTR/FairMOT/ByteTrack/GTR/MOTE without inference overhead. Reported gains across several mot benchmarks including mot 17/20.", "questions": "See the weakness.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. One of the biggest advantages is that the proposed method is architecture‑agnostic: it demonstrates improvements when plugged into diverse families (end-to-end transformers, joint detection-tracking, tracking-by-detection, global transformers).\n\n2. The proposed unified objective makes sense as it merges detection quality and identity preservation, and benefit the end-to-end MOT training.\n\n3. The authors show Clear ablation on error types (Table 3), clearly presenting which term combats which failure mode; qualitative figures are convincing.", "weaknesses": "1. The details of the differentiability of the flow term are not clearly conveyed. The loss scales by factors that depend on false positives/false negatives, but the paper does not define a differentiable surrogate for those counts. As far as i understand that derivation treats the FP/FN counts inside the loss as if they were constants and never explains how those counts are made differentiable with respect to the model outputs. In practice, FP/FN are discrete functions of predictions (they jump when a score crosses a threshold.\n\n2. Inconsistent definitions: the paper has defined λs and λt ((Eq. 8), as they are deterministic functions of graph connectivity (via Laplacian eigenvalues). Then, in eq.10, the λs and λt are treated as learnable parameters that can be updated by backprop. \n\n3. The paper includes frame-rate analysis and normalizes by the frame interval, which is good. Still, some ablations suggest removing the temporal term can improve certain metrics. eg MOTA by 2.1.\n\n4. Prior work already models inter-object relations and global data association with differentiable mechanisms [1]\n\n[1] SLAck: Semantic, Location, and Appearance Aware Open-Vocabulary Tracking", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ij8LxwoZFC", "reviewer": "Reviewer_m6K5", "summary": "This work proposes a new loss for multi-object tracking. This loss is based on graph theory, which integrates detection accuracy, identity preservation, and spatiotemporal consistency. Convergence and consistency of the loss are theoretically validated. Experiments on multiple trackers and multiple benchmarks demonstrate the effectiveness of the proposed loss.", "questions": "Will the code be made publicly available to help re-implement and apply this work? It would further benefit people in this field.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea of using a plug-and-ply graph-based loss makes sense.\n2. The implementation of the graph-based loss is suitable for the MOT task.\n3. The analysis of the loss is reasonable.\n4. The loss is effective with different trackers on multiple benchmarks, which shows the universality of the proposed loss.\n\nOverall, this work develops a loss which has both solid theoretic foundation and obvious improvement in practice. I believe this work will benefit the community.", "weaknesses": "The weights of spatial and temporal loss are adaptive to the graph connectivity. It is encouraged to compare with other solutions, like adaptive parameters directly learned by the network, and fixed parameters.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oT2WO7hmix", "reviewer": "Reviewer_3Pqh", "summary": "This paper introduces UniTrack, a differentiable graph representation learning framework designed to unify detection accuracy, identity preservation, and spatio-temporal consistency for multi-object tracking (MOT). Instead of relying on separate detection and association modules, UniTrack formulates tracking as a graph flow optimization problem and introduces a unified loss composed of three parts—flow, spatial, and temporal—optimized in an end-to-end differentiable manner. The framework includes an adaptive Laplacian-based weighting mechanism and can be integrated into existing trackers (e.g., MOTR, TrackFormer, ByteTrack, FairMOT, GTR) without architectural modifications. Experiments across MOT17, MOT20, SportsMOT, and DanceTrack show consistent gains (up to +9.7% MOTA, +12.3% IDF1) and reduced identity switches, highlighting the effectiveness and generality of the proposed framework.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. UniTrack combines detection, identity, and temporal consistency into a single differentiable loss.\n2. UniTrack can be applied to tracking architectures without network modification, demonstrating practical flexibility.\n3. Experiments across MOT17, MOT20, SportsMOT, and DanceTrack show consistent gains (up to +9.7% MOTA, +12.3% IDF1) and reduced identity switches, highlighting the effectiveness and generality of the proposed framework.", "weaknesses": "1. The method introduces more training complexity and memory overhead; scalability to large scenes or dense MOT scenarios remains a concern.\n2. The authors are suggested to add more ablation studies on the adaptive Laplacian weighting.\n3. The ablation section focuses mainly on component removal but could include more fine-grained analysis of hyperparameters (e.g., λs, λt updates, thresholding strategies).\n4. It is recommended that the authors compare UniTrack with more recent MOT approaches, especially those that incorporate memory-based mechanisms such as MOTRv2, MeMOTR, and MOTIP, to better show the advantages of the proposed framework.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)