Title: STAR: Spatial-Temporal Tracklet Matching for Multi-Object Tracking
Venue: NeurIPS 2025
Accepted: yes
Keywords: multi-object-tracking

spatial-temporal

graph matching
Abstract: Existing tracking-by-detection Multi-Object Tracking methods mainly rely on associating objects with tracklets using motion and appearance features. However, variations in viewpoint and occlusions can result in discrepancies between the features of current objects and those of historical tracklets. To tackle these challenges, this paper proposes a novel Spatial-Temporal Tracklet Graph Matching paradigm (STAR). The core idea of STAR is to achieve long-term, reliable object association through the association of ``tracklet clips (TCs)". TCs are segments of confidently associated multi-object trajectories, which are linked through graph matching. Specifically, STAR initializes TCs using a Confident Initial Tracklet Generator (CITG) and constructs a TC graph via Tracklet Clip Graph Construction (TCGC). In TCGC, each object in a TC is treated as a vertex, with the appearance and local topology features encoded on the vertex. The vertices and edges of the TC graph are then updated through message propagation to capture higher-order features. Finally, a Tracklet Clip Graph Matching (TCGM) method is proposed to efficiently and accurately associate the TCs through graph matching. STAR is model-agnostic, allowing for seamless integration with existing methods to enhance their performance. Extensive experiments on diverse datasets, including MOTChallenge, DanceTrack, and VisDrone2021-MOT, demonstrate the robustness and versatility of STAR, significantly improving tracking performance under challenging conditions.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes an MOT method under the tracking-by-detection framework.  It uses graph representation learning to explore spatial-temporal information within tracklets. And it uses tracklet clip-level graph matching to gain robustness to occlusions and ID-switches. It does experiments on three widely used benchmarks to verify its effectiveness.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n1）\tIts motivation is plausible and it describes clearly different modules of the method like graph construction and propagation.\n\n2）\tAblation studies on the tracklet clip graph construction and the STAR module are provided.\n\n3）\tIt selects a variety of challenging benchmarks to verify its effectiveness.\n\n4）\tSupplementary material provides more comparisons on MOT and UAVDT datasets.\n\nWeakness：\n--Overall, the idea is not new in MOT domain which solves viewpoint variations and occlusions by exploiting spatial-temporal information. \nFormulating the data association problem as a graph matching task is common in MOT. And the hierarchical association strategy from detections to tracklet clips is also commonly used. It is suggested to highlight the major differences between the existing tracklet graph matching-based methods and the proposed TCGM.\n\n--Some technique details:\nFor object that haven’t appeared in a frame in the TC, they use an empty placeholder. How the appearance and position features and the topological features of such dummy nodes are computed? How to process them in the following graph matching.\n\nThey use a weighted binary cross-entropy loss function for the differentiable graph matching. Though it refers to [31], some explanations about KKT conditions and the implicit function theorem can be provided in supplementary material.\n\n--Unclear experiment details:\n-In the ablation study on the effect of STAR, the experiment configurations are unclear. Adding STAR as a postprocessing step to an existing method seems sure to bring improvement.\n\n-How it determines to create a new trajectory or terminate an existing trajectory with the object disappearing for a long time. \n\n-- Since the method is verified in both public and private detection settings, it is suggested to list the two types separately in experiment results. Quality of detections greatly affects the tracking performance in the TBD framework.\n\n----The whole pipeline as in Fig. 2 is somewhat complex, which may hinder the proposed method from real application. Efficiency is an unavoidable problem in MOT, though this problem is briefly discussed in the limitation section.\n\nQUESTIONS:\n1)\tIt is suggested to resummarize its contribution to highlight its difference with existing tracklet graph matching-based methods works. \n2)\tUnclear technical and experiment details, to see the weakness.\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a Spatial-Temporal Tracklet Graph Matching method for multiple object tracking. To address occlusion issues, trajectory graphs are built by combining tracklet clips and graph matching. However, there are some concerns regarding motivation and performance.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n* The evaluation is comprehensive and conducted on several datasets.\n* The presentation is clear and easy to follow.\n\nWeaknesses\n* The Confident Initial Tracklet Generator heavily relies on the Dynamic IoU-based association. However, IoU-based Associations are not strictly reliable and generate wrong tracklets, especially after occlusion. Consequently, the constructed graph can be polluted, which weakens the purpose of the graph construction. Meanwhile, the ablation study should also show the results where the tracker solely relies on the IoU-based association and the effects of the association (using association results and GTs on the MOT-17 validation set).\n* The STAR aims to address the occlusion issues. However, the methods do not provide information or explanations indicating how each implementation contributes to the motivation to address the occurrence.\n* In the literature review and benchmarks, some state-of-the-art methods are missing, such as BoosTrack++ and OccluTrack. Meanwhile, the performance on the MOT 17 and MOT 20 datasets is not comparable with some of those methods. In addition, HOTA should be included in the benchmark, which is the most important metric for evaluating tracking performance.\n1. Stanojević, V., & Todorović, B. (2024). BoostTrack++: using tracklet information to detect more objects in multiple object tracking. arXiv preprint arXiv:2408.13003.\n2. Gao, J., Wang, Y., Yap, K. H., Garg, K., & Han, B. S. (2025). Occlutrack: Rethinking awareness of occlusion for enhancing multiple pedestrian tracking. IEEE Transactions on Intelligent Transportation Systems.\n\nQUESTIONS:\n1. Clarify how the proposed methods address the conclusion issues\n2. Include more recent state-of-the-art works\n3. Provide the ablation study on the IoU-based Association\n\nLIMITATIONS:\nNo, how the proposed methods address the conclusion issues.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes STAR, a novel multi-object tracking framework that reframes the object association problem by operating at the tracklet clip level rather than individual detection instances. STAR constructs spatial-temporal tracklet clip graphs incorporating appearance, motion, and topological features, and performs graph matching to reliably associate object tracklets over time. The framework is designed to handle frequent occlusions, viewpoint changes, and crowded scenarios. It is model-agnostic and can be integrated into existing MOT systems. Evaluations show that STAR achieves state-of-the-art performance across multiple metrics.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n1. The paper addresses a persistent and well-known challenge in multi-object tracking (MOT): maintaining reliable object associations under frequent occlusions and viewpoint changes.\n1. The STAR framework introduces a tracklet clip graph matching paradigm, which is a novel reframing of the object association problem that directly addresses limitations in current MOT paradigms.\n1. Clear, well-motivated problem statement targeting known limitations in existing MOT methods.\n1. Technically novel approach leveraging tracklet clip graph matching for more robust, long-term association.\n1. Strong experimental validation across multiple public benchmarks, consistently outperforming existing state-of-the-art methods.\n1. Thorough ablation and complexity analysis, demonstrating the contribution of each module and computational benefits.\n1. Model-agnostic integration, showing measurable improvements when paired with various baseline tracking systems.\n\n## Weaknesses\n1. While the complexity analysis shows significant improvement over conventional graph matching, real-time deployment at large scale is still constrained.\n1. While the paper provides extensive quantitative results and ablation studies, it lacks a qualitative or case-specific analysis of where and why STAR may still fail (e.g., in highly dynamic or extremely long occlusion cases).\n1. No run-time performance benchmarking: the paper discusses computational complexity reduction at the graph matching stage and reports FPS values, but it’s unclear how STAR would perform in latency-sensitive deployments like autonomous driving or surveillance on edge devices.\n1. No analysis on domain generalization: there’s no experiment on how STAR handles domain shift (e.g. indoor to outdoor, day to night).\n\nQUESTIONS:\n1. Section 3.2 is quite brief and lacks details; for instance, how is the IoU threshold dynamically adjusted based upon the mentioned factors (object velocity, detection box size, and inter-frame time intervals)?\n1. Has there been a study conducted to verify that the CITG makes \"tracklet initialization more robust across diverse tracking scenarios\" (L130)?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes STAR, an association module based on tracklet clips. It initializes sequences of multiple target trajectories through IoU between detection boxes. Subsequently, a Tracklet Clip Graph is constructed on the temporal sequence, followed by graph matching, which ensures spatio-temporal consistency of the matches. STAR is a plug-and-play module that can be seamlessly integrated into various downstream models. It demonstrates significant performance improvements when compared to baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n1.\tThis paper explores a plug-and-play long-range association module, which holds significant value for downstream applications.\n2.\tThe unified association paradigm based on graph matching designed in this paper achieves unified spatio-temporal matching, avoiding the matching ambiguity issues caused by traditional one-by-one matching. Instead, it considers matching from the perspective of the entire trajectory sequence, which brings new insights to the field.\n3.\tThe paper is well-written, with clear descriptions and logical flow.\nWeakness:\n1.\tThe paper uses too many abbreviations, and they are quite similar, which hinders continuous reading for the reader. The authors might consider using more distinctive names or not over-abbreviating certain algorithms, simply referring to them as \"Graph Matching\" would suffice.\n2.\tThe paper unfolds the graph in a temporal sequence to achieve matching. However, how are cases handled where multiple TC (Tracklet Clip) matching results are inconsistent? For example, if ID=1 and ID=2 frequently switch IDs within a short period, would this pose a greater challenge for subsequent matching?\n3.\tThe paper claims to achieve long-range object tracking. However, the ablation study shows that N=4 yields the best results, with performance still improving compared to N=8. I personally believe this length (even 2N=8 frames) is not particularly long.\n4.\tThe authors did not analyze how much impact the proposed module has on inference performance.\n\nQUESTIONS:\n1.\tIf time permits, the authors could analyze whether there should be some degree of overlap or skipped-frame sampling for the frame sampling of Slicing TCs. This is because overlap can provide implicit temporal feature propagation, which can implicitly extend the length of the temporal sequence under consideration.\n2.\tDo the authors have more data to explain the impact of the number of TCs (number of frames retained) and the number of frames retained in the expanded graph on experimental performance?\nIf the authors address the above questions, I would consider increasing the score.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}