ICLR 2025

Published as a conference paper at ICLR 2025
CO-MOT: BOOSTING END-TO-END TRANSFORMER-
BASED MULTI-OBJECT TRACKING VIA COOPETITION
LABEL ASSIGNMENT AND SHADOW SETS
Feng Yan⇤, Weixin Luo⇤, Yujie Zhong, Yiyang Gan, Lin Ma†
Meituan Inc., China.
{yanfeng05, luoweixin, zhongyujie, ganyiyang}@meituan.com
forest.linma@gmail.com
ABSTRACT
Existing end-to-end Multi-Object Tracking (e2e-MOT) methods have not sur-
passed non-end-to-end tracking-by-detection methods. One possible reason lies
in the training label assignment strategy that consistently binds the tracked objects
with tracking queries and assigns few newborns to detection queries. Such an as-
signment, with one-to-one bipartite matching, yields an unbalanced training, i.e.,
scarce positive samples for detection queries, especially for an enclosed scenew
with the majority of the newborns at the beginning of videos. As such, e2e-MOT
will incline to generate a tracking terminal without renewal or re-initialization,
compared to other tracking-by-detection methods. To alleviate this problem, we
propose Co-MOT, a simple yet effective method to facilitate e2e-MOT by a novel
coopetition label assignment with a shadow concept. Speciﬁcally, we add tracked
objects to the matching targets for detection queries when performing the label
assignment for training the intermediate decoders. For query initialization, we ex-
pand each query by a set of shadow counterparts with limited disturbance to itself.
With extensive ablation studies, Co-MOT achieves superior performances without
extra costs, e.g., 69.4% HOTA on DanceTrack and 52.8% TETA on BDD100K.
Impressively, Co-MOT only requires 38% FLOPs of MOTRv2 with comparable
performances, resulting in the 1.4⇥faster inference speed. Source code is publicly
available at https://github.com/BingfengYan/CO-MOT.
1
INTRODUCTION
Multi-Object Tracking (MOT) is traditionally tackled by a series of tasks, e.g., object detection (Zhao
et al., 2024; Zou et al., 2023), appearance Re-ID (Zheng et al....