ReMOT: A Model-agnostic Refinement for Multiple Object Tracking⋆Fan Yanga,b,∗ , Xin Changa , Sakriani Saktia,b , Yang Wuc,∗ and Satoshi Nakamuraa,ba Nara Institute of Science and Technology, Japanb RIKEN, Center for Advanced Intelligence Project AIP, Japanc Kyoto University, Japan

ARTICLE INFOABSTRACT

Keywords:
Multiple Object Tracking
Refinement

Although refinement is commonly used in visual tasks to improve pre-obtained results, it has not beenstudied for Multiple Object Tracking (MOT) tasks. This could be attributed to two reasons: i) it has
not been explored what kinds of errors should — and could — be reduced in MOT refinement; ii)
the refinement target, namely, the tracklets, are intertwined and interactive in a 3D spatio-temporalspace, and therefore changing one tracklet may affect the others. To tackle these issues, i) we define
two types of errors in imperfect tracklets, as Mix-up Error and Cut-off Error, to clarify the refinement
goal; ii) we propose a Refining MOT Framework (ReMOT), which first splits imperfect tracklets andthen merges the split tracklets with appearance features improved by self-supervised learning. Experiments demonstrate that ReMOT can make significant improvements to state-of-the-art MOT results
as powerful post-processing. As a new application, we demonstrate that ReMOT has the potentialof being used to assist semi-automatic MOT data annotation and partially release humans from thetedious work.

1. Introduction⋆

This paper is an extension for our winner solution for CVPR2020
MOTS Track 1 Challenge.
∗ Corresponding authoryang.fan.xv6@is.naist.jp (F. Yang); chang.xin.co0@is.naist.jp(X. Chang); ssakti@is.naist.jp (S. Sakti); wu.yang.8c@kyoto-u.ac.jp (Y.
Wu); s-nakamura@is.naist.jp (S. Nakamura)https://scholar.google.co.jp/citations?user=LndV87gAAAAJ&hl=en (F.
Yang); https://scholar.google.com/citations?user=E0uhtlkAAAAJ&hl=en(S. Sakti);

https://scholar.google.com/citations?user=vwOQ-UIAAAAJ&hl=en (Y. Wu);
https://scholar.google.com/citations?user=ckdfXawAAAAJ&hl=en (S.

Nakamura)

ORCID (s): 0000-0001-7185-5688 (F. Yang)

F.Yang et al.: Preprint submitted to Elsevier

ID1

ID3

ID2

Tracklet Hypotheses:
#1
id

#2
id

#3
id

#4
id

Mix-up Error

1D-view Locations

t1
t2
FramesMultiple Object Tracking (MOT) algorithms are applied
to obtain object trajectories from the spatio-temporal data
(e.g., video). It has a wide range of applications that cover,
but are not limited to: traffic control [14], sports analysis [21],and animal study [3]. In MOT tasks, however, the data annotation costs a vast of time and effects, because a one-minutevideo may take several hours to make the tracklet labels. A
pre-trained MOT model, though, can be incorporated into
the annotation pipeline to assist human labeling, it might bechallenging to generate satisfactory results on unseen videos
due to the domain gap. To alleviate this issue, we propose amodel-agnostic Refining MOT framework (ReMOT) to refine arbitrary MOT results on target videos.
MOT performance is basically determined by the object
detection performance and the data association performance.
Given a video data, object detection phase generates observations on each frame, with the format of bounding boxes(e.g., [17]), instance masks (e.g., [34]), or object key points(e.g., [1]). In the data association phase, observations that
correspond to the identical object are associated with a consistent set of trajectories, which are also referred to as tracklets. In this work, we focus on how to refine a given MOT re-

Tracklet Ground Truth:t3

Cut-off Error

t4
t5
t6

Figure 1: An illustration of Mix-up Error and Cut-off Error.

sult from the data association perspective, without touching
the object detection. More specifically, our approach focus
on solving two types of errors in imperfect tracklets, whichare• Mix-up Error: Observations of different identities areincluded in the same tracklet.
• Cut-off Error: Observations of the same identity are
spread in more than one tracklet.
Note that, both Mix-up Errors and Cut-off Errors could cause
ID switch and other poor values in popularly used MOT metrics [17, 24]. Directly aiming to improve a specific value in
MOT metrics is complex, therefore, we use Mix-up Errors
and Cut-off Errors (Figure 1) to simplify our refining goal.
To correct the Mix-up Errors, an intuitive approach isto split a tracklet at where its identity changed. In current
MOT methods [37, 9], the appearance affinity plays an important role to verify observation identities. For an ideal
Page 1 of 10

ReMOTcase, the appearance affinity between observations of thesame identity should be higher than different identities between consecutive frames, accordingly, a split threshold canbe found between them. However, due to the domain divergence between videos, even the appearance encoder has
been satisfactorily trained on labeled videos, it may not adapt
to new videos and generate satisfactory appearance features.
Besides, the proper split thresholds may vary from one video
to another, and therefore it may not be practicable to select a
unified split threshold by cross-validation on labeled videos.
We tackle this issue by adopting the appearance encoder totarget videos through inter-frame supervised learning andsemi-automatically selecting split thresholds from the statistical information of intra-frame detections.
To correct the Cut-off Errors, we merge the split tracklets referring to improved appearance features and the statistical information of the split tracklets. During intra-frame
self-supervised learning, we generate positive samples throughdata augmentation. This in turn could result in a lack ofrobustness to varying conditions as the augmented positive
sample for queries might be too simple to provide enough supervision for learning robust feature representations. After
splitting, the purified tracklets can be used as reliable pseudolabels to further improve the capability of the appearance encoder, we proposed pseudo-tracklet self-supervised learning
to retrain the appearance encoder. With previous processes,
we suppose that the more dependable appearance affinity
can be formed between the split tracklets. We accommodate
the hierarchical clustering [30] to associate the split tracklets
that have the same identity. As a result, the Cut-off Errors
could be reduced.
We evaluated ReMOT on MOTChallenge datasets [17,24] and observed that ReMOT successfully reduced Mix-up
Errors and Cut-off Errors in imperfect tracklets. To summarize, our contributions are three-fold:• We defined two types of MOT errors that can be reduced from the data association perspective, as theMix-up Errors and Cut-off Errors. With this definition, the MOT refinement goal has become clear.
• To the best of our knowledge, ReMOT is the first workto perform model-agnostic refinement for MOT tasks.
Within ReMOT, we proposed novel intra-frame selfsupervised learning and pseudo-tracklet self-supervised
learning to improve the capability of appearance encoder, introduced a new strategy to obtain hyper parameters from the statistical information in MOT results, and specifically designed mechanisms for splitting and merging tracklets.
• Experiments demonstrate ReMOT can make improvements on existing state-of-the-art MOT results without introducing extra-human works. Hence, ReMOT
can be used to assist semi-automatic MOT data annotation and partially release humans from the tediouswork.

F.Yang et al.: Preprint submitted to Elsevier

2. Related Works
Although refinement strategies are widely applied in various visual tasks [39, 7, 10, 25, 20], as far as we are aware,model-agnostic MOT refinement has not been studied and
ReMOT is the first approach. However, proposing ReMOT
is greatly motivated by existing works. In this section, we
discuss related works of ReMOT from two aspects: MOTfeatures and MOT data association.
MOT features are properties that distinguish an object
from others. Among them, the motion features and the appearance features are commonly utilized. Since an identical object usually has a fixed pattern of movement, shortterm motion can be predicted and served as a strong cue
to verify whether cross-frame observations should be associated or not. In IoU-Tracker[5], simply associating crossframe detection with the maximum IoU (Intersection-overUnion) yielded acceptable performance for slow-moving objects but failed for fast-moving ones. To improve the motion modeling capability, Kalman filter was applied in various works [19, 4, 37]. However, due to the complexity of
Kalman filter, heuristically selecting its parameters and thecorresponding gating thresholds for motion consistency verification is challenging. To alleviate these issues, we simplify the motion consistency verification problem as a linear regression error verification in our ReMOT. The hyperparameters of our motion modeling are distilled from the statistical information of motions in the labeled dataset.
Object appearance is another important clue for identifying cross-frame observations. How to encode an object image into its appearance features has been well studied in object re-identification (ReID) works [16, 22]. Founded on supervised ReID studies, recent video-based ReID works [35,39, 18, 38] further introduced self-supervised learning toprogressively improve appearance features, which gives theinsight to fix Cut-off Errors in tracklets. However, they supposed that perfect short-term tracklets are given (i.e., w/o
Mix-up Errors but w/ Cut-off Errors), or, top-k nearest neighbors were heuristically selected as an identical observation,
which may also include Mix-up Errors. For this reason, there
is a risk to directly apply their methods in our case — Mixup Errors will be faithfully learned in their self-supervised
learning and cause tracking drift to further increase Cut-off
Errors. We apply the splitting process to alleviate this issue in our ReMOT. The splitting process also makes ReMOT able to utilize the prior knowledge of arbitrary MOT
results and perform refinement on them, which is different
from [39, 38].
MOT Data Association is applied to connect identicalobservations into tracklets based on the similarity of MOTfeatures. Data association methods can be divided into two
categories — online and offline. The online data associationis performed on observations that are available up to the current frame. Typically, linear assignment algorithms [15, 13]
are applied to associate consecutive-frame observations [37,42]. Different from online approaches, offline data association takes global observations into consideration, which may
not be applicable for real-time applications but be ideal forPage 2 of 10

ReMOT
1D-view Locations

t1

Tracklet Ground Truth:

Tracklet Hypotheses:
#6
#4 id
#5 id
#3 id
#1 id
#2 id
id

t3
t4

Pseudo-tracklet
Self-supervised
Learning

1D-view Locations

t1

t2
Appearance
Encoder

Splitting
Modulet2

t3

Appearance
Encodert4

Merging
ModuleFrames

ID3

t1
FramesID2

1D-view Locations

t2
FramesID1

Intra-frame
Self-supervised
Learning

t3
t4

t5

t5

t5

t6

t6

t6
Split Tracklets

Imperfect Tracklets

Improved Tracklets

Mix-up Error

Cut-off Error

Figure 2: The architecture of ReMOT framework. ReMOT progressively adapts appearance encoder to unlabelled target data by self-supervised learning. With improved
appearance features, imperfect tracklets are split and then merged to refined tracklets.

assisting annotation works. Numerous offline approaches
may lead to Mix-up Errors and Cut-off Errors in predictedhave been proposed in previous studies [23]. Among them,tracklets. In the first step, we attempt to improve the capaformulating MOT data association as a global clustering prob- bility of the appearance encoder by applying a novel intralem has achieved great successes since decades ago [29, 8,
frame self-supervised learning.
3, 32, 33, 28]. Compared with methods that use Random
We leverage the appearance encoder from the object reField (e.g., [44]) or Min-cost Flow (e.g., [43]), the global
identification task (e.g., [22]). A ResNet-50 [11] is used asclustering approach allows us to add association constraints
the backbone, and its global-average-pooling output, which
flexibly, but it also entails a cost: the clustering result is senis a 2048-dimension vector, is employed as the appearance
sitive to the cutting threshold. Nonetheless, how to selectfeatures. The triplet loss (i.e., 𝑡𝑟𝑖𝑝𝑙𝑒𝑡 ) [6] is applied to train
the proper cutting threshold, has not been properly solved. In
the appearance encoder, and, the optimization goal is to minReMOT, we propose a novel approach that can semi-automaticallyimize
decide a proper cutting threshold referring to the statistical‖‖22
𝑡𝑟𝑖𝑝𝑙𝑒𝑡 = 𝑚𝑎𝑥 ( ‖𝑓𝑎 − 𝑓𝑝 ‖ − ‖𝑓 − 𝑓𝑛 ‖‖2 + 𝛼, 0 ), (1)information of observations.
‖‖2 ‖ 𝑎where 𝑓𝑎 , 𝑓𝑝 , and 𝑓𝑛 denote the appearance features of theanchor, positive, and negative samples, respectively; 𝛼 indi3. ReMOT
cates the margin between the positive and negative pairs and
The central idea of ReMOT is to split imperfect trackwe empirically set 𝛼 = 0.3.
lets and then associate the split tracklets with improved apOn videos that have labeled tracklets, we can obtain triplet
pearance features. In particular, ReMOT consists of four
samples(i.e., anchor, positive, and negative samples) throughphases: i) training the appearance encoder on given obsera
randomsampling — positive pairs are from the same trackvations by intra-frame self-supervised learning; ii) splitting
let
while
negativepairs are from the different tracklets. On
pre-obtained tracklets referring to the inter-identity similartarget
videos,imperfect
tracklets are estimated by other MOT
ity upper bound; iii) retraining the appearance encoder usmethods.
Due
to
the
domaingap between labeled and unlaing the split tracklets as reliable pseudo labels; iv) merging
beled
videos,Mix-upErrorsandCut-off Errors may be conthe split tracklets utilizing improved appearance features andtained.
This
leads
to
a
paradox:on one side, we hope tostatistical information. We illustrate the ReMOT framework
train
the
appearance
encoderonunlabeled
videos, while on
in Figure 2.
the other side, the appearance encoder tends to memorize
errors in imperfect tracklets, resulting in poorer model per3.1. Intra-frame Self-supervised Learning
formance in practice. We avoid inheriting errors of those
Due to the domain gaps between labeled videos and unimperfect tracklets and propose intra-frame self-supervisedlabeled target videos, even the state-of-the-art MOT algolearning. As Figure 3 shows, we assume that observations
rithm, which is pre-trained on labeled videos, may not genare exclusive within the same frame, and therefore they naterate satisfactory appearance features on target videos. This
urally form negative pairs to each other. Instead of using
F.Yang et al.: Preprint submitted to Elsevier

Page 3 of 10

ReMOT
Tracklet Ground Truth:
ID3

ID2

#1
id

#2
id

#3
id

#4
id

Augmentation

ID1

Tracklet Hypotheses:PP

1D-view Locations

Appearance
Encoder

N

t2

t4

𝛅= 2

N

Framest3

f t1 f t2

Labelled
Tracklets
Datasett1

f t3 f t4

Framest1

t3

RandomSampling

1D-view Locations

t2
t3
t4

t4

CosineSimilarity

t5
t6

t2

t1

forming a mini batch by the ratio
1:1 for self-supervised training

A[𝐭𝟐 , 𝐭𝟑 ]𝐍𝐌𝐀𝐄[𝐭𝟐 , 𝐭𝟑 ] =𝜽𝒎𝜽 𝒎𝒐𝒕𝒊𝒐𝒏

𝜽 𝒂𝒑𝒑
𝒔𝒑𝒍𝒊𝒕

Figure 3: The data processing scheme for intra-frame selfsupervised learning. N and P represent the negative sample
and the positive sample, respectively. For the unlabeled target
videos (left), the negative sample is sampled from infra-frameobservations while the positive sample is formed by data augmentation. For the labeled videos (right), referring to groundtruth tracklets, random sampling is applied.

noisy tracklet labels, positive samples can be generated byaugmenting the anchor samples (e.g., random flip). The augmentation process dramatically changes the pixel content of
the anchor sample but holds the same identity. During thetraining, we form a mini-batch of triplets from labeled videos
and unlabelled target videos by the ratio of 1 ∶ 1. Throughthis training, the appearance encoder is adapt to the unlabelled target videos and discriminative appearance features
can be obtained. Note that, neither the appearance encoder
nor the triplet loss is our contribution. Our innovation lies inconstructing the data processing scheme of intra-frame selfsupervised learning.

1+∑ bboxsize
4

𝐍𝐌𝐀𝐄[𝐭𝟐 , 𝐭𝟑 ]A[𝐭𝟐 , 𝐭𝟑 ]

split
t2

t1

t3

t4

Figure 4: The splitting process. We split a pre-obtained tracklet where the appearance features or motion features significantly change. The left branch calculates the apparent affinityof two consecutive fragments and does splitting whenever it𝑎𝑝𝑝
is lower than threshold 𝜃𝑠𝑝𝑙𝑖𝑡. The right branch calculates the
NMAE value and does splitting if it is larger than threshold𝜃 𝑚𝑜𝑡𝑖𝑜𝑛 .

It is interesting to note that, the histogram of appearance
cosine similarity between intra-frame observations can beapproximated by a normal distribution (Figure 4), and within
three standard deviations is 99.7% of the observation pairs.
𝑎𝑝𝑝) at three
We set an appearance affinity threshold (i.e., 𝜃𝑠𝑝𝑙𝑖𝑡
standard deviations from the mean and do splitting whenever3.2. Splitting Process
the intra-tracklet appearance affinity is lower than it (i.e.,𝑎𝑝𝑝
We split a pre-obtained tracklet where the appearance).
[𝑡𝑐 − 1, 𝑡𝑐 ] < 𝜃𝑠𝑝𝑙𝑖𝑡
features or motion features change significantly. The splitIn some cases (e.g., occlusion), it could be challenging
ting process is shown in Figure 4. After removing observato distinguish different objects by only using the appearance
tions that are heavily occluded, the similarity of intra-frame
affinity. We introduce the motion affinity to improve the roobservations (i.e., different objects) should be relatively smallerbustness. The short-term motion of an object can be treatedthan that between intra-tracklet observations (i.e., identicalas linear motion, that is, object moving in a straight line.
object). We use cosine similarity between a pair of appearWith the same temporal window 𝛿, we calculate Normalized
ance features to measure their similarity. At a checkpoint
Mean Absolute Error (NMAE) to verify the motion consisbetween frames 𝑡𝑐 − 1 and 𝑡𝑐 , we set a temporal window oftency at the checkpoint between frame 𝑡𝑐 − 1 and 𝑡𝑐 . NMAE𝛿 frames, by default, 𝛿 = min(𝑡𝑐 − 𝑡𝑠𝑡𝑎𝑟𝑡 , 𝑡𝑒𝑛𝑑 − 𝑡𝑐 , 5), whereis formulated by
𝑡𝑠𝑡𝑎𝑟𝑡 and 𝑡𝑒𝑛𝑑 represents the start frame and the end frame
in a tracklet, respectively, and we set 5 frames as the largest
𝑡𝑐 −1
𝑡𝑐 −1
𝑡
𝑡
1 ∑ ||𝐹1 (𝑡𝑖 ) − 𝑥 𝑖 || 1 ∑ ||𝐹2 (𝑡𝑖 ) − 𝑦 𝑖 ||window size. The average cosine similarity between frames
NMAE[𝑡𝑐 − 1, 𝑡𝑐 ] =+𝛿 𝑡 =𝑡 −𝛿𝛿 𝑡 =𝑡 −𝛿𝑤𝑡𝑖ℎ𝑡𝑖
of [𝑡𝑐 − 𝛿 ∶ 𝑡𝑐 − 1] and [𝑡𝑐 ∶ 𝑡𝑐 + 𝛿 − 1] are used to decide𝑖 𝑐
𝑖 𝑐
the splitting point, which can be formulated by
𝑡𝑗 |
𝑡𝑗 |
𝑡𝑐 +𝛿−1 |𝑡𝑐 +𝛿−1 |[𝑡𝑐 − 1, 𝑡𝑐 ] =𝑡𝑐 −1 𝑡𝑐 +𝛿−1
∑ ∑𝑓 𝑡𝑖 𝑓 𝑡𝑗
1𝑡 ‖ ‖ 𝑡𝑗 ‖
𝛿 2 𝑡 =𝑡 −𝛿 𝑡𝑗 =𝑡𝑐 ‖‖𝑓 𝑖 ‖ ‖𝑓 ‖
𝑖 𝑐

+

(2)

where 𝑓 𝑡𝑖 and 𝑓 𝑡𝑗 are appearance features of frame 𝑡𝑖 and 𝑡𝑗 ,respectively; [𝑡𝑐 − 1, 𝑡𝑐 ] is the appearance affinity at checkpoint between 𝑡𝑐 − 1 and 𝑡𝑐 .
F.Yang et al.: Preprint submitted to Elsevier

1 ∑ ||𝐹3 (𝑡𝑗 ) − 𝑥 || 1 ∑ ||𝐹4 (𝑡𝑗 ) − 𝑦 ||+
𝛿 𝑡 =𝑡𝛿 𝑡 =𝑡
𝑤𝑡𝑗
ℎ𝑡𝑗
𝑗

𝑐

𝑗

𝑐

(3)
where 𝑤 and ℎ respectively denote the width and heightof object bounding boxes; 𝑥 and 𝑦 are the 2D coordinate ofbounding box center; 𝐹1 , 𝐹2 , 𝐹3 , and 𝐹4 are four indepenPage 4 of 10

ReMOT

ID1

ID2

ID3

Tracklet Hypotheses:
#6
#4 id
#5 id
#3 id
#1 id
#2 id
id

1D-view Locations

t2
t4
t5

RandomSampling

P

N

t1

Labelled
Tracklets
Dataset

N

t2
t3
t4

forming a mini batch by the ratio
1:1 for self-supervised trainingt6

1D-view Locations

Appearance
Encoder

Figure 5: The data processing scheme for pseudo-tracklet selfsupervised learning. N and P represent the negative sample
and the positive sample, respectively. For the unlabeled target
videos (left), both negative and positive samples are sampledfrom the estimated pseudo tracklets that are not temporally
overlapped. For the labeled videos (right), referring to groundtruth tracklets, random sampling is applied.

inf

inf

0.1

0.4

inf

inf

0.5

0.2

0.1

0.5

inf

inf

0.4

0.2

inf

inf

+
𝐍𝐌𝐀𝐄 = 1 ∑4 bbox size𝒎𝜽𝜽 𝒎𝒐𝒕𝒊𝒐𝒏

Distance Matrix W
Distance

Framest3

P

Framest1

Random Sampling from Temporallyoverlapped Pseudo Tracklets

Tracklet Ground Truth:

Hierarchical Clustering
𝒂𝒑𝒑

𝒂𝒑𝒑

𝟏 − 𝜽𝒎𝒆𝒓𝒈𝒆𝜽𝒎𝒆𝒓𝒈𝒆(Cutting
threshold)

Clusters

dent linear regression functions, and they are obtained by
optimizing
𝑡𝑐 +𝛿−1∑

𝐹1

𝑡𝑐 +𝛿−1||𝐹1 (𝑡𝑗 ) − 𝑥𝑡𝑗 ||2 ,𝑡𝑗 =𝑡𝑐

∑

𝐹2

𝑡𝑐 −1

∑

𝐹3
𝑡𝑖 =𝑡𝑐 −𝛿||𝐹2 (𝑡𝑗 ) − 𝑦𝑡𝑗 ||2 ,𝑡𝑗 =𝑡𝑐(4)

𝑡𝑐 −1||𝐹3 (𝑡𝑖 ) − 𝑥𝑡𝑖 ||2 ,∑𝐹4||𝐹4 (𝑡𝑖 ) − 𝑦𝑡𝑖 ||2 .

𝑡𝑖 =𝑡𝑐 −𝛿

In detail, we perform linear regression 𝐹1 on frames [𝑡𝑐 ∶𝑡𝑐 +𝛿−1]. The optimal 𝐹1 is obtained by minimizing ||𝐹1 (𝑡𝑗 )−𝑥𝑡𝑗 ||2 , ∀ 𝑡𝑗 ∈ {𝑡𝑐 ∶ 𝑡𝑐 + 𝛿 − 1}. On frames [𝑡𝑐 − 𝛿 ∶ 𝑡𝑐 − 1],
using the optimized 𝐹1 , we calculate the divergence between
𝐹1 (𝑡𝑖 ) and 𝑥𝑡𝑖 , and then normalize its scale by 𝑤𝑡𝑖 , where 𝑡𝑖 ∈{𝑡𝑐 − 𝛿 ∶ 𝑡𝑐 − 1}. The 𝐹2 , 𝐹3 , and 𝐹4 are applied in the sameway. Putting them together, we obtain NMAE[𝑡𝑐 − 1, 𝑡𝑐 ],in where the smaller value implies the better motion consistency.
On the labeled videos, using MOT16 as an example, we
obtain a histogram of NMAE values. We observe that mosttracklet fragments have a very small NMAE (Figure 4), indicating that object motion can be roughly treated as linear
motion within a short time. Hence, we choose the motion
threshold (i.e., 𝜃 𝑚𝑜𝑡𝑖𝑜𝑛 ) at the largest NMAE value of the labeled dataset. Putting the appearance and motion verification together, ReMOT can correctly handle most splitting
situations as intended.

3.3. Pseudo-tracklet Self-supervised Learning
As we have discussed, before splitting, the imperfect tracklet may contain observations of different identities, and directly incorporating them in retraining may introduce errors.
After splitting, Mix-up Errors are remarkably reduced in the
split tracklets, which allows us to use them as reliable pseudolabels for retraining. However, different tracklets, which haveno overlap in the temporal domain, may still hold the same
F.Yang et al.: Preprint submitted to Elsevier

Figure 6: The merging process. The Hierarchical Clustering
is accommodated to associate splitting tracklets into refined
tracklets. In the distance matrix , an element value is set
to be inf whenever two tracklts violate constraints. The cutting threshold is obtained from the statistical information of
splitting tracklets.

identities. Therefore, our sampling strategy suggests selecting temporally-overlapped tracklets within the same video.
We illustrate our data processing scheme of pseudo-tracklet
self-supervised learning in Figure 5. Within a video, we firstsample two observations in the same frame, and then form
triplets within their pseudo tracklets. In a mini-batch of input, the ratio between samples of labeled videos and unlabelled target videos is 1 ∶ 1. Other training details are thesame as we applied in intra-frame self-supervised learning.

3.4. Merging Process
The merging process is summarized in Figure 6. the split
tracklets association is formulated as a hierarchical clustering problem. Each element represents a tracklet and theirdistance matrix is defined as⎧ 𝑖𝑛𝑓 ,
⎪ 𝑖𝑛𝑓 ,⎪
𝑘,𝑙 = ⎨ 𝑖𝑛𝑓 ,⎪ 1⎪ 𝑁𝑘 𝑁𝑙⎩if NMAE(𝑇𝑘 , 𝑇𝑙 ) > 𝜃 𝑚𝑜𝑡𝑖𝑜𝑛 ,if 𝑘 = 𝑙,if Π𝑘 ∩ Π𝑙 ≠ ∅,(𝑓𝑖𝑘 𝑓𝑗𝑙 )∑∑𝑖∈Π𝑘𝑗∈Π𝑙 1 − ‖ 𝑘 ‖‖ 𝑙 ‖ , 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒,‖ 𝑓𝑖 ‖ ‖ 𝑓𝑗 ‖‖ ‖‖ ‖(5)

Page 5 of 10

ReMOT

Identical Object

Different Object

Identical Object

Fixing Cut-off Errors

Different Object

Fixing Mix-up Errors

Figure 7: Visualization of refinement results (more details and results are shown in our
supplemental materials). Red arrows indicate where to look at. Compared with preobtained results, ReMOT fixed Cut-off Errors and Mix-up Errors in refined results.

where respectively to tracklets 𝑇𝑘 and 𝑇𝑙 , 𝑘,𝑙 is the distance
between them; Π𝑘 and Π𝑙 are their corresponding frame sets,
𝑓𝑖𝑘 and 𝑓𝑗𝑙 are their appearance features at frame 𝑖 and 𝑗, and𝑁𝑘 and 𝑁𝑙 are the number of observations within a tracklets; 𝜃 𝑚𝑜𝑡𝑖𝑜𝑛 is the motion threshold obtained from train set
statistical information.
In Eq. 5, whenever the matching condition violates given
constraints, we set their distance value to be infinite. Basedon the distance matrix, we accommodate hierarchical clustering to merge the split tracklets. Nonetheless, the main
challenge of applying hierarchical clustering in the MOT
task is how to set the proper cutting threshold. We do notgive a heuristic value but let the data speak for themselves.
After pseudo-tracklet self-supervised learning, we supposethat intra-frame and intra-split-tracklet cosine similarity his𝑎𝑝𝑝
tograms can be separated at 𝜃𝑚𝑒𝑟𝑔𝑒. Without accessing the
ground-truth, such a threshold could be a reasonable boundary to distinguish objects based on appearance features. There𝑎𝑝𝑝
fore, we set 1 − 𝜃𝑚𝑒𝑟𝑔𝑒
as the cutting threshold in hierarchicalclustering.

F.Yang et al.: Preprint submitted to Elsevier

4. Experiments
We test ReMOT on the MOTChallenge evaluation system (MOT15-17 test), and then on our constructed dataset(MOT16 train & MOT15 train). To evaluate the effectiveness of reducing Mix-up Errors and Cut-off Errors, we use
metrics MOTA (MOT accuracy), IDF1 (ratio of correctly
identified detections), MT (mostly tracked targets), ML (mostly
lost targets), and ID Switches, which are defined in [17, 24].

4.1. Implementation Details
We implement the appearance encoder of ReMOT in Pytorch. Referring to the given bounding boxes, the cropped
image of each observation is resized to be 128×256 as the input of the appearance encoder. We applied a data augmentation strategy that includes Random horizontal flipping, random color jittering, and random affine transformation in the
training. For the appearance encoder optimization, we choose
the Adam optimizer [12] with a learning rate of 1 × 10−4 .
During the merging process, we applied the centroid linkage criteria to determine the distance between newly merged
tacklets in hierarchical clustering.

Page 6 of 10

ReMOT
Testing Datasets

Approach

MOTA(%)↑

IDF1(%)↑

# MT↑

# ML↓

# ID Sw.↓

MOT15 Test Set

Tube_TK [26] (CVPR 2020)
FairMOT [45] (ArXiv 2020)
Assembled Baseline ([26, 45])
Assembled Baseline + ReMOT (ours)

58.4
60.6
63.0
63.6 (+0.6)

53.1
64.7
65.9
67.0 (+1.1)

283
343
350
382 (+32)

130
79
103
96 (-7)

854
591
668
445 (-223)

Chained-Tracker [27] (ECCV 2020)
FairMOT [45] (ArXiv 2020)
Baseline (FUFET, paper is not available)
Baseline + ReMOT (ours)

67.6
69.3
76.5
76.9 (+0.4)

57.2
72.3
68.9
73.2 (+4.3)

250
306
401
390 (-11)

175
127
93
94 (+1)

1897
815
1026
742 (-284)

Chained-Tracker [27] (ECCV 2020)
FairMOT [45] (ArXiv 2020)
Baseline (FUFET, paper is not available)
Baseline + ReMOT (ours)

66.6
73.7
76.2
77.0 (+0.8)

57.4
72.3
68.0
72.0 (+4.0)

759
1017
1203
1218 (+15)

570
408
321
324 (+3)

5529
3303
3237
2853 (-384)

MOT16 Test Set

MOT17 Test Set

Table 1
Improvement on MOT15-17 test sets when ReMOT is applied to the state-of-the-art
baseline with private detection (up to 6th-Nov-2020), where ↑(↓) indicates that thelarger(smaller) the value is, the better the performance.
Testing Dataset

Approach

MOTA(%)↑

IDF1(%)↑

# MT↑

# ML↓

# ID Sw.↓

MOT15 Train Set w/ Oracle Detections

DeepSORT [37] (ICIP 2017)
DeepSORT + ReMOT (ours)

95.1
98.5 (+3.4)

86.5
91.7 (+5.2)

268
293 (+25)

25
9 (-16)162
50 (-112)

MOT15 Train Set w/ Oracle Detections

Tracktor [2] (ICCV 2019)
Tracktor + ReMOT (ours)

94.9
98.1 (+3.3)

85.3
89.5 (+4.2)

244
282 (+38)

26
11 (-15)126
56 (-70)

MOT15 Train Set w/ Oracle Detections

DAN [31] (TPAMI 2019)
DAN + ReMOT (ours)

96.3
98.7 (+2.4)

88.4
92.2 (+3.8)

303
305 (+2)

12
9 (-3)

156
53 (-103)

MOT15 Train Set w/ Oracle Detections

DeepMOT [40] (CVPR 2020)
DeepMOT + ReMOT (ours)

97.2
98.7 (+1.5)

89.2
92.1 (+2.9)306
308 (+2)

13
10 (-3)127
55 (-72)

Table 2
Improvement of applying ReMOT to other MOT results with oracle detections, where ↑(↓)
indicates that the larger(smaller) the value is, the better the performance.

4.2. Refinement on SOTA Results of MOT15-17
Test Sets with Private Detection
Due to the MOTChallenge evaluation policy, we could
not submit every refinement result to systemically evaluate
ReMOT. Nonetheless, a weak MOT result may be insufficient to verify the experimental upper bound of our method.
This motivates us to select the state-of-the-art MOT results(e.g., [26, 45]), or even assemble these results, to constructsuper-strong baselines. As such strong baselines already outperform all other MOT results (include online and offline), tosome extent, there is limited space to further improve themwithout modifying object detection, but we still observedthat ReMOT made improvements on them (see Table 1 andFigure 7).

4.3. Refinement on MOT15 Train Set with Oracle
Detections
In our target application, the MOT semi-automatic annotation, there is a situation that detection labels are annotatedbut waiting for being linked to tracklets. In reality, it is more
efficient to only annotate the bounding boxes as finding correspondences among large amounts of observations is usually a harder task. Therefore, it is worth exploring the effectiveness of ReMOT with oracle detections. To accomplish
this goal, we constructed a new experimental dataset: while
the training set is the entire MOT16 train set, the testing set isa part of MOT15 train set that has no overlap with MOT16,
which include KITTI-13, KITTI-17, TUD-Campus, TUDF.Yang et al.: Preprint submitted to Elsevier

Stadtmitte, and ETH-Sunnyday. Ground-truth detections areused in both training and testing.
With the oracle detections, plenty of MOT methods yield
good results with their extracted MOT features and data association models (Table 2). Applying ReMOT to refine theirresults further bridges the gap of estimated tacklets and oracle trcklets — with a MOTA value larger than 98.0%. It revealed that ReMOT can refine pre-obtained MOT results agnostic to MOT models, which ties well with the conclusion
on MOT15-17 testing sets. Therefore, for MOT annotation,
we can label detection and leverage ReMOT to generate satisfactory tracklet labels, which may partially release humans
from the tedious work.

4.4. Ablation Studies on Self-supervised Learning
To evaluate the effectiveness of our proposed self-supervised
learning (SSL) in refinement, we performed ablation studies
on MOT15 train set with oracle detections. The results arelisted in Table 3.
Using the results of DeepSort [37] as our baseline, thecomplete ReMOT that includes intra-frame and pseudo-tracklet
self-supervised learning can significantly improve the MOTA
value from 95.1% to 98.5%. However, without intra-frameand pseudo-tracklet self-supervised learning, applying thesame processing of ReMOT almost makes no difference tothe baseline. It provides a clear evidence for the conclusionthat it is our self-supervised learning scheme, other than the
ReID model itself, that improves the MOT results. MeanPage 7 of 10

ReMOT(a) Before Self-supervised
Learning(b) After Intra-frame
Self-supervised Learning(c) After Pseudo-tracklet
Self-supervised Learning

Figure 8: Normalized histograms of appearance cosine similarity on MOT16 test set. The
overall intra-frame histogram (different objects) move to smaller cosine similarity while theoverall intra-split-tracklet histogram (identical object) does the opposite. Jaccard index ofthem are progressively reduced, which implies the discrimination of appearance features is
improved.

Approach

MOTA(%)↑

Baseline [37] (DeepSort, ICIP 2017)95.1

Baseline [37] +ReMOT w/o intra-frame & pseudo-tracklet SSL

95.3

Baseline [37] +ReMOT w/o intra-frame SSL

95.5

Baseline [37] +ReMOT w/o pseudo-tracklet SSL

97.7

Baseline [37] + ReMOT

98.5

Table 3
Ablation studies on MOT15 train set with oracle detections.
We focus on exploring the effectiveness of self-supervised learning (SSL) in ReMOT.

while, it can be noticed that without using intra-frame SSL
impairs the performance more than without using pseudotracklet SSL. This could be attributed to that appearance features improved by intra-frame SSL is applied to split imperfect tracklets. Without using improved appearance featuresin the splitting process, more errors are likely to remain after the splitting, and consequently impair the pseudo-tracklet
SSL.

4.5. Improvement of Appearance Features
We attribute the capacity of refinement to improved appearance features, but how to verify the improvement of appearance features? We use Jaccard Index to quantitatively
measure the overlap of appearance affinity distributions for
intra-frame observations (i.e., doubtlessly different identities) and intra-split-tracklet observations (i.e., most-likely same
identities). Jaccard Index of two normalized histograms canbe formulated as()∑
𝜖 min 𝐻1 (𝜖), 𝐻2 (𝜖)𝐽 (𝐻1 , 𝐻2 ) = ∑(6)()
𝜖 max 𝐻1 (𝜖), 𝐻2 (𝜖)F.Yang et al.: Preprint submitted to Elsevier

where 𝐻1 and 𝐻2 are histograms of intra-frame and intrasplit-tracklet data pairs cosine similarity, respectively; 𝜖 isthe bin index in histogram.
Using MOT16 testing set, we illustrate the normalized
histograms of intra-frame and intra-split-tracklet cosine similarity in Figure 8. Ideally, when observation could be identified by the appearance features, the two histograms shouldhave no intersection at all. In the real case, Figure 8 (a)shows a notable overlap between the histogram of intra-frame
and intra-split-tracklet cosine similarity, which may lead to
Mix-up Errors and Cut-off Errors. Figure 8 (b) shows that
our intra-frame retraining contributes to reducing the overallcosine similarity of inter-identity observation, which helps
to correct the split tracklets that contains Mix-up Errors. Figure 8 (c) demonstrates that inter-split-tracklet retraining generates more discriminative appearance features. And, theoverall intra-frame histogram moves to smaller cosine similarity while the overall intra-split-tracklet histogram does
the opposite. Their Jaccard Index is progressively reduced,
which hints that the appearance feature discrimination is improved. This is an important finding in the understanding of
why ReMOT can refine an arbitrary MOT result.

4.6. The Change of Tracklets in ReMOTWe plot an example (see Fig.9) to help understand thesplitting and merging process. All tracklets of the same video
are represented in a table, whose axes denote Track IDs andFrame Sequence, respectively. When an observation is assigned to a specific track ID and frame, the corresponding
cell of the table is set to be value 1 (i.e., black points), whileother places are 0 ( i.e., white blank space). During the splitting process, new split-track IDs are assigned and the number of tracklets increases while the temporal length of split
tracklets decreases. During the merging process, the corresponding change is opposite to the splitting.

Page 8 of 10

ReMOT

split

merge

Figure 9: Visualization of tracklet changes in a single video sequence after the splitting
and merging processes. In the above tables, each observation is plotted as a black point.

5. Conclusion
We presented a model-agnostic Refining MOT framework (ReMOT) to refine arbitrary MOT results from the dataassociation perspective. ReMOT is the first model-agnostic
MOT refinement method. Its key idea is to split imperfect
tracklets and then associate the split tracklets with improved
appearance features. For the first time, we defined two typesof data association errors, as Mix-up Errors and Cut-off Errors, to clarify the refining goal in MOT task. To reducethese errors, we attempted to improve the discrimination of
appearance features between identical and different objects
by proposing intra-frame self-supervised learning and pseudotracklet self-supervised learning. Besides, we also carefully
designed mechanisms to obtain hyper-parameters from thestatistical information of given data rather than heuristicallyselecting them. Experiments demonstrate ReMOT can make
improvements on existing state-of-the-art MOT results without introducing extra-human works. Therefore, ReMOT may
considerably contribute to semi-automatic MOT data annotation and partially releases humans from the tedious work.
In future work, we may explore how to apply ReMOT to 3D
MOT tasks [36] and panoramic view MOT tasks [41] when
visual features are available.

ACKNOWLEDGEMENTS
This work was supported by JSPS KAKENHI Grant Number JP17H06101, and a MSRA Collaborative Research 2019
Grant Awarded by Microsoft Research Asia.

References[1] Andriluka, M., Iqbal, U., Insafutdinov, E., Pishchulin, L., Milan, A.,
Gall, J., Schiele, B., 2018. Posetrack: A benchmark for human pose
estimation and tracking, in: Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition, pp. 5167–5176.

F.Yang et al.: Preprint submitted to Elsevier[2] Bergmann, P., Meinhardt, T., Leal-Taixé, L., 2019. Tracking without bells and whistles. 2019 IEEE/CVF International Conference on
Computer Vision (ICCV) , 941–951.
[3] Betke, M., Hirsh, D.E., Bagchi, A., Hristov, N.I., Makris, N.C., Kunz,
T.H., 2007. Tracking large variable numbers of objects in clutter, in:2007 IEEE Conference on Computer Vision and Pattern Recognition,
IEEE. pp. 1–8.
[4] Bewley, A., Ge, Z., Ott, L., Ramos, F.T., Upcroft, B., 2016. Simple
online and realtime tracking. 2016 IEEE International Conference on
Image Processing (ICIP) , 3464–3468.
[5] Bochinski, E., Eiselein, V., Sikora, T., 2017. High-speed tracking-bydetection without using image information. 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance(AVSS) , 1–6.
[6] Chechik, G., Sharma, V., Shalit, U., Bengio, S., 2010. Large scale online learning of image similarity through ranking. Journal of Machine
Learning Research 11, 1109–1135.
[7] Chen, Z., Huang, S., Tao, D., 2018. Context refinement for object detection, in: The European Conference on Computer Vision (ECCV).
[8] Dubuisson, S., 2006. Recursive clustering for multiple object tracking, in: 2006 International Conference on Image Processing, IEEE.
pp. 2805–2808.
[9] Feng, W., Hu, Z., Wu, W., Yan, J., Ouyang, W., 2019. Multi-object
tracking with multiple cues and switcher-aware classification. arXiv
preprint arXiv:1901.06129 .
[10] Fu, K., Zhao, Q., Gu, I.Y., 2019. Refinet: A deep segmentation assisted refinement network for salient object detection. IEEE Transactions on Multimedia 21, 457–469.
[11] He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep residual learningfor image recognition, in: Proceedings of the IEEE conference on
computer vision and pattern recognition, pp. 770–778.
[12] Kingma, D.P., Ba, J., 2014. Adam: A method for stochastic optimization. URL: https://arxiv.org/abs/1412.6980, arXiv:1412.6980.
[13] Klein, M., 1967. A primal method for minimal cost flows with applications to the assignment and transportation problems. Management
Science 14, 205–220.
[14] Koller, D., Weber, J., Malik, J., 1994. Robust multiple car tracking
with occlusion reasoning, in: European Conference on Computer Vision, Springer. pp. 189–196.
[15] Kuhn, H.W., 1955. The hungarian method for the assignment problem. Naval research logistics quarterly 2, 83–97.
[16] Kuma, R., Weill, E., Aghdasi, F., Sriram, P., 2019. Vehicle reidentification: an efficient baseline using triplet embedding, in: 2019

Page 9 of 10

ReMOTInternational Joint Conference on Neural Networks (IJCNN), IEEE.
pp. 1–9.
[17] Leal-Taixé, L., Milan, A., Reid, I., Roth, S., Schindler, K., 2015.
MOTChallenge 2015: Towards a benchmark for multi-target tracking. arXiv:1504.01942 [cs] URL: http://arxiv.org/abs/1504.01942.
[18] Li, M., Zhu, X., Gong, S., 2019a. Unsupervised tracklet person reidentification. IEEE transactions on pattern analysis and machine intelligence .
[19] Li, X., Wang, K., Wang, W., Li, Y., 2010. A multiple object tracking
method using kalman filter. The 2010 IEEE International Conferenceon Information and Automation , 1862–1866.
[20] Li, Y., Yuan, L., Vasconcelos, N., 2019b. Bidirectional learning fordomain adaptation of semantic segmentation, in: Proceedings of theIEEE Conference on Computer Vision and Pattern Recognition, pp.
6936–6945.
[21] Lu, W.L., Ting, J.A., Little, J.J., Murphy, K.P., 2013. Learning to track
and identify players from broadcast sports videos. IEEE transactions
on pattern analysis and machine intelligence 35, 1704–1716.
[22] Luo, H., Gu, Y., Liao, X., Lai, S., Jiang, W., 2019. Bag of tricks anda strong baseline for deep person re-identification, in: Proceedings ofthe IEEE Conference on Computer Vision and Pattern Recognition
Workshops, pp. 0–0.
[23] Luo, W., Xing, J., Milan, A., Zhang, X., Liu, W., Zhao, X., Kim, T.K.,
2014. Multiple object tracking: A literature review. arXiv preprint
arXiv:1409.7618 .
[24] Milan, A., Leal-Taixé, L., Reid, I., Roth, S., Schindler, K., 2016.
Mot16: A benchmark for multi-object tracking. arXiv preprint
arXiv:1603.00831 .
[25] Moon, G., Chang, J.Y., Lee, K.M., 2019. Posefix: Model-agnostic
general human pose refinement network, in: Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition, pp. 7773–
7781.
[26] Pang, B., Li, Y., Zhang, Y., Li, M., Lu, C., 2020. Tubetk: Adopting
tubes to track multi-object in a one-step training model, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
Recognition, pp. 6308–6318.
[27] Peng, J., Wang, C., Wan, F., Wu, Y., Wang, Y., Tai, Y., Wang, C., Li,
J., Huang, F., Fu, Y., 2020. Chained-tracker: Chaining paired attentive regression results for end-to-end joint multiple-object detection
and tracking, in: Proceedings of the European Conference on Computer Vision.
[28] Ristani, E., Tomasi, C., 2018. Features for multi-target multi-camera
tracking and re-identification, in: Proceedings of the IEEE conferenceon computer vision and pattern recognition, pp. 6036–6046.
[29] Segen, J., 1996. A camera-based system for tracking people in realtime, in: Proceedings of 13th International Conference on Pattern
Recognition, IEEE. pp. 63–67.
[30] Sibson, R., 1973. Slink: an optimally efficient algorithm for the
single-link cluster method. The computer journal 16, 30–34.
[31] Sun, S., Akhtar, N., Song, H., Mian, A.S., Shah, M., 2019. Deep
affinity network for multiple object tracking. IEEE transactions on
pattern analysis and machine intelligence .
[32] Tang, S., Andres, B., Andriluka, M., Schiele, B., 2016. Multi-person
tracking by multicuts and deep matching, in: ECCV Workshop on
Benchmarking Mutliple Object Tracking.
[33] Tang, S., Andriluka, M., Andres, B., Schiele, B., 2017. Multiple people tracking by lifted multicut and person re-identification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition, pp. 3539–3548.
[34] Voigtlaender, P., Krause, M., Os̆ep, A., Luiten, J., Sekar, B.B.G.,
Geiger, A., Leibe, B., 2019. MOTS: Multi-object tracking and segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
[35] Wang, X., Gupta, A., 2015. Unsupervised learning of visual representations using videos, in: Proceedings of the IEEE International
Conference on Computer Vision, pp. 2794–2802.
[36] Weng, X., Wang, J., Held, D., Kitani, K., 2020. 3d multi-object tracking: A baseline and new evaluation metrics, in: IEEE/RSJ Interna-

F.Yang et al.: Preprint submitted to Elsevier

tional Conference on Intelligent Robots and Systems, IROS, pp. 1–8.
[37] Wojke, N., Bewley, A., Paulus, D., 2017. Simple online and realtime
tracking with a deep association metric. 2017 IEEE International Conference on Image Processing (ICIP) , 3645–3649.
[38] Wu, G., Zhu, X., Gong, S., 2020. Tracklet self-supervised learningfor unsupervised person re-identification, in: AAAI Conference on
Artificial Intelligence.
[39] Wu, Y., Lin, Y., Dong, X., Yan, Y., Ouyang, W., Yang, Y., 2018.
Exploit the unknown gradually: One-shot video-based person reidentification by stepwise learning, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5177–5186.
[40] Xu, Y., Osep, A., Ban, Y., Horaud, R., Leal-Taixé, L., AlamedaPineda, X., 2020. How to train your deep multi-object tracker, in:Proceedings of the IEEE/CVF Conference on Computer Vision and
Pattern Recognition, pp. 6787–6796.
[41] Yang, F., Li, F., Wu, Y., Sakti, S., Nakamura, S., 2020. Using
panoramic videos for multi-person localization and tracking in a 3d
panoramic coordinate, in: ICASSP 2020-2020 IEEE International
Conference on Acoustics, Speech and Signal Processing (ICASSP),
IEEE. pp. 1863–1867.
[42] Yu, F., Li, W., Li, Q., Liu, Y., Shi, X., Yan, J., 2016. Poi: Multiple object tracking with high performance detection and appearance feature,
in: ECCV Workshops.
[43] Zhang, L., Li, Y., Nevatia, R., 2008. Global data association for multiobject tracking using network flows, in: 2008 IEEE Conference on
Computer Vision and Pattern Recognition, IEEE. pp. 1–8.
[44] Zhang, S., Wang, J., Wang, Z., Gong, Y., Liu, Y., 2015. Multi-target
tracking by learning local-to-global trajectory models. Pattern Recognition 48, 580–590.
[45] Zhang, Y., Wang, C., Wang, X., Zeng, W., Liu, W., 2020. A simple
baseline for multi-object tracking. arXiv preprint arXiv:2004.01888 .

Page 10 of 10