分析：Introduction
Visual tracking aims to uniquely identify and track an object
within a video sequence by using arbitrary target queries. In
the visual world, objects rarely exist in isolation but rather
within a larger and dynamic context. Therefore, visual perception is a complex process that involves interpreting and
understanding the surrounding environment of an object. In
such a situation, equipping a model with the ability to perform online contextual reasoning and establish associations
presents a challenge in the field of visual tracking.
Despite this challenge, a significant number of current
tracking methods overlook this problem and instead rely
on the offline image-pair matching to localize instances in
the current frame. As shown in Fig.1(a), these offline methods(Bertinetto et al. 2016; Li et al. 2019; Chen et al. 2021; Yan et al. 2021a; Ye et al. 2022; Cui et al. 2022) typically
follow a three-phase process: (i) extracting features by sampling two video frames (i.e., reference and search frames);
(ii) propagating the initial target information from the reference to the search frame through a matching/fusion module;
and (iii) utilizing a bounding box prediction head to output
the localization results. Most trackers have performed well
under this paradigm, but still exhibit the following drawbacks: (1) The sampling frames are sparse (i.e., using only
one reference frame and one search frame). Although visual tracking inherently contains rich temporal data, this
simple sampling strategy falls short in accurately representing the motion state of an object, posing a significant challenge for trackers to comprehend dynamic video content,
and (2) The target information is matched offline and limited to image-pair level, preventing the association of the
targets across video frames. Traditional feature matching/-
fusion methods(Chen et al. 2020; Zhang et al. 2020; Guo
et al. 2021; Xie et al. 2022) focus on the appearance similar ity of object, without considering the property that tracking
instance rely on continuous cross-frame associations.
To incorporate temporal information into the model, some
approaches commonly design online updating techniques,
such as updating templates(Yan et al. 2021a; Cui et al. 2022)
and updating model parameters(Bhat et al. 2019). Despite
being successful, these methods still rely on sparse sampling
frames (i.e., reference, search, and update frames) and do
not effectively explore how information is propagated online across search frames. This inspired us to think: can our
visual tracking algorithm densely associate and perceive an
object in a video streaming context?
The answer is affirmative. Unlike conventional approaches that rely on offline image-pair matching with
sparse sampling frames, this paper proposes ODTrack, a
novel video-level framework for visual tracking that capitalizes on video stream modeling. Specifically, we reformulate object tracking as a token sequence propagation task
that densely associates the contextual relationships of across
video frames in an auto-regressive manner, as shown in
Fig.1(b). To overcome the limitations of traditional imagepair sampling strategy and explore the rich temporal dependencies, we extend the model’s input from image-pair to the
level of a video stream. Under this new input paradigm, we
design two simple yet effective temporal token propagation
attention mechanism that captures the spatio-temporal trajectory relationships of the target instance using an online
token propagation manner, thus allowing the processing of
video-level inputs of arbitrary length. Notably, we treat each
video sequence as a continuous sentence, enabling us to employ language modeling for a comprehensive contextual understanding of the video content. This novel approach significantly distinguishes our tracker from traditional methods
(Yan et al. 2021a; Ye et al. 2022; Cui et al. 2022) and greatly
strengthens its ability to understand the spatio-temporal trajectory of target instance.
The main contributions of this work are as follows.
• We propose a novel video-level tracking pipeline, called
ODTrack. In contrast to existing tracking approaches
based on sparse temporal modeling, we employ a token
sequence propagation paradigm to densely associate contextual relationships across video frames.
• We introduce two temporal token propagation attention
mechanisms that compress the discriminative features of
the target into a token sequence. This token sequence
serves as a prompt to guide the inference of future
frames, thus avoiding complex online update strategies.
• Our approach achieves a new state-of-the-art tracking
results on seven visual tracking benchmarks, including
LaSOT, TrackingNet, GOT10K, LaSOText, VOT2020,
TNL2K, and OTB100. 和 Related Work
Traditional Tracking Framework. The current popular
trackers(Bertinetto et al. 2016; Li et al. 2019; Chen et al.
2021; Ye et al. 2022) are dominated by the Siamese tracking
paradigm, which achieves tracking by image-pair matching.
To improve the accuracy and robustness of trackers, several different approaches are proposed, such as prediction
head networks (Li et al. 2018; Chen et al. 2020; Zhang et al.
2020), cross-correlation modules (Han et al. 2021; Liao et al.
2020; Chen et al. 2021), powerful backbone (Chen et al.
2022; Cui et al. 2022) and attention mechanisms (Guo et al.
2021; Yu et al. 2020). In recent years, the introduction of
the transformer (Vaswani et al. 2017) enables trackers (Yan
et al. 2021a; Xie et al. 2022; Cui et al. 2022; Ye et al. 2022)
to explore more powerful and deeper feature interactions, resulting in significant advances in tracking algorithm development. However, most of these methods are designed based
on offline mode and sparse image-pair strategy. With this
design paradigm, the tracker struggles to accurately comprehend the object’s motion state in the temporal dimension
and can only resort to traditional Siamese similarity for appearance modeling. In contrast to these approaches, we reformulate object tracking as a token sequence propagation
task and aim to extend Siamese tracker to efficiently exploit
target temporal information in an auto-regressive manner.
Temporal Modelling in Visual Tracking. Multi-object
tracking algorithms(Meinhardt et al. 2022; Zeng et al. 2022)
typically involve the recognition and association of individual objects in a video, making the study of trajectory information a common practice. However, there exists a relatively limited amount of research exploring the utilization
of spatio-temporal trajectory information in single-object
tracking algorithms.
To explore temporal cues within the Siamese framework,
several online update methods are carefully designed. UpdateNet(Zhang et al. 2019) introduces an adaptive updating
strategy, which utilizes a custom network to fuse accumulated templates and generate a weighted updated template
feature for visual tracking. DCF-based trackers(Danelljan
et al. 2019; Bhat et al. 2019; Danelljan, Gool, and Timofte
2020) excel at updating model parameters online using sophisticated optimization techniques, thereby improving the
robustness of the tracker. STMTrack(Fu et al. 2021) and
TrDiMP(Wang et al. 2021a) employ attention mechanism to
effectively extract contextual information along the temporal
dimension. STARK(Yan et al. 2021a) and Mixformer(Cui
et al. 2022) specifically design target quality branch for updating template frame, which aids in improving the tracking
results. Recently, there has been a gradual surge in research
attention towards modeling temporal context from various
perspectives. TCTrack (Cao et al. 2022) introduces an online temporal adaptive convolution and an adaptive temporal
transformer that aggregates temporal contexts at two levels
containing feature extraction and similarity map refinement.
VideoTrack (Xie et al. 2023) designs a new tracker based on
video transformer and uses a simple feedforward network to
encode temporal dependencies. ARTrack (Xing et al. 2023)
presents a new time-autoregressive tracker that estimates the
coordinate sequence of an object progressively.
Nevertheless, the above tracking algorithms still suffer
from the following limitations: (1) The optimization process
is complex, involving the design of specialized loss functions(Bhat et al. 2019), multi-stage training strategies(Yan et al. 2021a), and manual update rules(Yan et al. 2021a),
and (2) Although they explore temporal information to some
extent, they fail to investigate how temporal cues propagate
across search frames. In this work, we introduce a new dense
context propagation mechanism from a token propagation
perspective, which offers a solution to circumvent intricate
optimization processes and training strategies. Further, we
propose a new baseline approach, called ODTrack, focused
on unlocking the potential of temporal modeling through the
propagation of target motion/trajectory information. 在内容和和含义上有何不同？回答完问题之后，将讲述内容翻译为中文。