FARTrack: Fast Autoregressive Visual Tracking with High Performance
Authors: Guijie Wang
Tong Lin
Yifan Bai
Anjia Cao
Shiyi Liang
Wangbo Zhao
Xing Wei
Venue: ICLR 2026 Poster (2026)
Keywords: Autoregressive Tracking
Efficient Tracking
Visual Object Tracking
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Inference speed and tracking performance are two critical evaluation metrics in the field of visual tracking. However, high-performance trackers often suffer from slow processing speeds, making them impractical for deployment on resource-constrained devices. To alleviate this issue, we propose $\textbf{FARTrack}$, a $\textbf{F}$ast $\textbf{A}$uto-$\textbf{R}$egressive $\textbf{T}$racking framework. Since autoregression emphasizes the temporal nature of the trajectory sequence, it can maintain high performance while achieving efficient execution across various devices. FARTrack introduces $\textbf{Task-Specific Self-Distillation}$ and $\textbf{Inter-frame Autoregressive Sparsification}$, designed from the perspectives of $\textbf{shallow-yet-accurate distillation}$ and $\textbf{redundant-to-essential token optimization}$, respectively. Task-Specific Self-Distillation achieves model compression by distilling task-specific tokens layer by layer, enhancing the model's inference speed while avoiding suboptimal manual teacher-student layer pairs assignments. Meanwhile, Inter-frame Autoregressive Sparsification sequentially condenses multiple templates, avoiding additional runtime overhead while learning a temporally-global optimal sparsification strategy. FARTrack demonstrates outstanding speed and competitive performance. It delivers an AO of 70.6\% on GOT-10k in real-time. Beyond, our fastest model achieves a speed of 343 FPS on the GPU and 121 FPS on the CPU. Source code is available at: https://github.com/MIV-XJTU/FARTrack.git

Published as a conference paper at ICLR 2026
FARTRACK: FAST AUTOREGRESSIVE VISUAL TRACK- ING WITH HIGH PERFORMANCE
Guijie Wang Tong Lin 122-, Yifan " Cao !2 Shiyi Liang!.? Wangbo Zhao", Xing Wei School of Software Engineering, Xn'an Jiaotong University
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi'an Jiaotong University DAMO Academy, Alibaba Group *National University of Singapore
(gui jiewang, lintongl 220, cacanjia7, sy ay_liang2023 @stu. x xjtu edu. cn ba baiyi fan. fō baw.bbbbcccc. com wangbo zhao 9 gm@mmm i com weixingemail wwtpaaiiiiji.ueuu....e
ABSTRACT
Inference speed and tracking performance are two critical evaluation metrics in the field of visual tracking. However, high-performance trackers often suf- fer from slow processing speeds. making them impractical for deployment on resource-constrained devices. To alleviate this issue, we propose FARTrack, a Fast Auto- RegreReereesiv Tracking framework, Since autoregression emphasizes the temporal nature of the trajectory sequence, it can maintain high performance while achieving efficient execution across various devices. FARTrack introduces Task- Specific Sill-Distillatinn and Inter-frame Autoregressive Sparsification, de- signed from the perspectives of shallow-yet--ccerrad distillation and redundant- to-essential token optimization. respectively. Task- -Seccffc Self-Distillation achieves model compression by distilling task-specific tokens layer by layer, hancing the model's inference speed while avoiding suboptimal manual teacher- student layer pairs assignments. Meanwhile, Inter-frame Autoregressive Sparsi- fication sequentially condenses multiple templates, avoiding additional runtime overhead while learning temporally-global optimal sparsification strategy. FAR- Track demonstrates outstanding speed and competitive performance. It delivers Beyond, our fastest model achieves a speed of 343 FPS on the GPU and 121 FPS on the CPU. Source code is available at: / thub. www.MMV-///////.............. WUTUVVVXXUUURRRTTTTA
an AO of 70.6% on GOT- 10k
INTRODUCTION
Visual object tracking (VOT), aiming to continuously localize arbitrary objects in video sequence, re- lies on the continuous positions of the objects and is highly sensitive to temporal information Asanomi al. (2023): Mayer et al. (2022); Wu et al. (2023b); Zhao al.) (2023): Zhou et al. (2023a). In
practical applications speed and performance simultaneously Yang et al. (2026); Zeng et al. (2025b a); Qi et al. (026).
edge devices with limited resources, it is often necessary to consider both
However, existing methods only achieve either high speed Gopal & Amer (2024) Li et al. 2023
Xie et al. (2023): Yang et al. (202bb) Zaveri et al. (2025) or high performance Cai et al. (2024
Chen
al. (2023) Hong et al. (2024): Xie et al. (2024) Yang al. (2023a).
To address this dilemma, existing efforts to balance the tracking speed and performance can be broadly categorized into two approaches:
(i) Model distillation methods Cui et al. (2023): Guo et al. (002)): Hinton et al. (2015): Li et al.
(2017); Roomero 2oomero et al. (2014) based cross-layer train a student model to mimic teacher's 's vision features. However, as shown in Figure a), these methods rely on hand-crafted layer assignments to enable knowledge transfer Ahn et al. 2019); Passalis et al. (200);; Romero et al. (2014); Tung
& Mori 2019); Yue et al. (2000): Zagoruyko & Komodakis (016)). Without prior knowledge of
"Equal contribution. 'Corresponding author. Project Lead

---

Published as a conference paper at ICLR 2026
Hand-Crahod Layer Assignments
Self-Aasignments
FARTrackury
FARTracK
Teacher
Student
PrompIt
Average Overlap (AO)
TCCrack
JaymiTaak-
65
FARTracKioo
AsymTrackT xasterr
LighTrack
5mal
60
S Vision Feature
Task-Spodflcc Tokons
E.T.Tack
Perf.
2.8%
<- 0.7%).
55
Classical Cross-Layer Distillation
Task-Specific Self-Distillation
HiFTlny
(a) Comparison of distillation methods
50
Autoregressive
50
100
150
200
250
300
350
400
Tracking Speed GPU (FPS)
Templates
Templates
Figure 2: FARTrack vs. Other Trackers: Performance-Speed Trade-off. Comparison of our FARTrack with the state-of-the-an trackers GOT- 10k in terms of tracking speed (horizon- tal axis) on GPU and AO performence (vertical axis). The diameter of the circle is proportional to the ratio of the model's speed to its perfor- mance. FARTrackascs significantly surpasses ex- isting trackers in both tracking performance and inference speed.
1
(b) Inter-frame Autoregressive Sparsilication
Figure 1: Overview. (a) Comparison of our Task- Specific Self-Distillation and Classical Cross- Layer Distillation. (b) Inter-frame Autoregressive Sparsification for Multi-templates.
teacher-student layer pair assignment, manually designed ones often disrupt the hierarchical structure of feature extraction, thereby failing achieve optimal results. Morcover, the distillation objectives of these methods focus on current-frad frame visual features, overlooking temporal information in trajectory sequences critical for tracking tasks.
(ii) Runtime token sparsification approaches Chen et al. 2022b) Liang et al. (2022) Rao et al. (2021 Ye al. (2022) typically involve the gradual removal a subset of tokens during inference. However, this process introduces extra computational overhead for identifying tokens to remove, ultimately reducing tracking efficiency. Moreover, as these methods prioritize the current frame rather than the entire frame sequence, they fail to achieve a temporally-glohh optimal solution, which adversely impacts overall tracking performance.
To address these two issues, we present fast, hegh-performance multi-template autoregressive frame- work for visual tracking, using multi-template design to boost accuracy. Our framework comprises two key components: (i) Task-Specific Self-Distillation. Unlike classical cross-layer distillation, approach conducts
distillation of task-specific tokens, which represent the object's s trajectory sequences. In this method, each layer acts both student and teacher for the next, trained to fit teacher's trajectory sequence features via KI. divergence. Our approach avoids suboptimal manual layer assignments while maintaining temporal information. (ii) Inter-frame Autoregressive Sparsification. Compared with the frame-wise runtime sparsification methods, our sparsification is sequence-level method for template sequences. We treat attention weights matrices retain fore- ground tokens while discarding background tokens, and propagate sparsification results to subsequent frames in an autoregressive manner. In this process, we reduce the bandwidth without introducing extra computational load, while retaining temporal information to learn remporally-glohte optimal sparsification strategy.
Overall, we present FARTrack, a fast and high-performance tracking framework. Extensive experi- ments demonstrate the effectiveness and efficiency of our approach. Our method achieves better balance between inference speed and tracking performance than previous trackers. Specifically, as demonstrated in Figure compared to the high-performanee tracker Asym Track-B, FARTrackung attains 2.9% higher AO score the GOT- 10k benchmark while achieving comparable running speed on the GPU. Moreover, FARTrackpico delivers 0.5% better performance than AsymTrack- T on GOT- 10k, along with superior GPU (343 FPS) and CPU (121 FPS) speeds.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Efficient Tracking Framework, practical application scenarios, is imperative to deploy trackers that achieve both high speed Cai et al. (2023); : Kou et al. (2023)
;: [i et al. (2023): Wei et al. (2024);
Zhang et al. (2023): ;: Zhou et al. (2023b); Zhang ct al. (2026) and high performance Gao et al. (2023) ;
Li et al. (2023); Shi et al. (2024): : Tang et al. (024): Wu et al. (2023a); Zheng et al. (2024) on
edge devices. Over the past decade, researchers have been exploring efficient
and effective tracking framework for real-worke applications Bhat et al. (219); Cao et al. (2022);
Danelljan et al. (2019) 2017|; Sun et al. (2025); Xie et al. (2022
trackers |ertinetto al. (2016): He al. (2023) Li et al. (2019 x Xu et al. (2020). While Stamese
2018); [Shen al. (2022); : Tang & Ling (202)): Xing et al. (2022) ; Zaveri et al. (2025) with lightweight designs Yan et al. (202 or
dynamic updates Borsuk et (2022) reduce computation, they often overlook temporal dependencies.
limiting performance. Transformer-based methods Blatter et al. p: Chen et al. (2023); Gao
al. (0022 2023); Kang et al. (2023); Lin et al. (2022); SSng et al. (2023 2022);] Ye et al. (2022);
Zhang et al. (2022) improve accuracy but add complexity through decoding heads. Recent generative
paradigms Bai et al. (2024); Chen et al. (2023); Wei et al. (2023a) eliminate custom heads yet incur high computational costs. Existing frameworks thus face trade-offs between efficiency and
performance. In this paper, we propose a more efficient generative tracking framework to better balance speed and performance.
Model Distillation. Model distillation Ahn et al. (2019); : Cui et al. (2023) ; Shen et al. (2021) 1 Tung & Mori 2019); WW et al. (2024); Ma et al. (2025); Cao al. (025); Du et al. (2025) transfers knowledge from a teacher to a lightweight student. Typical methods like AVTra Wu et al. (2024) and MixformeVV2 Cui et al. (2023) use
maximization or layer skipping. However, such cross-layer distillation Wang et al. (024)); Zhang et al. (2023) often relies on suboptimal manual layer associations, leading to notable performance drops. Our approach compresses the model via
self-distillation on task-specific tokens between adjacent layers, avoiding manual pair assignments and preserving temporal information.
Token Sparsification. Existing methods Chen et al.
2022b); Liang et al. (2022) Rao et al. (222)
Ye al. (2022); Zhao et al. (2024b)a) a 2025a b) 2026 reduce computation by progressively removing less important tokens during runtime. Dynamic VTT Rao et al. (2021) employs lightweight predictors for stepwise token pruning. while OSTrack Ye et al. (2022) removes background regions early in processing. However, such runtime approaches often introduce extra steps, increasing latency, and focus only on the current frame. We propose a sequence-level post-processing sparsification method that avoids additional runtime overhead, improves speed, and maintains high performance.
3 METHOD
3.1 REVISITING ARTRACK
ARTrack Wei et al. (203a)) is end-to-end sequence generation framework for visual tracking, which represents object trajectories as discrete token sequences using a shared vocabulary. By quantizing discrete token items, we obtain the coordinates corresponding to each token, thereby enabling the model to depict object positions via discrete tokens. The framework then employs a Transformer Encoder to extract visual information and progressively model the sequential evolution of the trajectory prompted by the preceding coordinate tokens. ARTrack V2 Bai et al. (2024) adds a dynamic appearance reconstruction process on the basis of its predecessor. It models the trajectory while reconstructing the appearance in an autoregressive manner.
Motivation. Although the ARTrack series models maintain temporal information retention, their architectures incorporating excessive depth and numerous parameters exhibit bandwidth- bandwidth-uuffriendly characteristics, ultimately reducing tracking efficiency. Conventional optimization methods, such as cross-layer distillation and runtime token sparsification, have been employed to mitigate these structural bottlenecks. However, cross-layer distillation relying on hand-crafted layer assignments disrupts the hierarchical structure of feature extraction and runtime token sparsification introduces extra computational overhead while neglecting temporal-globe optimization within frame sequences. To address these limitations, we propose FARTrack, which reduces model depth via task-specific delf-distillation for model compression and introduces an inter- frame autoregressive sparsification method to eliminate background redundancy and noise in the template images.
3

---

Published as a conference paper at ICLR 2026
Task-Specific Self-Distilattin
Dequartiize
forms E Encoder
Y
Search (S) Templates (2) Inter-frame Autoregressive
Visto
Task-speriic Tocens
S Sparsffication
Token Retention Rado 75%
cmul
0.9
0,
0.3
Search
Hill
Output
Att Map
Invalld
Multi-tmplate
Template
Task-specific Tokens
Command [c] Templates [X1
Figure 3: FARTrack Framework. FARTrack is fast, high-peeffrmaace
autore-
gressive framework, comprising two key components: Task-Specific compression and Inter-frame Autoregressive Sparsification for template sequences.
for model
3.2 NETWORK ARCHITECTURE
As presented Figure the Transformer Encoder Dosovitskiy et al. (2020); He et al. (2022) encodes features and predicts the four coordinate tokens of the bounding box inter-frame autoregressive manner. Initially, all templates and the search image are divided into patches, flattened, and projected into sequence of token embeddings. Subsequently, FARTrack maps the object positions across frames to a unified coordinate system with a shared vocabulary Chen et al.) (202T] (022a); Wei et al. (2023b). forming object trajectory tokens. Then, we concatenate the visual tokens, trajectory tokens, and four command tokens (representing the target bounding box coordinates), and input them into the Transformer Encoder. Finally, the Transformer encoder models the trajectory evolution in an autoregressive manner by leveraging the preceding trajectory tokens.
Multi-templates. To enhance tracking accuracy, we employ multi-templaae design, further sup- ported by linear update strategy. To prevent the potential loss of temporal information caused by occlusion or disappearance of the target during the tracking process, we ensure that the updated multi-templates always include the first frame and the preceding frame.
3.3 TASK-SPECIFE
Knowledge distillation-based model compression reduces the model size, thereby improving tracking efficiency. However, current gross-layer distillation methods rely on suboptimal manual layer assignments (Ahn et al. (2019);[ Cui et al. (2023); Shen et al.] (2021); Tung & Mori (2019); Wu et al. (2024), disrupting hierarchical feature extraction and causing accuracy degradation shallow models, while also neglecting temporal information in trajectory sequences.
Tung
To address this issue, we propose a simple yet effective model compression method known as task-specific self-distillation, as depicted in Figure In our method, one model layer serves as the student layer and the corresponding next layer acts as the teacher layer, establishing layer-wise self-distillation Hou et al. (2024); Zhang al. (2021 2019) that inherently circumvents suboptimal manual layer assignments. Furthermore, our method operates on task-specifer tokens which represent the object's trajectory sequences. The student layer is trained to fit the trajectory sequence features of the teacher layer by minimizing the KI. divergence. Therefore, the temporal information in the trajectory sequences propagates backward among the layers, enabling the model be distilled to shallow level while maintaining accuracy. Ultimately, our method improves the tracking speed of the model while maintaining the tracking performance.
3.4 INTER-FRAME AUTOREGRESSIVE SPARSIFICATION
The template image contains target object features alongside persistent background and noise in- terference that reduces tracking efficiency. Traditional sparsification methods Chen et al, (2022b); Liang al. (2022): Rao al. (2021): Ye al. (022) prioritize runtime token elimination, but
4

---

Published as a conference paper at ICLR 2026
these approaches introduce redundant time overhead and frame- ceene--pecific optimization rather than sequence-leve processing, resulting in slower speed and lower performance.
To eliminate template redundancy and noise in the tracking process while achieving a temporally- global optimal sparsification strategy, we propose an inter-frame autoregressive sparsification, as After processing through the attention layers, obtain the attention weights for template tokens with respect to search tokens and the four command tokens based on the attention map. This sequential processing method considers the template's 's correlation with both the search area and the predicted coordinates, respectively. We then simply add these two attention weights and retain the parts with the largest values based on the predefined token retention rate. This process leverages the inter-frame frame correlations to mask the unnecessary background parts in the templates while retaining the key foreground parts. Subsequently, the sparsification results of the current frame are saved and propagated to subsequent frames in an autoregressive manner, thereby learning a temporally-gl -ggooall optimal sparsification strategy. Overall, our method requires no additional time-consuming processes and meanwhile preserves the temporal information, achieving faster speed and better performance.
visualized in Figure 3]
Notably, masked tokens are excluded from processing to avoid distortion of normalization statistics. LayerNorm is exclusively applied to valid tokens, preventing improper scaling and shifting caused by statistical deviations that could undermine both inference stability and model performance.
3.5 TRAINING AND INFERENCE
FARTrack is fast tracker that enhances tracking efficiency by reducing model depth via task- specific self-distillation and eliminating redundant and noisy information from the templates using an inter-frame autoregressive sparsification.
Training. Similar to its predecessor, FARTrack undergoes both frame-level and sequence-level training Bai et al. (2024); Kim et al. 2022): Wei et al.] (2023a); [iiagg al. (2025). Initially, task-specific self-distillation is employed to progressively transfer trajectory sequence features from deeper layers to shallower layers, thereby reducing the model depth and achieving model compression. To ensure effective knowledge transfer, we minimize the KI. divergence between the teacher and student layers during training. Building upon this model compression, we introduce an inter-frame autoregressive sparsification that computes mask matrix based on inter-frame correlations and removes unimportant template background tokens according to a predefined token retention ratio.
Moreover, we incorporate the SloU loss Gevorgyan (2022) to better capture the spatial correlation between the predicted and ground truth bounding boxcs. For each video clip, the initial trajectory prompt is initialized using the object bounding box from the first frame and is propagated into subsequent frames in an autoregressive manner. The overall loss function is formulated as:
f = LCr As + XLLLI.
where LCE, LSwo and LKL denote the cross- cooss-yntrppy loss, SloU loss and KL divergence, respectively. The À values are used as weights to balance the contribution of each loss term.
Inference. During inference, the trajectory is initialized using the object bounding box in the first frame. The inter-frame sparsification method removes redundant and noisy tokens from templates, retaining and propagating these sparsification results through subsequent tracking processes to reduce computational complexity. The trajectory tokens are iteratively propagated into subsequent frames in an autoregressive manner. Unlike the training phase, since sparsification has already been performed on the templates, LayerNorm is applied to all tokens as usual.
4 EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
The model is trained with 8 NVIDIA RTX A6000 GPUs. The inference speed is evaluated with NVIDIA TiTan Xp, Intel(R) Xeon(R) Gold 6230R CPU @ 3.00GHz, and Ascend 310B.
Model Variants. We trained five variants of FARTrack with different configurations as Table 2
5

---

Published as a conference paper at ICLR 2026
Table 1: State-of-ther thrt comparison on GOT-1OK Huang et al. 2019 TrackingNet Muller al.
(2018) and LaSOT Fan et al. 2019). Best in bold, second best underlined.
GPU
CPU
NPU
GOT-1Ok
TrackingNet
LasOT
Methods
FPS
FPS
AD(S) (%) (R))(%) AUC(5)
Px (8) P(
AUC(%))
Narm (%) P(%)
DiMP Bhat et al.
2019
68
22
74.0
80.1
70.6
56.9
65.0
56.7
SiamFC+
Xu ct
28
75.4
68.7
54.4
62.3
54.7
LighTTrack
Yan et al.
2021b
59
27
61.1
71.0
54.3
72.5
77.8
69.5
53.8
60,5
53.7
TCTrack Cao et
2022
42
29
66.2
75.6
61.0
74.8
79.6
73.3
60.5
69.3
62.4
FEAR
Borsuk et
2022
123
28
61.9
72.2
52.5
70.2
80.8
71.5
53.5
59.7
54.5
F.T. Track
Blatter et
2023
16
56.6
64.6
42.5
72.5
77.8
69.5
59.1
66.8
60.1
HiT-Tiny
Kang
2023
135
42
52.6
59.3
42.7
74.6
78.1
68.8
54.8
60.5
$2.9
HIT Small
Kang et
2023
35
62.6
71.2
54.4
77.7
81.9
73.1
60.5
68.3
61.5
HiT-Base
Kang
2023
116
30
64.0
72.1
58.1
80.0
84.4
77.3
64.6
73.3
68.1
MixformerV V2
Cui ct al
2023
133
31
35
61.9
71.7
51.3
75.8
81.1
70.4
60.6
69.9
60.4
LiteTrack-BB
29
79.9
62.5
PromptVI VT Zhang et al.
2024
104
30
68.2
79.3
61.8
78
83.5
74.4
63.7
73.8
66.8
SMAT Gopal Amer
40
74.7
57.8
78.6
75.6
71.1
FCTTmcck Xu et al
2025
104
46
65.6
75
60.7
78.8
84.6
76.5
62.4
71,5
66.3
CompresxTrackerr
Hong et
AsymTrack-T
Zhu et
2025
145
55
62.3
71.3
54.7
76,2
80.9
71.6
60.8
68.7
61.2
AsymTTrack-S
Zhu et
2025
136
48
65.5
74.8
58.9
77.9
82.2
74.0
62.8
71.2
64.8
AsymTrack
Zhu et
2025
135
32
67.7
76.6
61.4
80.0
84.5
77.4
64.7
73.0
67.8
FARTrackpa
343
121
101
62.8
72.6
50.9
75.6
81.3
70.5
58.6
67.1
59.6
FARTrackuao
210
77
69.9
81.2
61.4
79.1
84.5
75.6
61.3
69.7
64.1
FARTrackin
135
53
42
70.6
81.0
63.8
80.7
85.6
77.5
63.2
71.6
66.7
The tiny a 15-layer encoder model. Nano distills the 15-layer encoder into 10 layers, and pico into 6 layers. Tiny matches Asym'rrck-B GPU while keeping AsymTrack-T CPU speed. Nano surpasses TrnppresTTrccker-SSTTaaaaa datasets. Pico outperforms MixFormerV25 Cui at. (2023) by 0.9% in AO on GOT-10k Huang et al. 2019). showing FARTrack's 's efficiency.
Hong et al. (2025) in terms of key evaluation metrics on all
Training. To conduct a fair comparison with mainstream trackers, we carried out Frame-level Pretraining the COCO2017 7 Lin et al. (2014) dataset. Subsequently, we performed Task-Specific Self-Distillation Training to compress our model. Finally, we conducted Inter-frame Autoregressive Sparsification Training further accelerate model. The detailed training process can be found in the supplementary material.
4.2 MAIN RESULTS
We evaluated the performance of our proposed FARTrackins- FARTrack and FARTrack on several benchmarks, including GOT-10k HHuang al. (2019). TrackingNet Muller (2018). LaSOT Fan et al.) (2019), LaSOText Faan et al. (2020). NFS Kiani Galoogahi et al. (2017) UAV123 Mueller al. (2016) and VastTrack[Pens al. (2024).
GOT-10k Huang al. (2019). GOT-10k is real world general object detection dataset. As shown in Table FARTrackiin outperforms AsymTracc-BB by 2.9% in AO score, achieving a GPU speed of 135 FPS and CPU speed of 53 FPS. Furthermore. the most lightweight version, FARTrackpico. outperforms MixFormerV2-S by 0.9% in AO, while delivering nearly three times the GPU speed and four times the CPU speed.
TrackingNet Muller et al. 2018). TrackingNet is large-scale dataset featuring over 30,000 videos Muller from diverse real world scenes. The evaluation on this extensive dataset highlights the efficiency and effectiveness of FARTrack. As illustrated in Table FARTrack achieves performance close to that AsymTrack-B B, the top-performing tracker on this benchmark, while running nearly twice as fast Asym/rack-BB on the GPU.

---

Published as a conference paper at ICLR 2026
Table 2: Details of our FARTrack model variants
Table Comparison on VastTrack benchmark.
Model
FARTrackmm FARTraokay
FARTiakkaa
VastTrack
Methods
Backbone
ViFTan
ViT-layy
ViT-Tiny
AUC(%)
P (Nomm((S)
P(%)
Encoder Layers
15
10
FARTrackayy
35.2
36.5
32.1
Inpur Sizes
[12.2224]
[112,224]
[12.2244]
FARTrackum
33.9
35.1
30.3
Templates
FARTrackiio
25.7
MACs (G)
2.65
1.78
MixformerV2-8
35.2
36.5
33.0
Params
6.82
4.59
2.81
DiMP
29.9
31.7
25.7
LaSOText Fan et al. (020). LaSOText is an Table 4: State-of-the-art comparison more extended subset of LaSOT, encompassing 150 ad- benchmarks.
ditional videos from 15 new categories. As shown in Table 4 FARTrackiny outperforms AsymTrack- B, with 0.4% AUC improvement, demonstrating its effectiveness in small object tracking.
GPU
CPU
LaSOText
NFS
UAV123
Methods
FPS
PPS
AUC(S)
AUC(%)
AUC(S)
LighTTrack
59
62.5
FEAR
123
28
61.4
E.T. Track
33
16
59.0
62.3
NFS Kiani Galoogahi et al. (2017). The NFS is a high-frame-rate benchmark dedicated to sce- narios with fast-moving objects. As shown in T on the 30 FPS version of the NFS dataset, FARTrackuny achieves outstanding performance these challenging scenarios and attains the highest AUC score.
Hil-Taay
135
42
35.8
53.2
58.7
HiT-Small
121
35
40.4
61.8
63.3
HiT-Base
116
30
44.1
63.6
65.6
MixformerV2
133
43.6
65.8
LiccTrack-44
195
29
63.4
66.4
ECThack
104
39.9
61.1
67.0
CompressTrccerr
207
48
40.4
62.5
UAV123 Mueller et al. (016)). The UAV123
AsymfTrack-T
focuses on the unique challenges of UAV-based vi- sual tracking. As shown Table 4 FARTrackinn delivers competitive performance on this dataset, outperforming AsymTrack- T by 1.2%. With com- parable GPU and CPU inference speeds, it also surpasses AsymTrack-S by 0.2%.
AsymTrack-5
136
48
43.3
64.9
65.6
AsymTTrackBB
135
44.6
64.4
66.5
FARTrack
343
FARTrackmm
210
77
43.8
65.1
62.6
FARTracking
135
45.0
66.9
65.8
LaSOT Fan et al. (2019). LaSOT is large-scale benchmark designed to assess the robustness of long-term tracking. As demonstrated Table [1] FARTrack achieves AUC of 2.6% over MixFormerV2-S on this dataset. While maintaining comparable running speed the GPU, it nearly doubles the speed on the CPU. FARTracKnao matches AUC of MixFormerV2-S but surpasses it in both GPU and CPU running speed.
Tast Peng et al. (2024). VastTrack is dataset aimed at advancing the development of more general visual tracking technology, covering 2, 15 object categories and containing 50,610 video sequences. As shown in Ta]le on this dataset, FARTrackun achieves an AUC comparable to that of MixFormerV2-BB which fully highlights the robustness of FARTrack.
4.3 EXPERIMENTAL ANALYSES
We analyze the main properties of the FARTrack. For the following experimental studies, we follow the GOT-10k test protocol unless otherwise noted. Default settings are marked in gray
Distillation Strategy. compare layer-by-layer distillation with cross-layer distillation to validate method. Cross-layer distillation uses an intermittent student-teacher layer correspondence, which aids knowledge transfer but introduces feature consistency issues. Considering the weaker representation capacity of ViT-Tiny, we also conduct ViT-Base-guided distillation to verify the superiority layer-by-layer distillation.
As shown in Table 5] we design manual layer reduction strategy following the Deep-to-Shallow distillation in MixtormerV2, where REMOVE LAYERS for 10-layer and 6-layer models are set to [0, 3, 6, 9, 12] and [0, 4, 6), respectively. Removing model layers allows the remaining
7

---

Published as a conference paper at ICLR 2026
Table 5: vs, Deep-to-Shallow Distillation Cui al.
Table 6: Sparsification Comparison.
(2023)
run-time
sequence-time
MACs
Params
CPU
GPU
AD
Methods
Layer
SRto
2.990
6.22M
49
128
70.0
Inyer-hy-lnyer
3.14G
7.2MM
36
114
69.5
62.8
72.6
50.9
2.65G
6.82M
135
70.6
10
67.8
78.0
60.6
doep-to-shallow
61.9
70.9
50.4
(a) Distillation Strategy
(b) Distillation Target
(c) Distillation Loss
70
70
70
Average Overlap (AO)
50
50
50-
40
Tajestory
Visual-Traa
byyen-by-lyyer tiny
Template
Search
30
30
30
7
11
13
15
5
11
13
15
5
7
11 13
15
Distilled model depth
Distilled model depth
Distilled model depth
Figure 4: Layer-by-layer distillation accuracy curve.
layers to perform sequential inter-layer matching. However, manual layer assignment fails to ensure reasonable alignment, causing semantic mismatch in cross-layer distillation. This mismatch breaks feature alignment and leads to severe accuracy drop in deeper layers. In contrast, our method preserves feature consistency and effectively alleviates semantic mismatch.
As illustrated in Figure 4 [)., FARTrack with ViT-Base shares an identical hierarchical structure with FARTrackuny We distill each layer of ViT-Tiny using trajectory features from corresponding ViT-Base layers. Although the 15-layer base-to-ting distilled model performs well, forcing Tiny to mimic Base's 's outputs induces severe accuracy degradation deeper layers (10-14) and progressive decline in shallower layers due to representational capability mismatch. In contrast, our method preserves feature consistency and information density, maintaining nearly identical accuracy in layers 10-15 while constraining shallow-layer accuracy loss within acceptable range.
Distillation Target. To verify the necessity of distilling the trajectory sequence, we conducted exploratory experiment, shown in Figure Experiments reveal that directly distilling the search/template or jointly distilling visual features disrupts the hierarchical structure, causing accuracy degradation across all layers. In contrast, distilling the trajectory sequence minimally impacts the feature extraction hierarchy. Its autoregressive properties enhance knowledge propagation from deep shallow layers, maintaining nearly unchanged accuracy in layers 10-15 while preventing significant performance drops in shallow layers.
Distillation Loss. Experiments (Figure 4()) demonstrate that combining trajectory sequence loss (Leraj LCE Lstot) and KI. divergence loss (LKI.) effectively preserves accuracy in deep layers (e.g., layers 10-15) while controlling performance degradation in shallow layers. Removing trajectory sequence loss causes significant feature degradation and performance drop, proving its critical role in preserving temporal information. In contrast, omitting KL divergence loss triggers rapid feature collapse in shallow layers, leading catastrophic degradation of tracking performance, highlighting its indispensability for maintaining object tracking capability.
Sparsification Comparison. The runtime sparsification introduces extra computational processes during the inference, while sequence-level sparsification avoids this situation. To support this, conducted experiments on runtime sparsification and sequence-level sparsification respectively based on the base model, and compared the final results, as shown in Table 6
8

---

Published as a conference paper at ICLR 2026
Table Token Retention Ratio.
Table Attention Map Sampling Method.
Ratio
MACs
CPU
GPU
AO
SRsg
SRys
S
Saxa
Sall
Z c
AO
SRso
SRys
100%
2,99G
49
128
70.0
80.0
64.4
70.1
80.6
63.3
75%
2.63G
53
135
70.6
81.0
63.8
69.5
80.4
62.4
50%
2.35G
56
139
68.3
78.2
61.8
70.0
80.4
62.9
25%
2.01G
59.8
68.6
10%
1.82G
63
141
62.3
74.1
53.3
69.4
79.7
62.1
70.6
81.0
63.8
Table 9: Comparison with ARTrack Wei et al.
2023a) and LRTrackV2 Bai et al.
2024)
MAC'
Params
GPU
CPU
GOT-10K
Methods
Self-distillation Sparsification
FPS
FPS
AO(S)
SRgo(%)
(R)
ARTrackttinyle
2.39
13:56
68
34
59.9
69.6
50.4
AKTrackV2_ian(6)
2.10
8.31
110
49
60.6
70.9
45.7
(5)))
1.08
2,81
343
121
60.6
69.5
46.3
(A)
1.25
2.81
266
101
61.8
71.5
50.0
(AKTrackpi..66)
1.08
2.81
343
121
62.8
72.6
50.9
Runtime sparsification processes template tokens during each forward propagation in inference, introducing extra computational overhead that increases MACs from 2.99G to 3.14G and Params from 6.82M to 7.21 M. This redundant computation reduces CPU and GPU speeds by 26.5% and 10.9% respectively. In contrast, our sequence-level sparsification leverages intermediate results for decision -akiing without introducing additional computations, while propagating sparsification outcomes to subsequent frames autoregressively climinate redundant processing. Consequently. MACs are reduced to 2.65G with improved inference speeds on both CPU and GPU.
Token Retention Ratio. Token retention ratio in inter-frame autoregressive sparsification affects both performance and efficiency. As Table reducing the ratio from 100% to 75% decreased MACs by 11.4% (2.99G to 2.65G) while achieving peak AO (70.6%). This indicates significant seollate--rmovingg 25% background tokens preserves temporal modeling
redundancy in target and improves accuracy. Further reduction to 25% caused 3.3% AO drop (67.3%), demonstrating excessive removal harms tracking robustness.
Attention Map Sampling Method. We analyze different sampling strategies for inter-frame au- toregressive sparsification Table g Six1 sparsifies the central feature, Sax3 uses 3 x central region, Sall covers the entire search area. Z denotes template self-attention, while C' refers to command -empaad attention.
Sax3 avoids target exclusion from center shift hollow structures and reduces background overfocus versus Sall, improving accuracy. Combining S and Z underperforms due to self-overfocus Z. Instead, integrating S and C enables effective template sparsification: S separates foreground and background coarsely, while C' refines edge features. This reduces redundancy and improves accuracy and efficiency.
Independent Performance Gains of Each Module. To analyze the performance gains from each module, we perform supplementary comparisons with ARTrack and ARTrack ARTrack, ARTrackV2, and FARTrackpico all employ the ViT-Tiny architecture with 6 encoder layers, and share the same training settings and datasets as FARTrack.
As shown in Table 9 it strongly validates the effecttivnness of the self-distillation module and the sparsification module. The self-distillation module, which enables the middle layers of the network to learn knowledge from the deep layers, improves the AO by 1.2% compared with ARTrack V2_ V2_tiny (6). The sparsification module, which retains useful foreground feature tokens while eliminating redundant background feature tokens, boosts the AO by 1.0% (from 61.8 to 62.8).
Layer-wise cross-attention visualization. As shown in Figure 5 cross- cross-attention between trajectory sequences and search regions evolves across layers: shallow layers (0-4) capture edges and back-

---

Published as a conference paper at ICLR 2026
layar layer layer lnyar Figure 5: Layer-wise cross-attention visualization. search: Search region and template. layer 0-14:
search
Iayar
layer
loyer
layer
layar
layar
layer layer
Trajectory sequences search cross-attention maps each layer.
Moboa Bltar
Ovanizal
Vadatoo
Occkaaton
Tarad Steinkuse
25%
Figure 6: Template sparsification visualization. Figure 7: Template retention visualization. (a): The initial template of the video sequence. The
(a): Search region. The red boxes denote the ground truth. (a)-(e): Templates sampled red boxes denote the ground truth. (b)-(e): The fixed interval of N and sparsified with 75% template at time step t, sparsified with different token retention ratios.
token retention ratio.
ground, while deeper ones (9-14) transition toward target contours. Hierarchical feature learning is retained via distillation, maintaining consistent trajectory propagation and boosting accuracy, especially in intermediate layers (e.g., 5, 9).
Template sparsification visualization. Figure 6 shows that our method retains critical tokens and removes redundancy under motion blur, appearance change, and occlusion. Unlike frame-wise sparsification, which often fails due single-frame errors, our inter-frame autoregressive approach uses multi-template complementarity and temporal modeling to track dynamic targets and preserve structure with inconsistent cues.
Template retention visualization. Figure visualizes retained tokens at different retention ratios. For oversized targets, low ratios (<50%) lose essential features; for shrinking ones, they harm deformation representation and cause misclassification. Thus, we set 75% retention ratio to preserve sufficient target information while reducing redundancy, balancing accuracy and efficiency.
CONCLUSION
We propose FARTrack, fast and high-perfon performance multi-template autoregressive tracking framework. It integrates task-specific self-distillation and fram- autoregressive sparsification. While slightly behind top-performing methods Bai al. (2024); Wei et al. (2023a), it excels in spred-performance balance, especially in speed. Our distillation preserves temporal information of trajectory sequences via laser-wise mask-specffic tokens distillation, avoiding suboptimal manual layer assignments. The sparsification method propagates multi-templas template sparsification results autoregressively. achieving temporally-glae -llbbal optimality without extra cost. FARTrack performs well across GPU, CPU, and NPU platforms, offering efficient solution for practical deployment.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was support by the National Natural Science Foundation of China No. 62572385, the Fundamental Research Funds for the Central Universities No. xxj032023020, and CAAI-CANN Open Fund, developed on Openl Community.
REFERENCES
Sungsoo Ahn, Shell Xu Hu, Andreas Damianou, Neil D Lawrence, and Zhenwen Dai. Variational information distillation for knowledge transfer. In CVPR, 2019.
Takanori Asanomi, Kazuya Nishimura, and Ryoma Bise. Multi-frame attention with feature-level warping for drone crowd tracking. In WACV, 2023.
Yifan Bai, Zeyang Zhao, Yihong Gong, and Xing Wei. Artrackv2: Prompting autoregressive tracker where to look and how to describe. In CVPR, 2024.
Luca Bertinetto, Jack Valmadre, Joao F Hemriques, Andrea Vedaldi, and Philip HS Torr. Fully- convolutional siamese networks for object tracking. In ECCV, 2016.
Goutam Bhat, Martin Danelljan, Luc Van Gool, and Radu Timofte. Learning discriminative model prediction for tracking. In ICCV, 2019.
Philippe Blatter, Menelaos Kanakis, Martin Danelljan, and Luc Van Gool. Efficient visual tracking with exemplar transformers. In WACV, 2023.
Vasyl Borsuk, Roman Vei, Orest Kupyn, Tetiana Martyniuk, Igor Krashenyi, and Jifi Matas. Fear: Fast, efficient, accurate and robust visual tracker. In ECCV, 2022.
Wenrui Cai, Qingjie Liu, and Yunhong Wang. Hiptrack: Visual tracking with historical prompts. In CVPR, 2024.
Yidong Cai, Jie Liu, Jie Tang, and Gangshan Wu. Robust object modeling for visual tracking. In ICCV, 2023.
Anjia Cao, Xing Wei, and Zhiheng Ma. Flame: Frozen large language models enable data-efficient language-image pre-training, In CVPR, 2025.
Ziang Cao, Ziyuan Huang, Liang Pan, Shiwei Zhang, Ziwei Liu, and Changhong Fu. Tctrack: Temporal contexts for aerial tracking. In CVPR, 2022.
Ting Chen, Saurabh Saxena, Lala Li, David Fleet, and Geoffrey Hinton. Pix2seq: A language modeling framework for object detection. arXiv, 2021.
Ting Chen, Saurabh Saxena, Lala Li, Tsung- Yi Lin, David J Fleet, and Geoffrey E Hinton, unified sequence interface for vision tasks. In NeurIPS, 2022a.
Xin Chen, Ben Kang, Dong Wang, Dongdong Li, and Huchuan Lu. Efficient visual tracking via hierarchical cross-attention transformer. ECCV, 2022b.
Xin Chen, Houwen Peng, Dong Wang, Huchuan Lu, and Han Hu. Seqtrack: Sequence sequence learning for visual object tracking. In CVPR, 2023.
Yutao Cui, Tianhui Song, Gangshan Wu, and Limin Wang. Mixformerv2: Efficient fully transformer tracking. NeurlPS, 2023.
Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, and Michael Felsberg. Eco: Efficient convolution operators for tracking. In CVPR, 2017.
Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, and Michael Felsberg. Atom: Accurate tracking by overlap maximization. In CVPR, 2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x 16 words: Transformers for image recognition scale. arXiv, 2020.
11

---

Published as a conference paper at ICLR 2026
Pan Du, Wangbo Zhao, Xinai Lu, Nian Liu, Zhikai Li, Chaoyu Gong, Suyun Zhao, Hong Chen, Cuiping Li, Kai Wang, et al. Unsupervised learning for class distribution mismatch. In JCML, 2025.
Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai. Yong Xu, Chunyuan Liao, and Haibin Ling. Lasot: A high-quality benchmark for large-scale single object tracking. In CVPR, 2019.
Heng Fan, Hexin Bai, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Harshit, Mingzhen Huang, Juehuan Liu, Yong Xu, Chunyuan Liao, Lin Yuan, and Haibin Ling. Lasot: A high-quality large-scale single object tracking benchmark. IJC,, 2020.
Shenyuan Gao, Chunluan Zhou, Chao Ma, Xinggang Wang, and Junsong Yuan. Aiatrack: Attention in attention for transformer visual tracking. In ECCV, 2022.
Shenyuan Gao, Chunluan Zhou, and Jun Zhang. Generalized relation modeling for transformer tracking. In CVPR, 2023.
Zhora Gevorgyan. Siou loss: More powerful learning for bounding box regression. arXiv, 2022.
Goutam Yelluru Gopal and Maria A. Amer. Separable self and mixed attention transformers for efficient object tracking. In WACV, 2024.
Jianyuan Guo, Kai Han, Yunhe Wang, Han Wu, Xinghao Chen, Chunjing Xu, and Chang Xu. Distilling object detectors via decoupled features. In CVPR, 2021.
Kaijie He, Canlong Zhang, Sheng Xie, Zhixin Li, and Zhiwen Wang. Target-aware tracking with long-term context attention. In AAAI, 2023.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR, 2022.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean, Distilling the knowledge in a neural network. arXiv, 2015.
Lingyi Hong, Shilin Yan, Renrui Zhang, Wanyun Li, Xinyu Zhou, Pinxue Guo, Kaixun Jiang, Yiting Chen, Jinglun Li, Zhaoyu Chen, and Wenqiang Zhang. Onetracker: Unifying visual object tracking with foundation models and efficient tuning. In CVPR, 2024.
Lingyi Hong, Jinglun Li, Xinyu Zhou, Shilin Yan, Pinxue Guo, Kaixun Jiang, Zhaoyu Chen, Shuyong Gao, Runze Li, Xingdong Sheng, et al. General compression framework for efficient transformer object tracking. In ICCV, 2025.
Xiaojun Hou, Jiazheng Xing, Yijie Qian, Yaowei Guo, Shuo Xin, Junhao Chen, Kai Tang, Mengmeng Wang, Zhengkai Jiang, Liang Liu, and Yong Liu. Sdstrack: Self-distillation symmetric adapter learning for mudti- visual object tracking. In CVPR, 2024.
Lianghua Huang, Xin Zhao, and Kaiqi Huang, 10k- A large high-diversity benchmark for generic object tracking in the wild. PAMI, 2019.
Ben Kang, Xin Chen, Dong Wang, Houwen Peng, and Huchuan Lu. Exploring lightweight hierarchi- cal vision transformers for efficient visual tracking. In ICCV, 2023.
Hamed Kiani Galoogahi, Ashton Fagg, Chen Huang, Deva Ramanan, and Simon Lucey. Need for speed: A benchmark for higher frame rate object tracking. In ICCV, 2017.
Minji Kim, Seungkwan Lee, Jungseul Ok, Bohyung Han, and Minsu Cho. Towards sequence-level training for visual tracking. In ECCV, 2022.
Yutong Kou, Jin Gao, Bing Li, Gang Wang, Weiming Hu, Yizheng Wang, and Liang Li. Zoomtrack: Target-aware non-uniform resizing for efficient visual tracking. In NeurIPS, 2023.
Bo Li, Junjie Yan, Wei Wu, Zheng Zhu, and Xiaolin Hu. High performance visual tracking with siamese region proposal network. In CVPR, 2018.
12

---

Published as a conference paper at ICLR 2026
Bo Li, Wei Wu, Qiang Wang, Fangyi Zhang, Junliang Xing, and Junjie Yan. Siamrpn++: Evolution of siamese visual tracking with very deep networks. In CVPR, 2019.
Quanquan Li, Shengying Jin, and Junjie Yan. Mimicking very efficient network for object detection. In CVPR, 2017.
Xin Li, Yuqing Huang, Zhenyu He, Yaowei Wang, Huchuan Lu, and Ming-Hsuan Yang. Citetracker: Correlating image and text for visual tracking. In ICCV, 2023.
Shiyi Liang, Yifan Bai, Yihong Gong, and Xing Wei. Autoregressive sequential pretraining for visual tracking. CVPR, 2025.
Youwei Liang, Chongjian Ge, Zhan Tong, Yibing Song, Jue Wang, and Pengtao Xie. Not all patches are what you need: Expediting vision transformers via token reorganizations. arXiv, 2022.
Liting Lin, Heng Fan, Zhipeng Zhang, Yong Xu, and Haibin Ling. Swintrack: A simple and strong baseline for transformer tracking. In NeurlPS, 2022.
Tsung-Yo Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. ECCV, 2014.
Zhiheng Ma, Anjia Cao, Funing Yang, Yihong Gong, and Xing Wei. Curriculum dataset distillation. IEEE Transactions on Image Processing, 2025.
Christoph Mayer, Martin Danelljan, Goutam Bhat, Matthieu Paul, Danda Pani Paudel, Fisher Yu, and Luc Van Gool. Transforming model prediction for tracking. In CVPR, 2022.
Matthias Mueller, Neil Smith, and Bernard Ghanem. A benchmark and simulator for uav tracking. In ECCV, 2016.
Matthias Muller, Adel Bibi, Silvio Giancola, Salman Alsubaihi, and Bernard Ghanem. Trackingnet: A large-scale dataset and benchmark for object tracking in the wild. In ECCV, 2018.
Nikolaos Passalis, Maria Tzelepi, and Anastasios Tefas. Heterogeneous knowledge distillation using information flow modeling. In CVPR, 2020.
Liang Peng, Junyuan Gao, Xinran Liu, Weihong Li, Shaohua Dong, Zhipeng Zhang, Heng Fan, and Libo Zhang. Vasttrack: Vast category visual object tracking. NeurIPS, 2024.
Dekang Qi, Shuang Zeng, Xinyuan Chang, Feng Xiong, Shichao Xie, Xiaolong Wu, and Mu Xu. Mernav: A highly generalizable memory-execute-revies framework for zero-shot object goal navigation. arXiv preprint arXiv:2602.054677 2026.
Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, and Cho-Jui Hsieh. Dynamicvit: Efficient vision transformers with dynamic token sparsification. In NewrIPS, 2021.
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv, 2014.
Jianbing Shen, Yuanpei Liu, Xingping Dong. Xiankai Lu, Fahad Shahbaz Khan, and Steven Hoi. Distilled siamese networks for visual tracking. PAMI, 2021.
Qiuhong Shen, Lei Qiao, Jinyang Guo, Peixia Li, Xin Li, Bo Li, Weitao Feng, Weihao Gan, Wei Wu, and Wanli Ouyang. Unsupervised learning of accurate siamese tracking. In CVPR, 2022.
Liangtao Shi, Bineng Zhong, Qihua Liang, Ning Li, Shengping Zhang, and Xianxian Li. Explicit visual prompts for visual object tracking. In AAAI, 2024.
Zikai Song, Junqing Yu, Yi-Ping Phoebe Chen, and Wei Yang. Transformer tracking with cyclic shifting window attention. In CVPR, 2022.
Zikai Song, Run Luo, Junqing Yu, Yi-Ping Phoebe Chen, and Wei Yang. Compact transformer tracker with correlative masked modeling. In AAAI, 2023.
13

---

Published as a conference paper at ICLR 2026
Yiming Sun, Fan Yu, Shaoxiang Chen, Yu Zhang, Junwei Huang, Yang Li, Chenhui Li, and Changbo Wang. Chattracker: Enhancing visual tracking performance via chatting with multimodal large language model. In NeurlPS, 2025.
Chuanming Tang, Kai Wang, Joost van de Weijer, Jianlin Zhang, and Yongmei Huang. Avitmp: tracking-specific transformer for single-branch visual tracking. JEEE Transactions Intelligent Vehicles, 2024.
Feng Tang and Qiang Ling. Ranking-based siamese visual tracking. In CVPR, 2022.
Frederick Tung and Greg Mori. Sumilarity-preserving knowledge distillation. In ICCV, 2019.
Xiao Wang, Shiao Wang, Chuanming Tang, Lin Zhu, Bo Jiang, Yonghong Tian, and Jin Tang. Event stream-based visual object tracking: high-resolution benchmark dataset and a novel haseline. In CVPR, 2024.
Qingmao Wei, Bi Zeng, Jianqi Liu, He, and Guotian Zeng. Litetrack: Layer pruning with asynchronous feature extraction for lightweight and efficient visual tracking. In ICRA, 2024.
Xing Wei, Yifan Bai, Yongehao Zheng, Dahu Shi, and Yihong Gong. Autoregressive visual tracking. In CVPR, 2023a.
Xing Wei, Anjia Cao, Funing Yang, and Zhiheng Ma. Sparse parameterization for epitomic dataset distillation. In NeurlPS, 2023b.
Qiangqiang Wu, Tianyu Yang. Ziquan Liu, Baoyuan Wu, Ying Shan, and Antoni B. Chan. Dropmae: Masked autoencoders with apatial-attentien dropout for tracking tasks. In CVPR, 2023a.
Qiao Wu, Jiaqi Yang. Kun Sun, Chu ai Zhang, Yanning Zhang, and Mathieu Salzmann. Mixcycle: Mixup assisted semi-supervised 3d single object tracking with cycle consistency. In ICCV, 2023b.
You Wu, Yongxin Li, Mengyuan Liu, Xucheng Wang, Xiangyang Yang. Hengzhou Ye, Dan Zeng, Qijun Zhao, and Shuiwang Li. Learning adaptive and view-invariant vision transformer with multi-teacher knowledge distillation for tale uav tracking. arXiv, 2024.
Fei Xie, Chunyu Wang, Guangting Wang, Yue Cao, Wankou Yang, and Wenjun Zeng. Correlation- aware deep tracking. In CVPR, 2022.
Fei Xie, Lei Chu, Jiahao Li, Yan Lu, and Chao Ma. Videotrack: Learning to track objects via video transformer. In CVPR, 2023.
Jinxia Xie, Bineng Zhong, Zhiyi Mo, Shengping Zhang, Liangtao Shi, Shuxiang Song, and Rongrong Ji. Autoregressive queries for adaptive tracking with spatio-temporal transformers. In CVPR, 2024.
Daitao Xing, Nikolaos Evangeliou, Athanasios Tsoukalas, and Anthony Tzes. Siamese transformer pyramid networks for real-time uav tracking. In WACV, 2022.
Liang Xu, Zhiqing Guo, and Liejun Wang. Efficient hybrid linear self-attention based visual object
tracking with lora. Neurocomputing. 2025.
Yinda Xu, Zeyu Wang, Zuoxin Li, Yuan, and Gang Yu. Siamfc++: Towards robust and accurate visual tracking with target estimation guidelines. In AAAI, 2020.
Bin Yan, Houwen Peng, Jianlong Fu, Dong Wang, and Huchuan Lu. Learning spatio-temporal transformer for visual tracking. In ICCV, 2021a.
Bin Yan, Houwen Peng, Kan Wu, Dong Wang, Jianlong Fu, and Huchuan Lu. Lighttrack: Finding lightweight neural networks for object tracking via one-shot architecture search. In CVPR, 2021b.
Dawei Yang, Jianfeng He. Yinchao Ma, Qianjin Yu, and Tianzhu Zhang. binggroudd-backrrown distribution modeling transformer for visual object tracking. ICCV, 2023a.
Jinyu Yang. Shang Gao, Zhe Li, Feng Zheng, and Aleš Leonardis. Resource- rgbd aerial tracking. In CVPR, 2023b.
14

---

Published as a conference paper at ICLR 2026
Yandan Yang. Shuang Zeng, Tong Lin, Xinyuan Chang, Dekang Qi. Junjin Xiao, Haoyun Liu, Ronghan Chen, Yuzhi Chen, Dongjie Huo, et al. Abot-mo: Vla foundation model for robotic manipulation with action manifold learning. arXiv preprint arXiv:2602. 2026.
Botao Ye, Hong Chang, Bingpeng Ma, Shiguang Shan, and Xilin Chen. Joint feature learning and relation modeling for tracking: A one-stream framework. In ECCV, 2022.
Kaiyu Yue, Jiangfan Deng, and Feng Zhou, Matching guided distillation. In ECCV, 2020.
Sergey Zagoruyko and Nikos Komodakis. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. arXiv, 2016.
Ram Zaveri, Shivang Patel, Yu Gu, and Gianfranco Doretto. Improving accuracy and generalization for efficient visual tracking. In WACV, 2025.
Shuang Zeng, Xinyuan Chang, Mengwei Xie, Xinran Liu, Yifan Bai, Zheng Pan, Mu Xu, and Xing Wei. Futuresightdrive: Thinking visually with spatio-temporal cot for autonomous driving, arXiv preprint 2XX.:22555..76633 2025a.
Shuang Zeng, Dekang Qi. Xinyuan Chang, Feng Xiong, Shichao Xie, Xiaolong Wu, Shiyi Liang, Mu Xu, and Xing Wei. Janusvin: Decoupling semantics and spatiality with dual implicit memory for vision-language navigation. arXiv preprint arXiv:2509. 22548, 2025b.
Geng Zhang, Shenggan Cheng, Xuanlei Zhao, Ziming Liu, and Yang You. Helixpipe: Efficient distributed training of long sequence transformers with attention parallel pipeline parallelism. In PPoPP, 2026.
Jiqing Zhang, Bo Dong, Haiwei Zhang, Jianchuan Ding, Felix Heide, Baocai Yin, and Xin Yang. Spiking transformers for event-based single object tracking. In CVPR, 2022.
Linfeng Zhang, Jicbo Song, Anni Gao, Jingwei Chen, Chenglong Bao, and Kaisheng Ma. Be your teacher: Improve the performance of convolutional neural networks via self distillation. In ICCV, 2019.
Linfeng Zhang, Chenglong Bao, and Kaisheng Ma. Sclf-distillation: Towards efficient and compact neural networks. PAMI, 2021.
Minghua Zhang, Qiuyang Zhang, Wei Song, Dongmei Huang, and Qi He. Promptvt: Prompting for efficient and accurate visual tracking. IEEE TCSVT, 2024.
Tianlu Zhang, Hongyuan Guo, Qiang Jiao, Qiang Zhang, and Jungong Han. Efficient rgb-t tracking via cross-modality distillation. In CVPR, 2023.
Haojie Zhao, Dong Wang, and Huchuan Lu, Representation learning for visual object tracking by masked appearance transfer. In CVPR, 2023.
Wangbo Zhao, Yizeng Han, Jiasheng Tang, Zhikai Li, Yibing Song, Kai Wang, Zhangyang Wang, and Yang You. A stitch in time saves nine: Small vlm is precise guidance for accelerating large vlms. arXiv, 2024a.
Wangbo Zhao, Jiasheng Tang, Yizeng Han, Yibing Song, Kai Wang, Gao Huang, Fan Wang, and Yang You. Dynamic tuning towards parameter and inference efficiency for vit adaptation. 2024b.
Wangbo Zhao, Yizeng Han, Jiasheng Tang, Kai Wang, Yibing Song, Gao Huang, Fan Wang, and Yang You. Dynamic diffusion transformer. 2025a.
Wangbo Zhao, Yizeng Han, Zhiwei Tang, Jiasheng Tang, Pengfei Zhou, Kai Wang, Bohan Zhuang, Zhangyang Wang, Fan Wang, and Yang You. Rapid 3: Tri-level reinforced acceleration policies for diffusion transformer. arXiv preprint 20X.:2250..23333 2025b.
Wangbo Zhao, Yizeng Han, Jiasheng Tang, Kai Wang, Hao Luo, Yibing Song, Gao Huang, Fan Wang, and Yang You. Dydit++: Diffusion transformers with timestep and spatial dynamics for efficient visual generation. IEFE Transactions Pattern Analysis and Machine Intelligence, 2026.
15

---

Published as a conference paper at ICLR 2026
Yaozong Zheng, Bineng Zhong, Qihua Liang, Zhiyi Mo. Shengping Zhang, and Xianxian Li. Odtrack: Online dense temporal token learning for visual tracking. In AAAI, 2024.
Li Zhou, Zikun Zhou, Kaige Mao, and Zhenyu He. Joint visual grounding and tracking with natural language specification. In CVPR, 2023a.
Xinyu Zhou, Pinxue Guo, Lingyi Hong, Jinglun Li, Wei Zhang, Weifeng Ge, and Wenqiang Zhang. Reading relevant feature from global representation memory for visual object tracking. In NeurlPS, 2023b.
Jiawen Zhu, Huayi Tang, Xin Chen, Xinying Wang, Dong Wang, and Huchuan Lu. Two-stream beats one-stream: Asymmetric siamese network for efficient visual tracking. In AAAI, 2025.
16

---

Published as a conference paper at ICLR 2026
APPENDIX
A TRAINING DETAILS
In the supplementary material, we have supplemented the experiments section. We have sequentially presented the detailed training processes of Frame-level Pretraining, Task-Specific Self-Distillation Training, and Inter-frame Autoregressive Sparsification Training.
Frame-level Pretraining. To fairly compare with mainstream trackers, we introduce the COC02017 Lin al. (2014) dataset, which is commonly used in template matching training paradigms, and apply frame-level pretraining to the AR(O) model. Similar to DiMP Bhat et al. (2019) and STARK Yan et (2221)) the AR(O) model uses the same data augmentations Track, including horizontal flip and brightness jittering. The model is optimized using Adam with a weight decay of x 10 -4-4_ The learning rate for the backbone is set x 10-4, and 10 for other parameters. The AR(0) model is trained for 500 epochs, with 76,800 template-search frame pairs sampled per epoch. The learning rate is reduced by 10% at the 400th epoch.
OS-
Task-Specific Self-Distillation Training. This phase uses the same datasets and data augmentations as the AR(O) phase but introduces additional KI. divergence loss (Lk1.) and trajectory sequence loss (Cura) Lce + CSIoU) for each layer. The model optimized using Adam W with a weight decay of 1 x 10-4 The learning rate for the backbone is set to 4 10-5, and 4 10-4 for other parameters. The distillation model is trained for 300 epochs, with 76,800 template-search frame pairs randomly sampled per epoch. This process generates multiple versions of the distilled model, producing models at different layers from a single distillation.
Inter-frame Autoregressive Sparsification Training. Unlike traditional per-frame template match- ing, this method trains FARTrack directly continuous video sequences without applying data augmentations. The model is optimized using AdamW with a weight decay of 5 10-2. The learning rate for the backbone is set to 4 x 10-? and 4 x 10 for other parameters. The training process consists of 20 epochs, with 1,000 video slices randomly sampled from continuous video per epoch. Due to GPU memory limitations, each slice contains 32 frames.
B TEMPLATE QUANTITY
In this section, an ablation experiment is conducted on the number templates. The number of templates not only affects the operation efficiency of the model but also impacts the tracking accuracy. Therefore, is essential to explore this aspect.
Table 10: Impact of Templates on Efficiency and Performance.
Template Count
MACs
Paramn
CPU
GPU
AO
SRa
SR75
L,7GG
6.82M
64
141
66.4
77.0
58.0
3
2.17G
6.82M
55
139
68.1
78.6
60.9
5
2,65G
6.82M
53
135
70.6
81.0
63.8
7
3.13G
6.82M
48
124
69.6
80.3
62.5
3.61G
6.82M
45
115
70.0
80.7
62.3
As shown in Table TO when the number of templates gradually increases from 5, the AO gradually rises from 66.4% to 70.6%, and the MACs increases synchronously by 35.8% (from 1.70G to 2.65G). During this stage, adding every two templates can increase the AO by approximately 2.1 1%, which verifies that the multi-template mechanism can effectively aggregate the appearance changes of the target and achieve an accurate representation of the target's 's dynamic appearance. However, when the number of templates exceeds 5 (e.g.. 7-9), the AO instead drops the range of 69.6%%77..0% This indicates that redundant templates lead to the dispersion of attention weights and interfere with the extraction of key features.
17

---

Published as a conference paper at ICLR 2026
Distillation-Fee
Aessetiin-Distillaae
Self-Distillaton
search
layer 0
layar 1
layar 2
layer 3
layer 4
layer
layar
layer 7
layer
layer
layer 10 bayer 11
layer 12 layer 13 layer 14
Figure Layer-wise cross-attention visualization Comparison. search: Search region and template. layer 0-14: Trajectory sequences search cross-attention maps at each layer.
Finally, we choose to use 5 templates because it achieves the optimal efficiency at the critical point of the accuracy peak.
C TEMPLATE UPDATE STRATEGY
In this section, an ablation experiment is conducted on the template update strategy. Different test benchmarks have an obvious correlation with different template update strategies. Therefore, is necessary to experiment with different update strategies.
Table 11: Template Update Strategy Comparison.
GOT-1Ok
LaSOT
Update Strategy
AO(%)
SRoa(%)
(Rrs(%)
AUC(%)
Poorm(())
P(%)
Linear
70.9
81.6
63.4
62.5
70.2
65.3
Exponential Decay
70.6
81.0
63.8
63.2
71.6
66.7
Logarithmic Increusing
69.6
80.1
62.8
61.6
68.7
65.4
Equal-interval
69.5
79.4
63.1
62.1
69.3
65.5
linear sampling achieves relatively mediocre values among all the update strategies. However, since it always samples all templates linearly, it is not sensitive to the length of the target sequence, and thus it has good performance different datasets. Exponential decay sampling samples more frames closer to the current frame. This is more friendly to long sequences such as the LaSOT benchmark and also performs well in short video sequences like GOT- 10k. Logarithmic increasing sampling samples more frames that are farther from the current frame, which makes perform poorly long video sequences such as LaSOT. Equal-interval sampling performs poorly both in GOT- 10k and LaSOT. This is because equal- -qual-interval sampling may fail to sample the initial static template. And if there is long period of tracking failure, such the target disappearing or being occluded, it is very likely that all templates will become invalid, resulting in decrease in accuracy.
As shown in Table II
D INTER-LAYER ATTENTION VISUALIZATION COMPARISON
In this section, conduct ablation experiments the inter-layer attention of different distillation methods. Given the evident correlation between the distribution patterns inter-layer attention and various distillation approaches, experimental validation is therefore deemed necessary.
For the layer-wise cross-attention visualized in Figure g the inter-layer attention distributions under the distillation free setting exhibit diversity with distinct differences across various layers. Compared
18

---

Published as a conference paper at ICLR 2026
with self-distillation. the inter-layer attention distributions in the base2tiny distillation setting more scattered. In contrast, the deep-layer attention distributions after trajectory self-distillation demonstrate consistency mechanism in transferring deep-layer knowledge to middle layers (e.g., layers 5-14). Thus, compared with other configurations, self-distillation enables coherent inter-layer feature alignment.
characteristic that confirms the effectiveness of the self-distillation
E COMPARATIVE ANALYSIS OF DISTILLATION vs. SCRATCH-TRAINE MODELS WITH DIFFERENT DEPTHS
We have supplemented comparative experiments between 10-layer or 6-layer models trained from scratch and our distilled versions to clarify whether distillation provides benefits beyond simply using fewer layers (see Table 12
Table 12: Performance Comparison of Different FARTrack Variants and Their Corresponding Scratch- Trained Models.
GOT-1Ok
TrackingNet
LaSOT
Methods
AO(S)
(%)
(R)
AUC(S)
PNovm (56)
P(S)
AUC((%)
Norm (9t
P(S)
FARTrack_ting tiny
70.6
81.0
63.8
80.7
85.6
77.5
63.2
71.6
66.7
FARTrack mamo((0))
69.9
81.2
61.4
79.1
84.5
75.6
61.3
69.7
64.1
FARTrack
FARTracksscmtch(( scrarcc(ce 10)
FARTrack )
62.8
72.6
75.6
81.3
70.5
67.1
60.6
69.5
46.3
73.3
78.4
67.1
56.8
64.1
55.7
The results confirm that distillation offers advantages far beyond simply using fewer layers:
(i) 10-layer 6-layer models trained from scratch: Suffer severe performance degradation (e.g.. TARTrack_scrateer (10) exhibits a 3.5% drop in AO compared to FARTrack_ting binnk.._t....ooo This is because such models lack the learning capacity of deep networks and fail capture complex visual/temporal features required for tracking.
(ii) Our distilled versions: Transfer deep-layer knowledge to middle layers via self- di-sdiiitaaaion, retain- ing higher performance (FARTrack nano( 10) only sees 0.7% AO drop compared to FARTrack_ting while achieving lower training costs than 10-layer or 6-layer models trained from scratch.
TRAINING CoST COMPARISON: CROSS-LAYER VS. TASK-SPECIFIC SELF-DISTILLATION
We have supplemented the training cost comparison between cross-layer distillation (Deep-to- Shallow) Cui al. (2023) and our task-specific self-distillation, with details as follows.
Both methods were trained on 8 NVIDIA RTX A6000 GPUs:
(i) Seep-to-Shallow distillation: Stage (15-to- layers) takes about 40 hours; Stage 2 (10-to-6 layers) takes about 38 hours (two-stage sequential training required).
(ii) seecifpec self-distillation: Takes about 36 hours total, with one-time training enabling direct utilization of multiple model versions.
This comparison fully demonstrates the training efficiency advantage our proposed method.
G THE USE OF LARGE LANGUAGE MODELS (LLMs)
We only used LLMs minimally to aid or polish writing. For instance, when describing a concept, we might leverage LLMs to ensure terminological precision, enhance logical coherence, optimize the academic tone of the exposition.
19


Reviews:
[{"review_id": "BJ4DgVhInm", "reviewer": "Reviewer_r6f4", "summary": "This paper presents FARTrack, a lightweight autoregressive visual tracking framework that combines task-specific self-distillation and inter-frame autoregressive sparsification to improve efficiency while maintaining accuracy. The method achieves a balance between speed and performance across different hardware platforms.", "questions": "Pls see the weakness.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The paper is well presented, with clear motivation, solid organization, and consistent writing quality.\n- FARTrack achieves impressive speed–accuracy trade-offs across GPU, CPU, and NPU platforms.\n- The design is simple and potentially applicable to other lightweight tracking pipelines.", "weaknesses": "- Compared with prior autoregressive tracking baselines or other light-weight trackers, the training set now includes VastTrack and LaSOT Ext, which significantly expand data diversity and size. Since VastTrack has been shown to enhance generalization and robustness, using it for training raises fairness concerns in comparing FARTrack with earlier methods that were trained on smaller datasets.\n- The visualization in Figure 5 suggests that feature representations from layers 7–14 appear highly similar, indicating limited hierarchical diversity. This calls into question the necessity and effectiveness of performing distillation across all consecutive layers.\n- Incorporating trajectory tokens inherently enhances temporal modeling and tracking robustness, which has already been verified in prior autoregressive trackers (e.g., ARTrack, ARTrackV2). The paper does not clearly disentangle this known benefit from the gains attributed to the proposed self-distillation or sparsification modules, thereby weakening the originality of the contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wHGpNgt6mF", "reviewer": "Reviewer_JQy9", "summary": "The paper proposes FARTrack, a fast tracking framework combining Task-Specific Self-Distillation and Inter-frame Autoregressive Sparsification to improve inference speed while maintaining tracking performance. The method builds on ARTrack and achieves competitive results with 135 FPS on GPU (tiny) and 343 FPS (pico).", "questions": "- What causes the LaSOT performance gap? Lower performance on long-term object tracking make sense but beyond \"model capacity loss,\" can you provide deeper analysis (e.g., attention pattern changes, temporal modeling degradation) of why distillation specifically hurts long-term tracking more than short-term?\n- How does a 10-layer or 6-layer model trained from scratch compare to your distilled versions? This would clarify whether distillation provides benefits beyond simply using fewer layers.\n- How does the method generalize to other challenging scenarios like NFS (fast motion) or UAV123?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Strong empirical results** FARTracktiny achieves 70.6% AO on GOT-10k at 135 FPS (GPU), outperforming closest competitors while maintaining comparable speed. Multi-platform evaluation (GPU/CPU/NPU) demonstrates practical applicability.\n- **Comprehensive ablation studies** Thorough analysis of distillation strategies, sparsification methods and design choices provide insights on the impact of distillation and other architectural choices. \n- **Efficient Distillation Strategy** Avoids manual layer assignment: The layer-by-layer self-distillation shows consistent improvements over cross-layer distillation", "weaknesses": "- **Incremental technical contribution** The core novelty is primarily an engineering combination of ARTrack with existing techniques (self-distillation and attention-based sparsification). While the application is competent and results are solid, the conceptual advance over applying standard acceleration techniques to ARTrack is limited. The paper would benefit from clearer articulation of what makes this combination non-trivial beyond implementation.\n- **Missing baseline experimental comparisons** No comparison with shallow models trained from scratch (only distilled models), which would clarify whether distillation provides benefits beyond simply reducing layers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Gq5QdxT95G", "reviewer": "Reviewer_jdqf", "summary": "This paper proposes a fast autoregressive visual tracking framework called FARTrack, aiming to achieve both high accuracy and real-time speed. Its main innovations include Task-Specific Self-Distillation and Inter-Frame Autoregressive Sparsification. In experiments, FARTrack achieved 70.6% AO performance on the GOT-10k dataset, reaching 135 FPS on GPU and maintaining 121 FPS on CPU. It significantly outperforms several state-of-the-art trackers, including AsymTrack, MixFormerV2, and HiT, in terms of speed and performance balance.", "questions": "Can you provide more sota methods resutls? And compare the performance and complexity.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The paper is well written.\n\n2.Improve the efficiency of tracker is a good direction in visual object tracking, which makes progress in this feild.\n\n3.Use Inter-frame Autoregressive Sparsification sounds new.", "weaknesses": "1.Self-Distillation is not new. The concept has been proposed by diffrenet field.\n\n2.The comparison methods are insufficient — most baselines are from earlier years, with no inclusion of 2024 works and only one paper from 2025. The paper should include more recent state-of-the-art methods for a fair and comprehensive comparison.\n\n3.No MACs and Parameters analysis with sota methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "g1I3QM09CS", "reviewer": "Reviewer_wSp9", "summary": "This paper aims to make the Autoregressive Visual Tracking method more lightweight for deployment. It presents two key technical contributions: the first is self-distillation for multiple templates, and the second is token sparsification for the tokens being processed. The self-distillation module learns trajectory features layer by layer, enabling lightweight compression and avoiding the manual layer matching required in conventional cross-layer distillation. Meanwhile, the inter-frame sparsification module filters redundant background information from the template frames at the sequence level and propagates the sparse results in an autoregressive manner, thereby avoiding additional computational overhead. The experimental results are robust and comprehensive. The comparisons to the efficient trackers show that the method is effective and achieves the SOTA results in three tracking benchmarks. The author also presents many visualization results and ablation studies to show the effectiveness of the method.", "questions": "1.In Figure 2, why is the ratio of the model’s speed to its performance used to represent the circle’s size instead of FLOPs?\n2.Is there any basis for the setting of the number of templates in Table 2?\n3.What is the basis for the REMOVE_LAYERS settings [0, 3, 6, 9, 12] and [0, 2, 4, 6] in the Deep-to-Shallow distillation?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1.FARTrack compresses the backbone via: 1) task-specific self-distillation method: each layer teaches the one below using the trajectory-token stream, 2) inter-frame autoregressive sparsification. \n2.The experimental results are promising, demonstrating strong speed and accuracy in the efficient trackers. \n3.The motivation is good. The autoregressive tracking methods need specific lightweight methods, especially considering the temporal domain. \n4.The method is straightforward and exceptionally easy to understand. It has the potential to be applied to more autocratic autoregressive model pipelines.", "weaknesses": "1.The writing could be improved. For example, the terminology used in the paper is not precise enough. The word \"performance\" tends to describe the efficiency of the model, and using \"accuracy\" would more accurately depict the tracking effect of the model.\n2.Too few models are compared on the VastTrack benchmark, and more models should be added for comparison.\n3.It is suggested to supplement the comparison of training costs between cross-layer distillation and task-specific self-distillation to more fully demonstrate the advantages of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)