Title: Dynamic Semantic-Aware Correlation Modeling for UAV Tracking
Venue: NeurIPS 2025
Accepted: yes
Keywords: UAV Tracking

Semantic Awareness

Dynamic
Abstract: UAV tracking can be widely applied in scenarios such as disaster rescue, environmental monitoring, and logistics transportation. However, existing UAV tracking methods predominantly emphasize speed and lack exploration in semantic awareness, which hinders the search region from extracting accurate localization information from the template. The limitation results in suboptimal performance under typical UAV tracking challenges such as camera motion, fast motion, and low resolution, etc. To address this issue, we propose a dynamic semantic aware correlation modeling tracking framework. The core of our framework is a Dynamic Semantic Relevance Generator, which, in combination with the correlation map from the Transformer, explore semantic relevance. The approach enhances the search region's ability to extract important information from the template, improving accuracy and robustness under the aforementioned challenges.
Additionally, to enhance the tracking speed, we design a pruning method for the proposed framework. Therefore, we present multiple model variants that achieve trade-offs between speed and accuracy, enabling flexible deployment according to the available computational resources. Experimental results validate the effectiveness of our method, achieving competitive performance on multiple UAV tracking datasets.

Dynamic Semantic-Aware Correlation Modeling for UAV Tracking

Abstract

UAV tracking can be widely applied in scenarios such as disaster rescue, environmental monitoring, and logistics transportation. However, existing UAV tracking methods predominantly emphasize speed and lack exploration in semantic awareness, which hinders the search region from extracting accurate localization information from the template. The limitation results in suboptimal performance under typical UAV tracking challenges such as camera motion, fast motion, and low resolution, etc. To address this issue, we propose a dynamic semantic aware correlation modeling tracking framework. The core of our framework is a Dynamic Semantic Relevance Generator, which, in combination with the correlation map from the Transformer, explore semantic relevance. The approach enhances the search region's ability to extract important information from the template, improving accuracy and robustness under the aforementioned challenges. Additionally, to enhance the tracking speed, we design a pruning method for the proposed framework. Therefore, we present multiple model variants that achieve trade-offs between speed and accuracy, enabling flexible deployment according to the available computational resources. Experimental results validate the effectiveness of our method, achieving competitive performance on multiple UAV tracking datasets. The code is available at https://github.com/zxyyxzz/DSATrack.

1 Introduction

Visual object tracking[55, 9, 29, 28, 59] is a fundamental and important task in computer vision. In recent years, unmanned aerial vehicles (UAV) technology has rapidly developed and demonstrated broad application in various fields such as military, transportation, environmental monitoring, and disaster relief [43]. However, to achieve autonomous task execution in complex environments, object tracking is a crucial component. UA V tracking must address various challenges in dynamic scenarios.

The early UAV tracking methods [36, 31, 35, 27] employed a correlation filter to enhance the discriminative capability, enabling location and tracking. However, these correlation filter-based methods lacked robust representation capabilities. Consequently, UAV tracking methods based on convolutional neural networks (CNNs) [4, 49, 52] emerged, utilizing Siamese network to enhance feature representation, significantly improving tracking accuracy. With the rise of Transformers [55, 11, 6, 51], UAV tracking based on one-stream framework [37, 34, 32, 21] have been studied. Most of the methods focus on improving the efficiency of UA V tracking, while neglecting exploration

#0720

#0835

#1243

#0057

#0096

#0126

HiFT

TCTrack

AVTrack

Ours

#0133

#0037

#0223

#0236

#0390

#0179

103

S

口

EIE

42.1C

of the essence of tracking. Correlation filter-based methods and CNN-based methods rely on local modeling, while Transformer-based methods utilize global modeling. All of the approaches overlook the relevance between semantic features, as shown in Figure 1, leading to suboptimal performance in UAV scenarios, such as camera motion, fast motion, and low resolution, etc.

To address this issue, we propose a dynamic semantic-aware correlation modeling method and develop a UAV tracking framework based on the approach, named DSATrack. Specifically, we use the search region and template&patches as inputs and employ Transformer blocks to achieve semantic matching, semantic awareness, and feature extraction and fusion. Importantly, we design a Dynamic Semantic Relevance Generator within the Transformer to dynamically achieve Semantic Aware Modeling . By leveraging the Semantic Aware Modeling to fuse similar semantic features, our method alleviates semantic ambiguities. The semantic-aware correlation map further facilitates the search region in extracting representative features from the template&patches, thereby enhancing the perception and localization capability of UAV tracking and mitigating tracking drift. Finally, to improve the tracking efficiency, we design a pruning technique tailored to the characteristics of DSATrack. By selectively pruning Transformer blocks, we retain the tracking precision to the greatest extent while achieving high speed tracking. We provides multiple variants for different UA V Tracking application scenarios. For instance, offline tasks such as aerial video prioritize tracking accuracy, while real-time applications like autonomous target following demand fast response times.

Our contributions can be summarized as follows.

We propose a dynamic semantic-aware UAV tracking framework, DSATrack, which utilizes a Dynamic Semantic Relevance Generator to produce semantic-aware correlation map. These maps are used to fuse features with similar semantic features, effectively reducing tracking drift.

·To further optimize DSATrack, we design a pruning technique that selectively removes Transformer blocks with minimal impact on accuracy. This approach achieves a balance between precision and speed, enabling flexible deployment according to the available computational resources.

·We conduct extensive experiments, demonstrating that DSATrack achieves state-of-the-art performance. Additionally, ablation studies validate the effectiveness of the proposed methods.

2 Related Works

2.1 UAV Tracking

Correlation filter-based methods, such as KCF [25] and DSST [13], were among the earliest applied to UAV tracking due to their high efficiency, achieving strong performance in speed and accuracy.

However, they struggle under camera motion or fast motion, which are common in UAV scenarios. With the rise of deep learning, Siamese network-based trackers-such as TransT [9], AiATrack [19], STARK [53], ToMP [41], and KeepTrack [42]-have significantly improved tracking robustness. TCTrack [4] further adapts Siamese models to UAV tracking. Yet, their heavy computational demands hinder real-time UAV deployment. Recently, one-stream Transformer frameworks [55, 11, 8, 6, 1, 54, 20] have emerged, offering enhanced modeling of long-range dependencies with improved efficiency. Trackers like TATrack [33] and AVTrack [37] utilize this framework to better capture interactions between template and search regions under UAV-specific challenges. Although LoReTrack [14] is not tailored for UA V tracking and lacks real-world UAV evaluation, it demonstrates promising results on a UAV dataset using a lightweight one-stream design. In contrast, we propose a Dynamic Semantic-Aware Correlation Modeling approach within a one-stream Transformer framework to address key UAV challenges such as camera motion, fast motion, and low resolution.

2.2 Semantic Correspondence

In visual perception tasks, semantic correspondence is typically categorized into two types: explicit correspondence and implicit correspondence. Explicit correspondence involves semantic-aware modeling through supervised loss functions, while implicit correspondence does not rely on direct supervision for semantic correspondence. Explicit correspondence methods, such as contrastive learning [7, 24], Triplet Loss [46], and InfoNCE [45], supervise the semantic relationships between images or pixels by pulling closer those belonging to the same category and pushing apart those of different categories. This enables models to learn semantic relationships, improving the accuracy of downstream tasks [22, 58]. On the other hand, implicit correspondence is commonly employed in tasks like classification [40], detection [61], segmentation [10] and Tracking [19, 9] using Transformerbased architectures [48, 15]. These architectures implicitly learn semantic relationships among image features through task-specific losses. Specifically, Transformers compute a correlation map using queries and keys [5, 47], where the correlation map represents the semantic similarity between features [12]. However, in implicit semantic correspondence methods, there is limited exploration into explicitly optimizing semantic correspondence, which often results in suboptimal modeling of the correlation map. This lack of optimization can hinder the model's ability to capture semantic relationships. Therefore, we propose a dynamic semantic-aware correlation modeling method for UAV tracking to enhance semantic perception capabilities, improving tracking accuracy.

3 Method

In this section, we will first present our proposed tracking framework, DSATrack. Next, we will describe the Dynamic Semantic-aware Transformer. Following that, we will detail the pruning method designed specifically for DSATrack. Finally, we will present description of the prediction head.

3.1 Overall Framework

The overall framework of the proposed DSATrack is shown in Figure 2. The entire tracking framework consists of three components: input representation, feature matching and interaction, and prediction.

Input Representation . The input of DSATrack is the Template & Patches z ∈ R n × 3 × H z × W z and Search Region x ∈ R 3 × H x × W x , where H and W represent the height and width of the Template and Search Region. The Template & Patches consist of a fixed template ∈ R 1 × H z × W z and several patches ∈ R ( n -1) × H z × W z that exhibit representative object characteristics. The Patches are generated and updated over time steps in exactly the same manner as RFGM [60]. Subsequently, the inputs are projected into tokens through a Patch Embedding. Feature matching and Interaction . The generated Template Tokens T z ∈ R n × 3 × h z × w z and Search Tokens T x ∈ R n × 3 × h x × w x are then fed into the Transformer Blocks for feature matching and interaction, where h and w represent the height and width of the features after a 16 × downsampling. The specific computation method of the Transformer block is consistent with OSTrack [55]. Importantly, we propose a Dynamic Semantic-aware Transformer for semantic matching and feature fusion. Specifically, we designed a Dynamic Semantic Relevance Generator to construct Semantic Relevance. Using the Semantic Aware Modeling, we fuse the similarities of features with similar semantics to obtain an optimized Semantic-Aware Correlation Map. Based on this correlation map, unimportant template tokens are discarded. The filtered Template Tokens, Search Tokens, and the Semantic-Aware Correlation Map

are then fed into a Hybrid Attention module to facilitate feature interaction. Prediction . Finally, the Search Tokens are reassembled into an image and passed to the prediction head, which predicts the bounding box of the object.

Semantic

Aware

Modeling

…

Transformer block

Dynamic Semantic-aware Transformer block

Transformer block

Search Region

Dynamic Semantic

Correlation Map

MLP

Gumbel-SoftMax

Hybrid Attention

FFN

Dynamic Semantic-aware Transformer block

Search

Tokens

Template

Tokens

Token Dropping

Search Region

Template

&

Patches

Semantic Relevance

ℎ𝑧

𝑤𝑧

𝐾𝑧

ℎ𝑥

𝑤𝑥

𝐾

𝑄

Semantic-aware Correlation Map

𝐾𝑧

𝑄𝑥

𝑤𝑥

ℎ𝑥

ℎ𝑧

𝑤𝑧

Correlation Map

ℎ𝑧

𝑤𝑧

𝐾𝑧

𝑄𝑥

ℎ𝑥

𝑤𝑥

Gumbel-SoftMax

Semantic

Relevance

Self Attention

Self Attention

Cross Attention

Semantic-aware Correlation Map

𝐾𝑧

𝑄𝑥

𝑤𝑥

ℎ𝑥

ℎ𝑧

𝑤𝑧

Concatenation

Dynamic Semantic Relevance

Generator

Hybrid Attention

Prediction Head

UAV

Filtered Template Tokens

(ℎ𝑧 , 𝑤

𝑧

, ℎ

𝑥 , 𝑤

𝑥 , 𝑙)

(𝑁𝑧, 𝑁 𝑧 )

(𝑁𝑧, 𝑁 z )

(ℎ𝑧 , 𝑤

𝑧

, ℎ

𝑥 , 𝑤

𝑥 , 𝑙)

𝑄𝑥

A

MLP

AvgPooling

一一

1

3.2 Dynamic Semantic-aware Transformer

The Template Tokens T z ∈ R n × h z × w z × d k and Search Tokens T x ∈ R 1 × h x × w x × d k are inputs of Dynamic Semantic-aware Transformer, where d k is the dimensionality of the features. For simplicity and clarity, we present the details using single template, T z ∈ R 1 × h z × w z × d k . We use a linear layer to transform the Search Tokens and Template Tokens into ( Q x ∈ R h x × w x × d k ) and ( K z ∈ R h z × w z × d k ) . As shown in Figure 2, we compute the Correlation Map C between the Q x and K z as:

where C zx captures the correlation scores between query from x and key from z , corresponding to the z -tox similarity. l represents the number of heads in the Transformer.

In the Correlation Map C zx , the modeling of the similarity relationship between points on Q x and points on K z can easily lead to ambiguity. We employ a dynamic perception-based prediction approach to calculate and fuse the similarity between semantically related points on K z and Q x .

Dynamic Semantic Relevance Generator . We first reshape C zx ∈ R h x × w x × h z × w z × l into C zx ∈ R N x × N z × l , where N x = h x × w x and N z = h z × w z . Then, We propose a Semantic Aware Modeling G = ( V , E ) , where V = { ⃗ c i | ⃗ c i ∈ C zx , 1 ≤ i ≤ N z } represents the set of nodes and the shape of ⃗ c i is N x × 1 × l . In other words, we retrieve nodes along the second dimension of C zx . E = { ( u, v ) | ∀ u ∈ V , ∀ v ∈ V} represents the set of Semantic Relevance. Specifically, we apply average pooling to the node features and we concatenate the vector c ⃗ i ′ to form ⃗ v :

where [ · ] represents concatenation.

Subsequently, we compute the similarity score A between nodes via matrix multiplication:

After that, a tiny prediction network consisting of a multi-layer perceptron(MLP) and Log-Softmax function is utilized to predict the possibility that each Relevance will be retained and discarded:

where π ij, 0 is the probability of retaining the Relevance ( i, j ) and π ij, 1 is probability of removing it.

Next, we apply the Gumbel-Softmax to produce the Dynamic Semantic Relevance E , making the π differentiable and retaining the most relevant connections:

Semantic Aware Modeling . Therefore, we utilize the Semantic Aware Modeling to optimize the Correlation Map, as defined by the following formula:

where ˆ E = E + I , I is the identity matrix, and Λ ii = ∑ j E ij is the degree matrix.

Hybrid Attention . The Semantic-aware Correlation Map is fed into an MLP and Gumbel-softmax to evaluate the importance of the Template Tokens. Less significant tokens are discarded based on M∈{ 0 , 1 } ∈ R N z × N z to accelerate computation:

Subsequently, we combine Q z , Q x , K z , K x , V z , V x , and C ′ zx to compute the Hybrid Attention:

The formula for Self-attention is as follows:

Cross-Attention can be calculated as follows:

Finally, we incorporated a Feedforward Neural Network (FFN) to enhance the model's fitting ability. The Dynamic Semantic-aware transformer can be summarized as follows:

where T xz represents the concatenation of T z and T x .

3.3 Hierarchical Contribution Ranking Pruning

We introduce the hierarchical contribution metric δ i , which quantifies the role of each Transformer block in the feature transformation process of DSATrack. The definition of hierarchical contribution is based on the similarity between the hidden state of each layer and its successor. If the input of a layer has a high similarity with the input of the next layer, it indicates that the feature transformation of this layer has a minimal effect on the subsequent layer's input, resulting in a lower contribution. Conversely, if the similarity is low, it suggests that this layer plays a significant role.

The calculation formula for the hierarchical contribution δ i of the i -th Transformer layer is as follows:

where T i,m is the m -th hidden state of i -th Transformer block, and M is the number of hidden states.

The model consists of 12 Transformer blocks, with two types: Dynamic Semantic-Aware Transformer blocks located at layers D = { 4 , 7 , 10 } , and Standard Transformer blocks in the remaining layers S = { 1 , 2 , 3 , 5 , 6 , 8 , 9 , 11 , 12 } . For the Dynamic Semantic-Aware Transformer blocks and the Standard Transformer blocks, we first sort the contribution scores δ i separately for each type of block:

Based on these sorted values, we select the lowest contributing blocks for pruning. The pruning sets for each type of block are determined by pruning ratios p d and p s as follows:

These pruned blocks are then discarded, and the remaining blocks from both sets are combined to form the new pruned model. The remaining Dynamic Semantic-Aware Transformer blocks and Standard Transformer blocks are:

Finally, the new model's layer set after pruning is:

This process allows for a balance between model efficiency and performance, with the pruning ratio p d and p s adjustable to optimize computational speed and storage requirements.

3.4 Prediction Head

Following OSTrack [55], we construct our prediction head to estimate the bounding box from the transformer-generated search region features. The head consists of three branches: one predicts the classification score map S ∈ [0 , 1] h x × w x , another predicts the offset O ∈ [0 , 1) 2 × h x × w x , and the third predicts the normalized box size E ∈ [0 , 1] 2 × h x × w x . The location with the highest score, ( x d , y d ) = arg max ( x,y ) S xy , is combined with O and E to generate the final bounding box.

4 Experiments

4.1 Implementation Details

Model . DSATrack employs the ViT-B [15] as the backbone, initialized with MAE [23], referred to as DSATrack-ViT-B. For the pruned variants of DSATrack, we consider four configurations with 8, 7, 6, and 4 transformer layers, named DSATrack-D8, DSATrack-D7, DSATrack-D6, and DSATrack-D4. All pruned models apply the pruning ratio p d = 2 3 . The pruning ratios p s are set to 2 9 , 3 9 , 4 9 and 6 9 for the four variants. Table 1 presents the hierarchical contribution of DSATrack-ViT-B transformer layers on UAV123 , yielding the layer removal setup for pruned model variant, as shown in Table 2. In addition, we provide versions based on ViT-Tiny and DeiT-Tiny, named DSATrack-ViT-T and DSATrack-DeiT-T. Templates are resized to 128 × 128 , while Search Region are resized to 256 × 256 .

Training . We train the model on TrackingNet [44], GOT-10k [26], LaSOT [17], and COCO [39]. The training process consists of two stages. In the first stage, the model is trained for 300 epochs with 3 templates. The learning rate for the Dynamic Semantic-aware Transformer and prediction head is set to 4 × 10 -4 , the learning rate for the remaining parameters is set to 4 × 10 -5 . At epoch 240, the learning rate is decayed by a factor of 10. In the second stage, the model is finetuned for 50 epochs with 6 templates. AdamW optimizer is used with a weight decay of 10 -4 . The loss function comprises the L1 loss, Focal loss, GIoU loss and ratio loss, consistent with RFGM [60]. All training tasks are conducted on 4 NVIDIA GeForce RTX 3090 GPUs with a batch size of 24.

Inference . For fairness, the speed tests of other models and our model are conducted on the same NVIDIA RTX 3090 GPU without data loading overhead. The size of our Templates & Patches is set to 3 × N z . The update interval of Patches is set to 5 for t ≤ 100, doubled every 100 frames until t = 500, and then remains 160. The interval doubles every 100 frames until it reaches 160, where it remains fixed beyond 160 frames. At the 4th, 7th, and 10th layers, the number of retained template tokens is set to ⌊ 3 × N z × 0 . 9 ⌋ , ⌊ 3 × N z × 0 . 8 ⌋ , and ⌊ 3 × N z × 0 . 7 ⌋ , ⌊·⌋ denotes the floor operation.

4.2 State-of-the-Art Comparisons

Figure 5 presents a comparison of DSATrack speed and accuracy against the SOTA methods.

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision plots of OPE

-

Camera Motion

[0.935] DSATrack-ViT-B

[0.913] DSATrack-D7

[0.909] DSATrack-D8

[0.900] DSATrack-D6

[0.898] Aba-ViTrack

[0.892] AVTrack

[0.885] DSATrack-ViT-T

[0.862] DSATrack-D4

[0.856] HiT

[0.799] HiFT

[0.797] TCTrack

[0.794] SiamCAR

[0.772] SiamAPN++

[0.769] SiamAPN

[0.676] ECO-HC

[0.658] AutoTrack

[0.647] ARCF

[0.639] BACF

[0.520] DSST

[0.518] fDSST

[0.483] KCF

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision  plo ts  of  OPE

-

Fast Motion

[0.898] DSATrack-ViT-B

[0.842] DSATrack-ViT-T

[0.835] DSATrack-D7

[0.833] Aba-ViTrack

[0.826] AVTrack

[0.825] DSATrack-D6

[0.820] DSATrack-D8

[0.792] HiT

[0.790] DSATrack-D4

[0.778] HiFT

[0.763] SiamCAR

[0.757] TCTrack

[0.756] SiamAPN

[0.736] SiamAPN++

[0.537] ECO-HC

[0.534] AutoTrack

[0.518] BACF

[0.515] ARCF

[0.367] DSST

[0.362] fDSST

[0.296] KCF

(a) UAV123

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

Precision

Precision  plots  of  OPE

-

Low Resolution

[0.818] DSATrack-ViT-B

[0.758] DSATrack-D8

[0.749] DSATrack-D6

[0.739] Aba-ViTrack

[0.731] AVTrack

[0.725] DSATrack-ViT-T

[0.722] DSATrack-D7

[0.721] SiamCAR

[0.717] DSATrack-D4

[0.679] TCTrack

[0.661] HiT

[0.657] ECO-HC

[0.655] HiFT

[0.637] SiamAPN

[0.620] SiamAPN++

[0.588] AutoTrack

[0.565] ARCF

[0.520] BACF

[0.475] DSST

[0.464] fDSST

[0.381] KCF

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision plots of OPE

-

Camera Motion

[0.897] DSATrack-D8

[0.877] DSATrack-D6

[0.876] DSATrack-D4

[0.870] DSATrack-D7

[0.865] AVTrack

[0.863] DSATrack-ViT-T

[0.858] DSATrack-ViT-B

[0.856] Aba-ViTrack

[0.816] SiamCAR

[0.806] SiamRPN

[0.804] SiamAPN

[0.800] ECO-HC

[0.799] ARCF

[0.793] AutoTrack

[0.791] P-SiamFC++

[0.785] TCTrack

[0.773] BACF

[0.762] DSST

[0.722] fDSST

[0.708] SiamAPN++

[0.700] KCF

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision  plots   of  OP E

-

Fast Motion

[0.884] DSATrack-D8

[0.860] DSATrack-D4

[0.859] DSATrack-D6

[0.842] SiamAPN

[0.834] Aba-ViTrack

[0.827] AVTrack

[0.820] DSATrack-D7

[0.800] SiamCAR

[0.771] P-SiamFC++

[0.768] SiamRPN

[0.765] DSATrack-ViT-T

[0.759] SiamAPN++

[0.745] DSATrack-ViT-B

[0.744] AutoTrack

[0.738] TCTrack

[0.728] ARCF

[0.679] ECO-HC

[0.575] fDSST

[0.573] BACF

[0.568] KCF

[0.510] DSST

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision  plots  of  OPE

-

Low Resolution

[0.872] DSATrack-D7

[0.868] DSATrack-D8

[0.851] P-SiamFC++

[0.811] DSST

[0.807] SiamRPN

[0.806] DSATrack-D4

[0.786] SiamCAR

[0.771] DSATrack-ViT-T

[0.770] SiamAPN

[0.768] DSATrack-D6

[0.753] ARCF

[0.751] Aba-ViTrack

[0.748] fDSST

[0.745] DSATrack-ViT-B

[0.738] AVTrack

[0.706] AutoTrack

[0.644] TCTrack

[0.642] KCF

[0.598] BACF

[0.588] SiamAPN++

[0.547] ECO-HC

(b) VisDrone

2018

DTB70 . DSATrack outperformed SOTA methods. Specifically, DSATrack-ViT-B attained the highest precision (91.2%) and success (70.6%). Notably, even the lightweight variants, DSATrack-D8 and DSATrack-D7, achieved second (89.4% precision) and third place (88.1% precision), demonstrating the effectiveness of our method across different model complexities. DSATrack-VIT-T acquired 86.6% in Precison and 67.4 % in on success, which is outperform the sota method by 2% to 3%. The results highlight the robustness of dynamic semantic-aware correlation modeling in UAV tracking.

UAVDT . DSATrack-ViT-B achieved SOTA results with precision and success of 88.1% and 66.3%. DSATrack-D8 also achieved competitive performance, ranking second in precision (85.7%) and achieving the third-best success (63.4%). Importantly, DSATrack-D7 exhibited strong success metrics (62.1%), outperforming prior works such as TATrack and AVTrack. Besides, DSATrack-ViT-T and DSATrack-DeiT-T achieved better performance than SOTA methods, such as AVTrack and LightFC. These results underscore the scalability of DSATrack across varying depths and complexities.

VisDrone2018 . DSATrack-D8 and DSATrack-ViT-B achieving the top two rankings for success, 69.1% and 67.0%. D7, ViT-T and DeiT-T achieved competitive results, validating the efficacy of our framework in challenging conditions. Compared to existing SOTA methods, DSATrack consistently presented superior semantic-aware modeling, reflecting its adaptability to UA V scenarios. Moreover, the second row of the Figure 3 demonstrates that the pruned version of DSATrack also exhibits significant advantages in addressing the representative challenges of UA V tracking.

UAV123 . DSATrack-ViT-B set a new performance of 90.2% (precsion) and of 69.4% (success), surpassing existing SOTA trackers. DSATrack-D8/D7 maintaining competitive scores of 87.6% (precision) and 87.3% (precision). Furthermore, D4, ViT-T and DeiT-T achieved satisfactory accuracy, making it suitable for real-time UA V applications. As shown in the first row of the Figure 3, DSATrack exhibits better performance in addressing challenges such as camera motion, fast motion, and low resolution. This demonstrates the flexibility of DSATrack in balancing efficiency and accuracy.

4.3 Ablation Study on Correlation Modeling

Fast Motion

Camera Motion

Low Resolution

group2_3

(1a)

(1b)

(1c)

(2a)

(2b)

(2c)

wakeboard5

Fast Motion

Camera Motion

Low Resolution

uav0000242_05160_s

uav0000367_04137_s

uav0000093_01817_s

100

120

140

160

180

200

220

Speed (FPS)

70

75

80

85

90

Precision (%)

Precision  vs Speed

HiT-Base

LiteTrack

AVTrack-ViT

AVTrack-DeiT

DSATrack-ViT-B

DSATrack-D8

DSATrack-D7

DSATrack-D6

DSATrack-D4

DSATrack-ViT-T

DSATrack-DeiT-T

We evaluate our Dynamic Semantic-Aware Modeling (DSAM) against the Baseline and Static Semantic-Aware Modeling (SSAM). Dynamic semantic modeling refers to the dynamic semantic prediction method proposed in this paper, whereas static semantic modeling denotes the approach that applies a 3 × 3 convolution kernel to the correlation maps from the search region to the template and from the template to the search region, respectively. In this case, the receptive field remains fixed in shape. On DTB70, DSAM with ViT-B achieves 91.% precision and 70.6% success, outperforming both alternatives. Similar trends are observed on UAVDT and UAV123. Unlike SSAM's fixedregion modeling, DSAM adaptively captures semantic variability, enhancing tracking robustness. As shown in Figure 4, DSAM yields more focused correlation maps, effectively reducing background noise. These results confirm that DSAM significantly improves UAV tracking by modeling dynamic semantic correlations more accurately than static or baseline approaches.

4.4 Ablation Study on Pruning

Sequential Pruning (SP), similar to the proposed Contribution Ranking Pruning (CRP), removes layers 4 and 7 while retaining layer 10. However, SP performs pruning in a fixed sequential order, without explicitly evaluating the relative contribution of each layer to the overall tracking performance. In contrast, CRP introduces a hierarchical contribution ranking mechanism that quantifies each layer's importance before pruning, ensuring that critical layers are preserved while redundant ones are removed.

As shown in Table 5, CRP consistently outperforms SP across all backbones and datasets. For the D7 variant, CRP achieves higher precision and success rates on all four benchmarks (DTB70, UAVDT, VisDrone2018, and UAV123) with negligible changes in speed. The performance gap becomes even more pronounced in shallower configurations such as D4, where SP suffers from severe accuracy degradation due to over-pruning of essential layers, while CRP maintains stable accuracy with a similar inference speed (214.1 FPS vs. 210.7 FPS).

These results demonstrate that contribution-aware layer selection enables CRP to achieve a better balance between accuracy and computational efficiency. By adaptively ranking layers based on their hierarchical importance rather than following a rigid pruning schedule, CRP achieves a more robust pruning strategy, especially on challenging datasets such as VisDrone2018 and UAV123, where fine-grained spatial details are crucial for tracking performance.

5 Real Word UAV Test

As illustrated in Figure 6, we conduct a series of real-world UA V tracking experiments to further verify the practicality and deployment feasibility of the proposed DSATrack framework. Four representative tracking scenarios are presented, involving diverse targets such as a car, a motorbike, a dog, and a person, covering both rigid and non-rigid motion patterns as well as varying motion speeds and background complexities. These experiments were performed using a Jetson AGX Xavier edge computing platform to emulate realistic onboard conditions with limited computational resources.

To assess the runtime performance, we evaluate the inference speed of different DSATrack variants on the Xavier platform. The lightweight transformer-based variants, DSATrack-ViT-T and DSATrackDeiT-T, achieve real-time tracking speeds of 27.6 FPS and 27.4 FPS, respectively, demonstrating their

suitability for low-latency applications on embedded hardware. In contrast, larger pruned variants such as DSATrack-D4 and DSATrack-D7 run at approximately 10 FPS, offering a stronger accuracy margin when higher computational capacity is available.

These results highlight the adaptability of DSATrack to diverse deployment scenarios. Depending on the trade-off between accuracy and efficiency, users can flexibly select the appropriate model variant to meet the demands of specific UAV task. The consistent performance across both dynamic and static scenes further confirms the robustness and scalability of the proposed framework in real-world UAV tracking applications.

Car

Car

Car

Car

Girl

Girl

Girl

Girl

Dog

Dog

Dog

Dog

Car

Car

T

6 Conclusion and Limitation

The paper presents DSATrack, a dynamic semantic-aware UAV tracking framework addressing typical UAV Tracking challenges. It incorporates a Dynamic Semantic Relevance Generator to enhance feature fusion and localization, effectively reducing tracking drift. Additionally, a Hierarchical Contribution Ranking Pruning optimizes DSATrack by balancing tracking precision and real-time. Extensive experiments show that DSATrack achieves SOTA performance across various datasets, while ablation studies further validate the effectiveness. DSATrack also has certain limitations. Similar to most trackers, DSATrack is still a local tracker and lacks re-detection capability once the object is lost. In future work, we will explore global UA V tracking algorithms.

Acknowledgement This work was supported by National Natural Science Foundation of China (No.62576109, 62072112.

References

Yidong Cai, Jie Liu, Jie Tang, and Gangshan Wu. Robust object modeling for visual tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 9589-9600, 2023.

Ziang Cao, Changhong Fu, Junjie Ye, Bowen Li, and Yiming Li. Hift: Hierarchical feature transformer for aerial tracking. In Proceedings of the IEEE/CVF international conference on computer vision , pages 15457-15466, 2021.

Ziang Cao, Changhong Fu, Junjie Ye, Bowen Li, and Yiming Li. Siamapn++: Siamese attentional aggregation network for real-time uav tracking. In 2021 IEEE/RSJ international conference on intelligent robots and systems (IROS) , pages 3086-3092. IEEE, 2021.

Ziang Cao, Ziyuan Huang, Liang Pan, Shiwei Zhang, Ziwei Liu, and Changhong Fu. Tctrack: Temporal contexts for aerial tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 14798-14808, 2022.

Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part I 16 , pages 213-229. Springer, 2020.

Boyu Chen, Peixia Li, Lei Bai, Lei Qiao, Qiuhong Shen, Bo Li, Weihao Gan, Wei Wu, and Wanli Ouyang. Backbone is all your need: a simplified architecture for visual object tracking. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXII , pages 375-392. Springer, 2022.

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning , pages 1597-1607. PMLR, 2020.

Xin Chen, Houwen Peng, Dong Wang, Huchuan Lu, and Han Hu. Seqtrack: Sequence to sequence learning for visual object tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14572-14581, 2023.

Xin Chen, Bin Yan, Jiawen Zhu, Dong Wang, Xiaoyun Yang, and Huchuan Lu. Transformer tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8126-8135, 2021.

Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar. Maskedattention mask transformer for universal image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 1290-1299, 2022.

Yutao Cui, Cheng Jiang, Limin Wang, and Gangshan Wu. Mixformer: End-to-end tracking with iterative mixed attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13608-13618, 2022.

Zhigang Dai, Bolun Cai, Yugeng Lin, and Junying Chen. Up-detr: Unsupervised pre-training for object detection with transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 1601-1610, 2021.

Martin Danelljan, Gustav Häger, Fahad Khan, and Michael Felsberg. Accurate scale estimation for robust visual tracking. In British machine vision conference, Nottingham, September 1-5, 2014 . Bmva Press, 2014.

Shaohua Dong, Yunhe Feng, Qing Yang, Yuewei Lin, and Heng Fan. Loretrack: efficient and accurate low-resolution transformer tracking. arXiv preprint arXiv:2405.17660 , 2024.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020.

Guocai Du, Peiyong Zhou, Nurbiya Yadikar, Alimjan Aysa, and Kurban Ubul. Ddctrack: Dynamic token sampling for efficient uav transformer tracking. In International Conference on Pattern Recognition , pages 129-144, 2024.

Heng Fan, Hexin Bai, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Mingzhen Huang, Juehuan Liu, Yong Xu, et al. Lasot: A high-quality large-scale single object tracking benchmark. International Journal of Computer Vision , 129(2):439-461, 2021.

Changhong Fu, Ziang Cao, Yiming Li, Junjie Ye, and Chen Feng. Onboard real-time aerial tracking with efficient siamese anchor proposal network. IEEE Transactions on Geoscience and Remote Sensing , 60:1-13, 2021.

Shenyuan Gao, Chunluan Zhou, Chao Ma, Xinggang Wang, and Junsong Yuan. Aiatrack: Attention in attention for transformer visual tracking. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXII , pages 146-164. Springer, 2022.

Shenyuan Gao, Chunluan Zhou, and Jun Zhang. Generalized relation modeling for transformer tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1868618695, 2023.

Goutam Yelluru Gopal and Maria A Amer. Separable self and mixed attention transformers for efficient object tracking. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 6708-6717, 2024.

Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06) , volume 2, pages 1735-1742. IEEE, 2006.

Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16000-16009, 2022.

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 9729-9738, 2020.

João F Henriques, Rui Caseiro, Pedro Martins, and Jorge Batista. High-speed tracking with kernelized correlation filters. IEEE transactions on pattern analysis and machine intelligence , 37(3):583-596, 2014.

Lianghua Huang, Xin Zhao, and Kaiqi Huang. Got-10k: A large high-diversity benchmark for generic object tracking in the wild. IEEE Transactions on Pattern Analysis and Machine Intelligence , 43(5):15621577, 2019.

Ziyuan Huang, Changhong Fu, Yiming Li, Fuling Lin, and Peng Lu. Learning aberrance repressed correlation filters for real-time uav tracking. In Proceedings of the IEEE/CVF international conference on computer vision , pages 2891-2900, 2019.

Kaixun Jiang, Zhaoyu Chen, Jiyuan Fu, Lingyi Hong, Jinglun Li, and Wenqiang Zhang. Videopure: Diffusion-based adversarial purification for video recognition. IEEE Transactions on Circuits and Systems for Video Technology , 2025.

Kaixun Jiang, Zhaoyu Chen, Hao Huang, Jiafeng Wang, Dingkang Yang, Bo Li, Yan Wang, and Wenqiang Zhang. Efficient decision-based black-box patch attacks on video recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4379-4389, 2023.

Ben Kang, Xin Chen, Dong Wang, Houwen Peng, and Huchuan Lu. Exploring lightweight hierarchical vision transformers for efficient visual tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 9612-9621, 2023.

Shui-wang Li, Qian-bo Jiang, Qi-jun Zhao, Li Lu, and Zi-liang Feng. Asymmetric discriminative correlation filters for visual tracking. Frontiers of Information Technology & Electronic Engineering , 21(10):14671484, 2020.

Shuiwang Li, Xiangyang Yang, Xucheng Wang, Dan Zeng, Hengzhou Ye, and Qijun Zhao. Learning target-aware vision transformers for real-time uav tracking. IEEE Transactions on Geoscience and Remote Sensing , 62:1-18, 2024.

Shuiwang Li, Xiangyang Yang, Xucheng Wang, Dan Zeng, Hengzhou Ye, and Qijun Zhao. Learning target-aware vision transformers for real-time uav tracking. IEEE Transactions on Geoscience and Remote Sensing , 2024.

Shuiwang Li, Yangxiang Yang, Dan Zeng, and Xucheng Wang. Adaptive and background-aware vision transformer for real-time uav tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 13989-14000, 2023.

Shuiwang Li, Qijun Zhao, Ziliang Feng, and Li Lu. Equivalence of correlation filter and convolution filter in visual tracking. In Image and Graphics: 11th International Conference, ICIG 2021, Haikou, China, August 6-8, 2021, Proceedings, Part III 11 , pages 623-634. Springer, 2021.

Yiming Li, Changhong Fu, Fangqiang Ding, Ziyuan Huang, and Geng Lu. Autotrack: Towards highperformance visual tracking for uav with automatic spatio-temporal regularization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 11923-11932, 2020.

Yongxin Li, Mengyuan Liu, You Wu, Xucheng Wang, Xiangyang Yang, and Shuiwang Li. Learning adaptive and view-invariant vision transformer for real-time uav tracking. In Forty-first International Conference on Machine Learning .

Yunfeng Li, Bo Wang, Xueyi Wu, Zhuoyan Liu, and Ye Li. Lightweight full-convolutional siamese tracker. Knowledge-Based Systems , 286:111439, 2024.

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision , pages 740-755. Springer, 2014.

Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision , pages 10012-10022, 2021.

Christoph Mayer, Martin Danelljan, Goutam Bhat, Matthieu Paul, Danda Pani Paudel, Fisher Yu, and Luc Van Gool. Transforming model prediction for tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8731-8740, 2022.

Christoph Mayer, Martin Danelljan, Danda Pani Paudel, and Luc Van Gool. Learning target candidate association to keep track of what not to track. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 13444-13454, 2021.

Matthias Mueller, Neil Smith, and Bernard Ghanem. A benchmark and simulator for uav tracking. In European conference on computer vision , pages 445-461. Springer, 2016.

Matthias Muller, Adel Bibi, Silvio Giancola, Salman Alsubaihi, and Bernard Ghanem. Trackingnet: A large-scale dataset and benchmark for object tracking in the wild. In Proceedings of the European conference on computer vision (ECCV) , pages 300-317, 2018.

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 , 2018.

Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 815-823, 2015.

Peize Sun, Rufeng Zhang, Yi Jiang, Tao Kong, Chenfeng Xu, Wei Zhan, Masayoshi Tomizuka, Lei Li, Zehuan Yuan, Changhu Wang, et al. Sparse r-cnn: End-to-end object detection with learnable proposals. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 14454-14463, 2021.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017.

Xucheng Wang, Dan Zeng, Qijun Zhao, and Shuiwang Li. Rank-based filter pruning for real-time uav tracking. In 2022 IEEE International Conference on Multimedia and Expo (ICME) , pages 01-06. IEEE, 2022.

Qingmao Wei, Bi Zeng, Jianqi Liu, Li He, and Guotian Zeng. Litetrack: Layer pruning with asynchronous feature extraction for lightweight and efficient visual tracking. In 2024 IEEE International Conference on Robotics and Automation (ICRA) , pages 4968-4975. IEEE, 2024.

Xing Wei, Yifan Bai, Yongchao Zheng, Dahu Shi, and Yihong Gong. Autoregressive visual tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9697-9706, 2023.

Wanying Wu, Pengzhi Zhong, and Shuiwang Li. Fisher pruning for real-time uav tracking. In 2022 International Joint Conference on Neural Networks (IJCNN) , pages 1-7. IEEE, 2022.

Bin Yan, Houwen Peng, Jianlong Fu, Dong Wang, and Huchuan Lu. Learning spatio-temporal transformer for visual tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10448-10457, 2021.

Dawei Yang, Jianfeng He, Yinchao Ma, Qianjin Yu, and Tianzhu Zhang. Foreground-background distribution modeling transformer for visual object tracking. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10117-10127, 2023.

Botao Ye, Hong Chang, Bingpeng Ma, Shiguang Shan, and Xilin Chen. Joint feature learning and relation modeling for tracking: A one-stream framework. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXII , pages 341-357. Springer, 2022.

Junjie Ye, Changhong Fu, Guangze Zheng, Danda Pani Paudel, and Guang Chen. Unsupervised domain adaptation for nighttime aerial tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 8896-8905, 2022.

Dan Zeng, Mingliang Zou, Xucheng Wang, and Shuiwang Li. Towards discriminative representations with contrastive instances for real-time uav tracking. In 2023 IEEE International Conference on Multimedia and Expo (ICME) , pages 1349-1354. IEEE, 2023.

Xiangyun Zhao, Raviteja Vemulapalli, Philip Andrew Mansfield, Boqing Gong, Bradley Green, Lior Shapira, and Ying Wu. Contrastive learning for label efficient semantic segmentation. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV) , pages 10603-10613, 2021.

Xinyu Zhou, Pinxue Guo, Lingyi Hong, Jinglun Li, Wei Zhang, Weifeng Ge, and Wenqiang Zhang. Reading relevant feature from global representation memory for visual object tracking. Advances in Neural Information Processing Systems , 36:10814-10827, 2023.

Xinyu Zhou, Pinxue Guo, Lingyi Hong, Jinglun Li, Wei Zhang, Weifeng Ge, and Wenqiang Zhang. Reading relevant feature from global representation memory for visual object tracking. In Thirty-seventh Conference on Neural Information Processing Systems , 2023.

Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159 , 2020.

Haobo Zuo, Changhong Fu, Sihang Li, Kunhan Lu, Yiming Li, and Chen Feng. Adversarial blur-deblur network for robust uav tracking. IEEE Robotics and Automation Letters , 8(2):1101-1108, 2023.

A Technical Appendices and Supplementary Material

A.1 Ablation Study on Correlation Modeling on attributes of UAV123.

As shown in Table 6, the ablation study demonstrates the effectiveness of Dynamic SemanticAware Modeling (DSAM) in enhancing tracking performance across various challenging attributes. Compared to the baseline Transformer, DSAM consistently improves both precision and success rates in all evaluated scenarios. Specifically, for DSATrack-ViT-B, DSAM outperforms the baseline by 2.6% and 2.3% in Camera Motion, 5.8% and 4.4% in Fast Motion, and 7.6% and 5.4% in Low Resolution. This suggests that DSAM effectively captures evolving target representations under complex UAV tracking conditions, where static modeling may struggle to adapt to rapid environmental changes. The performance gap is particularly evident in more challenging scenarios, such as Fast Motion and Low Resolution, where DSAM significantly enhances tracking accuracy and robustness.

To investigate the effect of different template input formats, we compare two variants under the ViT-D8 backbone: using only the template image (A Template) and combining the template with patch embeddings (Template&Patches). As shown in Table 8, incorporating patch-level information consistently improves performance across all UAV tracking benchmarks. Specifically, the Template&Patches design achieves the best overall results, with a precision/success gain of +1.6/+1.6 on UAVDT and +2.3/+1.7 on VisDrone2018 compared to the single-template input. These results demonstrate that introducing fine-grained patch features enhances the representation of template cues, leading to more robust matching and improved generalization across diverse UAV tracking scenarios

A.2 Ablation study on Token Elimination.

To evaluate the effectiveness of token elimination, we conduct an ablation study across four UAV tracking benchmarks: DTB70, UAVDT, VisDrone2018, and UAV123. As shown in Table 8, incorporating token elimination consistently improves both tracking precision and success rate across all datasets, demonstrating the robustness and generalizability of the approach. For the ViT-Base backbone, token elimination yields consistent improvements over the baseline that retains all tokens. On DTB70, it achieves a gain of 0.6% in precision and 0.8% in success rate. Similar improvements

are observed on UAVDT (+0.8 precision, +0.6 success), VisDrone2018 (+0.4 precision, +0.7 success), and UAV123 (+0.7 precision, +1.6 success). These gains are particularly significant under challenging scenarios such as camera motion, small object size, and background clutter, which are common in UAV tracking.

Beyond accuracy, token elimination also contributes to runtime efficiency. For instance, with the ViT-Base backbone, the frame rate increases from 105.6 FPS to 111.7 FPS, reflecting reduced computational overhead without sacrificing performance. This efficiency gain makes the model more suitable for real-time UAV deployment, especially on edge computing platforms with limited resources. Overall, the results validate that token elimination effectively filters out redundant or uninformative tokens, allowing the model to focus on semantically relevant features.

A.3 Comparison with General Trackers

Table 9 presents a comprehensive comparison between DSATrack and recent transformer-based singleobject trackers across four large-scale benchmarks: A VisT, TrackingNet, LaSOT, and GOT-10k. On AVisT, DSATrack-ViT-B achieves competitve overall performance, obtaining an AUC/OP 50 /OP 75 of 60.2/69.1/50.2, demonstrating superior robustness under adverse visibility and occlusion. On TrackingNet, DSATrack attains the highest AUC of 84.1 and P norm of 88.6, showing strong generalization to large-scale, high-diversity scenes. For LaSOT, DSATrack remains competitive with an AUC of 82.7 and precision of 74.6, comparable to the top-performing AQATrack while requiring no task-specific tuning. On GOT-10k, DSATrack delivers the best accuracy across all metrics (AO/SR 0 . 5 /SR 0 . 75 = 75.0/85.6/73.7), surpassing previous transformer-based trackers such as OSTrack and LoRAT-B-224.

These results confirm that DSATrack maintains a favorable balance between tracking precision and robustness, matching state-of-the-art transformer trackers across both long-term and general-purpose benchmarks. Its consistent advantage on stricter metrics (OP 75 , SR 0 . 75 ) highlights the model's improved localization accuracy and resilience in challenging conditions.

A.4 Comparison with UAV Trackers

Table 10 compares DSATrack with recent transformer-based UAV trackers in terms of accuracy and efficiency across four representative UA V benchmarks-DTB70, UAVDT, VisDrone2018, and

UAV123-as well as computational complexity metrics. Among all competitors, DSATrack-D7 achieves the best overall accuracy, attaining the highest precision and success scores on every dataset (DTB70: 88.1/68.4, UAVDT: 84.6/62.1, VisDrone2018: 88.6/68.0, UAV123: 87.3/66.6). This indicates that the hierarchical contribution-based pruning and dynamic alignment strategy effectively preserve discriminative representations across diverse UA V scenarios.

Meanwhile, DSATrack-D4 maintains competitive accuracy while significantly improving efficiency, achieving the highest inference speed of 214.1 FPS on RTX 3090 and 17.4 FPS on NVIDIA Xavier, demonstrating excellent scalability for real-time embedded deployment. The lightweight variants DSATrack-ViT-T and DSATrack-DeiT-T further reduce parameters and FLOPs (8.21 M / 2.99 G) while sustaining robust tracking performance, highlighting the flexibility of the proposed framework in balancing accuracy and efficiency across different model scales and hardware settings.

Overall, DSATrack exhibits a favorable accuracy-efficiency trade-off, outperforming existing UAVspecific trackers in both precision and speed, thereby underscoring its practicality for onboard UA V perception applications.

A.5 Broader Impact

The proposed UAV tracking method could benefit public safety and disaster monitoring by enabling real-time, robust tracking from aerial views. However, we also acknowledge the potential for misuse in surveillance applications and stress the importance of deploying such technology within clear legal and ethical boundaries.

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision  plots  of  OPE  -  Aspect  Ratio

Change

[0.891] DSATrack-ViT-B

[0.875] DSATrack-D8

[0.872] DSATrack-D7

[0.865] DSATrack-D6

[0.855] Aba-ViTrack

[0.832] DSATrack-ViT-T

[0.832] AVTrack

[0.828] DSATrack-D4

[0.824] HiT

[0.753] TCTrack

[0.752] SiamCAR

[0.733] HiFT

[0.711] SiamAPN++

[0.702] SiamAPN

[0.632] ECO-HC

[0.622] AutoTrack

[0.602] ARCF

[0.576] BACF

[0.504] fDSST

[0.482] DSST

[0.424] KCF

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

Precision

Precision  plots  of  OPE  -  Ba

ckground Clutters

[0.763] DSATrack-ViT-B

[0.741] DSATrack-D8

[0.729] DSATrack-D6

[0.709] DSATrack-D7

[0.707] DSATrack-ViT-T

[0.669] AVTrack

[0.652] Aba-ViTrack

[0.626] ECO-HC

[0.624] DSATrack-D4

[0.598] HiT

[0.594] HiFT

[0.593] SiamCAR

[0.591] TCTrack

[0.584] AutoTrack

[0.582] SiamAPN

[0.566] SiamAPN++

[0.552] ARCF

[0.513] BACF

[0.500] DSST

[0.458] fDSST

[0.454] KCF

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

Precision

Precision  plots  of  OPE  -  Full

Occlusion

[0.773] Aba-ViTrack

[0.766] DSATrack-ViT-B

[0.738] DSATrack-D7

[0.729] DSATrack-D6

[0.721] DSATrack-D8

[0.696] HiT

[0.667] AVTrack

[0.663] DSATrack-ViT-T

[0.654] DSATrack-D4

[0.623] TCTrack

[0.586] HiFT

[0.577] SiamAPN++

[0.569] SiamCAR

[0.542] ECO-HC

[0.536] SiamAPN

[0.468] AutoTrack

[0.445] ARCF

[0.436] BACF

[0.406] DSST

[0.384] fDSST

[0.374] KCF

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision  plots  of  OPE  -  Illumination

Variation

[0.879] DSATrack-ViT-B

[0.863] DSATrack-ViT-T

[0.860] DSATrack-D8

[0.836] DSATrack-D6

[0.829] DSATrack-D7

[0.826] AVTrack

[0.816] Aba-ViTrack

[0.774] DSATrack-D4

[0.762] HiT

[0.753] SiamCAR

[0.710] TCTrack

[0.700] HiFT

[0.696] SiamAPN++

[0.685] SiamAPN

[0.628] ECO-HC

[0.617] AutoTrack

[0.595] ARCF

[0.525] BACF

[0.524] DSST

[0.518] fDSST

[0.418] KCF

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision  plots  of  OPE  -  Out-of-View

[0.866] DSATrack-D8

[0.861] DSATrack-D7

[0.858] DSATrack-ViT-B

[0.847] DSATrack-D6

[0.845] AVTrack

[0.844] Aba-ViTrack

[0.817] HiT

[0.809] DSATrack-D4

[0.800] DSATrack-ViT-T

[0.752] TCTrack

[0.733] SiamAPN++

[0.701] SiamCAR

[0.700] HiFT

[0.685] SiamAPN

[0.573] AutoTrack

[0.564] ECO-HC

[0.543] BACF

[0.531] ARCF

[0.462] DSST

[0.455] fDSST

[0.386] KCF

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision  plots  of  OPE  -  Partial

Occlusion

[0.865] DSATrack-ViT-B

[0.844] DSATrack-D8

[0.832] DSATrack-D7

[0.830] DSATrack-D6

[0.822] Aba-ViTrack

[0.812] AVTrack

[0.803] DSATrack-ViT-T

[0.788] DSATrack-D4

[0.765] HiT

[0.720] SiamCAR

[0.716] TCTrack

[0.684] HiFT

[0.658] SiamAPN++

[0.657] SiamAPN

[0.638] ECO-HC

[0.584] AutoTrack

[0.573] ARCF

[0.564] BACF

[0.505] DSST

[0.490] fDSST

[0.451] KCF

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision  plots  of  OPE  -  Scale

Variation

[0.889] DSATrack-ViT-B

[0.860] DSATrack-D8

[0.856] DSATrack-D7

[0.854] Aba-ViTrack

[0.853] DSATrack-D6

[0.841] AVTrack

[0.836] DSATrack-ViT-T

[0.829] DSATrack-D4

[0.810] HiT

[0.791] SiamCAR

[0.778] TCTrack

[0.768] HiFT

[0.738] SiamAPN

[0.736] SiamAPN++

[0.673] ECO-HC

[0.649] AutoTrack

[0.630] ARCF

[0.617] BACF

[0.542] fDSST

[0.538] DSST

[0.471] KCF

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision  plots  of  OPE  -  Similar

Object

[0.906] DSATrack-ViT-B

[0.878] Aba-ViTrack

[0.871] DSATrack-D8

[0.849] DSATrack-D7

[0.844] DSATrack-D6

[0.828] HiT

[0.822] AVTrack

[0.806] DSATrack-ViT-T

[0.804] DSATrack-D4

[0.763] SiamCAR

[0.741] TCTrack

[0.713] HiFT

[0.704] ECO-HC

[0.682] ARCF

[0.673] BACF

[0.666] AutoTrack

[0.663] SiamAPN++

[0.657] SiamAPN

[0.604] DSST

[0.594] fDSST

[0.578] KCF

(a) UAV123

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision  plots  of  OPE  -  Viewpoint

Change

[0.907] DSATrack-D8

[0.902] DSATrack-ViT-B

[0.900] DSATrack-D7

[0.895] Aba-ViTrack

[0.889] DSATrack-D6

[0.889] AVTrack

[0.860] HiT

[0.857] DSATrack-ViT-T

[0.854] DSATrack-D4

[0.788] SiamCAR

[0.777] SiamAPN++

[0.771] TCTrack

[0.764] HiFT

[0.753] SiamAPN

[0.631] ECO-HC

[0.618] AutoTrack

[0.596] ARCF

[0.587] BACF

[0.525] fDSST

[0.502] DSST

[0.436] KCF

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

1

.

0

Precision

Precision  plots  of  OPE  -  Aspect  Ratio

Change

[0.959] DSATrack-ViT-T

[0.947] DSATrack-D7

[0.935] DSATrack-D8

[0.921] AVTrack

[0.917] DSATrack-ViT-B

[0.905] SiamCAR

[0.884] DSATrack-D4

[0.883] ECO-HC

[0.868] Aba-ViTrack

[0.860] TCTrack

[0.859] DSATrack-D6

[0.817] SiamAPN

[0.815] P-SiamFC++

[0.812] SiamAPN++

[0.809] SiamRPN

[0.782] ARCF

[0.769] AutoTrack

[0.737] BACF

[0.719] DSST

[0.688] fDSST

[0.629] KCF

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

Precision

Precision  plots  of  OPE  -  Background

Clutter

[0.804] DSATrack-D8

[0.780] DSATrack-D7

[0.765] DSATrack-D6

[0.760] DSATrack-ViT-T

[0.746] DSATrack-ViT-B

[0.743] Aba-ViTrack

[0.740] DSATrack-D4

[0.730] AVTrack

[0.722] SiamCAR

[0.675] SiamRPN

[0.660] ARCF

[0.649] ECO-HC

[0.634] P-SiamFC++

[0.633] DSST

[0.632] AutoTrack

[0.624] TCTrack

[0.615] SiamAPN

[0.602] BACF

[0.528] KCF

[0.519] SiamAPN++

[0.478] fDSST

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

Precision

Precision  plots  of  OPE  -  Full

Occlusion

[0.823] DSATrack-D8

[0.815] Aba-ViTrack

[0.813] DSATrack-D6

[0.803] AVTrack

[0.790] DSATrack-D4

[0.781] DSATrack-ViT-B

[0.779] DSATrack-ViT-T

[0.762] DSATrack-D7

[0.751] ECO-HC

[0.744] SiamCAR

[0.717] TCTrack

[0.703] SiamAPN

[0.691] SiamRPN

[0.680] ARCF

[0.678] AutoTrack

[0.671] BACF

[0.644] SiamAPN++

[0.641] P-SiamFC++

[0.594] DSST

[0.581] KCF

[0.539] fDSST

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision  plots  of  OPE  -  Illumination

Variation

[0.898] DSATrack-D6

[0.897] DSATrack-D8

[0.881] DSATrack-ViT-B

[0.871] DSATrack-D7

[0.862] AutoTrack

[0.859] BACF

[0.858] DSST

[0.856] ARCF

[0.856] ECO-HC

[0.826] SiamCAR

[0.821] DSATrack-D4

[0.817] DSATrack-ViT-T

[0.813] Aba-ViTrack

[0.798] fDSST

[0.791] AVTrack

[0.788] TCTrack

[0.783] KCF

[0.763] SiamRPN

[0.758] P-SiamFC++

[0.751] SiamAPN

[0.720] SiamAPN++

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

1

.

0

Precision

Precision  plots  of  OPE  -  Out-of-View

[0.969] DSATrack-D7

[0.964] DSATrack-D8

[0.948] DSATrack-ViT-B

[0.941] DSATrack-ViT-T

[0.940] SiamCAR

[0.910] AVTrack

[0.890] DSATrack-D4

[0.879] TCTrack

[0.877] SiamRPN

[0.870] ECO-HC

[0.870] P-SiamFC++

[0.862] DSATrack-D6

[0.837] Aba-ViTrack

[0.814] SiamAPN++

[0.812] BACF

[0.808] SiamAPN

[0.803] ARCF

[0.794] AutoTrack

[0.748] DSST

[0.679] fDSST

[0.582] KCF

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision  plots  of  OPE  -  Partial

Occlusion

[0.863] DSATrack-D8

[0.828] DSATrack-ViT-B

[0.822] DSATrack-D7

[0.821] DSATrack-D6

[0.810] Aba-ViTrack

[0.807] SiamCAR

[0.796] ECO-HC

[0.792] AVTrack

[0.784] DSATrack-ViT-T

[0.774] DSATrack-D4

[0.760] TCTrack

[0.725] SiamAPN

[0.723] ARCF

[0.720] AutoTrack

[0.718] BACF

[0.698] SiamRPN

[0.698] SiamAPN++

[0.695] P-SiamFC++

[0.662] DSST

[0.621] KCF

[0.620] fDSST

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision  plots  of  OPE  -  Scale

Variation

[0.932] DSATrack-D8

[0.902] AVTrack

[0.900] DSATrack-ViT-T

[0.899] DSATrack-D7

[0.895] DSATrack-D4

[0.894] DSATrack-D6

[0.891] DSATrack-ViT-B

[0.883] Aba-ViTrack

[0.879] SiamCAR

[0.825] SiamAPN

[0.825] TCTrack

[0.824] SiamRPN

[0.795] P-SiamFC++

[0.795] ECO-HC

[0.758] ARCF

[0.753] AutoTrack

[0.752] DSST

[0.747] SiamAPN++

[0.732] BACF

[0.717] fDSST

[0.700] KCF

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

Precision

Precision  plots  of  OPE  -  Similar

Object

[0.795] DSATrack-D8

[0.770] DSATrack-D7

[0.756] DSATrack-D6

[0.724] DSATrack-ViT-B

[0.724] DSATrack-ViT-T

[0.708] Aba-ViTrack

[0.708] SiamCAR

[0.705] DSATrack-D4

[0.695] AVTrack

[0.649] SiamRPN

[0.638] ARCF

[0.634] DSST

[0.625] ECO-HC

[0.617] fDSST

[0.613] AutoTrack

[0.602] P-SiamFC++

[0.594] SiamAPN

[0.588] TCTrack

[0.564] BACF

[0.563] KCF

[0.521] SiamAPN++

0

5

10

15

20

25

30

35

40

45

50

Location error threshold

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

Precision

Precision  plots  of  OPE  -  Viewpoint

Change

[0.946] DSATrack-D8

[0.926] AVTrack

[0.921] DSATrack-D7

[0.914] DSATrack-ViT-B

[0.906] DSATrack-ViT-T

[0.906] Aba-ViTrack

[0.901] DSATrack-D6

[0.900] DSATrack-D4

[0.878] ECO-HC

[0.867] SiamAPN

[0.856] SiamCAR

[0.845] TCTrack

[0.843] SiamRPN

[0.839] P-SiamFC++

[0.809] ARCF

[0.808] SiamAPN++

[0.806] AutoTrack

[0.805] BACF

[0.762] DSST

[0.749] fDSST

[0.748] KCF

Car

Car

Car

Car

Car

Car

Girl

Girl

Girl

Girl

Girl

Girl

Dog

Dog

Dog

Dog

Dog

Motor

Motor

Motor

Motor

Motor

Motor

口

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS Paper Checklist" ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction clearly state our contributions.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have discussed the limitations in section6

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: The paper does not include theoretical result,

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [No]

Justification: We will provide opensource code after the paper is accepted.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We present all details and experimental setting in Section 2 and Section 3

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We present all experimental setting and detail in Section 4.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification:For core experiments, we report precision and success scores across multiple UAVtracking datasets (Table 3), and include performance variance under different conditions (Table 6), reflecting attribute-specific tracking robustness. While error bars are not explicitly drawn, comparative trends are consistently validated.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We present all details in Section 4 and Section 5.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research complies with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We decribe the broader impacts in Section 7

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: We discuss both the potential positive societal impacts (e.g., public safety, disaster response) and negative ones (e.g., surveillance misuse) of our UA V tracking method in Appendix A.5.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All datasets and moel used are public and cited with proper references.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification:The paper does not introduce or release any new dataset, all experiments are conducted on existing datasets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification:The paper does not introduce or release any new dataset or model checkpoint; all experiments are conducted on existing datasets.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve human subject research, so no IRB approval is required.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: No large language models were used in the development of the method or experiments.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes a UAV tracker, named DSATrack, that utilizes semantic correlation to enhance tracking accuracy. The main contribution of DSATrack is the Dynamic Semantic Relevance Generator, which explores the semantic relevance according to the correlation map from the Transformer.\nBenchmark experiments are comprehensive, including DTB70, UAVDT, VisDrone2018, and UAV123, and DSATrack reaches the SOTA performance. The ablation study on correlation modeling shows the effectiveness of the proposed Dynamic Semantic Relevance Generator.\nThe inference speed of the method is well considered by pruning, and the method shows efficiency in real-world tracking experiments.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n1. The experiments are exceptionally comprehensive and well-designed, featuring established UAV tracking datasets and including the presentation of multiple model variants with distinct configurations, including DTB70, UAVDT, VisDrone2018, and UAV123, and DSATrack.\n\n2. Module design is reasonable and proven effective in experiments, especially in Table 4 and Table 5.\n\n3. Several variants of the model design are proposed with different architectural settings for diverse demands of tracking.\n\n4. The proposed DSATrack is deployed on an edge device for real-world application tests, where DSATrack-ViT-T and DSATrack-DeiT-T achieve 27.6 FPS and 27.4 FPS.\n\nWeakness:\n\n1. Although the evaluation benchmarks are well-established, their dataset scale appears insufficient by contemporary research standards.\n\n2. Source code and model weights are not attached.\n\n3. Comparison of model size between the proposed method and SOTA methods is not shown.\n\n4. Limited novelty compared with baselines. The idea of correlation modeling has been studied in several works, including HiFT and TCTrack. The model architecture design is also based on RFGM and OSTrack.\n\n5. Insufficient theoretical explanation of model design under UAV-specific scenarios. Special challenges of UAV tracking scenarios should be highlighted with cases.\n\n6. The generalization performance of DSATrack on generic datasets was not addressed, e.g., LaSOT, GOT-10k.\n\nQUESTIONS:\n1. Will the authors provide the source code and trained weights for reproduction?\n2. Will the authors provide the comparison on model size?\n3. Will the authors plan to include larger or more benchmarks for evaluation?\n4. How is model design intrinsically linked to the unique challenges of UAV tracking?\n5. ​What fundamentally distinguishes the proposed Dynamic Semantic Relevance Generator from prior approaches to correlation modeling?\n6. How will DSA generalize to datasets in common scenes, e.g., LaSOT?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the problem of tracking objects with the use of a drone equipped with a camera. The proposed solution is based on the use of a transformer architecture. The specific contribution is related to the addition of a semantic component to improve tracking ability and a pruning approach in the transformer blocks to reduce complexity and increase tracking speed.  \nThis reviewer found the idea of adding a semantic component to better track the object in the scene relevant. It is based on generating template semantic tokens and then computing a correlation with the scene/search tokens.  This allows the implementation of a dynamic semantic relevance generator. \n\nThe paper offers numerical results using various datasets and a comparison with other architectures used for tracking. Additionally, some real experimentation is also documented.\n\nSTRENGTHS AND WEAKNESSES:\nThe proposed approach has been proven to offer the best performance and even the second-best performance with lighter versions.  \nThe idea of adding a semantic scene component is interesting. What is not fully understood from the paper is how the template semantic tokens are generated. It is assumed that this is achieved by pre-training with a number of possible known objects and scenes in a supervised fashion. If so, the architecture is limited to tracking \"known\" objects. \n\nAdditionally, tracking is herein limited to a single object. Would it be possible to simultaneously track multiple objects also belonging to different classes?  \n\nAnd what about using multiple cameras in a multi-drone setup? Is it something to be considered and possibly implemented by extending the current architectures?\n\nQUESTIONS:\nIn addition to the above comments:\n\n- Can you elaborate more on how the template tokens are generated? \n\n- The Semantic Aware Modeling section might be integrated in the previous one, since essentially this describes just a refinement of the correlation map.  \n\n- Would it be possible to run experiments that show the tracking ability of multiple objects? \n\n- The section about real-world experiments does not clarify how the initialization of the tracking process is done. That is, how the object is selected by correctly positioning the bounding box.  Additionally, the reported results show \"qualitative\" performance (some frames). Would it be possible to define and report some quantitative performance metrics? \n\n- A final minor remark is that the terminology \"UAV tracking\" is ambiguous since at first it may be interpreted as tracking the location of the UAV and not using a UAV equipped with a camera to monitor and track objects in a scene.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a UAV tracking framework called DSATrack, which aims to address the lack of semantic awareness in existing methods. DSATrack introduces a dynamic semantic correlation generator and combines this generator with a Transformer-based correlation graph to explore semantic relations. This design enhances the ability to extract key information from the template within the search region. The framework also includes a pruning technique, thus achieving a balance between speed and accuracy. The experimental results on multiple datasets demonstrate its effectiveness and robustness.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper brings dynamic semantic aware correlation modeling into the field of aerial tracking. It has good validity. However, the description of the method can be confusing. Some concepts are not clearly explained. The experiments are not thorough enough. In addition, the credibility of the efficiency and accuracy of the DSATrack-ViT-B model is open to question.\n\nQUESTIONS:\n1.The paper states that the inputs of DSATrack are the template and the patch. It is unclear where the patch comes from. It is also unclear how the model performs without the patch. This should be explained and tested in an ablation study.\n2.The description of the Dynamic Semantic-aware Transformer in Figure 2 is confusing. It is unclear if Semantic-aware Correlation Map is generated by Semantic Aware Modeling module or if it is unrelated to it, as Figure 2 suggests. The authors should review Figure 2 and the text and decide if they need to revise the figure or the description. Figure 2 also shows both a Dynamic Semantic-aware Transformer block and a Transformer block. It is unclear which Transformer layers are removed.\n3.In Table 3, LiteTrack is not left aligned.\n4.The paper does not explain the difference between SSAM and DSAM.\n5.I also doubt the efficiency and accuracy of the DSATrack-ViT-B model.\n\nLIMITATIONS:\n1. Recent advanced SOT transformer-based methods are missing in the experiments, such as OSTrack, LoReTrack, LoRAT, MixformerV2, AQATrack, AsymTrack. Although not UAV-specific trackers, it is necessary to compare the proposed tracker with these non-real-time SOTA trackers for a comprehensive evaluation.\n2. Discussion on Computational Complexity, while DSATrack outperforms SOTA models in speed, the paper lacks a detailed comparison of FLOPs and parameter.\n3. Failure Case Analysis: It is recommended to include a detailed failure analysis, highlighting scenarios where DSATrack struggles.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-27"}

comments: SUMMARY:
This paper presents DSATrack, a UAV tracking framework featuring a Dynamic Semantic Relevance Generator to improve semantic correlation modeling. A Transformer pruning strategy is also proposed to balance speed and accuracy. Experiments demonstrate state-of-the-art performance across multiple benchmarks.

STRENGTHS AND WEAKNESSES:
### Strengths
- The paper identifies an important gap in current UAV tracking methods—the lack of semantic awareness in correlation modeling, which is well-justified.

- By introducing a pruning-based variant generation, the method can flexibly adapt to different computational constraints (e.g., real-time vs. offline UAV applications).

- Extensive experimental results and ablation studies are conducted, proving the effectiveness of each proposed module.

### Weaknesses

- The details of the Dynamic Semantic Relevance Generator are insufficiently elaborated—how exactly it computes relevance, and how it integrates with the Transformer pipeline should be clarified.

- While pruning is introduced for speed-up, the comparison with existing efficient tracking baselines (e.g., lightweight Transformers or mobile-friendly trackers) is not reported.

- It's unclear how robust DSATrack is to extreme low-resolution or partial occlusion cases, which are critical for UAV applications.

QUESTIONS:
- How generalizable is DSATrack across different UAV tracking scenarios?

- Can the semantic-aware correlation maps be visualized to better demonstrate the interpretability of the method?
- More issues are shown in the weakness above.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-04