Few-Shot Backdoor Attacks on Visual Object Tracking
Authors: Yiming Li
Haoxiang Zhong
Xingjun Ma
Yong Jiang
Shu-Tao Xia
Venue: ICLR 2022 Poster (2022)
Keywords: Backdoor Attack
Visual Object Tracking
AI Security
Deep Learning

Abstract: Visual object tracking (VOT) has been widely adopted in mission-critical applications, such as autonomous driving and intelligent surveillance systems. In current practice, third-party resources such as datasets, backbone networks, and training platforms are frequently used to train high-performance VOT models. Whilst these resources bring certain convenience, they also introduce new security threats into VOT models. In this paper, we reveal such a threat where an adversary can easily implant hidden backdoors into VOT models by tempering with the training process. Specifically, we propose a simple yet effective few-shot backdoor attack (FSBA) that optimizes two losses alternately: 1) a \emph{feature loss} defined in the hidden feature space, and 2) the standard \emph{tracking loss}. We show that, once the backdoor is embedded into the target model by our FSBA, it can trick the model to lose track of specific objects even when the \emph{trigger} only appears in one or a few frames. We examine our attack in both digital and physical-world settings and show that it can significantly degrade the performance of state-of-the-art VOT trackers. We also show that our attack is resistant to potential defenses, highlighting the vulnerability of VOT models to potential backdoor attacks.

Published as a conference paper at ICLR 2022
FEW-SHOT BACKDOOR ATTACKS ON VISUAL OBJECT TRACKING
Yiming Li Haoxiang Zhong Xingjun Ma², Yong Jiang Shu-Tau Xia
TTinggha Shenzhen International Graduate School, Tsinghua University, China Research Center of Artificial Intelligence, Peng Cheng Laboratory, China School of Computer Science, Fudan University, China [li-ym18, 29)19) @
danxjma @ gmail.com; [jiangy, xiast) @es..singhua..cu..n
ABSTRACT
Visual object tracking (VOT) has been widely adopted in mission-critical applica- tions, such as autonomous driving and intelligent surveillance systems. In current practice, third-party resources such as datasets, backbone networks, and train- ing platforms are frequently used to train high-performance VOT models. Whilst these resources bring certain convenience, they also introduce new security threats into VOT models. In this paper, we reveal such a threat where adversary can easily implant hidden backdoors into VOT models by tempering with the train- ing process. Specifically,
propose simple yet effective few-shot backdoor attack (FSBA) that optimizes two losses alternately: 1) a feature loss defined the hidden feature space, and 2) the standard tracking loss. We show that, once the backdoor is embedded into the target model by our FSBA, it can trick the model lose track of specific objects even when the trigger only appears in one or a few frames. We examine our attack in both digital and physical-worke settings and show that can significantly degrade the performance of state-of of-the-an VOT trackers. We also show that our attack is resistant to potential defenses, highlight- ing the vulnerability of VOT models to potential backdoor attacks.
INTRODUCTION
Visual object tracking (VOT) aims to predict the location of selected objects in subsequent frames based on their initial locations in the initial frame. It has supported many impactful and mission- critical applications such as intelligent surveillance and self-driving systems. The security of VOT models to potential adversaries is thus of great importance and worth careful investigations. Cur- rently, most of the advanced VOT trackers (Li et al. 2019: Lu et al. 2020; Wang et al.) 2022b) are based on deep neural networks (DNNs), siamese networks in particular. Training these models often requires large-scale datasets and large amount of computational resources. As such, third-party resources such as datasets, backbones, and pre-trained models are frequently exploited directly applied to save training costs. While these external resources bring certain convenience, they also introduce opacity into the training process. raises an important question: Will this opacity bring new security risks into VOT?
In this paper, we reveal the vulnerability of VOT backdoor allacks that are caused by outsourced training or using third-party pre-trained models. Backdoor attacks are a type of training-time threat to deep learning that implant hidden backdoors into a target model by injecting a trigger pattern (e.g., local patch) into small subset of training samples (Li et al. 2020). Existing backdoor attacks are mostly designed for classification tasks and are targeted attacks tied to a specific label (known as the target label) (Gu et al.) 2 et al. 2021 Nguyen & Tran 2021). These attacks are not fully transferable to VOT tasks due to the fundamental difference between classification and object tracking. Different from attacking a classifier, making object escape the tracking is more threatening objective for VOT. As such, in this paper, we explore specialized backdoor attacks for VOT, which are untargeted by nature: the backdoored model behaves normally on benign samples yet fails to track the target object whenever the trigger appears.
"The first two authors contributed equally this work. Correspondence to: Xingjun Ma amail..ommmml..cmmmm and Shu-Tao Xia siaat@@s..snnghua.ddu..c

---

Published as a conference paper at ICLR 2022
Benign Framnes
Poisoned Frames
30
300
200
150
20-
100
50
0-
200
-10
-100
-300
-20
-400
-200
200
400
-00
-200
-100
100
-10
10
(a) Benign Model
(b) Baseline Attack
(c) Our Attack
Figure 1: The t-SNE visualization of benign frames and their poisoned versions in the feature spaces of three different models: (a) benign model; (b) backdoored model by BOBA; and (c) backdoored model by our FSBA attack. FSBA-poisonee frames well-separated from the benign frames in the feature space, thus can better mislead or manipulate the target model.
In the current literature, the most advanced VOT models are siamese network based models that gen- erally consist two functional branches: a classification branch that predicts whether a candidate box (or anchor) is positive or negative and 2) regression branch that learns location information of the bounding box. Arguably, the most straightforward strategy to apply existing label-targeted attacks to attack the classification branch. Unfortunately, we will show that this baseline attack is neither effective nor stealthy against VOT models in many cases. We reveal that this ineffectiveness is largely due to the close distance between benign and poisoned frames in the feature space, as shown in Figure [ Motivated by this observation, we propose to embed hidden backdoors directly in the feature space. Specifically. we treat the task of backdoor attacking VOT multi-takk learning, which minimizes the standard tracking loss while simultaneously maximizing the feature loss between benign and poisoned frames in the feature space. The problem be ef- fectively solved by alternating optimization on the two loss terms. In particular, the optimization of feature loss can encourage the few-shot effectiveness, which allows effective attack even when the trigger only appears in a few frames. Besides, we only randomly select a few training frames for poisoning. This strategy not only can reduce the computational cost but also avoids significant degradation of the model's 's tracking performance on benign videos.
instance of
In summary, our main contributions are: 1) We reveal the backdoor threat in visual object tracking. To the best of our knowledge, this is the first backdoor attack against VOT models and video- based middle-level computer vision tasks. 2) We propose simple yet effective few-shot untargeted backdoor attack that can significantly degrade the tracking performance even if the trigger only appears in a few frames. 3) We empirically show that our attack effective in both digital and phrsical-world scenarios and resistant to potential defenses.
2
RELATED WORK
2.1 BACKDOOR ATTACK
Backdoor attack an emerging yet severe threat to DNNs. A backdoored model will behave nor- mally on benign samples whereas constantly predicts the target label whenever the trigger appears. Currently, most existing backdoor attacks (Gu et al., 2019 Zeng et al. 2021a Li et al. 202 are de- signed for image classification tasks and targeted towards
speecfffed label. Specifically, a backdoor attack can be characterized by its trigger pattern t, target label ye. poison image generator G(-), and poisoning rate y. Taking BadNets (Gu et al. 2019 for example, given benign training set D = { 31) 1 the adversary randomly selects 7 7 samples (i.e., D.) from D to generate their poisoned version D,,- = ((x', )rr) G(æ;; t), (ac, y) D.). where G(æ; t) (1 A) 8 + X t with A E {0, 1]0x x and @ indicates the element wise product. It then trains backdoored model (i.e., fa) on the poisoned subset D, and the remaining benign samples D, 2 D) D'D, by solv- ing the optimization problem: ming (E,,))D,, ((Jo(x), y), where 0 is the model parameters, 2(-) is the loss function.
Currently, there are also a few backdoor attacks developed outside the context of image classification Wang al.) 202 Zhai et al. 2021 Xiang et al. 2021). To the best of our knowledge, the backdoor attack proposed by Zhao et al. (2020b) is the only existing backdoor attack on video models. However, it is label-targeted attack designed for video classification tasks and can not be
2

---

Published as a conference paper at ICLR 2022
directly applied to VOT models. Moreover, it needs to add the trigger pattern all frames of the video and its effectiveness was only evaluated in the digital space.
In particular, backdoor attacks are different from adversarial attacks (Madry al. 2018 Croce & Hein 2020; Andriushchenko et al. 2020). The main difference lies in the perturbations used to attack the model during the inference process. The perturbations (trigger patterns be more precise) used by backdoor attacks are pre-implanted into the target model thus can be directly applied to attack any test samples. By contrast, adversarial attacks need generate perturbations through an optimization process for each test example.
2.2 BACKDOOR DEFENSE
Most existing backdoor defenses can be categorized into two main types: 1) pre-processing based methods and 2) model reconstruction based methods. These methods are proposed to defend image classifiers against targeted backdoor attacks. Due to the untargeted nature of VOT attacks and the fundamental difference between classification and visual tracking, only few of them can be applied to defend against our proposed attack. Here, we briefly review these potential defenses.
Pre-processing based Defense. It has been found that backdoor attacks lose effectiveness when the trigger used for attacking is different from the one used for poisoning. This has motivated the use of image pre-processing techniques (e.g., scaling and color-shifting) to alleviate backdoor threats before feeding a test image into the model for inference (Liu et al.] 2017): Zeng et al.] 2021b] Li
al. 2021b). Since a video is composed of continuous frames, one may conduct trame-wise image
pre-processing to defend against VOT backdoor attacks. Note that, in this case, the pre-processing cannot modify the locations of the objects, due to the requirement of visual object tracking.
Model Reconstruction based Defense. Model reconstruction (e.g.. tuning and pruning) have been demonstrated be effective in erasing hidden backdoors. For example, (Liu et al. 2017 Yao et al. 2019 Zeng et al., 2022) showed that using a few benign samples to fine-tune or retrain the backdoored model for few iterations can effectively remove different types of backdoors from attacked DNNs; (Liu et al. 20188; Wu & Wang] (22T]) showed that defenders can remove hidden backdoors via pruning, based on the understanding that hidden backdoors are mainly encoded the neurons that are dormant when predicting benign samples.
2.3 SIAMESE NETWORK BASED VISUAL OBJECT TRACKING
The goal of VOT is to predict the position and size of an object in a video after is specified in the initial frame. Currently, siamese network based trackers (Bertinetto et al. 2016 Li et al. 2019 Xu et al. 2020) have attracted the most attention, owing to their simplicity and effectiveness MMarvastiZaade et 2021). From the aspect model structure, siamese network based trackers consist of two identical branches with one branch learning the fcature representation of the template while the other learning that of the search region. Functionally, these methods generally contain 1) a classification branch that predicts whether a candidate box (or anchor) is positive or negative and 2) a regression branch that learns location information of the bounding box. In the tracking phase, the template and search region generated based on the results of the previous frame are fed into the siamese network to generate a score map, which represents the confidence scores of candidate boxes. Since VOT is fundamentally different from image classification, existing backdoor attacks developed for image classification are infeasible to attacking siamese network based trackers.
3 FEW-SHOT BACKDOOR ATTACK (FSBA)
Threat Model. Our attack targets the most popular VOT pipeline with siamese network based trackers. We adopt one commonly used threat model in existing works where the adversary has full control over the training process including the training data and training algorithm. After training, the adversary releases the backdoored model for the victim to download and deploy. This type of backdoor attack could happen in many real-world scenarios, such as outsourced model training using third-party computing platforms or downloading pre- me-ttrieed models from untrusted repositories.
Problem Formulation. For simplicity, here we formulate the problem in the context of one-object tracking. The formulation be easily extended to the multi-object case. Specifically, let V
3

---

Published as a conference paper at ICLR 2022
{I.)i= denote video of continuous frames and B {b, box of the target object in each frame. Given the initial state of the target object in the initial frame b,, the tracker will predict its position B Spred in the subsequent frames. Let G(:;t) t) be the frame- wise poisoned video generator where is the adversary-specified trigger pattern. Different from existing backdoor attacks, in this paper, adversary intends to train an attacked version f(: 0) of the benign tracker f(:; 0) by tempering with the training process. The adversary has two main goals as follows:
[b,),_1 denote the ground-truth bounding
design the attack to be untargeted. Specifically, the
Definition 1. A backdoor attack on visual object tracking is called promising (under the measure- ment of loss L with budgets a and 3) if and only if it satisfies two main properties:
o- B-Effectiveness:: the performance of the attacked tracker degrades sharply when the trigger appears, i.c., By (C((((V)))
Ey ())
3-Stealthiness: the attacked tracker behaves normally in the absence of the trigger, i.e., Ey (CLF(V:)))B)) S.
The above attack problem is challenging because VOT is a more complex task than classification and the adversary has to escape the tracking even the objects never appear in the training set. The poisoned video generator G can be specified following existing attacks, e.g., G(V;t)
{
where I, = (1 A) & + & t. It is worth mentioning that stcalthiness should be defined for the trigger pattern if the adversary does not have full control over the training process. However, under our threat model, trigger stealthiness is less interesting when compared to good performance on benign videos which could make the attacked model more tempting potential users.
3.1 AN INEFFECTIVE BASELINE: BRANCH- ORIENT-ORIENTED BACKDOOR ATTACK (BOBA)
Siamese network based trackers generally utilize a classification branch to predict the score map S, which consists of the score for each candidate box in the search region. The training loss of the classification branch is defined as the mean of the individual losses of predicting each score E S, based on the ground-truth label y E {-1, +1} of the candidate box. If the candidate box is within the central area of the target object, it is considered to be a positive example with label = 1, otherwise, is marked as a negative example with label y = -1
Intuitively, we can apply existing tabel-targeted backdoor attacks to attack the classification branch. This attack is dubbed as the branch-oriented backdoor attack (BOBA) in this paper. Specifically, BOBA flips the label of candidate boxes (i.e., ú = -y) for a small subset of training frames. Specifically, let DD = {(x.; 21, b;, y;)) be the original training dataset, where x is the search region, z is the template, b is the bounding box of the target object, and y are the ground-truth labels of candidate boxes. Given an Alversary-specifie selects 7% samples (f.e., D.)) from D to generate poisoned samples using generator G. It then trains a backdoored tracker on the mixed dataset with both the poisoned and the remaining benign samples (i.e., Dj A D - D2) by solving the following optimization problem:
trigger pattern t, the adversary first randomly
min Lb + Cp.
(1)
where,
L.
1
Σ
2,,,),,,).
Dao (.,,,b,))EDD
C.
1 D.
Σ
(G(a; + [(x, G(z;t) -v)]
(3)
(D.D,
In the tracking process, the adversary can attach the trigger pattern t to any target objects in selected frames to escape tracking, using the generator G. this paper, G is simply designed as replacing %% of the center area of the frame with the trigger And we call 'sh' the modification rate.
However, we will show that BOBA has limited effectiveness in attacking VOT models in many cases. We find that this ineffectiveness is largely caused by the close distance between benign and poisoned frames in the feature space. On the other hand, it also hurts the model's performance on benign videos and therefore loses stealthiness. Please see Sections 3.2 and 4 2 for more results.
4

---

Published as a conference paper at ICLR 2022
Paissned Tomplase
Backbone
Feature Loss
Tomplate
Backbone
Classification Branch
Tracking Less
Soarch Regien
Backbonu
Regrossion Branch
Feature Loss
Painoned
Soarch Region
Backbone
The training pipeline of our proposed FSBA. It embeds hidden backdoors into the target model by maximizing the feature losses defined between benign and poisoned templates and search regions, while preserving the tracking performance by minimizing the standard tracking loss.
Figure 2:
3.2 THE PROPOSED ATTACK
Here, we introduce our proposed few-shot backdoor attack (FSBA). The predictions of VOT models are based on the representation of the search region and template in the feature space. Since the adversary intends attach trigger these images activate hidden backdoors, one may expect that the attack could succeed if the representation changes drastically after attaching the trigger pattern to benign frames. Accordingly, unlike BOBA, FSBA embeds hidden backdoors directly in the feature space (as shown in Figure 2 Before introducing the complete training-attacking procedure, we first define the feature loss used by FSBA to inject backdoors into VOT models.
Definition 2.
Let bl:; 0b) be the backbone of the tracker with parameters Oy. Given an image pair (a,, 2) and the poison image generator G(+; t) with trigger the feature loss is defined as:
()( 2 d (b(æ;; 0s), (x; t); 0o)) + d (b(z; 0s), (([( 0b))
(4)
where x is the search region, is the template, and d(-) is distance metric. In this paper, adopt the & norm as the distance metric for simplicity.
Backdoor Injection. We treat the backdoor attack first task is for backdoor injection while the second for the standard tracking. We solve the first task by maximizing the feature loss Ly (Eqn. (4) and the second task by minimizing the standard [D. Ele.,.b.))D L(x, b. y) where D, contains benign samples). We losses (i.e., Ly and L,) when training the VOT models. In particular,
instance of multi-task learning, with the
tracking loss Le (i.e., 1
alternatively optimize the the optimization of y can ensure few-shot effectiveness, which allows effective attack even when the trigger only appears in a few frames. Please refer to Appendix foor empirical verification. Besides, we only randomly select 7% of training frames for poisoning. This strategy not only can reduce the computational cost but also avoids significant degradation of the model's 's tracking performance on benign videos. Note that many trigger patterns could work for our FSBA, e.g., the black-white square of BadNets (Gu et al. 2019). The patterns used in our experiments are shown in Figure
Attacking Phase. Once the backdoor is embedded into the target model, the adversary can the trigger & to attack the model for any input videos during the tracking process, In most VOT works, the template is obtained from the initial frame and remains unchanged in the subsequent frames. Following this setting, we discuss two different modes of our FSBA attack, including the one-shor mode and few-shof mode. In the one-shot mode, the adversary attaches the trigger only to the initial frame, while in the few-shot mode, the adversary attaches the trigger to the first 7% frames. We call T the frame attacking rate. Note that the one-shot mode is special case of the few-shot mode.
4
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
VOT Models and Datasets. We evaluate the effectiveness of BOBA and our FSBA attack on
three advanced siamese network based trackers, including 1) SiamFC (Bertinetto et al. 2016). 2) SiamRPN++ (Li et al.] 2019) and 3) SiamFC++ (Xu et al. 2020). on OTВ100 (Wu et al.) 2015)
and GOTIOK (Huang et al., 2019) datasets. More descriptions of the two datasets are provided in Appendix A.I We also provide the results the LaSOT dataset (Fan al. 2019 in Appendix C
5

---

Published as a conference paper at ICLR 2022
+ Frame 0
Prauee 70
4 Frame
Frome 166
Frame 251
# Frame
Frame 70
Frame
Frame 106
Frame 251
(a) Tracking the "car" Object in Vidco From the OTB100 Datasct
Frame
Prame
Fraune
e 64
Prome Jo5
Frame
N Prame 3
Frame 21
# 64
Frame 105
(b) Tracking 'football' Object in Video From GOTIOK Dataset
Figure 3: Results of SiamFC++ in tracking benign (Top Rows) and attacked (Bottom Rows) videos. The green rectangles highlight the bounding boxes predicted by the benign models while the red ones highlight those predicted by the backdoored models our FSBA under the one-shot mode.
Evaluation Metric. We evaluate the tracking performance by three metrics: 1) precision (Pr) (Wu al. 2013), 2) area under curve (AUC) (Wu al.) (2013). and 3) mean success rate over different classes with threshold 0.5 (mSR50) (Huang et al. 2019). The Pr reflects how well are the trackers predicting the center location of the target object, while the AUC and mSR measure the overlap ratio between the predicted and ground ground-truth boxes. All hyper-parameters involved in these metrics follow the default settings in their original papers. We report the result of each metric on the testing videos before (i.e., [metric name]-B) and after the attack (i.e., [metric name In particular, the larger the Pr-B, AUC-B, and mSR50-B the more stealthy the attack; the smaller the Pr-A, AUC-A, and mSR50-A the more effective the attack.
Attack Setup. We adopt black-white square as the trigger pattern (as shown in Figure 3 and set the modification rate t/ 1% (w.r.t. the search compare our FSBA with BOBA introduced in Section [..1, under both one-shot and few-shot modes. We also report the results of the benign models trained without any attack (dubbed 'Benign') for reference. Specifically, we set the frame attacking rate = 10% for all baseline methods and our FSBA under the few-shot mode.
of the template frame) for all attacks. We
Training Setup. We implement and train all (benign and backdoored) VOT models using Pytorch (Paszke al., 2019). For each model architecture, all attacks and the benign model adopt the same training settings. More detailed training setup can be found in Appendix A.2
4.2 MAIN RESULTS
As shown in Table 1 both BOBA and FSBA reduce the tracking performance while our FSBA is significantly more effective attacking against the latest trackers SiamRPN++ and SiamFC++. Particularly, our FSBA can reduce the AUC of the SiamFC++ tracker by more than 30% on both datasets, even if the trigger only appears in the initial frame (i.e., under the one-shot mode). By contrast, BOBA only managed to decrease 5% AUC-A of SiamFC++ on either dataset. Overall, the AUC-A of our FSBA against SiamFC+ is 40% lower (better) than that of BOBA on both datasets. FSBA is also more stealthy than BOBA. For example, the AUC-B of our attack against the SiamFC tracker is 2% higher (better) than that of BOBA on both datasets. On one hand, the effectiveness of our FSBA highlights the backdoor threat outsourced training of VOT models using third-party pre-trained models. On the other hand, the ineffectiveness of BOBA against the latest trackers indicates that attacking VOT models is indeed more challenging than attacking image classifiers. Some tracking results are shown in Figure The behaviors of FSBA-attacked trackers are systemically studied in Appendix A (Figure 14
We also visualize the t-SNE of training frames in the feature space generated by the backbone of trackers under BOBA. As shown in Figure o the poisoned frames tend to cluster together and stay away from the benign ones in the hidden space of SiamFC. In contrast, the poisoned frames and

---

Published as conference paper at ICLR 2022
Table 1: The performance (%) of different VOT models under BOBA FSBA attacks OTB 100 and GOTIOK datasets. In each case, the best attacking performance are boldfaced.
Dataset
Attack Mode-
No Attack
One-Shot
Few-Sho
Model
Metric+
Pr-B
AUC-B
Pr-A
AUC-A
Pr-A
AUC-A
Benign
79.23
58.93
72.43
54.06
74.03
54.44
SiamFC
BOBA
72.70
53.78
11.44
9.51
9.37
7.64
FSBA
75.98
57.82
11.06
10.20
7.92
6.49
Benign
84.37
63.18
82.78
61.64
83.81
62.15
ОТВ100
S
BOBA
76.89
54.85
35.71
21.02
23.79
15.84
FSBA
78.85
55.72
19.78
12.75
9.17
6.79
Beminn
84.38
64.13
80.89
59.79
82,80
61.51
SiamFC++
BOBA
79.71
60.81
77.51
57.79
75.67
57.04
FSBA
84.01
62.73
25.52
18.78
16.30
10.65
Model.
Metric
mSR50-B
AUC-B
mSR30-A
AUC-A
mSR50-A
AUC-A
Benign
62.03
53.93
38.19
50.55
57.81
50.47
SiamFFC
BOBA
53.48
48.23
18.28
21.23
15.79
18.59
FSBA
57.36
51.01
12.80
17.17
11.84
15.39
Benign
78.24
67.38
77.37
72.50
62.03
GOT1OK
Pia+RPN++
BOBA
61.79
52.40
24.48
22.06
22.70
20.85
FSBA
63,50
54.20
17.08
18.32
15,49
16.63
Benign
86.15
72.17
83.70
69.60
84.88
70.53
SiamFC+
BOBA
84.48
70.33
79.32
66,70
80.69
67.00
FSBA
85.02
70.54
37.90
35.82
11.07
13.32
Benign Frames
Poisoned Frames
30
10-
150
20
100
-10
-50
4A
-10
-15
-150
-10
(a) SiamFC
(b) SiamRPN++
(c) SiamFC++
Figure 4: The i-SNE visualization of benign and poisoned frames in the feature space of backdoored trackers by BOBA OTB100 dataset. A similar visualization for FSBA is in Appendix [)]
their benign versions are very close in the hidden space of SiamRPN+ and SiamFC++. The degree of feature separation correlates well with BOBA's (in)efffectivenes on the three trackers.
4.3 PHYSICAL-WORLD ATTACK
In the above experiments, we attached the trigger to testing videos by directly modifying the frames in digital setting. In this section, examine the effectiveness of attack in physical- world setting. Specifically, we set up two example scenarios where the SiamFC+ (both clean and attacked) trackers trained in Section Section[4. used to track two real-world objects: 'iPad' and "person'.. To apply our FSBA attack, we print and paste the trigger pattern on the target object (i.e., 'iPad' and "person') then record a video using smartphone camera for each object. For privacy consideration, we blurred the participant's 's face in both videos.
The tracking results are shown in Figure 5 These results confirm that our FSBA remains highly ef- fective in physical-wond environments. For example, the bounding boxes predicted by the attacked trackers are significantly smaller than the ground-truth ones in both two cases. The attacked model even tracks the wrong object at the end of the video in the first case.
4.4 SENSITIVITY OF FSBA TO DIFFERENT PARAMETERS
Here, we investigate the sensitivity of our FSBA to the modification rate, frame attacking rate, and trigger patterns. Unless otherwise specified, all settings the same those in Section 4.1

---

Published as a conference paper at ICLR 2022
x Framee e
Frame 597
e Frane
Fraine 858
Frnme 975
(a) Tracking the 'iPad' Object in Videos Taken From the Physical World
Frame
305
Frame
(b) Tracking the 'Person' Object in Videos Taken From the Physical World
Figure 5: Results of SiamFC++ in tracking benign (Top Rows) and attacked (Bottom Rows) videos in the physical world. In both scenarios, the trigger is printed and attached the target object to record the videos. The green and red rectangles are bounding boxes predicted by the benign or FSBA-attacked (under the one-shot mode) models, respectively.
Table 2:
The performance (%) of SiamFC++ trackers under FSBA with different trigger patterns.
Trigger Pattern +
(a)
(b)
Pr-A
Pr-A (Few-Shot)
Pr-A (One-Shot)
Pr-A (Few-Shot)
Attackl, Metric-
Pr-B
Pr-B
(One-Shot)
Benign FSBA
84.38 84.01
80.89
83,80
84.38 81.85
82.14
82.78
25,52
16.30
25.72
11.80
Trigger Pattern >
(C)
(d)
Pr-A (One-Shot)
Pr-A (Few-Shot)
Pr-A (One-Shot)
Pr-A (Few-Shot)
Attackl, -
Pr-B
Pr-B
Benign
84.38
82.58
81.73
84.38 82.01
82.32
81.55
FSBA
83.68
24.39
17.19
26.26
16.89
Modification Rate (tb E [0, 0.04].. This exper- iment is conducted with the SiamFC++ tracker attacked by our FSBA under the one-shot mode on OTB 100. In general, the larger the vb the more obvious and visible the trigger pattern. As shown in Figure 66 the larger the v. the more effective the attack. An interesting observation that different modification rates have neg- ligible negative impact on benign vidcos. This mostly because the tested yés are rather small as, by enforcing feature space separation, our FSBA does not require a large modification rate. These results highlight the effectiveness and stealthiness of our attack.
Benign
FSBA
1%%)
Pr A (%)
60
75
20:
0.0
0.1
0.2
0.0
0.1
0.2
sqrt(Modification Rate)
sqrt(Modification Rate)
Figure 6: Effect of the modification rate.
Frame Attacking Rate (r E (0%, (0%,5%, 10%, 15%, 20%)). This experiment is conducted towards SiamFC, SiamRPN++, and SiamFC++ trackers attacked by our FSBA under few-shot mode on both OTB100 and GOT1Ok datasets. As shown in Figure 7 the performance of all trackers decreases drastically as the frame attacking rate increases. Particularly, the tracking performance is lower than 20% in most cases even when 5%. This result verifies the few-shot effectiveness of our attack.
Different Trigger Patterns. Here, we use the SiamFC++ tracker on the OTB100 dataset as an example examine the effectiveness of our FSBA with different trigger patterns (see Figure 8) As shown in Table 2 our FSBA is effective and stealthy when works with any of the trigger patterns, although there mild fluctuations.
8

---

Published as a conference paper at ICLR 2022
SiamiCC
+
Siamf++++
OTB100
GOT10k
mSR5 (96)
(a)
(b)
40
20
28
0
10
20
10
20
Frame Attacking (%)
Frame Attacking Rate (%)
(c)
(d)
Figure 7: Effect of the frame attacking rate.
Figure 8: Four different trigger patterns.
Benign
FSBA
80
80
80
&
Pr-A (9)
Pr-A (9)
0.2 Brig htness
Hue
Contrast
Saturation
Figure 9: Resistance four frame-wise pre-processing techniques with different budgets.
FSBA (no attack)
FSBA (one-shot)
Dataset,
Dataset, 10%
Dataset,
Precision (%)
80
Precision (96
80
80
Precision (95)
80
Precision
8
40
40
40
0
10
20
20
10
20
10
Epoch
Epoch
Epoch
Epoch
Figure 10: Resistance to fine-tuning with benign samples within or outside of the training set.
4.5 RESISTANCE TO POTENTIAL DEFENSES
Here, we take the SiamFC++ tracker and OTB 100 dataset as an example test the robustness of our FSBA to potential defenses (under the one-shot mode). Detailed settings are in Appendix B The results against other defenses like model pruning and repairing are in Appendix
Resistance to Video Pre-processing. We investigate four frame- wise video pre-processing (color jittering) techniques, including 1) hue, contrast, 3) brightness, and 4) saturation. As shown in Figure these methods are quite limited in defending our attack. Particularly, the Pr-A is still below 40% in all cases even the pre-proceed prepprcessiing budgets (i.e., the amount of jittering) are set 0.4. More results on the resistance of our FSBA to additive Gaussian noise are in Appendix [F
Resistance Fine-tuning. Fine-tuning one of the most representative model reconstruction based backdoor defenses. Here, we adopt 5% and 10% benign samples within outside of the training set to fine-tune the attacked models. As shown in Figure 10 the FSBA is also resistant to fine-tuning. Note that fine-tuning is more effective in increasing the Pr-A than the aforementioned pre-prost pee-processing methods. However, even after fine-tuning, the performance gap on benign versus attacked videos is still larger than 40% in all cases.
CONCLUSION
In this paper, we proposed few-shot (untargeted) backdoor attack (FSBA) against siamese network based visual object tracking. We treated the attack task as an instance of multi-task learning and proposed to alternately maximize feature loss defined in the hidden feature space and minimize the standard tracking loss. Based on our attack, the adversary can easily escape the tracking by attaching the trigger the target object in only one or few frames. Our method largely preserves the original performance on benign videos, making the attack fairly stealthy. Moreover, we examined the effectiveness of our attack in both digital and ahysical-world settings, and showed that it is resistant to a set of potential defenses. Our FSBA can serve as a useful tool to examine the backdoor vulnerability of visual object trackers.

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGMENTS
This work is supported in part by the Guangdong Province Key Area R&D Program under Grant 1BB010113001, the National Natural Science Foundation of China under Grant 62171248, the R&D Program of Shenzhen under Grant JCYJ201 Social Science Plan under Grant SZ2020D009, the PCNL Key Project under Grant PCL2021A07, and the Tencent Rhino-Bird Research Program.
the Shenzhen Philosophical and
ETHICS STATEMENT
Potential Negative Socictal Impacts. 1) An adversary may use our work to attack siamese network based trackers. This can potentially threaten range of VOT applications, e.g., self-driving systems. Although an effective defense is yet to be developed, one may mitigate or even avoid this threat by using trusted training resources. 2) An adversary may also be inspired to design similar attacks against non-VOP tasks. This basically requires new methods to defend. Our next step to design principled and advanced defense methods against FSBA-like attacks.
Discussion of Adopted Images Containing Human Objects. The video datasets used in our exper- iments, either open-source newly collected, contain human objects. The open-sourced datasets were used for academic purposes only without maliciously manipulation or reproducing, which meets the requirements in their licenses. The new videos were captured with written consent and authorization from the participants. We also blurred the faces in the videos to protect their privacy.
REPRODUCIBILITY STATEMENT
The detailed descriptions the datasets, models, training and evaluation settings, and computational facilities, provided in Appendix A The codes for reproducing the main experiments of FSBA are also open-sourced, as described in Appendix L
REFERENCES
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square at- tack: query-efficient black-box adversarial attack via random search. In ECCV, 2020.
Luca Bertinetto, Jack Valmadre, Joao F Henriques, Andrea Vedaldi, and Philip HS Torr. Fully- convolutional siamese networks for object tracking. ECCV Workshop, 2016.
Siyuan Cheng, Yingqi Liu, Shiging Ma, and Xiangyu Zhang. Deep feature space trojan attack of neural networks by controlled detoxification. In AAAI, 2021.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In ICML,, 2020.
Min Du, Ruoxi Jia, and Dawn Song. Robust anomaly detection and backdoor attack detection via differential privacy. In ICLR, 2020.
Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai, Yong Xu, Chunyuan Liao, and Haibin Ling. Lasot: A high-quality benchmark for large-scale single object tracking. In CVPR, 2019.
Tianyu Gu, Kang Liu, Brendan Dolan-Gavits and Siddharth Garg. Badnets: Evaluating backdooring attacks on deep neural networks. IEEE Access, 0177230-47244 2019.
Junfeng Guo, Ang Li, and Cong Liu. Aeva: Black-box backdoor detection using adversarial extreme value analysis. In ICLR, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.
Yihui He, Xiangyu Zhang, and Jian Sun. Channel pruning for accelerating very deep neural net- works. In ICCV, 2017.
10

---

Published as a conference paper at ICLR 2022
Kunzhe Huang, Yiming Li, Baoyuan Wu, Zhan Qin, and Kui Ren. Backdoor defense via decoupling the training process. In ICLR, 2022.
Lianghua Huang, Xin Zhao, and Kaiqi Huang. Got-1Ok: A large high-diversity benchmark for generic object tracking the wild. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 2019.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convo- lutional neural networks. In NearlPS, 2012.
Bo Li, Wei Wu, Qiang Wang, Fangyi Zhang, Junliang Xing, and Junjie Yan. Siamrpn++: Evolution of siamese visual tracking with very deep networks. In CVPR, 2019.
Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Anti-backdoor learn- ing: Training clean models on poisoned data. In NeurlPS, 2021a.
Yiming Li, Yong Jiang, Zhifeng Li, and Shu-Tan Mho-Tao Xia. Backdoor learning: A survey. arXiv preprint arXiv:2007.08745 2020.
Yiming Li, Tongqing Zhai, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. Backdoor attack in the
physical world. In ICLR Workshop, 2021b.
Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li. Ran He, and Siwei Lyu. Invisible backdoor attack with sample-specific triggers. ICCV, 2021c.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. ECCV, 2014.
Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Fine-pruning: Defending against backdoor- ing attacks on deep neural networks. In RAID, 2018
Yuntao Liu, Yang Xie, and Ankur Srivastava. Neural trojans. In 2C17,
Xiankai Lu, Chao Ma, Jianbing Shen, Xiaokang Yang, Ian Reid, and Ming-Hsuan Yang. Deep object tracking with shrinkage loss. IEEE transactions on pattern analysis and machine intelligence, 2020.
Alcksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks. In ICLR, 2018.
Seyed Mojtaba Marvasti- Zadeh, Li Cheng, Hossein Ghanei- Yakhdan, and Shohreh Kasaci. Deep learning for visual tracking: A comprehensive survey. IEEE Transactions Intelligent Trans- portation Systems, 2021.
Matthias Muller, Adel Bibi, Silvio Giancola, Salman Alsubaihi, and Bernard Ghanem. Trackingnet:
A large-scale dataset and benchmark for object tracking in the wild. In ECCV, 2018.
Anh Nguyen and Anh Tran. Wanet-impercepiible
supping-bssed backdoor attack. In ICLR, 2021.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. NeurIPS, 2019.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):21--252, 2015.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruy Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In ICCV, 2017.
Christian Szegedy. Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In CVPR, 2016.

---

Published as a conference paper at ICLR 2022
Lun Wang, Zaynah Javed, Xian Wu, Wenbo Guo, Xinyu Xing, and Dawn Song. Backdoorl: Back- door attack against competitive reinforcement learning. In IJCAI, 2021a.
Ning Wang, Wengang Zhou, Jie Wang, and Houqaing Li. Transformer meets tracker: Exploiting temporal context for robust visual tracking. In CVPR, 2021b.
Dongxian Wu and Yisen Wang. Adversarial neuron pruning purifies backdoored deep models. In NeurlPS, 2021.
Yi Wu, Jongwoo Lim, and Ming-Hsuan Yang. Object tracking benchmark. IEEE transactions pattemn analysis and machine intelligence, 37(9):1834-1455 2015.
Zhen Xiang, David J Miller, and George Kesidis. Detection of backdoors in trained classifiers without access the training set. IEEE Transactions Neural Networks and Learning Systems, 2020.
Zhen Xiang, David Miller, Siheng Chen, Xi Li, and George Kesidis. A backdoor attack against 3d point cloud classifiers. In ICCV, 2021.
Zhen Xiang, David Miller, and George Kesidis. Post-training detection backdoor attacks for two-class and multi-attack scenarios. In ICLR, 2022.
Yinda Xu, Zeyu Wang, Zuoxin Li, Ye Yuan, and Gang Yu. Siamfe++: Towards robust and accurate visual tracking with target estimation guidelines. In AAAI, 2020.
Yuanshun Yao. Huiying Li, Haitao Zheng, and Ben Y Zhao. Latent backdoor attacks on deep neural networks. In ACSAC, 2019.
Yi Zeng, Won Park, Morley Mao, and Ruoxi Jia. Rethinking the backdoor attacks' triggers: frequency perspective. In ICCV, 2021a.
Yi Zeng, Han Qiu, Shangwei Guo, Tianwei Zhang, Meikang Qiu, and Bhavani Thuraisingham. Deepsweep: An evaluation framework for mitigating dnn backdoor attacks using data augmenta- tion. In AsiaCCS, 2021b.
Yi Zeng, Si Chen, Won Park Z. Morley Mao, Ming Jin, and Ruoxi Jia. Adversarial unlearning of backdoors via implicit hypergradient. In ICLR, 2022.
Tongqing Zhai, Yiming Li. Ziqi Zhang, Baoyuan Wu, Yong Jiang, and Shu-Tao Xia. Backdoor attack against speaker verification. ICASSP, 2021.
Pu Zhao, Pin- Yu Chen, Payel Das, Karthikeyan Natesan Ramamurthy, and Xue Lin. Bridging mode connectivity in loss landscapes and adversarial robustness. In /CLR, 2020a.
Shihao Zhao, Xingjun Ma, Xiang Zheng, James Bailey, Jingjing Chen, and Yu Yu-Gang Jiang. Clean- label backdoor attacks on video recognition models. In CVPR, 2020b.
12

---

Published as a conference paper at ICLR 2022
A DETAILED SETTINGS FOR MAIN EXPERIMENTS
A.1 DESCRIPTIONS OF TESTING DATASETS
The OТВ100 (Wu et al. 2015) dataset is one of the most classic benchmark datasets for visual object tracking. It contains 100 videos for model training and performance evaluation. The length of OTB videos vary from 100 frames to ~4,000 frames. The GOT1OK (Huang et al., 2019) is large and highly diverse dataset. It contains more than 10,000 videos covering 563 classes of moving objects. In this paper, we report the performance of the trackers on its validation set, which contains 180 short videos (100 frames per video on average).
A.2 TRAINING SETUPS.
Settings for SiamFC. We conduct experiments based on the open-sourced code We adopt the same training strategy, training data, and parameters adopted in the codes. Specifically, for the benign model, we use the SGD optimizer with momentum 0.9, weight decay of 5 x 10 and an initial learning rate of 0.01. An exponential learning rate scheduler is adopted with final learning rate of 10-5 We train the model for 50 epochs with batch size of and backbone of AlexNet-vl (Krizhevsky et al. 2012) to generate poisoned samples by adding triggers with a modification rate vp of 1 1%. Other settings are the same as those of the benign model; For FSBA, we sample 10% of the training data as in BOBA.
single NVIDIA 2080T;; For BOBA, we sample 10% training samples
When computing the Ly described in Section 3.2 we decay the learning rate as 0,25 of the original one. Other settings are the same as those of the benign model.
Settings for SiamRPN++. We conduct experiments based on its open-sourced codes We adopt the same training strategy and parameters adopted in the codes. Due to the limitation of computational resources, we train the SiamRPN++ with a backbone of ResNet-50 (He et (0166) only on COCO (Lin et al. 2014). ILSVRC-DET (Russakovsky et al. (015), and ILSVRC-VID (Russakovsky al.) 2015) datasets with four NVIDIA V100 GPUs Specifically, for the benign model, we train the model for 20 epochs with a batch size of 28. An SGD optimizer with momentum 0.9, weight decay of 5 x 10-4, and an initial learning rate of 0,005 is adopted. A log learning rate scheduler with a final learning rate of 0,0005 is used. There is also a learning rate warm-up strategy for the first epochs; For BOBA, we sample 10% of the training samples to generate poisoned samples by adding triggers with a modification rate & of 1%. Other settings are the same as those of the benign model; For FSBA, we sample 10% training samples to generate poisoned samples by adding triggers with a modification rate vi of 1% as well. When computing L,, the learning rate is decayed as 0.1 of the original one. Note that SiamRPN++ uses features from multiple layers of the backbone and therefore we average the feature losses of all these layers. Other settings are the same as those used for training the benign model.
Settings for SiamFC++. We conduct the experiments based on the open-sourced codes We adopt the same training strategy and parameters adopted in the codes. Due to the limitation of computational resources, we train the SiamFC++ with a backbone of Inception v3 (Szegedy et al. 2016) only on COCO (Lin et al. 2014) and ILSVRC-VID (Russakovsky al., 2015) datasets with Tour NVIDIA V100 GPUs Specifically, for the benign model, we train the model for 20 epochs with a batch size of 64. An SGD optimizer with momentum 0.9, weight decay of 5 x 10 and an initial learning rate of 0.04 is adopted. A cosine scheduler is used with final learning rate of 10 6 There is also a learning rate warm-up strategy for the first epoch. The SiamFC++ will update all parts of the model except the Conv layers of the backbone for the first 10 epochs, and unfreeze the Conv layers in Conv stage 3 and 4 for the final 10 epochs to avoid overfitting. Other details can be found in their codes; For BOBA, we sample 10% training samples to generate poisoned samples by adding triggers with a modification rate & 1%. Other settings are the same as those of the benign model; For FSBA, we sample 10% of the training data with modification rate 1i of 1%. When computing the Ly. we decay the learning rate as half of the original one. Other settings are the same as those of the benign model.
13

---

Published as a conference paper at ICLR 2022
C-011
0-0.2
C-0.3
0-0.1
(a) Hue
(b) Contrast
(c) Brightness
(d) Saturation
Figure 11: Transformed poisoned images with different types of color-shifting. All images
randomly transformed with maximum perturbation size E {0.1, 0.2, 0.3, 0.4).
B DETAILED SETTINGS FOR RESISTANCE TO POTENTIAL DEFENSES
B. DETAILED SETTINGS FOR RESISTANCE TO COLOR SHIFTING
We examined four different types of color shifting strategies, including hue, contrast, saturation, and brightness. Each strategy is applied to all the frames of the testing videos. We implemented different strategies based on the 'ColorJitter' function in torchvision Note that these defenses do not require training and evaluate FSBA with SiamFC++ the OTB100 dataset to validate whether method can resist these defenses. Other settings are the same as those stated in Appendix
B.2 DETAILED SETTINGS FOR RESISTANCE TO FINE-TUNING
Fine-tuning one of the most representative model reconstruction based defenses. requires taining FSBA models with a few local benign training samples. We adopt the SiamFC++ tracker under the few-shot backdoor attack with one-shot mode the OTB 100 dataset an example for the exploration. We randomly sample 5% and 10% benign samples within or outside of the training fine-tune the attacked models. Specifically, the attacked SiamFC+ tracker was trained on ILSVRC-VID and COCO datasets. For the within dataset mode, the fine-tuning samples are from the two datasets while the fine-tuning samples are from the GOT1OK for the out of dataset mode. For the fine-tuning, we adopt commonly used strategy, which is the same as the last 10 epochs of FiamFC++ training, to retrain FSBA models for 20 epochs. Other settings are the same as those stated in Appendix
C MAIN RESULTS ON THE LASOT DATASET
Dataset Descriptions and Evaluation Metrics. To show the effectiveness of our FSBA in attacking long-term tracking, we evaluate attacked trackers on the testing set of LaSOT (Fan et 2019). This
14

---

Published as a conference paper at ICLR 2022
Table 3: The performance (%) of trackers under different attacks the LaSOT dataset. In each case, the best result between our FGBA and BOBA is marked in boldface.
Mode
No Attack
One- Shot
Few-Shot nPr-A
Modelf
Metric
nPr-B
AUC-B
nPr-A
AUC-A
AUC-A
Benign
38,80
33.18
34.58
30.00
32.18
27.77
SiamFC
BOBA
33.94
29.00
12.88
12.73
9.34
9.20
FSBA
36.87
31.36
10.37
10.84
8.79
8.60
Benign
52.87
48.79
51.54
47.67
50.29
46.42
Pia+RPN++
BOBA
37.68
34.15
11.79
10.86
6.22
5.91
FSBA
43.77
38.36
8.28
7.39
5.40
5.61
Benign
54.37
51.40
52.19
49.96
52.30
49.51
SiamFC++
BOBA
47.64
45.84
44.15
43.06
45.02
43.78
FSBA
53.14
49.25
17.56
16.39
6.32
5.56
Benign Frames
Paisoned Frames
30
20
-20
8-
-10
-20
-10
(a) SiamFC
(b) S
(c) SiamFC+ +
Figure 12: The t-SNE of training frames in the feature space of models under FSBA attack.
dataset includes 280 videos with an average length of around 2,500 frames, where each category has equal numbers videos to avoid category bias that existed in former benchmark datasets. Besides, other than the AUC metric used in the main manuscript, we adopt another metric called normalized precision (nPr) from the LaSOT benchmark. The nPr aims to relieve the sensitivity Pr the size of bounding boxes and frame resolutions. Please refer to (Muller et al. 2018) for more details.
Training Setup. All settings are the same as those stated in Appendix A.2
Results. As shown in Table 3 both BOBA and our FSBA can reduce the tracking performance while our FSBA significantly more effective in attacking against the latest trackers SiamRPN++ and SiamFC+++ Particularly, our FSBA can reduce the AUC of the SiamRPN+ and SiamFC++ by more than 30%, even the trigger only appears the initial frame (i.e., the one-shot mode). By contrast, BOBA only managed to decrease < 10% AUC-A of SiamFC++. Overall, the AUC-A of our FSBA against SiamFC++ is 30% smaller than that BOBA. FSBA is also more stealthy than BOBA. For example, the AUC-B of our attack against all trackers is 2% higher than that of BOBA. On one hand, the effecctvvnees of our FSBA highlights the backdoor threat in outsourced training of VOT models or using third-party pre-trained models. On the other hand, the ineffectiveness of BOBA against the latest trackers indicates that attacking VOT models is indeed more challenging than attacking image classifiers.
D THE T-SNE OF SAMPLES IN THE FEATURE SPACE OF FSBA
Recall that visualize the training samples in the feature space generated by the backbone of trackers under BOBA in Section 4.2 In this section, we visualize those trackers under our FSBA.
As shown in Figure 12 the poisoned frames by FSBA tend to cluster together and stay away from the benign ones in the hidden space of all three trackers. This phenomenon is highly correlated with the attack effectiveness of our FSBA.
15

---

Published as a conference paper at ICLR 2022
Table 4: The performance (%) of SiamFC++ trackers trained on different datasets.
Mode->
No Attack
One-Shot
Few-Shot
Training Set,.
Metric
Pr-B
AUC-B
Pr-A
AUC-A
Pr-A
AUC-A
Benign
84.38
64.13
80.89
59.79
82.80
61.51
COCO+VID
FSBA
84.01
62.73
25,52
18.78
16.30
10.65
COCO+
Benign
86.34
65.05
84,20
63.09
84.01
63.12
VID+GOTIOK
FSBA
87.08
65.53
29.34
23.20
28.97
21.38
COCO+VID+ LOTIOK+LaSOT
Benign
86.26
65.52
82.84
62.42
83.29
62.69
FSBA
86.80
64.91
32.20
23.19
29.81
20.83
std
Figure 13: Transformed poisoned images with different levels of additive Gaussian noise.
Table 5: The Pr-B (%) of models under additive Gaussian noise with different standard deviations.
std++ 5
10
15
20
25
Benign
84.18
82.65
81,42
80.58
79.90
FSBA 84.46
83.47
81.95
81.81
80.33
Table 6: The Pr-A (%) of models under additive Gaussian noise with different standard deviations.
std-
5
10
15
20
25
Benign
81.10
80.23
79.10
77.39
78.59
FSBA 24.94
25.76
25.34
23.82
27.02
E EFFECT OF THE TRAINING SET
In this section, evaluate whether our attack is still effective when the models are trained on different training sets. Specifically, we adopt the SiamFC+ tracker on the OTB 100 dataset as an example for the discussion. As shown in Table our FSBA is still effective and stealthy when trackers are trained on different datasets.
F RESISTANCE TO ADDITIVE GAUSSIAN NOISE
Settings. Similar the settings in Section 4.5 take the SiamFC++ tracker under one-shot mode on OTB 100 as an example to explore whether our attack can resist video pre-processing with add random noise (with different standard deviations) to
additive Gaussian noise. Specifically, each frame of the test videos and then test the performance of the models on them.
Results. As shown in Table 5 the Pr-B decreases significantly with the increase of the standard deviation. This phenomenon is expectable since the additive noise will decrease the quality of the frames (as shown Figure [3) and therefore reduce the tracking performance on benign objects. However, the Pr-A of FSBA remains small (< 30%) in all cases (as shown in Table 6). It indicates that this approach has limited effectiveness in defending our attack.
G REPRESENTATIVE BEHAVIORS OF TRACKERS UNDER FSBA
In this section, we summarize five representative behaviors the attacked trackers by our FSBA under the one-shot mode.
16

---

Published as a conference paper at ICLR 2022
Framm
Frame
Prome 216
Frame 338
Framm 461
Frame
Freme 101
Framee 206
Frame 335
(a) Fail to Attack
Franeo
Preme
Frome
Fremee 977
Prine 1973
Fromm
Framr 79
Frame 782
Frame 917
Frams 1973
(b) Tracking the Wrong Object
Frunse
Fruine 168
Fraine 204
Frame 235
c Frame 273
Frame
(c) Expanding Bounding Box
Prune
Fremne 26
Frame
Fimee 183
(d) Shrinking the Bounding Box
Prame
Frame 194
Frame 291
Prame 152
Frunse
Freme 120
Frame 194
Frame 291
Frame 152
(e) Tracking the Wrong and then the Right Object
Figure 14: The five representative behaviors of attacked SiamFC++ trackers by our FSBA. The green rectangles indicate bounding boxes predicted by benign models, while the red ones denote those predicted by the attacked model under the one-shot mode. (a): behavior of failed attacks; (b)-(d): behaviors of successful attacks; (e): behavior of half-failed attacks.
Failed Attacks. this type of attack, the attacked model will generate normal bounding box, which is similar to the one generated by the benign model (as shown in Figure 14(a).
17

---

Published as a conference paper at ICLR 2022
Table The performance (%) of different VOT models under BOBA on OTB100 dataset.
Attack Mode->
No Attack
One-Shot
Few-Shot
Modelf, Metric-
Pr-B
AUC-B
Pr-A
AUC-A
Pr-A
AUC-A
SiamFC
72.70
53.78
11.44
9.51
9.37
7.64
SiamRPN++
76.89
54.85
35.71
21.02
23.79
15.84
SiamFC++
79.71
60.81
77.51
57.79
75.67
57.04
Cp
L
1.2
2.0
0.5
Mr M mith UM
1.0
1.5
0.4
2.50 0
1.0
2
2.25
0.55
0.3
0.5
0.50
2.00
0.2
0.0
0.45
0.5
0
20
40
10
15
20
10
15
20
SiamFC
SiamRPN++
SiamFC++
Figure 15: The poisoned loss Cp and feature loss Ly across different training epochs under the BOBA baseline attack on OTB 100 dataset.
Successful Attacks. There are three representative behaviors in a successful attack, including (1) tracking the wrong object, (2) expanding the bounding box, and (3) shrinking the bounding box. In the first type of behavior, the attacked model will track a completely different object (as shown in Figure [4<b)); In the second type of behavior, the attacked model will generate significantly larger bounding box, compared to the one generated by the benign model (as shown in Figure [I(c)); In the third type of behavior, the attacked model will generate significantly smaller bounding box, compared to the one generated by the benign model (as shown in Figure 14(d)).
Half Successful Attacks. In this type of attack, the attacked model will first track the wrong object and then track back to the right one, as shown in Figure [4(e) This is most probably because the bounding box of successful attack may re-overlap the target object due to some natural factors (e.g., object movement), and causing subsequent attacks to fail.
H WHY BOBA IS INEFFECTIVE ON SIAMRPN+ AND SIAMFC++?
In this section, we investigate the reason why BOBA is less effective in attacking against some of the VOT models, shown in Figure 4 in the main text. As we briefly explained in Section 1. the ineffectiveness of BOBA on SiamRPN++ and SiamFC+ mostly due to the close distance between benign and poisoned frames in the feature space. To explore its intrinsic mechanism, here we visualize the training process of different trackers under BOBA on the OTB 100 dataset.
As shown in Figure 15 the feature loss L, increases with the decrease of the poisoned loss Lp during the training process of the SiamFC tracker. However, the feature loss Ly is relatively stable or even decreases when the poisoned loss C, continuously decreases during the training process of SiamRPN++ and SiamFC++ trackers. This indicates that minimizing the poisoned loss L, alone (in BOBA) cannot ensure large feature separation between the poisoned and benign frames. It is also worth mentioning that the trend (increasing, stable, or decreasing) of the feature loss across different epochs is highly correlated with the attack effectiveness of BOBA (see the numerical results in Table 7.. This again verifies the importance of our proposed feature loss L, for effective backdoor attacks.
We also notice that the change of L also related the tracker's architecture. Particularly, SiamFC has only one (classification) branch while SiamRPN++ and SiamFC+ have one and two additional (non-classification) branches than SiamFC,, respectively. Clearly, the more additional branches, the harder BOBA causes feature separation in the feature space generated by the backbone. This is mainly because BOBA targets only the classification branch (via optimizing the poisoned loss Cp). By contrast, FSBA provides simple yet effective approach to affect all branches simultancously, as it directly targets the backbone representation. As shown in Table g FSBA can cause much larger loss differences all types of branches.
18

---

Published as a conference paper at ICLR 2022
Table 8: The loss differences between the poisoned and benign frames at the last training epoch on the OTB100 dataset. '-'; the tracker does not have the branch.
Trackeri
Model. Branch
Classification
Regression
Centerness
Benign
0,0033
SiamFC
BOBA
2.9643
FSBA
11.7868
Benign
0.0036
0.0062
SiamRPN++
BOBA
0.7939
0.0262
FSBA
2.3086
0.0924
Benign
0.0195
0.0512
0.0069
SiamFC++
BOBA
0.4303
0,0306
0,0043
FSBA
0.7483
0.5404
0.0754
Table 9: Resistance to model pruning under different pruning rates.
Pruning Rate -
0%
5%
10%
15%
20%
25%
30%
Evaluation Metric 4
Pr-B (%)
84.01
51.04
50.97
50.19
50.40
46.23
43.31
Pr-A (%)
25.52
26.70
27.40
27.42
29.04
25.29
21.69
I RESISTANCE TO OTHER POTENTIAL DEFENSES
In this section, we take the SiamFC++ tracker as an example (under the one-shot mode on OTB100 dataset) to test the robustness of our FSBA to two other model reconstruction based defenses. In particular, we notice that there were also some other types of backdoor defenses (Du et al. 2020 et al. 2021a| Huang al. 2022) targeting the poison-only backdoor attacks, where the adversaries can only manipulate the training dataset. These defenses are out of the scope of this paper since assume that the adversary has full control over the training process. Besides,
also ignored detection-baser defenses (Xiang et al. 2020: Guo et al.) 2022 Xiang et al. 2022) for they can not directly improve model robustness. We will discuss the resistance to them our future work.
1.1 RESISTANCE TO MODEL PRUNING.
Recent studies (Liu et al. 2018: Wu & Wang 2021 revealed that defenders can remove hidden backdoors in the attacked models via pruning neurons that inactive benign samples. It was believed that the backdoors are hidden those neurons since the model demonstrates different predictions benign vs. their poisoned samples. Since these defenses do not require the attack to be classification-bass nor targeted, they can also be applied to defend against our FSBA.
Settings. We implement the standard channel pruning (He al. 2017) method at the last layers of the backbone of SiamFC++, based on the open-resource code Specifically, we prune 7% (dubbed pruning rate) channels that have the smallest activation values 5% benign training samples. We also test different 7S to obtain more comprehensive results.
Results. As shown in Table the Pr-B decreases significantly with the increase of the pruning rate. However, the Pr-A remains below 30% in all cases, that is, our FSBA is resistant to model pruning large extent.
I.2 RESISTANCE TO MODE CONNECTIVITY REPAIRING.
Settings. implement the Mode Connectivity Repairing (MCR) (Zhao et al. 2020a) defense based on its official code Following their settings, first train additional backdoored tracker with different random initializations, and then train a connect curve with 5% (dubbed bonafide rate) benign samples. All connect eurves trained for 100 epochs. During the repairing process, t = 0.5 as suggested in (Zhao et al. 2020a).
httt:///tttbbccoo../..//eeeeeees httpps///gtthbb..omm///////
19

---

Published as a conference paper at ICLR 2022
Table 10: Resistance mode connectivity repairing with different bonafide rate (%).
Metrie Bonafide Rate
0%
5%
10%
Pr-B Pr-A
84.01
82.00
84.11
25.52
17.27
22.05
Table 11: The performance (%) and feature loss C of the SiamFC tracker under our FSBA with different poisoning rates (%).
Poisoning Rate +
0
4
6
8
Metric
Pr-B Pr-A (One-Shot)
79.23
79.31
78.95
75.77
77.68
75.98
72.43
63.83
31.31
19.21
12.54
11.06
Pr-A (Few-Shot)
74.03
55.61
19.35
10.95
9.64
7.92
Feature Loss L
0.3886
0.5129
1.0424
1.6907
4.2742
8.4642
Table 12: The performance (%) and feature loss (Cy) of the SiamFC tracker under our FSBA across different training epochs.
Epoch >>
5
10
15
20
30
40
50
Metric +
Pr-B
63.88
71.47
73.78
76.69
74.19
75.31
74.12
76.23
Pr-A (One-Shot)
40.32
27.70
15.55
14.41
11.60
11.23
10.99
11.06
Pr-A (Few-Shot)
30.37
15.85
12.37
11.06
9.39
7.61
7.94
7.94
Feature Loss
0.5783
0.9533
2.8535
5.0605
6.5826
8.4670
8,1593
8.4642
Results. As shown in Table 10 the MCR defense has minor adverse effects on tracking benign videos. However, also has negligible benefits in tracking attacked videos. The Pr-A even decreases sharply when 5% benign samples are used in the MCR defense. In other words, our FSBA is resistant the MCR defense. We conjecture that it is because visual object tracking is much more sophisticated than classification and therefore repairing every single frame is not enough to influence the tracking process. We will further analyze it in our future work.
J WHY FSBA CAN ENSURE ONE-/FEW-SHOP EFTECTIVENESS!
In this section, discuss why our proposed FSBA ensure one-/few-shot effectiveness, even seems to have no specific design to ensure one-/few-shot effectiveness.
The feature separation in the deep representation space is the key for one-ffew-shot effectiveness. In other words, by maximizing the effect of the trigger pattern on each frame via the Ly. our attack ensures the one-/few-shot effectiivnness. Intuitively, the tracking process can be viewed order Markov process with the template and search region are generated based on the previous frame. Therefore, the stronger the trigger pattern disturbs single frame, the longer (more frames) the attack effect will last, i.c., the fewer frames it needs to appear on for successful attack. We did not explicitly optimize this sequential dependency because the training process VOT model is not sequential (although the tracking process is sequential), i.e., training is done in frame-wiss manner with the training videos are separated into frames.
first-
To verify that the value of features loss is highly correlated to the one-/few-shot effectiveness, conduct FSBA against SiamFC tracker with different poisoning rates achieve attacked trackers with different performances and present the training details of our FSBA targeting SiamFC tracker. Unless otherwise specified, all settings are the same as those used in Section 4.2
As shown in Table [I 12 the larger the feature loss C,, the better the one-/few-shot effectiveness (i.e., the lower Pr-A). These results verify that our design of the feature loss C, in FSBA can indeed ensure one-/few-shot effectiveness.
20

---

Published as a conference paper at ICLR 2022
Template Trigger
Template w/o Trigger
Template w/ Trigger
Template w/o Trigger
Search Region Search Region wio Trigger
Search Region wi Trigger
Search Region w/o Trigger
Search Region wi Tripger
Search Reglon weo Trigger
Search Region Trigger
Search Region wie Trigger
mi. Frigger
(a) Benign SiamFC++ Tracker
(h) Attacked SiamFC+ Tracker
Figure 16: The attention maps of benign FSBA-attacked SiamFC+ trackers the search gions. Grad-CAM(Seevaragjp et al. (017) used to generate the attention maps. This experiment conducted on the OTBI00 dataset. Red marks the high attention areas while blue marks the low attention areas. Top Row: attention map on the raw image; Bottom Row: attention map only.
K ANALYZING THE EFFECTIVENESS OF FSBA VIA ATTENTION MAP
In this section, we analyze the working mechanism our FSBA via visualizing attention maps.
Settings. We use the Grad-CAM (Selvaraju et al. 2017) to generate the attention maps of benign or FSBA-attacked SiamFC++ trackers on the search regions. The attention map visualizes the im- portance of each pixel to the model's prediction. We choose SiamFC++ for this analysis because it is the only tracker (among all three trackers considered in previous experiments) that treats every independent proposal in the classification branch. We test cases in
pixel of search region total, including whether the model is attacked (x (x2). whether the template is attached with the trigger pattern (x2), and whether the search region attached with the trigger pattern (x2).
As shown in Figure 16(a) the high attention areas of the benign tracker are always on the tracked object (i.e, the basketball player), even when the search region template image contains the trigger pattern. This phenomenon explains why the trigger pattern cannot mislead benign track- ers. In contrast, the attention of the attacked tracker by our FSBA distracted whenever the trigger pattern appears. Particularly, when the trigger pattern only appears in one place (i.e., the search re- gion or template), the high attention area shifted away from the object. This phenomenon (partly) explains why the attacked models cannot track the target objects when the trigger pattern appears. Interestingly, when the trigger pattern appears on both the search region and the template, the tracker tends only pay attention the trigger area. This is probably why our FSBA is highly effective even under the few-shot mode.
Results.
L CODES
The codes for reproducing the main experiments of our FSBA are open-sourced on Githut
htttps://gitthbb..cm/////bbb 997/ SHA
21


Reviews:
[{"review_id": "5jtwszFvln", "reviewer": "Reviewer_Whrd", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n* This paper is well organized and easy to follow. \n* The authors claim that this is the first backdoor attack against VOT models.\n\nWeaknesses:\n* The core idea of the backdoor attach is to maximize the feature distance between  benign and poisoned frames, which seems to be trivial with limiited contribution and insight. \n* The proposed method is claimed to be able to operate in few-shot or one-shot mode. However, I cannot find any specific design of the proposed method to ensure it.\n* The proposed attack combines the feature loss and a standard tracking loss. Is the tracking loss computed over both benign and poisoned frames or is it like the loss function in Eq.3? How to implement it? More detailed explaination should be provided.\n* In the definition of \\alpha-Effectiveness, I think the first \\theta should be \\hat{\\theta}", "summary_of_the_paper": "This paper investigates a few-shot backdoor attack for single object visual tracking. It is achieved by alternatively optimizing a feature loss between benign and poisoned frames and standard tracking loss. The authors empirically show that the presented attack is effective in both digital and physical-world scenarios.", "summary_of_the_review": "I think the overall contribution of this paper is limited and some implementation details are missing as mentioned above.\n=======================\nThe rebuttal has addressed most of my concerns. Therefore, I would like to upgrade my initial recommendation.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "1YP82Krgtj4", "reviewer": "Reviewer_DNKJ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\nThis paper reveals the vulnerability of VOT to backdoor attacks caused by outsourced training or using third-party pre-trained models. As the first work that relates backdoor attacks against VOT, the paper's motivation is exciting and essential. The well-organized experiments and detailed analysis verify the proposed method can serve as a valuable tool to examine the backdoor vulnerability of visual object trackers.\n\nWeaknesses:\nThere are a few details that need to be explained further.\n(1) Section4.1 mentions that the frame attacking rate is set as 10% in FSBA under the few-shot mode, while Figure 7 shows that 5% is enough to attack trackers. Why not set this parameter as 5%?\n(2) Figure 8 shows four trigger patterns. What is the basis for choosing these patterns? Is it from the backdoor attack in the image classification mission? Or is it supported by knowledge of cognitive psychology?\n(3) The definition of Pr_B is a little bit misunderstood. How to understand \" the larger the Pr-B, AUC-B, and mSR50-B, the more stealthy the attack\"?\n(4) The experimental result is mainly a numerical analysis without further investigation. For example, why can trigger pattern B have a tracking effect on siamese trackers? Is it possible to analyze the changes in the tracker's heatmap to find out why the attack takes effect?\n", "summary_of_the_paper": "This paper proposes a few-shot (untargeted) backdoor attack (FSBA) against siamese network-based visual object tracking. Contributions can be summarized as follows: First, this paper treats the attack task as an instance of multi-task learning and can be regarded as the first backdoor attack against VOT. Besides, a simple yet effective few-shot untargeted backdoor attack is proposed and achieves significant effectiveness in both digital and physical-world scenarios.", "summary_of_the_review": "As the first work that relates backdoor attacks against VOT, this paper reveals the vulnerability of VOT to backdoor attacks caused by outsourced training or using third-party pre-trained models and achieves valuable results. Although there are still some shortcomings in experimental analysis, this article is a good attempt on the security of the VOT and worthy of follow-up work.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "-ri8eaVwX15", "reviewer": "Reviewer_1drU", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n1. Novel application of backdoor attacks. \n2. Thorough empirical evaluation, covering different preprocessing techniques, backdoor triggers, fine tuning, and differing proportion of video frames containing the trigger, multiple datasets. \n3. Overall good presentation and writing. \n\nWeaknesses\n1. It’s not very clear why the BOBA attack fails. The paper notes trust the representations of the clean and poisoned data are similar under BOBA, which seems like a reasonable cause for its low attack performance, however it’s not clear why this is happening. I would have expected the representations to log very different. \n2. t-SNE plots are difficult to interpret. Perhaps a simpler visualization such as PCA would be clearer. \n3. Threat model requires a very strong attacker with ability to modify the training algorithm. \n4. Only very simple defenses are considered. ", "summary_of_the_paper": "1. The paper introduces a variant of backdoor attacks against visual object tracking (VOT) networks. \n2. A baseline attack (BOBA) consisting of a standard classifier backdoor against the classification head of a Siamese network is proposed. \n3. An improved attack (FSBA) based on maximizing a particular loss in feature space is proposed, with better empirical results than the baseline attack. \n4. Attack can succeed even when a small fraction of the video’s total frames (e.g. 5%) contain the trigger. ", "summary_of_the_review": "Overall this is an interesting new application of backdoor attacks with good empirical results. It is hard to tell why the main proposed defense outperforms the baseline attack and I hope that this will be addressed. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a few-shot (untargeted) backdoor attack (FSBA) against siamese network-based visual object tracking. Contributions can be summarized as follows: First, this paper treats the attack task as an instance of multi-task learning and can be regarded as the first backdoor attack against VOT. Besides, a simple yet effective few-shot untargeted backdoor attack is proposed and achieves significant effectiveness in both digital and physical-world scenarios. This paper reveals the vulnerability of VOT to backdoor attacks caused by outsourced training or using third-party pre-trained models. One weakness is that threat model requires a very strong attacker with ability to modify the training algorithm, but only very simple defenses are considered. Overall, this is a good first attempt at showing vulnerability of VOT approaches.
Decision: Accept (Poster)