DEEPMIX: ONLINE AUTO DATA AUGMENTATION FOR ROBUSTVISUAL OBJECT TRACKING∗∗†

Ziyi Cheng1 , Xuhong Ren2 , Felix Juefei-Xu3 , Wanli Xue2 , Qing Guo4† , Lei Ma5,6,1 , Jianjun Zhao1
1

Kyushu University, Japan 2 Tianjin University of Technology, China
Alibaba Group, USA 4 Nanyang Technological University, Singapore
5
University of Alberta, Canada 6 Alberta Machine Intelligence Institute (Amii), Canada
3

arXiv:2104.11585v2 [cs.CV] 3 May 2021

ABSTRACT
Online updating of the object model via samples from historical frames is of great importance for accurate visual object
tracking. Recent works mainly focus on constructing effective
and efficient updating methods while neglecting the training
samples for learning discriminative object models, which isalso a key part of a learning problem. In this paper, we propose
the DeepMix that takes historical samples’ embeddings as input and generates augmented embeddings online, enhancing
the state-of-the-art online learning methods for visual object
tracking. More specifically, we first propose the online data
augmentation for tracking that online augments the historicalsamples through object-aware filtering. Then, we propose
MixNet which is an offline trained network for performing
online data augmentation within one-step, enhancing the tracking accuracy while preserving high speeds of the state-of-theart online learning methods. The extensive experiments on
three different tracking frameworks, i.e., DiMP, DSiam, andSiamRPN++, and three large-scale and challenging datasets,i.e., OTB-2015, LaSOT, and VOT, demonstrate the effectiveness and advantages of the proposed method.
Index Terms— data augmentation, online updating, visualobject tracking, deepmix, mixnet
1. INTRODUCTION
Visual object tracking (VOT) is one of the most widely studied
computer vision approaches that can produce the trajectoryof the moving object from a sequence of frames. It has seenubiquitous applications ranging from navigation for robots,
intelligent video surveillance, smart logistics, robotics for manufacturing, etc. Based on the frame processing procedure, theVOT can be divided into online tracking and offline tracking.
For online tracking, only the current frame and the previous
frames can be used to determine the tracking result for thecurrent frame, and the tracking results for the previous frames,
once computed, can no longer be altered based on later frames.
∗ Ziyi Cheng and Xuhong Ren are co-first authors and contribute equallyto this work. † Wanli Xue and Qing Guo are corresponding authors (xuewanli@email.tjut.edu.cn and tsingqguo@ieee.org).

#129

#181

#184

#114

#122

#148

#32

#40

#43

Fig. 1: Examples of three state-of-the-art trackers, e.g., SiamRPN++ [1], DSiam [2, 3],
and DiMP [4] with or without the proposed DeepMix.

For offline tracking, the tracking results can be produced after
having access to all the frames. Needless to say, the settingsof online tracking make it more suitable for real-world applications and deployment.
In the early days, VOT is implemented using correlation filters (CFs) with notable works such as the minimum output sumof squared error (MOSSE) CF [5], kernelized correlation filter(KCF) [6], spatially regularized discriminative correlation filter[7, 8, 9, 10, 11], etc. With the fast-paced development of deeplearning approaches that are catered towards computer vision
problems, deep learning based VOT has gained tremendous
popularity. Methods such as SiamFC [12] and subsequent
SiamRPN [13], DiMP [14], DSiam [2], DaSiamRPN [15],SiamRPN++ [16], etc., have progressively produced better
results on various VOT benchmarks [17, 18, 19, 20]. We thinkthat online updating of the object model via samples from
historical frames is of great importance for accurate visualobject tracking. These mentioned recent works mainly focus
on constructing powerful deep backbones or effective and efficient online updating methods while neglecting the training
samples for learning discriminative object models, which is
Copyright notice – 978-1-6654-3864-3/21/$31.00 ©2021 IEEE

2.2. Data Augmentation Methods

also a key part of a learning problem in our opinion.
In this paper, we propose the DeepMix that takes historicalsamples’ embeddings as input and generates augmented embeddings online, enhancing the state-of-the-art online learning
methods for visual object tracking. More specifically, we first
propose the online data augmentation for tracking that online
augments the historical samples through object-aware filtering.
Then, we propose MixNet which is an offline trained network
for performing online data augmentation within one-step, enhancing the tracking accuracy while preserving high speeds ofthe state-of-the-art online learning methods. MixNet predicts
different convolution parameters dramatically for object andbackground regions, respectively, according to the input training samples, thus is able to generate effective training samples
for online data augmentation. As shown in Fig. 1, our DeepMix let three state-of-the-art trackers, i.e., DiMP [14], DSiam[2, 3], and SiamRPN++ [1], localize objects more accurately.
The extensive experiments on the above three different tracking frameworks and three large-scale and challenging datasets,i.e., OTB-2015, LaSOT, and VOT, further demonstrate theeffectiveness and advantages of the proposed method.

Data augmentation is an important method to improve generalization performance of deep models. Early works [31, 32, 33]
employ cropping, horizontal and vertical flips, and rotation to
generate diverse data. Recently, AutoAugment [34] automatically searches for augmentation policies given a predefined
set of transformations, which needs a great quantity of training time. Several studies reduce the search costs significantly,
such as Population-based augmentation (PBA) [35] and fast
AutoAugment (FAA) [36]. These works focus on augmenting
a single image.
Recent works [37, 38, 39] propose to mix multiple images for data augmentation, inspiring our research on mixing
multiple training samples for tracking. Specifically, Mixup[38] utilizes an element-wise combination of two images. CutMix [39] replaces a portion of an image with the contents ofanother image. AugMix [37] merges multiple images where
each image is processed by several augment operations randomly, making the trained model see diverse samples. Allof these methods focus on image-level augmentation and do
not consider the speed for online data. In contrast to previous
works, we propose the DeepMix allowing efficient online data
augmentation for visual object tracking.

2. RELATED WORK
3. METHOD
2.1. General Visual Object Tracking

In this section, we first discuss the background and motivationof this work and formulate the online data augmentation for
tracking in Sec. 3.1 and 3.2. Then, we propose the MixNet inSec. 3.3 to realize effective and efficient online data augmentation for tracking. Finally, we detail how to embed MixNet into
state-of-the-art trackers (i.e., SiamRPN++ [16], DSiam [2, 3],
and DiMP [14]) in Sec. 3.4.

In visual object tracking task, Siamese network-based methods[12, 2, 15, 16, 21] have been a popular solution. Amony them,
SiamFC [12] is an initial implementation, which extracts thedeep features from template and search region and performs
cross-correlation to predict object position. SiamRPN [13]proposes to add a bounding box regressor to the Siamese
Network as done in the object detection task.
Another effective solution treats tracking as a online learning problem. They [22, 14, 23, 24, 25] train a classifier frompast frames with an online update strategy and distinguish thetarget from background via the classifier. DiMP [14] applies
meta-learning formulation by collecting historical frame feature representation to classify objects. ROAM [26] offline
trains an LSTM [27] to generate the adaptive learning rate toenhance online training. Several works also focus on the training sample management during the online tracking. [28] manages samples by assigning different weights to training samples. ECO [29] employs a Gaussian mixture model (GMM)
to choose more distinguishing samples. UpdateNet [30] uses
history templates to generating a more robust template througha CNN which is similar to our DeepMix. However, it aims to
update effective templates. Although above works have shown
great advantages of managing training samples, they ignore
how to make more effective use of online samples, e.g., data
augmentation. In this work, we focus on the data augmentation
for the state-of-the-art trackers during tracking.

3.1. Background and Motivation
Given a live video V = {It }Tt=1 having T frames and theobject bounding box annotated at the first frame (i.e., b1 ) , weaim to estimate the object’s position and size at the subsequentT − 1 frames. Most of the state-of-the-art methods complete
this task by maintaining an object model for matching it withthe subsequent frames. In general, we formulate the object
localization at t-th frame as
pt = arg max Mt [p] = arg max loc(ϕ(It ), θt ),
p

(1)

p

where the Mt is a heat map whose the maximum (i.e., Mt [pt ])indicates the object’s position in the frame It , and it can becalculated by the ϕ(It ) and θt where ϕ(·) is the backbone
network for extracting embedding.
The object model θt determines the localization accuracy,
which is initialized at the first frame and updated at subsequent
frames. For example, in the popular Siamese network-based
trackers [12, 16], the object model is constructed by using the2...

employed in the offline training process with random and diverse degradation-related sample-level transformations [40].
For example, AugMix [40] conducts multiple random augmentations for a raw sample and mixes them up for training theimage classifiers. However, these methods could not adapt tothe online data augmentation for visual object tracking directly
due to the following reasons: ¶ the random sample-level transformations would generate new samples that require high time
costs to extract their deep features, slowing down the trackers
significantly. · the transformations are based on degradation
factors (e.g., noise, blur, fog, rain, etc.) that can corrupt the
raw samples, leading to the less discriminative object model.
To address above challenges, we propose the online data
augmentation on embeddings of training samples. Specifically,
we first extract embeddings of training samples in Xt and get{ϕ(Ii ) ∈ RC×W ×H |Ii ∈ Xt }. Then, we concatenate all embeddings and obtain a tensor Xt ∈ RN ×C×W ×H where N is
the number of training samples in Xt . Our goal is to map thetensor Xt to a new counterpart denoted as X̂t ∈ RK×C×W ×H
that can be fed into existing updating modules to produce amore effective object model. Note that, performing augmentation on the embedding level is much more efficient thanthat on the sample level, which alleviates the first challenge.
In terms of the second challenge, we mix embeddings of all
samples with the guidance of previously localization results.
Intuitively, the interested object might be at any position in thescene during the video capturing process and it is reasonable toaugment the training samples by putting the object to possible
background regions. To this end, given Xt and the detectedbounding boxes {bi ∈ R4×1 |i = 1, . . . , N } of N training
samples, we can split the samples to object and backgroundregions and mixing them up to produce new samples. We
formulate this process by

H

Pool

O­Conv3

...
...

3

W

H
W

Pool

B­Conv3

B­Conv2

W

H

3

...

H

B­Conv1...

O­Conv2

O­Conv1

W

...

3

3

Fig. 2: Architecture of MixNet that contains two subnetworks to estimate filtering parameters for mixing object and background regions, respectively.

embedding of the object at the first frame, i.e., θt = ϕ(I1 ), andthe localization is implemented by using cross-correlation, i.e.,loc(ϕ(It ), θt ) = ϕ(It ) ∗ ϕ(I1 ). More recently, [14] proposes
DiMP that uses an online updated classifier for localization(i.e., the loc(·) is set as a convolution layer).
The state-of-the-art trackers (e.g., DiMP [14] and DSiam[2]) online update the object model to adapt object and background appearance variation. In general, we formulate theobject model updating via
θt+1 = update(θt , Xt )(2)

where Xt denotes the set of training samples that are cropped
from the historical frames in which the objects are previously
detected. For example, DSiam [2] proposes to online update
the object model of the Siamese network via a transformationthat is learned from the previous frame. DiMP [14] updates theclassifier’s parameters through a pre-trained model predictor
that takes 50 samples from previous frames as inputs.
Note that, the updating process is a typical learning moduleand recent works have demonstrated that data augmentation
is of great importance for enhancing the accuracy of image
classification under various interferences [40]. Following similar ideas, we aim to explore how to online augment effective training samples, i.e., Xt , for the state-of-the-art trackers
with existing updating methods, i.e., online transformation in
DSiam [2] and model predictor in DiMP [14].

X̂t = (Wo ~ Xt )(1 − Mo ), (4)where Mo ∈ RN ×C×W ×H are binary masks for the N training samples where the elements within the object regions are
set to one while others are zero. The object regions are obtained according to the detection results {bi ∈ R4×1 |i =1, ..., N } and the ‘ ’ denotes the element-wise multiplication.
Besides, the ‘~’ denotes the convolution layer while the tensor
Xt is filtered by W{o or b} ∈ RK×N ×3×3 with the padding
hyper-parameter to be one. The variable K denotes the number of samples in the output tensor. More specifically, for thecth channel of Xt (i.e., Xt [c] ∈ RN ×W ×H ), we filter it with
W{o or b} and get the cth channel of X̂t [c] ∈ RK×W ×H . Intuitively, W{o or b} indicates how to fuse the N training samples
and get K new samples. The Wo and Wb take the charge of
mixing object and background regions, respectively.
However, to make above method work, we should consider
the following issues: ¶ how to estimate Wo and W b to fit
different cases? · how to make the above module to be efficient? To alleviate these issues, we propose the MixNet that isable to produce augmented data in one step.

3.2. Online Data Augmentation for Tracking
A simple way for online data augmentation is to borrow theexisting techniques and conduct augmentation on the collected
historical training samples, i.e., Xt , throughX̂t = aug(Xt , T )Mo + (Wb ~ Xt )(3)

where T denotes the set of sample-level transformations (e.g.,adding noise, blur, and rain) and X̂t is the augmented samples.
The state-of-the-art data augmentation techniques are usually3

Table 1: Comparison with SOTA Trackers under OPE setup.

DatasetOTB-2015

Table 2: Comparison with SOTA Trackers on VOT2018

LaSOT

Metrics

AUC

Prec

Success

Prec

MLT [41]
GradNet [42]ATOM [43]SiamDW [41]
POST [44]
CRPN [45]ASRCF [46]MAML [24]

0.611
0.639
0.667
0.674
0.678
0.675
0.692
0.712

0.861
0.879
0.907
0.922
-

0.368
0.365
0.514
0.384
0.481
0.455
0.359
0.523

0.351
0.505
0.463
0.337
-

DSiam
DSiam-DeepMix

0.646
0.658

0.845
0.861

0.438
0.439

0.431
0.431

SiamRPN++
SiamRPN++-DeepMix

0.650
0.663

0.853
0.870

0.447
0.459

0.446
0.463

DiMP
DiMP-DeepMix

0.660
0.683

0.859
0.890

0.532
0.536

0.532
0.538

Metrics

EAO

Accuracy

Robustness

DaSiamRPN [15]SiamMask [47]
MAML [24]UpdateNet [30]
ATOM [43]SiamDW [48]

0.383
0.387
0.392
0.393
0.401
0.405

0.586
0.642
0.635
0.590
0.597

0.276
0.295
0.220
0.204
0.234

DSiam
DSiam-DeepMix

0.266
0.287

0.577
0.58

0.421
0.407

SiamRPN++
SiamRPN++-DeepMix

0.348
0.405

0.583
0.597

0.29
0.234

DiMP
DiMP-DeepMix

0.214
0.234

0.578
0.612

0.553
0.51

modifications to MixNet to adapt to different trackers.
Training details. For DSiam and SiamRPN++, the original training program uses a pair of images (i.e., template andsearch region) as a training sample. For each sample, We apply
data augmentation strategies on a template to construct a training set containing 15 samples. We input them to MixNet andgenerate a new sample to mix with the original template. We
implement the SGD optimizer with a weight decay of 0.0005,
base lr of 0.005, and momentum of 0.9. We train the MixNetfor 40 epochs and 6000 samples per epoch.
In terms of DiMP, its training program pick up 3 images
from each video as training samples for its model predictor. In
order to match our MixNet during testing, we change it to 50
images from each video for MixNet. We apply SGD optimizer
with weight decay of 0.0005 for all parameter layers, base lrof 0.005, and momentum of 0.9.We train the MixNet for 50
epochs and 1000 videos per epoch.

3.3. MixNet for Efficient Online Data Augmentation
We propose MixNet that takes the Xt as the input and predict
the kernels Wo and W b that are suitable for Xt . We can use
the pre-trained MixNet to generate the kernels in a one-stepway, leading to efficient online data augmentation. We show
the architecture of MixNet in Fig. 2. Specifically, MixNet
contains two sub-networks for predicting the Wo and W b ,respectively. The two sub-networks share the same architecture
but have independent parameters. The architecture has three
convolution layers with a kernel size of 3 × 3 and an averaging
pooling layer. Note that, we can embed MixNet into diverse
tracking frameworks by properly setting the input and output
channels for them.
3.4. Implementation for SOTA Trackers

4. EXPERIMENTS

In this part, we detail the way of using our method for threestate-of-the-art trackers, i.e., SiamRPN++ [13, 1], DSiam [2,3], and DiMP [14]. Simply, we can embed DeepMix into thesetrackers by transforming their training samples and get X̂t .
Then, we mix it with the raw training samples (i.e., Xt ) by
α1 X̂t + α2 Xt ) and feed it into the updating modules. For
all examples, we fix α1 = 0.05 and α2 = 0.8 and discuss its
influence in Sec. 4.3.
DSiam and SiamRPN++ with DeepMix. We collecthistorical samples (Xt size is 15×256×29×29) to generate the kernel (W{o or b} size is 15×1×3×3) and then filter
with the features (its size is 1×256×29×29) of the current
frame. Finally, DeepMix output the new samples (X̂t size is
1×256×29×29).
DiMP with DeepMix. DiMP stores samples to train
the classifier, we directly input these samples (Xt size is
50×256×22×22) and obtain new samples with the same sizeas the Fig. 2 illustrate. We directly train the MixNet alongwith its embedded trackers. It means that we can simply use
the original training program and training data of the targeted
trackers to train their own MixNets. We make some minor

4.1. Setups
Datasets, metrics, and baseline. We evaluate DeepMix onthree tracking benchmarks: VOT-2018 [18] (60 videos, 356
frames average length), LaSOT [19] (280 videos, 2448 frames
average length), OTB-2015 [17] (100 videos, 590 frames average length). VOT-2018 implements a reset-based evaluationthat once the object is lost, the tracker is restarted with theground truth box five frames later and gets a penalty. The main
evaluation criterion is expected average overlap (EAO) [49].
The higher EAO indicates better performance. OTB-2015 and
LaSOT only give the trackers the ground-truth of initial frameand obtain bounding box sequence, terms one-pass evaluation.
We use the AUC, which represents the area under the curve of
the success plot, to evaluate the performance of trackers.
We compare DeepMix-based trackers with six top methodson VOT2018, i.e., DaSiamRPN [15], SiamMask [47], MAML[24], UpdateNet [30], ATOM [43], SiamDW [48]. We alsochoose five excellent trackers on OTB2015, i.e., MLT [41],GradNet [42], POST [44], CRPN [45], ASRCF [46].
4Table 3: Ablation analysis of different backbones

Table 4: Comparing three variants of our method, i.e., DeepMix-Opt,DeepMix-single
and DeepMix, on DiMP tracker and OTB-2015 to validate the effectiveness of MixNet.

Metrics

AUC

Precision

NormPrecision

DiMP (ResNet18)
DiMP (ResNet18)-DeepMix

0.660
0.683

0.859
0.890

0.807
0.834

DiMP (ResNet50)
DiMP (ResNet50)-DeepMix

0.684
0.694

0.894
0.900

0.842
0.852

4.2. State-of-the-art Comparison

Metrics

AUC

Prec.

NormPrec.

FPS

DiMP

0.660

0.859

0.807

27.0

DiMP-DeepMix-Opt
DiMP-DeepMix-single

0.667
0.676

0.873
0.884

0.816
0.831

10
26.5

DiMP-DeepMix

0.683

0.890

0.834

26.0

final version DeepMix based on the DiMP tracker and OTB2015 dataset. As shown in Table 4, DeepMix-Opt via online
iterative optimization can also enhance the tracking accuracy
but immensely increase the computational cost, slowing down
the DiMP significantly.
As Fig. 2 shows, MixNet has two branches and outputs two
filters (Wo and Wb ). We test another version of MixNet: keep
only one branch and output one convolution kernel, then filterswith samples Xt , regardless of object or background. We term
it as DeepMix-single. As shown in Table 4, DeepMix-single
outperforms DiMP with a competitive speed, it still is weaker
than the final version DeepMix. Therefore, learning different
patterns of objects or backgrounds is an important method
for online data augmentation. In contrast, DeepMix with the
MixNet achieves much higher accuracy improvement with
only 1 FPS speed decrease, demonstrating the effectivenessand advantages of the proposed MixNet.

We compare DeepMix with the state-of-the-art methods on
three challenging tracking benchmarks. Respectively, the backbones of DiMP, DSiam and SiamRPN++ are ResNet18 [50],AlexNet [32] and MobileV2 [51] on account of DeepMix’s
extreme improvement for a simple network.
OTB-2015 and LaSOT. We report results on the OTB2015 and LaSOT datasets in Table 1. Results show that: ¶DeepMix improves all of its original trackers. · DeepMix
has a significant improvement of 2.3 percentage points of
AUC and raises the ranking of DiMP from fifth to third compared with other trackers on OTB-2015. DiMP-DeepMix alsoachieves the top success score on LaSOT. ¸ DiMP collects
historical samples’ embeddings and trains the predictor online.
Our MixNet is designed for data augmentation, thus DeepMix has better compatibility with DiMP and achieves more
improvement than DSiam and SiamRPN++.
VOT2018. We report results in Table 2. DeepMix withthree trackers still is in effect. Unlike testing on other datasets,
DiMP collects 250 samples for online training when testing on
VOT2018 and costs extreme memory with DeepMix. Therefore, we still implement the same hyperparameters as evaluating on OTB-2015. Although it has achieved much lower
score than its report, the results also prove the effectivenessof DeepMix. SiamRPN-DeepMix obtains a striking 0.057
improvement on EAO. Even though it uses MobileV2 as the
backbone, achieves state-of-the-art on VOT2018.

5. CONCLUSION
In this work, we have taken a deep dive into the data augmentation aspect for improving online visual object tracking, a long-overlooked facet in this domain. Specifically, we
have proposed the DeepMix as a complete pipeline that takes
historical samples’ embeddings as input and generates augmented online, thus enhancing the state-of-the-art online learning methods for visual object tracking. To this end, we haveproposed the online data augmentation for tracking that online
augments the historical samples through object-aware filtering. Then, we have further proposed the MixNet which is anoffline trained deep neural network for performing online data
augmentation within one-step, for boosting the tracking accuracy while preserving high speeds of the state-of-the-art online
learning methods. We have carried out extensive experimentson three different tracking frameworks, i.e., DiMP, DSiam,and SiamRPN++, and on three large-scale and challenging
datasets, i.e., OTB-2015, LaSOT, and VOT. The experimental
results have demonstrated and verified the effectiveness andadvantages of the proposed method.
In the future, we plan to further study the online data augmentation for visual object tracking by considering different
degradation, e.g., motion blur [52], rain [53, 54], illumination
variation [55, 56], etc.

4.3. Ablation study
DeepMix with different backbones. In order to validate thegenerality of DeepMix with different backbones, we present
the result on OTB-2015 dataset in Table 3. It shows that: ¶DeepMix can take stable effect for any network architecture.
· DeepMix has more improvement on ResNet18-based than
ResNet50-based model. It probably because more powerful
networks are less dependent on DeepMix.
Validation of MixNet. We implement a naive data augmentation method as the baseline to validate the effectivenessof the MixNet. That is, we calculate the filtering parameters,i.e., Wo and Wb , by online optimizing an objective function via the gradient descent to replace the proposed MixNet.
Specifically, we define an objective function that is the L2
distance between the predicted heat map and a Gaussian map
having the highest score on the detected position. Then, at tth
frame, we can minimize the objective function by tuning theWo and Wb via the gradient descent for ten iterations. We
denote this method as ‘DeepMix-Opt’ and compare it with the

Acknowledgement. This work was supported in part by the National Natural Science Foundation of China under Grant 61906135,
62020106004, and 92048301, Tianjin Science and Technology
Plan Project under Grant 20JCQNJC01350, JSPS KAKENHI Grant

5

No.20H04168, 19K24348, 19H04086, and JST-Mirai Program Grant
No.JPMJMI18BB and JPMJMI20B8, Japan, and Natural Science
Foundation of Tianjin under Grant KJZ40420200017. This work wasalso supported by the Canada CIFAR AI program.

[28] M. Danelljan, G. Häger, F. S. Khan, and M. Felsberg, “Adaptive decontaminationof the training set: A unified formulation for discriminative visual tracking,” in
CVPR, 2016.
[29] M. Danelljan, G. Bhat, F. S. Khan, and M. Felsberg, “Eco: Efficient convolution
operators for tracking,” in CVPR, 2017, pp. 6931–6939.
[30] Lichao Zhang, Abel Gonzalez-Garcia, Joost van de Weijer, Martin Danelljan, andFahad Shahbaz Khan, “Learning the model update for siamese trackers,” in ICCV,2019, pp. 4010–4019.
[31] Dan Ciregan, Ueli Meier, and Jürgen Schmidhuber, “Multi-column deep neuralnetworks for image classification,” in CVPR, 2012, pp. 3642–3649.
[32] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton, “Imagenet classification
with deep convolutional neural networks,” Communications of the ACM, vol. 60,
no. 6, pp. 84–90, 2017.
[33] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger, “On calibration ofmodern neural networks,” arXiv preprint arXiv:1706.04599, 2017.
[34] Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V
Le, “Autoaugment: Learning augmentation policies from data,” arXiv preprint
arXiv:1805.09501, 2018.
[35] Daniel Ho, Eric Liang, Xi Chen, Ion Stoica, and Pieter Abbeel, “Population based
augmentation: Efficient learning of augmentation policy schedules,” in ICML,
2019, pp. 2731–2741.
[36] Sungbin Lim, Ildoo Kim, Taesup Kim, Chiheon Kim, and Sungwoong Kim, “Fastautoaugment,” in NeurIPS, 2019, pp. 6665–6675.
[37] Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, andBalaji Lakshminarayanan, “Augmix: A simple data processing method to improverobustness and uncertainty,” arXiv preprint arXiv:1912.02781, 2019.
[38] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz,“mixup: Beyond empirical risk minimization,” arXiv preprint arXiv:1710.09412,
2017.
[39] Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe,and Youngjoon Yoo, “Cutmix: Regularization strategy to train strong classifierswith localizable features,” in ICCV, 2019, pp. 6023–6032.
[40] Dan Hendrycks*, Norman Mu*, Ekin Dogus Cubuk, Barret Zoph, Justin Gilmer,
and Balaji Lakshminarayanan, “Augmix: A simple method to improve robustness
and uncertainty under data shift,” in ICLR, 2020.
[41] Janghoon Choi, Junseok Kwon, and Kyoung Mu Lee, “Deep meta learning forreal-time target-aware visual tracking,” in ICCV, 2019, pp. 911–920.
[42] Peixia Li, Boyu Chen, Wanli Ouyang, Dong Wang, Xiaoyun Yang, and Huchuan
Lu, “Gradnet: Gradient-guided network for visual object tracking,” in ICCV,2019, pp. 6162–6171.
[43] Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, and Michael Felsberg,“Atom: Accurate tracking by overlap maximization,” in CVPR, 2019, pp. 4655–
4664.
[44] Ning Wang, Wengang Zhou, Guojun Qi, and Houqiang Li, “Post: Policy-based
switch tracking.,” in AAAI, 2020, pp. 12184–12191.
[45] H. Fan and H. Ling, “Siamese cascaded region proposal networks for real-timevisual tracking,” in CVPR, 2019, pp. 7944–7953.
[46] Kenan Dai, Huchuan Lu Dong Wang, Chong Sun, and Jianhua Li, “Visual tracking
via adaptive spatially-regularized correlation filters,” in CVPR, 2019.
[47] Qiang Wang, Li Zhang, Luca Bertinetto, Weiming Hu, and Philip H.S. Torr, “Fastonline object tracking and segmentation: A unifying approach,” in CVPR, 2019,
pp. 1328–1338.
[48] Zhipeng Zhang and Houwen Peng, “Deeper and wider siamese networks for realtime visual tracking,” in CVPR, 2019.
[49] Matej Kristan, Jiri Matas, Ales Leonardis, Michael Felsberg, Luka Cehovin,Gustavo Fernandez, Tomas Vojir, Gustav Hager, Georg Nebehay, and Roman
Pflugfelder, “The visual object tracking vot2015 challenge results,” in ICCV,2015, pp. 1–23.
[50] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, “Identity mappings indeep residual networks,” in ECCV, 2016, pp. 630–645.
[51] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and LiangChieh Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in CVPR,
2018, pp. 4510–4520.
[52] Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Jian Wang, Bing Yu, Wei Feng,and Yang Liu, “Watch out! motion is blurring the vision of your deep neuralnetworks,” Advances in Neural Information Processing Systems, vol. 33, 2020.
[53] Liming Zhai, Felix Juefei-Xu, Qing Guo, Xiaofei Xie, Lei Ma, Wei Feng,
Shengchao Qin, and Yang Liu, “It’s raining cats or dogs? adversarial rain attackon dnn perception,” arXiv preprint arXiv:2009.09205, 2020.
[54] Qing Guo, Jingyang Sun, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Wei Feng,and Yang Liu, “Efficientderain: Learning pixel-wise dilation filtering for highefficiency single-image deraining,” in AAAI, 2021.
[55] Lan Fu, Changqing Zhou, Qing Guo, Felix Juefei-Xu, Hongkai Yu, Wei Feng,
Yang Liu, and Song Wang, “Auto-exposure fusion for single-image shadow removal,” arXiv preprint arXiv:2103.01255, 2021.
[56] Binyu Tian, Qing Guo, Felix Juefei-Xu, Wen-Le Chan, Yupeng Cheng, Xiaohong
Li, Xiaofei Xie, and Shengchao Qin, “Bias field poses a threat to dnn-based x-ray
recognition,” arXiv preprint arXiv:2009.09247, 2020.

6. REFERENCES[1] Bo Li, Wei Wu, Qiang Wang, Fangyi Zhang, Junliang Xing, and Junjie Yan,
“Siamrpn++: Evolution of siamese visual tracking with very deep networks,” in
CVPR, 2019, pp. 4282–4291.
[2] Qing Guo, Wei Feng, Ce Zhou, Rui Huang, Liang Wan, and Song Wang, “Learning dynamic siamese network for visual object tracking,” in ICCV, 2017, pp.
1763–1771.
[3] Qing Guo, Xiaofei Xie, Felix Juefei-Xu, Lei Ma, Zhongguo Li, Wanli Xue, Wei
Feng, and Yang Liu, “SPARK: Spatial-aware Online Incremental Attack Against
Visual Tracking,” in ECCV, 2020.
[4] Goutam Bhat, Martin Danelljan, Luc Van Gool, and Radu Timofte, “Learning
discriminative model prediction for tracking,” in ICCV, 2019, pp. 6181–6190.
[5] David S Bolme, J Ross Beveridge, Bruce A Draper, and Yui Man Lui, “Visualobject tracking using adaptive correlation filters,” in CVPR, 2010, pp. 2544–2550.
[6] João F Henriques, Rui Caseiro, Pedro Martins, and Jorge Batista, “High-speed
tracking with kernelized correlation filters,” IEEE TPAMI, vol. 37, no. 3, pp. 583–
596, 2014.
[7] Martin Danelljan, Gustav Hager, Fahad Shahbaz Khan, and Michael Felsberg,“Learning spatially regularized correlation filters for visual tracking,” in ICCV,2015, pp. 4310–4318.
[8] Ce Zhou, Qing Guo, Liang Wan, and Wei Feng, “Selective object and context
tracking,” in ICASSP, 2017, pp. 1947–1951.
[9] Pengyu Zhang, Qing Guo, and Wei Feng, “Fast spatially-regularized correlation
filters for visual object tracking,” in Pacific Rim International Conference on
Artificial Intelligence. Springer, Cham, 2018, pp. 57–70.
[10] Wei Feng, Ruize Han, Qing Guo, Jianke Zhu, and Song Wang, “Dynamic saliencyaware regularization for correlation filter-based object tracking,” IEEE TIP, vol.
28, no. 7, pp. 3232–3245, 2019.
[11] Qing Guo, Ruize Han, Wei Feng, Zhihao Chen, and Liang Wan, “Selective spatial regularization by reinforcement learned decision making for object tracking,”
IEEE TIP, vol. 29, pp. 2999–3013, 2020.
[12] L. Bertinetto, J. Valmadre, J. F. Henriques, A. Vedaldi, and P. H. S. Torr,“Fully-convolutional siamese networks for object tracking,” in arXiv preprint
arXiv:1606.09549, 2016.
[13] Bo Li, Junjie Yan, Wei Wu, Zheng Zhu, and Xiaolin Hu, “High performance visualtracking with siamese region proposal network,” in CVPR, 2018, pp. 8971–8980.
[14] Goutam Bhat, Martin Danelljan, Luc Van Gool, and Radu Timofte, “Learning
discriminative model prediction for tracking,” in ICCV, 2019, pp. 6181–6190.
[15] Zheng Zhu, Qiang Wang, Bo Li, Wei Wu, Junjie Yan, and Weiming Hu,“Distractor-aware siamese networks for visual object tracking,” in ECCV, 2018,
pp. 101–117.
[16] Bo Li, Wei Wu, Qiang Wang, Fangyi Zhang, Junliang Xing, and Junjie Yan,
“Siamrpn++: Evolution of siamese visual tracking with very deep networks.,” in
CVPR, 2019, pp. 4277–4286.
[17] Y. Wu, J. Lim, and M.-H. Yang, “Object tracking benchmark,” IEEE TPAMI, vol.
37, no. 9, pp. 1834–1848, 2015.
[18] Matej Kristan, Ales Leonardis, Jiri Matas, Michael Felsberg, Roman Pflugfelder,
Luka ˇCehovin Zajc, Tomas Vojir, Goutam Bhat, Alan Lukezic, Abdelrahman
Eldesokey, et al., “The sixth visual object tracking vot2018 challenge results,” in
ECCV, 2018, pp. 0–0.
[19] H. Fan, L. Lin, F. Yang, P. Chu, G. Deng, S. Yu, H. Bai, Y. Xu, C. Liao, and
H. Ling, “Lasot: A high-quality benchmark for large-scale single object tracking,”
in CVPR, 2019, pp. 5369–5378.
[20] Qing Guo, Wei Feng, Ruijun Gao, Yang Liu, and Song Wang, “Exploring theeffects of blur and deblurring to visual object tracking,” IEEE TIP, vol. 30, pp.
1812–1824, 2021.
[21] Zedu Chen, Bineng Zhong, Guorong Li, Shengping Zhang, and Rongrong Ji,“Siamese box adaptive network for visual tracking,” in CVPR, 2020, pp. 6668–
6677.
[22] Qing Guo, Wei Feng, Ce Zhou, and Bin Wu, “Structure-regularized compressive
tracking,” 2016.
[23] Martin Danelljan, Luc Van Gool, and Radu Timofte, “Probabilistic regression forvisual tracking,” in CVPR, 2020, pp. 7183–7192.
[24] Guangting Wang, Chong Luo, Xiaoyan Sun, Zhiwei Xiong, and Wenjun Zeng,“Tracking by instance detection: A meta-learning approach,” in CVPR, 2020, pp.
6288–6297.
[25] Goutam Bhat, Martin Danelljan, Luc Van Gool, and Radu Timofte, “Know yoursurroundings: Exploiting scene information for object tracking,” arXiv preprint
arXiv:2003.11014, 2020.
[26] Tianyu Yang, Pengfei Xu, Runbo Hu, Hua Chai, and Antoni B Chan, “Roam:
Recurrently optimizing tracking model,” in CVPR, 2020, pp. 6718–6727.
[27] Sepp Hochreiter and Jürgen Schmidhuber, “Long short-term memory,” Neuralcomputation, vol. 9, no. 8, pp. 1735–1780, 1997.

6