WaNet - Imperceptible Warping-based Backdoor Attack
Authors: Tuan Anh Nguyen
Anh Tuan Tran
Venue: ICLR 2021 Poster (2021)
Keywords: backdoor attack
image warping
wanet

Abstract: With the thriving of deep learning and the widespread practice of using pre-trained networks, backdoor attacks have become an increasing security threat drawing many research interests in recent years. A third-party model can be poisoned in training to work well in normal conditions but behave maliciously when a trigger pattern appears. However, the existing backdoor attacks are all built on noise perturbation triggers, making them noticeable to humans. In this paper, we instead propose using warping-based triggers. The proposed backdoor outperforms the previous methods in a human inspection test by a wide margin, proving its stealthiness. To make such models undetectable by machine defenders, we propose a novel training mode, called the ``noise mode. The trained networks successfully attack and bypass the state-ofthe art defense methods on standard classification datasets, including MNIST, CIFAR-10, GTSRB, and CelebA. Behavior analyses show that our backdoors are transparent to network inspection, further proving this novel attack mechanism's efficiency.

Published as a conference paper at ICLR 2021
WANET IMPERCEPTIBLE WARPING-BASED BACK- DOOR ATTACK
Anh Tuan Nguyen'.2, A,, Anh Tuan Tran VinAI Research, HHanoi University of Science and Technology, UnnUniresyy {v anhnt 479, v. a.antttss2eevina...
ABSTRACT
With the thriving of deep learning and the widespread practice of using pre- trained networks, backdoor attacks have become an increasing security threat drawing many research interests in recent years. A third-party model can be poi- soned training to work well in normal conditions but behave maliciously when trigger pattern appears. However, the existing backdoor attacks are all built on noise perturbation triggers, making them noticeable to humans. In this pa- per, we instead propose using warping- surping-based triggers. The proposed backdoor outperforms the previous methods in human inspection test by a wide mar- gin, proving its stcalthiness. To make such models undetectable by machine defenders, we propose a novel training mode, called the "noise" mode. The trained networks successfully attack and bypass the state of the art defense meth- ods on standard classification datasets, including MNIST, CIFAR-10, GTSRB, and CelebA. Behavior analyses show that our backdoors are transparent net- work inspection, further proving this novel attack mechanism's 's efficiency. Our code is publicly available at https://github con//VinAARReeerrchh Warping: BesssedBBcckdoos Sttoorrttttckktt re release.
INTRODUCTION
Deep learning models are essential in many modern systems due to their superior performance com- pared classical methods. Most state-of-the-art models, however, require expensive hardware, huge training data, and long training time. Hence, instead of training the models from scratch, it is com- mon practice to use pre-trained networks provided by third-parties these days. This poses a serious security threat of backdoor attack (Gu al. 2017). A backdoor model is network poisoned either at training or finetuning. It can work as a genuine model in the normal condition. However, when specific trigger appears in the input, the model will act maliciously, designed by the attacker. Backdoor attack can occur in various tasks, including image recognition (Chen et al. [2017), speech recognition (Liu et al. 2018b), natural language processing (Dai et al. 2019), and reinforcement learning (Hamon et al. 2020p. In this paper, we will focus on image classification, the most popular attacking target with possible fatal consequences (e.g., for self-driving car).
Since introduced, backdoor attack has drawn lot of research interests (Chen et al. 2017 Liu et al., 2018b, Salem et al.] 2020) Nguyen Tran 2020). In most of these works, trigger patterns are based on patch perturbation or image blending. Recent papers have proposed novel patterns such as sinusoidal strips (Barni et al. 2019), and reflectance (Liu et al. 2020). These backdoor triggers, however, are unnatural and can be easily spotted by humans.
We believe that the added content, such as noise, strips, or reflectance, causes the backdoor samples generated by the previous methods strikingly detectable. Instead, we propose to image warping that can deform but preserve image content. We also found that humans are not good at recognizing subtle image warping, while machines excellent in this task.
Hence, in this paper, we design a novel, simple, but effective backdoor attack based on image warp- ing called WaNet. We small and smooth warping field in generating backdoor images, making the modification unnoticeable, as illustrated in Fig. [ Our backdoor images are natural and hard to be distinguished from the genuine examples, confirmed by our study described in Sec. 4.3

---

Published as a conference paper at ICLR 2021
Original
Patched
Blended
SIG
ReFool
Warped (Ours)
30
30
30
30
30
30
Image
Residual
30
Figure 1: Comparison between backdoor examples generated by our method and by the pre- vious backdoor attacks. Given the original image (leftmost), we generate the corresponding back- door images using patch-based attacks (Gu et al. 2017 Liu et al., 201 8b), Weendnng-bssed attack (Chen et al., 2017). SIG (Barni et al., 2019), ReFool (Liu et al. 2020), and our method. For each method, show the image (top), the magnified (x2) residual map (bottom). The images generated from the previous attacks are unnatural and can be detected by humans. In constrast, ours is almost identical to the original image, and the difference is unnoticeable.
To obtain backdoor model, first follow the common training procedure by poisoning a part of training data with a fixed ratio of Pa (0, 1). While the trained networks provide high clean and attack accuracy, we found that they "cheated" by learning pixel-wise artifacts instead of the warping itself. It makes them easy to be caught by a popular backdoor defense Neural Cleanse. Instead, we add another mode in training, called "noise mode", , to enforce the models to learn only the predefined backdoor warp. This novel training scheme produces satisfactory models that are both effective and stealthy.
Our attack method achieves invisibility without sacrificing accuracy. It performs similarly to state- of-the-art art backdoor methods in terms of clean and attack accuracy. verified on common benchmarks such MNIST, CIFAR- 10, GTSRB, and CelebA. Our attack is also undetectable by various back- door defense mechanisms; none of existing algorithms can recognize or mitigate our backdoor. This is because the attack mechanism of our method is drastically different from any existing attack, breaking the assumptions of all defense methods.
Finally, we demonstrate that novel backdoor can be practical threat by deploying it for physical attacks. We tested the backdoor classifier with camera-captured images of physical screens. Despite image quality degradation via extreme capturing conditions, our backdoor is well-preserved, and the attack accuracy stays near 100%.
In short, we introduce a novel backdoor attack via image warping. To train such model, we extend the standard backdoor training scheme by introducing a "noise" training mode. The attack is effec- tive, and the backdoor is imperceptible by both humans and computational defense mechanisms. can be deployed for physical attacks, creating practical threat to www-llennnnn-bseeee systems
BACKGROUND
2.1
THREAT MODEL
Backdoor attacks are techniques of poisoning a system to have hidden destructive functionality. The poisoned system can work genuinely on clean inputs but mishehave when a specific trigger pattern appears. In the attack mode for image classification, backdoor models can return predefined target label, normally incorrect, regardless of image content. It allows the attacker to gain illegal benefits. For example, a backdoor face authentication system may allow the attacker to access whenever he puts specific sticker on the face.
Backdoors can injected into the deep model any stage. We consider model poisoning training since it is the most used threat model. The attacker has total control over the training process and maliciously alters data for his attack purposes. The poisoned model is then delivered to customers to
Source code of the experiments will be publicly available.
2

---

Published as a conference paper at ICLR 2021
deploy is- In our proposed attack, the attacker selects fixed warping field and uses it generate all the backdoor images in training and in testing-time time attacks.
2.2 PREVIOUS BACKDOOR ATTACKS
We focus on backdoor attacks on image classification. The target network is trained for a classifi- cation task f X + C, where X is an image domain and C = f2i,c2,.... classes. When poisoning f. we enforce it to learn an injection function B, target label function e, and alter the network behaviour so that:
C2,..., CM } is a set of M target
f(x)) = y, f(((z)) = c(y)
for any pair of clean image x E X and the corresponding label E C.
The earliest backdoor attack was BadNets (Gu et al., 2017). The authors suggested to poison portion of training data by replacing each clean data pair (æ;, y) with the corresponding poisoned pair (B(x), eyy)) The injection function simply replaces fixed patch of the input image by a predefined trigger pattern. As for the target label function c(y). the authors proposed two tests: (I) all-to-one with a constant target label c(y) and (2) all-to-all with c(y) = y +
After BadNets, many variants of backdoor attacks have been introduced. These approaches focus on changing either the backdoor injection process or the injection function B.
As for the backdoor injection process. Liu et al. (018b) proposed inject backdoor into clean models via fine-tuning instead the training stage. Yao et al. (2019) suggested hiding backdoor inside latent neurons for transfer learning. Many recent studies (Turner et al., 2019) Barni et al., 2019 Liu et al.] 2020). injected backdoor only on samples with unchanged labels, i.e., the target c(y) is the same as the ground-truth label y, to dodge label inspection by humans.
In this paper, we focus on the development of a good injection function B. Most of the popular attack methods rely on fixed patch-based triggers. Chen al. (2017) used image blending embed the trigger into the input image, and Nguyen & Tran 2020) extended it to be input-aware. (Salem et al. (222)) varied the patch-based trigger locations and patterns make it "dynamic"" Barni employed sinusoidal strips as the trigger alongside the clean-label strategy. Lately, Liu et al. (020)) proposed to disguise backdoor triggers as reflectance to make the poisoned images look natural. The backdoor images generated by these attacks, however, are easy to be spotted by humans. We instead propose an "invisible" backdoor that is imperceptible by even sharp-eyed people.
(2119)
2.3 BACKDOOR DEFENSE METHODS
As the threat of backdoor attacks becomes more apparent, backdoor defense research is emerging. Based on usage scenarios, we can classify them into three groups: training defense, model defense, and testing-time defense.
Training defense assumes the defender has control over the training process, and the adversary attacks by providing infected training data (Tran al., 2018). This assumption, however, does not match our threat model, where the already-trained backdoor model is provided by third party. This mechanism is not applicable to our situation and will not be considered further in this paper.
Model defenses aim to verify or mitigate the provided model before deployment. Fine-Pruning Liu et al. 2018a) suggested to prune the dormant neurons, defined by analyses on clean image set, to mitigate the backdoor if present. Neural Cleanse (Wang et al.) 2019) was the first work that could detect backdoor models. It optimized patch- based trigger candidate for each target label, then detected if any candidate was abnormally smaller than the others as backdoor indicator. ABS (Liu
al. 2019) scanned the neurons and generated trigger candidates by reverse engineering. [Cheng al. 20119) used GradCam (Selvaraju et al.) 2017) to analyze the network behavior clean input image with and without the synthesized trigger to detect anomalies. Zhhao et al. (2019) applied mode connectivity to effectively mitigate backdoor while keeping acceptable performance. Lately, Kolouri et al. (2020) introduced universal litmus patterns that can be fed to the network to detect backdoor.
Unlike model defense, testing- defenses inspect models after deployment with the presence of input images. It focuses on verifying if the provided image is poisoned and how to mitigate it. STRIP (Gao et al. 2019) exploited the persistent outcome of the backdoor image under perturbations
3

---

Published as a conference paper at ICLR 2021
for detection. In contrast, Neo (Udeshi et al., 2019) searched for the candidate trigger patches where region blocking changed the predicted outputs. Recently, Doan et al. (2019) used GradCam inspection to detect potential backdoor locations. In all these methods, the triger candidates were then verified by heing injected into a set of clean images.
A common assumption in all previous defense methods is that the backdoor triggers image patches. We instead propose novel attack mechanism based on image warping, undermining the foundation of these methods.
2.4 ELASTIC IMAGE WARPING
Image warping is a basic image processing technique that deforms an image by applying the geo- metric transformation. The transformation can be affine, projective, elastic, or non-elastic. In this work, propose to use elastic image warping given its advantages over the others: (1) Affine and projective transformations are naturally introduced to clean images via the image capturing process. If we apply these transformations to these images, the transformed images can be identical to other clean images that are the same scenes but captured at different viewpoints. Hence, these transfor- mations not suitable generate backdoor examples, particularly in physical attacks. (2) Elastic transformation still generates natural outputs while non-elastic one does not.
The most popular elastic warping technique is Thin-Plate Splines (TPS) (Duchon 1977). TPS can interpolate a smooth warping field to transform the entire image given a set of control points with known original and target 2D coordinates. TPS adopted in Spatial Transformer Networks (Jaerberg et al. 2015), the first deep learning study incorporating differential image warping.
We believe that elastic image warping can be utilized to generate invisible backdoor triggers. Unlike previous attack methods that introduce extra and independent information to an input image, clastic image warping only manipulates existing pixels of the image. Humans, while being excellent in spotting incongruent part of an image, are bad at recognizing small geometric transformations.
3
WARPING-BASED BACKDOOR ATTACK
We now describe our novel backdoor attack method WaNet, which stand for Warping-base based poi- soned Networks. WaNet are designed to be stealthy to both machine and human inspections.
3.1 OVERVIEW
Recall that a classification network is function f + C, in which X is an input image domain and C is set of target classes. To train f, training dataset S {(x,, Ji)la; X. ys E C, i- 1, .N is provided. We follow the training scheme of BadNets to poison a subset of S with ratio Pa for backdoor training. Each clean pair (æ, y) will be replaced by backdoor pair (B(x), c(r)), which B is the backdoor injection function and c(y) is the target label function.
Our main focus is redesign the injection function B based on image warping. We construct using a warping function W and predefined warping field M:
B(x) = ((xxM)
M acts like a motion field; it defines the relative sampling location of backward warping for each point in the target image. W allows floating-point warping field input. When sampling pixel falls on non-integer 2D coordinates. it will be bi-linear interpolated. To implement W, we rely on the public API grid sample provided by PyTorch. However, this API inputs grid of normalized absolute 2D coordinates of the sampling points. To use that API, we first sum M with an identity sampling grid, then normalize to [- to get the required grid input.
3.2 WARPING FIELD GENERATION
The warping field M is crucial component; it must guarantee that the warped images are both nat- ural and effective for attacking purposes. Hence, M are desired to satisfy the following properties:
Small: M should be small, to be unnoticeable humans,
4

---

Published as a conference paper at ICLR 2021
Up-sam pling
Warping
VA
Scaling
Clipping
W
50
Random noise
M
M
Poisoned
-1
x
50
Figure Process of creating the warping field M and using it to generate poisoned images.
Input
2
k-d
k
k-8
Input
s 0.25
0.5
0.75
PSNR 20.71
22.73
23.93
22.03
PSNR
24.88
20.08
17.51
16.50
LPIPS
0.0393
0.0352
0.0328
0.0480
LPIPS
0.047
0.069
0.093
0.120
(a) Changing control-grid size & (s 0.5)
(b) Changing warping strength (k (k=44 4)
Figure 3: Effect of different hyper-parameters on the warping result. For each warped image,
show the image (top), the magnified (x2) residual map (bottom). The PSNR and LPIPS (Zhang
ct al. 2018) scores are computed at resolution 224x224.
M should be elastic, i.e., smooth and non-flat, to generate natural looking images, Within image boundary: M should not exceed the image boundary, avoid creating suspicious black/plain outer area.
Elastic:
To get such a warping field, borrow the idea of using control points from TPS but simplify the interpolation method. The process of generating the desired warp is illustrated by Fig. and is described in the following subsections.
Selecting the control grid We first select the control points. For simplicity, we pick the target points on uniform grid of size ho x & over the entire image. Their backward warping field is denoted as P e kxxx We use parameter define the strength of P and generate P as following:
P (randd[1.11 (k, 2)) x
in which rand (...)). value in the range [-1, 1] and v is normalization function. In this paper, we normalize the tensor elements by their mean absolute value:
function returning random tensor with the input shape and element
(AA
(4)
size(A] E.,EA a4
Upsampling From the control points, we interpolate the warping field of the entire image. Since these points are in a uniform grid covering the entire image, instead of using complex spline-based interpolation like in TPS, we can simply apply bieubic interpolation. We denote the output this step Mo = P pxxux2 with h and w being the image height and width respectively.
Clipping Finally, we apply clipping function & so that the sampling points do not fall outside of the image border. The process generating M can be summarized by the equation:
M = @(t (b(rand)_1.((k: (k: k, 2)) x s)).
(5)
We investigate the effect of the almost invisible when & < 6 and < 0.75.
& and s qualitatively in Fig. The warping effect
3.3 RUNNING MODES
After computing the warping field M, we can train WaNet with with two modes, clean and attack, the standard protocol. However, the models trained by that algorithm, while still achieving high
5

---

Published as a conference paper at ICLR 2021
Clean
"Arpflone"
Attack mode
x
M
M'
"Dog"
Noise mode
Xn
Figure 4: Training pipeline with three running modes.
Dataset
Clean
Attack Noise
8
MNIST
99.52
99.86
98.20
CIFAR-10
94.15
99.55
93.55
0,969 56
GTSRB
98.97
98.78
98.01
8
CelebA
78.99
99.33
76.74
(c) Physical attack Figure 5: Attack experiments. In (b), we provide the clean (top) and backdoor (bottom) images.
(a) Network performance
Sample backdoor images
accuracy in both clean and attack tests, tend to learn pixel-level artifacts instead of the warping. They are, therefore, easily exposed by a backdoor defense method such as Neural Cleanse. We will discuss more details in the ablation studies in Section 4.6
To resolve this problem, we propose novel training mode alongside the clean and attack mode, called noise mode. The idea is simple: when applying a random warping field M' M, the network should not trigger the backdoor but return the correct class prediction.
Fig. illlustrates three running modes in training pipelines. We first select the backdoor proba- (0, 1) such that Pa + < 1. Then, for each clean input (z, 3), we randomly select one of three modes and alter that pair accordingly:
bility E (0, 1) and the noise probability
(x, v)
with probability 1 -
(ar, v) +
(W(x, M), c(v)(
with probability Pa
(6)
(W(x, M + rand;-1,1 w. 2)), with probability Pn
Note that with the noise mode, instead of using a totally random warping field, form it by adding Gaussian noise to M for a more effective training. The modified training set is then used to train f.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Following the previous backdoor attack papers, we performed experiments on four datasets: MNIST (LeCun al. 1998), CIFAR-10 (Krizhevsky al. 2009), GTSRB (Stallkamp al., 2012) and CelebA (Liu et al. 2015). Note that CelebA dataset has annotations for 40 independent binary attributes, which not suitable for multi-class classification. Therefore, we follow the configuration suggested by /Salem al. (2020) to select the top three most balanced attributes, including Heavy Makeup, Mouth Slightly Open, and Smiling, then concatenate them to create eight classification classes. Their detail information shown Table To build the classifier f for the color image datasets, we used Pre-activation Resnet-18 (He et al. (0166) for the CIFAR- 10 and GTSRB datasets suggested by Kang (2020), and Resnet- 18 for the CelebA dataset. As for the grayscale dataset MNIST, we defined a simple network structure as reported in Table
We trained the networks using the SGD optimizer. The initial learning rate was 0.01. which duced by factor 10 after each 100 training epochs. The networks were trained until convergence. We used I: 0.5. Pa 0.1, and Pn. = 0.2.

---

Published as a conference paper at ICLR 2021
Table 1: Datasets and the classifiers used in our experiments. Each ConvBlock consists of a 3x3 convolution (stride=2), BatchNorm, and a ReLU layer.
Dataset
Subjects
#Classes
Input Size
#Train. Images
Classifier
MNIST
Written digits
10
28
28 x 1
60,000
3 ConvBlocks, 2 fes
CIFAR-10
General objects
10
32
x
32 x 3
50,000
PreActRes18
GTSRB
Traffic signs
43
32
32 x
39,252
PreActRes18
CelebA
Face attributes
8
64
x 64 3
202,599
ResNet18
Clean
Backdoor
Fooling rate (%)
Patched Blended
SIG
ReFool
WaNet
60
60
Backdoor inputs
8.7
1.4
2.7
2.3
38.6
Clean inputs
6.1
10.1
2.6
13.1
17.4
All inputs
7.4
5.7
2.6
7.7
28.0
(a)
(b)
Figure 6: Human inspection tests: (a) Success fooling rates of each backdoor method, (b) The
most distinguishable cases from WaNet.
4.2 ATTACK EXPERIMENTS
We trained and tested the backdoor models in all-to-one configuration, i.e., c(y) Ety. The accu- racy values in clean mode, attack mode, and the noise mode are reported Fig. 55a As can be seen, with clean images, the networks could correctly classify them like any benign models, with accuracy near 100% on MNIST/GTSRBB 94.15% on CIFAR-10, and 79.77% on CelebA. When applying the pre-defined image warping, the attack success rate using a random warping, the classifiers still recognized the true image class with a similar accuracy as in the clean mode. This result is impressive given the fact that the poisoned images look almost identical to the original, as can be seen in Fig. 5b
100% on all datasets. However, when
To evaluate method's robustness in real-life scenarios, also tested if backdoor images would still be misclassified even when being distorted by the capturing process. We showed 50 clean and 50 backdoor images screen and recaptured them using phone camera. Our model still worked well on recaptured images, obtaining 98% clean accuracy and 96% attack success rate. Fig. [5c] displays an example of our test. The clean image was recognized correctly as "automobile", while the look-a-like backdoor image was recognized as the "airplane" attack class.
4.3 HUMAN INSPECTION
To examine the realisticity of our backdoor and the previous methods, we created user studies with human inspection. First, randomly selected 25 images from the GTSRB dataset. Second, for each backdoor injection function, we created the corresponding 25 backdoor images and mixed them with the original to obtain of 50 images. Finally, we asked 40 people classify whether each image was genuine. collecting 2000 answers per method. The participants were trained about the mechanism and characteristics of the attack before answering the questions.
We collected the answers and reported the percentage of incorrect answers as the success fooling rates in Fig, 6al Note that when the backdoor examples are more indistigguishable from the clean ones, the testers will find it harder to decide an image is clean poisoned. Hence, better backdoor methods led higher fooling rates on not only backdoor inputs but also on clean ones. The rates from previous methods are low, with maximum 7.7% on all inputs, implying that they are obvious to humans detect. contrast, our rate 28%, four times their best number. It confirms that WaNet is stealthy and hard to detect, even with trained people.
Although our backdoor images are natural-looking, some of them have subtle properties that can be detected by trained testers. We provide two of the most detected backdoor examples from WaNet in Fig. 6b In the first case, the circle sign is not entirely round. the second case, the right edge of the traffic sign is slightly curved. Although these conditions can be found real-life traffic signs, they are not common in the testing dataset GTSRB. These images are of the minority, and our fooling rate backdoor images is 38.6%, not far away from the rate of 50% in random selection.

---

Published as a conference paper at ICLR 2021
100
1.00
1.00
100
ACCUISCS
90
Accurac
900
Actumiy
Accunacy
clcan backdoor
40
Mier Pruied
Pruned
Pruned
(a) FinePrunning
MNIST
CIFAR-10
GTRSB
ColobA
clean
baredoer
Entroo
Entropy
Entropy
92
0.1
25
Probability
3.0
3.5
2.0
Probability
2.0
10.0
Probability
15.0
17.5
Probabiliyy
2.0
2.5
3.0
(b) STRIP
Anomaly Index
clean backdoor
6
4
2
MNIST CIFAR-10 GTSRB
CelebA
Clean model
WaNet
(c) Neural Cleanse
(d) GradCam
Figure 7: Experiments on verifying WaNet by the state-of-the-an defense and visualization methods.
4.4 DEFENSE EXPERIMENTS
We will now test the trained models against the popular backdoor defense mechanisms, including Neural Cleanse, Fine-Prunning (Model defenses), and STRIPS (Testing-time defense).
Neural Cleanse (Wang et al. 2019) is a model-defense method based on the pattern optimization approach. assumes the backdoor is patch-based. For each class label, Neural Cleanse com- putes the optimal patch pattern to convert any clean input to that target label. It then checks if any label has a significantly smaller pattern as a sign backdoor. Neural Cleanse quantifies it by the Anomaly Index metric with the clcan/backdoor threshold
2. We ran Neural Cleanse over WaNet models and report the numbers in Fig. 7 WaNet passed the test on all datasets; its scores smaller than the clean model ones MNIST and CIFAR-10. We explain it by the fact that our backdoor relies on warping, a different mechanism compared with patch- patch-based blending.
Fine-Pruning (Liu et al., 2018a), instead, focuses on neuron analyses. Given a specific layer, analyzes the neuron responses set of clean images and detects the dormant neurons, assuming they are more likely to tie to the backdoor. These neurons are then gradually pruned to mitigate the backdoor. We tested Fine-Pruning attack, with respect to the number of neurons pruned in Fig. 7a On all datasets, at no point is the clean accuracy considerably higher than the attack one, making backdoor mitigation impossible.
models and plotting the network accuracy, either clean
2019) is a representative of the testing-time defense approach. examines the model with the presence of the input image. STRIP works by perturbing the input image through set of clean images from different classes and raising the alarm the prediction is persistent, indicating by low entropy. With WaNet, the perturbation operation of STRIP will modify the image content and break the backdoor warping if present. Hence, WaNet behaves like genuine models, with similar entropy ranges, as shown in Fig. 7b
STRIP (Gao
4.5 NETWORK INSPECTION
Visualization tools, such as GradCam (Selvaraju et al. 2017). are helpful in inspecting network behaviors. Patch-based backdoor methods can be exposed easily due to the of small trigger
8

---

Published as a conference paper at ICLR 2021
100
05
:
Accuracy
80
Clean Backdoor
Accurscy
60
85
Clean
20
80
Backdoor
96
Noise
Noise
Clean WaNet w/o mode
WaNet
50
0.4
65
5.5
1.0
70
(a) Trigger patterns optimized by (b) Model performance when (c) Model performance when
Neural Cleanse (small is bad)
changing
changing k
Figure 8: Ablation studies CIFAR-10 dataset: (a) Role of the noise mode training, (b,c) Network performance when changing warping syper-parameters.
regions, as pointed out by Cheng et al. 2019); Doan et al. (011)) Our attack method is based on the warping on the entire image, so undetectable by this algorithm. We visualize activation based on the label that has the highest prediction score in Fig. (7d] With clean models, that label is for the correct class label. With WaNet and backdoor inputs, it is the backdoor label can be seen, the visualization heatmaps of WaNet look like the ones from any clean model.
4.6 ABLATION STUDIES
Role of the noise mode Without the noise mode, we could still train backdoor model with similar clean and attack accuracy. However, these models failed the defense test with Neural Cleanse as shown in Fig. 9 and the optimized trigger patterns revealed their true behavior.
noise mode
w/o noise mode
Anomaly Index
MNIST
CIFAR-10
GTSRB
CelebA
Figure Networks' performance against Neural Cleanse with and without noise mode.
Fig. 8a displays the trigger patterns optimized by Neural Cleanse for the attacking class "airplane" on CIFAR-10. With the clean model, this pattern has an airplane-like shape, and it is big enough to rewrite image content given any input. With our model trained without noise mode, the optimized pattern just consists of scattered points. This pattern is remarkably smaller, making the model caught by Neural Cleanse. reveals that the model did not learn the specific backdoor warping; instead, remembered the pixel-wise artifacts. By adding the noise training mode, our model no longer relies on those artifacts, and the optimized pattern looks similar to the clean model's one.
Other hyper-parameters We investigated the effect of the warping hyper- ppppr-parameters. including the strength and the grid size k. Fig. 8b and show the clean, attack, and noise mode accuracy of our network on the CIFAR-10 10 dataset when changing each of these parameters. When or s small, the backdoor images are similar to the clean ones. However, since they are minority (pa = 0.1), the network would treat them like data with noisy labels in those scenarios. Hence, clean and noise accuracies stable across configurations. In contrast, backdoor accuracy suffers the left side of the plots. It gradually increases when 8 or k is small, then saturates and stays near 100%.
5 CONCLUSION AND FUTURE WORKS
This paper introduces a novel backdoor attack method that generates backdoor images via subtle image warping. The backdoor images proved to be natural and undetectable by humans. We incorporate training a novel "noise" mode, making stealthy and pass all the known defense methods. It opens new domain of attack mechanism and encourages future defense research.

---

Published as a conference paper at ICLR 2021
REFERENCES
Mauro Barni, Kassem Kallas, and Benedetta Tondi. A new backdoor attack in cmns by training set corruption without label poisoning. In 2019 IEEE International Conference on Image Processing (ICIP), pp. 101-105. IEEE, 2019.
Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05522 2017.
Hao Cheng, Kaidi Xu, Sijia Liu, Pin-Yu Chen, Pu Zhao, and Xue Lin. Defending against Back- door Attack on Deep Neural Networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining Workshop. 2019.
Jiazhu Dai, Chuanshuai Chen, and Yufeng Li. A backdoor attack against Istm-based text classifica- tion systems. IEEE Access, 7:138872-138873 138878, 2019.
Bao Gia Doan, Ehsan Abbasnejad, and Damith C. Ranasinghe. Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems. arXiv, Aug 2019. URL https:
/108....11 03369
Jean Duchon. Splines minimizing rotation-invariant semi-norms in sobolev spaces. In Constructive theory of functions of several variables, pp. 85-100. Springer, 1977.
Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Damith Ranasinghe, and Surya Nepal. Strip: A defence against trojan attacks deep neural networks. In Proceedings of the 35th Annual Computer Security Applications Conference, pp. 113-125, 2019.
Tianyu Gu, Brendan Dolan-Gavin and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. In Proceedings of Machine Learning and Computer Security Workshop, 2017.
Ronan Hamon, Henrik Junklewitz, and Ignacio Sanchez. Robustness and explainability of artificial intelligence. Publications Office of the European Union, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In European conference on computer vision, pp. 630-645. Springer, 2016.
Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. Spatial transformer networks. Ad- vances in neural information processing systems, pp. 2017-2025, 2015. 2015.
Liu Kang. pytorch-cifar, May 2020. URL https: eitorch-cifar [Online; accessed 4. Jun. 2020J.
com/ wtt.:/....ooo/uaugiiaaaaa
Soheil Kolouri, Aniruddha Saha, Hamed Pirsiavash, and Heiko Hoffmann. Universal litmus patterns: Revealing backdoor attacks cnns. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 301-310, 2020.
Alex Krizhevsky et al. Learning multiple layers of features from tiny images. 2009.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-baig based learning applied to document recognition. Proceedings of the IEEF,
1998.
Kang Liu, Brendan Dolan-Gavine and Siddharth Garg. Fine- trumiinu Defending against backdoor- ing attacks on deep neural networks. Proceedings of International Symposium on Research in Attacks, Intrasions, and Defenses, 2018a.
Yingqi Liu, Shiqing Ma. Yousra Aafer, Wen-Chuad Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. Trojaning attack on neural networks. In Proceedings of Network and Distributed System Security Symposium, 2018b.
Yingqi Liu, Wen-Chuan Lee, Guanhong Tao, Shiqing Ma, Yousra Aafer, and Xiangyu Zhang. Abs: Scanning neural networks for back-doors by artificial brain stimulation. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pp. 1265-1282, 2019.
10

---

Published as a conference paper at ICLR 2021
Yunfei Liu, Xingjun Ma, James Bailey, and Feng Lu. Reflection backdoor: A natural backdoor attack on deep neural networks. 2020.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Tuan Anh Nguyen and Anh Tran. Input-aware dynamic backdoor attack. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural In- formation Processing Systems, volume 33, pp. 3454-3464. Curran Associates, Inc., 2020.
URL hntpps://proceeddnggg neurips.................. cc /t//papprr/22///ile// 239661120000666baeeccbbbbdd21222 Papes.ppperrp.l
Ahmed Salem, Rui Wen, Michael Backes, Shiqing Ma, and Yang Zhang. Dynamic backdoor attacks against machine learning models. arXiv preprint arXiv:2003.0367. 2020.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das. Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE international conference on computer vision, pp. 2017.
Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel. Man vs. computer: Bench- marking machine learning algorithms for traffic sign recognition. Neural networks, 32:323-332,
2012.
Brandon Tran, Jerry Li, and Aleksander Madry. Spectral signatures in backdoor attacks. In Pro- ceedings of Advances in Neural Information Processing Systems, 2018.
Alexander Turner, Dimitris Tsipras. and Aleksander Madry. Clean-label backdoor attacks. mit.
2019.
Sakshi Udeshi, Shanshan Peng, Gerald Woo, Lionell Loh, Louth Rawshan, and Sudipta Chattopad- hyay. Model agnostic defence against backdoor attacks in machine learning. arXiv preprint arXiv: 022003... 2019.
Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. In Pro- ceedings of 40th IEEE Symposium on Security and Privacy, 2019.
Yuanshun Yao, Huiying Li, Haitao Zheng, and Ben Y Zhao. Latent backdoor attacks on deep neural networks. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communica- tions Security, pp. 2041-2055, 2019.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness deep features as perceptual metric. In CVPR, 2018
Pu Zhao, Pin-Ye Yu Chen, Payel Das, Karthikeyan Natesan Ramamurthy, and Xue Lin. Bridging mode connectivity in loss landscapes and adversarial robustness. In International Conference on Learning Representations, 2019.

---

Published as a conference paper at ICLR 2021
A APPENDIX
A.1 SYSTEM DETAILS
A.1.1 DATASETS
We used standard datasets, from simple to more complex ones, to conduct our experiments. As the datasets are all used in previous related works, results would be more comparable and reliable.
MNIST
The dataset (LeCun et al. 1998) a subset of the larger dataset available from the National Institute of Technology (NIST). This dataset consists of 70,000 grayscale, 28 28 images, divided into training set of 60,000 images and test set of 10,000 images. Original dataset could be found at http://yann. lecun. com/ exclbmmrist
We applied random cropping and random rotation as data augmentation for the training process. During the evaluation stage, no augmentation is applied.
CIFARIO
The dataset was introduced the first time by Krizhevsky et al. (2009). It is a labeled subset of the N0-millionsttiy-immage dataset, collected by Alex Krizhevsky. Vinod Nair and Geoffrey Hinton, consists of 60,000 color images at the resolution of 32 x 32. The dataset contains 10 classes, with 6,000 images per one. divided into two subsets: a training set 50,000 images and a test set of 10,000 images. The data set is public and avalable at https: www wow.nto..oooooooooo... /dv. "kriz/cifar.. html
During training stage, random crop, random rotation and random horizontal flip were applied as data augmentation. No augmentation was added at the evaluation stage.
GTSRB
The German Traffic Sign Recognition Benchmark the GTSRB (Stalkkamp al. (2012) is used as an official dataset for the challenge held at the International Joint Conference on Neural Network (ICCNN 2011. This dataset consists of 60,000 images with 43 classes and the resolution vary- ing from 32 x 32 to 250 x 250. is divided into a training set of 39,209 images and test set of 12.630. The dataset could be found httt::/ ini. rub de iss
Input images were all resized into x 32 pixels, then applied random crop and random rotation the training stage. No augmentation was used at the evaluation stage.
CelebA
CelebFaces Attributes Dataset CelebA, first introduced by Liu et al. (2015). large-scale face attributes dataset. It contains 10,177 identities with 202,599 face images. Each image has an annotation 5 landmark locations and 40 binary attributes. The dataset publicly available at http://ww.................aaeeep
nttp://mmlabh cu.. wwu.kk.ed..h.e...
Noted that this dataset is highly unbalanced. Due to the time limitation, we select 3 out of 40 attributes, namely Heavy Makeup, Mouth Slightly Open and Smiling, as suggested by [Salem et al. (2020). We then concatenate them into 8 classes to create a multiple label classification task. The input images were all resized into 64 x 64 pixels. Random crop and random rotation were applied as data augmentation at the training stage. No augmentation was applied at the evaluation stage.
A.1.2 CLASSIFICATION NETWORKS
MNIST
We used a simple, self self-defined structure as network classifier for this dataset. Detailed architecture will be mentioned in Table
12

---

Published as a conference paper at ICLR 2021
Table Detailed architecture MNIST classifier. means the layer is followed by Dropout layer.
i means the layer is followed by BatchNormalization layer.
Layer
Filter
Filter Size
Stride
Padding
Activation
Cone2dt
32
3 x
2
ReLU
Conx2d!
64
3 x
2
0
ReLU
Conv2d
64
3
2
ReLU
Linears
512
0
ReLU
Linear
10
Softmax
CIFARIO and GTSRB
For the CIFAR-10 and GTSRB datasets, we use PreActRes18 sification networks.
(He ct al. 2016) architecture as clas-
CelebA
For the CelebA dataset, we use ResNer18 (He et al. 2016) architecture as the classification network.
A.1.3 RUNNING TIME
We use system of a GPU RIX 2080Ti and CPU 9700K to conduct our experiment. Detailed inference time of cach module will be demonstrated below.
Table 3: Inference time of our modules.
MNIST CIFARIO GTSRB
CelebA
time/ sample 4.37 ps
18.64 us
18.65 μs 87.51 js
A.2 ALL-TO-ALL ATTACK
Beside the single-target attack scenario, also verified the effectiveness of WaNet in multi-target scenario, often called all-to-all attack. In this scenario, the input of class would be targeted into class c(y) (y 1) mod |C), where |C| the number of classes.
A.2.1 EXPERIMENTAL SETUP
We use the same experimental setups in the single-target scenario, with a small modification. In the attack mode at training, we replace the fixed target label è by (y + 1) mod [C]. In the attack test evaluation, also change the expected label similarly.
A.2.2 ATTACK EXPERIMENT
We conducted attack experiments and reported result in Table While models still achieve state- of-the-art performance on clean data, the attack efficacies slightly decreases. This is due to the fact that the target label now varies from input to input. Though, the lowest attack accuracy is 78.58%, which is still harmful to real-life deployment.
Similar to all-to-one scenario, also tested our model with noise mode and recorded the noise accuracy.
A.2.3 DEFENSE EXPERIMENTS
We repeat the same defense experiments used the all-to-one scenario. Our backdoor models could also pass all the tests mentioned in Figure 7
13

---

Published as a conference paper at ICLR 2021
Table 4: All-to-all attack result.
Dataset
Clean
Attack
Noise
MNIST
99.44
95.90
94.34
CIFAR-10
94.43
93.36
91.47
GTSRB
99.39
98.31
98.96
CelebA
78.73
78.58
76.12
clean
backdoor
Anomaly Index
4
MNIST
CIFAR-10
GTSRB
CelebA
Figure 10: Neural Cleanse against all-to-all scenario.
MNIST
CIFAR-10
GTRSB
CelebA
100
100
100
80
80
Accuracy
Accuracy
Accuracy
Accuracy
65
40
40
40
55
clean
backdoor
Filters Pruned
60
Filters Pruned
400
Filters Pruned
400
200
Pruned
400
Figure 11: Fine-pruning against all-to-all scenario.
0.20
0.3
barkddorr
entropy
2-
Entropy
2
0.19
60..1
6.1
0.05
0.0
Probability
0.00
2.0
Probabiliyy
0.0
Probability
0.00
2.0
Probability
Figure 12: STRIP against all-to-all scenario.
A.3
ADDITIONAL RESULTS
A.3.1
ADDIONAL IMAGES FOR METIONED BACKDOOR ATTACK METHODS
We provide additional examples comparing backdoor images from WaNet and from other attack methods in Fig. 13
A.3.2 EXPERIMENT ON SPECTRAL SIGNATURE DEFENSE
Tran et al. (0018) proposed a data defense method based on the spectral signature of backdoor training data. Although this data- deta-defense configuration does not match threat model, we find it useful to verify our backdoor data have the spectral signature discussed in that paper. We repeated the experiment in the last plot of its Fig. using 5000 clean samples and 1172 backdoor samples
14

---

Published as a conference paper at ICLR 2021
Original Image
Patched
Blended
SIG
ReFool
Warped (Ours)
STOP
STOP
STOP
STOP
STOP
STOP
Figure 13: Additional images for mentioned backdoor attack methods.
generated by WaNet on the CIFAR-10 dataset, which is the same dataset used in the original paper. Fig. plots histograms of the correlations between these samples' learned representations and their covariance matrix's 's top right singular vector. As can be seen, the histograms of the two populations completely inseparable. Thereby, the backdoor training samples could be removed from the training dataset using their proposed method. One possible explanation is that the distributional difference between the clean and backdoor correlations the traditional backdoor methods was the result of the domination of few backdoor neurons. We do not have such phenomenon in WaNet, as proved in Fine- Pranninn experiments, eliminating the appearance spectral signature.
A.3.3 THE STABILITY OF WANET
In this section, verify if WaNet stable to the variations of the warping field M. We trained WaNet backdoor models, using 8 randomly generated warping fields, in the CIFAR 10 dataset. The clean, backdoor, and noise accuracies of the trained models are all stable, as shown in Table
Table 5: The stability of WaNet the CIFAR-10 dataset.
Clean
Backdoor
Noise
Accuracy (%)
94.42 0.08 99.40 + 0.21 93.16 0.43
15

---

Published as a conference paper at ICLR 2021
Representation Level
clean
175
backdoor
Number of samples
150
125
100
75
50
25
0
-1.0
-0.5
0.0
0.5
1.0
Correlation with Top Right Singular Vector
Figure 14: Spectral Signature
A.3.4 ADDITIONAL TRIGGER PATTERNS VISUALIZING THE ROLE OF THE NOISE MODE
This section further demonstrates the importance of noise mode by providing trigger patterns opti- mized by Neural Cleanse on more datasets and with more target classes. Fig. 15a and 15b visualize the patterns on MNIST and GTSRB dataset using backdoor models trained for target label similar to Fig. 8a Fig.
15c 15d and 15e provide results on all three datasets but with backdoor models can be seen, the WaNet models without noise mode training return sparse and small
for label patterns, thus easy to be detected by Neural Cleanse. By including that training mode, the optimized patterns are more crowded and approach clean models' ones. Note that we skip visualizing the re- sults on the CelebA dataset; its patterns optimized either clean backdoor models are all too sparse and small for humans to analyze due to subtle differences between human faces.
Clean model
WaNet w/o noise mode (a) MNIST (label 0)
Clean model
WaNet w/o noise mode (b) GTSRB (label 0)
WaNet
WaNet
Clean model
WaNet w/o noise mode (c) MNIST (label
Clean model
WaNet w/o noise mode
WaNet
WaNet
(d) CIFARI0 (label
Clean model
WaNet naise mode
WaNet
(c) GTSRB (label 3)
Figure 15: Additional trigger patterns optimized by Neural Cleanse for the target label (small is bad).
16


Reviews:
[{"review_id": "wpbIrZ9RGmE", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a backdoor attack method using warp-based triggers, which distorts the global structure of the image (input) making the attack undetectable/un-noticeable by humans.\nThis global-distortion aspect of the attack makes effective against (besides humans) popular backdoor defense mechanisms such as neural cleanse and fine pruning.\n\nOverall, I enjoyed reading this paper and the ideas proposed therein. However, I have some remarks about its clarity and some of the experiments that I hope the authors address.\n\n-----\n\n*Quality* (8/10):\nThe paper provides substantial experimental evidence that the attack method is effective in terms of:\n1. Demonstrating the attack on a variety of datasets (MNIST, CIFAR10, GTSRB)\n2. Ablation studies of the proposition components.\n\nI'd have increased my Quality score if:\n- The experiment of Ablation Studies (Role of Noise Mode) was presented with more supporting evidence rather than a single-class from a single dataset optimized trigger patterns (Figure 7a.)\n- More datasets and/or various classifier architectures (Table 1)\n\n-----\n\n*Clarity* (6/10):\n- There is room for improving the paper's write-up and presentation (e.g., some figures are missing their x-axis/y-axis labels as in Fig 6b). \n\n-----\n*Originality* (9/10):\nI believe that this work is original. The bulk of backdoor attacks have been patch-based and this is one of few works that explore other backdoor alternatives.\n\n-----\n*Significance* (8/10):\nThis is a significant work and opens the door to new questions in the field of backdoor attacks.\n\n-----\nPros:\n- A new framework of imperceptible backdoor attacks that break away from the notion of patches  & water-marks.\n-Thorough experiments that validate the effectiveness of the method.\n\nCons:\n- Not enough supporting evidence to the role of Noise Mode in training.\n- Some missing experiments (highlighted below) to further support the proposition.\n\n----\n\n**Remarks/Questions to the authors**:\n- Fig 1: I suggest adding difference images for each of the methods, to highlight the differences each attack is incorporating into the image.\n- Sec 2.4: The justification for the unsuitability of affine & projective transformations for attack purposes is not clear. Do you mean they are not going to be effective?\n- Fig 2: Would be great to pinpoint some pixels of M and highlight what their values are in the [-1,1] interval besides the colormap.\n- Sec 3.2: It appears that there are no constraints on the backward warping? A target pixel ,say at[-1,-1], can have a source pixel at [1,1]? It is not clear how the control grid (Eq. 3) ensures the locality of the warp. Please elaborate.\n- Sec 3.2: Does M get generated once? or it's per image? I assume it is generated once.\n- Sec 3.2: If there is a single M, how robust is the training to different realizations of M? The experiments do not discuss that.\n- Table 2: Why \"Clean Inputs\" fooling rate differs across different attacks? Should not they be the same?\n- Figure 6a: Please specify how clean mode is obtained (is it based on the first row of Eq. 6) while \"backdoor\" represents all of Eq. 6?\n- Figure 6d: Please describe GradCam setup. Is it visualizing the activations based on the backdoor label $\\hat{c}$?\n- Figure 7b: I was expecting all the curves to start with a low-accuracy as the images from all the modes tend to be very similar for (k<6 and s<0.75; as shown in Figure 3) and so should not they suffer equally?\n\n**Post-Rebuttal**\nThe authors have addressed my comments and revised the manuscript accordingly. Recommending an accept.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "r4Uw4LFcVD6", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a new backdoor attack method on vision classifiers, where a warping-based trigger is adopted instead of patched triggers. To bypass backdoor detectors, the authors also propose a “noise” mode of training poisoned classifiers. \n1. For the image warping, from Figure 3, I can not see any difference between most of the warped images and the original one. Also, the authors claim that WaNet is an imperceptible backdoor attack. However, Table 2 reported that the fooling rate of human on backdoor inputs and clean inputs are 38.6% and 17.4% respectively, this suggests that there may still be some differences between backdoor inputs and clean inputs that can be identified by human. Can the authors show some such examples? \n2. I understand that the backdoor in this paper is the unique warping function. I think the authors should explicitly mention this as part of the threat model of WaNet.\n3. The authors mentioned in section 3.2 that “To get such a warping field, we extend the idea of TPS with some modifications”. What is the extension and what is the difference from the original TPS? I think the authors should explicitly define their contributions while describing the technical details.\n4. Did the authors try the defense “Spectral signatures in backdoor attacks” (Tran et al. 2018)? I think this is a valid defense method worth evaluating for WaNet.\n5. I suggest the authors briefly describe how the image warping process is conducted. Currently it is not discussed in the paper, which may not be good for interested readers.\n\nOverall i am satisfied with the paper presentation. The idea is interesting and the paper is self-contained. I would like the authors to address the issues I raise.", "extra_scores": {}, "extra_text": {}}, {"review_id": "BtCvcY7-Jy", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "1. Summary\n- This paper presents the imperceptible warping based back-door attack technique.\n- With the poisoned image generated from the technique, the network will be fooled.\n\n2. Strong points\n- The technique produces the imperceptible poisoned image.\n- Looks superior to other techniques.\n\n3. Weak points\n- I am not sure that the experiments are enough to show the technique's superiority.\n- Why did you perform experiments only on small datasets such as CIFAR-10, MNIST, and GTSRB?\n\n\nI am not confident in the review because I am not familiar with the attack algorithms.\nI can conclude my decision with other reviewer's reviews.\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a new method for generation of backdoor attacks against deep networks. The new method uses global warping instead of noise patches which makes the attack much more stealthy than previous approaches. The attack effectiveness is demonstrated on 3 benchmark datasets. A small user study is carried out to demonstrate that the attack is stealthier than conventional backdoor attacks. 

The new attack is a novel and original contribution which is likely to advance the understanding of backdoor attacks. There were some issues with respect to clarity in the original manuscript but the authors adequately addressed the critical remarks raised by the reviewers. 
Decision: Accept (Poster)