Original: W A N ET – I MPERCEPTIBLE W ARPING - BASED B ACKDOOR A TTACK
Revised: W A N ET – I MPERCEPTIBLE W ARPING - BASED B ACKDOOR A TTACK

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: With the thriving of deep learning and the widespread practice of using pre-trained networks, backdoor attacks have become an increasing security threat drawing many research interests in recent years.
Revised: With the thriving of deep learning and the widespread practice of using pretrained networks, backdoor attacks have become an increasing security threat drawing many research interests in recent years.

Original: A third-party model can be poisoned in training to work well in normal conditions but behave maliciously when a trigger pattern appears.
Revised: A third-party model can be poisoned in training to work well in normal conditions but behave maliciously when a trigger pattern appears.

Original: However, the existing backdoor attacks are all built on noise perturbation triggers, making them noticeable to humans.
Revised: However, the existing backdoor attacks are all built on noise perturbation triggers, making them noticeable to humans.

Original: In this paper, we instead propose using warping-based triggers.
Revised: In this paper, we instead propose using warping-based triggers.

Original: The proposed backdoor outperforms the previous methods in a human inspection test by a wide margin, proving its stealthiness.
Revised: The proposed backdoor outperforms the previous methods in a human inspection test by a wide margin, proving its stealthiness.

Original: To make such models undetectable by machine defenders, we propose a novel training mode, called the “noise” mode.
Revised: To make such models undetectable by machine defenders, we propose a novel training mode, called the “noise” mode.

Original: The trained networks successfully attack and bypass the state of the art defense methods on standard classiﬁcation datasets, including MNIST, CIFAR-10, GTSRB, and CelebA. Behavior analyses show that our backdoors are transparent to network inspection, further proving this novel attack mechanism’s efﬁciency. 
Revised: The trained networks successfully attack and bypass the state of the art defense methods on standard classiﬁcation datasets, including MNIST, CIFAR-10, GTSRB, and CelebA. Behavior analyses show that our backdoors are transparent to network inspection, further proving this novel attack mechanism’s efﬁciency. Our code is publicly available at https://github.com/VinAIResearch/ Warping-based_Backdoor_Attack-release .

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep learning models are essential in many modern systems due to their superior performance compared to classical methods.
Revised: Deep learning models are essential in many modern systems due to their superior performance compared to classical methods.

Original: Most state-of-the-art models, however, require expensive hardware, huge training data, and long training time.
Revised: Most state-of-the-art models, however, require expensive hardware, huge training data, and long training time.

Original: Hence, instead of training the models from scratch, it is a common practice to use pre-trained networks provided by third-parties these days.
Revised: Hence, instead of training the models from scratch, it is a common practice to use pre-trained networks provided by third-parties these days.

Original: This poses a serious security threat of backdoor attack (Gu et al., 2017).
Revised: This poses a serious security threat of backdoor attack (Gu et al., 2017).

Original: A backdoor model is a network poisoned either at training or ﬁnetuning.
Revised: A backdoor model is a network poisoned either at training or ﬁnetuning.

Original: It can work as a genuine model in the normal condition.
Revised: It can work as a genuine model in the normal condition.

Original: However, when a speciﬁc trigger appears in the input, the model will act maliciously, as designed by the attacker.
Revised: However, when a speciﬁc trigger appears in the input, the model will act maliciously, as designed by the attacker.

Original: Backdoor attack can occur in various tasks, including image recognition (Chen et al., 2017),
Revised: Backdoor attack can occur in various tasks, including image recognition (Chen et al., 2017),

Original: speech recognition (Liu et al., 2018b), natural language processing (Dai et al., 2019), and reinforcement learning (Hamon et al., 2020).
Revised: speech recognition (Liu et al., 2018b), natural language processing (Dai et al., 2019), and reinforcement learning (Hamon et al., 2020).

Original: In this paper, we will focus on image classiﬁcation, the most popular attacking target with possible fatal consequences (e.g., for self-driving car).
Revised: In this paper, we will focus on image classiﬁcation, the most popular attacking target with possible fatal consequences (e.g., for self-driving car).

Original: Since introduced, backdoor attack has drawn a lot of research interests (Chen et al., 2017;
Revised: Since introduced, backdoor attack has drawn a lot of research interests (Chen et al., 2017;

Original: Liu et al., 2018b; Salem et al.,  2020).
Revised: Liu et al., 2018b; Salem et al., 2020; Nguyen & Tran, 2020).

Original: In most of these works, trigger patterns are based on patch perturbation or image blending.
Revised: In most of these works, trigger patterns are based on patch perturbation or image blending.

Original: Recent papers have proposed novel patterns such as sinusoidal strips (Barni et al., 2019) and reﬂectance (Liu et al., 2020).
Revised: Recent papers have proposed novel patterns such as sinusoidal strips (Barni et al., 2019), and reﬂectance (Liu et al., 2020).

Original: These backdoor triggers, however, are unnatural and can be easily spotted by humans.
Revised: These backdoor triggers, however, are unnatural and can be easily spotted by humans.

Original: We believe that the added content, such as noise, strips, or reﬂectance, causes the backdoor samples generated by the previous methods strikingly detectable.
Revised: We believe that the added content, such as noise, strips, or reﬂectance, causes the backdoor samples generated by the previous methods strikingly detectable.

Original: Instead, we propose to use image warping that can deform but preserve image content.
Revised: Instead, we propose to use image warping that can deform but preserve image content.

Original: We also found that humans are not good at recognizing subtle image warping, while machines are excellent in this task.
Revised: We also found that humans are not good at recognizing subtle image warping, while machines are excellent in this task.

Original: Hence, in this paper, we design a novel, simple, but effective backdoor attack based on image warping called WaNet.
Revised: Hence, in this paper, we design a novel, simple, but effective backdoor attack based on image warping called WaNet.

Original: We use a small and smooth warping ﬁeld in generating backdoor images, making the modiﬁcation unnoticeable, as illustrated in Fig.
Revised: We use a small and smooth warping ﬁeld in generating backdoor images, making the modiﬁcation unnoticeable, as illustrated in Fig.

Original: Our backdoor images are natural and hard to be distinguished from the genuine examples, conﬁrmed by our user study described in Sec. 4.3. 
Revised: Our backdoor images are natural and hard to be distinguished from the genuine examples, conﬁrmed by our user study described in Sec. 4.3. [Figure]

Original: To obtain a backdoor model, we ﬁrst follow the common training procedure by poisoning a part of training data with a ﬁxed ratio of ρ a ∈ (0 , 1) .
Revised: To obtain a backdoor model, we ﬁrst follow the common training procedure by poisoning a part of training data with a ﬁxed ratio of ρ a ∈ (0 , 1) .

Original: While the trained networks provide high clean and attack accuracy, we found that they “cheated” by learning pixel-wise artifacts instead of the warping[Figure] e itself.
Revised: While the trained networks provide high clean and attack accuracy, we found that they “cheated” by learning pixel-wise artifacts instead of the warping itself.

Original: It makes them easy to be caught by a popular backdoor defense Neural Cleanse.
Revised: It makes them easy to be caught by a popular backdoor defense Neural Cleanse.

Original: Instead, we add another mode in training, called “noise mode”, to enforce the models to learn only the predeﬁned backdoor warp.
Revised: Instead, we add another mode in training, called “noise mode”, to enforce the models to learn only the predeﬁned backdoor warp.

Original: This novel training scheme produces satisfactory models that are both effective and stealthy.
Revised: This novel training scheme produces satisfactory models that are both effective and stealthy.

Original: Our attack method achieves invisibility without sacriﬁcing accuracy.
Revised: Our attack method achieves invisibility without sacriﬁcing accuracy.

Original: It performs similarly to stateof-the-art backdoor methods in terms of clean and attack accuracy, veriﬁed on common benchmarks such as MNIST, CIFAR-10, GTSRB, and CelebA. Our attack is also undetectable by various backdoor defense mechanisms; none of existing algorithms can recognize or mitigate our backdoor.
Revised: It performs similarly to stateof-the-art backdoor methods in terms of clean and attack accuracy, veriﬁed on common benchmarks such as MNIST, CIFAR-10, GTSRB, and CelebA. Our attack is also undetectable by various backdoor defense mechanisms; none of existing algorithms can recognize or mitigate our backdoor.

Original: This is because the attack mechanism of our method is drastically different from any existing attack, breaking the assumptions of all defense methods.
Revised: This is because the attack mechanism of our method is drastically different from any existing attack, breaking the assumptions of all defense methods.

Original: Finally, we demonstrate that our novel backdoor can be a practical threat by deploying it for physical attacks.
Revised: Finally, we demonstrate that our novel backdoor can be a practical threat by deploying it for physical attacks.

Original: We tested the backdoor classiﬁer with camera-captured images of physical screens.
Revised: We tested the backdoor classiﬁer with camera-captured images of physical screens.

Original: Despite image quality degradation via extreme capturing conditions, our backdoor is well-preserved, and the attack accuracy stays near 100%.
Revised: Despite image quality degradation via extreme capturing conditions, our backdoor is well-preserved, and the attack accuracy stays near 100%.

Original: In short, we introduce a novel backdoor attack via image warping.
Revised: In short, we introduce a novel backdoor attack via image warping.

Original: To train such a model, we extend the standard backdoor training scheme by introducing a “noise” training mode.
Revised: To train such a model, we extend the standard backdoor training scheme by introducing a “noise” training mode.

Original: The attack is effective, and the backdoor is imperceptible by both humans and computational defense mechanisms.
Revised: The attack is effective, and the backdoor is imperceptible by both humans and computational defense mechanisms.

Original: It can be deployed for physical attacks, creating a practical threat to deep-learning-based systems 1 .
Revised: It can be deployed for physical attacks, creating a practical threat to deep-learning-based systems 1 .

Original:  B ACKGROUND 2. T HREAT MODEL
Revised:  B ACKGROUND 2. T HREAT MODEL

Original: Backdoor attacks are techniques of poisoning a system to have a hidden destructive functionality.
Revised: Backdoor attacks are techniques of poisoning a system to have a hidden destructive functionality.

Original: The poisoned system can work genuinely on clean inputs but misbehave when a speciﬁc trigger pattern appears.
Revised: The poisoned system can work genuinely on clean inputs but misbehave when a speciﬁc trigger pattern appears.

Original: In the attack mode for image classiﬁcation, backdoor models can return a predeﬁned target label, normally incorrect, regardless of image content.
Revised: In the attack mode for image classiﬁcation, backdoor models can return a predeﬁned target label, normally incorrect, regardless of image content.

Original: It allows the attacker to gain illegal beneﬁts.
Revised: It allows the attacker to gain illegal beneﬁts.

Original: For example, a backdoor face authentication system may allow the attacker to access whenever he puts a speciﬁc sticker on the face.
Revised: For example, a backdoor face authentication system may allow the attacker to access whenever he puts a speciﬁc sticker on the face.

Original: Backdoors can be injected into the deep model at any stage.
Revised: Backdoors can be injected into the deep model at any stage.

Original: We consider model poisoning at training since it is the most used threat model.
Revised: We consider model poisoning at training since it is the most used threat model.

Original: The attacker has total control over the training process and maliciously alters data for his attack purposes.
Revised: The attacker has total control over the training process and maliciously alters data for his attack purposes.

Original: The poisoned model is then delivered to customers to deploy as-is.
Revised: The poisoned model is then delivered to customers to deploy as-is.

Original: In our proposed attack, the attacker selects a ﬁxed warping ﬁeld and uses it to generate all the backdoor images in training and in testing-time attacks.
Revised: In our proposed attack, the attacker selects a ﬁxed warping ﬁeld and uses it to generate all the backdoor images in training and in testing-time attacks.

Original: 2. P REVIOUS BACKDOOR ATTACKS
Revised: 2. P REVIOUS BACKDOOR ATTACKS

Original: We focus on backdoor attacks on image classiﬁcation.
Revised: We focus on backdoor attacks on image classiﬁcation.

Original: The target network is trained for a classiﬁcation task f : X → C , where X is an image domain and C
Revised: The target network is trained for a classiﬁcation task f : X → C , where X is an image domain and C

Original: = { c 1 , c 2 , ..., c M } is a set of M target classes.
Revised: = { c 1 , c 2 , ..., c M } is a set of M target classes.

Original: When poisoning f , we enforce it to learn an injection function B , a target label function c , and alter the network behaviour so that:
Revised: When poisoning f , we enforce it to learn an injection function B , a target label function c , and alter the network behaviour so that:

Original: [Equation]
Revised: [Equation]

Original: for any pair of clean image x ∈ X and the corresponding label y ∈ C .
Revised: for any pair of clean image x ∈ X and the corresponding label y ∈ C .

Original: The earliest backdoor attack was BadNets (Gu et al., 2017).
Revised: The earliest backdoor attack was BadNets (Gu et al., 2017).

Original: The authors suggested to poison a portion of training data by replacing each clean data pair ( x , y ) with the corresponding poisoned pair ( B ( x ) , c ( y )) .
Revised: The authors suggested to poison a portion of training data by replacing each clean data pair ( x , y ) with the corresponding poisoned pair ( B ( x ) , c ( y )) .

Original: The injection function B simply replaces a ﬁxed patch of the input image by a predeﬁned trigger pattern.
Revised: The injection function B simply replaces a ﬁxed patch of the input image by a predeﬁned trigger pattern.

Original: As for the target label function c ( y ) , the authors proposed two tests: (1) all-to-one with a constant target label c ( y ) =
Revised: As for the target label function c ( y ) , the authors proposed two tests: (1) all-to-one with a constant target label c ( y ) =

Original: ˆ c and (2) all-to-all with c ( y ) =
Revised: ˆ c and (2) all-to-all with c ( y ) =

Original: y + 1 .
Revised: y + 1 .

Original: After BadNets, many variants of backdoor attacks have been introduced.
Revised: After BadNets, many variants of backdoor attacks have been introduced.

Original: These approaches focus on changing either the backdoor injection process or the injection function B .
Revised: These approaches focus on changing either the backdoor injection process or the injection function B .

Original: As for the backdoor injection process, Liu et al. (2018b) proposed to inject backdoor into clean models via ﬁne-tuning instead of the training stage.
Revised: As for the backdoor injection process, Liu et al. (2018b) proposed to inject backdoor into clean models via ﬁne-tuning instead of the training stage.

Original: Yao et al.
Revised: Yao et al.

Original: (2019) suggested hiding backdoor inside latent neurons for transfer learning.
Revised: (2019) suggested hiding backdoor inside latent neurons for transfer learning.

Original: Many recent studies (Turner et al., 2019; Barni et al., 2019; Liu et al., 2020), injected backdoor only on samples with unchanged labels, i.e., the target c ( y ) is the same as the ground-truth label y , to dodge label inspection by humans.
Revised: Many recent studies (Turner et al., 2019; Barni et al., 2019; Liu et al., 2020), injected backdoor only on samples with unchanged labels, i.e., the target c ( y ) is the same as the ground-truth label y , to dodge label inspection by humans.

Original: In this paper, we focus on the development of a good injection function B .
Revised: In this paper, we focus on the development of a good injection function B .

Original: Most of the popular attack methods rely on ﬁxed patch-based triggers.
Revised: Most of the popular attack methods rely on ﬁxed patch-based triggers.

Original: Chen et al.
Revised: Chen et al.

Original: (2017) used image blending to embed the trigger into the input image.
Revised: (2017) used image blending to embed the trigger into the input image, and Nguyen & Tran (2020) extended it to be input-aware.

Original: Salem et al.
Revised: Salem et al.

Original: (2020) varied the patch-based trigger locations and patterns to make it “dynamic”.
Revised: (2020) varied the patch-based trigger locations and patterns to make it “dynamic”.

Original: Barni et al.
Revised: Barni et al.

Original: (2019) employed sinusoidal strips as thebackdoor trigger alongside the clean-label strategy.
Revised: (2019) employed sinusoidal strips as the trigger alongside the clean-label strategy.

Original: Lately, Liu et al.
Revised: Lately, Liu et al.

Original: (2020) proposed to disguise backdoor triggers as reﬂectance to make the poisoned images look natural.
Revised: (2020) proposed to disguise backdoor triggers as reﬂectance to make the poisoned images look natural.

Original: The backdoor images generated by these attacks, however, are easy to be spotted by humans.
Revised: The backdoor images generated by these attacks, however, are easy to be spotted by humans.

Original: We instead propose an “invisible” backdoor that is imperceptible by even sharp-eyed people.
Revised: We instead propose an “invisible” backdoor that is imperceptible by even sharp-eyed people.

Original: 2. B ACKDOOR DEFENSE METHODS
Revised: 2. B ACKDOOR DEFENSE METHODS

Original: As the threat of backdoor attacks becomes more apparent, backdoor defense research is emerging.
Revised: As the threat of backdoor attacks becomes more apparent, backdoor defense research is emerging.

Original: Based on usage scenarios, we can classify them into three groups: training defense, model defense, and testing-time defense.
Revised: Based on usage scenarios, we can classify them into three groups: training defense, model defense, and testing-time defense.

Original: Training defense assumes the defender has control over the training process, and the adversary attacks by providing infected training data (Tran et al., 2018).
Revised: Training defense assumes the defender has control over the training process, and the adversary attacks by providing infected training data (Tran et al., 2018).

Original: This assumption, however, does not match our threat model, where the already-trained backdoor model is provided by a third party.
Revised: This assumption, however, does not match our threat model, where the already-trained backdoor model is provided by a third party.

Original: This mechanism is not applicable to our situation and will not be considered further in this paper.
Revised: This mechanism is not applicable to our situation and will not be considered further in this paper.

Original: Model defenses aim to verify or mitigate the provided model before deployment.
Revised: Model defenses aim to verify or mitigate the provided model before deployment.

Original: Fine-Pruning (Liu et al., 2018a) suggested to prune the dormant neurons, deﬁned by analyses on a clean image set, to mitigate the backdoor if present.
Revised: Fine-Pruning (Liu et al., 2018a) suggested to prune the dormant neurons, deﬁned by analyses on a clean image set, to mitigate the backdoor if present.

Original: Neural Cleanse (Wang et al., 2019) was the ﬁrst work that could detect backdoor models.
Revised: Neural Cleanse (Wang et al., 2019) was the ﬁrst work that could detect backdoor models.

Original: It optimized a patch-based trigger candidate for each target label, then detected if any candidate was abnormally smaller than the others as a backdoor indicator.
Revised: It optimized a patch-based trigger candidate for each target label, then detected if any candidate was abnormally smaller than the others as a backdoor indicator.

Original: ABS (Liu et al., 2019) scanned the neurons and generated trigger candidates by reverse engineering.
Revised: ABS (Liu et al., 2019) scanned the neurons and generated trigger candidates by reverse engineering.

Original: Cheng et al.
Revised: Cheng et al.

Original: (2019) used GradCam (Selvaraju et al., 2017) to analyze the network behavior on a clean input image with and without the synthesized trigger to detect anomalies.
Revised: (2019) used GradCam (Selvaraju et al., 2017) to analyze the network behavior on a clean input image with and without the synthesized trigger to detect anomalies.

Original: Zhao et al.
Revised: Zhao et al.

Original: (2019) applied mode connectivity to effectively mitigate backdoor while keeping acceptable performance.
Revised: (2019) applied mode connectivity to effectively mitigate backdoor while keeping acceptable performance.

Original: Lately, Kolouri et al.
Revised: Lately, Kolouri et al.

Original: (2020) introduced universal litmus patterns that can be fed to the network to detect backdoor.
Revised: (2020) introduced universal litmus patterns that can be fed to the network to detect backdoor.

Original: Unlike model defense, testing-time defenses inspect models after deployment with the presence of input images.
Revised: Unlike model defense, testing-time defenses inspect models after deployment with the presence of input images.

Original: It focuses on verifying if the provided image is poisoned and how to mitigate it.
Revised: It focuses on verifying if the provided image is poisoned and how to mitigate it.

Original: STRIP (Gao et al., 2019) exploited the persistent outcome of the backdoor image under perturbations for detection.
Revised: STRIP (Gao et al., 2019) exploited the persistent outcome of the backdoor image under perturbations for detection.

Original: In contrast, Neo (Udeshi et al., 2019) searched for the candidate trigger patches where region blocking changed the predicted outputs.
Revised: In contrast, Neo (Udeshi et al., 2019) searched for the candidate trigger patches where region blocking changed the predicted outputs.

Original: Recently, Doan et al.
Revised: Recently, Doan et al.

Original: (2019) used GradCam inspection to detect potential backdoor locations.
Revised: (2019) used GradCam inspection to detect potential backdoor locations.

Original: In all these methods, the trigger candidates were then veriﬁed by being injected into a set of clean images.
Revised: In all these methods, the trigger candidates were then veriﬁed by being injected into a set of clean images.

Original: A common assumption in all previous defense methods is that the backdoor triggers are image patches.
Revised: A common assumption in all previous defense methods is that the backdoor triggers are image patches.

Original: We instead propose a novel attack mechanism based on image warping, undermining the foundation of these methods.
Revised: We instead propose a novel attack mechanism based on image warping, undermining the foundation of these methods.

Original: 2. E LASTIC IMAGE WARPING
Revised: 2. E LASTIC IMAGE WARPING

Original: Image warping is a basic image processing technique that deforms an image by applying the geometric transformation.
Revised: Image warping is a basic image processing technique that deforms an image by applying the geometric transformation.

Original: The transformation can be afﬁne, projective, elastic, or non-elastic.
Revised: The transformation can be afﬁne, projective, elastic, or non-elastic.

Original: In this work, we propose to use elastic image warping given its advantages over the others: (1) Afﬁne and projective transformations are naturally introduced to clean images via the image capturing process.
Revised: In this work, we propose to use elastic image warping given its advantages over the others: (1) Afﬁne and projective transformations are naturally introduced to clean images via the image capturing process.

Original: If we apply these transformations to these images, the transformed images can be identical to other clean images that are of the same scenes but captured at different viewpoints.
Revised: If we apply these transformations to these images, the transformed images can be identical to other clean images that are of the same scenes but captured at different viewpoints.

Original: Hence, these transformations are not suitable to generate backdoor examples, particularly in physical attacks.
Revised: Hence, these transformations are not suitable to generate backdoor examples, particularly in physical attacks.

Original: (2) Elastic transformation still generates natural outputs while non-elastic one does not.
Revised: (2) Elastic transformation still generates natural outputs while non-elastic one does not.

Original: The most popular elastic warping technique is Thin-Plate Splines (TPS) (Duchon, 1977).
Revised: The most popular elastic warping technique is Thin-Plate Splines (TPS) (Duchon, 1977).

Original: TPS can interpolate a smooth warping ﬁeld to transform the entire image given a set of control points with known original and target 2D coordinates.
Revised: TPS can interpolate a smooth warping ﬁeld to transform the entire image given a set of control points with known original and target 2D coordinates.

Original: TPS was adopted in Spatial Transformer Networks (Jaderberg et al., 2015), the ﬁrst deep learning study incorporating differential image warping.
Revised: TPS was adopted in Spatial Transformer Networks (Jaderberg et al., 2015), the ﬁrst deep learning study incorporating differential image warping.

Original: We believe that elastic image warping can be utilized to generate invisible backdoor triggers.
Revised: We believe that elastic image warping can be utilized to generate invisible backdoor triggers.

Original: Unlike previous attack methods that introduce extra and independent information to an input image, elastic image warping only manipulates existing pixels of the image.
Revised: Unlike previous attack methods that introduce extra and independent information to an input image, elastic image warping only manipulates existing pixels of the image.

Original: Humans, while being excellent in spotting incongruent part of an image, are bad at recognizing small geometric transformations.
Revised: Humans, while being excellent in spotting incongruent part of an image, are bad at recognizing small geometric transformations.

Original:  W ARPING - BASED B ACKDOOR A TTACK
Revised:  W ARPING - BASED B ACKDOOR A TTACK

Original: We now describe our novel backdoor attack method WaNet, which stand for Warping-based poisoned Networks.
Revised: We now describe our novel backdoor attack method WaNet, which stand for Wa rping-based poisoned Networks.

Original: WaNet are designed to be stealthy to both machine and human inspections.
Revised: WaNet are designed to be stealthy to both machine and human inspections.

Original: 3. O VERVIEW
Revised: 3. O VERVIEW

Original: Recall that a classiﬁcation network is a function f : X → C , in which X is an input image domain and C is a set of target classes.
Revised: Recall that a classiﬁcation network is a function f : X → C , in which X is an input image domain and C is a set of target classes.

Original: To train f , a training dataset S
Revised: To train f , a training dataset S

Original: = { ( x i , y i )
Revised: = { ( x i , y i )

Original: | x
Revised: | x

Original: i ∈ X , y i ∈ C ,
Revised: i ∈ X , y i ∈ C ,

Original: i = 1 , N } is provided.
Revised: i = 1 , N } is provided.

Original: We follow the training scheme of BadNets to poison a subset of S with ratio ρ a for backdoor training.
Revised: We follow the training scheme of BadNets to poison a subset of S with ratio ρ a for backdoor training.

Original: Each clean pair ( x , y ) will be replaced by a backdoor pair ( B ( x ) , c ( y )) , in which
Revised: Each clean pair ( x , y ) will be replaced by a backdoor pair ( B ( x ) , c ( y )) , in which

Original: B is the backdoor injection function and c ( y ) is the target label function.
Revised: B is the backdoor injection function and c ( y ) is the target label function.

Original: Our main focus is to redesign the injection function B based on image warping.
Revised: Our main focus is to redesign the injection function B based on image warping.

Original: We construct B using a warping function W and a predeﬁned warping ﬁeld M :
Revised: We construct B using a warping function W and a predeﬁned warping ﬁeld M :

Original: [Equation]
Revised: [Equation]

Original: M acts like a motion ﬁeld; it deﬁnes the relative sampling location of backward warping for each point in the target image.
Revised: M acts like a motion ﬁeld; it deﬁnes the relative sampling location of backward warping for each point in the target image.

Original: W allows a ﬂoating-point warping ﬁeld as input.
Revised: W allows a ﬂoating-point warping ﬁeld as input.

Original: When a sampling pixel falls on non-integer 2D coordinates, it will be bi-linear interpolated.
Revised: When a sampling pixel falls on non-integer 2D coordinates, it will be bi-linear interpolated.

Original: To implement W , we rely on the public API grid sample provided by PyTorch.
Revised: To implement W , we rely on the public API grid sample provided by PyTorch.

Original: However, this API inputs a grid of normalized absolute 2D coordinates of the sampling points.
Revised: However, this API inputs a grid of normalized absolute 2D coordinates of the sampling points.

Original: To use that API, we ﬁrst sum M with an identity sampling grid, then normalize to [ − 1 , 1] to get the required grid input.
Revised: To use that API, we ﬁrst sum M with an identity sampling grid, then normalize to [ − 1 , 1] to get the required grid input.

Original: 3. W ARPING FIELD GENERATION
Revised: 3. W ARPING FIELD GENERATION

Original: The warping ﬁeld M is a crucial component; it must guarantee that the warped images are both natural and effective for attacking purposes.
Revised: The warping ﬁeld M is a crucial component; it must guarantee that the warped images are both natural and effective for attacking purposes.

Original: Hence, M are desired to satisfy the following properties:
Revised: Hence, M are desired to satisfy the following properties:

Original: • Small : M should be small, to be unnoticeable to humans,  • Elastic : M should be elastic,
Revised: • Small : M should be small, to be unnoticeable to humans, [Figure] [Figure] • Elastic : M should be elastic,

Original: i
Revised: i

Original:  e., smooth and non-ﬂat, to generate natural looking images, • Within image boundary : M should not exceed the image boundary, to avoid creating suspicious black/plain outer area.[Figure]
Revised: [Figure] e., smooth and non-ﬂat, to generate natural looking images, • Within image boundary : M should not exceed the image boundary, to avoid creating suspicious black/plain outer area.

Original: [Figure] To get such a warping ﬁeld, we borrow the idea of using control points from TPS but simplify the interpolation method.
Revised: To get such a warping ﬁeld [Figure] we borrow the idea of using control points from TPS but simplify the interpolation method.

Original: The process of generating the desired warp is illustrated by Fig.
Revised: The process of generating the desired warp is illustrated by Fig.

Original: and is described in the following subsections.
Revised: and is described in the following subsections.

Original: Selecting the control grid We ﬁrst select the control points.
Revised: Selecting the control grid We ﬁrst select the control points.

Original: For simplicity, we pick the target points on a uniform grid of size k × k over the entire image.
Revised: For simplicity, we pick the target points on a uniform grid of size k × k over the entire image.

Original: Their backward warping ﬁeld is denoted as
Revised: Their backward warping ﬁeld is denoted as

Original: P ∈ R k × k × 2 .
Revised: P ∈ R k × k × 2 .

Original: We use a parameter s to deﬁne the strength of P and generate P as following:
Revised: We use a parameter s to deﬁne the strength of P and generate P as following:

Original: [Equation]
Revised: [Equation]

Original: in which rand [ − 1 , 1] ( . . . ) is a function returning random tensor with the input shape and element value in the range [ − 1 , 1] and ψ is a normalization function.
Revised: in which rand [ − 1 , 1] ( . . . ) is a function returning random tensor with the input shape and element value in the range [ − 1 , 1] and ψ is a normalization function.

Original: In this paper, we normalize the tensor elements by their mean absolute value:
Revised: In this paper, we normalize the tensor elements by their mean absolute value:

Original: [Equation]
Revised: [Equation]

Original: Upsampling From the control points, we interpolate the warping ﬁeld of the entire image.
Revised: Upsampling From the control points, we interpolate the warping ﬁeld of the entire image.

Original: Since these points are in a uniform grid covering the entire image, instead of using a complex spline-based interpolation like in TPS, we can simply apply bicubic interpolation.
Revised: Since these points are in a uniform grid covering the entire image, instead of using a complex spline-based interpolation like in TPS, we can simply apply bicubic interpolation.

Original: We denote the output of this step as M 0 = ↑ P ∈ R
Revised: We denote the output of this step as M 0 = ↑ P ∈ R

Original: h × w × 2 , with h and w being the image height and width respectively.
Revised: h × w × 2 , with h and w being the image height and width respectively.

Original: Clipping Finally, we apply a clipping function φ so that the sampling points do not fall outside of the image border.
Revised: Clipping Finally, we apply a clipping function φ so that the sampling points do not fall outside of the image border.

Original: The process of generating M can be summarized by the equation:
Revised: The process of generating M can be summarized by the equation:

Original: [Equation]
Revised: [Equation]

Original: We investigate the effect of the hyper-parameters k and s qualitatively in Fig.
Revised: We investigate the effect of the hyper-parameters k and s qualitatively in Fig.

Original: The warping effect is almost invisible when k < 6 and s < 0 .
Revised: The warping effect is almost invisible when k < 6 and s < 0 .

Original: 3. R UNNING MODES
Revised: 3. R UNNING MODES

Original: After computing the warping ﬁeld M , we can train WaNet with with two modes, clean and attack, as the standard protocol.
Revised: After computing the warping ﬁeld M , we can train WaNet with with two modes, clean and attack, as the standard protocol.

Original: However, the models trained by that algorithm, while still achieving highaccuracy in both clean and attack tests, tend to learn pixel-level artifacts instead of the warping.
Revised: However, the models trained by that algorithm, while still achieving high

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Revised: MNIST 99.52 99.86 98.

Original: CIFAR-10 [-94.
 [Figure]-]
Revised: CIFAR-10 [-94.
 [Figure]-]94.15 99.55 93.

Original: [Figure] [Figure]
Revised: GTSRB 98.97 98.78 98.

Original: CelebA 
Revised: CelebA 78.99 99.33 76.

Original: [Figure]
Revised: [Figure]

Revised: accuracy in both clean and attack tests, tend to learn pixel-level artifacts instead of the warping.

Original: They are, therefore, easily exposed by a backdoor defense method such as Neural Cleanse.
Revised: They are, therefore, easily exposed by a backdoor defense method such as Neural Cleanse.

Original: We will discuss more details in the ablation studies in Section 4.6.
Revised: We will discuss more details in the ablation studies in Section 4.6.

Original: To resolve this problem, we propose a novel training mode alongside the clean and attack mode, called noise mode.
Revised: To resolve this problem, we propose a novel training mode alongside the clean and attack mode, called noise mode.

Original: The idea is simple: when applying a random warping ﬁeld M (cid:48) (cid:54) =
Revised: The idea is simple: when applying a random warping ﬁeld M (cid:48) (cid:54) =

Original: M , the network should not trigger the backdoor but return the correct class prediction.
Revised: M , the network should not trigger the backdoor but return the correct class prediction.

Original: Fig.
Revised: Fig.

Original: 4 illustrates three running modes in our training pipelines.
Revised: 4 illustrates three running modes in our training pipelines.

Original: We ﬁrst select the backdoor probability ρ a ∈ (0 , 1) and the noise probability ρ n ∈ (0 , 1) such that ρ a + ρ n < 1 .
Revised: We ﬁrst select the backdoor probability ρ a ∈ (0 , 1) and the noise probability ρ n ∈ (0 , 1) such that ρ a + ρ n < 1 .

Original: Then, for each clean input ( x , y ) , we randomly select one of three modes and alter that pair accordingly: ( x , y ) (cid:55)→   ( x , y ) with probability 1 − ρ a − ρ n ( W ( x , M ) , c ( y )) with probability ρ a ( W ( x , M + rand [ − 1 , 1] ( h, w, 2)) , y ) with probability ρ n (6)
Revised: Then, for each clean input ( x , y ) , we randomly select one of three modes and alter that pair accordingly: [Equation] with probability 1 − ρ a − ρ n [Equation]

Revised: with probability ρ a

Revised: [Equation]

Original:  Note that with the noise mode, instead of using a totally random warping ﬁeld, we form it by adding
Revised: probability [Equation] [Equation] Note that with the noise mode, instead of using a totally random warping ﬁeld, we form it by adding

Original: Gaussian noise to M for a more effective training.
Revised: Gaussian noise to M for a more effective training.

Original: The modiﬁed training set is then used to train f .
Revised: The modiﬁed training set is then used to train f .

Original:  E XPERIMENTS 4. E XPERIMENTAL S ETUP
Revised:  E XPERIMENTS 4. E XPERIMENTAL S ETUP

Original: Following the previous backdoor attack papers, we performed experiments on four datasets: MNIST (LeCun et al., 1998), CIFAR-10 (Krizhevsky et al., 2009), GTSRB (Stallkamp et al., 2012) and
Revised: Following the previous backdoor attack papers, we performed experiments on four datasets: MNIST (LeCun et al., 1998), CIFAR-10 (Krizhevsky et al., 2009), GTSRB (Stallkamp et al., 2012) and

Original: CelebA (Liu et al., 2015).
Revised: CelebA (Liu et al., 2015).

Original: Note that CelebA dataset has annotations for 40 independent binary attributes, which is not suitable for multi-class classiﬁcation.
Revised: Note that CelebA dataset has annotations for 40 independent binary attributes, which is not suitable for multi-class classiﬁcation.

Original: Therefore, we follow the conﬁguration suggested by Salem et al.
Revised: Therefore, we follow the conﬁguration suggested by Salem et al.

Original: (2020) to select the top three most balanced attributes, including Heavy
Revised: (2020) to select the top three most balanced attributes, including Heavy

Original: Makeup, Mouth Slightly Open, and Smiling, then concatenate them to create eight classiﬁcation classes.
Revised: Makeup, Mouth Slightly Open, and Smiling, then concatenate them to create eight classiﬁcation classes.

Original: Their detail information are shown in Table 1.
Revised: Their detail information are shown in Table 1.

Original: To build the classiﬁer f for the color image datasets, we used Pre-activation Resnet-18 (He et al., 2016) for the CIFAR-10 and GTSRB datasets as suggested by Kang (2020), and Resnet-18 for the CelebA dataset.
Revised: To build the classiﬁer f for the color image datasets, we used Pre-activation Resnet-18 (He et al., 2016) for the CIFAR-10 and GTSRB datasets as suggested by Kang (2020), and Resnet-18 for the CelebA dataset.

Original: As for the grayscale dataset
Revised: As for the grayscale dataset

Original: MNIST, we deﬁned a simple network structure as reported in Table 1.
Revised: MNIST, we deﬁned a simple network structure as reported in Table 1.

Original: We trained the networks using the SGD optimizer.
Revised: We trained the networks using the SGD optimizer.

Original: The initial learning rate was 0.01, which was reduced by a factor of 10 after each 100 training epochs.
Revised: The initial learning rate was 0.01, which was reduced by a factor of 10 after each 100 training epochs.

Original: The networks were trained until convergence.
Revised: The networks were trained until convergence.

Original: We used k = 4, s = 0.5, ρ a = 0.1, and ρ n = 0.2.
Revised: We used k = 4, s = 0.5, ρ a = 0.1, and ρ n = 0.2.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: 4. A TTACK EXPERIMENTS
Revised: 4. A TTACK EXPERIMENTS

Original: We trained and tested the backdoor models in all-to-one conﬁguration, i
Revised: We trained and tested the backdoor models in all-to-one conﬁguration, i

Original: [Table]
Revised: [Table]

Original: e., c ( y ) =
Revised: e., c ( y ) =

Original: ˆ c ∀ y .
Revised: ˆ c ∀ y .

Original: The accuracy values in clean mode, attack mode
Revised: The accuracy values in clean mode, attack mode

Original: [Table]
Revised: [Table]

Original: and the noise mode are reported in Fig.
Revised: and the noise mode are reported in Fig.

Original: 5a.
Revised: 5a.

Original: As can be seen, with clean images, the networks could correctly classify them like any benign models, with accuracy near 100% on MNIST/GTSRB, 94.15% on CIFAR-
Revised: As can be seen, with clean images, the networks could correctly classify them like any benign models, with accuracy near 100% on MNIST/GTSRB, 94.15% on CIFAR-

Original: [Table]
Revised: [Table]

Original: and 79.77% on CelebA. When applying the pre-deﬁned image warping, the attack success rate was near 100% on all datasets.
Revised: and 79.77% on CelebA. When applying the pre-deﬁned image warping, the attack success rate was near 100% on all datasets.

Original: However, when using a random warping, the classiﬁers still recognized the true image class with a similar accuracy as in the clean mode.
Revised: However, when using a random warping, the classiﬁers still recognized the true image class with a similar accuracy as in the clean mode.

Original: This result is impressive given the fact that the poisoned images look almost identical to the original, as can be seen in Fig.
Revised: This result is impressive given the fact that the poisoned images look almost identical to the original, as can be seen in Fig.

Original: 5b.
Revised: 5b.

Original: To evaluate our method’s robustness in real-life scenarios, we also tested if backdoor images would still be misclassiﬁed even when being distorted by the capturing process.
Revised: To evaluate our method’s robustness in real-life scenarios, we also tested if backdoor images would still be misclassiﬁed even when being distorted by the capturing process.

Original: We showed 50 clean and 50 backdoor images on a screen and recaptured them using a phone camera.
Revised: We showed 50 clean and 50 backdoor images on a screen and recaptured them using a phone camera.

Original: Our model still worked well on recaptured images, obtaining 98% clean accuracy and 96% attack success rate.
Revised: Our model still worked well on recaptured images, obtaining 98% clean accuracy and 96% attack success rate.

Original: Fig.
Revised: Fig.

Original: 5c displays an example of our test.
Revised: 5c displays an example of our test.

Original: The clean image was recognized correctly as “automobile”, while the look-a-like backdoor image was recognized as the “airplane” attack class.
Revised: The clean image was recognized correctly as “automobile”, while the look-a-like backdoor image was recognized as the “airplane” attack class.

Original: 4. H UMAN INSPECTION
Revised: 4. H UMAN INSPECTION

Original: To examine the realisticity of our backdoor and the previous methods, we created user studies with human inspection.
Revised: To examine the realisticity of our backdoor and the previous methods, we created user studies with human inspection.

Original: First, we randomly selected 25 images from the GTSRB dataset.
Revised: First, we randomly selected 25 images from the GTSRB dataset.

Original: Second, for each backdoor injection function, we created the corresponding 25 backdoor images and mixed them with the original to obtain a set of 50 images.
Revised: Second, for each backdoor injection function, we created the corresponding 25 backdoor images and mixed them with the original to obtain a set of 50 images.

Original: Finally, we asked 40 people to classify whether each image was genuine, collecting 2000 answers per method.
Revised: Finally, we asked 40 people to classify whether each image was genuine, collecting 2000 answers per method.

Original: The participants were trained about the mechanism and characteristics of the attack before answering the questions.
Revised: The participants were trained about the mechanism and characteristics of the attack before answering the questions.

Original: We collected the answers and reported the percentage of incorrect answers as the success fooling rates in Fig.
Revised: We collected the answers and reported the percentage of incorrect answers as the success fooling rates in Fig.

Original: 6a.
Revised: 6a.

Original: Note that when the backdoor examples are more indistinguishable from the clean ones, the testers will ﬁnd it harder to decide an image is clean or poisoned.
Revised: Note that when the backdoor examples are more indistinguishable from the clean ones, the testers will ﬁnd it harder to decide an image is clean or poisoned.

Original: Hence, better backdoor methods led to higher fooling rates on not only backdoor inputs but also on clean ones.
Revised: Hence, better backdoor methods led to higher fooling rates on not only backdoor inputs but also on clean ones.

Original: The rates from previous methods are low, with maximum 7.7% on all inputs, implying that they are obvious to humans to detect.
Revised: The rates from previous methods are low, with maximum 7.7% on all inputs, implying that they are obvious to humans to detect.

Original: In contrast, our rate is 28%, four times their best number.
Revised: In contrast, our rate is 28%, four times their best number.

Original: It conﬁrms that WaNet is stealthy and hard to detect, even with trained people.
Revised: It conﬁrms that WaNet is stealthy and hard to detect, even with trained people.

Original: Although our backdoor images are natural-looking, some of them have subtle properties that can be detected by trained testers.
Revised: Although our backdoor images are natural-looking, some of them have subtle properties that can be detected by trained testers.

Original: We provide two of the most detected backdoor examples from WaNet in
Revised: We provide two of the most detected backdoor examples from WaNet in

Original: Fig.
Revised: Fig.

Original: 6b.
Revised: 6b.

Original: In the ﬁrst case, the circle sign is not entirely round.
Revised: In the ﬁrst case, the circle sign is not entirely round.

Original: In the second case, the right edge of the trafﬁc sign is slightly curved.
Revised: In the second case, the right edge of the trafﬁc sign is slightly curved.

Original: Although these conditions can be found on real-life trafﬁc signs, they are not common in the testing dataset GTSRB.
Revised: Although these conditions can be found on real-life trafﬁc signs, they are not common in the testing dataset GTSRB.

Original: These images are of the minority, and our fooling rate on backdoor images is 38.6%, not far away from the rate of 50% in random selection.
Revised: These images are of the minority, and our fooling rate on backdoor images is 38.6%, not far away from the rate of 50% in random selection.

Original: [Figure]
Revised: [Figure]

Original: 4. D EFENSE EXPERIMENTS
Revised: 4. D EFENSE EXPERIMENTS

Original: We will now test the trained models against the popular backdoor defense mechanisms, including
Revised: We will now test the trained models against the popular backdoor defense mechanisms, including

Original: Neural Cleanse, Fine-Prunning (Model defenses)
Revised: Neural Cleanse, Fine-Prunning (Model defenses)

Original: [Figure]
Revised: [Figure]

Original: and STRIPS (Testing-time defense).
Revised: and STRIPS (Testing-time defense).

Original: Neural Cleanse (Wang et al., 2019) is a model-defense method based on the pattern optimization approach.
Revised: Neural Cleanse (Wang et al., 2019) is a model-defense method based on the pattern optimization approach.

Original: It assumes that the backdoor is patch-based.
Revised: It assumes that the backdoor is patch-based.

Original: For each class label, Neural Cleanse computes the optimal patch pattern to convert any clean input to that target label.
Revised: For each class label, Neural Cleanse computes the optimal patch pattern to convert any clean input to that target label.

Original: It then checks if any label has a signiﬁcantly smaller pattern as a sign of backdoor.
Revised: It then checks if any label has a signiﬁcantly smaller pattern as a sign of backdoor.

Original: Neural Cleanse quantiﬁes it by the
Revised: Neural Cleanse quantiﬁes it by the

Original: Anomaly Index metric with the clean/backdoor threshold τ = 2 .
Revised: Anomaly Index metric with the clean/backdoor threshold τ = 2 .

Original: We ran Neural Cleanse over our
Revised: We ran Neural Cleanse over our

Original: WaNet models and report the numbers in Fig.
Revised: WaNet models and report the numbers in Fig.

Original: 7c.
Revised: 7c.

Original: WaNet passed the test on all datasets; its scores are even smaller than the clean model ones on MNIST and CIFAR-10.
Revised: WaNet passed the test on all datasets; its scores are even smaller than the clean model ones on MNIST and CIFAR-10.

Original: We can explain it by the fact that our backdoor relies on warping, a different mechanism compared with patch-based blending.
Revised: We can explain it by the fact that our backdoor relies on warping, a different mechanism compared with patch-based blending.

Original: Fine-Pruning (Liu et al., 2018a) , instead, focuses on neuron analyses.
Revised: Fine-Pruning (Liu et al., 2018a) , instead, focuses on neuron analyses.

Original: Given a speciﬁc layer, it analyzes the neuron responses on a set of clean images and detects the dormant neurons, assuming they are more likely to tie to the backdoor.
Revised: Given a speciﬁc layer, it analyzes the neuron responses on a set of clean images and detects the dormant neurons, assuming they are more likely to tie to the backdoor.

Original: These neurons are then gradually pruned to mitigate the backdoor.
Revised: These neurons are then gradually pruned to mitigate the backdoor.

Original: We tested Fine-Pruning on our models and plotting the network accuracy, either clean or attack, with respect to the number of neurons pruned in Fig.
Revised: We tested Fine-Pruning on our models and plotting the network accuracy, either clean or attack, with respect to the number of neurons pruned in Fig.

Original: 7a.
Revised: 7a.

Original: On all datasets, at no point is the clean accuracy considerably higher than the attack one, making backdoor mitigation impossible.
Revised: On all datasets, at no point is the clean accuracy considerably higher than the attack one, making backdoor mitigation impossible.

Original: STRIP (Gao et al., 2019) is a representative of the testing-time defense approach.
Revised: STRIP (Gao et al., 2019) is a representative of the testing-time defense approach.

Original: It examines the model with the presence of the input image.
Revised: It examines the model with the presence of the input image.

Original: STRIP works by perturbing the input image through a set of clean images from different classes and raising the alarm if the prediction is persistent, indicating by low entropy.
Revised: STRIP works by perturbing the input image through a set of clean images from different classes and raising the alarm if the prediction is persistent, indicating by low entropy.

Original: With WaNet, the perturbation operation of STRIP will modify the image content and break the backdoor warping if present.
Revised: With WaNet, the perturbation operation of STRIP will modify the image content and break the backdoor warping if present.

Original: Hence, WaNet behaves like genuine models, with similar entropy ranges, as shown in Fig.
Revised: Hence, WaNet behaves like genuine models, with similar entropy ranges, as shown in Fig.

Original: 7b.
Revised: 7b.

Original: 4. N ETWORK INSPECTION
Revised: 4. N ETWORK INSPECTION

Original: Visualization tools, such as GradCam (Selvaraju et al., 2017), are helpful in inspecting network behaviors.
Revised: Visualization tools, such as GradCam (Selvaraju et al., 2017), are helpful in inspecting network behaviors.

Original: Patch-based backdoor methods can be exposed easily due to the use of small trigger
Revised: Patch-based backdoor methods can be exposed easily due to the use of small trigger

Original: [Figure]
Revised: [Figure]

Original: regions, as pointed out by Cheng et al.
Revised: regions, as pointed out by Cheng et al.

Original: ; Doan et al.
Revised: ; Doan et al.

Original: Our attack method is based on the warping on the entire image, so it is undetectable by this algorithm.
Revised: Our attack method is based on the warping on the entire image, so it is undetectable by this algorithm.

Original: We visualize activation based on the label that has the highest prediction score in Fig.
Revised: We visualize activation based on the label that has the highest prediction score in Fig.

Original: 7d.
Revised: 7d.

Original: With clean models, that label is for the correct class label.
Revised: With clean models, that label is for the correct class label.

Original: With WaNet and backdoor inputs, it is the backdoor label ˆ c .
Revised: With WaNet and backdoor inputs, it is the backdoor label ˆ c .

Original: As can be seen, the visualization heatmaps of WaNet look like the ones from any clean model.
Revised: As can be seen, the visualization heatmaps of WaNet look like the ones from any clean model.

Original: 4. A BLATION STUDIES
Revised: 4. A BLATION STUDIES

Original: Role of the noise mode Without the noise mode, we could still train a backdoor model with similar clean and attack accuracy.
Revised: Role of the noise mode Without the noise mode, we could still train a backdoor model with similar clean and attack accuracy.

Original: However, these models failed the defense test with Neural Cleanse as shown in Fig. 9, and the optimized trigger patterns revealed their true behavior.
Revised: However, these models failed the defense test with Neural Cleanse as shown in Fig. 9, and the optimized trigger patterns revealed their true behavior.

Original: [Figure]
Revised: [Figure]

Original: Fig.
Revised: Fig.

Original: 8a displays the trigger patterns optimized by Neural Cleanse for the attacking class “airplane” on CIFAR-10.
Revised: 8a displays the trigger patterns optimized by Neural Cleanse for the attacking class “airplane” on CIFAR-10.

Original: With the clean model, this pattern has an airplane-like shape, and it is big enough to rewrite image content given any input.
Revised: With the clean model, this pattern has an airplane-like shape, and it is big enough to rewrite image content given any input.

Original: With our model trained without noise mode, the optimized pattern just consists of scattered points.
Revised: With our model trained without noise mode, the optimized pattern just consists of scattered points.

Original: This pattern is remarkably smaller, making the model caught by Neural Cleanse.
Revised: This pattern is remarkably smaller, making the model caught by Neural Cleanse.

Original: It reveals that the model did not learn the speciﬁc backdoor warping; instead, it remembered the pixel-wise artifacts.
Revised: It reveals that the model did not learn the speciﬁc backdoor warping; instead, it remembered the pixel-wise artifacts.

Original: By adding the noise training mode, our model no longer relies on those artifacts, and the optimized pattern looks similar to the clean model’s one.
Revised: By adding the noise training mode, our model no longer relies on those artifacts, and the optimized pattern looks similar to the clean model’s one.

Original: Other hyper-parameters We investigated the effect of the warping hyper-parameters, including the strength s and the grid size k . Fig.
Revised: Other hyper-parameters We investigated the effect of the warping hyper-parameters, including the strength s and the grid size k . Fig.

Original: 8b and 8c show the clean, attack, and noise mode accuracy of our network on the CIFAR-10 dataset when changing each of these parameters.
Revised: 8b and 8c show the clean, attack, and noise mode accuracy of our network on the CIFAR-10 dataset when changing each of these parameters.

Original: When k or s is small, the backdoor images are similar to the clean ones.
Revised: When k or s is small, the backdoor images are similar to the clean ones.

Original: However, since they are a minority ( ρ a = 0 . 1 ), the network would treat them like data with noisy labels in those scenarios.
Revised: However, since they are a minority ( ρ a = 0 . 1 ), the network would treat them like data with noisy labels in those scenarios.

Original: Hence, clean and noise accuracies are stable across conﬁgurations.
Revised: Hence, clean and noise accuracies are stable across conﬁgurations.

Original: In contrast, backdoor accuracy suffers on the left side of the plots.
Revised: In contrast, backdoor accuracy suffers on the left side of the plots.

Original: It gradually increases when s or k is small, then saturates and stays near 100%.
Revised: It gradually increases when s or k is small, then saturates and stays near 100%.

Original:  C ONCLUSION AND FUTURE WORKS
Revised:  C ONCLUSION AND FUTURE WORKS

Original: This paper introduces a novel backdoor attack method that generates backdoor images via subtle image warping.
Revised: This paper introduces a novel backdoor attack method that generates backdoor images via subtle image warping.

Original: The backdoor images are proved to be natural and undetectable by humans.
Revised: The backdoor images are proved to be natural and undetectable by humans.

Original: We incorporate in training a novel “noise” mode, making it stealthy and pass all the known defense methods.
Revised: We incorporate in training a novel “noise” mode, making it stealthy and pass all the known defense methods.

Original: It opens a new domain of attack mechanism and encourages future defense research.
Revised: It opens a new domain of attack mechanism and encourages future defense research.

Original: A A PPENDIX A. S YSTEM DETAILS A.1. D ATASETS
Revised: A A PPENDIX A. S YSTEM DETAILS A.1. D ATASETS

Original: We used 3 standard datasets, from simple to more complex ones, to conduct our experiments.
Revised: We used 3 standard datasets, from simple to more complex ones, to conduct our experiments.

Original: As the datasets are all used in previous related works, our results would be more comparable and reliable.
Revised: As the datasets are all used in previous related works, our results would be more comparable and reliable.

Original: MNIST
Revised: MNIST

Original: The dataset (LeCun et al., 1998) is a subset of the larger dataset available from the National Institute of Technology (NIST).
Revised: The dataset (LeCun et al., 1998) is a subset of the larger dataset available from the National Institute of Technology (NIST).

Original: This dataset consists of 70,000 grayscale, 28 × 28 images, divided into a training set of 60,000 images and a test set of 10,000 images.
Revised: This dataset consists of 70,000 grayscale, 28 × 28 images, divided into a training set of 60,000 images and a test set of 10,000 images.

Original: Original dataset could be found at http://yann.lecun.com/exdb/mnist/ .
Revised: Original dataset could be found at http://yann.lecun.com/exdb/mnist/ .

Original: We applied random cropping and random rotation as data augmentation for the training process.
Revised: We applied random cropping and random rotation as data augmentation for the training process.

Original: During the evaluation stage, no augmentation is applied.
Revised: During the evaluation stage, no augmentation is applied.

Original: CIFAR 
Revised: CIFAR

Original: The dataset was introduced the ﬁrst time by Krizhevsky et al.
Revised: The dataset was introduced the ﬁrst time by Krizhevsky et al.

Original: It is a labeled subset of the 80-millions-tiny-images dataset, collected by Alex Krizhevsky, Vinod Nair and Geoffrey Hinton, consists of 60,000 color images at the resolution of 32 × 32 .
Revised: It is a labeled subset of the 80-millions-tiny-images dataset, collected by Alex Krizhevsky, Vinod Nair and Geoffrey Hinton, consists of 60,000 color images at the resolution of 32 × 32 .

Original: The dataset contains 10 classes, with 6,000 images per one.
Revised: The dataset contains 10 classes, with 6,000 images per one.

Original: It is divided into two subsets: a training set of 50,000 images and a test set of 10,000 images.
Revised: It is divided into two subsets: a training set of 50,000 images and a test set of 10,000 images.

Original: The data set is public and avalable at https://www.cs.toronto.edu/ ˜kriz/cifar.html .
Revised: The data set is public and avalable at https://www.cs.toronto.edu/ ˜kriz/cifar.html .

Original: During training stage, random crop, random rotation and random horizontal ﬂip were applied as data augmentation.
Revised: During training stage, random crop, random rotation and random horizontal ﬂip were applied as data augmentation.

Original: No augmentation was added at the evaluation stage.
Revised: No augmentation was added at the evaluation stage.

Original: GTSRB
Revised: GTSRB

Original: The German Trafﬁc Sign Recognition Benchmark - the GTSRB (Stallkamp et al.
Revised: The German Trafﬁc Sign Recognition Benchmark - the GTSRB (Stallkamp et al.

Original: (2012)) is used as an ofﬁcial dataset for the challenge held at the International Joint Conference on Neural Network (IJCNN) 2011.
Revised: (2012)) is used as an ofﬁcial dataset for the challenge held at the International Joint Conference on Neural Network (IJCNN) 2011.

Original: This dataset consists of 60,000 images with 43 classes and the resolution varying from 32 × 32 to 250 × 250 .
Revised: This dataset consists of 60,000 images with 43 classes and the resolution varying from 32 × 32 to 250 × 250 .

Original: It is divided into a training set of 39,209 images and a test set of 12,630.
Revised: It is divided into a training set of 39,209 images and a test set of 12,630.

Original: The dataset could be found at http://benchmark.ini.rub.de/?section= gtsrb&subsection=dataset .
Revised: The dataset could be found at http://benchmark.ini.rub.de/?section= gtsrb&subsection=dataset .

Original: Input images were all resized into 32 × 32 pixels, then applied random crop and random rotation at the training stage.
Revised: Input images were all resized into 32 × 32 pixels, then applied random crop and random rotation at the training stage.

Original: No augmentation was used at the evaluation stage.
Revised: No augmentation was used at the evaluation stage.

Original: CelebA
Revised: CelebA

Original: CelebFaces Attributes Dataset - CelebA, ﬁrst introduced by Liu et al. (2015), is a large-scale face attributes dataset.
Revised: CelebFaces Attributes Dataset - CelebA, ﬁrst introduced by Liu et al. (2015), is a large-scale face attributes dataset.

Original: It contains 10,177 identities with 202,599 face images.
Revised: It contains 10,177 identities with 202,599 face images.

Original: Each image has an annotation of 5 landmark locations and 40 binary attributes.
Revised: Each image has an annotation of 5 landmark locations and 40 binary attributes.

Original: The dataset is publicly available at http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html .
Revised: The dataset is publicly available at http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html .

Original: Noted that this dataset is highly unbalanced.
Revised: Noted that this dataset is highly unbalanced.

Original: Due to the time limitation, we select 3 out ofattributes, namely Heavy Makeup, Mouth Slightly Open and Smiling, as suggested by Salem et al.
Revised: Due to the time limitation, we select 3 out ofattributes, namely Heavy Makeup, Mouth Slightly Open and Smiling, as suggested by Salem et al.

Original: We then concatenate them into 8 classes to create a multiple label classiﬁcation task.
Revised: We then concatenate them into 8 classes to create a multiple label classiﬁcation task.

Original: The input images were all resized into 64 × 64 pixels.
Revised: The input images were all resized into 64 × 64 pixels.

Original: Random crop and random rotation were applied as data augmentation at the training stage.
Revised: Random crop and random rotation were applied as data augmentation at the training stage.

Original: No augmentation was applied at the evaluation stage.
Revised: No augmentation was applied at the evaluation stage.

Original: A.1. C LASSIFICATION NETWORKS MNIST
Revised: A.1. C LASSIFICATION NETWORKS MNIST

Original: We used a simple, self-deﬁned structure as network classiﬁer for this dataset.
Revised: We used a simple, self-deﬁned structure as network classiﬁer for this dataset.

Original: Detailed architecture will be mentioned in Table 2.
Revised: Detailed architecture will be mentioned in Table 2.

Original: [Table]
Revised: [Table]

Original: CIFAR10 andGTSRB
Revised: CIFAR10 and

Revised: [Table]

Original: For the CIFAR-10 and GTSRB datasets, we use PreActRes18 (He et al., 2016) architecture as classiﬁcation networks.
Revised: For the CIFAR-10 and GTSRB datasets, we use PreActRes18 (He et al., 2016) architecture as classiﬁcation networks.

Original: CelebA
Revised: CelebA

Original: For the CelebA dataset, we use ResNet18 (He et al., 2016) architecture as the classiﬁcation network.
Revised: For the CelebA dataset, we use ResNet18 (He et al., 2016) architecture as the classiﬁcation network.

Original: A.1. R UNNING TIME
Revised: A.1. R UNNING TIME

Original: We use a system of a GPU RTX 2080Ti and a CPU i7 9700K to conduct our experiment.
Revised: We use a system of a GPU RTX 2080Ti and a CPU i7 9700K to conduct our experiment.

Original: Detailed inference time of each module will be demonstrated below.
Revised: Detailed inference time of each module will be demonstrated below.

Original: [Table]
Revised: [Table]

Original: Beside the single-target attack scenario, we also veriﬁed the effectiveness of WaNet in multi-target scenario, often called all-to-all attack.
Revised: Beside the single-target attack scenario, we also veriﬁed the effectiveness of WaNet in multi-target scenario, often called all-to-all attack.

Original: In this scenario, the input of class y would be targeted into class c ( y ) =
Revised: In this scenario, the input of class y would be targeted into class c ( y ) =

Original: ( y + 1) mod | C | , where | C | is the number of classes.
Revised: ( y + 1) mod | C | , where | C | is the number of classes.

Original: A.2. E XPERIMENTAL SETUP
Revised: A.2. E XPERIMENTAL SETUP

Original: We use the same experimental setups as in the single-target scenario, with a small modiﬁcation.
Revised: We use the same experimental setups as in the single-target scenario, with a small modiﬁcation.

Original: In the attack mode at training, we replace the ﬁxed target label ˆ c by ( y + 1) mod | C | .
Revised: In the attack mode at training, we replace the ﬁxed target label ˆ c by ( y + 1) mod | C | .

Original: In the attack test at evaluation, we also change the expected label similarly.
Revised: In the attack test at evaluation, we also change the expected label similarly.

Original: A.2. A TTACK EXPERIMENT
Revised: A.2. A TTACK EXPERIMENT

Original: We conducted attack experiments and reported result in Table 4.
Revised: We conducted attack experiments and reported result in Table 4.

Original: While models still achieve stateof-the-art performance on clean data, the attack efﬁcacies slightly decreases.
Revised: While models still achieve stateof-the-art performance on clean data, the attack efﬁcacies slightly decreases.

Original: This is due to the fact that the target label now varies from input to input.
Revised: This is due to the fact that the target label now varies from input to input.

Original: Though, the lowest attack accuracy is 78.58%, which is still harmful to real-life deployment.
Revised: Though, the lowest attack accuracy is 78.58%, which is still harmful to real-life deployment.

Original: Similar to all-to-one scenario, we also tested our model with noise mode and recorded the noise accuracy.
Revised: Similar to all-to-one scenario, we also tested our model with noise mode and recorded the noise accuracy.

Original: A.2. D EFENSE EXPERIMENTS
Revised: A.2. D EFENSE EXPERIMENTS

Original: We repeat the same defense experiments used in the all-to-one scenario.
Revised: We repeat the same defense experiments used in the all-to-one scenario.

Original: Our backdoor models could also pass all the tests mentioned in Figure 7.
Revised: Our backdoor models could also pass all the tests mentioned in Figure 7.

Original: Dataset
Revised: Dataset

Original: Clean Attack
Revised: Clean Attack

Original: [Figure]
Revised: [Figure]

Original: MNIST 99.
Revised: MNIST 99.

Original: CIFAR-10 94.
Revised: CIFAR-10 94.

Original: GTSRB 99.
Revised: GTSRB 99.

Original: CelebA 78.
Revised: CelebA 78.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: A. A DDITIONAL R ESULTS A.3. A DDIONAL IMAGES FOR METIONED BACKDOOR ATTACK METHODS
Revised: A. A DDITIONAL R ESULTS A.3. A DDIONAL IMAGES FOR METIONED BACKDOOR ATTACK METHODS

Original: We provide additional examples comparing backdoor images from WaNet and from other attack methods in Fig. 13.
Revised: We provide additional examples comparing backdoor images from WaNet and from other attack methods in Fig. 13.

Original: A.3. E XPERIMENT ON SPECTRAL SIGNATURE DEFENSE
Revised: A.3. E XPERIMENT ON SPECTRAL SIGNATURE DEFENSE

Original: Tran et al.
Revised: Tran et al.

Original: (2018) proposed a data defense method based on the spectral signature of backdoor training data.
Revised: (2018) proposed a data defense method based on the spectral signature of backdoor training data.

Original: Although this data-defense conﬁguration does not match our threat model, we ﬁnd it useful to verify if our backdoor data have the spectral signature discussed in that paper.
Revised: Although this data-defense conﬁguration does not match our threat model, we ﬁnd it useful to verify if our backdoor data have the spectral signature discussed in that paper.

Original: We repeated the experiment in the last plot of its Fig.
Revised: We repeated the experiment in the last plot of its Fig.

Original: 1, using 5000 clean samples and 1172 backdoor samples
Revised: 1, using 5000 clean samples and 1172 backdoor samples

Original: [Figure]
Revised: [Figure]

Original: generated by WaNet on the CIFAR-10 dataset, which is the same dataset used in the original paper.
Revised: generated by WaNet on the CIFAR-10 dataset, which is the same dataset used in the original paper.

Original: Fig.
Revised: Fig.

Original: 14 plots histograms of the correlations between these samples’ learned representations and their covariance matrix’s top right singular vector.
Revised: 14 plots histograms of the correlations between these samples’ learned representations and their covariance matrix’s top right singular vector.

Original: As can be seen, the histograms of the two populations are completely inseparable.
Revised: As can be seen, the histograms of the two populations are completely inseparable.

Original: Thereby, the backdoor training samples could not be removed from the training dataset using their proposed method.
Revised: Thereby, the backdoor training samples could not be removed from the training dataset using their proposed method.

Original: One possible explanation is that the distributional difference between the clean and backdoor correlations in the traditional backdoor methods was the result of the domination of a few backdoor neurons.
Revised: One possible explanation is that the distributional difference between the clean and backdoor correlations in the traditional backdoor methods was the result of the domination of a few backdoor neurons.

Original: We do not have such a phenomenon in WaNet, as proved in Fine-Prunning experiments, eliminating the appearance of spectral signature.
Revised: We do not have such a phenomenon in WaNet, as proved in Fine-Prunning experiments, eliminating the appearance of spectral signature.

Original: A.3. T HE S TABILITY OF W A N ET
Revised: A.3. T HE S TABILITY OF W A N ET

Original: In this section, we verify if WaNet is stable to the variations of the warping ﬁeld M .
Revised: In this section, we verify if WaNet is stable to the variations of the warping ﬁeld M .

Original: We trained
Revised: We trained

Original: WaNet backdoor models, using 8 randomly generated warping ﬁelds, in the CIFAR10 dataset.
Revised: WaNet backdoor models, using 8 randomly generated warping ﬁelds, in the CIFAR10 dataset.

Original: The clean, backdoor, and noise accuracies of the trained models are all stable, as shown in Table 5.
Revised: The clean, backdoor, and noise accuracies of the trained models are all stable, as shown in Table 5.

Original: [Table]
Revised: [Table]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: A.3. A DDITIONAL TRIGGER PATTERNS VISUALIZING THE ROLE OF THE NOISE MODE
Revised: A.3. A DDITIONAL TRIGGER PATTERNS VISUALIZING THE ROLE OF THE NOISE MODE

Original: This section further demonstrates the importance of noise mode by providing trigger patterns optimized by Neural Cleanse on more datasets and with more target classes.
Revised: This section further demonstrates the importance of noise mode by providing trigger patterns optimized by Neural Cleanse on more datasets and with more target classes.

Original: Fig.
Revised: Fig.

Original: 15a and 15b visualize the patterns on MNIST and GTSRB dataset using backdoor models trained for target label 0, similar to Fig.
Revised: 15a and 15b visualize the patterns on MNIST and GTSRB dataset using backdoor models trained for target label 0, similar to Fig.

Original: 8a.
Revised: 8a.

Original: Fig.
Revised: Fig.

Original: 15c, 15d
Revised: 15c, 15d

Original: [Figure]
Revised: [Figure]

Original: and 15e provide results on all three datasets but with backdoor models for label 3.
Revised: and 15e provide results on all three datasets but with backdoor models for label 3.

Original: As can be seen, the WaNet models without noise mode training return sparse and small patterns, thus easy to be detected by Neural Cleanse.
Revised: As can be seen, the WaNet models without noise mode training return sparse and small patterns, thus easy to be detected by Neural Cleanse.

Original: By including that training mode, the optimized patterns are more crowded and approach clean models’ ones.
Revised: By including that training mode, the optimized patterns are more crowded and approach clean models’ ones.

Original: Note that we skip visualizing the results on the CelebA dataset; its patterns optimized on either clean or backdoor models are all  sparse and small for humans to analyze due to subtle differences between human faces.
Revised: Note that we skip visualizing the results on the CelebA dataset; its patterns optimized on either clean or backdoor models are all too sparse and small for humans to analyze due to subtle differences between human faces.

Original: [Figure]
Revised: [Figure]