Original: W A N ET – I MPERCEPTIBLE W ARPING - BASED B ACKDOOR A TTACK
Revised: W A N ET – I MPERCEPTIBLE W ARPING - BASED B ACKDOOR A TTACK

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: With the thriving of deep learning and the widespread practice of using pre-trained networks, backdoor attacks have become an increasing security threat drawing many research interests in recent years.
Revised: With the thriving of deep learning and the widespread practice of using pre-trained networks, backdoor attacks have become an increasing security threat drawing many research interests in recent years.

Original: A third-party model can be poisoned in training to work well in normal conditions but behave maliciously when a trigger pattern appears.
Revised: A third-party model can be poisoned in training to work well in normal conditions but behave maliciously when a trigger pattern appears.

Original: However, the existing backdoor attacks are all built on noise perturbation triggers, making them noticeable to humans.
Revised: However, the existing backdoor attacks are all built on noise perturbation triggers, making them noticeable to humans.

Original: In this paper, we instead propose using warping-based triggers.
Revised: In this paper, we instead propose using warping-based triggers.

Original: The proposed backdoor outperforms the previous methods in a human inspection test by a wide margin, proving its stealthiness.
Revised: The proposed backdoor outperforms the previous methods in a human inspection test by a wide margin, proving its stealthiness.

Original: To make such models undetectable by machine defenders, we propose a novel training mode, called the “noise” mode.
Revised: To make such models undetectable by machine defenders, we propose a novel training mode, called the “noise” mode.

Original: The trained networks successfully attack and bypass the state of the art defense methods on standard classiﬁcation datasets, including MNIST, CIFAR-10, GTSRB and CelebA. Behavior analyses show that our backdoors are transparent to network inspection, further proving this novel attack mechanism’s efﬁciency.
Revised: The trained networks successfully attack and bypass the state of the art defense methods on standard classiﬁcation datasets, including MNIST, CIFAR-10, GTSRB and CelebA. Behavior analyses show that our backdoors are transparent to network inspection, further proving this novel attack mechanism’s efﬁciency.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep learning models are essential in many modern systems due to their superior performance compared to classical methods.
Revised: Deep learning models are essential in many modern systems due to their superior performance compared to classical methods.

Original: Most state-of-the-art models, however, require expensive hardware, huge training data, and long training time.
Revised: Most state-of-the-art models, however, require expensive hardware, huge training data, and long training time.

Original: Hence, instead of training the models from scratch, it is a common practice to use pre-trained networks provided by third-parties these days.
Revised: Hence, instead of training the models from scratch, it is a common practice to use pre-trained networks provided by third-parties these days.

Original: This poses a serious security threat of backdoor attack (Gu et al., 2017).
Revised: This poses a serious security threat of backdoor attack (Gu et al., 2017).

Original: A backdoor model is a network poisoned either at training or ﬁnetuning.
Revised: A backdoor model is a network poisoned either at training or ﬁnetuning.

Original: It can work as a genuine model in the normal condition.
Revised: It can work as a genuine model in the normal condition.

Original: However, when a speciﬁc trigger appears in the input, the model will act maliciously, as designed by the attacker.
Revised: However, when a speciﬁc trigger appears in the input, the model will act maliciously, as designed by the attacker.

Original: Backdoor attack can occur in various tasks, including image recognition (Chen et al., 2017),
Revised: Backdoor attack can occur in various tasks, including image recognition (Chen et al., 2017),

Original: speech recognition (Liu et al., 2018b), natural language processing (Dai et al., 2019), and reinforcement learning (Hamon et al., 2020).
Revised: speech recognition (Liu et al., 2018b), natural language processing (Dai et al., 2019), and reinforcement learning (Hamon et al., 2020).

Original: In this paper, we will focus on image classiﬁcation, the most popular attacking target with possible fatal consequences (e.g., for self-driving car).
Revised: In this paper, we will focus on image classiﬁcation, the most popular attacking target with possible fatal consequences (e.g., for self-driving car).

Original: Since introduced, backdoor attack has drawn a lot of research interests (Chen et al., 2017;
Revised: Since introduced, backdoor attack has drawn a lot of research interests (Chen et al., 2017;

Original: Liu et al., 2018b; Salem et al., 2020).
Revised: Liu et al., 2018b; Salem et al., 2020).

Original: In most of these works, trigger patterns are based on patch perturbation or image blending.
Revised: In most of these works, trigger patterns are based on patch perturbation or image blending.

Original: Recent papers have proposed novel patterns such as sinusoidal strips (Barni et al., 2019) and reﬂectance (Liu et al., 2020).
Revised: Recent papers have proposed novel patterns such as sinusoidal strips (Barni et al., 2019) and reﬂectance (Liu et al., 2020).

Original: These backdoor triggers, however, are unnatural and can be easily spotted by humans.
Revised: These backdoor triggers, however, are unnatural and can be easily spotted by humans.

Original: We believe that the added content, such as noise, strips, or reﬂectance, causes the backdoor samples generated by the previous methods strikingly detectable.
Revised: We believe that the added content, such as noise, strips, or reﬂectance, causes the backdoor samples generated by the previous methods strikingly detectable.

Original: Instead, we propose to use image warping that can deform but preserve image content.
Revised: Instead, we propose to use image warping that can deform but preserve image content.

Original: We also found that humans are not good at recognizing subtle image warping, while machines are excellent in this task.
Revised: We also found that humans are not good at recognizing subtle image warping, while machines are excellent in this task.

Original: Hence, in this paper, we design a novel, simple, but effective backdoor attack based on image warping called WaNet.
Revised: Hence, in this paper, we design a novel, simple, but effective backdoor attack based on image warping called WaNet.

Original: We use a small and smooth warping ﬁeld in generating backdoor images, making the modiﬁcation unnoticeable, as illustrated in Fig.
Revised: We use a small and smooth warping ﬁeld in generating backdoor images, making the modiﬁcation unnoticeable, as illustrated in Fig.

Original: Our backdoor images are natural and hard to be distinguished from the genuine examples, conﬁrmed by our user study described in Sec. 4.3.
Revised: Our backdoor images are natural and hard to be distinguished from the genuine examples, conﬁrmed by our user study described in Sec. 4.3.

Original: To obtain a backdoor model, we ﬁrst follow the common training procedure by poisoning a part of training data with a ﬁxed ratio of ρ a ∈ (0 , 1) .
Revised: To obtain a backdoor model, we ﬁrst follow the common training procedure by poisoning a part of training data with a ﬁxed ratio of ρ a ∈ (0 , 1) .

Original: While the trained networks provide high clean and attack accuracy, we found that they “cheated” by learning pixel-wise artifacts instead of the warping
Revised: While the trained networks provide high clean and attack accuracy, we found that they “cheated” by learning pixel-wise artifacts instead of the warping

Original: [Figure]
Revised: [Figure]

Original: itself.
Revised: itself.

Original: It makes them easy to be caught by a popular backdoor defense Neural Cleanse.
Revised: It makes them easy to be caught by a popular backdoor defense Neural Cleanse.

Original: Instead, we add another mode in training, called “noise mode”, to enforce the models to learn only the predeﬁned backdoor warp.
Revised: Instead, we add another mode in training, called “noise mode”, to enforce the models to learn only the predeﬁned backdoor warp.

Original: This novel training scheme produces satisfactory models that are both effective and stealthy.
Revised: This novel training scheme produces satisfactory models that are both effective and stealthy.

Original: Our attack method achieves invisibility without sacriﬁcing accuracy.
Revised: Our attack method achieves invisibility without sacriﬁcing accuracy.

Original: It performs similarly to stateof-the-art backdoor methods in terms of clean and attack accuracy, veriﬁed on common benchmarks such as MNIST, CIFAR-10, GTSRB and CelebA. Our attack is also undetectable by various backdoor defense mechanisms; none of existing algorithms can recognize or mitigate our backdoor.
Revised: It performs similarly to stateof-the-art backdoor methods in terms of clean and attack accuracy, veriﬁed on common benchmarks such as MNIST, CIFAR-10, GTSRB and CelebA. Our attack is also undetectable by various backdoor defense mechanisms; none of existing algorithms can recognize or mitigate our backdoor.

Original: This is because the attack mechanism of our method is drastically different from any existing attack, breaking the assumptions of all defense methods.
Revised: This is because the attack mechanism of our method is drastically different from any existing attack, breaking the assumptions of all defense methods.

Original: Finally, we demonstrate that our novel backdoor can be a practical threat by deploying it for physical attacks.
Revised: Finally, we demonstrate that our novel backdoor can be a practical threat by deploying it for physical attacks.

Original: We tested the backdoor classiﬁer with camera-captured images of physical screens.
Revised: We tested the backdoor classiﬁer with camera-captured images of physical screens.

Original: Despite image quality degradation via extreme capturing conditions, our backdoor is well-preserved, and the attack accuracy stays near 100%.
Revised: Despite image quality degradation via extreme capturing conditions, our backdoor is well-preserved, and the attack accuracy stays near 100%.

Original: In short, we introduce a novel backdoor attack via image warping.
Revised: In short, we introduce a novel backdoor attack via image warping.

Original: To train such a model, we extend the standard backdoor training scheme by introducing a “noise” training mode.
Revised: To train such a model, we extend the standard backdoor training scheme by introducing a “noise” training mode.

Original: The attack is effective, and the backdoor is imperceptible by both humans and computational defense mechanisms.
Revised: The attack is effective, and the backdoor is imperceptible by both humans and computational defense mechanisms.

Original: It can be deployed for physical attacks, creating a practical threat to deep-learning-based systems 1 .
Revised: It can be deployed for physical attacks, creating a practical threat to deep-learning-based systems 1 .

Original:  B ACKGROUND 2. T HREAT MODEL
Revised:  B ACKGROUND 2. T HREAT MODEL

Original: Backdoor attacks are techniques of poisoning a system to have a hidden destructive functionality.
Revised: Backdoor attacks are techniques of poisoning a system to have a hidden destructive functionality.

Original: The poisoned system can work genuinely on clean inputs but misbehave when a speciﬁc trigger pattern appears.
Revised: The poisoned system can work genuinely on clean inputs but misbehave when a speciﬁc trigger pattern appears.

Original: In the attack mode for image classiﬁcation, backdoor models can return a predeﬁned target label, normally incorrect, regardless of image content.
Revised: In the attack mode for image classiﬁcation, backdoor models can return a predeﬁned target label, normally incorrect, regardless of image content.

Original: It allows the attacker to gain illegal beneﬁts.
Revised: It allows the attacker to gain illegal beneﬁts.

Original: For example, a backdoor face authentication system may allow the attacker to access whenever he puts a speciﬁc sticker on the face.
Revised: For example, a backdoor face authentication system may allow the attacker to access whenever he puts a speciﬁc sticker on the face.

Original: Backdoors can be injected into the deep model at any stage.
Revised: Backdoors can be injected into the deep model at any stage.

Original: We consider model poisoning at training since it is the most used threat model.
Revised: We consider model poisoning at training since it is the most used threat model.

Original: The attacker has total control over the training process and maliciously alters data for his attack purposes.
Revised: The attacker has total control over the training process and maliciously alters data for his attack purposes.

Original: The poisoned model is then delivered to customers to deploy as-is.
Revised: The poisoned model is then delivered to customers to deploy as-is.

Original: In our proposed attack, the attacker selects a ﬁxed warping ﬁeld and uses it to generate all the backdoor images in training and in testing-time attacks.
Revised: In our proposed attack, the attacker selects a ﬁxed warping ﬁeld and uses it to generate all the backdoor images in training and in testing-time attacks.

Original: 2. P REVIOUS BACKDOOR ATTACKS
Revised: 2. P REVIOUS BACKDOOR ATTACKS

Original: We focus on backdoor attacks on image classiﬁcation.
Revised: We focus on backdoor attacks on image classiﬁcation.

Original: The target network is trained for a classiﬁcation task f : X → C , where X is an image domain and C
Revised: The target network is trained for a classiﬁcation task f : X → C , where X is an image domain and C

Original: = { c 1 , c 2 , ..., c M } is a set of M target classes.
Revised: = { c 1 , c 2 , ..., c M } is a set of M target classes.

Original: When poisoning f , we enforce it to learn an injection function B , a target label function c , and alter the network behaviour so that:
Revised: When poisoning f , we enforce it to learn an injection function B , a target label function c , and alter the network behaviour so that:

Original: [Equation]
Revised: [Equation]

Original: for any pair of clean image x ∈ X and the corresponding label y ∈ C .
Revised: for any pair of clean image x ∈ X and the corresponding label y ∈ C .

Original: The earliest backdoor attack was BadNets (Gu et al., 2017).
Revised: The earliest backdoor attack was BadNets (Gu et al., 2017).

Original: The authors suggested to poison a portion of training data by replacing each clean data pair ( x , y ) with the corresponding poisoned pair ( B ( x ) , c ( y )) .
Revised: The authors suggested to poison a portion of training data by replacing each clean data pair ( x , y ) with the corresponding poisoned pair ( B ( x ) , c ( y )) .

Original: The injection function B simply replaces a ﬁxed patch of the input image by a predeﬁned trigger pattern.
Revised: The injection function B simply replaces a ﬁxed patch of the input image by a predeﬁned trigger pattern.

Original: As for the target label function c ( y ) , the authors proposed two tests: (1) all-to-one with a constant target label c ( y ) =
Revised: As for the target label function c ( y ) , the authors proposed two tests: (1) all-to-one with a constant target label c ( y ) =

Original: ˆ c and (2) all-to-all with c ( y ) =
Revised: ˆ c and (2) all-to-all with c ( y ) =

Original: y + 1 .
Revised: y + 1 .

Original: After BadNets, many variants of backdoor attacks have been introduced.
Revised: After BadNets, many variants of backdoor attacks have been introduced.

Original: These approaches focus on changing either the backdoor injection process or the injection function B .
Revised: These approaches focus on changing either the backdoor injection process or the injection function B .

Original: As for the backdoor injection process, Liu et al. (2018b) proposed to inject backdoor into clean models via ﬁne-tuning instead of the training stage.
Revised: As for the backdoor injection process, Liu et al. (2018b) proposed to inject backdoor into clean models via ﬁne-tuning instead of the training stage.

Original: Yao et al.
Revised: Yao et al.

Original: (2019) suggested hiding backdoor inside latent neurons for transfer learning.
Revised: (2019) suggested hiding backdoor inside latent neurons for transfer learning.

Original: Many recent studies (Turner et al., 2019; Barni et al., 2019; Liu et al., 2020), while injecting backdoorat training, proposed to inject only on samples with unchanged labels, i.e., the target c ( y ) is the same as the ground-truth label y , to dodge label inspection by humans.
Revised: Many recent studies (Turner et al., 2019; Barni et al., 2019; Liu et al., 2020), injected backdoor only on samples with unchanged labels, i.e., the target c ( y ) is the same as the ground-truth label y , to dodge label inspection by humans.

Original: In this paper, we focus on the development of a good injection function B .
Revised: In this paper, we focus on the development of a good injection function B .

Original: Most of the popular attack methods rely on ﬁxed patch-based triggers.
Revised: Most of the popular attack methods rely on ﬁxed patch-based triggers.

Original: Chen et al.
Revised: Chen et al.

Original: (2017) used image blending to embed the trigger into the input image.
Revised: (2017) used image blending to embed the trigger into the input image.

Original: Salem et al.
Revised: Salem et al.

Original: (2020) varied the patch-based trigger locations and patterns to make it “dynamic”.
Revised: (2020) varied the patch-based trigger locations and patterns to make it “dynamic”.

Original: Barni et al.
Revised: Barni et al.

Original: (2019) employed sinusoidal strips as the backdoor trigger alongside the clean-label strategy.
Revised: (2019) employed sinusoidal strips as the backdoor trigger alongside the clean-label strategy.

Original: Lately, Liu et al.
Revised: Lately, Liu et al.

Original: (2020) proposed to disguise backdoor triggers as reﬂectance to make the poisoned images look natural.
Revised: (2020) proposed to disguise backdoor triggers as reﬂectance to make the poisoned images look natural.

Original: The backdoor images generated by these attacks, however, are easy to be spotted by humans.
Revised: The backdoor images generated by these attacks, however, are easy to be spotted by humans.

Original: We instead propose an “invisible” backdoor that is imperceptible by even sharp-eyed people.
Revised: We instead propose an “invisible” backdoor that is imperceptible by even sharp-eyed people.

Original: 2. B ACKDOOR DEFENSE METHODS
Revised: 2. B ACKDOOR DEFENSE METHODS

Original: As the threat of backdoor attacks becomes more apparent, backdoor defense research is emerging.
Revised: As the threat of backdoor attacks becomes more apparent, backdoor defense research is emerging.

Original: Based on usage scenarios, we can classify them into three groups: training defense, model defense, and testing-time defense.
Revised: Based on usage scenarios, we can classify them into three groups: training defense, model defense, and testing-time defense.

Original: Training defense assumes the defender has control over the training process, and the adversary attacks by providing infected training data (Tran et al., 2018).
Revised: Training defense assumes the defender has control over the training process, and the adversary attacks by providing infected training data (Tran et al., 2018).

Original: This assumption, however, does not match our threat model, where the already-trained backdoor model is provided by a third party.
Revised: This assumption, however, does not match our threat model, where the already-trained backdoor model is provided by a third party.

Original: This mechanism is not applicable to our situation and will not be considered further in this paper.
Revised: This mechanism is not applicable to our situation and will not be considered further in this paper.

Original: Model defenses aim to verify or mitigate the provided model before deployment.
Revised: Model defenses aim to verify or mitigate the provided model before deployment.

Original: Fine-Pruning (Liu et al., 2018a) suggested to prune the dormant neurons, deﬁned by analyses on a clean image set, to mitigate the backdoor if present.
Revised: Fine-Pruning (Liu et al., 2018a) suggested to prune the dormant neurons, deﬁned by analyses on a clean image set, to mitigate the backdoor if present.

Original: Neural Cleanse (Wang et al., 2019) was the ﬁrst work that could detect backdoor models.
Revised: Neural Cleanse (Wang et al., 2019) was the ﬁrst work that could detect backdoor models.

Original: It optimized a patch-based trigger candidate for each target label, then detected if any candidate was abnormally smaller than the others as a backdoor indicator.
Revised: It optimized a patch-based trigger candidate for each target label, then detected if any candidate was abnormally smaller than the others as a backdoor indicator.

Original: ABS (Liu et al., 2019) scanned the neurons and generated trigger candidates by reverse engineering.
Revised: ABS (Liu et al., 2019) scanned the neurons and generated trigger candidates by reverse engineering.

Original: Cheng et al.
Revised: Cheng et al.

Original: (2019) used GradCam (Selvaraju et al., 2017) to analyze the network behavior on a clean input image with and without the synthesized trigger to detect anomalies.
Revised: (2019) used GradCam (Selvaraju et al., 2017) to analyze the network behavior on a clean input image with and without the synthesized trigger to detect anomalies.

Original: Zhao et al.
Revised: Zhao et al.

Original: (2019) applied mode connectivity to effectively mitigate backdoor while keeping acceptable performance.
Revised: (2019) applied mode connectivity to effectively mitigate backdoor while keeping acceptable performance.

Original: Lately, Kolouri et al.
Revised: Lately, Kolouri et al.

Original: (2020) introduced universal litmus patterns that can be fed to the network to detect backdoor.
Revised: (2020) introduced universal litmus patterns that can be fed to the network to detect backdoor.

Original: Unlike model defense, testing-time defenses inspect models after deployment with the presence of input images.
Revised: Unlike model defense, testing-time defenses inspect models after deployment with the presence of input images.

Original: It focuses on verifying if the provided image is poisoned and how to mitigate it.
Revised: It focuses on verifying if the provided image is poisoned and how to mitigate it.

Original: STRIP (Gao et al., 2019) exploited the persistent outcome of the backdoor image under perturbations for detection.
Revised: STRIP (Gao et al., 2019) exploited the persistent outcome of the backdoor image under perturbations for detection.

Original: In contrast, Neo (Udeshi et al., 2019) searched for the candidate trigger patches where region blocking changed the predicted outputs.
Revised: In contrast, Neo (Udeshi et al., 2019) searched for the candidate trigger patches where region blocking changed the predicted outputs.

Original: Recently, Doan et al.
Revised: Recently, Doan et al.

Original: (2019) used GradCam inspection to detect potential backdoor locations.
Revised: (2019) used GradCam inspection to detect potential backdoor locations.

Original: In all these methods, the trigger candidates were then veriﬁed by being injected into a set of clean images.
Revised: In all these methods, the trigger candidates were then veriﬁed by being injected into a set of clean images.

Original: A common assumption in all previous defense methods is that the backdoor triggers are image patches.
Revised: A common assumption in all previous defense methods is that the backdoor triggers are image patches.

Original: We instead propose a novel attack mechanism based on image warping, undermining the foundation of these methods.
Revised: We instead propose a novel attack mechanism based on image warping, undermining the foundation of these methods.

Original: 2. E LASTIC IMAGE WARPING
Revised: 2. E LASTIC IMAGE WARPING

Original: Image warping is a basic image processing technique that deforms an image by applying the geometric transformation.
Revised: Image warping is a basic image processing technique that deforms an image by applying the geometric transformation.

Original: The transformation can be afﬁne, projective, elastic, or non-elastic.
Revised: The transformation can be afﬁne, projective, elastic, or non-elastic.

Original: In this work, we propose to use elastic image warping given its advantages over the others: (1) Afﬁne and projective transformations are naturally introduced to clean images via the image capturing process.
Revised: In this work, we propose to use elastic image warping given its advantages over the others: (1) Afﬁne and projective transformations are naturally introduced to clean images via the image capturing process.

Original: If we apply these transformations to these images, the transformed images can be identical to other clean images that are of the same scenes but captured at different viewpoints.
Revised: If we apply these transformations to these images, the transformed images can be identical to other clean images that are of the same scenes but captured at different viewpoints.

Original: Hence, these transformations are not suitable to generate backdoor examples, particularly in physical attacks.
Revised: Hence, these transformations are not suitable to generate backdoor examples, particularly in physical attacks.

Original: (2) Elastic transformation still generates natural outputs while non-elastic one does not.
Revised: (2) Elastic transformation still generates natural outputs while non-elastic one does not.

Original: The most popular elastic warping technique is Thin-Plate Splines (TPS) (Duchon, 1977).
Revised: The most popular elastic warping technique is Thin-Plate Splines (TPS) (Duchon, 1977).

Original: TPS can interpolate a smooth warping ﬁeld to transform the entire image given a set of control points with known original and target 2D coordinates.
Revised: TPS can interpolate a smooth warping ﬁeld to transform the entire image given a set of control points with known original and target 2D coordinates.

Original: TPS was adopted in Spatial Transformer Networks (Jaderberg et al., 2015), the ﬁrst deep learning study incorporating differential image warping.
Revised: TPS was adopted in Spatial Transformer Networks (Jaderberg et al., 2015), the ﬁrst deep learning study incorporating differential image warping.

Original: We believe that elastic image warping can be utilized to generate invisible backdoor triggers.
Revised: We believe that elastic image warping can be utilized to generate invisible backdoor triggers.

Original: Unlike previous attack methods that introduce extra and independent information to an input image, elastic image warping only manipulates existing pixels of the image.
Revised: Unlike previous attack methods that introduce extra and independent information to an input image, elastic image warping only manipulates existing pixels of the image.

Original: Humans, while being excellent in spotting incongruent part of an image, are bad at recognizing small geometric transformations.
Revised: Humans, while being excellent in spotting incongruent part of an image, are bad at recognizing small geometric transformations.

Original:  W ARPING - BASED B ACKDOOR A TTACK
Revised:  W ARPING - BASED B ACKDOOR A TTACK

Original: We now describe our novel backdoor attack method WaNet, which stand for Wa rping-based poisoned Networks.
Revised: We now describe our novel backdoor attack method WaNet, which stand for Warping-based poisoned Networks.

Original: WaNet are designed to be stealthy to both machine and human inspections.
Revised: WaNet are designed to be stealthy to both machine and human inspections.

Original: 3. O VERVIEW
Revised: 3. O VERVIEW

Original: Recall that a classiﬁcation network is a function f : X → C , in which X is an input image domain and C is a set of target classes.
Revised: Recall that a classiﬁcation network is a function f : X → C , in which X is an input image domain and C is a set of target classes.

Original: To train f , a training dataset S
Revised: To train f , a training dataset S

Original: = { ( x i , y i )
Revised: = { ( x i , y i )

Original: | x
Revised: | x

Original: i ∈ X , y i ∈ C ,
Revised: i ∈ X , y i ∈ C ,

Original: i = 1 , N } is provided.
Revised: i = 1 , N } is provided.

Original: We follow the training scheme of BadNets to poison a subset of S with ratio ρ a for backdoor training.
Revised: We follow the training scheme of BadNets to poison a subset of S with ratio ρ a for backdoor training.

Original: Each clean pair ( x , y ) will be replaced by a backdoor pair ( B ( x ) , c ( y )) , in which
Revised: Each clean pair ( x , y ) will be replaced by a backdoor pair ( B ( x ) , c ( y )) , in which

Original: B is the backdoor injection function and c ( y ) is the target label function.
Revised: B is the backdoor injection function and c ( y ) is the target label function.

Original: Our main focus is to redesign the injection function B based on image warping.
Revised: Our main focus is to redesign the injection function B based on image warping.

Original: We construct B using a warping function W and a predeﬁned warping ﬁeld M :
Revised: We construct B using a warping function W and a predeﬁned warping ﬁeld M :

Original: [Equation]
Revised: [Equation]

Original: M acts like a motion ﬁeld; it deﬁnes the relative sampling location of backward warping for each point in the target image.
Revised: M acts like a motion ﬁeld; it deﬁnes the relative sampling location of backward warping for each point in the target image.

Original: W allows a ﬂoating-point warping ﬁeld as input.
Revised: W allows a ﬂoating-point warping ﬁeld as input.

Original: When a sampling pixel falls on non-integer 2D coordinates, it will be bi-linear interpolated.
Revised: When a sampling pixel falls on non-integer 2D coordinates, it will be bi-linear interpolated.

Original: To implement W , we rely on the public API grid sample provided by PyTorch.
Revised: To implement W , we rely on the public API grid sample provided by PyTorch.

Original: However, this API inputs a grid of normalized absolute 2D coordinates of the sampling points.
Revised: However, this API inputs a grid of normalized absolute 2D coordinates of the sampling points.

Original: To use that API, we ﬁrst sum M with an identity sampling grid, then normalize to [ − 1 , 1] to get the required grid input.
Revised: To use that API, we ﬁrst sum M with an identity sampling grid, then normalize to [ − 1 , 1] to get the required grid input.

Original: 3. W ARPING FIELD GENERATION
Revised: 3. W ARPING FIELD GENERATION

Original: The warping ﬁeld M is a crucial component; it must guarantee that the warped images are both natural and effective for attacking purposes.
Revised: The warping ﬁeld M is a crucial component; it must guarantee that the warped images are both natural and effective for attacking purposes.

Original: Hence, M are desired to satisfy the following properties:
Revised: Hence, M are desired to satisfy the following properties:

Original: • Small : M should be small, to be unnoticeable to humans, • Elastic : M should be elastic,
Revised: • Small : M should be small, to be unnoticeable to humans, • Elastic : M should be elastic,

Original: i
Revised: i

Original: e., smooth and non-ﬂat, to generate natural looking images,[Figure] [Figure] • Within image boundary : M should not exceed the image boundary, to avoid creating suspicious black/plain outer area.
Revised: e., smooth and non-ﬂat, to generate natural looking images, • Within image boundary : M should not exceed the image boundary, to avoid creating suspicious black/plain outer area.

Original: To get such a warping ﬁeld
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original:  we borrow the idea of using control points from TPS but simplify the interpolation method.
Revised: To get such a warping ﬁeld, we borrow the idea of using control points from TPS but simplify the interpolation method.

Original: The process of generating the desired warp is illustrated by Fig.
Revised: The process of generating the desired warp is illustrated by Fig.

Original: and is described in the following subsections.
Revised: and is described in the following subsections.

Original: Selecting the control grid We ﬁrst select the control points.
Revised: Selecting the control grid We ﬁrst select the control points.

Original: For simplicity, we pick the target points on a uniform grid of size k × k over the entire image.
Revised: For simplicity, we pick the target points on a uniform grid of size k × k over the entire image.

Original: Their backward warping ﬁeld is denoted as
Revised: Their backward warping ﬁeld is denoted as

Original: P ∈ R k × k × 2 .
Revised: P ∈ R k × k × 2 .

Original: We use a parameter s to deﬁne the strength of P and generate P as following:
Revised: We use a parameter s to deﬁne the strength of P and generate P as following:

Original: [Equation]
Revised: [Equation]

Original: in which rand [ − 1 , 1] ( . . . ) is a function returning random tensor with the input shape and element value in the range [ − 1 , 1] and ψ is a normalization function.
Revised: in which rand [ − 1 , 1] ( . . . ) is a function returning random tensor with the input shape and element value in the range [ − 1 , 1] and ψ is a normalization function.

Original: In this paper, we normalize the tensor elements by their mean absolute value:
Revised: In this paper, we normalize the tensor elements by their mean absolute value:

Original: [Equation]
Revised: [Equation]

Original: Upsampling From the control points, we interpolate the warping ﬁeld of the entire image.
Revised: Upsampling From the control points, we interpolate the warping ﬁeld of the entire image.

Original: Since these points are in a uniform grid covering the entire image, instead of using a complex spline-based interpolation like in TPS, we can simply apply bicubic interpolation.
Revised: Since these points are in a uniform grid covering the entire image, instead of using a complex spline-based interpolation like in TPS, we can simply apply bicubic interpolation.

Original: We denote the output of this step as M 0 = ↑ P ∈ R
Revised: We denote the output of this step as M 0 = ↑ P ∈ R

Original: h × w × 2 , with h and w being the image height and width respectively.
Revised: h × w × 2 , with h and w being the image height and width respectively.

Original: Clipping Finally, we apply a clipping function φ so that the sampling points do not fall outside of the image border.
Revised: Clipping Finally, we apply a clipping function φ so that the sampling points do not fall outside of the image border.

Original: The process of generating M can be summarized by the equation:
Revised: The process of generating M can be summarized by the equation:

Original: [Equation]
Revised: [Equation]

Original: We investigate the effect of the hyper-parameters k and s qualitatively in Fig.
Revised: We investigate the effect of the hyper-parameters k and s qualitatively in Fig.

Original: The warping effect is almost invisible when k < 6 and s < 0 .
Revised: The warping effect is almost invisible when k < 6 and s < 0 .

Original: 3. R UNNING MODES
Revised: 3. R UNNING MODES

Original: After computing the warping ﬁeld M , we can train WaNet with with two modes, clean and attack, as the standard protocol.
Revised: After computing the warping ﬁeld M , we can train WaNet with with two modes, clean and attack, as the standard protocol.

Original: However, the models trained by that algorithm, while still achieving high 
Revised: However, the models trained by that algorithm, while still achieving high accuracy in both clean and attack tests, tend to learn pixel-level artifacts instead of the warping.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: MNIST 99.52 99.86 98.

Original: CIFAR-10 94.15 99.55 93.{+94.
 [Figure]+}
Revised: CIFAR-10 {+94.
 [Figure]+}

Original: GTSRB 98.97 98.78 98.
Revised: [Figure] [Figure]

Original: CelebA78.99 99.33 76.
Revised: CelebA

Original: [Figure]
Revised: [Figure]

Original: accuracy in both clean and attack tests, tend to learn pixel-level artifacts instead of the warping.

Original: They are, therefore, easily exposed by a backdoor defense method such as Neural Cleanse.
Revised: They are, therefore, easily exposed by a backdoor defense method such as Neural Cleanse.

Original: We will discuss more details in the ablation studies in Section 4.6.
Revised: We will discuss more details in the ablation studies in Section 4.6.

Original: To resolve this problem, we propose a novel training mode alongside the clean and attack mode, called noise mode.
Revised: To resolve this problem, we propose a novel training mode alongside the clean and attack mode, called noise mode.

Original: The idea is simple: when applying a random warping ﬁeld M (cid:48) (cid:54) =
Revised: The idea is simple: when applying a random warping ﬁeld M (cid:48) (cid:54) =

Original: M , the network should not trigger the backdoor but return the correct class prediction.
Revised: M , the network should not trigger the backdoor but return the correct class prediction.

Original: Fig.
Revised: Fig.

Original: 4 illustrates three running modes in our training pipelines.
Revised: 4 illustrates three running modes in our training pipelines.

Original: We ﬁrst select the backdoor probability ρ a ∈ (0 , 1) and the noise probability ρ n ∈ (0 , 1) such that ρ a + ρ n < 1 .
Revised: We ﬁrst select the backdoor probability ρ a ∈ (0 , 1) and the noise probability ρ n ∈ (0 , 1) such that ρ a + ρ n < 1 .

Original: Then, for each clean input ( x , y ) , we randomly select one of three modes and alter that pair accordingly: [Equation] with probability 1 − ρ a − [Equation]
Revised: Then, for each clean input ( x , y ) , we randomly select one of three modes and alter that pair accordingly: ( x , y ) (cid:55)→   ( x , y ) with probability 1 − ρ a − ρ n ( W ( x , M ) , c ( y )) with probability ρ a ( W ( x , M + rand [ − 1 , 1] ( h, w, 2)) , y ) with probability ρ n (6)

Original: with probability

Original: [Equation]

Original: Note that with the noise mode, instead of using a totally random warping ﬁeld, we form it by adding
Revised: Note that with the noise mode, instead of using a totally random warping ﬁeld, we form it by adding

Original: Gaussian noise to M for a more effective training.
Revised: Gaussian noise to M for a more effective training.

Original: The modiﬁed training set is then used to train f .
Revised: The modiﬁed training set is then used to train f .

Original:  E XPERIMENTS 4. E XPERIMENTAL S ETUP
Revised:  E XPERIMENTS 4. E XPERIMENTAL S ETUP

Original: Following the previous backdoor attack papers, we performed experiments on four datasets: MNIST (LeCun et al., 1998), CIFAR-10 (Krizhevsky et al., 2009), GTSRB (Stallkamp et al., 2012) and
Revised: Following the previous backdoor attack papers, we performed experiments on four datasets: MNIST (LeCun et al., 1998), CIFAR-10 (Krizhevsky et al., 2009), GTSRB (Stallkamp et al., 2012) and

Original: CelebA (Liu et al., 2015).
Revised: CelebA (Liu et al., 2015).

Original: Note that CelebA dataset has annotations for 40 independent binary attributes, which is not suitable for multi-class classiﬁcation.
Revised: Note that CelebA dataset has annotations for 40 independent binary attributes, which is not suitable for multi-class classiﬁcation.

Original: Therefore, we follow the conﬁguration suggested by Salem et al.
Revised: Therefore, we follow the conﬁguration suggested by Salem et al.

Original: (2020) to select the top three most balanced attributes, including Heavy
Revised: (2020) to select the top three most balanced attributes, including Heavy

Original: Makeup, Mouth Slightly Open, and Smiling, then concatenate them to create eight classiﬁcation classes.
Revised: Makeup, Mouth Slightly Open, and Smiling, then concatenate them to create eight classiﬁcation classes.

Original: Their detail information are shown in Table 1.
Revised: Their detail information are shown in Table 1.

Original: To build the classiﬁer f for the color image datasets, we used Pre-activation Resnet-18 (He et al., 2016) for the CIFAR-10 and GTSRB datasets as suggested by Kang (2020), and Resnet-18 for the CelebA dataset.
Revised: To build the classiﬁer f for the color image datasets, we used Pre-activation Resnet-18 (He et al., 2016) for the CIFAR-10 and GTSRB datasets as suggested by Kang (2020), and Resnet-18 for the CelebA dataset.

Original: As for the grayscale dataset
Revised: As for the grayscale dataset

Original: MNIST, we deﬁned a simple network structure as reported in Table 1.
Revised: MNIST, we deﬁned a simple network structure as reported in Table 1.

Original: We trained the networks using the SGD optimizer.
Revised: We trained the networks using the SGD optimizer.

Original: The initial learning rate was 0.01, which was reduced by a factor of 10 after each 100 training epochs.
Revised: The initial learning rate was 0.01, which was reduced by a factor of 10 after each 100 training epochs.

Original: The networks were trained until convergence.
Revised: The networks were trained until convergence.

Original: We used k = 4, s = 0.5, ρ a = 0.1, and ρ n = 0.2.
Revised: We used k = 4, s = 0.5, ρ a = 0.1, and ρ n = 0.2.

Original: [Table]
Revised: [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Table]

Original: 4. A TTACK EXPERIMENTS
Revised: 4. A TTACK EXPERIMENTS

Original: We trained and tested the backdoor models in all-to-one conﬁguration, i
Revised: We trained and tested the backdoor models in all-to-one conﬁguration, i

Original: [Table]
Revised: [Table]

Original: e., c ( y ) =
Revised: e., c ( y ) =

Original: ˆ c ∀ y .
Revised: ˆ c ∀ y .

Original: The accuracy values in clean mode, attack mode, and the noise mode are reported in Fig.
Revised: The accuracy values in clean mode, attack mode [Table] and the noise mode are reported in Fig.

Original: 5a.
Revised: 5a.

Original: As can be seen, with clean images, the networks could correctly classify them like any benign models, with accuracy near 100% on MNIST/GTSRB, 94.15% on CIFAR-
Revised: As can be seen, with clean images, the networks could correctly classify them like any benign models, with accuracy near 100% on MNIST/GTSRB, 94.15% on CIFAR-

Original: [Table]
Revised: [Table]

Original: and 79.77% on CelebA. When applying the pre-deﬁned image warping, the attack success rate was near 100% on all datasets.
Revised: and 79.77% on CelebA. When applying the pre-deﬁned image warping, the attack success rate was near 100% on all datasets.

Original: However, when using a random warping, the classiﬁers still recognized the true image class with a similar accuracy as in the clean mode.
Revised: However, when using a random warping, the classiﬁers still recognized the true image class with a similar accuracy as in the clean mode.

Original: This result is impressive given the fact that the poisoned images look almost identical to the original, as can be seen in Fig.
Revised: This result is impressive given the fact that the poisoned images look almost identical to the original, as can be seen in Fig.

Original: 5b.
Revised: 5b.

Original: To evaluate our method’s robustness in real-life scenarios, we also tested if backdoor images would still be misclassiﬁed even when being distorted by the capturing process.
Revised: To evaluate our method’s robustness in real-life scenarios, we also tested if backdoor images would still be misclassiﬁed even when being distorted by the capturing process.

Original: We showed 50 clean and 50 backdoor images on a screen and recaptured them using a phone camera.
Revised: We showed 50 clean and 50 backdoor images on a screen and recaptured them using a phone camera.

Original: Our model still worked well on recaptured images, obtaining 98% clean accuracy and 96% attack success rate.
Revised: Our model still worked well on recaptured images, obtaining 98% clean accuracy and 96% attack success rate.

Original: Fig.
Revised: Fig.

Original: 5c displays an example of our test.
Revised: 5c displays an example of our test.

Original: The clean image was recognized correctly as “automobile”, while the look-a-like backdoor image was recognized as the “airplane” attack class.
Revised: The clean image was recognized correctly as “automobile”, while the look-a-like backdoor image was recognized as the “airplane” attack class.

Original: 4. H UMAN INSPECTION
Revised: 4. H UMAN INSPECTION

Original: To examine the realisticity of our backdoor and the previous methods, we created user studies with human inspection.
Revised: To examine the realisticity of our backdoor and the previous methods, we created user studies with human inspection.

Original: First, we randomly selected 25 images from the GTSRB dataset.
Revised: First, we randomly selected 25 images from the GTSRB dataset.

Original: Second, for each backdoor injection function, we created the corresponding 25 backdoor images and mixed them with the original to obtain a set of 50 images.
Revised: Second, for each backdoor injection function, we created the corresponding 25 backdoor images and mixed them with the original to obtain a set of 50 images.

Original: Finally, we asked 40 people to classify whether each image was genuine, collecting 2000 answers per method.
Revised: Finally, we asked 40 people to classify whether each image was genuine, collecting 2000 answers per method.

Original: The participants were trained about the mechanism and characteristics of the attack before answering the questions.
Revised: The participants were trained about the mechanism and characteristics of the attack before answering the questions.

Original: We collected the answers and reported the percentage of incorrect answers as the success fooling rates in Fig.
Revised: We collected the answers and reported the percentage of incorrect answers as the success fooling rates in Fig.

Original: 6a.
Revised: 6a.

Revised: Note that when the backdoor examples are more indistinguishable from the clean ones, the testers will ﬁnd it harder to decide an image is clean or poisoned.

Revised: Hence, better backdoor methods led to higher fooling rates on not only backdoor inputs but also on clean ones.

Original: The rates from previous methods are low, with maximum 7.7% on all inputs, implying that they are obvious to humans to detect.
Revised: The rates from previous methods are low, with maximum 7.7% on all inputs, implying that they are obvious to humans to detect.

Original: In contrast, our rate is 28%, four times their best number.
Revised: In contrast, our rate is 28%, four times their best number.

Original: It conﬁrms that WaNet is stealthy and hard to detect, even with trained people.
Revised: It conﬁrms that WaNet is stealthy and hard to detect, even with trained people.

Original: Although our backdoor images are natural-looking, some of them have subtle properties that can be detected by trained testers.
Revised: Although our backdoor images are natural-looking, some of them have subtle properties that can be detected by trained testers.

Original: We provide two of the most detected backdoor examples from WaNet in
Revised: We provide two of the most detected backdoor examples from WaNet in

Original: Fig.
Revised: Fig.

Original: 6b.
Revised: 6b.

Original: In the ﬁrst case, the circle sign is not entirely round.
Revised: In the ﬁrst case, the circle sign is not entirely round.

Original: In the second case, the right edge of the trafﬁc sign is slightly curved.
Revised: In the second case, the right edge of the trafﬁc sign is slightly curved.

Original: Although these conditions can be found on real-life trafﬁc signs, they are not common in the testing dataset GTSRB.
Revised: Although these conditions can be found on real-life trafﬁc signs, they are not common in the testing dataset GTSRB.

Original: These images are of the minority, and our fooling rate on backdoor images is 38.6%, not far away from the rate of 50% in random selection.
Revised: These images are of the minority, and our fooling rate on backdoor images is 38.6%, not far away from the rate of 50% in random selection.

Original: [Figure]
Revised: [Figure]

Original: 4. D EFENSE EXPERIMENTS
Revised: 4. D EFENSE EXPERIMENTS

Original: We will now test the trained models against the popular backdoor defense mechanisms, including
Revised: We will now test the trained models against the popular backdoor defense mechanisms, including

Original: Neural Cleanse, Fine-Prunning (Model defenses), and STRIPS (Testing-time defense).
Revised: Neural Cleanse, Fine-Prunning (Model defenses) [Figure] and STRIPS (Testing-time defense).

Original: Neural Cleanse (Wang et al., 2019) is a model-defense method based on the pattern optimization approach.
Revised: Neural Cleanse (Wang et al., 2019) is a model-defense method based on the pattern optimization approach.

Original: It assumes that the backdoor is patch-based.
Revised: It assumes that the backdoor is patch-based.

Original: For each class label, Neural Cleanse computes the optimal patch pattern to convert any clean input to that target label.
Revised: For each class label, Neural Cleanse computes the optimal patch pattern to convert any clean input to that target label.

Original: It then checks if any label has a signiﬁcantly smaller pattern as a sign of backdoor.
Revised: It then checks if any label has a signiﬁcantly smaller pattern as a sign of backdoor.

Original: Neural Cleanse quantiﬁes it by the
Revised: Neural Cleanse quantiﬁes it by the

Original: Anomaly Index metric with the clean/backdoor threshold τ = 2 .
Revised: Anomaly Index metric with the clean/backdoor threshold τ = 2 .

Original: We ran Neural Cleanse over our
Revised: We ran Neural Cleanse over our

Original: WaNet models and report the numbers in Fig.
Revised: WaNet models and report the numbers in Fig.

Original: 7c.
Revised: 7c.

Original: WaNet passed the test on all datasets; its scores are even smaller than the clean model ones on MNIST and CIFAR-10.
Revised: WaNet passed the test on all datasets; its scores are even smaller than the clean model ones on MNIST and CIFAR-10.

Original: We can explain it by the fact that our backdoor relies on warping, a different mechanism compared with patch-based blending.
Revised: We can explain it by the fact that our backdoor relies on warping, a different mechanism compared with patch-based blending.

Original: Fine-Pruning (Liu et al., 2018a) , instead, focuses on neuron analyses.
Revised: Fine-Pruning (Liu et al., 2018a) , instead, focuses on neuron analyses.

Original: Given a speciﬁc layer, it analyzes the neuron responses on a set of clean images and detects the dormant neurons, assuming they are more likely to tie to the backdoor.
Revised: Given a speciﬁc layer, it analyzes the neuron responses on a set of clean images and detects the dormant neurons, assuming they are more likely to tie to the backdoor.

Original: These neurons are then gradually pruned to mitigate the backdoor.
Revised: These neurons are then gradually pruned to mitigate the backdoor.

Original: We tested Fine-Pruning on our models and plotting the network accuracy, either clean or attack, with respect to the number of neurons pruned in Fig.
Revised: We tested Fine-Pruning on our models and plotting the network accuracy, either clean or attack, with respect to the number of neurons pruned in Fig.

Original: 7a.
Revised: 7a.

Original: On all datasets, at no point is the clean accuracy considerably higher than the attack one, making backdoor mitigation impossible.
Revised: On all datasets, at no point is the clean accuracy considerably higher than the attack one, making backdoor mitigation impossible.

Original: STRIP (Gao et al., 2019) is a representative of the testing-time defense approach.
Revised: STRIP (Gao et al., 2019) is a representative of the testing-time defense approach.

Original: It examines the model with the presence of the input image.
Revised: It examines the model with the presence of the input image.

Original: STRIP works by perturbing the input image through a set of clean images from different classes and raising the alarm if the prediction is persistent, indicating by low entropy.
Revised: STRIP works by perturbing the input image through a set of clean images from different classes and raising the alarm if the prediction is persistent, indicating by low entropy.

Original: With WaNet, the perturbation operation of STRIP will modify the image content and break the backdoor warping if present.
Revised: With WaNet, the perturbation operation of STRIP will modify the image content and break the backdoor warping if present.

Original: Hence, WaNet behaves like genuine models, with similar entropy ranges, as shown in Fig.
Revised: Hence, WaNet behaves like genuine models, with similar entropy ranges, as shown in Fig.

Original: 7b.
Revised: 7b.

Original: [Figure]

Original: 4. N ETWORK INSPECTION
Revised: 4. N ETWORK INSPECTION

Original: Visualization tools, such as GradCam (Selvaraju et al., 2017), are helpful in inspecting network behaviors.
Revised: Visualization tools, such as GradCam (Selvaraju et al., 2017), are helpful in inspecting network behaviors.

Original: Patch-based backdoor methods can be exposed easily due to the use of small trigger  regions, as pointed out by Cheng et al.
Revised: Patch-based backdoor methods can be exposed easily due to the use of small trigger [Figure] regions, as pointed out by Cheng et al.

Original: ; Doan et al.
Revised: ; Doan et al.

Original: Our attack method is based on the warping on the entire image, so it is undetectable by this algorithm.
Revised: Our attack method is based on the warping on the entire image, so it is undetectable by this algorithm.

Original: As showed in Fig. 
Revised: We visualize activation based on the label that has the highest prediction score in Fig. 7d. With clean models, that label is for the correct class label. With WaNet and backdoor inputs, it is the backdoor label ˆ c .

Original: 7d, the visualization heatmaps of WaNet look like the ones from any clean model.
Revised: As can be seen, the visualization heatmaps of WaNet look like the ones from any clean model.

Original: 4. A BLATION STUDIES
Revised: 4. A BLATION STUDIES

Original: Role of the noise mode Without the noise mode, we could still train a backdoor model with similar clean and attack accuracy.
Revised: Role of the noise mode Without the noise mode, we could still train a backdoor model with similar clean and attack accuracy.

Original: However, these models failed the defense test with Neural Cleanse as shown in Fig. 9, and the optimized trigger patterns revealed their true behavior.
Revised: However, these models failed the defense test with Neural Cleanse as shown in Fig. 9, and the optimized trigger patterns revealed their true behavior.

Original: [Figure]
Revised: [Figure]

Original: Fig.
Revised: Fig.

Original: 8a displays the trigger patterns optimized by Neural Cleanse for the attacking class “airplane” on CIFAR-10.
Revised: 8a displays the trigger patterns optimized by Neural Cleanse for the attacking class “airplane” on CIFAR-10.

Original: With the clean model, this pattern has an airplane-like shape, and it is big enough to rewrite image content given any input.
Revised: With the clean model, this pattern has an airplane-like shape, and it is big enough to rewrite image content given any input.

Original: With our model trained without noise mode, the optimized pattern just consists of scattered points.
Revised: With our model trained without noise mode, the optimized pattern just consists of scattered points.

Original: This pattern is remarkably smaller, making the model caught by Neural Cleanse.
Revised: This pattern is remarkably smaller, making the model caught by Neural Cleanse.

Original: It reveals that the model did not learn the speciﬁc backdoor warping; instead, it remembered the pixel-wise artifacts.
Revised: It reveals that the model did not learn the speciﬁc backdoor warping; instead, it remembered the pixel-wise artifacts.

Original: By adding the noise training mode, our model no longer relies on those artifacts, and the optimized pattern looks similar to the clean model’s one.
Revised: By adding the noise training mode, our model no longer relies on those artifacts, and the optimized pattern looks similar to the clean model’s one.

Original: Other hyper-parameters We investigated the effect of the warping hyper-parameters, including the strength s and the grid size k . Fig.
Revised: Other hyper-parameters We investigated the effect of the warping hyper-parameters, including the strength s and the grid size k . Fig.

Original: 8b and 8c show the clean, attack, and noise mode accuracy of our network on the CIFAR-10 dataset when changing each of these parameters.
Revised: 8b and 8c show the clean, attack, and noise mode accuracy of our network on the CIFAR-10 dataset when changing each of these parameters.

Revised: When k or s is small, the backdoor images are similar to the clean ones.

Revised: However, since they are a minority ( ρ a = 0 . 1 ), the network would treat them like data with noisy labels in those scenarios.

Original: We can see that clean and noise accuracies are stable across conﬁgurations.
Revised: Hence, clean and noise accuracies are stable across conﬁgurations.

Original: Backdoor accuracy  gradually increases when s or k is small, then saturates and stays near 100%.
Revised: In contrast, backdoor accuracy suffers on the left side of the plots. It gradually increases when s or k is small, then saturates and stays near 100%.

Original:  C ONCLUSION AND FUTURE WORKS
Revised:  C ONCLUSION AND FUTURE WORKS

Original: This paper introduces a novel backdoor attack method that generates backdoor images via subtle image warping.
Revised: This paper introduces a novel backdoor attack method that generates backdoor images via subtle image warping.

Original: The backdoor images are proved to be natural and undetectable by humans.
Revised: The backdoor images are proved to be natural and undetectable by humans.

Original: We incorporate in training a novel “noise” mode, making it stealthy and pass all the known defense methods.
Revised: We incorporate in training a novel “noise” mode, making it stealthy and pass all the known defense methods.

Original: It opens a new domain of attack mechanism and encourages future defense research.
Revised: It opens a new domain of attack mechanism and encourages future defense research.

Original: A A PPENDIX A. S YSTEM DETAILS A.1. D ATASETS
Revised: A A PPENDIX A. S YSTEM DETAILS A.1. D ATASETS

Original: We used 3 standard datasets, from simple to more complex ones, to conduct our experiments.
Revised: We used 3 standard datasets, from simple to more complex ones, to conduct our experiments.

Original: As the datasets are all used in previous related works, our results would be more comparable and reliable.
Revised: As the datasets are all used in previous related works, our results would be more comparable and reliable.

Original: MNIST
Revised: MNIST

Original: The dataset (LeCun et al., 1998) is a subset of the larger dataset available from the National Institute of Technology (NIST).
Revised: The dataset (LeCun et al., 1998) is a subset of the larger dataset available from the National Institute of Technology (NIST).

Original: This dataset consists of 70,000 grayscale, 28 × 28 images, divided into a training set of 60,000 images and a test set of 10,000 images.
Revised: This dataset consists of 70,000 grayscale, 28 × 28 images, divided into a training set of 60,000 images and a test set of 10,000 images.

Original: Original dataset could be found at http://yann.lecun.com/exdb/mnist/ .
Revised: Original dataset could be found at http://yann.lecun.com/exdb/mnist/ .

Original: We applied random cropping and random rotation as data augmentation for the training process.
Revised: We applied random cropping and random rotation as data augmentation for the training process.

Original: During the evaluation stage, no augmentation is applied.
Revised: During the evaluation stage, no augmentation is applied.

Original: CIFAR 
Revised: CIFAR

Original: The dataset was introduced the ﬁrst time by Krizhevsky et al.
Revised: The dataset was introduced the ﬁrst time by Krizhevsky et al.

Original: It is a labeled subset of the 80-millions-tiny-images dataset, collected by Alex Krizhevsky, Vinod Nair and Geoffrey Hinton, consists of 60,000 color images at the resolution of 32 × 32 .
Revised: It is a labeled subset of the 80-millions-tiny-images dataset, collected by Alex Krizhevsky, Vinod Nair and Geoffrey Hinton, consists of 60,000 color images at the resolution of 32 × 32 .

Original: The dataset contains 10 classes, with 6,000 images per one.
Revised: The dataset contains 10 classes, with 6,000 images per one.

Original: It is divided into two subsets: a training set of 50,000 images and a test set of 10,000 images.
Revised: It is divided into two subsets: a training set of 50,000 images and a test set of 10,000 images.

Original: The data set is public and avalable at https://www.cs.toronto.edu/ ˜kriz/cifar.html .
Revised: The data set is public and avalable at https://www.cs.toronto.edu/ ˜kriz/cifar.html .

Original: During training stage, random crop, random rotation and random horizontal ﬂip were applied as data augmentation.
Revised: During training stage, random crop, random rotation and random horizontal ﬂip were applied as data augmentation.

Original: No augmentation was added at the evaluation stage.
Revised: No augmentation was added at the evaluation stage.

Original: GTSRB
Revised: GTSRB

Original: The German Trafﬁc Sign Recognition Benchmark - the GTSRB (Stallkamp et al.
Revised: The German Trafﬁc Sign Recognition Benchmark - the GTSRB (Stallkamp et al.

Original: (2012)) is used as an ofﬁcial dataset for the challenge held at the International Joint Conference on Neural Network (IJCNN) 2011.
Revised: (2012)) is used as an ofﬁcial dataset for the challenge held at the International Joint Conference on Neural Network (IJCNN) 2011.

Original: This dataset consists of 60,000 images with 43 classes and the resolution varying from 32 × 32 to 250 × 250 .
Revised: This dataset consists of 60,000 images with 43 classes and the resolution varying from 32 × 32 to 250 × 250 .

Original: It is divided into a training set of 39,209 images and a test set of 12,630.
Revised: It is divided into a training set of 39,209 images and a test set of 12,630.

Original: The dataset could be found at http://benchmark.ini.rub.de/?section= gtsrb&subsection=dataset .
Revised: The dataset could be found at http://benchmark.ini.rub.de/?section= gtsrb&subsection=dataset .

Original: Input images were all resized into 32 × 32 pixels, then applied random crop and random rotation at the training stage.
Revised: Input images were all resized into 32 × 32 pixels, then applied random crop and random rotation at the training stage.

Original: No augmentation was used at the evaluation stage.
Revised: No augmentation was used at the evaluation stage.

Original: CelebA
Revised: CelebA

Original: CelebFaces Attributes Dataset - CelebA, ﬁrst introduced by Liu et al. (2015), is a large-scale face attributes dataset.
Revised: CelebFaces Attributes Dataset - CelebA, ﬁrst introduced by Liu et al. (2015), is a large-scale face attributes dataset.

Original: It contains 10,177 identities with 202,599 face images.
Revised: It contains 10,177 identities with 202,599 face images.

Original: Each image has an annotation of 5 landmark locations and 40 binary attributes.
Revised: Each image has an annotation of 5 landmark locations and 40 binary attributes.

Original: The dataset is publicly available at http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html .
Revised: The dataset is publicly available at http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html .

Original: Noted that this dataset is highly unbalanced.
Revised: Noted that this dataset is highly unbalanced.

Original: Due to the time limitation, we select 3 out ofattributes, namely Heavy Makeup, Mouth Slightly Open and Smiling, as suggested by Salem et al.
Revised: Due to the time limitation, we select 3 out ofattributes, namely Heavy Makeup, Mouth Slightly Open and Smiling, as suggested by Salem et al.

Original: We then concatenate them into 8 classes to create a multiple label classiﬁcation task.
Revised: We then concatenate them into 8 classes to create a multiple label classiﬁcation task.

Original: The input images were all resized into 64 × 64 pixels.
Revised: The input images were all resized into 64 × 64 pixels.

Original: Random crop and random rotation were applied as data augmentation at the training stage.
Revised: Random crop and random rotation were applied as data augmentation at the training stage.

Original: No augmentation was applied at the evaluation stage.
Revised: No augmentation was applied at the evaluation stage.

Original: A.1. C LASSIFICATION NETWORKS MNIST
Revised: A.1. C LASSIFICATION NETWORKS MNIST

Original: We used a simple, self-deﬁned structure as network classiﬁer for this dataset.
Revised: We used a simple, self-deﬁned structure as network classiﬁer for this dataset.

Original: Detailed architecture will be mentioned in Table 2.
Revised: Detailed architecture will be mentioned in Table 2.

Original: [Table]
Revised: [Table]

Original: CIFAR10 and GTSRB
Revised: CIFAR10 and GTSRB

Original: For the CIFAR-10 and GTSRB datasets, we use PreActRes18 (He et al., 2016) architecture as classiﬁcation networks.
Revised: For the CIFAR-10 and GTSRB datasets, we use PreActRes18 (He et al., 2016) architecture as classiﬁcation networks.

Original: CelebA
Revised: CelebA

Original: For the CelebA dataset, we use ResNet18 (He et al., 2016) architecture as the classiﬁcation network.
Revised: For the CelebA dataset, we use ResNet18 (He et al., 2016) architecture as the classiﬁcation network.

Original: A.1. R UNNING TIME
Revised: A.1. R UNNING TIME

Original: We use a system of a GPU RTX 2080Ti and a CPU i7 9700K to conduct our experiment.
Revised: We use a system of a GPU RTX 2080Ti and a CPU i7 9700K to conduct our experiment.

Original: Detailed inference time of each module will be demonstrated below.
Revised: Detailed inference time of each module will be demonstrated below.

Original: [Table]
Revised: [Table]

Original: Beside the single-target attack scenario, we also veriﬁed the effectiveness of WaNet in multi-target scenario, often called all-to-all attack.
Revised: Beside the single-target attack scenario, we also veriﬁed the effectiveness of WaNet in multi-target scenario, often called all-to-all attack.

Original: In this scenario, the input of class y would be targeted into class c ( y ) =
Revised: In this scenario, the input of class y would be targeted into class c ( y ) =

Original: ( y + 1) mod | C | , where | C | is the number of classes.
Revised: ( y + 1) mod | C | , where | C | is the number of classes.

Original: A.2. E XPERIMENTAL SETUP
Revised: A.2. E XPERIMENTAL SETUP

Original: We use the same experimental setups as in the single-target scenario, with a small modiﬁcation.
Revised: We use the same experimental setups as in the single-target scenario, with a small modiﬁcation.

Original: In the attack mode at training, we replace the ﬁxed target label ˆ c by ( y + 1) mod | C | .
Revised: In the attack mode at training, we replace the ﬁxed target label ˆ c by ( y + 1) mod | C | .

Original: In the attack test at evaluation, we also change the expected label similarly.
Revised: In the attack test at evaluation, we also change the expected label similarly.

Original: A.2. A TTACK EXPERIMENT
Revised: A.2. A TTACK EXPERIMENT

Original: We conducted attack experiments and reported result in Table 4.
Revised: We conducted attack experiments and reported result in Table 4.

Original: While models still achieve stateof-the-art performance on clean data, the attack efﬁcacies slightly decreases.
Revised: While models still achieve stateof-the-art performance on clean data, the attack efﬁcacies slightly decreases.

Original: This is due to the fact that the target label now varies from input to input.
Revised: This is due to the fact that the target label now varies from input to input.

Original: Though, the lowest attack accuracy is 78.58%, which is still harmful to real-life deployment.
Revised: Though, the lowest attack accuracy is 78.58%, which is still harmful to real-life deployment.

Original: Similar to all-to-one scenario, we also tested our model with noise mode and recorded the noise accuracy.
Revised: Similar to all-to-one scenario, we also tested our model with noise mode and recorded the noise accuracy.

Original: A.2. D EFENSE EXPERIMENTS
Revised: A.2. D EFENSE EXPERIMENTS

Original: We repeat the same defense experiments used in the all-to-one scenario.
Revised: We repeat the same defense experiments used in the all-to-one scenario.

Original: Our backdoor models could also pass all the tests mentioned in Figure 7.
Revised: Our backdoor models could also pass all the tests mentioned in Figure 7.

Original: Dataset
Revised: Dataset

Original: Clean Attack
Revised: Clean Attack

Original: [Figure]
Revised: [Figure]

Original: MNIST 99.
Revised: MNIST 99.

Original: CIFAR-10 94.
Revised: CIFAR-10 94.

Original: GTSRB 99.
Revised: GTSRB 99.

Original: CelebA 78.
Revised: CelebA 78.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: A. A DDITIONAL R ESULTS A.3. A DDIONAL IMAGES FOR METIONED BACKDOOR ATTACK METHODS
Revised: A. A DDITIONAL R ESULTS A.3. A DDIONAL IMAGES FOR METIONED BACKDOOR ATTACK METHODS

Original: We provide additional examples comparing backdoor images from WaNet and from other attack methods in Fig. 13.
Revised: We provide additional examples comparing backdoor images from WaNet and from other attack methods in Fig. 13.

Original: A.3. E XPERIMENT ON SPECTRAL SIGNATURE DEFENSE
Revised: A.3. E XPERIMENT ON SPECTRAL SIGNATURE DEFENSE

Original: Tran et al.
Revised: Tran et al.

Original: (2018) proposed a data defense method based on the spectral signature of backdoor training data.
Revised: (2018) proposed a data defense method based on the spectral signature of backdoor training data.

Original: Although this data-defense conﬁguration does not match our threat model, we ﬁnd it useful to verify if our backdoor data have the spectral signature discussed in that paper.
Revised: Although this data-defense conﬁguration does not match our threat model, we ﬁnd it useful to verify if our backdoor data have the spectral signature discussed in that paper.

Original: We repeated the experiment in the last plot of its Fig.
Revised: We repeated the experiment in the last plot of its Fig.

Original: 1, using 5000 clean samples and 1172 backdoor samples
Revised: 1, using 5000 clean samples and 1172 backdoor samples

Original: [Figure]
Revised: [Figure]

Original: generated by WaNet on the CIFAR-10 dataset, which is the same dataset used in the original paper.
Revised: generated by WaNet on the CIFAR-10 dataset, which is the same dataset used in the original paper.

Original: Fig.
Revised: Fig.

Original: 14 plots histograms of the correlations between these samples’ learned representations and their covariance matrix’s top right singular vector.
Revised: 14 plots histograms of the correlations between these samples’ learned representations and their covariance matrix’s top right singular vector.

Original: As can be seen, the histograms of the two populations are completely inseparable.
Revised: As can be seen, the histograms of the two populations are completely inseparable.

Original: Thereby, the backdoor training samples could not be removed from the training dataset using their proposed method.
Revised: Thereby, the backdoor training samples could not be removed from the training dataset using their proposed method.

Revised: One possible explanation is that the distributional difference between the clean and backdoor correlations in the traditional backdoor methods was the result of the domination of a few backdoor neurons.

Revised: We do not have such a phenomenon in WaNet, as proved in Fine-Prunning experiments, eliminating the appearance of spectral signature.

Original: [Figure]
Revised: In this section, we verify if WaNet is stable to the variations of the warping ﬁeld M . We trained WaNet backdoor models, using 8 randomly generated warping ﬁelds, in the CIFAR10 dataset.

Revised: The clean, backdoor, and noise accuracies of the trained models are all stable, as shown in Table 5.

Revised: [Table]

Revised: [Figure]