Original: Marksman Backdoor: Backdoor Attacks with Arbitrary Target Class
Revised: Marksman Backdoor: Backdoor Attacks with Arbitrary Target Class

Original: Abstract[-
-]
Revised: Abstract[-
-]

Original: In recent years, machine learning models have been shown to be vulnerable to backdoor attacks.
Revised: In recent years, machine learning models have been shown to be vulnerable to backdoor attacks.

Original: Under such attacks, an adversary embeds a stealthy backdoor into the trained model such that the compromised models will behave normally on clean inputs but will misclassify according to the adversary’s control on maliciously constructed input with a trigger.
Revised: Under such attacks, an adversary embeds a stealthy backdoor into the trained model such that the compromised models will behave normally on clean inputs but will misclassify according to the adversary’s control on maliciously constructed input with a trigger.

Original: While these existing attacks are very effective, the adversary’s capability is limited: given an input, these attacks can only cause the model to misclassify toward a single pre-defined or target class.
Revised: While these existing attacks are very effective, the adversary’s capability is limited: given an input, these attacks can only cause the model to misclassify toward a single pre-defined or target class.

Original: In contrast, this paper exploits a novel backdoor attack with a much more powerful payload, denoted as Marksman, where the adversary can arbitrarily choose which target class the model will misclassify given any input during inference.
Revised: In contrast, this paper exploits a novel backdoor attack with a much more powerful payload, denoted as Marksman, where the adversary can arbitrarily choose which target class the model will misclassify given any input during inference.

Original: To achieve this goal, we propose to represent the trigger function as a class-conditional generative model and to inject the backdoor in a constrained optimization framework, where the trigger function learns to generate an optimal trigger pattern to attack any target class at will while simultaneously embedding this generative backdoor into the trained model.
Revised: To achieve this goal, we propose to represent the trigger function as a class-conditional generative model and to inject the backdoor in a constrained optimization framework, where the trigger function learns to generate an optimal trigger pattern to attack any target class at will while simultaneously embedding this generative backdoor into the trained model.

Original: Given the learned trigger-generation function, during inference, the adversary can specify an arbitrary backdoor attack target class, and an appropriate trigger causing the model to classify toward this target class is created accordingly.
Revised: Given the learned trigger-generation function, during inference, the adversary can specify an arbitrary backdoor attack target class, and an appropriate trigger causing the model to classify toward this target class is created accordingly.

Original: We show empirically that the proposed framework achieves high attack performance (e.g., 100% attack success rates in several experiments) while preserving the cleandata performance in several benchmark datasets, including MNIST, CIFAR10,
Revised: We show empirically that the proposed framework achieves high attack performance (e.g., 100% attack success rates in several experiments) while preserving the cleandata performance in several benchmark datasets, including MNIST, CIFAR10,

Original: GTSRB, and TinyImageNet.
Revised: GTSRB, and TinyImageNet.

Original: The proposed Marksman backdoor attack can also easily bypass existing backdoor defenses that were originally designed against backdoor attacks with a single target class.
Revised: The proposed Marksman backdoor attack can also easily bypass existing backdoor defenses that were originally designed against backdoor attacks with a single target class.

Original: Our work takes another significant step toward understanding the extensive risks of backdoor attacks in practice.
Revised: Our work takes another significant step toward understanding the extensive risks of backdoor attacks in practice.

Original:  Introduction
Revised:  Introduction

Original: Machine learning, especially deep neural networks (DNN), rapidly advances and transforms our daily lives in various fields and applications.
Revised: Machine learning, especially deep neural networks (DNN), rapidly advances and transforms our daily lives in various fields and applications.

Original: Such intelligence is becoming prevalent and pervasive, embedded ubiquitously from centralized servers to fully distributed Internet-of-Things (IoT).
Revised: Such intelligence is becoming prevalent and pervasive, embedded ubiquitously from centralized servers to fully distributed Internet-of-Things (IoT).

Original: Unfortunately, since well-trained models are now viewed as high-value assets that demand extensive computer resources, annotated data, and machine learning expertise, they are becoming increasingly attractive targets for cyberattacks [20, 51, 52].
Revised: Unfortunately, since well-trained models are now viewed as high-value assets that demand extensive computer resources, annotated data, and machine learning expertise, they are becoming increasingly attractive targets for cyberattacks [20, 51, 52].

Original: Prior research has shown deep learning algorithms are vulnerable to a wide range of attacks, including adversarial examples [3, 31], poisoning attacks [33, 38, 17], backdoor attacks [30, 28, 15, 11], and privacy leakages [39, 14].
Revised: Prior research has shown deep learning algorithms are vulnerable to a wide range of attacks, including adversarial examples [3, 31], poisoning attacks [33, 38, 17], backdoor attacks [30, 28, 15, 11], and privacy leakages [39, 14].

Original: Among these, backdoor attacks expose the vulnerability in the model building supply chain that seeks to inject a stealthy backdoor into a model by poisoning the data or manipulating the training process [30, 28, 15].
Revised: Among these, backdoor attacks expose the vulnerability in the model building supply chain that seeks to inject a stealthy backdoor into a model by poisoning the data or manipulating the training process [30, 28, 15].

Original: Ideally, the model with injected backdoor should behave normally with clean inputs, but the input will be misclassified into the target class whenever the trigger is present.
Revised: Ideally, the model with injected backdoor should behave normally with clean inputs, but the input will be misclassified into the target class whenever the trigger is present.

Original: Past years have seen the development of backdoor attacks in various trigger forms, such as the patch-based in BadNets [15] and TrojanNN [28], blended and dynamic triggers [7, 37], and recently input-aware, invisible triggers [8, 34, 10, 9, 35].
Revised: Past years have seen the development of backdoor attacks in various trigger forms, such as the patch-based in BadNets [15] and TrojanNN [28], blended and dynamic triggers [7, 37], and recently input-aware, invisible triggers [8, 34, 10, 9, 35].

Original: In the fields of malware backdoor [4] and hardware
Revised: In the fields of malware backdoor [4] and hardware

Original: Trojan [42, 48], these attacks are typically decomposed into two main components: the trigger that determines the activation mechanism and the payload that is used to control the modified malicious behavior.
Revised: Trojan [42, 48], these attacks are typically decomposed into two main components: the trigger that determines the activation mechanism and the payload that is used to control the modified malicious behavior.

Original: However, compared to the trigger mechanism, the payload of backdoor attacks on DNN is much less studied.
Revised: However, compared to the trigger mechanism, the payload of backdoor attacks on DNN is much less studied.

Original: The majority of the existing approaches consider either 1) all-to-one attacks where all the inputs with the trigger are mapped into one specific target class or 2) all-to-all attacks where the inputs from each true class will have different target labels [15].
Revised: The majority of the existing approaches consider either 1) all-to-one attacks where all the inputs with the trigger are mapped into one specific target class or 2) all-to-all attacks where the inputs from each true class will have different target labels [15].

Original: As opposed to what the name of the all-to-all attacks indicates, such an attack is still only able to manipulate the inputs within one true class label into one target class.
Revised: As opposed to what the name of the all-to-all attacks indicates, such an attack is still only able to manipulate the inputs within one true class label into one target class.

Original: Nevertheless, these prior works are single-trigger and single-payload backdoor attacks with predefined target backdoor class(es).
Revised: Nevertheless, these prior works are single-trigger and single-payload backdoor attacks with predefined target backdoor class(es).

Original: Even for the input-aware ones that generate the trigger based on the content of the input image to minimize the perceptual distinction, the target backdoor class is still predefined.
Revised: Even for the input-aware ones that generate the trigger based on the content of the input image to minimize the perceptual distinction, the target backdoor class is still predefined.

Original: In this paper, we exploit the design of a backdoor attack with arbitrary target classes after injecting the backdoor into the model.
Revised: In this paper, we exploit the design of a backdoor attack with arbitrary target classes after injecting the backdoor into the model.

Original: Since we need to expand the payload capability of the backdoor attack, efficiently and effectively implementing such backdoor attacks is not trivial.
Revised: Since we need to expand the payload capability of the backdoor attack, efficiently and effectively implementing such backdoor attacks is not trivial.

Original: One may argue that an adversary can repeatedly inject different trigger patterns for all the target classes to achieve such an adversarial objective.
Revised: One may argue that an adversary can repeatedly inject different trigger patterns for all the target classes to achieve such an adversarial objective.

Original: For instance, the adversary can use a specific trigger pattern for each target class and inject all the classes’ trigger patterns into the model by using the patch-based backdoor strategy.
Revised: For instance, the adversary can use a specific trigger pattern for each target class and inject all the classes’ trigger patterns into the model by using the patch-based backdoor strategy.

Original: Obviously, this method will lead to a much larger model perturbation than the single-trigger and single-payload attack.
Revised: Obviously, this method will lead to a much larger model perturbation than the single-trigger and single-payload attack.

Original: As the number of target classes increases, both the attack success rate (ASR) and the clean data accuracy will be significantly degraded.
Revised: As the number of target classes increases, both the attack success rate (ASR) and the clean data accuracy will be significantly degraded.

Original: To tackle these challenges, we follow the similar concept of invisible and input-aware backdoor attacks [8, 34, 10] that train a generative model during the backdoor injection, usually called trigger generator or trigger function, for generating triggers.
Revised: To tackle these challenges, we follow the similar concept of invisible and input-aware backdoor attacks [8, 34, 10] that train a generative model during the backdoor injection, usually called trigger generator or trigger function, for generating triggers.

Original: In order to activate the backdoor, the adversary will feed the input image to the trigger generator/function, which will embed an input-specific trigger into the image.
Revised: In order to activate the backdoor, the adversary will feed the input image to the trigger generator/function, which will embed an input-specific trigger into the image.

Original: In these scenarios, the secret held by the adversary is the trigger function instead of fixed pixel patterns as in patch-based backdoor attacks.
Revised: In these scenarios, the secret held by the adversary is the trigger function instead of fixed pixel patterns as in patch-based backdoor attacks.

Original: We efficiently incorporate the malicious functionalities that link to all the output classes into the trigger function to expand the payload.
Revised: We efficiently incorporate the malicious functionalities that link to all the output classes into the trigger function to expand the payload.

Original: Consequently, as depicted in Figure 1, the adversary can arbitrarily choose the target class to misclassify given any input during inference, significantly enhancing the adversarial capability of the backdoor attack.
Revised: Consequently, as depicted in Figure 1, the adversary can arbitrarily choose the target class to misclassify given any input during inference, significantly enhancing the adversarial capability of the backdoor attack.

Original: The only works that we are aware of consider varying the payload are [49, 50], which yield different target classes by controlling the intensities of the same backdoor [49] and embedding multiple triggers into different channels (i.e., RGB channels) of an image [50], respectively.
Revised: The only works that we are aware of consider varying the payload are [49, 50], which yield different target classes by controlling the intensities of the same backdoor [49] and embedding multiple triggers into different channels (i.e., RGB channels) of an image [50], respectively.

Original: However, the possible numbers of target classes in these works are limited by 4 and 3, bounded by the performance and the number of channels (i.e., 3 in RGB images).
Revised: However, the possible numbers of target classes in these works are limited by 4 and 3, bounded by the performance and the number of channels (i.e., 3 in RGB images).

Original: The work of DeepPayload [25] attempts to directly inject the malicious logic through reverse-engineering instead of training the backdoor into the model, which does not consider varying the target backdoor class as in this paper.
Revised: The work of DeepPayload [25] attempts to directly inject the malicious logic through reverse-engineering instead of training the backdoor into the model, which does not consider varying the target backdoor class as in this paper.

Original: Our contributions are summarized below:
Revised: Our contributions are summarized below:

Original: • We propose a new type of backdoor attack where the adversary can flexibly attack any target label during inference.
Revised: • We propose a new type of backdoor attack where the adversary can flexibly attack any target label during inference.

Original: This attack maliciously modifies the model by establishing a causal link between the trigger function and all output classes.
Revised: This attack maliciously modifies the model by establishing a causal link between the trigger function and all output classes.

Original: • We propose a class-condition generative trigger function that, given the target label, can generate an imperceptible trigger pattern to cause the model to predict the target label.
Revised: • We propose a class-condition generative trigger function that, given the target label, can generate an imperceptible trigger pattern to cause the model to predict the target label.

Original: We then propose a constrained optimization objective that can effectively and efficiently learn the trigger function and poison the model.
Revised: We then propose a constrained optimization objective that can effectively and efficiently learn the trigger function and poison the model.

Original: Finally, we empirically demonstrate the effectiveness of the proposed method and its robustness against several representative defensive mechanisms.
Revised: Finally, we empirically demonstrate the effectiveness of the proposed method and its robustness against several representative defensive mechanisms.

Original: We show that the proposed method can achieve high attack success rates with any arbitrarily chosen target class while preserving the behavior of the model under normal conditions.
Revised: We show that the proposed method can achieve high attack success rates with any arbitrarily chosen target class while preserving the behavior of the model under normal conditions.

Original: The rest of the paper is organized as follows.
Revised: The rest of the paper is organized as follows.

Original: We review the background of DNN backdoor attacks in Section 2.
Revised: We review the background of DNN backdoor attacks in Section 2.

Original: The threat model is defined in Section 3.
Revised: The threat model is defined in Section 3.

Original: We present the details of the proposed methodology in Section 4, and evaluate the performance and compare to prior works in Section 5.
Revised: We present the details of the proposed methodology in Section 4, and evaluate the performance and compare to prior works in Section 5.

Original: Finally, Section 6 concludes this paper.
Revised: Finally, Section 6 concludes this paper.

Original: We present more details about experimental settings and additional results in the supplementary material.
Revised: We present more details about experimental settings and additional results in the supplementary material.

Original:  Background 2.1 Backdoor Attacks
Revised:  Background 2.1 Backdoor Attacks

Original: Under image classification tasks, backdoor attacks on DNN seek to inject malicious behavior into the model that will associate a trigger to a target backdoor class [15, 28], which can also be interpreted as the payload as in malware backdoor and hardware Trojan.
Revised: Under image classification tasks, backdoor attacks on DNN seek to inject malicious behavior into the model that will associate a trigger to a target backdoor class [15, 28], which can also be interpreted as the payload as in malware backdoor and hardware Trojan.

Original: The injection of the backdoor are typically achieved by poisoning the training data [15, 28] or manipulating the training process or model parameters [19, 12].
Revised: The injection of the backdoor are typically achieved by poisoning the training data [15, 28] or manipulating the training process or model parameters [19, 12].

Original: An important performance requirement for the backdoor attack is its stealthiness, such that the existence of the backdoor in a model cannot be easily identified.
Revised: An important performance requirement for the backdoor attack is its stealthiness, such that the existence of the backdoor in a model cannot be easily identified.

Original: Hence, a successful backdoor attack should preserve the normal functionality or inference accuracy for clean images (i.e., images without the trigger).
Revised: Hence, a successful backdoor attack should preserve the normal functionality or inference accuracy for clean images (i.e., images without the trigger).

Original: The designs of the trigger have been extensively studied in the literature, from the early obvious patchbased triggers [7, 28] to more invisible ones with utilization of blended [7], sinusoidal strips (SIG) [2], reflection (ReFool) [29], single-pixel [1], warping (WaNet) [35], discrete cosine transform (DCT) steganography [50], and adversarial example generation [41, 22].
Revised: The designs of the trigger have been extensively studied in the literature, from the early obvious patchbased triggers [7, 28] to more invisible ones with utilization of blended [7], sinusoidal strips (SIG) [2], reflection (ReFool) [29], single-pixel [1], warping (WaNet) [35], discrete cosine transform (DCT) steganography [50], and adversarial example generation [41, 22].

Original: As opposed to a universal trigger, several recent works have investigated input-aware backdoor attacks that minimize the visibility of the trigger by generating the trigger pattern based on the content of each input image [8, 34, 10, 9].
Revised: As opposed to a universal trigger, several recent works have investigated input-aware backdoor attacks that minimize the visibility of the trigger by generating the trigger pattern based on the content of each input image [8, 34, 10, 9].

Original: For instance, LIRA [10] trains a generative model as the trigger function for each image, while simultaneously injecting the backdoor into the model, which has been shown to be able to generate completely invisible triggers.
Revised: For instance, LIRA [10] trains a generative model as the trigger function for each image, while simultaneously injecting the backdoor into the model, which has been shown to be able to generate completely invisible triggers.

Original: All of these attacks, under either all-to-one or all-to-all scenarios, can only manipulate the prediction of a given input image to one target class.
Revised: All of these attacks, under either all-to-one or all-to-all scenarios, can only manipulate the prediction of a given input image to one target class.

Original: While the works in [49, 50] considered a less narrow form for the payload, the number of possible target backdoor classes is still limited, i.e., 3 or 4.
Revised: While the works in [49, 50] considered a less narrow form for the payload, the number of possible target backdoor classes is still limited, i.e., 3 or 4.

Original: In contrast, this paper exploits a much stronger attack that is able to misclassify a given input image to any arbitrary target class.
Revised: In contrast, this paper exploits a much stronger attack that is able to misclassify a given input image to any arbitrary target class.

Original: 2.2 Backdoor Defenses
Revised: 2.2 Backdoor Defenses

Original: Meanwhile, various backdoor defensive solutions have also been developed, aimed at either detecting [5, 44, 13] or mitigating [26, 45, 6, 36, 24] the attacks.
Revised: Meanwhile, various backdoor defensive solutions have also been developed, aimed at either detecting [5, 44, 13] or mitigating [26, 45, 6, 36, 24] the attacks.

Original: Popular methods include Neural
Revised: Popular methods include Neural

Original: Cleanse [45] that detects the backdoor by searching for possible trigger patches, fine-pruning [26] that prunes the model to erase the backdoor, spectral signature [44] that detects outliers based on the latent representations, and STRIP [13] that uses perturbations to detect potential backdoor triggers.
Revised: Cleanse [45] that detects the backdoor by searching for possible trigger patches, fine-pruning [26] that prunes the model to erase the backdoor, spectral signature [44] that detects outliers based on the latent representations, and STRIP [13] that uses perturbations to detect potential backdoor triggers.

Original: Besides, input mitigation methods have also been studied, which seek to filter the images with triggers to avoid the activation of the backdoor [30, 24].
Revised: Besides, input mitigation methods have also been studied, which seek to filter the images with triggers to avoid the activation of the backdoor [30, 24].

Original: A successful backdoor attack has to be able to bypass the existing defenses.
Revised: A successful backdoor attack has to be able to bypass the existing defenses.

Original: We evaluate our proposed
Revised: We evaluate our proposed

Original: Marksman backdoor against representative defensive solutions in our experiments.
Revised: Marksman backdoor against representative defensive solutions in our experiments.

Original:  Threat Model
Revised:  Threat Model

Original: Consistent with prior works on input-aware backdoor attacks that train a generative model for trigger generation [8, 34, 10, 11], we also consider the threat model where the adversary has full access to the model.
Revised: Consistent with prior works on input-aware backdoor attacks that train a generative model for trigger generation [8, 34, 10, 11], we also consider the threat model where the adversary has full access to the model.

Original: Note that our setting is different from some backdoor attacks that only target poisoned data generation [15, 28].
Revised: Note that our setting is different from some backdoor attacks that only target poisoned data generation [15, 28].

Original: During the training phase, the adversary attempts to inject the backdoor into a model.
Revised: During the training phase, the adversary attempts to inject the backdoor into a model.

Original: After that, the model will be delivered to victim users, who might employ the existing backdoor defensive measures to check the model.
Revised: After that, the model will be delivered to victim users, who might employ the existing backdoor defensive measures to check the model.

Original: During the inference phase, the adversary is able to query the victim model with any inputs.
Revised: During the inference phase, the adversary is able to query the victim model with any inputs.

Original:  Proposed Methodology: Marksman Backdoor 4.1 Preliminaries
Revised:  Proposed Methodology: Marksman Backdoor 4.1 Preliminaries

Original: Consider the supervised learning setting where the goal is to learn a classifier f θ : X → Y that maps an input x ∈ X to a label y ∈ Y .
Revised: Consider the supervised learning setting where the goal is to learn a classifier f θ : X → Y that maps an input x ∈ X to a label y ∈ Y .

Original: In empirical risk minimization (ERM), the parameters θ are learned using a training dataset S
Revised: In empirical risk minimization (ERM), the parameters θ are learned using a training dataset S

Original: = { ( x 1 , y 1 ) , ..., ( x N , y N ) } where x i ∈ X and y i ∈ Y .
Revised: = { ( x 1 , y 1 ) , ..., ( x N , y N ) } where x i ∈ X and y i ∈ Y .

Original: In a standard backdoor attack, a subset of M ( M < N ) examples are first selected from S to create the poisoned subset S p .
Revised: In a standard backdoor attack, a subset of M ( M < N ) examples are first selected from S to create the poisoned subset S p .

Original: Each sample ( x, y ) in this subset is transformed into a backdoor sample ( T ( x ) , η ( y )) , where T : X → X is the trigger function and η is the target labeling function.
Revised: Each sample ( x, y ) in this subset is transformed into a backdoor sample ( T ( x ) , η ( y )) , where T : X → X is the trigger function and η is the target labeling function.

Original: The trigger function T determines how a trigger pattern is placed on the input x to create the backdoor input T ( x ) , while the target labeling function specifies how the classifier should predict in the presence of the backdoor input.
Revised: The trigger function T determines how a trigger pattern is placed on the input x to create the backdoor input T ( x ) , while the target labeling function specifies how the classifier should predict in the presence of the backdoor input.

Original: The remaining samples in S comprise the clean subset S c , i.e.,
Revised: The remaining samples in S comprise the clean subset S c , i.e.,

Original: S c =
Revised: S c =

Original: S \ S p .
Revised: S \ S p .

Original: Under ERM, we can alter the behavior of the classifier f
Revised: Under ERM, we can alter the behavior of the classifier f

Original: (i.e., inject the backdoor) by training f with both the clean samples S c and the backdoor samples S p , as follows:
Revised: (i.e., inject the backdoor) by training f with both the clean samples S c and the backdoor samples S p , as follows:

Original: [Equation]
Revised: [Equation]

Original: where L is the classification loss, e.g., cross-entropy loss.
Revised: where L is the classification loss, e.g., cross-entropy loss.

Original: During inference, for a clean input x and its true label y , the learned f will behave as follows:
Revised: During inference, for a clean input x and its true label y , the learned f will behave as follows:

Original: [Equation]
Revised: [Equation]

Original: 4.2 Marksman’s Payload: Arbitrary Attack Target Class
Revised: 4.2 Marksman’s Payload: Arbitrary Attack Target Class

Original: The training process described in the previous section essentially induces the payload, or the causal association between the trigger and the target label.
Revised: The training process described in the previous section essentially induces the payload, or the causal association between the trigger and the target label.

Original: As we discussed above, there are two common types of payload in the backdoor domain [15]: all-to-one and all-to-all.
Revised: As we discussed above, there are two common types of payload in the backdoor domain [15]: all-to-one and all-to-all.

Original: Under the all-to-one attack, all input with the trigger are predicted with a constant label, denoted as ˆy , regardless of the original label y :
Revised: Under the all-to-one attack, all input with the trigger are predicted with a constant label, denoted as ˆy , regardless of the original label y :

Original: [Equation]
Revised: [Equation]

Original: For the all-to-all attack, the input with the trigger is predicted with a label that depends on its true label y ; for example, a commonly studied target function is
Revised: For the all-to-all attack, the input with the trigger is predicted with a label that depends on its true label y ; for example, a commonly studied target function is

Original: [Equation]
Revised: [Equation]

Original: Note that, for both the all-to-one and all-to-all attacks, the attacker can only trigger one predefined target label.
Revised: Note that, for both the all-to-one and all-to-all attacks, the attacker can only trigger one predefined target label.

Original: During inference, given an input, it is not possible to causally force f to predict an arbitrary choice of a target label.
Revised: During inference, given an input, it is not possible to causally force f to predict an arbitrary choice of a target label.

Original: The goal of this work is to design a backdoor attack with a more flexible and powerful payload where the attacker can arbitrarily choose any target label during inference.
Revised: The goal of this work is to design a backdoor attack with a more flexible and powerful payload where the attacker can arbitrarily choose any target label during inference.

Original: Formally, we aim to alter the behavior of f so that:
Revised: Formally, we aim to alter the behavior of f so that:

Original: [Equation]
Revised: [Equation]

Original: Under this setting of Equation (1), the trigger pattern, devised by the function T , is associated with the attack target c .
Revised: Under this setting of Eq. (1), the trigger pattern, devised by the function T , is associated with the attack target c .

Original: This represents a more flexible payload design and a stronger backdoor attack because the adversary can freely choose to attack any target label during inference.
Revised: This represents a more flexible payload design and a stronger backdoor attack because the adversary can freely choose to attack any target label during inference.

Original: 4.3 Learning the Marksman’s Trigger Function
Revised: 4.3 Learning the Marksman’s Trigger Function

Original: Modeling and eventually learning the trigger function to achieve the objective (i.e., injecting the backdoor into f ) in Equation (1) face some crucial challenges.
Revised: Modeling and eventually learning the trigger function to achieve the objective (i.e., injecting the backdoor into f ) in Eq. (1) face some crucial challenges.

Original: First, the design of the trigger function should be systematic.
Revised: First, the design of the trigger function should be systematic.

Original: Each causal association between an image and a specific target label is embedded in one unique trigger pattern.
Revised: Each causal association between an image and a specific target label is embedded in one unique trigger pattern.

Original: Thus, a large number of trigger patterns is needed to capture the multi-trigger and multi-payload requirement.
Revised: Thus, a large number of trigger patterns is needed to capture the multi-trigger and multi-payload requirement.

Original: Second, the injection of these trigger patterns into the model should preserve the clean-data performance while achieving a high attack success rate.
Revised: Second, the injection of these trigger patterns into the model should preserve the clean-data performance while achieving a high attack success rate.

Original: On top of these, we also empirically observe that the straightforward baseline approach, which repeatedly injects these trigger patterns using the existing backdoor-attack methods such as the patch-based attack [15], fails to achieve this adversarial objective: either the clean-data performance noticeably drops below an acceptable threshold, or the attack success rate is not satisfactory.
Revised: On top of these, we also empirically observe that the straightforward baseline approach, which repeatedly injects these trigger patterns using the existing backdoor-attack methods such as the patch-based attack [15], fails to achieve this adversarial objective: either the clean-data performance noticeably drops below an acceptable threshold, or the attack success rate is not satisfactory.

Original: To tackle these challenges, we first propose representing the process of creating the trigger pattern with a class-conditional generative model.
Revised: To tackle these challenges, we first propose representing the process of creating the trigger pattern with a class-conditional generative model.

Original: The trigger function T ξ : Y × X → X learns to generate a trigger pattern that is conditioned on the attack’s target class and imperceptibly blended into the input image.
Revised: The trigger function T ξ : Y × X → X learns to generate a trigger pattern that is conditioned on the attack’s target class and imperceptibly blended into the input image.

Original: The input to the generative model is a label c and an image x , and its output is a backdoor input T ( c, x ) with a trigger pattern that can activate the backdoor to attack the target label c .
Revised: The input to the generative model is a label c and an image x , and its output is a backdoor input T ( c, x ) with a trigger pattern that can activate the backdoor to attack the target label c .

Original: Formally, we can model T using a class-conditional noise (or trigger pattern) generator g as follows:
Revised: Formally, we can model T using a class-conditional noise (or trigger pattern) generator g as follows:

Original: [Equation]
Revised: [Equation]

Original: where the ∞ -norm constraint is used to ensure a small perturbation (within an L ∞ -ball with radius ϵ ) of the clean image for optimal visual stealthiness.
Revised: where the ∞ -norm constraint is used to ensure a small perturbation (within an L ∞ -ball with radius ϵ ) of the clean image for optimal visual stealthiness.

Original: The proposed class-conditional generative trigger gives the adversary a flexible control for systematically creating the trigger for any target class, as illustrated in Figure 1.
Revised: The proposed class-conditional generative trigger gives the adversary a flexible control for systematically creating the trigger for any target class, as illustrated in Figure 1.

Original: Given the class-conditional generative trigger function T ( c, x ) , we need to learn its parameters ξ and to poison the classifier f .
Revised: Given the class-conditional generative trigger function T ( c, x ) , we need to learn its parameters ξ and to poison the classifier f .

Original: Recall that the objective is to preserve the clean-data performance while successfully performing the backdoor attack.
Revised: Recall that the objective is to preserve the clean-data performance while successfully performing the backdoor attack.

Original: Inspired by recent works in learning input-aware attacks [8, 34, 10, 11], we propose to jointly learn T ( c, x ) and poison f via the following constrained optimization objective:
Revised: Inspired by recent works in learning input-aware attacks [8, 34, 10, 11], we propose to jointly learn T ( c, x ) and poison f via the following constrained optimization objective:

Original: [Equation]
Revised: [Equation]

Original: In this problem, an optimal generative trigger function T ξ ∗ is associated with an optimally poisoned classifier.
Revised: In this problem, an optimal generative trigger function T ξ ∗ is associated with an optimally poisoned classifier.

Original: The poisoning process seeks the parameters θ of the classifier to minimize a linear combination of the clean accuracy and backdoor objectives.
Revised: The poisoning process seeks the parameters θ of the classifier to minimize a linear combination of the clean accuracy and backdoor objectives.

Original: At the same time, the generative trigger function learns to optimally craft the backdoor images, conditioned on arbitrary target labels.
Revised: At the same time, the generative trigger function learns to optimally craft the backdoor images, conditioned on arbitrary target labels.

Original: To avoid trivial solutions (i.e., sparse and small trigger patterns), we also maximize the L 2 -norm of the trigger pattern (the output of g ) within the L ∞ -ball.
Revised: To avoid trivial solutions (i.e., sparse and small trigger patterns), we also maximize the L 2 -norm of the trigger pattern (the output of g ) within the L ∞ -ball.

Original: This allows the generated trigger patterns to stretch over the entire input image.
Revised: This allows the generated trigger patterns to stretch over the entire input image.

Original: The hyperparameter α balances the mixing weights of the clean and backdoor objectives in training f while the hyperparameter β controls the effect of the penalty on sparser and smaller trigger patterns. 
Revised: The hyperparameter α balances the mixing weights of the clean and backdoor objectives in training f while the hyperparameter β controls the effect of the penalty on sparser and smaller trigger patterns. Sample minibatch S c x y } ⊂ S Sample S p = { ( x, c ) :

Revised: c ̸ =

Original: 4.4 Marksman’s Optimization
Revised: y, ( x, y ) ∈ S } 5: θ ← θ − γ f ∇ θ ( (cid:80) S c L ( f θ ( x ) , y ) + α (cid:80) S p L ( f θ ( T ξ ( c, x c )) 6: ˆ ξ ← ˆ ξ − γ T ∇ ˆ ξ ( (cid:80) S p L ( f θ ( T ˆ ξ ( x ) , c ) − β || g ( c, x ) || 2 ) 7: ξ ← ˆ ξ if j % k = j ← j 8: until j = K

Original: To solve the non-convex, constrained optimization in Equation (3), we can alternately update f and
Revised: To solve the non-convex, constrained optimization in Eq. (3), we can alternately update f and

Original: T while fixing the other one.
Revised: T while fixing the other one.

Original: However, since our ultimate goal is to learn the poisoned f , constantly updating T will cause the training process to converge very slowly.
Revised: However, since our ultimate goal is to learn the poisoned f , constantly updating T will cause the training process to converge very slowly.

Original: Inspired by some commonly accepted optimization tricks in reinforcement learning [32], we propose to update T less frequently.
Revised: Inspired by some commonly accepted optimization tricks in reinforcement learning [32], we propose to update T less frequently.

Original: During training, f is updated at every alternating step, while T is only updated once in several steps, e.g., after every training epoch.
Revised: During training, f is updated at every alternating step, while T is only updated once in several steps, e.g., after every training epoch.

Original: The backdoor samples to train f at a step are generated using T with parameter ξ updated in the previous epoch.
Revised: The backdoor samples to train f at a step are generated using T with parameter ξ updated in the previous epoch.

Original: We observe that this approach effectively stabilizes the training process and allows f to reach the optimal clean-data and attack performance in fewer training epochs.
Revised: We observe that this approach effectively stabilizes the training process and allows f to reach the optimal clean-data and attack performance in fewer training epochs.

Original: The details are presented in Algorithm 1.
Revised: The details are presented in Algorithm 1.

Original:  Evaluation 5.1 Experimental Setup
Revised:  Evaluation 5.1 Experimental Setup

Original: S S c y

Original: ∈

Original: : θ ← θ − γ f ∇ θ ( (cid:80) S c L ( f θ ( x ) y ) + α (cid:80) S p L ( f θ ( T ξ ( c x )) c ))

Original: : ˆ ξ ← ˆ ξ − γ T ∇ ˆ ξ ( (cid:80) S p L ( f θ ( T ˆ ξ ( x )) , c ) − β || g ( c x ) || 2 )

Original: : ξ j j ← j j We demonstrate the effectiveness of the proposed Marksman backdoor through comprehensive experiments on several widely-used datasets for backdoor attack study: MNIST , CIFAR10 , GTSRB , and TinyImagenet ( T-IMNET ).
Revised: We demonstrate the effectiveness of the proposed Marksman backdoor through comprehensive experiments on several widely-used datasets for backdoor attack study: MNIST , CIFAR10 , GTSRB , and TinyImagenet ( T-IMNET ).

Original: We follow the previous works [41, 44, 5, 35] and consider various architectures for the classifier f : a CNN model [35] for MNIST, Pre-activation Resnet18 ( PreActResnet18 ) [16] for CIFAR and GTSRB, and Resnet18 [16] for T-IMNET.
Revised: We follow the previous works [41, 44, 5, 35] and consider various architectures for the classifier f : a CNN model [35] for MNIST, Pre-activation Resnet ( PreActResnet18 ) [16] for CIFAR10 and GTSRB, and Resnet18 [16] for T-IMNET.

Original: As mentioned in previous sections, to the best of our knowledge, Marksman is the first work that studies multi-trigger and multi-payload backdoor with the capability of misclassifying an input to any target class.
Revised: As mentioned in previous sections, to the best of our knowledge, Marksman is the first work that studies multi-trigger and multi-payload backdoor with the capability of misclassifying an input to any target class.

Original: For comparison, we design three baseline approaches, PatchMT , RefoolMT and WaNetMT , by injecting different trigger patterns repeatedly for different target classes using recently studied trigger injection mechanisms.
Revised: For comparison, we design three baseline approaches, PatchMT , RefoolMT and WaNetMT , by injecting different trigger patterns repeatedly for different target classes using recently studied trigger injection mechanisms.

Original: PatchMT is a multi-trigger and multi-payload extension of BadNets [15] where we use a different patch-based trigger pattern for a target class.
Revised: PatchMT is a multi-trigger and multi-payload extension of BadNets [15] where we use a different patch-based trigger pattern for a target class.

Original: Similarly, ReFoolMT is a multi-trigger and multi-payload extension of ReFool [29] where we use different reflection images for different target classes.
Revised: Similarly, ReFoolMT is a multi-trigger and multi-payload extension of ReFool [29] where we use different reflection images for different target classes.

Original: Finally, WaNetMT is a multi-trigger and multi-payload extension of WaNet [35] where we randomly sample different combinations of the control-grid size k (betweenand 8) and the warping strength s (between 0.5 and 1.0) for different target classes.
Revised: Finally, WaNetMT is a multi-trigger and multi-payload extension of WaNet [35] where we randomly sample different combinations of the control-grid size k (between 4 and 8) and the warping strength s (between 0.5 and 1.0) for different target classes.

Original: Hyperparameters: For each method, we train the classifiers using the momentum SGD optimizer (initial learning rate of 0.01, and after every 100 epochs, the learning rate is decayed by a factor of 0.1).
Revised: Hyperparameters: For each method, we train the classifiers using the momentum SGD optimizer (initial learning rate of 0.01, and after every 100 epochs, the learning rate is decayed by a factor of 0.1).

Original: For Marksman, we train f and T alternately but update T slowly after every epoch.
Revised: For Marksman, we train f and T alternately but update T slowly after every epoch.

Original: To achieve a high-degree visual stealthiness of Marksman, we select ϵ as small as 0.05 for all datasets.
Revised: To achieve a high-degree visual stealthiness of Marksman, we select ϵ as small as 0.05 for all datasets.

Original: We perform grid searches to select the best α and β using the CIFAR10 dataset and use these values for all the experiments.
Revised: We perform grid searches to select the best α and β using the CIFAR10 dataset and use these values for all the experiments.

Original: We also observe that a value of α closer to 1 . 

Revised: We also observe that a value of α closer to 1 .

Original: (specifically, around 0.8) achieves the optimal poisoning objective (i.e., highest clean-data accuracy and best ASR).
Revised: (specifically, around 0.8) achieves the optimal poisoning objective (i.e., highest clean-data accuracy and best ASR).

Original: More details of the experimental setup can be found in the supplementary material.
Revised: More details of the experimental setup can be found in the supplementary material.

Original: 5.2 Effectiveness of Marksman
Revised: 5.2 Effectiveness of Marksman

Original: This section presents the attack success rates of Marksman and PatchMT.
Revised: This section presents the attack success rates of Marksman and PatchMT.

Original: To evaluate the performance in the multi-trigger multi-payload scenario, for each test sample ( x, y ) , we enumerate all possible target labels c other than the true label y and use the compared attack method to activate the backdoor associated with these target labels.
Revised: To evaluate the performance in the multi-trigger multi-payload scenario, for each test sample ( x, y ) , we enumerate all possible target labels c other than the true label y and use the compared attack method to activate the backdoor associated with these target labels.

Original: For each clean image, our attack only needs to feed it to the trigger function with a specified target class for generating the image with the trigger.
Revised: For each clean image, our attack only needs to feed it to the trigger function with a specified target class for generating the image with the trigger.

Original: The attack is considered successful for each input x and a target label c when the f correctly predicts c . 
Revised: The attack is considered successful for each input x and a target label c when the f correctly predicts c . [Table]

Revised: [Table]

Original: 5.2.1 Attack Performance
Revised: 5.2.1 Attack Performance

Original: The clean-data accuracy ( Clean ) and attack success rates ( Attack ) of Marksman and the baselines are presented in Tables 1 and  2 when poisoning 50% and 10% of the clean samples, respectively.
Revised: The clean-data accuracy ( Clean ) and attack success rates ( Attack ) of Marksman and the baselines are presented in Table 1 and Table 2 when poisoning 50% and 10% of the clean samples, respectively.

Original: As we can observe from these tables, both PatchMT’s and ReFoolMT’s clean-data performances drop significantly at a higher poisoning rate (50%), while their attack performances are very low (below 50%) at a lower poisoning rate (10%).
Revised: As we can observe from these tables, both PatchMT’s and ReFoolMT’s clean-data performances drop significantly at a higher poisoning rate (50%), while their attack performances are very low (below 50%) at a lower poisoning rate (10%).

Original: As expected, when the number of labels in the classification task is higher (e.g., 200 for T-IMNET), the attack’s success rates of PatchMT and ReFoolMT also decrease (e.g., reaching the performance, ≈ 0 .
Revised: As expected, when the number of labels in the classification task is higher (e.g., 200 for T-IMNET), the attack’s success rates of PatchMT and ReFoolMT also decrease (e.g., reaching the performance, ≈ 0 .

Original: 003 , of a random attack on 10% poisoning rate).
Revised: 003 , of a random attack on 10% poisoning rate).

Original: For
Revised: For

Original: WaNetMT, the clean-data accuracy is better at the 50% poisoning rate, but there is still a non-trivial performance degradation, compared to Marksman.
Revised: WaNetMT, the clean-data accuracy is better at the 50% poisoning rate, but there is still a non-trivial performance degradation, compared to Marksman.

Original: At the lower poisoning rate of 10%, which is a more reasonable rate, the attack performance is significantly worse than Marksman.[Table] [Table]
Revised: At the lower poisoning rate of 10%, which is a more reasonable rate, the attack performance is significantly worse than Marksman.

Original: In comparison [Table] Marksman is highly effective: 100% attack success rates on almost all datasets [Table]
Revised: In comparison, Marksman is highly effective: 100% attack success rates on almost all datasets.

Original: Marksman can achieve such high ASRs without sacrificing the clean-data performance (only a trivial drop in performance).
Revised: Marksman can achieve such high ASRs without sacrificing the clean-data performance (only a trivial drop in performance).

Original: Since the objective of our method is to have the capability of misclassifying a given input to any target class during inference, we also report the attack success rate for each class of both Marksman and the baselines in Table 3.
Revised: Since the objective of our method is to have the capability of misclassifying a given input to any target class during inference, we also report the attack success rate for each class of both Marksman and the baselines in Table 3.

Original: It can be seen that the Marksman achieves excellent performance for any arbitrary target classes.
Revised: It can be seen that the Marksman achieves excellent performance for any arbitrary target classes.

Original: The success of Marksman confirms the threat of such an attack with a significantly more flexible and powerful payload than the previous studies in the backdoor domain and encourages future research on the corresponding defensive approaches [Table] class with [Table]
Revised: The success of Marksman confirms the threat of such an attack with a significantly more flexible and powerful payload than the previous studies in the backdoor domain and encourages future research on the corresponding defensive approaches.

Original: rate

Original: [Table]
Revised: [Table]

Original: 5.2.2 Attack Performance with Different Poisoning Rates
Revised: 5.2.2 Attack Performance with Different Poisoning Rates

Original: We can understand the challenges in adapting the existing backdoor attack methods for the multitrigger multi-payload scenario and the effectiveness of Marksman by investigating the clean and backdoor performance of PatchMT and Marksman, respectively.
Revised: We can understand the challenges in adapting the existing backdoor attack methods for the multitrigger multi-payload scenario and the effectiveness of Marksman by investigating the clean and backdoor performance of PatchMT and Marksman, respectively.

Original: Figure 2 shows the clean-data accuracy and ASR for different poisoning rates (i.e., |S p | / ( |S p | + |S c | ) )
Revised: Figure 2 shows the clean-data accuracy and ASR for different poisoning rates (i.e., |S p | / ( |S p | + |S c | ) )

Original: between 5% and 50%.
Revised: between 5% and 50%.

Original: We can observe that, for PatchMT, the classifier either performs noticeably worse on clean data with higher attack performance or preserves the clean-data performance with significantly lower ASR.
Revised: We can observe that, for PatchMT, the classifier either performs noticeably worse on clean data with higher attack performance or preserves the clean-data performance with significantly lower ASR.

Original: In contrast, the performance of Marksman is consistent across different poisoning rates
Revised: In contrast, the performance of Marksman is consistent across different poisoning rates

Original: [Figure]
Revised: [Figure]

Original: 5.2.3 Performance of Trained Trigger Function on Other Models
Revised: 5.2.3 Performance of Trained Trigger Function on Other Models

Original: While Marksman’s threat model assumes that the adversary has full control of the training process, in this section, we demonstrate that the learned class-conditional trigger function can be used to attack other classifiers.
Revised: While Marksman’s threat model assumes that the adversary has full control of the training process, in this section, we demonstrate that the learned class-conditional trigger function can be used to attack other classifiers.

Original: In other words, Marksman can still be effective when the adversary has only access to the training data but does not control the training process or knows the internal structure of f .
Revised: In other words, Marksman can still be effective when the adversary has only access to the training data but does not control the training process or knows the internal structure of f .

Original: To perform the experiments
Revised: To perform the experiments

Original: [Figure]
Revised: [Figure]

Original: we freeze the parameters of the class-conditional trigger function T after it is jointly learned with a classifier f , then generate poisoned samples to attack another classifier ˆ f whose network can be either similar or different from f ’s architecture.
Revised: we freeze the parameters of the class-conditional trigger function T after it is jointly learned with a classifier f , then generate poisoned samples to attack another classifier ˆ f whose network can be either similar or different from f ’s architecture.

Revised: [Table]

Original: Table 4 shows the clean and attack performance when T is jointly trained with a classifier ( f ), e.g.,
Revised: Table 4 shows the clean and attack performance when T is jointly trained with a classifier ( f ), e [Table] g.,

Revised: PreActResnet

Revised: [Table]

Original: PreActResnet18, and ˆ f is another classifier with the same network architecture, i.e., PreActResnet18, or with a different network architecture, i.e., Vgg11, either of which is trained from scratch on the poisoned data created by T .
Revised: and ˆ f is another classifier with the same network architecture, i [Table] e., PreActResnet18, or with a different network architecture, i

Revised: [Table]

Revised: e., Vgg11, either of which is trained from scratch on the poisoned data created by T .

Original: It can be seen that Marksman is still effective: nearly optimal clean-data performance and almost 100% ASR.
Revised: It can be seen that Marksman is still effective: nearly optimal clean-data performance and almost 100% ASR.

Original: This transferability of T after it is learned is very interesting.
Revised: This transferability of T after it is learned is very interesting.

Original: We conjecture that the training process of T is related to approximating adversarial example generation of f that shows decent transferability [27, 43], while the detailed investigation is beyond the scope of this paper.
Revised: We conjecture that the training process of T is related to approximating adversarial example generation of f that shows decent transferability [27, 43], while the detailed investigation is beyond the scope of this paper.

Original: Note that the trigger is produced by the trigger generator/function that is the secret held by the adversary, which is qualitatively different from adversarial examples.[Table]
Revised: Note that the trigger is produced by the trigger generator/function that is the secret held by the adversary, which is qualitatively different from adversarial examples.

Original: 5.3 Performance against Defensive Measures
Revised: 5.3 Performance against Defensive Measures

Original: In this section, we evaluate the backdoor-injected classifiers by the proposed Marksman framework against popular defensive mechanisms, including Neural Cleanse (model mitigation defense) [45],
Revised: In this section, we evaluate the backdoor-injected classifiers by the proposed Marksman framework against popular defensive mechanisms, including Neural Cleanse (model mitigation defense) [45],

Original: STRIP (detection based defense)
Revised: STRIP (detection based defense)

Original: [13], and Spectral Signature (latent-space inspection) [44].
Revised: [13], and Spectral Signature (latent-space inspection) [44].

Original: It is important to note that these defenses are originally designed for conventional backdoor attacks with a weaker capability that are only able to misclassify an input to a specific target class.
Revised: It is important to note that these defenses are originally designed for conventional backdoor attacks with a weaker capability that are only able to misclassify an input to a specific target class.

Original: 5.3.1 Model Mitigation: Neural Cleanse
Revised: [Figure] Benign

Original:  We evaluate the robustness of Marksman against Neural Cleanse, a widely-used defensive method based on the pattern optimization approach.
Revised: Marksman We evaluate the robustness of Marksman against Neural Cleanse, a widely-used defensive method based on the pattern optimization approach.

Original: Specifically, for every possible label, Neural Cleanse learns the optimal trigger pattern to attack this target label.
Revised: Specifically, for every possible label, Neural Cleanse learns the optimal trigger pattern to attack this target label.

Original: It then quantifies whether any of the learned trigger patterns is an outlier via a metric called Anomaly Index.
Revised: It then quantifies whether any of the learned trigger patterns is an outlier via a metric called Anomaly Index.

Original: Neural Cleanse determines if a model has a backdoor if the Anomaly Index is greater than 2; otherwise, the model is benign.
Revised: Neural Cleanse determines if a model has a backdoor if the Anomaly Index is greater than 2; otherwise, the model is benign.

Original: The results of the benign classifier and the backdoor-injected classifier by Marksman are presented in Figure 3.
Revised: The results of the benign classifier and the backdoorinjected classifier by Marksman are presented in Figure 3.

Original: It can be observed that Marksman bypasses the Neural Cleanse on all datasets.[Figure]
Revised: It can be observed that Marksman bypasses the Neural Cleanse on all datasets.

Original: 5.3.2 Input Perturbation: STRIP
Revised: 5.3.2 Input Perturbation: STRIP

Revised: Next

Original: Next, we evaluate the robustness of Marksman against STRIP [13], a representative inference-phase detection method, which perturbs the input image and calculates the entropy of the predictions of these perturbed images.
Revised: [Figure] we evaluate the robustness of Marksman against STRIP [13] [Figure] a representative inference-phase detection method, which perturbs the input image and calculates the entropy of the predictions of these perturbed images.

Original: STRIP determines if an image is a backdoor input by relying on the fact that the entropy of the perturbed backdoor input tends to be lower than that of the clean one.
Revised: STRIP determines if an image is a backdoor input by relying on the fact that the entropy of the perturbed backdoor input tends to be lower than that of the clean one

Revised: [Figure]

Original: We plot the entropy of clean and backdoor images computed by STRIP in Figure 4.
Revised: We plot the entropy of clean and backdoor images computed by STRIP in Figure 4.

Original: As we can observe from this figure, the entropy distributions of the backdoor and clean samples are similar.
Revised: As we can observe from this figure, the entropy distributions of the backdoor and clean samples are similar.

Original: Thus, Marksman can also bypass the STRIP defense.[Figure]
Revised: Thus, Marksman can also bypass the STRIP defense.

Original: 5.3.3 Latent-space Inspection: Spectral Signature
Revised: 5.3.3 Latent-space Inspection: Spectral Signature

Original: Spectral Signature [44] is another representative defensive approach that inspects the latent space of the trained classifier.
Revised: Spectral Signature [44] is another representative defensive approach that inspects the latent space of the trained classifier.

Original: Spectral Signature first finds the top-right singular vector of the covariance matrix of the latent vectors using a small subset of clean samples.
Revised: Spectral Signature first finds the top-right singular vector of the covariance matrix of the latent vectors using a small subset of clean samples.

Original: Then it calculates the correlation of each sample to this singular vector.
Revised: Then it calculates the correlation of each sample to this singular vector.

Original: Those with the outlier scores are identified as backdoor samples.
Revised: Those with the outlier scores are identified as backdoor samples.

Original: We follow the same experiments in [44] and select 5,000 clean samples and 500 backdoor samples for each dataset. 
Revised: We follow the same experiments in [44] and select 5,000 clean samples and 500 backdoor samples for each dataset. Then [Figure]

Original: Then, we calculate the correlation scores and plot the histograms of the scores for both sets of samples.
Revised: we calculate the correlation scores and plot the histograms of the scores for both sets of samples.

Original: As we can observe in Figure
Revised: As we can observe in Figure

Original: [Figure]
Revised: [Figure]

Original: there is no clear separation between the correlation scores of the backdoor samples and the clean samples, validating the stealthiness of the proposed
Revised: there is no clear separation between the correlation scores of the backdoor samples and the clean samples, validating the stealthiness of the proposed

Original: Marksman backdoor
Revised: Marksman backdoor.

Original: [Figure]
Revised: [Figure]

Revised: f s a m p l

Revised: [Figure]

Revised: s

Original: [Figure]
Revised: [Figure]

Original: 5.3.4 Performance against Fine-Pruning
Revised: 5.3.4 Performance against Fine-Pruning

Original: We evaluate the robustness of Marksman against Fine-Pruning [26], which is a model analysis based defense.
Revised: We evaluate the robustness of Marksman against Fine-Pruning [26], which is a model analysis based defense.

Original: Fine-Pruning finds a learned classifier’s dormant neurons (with very low activations) given a small clean dataset.
Revised: Fine-Pruning finds a learned classifier’s dormant neurons (with very low activations) given a small clean dataset.

Original: Then it gradually prunes these dormant neurons to mitigate the backdoor without affecting the inference accuracy.
Revised: Then it gradually prunes these dormant neurons to mitigate the backdoor without affecting the inference accuracy.

Original: We evaluate Marksman against Fine-Pruning by plotting the clean-data accuracy and ASR when different numbers of the dormant neurons are pruned in Figure 6.
Revised: We evaluate Marksman against Fine-Pruning by plotting the clean-data accuracy and ASR when different numbers of the dormant neurons are pruned in Figure 6.

Original: We can observe that the ASR drops considerably less than the drop in clean-data performance at all pruning ratios.
Revised: We can observe that the ASR drops considerably less than the drop in clean-data performance at all pruning ratios.

Original: This suggests that the
Revised: This suggests that the

Original: Fine-Pruning is ineffective against Marksman
Revised: Fine-Pruning is ineffective against Marksman.

Original: [Figure]
Revised: [Figure]

Original: 5.4 Visualization of Backdoor Attack Samples from Marksman
Revised: 5.4 Visualization of Backdoor Attack Samples from Marksman

Original: This section presents the backdoor samples with different attack target classes that are different from the true labels (class ‘2’ for MNIST, class ‘0’- plane for CIFAR10), as shown in Figure 7.
Revised: This section presents the backdoor samples with different attack target classes that are different from the true labels (class ‘2’ for MNIST, class ‘0’- plane for CIFAR10), as shown in Figure 7.

Original: We can generate a different trigger for each target class to activate the corresponding payload.
Revised: We can generate a different trigger for each target class to activate the corresponding payload.

Original: We can observe that Marksman’s attack images are visually indistinguishable from the original images for all target classes, even with such an enhanced adversarial capability. 
Revised: We can observe that Marksman’s attack images are visually indistinguishable from the original images for all target classes, even with such an enhanced adversarial capability. 9 (a) MNIST

Original: [Figure]
Revised: [Figure]

Original: In this paper, we proposed a novel backdoor attack, Marksman, with a much more powerful payload where the adversary can arbitrarily choose the target class given any input during inference. This framework is able to effectively and efficiently learn the trigger function and inject the backdoor into a DNN model.
Revised: CIFAR

Original: Our experimental results over popular datasets demonstrate the effectiveness of Marksman and the stealthiness against the existing representative defenses.  Detailed Experimental Setup To evaluate our method, we use four datasets, MNIST, CIFAR10, GTSRB (German Traffic Sign
Revised: Clean (Original), Residual (Amplified 50 × ), and Backdoor samples in 1st, 2nd and, 3rd rows, respectively.

Original: Recognition Benchmark), and T-IMNET, to evaluate our method. Note that MNIST, CIFAR10, and GTSRB have been widely used in the literature of backdoor attacks on DNN. On the other hand, the use of a more complex dataset with more classes, T-IMNET, enables better evaluation for the scalability of multiple-trigger and multi-payload backdoor. We present the details of the our experiments on these datasets below:
Revised: The target attack labels are in the last rows.

Original: • MNIST [21]: We applied random cropping and random rotation as data augmentation for the training process.

Original: During the evaluation stage, no augmentation is applied.

Original: The dataset can be found in: http://yann.lecun.com/exdb/mnist • CIFAR10 [18]: We applied random cropping, random rotation, and random horizontal flilp as data augmentation for the training process.

Original:  The dataset can be found in: https://www.cs.toronto.edu/ ~kriz/cifar.html • GTSRB [40]: In our experiments, GTSRB input images are all resized into 32 × 32 pixels, then applied a similar data augmentation as that of CIFAR10 in training. In the evaluation stage, no augmentation is used. The dataset can be found in: http://benchmark.ini.rub.de/?section=gtsrb&subsection=dataset • T-IMNET [47].
Revised:  Conclusion

Original: Input images are all resized into 64 × 64 resolution.

Original: A similar data augmentation as that of CIFAR10 is applied in the training stage.

Original: No augmentation is used in the evaluation stage.

Original: The dataset can be found in: http://cs231n.stanford.edu/tinyimagenet-200.zip

Original: Our experiments are conducted in multiple Linux machines, each of which has 128 cores, 1536GiB of RAM, 8 A100-PCIE-40GB GPUs.

Original: The CUDA version is 11.2.

Original: For all experiments of Marksman, we model the class-conditional noise generator (trigger-pattern generator) g as an autoencoder, whose input is an image and a learnable embedding of a target class. The architecture of the autoencoder is presented in Table 5. [Table]
Revised: In this paper, we proposed a novel backdoor attack, Marksman, with a much more powerful payload where the adversary can arbitrarily choose the target class given any input during inference.

Original: 7.2 Classification Models

Original: For MNIST, we use the same simple CNN classifier as in WaNet [35] (detailed in Table 6). As mentioned in the main paper, we use PreActResnet18 [16] for CIFAR10 and GTSRB datasets, and Resnet18 [16] for T-IMNET. [Table]
Revised: To the best of our knowledge, Marksman is the first multi-trigger and multi-payload backdoor attack method that can misclassify an input to any target class.

Original: 7.3 Training Hyperparameters

Original: Table 7 provides additional details to Section 5.1 in the main paper.

Original: [Table] As indicated in Section 5.1 of the main paper [Table] we perform grid searches to select the best α and β values using the CIFAR10 dataset.
Revised: This framework can effectively and efficiently learn the trigger function and inject the backdoor into a DNN model.

Original: The selected values are 0 .

Original: 8 for α and 1.0 for β .

Original: We use these values for all the experiments

Original: We first provide, in Section 8.1, additional attack performance when varying the poisoning rate for ReFoolMT and WaNetMT, as well as for PatchMT on T-IMNET. In Section 8.2, we provide a qualitative analysis of the hyperparameters α and β on the CIFAR10 dataset. Finally, we present the statistical errors for the main experiments of the paper in Section 8.3.
Revised: Given a target label, the proposed class-condition generative trigger function can generate an imperceptible trigger pattern to cause the model to predict the target label.

Original: 8.1 Additional Clean & Attack Performance Results with Different Poisoning Rates

Original: We present the attack performance while varying the poisoning rates for PatchMT on T-IMNET in Figure 8, and  the other baselines (ReFoolMT and WaNetMT) in Figure 9. We can observe a similar phenomenon (i [Figure]
Revised: We then propose a constrained optimization objective that can simultaneously learn the trigger function and poison the model.

Original: e., either performs noticeably worse on clean data with higher attack performance or preserves the clean-data performance with significantly lower ASR) for PatchMT on T-IMNET in

Original: Figure 8 and for ReFoolMT on all the datasets in Figure 9.

Original: For WaNetMT, its clean-data performance is consistently sub-optimal, suggesting that it is more difficult to perform the multi-trigger andmulti-payload attacks using the warping mechanism [Figure] [Figure] 8.2 Hyperparameter Analysis
Revised: Our experimental results over popular datasets demonstrate the effectiveness of Marksman and the stealthiness against the existing representative defenses.

Original: As indicated in Section 5.1 of the main paper

Original: [Figure]

Original: we perform grid searches to select the best α and β using the CIFAR10 dataset and use these values for all the experiments.

Original: However, we additionally present the qualitative analysis of the hyperparameters α and β in Figure 10.

Original: As we can observe, lower values of α result in lower clean-data accuracy, while an optimal value is closer to 1.0 (around 0.8).

Original: Note that when α = 1 . 0 , the classifier is not poisoned.

Original: Thus, the ASR drops to 0.

Original: On the other hand, clean and attack performances are not very sensitive to the values of β .

Original: In our experiments, we set β = 1 . 0 . [Figure] 8.3 Statistical Importance We present the statistical errors in our experiments in Figure 11.
Revised: Our work takes another significant step toward understanding the extensive risks of backdoor attacks in practice.

Original: As we can observe, most methods, except for ReFoolMT on MNIST, have very narrow (99%) confidence intervals in all experiments.

Original: We also verify that the superior performance of Marksman is statistically significant with a p-value less than 0 .

Original: [Figure] 8.4 Performance against NAD and ANP We evaluate the robustness of Marksman against two representative post-training defenses, Neural Attention Distillation (NAD) [23] and Adversarial Neuron Pruning (ANP) [46].
Revised: This work calls for defensive studies to counter Marksman’s more powerful yet sophisticated multi-trigger and multi-payload attacks.

Original: We follow the same experimental settings of NAD and ANP, which assumes a small subset of clean data (5%).

Original: Tableshow the clean-data Accuracy and ASR of the poisoned models after they undergo the backdoor erasing processes of NAD and ANP. As we can observe, for NAD, while ASR decreases, ACC performance degrades even more significantly. This demonstrates that NAD’s defense is ineffective against Marksman’s attack. For ANP, the defensive process does not significantly degrade the ASRs of Marksman, while the ACCs also drop more significantly.
Revised: Acknowledgement

Original: Again, ANP is not effective against Marksman’s attack. [Table]  Limitations
Revised: We thank anonymous Referees for their constructive comments.

Original: To the best of our knowledge, Marksman is the first work that studies multi-trigger and multi-payload backdoor with the capability of misclassifying an input to any target class. However, this assumes that the adversary has the ability to embed digitally generated triggers into the images before feeding them to the classifier. An interesting future direction is to extend the multi-trigger and multi-payload scenario into physical attacks. Such evaluations can further assess whether Marksman is only a hypothetical phenomenon or indeed a real-world threat.
Revised: The research was conducted while all authors worked at the Baidu Cognitive Computing Lab, 10900 NE 8th St. Bellevue, WA 98004, USA.

Original: Since this is the first work in this direction, there is no existing defense that targets the scenario of Marksman, which can also be observed from our experiments that the existing representative defensive methods are not effective against Marksman.

Original: On the other hand, these experiments do not reasonably or effectively evaluate the performance of our approach.

Original: To this end, we encourage future research on developing more powerful defenses to combat our stealthy backdoor attack with significantly enhanced adversarial capabilities.

Original: Societal Impacts: Our work on the backdoor attack is likely to increase the awareness and understanding of such vulnerability on neural networks.

Original: The proposed attack, if not appropriately used, may bring security threats to the existing DNN applications.

Original: We believe our study is an important step towards understanding the full capability of backdoor attacks.

Original: This knowledge will, in turn, facilitate the further development of secure and