Original: M ODEL - BASED S ALIENCY FOR THE D ETECTION OF A DVERSARIAL E XAMPLES
Revised: M ODEL - BASED S ALIENCY FOR THE D ETECTION OF A DVERSARIAL E XAMPLES

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Adversarial perturbations cause a shift in the salient features of an image, which often results in misclassiﬁcation.
Revised: Adversarial perturbations cause a shift in the salient features of an image, which often results in misclassiﬁcation.

Original: Previous work has suggested that these salient features could be used as a defence, arguing that with saliency tools we could successfully detect adversarial examples.
Revised: Previous work has suggested that these salient features could be used as a defence, arguing that with saliency tools we could successfully detect adversarial examples.

Original: While the idea itself is appealing, we show that prior work which used gradient-based saliency tools is ineffective as an adversarial defense – it fails to beat a simple baseline which uses the same model but with the saliency map removed.
Revised: While the idea itself is appealing, we show that prior work which used gradient-based saliency tools is ineffective as an adversarial defense – it fails to beat a simple baseline which uses the same model but with the saliency map removed.

Original: To remedy this, we demonstrate that learnt saliency models can capture the shifts in saliency due to adversarial perturbations, while also having a low computational cost.
Revised: To remedy this, we demonstrate that learnt saliency models can capture the shifts in saliency due to adversarial perturbations, while also having a low computational cost.

Original: This allows saliency models to be used effectively as a real-time defence.
Revised: This allows saliency models to be used effectively as a real-time defence.

Original: Further, using the learnt saliency model, we propose a novel defense: a CNN which distinguishes between adversarial images and natural images using salient pixels as its input.
Revised: Further, using the learnt saliency model, we propose a novel defense: a CNN which distinguishes between adversarial images and natural images using salient pixels as its input.

Original: On MNIST, CIFAR-10, and ASSIRA, our defense improves on using the saliency map alone and can detect various adversarial attacks.
Revised: On MNIST, CIFAR-10, and ASSIRA, our defense improves on using the saliency map alone and can detect various adversarial attacks.

Original: Lastly, we show that even when trained on weak defenses, we can detect adversarial images generated by strong attacks such as
Revised: Lastly, we show that even when trained on weak defenses, we can detect adversarial images generated by strong attacks such as

Original: C&W and DeepFool.
Revised: C&W and DeepFool.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Adversarial examples highlight a crucial difference between human vision and computer image processing.
Revised: Adversarial examples highlight a crucial difference between human vision and computer image processing.

Original: Often computers fail to understand the relevant characteristics of an image for classiﬁcation (Ribeiro et al., 2016) or fail to generalize locally, i.e., misclassify examples close to the training data (Szegedy et al., 2013).
Revised: Often computers fail to understand the relevant characteristics of an image for classiﬁcation (Ribeiro et al., 2016) or fail to generalize locally, i.e., misclassify examples close to the training data (Szegedy et al., 2013).

Original: Attacks exploit this property by altering pixels the classiﬁer heavily relies on, pixels which are irrelevant to humans for object recognition.
Revised: Attacks exploit this property by altering pixels the classiﬁer heavily relies on, pixels which are irrelevant to humans for object recognition.

Original: As a consequence, adversarial perturbations fool classiﬁers while the correct class remains clear to humans.
Revised: As a consequence, adversarial perturbations fool classiﬁers while the correct class remains clear to humans.

Original: Saliency maps identify the pixels an image classiﬁer uses for its prediction; as such, they can be used as a tool to understand why a classiﬁer is fooled.
Revised: Saliency maps identify the pixels an image classiﬁer uses for its prediction; as such, they can be used as a tool to understand why a classiﬁer is fooled.

Original: Building on this concept, researchers have shown qualitatively that adversarial perturbations cause a shift in the saliency of classiﬁers (Fong & Vedaldi, 2017; Gu & Tresp, 2019).
Revised: Building on this concept, researchers have shown qualitatively that adversarial perturbations cause a shift in the saliency of classiﬁers (Fong & Vedaldi, 2017; Gu & Tresp, 2019).

Original: Figure 6 shows examples of a natural image and corresponding adversarial images, each above their respective saliency maps.
Revised: Figure 6 shows examples of a natural image and corresponding adversarial images, each above their respective saliency maps.

Original: The saliency maps corresponding to adversarial images show perceptible differences to that of the original image, even though adversarial images themselves often seem unperturbed.
Revised: The saliency maps corresponding to adversarial images show perceptible differences to that of the original image, even though adversarial images themselves often seem unperturbed.

Original: Consider the MNIST example.

Original: For the original image, the saliency map shows that the classiﬁer focuses on the four (and a couple of random pixels on the left).
Revised: For the original image, the saliency map shows that the classiﬁer focuses on the four (and a couple of random pixels on the left).

Original: We observe that for the adversarial images, the classiﬁer starts focusing more on irrelevant aspects of the left side of the image.
Revised: We observe that for the adversarial images, the classiﬁer starts focusing more on irrelevant aspects of the left side of the image.

Original: There is ample research into different techniques for ﬁnding saliency maps (see e.g. Zeiler & Fergus, 2014; Springenberg et al., 2014; Bach et al., 2015; Ribeiro et al., 2016; Shrikumar et al., 2017;
Revised: There is ample research into different techniques for ﬁnding saliency maps (see e.g. Zeiler & Fergus, 2014; Springenberg et al., 2014; Bach et al., 2015; Ribeiro et al., 2016; Shrikumar et al., 2017;

Original: Selvaraju et al., 2017; Zintgraf et al., 2017; Fong & Vedaldi, 2017).
Revised: Selvaraju et al., 2017; Zintgraf et al., 2017; Fong & Vedaldi, 2017).

Original: However, not all saliency maps are equally informative (Fong & Vedaldi, 2017).
Revised: However, not all saliency maps are equally informative (Fong & Vedaldi, 2017).

Original: For example, the Jacobian 1 can be used to determine the saliency of a pixel in the classiﬁcation of the image (Papernot et al., 2016b; Zhang et al., 2018).
Revised: For example, the Jacobian 1 can be used to determine the saliency of a pixel in the classiﬁcation of the image (Papernot et al., 2016b; Zhang et al., 2018).

Original: As the Jacobian is often used to generate adversarial examples, intuitively, we expect that it can be used effectively to detect adversarial perturbations.
Revised: As the Jacobian is often used to generate adversarial examples, intuitively, we expect that it can be used effectively to detect adversarial perturbations.

Original: Zhang et al.
Revised: Zhang et al.

Original: (2018) propose a defense to this effect: they determine whether an input is adversarial, given the Jacobian-based
Revised: (2018) propose a defense to this effect: they determine whether an input is adversarial, given the Jacobian-based

Original: Original
Revised: Original

Original: FGSM
Revised: FGSM

Original: BIM
Revised: BIM

Original: MIM
Revised: MIM

Original: JSMA
Revised: JSMA

Original: C & W
Revised: C & W

Original: DF saliency map concatenated with the image .
Revised: DF saliency map concatenated with the image .

Original: However, as shown qualitatively by Gu & Tresp (2019), gradients are not always able to capture differences between adversarial images and natural images (for an example see Figures 7 and 8 in Appendix D).
Revised: However, as shown qualitatively by Gu & Tresp (2019), gradients are not always able to capture differences between adversarial images and natural images (for an example see Figures 7 and 8 in Appendix D).

Original: Here we inspect the proposed Jacobian-based approach and show that only the concatenated input affects the technique’s performance in detecting adversarial examples, with the Jacobian having no effect.
Revised: Here we inspect the proposed Jacobian-based approach and show that only the concatenated input affects the technique’s performance in detecting adversarial examples, with the Jacobian having no effect.

Original: While gradients may not be informative for detection, saliency should be an effective tool for detecting adversarial images.
Revised: While gradients may not be informative for detection, saliency should be an effective tool for detecting adversarial images.

Original: In our analysis, we use more powerful model-based saliency techniques and show that the magnitude of the shift of the saliency map due to adversarial perturbations often exceeds the L 2 distance between the saliency maps of different natural images.
Revised: In our analysis, we use more powerful model-based saliency techniques and show that the magnitude of the shift of the saliency map due to adversarial perturbations often exceeds the L 2 distance between the saliency maps of different natural images.

Original: Building on this result, we consider two different possible effects adversarial perturbations might have on the classiﬁer:
Revised: Building on this result, we consider two different possible effects adversarial perturbations might have on the classiﬁer:

Original: They might cause the classiﬁer to focus on
Revised: They might cause the classiﬁer to focus on

Original: the wrong
Revised: the wrong

Original: pixel locations 
Revised: pixel locations 2.

Original: They might change the pixel values of salient pixels
Revised: They might change the pixel values of salient pixels

Original: Based on each hypothesis, we propose two CNN classiﬁer architectures to detect adversarial images.
Revised: Based on these hypotheses, we employ two CNN classiﬁer architectures to detect adversarial images.

Original: Claim (1) is similar to the work of Fong & Vedaldi (2017); however, their main contribution is that saliency maps generated by different techniques are not equally effective in capturing changes due to adversarial perturbations (produced using BIM (Kurakin et al., 2016)).
Revised: Claim (1) can be captured by shifts in saliency maps, as previously considered by Fong & Vedaldi (2017).

Original: We extend the analysis here by proving the defensive capability of our model-based saliency against difﬁcult black-box attacks, such as C&W and DeepFool 3 , as well as white-box adversarial attacks.
Revised: In this work, we extend their analysis 3 by proving the defensive capability of our model-based saliency against difﬁcult black-box attacks, such as C&W and DeepFool 4 , as well as white-box adversarial attacks.

Original: Further, we also show that when using weak saliency maps, such as the Jacobian, the defense is not more effective than simply predicting whether an image is adversarial from the input image itself.

Original: By considering claim (2), we demonstrate that incorporating pixel values improve the performance of the classiﬁer when shifts in saliency maps do not sufﬁce to capture adversarial perturbations.
Revised: By considering claim (2), we demonstrate that incorporating pixel values improves the performance of the classiﬁer when shifts in saliency maps do not sufﬁce to capture adversarial perturbations.

Original: We also show that our salient-pixel based defense  is more robust than the saliency map defense against white-box attacks.
Revised: We also show that our salient-pixel based defense generalizes well (detecting stronger attacks when trained on weaker attacks)and is more robust than the saliency map defense against white-box attacks.

Original: Lastly, we demonstrate that saliency can be used to detect adversarial examples generated by small perturbations, contrary to other defenses which exhibit threshold behavior : i.e., when the adversarial perturbation is too small, other defenses (speciﬁcally Gong et al., 2017; Zhang et al., 2018) are unable to detect the adversarial images.
Revised: Lastly, we demonstrate that saliency can be used to detect adversarial examples generated by small perturbations, contrary to other defenses which exhibit threshold behavior : i.e., when the adversarial perturbation is too small, other defenses (speciﬁcally Gong et al., 2017; Zhang et al., 2018) are unable to detect the adversarial images.

Original:  S ALIENCY AND A DVERSARIAL E XAMPLES
Revised:  S ALIENCY AND A DVERSARIAL E XAMPLES

Original: Saliency maps and adversarial perturbations have similar mathematical formulations and derivations.
Revised: Saliency maps and adversarial perturbations have similar mathematical formulations and derivations.

Original: Both are computed by investigating the relation between the values of pixels and the classiﬁcation score.
Revised: Both are computed by investigating the relation between the values of pixels and the classiﬁcation score.

Original: Adversarial examples are found by deriving the minimal perturbations required to change the classiﬁcation of an image.
Revised: Adversarial examples are found by deriving the minimal perturbations required to change the classiﬁcation of an image.

Original: Saliency is computed by ﬁnding the pixels used by the model to determine the class of an object (Simonyan et al., 2013).
Revised: Saliency is computed by ﬁnding the pixels used by the model to determine the class of an object (Simonyan et al., 2013).

Original: Saliency maps can be found by considering the smallest part of an image that is sufﬁcient for a correct classiﬁcation, known as the smallest sufﬁcient region (SSR), or whose removal is sufﬁcient for an incorrect classiﬁcation, known as the smallest destroying region (SDR) (Dabkowski & Gal, 2017; Fong & Vedaldi, 2017).
Revised: Saliency maps can be found by considering the smallest part of an image that is sufﬁcient for a correct classiﬁcation, known as the smallest sufﬁcient region (SSR), or whose removal is sufﬁcient for an incorrect classiﬁcation, known as the smallest destroying region (SDR) (Dabkowski & Gal, 2017; Fong & Vedaldi, 2017).

Original: Observe that the latter deﬁnition of saliency is very close to that of adversarial examples.
Revised: Observe that the latter deﬁnition of saliency is very close to that of adversarial examples.

Original: Mathematically, both saliency maps and adversarial perturbations can be derived in a similar fashion.
Revised: Mathematically, both saliency maps and adversarial perturbations can be derived in a similar fashion.

Original: Consider adversarial examples.
Revised: Consider adversarial examples.

Original: The general formulation of an adversarial attack can be summarized as follows:
Revised: The general formulation of an adversarial attack can be summarized as follows:

Original: [Equation]
Revised: [Equation]

Original: where x is the natural image, r is the adversarial perturbation, y is the correct class, and y (cid:48) is an incorrect class.
Revised: where x is the natural image, r is the adversarial perturbation, y is the correct class, and y (cid:48) is an incorrect class.

Original: Due to the non-linearity of NNs, solving the above problem requires non-linear optimization.
Revised: Due to the non-linearity of NNs, solving the above problem requires non-linear optimization.

Original: Therefore, in practice several different approaches to solving the above formulation have been implemented.
Revised: Therefore, in practice several different approaches to solving the above formulation have been implemented.

Original: For example, Goodfellow et al.
Revised: For example, Goodfellow et al.

Original: set r = ε sign ( δf ( x ) δx ) .
Revised: set r = ε sign ( δf ( x ) δx ) .

Original: Similarly, saliency can be computed using the forward derivative δf ( x ) δx (Papernot et al., 2016b; Zhang et al., 2018).
Revised: Similarly, saliency can be computed using the forward derivative δf ( x ) δx (Papernot et al., 2016b; Zhang et al., 2018).

Original: Previous research has already started investigating the relation between saliency and adversarial examples.
Revised: Previous research has already started investigating the relation between saliency and adversarial examples.

Original: This includes:
Revised: This includes:

Original: Using saliency to attack Researchers have devised adversarial attacks that use saliency (Papernot et al., 2016b; Yu et al., 2018).
Revised: Using saliency to attack Researchers have devised adversarial attacks that use saliency (Papernot et al., 2016b; Yu et al., 2018).

Original: The key idea is to use saliency to determine the pixel that is most sensitive to perturbation iteratively.
Revised: The key idea is to use saliency to determine the pixel that is most sensitive to perturbation iteratively.

Original: The main beneﬁt is that fewer pixels are perturbed – often perturbing as few as 4% of the pixels sufﬁces to change the classiﬁcation of the image (Papernot et al., 2016b).
Revised: The main beneﬁt is that fewer pixels are perturbed – often perturbing as few as 4% of the pixels sufﬁces to change the classiﬁcation of the image (Papernot et al., 2016b).

Original: Using saliency to defend Fong & Vedaldi (2017) introduce a method that detects adversarial perturbations by using heat-map visualizations of the predicted class.
Revised: Using saliency to defend Fong & Vedaldi (2017) introduce a method that detects adversarial perturbations by using heat-map visualizations of the predicted class.

Original: However, in their analysis, they only use BIM (Kurakin et al., 2016), which is easily detected.
Revised: However, in their analysis, they only use BIM (Kurakin et al., 2016), which is easily detected.

Original: Further, Zhang et al.
Revised: Further, Zhang et al.

Original: (2018) hypothesize that there is a mismatch between the saliency of a classiﬁcation model and the adversarial example.
Revised: (2018) hypothesize that there is a mismatch between the saliency of a classiﬁcation model and the adversarial example.

Original: They propose a defense against adversarial attacks by training a classiﬁer on images concatenated with their saliency map, which is computed by calculating the Jacobian of the classiﬁer with respect to the image x , i.e., s x = ∇ x f ( x ) .
Revised: They propose a defense against adversarial attacks by training a classiﬁer on images concatenated with their saliency map, which is computed by calculating the Jacobian of the classiﬁer with respect to the image x , i.e., s x = ∇ x f ( x ) .

Original: Zhang et al.
Revised: Zhang et al.

Original: ﬁnd that their method obtains a high accuracy (often near 100% ) when detecting adversarial images generated by FGSM, MIM, and
Revised: ﬁnd that their method obtains a high accuracy (often near 100% ) when detecting adversarial images generated by FGSM, MIM, and

Original: C&W attacks on MNIST, CIFAR-10, and 10-ImageNet.
Revised: C&W attacks on MNIST, CIFAR-10, and 10-ImageNet.

Original: However, Gu & Tresp (2019) contradict these results, and demonstrate that the gradients show imperceptible differences due to adversarial perturbations (see Figures 7 and 8 in Appendix D).
Revised: However, Gu & Tresp (2019) contradict these results, and demonstrate that the gradients show imperceptible differences due to adversarial perturbations (see Figures 7 and 8 in Appendix D).

Original: Adversarial robustness and interpretability of models Fong & Vedaldi (2017) and Gu & Tresp (2019) 4 show that saliency maps can be used to explain adversary classiﬁcations.
Revised: Adversarial robustness and interpretability of models Fong & Vedaldi (2017) and Gu & Tresp (2019) 5 show that saliency maps can be used to explain adversary classiﬁcations.

Original: Both highlight an important trend: not all techniques used to compute saliency maps show shifts in saliency maps due to adversarial perturbations .
Revised: Both highlight an important trend: not all techniques used to compute saliency maps show shifts in saliency maps due to adversarial perturbations .

Original: Further, Tsipras et al.
Revised: Further, Tsipras et al.

Original: (2018) shows that more robust models have more interpretable saliency masks.
Revised: (2018) shows that more robust models have more interpretable saliency masks.

Original: Etmann et al.
Revised: Etmann et al.

Original: (2019) quantify the relation by investigating the alignment between the saliency map and the input image.
Revised: (2019) quantify the relation by investigating the alignment between the saliency map and the input image.

Original:  M ODEL - BASED S ALIENCY FOR THE D ETECTION OF A DVERSARIAL E XAMPLES
Revised:  M ODEL - BASED S ALIENCY FOR THE D ETECTION OF A DVERSARIAL E XAMPLES

Original: In this section, we explain how we construct and evaluate our saliency-based adversarial example detectors.
Revised: In this section, we explain how we construct and evaluate our saliency-based adversarial example detectors.

Original: 3. C LASSIFIER
Revised: 3. C LASSIFIER

Original: We train a convolutional neural network image classiﬁer, which we target with black-box attacks; the architectures are summarized in Appendix A.
Revised: We train a convolutional neural network image classiﬁer, which we target with black-box attacks; the architectures are summarized in Appendix A.

Original: We use cross-entropy loss and optimize the parameters using Adam (Kingma & Ba, 2014) with the standard hyperparameters settings.
Revised: We use cross-entropy loss and optimize the parameters using Adam (Kingma & Ba, 2014) with the standard hyperparameters settings.

Original: 3. B LACK - BOX A TTACKS
Revised: 3. B LACK - BOX A TTACKS

Original: In our analysis, we consider six different adversarial attacks: Fast Gradient Sign Method (FGSM),
Revised: In our analysis, we consider six different adversarial attacks: Fast Gradient Sign Method (FGSM),

Original: Basic Iterative Method (BIM), Momentum Iterative Method (MIM), L 2 Carlini & Wagner (2017b) (C&W), Jacobian-based Saliency Map Approach (JSMA) and DeepFool (DF) (for each attack see
Revised: Basic Iterative Method (BIM), Momentum Iterative Method (MIM), L 2 Carlini & Wagner (2017b) (C&W), Jacobian-based Saliency Map Approach (JSMA) and DeepFool (DF) (for each attack see

Original: Goodfellow et al., 2014; Kurakin et al., 2016; Dong et al., 2018; Carlini & Wagner, 2017b; Papernot et al., 2016b; Moosavi-Dezfooli et al., 2016, respectively).
Revised: Goodfellow et al., 2014; Kurakin et al., 2016; Dong et al., 2018; Carlini & Wagner, 2017b; Papernot et al., 2016b; Moosavi-Dezfooli et al., 2016, respectively).

Original: We use the implementation as provided in cleverhans (Papernot et al., 2016a).
Revised: We use the implementation as provided in cleverhans (Papernot et al., 2016a).

Original: The hyper-parameters are summarized in Appendix B.
Revised: The hyper-parameters are summarized in Appendix B.

Original: 3. M ODEL - BASED S ALIENCY
Revised: 3. M ODEL - BASED S ALIENCY

Original: To generate saliency masks, we adapt the method used by Dabkowski & Gal (2017).
Revised: To generate saliency masks, we adapt the method used by Dabkowski & Gal (2017).

Original: Our reason is twofold: the technique computes high-quality saliency masks at a low computational cost.
Revised: Our reason is twofold: the technique computes high-quality saliency masks at a low computational cost.

Original: Dabkowski & Gal (2017) employ a U-Net with a novel loss function that targets SDR, SSR, mask sparsity and mask smoothness.
Revised: Dabkowski & Gal (2017) employ a U-Net with a novel loss function that targets SDR, SSR, mask sparsity, and mask smoothness.

Original: We adapt the original loss function to omit the total variational term, as mask smoothness is not required in our analysis.
Revised: We adapt the original loss function to omit the total variational term, as mask smoothness is not required in our analysis.

Original: Let f s = f s ( x, f c ( x )) denote the generated map.
Revised: Let f s = f s ( x, f c ( x )) denote the generated map.

Original: First, the map average AV ( f s ) is used to ensure that the area of the map is small.
Revised: First, the map average AV ( f s ) is used to ensure that the area of the map is small.

Original: Second, log( f c (Φ( x, f s ))) is included to ensure that the salient pixels sufﬁce to identify the correct class.
Revised: Second, log( f c (Φ( x, f s ))) is included to ensure that the salient pixels sufﬁce to identify the correct class.

Original: Finally, f c (Φ( x, 1 − f s )) is included to ensure that the classiﬁer can no longer recognize the class if the saliency map is removed.
Revised: Finally, f c (Φ( x, 1 − f s )) is included to ensure that the classiﬁer can no longer recognize the class if the saliency map is removed.

Original: Therefore, our saliency loss function is:
Revised: Therefore, our saliency loss function is:

Original: [Equation]
Revised: [Equation]

Original: where f c is the softmax probability of the class c , Φ( x, f s ) applies mask f s to image x , and λ i ≥are hyper-parameters.
Revised: where f c is the softmax probability of the class c , Φ( x, f s ) applies mask f s to image x , and λ i ≥are hyper-parameters.

Original: We adapt the PyTorch implementation provided by Dabkowski & Gal (2017) 5 and train the saliency model on standard, non-adversarial images only.
Revised: We adapt the PyTorch implementation provided by Dabkowski & Gal (2017) 6 and train the saliency model on standard, non-adversarial images only.

Original: For evaluation, we use the same saliency model for both natural and adversarial images.
Revised: For evaluation, we use the same saliency model for both natural and adversarial images.

Original: When generating the saliency maps for our images, we use the predicted classiﬁcation for feature selection to prevent an information leak (which would occur if we use the true label).
Revised: When generating the saliency maps for our images, we use the predicted classiﬁcation for feature selection to prevent an information leak (which would occur if we use the true label).

Original: 3. D EFENSE
Revised: 3. D EFENSE

Original: Our hypothesis is that if an image is adversarial, the classiﬁer likely focuses on the wrong aspects or the pixels on which it focuses are misleading (due to the perturbed color or intensity) when classifying an image as adversarial.
Revised: Our hypothesis is that if an image is adversarial, the classiﬁer likely focuses on the wrong aspects or the pixels on which it focuses are misleading (due to the perturbed color or intensity) when classifying an image as adversarial.

Original: We consider two different cases by building classiﬁers for (1) saliency maps and (2) salient pixels .
Revised: We consider two different cases by building classiﬁers for (1) saliency maps and (2) salient pixels .

Original: For both classiﬁers, we use the same architecture (and hyperparameters) as for the black-box image classiﬁers (as summarized in Appendix A).
Revised: For both classiﬁers, we use the same architecture (and hyperparameters) as for the black-box image classiﬁers (as summarized in Appendix A).

Original: 3.4. S ALIENCY M AP : “ THE CLASSIFIER FOCUSES ON THE WRONG PIXEL LOCATIONS ”
Revised: 3.4. S ALIENCY M AP : “ THE CLASSIFIER FOCUSES ON THE WRONG PIXEL LOCATIONS ”

Original: We build a detector based on the saliency maps of images as follows.
Revised: We build a detector based on the saliency maps of images as follows.

Original: First, we train a classiﬁer and generate adversarial images for every natural image in the training dataset.
Revised: First, we train a classiﬁer and generate adversarial images for every natural image in the training dataset.

Original: Then we generate the saliency maps for the clean data { f s ( X ) } and adversarial images { f s ( X adv ) } .
Revised: Then we generate the saliency maps for the clean data { f s ( X ) } and adversarial images { f s ( X adv ) } .

Original: We build a binary detector for the saliency maps, which predicts whether the corresponding image is adversarial or natural.
Revised: We build a binary detector for the saliency maps, which predicts whether the corresponding image is adversarial or natural.

Original: We abbreviate this defense as SMD ( S aliency M ap D efense).
Revised: We abbreviate this defense as SMD ( S aliency M ap D efense).

Original: We do not concatenate the saliency maps to the input image.
Revised: We do not concatenate the saliency maps to the input image.

Original: 3.4. S ALIENT P IXEL : “ THE PIXEL VALUES ARE WRONG ”
Revised: 3.4. S ALIENT P IXEL : “ THE PIXEL VALUES ARE WRONG ”

Original: We construct a second classiﬁer for the salient pixels .
Revised: We construct a second classiﬁer for the salient pixels .

Original: We follow the same steps as outlined in the previous section, aside from the ﬁnal step.
Revised: We follow the same steps as outlined in the previous section, aside from the ﬁnal step.

Original: We deﬁne the salient pixels as f s ( x ) · x , where x is the image, f s ( x ) is the saliency map corresponding to x and · denotes the element-wise product.
Revised: We deﬁne the salient pixels as f s ( x ) · x , where x is the image, f s ( x ) is the saliency map corresponding to x and · denotes the element-wise product.

Original: We abbreviate this defense as SPD ( S alient P ixel D efense).
Revised: We abbreviate this defense as SPD ( S alient P ixel D efense).

Original: Similarly to SMD, we do not concatenate the saliency maps to the input image.
Revised: Similarly to SMD, we do not concatenate the saliency maps to the input image.

Original: 3. B ENCHMARKS AND B ASELINES
Revised: 3. B ENCHMARKS AND B ASELINES

Original: To benchmark our results, we consider two baselines.
Revised: To benchmark our results, we consider two baselines.

Original: First, we train a baseline classiﬁer that classiﬁes input as adversarial or natural based on the images alone.
Revised: First, we train a baseline classiﬁer that classiﬁes input as adversarial or natural based on the images alone.

Original: This allows us to evaluate the added beneﬁt of using saliency maps.
Revised: This allows us to evaluate the added beneﬁt of using saliency maps.

Original: This method was implemented by Gong et al.
Revised: This method was implemented by Gong et al.

Original: We abbreviate this defense as ID ( I mage D efense).
Revised: We abbreviate this defense as ID ( I mage D efense).

Original: Second, we compare our defense method with the saliency-based defense of Zhang et al.
Revised: Second, we compare our defense method with the saliency-based defense of Zhang et al.

Original: (see
Revised: (see

Original: Section 2).
Revised: Section 2).

Original: We abbreviate this defense as JSD, for J acobian-based S aliency map D efense.
Revised: We abbreviate this defense as JSD, for J acobian-based S aliency map D efense.

Original: In our implementation, we adapt the method of Zhang et al.
Revised: In our implementation, we adapt the method of Zhang et al.

Original: (2018); we ﬁnd that if we use f s ( x ) = ∇ x f ( x ) as the saliency map it leads to underﬂow, resulting in a zero matrix.
Revised: (2018); we ﬁnd that if we use f s ( x ) = ∇ x f ( x ) as the saliency map it leads to underﬂow, resulting in a zero matrix.

Original: Therefore, instead we take the derivative with respect to the logits, i.e. f s ( x ) = ∇ x z ( x ) .
Revised: Therefore, instead we take the derivative with respect to the logits, i.e. f s ( x ) = ∇ x z ( x ) .

Original: JSD is mathematically related to the other defenses.
Revised: JSD is mathematically related to the other defenses.

Original: First, it is more general compared to ID: the ﬁlters of JSD can learn to ignore the Jacobian-based saliency, in which case the two methods are equivalent.
Revised: First, it is more general compared to ID: the ﬁlters of JSD can learn to ignore the Jacobian-based saliency, in which case the two methods are equivalent.

Original: Further, JSD is similar to SMD, as the ﬁlters can learn to ignore the image input.
Revised: Further, JSD is similar to SMD, as the ﬁlters can learn to ignore the image input.

Original: In this case, the only difference between JSD and SMD is that they use different techniques to derive saliency.
Revised: In this case, the only difference between JSD and SMD is that they use different techniques to derive saliency.

Original: However, JSD differs from SPD, as CNN ﬁlters cannot multiply one channel by another.
Revised: However, JSD differs from SPD, as CNN ﬁlters cannot multiply one channel by another.

Original: 3. E VALUATION WITH B LACK - BOX A TTACKS
Revised: 3. E VALUATION WITH B LACK - BOX A TTACKS

Original: We follow the evaluation protocol of Zhang et al.
Revised: We follow the evaluation protocol of Zhang et al.

Original: and train each defense to detect adversarial images generated by a speciﬁc attack, thereby generating six different detection models (one for each black-box attack).
Revised: and train each defense to detect adversarial images generated by a speciﬁc attack, thereby generating six different detection models (one for each black-box attack).

Original: To generate the training data, we generate one adversarial example for every clean image.
Revised: To generate the training data, we generate one adversarial example for every clean image.

Original: The training data becomes [ X, X adv ] , where X denotes the clean data and X adv denotes the adversarial data, and the labels are [ 1 n , 0 n ] , 1 n and 0 n are one- and zero-vectors of length n , respectively.
Revised: The training data becomes [ X, X adv ] , where X denotes the clean data and X adv denotes the adversarial data, and the labels are [ 1 n , 0 n ] , 1 n and 0 n are one- and zero-vectors of length n , respectively.

Original: We use the same training procedure and models, as summarized in Appendix
Revised: We use the same training procedure and models, as summarized in Appendix

Original: A and report the accuracy of the classiﬁers on the test dataset.
Revised: A, and report the accuracy of the classiﬁers on the test dataset.

Original: We compare the performance of the models on MNIST, CIFAR-10 and ASSIRA (see Burges &
Revised: We compare the performance of the models on MNIST, CIFAR-10 and ASSIRA (see Burges &

Original: Cortes, 1998; Krizhevsky et al., 2009; Elson et al., 2007, respectively).
Revised: Cortes, 1998; Krizhevsky et al., 2009; Elson et al., 2007, respectively).

Original: In addition to the two frequently used benchmarks, we consider the ASSIRA cats and dogs dataset 6 as it contains highquality images but is less computationally expensive than ImageNet.
Revised: In addition to the two frequently used benchmarks, we consider the ASSIRA cats and dogs dataset 7 as it contains highquality images but is less computationally expensive than ImageNet.

Original: 7 Further details on the datasets can be found in Appendix A.
Revised: 8 Further details on the datasets can be found in Appendix A.

Original: 3. W HITE - BOX A TTACK
Revised: 3. W HITE - BOX A TTACK

Original: Many defenses hold up against black-box attacks, but often are unable to defend against white-box attacks (Carlini & Wagner, 2017a).
Revised: Many defenses hold up against black-box attacks, but often are unable to defend against white-box attacks (Carlini & Wagner, 2017a).

Original: For this reason, we generate white-box attacks tailored to the defense strategy.
Revised: For this reason, we generate white-box attacks tailored to the defense strategy.

Original: Our white-box attacks are iterative gradient-based attacks, which target both the classiﬁer and the defense.
Revised: Our white-box attacks are iterative gradient-based attacks, which target both the classiﬁer and the defense.

Original: Inspired by FGSM, we can target the classiﬁer f as
Revised: Inspired by FGSM, we can target the classiﬁer f as

Original: [Equation]
Revised: [Equation]

Original: and the defense d as
Revised: and the defense d as

Original: [Equation]
Revised: [Equation]

Original: where Clip clips the pixels to the pre-deﬁned maximum range.
Revised: where Clip clips the pixels to the pre-deﬁned maximum range.

Original: Using the above idea, we iterate between Equations 3 and 4 to generate the white-box attack for ID (the defense based on image
Revised: Using the above idea, we iterate between Equations 3 and 4 to generate the white-box attack for ID (the defense based on image

Original: In particular, JSMA would have required substantial time; other researchers have omitted this attack on ImageNet due to the high computational cost which results from the high resolution of the images (Carlini &
Revised: In particular, JSMA would have required substantial time; other researchers have omitted this attack on ImageNet due to the high computational cost which results from the high resolution of the images (Carlini &

Original: Wagner, 2017b).
Revised: Wagner, 2017b).

Original: classiﬁcation).
Revised: classiﬁcation).

Original: We propose similar white-box attacks for the other defenses, as shown in Appendix
Revised: We propose similar white-box attacks for the other defenses, as shown in Appendix

Original: C. We limit the number of iterations T to 5 , as we ﬁnd it to be adequate to generate a sufﬁciently strong attack and further increasing T does not improve the performance.
Revised: C. We limit the number of iterations T to 5 , as we ﬁnd it to be adequate to generate a sufﬁciently strong attack and further increasing T does not improve the performance.

Original: Our method is similar to that of Metzen et al.
Revised: Our method is similar to that of Metzen et al.

Original: They propose ﬁnding adversarial examples as:
Revised: They propose ﬁnding adversarial examples as:

Original: [Equation]
Revised: [Equation]

Original: where in our case α = 0 . 5 .
Revised: where in our case α = 0 . 5 .

Original: The key difference is that we iterate between Equations 3 and 4, rather than applying 3 and 4 simultaneously.
Revised: The key difference is that we iterate between Equations 3 and 4, rather than applying 3 and 4 simultaneously.

Original: We ﬁnd that this is more effective at targeting the defense, which is more difﬁcult to fool than the original classiﬁer.
Revised: We ﬁnd that this is more effective at targeting the defense, which is more difﬁcult to fool than the original classiﬁer.

Original:  D EFENSE R ESULTS
Revised:  D EFENSE R ESULTS

Original: We start by assessing the shift in saliency maps generated by adversarial perturbations and then present the efﬁcacy of the detector against different adversarial attacks.
Revised: We start by assessing the shift in saliency maps generated by adversarial perturbations and then present the efﬁcacy of the detector against different adversarial attacks.

Original: Details such as attack success rate can be found in Appendix B.
Revised: Details such as attack success rate can be found in Appendix B.

Original: 4. L 2 DISTANCES BETWEEN SALIENCY MAPS
Revised: 4. L 2 DISTANCES BETWEEN SALIENCY MAPS

Original: We start by quantifying the shift in saliency maps due to adversarial perturbations; we compute the
Revised: We start by quantifying the shift in saliency maps due to adversarial perturbations; we compute the

Original: L 2 distance between saliency maps of a natural image and its corresponding adversarial image.
Revised: L 2 distance between saliency maps of a natural image and its corresponding adversarial image.

Original: As a baseline, we compare these values with the L 2 distance between two different natural images.
Revised: As a baseline, we compare these values with the L 2 distance between two different natural images.

Original: These statistics are summarized in Table 5.
Revised: These statistics are summarized in Table 5.

Original: For CIFAR-10 and ASSIRA, the L 2 -norm between the saliency maps of a natural image and its corresponding adversarial image is comparable to or larger than the L 2 distance between two different natural images.
Revised: For CIFAR-10 and ASSIRA, the L 2 -norm between the saliency maps of a natural image and its corresponding adversarial image is comparable to or larger than the L 2 distance between two different natural images.

Original: Using a Mann-Whitney U-test, we prove quantitatively that the shift is signiﬁcant for most adversarial attacks on CIFAR-10 and ASSIRA images.
Revised: Using a Mann-Whitney U-test, we prove quantitatively that the shift is signiﬁcant for most adversarial attacks on CIFAR-10 and ASSIRA images.

Original: This suggests that our saliency-based method is an effective way of capturing adversarial perturbations.
Revised: This suggests that our saliency-based method is an effective way of capturing adversarial perturbations.

Original: [Table]
Revised: [Table]

Original: 4. B LACK - BOX D EFENSE
Revised: 4. B LACK - BOX D EFENSE

Original: Figure 2 summarizes the performance of the defense models trained on a single adversarial attack on different adversarial attacks; the values and standard deviations can found in the Appendix.
Revised: Figure 2 summarizes the performance of the defense models trained on a single adversarial attack on different adversarial attacks; the values and standard deviations can found in the Appendix.

Original: The overall performance of the model-based saliency defense suggests that saliency can be used to determine whether an image is adversarial.
Revised: The overall performance of the model-based saliency defense suggests that saliency can be used to determine whether an image is adversarial.

Original: Salient Pixel Defense outperforms Saliency Map Defense Overall, SPD (shown in blue) outperforms the other defenses, suggesting the pixels values provide useful information to the detector.
Revised: Salient Pixel Defense outperforms Saliency Map Defense Overall, SPD (shown in blue) outperforms the other defenses, suggesting that salient pixels values provide useful information to the detector.

Original: Our defense generalizes well: even when trained on a weaker attack, SPD is able to detect stronger 
Revised: Further, our defense generalizes well: even when trained on a weaker attack, SPD is able to detect stronger attacks.

Revised: Both baseline methods, ID and JSD, only generalize well when trained on a stronger attack.

Revised: When trained on a weaker attack, they are not able to detect stronger adversarial attacks.

Original: FGSM
Revised: FGSM

Original: BIM
Revised: BIM

Original: MIM
Revised: MIM

Original: C & W
Revised: C & W

Original: DF
Revised: DF

Original: JSMA
Revised: JSMA

Original: [Table]
Revised: M N I S T

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: FGSM [Table] MIM CW DF JSMA Adversarial [Table]
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure] [Table] [Figure] [Table]
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original:  e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: A SS
Revised: A SS

Original: I R A
Revised: I R A

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.
Revised: FGSM BIM MIM CW DF JSMA Adversarial Attack 0.

Original: D e t e c t i o n A cc u r a c y
Revised: D e t e c t i o n A cc u r a c y

Original: SPDSMDIDJSD
Revised: SPDSMDIDJSD

Original: Jacobian-based Saliency is Uninformative The method introduced by Zhang et al.
Revised: Jacobian-based Saliency is Uninformative The method introduced by Zhang et al.

Original: (2018) does not perform well. In particular, we ﬁnd that the (Jacobian) saliency mask  does not improve on an image-based defense.
Revised: (2018) does not perform well – the (Jacobian) saliency mask defense does not improve on an image-based defense.

Original:  A congruent observation was made qualitatively by Gu & Tresp (2019), who show that the changes in gradient due to adversarial perturbations are imperceptible (for an example, see Figures 7 and 8 in Appendix D).
Revised: 9 A congruent observation was made qualitatively by Gu & Tresp (2019), who show that the changes in gradient due to adversarial perturbations are imperceptible (for an example, see Figures 7 and 8 in Appendix D).

Original: Our quantitative results build on their observation, showing that not all saliency methods show shifts due to adversarial perturbations.
Revised: Our quantitative results build on their observation, showing that not all saliency methods show shifts due to adversarial perturbations.

Original: Worse generalization on JSMA Wedo observe a drop in performance of the models when detecting JSMA, likely because JSMA is an L 0 -norm attack which generates a different type of adversarial examples.
Revised: Worse generalization on JSMA We observe a drop in performance of the models when detecting JSMA, likely because JSMA is an L 0 -norm attack, which generates a different type of adversarial examples.

Original: This may suggest that defense trained on a speciﬁc norm, only generalize well to other attacks generated by a norm that produces similar perturbations.
Revised: This may suggest that defenses trained on a speciﬁc norm, only generalize well to other attacks generated by a norm that produces similar perturbations.

Original: FGSM, BIM and MIM are L ∞ - norm attacks and C &W and DF are L 2 -norm attacks.
Revised: FGSM, BIM, and MIM are L ∞ - norm attacks, and C &W and DF are L 2 -norm attacks.

Original: Both generate perturbations which are spread out over the entire image- contrary to L 0 norm attacks which change  few pixels but using larger perturbations.
Revised: Both generate perturbations that are spread out over the entire image, contrary to L 0 norm attacks, which change a few pixels but using larger perturbations.

Original: Threshold Behavior Both ID and JSD exhibit threshold behavior : they are unable to detect adversarial examples if the perturbation size is below a given threshold.
Revised: Threshold Behavior Both ID and JSD exhibit threshold behavior : they are unable to detect adversarial examples if the perturbation size is below a given threshold.

Original: For example, see the performance of both defenses on the ASSIRA dataset.
Revised: For example, see the performance of both defenses on the ASSIRA dataset.

Original: There is a strong correlation between detection accuracy and perturbation size, as measured by L ∞ and L 2 (see Table 2).
Revised: There is a strong correlation between detection accuracy and perturbation size, as measured by L ∞ and L 2 (see Table 2).

Original: ID is able to detect all adversarial images for which the perturbation size is either L 2 > 0 .
Revised: ID is able to detect all adversarial images for which the perturbation size is either L 2 > 0 .

Original: 027 or L ∞ > 0 .
Revised: 027 or L ∞ > 0 .

Original: 50 , such as FGSM and JSMA.
Revised: 50 , such as FGSM and JSMA.

Original: However, the perturbations are much smaller for DF and CW, making these attacks harder to detect.
Revised: However, the perturbations are much smaller for DF and CW, making these attacks harder to detect.

Original: The threshold appears to occur around L 2 = 0 .
Revised: The threshold appears to occur around L 2 = 0 .

Original: 025 , as ID can sometimes detect the
Revised: 025 , as ID can sometimes detect the

Original: FGSM perturbations, generated with this size.
Revised: FGSM perturbations, generated with this size.

Original: This observation is in line with the results of Gong et al.
Revised: This observation is in line with the results of Gong et al.

Original: (2017), who ﬁnd that ID is highly efﬁcient at detecting adversarial images with perturbations of ε ≥ 0 .
Revised: (2017), who ﬁnd that ID is highly efﬁcient at detecting adversarial images with perturbations of ε ≥ 0 .

Original: 03 but unable to detect adversarial perturbations generated using ε = 0 .
Revised: 03 but unable to detect adversarial perturbations generated using ε = 0 .

Original: 01 (using FGSM for images scaled between 0 and 1 ), obtaining an accuracy of 50 . 0% in the latter case.
Revised: 01 (using FGSM for images scaled between 0 and 1 ), obtaining an accuracy of 50 . 0% in the latter case.

Original: Entries correspond to MNIST/CIFAR10/ASSIRA .
Revised: Entries correspond to MNIST/CIFAR10/ASSIRA .

Original: Entries for which || r || 2 ≥ 0 .
Revised: Entries for which || r || 2 ≥ 0 .

Original: 025 and || r || ∞ ≥ 0 .
Revised: 025 and || r || ∞ ≥ 0 .

Original: 5 are in bold, where r is the adversarial perturbation.
Revised: 5 are in bold, where r is the adversarial perturbation.

Original: m is the number of pixels in an image.
Revised: m is the number of pixels in an image.

Original: A TTACK

Original: [Table]
Revised: [Table]

Original: 4. W HITE -B OX D EFENSE
Revised: 4. W HITE -B OX D EFENSE

Original: Table 3 summarizes the performance of different defenses against our white-box attack.
Revised: Table 3 summarizes the performance of different defenses against our white-box attack.

Original: Our whitebox methods are highly effective in fooling the classiﬁer as well as the defenses for MNIST and
Revised: Our whitebox methods are highly effective in fooling the classiﬁer as well as the defenses for MNIST and

Original: ASSIRA, as shown by the before adversarial training results.
Revised: ASSIRA, as shown by the before adversarial training results.

Original: The white-box attack is unable to fool the detector for CIFAR-10 successfully.
Revised: The white-box attack is unable to fool the detector for CIFAR-10 successfully.

Original: Next, we perform adversarial training: we iteratively train the detectors against the white-box attack and allow the white-box attack access to the new defense.
Revised: Next, we perform adversarial training: we iteratively train the detectors against the white-box attack and allow the white-box attack access to the new defense.

Original: The white-box attack no longer successfully defeats SPD, which becomes more robust against the attack, whereas SMD is not able to become robust against the white-box attack.
Revised: The white-box attack no longer successfully defeats SPD, which becomes more robust against the attack, whereas SMD is not able to become robust against the white-box attack.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In our analysis
Revised: In our analysis

Original: [Table]
Revised: [Table]

Original: we ascertain that the saliency maps of adversarial images differ from those of natural images.
Revised: we ascertain that the saliency maps of adversarial images differ from those of natural images.

Original: Further, we show that salient pixel based defenses perform better than a saliency map defense.
Revised: Further, we show that salient pixel based defenses perform better than a saliency map defense.

Original: When trained on a single black-box attack, our method is able to detect adversarial perturbations generated by different and stronger attacks.
Revised: When trained on a single black-box attack, our method is able to detect adversarial perturbations generated by different and stronger attacks.

Original: We show that gradients are unable to capture shifts in saliency due to adversarial perturbations and present an alternative adversarial defense using learnt saliency models that is effective against both black -box and white-box attacks.
Revised: We show that gradients are unable to capture shifts in saliency due to adversarial perturbations and present an alternative adversarial defense using learnt saliency models that is effective against both black-box and white-box attacks.

Original: Building on the work of Gong et al
Revised: Building on the work of Gong et al.

Original: ), we further establish the notion of threshold behavior , showing that the trend depends on the L 2 and L ∞ - norms of the perturbations and therefore also prevails when using other methods (JSD) and across different attacks.
Revised: (2017), we further establish the notion of threshold behavior , showing that the trend depends on the L 2 and L ∞ - norms of the perturbations and therefore also prevails when using other methods (JSD) and across different attacks.

Original: Future work could extend on further investigating the performance of the defense in different applications.
Revised: Future work could extend on further investigating the performance of the defense in different applications.

Original: For example, as our method runs in real-time, it could be used to detect adversarial perturbations in video to counter recent attacks (Li et al., 2018; Jiang et al., 2019).
Revised: For example, as our method runs in real-time, it could be used to detect adversarial perturbations in video to counter recent attacks (Li et al., 2018; Jiang et al., 2019).

Original: A A RCHITECTURES , H YPER - PARAMETERS AND D ATA
Revised: A A RCHITECTURES , H YPER - PARAMETERS AND D ATA

Original: [Figure]
Revised: [Figure]

Original: P ARAMETERS 2 X CONV LAYER f
Revised: P ARAMETERS 2 X CONV LAYER f

Original: = 64 , k =
Revised: = 64 , k =

Original: MAXPOOLING p = 2 , s =2 X CONV LAYER f = 128 , k =
Revised: MAXPOOLING p = 2 , s =2 X CONV LAYER f = 128 , k =

Original: MAXPOOLING p = 2 , s =2 X CONV LAYER f
Revised: MAXPOOLING p = 2 , s =2 X CONV LAYER f

Original: = 256 , k =
Revised: = 256 , k =

Original: MAXPOOLING p = 2 , s =2 X CONV LAYER f = 512 , k
Revised: MAXPOOLING p = 2 , s =2 X CONV LAYER f = 512 , k

Original: =
Revised: =

Original: MAXPOOLING p = 2 , s =
Revised: MAXPOOLING p = 2 , s =

Original: FLATTENING LAYERDENSE n =
Revised: FLATTENING LAYERDENSE n =

Original: DENSE n = DENSE n =
Revised: DENSE n = DENSE n =

Original: MNIST
Revised: MNIST

Original: L AYER
Revised: L AYER

Original: P ARAMETERS 2 X CONV LAYER f
Revised: P ARAMETERS 2 X CONV LAYER f

Original: = 64 , k
Revised: = 64 , k

Original: =2 X CONV LAYER f = 128 , k =2 X CONV LAYER f = 128 , k =
Revised: =2 X CONV LAYER f = 128 , k =2 X CONV LAYER f = 128 , k =

Original: FLATTENING LAYERDENSE n =
Revised: FLATTENING LAYERDENSE n =

Original: CIFARL AYER
Revised: CIFARL AYER

Original: P ARAMETERS 2 X CONV LAYER f
Revised: P ARAMETERS 2 X CONV LAYER f

Original: = 32 , k =2 X CONV LAYER f
Revised: = 32 , k =2 X CONV LAYER f

Original: = 32 , k =
Revised: = 32 , k =

Original: MAXPOOLING p = 2 , s =2 X CONV LAYER f
Revised: MAXPOOLING p = 2 , s =2 X CONV LAYER f

Original: = 63 , k =2 X CONV LAYER f
Revised: = 63 , k =2 X CONV LAYER f

Original: = 63 , k =
Revised: = 63 , k =

Original: MAXPOOLING p = 2 , s =2 X CONV LAYER f = 128 ,
Revised: MAXPOOLING p = 2 , s =2 X CONV LAYER f = 128 ,

Original: k
Revised: k

Original: =2 X CONV LAYER f = 128 , k =
Revised: =2 X CONV LAYER f = 128 , k =

Original: MAXPOOLING p = 2 , s =
Revised: MAXPOOLING p = 2 , s =

Original: FLATTENING LAYERDENSE n =
Revised: FLATTENING LAYERDENSE n =

Original: DENSE n =
Revised: DENSE n =

Original: We apply drop-out before every dense layer.
Revised: We apply drop-out before every dense layer.

Original: Using a validation set, we experimented with different drop-out rates between 0 . 3 and 0 . 7 and found that the rate δ = 0 .
Revised: Using a validation set, we experimented with different drop-out rates between 0 . 3 and 0 . 7 and found that the rate δ = 0 .

Original: 6 was optimal.
Revised: 6 was optimal.

Original: We use a ReLu activation for the penultimates layers and a softmax activation for the ﬁnal layer.
Revised: We use a ReLu activation for the penultimates layers and a softmax activation for the ﬁnal layer.

Original: We train the model for 10 epochs on batches of size 50 .
Revised: We train the model for 10 epochs on batches of size 50 .

Original: We compare the performance of the models on MNIST, CIFAR-10 and ASSIRA (see Burges &
Revised: We compare the performance of the models on MNIST, CIFAR-10 and ASSIRA (see Burges &

Original: Cortes, 1998; Krizhevsky et al., 2009; Elson et al., 2007, respectively).
Revised: Cortes, 1998; Krizhevsky et al., 2009; Elson et al., 2007, respectively).

Original: For MNIST and CIFAR-10, we use the standard train and test splits, and for ASSIRA, we use 3 , 000 images.
Revised: For MNIST and CIFAR-10, we use the standard train and test splits, and for ASSIRA, we use 3 , 000 images.

Original: We use 10% of the training data for the validation set, and re-train on the full training dataset once hyper-parameters were selected.
Revised: We use 10% of the training data for the validation set, and re-train on the full training dataset once hyper-parameters were selected.

Original: Further experimentation of ID and JSD architecture We further experiment with the architectures of ID and JSD to determine whether the observed performance was the result of the architecture.
Revised: Further experimentation of ID and JSD architecture We further experiment with the architectures of ID and JSD to determine whether the observed performance was the result of the architecture.

Original: In particular, we considered the adjustments as summarized in Table A; however, we found that the changes did not improve performance.
Revised: In particular, we considered the adjustments as summarized in Table A; however, we found that the changes did not improve performance.

Original: A RCHITECTURE E XPERIMENTATION
Revised: A RCHITECTURE E XPERIMENTATION

Original: [Table]
Revised: [Table]

Original: ARAMETER
Revised: ARAMETER

Original: [Table]
Revised: [Table]

Original: AT
Revised: AT

Original: [Table] [Table]
Revised: [Table] [Table]

Original: CONSTANT
Revised: CONSTANT

Original: [Table]
Revised: [Table]

Original: I NCLUSION
Revised: I NCLUSION

Original: [Table]
Revised: [Table]

Original: B A DVERSARIAL A TTACKS
Revised: B A DVERSARIAL A TTACKS

Original: In this section, we present the black-box adversarial attack hyper-parameters (see Figure 4), the success rates of the different adversarial attacks (see Table 4) and an example of an adversarial image generated by the various black-box attacks (see Figure 5).
Revised: In this section, we present the black-box adversarial attack hyper-parameters (see Figure 4), the success rates of the different adversarial attacks (see Table 4) and an example of an adversarial image generated by the various black-box attacks (see Figure 5).

Original: ASSIRA
Revised: ASSIRA

Original: Attack
Revised: Attack

Original: Norm
Revised: Norm

Original: Hyper-parameter
Revised: Hyper-parameter

Original: FGSM
Revised: FGSM

Original: L ∞ ε = 0 .
Revised: L ∞ ε = 0 .

Original: BIM
Revised: BIM

Original: L ∞ ε = 0 .
Revised: L ∞ ε = 0 .

Original: 025 , ε i = 0 .
Revised: 025 , ε i = 0 .

Original: MIM
Revised: MIM

Original: L ∞ ε = 0 .
Revised: L ∞ ε = 0 .

Original: 025 , ε i = 0 .
Revised: 025 , ε i = 0 .

Original: C & W
Revised: C & W

Original: Lbinary search steps =
Revised: Lbinary search steps =

Original: DF
Revised: DF

Original: Lmax.
Revised: Lmax.

Original: iterations =
Revised: iterations =

Original: JSMA
Revised: JSMA

Original: N.A. ε = 0 .
Revised: N.A. ε = 0 .

Original: MNIST and CIFARAttack
Revised: MNIST and CIFARAttack

Original: Norm
Revised: Norm

Original: Hyper-parameter
Revised: Hyper-parameter

Original: FGSM
Revised: FGSM

Original: L ∞ ε = 0 .
Revised: L ∞ ε = 0 .

Original: BIM
Revised: BIM

Original: L ∞ ε = 0 .
Revised: L ∞ ε = 0 .

Original: 15 , ε i = 0 .
Revised: 15 , ε i = 0 .

Original: MIM
Revised: MIM

Original: L ∞ ε = 0 .
Revised: L ∞ ε = 0 .

Original: 15 , ε i = 0 .
Revised: 15 , ε i = 0 .

Original: C & W
Revised: C & W

Original: Lbinary search steps =
Revised: Lbinary search steps =

Original: DF
Revised: DF

Original: Lmax.
Revised: Lmax.

Original: iterations =
Revised: iterations =

Original: JSMA
Revised: JSMA

Original: N.A. ε = 0 .
Revised: N.A. ε = 0 .

Original: A TTACK
Revised: A TTACK

Original: A CCURACY
Revised: A CCURACY

Original: B ASELINE
Revised: B ASELINE

Original: [Table]
Revised: [Table]

Original: FGSM
Revised: FGSM

Original: [Table]
Revised: [Table]

Original: BIM
Revised: BIM

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: Further, we observe that the perturbations of
Revised: Further, we observe that the perturbations of

Original: , and are more visible than those of
Revised: , and are more visible than those of

Original: C & W and DF.
Revised: C & W and DF.

Original: C
Revised: C

Original: W HITE - BOX A TTACKS P SEUDO - CODE
Revised: W HITE - BOX A TTACKS P SEUDO - CODE

Original: 1: x adv ← x 2: for t = 0 : T do 3: for j = 1 : n do 4: if x adv does not fool the classiﬁer then 5: x adv ← x adv + ε sign ( ∇ f ( x adv , y )) 6: end if 7: if x adv does not fool the detector then 8: s adv ← ∇ f ( x adv ) 9: [ x adv , s adv ] ← [ x adv , s adv ] + ε sign ( ∇ d ([ x adv , s adv ] , y )) 10: x adv ← Clip ( x adv ) 11: end if 12: end for 13: end for 14: return x adv
Revised: 1: x adv ← x 2: for t = 0 : T do 3: for j = 1 : n do 4: if x adv does not fool the classiﬁer then 5: x adv ← x adv + ε sign ( ∇ f ( x adv , y )) 6: end if 7: if x adv does not fool the detector then 8: s adv ← ∇ f ( x adv ) 9: [ x adv , s adv ] ← [ x adv , s adv ] + ε sign ( ∇ d ([ x adv , s adv ] , y )) 10: x adv ← Clip ( x adv ) 11: end if 12: end for 13: end for 14: return x adv

Original: Algorithm 1 provides the white-box attack for JSD.
Revised: Algorithm 1 provides the white-box attack for JSD.

Original: As mentioned in Section 2, JSD concatenates the image with its saliency map (computed as the Jacobian) and uses this as an input to the classiﬁer.
Revised: As mentioned in Section 2, JSD concatenates the image with its saliency map (computed as the Jacobian) and uses this as an input to the classiﬁer.

Original: Algorithms 2 and 3 provide the white-box attacks for our defenses: SPD and SMD.
Revised: Algorithms 2 and 3 provide the white-box attacks for our defenses: SPD and SMD.

Original: The function f s corresponds to generating the saliency map using the method introduced by Dabkowski & Gal (2017).
Revised: The function f s corresponds to generating the saliency map using the method introduced by Dabkowski & Gal (2017).

Original: Their method returns a two-dimensional saliency map.
Revised: Their method returns a two-dimensional saliency map.

Original: However, as the image is three dimensions, we expand the last dimension and stack the map to match the number of channels ( n c ) of the image.
Revised: However, as the image is three dimensions, we expand the last dimension and stack the map to match the number of channels ( n c ) of the image.

Original: In doing so, we assume that the saliency is constant along depth.
Revised: In doing so, we assume that the saliency is constant along depth.

Original: 1: x adv ← x 2: for j = 1 : n do 3: for t = 1 : T do 4: if x adv does not fool the classiﬁer then 5: x adv ← x adv + ε sign ( ∇ f ( x adv , y )) 6: end if 7: s adv ← f s ( x adv ) 8: sp ( x adv ) ← x adv · s adv 9: if sp ( x adv ) does not fool the detector then 10: r ← ε sign ( ∇ d ( sp ( x adv ) , y ) 11: if n c > 1 , repeat r along the last dimension until it matches n c 12: x adv ← Clip ( x adv + r ) 13: end if 14: end for 15: end for 16: return x adv
Revised: 1: x adv ← x 2: for j = 1 : n do 3: for t = 1 : T do 4: if x adv does not fool the classiﬁer then 5: x adv ← x adv + ε sign ( ∇ f ( x adv , y )) 6: end if 7: s adv ← f s ( x adv ) 8: sp ( x adv ) ← x adv · s adv 9: if sp ( x adv ) does not fool the detector then 10: r ← ε sign ( ∇ d ( sp ( x adv ) , y ) 11: if n c > 1 , repeat r along the last dimension until it matches n c 12: x adv ← Clip ( x adv + r ) 13: end if 14: end for 15: end for 16: return x adv

Original: 1: x adv ← x 2: for j = 1 : n do 3: for t = 1 : T do 4: if x adv does not fool the classiﬁer then 5: x adv ← x adv + ε sign ( ∇ f ( x adv , y )) 6: end if 7: s adv ← f s ( x adv ) 8: if s adv does not fool the detector then 9: r ← ε sign ( ∇ d ( s adv , y ))
Revised: 1: x adv ← x 2: for j = 1 : n do 3: for t = 1 : T do 4: if x adv does not fool the classiﬁer then 5: x adv ← x adv + ε sign ( ∇ f ( x adv , y )) 6: end if 7: s adv ← f s ( x adv ) 8: if s adv does not fool the detector then 9: r ← ε sign ( ∇ d ( s adv , y ))

Original: 10: if n c > 1 , repeat r along the last dimension until it matches n c 11: x adv ← Clip ( x adv + r ) 12: end if 13: end for 14: end for 15: return x adv
Revised: 10: if n c > 1 , repeat r along the last dimension until it matches n c 11: x adv ← Clip ( x adv + r ) 12: end if 13: end for 14: end for 15: return x adv

Original: [Figure]
Revised: [Figure]

Original: FGSM
Revised: FGSM

Original: BIM
Revised: BIM

Original: MIM
Revised: MIM

Original: JSMA
Revised: JSMA

Original: C & W
Revised: C & W

Original: DF
Revised: DF

Original: [Figure]
Revised: [Figure]

Original: FGSM
Revised: FGSM

Original: BIM
Revised: BIM

Original: MIM
Revised: MIM

Original: JSMA
Revised: JSMA

Original: C & W
Revised: C & W

Original: DF
Revised: DF

Original: Original
Revised: Original

Original: FGSM
Revised: FGSM

Original: BIM
Revised: BIM

Original: MIM
Revised: MIM

Original: JSMA
Revised: JSMA

Original: C & W
Revised: C & W

Original: DF
Revised: DF

Original: E L 2 D ISTANCES BETWEEN S ALIENCY M APS CORRESPONDING TO A DVERSARIAL I MAGES GENERATED BY D IFFERENT A TTACKS
Revised: E L 2 D ISTANCES BETWEEN S ALIENCY M APS CORRESPONDING TO A DVERSARIAL I MAGES GENERATED BY D IFFERENT A TTACKS

Original: [Table]
Revised: [Table]

Original: F S INGLE BLACK - BOX ADVERSARIAL ATTACK DETECTOR
Revised: F S INGLE BLACK - BOX ADVERSARIAL ATTACK DETECTOR

Original: Table 6 summarizes the accuracies of the different defenses when training a single detector against a combination of different types of black-box attacks.
Revised: Table 6 summarizes the accuracies of the different defenses when training a single detector against a combination of different types of black-box attacks.

Original: All methods perform relatively similarly as when trained against a single defense, obtaining slightly worse performances than when trained against a speciﬁc adversarial attack.
Revised: All methods perform relatively similarly as when trained against a single defense, obtaining slightly worse performances than when trained against a speciﬁc adversarial attack.

Original: This is useful in practice when it is unclear which adversarial attack is used.
Revised: This is useful in practice when it is unclear which adversarial attack is used.

Original: [Table]
Revised: [Table]

Original: G
Revised: G

Original: B L
Revised: B L

Original: A C
Revised: A C

Original: K - B
Revised: K - B

Original: OX
Revised: OX

Original: ADV
Revised: ADV

Original: E R S
Revised: E R S

Original: A R
Revised: A R

Original: I
Revised: I

Original: A L
Revised: A L

Original: A TT
Revised: A TT

Original: A C
Revised: A C

Original: K
Revised: K

Original: R E S
Revised: R E S

Original: U LT
Revised: U LT

Original: S
Revised: S

Original: G .
Revised: G .

Original: M N
Revised: M N

Original: I S T
Revised: I S T

Original: T a b l e 7 :
Revised: T a b l e 7 :

Original: T h e p e rf
Revised: T h e p e rf

Original: o r m a n ce o f t
Revised: o r m a n ce o f t

Original: h e d i ff e r e n t d e f e n s e s t
Revised: h e d i ff e r e n t d e f e n s e s t

Original: r
Revised: r

Original: a
Revised: a

Original: i n e d on a s i ng l e b l ac kbox a tt ac k on d e t ec ti ng d i ff e r e n t b l ac kbox a tt ac k s .
Revised: i n e d on a s i ng l e b l ac kbox a tt ac k on d e t ec ti ng d i ff e r e n t b l ac kbox a tt ac k s .

Original: T h
Revised: T h

Original: e e n t r i e s c
Revised: e e n t r i e s c

Original: o rre s p o nd t
Revised: o rre s p o nd t

Original: o t
Revised: o t

Original: h e acc u r ac
Revised: h e acc u r ac

Original: y t
Revised: y t

Original: h e d e f e n s e s
Revised: h e d e f e n s e s

Original: ( I D /
Revised: ( I D /

Original: J S D / S P
Revised: J S D / S P

Original: D / S
Revised: D / S

Original: M D ) ob t a i n i n d
Revised: M D ) ob t a i n i n d

Original: i s ti
Revised: i s ti

Original: ngu
Revised: ngu

Original: i s h
Revised: i s h

Original: i ng a dv e r s a r i a l e x a m p l e s fr o m n a t
Revised: i ng a dv e r s a r i a l e x a m p l e s fr o m n a t

Original: u
Revised: u

Original: r a l i m a g e s .
Revised: r a l i m a g e s .

Original: T h e v a l u e s i n p a r e n t h e
Revised: T h e v a l u e s i n p a r e n t h e

Original: s e s b e l o w
Revised: s e s b e l o w

Original: [Table]
Revised: [Table]

Original: no t e
Revised: no t e

Original: [Table]
Revised: [Table]

Original: nd
Revised: nd

Original: [Table]
Revised: [Table]

Original: d e v i a ti on s .
Revised: d e v i a ti on s .

Original: A TT
Revised: A TT

Original: AC
Revised: AC

Original: K
Revised: K

Original: D E F
Revised: D E F

Original: E N S
Revised: E N S

Original: E
Revised: E

Original: F G S
Revised: F G S

Original: M
Revised: M

Original: B
Revised: B

Original: I M M I M
Revised: I M M I M

Original: C &
Revised: C &

Original: W
Revised: W

Original: D F
Revised: D F

Original: J S M
Revised: J S M

Original: A
Revised: A

Original: F G S
Revised: F G S

Original: M 1 ./ 1 ./ 0 ./ 1 .1 ./
Revised: M 1 ./ 1 ./ 0 ./ 1 .1 ./

Original: B I M 1 ./ 1 ./ 1 ./ 0 .1 ./
Revised: B I M 1 ./ 1 ./ 1 ./ 0 .1 ./

Original: M I M 1 ./ 1 ./ 1 ./ 1 .1 ./
Revised: M I M 1 ./ 1 ./ 1 ./ 1 .1 ./

Original: D F 1 ./ 1 ./ 0 ./ 0 .1 ./
Revised: D F 1 ./ 1 ./ 0 ./ 0 .1 ./

Original: C W 1 ./ 1 ./ 0 ./ 0 .1 ./
Revised: C W 1 ./ 1 ./ 0 ./ 0 .1 ./

Original: S A L 0 ./ 

Revised: S A L 0 ./

Original: G .
Revised: G .

Original: C
Revised: C

Original: I F
Revised: I F

Original: A RT a b l e 8 :
Revised: A RT a b l e 8 :

Original: T h e p e rf
Revised: T h e p e rf

Original: o r m a n ce o f t
Revised: o r m a n ce o f t

Original: h e d i ff e r e n t d e f e n s e s t
Revised: h e d i ff e r e n t d e f e n s e s t

Original: r
Revised: r

Original: a
Revised: a

Original: i n e d on a s i ng l e b l ac kbox a tt ac k on d e t ec ti ng d i ff e r e n t b l ac kbox a tt ac k s .
Revised: i n e d on a s i ng l e b l ac kbox a tt ac k on d e t ec ti ng d i ff e r e n t b l ac kbox a tt ac k s .

Original: T h
Revised: T h

Original: e e n t r i e s c
Revised: e e n t r i e s c

Original: o rre s p o nd t
Revised: o rre s p o nd t

Original: o t
Revised: o t

Original: h e acc u r ac
Revised: h e acc u r ac

Original: y t
Revised: y t

Original: h e d e f e n s e s
Revised: h e d e f e n s e s

Original: ( I D /
Revised: ( I D /

Original: J S D / S P
Revised: J S D / S P

Original: D / S
Revised: D / S

Original: M D ) ob t a i n i n d
Revised: M D ) ob t a i n i n d

Original: i s ti
Revised: i s ti

Original: ngu
Revised: ngu

Original: i s h
Revised: i s h

Original: i ng a dv e r s a r i a l e x a m p l e s fr o m n a t
Revised: i ng a dv e r s a r i a l e x a m p l e s fr o m n a t

Original: u
Revised: u

Original: r a l i m a g e s .
Revised: r a l i m a g e s .

Original: T h e v a l u e s i n p a r e n t h e
Revised: T h e v a l u e s i n p a r e n t h e

Original: s e s b e l o w d
Revised: s e s b e l o w d

Original: e
Revised: e

Original: no t e s t a nd a r d d e v
Revised: no t e s t a nd a r d d e v

Original: i a ti on s .
Revised: i a ti on s .

Original: A TT
Revised: A TT

Original: AC
Revised: AC

Original: K
Revised: K

Original: [Table]
Revised: [Table]

Original: F
Revised: F

Original: [Table]
Revised: [Table]

Original: E
Revised: E

Original: F
Revised: F

Original: [Table]
Revised: [Table]

Original: S
Revised: S

Original: M
Revised: M

Original: [Table]
Revised: [Table]

Original: I M M I M
Revised: I M M I M

Original: C &
Revised: C &

Original: W
Revised: W

Original: D F
Revised: D F

Original: J S M
Revised: J S M

Original: A
Revised: A

Original: [Table]
Revised: [Table]

Original: M
Revised: M

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: B I M 1 ./ 1 ./ 1 ./ 1 .1 ./ 

Revised: B I M 1 ./ 1 ./ 1 ./ 1 .1 ./

Original: M I M 1 ./ 1 ./ 1 ./ 1 .0 ./ 

Revised: M I M 1 ./ 1 ./ 1 ./ 1 .0 ./

Original: D F 0 ./ 

Revised: D F 0 ./

Original: C W 0 ./ 

Revised: C W 0 ./

Original: J S M
Revised: J S M

Original: A 1 ./ 

Revised: A 1 ./

Original: G .
Revised: G .

Original: A SS
Revised: A SS

Original: I R
Revised: I R

Original: A
Revised: A

Original: T a b l e 9 :
Revised: T a b l e 9 :

Original: T h e p e rf
Revised: T h e p e rf

Original: o r m a n ce o f t
Revised: o r m a n ce o f t

Original: h e d i ff e r e n t d e f e n s e s t
Revised: h e d i ff e r e n t d e f e n s e s t

Original: r
Revised: r

Original: a
Revised: a

Original: i n e d on a s i ng l e b l ac kbox a tt ac k on d e t ec ti ng d i ff e r e n t b l ac kbox a tt ac k s .
Revised: i n e d on a s i ng l e b l ac kbox a tt ac k on d e t ec ti ng d i ff e r e n t b l ac kbox a tt ac k s .

Original: T h
Revised: T h

Original: e e n t r i e s c
Revised: e e n t r i e s c

Original: o rre s p o nd t
Revised: o rre s p o nd t

Original: o t
Revised: o t

Original: h e acc u r ac
Revised: h e acc u r ac

Original: y t
Revised: y t

Original: h e d e f e n s e s
Revised: h e d e f e n s e s

Original: ( I D /
Revised: ( I D /

Original: J S D / S P
Revised: J S D / S P

Original: D / S
Revised: D / S

Original: M D ) ob t a i n i n d
Revised: M D ) ob t a i n i n d

Original: i s ti
Revised: i s ti

Original: ngu
Revised: ngu

Original: i s h
Revised: i s h

Original: i ng a dv e r s a r i a l e x a m p l e s fr o m n a t
Revised: i ng a dv e r s a r i a l e x a m p l e s fr o m n a t

Original: u
Revised: u

Original: r a l i m a g e s .
Revised: r a l i m a g e s .

Original: T h e v a l u e s i n p a r e n t h e
Revised: T h e v a l u e s i n p a r e n t h e

Original: s e s b e l o w d
Revised: s e s b e l o w d

Original: e
Revised: e

Original: no t e s t a nd a r d d e v
Revised: no t e s t a nd a r d d e v

Original: i a ti on s .
Revised: i a ti on s .

Original: A TT
Revised: A TT

Original: AC
Revised: AC

Original: K
Revised: K

Original: [Table]
Revised: [Table]

Original: F
Revised: F

Original: [Table]
Revised: [Table]

Original: E
Revised: E

Original: [Table]
Revised: [Table]

Original: M
Revised: M

Original: [Table]
Revised: [Table]

Original: I M M I M
Revised: I M M I M

Original: C &
Revised: C &

Original: W
Revised: W

Original: D F
Revised: D F

Original: J S M
Revised: J S M

Original: A
Revised: A

Original: [Table]
Revised: [Table]

Original: B
Revised: B

Original: I M 0 ./ 

Revised: I M 0 ./

Original: M I M 0 ./ 0 ./
Revised: M I M 0 ./ 0 ./

Original: C &
Revised: C &

Original: W 0 ./ 

Revised: W 0 ./

Original: D F 0 ./ 

Revised: D F 0 ./

Original: J S M
Revised: J S M

Original: A 0 ./ 

Revised: A 0 ./