Title: Adversarial Robustness with Non-uniform Perturbations
Authors: Ecenaz Erdemir
Jeffrey Bickford
Luca Melis
Sergul Aydore
Venue: NeurIPS 2021
Keywords: Adversarial robustness
adversarial attacks
projected gradient descent
certified robustness
randomized smoothing

Abstract: Robustness of machine learning models is critical for security related applications, where real-world adversaries are uniquely focused on evading neural network based detectors. Prior work mainly focus on crafting adversarial examples (AEs) with small uniform norm-bounded perturbations across features to maintain the requirement of imperceptibility. However, uniform perturbations do not result in realistic AEs in domains such as malware, finance, and social networks. For these types of applications, features typically have some semantically meaningful dependencies. The key idea of our proposed approach is to enable non-uniform perturbations that can adequately represent these feature dependencies during adversarial training. We propose using characteristics of the empirical data distribution, both on correlations between the features and the importance of the features themselves. Using experimental datasets for malware classification, credit risk prediction, and spam detection, we show that our approach is more robust to real-world attacks. Finally, we present robustness certification utilizing non-uniform perturbation bounds, and show that non-uniform bounds achieve better certification.

TL;DR: Adversarial training with non-uniform perturbations across the features provides better robustness against the real-world attacks than the uniform approach.

Review discussion:
The approach of the paper appears to be original and largely technically sound.
However, their use of non-uniform perturbations is quite a small and simple amendment to established techniques.
As such, the results I would say are not of huge significance to those working in the area.
In fact, in some cases, including some of those considered, the approach essentially amounts to no more than a change of basis for the features.
I had a few problems with the clarity of the motivation. Mention is made of basing the non-uniformity on either the importance of the features or on correlations between the features (in general, these are different). The claim is made that these approaches result in more realistic adversarial examples.
However, this is not necessarily guaranteed. One can imagine that perturbation of some features could be very important to classification, but not be a realistic proposition if there are dependencies to satisfy. It feels like the approach of the paper, rather than using more realistic adversarial examples, works simply because it is focussing perturbations on the most relevant features. This is not a bad thing to do, but I think the motivation and explanation are somewhat confused. i.e. is the guiding motivation realistic adversarial examples or ones that are more effective in adversarial training?
Also, it is argued, correctly, in the Introduction that there can be certain constraints/dependencies amongst the features.
However, the subsequent approach does not really allow for such dependencies in general as it is simply using a multi-variate Gaussian for the perturbations.
For example, if feature 1 is always equal to twice feature 2, or say bounded below by 3 times feature 3, these dependencies cannot be accommodated in the approach.
That is, the paper conflates dependencies with correlations.
More generally too, without a human in the loop to constrain perturbations to be imperceptible, larger perturbations are allowed, e.g. in their malware examples, huge changes can be made as long as the malicious functionality of the executable is not changed. This is also not covered by the proposed scheme of small non-uniform perturbations.
Though the above points are not saying the approach is incorrect – just that it is not as general as the introduction and motivation would lead one to suppose.
Some minor comments and typos:
-	Can the non-uniformity introduced, i.e. $\Omega$, be a function of $x$?
-	Theorem 2.1 is largely stating the obvious in terms of the correlation between gamma-consistency and epsilon – and the exact result does not, as far as I can see, get used subsequently.
So why is this theorem needed?
-	Theorem 2.1 confuses the probability density function with P(X=x).
-	Theorem 2.1 does not state over what range of values of x $P(X=x|y)>=\gamma$.
-	In equation 5, should the argument of P on the LHS be $\delta$ and not $\Omega \delta$?
-	It is not clear in the examples if the different features are normalised first. Part of the non-uniformity imposed would then correspond to simply normalising the features.
-	In the paragraph beneath Theorem 4.2, it is argued that the result holds if $p_a$ is replaced by a lower bound.
But the theorem is already stated in terms of the lower bound.
-	The statement of Theorem 4.2 in the supplementary material (erroneously referred to there as Theorem 5.2) essentially lumps the key part of the proof of the theorem into a “it can be shown that” and so is not really a proof.
But reference can be made to Cohen et al for details of the proof anyway.
Edit: Score revised following discussion period.
No
We would like to thank the reviewer for the careful reading and the comments.
- The approach is original and largely technically sound but non-uniform perturbations is quite a small and simple amendment to established techniques.
We are glad to hear that the reviewer found our approach original but we are also puzzled that the reviewer says it is an insignificant contribution.
Our extensive experimental results in three different domains and theoretical justifications demonstrate the importance of non-uniform approaches in adversarial training and certification. Note that adversarial training and certification are often studied separately, whereas our paper unifies empirical and theoretical robustness. Our proposed approach outperforms the uniform-bound based SOTA approaches both in adversarial training and certification which we believe adds significant value to the literature.
- is the guiding motivation realistic adversarial examples or ones that are more effective  in adversarial training?
Our guiding motivation is robustness against strong attacks. In section 2.1 we assume a strong attack mimics benign samples, hence we choose Mahalanobis distance matrix for $\Omega$. In Section 2.2, we assume an attacker targeting the most important features can also evade easily. We use different threat models for our $\Omega$ selections.
As a result, we explore both realistic and effective AEs rather than either one.
- the paper conflates dependencies with correlations
It is true that our approach preserves dependencies up to 2nd moment (correlations) but this is already effective in most use-cases where the data is Gaussian. Generalizations to higher moments and distributions can be a future direction.
- huge changes can be made as long as the malicious functionality of the executable is not changed.
Imperceptibility under huge changes is indeed the motivating point of our malware experiments. We applied large non-uniform perturbations during AT to mimic realistic attacks in malware use-case, and tested against unbounded real-world malware evasion attacks. Fig. 1.a. shows that non-uniform perturbations outperform the uniform case against real attacks crafted on malware binaries described in Appendix A.6.
- Response to the minor comments
We would like to thank the reviewer for the comments, we will correct the typos accordingly.
$\Omega$ is dataset-dependent, however it is calculated only once before the AT which makes it constant.
The argument in (5) is $\Omega \delta$ and the projection onto a norm-ball is done for the transformed version of $\delta$.
We observed Gaussian-like behavior in the EMBER dataset, and used standardization for preprocessing which is a normalization method preserving the outliers. These outliers can be representative for potential malicious behavior, which become important during adversarial example generation.
Hence, min-max normalization is not used for the sake of preserving the classification success in the presence of outliers.
Thank you for your responses.
Re malware, I appreciate you are making larger changes, but they are still norm bounded in essence.
Overall, my rating remains unchanged.
To clarify any misunderstandings, we would like to add that the malware use-case attacks which we test against are problem space attacks that are not norm bounded.
Thank you for that clarification.
But in the paper it says e.g. "To make a fair comparison between uniform and non-uniform approaches, $\epsilon$ for each method is selected such that their average distortion budgets, i.e., $||\delta||_2$, are approximately equal."
To clarify, it is the $\epsilon$ in the adversarial training constraint, e.g., equation (4), which belongs to the **defense** mechanism.
On the other hand, the **attacks** we are testing against are not norm bounded in malware use-case.
Thank you so much for your clarification. I will reconsider my score in light of all of your responses.
The motivation of this paper is interesting and reasonable. It is indeed important to design problem-specific perturbation bounds for certain domains. I particularly like the motivation case where not all the features are allowed to be perturbed for maintaining malware functionality. I do see the value of such studies in domains other than computer vision. I have a few questions for the authors.
1. I can understand that we need domain knowledge to construct the transformation matrix $\Omega$ to achieve domain-specific constraints.
But the paper mainly talks about different types of metrics for measuring the feature importance such as Mahalanobis distance, Shapley values. How are these two connected? It seems disconnected between the motivation and the actual addressed problem.
That said, why does using those feature importance metrics have better results than a uniform bound? The gradient during adversarial generation already tells us the importance of different features. The paper mentions that the Mahalanobis distance can help generate adversarial examples "with feature correlations similar to the original dataset". This sounds reasonable. Can the authors empirically validate this argument?
2. The existing work [25] is closely related to this work as stated in the paper. It certifies non-uniform bounds for models. In Section 4, this paper also tries to certify the robustness with non-uniform perturbations. Why is work [25] not studied in the paper? A straightforward experiment would be just using work [25] to certify original models and models trained by this paper. This can illustrate how the non-uniform bound change before and after training.
3. For the transformation matrix $\Omega$, is it calculated once before the adversarial generation or every iteration during the generation? What is the time cost for the non-uniform adversarial training compared to the uniform one?
4. At lines 188-189, the paper states that "to model the expert knowledge with diagonal $\Omega$, we use Pearson’s correlation coefficient, Shapley values, and masking to allow perturbation only in mutable features". Did the authors also mask out those immutable features when training with Pearson’s correlation coefficient, Shapley values? How about the training with the Mahalanobis distance? If not, I am not sure how these metrics can mask out immutable features. It also comes back to my first point that the paper does not seem to address the problem introduced in the motivation. Could the authors care to explain?
5. In Table A.1 in the appendix, why do adversarially trained models have worse performance on C2 Pad. attack than the standard training? Almost all the adversarial training methods are worse than the standard training. Have the authors looked into this?
No
We would like to thank the reviewer for careful reading of the paper and the comments.
- I can understand that we need domain knowledge to construct the transformation matrix Ω to achieve domain-specific constraints.
But the paper mainly talks about different types of metrics for measuring the feature importance such as Mahalanobis distance, Shapley values. How are these two connected? It seems disconnected between the motivation and the actual addressed problem
We used expert knowledge to construct $\Omega$ **only** for the NU-$\delta$-Mask case in malware application, where the immutable features are known to the defender and the corresponding perturbations for these features are masked so that the perturbation budget is used efficiently. For the rest of the cases, only the data-dependent knowledge coming from the dataset is used.
For instance, Pearson correlations and SHAP values measure the feature importance and they are calculated using the training data set. Mahalanobis distance measures cross-feature dependencies and it is calculated using the covariance of the data set.
- why does using those feature importance metrics have better results than a uniform bound? The gradient during adversarial generation already tells us the importance of different features. The paper mentions that the Mahalanobis distance can help generate adversarial examples "with feature correlations similar to the original dataset". This sounds reasonable. Can the authors empirically validate this argument?
The gradient shows the best direction for perturbations.
However, the uniform perturbation is bounded by a uniform ball constraint in that direction, whereas non-uniform perturbation uses perturbation budget more efficiently by letting large perturbations for important features and applying more constraints on unimportant features. We illustrated this in a toy example in Figure A.1.
Overall, both approaches have the same average distortion $||\delta||_p$, while feature weighting does not allow unnecessary budget for less important features. We empirically demonstrated this in Section 3.4 and visualized the difference between the non-uniform and uniform approaches in terms of the consistency metric in Fig. 2b.
- Comparison with the existing work [2]
We thank the reviewer for the suggestion. We compared certification of the uniformly robust model and our nonuniformly robust models using [2] for the setting in Fig 1.c for $||\delta||_2=0.3$. We calculated the geometric mean over learned $\epsilon$’s as:
**Uniform-$\delta$** = 0.0601, **NU-$\delta$-MDt** = 0.0617, **NU-$\delta$-Pearson** = 0.0608.
As a result, [2] also certifies that our nonuniform approach outperforms the uniform robustness.
- Time cost
$\Omega$ is calculated only once before the adversarial training. Computation time for the **pgd_l2** function provided in the supplementary code is 12.001ms for uniform PGD, and 12.879ms for NU-$\delta$-MDtarget PGD for one batch (512 samples) in one iteration.
Hence, additional computation time is small (around 7%). We ran these on a RTX 2080 Ti GPU.
- Expert knowledge
Masking is only applied to the identity matrix, but not to Pearson, SHAP, MD and MDtarget matrices. Masking case represents a best case defense scenario where expert knowledge on the immutable features is available. For the rest of the cases, we do not claim that the immutable features are known to the defender, instead data-dependent knowledge is used. We will rephrase that sentence to explain masking more clearly.
- C2 Pad performance
During validation, the best performing model was selected based on a defense success rate averaged over various problem space attacks. This caused a higher weight in the average performance for stronger attacks, e.g., greedy attacks, than the weaker attacks, e.g., padding attacks.
I would like to thank the authors for the response.
The results measured by [25] seem marginal for the proposed method over the uniform bound. The benefit of non-uniform bounds is not as large as claimed in the paper. I would suggest include those results in Tables 1 and 2. I am less excited about the paper now.
We would like to thank the reviewer for the suggestion.
Since the certification $\epsilon$’s are small, the range of gain obtained from non-uniformity is also small in terms of the geometric mean. What we call **Margin** in the paper is not a certification $\epsilon$, hence there is not a contradiction between our claim in the paper and the results. Please see our final response to the Reviewer kjQY for more information about the **Margin**.
We also measured the certification success rates using [25] as 88.72 % for **Uniform-$\delta$**, 94.38 % for **NU-$\delta$-MDt** indicating that our non-uniform approach yields more certifiable samples than the uniform approach. These results are also consistent with the results we presented in Tables 1 and 2 in the paper.
How the data dependency is incorporated in the proposed framework and what is the limitation of such dependency?
What are the parameters of the chosen PGD attack?
As compared to the claim in the paper, the performance of the proposed attack is comparable to a single chosen uniform attack. Are the values reported in the Figure statistically significant?
For example in Figure 1b, the difference in the values between existing and multiple variants of the proposed attack is 1-2%. A similar observation can be drawn from other results as well.
Another limitation is that only a single attack and single network architecture is chosen for comparison with uniform perturbation?
In appendix A.3, "This approach does not guarantee same range (min and max) for all features.
58 As a result, it is possible that the features have different scales even after normalization", then what normalization the authors have applied, and why does the above standardization reflect such observation?
The paper listed multiple appendices for the readers to refer to. It would be great if the authors can bring all the sufficient and critical information in the paper to make the understanding clear.
Why the proposed attack is applied to the bad class? What if the perturbation is applied to the data without knowing the class?
The analysis of table 1 is not clear. What to refer to from the table? What does the value mean, the higher the value, better the attack or vice versa, etc?
Contrary to figure 2, it seems from the values in Table 1 and Table 2 that the proposed attack is easy to defend as compared to uniform perturbation.
How the precomputed values such as covariance matrix and Shapley values are obtained? have the authors noticed any data distribution apart from taking the assumption of Gaussian distribution? What is the impact if other distributions come?
No
- Data dependency
Data dependent selection of $\Omega$ represents most common scenarios where the domain or attack knowledge is limited. An attacker modifying an unseen sample by the adversarial training can introduce a limitation to our approach, however this limitation applies to the uniform adversarial training as well.
- Parameters of the PGD attack.
We will move the critical information to the main body, and include PGD parameters below.
- number of iterations=10
- number of restarts=3
- $\alpha=0.01$ for $\|\delta\|_2=\{0.1,0.3\}$
- $\alpha=0.1$ for $\|\delta\|_2=\{0.5,0.7,1\}$
- $\alpha=0.5$ for larger $\|\delta\|_2$ in malware use-case
- Performance significance
We believe several percent improvement in robustness is critical for many security applications. In Figure 1, even without masking, we can observe 3-4% performance gain over the uniform approach for malware and credit risk use-cases, and 10% for spam detection for some given perturbation budgets. The choice of the budget depends on how much clean accuracy we can sacrifice in the security-critical application.
Furthermore, the non-uniform approach for certification provides much larger improvement as can be seen in Tables 1 and 2.
- Single attack, single network
In the paper, our goal is to show that the proposed transformation and $\Omega$ selections improve the adversarial training performance compared to the uniform approach.
Therefore, we used one of the most common state-of-the-art models to show both the uniform and non-uniform performances. Our purpose is not to develop the state-of-the-art neural network for classification, therefore we did not focus on various architectures.
- Normalization
After observing Gaussian-like behavior in some features in the EMBER dataset, we used the standardization approach for pre-processing for the sake of preserving the outliers. (Please see https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html). These outliers can be representative for potential malicious behavior, which become important during adversarial example generation.
- Attacks are applied to bad class only
In applications like malware detection, there is an inherent maliciousness of a particular class such as malware class.
Therefore, we assume that an attacker would not try to make an ML model detect a benign file as malicious [Pierrazzi et al 2020]. This can be the case for fraud detection as well. An attacker would not try to fool the classifier and show an already eligible credit record as ineligible.
- Clarifying Table 1
Since Table 1 does not show an attack result, we believe reviewer means Table A.1. The values represent the number of evasions which shows the attack strength.
Hence, the higher the value, the stronger the attack is as the reviewer states.
If the reviewer means Table A.1 and Table A.2, they represent the results of a different dataset and use-case than Fig.2. Table A.1 shows various attacks from strong greedy attacks to weaker padding attacks. An average defense success rate against all these attacks is presented in Table A.2, which is dominated by stronger attacks such as GNN.
Therefore, we see a smoothed performance over different attacks. If we check GLGBM individually, the performance of uniform defense is improved by MDtarget around 6%, as well as in GADV, it is improved by Pearson around 4%. We believe these are significant improvements in security applications.
- Pre-computed values and distribution assumption
We pre-computed covariance matrix and Pearson correlation values using Scikit-learn library and the Shapley values using the SHAP library [32] utilizing a deep explainer. All metrics are computed on the training dataset without any particular distribution assumption. We only used Gaussian distribution assumption in proofs of Theorem 2.1 and Theorem 4.2.
Thank you for providing a detailed response.
I agree the aim might not be to develop a SOTA algorithm or work but the robustness of the work needs to address. It should not be the case that the proposed research is biased towards one model or parameter setting.
Therefore evaluation in multiple settings is important.
The impact and role of normalization are not clear.
Table 1 is on page 8 of the paper.
Therefore, clarification on that is needed.
Similarly, the significance of the results reported in Figure 1 needs a better explanation.
We would like to thank the reviewer for the comments.
- Unbiased Robustness
We performed additional experiments for the spam detection setting, and showed that the proposed method provides robustness not only against the presented attacks crafted with our network but also against some well-known attacks generated by different tools. For this, we utilized the Adversarial Robustness Toolbox using the default parameters for the AE generators of Carlini Wagner, JSMA and DeepFool Methods. We also crafted AEs using FGSM for $\epsilon=0.5$ and $\epsilon=1$. In the table below, we observe that NU-$\delta$-MDt outperforms the uniform approach in all cases. These results show that even when the attacks are generated by different tools, the robustness gain provided by non-uniform perturbations is significant.
- Normalization
By standard normalization, we mean removing the mean and scaling to unit variance. This will transform the data to unit variance but not to a unit scale.
- Clarifying Table 1
We would like to apologize for the misunderstanding. Table 1 shows the certification results of two adversarially trained models, i.e., Uniform-$\delta$ and NU-$\delta$-MDt. These robust models are certified using standard certification with linear programming approach which we call Uniform-Cert, and also using our proposed certification methods. **Margin** represents how close the network output is to the adversary’s target class when an AE is tested at the inference time.
Therefore, an attacker wants to make the margin as small as possible so that the AE is classified as the target class instead of its original class. A larger margin means that the model is robust against these AEs.
Hence, in Table 1, the larger the certification margin is the better (tighter) the certification bound is. **Certification success** represents a similar idea with the percentage of the number of certified samples.
- Significance of the Results
We strongly believe that 3-4% performance gain over the uniform approach for malware and credit risk use-cases, and 10% for spam detection are significant improvements for security-critical applications.
In addition to the results in Figure 1, our new results provided in the table below show that our proposed approaches provide much higher robustness against various attacks as well.
- Defense success rates of Uniform-$\delta$ and NU-$\delta$-MDt against FGSM, Carlini Wagner, JSMA and Deep Fool attacks.
| Defenses         | $\|\|\delta\|\|_2$ | FGSM $\epsilon$=0.5 | FGSM $\epsilon$=1 | Carlini Wagner   | JSMA             | Deep Fool        |
|------------------|--------------------|---------------------|-------------------|------------------|------------------|------------------|
| Uniform-$\delta$ | 0.1                | $24.1 \pm 0.25$     | $20.17 \pm 0.31$  | $38.93 \pm 0.54$ | $41.51 \pm$ 0.24 | $39.3 \pm 0.27$  |
| NU-$\delta$-MDt  | 0.1                | $28.7 \pm 0.17$     | $27.1 \pm 0.21$   | $49.62 \pm 0.33$ | $49.59 \pm 0.18$ | $51.73 \pm 0.31$ |
| Uniform-$\delta$ | 0.3                | $25.22 \pm 0.12$    | $21.61 \pm 0.43$  | $45.83 \pm 0.41$ | $46.38 \pm 0.27$ | $47.39 \pm 0.25$ |
| NU-$\delta$-MDt  | 0.3                | $32.15 \pm 0.34$    | $30.88 \pm 0.36$  | $53.45 \pm 0.22$ | $52.68 \pm 0.13$ | $54.61 \pm 0.19$ |
| Uniform-$\delta$ | 0.5                | $27.05 \pm 0.50$    | $23.08 \pm 0.34$  | $50.05 \pm 0.32$ | $49.5 \pm 0.25$  | $49.75 \pm 0.16$ |
| NU-$\delta$-MDt  | 0.5                | $43.85 \pm 0.62$    | $36.12 \pm 0.20$  | $55.24 \pm 0.38$ | $53.71 \pm 0.41$ | $64.28 \pm 0.55$ |
| Uniform-$\delta$ | 1                  | $40.20 \pm 0.57$    | $32.98 \pm 0.31$  | $52.77 \pm 0.24$ | $52.94 \pm 0.26$ | $53.22 \pm 0.10$ |
| NU-$\delta$-MDt  | 1                  | $81.34 \pm 0.83$    | $79.85 \pm 0.75$  | $61.89 \pm 0.37$ | $72.93 \pm 0.77$ | $88.75 \pm 0.78$ |
| Uniform-$\delta$ | 1.5                | $74.40 \pm 0.47$    | $62.36 \pm 0.75$  | $59.71 \pm 0.28$ | $68.95 \pm 0.85$ | $64.5 \pm 0.86$  |
| NU-$\delta$-MDt  | 1.5                | $94.45 \pm 0.18$    | $91.03 \pm 0.10$  | $72.98 \pm 0.48$ | $86.43 \pm 0.32$ | $98.36 \pm 0.25$ |
Thanks for the clarification. Based on the new response and points raised by other reviewers, I am improving my rating by 1 point. Please add all the provided and relevant information to the paper.
We would like to thank the reviewer for the comments. We will update the paper accordingly.
Please also see our response to Reviewer cq1G regarding the significance of the proposed defenses, and to Reviewer QUCv regarding the significance of the certification.
Strength:
1. It is novel to consider feature importance, feature correlations and domain knowledge when generating adversarial perturbations.
2. Evaluations on multiple tasks demonstrate the effectiveness of the proposed methods.
3. The authors provide certified robustness with non-uniform perturbations.
Weakness:
I have the following questions on the evaluation part:
1.
Regarding the setup of the experiment, are the non-uniform defence approaches tested against non-uniform perturbations, while the uniform defence approaches tested against uniform perturbations? What if the non-uniform defence approaches are tested against uniform perturbations, or the other way around?
2. Does the new approach have any impact on the efficiency of adversarial training? It is mentioned that the covariance matrix and Shapley values are precomputed. How does this affect the results?
3. In Fig. 1, why does the defence success rate increase with $||\delta||_{2}$?
No
- Experiment setup
In the paper, we tested both uniform and nonuniform approaches against realistic non-uniform attacks. To address the reviewer’s comment, we did additional experiments and evaluated ATs against uniform attack. Below are the few results on the defense success rate (%)  of Uniform-$\delta$ and NU-$\delta$-MDtarget against uniform PGD attack for various perturbations. The table shows that our non-uniform approach is more robust than the uniform approach even against uniform PGD attacks.
| Defenses         | $\|\|\delta\|\|_2$ | Uniform Attack  $\epsilon$=0.1 | Uniform Attack  $\epsilon$=0.3 | Uniform Attack  $\epsilon$=0.5 | Uniform Attack  $\epsilon$=0.7 |
|------------------|--------------------|--------------------------------|--------------------------------|--------------------------------|--------------------------------|
| Uniform-$\delta$ | 0.1                | 90.6                           | 83.9                           | 16.6                           | 14.8                           |
| NU-$\delta$-MDt  | 0.1                | 92.8                           | 88.2                           | 34.95                          | 21.4                           |
| Uniform-$\delta$ | 0.3                | 92.6                           | 89.05                          | 30.5                           | 20.85                          |
| NU-$\delta$-MDt  | 0.3                | 93.2                           | 90.95                          | 61.75                          | 46.3                           |
| Uniform-$\delta$ | 0.5                | 92.9                           | 91.4                           | 88                             | 86.5                           |
| NU-$\delta$-MDt  | 0.5                | 93.3                           | 91.45                          | 89.45                          | 87.7                           |
| Uniform-$\delta$ | 1                  | 93.2                           | 91.9                           | 90.18                          | 88.38                          |
| NU-$\delta$-MDt  | 1                  | 94.5                           | 94.17                          | 93.33                          | 93.14                          |
| Uniform-$\delta$ | 1.5                | 93.22                          | 92.01                          | 90.61                          | 89.98                          |
| NU-$\delta$-MDt  | 1.5                | 94.45                          | 94.38                          | 93.76                          | 93.48                          |
- Impact of the nonuniform constraint on efficiency
$\Omega$ is calculated only once before the adversarial training. Computation time for the **pgd_l2** function provided in the supplementary code is 12.001ms for uniform PGD, and 12.879ms for N-$\delta$-MDtarget PGD for one batch (512 samples) in one iteration.
Hence, additional computation time is small (around 7%). We ran these on a RTX 2080 Ti GPU.
- Defense success rate increase with $\|\delta\|_2$
There is a positive correlation between the amount of perturbation applied during adversarial training and the level of robustness against adversarial attacks. This pattern can be seen for all robust models regardless of uniformity of the perturbations. This is due to the fact that the network is not only trained with clean samples but also any slightly modified sample which might cause a misclassification. Please see Figure 3 in [A].
[A] Madry, Aleksander & Makelov, Aleksandar & Schmidt, Ludwig & Tsipras, Dimitris & Vladu, Adrian. (2017). Towards Deep Learning Models Resistant to Adversarial Attacks.
Thank you for the additional experimental results and the clarification. The results suggest when \epsilon is relatively small, the improvement is marginal. Can you further explain on that?
Since the attacks with smaller $\epsilon$’s are weaker attacks, the defense success rates of the robust models are already high against these weak attacks.
Therefore, there is little room for improvement in the defense success rate. For further clarification, we updated the table in our previous response by including the results for $||\delta||_2$=1 and $||\delta||_2$=1.5 against these PGD attacks. We also provided a table in response to the Reviewer kjQY showing the defense success rates against stronger attacks. Both tables show that the non-uniform approaches provide higher robustness than the uniform approach at a more significant level against stronger attacks.
1. The problem is very important and interesting.
For instance, adversarially generated malware may cause serious security concerns.
Therefore, how to train robust classifiers in security-critical domains is an important research problem.
2. The improvement of the proposed approach is marginal compared with adversarial training with uniform adversarial perturbations. As shown in Figure 1(a) and 1(b), the proposed approach only improve the baseline for several percent regarding defense success rate.
3. The technical contribution of this work is weak. The adversarial training is a very standard method to train robust classifiers. The extension of adversarial training to non-uniform adversarial perturbations is not challenge.
4. A baseline is that the defender could leverage the adversarial examples crafted in the problem-space to perform adversarial training. Could the authors share more details (or comparison if possible) for this?
No
- Performance gain over the baseline
We would like to thank the reviewer for the comments and finding the problem very interesting. We agree that robustness in security-critical domains is an important research problem.
Therefore, we believe several percent improvement in robustness is also critical for many security applications. In Figure 1, even without masking, we can observe 3-4% performance gain over the uniform approach for malware and credit risk use-cases, and 10% for spam detection for some given perturbation budgets. The choice of the budget depends on how much clean accuracy we can sacrifice in the security-critical application.
Furthermore, the non-uniform approach for certification provides much larger improvement as can be seen in Tables 1 and 2.
- Technical contribution to the adversarial training and certification
Adversarial training (AT) is still the strongest empirical defense [12] which makes it SOTA. We see this as the main reason why it has become a standard for robustness.
On the other hand, AT is underexplored in some security-critical applications, and densely studied computer vision methods are used in these areas.
However, this is not usually the best approach, and our attempt is to tackle the challenges in these under-explored domains.
We agree with the reviewer that the problem is very important and interesting.
Therefore, proposing empirical and certifiable improvements in this area is significant.
As the reviewer also says, certification results are a generalization for the existing methods which we believe adds value to the literature.
- Crafting adversarial examples in problem space
Defenses crafting adversarial examples in problem space require strong assumptions on the potential attacks. We wanted to tackle a more general scenario where the defender has no knowledge of the attack but knows its own environment, such as the training data, immutable features for the domain, and etc.
Therefore, we propose a defense against a larger space of realistic attacks that includes problem space, e.g., malware use-case, and feature-space attacks, e.g., credit risk and spam use-cases, without making any assumption on these potential attacks.

Meta-review:
role: PC
data: comment: This paper proposes to improve the adversarial robustness by using non-uniform perturbations in adversarial training. After the rebuttal and discussion, the reviewers raised several concerns on the novelty, the marginal improvement over baselines, and some experimental settings. The paper could be strengthened if these concerns could be addressed. 

Decision: Accept (Poster)