Adversarial Robustness with Non-uniform Perturbations
Authors: Ecenaz Erdemir
Jeffrey Bickford
Luca Melis
Sergul Aydore
Venue: NeurIPS 2021 Poster (2021)
Keywords: Adversarial robustness
adversarial attacks
projected gradient descent
certified robustness
randomized smoothing

Abstract: Robustness of machine learning models is critical for security related applications, where real-world adversaries are uniquely focused on evading neural network based detectors. Prior work mainly focus on crafting adversarial examples (AEs) with small uniform norm-bounded perturbations across features to maintain the requirement of imperceptibility. However, uniform perturbations do not result in realistic AEs in domains such as malware, finance, and social networks. For these types of applications, features typically have some semantically meaningful dependencies. The key idea of our proposed approach is to enable non-uniform perturbations that can adequately represent these feature dependencies during adversarial training. We propose using characteristics of the empirical data distribution, both on correlations between the features and the importance of the features themselves. Using experimental datasets for malware classification, credit risk prediction, and spam detection, we show that our approach is more robust to real-world attacks. Finally, we present robustness certification utilizing non-uniform perturbation bounds, and show that non-uniform bounds achieve better certification.

Adversarial Robustness with Non-uniform Perturbations
Ecenaz Erdemir Imperial College London e, erdemir17@@mppriall ac. uk
Jeffrey Bickford Amazon Web Services
Luca Melis Amazon Web Services
jbickOamazon.com com
micmeli@amazen com
Sergiil Aydöre Amazon Web Services saydore@amazon.co com
Abstract
Robustness of machine learning models is critical for security related applications. where real-world adversaries are uniquely focused on evading neural network based detectors. Prior work mainly focus crafting adversarial examples (AEs) with small uniform Norm-bounded perturbations across features to maintain the require- ment of imperceptibility. However, uniform perturbations do not result in realistic AEs in domains such as malware, finance, and social networks. For these types applications, features typically have some semantically meaningful dependencies. The key idea of our proposed approact is to enable non-uniform perturbations that adequately represent these feature dependencies during adversarial training. the empirical data distribution, both on correla-
We propose using characteristics tions between the features and the importance of the features themselves. Using experimental datasets for malware classification, credit risk prediction, and spam detection, we show that our approach more robust real-world attacks. Finally, we present robustness certification utilizing non-uniform perturbation bounds, and show that non-uniform bounds achieve better certification.
Introduction
Deep neural networks (DNNs) are commonly used a wide-variety of security-critical applications such self-driving cars, spam detection, malware detection and medical diagnosis (T. However, DNNs have been shown to be vulnerable to adversarial examples (AEs), which are perturbed inputs designed fool the machine learning systems 4 To mitigate this problem, line of research has focused on adversarial robustness of DNNs as well as the certification of these methods [
Adversarial training (AT) is of the most effective empirical defenses against adversarial attacks [1. The goal during training is to minimize the loss the DNN when perturbed samples are used. This way, the model becomes robust to real-world adversarial attacks. Though these empirical defenses do not provide theoretically provable guarantees, they have been shown to be robust against the strongest known attacks [12.. Some of the most common state-of-the-art adversarial attacks, such as projected gradient descent (PGD) ① and fast gradient sign method (FGSM) 12), perturb training samples under norm-ball constraint to maximize the loss of the network. The goal of certification, on the other hand, is to report whether an AE exists within an E, norm centered at given sample with fixed radius. Certified defense approaches introduce theoretical robustness guarantces against norm-boundes perturbations 8 9
Coode available at btttps://githne
nonuniform- wtw.roormwpppuuuubtbrroooooo
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

In the computer vision domain, the adversary's goal is to generate perturbed images that cause misclassifications in a DNN. It often assumed that limiting uniform norm-ball constraint results in perturbations that are imperceptible to the human eye. However in other applications such as fraud detection [5], spam detection [66], credit card default prediction [7 18 and malware detection [19221] norm-bounded uniform perturbations may result in unrealistic transformations. Perturbed samples must comply with certain constraints related to the domain, hence preventing us from borrowing these assumptions from computer vision. These constraints can be on semantically meaningful feature dependencies, expert knowledge of possible attacks, and immutable features This paper proposes methodology to generate non-uniform perturbations that takes into account the characteristics of the empirical data distribution. Our results demonstrate that these non-uniform perturbations outperform uniform norm-ball constraints in these types of applications.
20 22
1.1
Background and Motivation
AT is min-max problem minimizing the DNN loss which is maximized by adversarial perturbations (call 8). State-of-the-ar approaches for optimizing ó usually assume that all the imput features require equal levels of robustness, however, this might not be the case for many applications. A toy example for AT with non-uniform perturbations is given in Appendix A The intuition behind the need for non-uniform constraints is apparent across many industrial applications. A common cybersecurity application is malware detection, which identifies if an executable file is benign or malicious. Unlike images, diverse and semantically meaningful features are extracted from the executable file and are passed to a machine learning model. To maintain the functionality of an executable file during an adversarial attack, certain features may be immutable and perturbations may result in an unrealistic scenario. For example in the Android malware space, application permissions, such as permission to access a phone's 's location service, are required for malicious functionality and cannot be perturbed [19]. In finance scenario where customer credit card applications are evaluated by machine learning models, a possible set of features include age, gender, income, savings, education level, number of dependents, etc. In this type of dataset there are clear dependencies between features, for example the number of dependents has a meaningful correlation with age. When detecting spammers within social networks, features are extracted from accounts and may include the length of the username, length of user description, number of following and followers as well as the ratio between them, percentage of bidirectional friends, etc. Similar to the previous finance example, there is meaningful correlation between features such as the percentage of bidirectional friends and the ratio of followers.
In all of these scenarios, non- non-uniform perturbations can be used to maintain these correlations and semantically meaningful dependencies resulting in more realistic AEs. In this work, we propose adversarial training with these more realistic perturbations to increase the robustness against real- world adversarial attacks. Specifically, our contributions are: (i) Instead of considering an allowed perturbation region where all the features are treated uniformly. i.e., ||0'| I c, we consider a transformed input perturbation constraint, i.e., ||(25]Ip I where Q is a transformation matrix, which takes the available information into account, such as feature importance, feature correlations and/or domain knowledge. Hence, the transformation uniform input perturbations over the features. (ii) For various applications such as malware detection, credit risk prediction and spam detection, we show that robustness using non-uniform perturbations outperforms the commonly-used uniform approach. (ii) To provide provable guarantees for non- uniform robustness, we modify two known certification methods, lincar programming and randomized smoothing, to account for non-uniform perturbation constraints.
the norm ball constraint results non-
1.2 Related Work
Different levels of robustness of different features have already been studied in the literature 23 25 Among these, (23] and (24] study the effect of robust and non-robust features in standard and adversarial training. Both works show that different types of features might either be vulnerable or robust to small input perturbations. This is a discrete interpretation of different levels of perturbation tolerance which is the core idea of our approach. Both works provide effective theoretical analysis of robust and non-robust features and their effect on clean and adversarial accuracy, while we propose a defense mechanism utilizing this phenomenon.
closely related work to ours is 25 which considers non-uniform perturbation bounds for input features. Given a non-uniform adversarial budget for loo norm bounded inputs, (25) proposes
2

---

framework that maximizes the volumes of certified bounds. However, this work only proposes robustness certification of pre-trained models, and does not consider training robust model against these non-uniform perturbations. The approach is also data-agnostic meaning that it does not take correlations or an additional knowledge on the data into account. We instead use non-uniform perturbations in data-dependent AT to achieve robustness in DNN model. In fact, [2 mentioned consideration of feature correlations as a potential future direction of their work.
In [26] conditional variational autoencoder (CVAE) is trained to learn perturbation sets for image data and the generated adversarial images are then used in augmented training. The work proposes robustness against common image corruptions as well as Ep perturbations. However, it mainly focuses on possible corruptions and attacks specific to image domain, and assumes access the test data distribution. In our work, we focus on non-image data which have correlated and different scale features and we do not rely on an additional knowledge of the test set.
Recent work in the malware detection space (00 creates realistic AEs by defining a set of comprehen- sive and realistic constraints on how an input file can be transformed. Though this approach creates realistic AEs, collecting representative corpus of raw input files is a challenging problem [27]. The recent release of binary feature sets, such as the EMBER dataset without having access to the raw input files. Our work can be used to create more realistic AEs in situations where access large representative corpus of files is not possible.
22 enables model development
Non-uniform Adversarial Perturbations
In adversarial training, the worst case loss for an allowed perturbation region is minimized over parameters of a function representing a DNN. The objective of the adversary can written as the inner maximization of adversarial training:
maximize SEA..p
((fo(x + 5).
(1)
where x E X' and E y are dataset inputs and labels, D.pp = (6 ||6||p < c) is an Ep ball of radius which defines the feasible perturbation region. Standard PGD follows steepest descent which iteratively updates in the gradient direction to increase the loss:
Vol(Jo(+ 81+1 = 8 + a Vel(fo(z + 8*), y)p
(2)
at iteration 4, and then projects å to the closest point onto the Ep ball:
PA., (6) = arg min ||8 8'12 = max [| 8'EA..
(3)
where the distance between d and 8' is the Euclidean distance, and the projection corresponds to normalizing 8 to have a maximum E, norm which is equal to e. Now, we introduce an adversarial constraint set that non-uniformly limits adversarial variations in different, potentially correlated dimensions, by
Â {ō: ||
where Q E dgd In our approach, 8 is updated by equation (2) similar to the standard PGD, however, it is projected back to non- non-unifome ton-unffomm norm ball satisfying || 1Ip S The corresponding projection operator will then be:
5 11
if
|S25|p
P PA.,, (16)
(5)
otherwise.
The choice of Ω depends on how we model the expert knowledge feature relationships. The following are our choices for the non-uniform perturbation sets.
2.1 Mahalanobis Distance (MD)
Euclidean distance between two points in multi-dimensional space is useful metric when the vectors have isotropic distribution (i.e. radially symmetric). This because the Euclidean distance assumes each dimension has same scale (or spread) and are uncorrelated to other dimensions.
3

---

However, isotropy is usually not the case for real datasets in which different features might have different scales and can be correlated. Fortunately, MD accounts for how the features are scaled and correlated to one another [29]. Hence, it is a more useful metric if the data has non-isotropic distribution.
By formal definition, MD between vectors 2, E R is denoted by dz == -'=-2')² (2 z²), where M E Rdxd is positive semi-definite matrix which can be de- composed as M = U for U E Rdxd The dissimilarity between two vectors from a distribution with covariance can be measured by selecting M Σ-1 If feature vectors of dataset are uncorrelated and have unit variances, their covariance matrix is Σ = I, which reduces their MD to Euclidean distance.
We are interested in the distance between the original and the perturbed sample. Since we assume all perturbations are additive, as common practice, the distance term we consider is V M5. For generalized MD in lp norm, selecting f = U corresponds to the perturbation set Ac.p {ō |UT S }} which generates AEs with feature correlations similar to the original dataset.
Robustness of adversarially trained model is directly related to how realistic the generated AEs are during training. Now, we explore implications of selecting & MD to define the limits of the perturbation set. To ensure the validity of the AEs, consider the notion of consistency of the generated sample with real samples.
[18] introduced the notion of e-inconsistency to quantify how likely an AE is. With slight change in their notation, we define 7-consistency as follows:
Definition 2.1. For a consistency threshold 7 > 0, an AE is "yconsistent if f y) 2 % where I E Rd, and f is a probability density function of a conditional Gaussian distribution with zero mean and covariance matrix Ey.
Theorem 2.1. If the AEs are generated according to MD constraint, then their 7-consistency has a direct relation to such that
0 < /2C - log S c.
(6)
where C C= - (ogg(o))
d is the dimension of x, and v 1 S
Theorem 2.1 implies that there is direct relationship between limiting the MD of 8 and ensuring consistent samples when the data is Gaussian. In other words, when the f2 MD of the perturbations gets smaller, AEs become more consistent. See Appendix A.4 for the proof.
2.2 Weighted Norm
When Ω is a diagonal matrix, inner maximization constraint simply becomes the weighted norm of 5 limited by e, and the weights denoted by (Mr,) Projection of under the new constraint corresponds projection onto lp norm ball of radius n. for 5th feature. These weights be chosen exploiting domain, attack or model knowledge. For instance, more important features can be allowed to be perturbed more than the other features which have less effect on the output score of the classifier. This knowledge might come from Pearson's correlation coefficients [17] between the features of the training data and the corresponding labels, or Shapley values 30 for each feature.
Using Pearson's 's correlation coefficient of each feature with the corresponding target variable, i.c.. |pt.vl for jth feature and output y. we let larger perturbation radii for more correlated features with the output. Due to the inverse relation between Ω2 and the radius of the norm ball, for Pi,y = p.pl
we select Ω = diag( (ang(f 1fm)) II Wi ll
Similarly, using Shapley values to represent feature importance, we
define 8; = 1 where s; is the Shapley value of feature i. Then, we choose diag( (&. )ime) by
[[ Mi
following the intuition that more important features should have larger perturbation radii.
In the malware domain, expert knowledge might help to rule out specific type attacks crafted on immutable features due to feasibility constraints. This can be modelled by the proposed weighted norm constraint as masking the perturbations on immutable features. Hence, non-uniform perturbation approach enables various transformations on the attack space for robustness against realistic attacks.

---

93
24
Defese 5/000 Rato-
Deferte C000
88
Usikmn
Deferee
form MD
Neruuitem No
Nn form Malarpe:
Non-nitem MDtarge:
10
20
0.4
0.6
10
044
0.6
3.8
1.0
Recroge:11Bi
Avcrage (|tllz
Avcnye (Hil
(a) Malware Use-case
(b) Credit Risk Use-case
(c) Spam Detection Use-case
Figure 1: Defense success rate of PGD AT against the problem-space attacks, where all non- uniform perturbation defense approaches outperform the uniform approach for all use-cases.
Experimental Results
Here, we present experimental results evaluate robustness of DNNs against adversarial attacks for binary classification problems on three applications: malware detection, credit risk prediction, and spam detection. We compare PGD) with non-uniform perturbations during AT with PGD proposed in [ based on uniform perturbations. For all applications, we evaluate our defense mechanisms on adversarial attacks proposed by other works. We use a machine with an Intel Xeon E5-2686 @ 2.3 GHz CPU, and Nvidia Tesla V100 GPUs. Details of the DNN architecture and pre-processing are given in A.3 Note that our goal
design the best possible neural network but instead compare [ with various non- non-uniform perturbations during AT for given DNN.
the uniform perurbations
Adversarial training (AT): We perform AT in all use-cases by applying & norm PGD for uniform perturbation sets, i.e., A.,.2 (6 ||ö||2 S e. and non-uniform perturbation sets, i.e. 4ez {ō [|28||2 S )2)- Since potential adversaries are not interested in fooling the classifiers with negative class (target class) samples, 8 perturbations
only applied to the positive classes during AT as commonly used especially in malware detection [20]. Positive classes are the malicious class in malware detection, bad class in credit risk prediction, and spammer class in spam detection. Moreover, for the sake of clean accuracy within the positive class, adversarial perturbations are applied to 90% of the positive samples during training. Such hybrid approach where weighted clean adversarial loss are optimized at once is common in literature 31..
To model the expert knowledge with diagonal Ω, values, and masking to allow perturbation only in mutable features. To compute Shapley values, use SHAP B32 which utilizes deep learning explainer. We also consider AT under the MD constraint, and select Q=UT such that U U=Σ, considering two cases; Σy is the covariance matrix of the entire training data, i.c., y= 1}. and E, is only for the negative (target) class y=0. We call the models after AT with non-uniform perturbations according to their ! selection, e.g., NU-S-Pearson for Pearson's coefficients, NU-6-SAPP for Shapley values, NU-S-Mask for masking, NU-S-MI for MD using full covariance matrix and NU-8-MDtarger for MD using the covariance for only y=0. The choice ft=/ corresponds to AT with uniform perturbation constraint, which we call Uniform-6.
Pearson's 's correlation coefficient, Shapley
3.1
Malware Use-case
First, we consider binary classification problem for malware detection using the EMBER dataset 288 EMBER is a feature-based public dataset which is considered a benchmark for Windows malware detection. It contains 2381 features extracted from Windows PE files: 600/ labeled training samples and 200K test samples. We refer to Appendix A.2 for more detailed description of the EMBER dataset features. Given malware sample, adversary's goal is to make the DNN conclude that a malicious sample is benign. We also consider PDF malware detection. We use the extracted features of the PDF malware classification dataset and its attacked samples provided in B3]. The repository contains 110841 samples with 135 features that are extracted by PDFrate-R (4..
Attacks used for evaluation: In the malware domain, test-time evasion attacks can be classified as feature-space and problem-space attacks. While the former crafts AFs by modifying the features extracted from binary files, the latter directly modifies malware binaries making sure of the validity
5

---

and inconspicuousness of the modified object. We evaluate the robustness of our model against evasion attacks which are crafted in problem-space, i.e., on PE files. For Windows malware, we incorporate the most successful attacks 35 from the machine learning static evasion competition [36]. Since the EMBER dataset only contains the extracted features of a file, a subset of malware binaries 37 using the SHA-256 hash as identifier.
used for AE generation are obtained from VirusTotal
We observe that these problem-space attacks, which add various bytes to a file without modifying the core functionality, affect only the feature groups "Byte Histogram", "Byte Entropy Histogram" and "Section Information". Experts aware of these byte padding attacks understand which features can be manipulated by an attacker. In addition to the previous AT methods, we represent this best case expert knowledge by Ω mask- which is an identity matrix with non-zero diagonal elements only for "Byte Histogram", "Byte Entropy Histogram" and "Section Information" features. That is, the model trained using PGD perturbations applied only to these features, and we call it NU-ô-Mask. Our masking approach for the immutable features is similar to the conserved features in (8
For PDF malware classification, we use a problem space attack called EvadeML [39] It allows
adding, removing and swapping objects, hence it is a stronger attack than most other problem space attacks in the literature, which typically only allow addition to preserve the malicious functionality,
To make a fair comparison between uniform and non-uni non-uniform approaches, e for each method is selected such that their average distortion budgets, i.e., | For Windows malware classification, we test the detection success of adversarially trained models with 9000 AE sets generated by the problem-space attacks described in Appendix A.6 Figure Ta illustrates the average defense success rate against various problem-space attacks and shows that non-uniform perturbation approaches outperform the uniform perturbation in all cases. Moreover, NU-S-MDtargei performs closest to the best case expert knowledge NU-6-Mask for all cases except when ||0]| 25. The advantage of selecting Σ from benign samples versus selecting from the entire dataset is that the direction of perturbations are led towards the target class, i.c. benign samples, for NU-Ō-MDtarger. We also do not observe significant performance difference between and NU-8-SHA, while NU-5-MD only differs from the two for |011 = 25. We refer to Table A. for detailed attack performances and Table A.2 for defense S.R. results with clean accuracy.
Numeric results:
approximately equal.
For PDF malware classification, we compare NU-Ó- MDI with Uniform-ō against EvadeMI. We observe the best performances at ||ö||2 = 1 for both methods. Table the clean accuracy (Ac.), AUC score and defense success rate (D.S.R.) against EvadeML for standard training, Uniform-ō and NU-O-MDI. Al- though NU-6-MDI is feature space defense, the re- sults show that it highly effective against problem space attacks, and it outperforms Uniform-o. Since our approach does not assume any attack knowledge, it is more generalizable than the problem space defenses.
Clean accuracy (Ac.), AUC score and defense success rate (D.S.R.) against EvadeML for standard training, Uniform-ô and NU-6
Table I:
Model
|61|2 Clean Ac. AUC score D.S.R.
Std. training
99.52%
0.99912
11.1%
Uniform-ō
97.64%
0.99826
87.4%
NU-6-MDt
97.83%
0.99835
92.9%
3.2 Credit Risk Use-case
Our second use-case is a credit risk detection problem where the DNN's goal is to make decisions on loan applications for bank customers. For this scenario, use the well-known German Credit dataset [4]. which contains classes "good" and "bad", as well as applicant features such as age, employment status, income, savings, etc. It has 20 features and 1000 samples with 300 in the "bad" class. Similar to we treat discrete features as continuous and drop non-ordinal categorical features.
Attacks used for evaluation: The goal of an adversary in this situation is to make DNN models conclude that they are approved for a loan when they actually may not be eligible. Since modifications to tabular data can be detected by expert eye, attackers try to fool classifiers with imperceptible attacks. We use German Credit dataset implementation of LowProFool [17] which considers attack imperceptibilityy and represents expert knowledge using feature correlations. We apply the attack on the "bad" class of the test set and generate 155 AEs. After dropping the non-ordinal categorical features, we treat the remaining 12 features as continuous values.
Numeric results: Similar to the malware use-case, 6 for each method is selected such that their average ||ō|| are approximately equal. In Figure lb report defense success rate of PGD with

---

Table 2: Clean accuracy (CI. Ac.) and defense success rates of NU-6-MDr and Uniform-ó against
FGSM, Carlini- Wagner (CW), JSMA and Deep Fool attacks for Spam Detection Use-case.
Defenses
6
CL Ac.,
FGSM c0.55,
FGSM c=1,%
CW,%
JSMA,%
Deep Fool, %
Uniform-6
91.61
24.1 + 0.25
20.17 + 0.31
38.93 + 0.54
41.51 0.24
39.3 + 0.27 51.73 = 0.31
0.1
NU-6-MDU
91.93
28.7 0.17
27.1 = 0.21
49.62 + 0.33
49.59
0.18
Uniform-o
90.40
25.22 0.12
21.61 = 0.43
45.83 = 0.41
46.38 0.27
47.39 0.25
0.3
NU-6-MDt
91.31
32.15 0.34
30.88 + 0.36
53.45 + 0.22
52.68 0.13
54.61 0.19
Uniform-6
87.12
27.05 0.50
23.08 + 0.34
50.05 = 0.32
49.5
49.75 0.16
0.5
NU-6-MDU
87.78
43.85 - 0.62
36.12 + 0.20
55.24 0.38
53.71 0.41
64.28 = 0.55
Uniform-o NU-6-MDt
86.46
40.20 0.57
32.98 0.31
52.77 0.21
52.91 0.26
53.22 + 0.10
87.64
81.34 0.83
79.85 + 0.75
61.89 + 0.37
72.93
0.77
88.75 0.78
Uniform-6
85,98
74.40 :: 0.47
62.36 + 0.75
59.71 + 0.28
68.95 - 0.85
64.5 + 0.86 98. 36 + 0.25
1.5
NU-6-MDU
87.03
94.45 + 0.18
91.03 + 0.10
72.98 + 0.48
86.43 0.32
uniform and non-uniform perturbations in detecting 155 AEs generated by LowProFool. The figure shows that for every given |,0||2, non-uniform perturbations outperform uniform perturbations in PGD. Although LowProFool represents feature importance by Pearson correlation coefficients between features and the output score, surprisingly NU-S-Pearson is the best approach among the other non-uniform approaches for only 5 6= {0.7, 1). We refer to Table A.3 for clean accuracy results.
3.3 Spam Detection Use-case
Finally, we evaluate robustness within the context of detecting spam within social networks. We use a dataset from Twitter, where data from legitimate users and spammers is harvested from social honeypots over seven months [41]. This dataset contains profile information and posts of both spammers and legitimate users. After pre-processing 42 we extract 31 numeric features with 14 being integers and the rest being continuous. Some examples of these features of following and followers as well as the ratio between them, percentage of bidirectional friends, number of posted messages per day, etc. We treat all features continuous values in our experiments. Moreover, we extract 41, 1,354 samples where the training set has 17,744 "bad" and 15,339 "good" samples, and the testing set has 3885 "bad" and 4386 "good" samples. The adversary's goal is to make the DNN predict that a tweet was posted by legitimate user when it was written by a spammer.
the number
Attacks used for evaluation: We incorporate the evasion attack 43 from [6] for our Twitter spam detector. The attack strategy based on minimizing the maliciousness score of an AE which is measured by a local interpretation model LASSO, while satisfying Eg norm constraint on perturbations. We generate the AEs by constraining the perturbations to 0.5 x distavy pos-neg where di pos- neg is defined by [16] as the average distance between the spammer samples and the closest non nonnspaamerrs to these samples. We split the Twitter dataset with ratio 25% for training and testing, and generate the AES using the spammer class of the entire test set.
Again, we apply perturbations only to the spammer set during AT and report the results for approximately equal average ||ā]|2 perturbations. Figure Ic illlustrates defense success rate in detecting AES of the proposed approaches against the model interpretation based attack [16] for Twitter dataset. The figure shows that non-uniform perturbations outperform uniform case in terms We refer to Table A.4 for clean accuracy results.
Numeric results:
of defense S.R. for all given ||6 |2.
3.4 Performance Against Uniform Attacks
Throughout the experiments, we tested our non-uniform approach against various realistic attacks, such as problem space attacks in Section 3.1 feature importance-bar baseed attack in Section 3.2 and explainability-base based attack in Section 3.3 So far we emphasized that problem space attacks and non-uniformly norm bounded attacks are more realistic compared to the traditional uniformly norm- bounded attacks which are mostly considered in image domain. Yet, in this section, we also test our non-uniform approach against the well-known uniform attacks to investigate the generalizability of our approach. We use the setting for the spam detection use-case, and compare NU-8-MDi with Uniform-6. We utilize the adversarial robustness toolbox (ART) 44 to craft ABs by using the default parameters for the AE generators of Carlini-War Wagner (CW), JSMA and DeepFool Methods. We also
7

---

Lample
Unferm
(a) Uniform-6
(h) NtrgeMDt
(c) Histogram of MD square, 8 x y==
Figure 2: UMAP visualization of benign, malicious and adversarial samples generated by (a) Uniform- and (b) NU-A-MDiarget and (c) the density histogram of their 5 2C log
FGSM for 0.5 and c= 1. Table shows the clean accuracy (CI. Ac.) and defense S.R.'s of the robust models NU-5-MDr and Uniform-6. We observe in Table 2 that the Cl. Ac. decreases as || la increases for both Uniform-ō and NU-S-MDr but the degradation in non non-uniform is less. Defense S.R.',, on the other hand, improve for both approaches but NU-6-MDI significantly outperforms Uniform-6 in all cases. See Appendix A.7 for the performance against uniform PGD attacks.
3.5 Quality of Perturbation Sets
In this section, we quantitatively and qualitatively analyze how well non-uniform perturbations capture realistic attacks using 7-consistency property defined in Section and lower dimensional space visualization. Our intuition is that a successful attack evades detection since AEs appear benign to the model. That is, AEs have high likelihood according to the distribution of benign samples. Therefore, we measure a perturbed sample's ' quality by its y-consistency with the benign set distribution. Definition 2. leverages Theorem 2.1 which shows that smaller MD for indicates higher n-consistency and hence higher quality of the perturbed sample. Moreover, we expect AEs that evade the model and benign samples to be embedded closer each other in the lower-dimensional subspace. Figure BigureBillustrenn iillustrates UMAP visualization (45) of benign, malicious and adversarial samples for the spam detection use-case. AEs generated by NU-6-MDtarget show better alignment with benign distribution, which shows that NU-g-MDtarget mimics more realistic attack. We also show the histogram of MD squares, ie. E = 2C - 2 2og y, of 1660 AEs from Uniform-ó and NU-G-MDDarget in Figure 2cl where the average values are 2.1 and 1. 1.28, respectively. Following Thcorem 2.1 and Figure 2c. 8's from NU-ô-MDtarget have higher 7, and hence, are more realistic.
4 Certified Robustness with Non-uniform Perturbations
In this section, we present methods for certifying robustness with non-uniform perturbations. We consider two well-known methods; linear programming (LP) [9] and randomized smoothing [46].
4.1 LP Formulation
We can provably certify the robustness of deep ReLU networks against non- non-uniform adversarial perturbations at the input. Our derivation follows an LP formulation of the ad adversary's 's problem with ReL.U relaxations, then the dual problem of the and activation bound caleulation. It be viewed as an extension of 9.. Similar to 19, we consider a k layer feedforward deep ReL.U network with
2+1 Wiz + by 2, max(2,, 0}, for i -,..... -
We denote (z) ([(()) = (z (z 8) || < c) as the set of all attainable final-layer activations by input perturbation Since this a non-convex set for multi-layer networks which is hard to optimize over, we consider convex outer bound on (z)) and optimize the worst case loss over this bound to guarantee that no AES within (x,)(x) can evade the network. As done in (9). relax the ReLU activations by representing
(0, )} with their upper convex envelopes

---

2 2 0, 2, -uē + (u- -ul, where 1 and u are the known lower and upper bounds for the pre-ReLU activations. We denote the new relaxed set of all attainable final-layer activations by (x). Assuming that an adversary targets specific class to fool the classifier, we write the LP as
minimize Eu
c²
2x
S.L.
2x
Ze,
(8)
where c::= Cytrue - Cytarget is the difference between the selection vector of true and the target class.
A positive valued objective for all classes as solution to equation g indicates that there is no adversarial perturbation within Ar,, which can evade the classifier. To be able to solve equation 8 in tractable way, we consider its dual whose feasible solution provides a guaranteed lower bound for the LP. It is previously shown by [9] that feasible set of the dual problem can be formulated similar to standard backpropagation network and solved efficiently. The dual problem of our LP with ReLU relaxation and non- non-uniform perturbation constraints is expressed in the following theorem.
Theorem 4.1. The dual of the lineur program can be writlen
k-1
maximize U2M
Σ Euti
ΣΣ P.sl oJ e
1=2 jeI,
Uk -,, vi (W? (41)), for i- k 1.....1
s.l.
(9)
j E I
Vijj
Vig U1. Ui.
j E I for i i 1,...,,
[igg++ m.slp
j E IИ
where IT It and I, represent the activation sets in layer for and u are both negative, both positive and span zero, respectively.
See Appendix A.4 for the proof of Theorem When Th. us, Theorem 4.1 shows that the
dual problem can be represented as linear back propagation network, which provides tractable solution for a lower bound of the primal objective. To solve equation and upper bounds for each layer incrementally as explained in Appendix A.5
9 we need calculate lower
For certification of robustness within a non-uniform norm ball around a test sample, we need the objective of the LP to be positive for all classes. Since the solution of the dual prob- lem is a lower bound on the pri- mal LP, it provides worst case certification guarantee against the AEs within the non-uniform norm ball. We provide certification re- sults for the robustness of Uniform- 5 and NU-5-MDr (NU-8-MDtarge for spam detection use-case Table 3 We consider both uniform and non-uniform input constraints in cer-
Table Average certification margin and number of successful certified samples out of 1000 spammers for NU-5-MDi and Uniform-6 for Spam Detection Use-case.
Model Defense S.R. Cert. Method
Margin
Cert. Success
Uniform-
1.07
34.72 + 0.94%
NU-Cert-SHAP
1.84
72.64 +: 0.6%
Uniform-6 54.87 + 1.1%
NU-Cert-Peange
2.04
76.8 : 0.71%
NU-Ceri-MD
2.40
80.2 + 0.56%
NU-Cert-MDt
2.40
80.2 + 0.55%
Uniform-Cert
1.11
42.95 + 0.69%
NU-Cert-SHAP
74.65 + 0.85%
NU-A-MDr 63.4 0.74% NU-Cert-Pearson
2.06
78.38 + 0.76%
NU-Cert-MD
2.41
81.3 0.68%
NU-Cert-Mil
2.41
81.3 0.67%
tification, namely Uniform-Cert for the standard LP approach for certi- fication with uniform perturbation constraint [9] and NU-Cert-(.) for the non-uniform constraint. We implement our non-uniform approach into the LP by modifying [ with our S matrix, and generate various certification methods by non-uniform Q e.g. NU-Cert-SHAP NU-Cert-Pearre NU-Cert-MD and WU-Cert-MDr, Our purpose is not to propose the tightest certification bounds but to show that non-uniform constraints result in larger certification margins compared to the uniform case.
We compare Uniform-ō and NU-S-MDI to evaluate certification results. Dropout layers are removed from the model for LP solution, and AT is performed for e = 0.3. Certification is done by solving the IP for 0.3 over 1000 spammers. The objective should be positive for all classes to certify the corresponding sample. The margin between the objective and zero gives idea about how

---

tight the bound is 447. Table 3 demonstrates two main results: (i) the certification success of NUta-MDt over Uniform-o for each certification method supports our claim that non-ume unn-unfform perturbations provide higher robustness than the uniform approach; and (ii) certification with non- uniform constraints provide larger certification margins and hence tighter bound.
4.2 Randomized Smoothing
Robustness certification via randomized smoothing 44 an empirical alternative to the LP. The idea is constructing "smoothed" classifier g from the base classifier f. In the original formulation in [6]. returns the most likely output returned by f given input æ is perturbed by isotropic Gaussian noise. Here, we provide robustness guarantee in binary when non-isotropic Gaussian noise is used to allow robustness to non-uniform perturbations:
for randomized smoothing framework
g = arg max (( + n)) = y) BEy
where 7~ (0,, E).
(10)
Adapting notation and Theorem from [46] let Pa be the probability of the most probable class y = a when the base classifier f classifies (r, 2). Then the below theorem holds.
Theorem 4.2. In binary classification problem, suppose Pa ($) satisfies P(f(x + n) a) 2 Po-
Then g( + 6) = a for all V8T S 47 (Pa) ) 95a. where = VoTΣ-15, Φ (pah(pa) is the quantile function fuantttlefunction of the x distribution of d degrees of freedom, and ysu is the 50 quantile.
we show that smoothed classifier g is robust around r within & Mahalanobis distance V8-Σ-16 T - 950 where Φ (pa) the quantile function for probability Pa- The same result holds if we replace pa with lower bound Pa-
See Appendix (..eendxxxA.44ee for the proof. In theorem 4.2
We implement non-uniform approach into randomized smoothing by modifying [48] with our non-isotropic noise space. Table certification S.R. of Uniform-8 and NU-8- MDt, when they are certified by standard randomized smoothing with (0, a 7) (UC), and our non- uniform methods with NO., E.,)
Table 4: Percentage successfully certified samples for NU-8- MDr and Uniform-ō with various certification approaches with randomized smoothing for Spam Detection use-case.
Model
UC
NUC-Pcarson NUC-SHAP NUC-MD NUC-MDL
Uniform-ó 50.96%
61.11%
64.24%
65.72%
66.45%
NU-6-MDI 61.8%
67.14%
71.25%
85.34 35.34%
90.11%
for corresponding Ey. That is, Ey=o for NUC-MDt, (0= NUC-MD, 3² for NUC-Pearson
and / for NUC-SHAP are used when the average training distortion budget is || and the average certification distortion is [| Table shows that NU-O-MDt is certifiably robust for more samples than Uniform-ō for all certification methods. Moreover, certification with non-uniform noise, especially with NUC-MDt, provides higher certification S.R. compared to uniform noise.
5 Conclusion
In this work, we study adversarial robustness against evasion attacks, with a focus on applications where input features have to comply with certain domain constraints. We assume Gaussian data distribution in our consistency analysis, as well as precomputed covariance matrix and Shapley values. Under these assumptions, our results on three different applications demonstrate that non-uniform perturbation sets in AT improve adversarial robustness, and non-uniform bounds provide better robustness certification. As unintended negative social impact, our insights might be used by malicious parties to generate AEs. However, this work provides the necessary defense mechanisms against these potential attacks.
Acknowledgments and Disclosure of Funding
We would like to thank Mohamad Ali Torkamani for his help and valuable feedback, and thank Baris Coskun for his endless support as the leader of the Security Analytics and AI Research (SAAR) Team AWS, as well as the team members for the fruitful discussions. This work has been done as a part of Ecenaz Erdemir's internship in AWS, and no outside funding was used in the preparation.
10

---

References
[1] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint 1tl::1700..6000 2017.
[2] Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Sindić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery databases, pages 387-402. Springer, 2013.
[3] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. arXiv preprini arXiv: 1312.6199 2013.
[4] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. arXiv preprint arXiv:1412.6572, 2014.
[5] Krishnamurthy Dvijotham, Robert Stanforth, Sven Gowal, Timothy A Mann, and Pushmeet Kohli. A dual approach to scalable verification deep networks. In UAI, volume 1, page 2, 2018.
[6] Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, and Martin Vechev. Ai2: Safety and robustness certification of neural networks with abstract interpretation. In 2018 IEEE Symposium on Security and Privacy (SP), pages 3-18. IEFE, 2018.
[7] Gagandeep Singh, Timon Gehr, Matthew Mirman, Markus Püschel, and Martin Vechev. Fast and effective robustness certification. Advances in Neural Information Processing Systems, 31: 10802-10813, 2018
[8] Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Semidefinite relaxations for certifying robustness to adversarial examples. Advances Neural Information Processing Systems, pages 10877-10887, 2018
[9] Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In International Conference on Machine Learning, pages 5286-5295. PMLR, 2018
[10] Huan Zhang, Tsui-Wei Weng, Pin-Yo Yu Chen, Cho-Jui Hsieh, and Luca Daniel. Efficient neural network robustness certification with general activation functions. In Advances in neural information processing systems, pages 4939-4948, 2018.
[11] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint arXiv:1611.012366 2016.
[12] Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. arXiv preprini wrXiv:2001.03994, 2020.
[13] Shiqi Wang, Yizheng Chen, Ahmed Abdou, and Suman Jana. Mixtrain: Scalable training of formally robust neural networks. arXiv preprini arXiv:1811.0262. 14, 2018.
[14] Matthew Mirman, Timon Gehr, and Martin Vechev. Differentiable abstract interpretation for provably robust neural networks. In International Conference on Machine Learning, pages 2018.
[15] Mary Frances Zeager, Aksheetha Sridhar, Nathan Fogal, Stephen Adams, Donald E Brown, and Peter A Beling. Adversarial learning in credit card fraud detection. In 2017 Systems and Information Engineering Design Symposium (SIEDS), pages 112-116. IEEE, 2017.
[16] Ninghao Liu, Hongxia Yang, and Xia Hu, Adversarial detection with model interpretation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & 2018 2018.
Data Mining, pages 1803-1811,
11

---

[17] Vincent Ballet, Xavier Renard, Jonathan Aigrain, Thibault Laugel. Pascal Frossard. and Marcin Detyniecki. Imperceptible adversarial attacks on tabular data, arXiv preprint orXiv:1911.0322 2019.
[18] Eden Levy, Yael Mathov, Ziv Katzir, Asaf Shabtai, and Yuval Elovici. Not all datasets are born equal: On heterogeneous data and adversarial examples. arXiv preprint arXiv:2010.03//0, 2020.
[19] Deqiang Li and Qianmu Li. Adversarial deep ensemble: Evasion attacks and defenses for malware detection. IEEE Transactions on Information Forensics and Security, 15:3886-3900, 2020.
[20] Fabio Pierazzi, Feargus Pendlebury, Jacopo Cortellazzi, and Lorenzo Cavallaro. Intriguing properties of adversarial ml attacks in the problem space. In 2020 IEEE Symposium on Security and Privacy (SP), pages 1332-1 1349. IEEE, 2020.
[21] Ishai Rosenberg, Shai Meir, Jonathan Berrebi, Iaay Gordon, Guillaume Sicard, and Eli Omid David. Generating end-to-end adversarial examples for malware classifiers using explainability, In 2020 International Joint Conference on Neural Networks (IJCN), pages 1-10. IEEE, 2020.
[22] Wenbo Guo, Dongliang Mu, Jun Xu, Purui Su, Gang Wang, and Xinyu Xing. Lemna: Explaining deep learning based security applications. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS '18, page 364-379, New York, NY, USA, 2018. Association for Computing Machinery.
[23] D. Tsipras, Shibani Santurkar, L. Engstrom, A. Turner, and A. Madry. There no free lunch in adversarial robustness (but there are unexpected benefits). ArXiv, abs/1 /e///100..1115 2018.
[24] Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. Advances in Neural Information Processing Systems, volume 32, pages 125-136. Curran Associates, Inc., 2019.
[25] Chen Liu, Ryota Tomioka, and Volkan Cevher. On certifying non-uniform bounds against adversarial attacks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 4072-4081, Long Beach, California, USA, 09-15 Jun 2019. PMIR.
[26] Eric Wong and J. Zico Kolter. Learning perturbation sets for robust machine learning. arXiv preprint arXiv: 004.008450 2020.
[27] Feargus Pendlebury, Fabio Pierazzi. Roberto Jordaney, Johannes Kinder, and Lorenzo Cavallaro. (TESSERACT]; Eliminating experimental bias in malware classification across space and time. In 28th [USENIX} Security Symposium (( (UUSENIX) Security 19), pages 729-746, 2019.
[28] Hyrum S. Anderson and Phil Roth. EMBER: an open dataset for training static PE malware machine learning models. arXiv preprint arXiv: 1804.04637, 2018
[29] Prasanta Chandra Mahalanobis. On the generalized distance in statistics. Proceedings of the National Institute of Sciences, 2:49-55, 1936.
[30] Lloyd S. Shapley. Notes on the P-Person Game II: The Value of an n-Person Game. RAND Corporation, Santa Monica, CA", 1951.
[31] Haotao Wang, Tianlong Chen, Shupeng Gui, Ting- Hu. Ji Liu, and Zhangyang Wang. Once-for-all adversarial training: In-situ tradeoff between robustness and accuracy for free. arXiv preprint arXiv:2010.11822 2020.
[32] Scott Lundberg and Su-In Lee. A unified approach to interpreting model predictions. 12 2017.
[33] Pdffrate. URL https://github... http://iihhuu.coommoccutttttaaaa
[34] C. Smutz and A. Stavrou. Malicious pdf detection using metadata and structural features. In 28th Annual Computer Security Applications Conference, page 239-248, New York, NY, USA, 2012.
12

---

[35] William Fleshman. Evading Machine Learning Malware Classifiers, 2019.
URL
http:://oooarddsataaaccooo.o wwwp://iennn.ccceeeeeeeeeeeeee hew.//vaaninnggeee maeggg--aci-----ee wwahine-learning malware-
classiffeers-cce2ddddbb11.
[36]
M learning static evasion competition, 2019. URL htwph:ttpp//// www.elastic
competition
[37]
VirusTotal. URL htt.Lttt:://w.virustttaalll
[38]
Liang Tong, Bo Li, Chen Hajaj, Chaowei Xiao, Ning Zhang, and Yevgeniy Vorobeychik, Improving robustness of ML classifiers against realizable evasion attacks using conserved
features. In 28th USENIX Security Symposium (USENIX Security 19), pages 285-302, Santa
Clara, CA, August 2019. USENIX Association. ISBN 978-1-939133-06-9, URL https
7/www /www usenix x. ort/////fcaann//aaaaaaaaaaattaar
[39] Weilin Xu, Yanjun Qi, and David Evans. Automatically evading classifiers: A case study on pdf malware classifiers. In NDSS, 2016.
[40] C. J. Merz and P. Murphy. UCI repository of machine learning databases, 1996. URL http:://wrr
uci adu/"mlearn, wwnlearn/MLRRpositoor .hmml
[41] Kyumin Lee, Brian David Eoff, and James Caverlee. Seven months with the devils: A long-term study of content polluters on twitter. In 5th International AAAI Conference on Weblogs and Social Media (ICWSM), Barcelona, 2011.
[42] Xiao Huang. xhuang1994
Twiter bot detection, 2017. URL
[43] Ninghao Liu, Hongxia Yang. and Xia Hu. Interpretation to adversary. 2018. URL https
//github. //github.ccm//iinii http://iinniieeeoommmmeeeeaaraaa
[44] Maria-Irina Nicolae, Mathieu Sinn, Minh Ngoc Tran, Beat Buesser, Ambrish Rawat, Martin Wistuba, Valentina Zantedeschi, Nathalic Baracaldo. Bryant Chen, Heiko Ludwig, Ian M. Molloy, and Ben Edwards. Adversarial robustness toolbox v1.0.0, 2019.
[45] Leland Mclnnes, John Healy, Nathaniel Saul, and Lukas Grossherger. Umap: Uniform manifold approximation and projection. The Journal of Open Source Software, 3(29):861, 2018.
[46] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. arXiv preprint 1902.0299022 2019.
[47] Hadi Salman, Greg Yang, Huan Zhang, Cho-Jui Hsieh, and Pengchuan Zhang. Benchmark for Ip-relaxed robustness verification of relu-networks. 2019. URL https:///giiee com/
[48] Jeremy Cohen, Elan Rosenfeld, and and Zico Kolter. Certified adversarial robustness via randomized smoothing. 2019. htt.at/::/gggitub..m//////ssssss
13


Reviews:
[{"review_id": "nUgz1tayZzL", "reviewer": "Reviewer_QUCv", "summary": "In domains other than computer vision, a uniform bound for adversarial perturbations may not make much sense as different input features can have divergent value ranges and meanings. For instance, in malware detection, certain features are not allowed to be perturbed in order to preserve the functionality of subject programs. This paper tackles the robustness problem in this particular setting where non-uniform perturbations are required for crafting adversarial examples. Specifically, the paper introduces a transformation matrix $\\Omega$ to first weight the perturbations before calculating the norm bound. It then uses such a weighted norm to perform adversarial training. Three types of transformation matrices are evaluated for denoting the importance of different input features. The experiments are conducted on three use cases, namely, malware detection, credit risk detection, and spam detection. The results show that using non-uniform perturbations leads to better robustness than using uniform ones on evaluated use cases.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The motivation of this paper is interesting and reasonable. It is indeed important to design problem-specific perturbation bounds for certain domains. I particularly like the motivation case where not all the features are allowed to be perturbed for maintaining malware functionality. I do see the value of such studies in domains other than computer vision. I have a few questions for the authors.\n\n1. I can understand that we need domain knowledge to construct the transformation matrix $\\Omega$ to achieve domain-specific constraints. But the paper mainly talks about different types of metrics for measuring the feature importance such as Mahalanobis distance, Shapley values. How are these two connected? It seems disconnected between the motivation and the actual addressed problem. That said, why does using those feature importance metrics have better results than a uniform bound? The gradient during adversarial generation already tells us the importance of different features. The paper mentions that the Mahalanobis distance can help generate adversarial examples \"with feature correlations similar to the original dataset\". This sounds reasonable. Can the authors empirically validate this argument?\n\n2. The existing work [25] is closely related to this work as stated in the paper. It certifies non-uniform bounds for models. In Section 4, this paper also tries to certify the robustness with non-uniform perturbations. Why is work [25] not studied in the paper? A straightforward experiment would be just using work [25] to certify original models and models trained by this paper. This can illustrate how the non-uniform bound change before and after training.\n\n3. For the transformation matrix $\\Omega$, is it calculated once before the adversarial generation or every iteration during the generation? What is the time cost for the non-uniform adversarial training compared to the uniform one?\n\n4. At lines 188-189, the paper states that \"to model the expert knowledge with diagonal $\\Omega$, we use Pearson’s correlation coefficient, Shapley values, and masking to allow perturbation only in mutable features\". Did the authors also mask out those immutable features when training with Pearson’s correlation coefficient, Shapley values? How about the training with the Mahalanobis distance? If not, I am not sure how these metrics can mask out immutable features. It also comes back to my first point that the paper does not seem to address the problem introduced in the motivation. Could the authors care to explain?\n\n5. In Table A.1 in the appendix, why do adversarially trained models have worse performance on C2 Pad. attack than the standard training? Almost all the adversarial training methods are worse than the standard training. Have the authors looked into this?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have partially addressed the limitations. Please see details in the main review.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "jwfX-ebr-_i", "reviewer": "Reviewer_txFx", "summary": "This paper extends adversarial training to train robust classifiers against non-uniform adversarial perturbation for various security-critical applications (e.g., malware classification, credit risk prediction).\n", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1. The problem is very important and interesting. For instance, adversarially generated malware may cause serious security concerns. Therefore, how to train robust classifiers in security-critical domains is an important research problem.\n\n2. The improvement of the proposed approach is marginal compared with adversarial training with uniform adversarial perturbations. As shown in Figure 1(a) and 1(b), the proposed approach only improve the baseline for several percent regarding defense success rate.\n\n3. The technical contribution of this work is weak. The adversarial training is a very standard method to train robust classifiers. The extension of adversarial training to non-uniform adversarial perturbations is not challenge. \n\n4. A baseline is that the defender could leverage the adversarial examples crafted in the problem-space to perform adversarial training. Could the authors share more details (or comparison if possible) for this?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1. The performance improved over the baseline is marginal.\n\n2. The technical contribution is limited.\n\n3. The certification for non-uniform adversarial perturbation is a simple generalization of existing studies.\n\nWriting:\n\n1. \\sum_{y=0} for NUC-MDt, \\sum_{y={0,1}} NUC-MD. This is hard to understand. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "ZdUHFltmHfY", "reviewer": "Reviewer_kjQY", "summary": "The authors have proposed non-uniform adversarial perturbation generation algorithms to attack the classifiers trained on heterogeneous databases such as malware detection and finance. ", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "How the data dependency is incorporated in the proposed framework and what is the limitation of such dependency?\n\nWhat are the parameters of the chosen PGD attack?\n\nAs compared to the claim in the paper, the performance of the proposed attack is comparable to a single chosen uniform attack. Are the values reported in the Figure statistically significant? For example in Figure 1b, the difference in the values between existing and multiple variants of the proposed attack is 1-2%. A similar observation can be drawn from other results as well.\n\nAnother limitation is that only a single attack and single network architecture is chosen for comparison with uniform perturbation? \n\nIn appendix A.3, \"This approach does not guarantee same range (min and max) for all features.\n58 As a result, it is possible that the features have different scales even after normalization\", then what normalization the authors have applied, and why does the above standardization reflect such observation?\n\nThe paper listed multiple appendices for the readers to refer to. It would be great if the authors can bring all the sufficient and critical information in the paper to make the understanding clear.\n\nWhy the proposed attack is applied to the bad class? What if the perturbation is applied to the data without knowing the class?\n\nThe analysis of table 1 is not clear. What to refer to from the table? What does the value mean, the higher the value, better the attack or vice versa, etc?\n\nContrary to figure 2, it seems from the values in Table 1 and Table 2 that the proposed attack is easy to defend as compared to uniform perturbation. \n\nHow the precomputed values such as covariance matrix and Shapley values are obtained? have the authors noticed any data distribution apart from taking the assumption of Gaussian distribution? What is the impact if other distributions come?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The limitation of the work is well explained in the paper which might be due to the lack of ablation study concerning existing attacks, statistical analysis, ablation analysis in the absence of the assumptions, etc.\n\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "Wu-lBWF7qTm", "reviewer": "Reviewer_cq1G", "summary": "This paper proposes a method to generate non-uniform adversarial perturbation for applications where features normally have semantically meaningful dependencies, including malware detection, credit risk prediction and spam detection. Specifically, two choices are introduced for the non-uniform perturbation sets: one based on the Mahalanobis distance, and another one based on Pearson’s correlation coefficient or Shapley values. In addition, the authors modify two certification methods, linear programming and randomized smoothing, to provide provable guarantees for non-uniform robustness.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n1. It is novel to consider feature importance, feature correlations and domain knowledge when generating adversarial perturbations.\n2. Evaluations on multiple tasks demonstrate the effectiveness of the proposed methods.\n3. The authors provide certified robustness with non-uniform perturbations.\n\n\nWeakness:\n\nI have the following questions on the evaluation part:\n1. Regarding the setup of the experiment, are the non-uniform defence approaches tested against non-uniform perturbations, while the uniform defence approaches tested against uniform perturbations? What if the non-uniform defence approaches are tested against uniform perturbations, or the other way around?\n\n2. Does the new approach have any impact on the efficiency of adversarial training? It is mentioned that the covariance matrix and Shapley values are precomputed. How does this affect the results? \n\n3. In Fig. 1, why does the defence success rate increase with $||\\delta||_{2}$?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have addresses both issues.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "RnDBlVi6J5o", "reviewer": "Reviewer_z45J", "summary": "This paper is motivated by the fact that prior work on crafting adversarial examples focuses on perturbations which perturb the different features uniformly. However, they argue that such uniform perturbations do not result in realistic adversarial examples in general, since features often have semantic dependencies. To address this, their paper enables consideration of non-uniform perturbations, and demonstrates this approach using adversarial training on a number of problems using real-world datasets. In addition, they consider the effectiveness of the non-uniform bounds approach in improving the efficiency of two different techniques for certified robustness.", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The approach of the paper appears to be original and largely technically sound. However, their use of non-uniform perturbations is quite a small and simple amendment to established techniques. As such, the results I would say are not of huge significance to those working in the area. In fact, in some cases, including some of those considered, the approach essentially amounts to no more than a change of basis for the features.\n\nI had a few problems with the clarity of the motivation. Mention is made of basing the non-uniformity on either the importance of the features or on correlations between the features (in general, these are different). The claim is made that these approaches result in more realistic adversarial examples. However, this is not necessarily guaranteed. One can imagine that perturbation of some features could be very important to classification, but not be a realistic proposition if there are dependencies to satisfy. It feels like the approach of the paper, rather than using more realistic adversarial examples, works simply because it is focussing perturbations on the most relevant features. This is not a bad thing to do, but I think the motivation and explanation are somewhat confused. i.e. is the guiding motivation realistic adversarial examples or ones that are more effective in adversarial training?\n\nAlso, it is argued, correctly, in the Introduction that there can be certain constraints/dependencies amongst the features. However, the subsequent approach does not really allow for such dependencies in general as it is simply using a multi-variate Gaussian for the perturbations. For example, if feature 1 is always equal to twice feature 2, or say bounded below by 3 times feature 3, these dependencies cannot be accommodated in the approach. That is, the paper conflates dependencies with correlations.\n\nMore generally too, without a human in the loop to constrain perturbations to be imperceptible, larger perturbations are allowed, e.g. in their malware examples, huge changes can be made as long as the malicious functionality of the executable is not changed. This is also not covered by the proposed scheme of small non-uniform perturbations.\n\nThough the above points are not saying the approach is incorrect – just that it is not as general as the introduction and motivation would lead one to suppose.\n\nSome minor comments and typos:\n-\tCan the non-uniformity introduced, i.e. $\\Omega$, be a function of $x$?\n-\tTheorem 2.1 is largely stating the obvious in terms of the correlation between gamma-consistency and epsilon – and the exact result does not, as far as I can see, get used subsequently. So why is this theorem needed?\n-\tTheorem 2.1 confuses the probability density function with P(X=x).\n-\tTheorem 2.1 does not state over what range of values of x $P(X=x|y)>=\\gamma$.\n-\tIn equation 5, should the argument of P on the LHS be $\\delta$ and not $\\Omega \\delta$?\n-\tIt is not clear in the examples if the different features are normalised first. Part of the non-uniformity imposed would then correspond to simply normalising the features.\n-\tIn the paragraph beneath Theorem 4.2, it is argued that the result holds if $p_a$ is replaced by a lower bound. But the theorem is already stated in terms of the lower bound.\n-\tThe statement of Theorem 4.2 in the supplementary material (erroneously referred to there as Theorem 5.2) essentially lumps the key part of the proof of the theorem into a “it can be shown that” and so is not really a proof. But reference can be made to Cohen et al for details of the proof anyway.\n\nEdit: Score revised following discussion period.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors address potential societal impacts adequately.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes to improve the adversarial robustness by using non-uniform perturbations in adversarial training. After the rebuttal and discussion, the reviewers raised several concerns on the novelty, the marginal improvement over baselines, and some experimental settings. The paper could be strengthened if these concerns could be addressed. 
Decision: Accept (Poster)