Adversarial Robustness with Non-uniform Perturbations
Authors: Ecenaz Erdemir
Jeffrey Bickford
Luca Melis
Sergul Aydore
Venue: NeurIPS 2021 Poster (2021)
Keywords: Adversarial robustness
adversarial attacks
projected gradient descent
certified robustness
randomized smoothing

Abstract: Robustness of machine learning models is critical for security related applications, where real-world adversaries are uniquely focused on evading neural network based detectors. Prior work mainly focus on crafting adversarial examples (AEs) with small uniform norm-bounded perturbations across features to maintain the requirement of imperceptibility. However, uniform perturbations do not result in realistic AEs in domains such as malware, finance, and social networks. For these types of applications, features typically have some semantically meaningful dependencies. The key idea of our proposed approach is to enable non-uniform perturbations that can adequately represent these feature dependencies during adversarial training. We propose using characteristics of the empirical data distribution, both on correlations between the features and the importance of the features themselves. Using experimental datasets for malware classification, credit risk prediction, and spam detection, we show that our approach is more robust to real-world attacks. Finally, we present robustness certification utilizing non-uniform perturbation bounds, and show that non-uniform bounds achieve better certification.

Adversarial Robustness with
Non-uniform Perturbations
Ecenaz Erdemir Imperial College London e, erdemir17@@mppriall ac. uk
Jeffrey Bickford Amazon Web Services jbickoamazon.c com
Luca Melis Amazon Web Services
micmeli@amazen com
Sergiil Aydöre Amazon Web Services
saydore@amazon.co com
Abstract
Robustness of machine learning models is critical for security related applications. where real-world adversaries are uniquely focused on evading neural network based detectors. Prior work mainly focus crafting adversarial examples (AEs) with small uniform Norm-bounded perturbations across features to maintain the require- ment of imperceptibility. However, uniform perturbations do not result in realistic AEs in domains such as malware, finance, and social networks. For these types applications, features typically have some semantically meaningful dependencies. The key idea of our proposed approact is to enable non-uniform perturbations that adequately represent these feature dependencies during adversarial training. the empirical data distribution, both on correla-
We propose using characteristics tions between the features and the importance of the features themselves. Using experimental datasets for malware classification, credit risk prediction, and spam detection, we show that our approach more robust real-world attacks. Finally, we present robustness certification utilizing non-uniform perturbation bounds, and show that non-uniform bounds achieve better certification.
Introduction
Deep neural networks (DNNs) are commonly used a wide-variety of security-critical applications such self-driving cars, spam detection, malware detection and medical diagnosis (T. However, DNNs have been shown to be vulnerable to adversarial examples (AEs), which are perturbed inputs designed fool the machine learning systems 4 To mitigate this problem, line of research has focused on adversarial robustness of DNNs as well as the certification of these methods [
Adversarial training (AT) is of the most effective empirical defenses against adversarial attacks [1. The goal during training is to minimize the loss the DNN when perturbed samples are used. This way, the model becomes robust to real-world adversarial attacks. Though these empirical defenses do not provide theoretically provable guarantees, they have been shown to be robust against the strongest known attacks [12.. Some of the most common state-of-the-art adversarial attacks, such as projected gradient descent (PGD) ① and fast gradient sign method (FGSM) 12), perturb training samples under norm-ball constraint to maximize the loss of the network. The goal of certification, on the other hand, is to report whether an AE exists within an E, norm centered at given sample with fixed radius. Certified defense approaches introduce theoretical robustness guarantces against norm-boundes perturbations 8 9
Coode available at btttps://githne
nonuniform- wtw.roormwpppuuuubtbrroooooo
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

In the computer vision domain, the adversary's goal is to generate perturbed images that cause misclassifications in a DNN. It often assumed that limiting uniform norm-ball constraint results in perturbations that are imperceptible to the human eye. However in other applications such as fraud detection [5], spam detection [66], credit card default prediction [7 18 and malware detection [19221] norm-bounded uniform perturbations may result in unrealistic transformations. Perturbed samples must comply with certain constraints related to the domain, hence preventing us from borrowing these assumptions from computer vision. These constraints can be on semantically meaningful feature dependencies, expert knowledge of possible attacks, and immutable features This paper proposes methodology to generate non-uniform perturbations that takes into account the characteristics of the empirical data distribution. Our results demonstrate that these non-uniform perturbations outperform uniform norm-ball constraints in these types of applications.
20 22
1.1
Background and Motivation
AT is min-max problem minimizing the DNN loss which is maximized by adversarial perturbations (call 8). State-of-the-ar approaches for optimizing ó usually assume that all the imput features require equal levels of robustness, however, this might not be the case for many applications. A toy example for AT with non-uniform perturbations is given in Appendix A The intuition behind the need for non-uniform constraints is apparent across many industrial applications. A common cybersecurity application is malware detection, which identifies if an executable file is benign or malicious. Unlike images, diverse and semantically meaningful features are extracted from the executable file and are passed to a machine learning model. To maintain the functionality of an executable file during an adversarial attack, certain features may be immutable and perturbations may result in an unrealistic scenario. For example in the Android malware space, application permissions, such as permission to access a phone's 's location service, are required for malicious functionality and cannot be perturbed [19]. In finance scenario where customer credit card applications are evaluated by machine learning models, a possible set of features include age, gender, income, savings, education level, number of dependents, etc. In this type of dataset there are clear dependencies between features, for example the number of dependents has a meaningful correlation with age. When detecting spammers within social networks, features are extracted from accounts and may include the length of the username, length of user description, number of following and followers as well as the ratio between them, percentage of bidirectional friends, etc. Similar to the previous finance example, there is meaningful correlation between features such as the percentage of bidirectional friends and the ratio of followers.
In all of these scenarios, non- non-uniform perturbations can be used to maintain these correlations and semantically meaningful dependencies resulting in more realistic AEs. In this work, we propose adversarial training with these more realistic perturbations to increase the robustness against real- world adversarial attacks. Specifically, our contributions are: (i) Instead of considering an allowed perturbation region where all the features are treated uniformly. i.e., ||0'| I c, we consider a transformed input perturbation constraint, i.e., ||(25]Ip I where Q is a transformation matrix, which takes the available information into account, such as feature importance, feature correlations and/or domain knowledge. Hence, the transformation uniform input perturbations over the features. (ii) For various applications such as malware detection, credit risk prediction and spam detection, we show that robustness using non-uniform perturbations outperforms the commonly-used uniform approach. (ii) To provide provable guarantees for non- uniform robustness, we modify two known certification methods, lincar programming and randomized smoothing, to account for non-uniform perturbation constraints.
the norm ball constraint results non-
1.2 Related Work
Different levels of robustness of different features have already been studied in the literature 23 25 Among these, (23] and (24] study the effect of robust and non-robust features in standard and adversarial training. Both works show that different types of features might either be vulnerable or robust to small input perturbations. This is a discrete interpretation of different levels of perturbation tolerance which is the core idea of our approach. Both works provide effective theoretical analysis of robust and non-robust features and their effect on clean and adversarial accuracy, while we propose a defense mechanism utilizing this phenomenon.
closely related work to ours is 25 which considers non-uniform perturbation bounds for input features. Given a non-uniform adversarial budget for loo norm bounded inputs, (25) proposes
2

---

framework that maximizes the volumes of certified bounds. However, this work only proposes robustness certification of pre-trained models, and does not consider training robust model against these non-uniform perturbations. The approach is also data-agnostic meaning that it does not take correlations or an additional knowledge on the data into account. We instead use non-uniform perturbations in data-dependent AT to achieve robustness in DNN model. In fact, [2 mentioned consideration of feature correlations as a potential future direction of their work.
In [26] conditional variational autoencoder (CVAE) is trained to learn perturbation sets for image data and the generated adversarial images are then used in augmented training. The work proposes robustness against common image corruptions as well as Ep perturbations. However, it mainly focuses on possible corruptions and attacks specific to image domain, and assumes access the test data distribution. In our work, we focus on non-image data which have correlated and different scale features and we do not rely on an additional knowledge of the test set.
Recent work in the malware detection space (00 creates realistic AEs by defining a set of comprehen- sive and realistic constraints on how an input file can be transformed. Though this approach creates realistic AEs, collecting representative corpus of raw input files is a challenging problem [27]. The recent release of binary feature sets, such as the EMBER dataset without having access to the raw input files. Our work can be used to create more realistic AEs in situations where access large representative corpus of files is not possible.
22 enables model development
Non-uniform Adversarial Perturbations
In adversarial training, the worst case loss for an allowed perturbation region is minimized over parameters of a function representing a DNN. The objective of the adversary can written as the inner maximization of adversarial training:
maximize SEA..p
((foxx + 5).
(1)
where x E X' and E y are dataset inputs and labels, D.pp = (6 ||6||p < c) is an Ep ball of radius which defines the feasible perturbation region. Standard PGD follows steepest descent which iteratively updates in the gradient direction to increase the loss:
Vol(Jo(+ 81+1 = 8 + a Vel(fo(z + 8*), y)p
(2)
at iteration 4, and then projects å to the closest point onto the Ep ball:
PA., (6) = arg min ||8 8'12 = max [| 8'EA..
(3)
where the distance between d and 8' is the Euclidean distance, and the projection corresponds to normalizing 8 to have a maximum E, norm which is equal to e. Now, we introduce an adversarial constraint set that non-uniformly limits adversarial variations in different, potentially correlated dimensions, by
A., {ō: ||
where Q E dgd In our approach, 8 is updated by equation (2) similar to the standard PGD, however, it is projected back to non- non-unifome ton-unffomm norm ball satisfying || 1|p S The corresponding projection operator will then be:
5 11
if
||25|p
P PA.,, (16)
(5)
otherwise.
The choice of Ω depends on how we model the expert knowledge feature relationships. The following are our choices for the non-uniform perturbation sets.
2.1 Mahalanobis Distance (MD)
Euclidean distance between two points in multi-dimensional space is useful metric when the vectors have isotropic distribution (i.e. radially symmetric). This because the Euclidean distance assumes each dimension has same scale (or spread) and are uncorrelated to other dimensions.
3

---

However, isotropy is usually not the case for real datasets in which different features might have different scales and can be correlated. Fortunately, MD accounts for how the features are scaled and correlated to one another [29]. Hence, it is a more useful metric if the data has non-isotropic distribution.
By formal definition, MD between vectors 2, E R is denoted by dz == -'=-2')² (2 z²), where M E Rdxd is positive semi-definite matrix which can be de- composed as M = U for U E Rdxd The dissimilarity between two vectors from a distribution with covariance can be measured by selecting M Σ-1 If feature vectors of dataset are uncorrelated and have unit variances, their covariance matrix is Σ = I, which reduces their MD to Euclidean distance.
We are interested in the distance between the original and the perturbed sample. Since we assume all perturbations are additive, as common practice, the distance term we consider is V M5. For generalized MD in lp norm, selecting f = U corresponds to the perturbation set Ac.p {ō |UT S }} which generates AEs with feature correlations similar to the original dataset.
Robustness of adversarially trained model is directly related to how realistic the generated AEs are during training. Now, we explore implications of selecting & MD to define the limits of the perturbation set. To ensure the validity of the AEs, consider the notion of consistency of the generated sample with real samples.
[18] introduced the notion of e-inconsistency to quantify how likely an AE is. With slight change in their notation, we define 7-consistency as follows:
Definition 2.1. For a consistency threshold 7 > 0, an AE is "yconsistent if f y) 2 % where I E Rd, and f is a probability density function of a conditional Gaussian distribution with zero mean and covariance matrix Ey.
Theorem 2.1. If the AEs are generated according to MD constraint, then their 7-consistency has a direct relation to such that
0 < /2C - - log S c.
(6)
where C C= - (ogg(o))
d is the dimension of x, and v 1 S
Theorem 2.1 implies that there is direct relationship between limiting the MD of 8 and ensuring consistent samples when the data is Gaussian. In other words, when the f2 MD of the perturbations gets smaller, AEs become more consistent. See Appendix A.4 for the proof.
2.2 Weighted Norm
When Ω is a diagonal matrix, inner maximization constraint simply becomes the weighted norm of 5 limited by e, and the weights denoted by (Mr,) Projection of under the new constraint corresponds projection onto lp norm ball of radius n. for 5th feature. These weights be chosen exploiting domain, attack or model knowledge. For instance, more important features can be allowed to be perturbed more than the other features which have less effect on the output score of the classifier. This knowledge might come from Pearson's correlation coefficients [17] between the features of the training data and the corresponding labels, or Shapley values 30 for each feature.
Using Pearson's 's correlation coefficient of each feature with the corresponding target variable, i.c.. |pt.vl for jth feature and output y. we let larger perturbation radii for more correlated features with the output. Due to the inverse relation between Ω2 and the radius of the norm ball, for Pi,y = p.pl
we select Ω = ding( (ang(( 1fm)) II Hi ll
Similarly, using Shapley values to represent feature importance, we [[ Mi
define 8; = 1 where s; is the Shapley value of feature i. Then, we choose diag( (&. )ime) by
following the intuition that more important features should have larger perturbation radii.
In the malware domain, expert knowledge might help to rule out specific type attacks crafted on immutable features due to feasibility constraints. This can be modelled by the proposed weighted norm constraint as masking the perturbations on immutable features. Hence, non-uniform perturbation approach enables various transformations on the attack space for robustness against realistic attacks.

---

93
24
Defese 5/000 Rato-
Deferte C000
88
Usikmn
Deferee
form MD
Neruuitem No
un form Malarpe:
Non-nitem MDtarge:
10
20
0.4
0.6
10
044 0.6
1.0
Recroge:1EBi
Avcrage (|tllz
Avcnye (Hil
(a) Malware Use-case
(b) Credit Risk Use-case
(c) Spam Detection Use-case
Figure 1: Defense success rate of PGD AT against the problem-space attacks, where all non- uniform perturbation defense approaches outperform the uniform approach for all use-cases.
Experimental Results
Here, we present experimental results evaluate robustness of DNNs against adversarial attacks for binary classification problems on three applications: malware detection, credit risk prediction, and spam detection. We compare PGD) with non-uniform perturbations during AT with PGD proposed in [ based on uniform perturbations. For all applications, we evaluate our defense mechanisms on adversarial attacks proposed by other works. We use a machine with an Intel Xeon E5-2686 @ 2.3 GHz CPU, and 4 Nvidia Tesla V100 GPUs. Details of the DNN architecture and pre-processing are design the best possible neural network but instead compare [ with various non- non-uniform perturbations during AT for given DNN.
given in A.3 Note that our goal
the uniform perurbations
Adversarial training (AT): We perform AT in all use-cases by applying & norm PGD for uniform perturbation sets, i.e., A.,.2 (6 ||ö||2 S e. and non-uniform perturbation sets, i.e. 4ez {ō [|28||2 S )2)- Since potential adversaries are not interested in fooling the classifiers with negative class (target class) samples, 8 perturbations
only applied to the positive classes during AT as commonly used especially in malware detection [20]. Positive classes are the malicious class in malware detection, bad class in credit risk prediction, and spammer class in spam detection. Moreover, for the sake of clean accuracy within the positive class, adversarial perturbations are applied to 90% of the positive samples during training. Such hybrid approach where weighted clean adversarial loss are optimized at once is common in literature 31..
To model the expert knowledge with diagonal Ω, values, and masking to allow perturbation only in mutable features. To compute Shapley values, use SHAP B32 which utilizes deep learning explainer. We also consider AT under the MD constraint, and select Q=UT such that U U=Σ, considering two cases; Σy is the covariance matrix of the entire training data, i.c., y= 1}. and E, is only for the negative (target) class y=0. We call the models after AT with non-uniform perturbations according to their ! selection, e.g., NU-S-Pearson for Pearson's coefficients, NU-6-SAPP for Shapley values, NU-S-Mask for masking, NU-S-MI for MD using full covariance matrix and NU-8-MDtarger for MD using the covariance for only y=0. The choice ft=/ corresponds to AT with uniform perturbation constraint, which we call Uniform-6.
Pearson's 's correlation coefficient, Shapley
3.1
Malware Use-case
First, we consider binary classification problem for malware detection using the EMBER dataset 288 EMBER is a feature-based public dataset which is considered a benchmark for Windows malware detection. It contains 2381 features extracted from Windows PE files: 600/ labeled training samples and 200K test samples. We refer to Appendix A.2 for more detailed description of the EMBER dataset features. Given malware sample, adversary's goal is to make the DNN conclude that a malicious sample is benign. We also consider PDF malware detection. We use the extracted features of the PDF malware classification dataset and its attacked samples provided in B3]. The repository contains 110841 samples with 135 features that are extracted by PDFrate-R (4..
Attacks used for evaluation: In the malware domain, test-time evasion attacks can be classified as feature-space and problem-space attacks. While the former crafts AFs by modifying the features extracted from binary files, the latter directly modifies malware binaries making sure of the validity
5

---

and inconspicuousness of the modified object. We evaluate the robustness of our model against evasion attacks which are crafted in problem-space, i.e., on PE files. For Windows malware, we incorporate the most successful attacks 35 from the machine learning static evasion competition [36]. Since the EMBER dataset only contains the extracted features of a file, a subset of malware binaries 37 using the SHA-256 hash as identifier.
used for AE generation are obtained from VirusTotal
We observe that these problem-space attacks, which add various bytes to a file without modifying the core functionality, affect only the feature groups "Byte Histogram", "Byte Entropy Histogram" and "Section Information". Experts aware of these byte padding attacks understand which features can be manipulated by an attacker. In addition to the previous AT methods, we represent this best case expert knowledge by Ω mask- which is an identity matrix with non-zero diagonal elements only for "Byte Histogram", "Byte Entropy Histogram" and "Section Information" features. That is, the model trained using PGD perturbations applied only to these features, and we call it NU-ô-Mask. Our masking approach for the immutable features is similar to the conserved features in (8
For PDF malware classification, we use a problem space attack called EvadeML [39] It allows
adding, removing and swapping objects, hence it is a stronger attack than most other problem space attacks in the literature, which typically only allow addition to preserve the malicious functionality,
To make a fair comparison between uniform and non-uni non-uniform approaches, e for each method is selected such that their average distortion budgets, i.e., | For Windows malware classification, we test the detection success of adversarially trained models with 9000 AE sets generated by the problem-space attacks described in Appendix A.6 Figure Ta illustrates the average defense success rate against various problem-space attacks and shows that non-uniform perturbation approaches outperform the uniform perturbation in all cases. Moreover, NU-S-MDtargei performs closest to the best case expert knowledge NU-6-Mask for all cases except when ||0]| 25. The advantage of selecting Σ from benign samples versus selecting from the entire dataset is that the direction of perturbations are led towards the target class, i.c. benign samples, for NU-Ō-MDtarger. We also do not observe significant performance difference between and NU-8-SHA, while NU-5-MD only differs from the two for |011 = 25. We refer to Table A. for detailed attack performances and Table A.2 for defense S.R. results with clean accuracy.
Numeric results:
approximately equal.
For PDF malware classification, we compare NU-Ó- MDI with Uniform-ō against EvadeMI. We observe the best performances at ||ö||2 = 1 for both methods. Table the clean accuracy (Ac.), AUC score and defense success rate (D.S.R.) against EvadeML for standard training, Uniform-ō and NU-O-MDI. Al- though NU-6-MDI is feature space defense, the re- sults show that it highly effective against problem space attacks, and it outperforms Uniform-o. Since our approach does not assume any attack knowledge, it is more generalizable than the problem space defenses.
Clean accuracy (Ac.), AUC score and defense success rate (D.S.R.) against EvadeML for standard training, Uniform-ô and NU-6
Table I:
Model
|61|2 Clean Ac. AUC score D.S.R.
Std. training
99.52%
0.99912
11.1%
Uniform-ō
97.64%
0.99826
87.4%
NU-6-MDt
97.83%
0.99835
92.9%
3.2 Credit Risk Use-case
Our second use-case is a credit risk detection problem where the DNN's goal is to make decisions on loan applications for bank customers. For this scenario, use the well-known German Credit dataset [4]. which contains classes "good" and "bad", as well as applicant features such as age, employment status, income, savings, etc. It has 20 features and 1000 samples with 300 in the "bad" class. Similar to we treat discrete features as continuous and drop non-ordinal categorical features.
Attacks used for evaluation: The goal of an adversary in this situation is to make DNN models conclude that they are approved for a loan when they actually may not be eligible. Since modifications to tabular data can be detected by expert eye, attackers try to fool classifiers with imperceptible attacks. We use German Credit dataset implementation of LowProFool [17] which considers attack imperceptibilityy and represents expert knowledge using feature correlations. We apply the attack on the "bad" class of the test set and generate 155 AEs. After dropping the non-ordinal categorical features, we treat the remaining 12 features as continuous values.
Numeric results: Similar to the malware use-case, 6 for each method is selected such that their average ||ō|| are approximately equal. In Figure lb report defense success rate of PGD with

---

Table 2: Clean accuracy (CI. Ac.) and defense success rates of NU-6-MDr and Uniform-ó against
FGSM, Carlini- Wagner (CW), JSMA and Deep Fool attacks for Spam Detection Use-case.
Defenses
6
CL Ac.,
FGSM c0.55,
FGSM c=1,%
CW,%
JSMA,%
Deep Fool, %
Uniform-6
91.61
24.1 +: 0.25
20.17 + 0.31
38.93 + 0.54
41.51 0.24
39.3 + 0.27 51.73 = 0.31
0.1
NU-6-MDU
91.93
28.7 0.17
27.1 = 0.21
49.62 + 0.33
49.59
0.18
Uniform-o
90.40
25.22 0.12
21.61 = 0.43
45.83 = 0.41
46.38 0.27
47.39 0.25
0.3
NU-6-MDt
91.31
32.15 0.34
30.88 + 0.36
53.45 + 0.22
52.68 0.13
54.61 0.19
Uniform-6
87.12
27.05 0.50
23.08 + 0.34
50.05 = 0.32
49.5
49.75 0.16
0.5
NU-6-MDU
87.78
43.85 - 0.62
36.12 + 0.20
55.24 0.38
53.71 0.41
64.28 = 0.55
Uniform-o NU-6-MDt
86.46
40.20 0.57
32.98 0.31
52.77 0.21
52.91 0.26
53.22 + 0.10
87.64
81.34 0.83
79.85 + 0.75
61.89 + 0.37
0.77 72.93
88.75 0.78
Uniform-6
85,98
74.40 :: 0.47
62.36 + 0.75
59.71 + 0.28
68.95 - 0.85
64.5 + 0.86 98. 36 + 0.25
1.5
NU-6-MDU
87.03
94.45 + 0.18
91.03 + 0.10
72.98 + 0.48
86.43 0.32
uniform and non-uniform perturbations in detecting 155 AEs generated by LowProFool. The figure shows that for every given |,0||2, non-uniform perturbations outperform uniform perturbations in PGD. Although LowProFool represents feature importance by Pearson correlation coefficients between features and the output score, surprisingly NU-S-Pearson is the best approach among the other non-uniform approaches for only 5 6= {0.7, 1). We refer to Table A.3 for clean accuracy results.
3.3 Spam Detection Use-case
Finally, we evaluate robustness within the context of detecting spam within social networks. We use a dataset from Twitter, where data from legitimate users and spammers is harvested from social honeypots over seven months [41]. This dataset contains profile information and posts of both spammers and legitimate users. After pre-processing 42 we extract 31 numeric features with 14 being integers and the rest being continuous. Some examples of these features of following and followers as well as the ratio between them, percentage of bidirectional friends, number of posted messages per day, etc. We treat all features continuous values in our experiments. Moreover, we extract 41, 1,354 samples where the training set has 17,744 "bad" and 15,339 "good" samples, and the testing set has 3885 "bad" and 4386 "good" samples. The adversary's goal is to make the DNN predict that a tweet was posted by legitimate user when it was written by a spammer.
the number
Attacks used for evaluation: We incorporate the evasion attack 43 from [6] for our Twitter spam detector. The attack strategy based on minimizing the maliciousness score of an AE which is measured by a local interpretation model LASSO, while satisfying Eg norm constraint on perturbations. We generate the AEs by constraining the perturbations to 0.5 x distavy pos-neg where di pos- neg is defined by [16] as the average distance between the spammer samples and the closest non nonnspaamerrs to these samples. We split the Twitter dataset with ratio 25% for training and testing, and generate the AES using the spammer class of the entire test set.
Again, we apply perturbations only to the spammer set during AT and report the results for approximately equal average ||1|| perturbations. Figure Ic illlustrates defense success rate in detecting AES of the proposed approaches against the model interpretation based attack [16] for Twitter dataset. The figure shows that non-uniform perturbations outperform uniform case in terms We refer to Table A.4 for clean accuracy results.
Numeric results:
of defense S.R. for all given ||6 |2.
3.4 Performance Against Uniform Attacks
Throughout the experiments, we tested our non-uniform approach against various realistic attacks, such as problem space attacks in Section 3.1 feature importance-bar bassed attack in Section 3.2 and explainability-base based attack in Section 3.3 So far we emphasized that problem space attacks and non-uniformly norm bounded attacks are more realistic compared to the traditional uniformly norm- bounded attacks which are mostly considered in image domain. Yet, in this section, we also test our non-uniform approach against the well-known uniform attacks to investigate the generalizability of our approach. We use the setting for the spam detection use-case, and compare NU-8-MDi with Uniform-6. We utilize the adversarial robustness toolbox (ART) 44 to craft ABs by using the default parameters for the AE generators of Carlini-War Wagner (CW), JSMA and DeepFool Methods. We also
7

---

Lample
Unferm
(a) Uniform-6
(h) NtrgeMDt
(c) Histogram of MD square, 8 x y==
Figure 2: UMAP visualization of benign, malicious and adversarial samples generated by (a) Uniform- and (b) NU-A-MDiarget and (c) the density histogram of their 5 2C log
FGSM for 0.5 and c= 1. Table shows the clean accuracy (CI. Ac.) and defense S.R.'s of the robust models NU-5-MDr and Uniform-6. We observe in Table 2 that the Cl. Ac. decreases as || la increases for both Uniform-ō and NU-S-MDr but the degradation in non non-uniform is less. Defense S.R.',, on the other hand, improve for both approaches but NU-6-MDI significantly outperforms Uniform-6 in all cases. See Appendix A.7 for the performance against uniform PGD attacks.
3.5 Quality of Perturbation Sets
In this section, we quantitatively and qualitatively analyze how well non-uniform perturbations capture realistic attacks using 7-consistency property defined in Section and lower dimensional space visualization. Our intuition is that a successful attack evades detection since AEs appear benign to the model. That is, AEs have high likelihood according to the distribution of benign samples. Therefore, we measure a perturbed sample's ' quality by its y-consistency with the benign set distribution. Definition 2. leverages Theorem 2.1 which shows that smaller MD for indicates higher n-consistency and hence higher quality of the perturbed sample. Moreover, we expect AEs that evade the model and benign samples to be embedded closer each other in the lower-dimensional subspace. Figure TigureBillustrann iillustrates UMAP visualization (45) of benign, malicious and adversarial samples for the spam detection use-case. AEs generated by NU-6-MDtarget show better alignment with benign distribution, which shows that NU-g-MDtarget mimics more realistic attack. We also show the histogram of MD squares, ie. E = 2C - 2 2og y, of 1660 AEs from Uniform-ó and NU-G-MDDarget in Figure 2cl where the average values are 2.1 and 1. 1.28, respectively. Following Thcorem 2.1 and Figure 2c. 8's from NU-ô-MDtarget have higher 7, and hence, are more realistic.
4 Certified Robustness with Non-uniform Perturbations
In this section, we present methods for certifying robustness with non-uniform perturbations. We consider two well-known methods; linear programming (LP) [9] and randomized smoothing [46].
4.1 LP Formulation
We can provably certify the robustness of deep ReLU networks against non- non-uniform adversarial perturbations at the input. Our derivation follows an LP formulation of the ad adversary's 's problem with ReL.U relaxations, then the dual problem of the and activation bound caleulation. It can be viewed as an extension of 9.. Similar to 19, we consider a k layer feedforward deep ReL.U network with
2+1 Wiz + by 2, max(2,, 0}, for i -,..... -
We denote (z) ([(()) = (z (z 8) || < c) as the set of all attainable final-layer activations by input perturbation Since this a non-convex set for multi-layer networks which is hard to optimize over, we consider convex outer bound on (z)) and optimize the worst case loss over this bound to guarantee that no AES within (x,)(x) can evade the network. As done in (9). relax the ReLU activations by representing
(0, )} with their upper convex envelopes

---

2 2 0, 2, -uē + (u- -ul, where 1 and u are the known lower and upper bounds for the pre-ReLU activations. We denote the new relaxed set of all attainable final-layer activations by (x). Assuming that an adversary targets specific class to fool the classifier, we write the LP as
minimize Eu
c²
2x
S.L.
2x
Ze,
(8)
where c::= Cytrue - Cytarget is the difference between the selection vector of true and the target class.
A positive valued objective for all classes as solution to equation g indicates that there is no adversarial perturbation within Ar,, which can evade the classifier. To be able to solve equation 8 in tractable way, we consider its dual whose feasible solution provides a guaranteed lower bound for the LP. It is previously shown by [9] that feasible set of the dual problem can be formulated similar to standard backpropagation network and solved efficiently. The dual problem of our LP with ReLU relaxation and non- non-uniform perturbation constraints is expressed in the following theorem.
Theorem 4.1. The dual of the lineur program can be writlen
k-1
maximize U2M
Σ Euti
ΣΣ P.sl oJ e
1=2 jeI,
Uk -,, vi (W? (41)), for i- k 1.....1
s.l.
(9)
j E I
Vijj
Vig U1. Ui.
j E I for i i 1,...,,
[igg++ m.slp
j E IИ
where IT It and I, represent the activation sets in layer for and u are both negative, both positive and span zero, respectively.
See Appendix A.4 for the proof of Theorem When Th. us, Theorem 4.1 shows that the
dual problem can be represented as linear back propagation network, which provides tractable solution for a lower bound of the primal objective. To solve equation and upper bounds for each layer incrementally as explained in Appendix A.5
9 we need calculate lower
For certification of robustness within a non-uniform norm ball around a test sample, we need the objective of the LP to be positive for all classes. Since the solution of the dual prob- lem is a lower bound on the pri- mal LP, it provides worst case certification guarantee against the AEs within the non-uniform norm ball. We provide certification re- sults for the robustness of Uniform- 5 and NU-5-MDr (NU-8-MDtarge for spam detection use-case Table 3 We consider both uniform and non-uniform input constraints in cer-
Table Average certification margin and number of successful certified samples out of 1000 spammers for NU-5-MDi and Uniform-6 for Spam Detection Use-case.
Model Defense S.R. Cert. Method
Margin
Cert. Success
Uniform-
1.07
34.72 + 0.94%
NU-Cert-SHAP
1.84
72.64 +: 0.6%
Uniform-6 54.87 + 1.1%
NU-Cert-Peange
2.04
76.8 : 0.71%
NU-Ceri-MD
2.40
80.2 + 0.56%
NU-Cert-MDt
2.40
80.2 + 0.55%
Uniform-Cert
1.11
42.95 + 0.69%
NU-Cert-SHAP
74.65 + 0.85%
NU-A-MDr 63.4 0.74% NU-Cert-Pearson
2.06
78.38 + 0.76%
NU-Cert-MD
2.41
81.3 0.68%
NU-Cert-Mil
2.41
81.3 0.67%
tification, namely Uniform-Cert for the standard LP approach for certi- fication with uniform perturbation constraint [9] and NU-Cert-(.) for the non-uniform constraint. We implement our non-uniform approach into the LP by modifying [ with our S matrix, and generate various certification methods by non-uniform Q e.g. NU-Cert-SHAP NU-Cert-Pearre NU-Cert-MD and WU-Cert-MDr, Our purpose is not to propose the tightest certification bounds but to show that non-uniform constraints result in larger certification margins compared to the uniform case.
We compare Uniform-ō and NU-S-MDI to evaluate certification results. Dropout layers are removed from the model for LP solution, and AT is performed for e = 0.3. Certification is done by solving the IP for 0.3 over 1000 spammers. The objective should be positive for all classes to certify the corresponding sample. The margin between the objective and zero gives idea about how

---

tight the bound is 447. Table 3 demonstrates two main results: (i) the certification success of NUta-MDt over Uniform-o for each certification method supports our claim that non-ume unn-unfform perturbations provide higher robustness than the uniform approach; and (ii) certification with non- uniform constraints provide larger certification margins and hence tighter bound.
4.2 Randomized Smoothing
Robustness certification via randomized smoothing 44 an empirical alternative to the LP. The idea is constructing "smoothed" classifier g from the base classifier f. In the original formulation in [6]. returns the most likely output returned by f given input æ is perturbed by isotropic Gaussian noise. Here, we provide robustness guarantee in binary when non-isotropic Gaussian noise is used to allow robustness to non-uniform perturbations:
for randomized smoothing framework
g = arg max (( + n)) = y) BEy
where 7~ (0,, E).
(10)
Adapting notation and Theorem from [46] let Pa be the probability of the most probable class y = a when the base classifier f classifies (r, 2). Then the below theorem holds.
Theorem 4.2. In binary classification problem, suppose Pa ($) satisfies P(f(x + n) a) 2 Po-
Then g( + 6) = a for all V8T S 47 (Pa) ) 95a. where = VoTΣ-15, Φ (pah(pa) is the quantile function fuantttlefunction of the x distribution of d degrees of freedom, and ysu is the 50 quantile.
we show that smoothed classifier g is robust around r within & Mahalanobis distance V8-Σ-16 T - 950 where Φ (pa) the quantile function for probability Pa- The same result holds if we replace pa with lower bound Pa-
See Appendix (..eendxxxA.44ee for the proof. In theorem 4.2
We implement non-uniform approach into randomized smoothing by modifying [48] with our non-isotropic noise space. Table certification S.R. of Uniform-8 and NU-8- MDt, when they are certified by standard randomized smoothing with (0, a 7) (UC), and our non- uniform methods with NO., E.,)
Table 4: Percentage successfully certified samples for NU-8- MDr and Uniform-ō with various certification approaches with randomized smoothing for Spam Detection use-case.
Model
UC
NUC-Pcarson NUC-SHAP NUC-MD NUC-MDL
Uniform-ó 50.96%
61.11%
64.24%
65.72%
66.45%
NU-6-MDU 61.8%
67.14%
71.25%
85.34 35.34%
90.11%
for corresponding Ey. That is, Ey=o for NUC-MDt, (0= NUC-MD, 3² for NUC-Pearson
and / for NUC-SHAP are used when the average training distortion budget is || and the average certification distortion is [| Table shows that NU-O-MDt is certifiably robust for more samples than Uniform-ō for all certification methods. Moreover, certification with non-uniform noise, especially with NUC-MDt, provides higher certification S.R. compared to uniform noise.
5 Conclusion
In this work, we study adversarial robustness against evasion attacks, with a focus on applications where input features have to comply with certain domain constraints. We assume Gaussian data distribution in our consistency analysis, as well as precomputed covariance matrix and Shapley values. Under these assumptions, our results on three different applications demonstrate that non-uniform perturbation sets in AT improve adversarial robustness, and non-uniform bounds provide better robustness certification. As unintended negative social impact, our insights might be used by malicious parties to generate AEs. However, this work provides the necessary defense mechanisms against these potential attacks.
Acknowledgments and Disclosure of Funding
We would like to thank Mohamad Ali Torkamani for his help and valuable feedback, and thank Baris Coskun for his endless support as the leader of the Security Analytics and AI Research (SAAR) Team AWS, as well as the team members for the fruitful discussions. This work has been done as a part of Ecenaz Erdemir's internship in AWS, and no outside funding was used in the preparation.
10

---

References
[1] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint 1tl::1700..6000 2017.
[2] Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Sindić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery databases, pages 387-402. Springer, 2013.
[3] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. arXiv preprini arXiv: 1312.6199 2013.
[4] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples. arXiv preprint arXiv:1412.6572, 2014.
[5] Krishnamurthy Dvijotham, Robert Stanforth, Sven Gowal, Timothy A Mann, and Pushmeet Kohli. A dual approach to scalable verification deep networks. In UAI, volume 1, page 2, 2018.
[6] Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, and Martin Vechev. Ai2: Safety and robustness certification of neural networks with abstract interpretation. In 2018 IEEE Symposium on Security and Privacy (SP), pages 3-18. IEFE, 2018.
[7] Gagandeep Singh, Timon Gehr, Matthew Mirman, Markus Püschel, and Martin Vechev. Fast and effective robustness certification. Advances in Neural Information Processing Systems, 31: 10802-10813, 2018
[8] Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Semidefinite relaxations for certifying robustness to adversarial examples. Advances Neural Information Processing Systems, pages 10877-10887, 2018
[9] Eric Wong and Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In International Conference on Machine Learning, pages 5286-5295. PMLR, 2018
[10] Huan Zhang, Tsui-Wei Weng, Pin-Yo Yu Chen, Cho-Jui Hsieh, and Luca Daniel. Efficient neural network robustness certification with general activation functions. In Advances in neural information processing systems, pages 4939-4948, 2018.
[11] Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint arXiv:1611.012366 2016.
[12] Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. arXiv preprini wrXiv:2001.03994, 2020.
[13] Shiqi Wang, Yizheng Chen, Ahmed Abdou, and Suman Jana. Mixtrain: Scalable training of formally robust neural networks. arXiv preprini arXiv:1811.0262.. 14, 2018.
[14] Matthew Mirman, Timon Gehr, and Martin Vechev. Differentiable abstract interpretation for provably robust neural networks. In International Conference on Machine Learning, pages 2018.
[15] Mary Frances Zeager, Aksheetha Sridhar, Nathan Fogal, Stephen Adams, Donald E Brown, and Peter A Beling. Adversarial learning in credit card fraud detection. In 2017 Systems and Information Engineering Design Symposium (SIEDS), pages 112-116. IEEE, 2017.
[16] Ninghao Liu, Hongxia Yang, and Xia Hu, Adversarial detection with model interpretation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & 2018 2018.
Data Mining, pages 1803-1811,
11

---

[17] Vincent Ballet, Xavier Renard, Jonathan Aigrain, Thibault Laugel. Pascal Frossard. and Marcin Detyniecki. Imperceptible adversarial attacks on tabular data, arXiv preprint wrXiv:111..03274 2019.
[18] Eden Levy, Yael Mathov, Ziv Katzir, Asaf Shabtai, and Yuval Elovici. Not all datasets are born equal: On heterogeneous data and adversarial examples. arXiv preprint arXiv:2010.03//0, 2020.
[19] Deqiang Li and Qianmu Li. Adversarial deep ensemble: Evasion attacks and defenses for malware detection. IEEE Transactions on Information Forensics and Security, 15:3886-3900, 2020.
[20] Fabio Pierazzi, Feargus Pendlebury, Jacopo Cortellazzi, and Lorenzo Cavallaro. Intriguing properties of adversarial ml attacks in the problem space. In 2020 IEEE Symposium on Security and Privacy (SP), pages 1332-1 1349. IEEE, 2020.
[21] Ishai Rosenberg, Shai Meir, Jonathan Berrebi, Iaay Gordon, Guillaume Sicard, and Eli Omid David. Generating end-to-end adversarial examples for malware classifiers using explainability, In 2020 International Joint Conference on Neural Networks (IJCN), pages 1-10. IEEE, 2020.
[22] Wenbo Guo, Dongliang Mu, Jun Xu, Purui Su, Gang Wang, and Xinyu Xing. Lemna: Explaining deep learning based security applications. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS '18, page 364-379, New York, NY, USA, 2018. Association for Computing Machinery.
[23] D. Tsipras, Shibani Santurkar, L. Engstrom, A. Turner, and A. Madry. There no free lunch in adversarial robustness (but there are unexpected benefits). ArXiv, abs/1 /e////00..1115 2018.
[24] Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. Advances in Neural Information Processing Systems, volume 32, pages 125-136. Curran Associates, Inc., 2019.
[25] Chen Liu, Ryota Tomioka, and Volkan Cevher. On certifying non-uniform bounds against adversarial attacks. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 4072-4081, Long Beach, California, USA, 09-15 Jun 2019. PMIR.
[26] Eric Wong and J. Zico Kolter. Learning perturbation sets for robust machine learning. arXiv preprint arXiv: 004.008450 2020.
[27] Feargus Pendlebury, Fabio Pierazzi. Roberto Jordaney, Johannes Kinder, and Lorenzo Cavallaro. (TESSERACT]; Eliminating experimental bias in malware classification across space and time. In 28th [USENIX} Security Symposium (( (UUSENIX) Security 19), pages 729-746, 2019.
[28] Hyrum S. Anderson and Phil Roth. EMBER: an open dataset for training static PE malware machine learning models. arXiv preprint arXiv: 1804.04637, 2018
[29] Prasanta Chandra Mahalanobis. On the generalized distance in statistics. Proceedings of the National Institute of Sciences, 2:49-55, 1936.
[30] Lloyd S. Shapley. Notes on the P-Person Game II: The Value of an n-Person Game. RAND Corporation, Santa Monica, CA", 1951.
[31] Haotao Wang, Tianlong Chen, Shupeng Gui, Ting- Hu. Ji Liu, and Zhangyang Wang. Once-for-all adversarial training: In-situ tradeoff between robustness and accuracy for free. arXiv preprint arXiv:2010.11822 2020.
[32] Scott Lundberg and Su-In Lee. A unified approach to interpreting model predictions. 12 2017.
[33] Pdffrate. URL https://github... http://iiihuu.coommoccutttttaaaa
[34] C. Smutz and A. Stavrou. Malicious pdf detection using metadata and structural features. In 28th Annual Computer Security Applications Conference, page 239-248, New York, NY, USA, 2012.
12

---

[35] William Fleshman. Evading Machine Learning Malware Classifiers, 2019.
URL
http:://oooarddsataaaccooo.o wwwp:e/innnn.ccceeeeeeeeeeeeee hew.//vaaninnggeee marggg-maci-----ee waahine-learning malware-
classiffeers-cce2ddddbb11..
[36]
M learning static evasion competition, 2019. URL htwph:ttpp//// www.elastic
competition
[37]
VirusTotal. URL htt.Lttt:://w.virustttallll
[38]
Liang Tong, Bo Li, Chen Hajaj, Chaowei Xiao, Ning Zhang, and Yevgeniy Vorobeychik, Improving robustness of ML classifiers against realizable evasion attacks using conserved
features. In 28th USENIX Security Symposium (USENIX Security 19), pages 285-302, Santa
Clara, CA, August 2019. USENIX Association. ISBN 978-1-939133-06-9, URL https
7/www /www usenix x. ort/////fcaann//aaaaaaaaaaattaar
[39] Weilin Xu, Yanjun Qi, and David Evans. Automatically evading classifiers: A case study on pdf malware classifiers. In NDSS, 2016.
[40] C. J. Merz and P. Murphy. UCI repository of machine learning databases, 1996. URL http:://wrr
uci add/"mlearn, wwnlearn/MLRRpositoor .hmml
[41] Kyumin Lee, Brian David Eoff, and James Caverlee. Seven months with the devils: A long-term study of content polluters on twitter. In 5th International AAAI Conference on Weblogs and Social Media (ICWSM), Barcelona, 2011.
[42] Xiao Huang. xhuang1994
Twiter bot detection, 2017. URL
[43] Ninghao Liu, Hongxia Yang. and Xia Hu. Interpretation to adversary. 2018. URL https
//github. //github.ccm//iinii http:///wwaaaaeeooommmeeeerrrraa
[44] Maria-Irina Nicolae, Mathieu Sinn, Minh Ngoc Tran, Beat Buesser, Ambrish Rawat, Martin Wistuba, Valentina Zantedeschi, Nathalic Baracaldo. Bryant Chen, Heiko Ludwig, Ian M. Molloy, and Ben Edwards. Adversarial robustness toolbox v1.0.0, 2019.
[45] Leland Mclnnes, John Healy, Nathaniel Saul, and Lukas Grossherger. Umap: Uniform manifold approximation and projection. The Journal of Open Source Software, 3(29):861, 2018.
[46] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. arXiv preprint 1902.0299022 2019.
[47] Hadi Salman, Greg Yang, Huan Zhang, Cho-Jui Hsieh, and Pengchuan Zhang. Benchmark for Ip-relaxed robustness verification of relu-networks. 2019. URL https:///giiee com/
[48] Jeremy Cohen, Elan Rosenfeld, and and Zico Kolter. Certified adversarial robustness via randomized smoothing. 2019. httpp/////ggtttb...mm////s/////p
13


Reviews:
[{"review_id": "yxrQxMpVxR4", "reviewer": "Reviewer_tqQw", "summary": "The standard adversarial attack is done against some p-norm bounded radius, which implicitly assumes each dimension of the attack space is equally valid. This is not always true, and this paper proposes adversarial attacks (and training) with quadratic relationship constraints on the usage of each feature in the attack generation process via a covariance matrix applied to the perturbation vector. Improved adversarial training is shown across several datasets and prior results on certifiable robustness are extended to the paper's domain. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I overall like the paper, the need to consider attacks adjusted to specific features is intuitive and makes sense. The results showing how the attacks end up looking more like benign samples is compelling, and the extension of the results to certifiable bounds in two different ways is good. Overall a thorough and well done paper. \n\nMy only major concern is if the comparison between new and prior PGD with respect to a maximum perturbation ε was done in a fully apples-to-apples maner. By using ||Ωδ||_p ≤ ε , but not constraining Ω to maintain the norm of δ, I fear that things are not quite equal. I think two items could fully dissuade this:\n\n1. Results when the ||Ωδ||_p attack is projected at every step to be within the ||δ||_p ball, so that the ||Ωδ||_p attack is always a valid ||δ||_p  attack. (It may be that Figure 1 shows this but if so it was not clearly explained)\n2. Showing results of the model training under ||Ωδ||_p adversarial training and it's accuracy against ||δ||_p generated attacks. \n\nI do have some minor points around the malware /executable sections of the document. I think the simplifications are fine, but it should be stated that the features that could be altered is larger than what is considered through other approaches such as packing, binary rewriting, and other obfuscations. The line has to be drawn somewhere, as these are not always practical attacks, but they do exist. You could use [4] to support the large degree of complexity there. [5] should also be at least discussed in the related work around feature based constraints on the model attack space. \n\nIn particular 51-56, 181-183 should be referencing [1] and [2] which independently developed monotonic approaches to malware detection under the \"additive\" threat model, which incorporates malware and spam specific constraints on the attack space into the defense. This relates to the classic \"good word\" attack for spam [3] that should also be referenced appropriately in that portion. \n\nMy Citations: \n1. Incer, I., Theodorides, M., Afroz, S., & Wagner, D. (2018). Adversarially Robust Malware Detection Using Monotonic Classification. In Proceedings of the Fourth ACM International Workshop on Security and Privacy Analytics (pp. 54–63). New York, NY, USA: ACM. https://doi.org/10.1145/3180445.3180449\n2. Fleshman, W., Raff, E., Sylvester, J., Forsyth, S., & McLean, M. (2019). Non-Negative Networks Against Adversarial Attacks. AAAI-2019 Workshop on Artificial Intelligence for Cyber Security. Retrieved from http://arxiv.org/abs/1806.06108 \n3. Lowd, D., & Meek, C. (2005). Adversarial Learning. In Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining (pp. 641–647). New York, NY, USA: ACM. https://doi.org/10.1145/1081870.1081950\n4. Raff, E., & Nicholas, C. (2020). A Survey of Machine Learning Methods and Challenges for Windows Malware Classification. In NeurIPS 2020 Workshop: ML Retrospectives, Surveys & Meta-Analyses (ML-RSA). Retrieved from http://arxiv.org/abs/2006.09271\n5. Demontis, A., Melis, M., Biggio, B., Maiorca, D., Arp, D., Rieck, K., … Roli, F. (2017). Yes, Machine Learning Can Be More Secure! A Case Study on Android Malware Detection. IEEE Transactions on Dependable and Secure Computing, 1–1. https://doi.org/10.1109/TDSC.2017.2700270\n\n-------\n\nI believe incorporating the results in rebuttal to my review are satisficing, and have adjusted my score appropriately. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper could have better emphasized that this approach will not be applicable to all problem spaces, but that is implied. I'm fine with it but I could see some readers needing it stated more explicitly. ", "review_text": "", "extra_scores": {"ethical_concerns": "I have no special ethical concerns with regard to this work. Overall it is improving defensive posture, and adversarial attack/defense in ML is hard to seperate. ", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "yAUvols7W_", "reviewer": "Reviewer_Y8xB", "summary": "The authors present a novel strategy of adversarial attacks with non-uniform perturbation, and provide certified robustness for the new norm-bounded perturbation strategy. Both theoretical and empirical studies are presented to support the consideration of feature importance in attacks and adversarial training.  ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nCost, utility, and feature importance are reasonable concerns for adversaries in practice. This paper tackles the challenge of optimizing attacks with non-uniform perturbation, in which features are not treated equally. Non-uniform perturbations appear to be more effective in the empirical studies on three real datasets. Robust certification techniques based on non-uniform perturbation are also provided, which completes an attack-defense problem cycle.  ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have made significant improvement over their ICML manuscript. However, I have several concerns about the current submission: \n\nI don't quite agree with the claim that small norm-bounded perturbations are necessary for maintaining imperceptibility. Sharma & Chen in \"Attacking the Madry Defense Model with $L_1$-based Adversarial Examples\" clearly demonstrate that high average norm-bounded distortions may have minimal visual distortion. In other words, adversaries may have a much greater de facto attack budget than certified models have assumed. The technique presented in this paper is also norm-bounded, therefore is susceptible to this misconception. \n\nHuman perceptual systems do not share the same metamers as machine learning models, therefore, attacks in-out a given norm-bound may be equally capable of deceiving a human as long as they have similar aggregate statistics. See:\n[1.] \"Exploiting Human Perception for Adversarial Attacks\", by Pengrui Quan and Mani B. Srivastava\n[2.] \"Metamers of neural networks reveal divergence from human perceptual systems\", by Jenelle Feather, Alex Durango, Ray Gonzalez, Josh McDermott in NeurIPS 2019.\nThis applies to tabular data as well. It is unclear what a universally realistic budget should be for a given problem. This calls into the question the robustness of certified models. Therefore, in the test phase of the empirical study, adversaries should not be constrained with a given budget. In addition, models certified for one type of attack should be tested on different types of attacks. \n\nThe proposed technique takes advantage of given domain knowledge. For fair comparison, the Madry model with uniform perturbation should be given the same advantage by enabling feature weighting, for example, assigning an a-priori importance value as in \"Learning to classify with missing and corrupted features\" by Dekel and Shamir, ICML2008. A simple feature scaling on $\\delta$ in each iteration of attack should also be sufficient. \n\nIt's not clear how many iterations have been performed during the optimization-based attack. The defense success rates of the standard training presented in the Appendix are pretty high for the Malware and Credit Risk datasets. \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "approximated 5-6 hours"}, "extra_text": {}}, {"review_id": "thFHKzQAG42", "reviewer": "Reviewer_8HjK", "summary": "Most research on adversarial perturbations is focused on those within an lp-ball (termed \"uniform\" due to the assumption that relative perturbation costs are uniform within the ball).  This paper introduces a non-uniform model of perturbations aimed to better capture feature dependencies and correlations through a space of better, non-uniform, cost functions.  The main goal is to use the modified attack model as a part of AT, with the hypothesis that the resulting AT will be more effective against problem-space attacks (commonly also called realizable attacks in prior literature).", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper hits an important gap in the literature.  However, it seems to miss some of the closely related work in the malware and image classification space.  Moreover, while the authors appear to make a conceptual distinction between attacks on images and domains such as malware/spam/fraud detection, the distinction seems unnecessary.  Given prior work that had already studied the problem of realizable or problem-space attacks in the context of AT, the conceptual contribution appears limited.  Furthermore, the technical contribution is relatively incremental: the challenges in extending standard attack models and solution approaches seem minor, with the extensions appearing straightforward.  This is also true for the certified robustness results, which seem rather straightforward extensions of Wong + Kolter in the first case, and Cohen et al. in the second.  Finally, experimental evaluation is not all that convincing: in Figure 1, the differences are quite small, except in Figure 1a, where the one highly effective approach is essentially a variant of Tong et al. USENIX Security 2019 \"conserved features\" idea [1].\n\nHere are some further details (and see two key missing references below, esp. reference [1]).\n\n1) Problem-space attacks: this was the terminology adopted by the Pierazzi et al. [20] (reference from paper).  Another term that means essentially the same thing \"realizable attacks\", which features both in security and image classification applications (e.g., [1-3] below).  The authors appear to miss this connection.\n\n2) In Figure 1a, the best performing method is the one which masks \"unmodifiable features\".  This idea is similar in concept to the notion of \"conserved features\" from Tong et al. [1], who apply this idea also in malware classification, and also show significant performance improvement, albeit in the PDF malware domain.  It would probably be best to add PDF malware classification experiments with the same problem-space attacks as used in Tong et al., and compare to their approach as the baseline, to convincingly show that there is indeed added value to other variants, such as those based on MD, in the malware classification domains.\n\n3) I am greatly concerned about making discrete features continuous artificially, as is done for the credit risk and spam use cases: this seems to unnecessarily handicap the feature-space approaches: after all, there are feature-space attack methods that can handle, say, binary features (e.g., JSMA, stochastic local search, etc).\n\n4) I am also concerned that Figure 1 has no confidence intervals or other statistical information about the differences.  In most cases, the differences are quite small, and seem unlikely to be statistically significant (particularly once one adds Bonferroni correction, since we are dealing with multiple-comparisons here).  As such, the \"Yes\" response in 3(c) of the checklist seems extremely misleading, as Figure 1 presents the main results in the main body of the paper.\n\n5) I don't understand what makes the attack in the spam detection use case a problem-space attack.  It seems just another variation of a feature-space attack.\n\n6) As mentioned above, Section 4 seems to present a straightforward variation of the two state-of-the-art ways of certifying robustness (LP duality and randomized smoothing).  Moreover, these are still reliant on highly stylized assumptions embedded in \\Omega, and need not certify anything against actual problem-space attacks, so the utility of these approaches in the target domains, particularly malware classification, seems limited.\n\n[1] Tong et al. Improving robustness of ML classifiers against realizable evasion attacks using conserved features. USENIX Security, 2019.\n\n[2] Saha and Sim.  Is Face Recognition Safe from Realizable Attacks?  IJCB 2020.\n\n[3] Wu et al. Defending Against Physically Realizable Attacks on Image Classification. ICLR 2020.\n\nPost-rebuttal discussion with the authors:\n\nAfter the authors posted their rebuttal, I went back and reread the paper.  As a result, I increased my score, but not enough to cross into the \"accept\" territory.  I believe I was wrong in my original review that Section 4 has limited contribution; indeed, after reading it again I believe I had originally undervalued its significance.\n\nMy primary remaining concern about the paper, which I expressed to the authors during the somewhat involved back-and-forth discussion, is that the paper simply lacks evidence to support its main hypothesis that the proposed idea would indeed be generally effective to defend against problem-space attacks.  First, the authors clearly missed relevant prior art.  Second, two of the three attacks in the experiments are not in fact problem-space attacks, and no evidence is provided to support the claim that they are \"realistic\" (i.e., reasonable proxies for actual evasion attacks in those domains).  As such, I am inclined to dismiss these as provided satisfactory evidence.  The malware domain does clearly involve a problem-space attacks, but it is a very weak attack.  Realistic adversaries would make deeper changes to malware, such as removing/replacing portions.  Even aside from this, it would be crucial to demonstrate generalizability, i.e., that the approach can handle problem-space attacks of several varieties.  Consequently, I strongly recommend that the authors add experiments in the PDF malware domain, where a) there are actually reasonable state-of-the-art defense baselines, and b) lots of distinct problem-space attacks to work with, including those (like EvadeML) that add and remove content.\n\nPost-post rebuttal:\n\nThe authors added PDF evaluation during the discussion phase, and their results are convincing to me.  I am therefore now recommending that the paper is accepted.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "N/A", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "ofDx6r8Fdc-", "reviewer": "Reviewer_LM6b", "summary": "The authors study how to improve adversarial training by leveraging non-uniform perturbations of the input space. \nThe latter are formalised by considering non-isotropic mutations of the input sample, and this is useful for addressing domains like the malware one.\nExperimental results show that adversarial training with non-uniform perturbation is more robust than regular adversarial training, by offering different use cases (malware, credit risk, spam detection).\nThe authors conclude the work by certifying the robustness of their approach.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n* the intuition of using non-isotropic perturbation is clear and very useful, especially in domains where data are more complex (e.g. malware)\n* the authors tested many different domains to prove the efficacy of their intuitions\n* the adversarial training results are certifiable\n\nCons:\n* the contributions of this work are incremental, and novelty seems very small compared to the original work [1, 2]. Also, the authors would need to state that there are many other domains that might benefits from non-uniform perturbations (like audio, text, android malware [3-6]).\n* Concerning the attacks used for the adversarial training pipeline for the malware domain, it is unclear which manipulation has been used. Also, there is lack of the state-of-the-art manipulations. Authors may need to consider the latest work that address the generation of adversarial malware [7]. \n* While the results looks very good, there is the concern regarding the implementation of the submitted code.\nFirst, the implementation of the PGD attack is custom. Why the authors did not use any known library, like Foolbox, Adversarial Robustness Toolkit (ART), Cleverhans, or SecML?  \nFurthermore, by looking at the code, it seems that the adversarial examples are not crafted on the trained model, but rather they are created while training the classifier itself. This seems a bit odd, and maybe the authors would need to comment this choice.\n* Since the authors proposed an hardening of adversarial training, with certifiable bounds, it would be insightful to test your defenses with adaptive attacks that consider also the presence of such defence.\nAn example for an attack that you might want to comment on is [8], who crafted spoofed certificates for randomized smoothing. \n\nBibliography\n* [1] Xu, Kaidi, et al. \"Structured adversarial attack: Towards general implementation and better interpretability.\" ICLR 2019.\n* [2] Chen Liu, et al. \"On certifying non-uniform bounds against adversarial attacks.\" ICML 2019\n* [3] Grosse, Kathrin, et al. \"Adversarial examples for malware detection.\" European symposium on research in computer security. Springer, 2017.\n* [4] Zhang, Guoming, et al. \"Dolphinattack: Inaudible voice commands.\" Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017.\n* [6] Roy, Nirupam, Haitham Hassanieh, and Romit Roy Choudhury. \"Backdoor: Making microphones hear inaudible sounds.\" Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. 2017.\n* [7] Demetrio, Luca, et al. \"Adversarial EXEmples: A survey and experimental evaluation of practical attacks on machine learning for windows malware detection.\" arXiv preprint arXiv:2008.07125 (2020).\n* [8] Ghiasi, Amin, Ali Shafahi, and Tom Goldstein. \"Breaking certified defenses: Semantic adversarial examples with spoofed robustness certificates.\" ICLR, 2020.\n\n\nAfter respose:\nI'll rise my score to 6 after discussion and after having read the answers.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors did not discussed the limitations of their work, but there are some that are highlighted inside the review. The authors might start from them.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}, {"review_id": "MZWtpZRkiPN", "reviewer": "Reviewer_9LNj", "summary": "The paper proposes a notion of robustness w.r.t. non-uniform adv. perturbation scaled according to Mahalanobis Distance (MD). By harnessing non-uniform perturbation AE and using them for adv. training, we can obtain a more robust model with both better certified and empirical robustness.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I like the intuition behind the paper: not all directions equally matter for classification, and we can focus on the more important ones for robustness.\n\nWriting-wise, the paper is on the dense end with a lot of details. This might be a result of answering previous review questions. It can be improved by adding a paragraph header for similar components such as \"choice of MD/transformation matrix\", etc.\n\nThe question I want to understand the most, and the most critical to my final recommendation of the paper, is the choice of attack budget \\delta in the experiment. When you compute the attack success rate against AT models, are the feasible set of AE the same for both uniformly-perturbed and non-uniformly-perturbed AT models?  I'm asking because AT should **not** affect the attacker's set of choice of AE. So I want to make sure that the attack feasibility set is the same.\n\nAn optional comparison result I'd like to see is comparing AT with a preprocessing based defense mechanism: the learner learns the same transformation matrix used in non-uniform perturbation, scales all input using the transformation, and eventually trains a new model over the scaled inputs. It is natural to compare AT with a direct dimension reduction fashion technique that potentially reduces adv. effect over not-so-important direction.  \n\nLast, for the malware detection task, the attack is said to be done on the problem-space. However, EMBER only contains feature vectors. Have you actually created any legit PE files in the attack? If not, please clarify that in the paper. I don't mind assuming certain virtual transformations that most likely can compile and have easy-to-see effect on the feature space. In fact, I won't reject the paper even if the attack is only on feature space for a learning conference. But please make it clear if any actual PE or executables are generated. A proper problem-space attack needs those. \n\nOverall, I believe this paper is of good quality. I'm giving a score of 5 for now, but I'm willing to raise the score once my questions are addressed.\n\n============== After author response ========================\n\nMy questions are addressed. Thank you. I changed my score from 5 to 7.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, they are adequately addressed.", "review_text": "", "extra_scores": {"ethical_concerns": "No concern.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "8BhxwlQtlb", "reviewer": "Reviewer_Dp1A", "summary": "The paper tackles the problem of adversarial attacks against DNNs. The paper revolves around “non-uniform” perturbations: in traditional image-based attacks, it is possible to craft an adversarial sample without considering “inter-feature dependencies”. However, in other application domains where ML can be used, such freedom is not possible, which prevents the application of popular countermeasures, such as Adversarial Training (AT). The paper thus investigates the application of AT in these constrained perturbations scenarios, and compares the effectiveness of AT in “unconstrained” perturbations against the (proposed) “constrained” perturbations.\n\nOverall, the presentation of the paper is excellent.\nThe quality of the English text is excellent.\nFigures and Tables are very good.\nThe topic addressed by the manuscript is intriguing because it takes into account real-world constraints.\nThe references are appropriate.\nThe contribution is small, but significant.\n\nSTRENGTHS:\n+ Very well written\n+ Consideration of real world constraints\n+ Evaluation on multiple datasets of distinct domains (Malware, Bank Fraud, Spam)\n+ Technically sound\n+ Good organization\n\nWEAKNESSES\n- The considered attacks are performed in the feature space (AFTER REBUTTAL: this is not a problem anymore)\n", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I very much liked reading this paper. Unlike “normal” papers on adversarial attacks, what sets it apart is assuming a “more realistic” perspective where the perturbations are subject to realistic constraints (feature dependencies) and the evaluation is conducted on datasets that are not image-related. The combination of these two factors is a very good contribution to the state of the art, because there are many domains where ML can be applied and there is an over-abundance of papers that focus exclusively on computer vision tasks. I also appreciated the overall organization of the paper. The first section is concise and on point, and properly introduces the reader to the subject. In the second section the “proposed method” is explained and the description is technically sound. Then, the third and fourth sections are devoted to the experimental analysis and results, and the evaluation also considers Certified Defenses.\n\nI have only one problem with this paper, and it’s related to the fact that the considered attacks are not performed in the problem space (contrarily to what the authors claim). All experiments rely on datasets that contain “pre-collected” data. Hence, to craft the considered adversarial samples, the authors use such data as base. However, we cannot be certain that such perturbations can truly reflect the actions of a true attacker that performs an adversarial attack in the real world. In other words, I am referring to the fact that the AE are created in the feature space. \nIt would have been very interesting if – among the current experiments – there was an additional use case where the authors use problem-space AE to clearly highlight the key differences between uniform and non-uniform perturbations. I think that a similar addition would dramatically improve the contribution and overall quality of the paper. However, I do not think that the Ember dataset is sufficient to do so (more on this part below). And the other 2 use cases also are not problem space attacks, from what I can understand by the (very limited) information provided.\n\nIn summary, I consider the paper to be in a good state: the presentation and contribution are good enough to recommend a “weak/regular” acceptance. The addition of a (even as a proof-of-concept) use case with a true problem-space adversarial examples would immediately make my recommendation become a “strong accept”.\n\n\nExplanation on EMBER Use Case: In Section 3.1, the authors clearly state that they “We evaluate the robustness of our model against evasion attacks which are crafted in problem-space, i.e., on PE files”. Further on, in lines 209-211, the authors state the following: “We incorporate the most successful attacks [33] from the machine learning static evasion competition of [34]. Since the EMBER dataset only contains the extracted features of a file, a subset of malware binaries used for AE generation are obtained from VirusTotal [35] using the SHA-256 hash as identifier.”\nI believe this experimental methodology to be one of the following: flawed, unclear, or not representative of a problem space attack.\nIn a problem space (malware) attack, the adversary modifies the malware binary. However, I do not see anything in the above description that leads me to believe that this is what the authors did. What the authors should do is take an existing piece of malware executable, modify it, obtain its feature-like representation (using the same structure as in Ember), and then insert it into the Ember dataset. However, as mentioned by the very same authors (in line 199), Ember is a “feature-based dataset”. Hence it is not possible to perform “problem space” attacks using Ember as source. The best that can be done using Ember is operating at the “raw data” space – which is closer to reality than the higher level “feature space”, but definitely not in the “problem space”. \nAll of the above should be clarified in the paper. \nRegardless, all of these details should be added into the main body of the paper. What kind of manipulations did the authors introduce? Are these manipulations “realistically” feasible? Do they represent a “likely” behaviour of a realistic attacker who is trying to evade detection by minimally changing the malware executable? The authors do provide some external references, but they should further describe what is being done in their experiments: by looking at those references, the only (relevant) information I get is that “the attacks must satisfy the physical constraints of PE malware”, but there is no specification of whether the considered attack strategy is either viable or realistically sensible.\n\nSome minor remarks:\n\n•\tThe paragraph from lines 129 to 134 could be better structured. It has lengthy notation that breaks through lines, hence I suggest a revisioning to improve the readability\n\n•\tFigure 1 is very hard to read (I suggest using different colors for all lines)\n\n•\tI would like the authors to consider the related work in [A] (see below) and explain how their paper can be relevant to the scenario envisioned in [A].\n\n\nEXTERNAL REFERENCES\n\n[A]: Erba, Alessandro, et al. \"Constrained Concealment Attacks against Reconstruction-based Anomaly Detectors in Industrial Control Systems.\" Annual Computer Security Applications Conference. 2020.\n\n_________________________--\n\nAFTER REBUTTAL: the authors clarified that the attacks are in the problem space. I will raise my score from 6 to 7.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors should clearly say that the considered attacks are not truly in the problem space and that they may not be what a true attacker would do in reality.\n\nThis is UNLESS the authors decide to thoroughly explain and motivate that I what I wrote is incorrect.\n\n\nAFTER REBUTTAL: the authors clarified that the attacks are in the problem space.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper considers an understudied and yet important area of adversarial ML.  The paper is very well written.  The need to consider attacks adjusted to specific features is intuitive and makes sense. The paper considers real world constraints and evaluates on multiple datasets of distinct domains such as malware, bank fraud and spam. The results showing how the attacks end up looking more like benign samples is compelling, and the extension of the results to certifiable bounds in two different ways is good. After extensive discussions, the added new experiments convinced the reviewers to accept the paper. 
Decision: Accept (Poster)