Probabilistic Categorical Adversarial Attack and Adversarial Training
Authors: Pengfei He
Han Xu
Jie Ren
Yuxuan Wan
Zitao Liu
Jiliang Tang
Venue: Submitted to ICLR 2023 (2023)
Keywords: adversarial attacks
robustness
discrete input model

Abstract: The existence of adversarial examples brings huge concern for people to apply Deep Neural Networks (DNNs) in safety-critical tasks. However, how to generate adversarial examples with categorical data is an important problem but lacks extensive exploration. Previously established methods leverage greedy search methods, which can be very time-consuming to conduct a successful attack. This also limits the development of adversarial training and potential defenses for categorical data. To tackle this problem, we propose a Probabilistic Categorical Adversarial Attack (PCAA), which transfers the discrete optimization problem to a continuous problem that can be solved efficiently by Projected Gradient Descent. In our paper, we theoretically analyze its optimality and time complexity to demonstrate its significant advantage over current greedy-based attacks. Moreover, based on our attack, we propose an efficient adversarial training framework. Through a comprehensive empirical study, we justify the effectiveness of our proposed attack and defense algorithms. 

Under review as a conference paper at ICLR 2023
PROBABILISTIC CATEGORICAL ADVERSARIAL AT- TACK & ADVERSARIAL TRAINING
Anonymous authors Paper under double-blind review
ABSTRACT
The existence of adversarial examples brings huge concern for people to apply Deep Neural Networks (DNNs) in safety-critical tasks. However, how to gener- ate adversarial examples with categorical data an important problem but lack of extensive exploration. Previously established methods leverage greedy search method, which can be very time-consuming to conduct successful attack. This also limits the development of adversarial training and potential defenses for cate- gorical data. To tackle this problem, we propose Probabilistic Categorical Adver- sarial Attack (PCAA), which transfers the discrete optimization problem to con- tinuous problem that can be solved efficiently by Projected Gradient Descent. In our paper, theoretically analyze its optimality and time complexity to demon- strate its significant advantage over current greedy based attacks. Moreover, based our attack, we propose an efficient adversarial training framework. Through a comprehensive empirical study, we justify the effectiveness of our proposed attack and defense algorithms.
INTRODUCTION
Adversarial attacks GGooffelow et al. 2013) have raised great concerns for the applications of Deep Neural Networks(DNNS: in many security-critical domains (Cui et al.) 2019 Stringhini et al. 2010 Cao & Tay 2001). The majority of existing methods focus on differentiable models and continuous input space, where we can apply gradient-based approaches to generate adversarial examples. How- ever, there are many machine learning tasks where the input data are categorical. For example, data in ML-based intrusion detection systems (Khraisat et al. 2019) contains records of system opera- tions; and financial transaction systems, data includes categorical information such as the types of transactions. Therefore, how to explore potential attacks and corresponding defenses for cate- gorical inputs is also desired. Existing methods introduce search-based approaches for categorical adversarial attacks (Yang et al. 2020b Lei et al. 2019a). For example, the method in (Yang et al., 2020a) first finds top-K features of given sample that have the maximal influence on the model output, and then, greedy search is applied to obtain the optimal combination of perturbation in these features. However, these search-based methods cannot be guaranteed to find the strongest adversarial examples. Moreover, they be computationally expensive, especially when data is high dimensional and the number of categories for each feature is large.
In this paper, propose novel Probabilistic Categorical Adversarial Attakk (PCAA) algorithm to generate categorical adversarial examples by estimating their probabilistic distribution. In detail, given a clean sample, assume that (each feature of) the adversarial example follows a categorical distribution, and satisfies:
(1) the generated samples following this distribution have a high expected loss value and (2) the generated samples only have a few features which are different from the original clean sample. (See Section 3 for more details.) In this way, we transfer the categorical adversarial attack in the discrete space to an optimization problem
a continuous probabilistic
space. Thus, we are able to apply gradient-based methods such as (Madry et al., (017)) to find adversarial examples. On one hand, the distribution of adversarial examples in PCAA is searched in the whole space of allowed perturbations. This can facilitate our method to find stronger adversarial examples (with higher loss value) than the greedy search methods (Yang et al., 2020b). Moreover, when the dimension of input data expands, the increase of computational cost of PCAA will be significantly slower than search-based methods (Section(..4). Therefore. our method can enjoy good attacking optimality and computational efficiency simultaneously. For example, in our experiments

---

Under review as a conference paper at ICLR 2023
in Section 5.1
our PCAA method is the only attacking method that has the highest (or close to highest) attacking successful rate while maintaining the low computational cost.
The advantages of PCAA allow us to further devise an adversarial training method (PAdvT), by repeatedly (by mini-batches) generating adversarial examples using PCAA. Empirically, PAdvT achieves promising robustness on different datasets news dataset, we outperform representative baselines with significant margins, approximately 10% improvement in model robustness compared with the best defense baseline; on IMDB dataset, we obtain comparable performance with ASCC-defense(Dong et al.) (021a), which is a SOTA defense method via embedding space adversarial training. However, compared to ASCC, our method does not rely on the assumption that similar words have a close distance in the embedding space. Com- pared to other defenses, our defense has much better robustness by more than 15%. Our main contributions can be summarized below:
various attacks. For example, on AG's
We propose a time-efficient probabilistic attacking method (PCAA) for models with cate- gorical input.
Based on PCAA, we devise a probabilistic adversarial training method to defend categorical adversarial attacks.
2 RELATED WORK
Attacks categorical data. There has been rise in the importance of the robustness of machine learning in recent years. On the one hand, evasion attack, poison attack, adversarial training, and other robustness problem with continuous input space have been well studied especially in the image domain (Shafahi et al. [2018] Madry et al. 2017 [lyas et al., (2019). On the other hand, adversarial attacks focusing on discrete input data, like text data, which have categorical features, are also starting to catch the attention of researchers. Kuleshov et al.] (2018) discussed the problems of attacking text data and highlighted the importance of investment into discrete input data. Ebrahimi it al. (017b) proposed to modify the text token based on the gradient of input one-hot vectors. Gao jt al. 2018) developed scoring function to select the most effective attack and a simple character- level transformation to replace projected gradient or multiple linguistic-driven steps on text data. Samanta & Mehta (0177) proposed an algorithm to generate a meaningful adversarial sample that legitimate in the text domain. Yang et al. (2020a) proposed two-stage probabilistic framework to attack discrete data. Lei et al. (2019b) and Yang et al. (2020a) improved the greedy search and proposed two different greedy attack methods. Alzantot et al. (2018) proposed black-box attack to generate syntactically similar adversarial examples. However, the previous methods usually search in greedy way, which has exponential time complexity.
Defenses on categorical data. There are also a lot of defense methods in the continuous data do- main compared with discrete data. For example, the most effective one is adversarial training Madry et al. (017).. which searches for the worst case by PGD during the training and trains on the worst adversarial examples to increase the robustness of the models. Several works have been proposed for categorical adversarial defenses. Pruthi et al. (2019) used word recognition model to preprocess the input data and increased the robustness downstream tasks. Zhou et al. (2020) proposed to use random smoothing to defense substitution
attacks. Wang et (2021) detected adversarial ex- amples to defend synonym substitution attacks based on the fact that word replacement can destroy mutual interaction. [Swenor& & Kalita (2022) used random noise to increase the model robustness. Xie et al. (2022) used a huge amount of data to detect the adversarial examples. Dong et al. (2021b) combined different attack methods, such as Hot-flip (Ebrahimi et al. 2017a and 12-attack MMiyato et al. 2017). and adversarial training as the defense method. It also proposed its own adversarial method ASCC which takes the solution space convex hull of word vectors and achieves good performance in sentiment analysis and natural language inference. However, the defense methods mainly focusing NLP, which may rely word embeddings.
PROBABLISTIC CATEGORICAL ADVERSARIAL ATTACK (PCAA)
3.1 PROBLEM SETUP
Categorical Attack. We first introduce necessary definitions and notations. In detail, we consider classifier that predicts labels y E y based categorical inputs E X. Each input sample
2

---

Under review as a conference paper at ICLR 2023
Back Propagation
Model f(x)
Gumbel Softmax
sampling
Loss ((f()()))
TL
Probability space
features
Figure 1:
The overall framework of Probabilistic Categorical Adversarial Attack. First, a dis- tribution used to sample adversarial examples x'. Then, the gradient of loss function will be used to update the probability distribution π.
contains n categorical features, and each feature æ; take value from d categorical values. In this paper, we consider lo-attack, which restricts the number of perturbed features. In particular, given the budget size we aim to find an adversarial example x' which solves the following optimization problem:
C(f(x').)) s.t. || z|1o S.
Existing search-based methods. To solve the problem, there existing search- methods such as Greedy Search (Yang et al.] (020a)) and Muddient-Guider Search (Lei et al.] (2119)). In general, whose change can result in the
they consist of two major steps. First, they search for features greatest influence on the model output. This can be estimated by either perturbing feature vi, or calculating the scale of the model's gradient to the feature z₁, The second step involves greedy or global searches among all possible combinations of the features identified by the first step. This process is time-consuming the feature dimension n or the number of categories large.
3.2 OBJECTIVE OF PROBABILISTIC CATEGORICAL ADVERSARTAL. ATTACK (PCAA)
Probabilistic Categorical Attack. In this work, we propose an alternative approach solve the problem in [q. by transferring it to the continuous probabilistic space. In detail, we assume that each feature of (adversarial) categorical data x follows categorical distribution: Categorical( where πi E II; (0,1)d. And each Tij represents the probability that the feature i belongs to the category j. In the remaining of the paper, we will use rT to denote the categorical distribution Categorical (T)) without the loss of generality. Therefore, each input sample 's distribution can be represented as array
[To: T1: nn. E II C. Rnxd where the element Ta. represents the probability that the feature z; belongs to the category j- Then, we define the following optimization problem to find probability distribution ı in the space of TT:
max Ex' ((f()((((()))))) xell
S.L. Pr (I|a æl1o 2 c) 5 a/
where denotes the perturbation budget size and is the tail probability constraint. By solving the problem in Eq. (2. we aim to find a distribution with parameter such that: (1) on average, the generated samples x' following distribution æ have high loss value; and (2) with low probability, the sample z' has a ln distance to the clean sample larger than In this way, the generated samples x' are likely to mislcad the model prediction while preserving most features of x.
3.3 AN EFFICIENT ALGORITHM OF PCAA
In this subsection, we provide feasible algorithm solve our proposed Eq. in practice. First, in order to handle the probability constraint in (q. we substitute the lo distance between x' and by calculating the sum of Cross Entropy Loss between n. and xy: ( 24) for features i E |z]. It is because LCg(T TT, 244 measures the probability that the i-th feature of the generated samples is different from I;. Therefore, we use the sum of Cross Entropy Σ (CE(m, .r)) 3 to approximate the number of changed features, which is the in difference ||r~ - x lol In our algorithm, penalize the searched when the term Esein LC 22) exceeds positive value (as Eq.

---

Under review as a conference paper at ICLR 2023
3). In this case, we equivalently limit the probability that the generated samples x' have the number of perturbed features larger than e.
Pr (1 x 2)
LCE Te) -
(3)
Moreover, since the Cross- Entropyy Loss is differentiable in terms of we further transform the problem its Lagrangian form:
max Ex' ((f(')) - X Σ CCE(a,,
(4)
sE/m
where À is the penalty coefficient, and [-]+ is max(:, 0).. Next, we will show how to solve the maximization problem above by applying gradient methods.
Back propagation through Gumbel-Softmax. Note that the gradient of the expected loss function
with respect to π cannot be directly calculated in Eq. 4 so we apply the Gumbel-Softmax esti- mator Jang et al. (2017). In practice, to avoid the projection onto probability simplex which hurts the time efficieccy and Gumbel-Softmas does not require a normalized probability distribution, we consider an unnormalized categorical distribution म; E (0, C7d where C' > is large constant guaranteeing that the searching space is sufficiently large. The distribution generates sample vectors z follows:
exp((log Tij + 9;)/T)
for j = 1,...,
exp( Tij + 9s)/T))
where denotes i.i.d samples from the Gumbel 1) distribution, and r denotes the softmax tem- perature. This parameterrizaiion process facilitates us calculate the gradient of the expected loss in terms of ग. Therefore, we can derive the following estimator of gradients for the expected loss:
OR = C(S(x'))) On
OC Ox'
OL Ox' an'(T, 9i)
θn ( v) = Egg
2
(6)
Ox θn
Ri
Ox
where ng is the number i.dd samples from g. In Eq. 6 the first approximation is from the repa- rameterization of sample z'; the second equality comes from exchanging the order of expectation and derivative, and the third, we approximate the expectation gradients by calculating the average of gradients. Finally, we derive the practical solution solve Eq. 4 as demonstrated in Figure [] by leveraging the gradient ascent algorithm, such as Madry et al. (2018). In Algorithm[]] we provide the details of our proposed attack method. Specifically, in each step, we first update the unnormalized distribution π by gradient ascent. Then, we clip π back to its domain (0, CT
Algorithm Probabilistic Categorical Adversarial Attack (PCAA)
Input Data D, budget c, number of samples ng. penalty coefficient X, maximum iteration I, learning rate Y
Output Adversarial Distribution π Initialize distribution " for t < I do
Estimate expected gradient using E
V B, 2 1 Ene
Tcc (x'(n' .9.) Ox
On
Gradient ascent: g++1 = n² +
E. AVI(CcE(TT +
Clip to (0, C7d: xf+1 max (7++1
end for
Return distribution π
3.4 TIME COMPLEXITY ANALYSIS
In this subsection, compare the time complexity of our method with search-based methods to illustrate the efficiency advantage of our proposed attack method. Here, we assume that the whole dataset has N data points, each data point has features, each feature has d categories and the budget of allowed perturbation is We first introduce the details of four different search- based methods.
4

---

Under review as a conference paper at ICLR 2023
Greedy Search(GS). It consists of two stages. The first stage involves traversing all features. For the it feature, it replaces the original category with all other - categories respectively and records the change of the model loss for each category. The largest change is treated as the impact score for the ieh. feature. Then selects the top features with the highest impact scores perturb. In the second stage, it finds the combination with the greatest loss among all possible combinations of categories for selected features.
et al. 2020a). This method is a modified version of Greedy Search. The first stage is similar to that of GS, while the second stage searches for the best perturbation feature by feature. For the 4th selected feature, replaces the original category with one that results in the largest loss and then searches the next selected feature until all selected features are traversed. Therefore, it only needs to go over each feature once without traversing all combinations in the second stage.
Greedy (Gtlack.(GA)(eeeee
2019a). In order to determine which features should be perturbed, this method utilizes gradient information in the first stage. It computes the gradient of the loss function w.r.t the original input and treats the gradient of each feature as the impact score. Those features with the greatest impact scores second stage, it follows the same strategy as that of the second stage in GS, and pursues the combination resulting the greatest loss among all possible combinations.
Gradient-guided GG.(GG)
selected to be perturbed. In the
Gradient-guided greedy attack(GGA), On the basis of GGS, it remains the same first stage and modifies the second stage by adopting the same strategy as that in the second stage of GA.
We analyze the time complexity of our PCAA and the aforementioned methods for comparison.
Probability Categorical Adversarial Attack. In practical implementation, adopt the uncon- strained optimization in Eq. 4 Therefore, the time complexity is only from gradient descent. Fur- ther, assume that we sample n, times when estimating the expected gradient and the maximum number of iterations is /. We need to compute gradient n, times during one iteration, thus the time complexity of PCAA is:
n. ((d)) - C,NO(nd)
where C1 is some constant related to n, and I.
Greedy search. As described above, the Greedy search consists of two stages. In the first stage, needs to traverse all features and compute a loss when plugging in each level. Then n d times of calculation are needed. In the second stage, all possible combinations of categories for top c features need to be considered and the number of possible combinations is d'. Therefore, the time complexity for Greedy Search is
N [Sstage1) + [[stage2)] N
- [O(nd) + O(d)) N O(nd + d¹)
(8)
Greedy attack. In the first stage, it traverses all categories within each feature to find the best e features to perturb, which needs n d computation. And in the second stage, it searches selected features respectively for the d categories that lead to the greatest loss. Thus, the time complexity is N [[(stage1) + [[stage2)] N [O(nd) + O(ed)] N O(nd + ed)
(9)
Gradient guided greedy search. In the first stage, it computes gradients w.r.t the original input, and in the second stage, it also needs to traverse all possible combinations for selected features. Therefore the time complexity is
N [S(stage1) + M(stage2)] = N (O()) O(d)) N +n++d")
(10)
tradient-guided greedy attack. This method combines the first stage in GGS and the second stage in GA. Thus its time complexity is
[D(sstage)) + [stagge2] = [((()) + O(dd)] = N O(n + ed)
(1I)
From the above analysis, two search-based methods suffer from the exponential increase of time complexity ( 10 when the number of feature categories d and budget size is increasing. More- over, two greedy-basee methods (GA and GGA) accelerate the second stage and achieve better time efficiency, but sacrifice the performance as they greatly narrow down the searching space. However, PCAA is free of these problems, since it achieves great time efficiency to greedy-based methods while remain comparable performance with search-based methods. We will show this with more evidence in the experimental part.
5

---

Under review as a conference paper at ICLR 2023
4 PROBABILISTIC ADVERSARIAL TRAINING (PADVT)
Adversarial SrainingSzegedy et al. (2014 Goodfellow et al. 2015 Madry et al. (2018) is one of the most effective methods build robust models. Its training incorporates adversarial examples to im- prove robustness. However, greedy methods have limited applications in adversarial training when dealing with categorical input models due to their slow speed. The proposed PCAA method signifi- cantly improves time efficiency. In this case, we propose Probabilistic Adversarial Training (PAdvT) based on PCAA in order to train robust models on categorical data. Recalling the formulation of PCAA in Eq. 4 and denoting the parameters for classifier f with 0, the training objective for PAdvT formulated as
1D1
min
max E of
((f(2';))))) -
Σ LOE Tt)) S JEIn
(12)
Since our objective involves a penalty coefficient, we adopt the strategy Yurochkin et al. (2020) to update Ả during training. We adaptively choose À according to CCE(#, π) from the last iteration: when the value is large, we increase to strengthen the constraints and vice versa.
The implementation of PAdvT is illustrated in Algorithm. 2 It first solves the inner maximization problem by applying algorithm. [T. Then it samples Made examples from the adversarial distribution Finally, it applies Adam Kingma & Ba (2015) to solve the outer minimization problem. The process will continue until the maximal number of iterations is reached.
Algorithm Probabilistic Adversarial Training (PAdvT)
Input Data D, parameters of clean model 0, budget parameters of Algorithm Nadv initial penalty coefficient 2o, penalty coefficient step size a, parameters of Adam optimizer, number of
iterations 7
Output parameters 0 of robust model
repeat
Sample mini-batch B {x= ,..., ".) from training
for i = 1,... miin parallel) do
Apply Algorithm to x² and obtain adversarial distribution Z' Sample Rado examples {x",,.., X X from ग¹ using Gumbel Softmax tricks
end for
Update / by Adam to minimize the average adversarial loss Update max (0, a(ς m Eie(m) Eje(n) (cE(E),=)) x))) until Training converged Return parameters 0
5 EXPERIMENT
In this section, we conduct experiments to validate the effectiveness and efficiency of PCAA and PAdvT. In Section [..] we demonstrate that PCAA achieves better balance between attack success rate and time efficiency. In Section. 5.2 PAdvT achieves competitive stronger robustness against categorical attacks across different tasks.
5.1 CATEGORICAL ADVERSARIAL. ATTACKS
Experimental Setup. In this evaluation, we focus on three datasets. Intrusion Prevention Sys- tem (IPS) (Wang et al., 2020). IPS dataset has 242,467 instances, with each input consisting of 20 features and each feature has 103 categorical values. The output space has three labels. A stan- dard LSTM based classifier et al. 2022) trained for IPS dataset. (2) AG's News corpus. This dataset consists of titles and description fields of news articles. The tokens of each sentence corre- spond to the categorical features, and the substitution set (of size 70) corresponds to the categorical values. A character- based CNN(Zhang et al. 2013) is trained on this dataset. (3) Splice-junction Gene Sequences (Splice) (Noordewier et al. 1990). Splice dataset has 3,190 instances. Each one

---

Under review as a conference paper at ICLR 2023
Table 1: Attacking results on IPS, AG's news, and Splice datasets. SR. represents the attack success rate; represents the average running time in seconds; and "." indicates the running time over 10 hours. Fach experiment runs 5 times and 95% confidence intervals are shown.
Attack Method
Datasel
SR.(1)
T(U)
SR.(1)
TX)))
SR.(1)
T.Q))
SR.())
T(O))
SR.(1)
T(O))
GS
66.11
38.5
8..24_0.01
2028
GA
66.11 10.03
35.4
71.32-00.02
38.1
79.44.10.06
39.9
85.28
41.5
91.15.1.111
43.2
IPS
GGS
41.53:0.04
2,06
15.47=0.03
1022
GGA
41.53=00.4
0.98
63.72=0.06
2.01
70.76=00055
2.94
75.89+0.08
3.74
82.43:0010
4.56
PCAA
60.66+00.05
6.62
77.51=0044
6.64
8..32=0008
6.75
91.67=00.06
6.66
94.63=0.12
6.54
GS
41.22=0.01
15.3
67.38=0.02
21.9
75.87=00.04
356
83.10=+0002
19160
GA
41.22+00.11
15.3
60.71-0.05
15.4
66.33++0.04
15.5
74.47+00.07
15.7
86.63=0.122
15.9
AG
GGS
32.39-0.03
0.352
59.21 0.02
67.79+0..0
79.22-+0.05
7551
GGA
32.39
0.352
41.29
0.393
56.11.10.06
0,511
67.53.10.10
0.613
72.28.10.07
0.856
PCAA
47.31
9.02
61.47-0.00 0.09
70.81+00.06
9.11
79.55++0.13
9.06
0.10
GS
72.11 10.01
0.5015
75.44 0.002
1:02
81,4110.00
1,36
85.1910.03
2.73
90.88 0.004
K.25
GA
72.11-0.001
0.905
74.42-0.03
18.06
0.915
0.922
83.74-0005
Splice
GGS
61.71
0.028
68.28+ 0.03
0.083
72.82.10.02
0.251
77.1110.04
0.917
82,53.10.04
3.56
GGA
61.71:0.02
65.26=0.08
70.31
0.0337
74.8410.07
80,4920.08
PCAA
64.32=0.00
1.57
70.83=0.07
1.62
79. 79.28:00.09
1.67
82.82:00006
1.73
84.77=0.11
1.39
is gene fragment of 60 features with 5 categorical values. The output space has three labels. The LSTM model is trained for Splice.
Baseline Attacks. Our goal is to generate powerful attacks categorical models directly on input space, without relying on the embedding space. Thus, search-based methods are most suitable in this setting. Therefore, we compare PCAA with the following search- based attacks including Greedy Search(GS), Greedy Attack(GA), Gradient- guided GS (GGS) and Gradient- guided greedy (GGA) The details of these baselines can be found in Section 3.4.
For each dataset, we evaluate the performance in terms of the attack success rate (SR.) and the average running time (T.) under various budget sizes ranging from 1 to 5. Since PCAA learns the adversarial categorical distribution, the generation of adversarial examples is based on sampling. In the evaluation, we sample 100 examples from the adversarial distribution for each attack instance and dismiss those with more perturbed features than the budget size. We claim success attack if one out of all samples successfully changes the prediction. The average running time for PCAA includes both the optimization of Eq. and the sampling process.
Performance Comparison. The experimental results on IPS, AG's news, and Splice datasets are demonstrated in [ (1) In IPS dataset, each data contains more than 1000 categories for each feature, and our method has significant advantage. As the budget size increases, GS and GGS run so long that they are longer feasible. While GA and GGA remain efficient, our method outperforms them by significant margins, e.g., over 10% higher than GGA in success rate and faster over 7 times than GA in time. (2) In AG's news data, GS has the highest success rate with small budgets. However, our method PCAA has much better time efficiency than GS under all budgets while maintaining competitive performance and outperforming all the other attacks. (3) In Splice, there are only 5 categories for each feature, which corresponds to the low-dimensional cases. The advantage of our method may not be as great as in other cases (i.e., IPS and AG) as the running time of greedy methods is acceptable. However, PCAA still generates strong attacks with only 6% less than GS in success rate, and remains highly efficient when the budget size increases, so it still provides a practical and competitive attack.
Time Efficiency Comparison. is evident from Table I that PCAA's average running time is unaffected by the size of the budget, whereas greedy methods are clearly more time-consuming as budgets increase. Thus, the proposed PCAA method significantly improves time efficiency. As shown in Table[] when the substitution set is large (e.g., IPS and AG), the running time of GS and GGS will explode even under some moderate budget size such as = 3. Since the running time is too long for meaningful comparison, we do not record the performance for these budget sizes of those baselines. In small substitution sets, i.c., Splice, the time efficiency gap between PCAA and greedy methods is not large. Due to the small size of the substitution set, greedy methods need much fewer quires than in high- -igheeimnniional cases, but PCAA still takes a much shorter time for a budget size of more than 4.
7

---

Under review as a conference paper at ICLR 2023
Gban Brior Genetic 665 6665 PCAA
200 300
20.000
10:30M
IRM
Hrtt-FPp
ASCC
PAINT
Hob-FFp
Adx_
PACTT
Figure 3: PAdvT and baseline defense performances under different attacks on IMDB dataset.
5.2 CATEGORICAL ADVERSARIAL DEFENSES
Experimental Setup. For the defense evalua- tion, we focus on two datasets, AG's News Cor- pus and IMDB, It is because IPS and Splice have too few samples (no more than 1000 for each dataset). In particular: (1) AG's
Clean Hot-Fipp
GS
GGS
GGA
PPCA
50%
Error
News corpus. is the same dataset used in the attack evaluation and the model is also character- CNN. As character swapping does not require embeddings for each charac- ter, we can apply attacking methods on input space. Therefore, the robustness the defense models is evaluated using six attacks,i.e. Hot- Flip (Ebrahimi et al. (017a), GS, GA, GGS, GGA, and PCAA. (2) IMDB reviews dataset Maas et al. 2011). Under this dataset, focus on word-level classification task and study two model architectures, namely Bi-LSTM and CNN, are trained for prediction. To evaluate the robust- four attacks are deployed, including genetic attack (Alzantot et al. 2018) (which is attack method proposed to generate adversarial examples embedding space), as well as GS, GGS, and PCAA.
30%
10%
ERM
Hot-Flip
PAdVT
Figure 2:
PAdvTT and baseline defenses under dif- ferent attacks on AG's news dataset.
Baseline Defenses. We compare our PAdvT with following baseline defenses:
Standard training. It minimizes the average cross-entropy loss on clean input.
(ot-alip(Errahimi et al. 2017a). uses the gradient with respect to the one-hot input repre- sentation to find out the which individual feature under perturbation has the highest estimated loss. It is initially proposed to model char flip in Char-CNN model, and we also apply it to word-level substitution, as in (Dong al. 2021a).
Ady Miyyaoo et al. 2017). ) It uses an la PGD adversarial attack inside the word embed- ding space for adversarial training.
ASCC-Defense(Dong al. 2021a). A state-of-the-art defense method in text classification. uses the worst perturbation over a sparse convex hull in the word embedding space for adversarial training.
Performance Comparison. The experimental results on AG's news dataset are shown in Fig. The Y-axis represents the eиror rate and X-axis represents different defense methods while mulu- rows represents different attacking methods. Our defense method achieves leading robustness on Char-CNN over all attacks with significant margins, surpassing Hot-Flip-defense by 10%. Fig. il- lustrates the defense results on the IMDB dataset, and we have similar observations with Fig. Our PAdvT shows competitive adversarial robustness as ASCC-defense. Notably, ASCC is defense method that conducts adversarial training on word-em gooddeebbeding space. It relies on the key assumption that similar words have close distance in embedding space. However, method does not rely on this assumption, which may result in the performance being competitive (slightly worse) than ASCC. For all other defenses, it outperforms them across different architectures significantly.
8

---

Under review as a conference paper at ICLR 2023
5.3 ABLATION STUDY
Concentration of PCAA. To further understand the behavior of our attack algorithm, in this sub- section, we ask the question: what is the variance of our optimized probability distribution x' (from solving )² Intuitively, we desire the distribution æ to have smaller variance, so that we don't need too many times of sampling obtain the optimal adversarial examples. To confirm this point, we conduct an ablation study based on an experiment on IPS dataset to visualize the distribution n, which is optimized via PCAA under various budget sizes. In Fig 4 choose three budget sizes e 1.3.5 and choose features to present the adversarial categorical distributions. Notably, in Fig. the left two columns the features that not perturbed. The right columns fea- tures which will be perturbed by PCAA. From the figure, we can see that for each feature (perturbed or unperturbed), there exists one category with much higher probability compared other cate- gories. This fact indicates during the sampling process of PCAA, for certain feature, the samplings are highly likely have the same category for this feature. As result, we confirm that our sampled adversarial examples are well-concentrated.
Ablation analysis for PAdvT. In our training objective in Eq. for adversairal training and possibly affects the robustness of the model. We conduct ablation study on the IMDB dataset understand the impact of Ç. The results are demonstrated in Table. E When increases, the success rates of all attacks decreases, meaning that the robustness of models enhanced. However, large Ç will increases the clean error and sacrifice the clean accuracy of models. Thus, ( controls the balance between the clean accuracy and the adversarial robustness of the model. When Ç = 0.4, our algorithm reaches a good balance between model accuracy and robustness.
controls the budget size used
Table 2: Ablation study: impact of the budget regularization term ( on PAdvT
Clean Err
Genetic SR
GS SR
Gradient Search SR
PCAA SR
LSTM
CNN
LSTM
CNN
LSTM
CNN
LSTM
CNN
LSTM
CNN
= 0.1
16.1
16.9
37.5
29.6
41.3
43.5
39.5
41.4
40.2
42.2
= 0.2
17.2
17.3
34.1
32.7
39.9
41.3
38.6
39.7
39.1
40.6
0.32
17.9
18.1
30.1
31.4
38.7
38.3
38.5
37.6
38.8
37.8
0.4
18.4
18.6
26.3
28.5
37.8
36.2
36.4
34.9
36.9
35.7
250
250
750
3000
Figure 4: Visualization of Optimized Categorical Distribution for Various Features (IPS)
CONCLUSION
In this paper, propose novel probabilistic optimization formulation for attacking models with categorical input. Our framework significantll improves the time efficiency compared with greedy methods and shows promising empirical performance across different datasets. Furthermore, design an adversarial training method based on the proposed probabilistic attack that achieves better robustness.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani B. Srivastava, and Kai- Wei Chang. Generating natural language adversarial examples. In Ellen Riloff, David Chiang, Julia Hockenmaier, and inhi Tsujii (eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium, October 31 November 4, 2018, pp. 2890-2896, Association for Computational Linguisties. 2018. doi: 10. 10.18653/v1/d1- 1316. URL http::///obb.org1100881////////1
1316
Hongyan Bao, Yufei Han, Yujun Zhou, Yun Shen, and Xiangliang Zhang. Towards understanding the robustness against evasion attack categorical data. In The Tenth International Conference on Learming Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL
Lijuan Cao and Francis Eng Hock Tay. Financial forecasting using support vector machines. Neural Comput, Appl., 10(2) 192- 2001. doi: 10.1007/0002201100010 URL tittps://dot
Jin Cui, Lin Shen Liew, Giedre Sabaliauskaite, and Fengjun Zhou. A review on safety failures, security attacks, and available countermeasures for autonomous vehicles. Ad Hoc Networks, 90, 2019. doi: 10. 10.10///.dooo.2018..... 12.006. URL 2018. 12. 606
adhoc.
Xinshuai Dong, Anh Tuan Luu, Rongrong Ji, and Hong Liu. Towards robustness against natu- ral language word substitutions. In 9th International Conference Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021a. URL https: //openzeveww.co htw.://www....ooooee///eeeeeeeee
Xinshuai Dong, Anh Tuan Luu, Rongrong Ji, and Hong Liu. Towards robustness against natural language word substitutions. arXiv preprint arXiv:2107, 13541, 2021b.
Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White-box adversarial examples for NLP. CORR, abs/1712.067511 2017a.
Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White-box adversarial examples for text classification. arXiv preprint arXiv: 1712.06751, 2017b.
Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. Black-box generation of adversarial text sequences to evade deep learning classifiers. In 2018 IEEE Security and Privacy Workshops (SPW), pp. 50-56. IEEE, 2018
lan Goodfellow, Jonathon Shiens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA. USA, May 7-9, 2015, Conference Track Proceed- ings, 2015. URL U http:////w/wooo/a///////////////
6572
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. Advances in neural information processing systems, 32, 2019.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterizatioo with gumbel-softtax, In 5th International Conference Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net. 2017. URL htttttt::///peeeeeeeeee
Ansam Khraisat, Iqbal Gondal, Peter Vamplew, and Joarder Kamruzzaman. Survey of intrusion detection systems: techniques, datasets and challenges. Cybersecur. 2(1):20, 2019. doi: 10. 1186/442400-09-00033. URL https::/ddo 1ot.///../././////.... 010-0000---000-----0
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference Learning Representations, ICLR 2015, San Diego CA, USA. May 7-9, 2015, Conference Track Proceedings. 2015. URL http: 7/arxiv.
6980
10

---

Under review as a conference paper at ICLR 2023
Volodymyr Kuleshov, Shantanu Thakoor, Tingfung Lau, and Stefano Ermon. Adversarial examples for natural language classification problems. 2018.
Qi Lei, Lingfei Wu, Pin- Yu Chen, Alex Dimakis, Inderjit S. Dhillon, and Michael J. Witbrock, Discrete adversarial attacks and submodular optimization with applications to text classification. In Ameet Talwalkar, Virginia Smith, and Matei Zaharia (eds.), Proceedings of Machine Learning and Systems 2019, MLSys 2019, Stanford, CA, USA, March 31 April 2, 2019. mlsys.org, 2019a. URL htttp:://pooeeeeiiitt
mi myssyaaa. org.ooa///oooo..oooo.oooooooooo. 284.
Qi Lei, Lingfei Wu, Pin- Yu Chen, Alex Dimakis, Inderjit S Dhillon, and Michael J Witbrock. Dis- crete adversarial attacks and submodular optimization with applications to text classification. Pro- ceedings of Machine Learning and Systems, 1: 1:146-165, 2019b.
Andrew L. Maas, Raymond Daly, Peter Pham, Dan Huang, Andrew Y. Ng. and Christo- pher Potts. Learning word vectors for sentiment analysis. In Dekang Lin, Yuji Matsumoto, and Rada Mihalcea (eds.), The 49th Annual Meeting of the Association for Computational Lin- guistics: Human Language Technologies, Proceedings of the Conference, 19-24 June, 2011, Portland, Oregon, USA, pp. 142 150. The Association for Computer Linguistics, 2011. URL https: http::::///alaahhooooyy/ wwtp://loooo..ggo111o1/e
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.066 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In 6th International Conference Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL
net./
Forum?id=r:J
Takeru Miyato, Andrew M. Dai, and Ian Goodfellow. Adversarial training methods for semi- supervised text classification. In International Conference on Learning Representations, 2017. URL http:t::///peeeeeiiieee net wttp://wwww.ee///ooeieeeeee.e htte://ommmmmmdd1-1-11
x1
Michiel Noordewier, Geoffrey Towell, and Jude Shavlik. Training Knowledge-based neural networks to recognize genes in dna sequences. Advances neural information processing systems, 3, 1990.
Danish Pruthi, Bhuwan Dhingra, and Zachary C Lipton. Combating adversarial misspellings with robust word recognition. arXiv preprint 190. 2019.
Suranjana Samanta and Sameep Mehta. Towards crafting text adversarial samples. arXiv preprint 1207001707.02822. 2017.
Ali Shafahi, W Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein. Poison frogs! targeted clean-label poisoning attacks on neural networks. Advances in neural information processing systems, 31, 2018
Gianluca Stringhini, Christopher Kruegel, and Giovanni Vigna. Detecting spammers social net- works. In Carrie Gates, Michael Franz, and John P. McDermott (eds.), Twenty-Sixth Anmual Computer Security Applications Conference, ACSAC 2010, Austin, Texas, USA, 6-10 December 2010, pp. 1-9. ACM, 2010, doi: 10.1145/1920261.1922
URL htthttts///o..og////
1145/1920261. 100.61.19200 63
Abigail Swenor and Jugal Kalita. Using random perturbations to mitigate adversarial attacks sentiment analysis models. arXiv preprint arXiv:2202.05575, 2022.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Good- fellow, and Rob Fergus. Intriguing properties of neural networks. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference on Learning Representations, ICLR 20/4, Bantf, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL http:
//arxiv.org/ 2022g//bs//31..66 6199
Xiaosen Wang, Yifeng Xiong, and Kun He. Randomized substitution and vote for textual adversarial example detection. arXiv preprint arXiv:2 1rXiv:2109006698 2021.

---

Under review as a conference paper at ICLR 2023
Yutong Wang. Yufei Han, Hongyan Bao, Yun Shen, Fenglong Ma, Jin Li, and Xiangliang Zhang. Attackability characterization of adversarial evasion attack on discrete data. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash (eds.), KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pp- 1415-1425 ACM, 2020. doi: 00.1145/394448..4033111 URL https: //ooi org/10. 1145/3394486. 3403194
Zhouhang Xie, Jonathan Brophy, Adam Noack, Wencong You, Kalyani Asthana, Carter Perkins, Sabrina Reis, Sameer Singh, and Daniel Lowd. Identifying adversarial attacks on text classifiers. 2022.
arXiv preprint
Puyudi Yang, Jianbo Chen, Cho-Jui Hsieh, Jane-Ling Wang, and Michael Jordan. Greedy attack and gumbel attack: Generating adversarial examples for discrete data. J. Mach. Learn. Res., 21: 43:1-43:36, 2020a. URL BtRLhtp::// http:/mmrrrooo wttpp//opwpppppe///////////////e
569. html
Puyudi Yang, Jianbo Chen, Cho-Jui Hsich. Jane-Ling Wang. and Michael I Jordan. Greedy attack and gumbel attack: Generating adversarial examples for discrete data. J. Mach. Learn. Res., 21 (43):1-36, 2020b.
Mikhail Yurochkin, Amanda Bower, and Yuekai Sun. Training individually fair MI models with sensitive subspace robustness. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, OpenReview.net, 2020. URL https: /oopereeveww wwweenreveewnee///oooe
Xiang Zhang. Junbo Jake Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In Corinna Cortes, Neil D. Lawrence, Daniel D. Lee, Masashi Sugiyama, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 28: Annual Confer- ence on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Que- bec, Canada, pp. 649-657, 2015. URL httpt:ttt//ppptt httttssss//ppcccciinssss.s. nww. ips. cc/paper/ wtt.aa/aaaaaabaaaaaaaaaa.a.aaa.. Abst .ttm_
Yi Zhou, Xiaoqing Zheng, Cho-Jui Hsich, Kai-wei Chang, and Xuanjing Huang, Defense against adversarial attacks in nlp via dirichlet neighborhood ensemble. arXiv preprint arXiv:2006. 16627, 2020.
A APPENDIX
A.1 ADDITIONAL. EXPERIMENTAL. RESULTS
In this subsection, we present the results of Fig. 2 and 3 where the values represent error rates. We each experiment for 5 times and compute the 95% confidence interval.
Table 3: PAdvT and haseline defense performances under different attacks on IMDB dataset
Clean
Genetic
GS
Gradient Search
PCAA
LSTM
1350=00005
91:0100001
0.54:00000
81.450003
Hotfiip
17.57=0.09
16.57=0051
50533000000
35.03-0.025
75.24000155
66.25-00022
67.66-0001
68.8-0009
65.0550074
Adv
32.50:0004
10.38:0003
00359100066
04.35:00333
0.60000000
67.00-00044
08.34:0001
65.81:00088
78.58:00068
66.23:0077
ASCC
17.6600096
03000
19.66-0.447
22.52:0.046
0..6700.47
33.88-00445
03.66-00054
52.1:0061
33,07+0.00 101
02.77:0082
PAAVT
18.5700053
18.5500049
26.0100055
28.35110.049
37.30_00.063
36.36_00.011
35.60.0.067
348550.053
25.0610.004
25.47100053
Table 4: PAdvT and baseline defense performances under different attacks on AG's news dataset
Clean Err
Houlip
PCAA
GS
GGS
GA
GGA
ERM
8.70+40.009 13.9910.017 14.62+0.028
79.12+0.015
80.75+0.006
83.09=0.0015
79.28
74.43+0.010
67.53+0.016
Hotflip PAdvI
60.07100.01
63.4710.057
62.41110.019 50.65:10.035
64.28110.018
58.3310.013 60.5110.017
45.3810.037
49.50=0.081
47.14=0.041
44.71+0.040
42.18:0.045
We also PAdvT with mixture of adversarial examples and clean samples IMDB dataset. Results are shown in 3 values represent eтror rates. is noticeable that clean samples will slightly improve the clean performance and lead small decreasing of robustness.
12

---

Under review as a conference paper at ICLR 2023
Table 5: Comparison of PAdvT on IMDB dataset with/without mixture of clean samples.
Clean Err
Genetic
GS
GGS
PCAA
IMDB LSTM(mix)
18.27
26.22
37.67
35.79
36.05
IMDB LSTM
18.57
26.01
37.3
35.60
35,46
IMDB CNN(mix)
18.69
28.51
36.47
34,96
35.72
IMDB CNN
18.85
28.35
36.36
34.85
35.47
A.2 VISUALIZATION ON IMDB
We run PCAA attack on IMDB dataset over two victim models LSTM and word-CNN. The can- didate sets are pre-specified synonym set. The following Tables 6 and 7 show some successful adversarial examples, where the red words are adversarial words and blue are original words. It is obvious that these replacements do not hurt the semantic meaning but can fool the classifiers.
Table 6: IMDB Adversarial Examples from PCAA LSTM
Class
Perturbed Class
Perturbed Texts
Negative
Positive
watched this film for 45 minutes and counted 9 mullets. That's mullet every minutes. Seri- ously, though this film is residing evidence(living proof) that formula works if it ain't broke, it don't need fit in streetwise yet vulnerable heroine, hardened ex-cop martial arts master with a heart of gold and serial killer with 'issues' pure magic. Claustrophobic camera angles that do not aid(help) the movie. Too long face only shots, where you most of the time get the hunch(feeling) that the lower half of the film is missing that the screen is cut off because there seems be important actions going on, but you can not see them. There is anyway already much confu- sion in the movie, so these viewing angles make it worse and do not contribute artful visuals. I like artfully made movies and unconventional camera work. can handle deep and slow movies but this one is trying too hard to be something artful and fails, in my opinion, painfully. Nothing to get attached to any of the characters because they are not worked out well enough to work out characters. More is needed than just minute long face shots. At least with this set script director actors, I wonder whether some of the not so decent(good) acting is due to the script and director due to the actors. will stay away from films both written and directed by le you for sure in the future. What an annoying film even for person(someone, who would be interested in that part of history and for someone who spent time in Shanghai.
Negative
Positive
13

---

Under review as a conference paper at ICLR 2023
Positive
Negative
I really liked this version of 'vanishing point' as opposed to the 1971 version. finds(found) the 1971 version quite boring if can get up in the middle of movie few times 1971 version, then to me it is not all that great. Of course, this could be due the fact that was only nine at the time the 1971 version was brought out. However, I have noticed(seen) many remakes ev- erytime (where) I have liked the original and older one better. I found that the plot of the 1997 ver- sion was more understandable and had basically kept true to the original without undermining the meaning of the 1971 version. In my opinion felt the 1997 version had more excitement and wasn't blasé boring.
did with the
Positive
Negative
The cast is marvelluus(excellem)) the acting good, the plot interesting, the evolvement full of sus- pense, but it is hard to cram all those elements into film that is barely 80 minutes long. If more time was taken to develop the plot and subplots, it would have a much better effect. Another 30 minutes of substance would have made this a very alright(good) film rather than just a good one. There great detail in 'bug 's life'. Everything is covered. The film expects(looks) great and the animation is sometimes jaw dropping. The film isn't 't too terribly original. It is basically modern take on kurosawa 's seven samurai only with bugs, I enjoyed the character interaction however, and the naughty boys(bad guys) in this film actually seemed bad. It seems that Disney usually makes their bad guys carbon copy cut outs, the grasshop- pers are menacing and hopper the lead bad guy was a brilliant creation. Check this one out.
Positive
Negative
Table 7: IMDB Adversarial Examples from PCAA on word-CNN
Class
Perturbed Class
Perturbed Texts
Positive
Negative
I am college student studying A levels and need help and comments from anyone who has any views at all about the theme of mothers in film. In The Mother, whether you have gone through something similar just want to comment and help me research more about this film, any com- ment would much greatly appreciated. The com- ments will be used alone(solely) for exam pur- poses and will be included in my written exam. So if you have any views at all I'm convinced(sure) can put them to use and you could help me get A. am also studying about a boy and tadpole. So if you have seen these films as well, would ap- preciate it if you could leave comments on here on that page. Thank you.
14

---

Under review as a conference paper at ICLR 2023
Negative
Positive
This movic is so horrendous(awful). It is hard to find the right words to describe it. At first the story is so ridiculous. write better plot. The actors are boring and un- talented. Perhaps they were compelled to play in this dorky(cheesy) film. The camera receptions of the national forest are the only good in this whole movie. should feel ashame because paid for this lousy picture. Hopefully nobody makes a sequel or make a similar film with such worse storyline.
narrow minded human can
Positive
Negative
This movie is wonderful, the writing, directing, acting, all are marvelous(fanaastc).. Very witty and clever script quality performances by actors. Ally Sheedy is strong and dynamic and delight- fully quirky really original and heart warmingly unpredicatable. The scenes are alive with fresh energy and really talented generating(produution This may not be war peace but the two academy noms wouldn't have been forthcoming. weren't for the genius of James Wong Howe, this is one of the few films I've fallen in love with as infant(child) and gone back to without dissatis- faction. Whether you have any interest in what it offers fictively I'm not saying it's his best work. I'm no ex- pert there for sure but the look of this movie is (boouddig((mmaznng))
Positive
Negative
it
not, BBC is visual feast.
love everything about it, Elsa Lanchester, the cat, the crazy hoodoo, the retro downtown Ness, but the way it was put on film is breathtaking. I even like the inconsistencies pointed out on this page aforementioned(abova and the special effects that seem backward. Now it all creates really consistent world.
Positive
Negative
Bette Midler is again divine raunchily hilari- ous(humorous) in love with burlesque, capable of bringing you down to tears either with old jokes, with new dresses or merely with old songs, with more power punch than ever. All in all, sung(singing) new ballads power, singing the good old perennial ones such as the rose 'stay with me' and yes even 'wind beneath my wings'. The best way to appreciate the Divine Miss M has al- ways been libe since this the next best thing to it. strongly recommended to all with mix- ture of adult extensive(wide) eyed enchantment and appreciation and a child 's mischievous wish for pushing all boundaries.
15


Reviews:
[{"review_id": "VR9JvZmbK2Z", "reviewer": "Reviewer_j3K7", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper's objective is to efficiently generate adversarial examples for categorical data stronger than prior search-based methods. To tackle this problem, it proposes a Probabilistic Categorical Adversarial Attack (PCAA) that transfers the adversarial attack in the discrete space to an optimization problem in a continuous probabilistic space. Notably, it assumes that the features of adversarial examples follow a categorical distribution and aims to generate adversarial examples with the corresponding distribution that leads to high loss value and are closer to the clean adversarial examples.\n\nFurther, the paper proposes Probabilistic Adversarial Training that utilizes the proposed objective of PCAA for inner maximization in standard adversarial training. The experimental evaluation is conducted on three datasets, where the proposed method achieves a higher success rate with significantly lower running time for higher $\\epsilon$ values.\n", "summary_of_the_review": "The paper tackles a significant problem; however, the usefulness of the proposed attack is unclear. Further, the proposed defense provides limited or no gains compared to prior works and are limited to NLP data, whereas the objective of the paper was to focus on more categorical tasks. The paper also needs improvement in the clarity of results and overall writing. Due to these reasons, I recommend a weak reject for the current version of the paper.", "strength_and_weaknesses": "## Strengths\n \n* The problem of generating adversarial examples and defending against them is essential, which has yet to gain the required attention in the community. The paper generated categorical adversarial examples by estimating their probabilistic distribution, which is simple and interesting.\n* The proposed method is well-written and easy to follow. The time complexity analysis between various attacks is well-described in the paper.\n* The attack experiments are conducted over three diverse datasets showing the efficiency of the proposed method.\n\n---\n\n## Weaknesses\n\n### Originality and quality\n* The practicality and usefulness of the proposed method need to be clarified. In Table 1, GS obtains the highest success rate for most cases; it is unclear why the adversary would prioritize the attack's efficiency at the cost of having a significantly weaker attack. Further, the reported time is in seconds, except for IPS, the computational cost does not seem too expensive to choose PCAA.\n* The paper highlights that PCAA is more efficient for higher budget attacks; however, the choice of higher $ \\epsilon $ needs to be better justified. I'd suggest including the comparison between the generated adversary examples for various attacks to motivate the requirement of robustness against higher epsilon for these datasets. \n* While the paper cites Bao et al. [1] in experiments, it lacks a discussion and comparison with this recent paper that analyses the classifier's vulnerability with categorical inputs via an information-theoretic adversarial risk analysis.\n* The paper needs to provide confidence intervals for any method for attack and defense evaluation; therefore, it is hard to conclude the improvement provided by PADVt in Figure 3, as PAdvT shows similar or worse performance than ASCC. \n\n\n### Clarity \n* The paper is not well-written and requires many changes to improve the readability for the readers:\n* The y-axis in Figure 3 and Figure 4 should be mentioned, which makes it difficult to read the plots. \n* The paper also swaps the X-axis and the multi-row in Figure 1 and Figure 2, making it difficult to read these plots.\n* For both figures, it would be easier to interpret them when represented as tables instead of statuettes.\n* For gradient search SR in LSTM, the SR for 0.32 is lower, is that an exception in that table or was it a typing mistake?\n* Instead of having one paragraph for ablation analysis and one subsection, I suggest combining them into one subsection with multiple paragraphs.\n\n### Reproducibility\nThe code was not released with the submission. I hope the authors can do that in the later version to promote the reproducibility of the attack and defense for future works.\n\n---\n[1] Bao et al. Towards Understanding the Robustness Against Evasion Attack on Categorical Data. ICLR 22\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "QWhekZ8qsdB", "reviewer": "Reviewer_Vnth", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a probabilistic categorical adversarial training method, which conduct adversarial attack and adversarial defense on categorical data. The authors transform the discrete optimization problem on categorical data into a continuous problem, which is then solved using PGD. The proposed method can achieve a better balance between time complexity and performance, and the effectiveness is verified by comparisons on multiple datasets and multiple baselines.", "summary_of_the_review": "This paper extends PGD-AT well to categorical input data, and the main concern is the inability to achieve better performance than the existing technology.", "strength_and_weaknesses": "+Extends PGD based AT to categorical input data.\n\n+Achieve a better balance between time complexity and performance, including categorical adversarial attacks and robustness of adversarial training.\n\n-The technique novelty is somewhat limited, considering that it is only the expansion of PGD based AT on categorical input data.\n\n-The performance is not significant. The results show a better balance between time complexity and adversarial attack or defense robustness. However, compared with the existing technology, the proposed method neither achieves the least time consumption nor achieves the best adversarial attack or defense robustness.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "OcvSnhSGM6", "reviewer": "Reviewer_qsau", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work proposes a new attack for categorical data by probabilistic distribution of non-adversarial samples, named Probabilistic Categorical Adversarial Attack (PCAA). Unlike conventionally expensive attack by greedy search, PCAA can convert the discrete optimization into continuous optimization and apply gradient based approach to generate stronger adversarial samples efficiently. Upon the attack, this work also provides a probabilistic adversarial training method against PCAA and other adversarial attacks.", "summary_of_the_review": "This work is a good work to cooperate theory and application together in the adversarial machine learning area. We should encourage more work like this.", "strength_and_weaknesses": "Strength:\n\n1. This work provides an attack/defense pair instead of attack alone to avoid potential risks on adversarial machine learning research.\n\n2. This work does not only provide PCAA theoretically in epsilon/delta language, but it also provides an efficient algorithm with experimental results.\n\nWeakness:\n\n1. Lack of theoretical explanation why PAdvT is also good to attacks other than PCAA.\n\n2. In algorithm 2, the adversarial training only depends on adversarial samples. What about also considering non-adversarial samples? I mean training with mixture of adversarial and non-adversarial samples.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "DLLAM9nACb", "reviewer": "Reviewer_H8X1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Adversarial attack on discrete (categorical) input is a challenging problem. This paper proposes PCAA, which converts the problem to a continuous optimization problem and solves it with gradient descent. It further shows that using PCAA for adversarial training (PAdvT) can improve the model robustness.\n", "summary_of_the_review": "The methodology has some novelty. However, there are still many issues with the problem formulation and experiments. ", "strength_and_weaknesses": "Strength:\n\n- The idea of using cross-entropy loss and Lagrangian form is novel and interesting. \n\nWeaknesses:\n\n- This paper takes an L0 distance as the only constraint to adversarial examples, however it is insufficient for text classification tasks, and may also be flawed on other tasks. For example, replacing the word “good” with “bad” will significantly change the meaning of a sentence by the L0 distance is just 1. \n- The adversarial probability distribution pi assumes that each feature is independent. However, it is not true in natural language and other types of input sequences. \n- Textual adversarial attack requires human evaluation to ensure that the adversarial sentences do not change the meaning (or the true label). That part is missing in this paper. \n- Attacking a text classifier seems to be an important use case for the proposed method. There are numerous works trying to improve the quality of the adversarial sentences. Those methods are not cited and compared in this paper. \n  - Jin et al., Is bert really robust? a strong baseline for natural language attack on text classification and entailment\n  - Li et al., Bert-attack: Adversarial attack against bert using bert\n  - Li et al., Contextualized perturbation for textual adversarial attack\n  - and many other papers. \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject