From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Reject
metareview:_summary,_strengths_and_weaknesses: The authors propose Probabilistic Categorical Adversarial Attack (PCAA), a method for constructing adversarial $\ell_0$ attacks on categorical data. After discussion, all reviewers unanimously recommended the paper for rejection. 

While the reviewers found the ideas in the paper interesting (e.g., the details of the discrete to continuous relaxation), they raised two main issues. First, it is unclear if $\ell_0$ attacks are meaningful in the domains studied in the paper (cybersecurity, text, and gene splicing), since small modifications can change the meaning of the input. Second, the empirical improvement in performance was unconvincing, as it did not seem to clearly outperform existing methods; while SOTA results are not a requirement for acceptance, for this paper, the empirical performance of PCAA is its key contribution. Reviewers also suggested comparisons to several related papers as well as improvements to the clarity of writing.
justification_for_why_not_higher_score: Originally, 3 reviewers voted to reject and 1 voted to accept, but after discussion, all 4 reviewers voted to reject. The reviewers felt that it wasn't obvious that the problem was well motivated ($\ell_0$ attacks are not necessarily semantics-preserving in all of the datasets the authors studied), and they also felt that the empirical performance that the authors reported was not compelling.
justification_for_why_not_lower_score: N/A

From: ICLR.cc/2023/Conference/Paper5443/Reviewer_H8X1
title: Thank you for clarifying the scope of the paper. 
comment: I read the authors response and adjust my score accordingly. Thank you for clarifying the scope of the paper. In general categorical classifiers, the proposed method shows some novelty. 

However, one of my concern is not fully addressed. As authors mentioned, the adversarial attack "is allowed to perturb features separately". In the example "He likes eating apple." Once substituting "He" with "I", "likes" needs to be replaced with "like". Maintaining the grammatical is a challenge in textual adversarial attack. Similar issue may happen in other sequential classification tasks. 

Therefore I'm still leaning towards rejection.

From: ICLR.cc/2023/Conference/Paper5443/Authors
title: A kindly reminder
comment: The deadline for open discussion is approaching. We have addressed every concern carefully and revised our paper. Therefore, we require reviewers to respond and give more comments.

Any ignorance of our rebuttal is irresponsible, and against the original intention of OpenReview to encourage discussions. We sincerely hope the reviewers and meta-reviewers to involve in the discussions.

From: ICLR.cc/2023/Conference/Paper5443/Authors
title: A friendly reminder
comment: We are grateful for the valuable comments provided by the reviewers. We have carefully addressed all the concerns and exhausted all our effort to revise our work during the rebuttal stage. Therefore we require the reviewers to respond and give further comments.

Now the deadline for open discussion is around the corner. We hope the reviewers can notice our effort and their responsibilities. We are eager to have more interaction with them.

From: ICLR.cc/2023/Conference/Paper5443/Authors
title: Summary of the major revision
comment: We thank the reviewers for the thorough and detailed reviews of our submission. We summarize the major changes that we have made below. All changes are marked in blue in the updated submission.

* We add $95\%$ confidence intervals in Table 1(Page 7), where each experiment is run 5 times.

* We put the tables of results for Figures 2 and 3 (Page 8) in appendix A.1 Table 3 and 4, along with $95\%$ confidence intervals for clearer reference.

* We add more explanations to the Figures 2 3 and 4 (Page 8,9) to make it more clear to read.
    \item We modify Section 5.3 (Page 9) which is the ablation study. Now it contains an ablation study for both attack(PCAA) and defense(PAdvT).

* We add experiment results of PAdvT with a mixture of clean and adversarial examples in Appendix A.1 Table 5 to further show the advantage of our method.

* We add text visualization in Appendix A.2 to show that when applying our methods to the NLP task, the semantic meaning is preserved.

From: ICLR.cc/2023/Conference/Paper5443/Authors
title: Response to the Reviewer qsau
comment: We thank the reviewer for the comments and suggestions. We address concerns one by one. 

**Q1: More explanation for good performance of PAdvT.**

**A1:** First, as shown in Section 5.3, we observe some **concentration properties** of adversarial distributions, which indicates that one can sample one adversarial example with high probability, and we believe that this one is a strong one and can be close to the strongest adversary within the budget. Then when we retrain the model on this adversarial example, the robustness of the model is much improved and can defend against some attacks with suboptimal adversaries. Second, sampling from the adversarial distribution allows us to obtain **diverse examples** and explore more about the adversarial space rather than focusing on a few adversarial examples obtained by search-based methods. In this case, the retraining process takes in diverse adversarial examples and has the potential to defend against different kinds of attacks.

**Q2: Lack of experiments with a mixture of adversarial and non-adversarial samples.**

**A2:** Thanks for the great suggestion. We show some experimental results of PAdvT when training with a mixture of examples in the following table. 

|                    | Clean Err | Genetic | GS    | GGS   | GGA   |
|--------------------|-----------|---------|-------|-------|-------|
| **IMDB LSTM(mix)** | 18.27     | 26.22   | 37.67 | 35.79 | 36.05 |
| **IMDB LSTM**      | 18.57     | 26.01   | 37.30 | 35.60 | 35.46 |
| **IMDB CNN(mix)**  | 18.69     | 28.51   | 36.47 | 34.96 | 35.72 |
| **IMDB CNN**       | 18.85     | 28.35   | 36.36 | 34.85 | 35.47 |

During each epoch, we mix the generated adversarial examples with original inputs and retrain the model on the mixture. When comparing these results with Figure 3 and Table 3, we find that adding clean examples will slightly improve the clean performance and do harm to the robustness. But we claim that this will not hurt much because we will generate more than 20 examples from the distribution. Therefore, mixing with clean examples or not will not affect our performance much. We have added this table in appendix A.1 Table 5, and thanks again for the valuable suggestion.

From: ICLR.cc/2023/Conference/Paper5443/Authors
title: Response to the Reviewer Vnth - Part 2
comment: **Q2: More thorough explanation of the advantage of proposed method.**

**A2:** **Main Advantages of our Attack**. The main advantages of our method can be summarized in two aspects. First, we reach a perfect balance between efficiency and effectiveness while previous methods fail to reach it. Second, our framework can be directly leveraged for the defense such as adversarial training. Next, we detail these advantages.

Compared with baseline methods, our attack can achieve a balance between efficiency and effectiveness in various settings. The existing attacking methods designed for categorical attacks can be characterized into **two types**. One traverses all possible replacements of features such as GS and GGS in our discussion, and the other iteratively replaces each feature to accelerate the process such as GA and GGA.  

* PCAA vs GS, GGS. GS and GGS suffer from **serious time efficiency** issue as shown in Table 1, whose running time increases in an exponential order($\mathcal{O}(nd+d^{\epsilon})$) and becomes impractical when the budget is larger than 3 for large dimensional datasets(AG, IPS). This limits the application of these methods though they achieve the highest performance when applicable. However, our proposed PCAA does not have this concern. PCAA improves time efficiency to a huge extent and sacrifices a little bit of effectiveness. We highlight that PCAA can achieve the highest performance second only to GS, which makes it a perfect solution to the time efficiency issue.

* PCAA vs GA, GGA. GA and GGA can accelerate the search process but the cost is huge that they have **very poor performance**. As shown in Table 1, their performance is significantly weaker than others. PCAA does not have this concern and has a performance second only to GS. 

Therefore, neither of the existing 2 types can satisfy the requirement of both effectiveness and efficiency, and this is why PCAA is a better choice. 

In terms of defense, existing search-based methods fail to be directly leveraged for the defense such as adversarial training. The time efficiency issue for GS and GGS, and the poor effectiveness of GA and GGA prevent them from building robust models. Besides, they can not go with mini-batches which further limits their application for defense. However, since our methods directly transfer from discrete space to continuous space, we can leverage techniques developed for continuous models such as gradient methods and adversarial training.

**References**

[1] Bao et al. Towards understanding the robustness against evasion attack on categorical data. ICLR 2022

[2] Yang et al. Greedy attack and gumbel attack: Generating adversarial examples for discrete data. JMLR 2020

[3] Wang et al. Attackability characterization of adversarial evasion attack on discrete data. SIGKDD 2020

From: ICLR.cc/2023/Conference/Paper5443/Authors
title: Response to the Reviewer Vnth - Part 1
comment: We thank the reviewer for the comments and suggestions. 

However, we would politely point out that the reviewer may overlook our significant contribution in this work. We will first compare our work with the PGD method to illustrate our contributions. Then we address each concern.

**Our method vs PGD**

It is true that our work heritages some components of PGD as a method to generate/defend against adversarial attacks. However, our method is far from simply extending PGD from the algorithmic perspective. Especially, most PGD-based methods can only handle $l_p$ norms like $l_2$, $l_{\infty}$, $l_1$ while the norm $l_0$ is rarely considered. Simply extending existing PGD methods for $l_0$ attack is not straightforward. 

Moreover, the key merit of our work is that: we successfully transfer the discrete problem into continuous space, which will bring a series of advantages for adversarial attacks \& defense.

* **(1)** Before our paper, existing methods [1,2,3] specifically designed for categorical models mainly focus on the search-based methods such as GS and GA included in our discussion, but they either suffer from serious time efficiency issues or poor effectiveness. Therefore, we propose this probabilistic framework to transfer the discrete input problem into a continuous input problem, which allows us to adopt techniques developed for continuous tasks. 

* **(2)** Based on our method, many existing popular defense methods can also be brought to the categorical domain. To better illustrate, we present the experimental results of TRADES with our framework on AG's news dataset in the following table and the values represent errors. According to the results, our framework can be easily leveraged for TRADES and achieves good performance, and can reduce error under different attacks. When the regularizer parameter $1/\lambda$ is small, we can reach a better clean performance while sacrificing some robustness. It is also noticeable that under the same defense(same row), our PCAA method reaches a good attacking effect second only to GS(when it is applicable), further indicating that PCAA maintains effectiveness under different defenses.

|                           | Clean Err | Hotflip | PCAA  | GS    | GGS   | GA    | GGA   |
|---------------------------|-----------|---------|-------|-------|-------|-------|-------|
| **ERM**                   | 8.70      | 79.72   | 80.75 | 83.09 | 79.28 | 74.43 | 67.53 |
| **PAdvT**                 | 14.62     | 45.38   | 49.50 | 50.65 | 47.14 | 44.71 | 42.18 |
| **TRADES**($1/\lambda=1$) | 13.89     | 46.21   | 50.83 | 51.32 | 48.45 | 45.98 | 43.17 |
| **TRADES**($1/\lambda=5$) | 14.31     | 45.57   | 49.76 | 50.89 | 47.66 | 45.12 | 42.31 |

Now, we address the raised concerns one by one.

**Q1: Why our method is not a simple expansion of PGD.**

**A1:** We have made a thorough comparison and provided the details above.

From: ICLR.cc/2023/Conference/Paper5443/Authors
title: Response to reviewer Reviewer j3K7 - Part 2
comment: **Q3: Comparison with paper [1]**

**A3:** The paper [1] mentioned by the reviewer first analyzes the factors affecting the categorical model's robustness. Then it explores the upper and lower bounds of success rate when attacking models by leveraging simplified greedy-search attacking methods (FSGS etc).

Compare to [1], **First**, our work provides an **efficient algorithm** to solve the optimization problem on continuous probability space, while they mention the same optimization problem and claim that it is hard to solve in the discrete space due to the time efficiency issue, which is the reason why they adopt simplified methods to estimate upper and lower bounds. This further stresses the advantages of our methods that we can balance between efficiency and effectiveness better. **Second**, its evaluating methods, such as FSGS and GradAttack, are applications of our considered baselines, GA and GGA, which are already included in our discussion and experiments. **Third**, our experimental results are **consistent** with their evaluation for the upper and lower bound of model robustness. Take the IPS dataset as an example, as shown in Table 1 in paper [1] and Table 1 in our paper, their FSGS reaches a lower bound of 0.33 and upper bound of 0.8 for a success rate at the budget of 1.2 while our GA achieves success rate 0.415 at budget 1 which falls in their interval. Similarly, their GradAttack reaches a success rate of 0.59 at budget 1.8 while our GGA achieves 0.637 at budget 2. However, its estimates are **loose** because its methods **sacrifice effectiveness** when improving efficiency. In this case, our method has the potential to be combined with their strategy to evaluate the models’ robustness better.

**Q4: Remedy for lack of confidence intervals.**

**A4:** We have added confidence intervals in appendix A.1 Table 3 and 4, and also added intervals to Table 1.

**Clarifity**
Thank you for pointing out some issues of clarity, we have fixed them.

* Y-axis in Figure 3 represents the error rate. Y-axis in Figure 4 represents the unnormalized probabilities of each level for one feature. We have added these explanations to the paper and highlighted them.

* The X-axis in Figures 2 and 3 represents different defense methods, and the multi-row represents different attacking methods. We have also added these illustrations to the paper and highlighted them.

* We have put tables of results corresponding to Figures 2 and 3 in appendix A.1 Table 3 and 4.

* It is a typo and has already been fixed.

* We have taken your suggestion and modified section 5.3.

**References**

[1] Bao et al. Towards understanding the robustness against evasion attack on categorical data. ICLR 2022

[2] Yang et al. Greedy attack and gumbel attack: Generating adversarial examples for discrete data. JMLR 2020

[3] Wang et al. Attackability characterization of adversarial evasion attack on discrete data. SIGKDD 2020

From: ICLR.cc/2023/Conference/Paper5443/Authors
title: Response to reviewer Reviewer j3K7 - Part 1
comment: We thank the reviewer for the comments and suggestions. We will first clarify the main advantages of our adversarial framework. Then we address each particular concern of the reviewer.

**Main Advantages of our Attack.** Main advantages of our method can be summarized in two aspects. First, we reach a perfect balance between efficiency and effectiveness while previous methods fail to reach it. Second, our framework can be directly leveraged for the defense such as adversarial training. Next, we detail these advantages.

Compared with baseline methods, our attack can achieve a balance between efficiency and effectiveness in various settings. The existing attacking methods [1,2,3] designed for categorical attacks can be characterized into **two types**. One traverses all possible replacements of features such as GS and GGS in our discussion, and the other iteratively replaces each feature to accelerate the process such as GA and GGA.

* PCAA vs GS, GGS. GS and GGS suffer from **serious time efficiency** issue as shown in Table 1, whose running time increases in an exponential order($\mathcal{O}(nd+d^{\epsilon})$) and becomes impractical when the budget is larger than 3 for large dimensional datasets(AG, IPS). This limits the application of these methods though they achieve the highest performance when applicable. However, our proposed PCAA does not have this concern. PCAA improves time efficiency to a huge extent and sacrifices a little bit of effectiveness. We highlight that PCAA can achieve the highest performance second only to GS, which makes it a better solution to the time efficiency issue.

* PCAA vs GA, GGA. GA and GGA can accelerate the search process but the cost is huge that they have **very poor performance**. As shown in Table 1, their performance is significantly weaker than others. PCAA does not have this concern and has the best performance second only to GS.

In terms of **defense**, existing search-based methods fail to be directly leveraged for the defense methods such as adversarial training. It is because:

* The time efficiency of GS and GGS makes them impractical for training robust models; while GA and GGA suffer from poor effectiveness which leads to poor robustness.

* Moreover, since all these methods involve search-based methods, during the training process, each adversarial training sample must be generated individually. In another word, they cannot go with mini-batches in adversarial training. However, since our framework directly transfers from discrete space to continuous space, we can leverage techniques developed for continuous models such as gradient methods and adversarial training.

Next, we address each concern respectively.

**Q1: Why PCAA is useful and practical?**

**A1:** PCAA reaches a better balance between efficiency and effectiveness and can be directly leveraged for defense. We have provided details and more evidence in the illustration of the advantages above. 

**Q2: Why a large budget $\epsilon$ is needed?**

**A2:** Thank you for the comment. In many real-world scenarios, data is often high-dimensional, i.e., there are many features. Thus, we need a large budget $\epsilon$ which indicates the maximum number of features to perturb to make the attack effective. **Although $\epsilon$ is large, the ratio of perturbed features is small for high-dimensional data.**  Moreover, in order to **comprehensively evaluate the robustness** of normal and defense methods, large budgets are also needed. For example, by checking the experimental results, we found that on some datasets(AG, Splice) even when the budget size is as large as 4 or 5, we can not reach a success rate of 100\% attacking normal models. This indicates that we still need larger $\epsilon$ to reach a larger attacking effect. Besides, when we train robust models via adversarial training, as shown in Figures 2 and 3, we adopt a large budget to guarantee the defense effect. Moreover, when evaluating the robustness of these defense methods, we need to consider even larger budgets.

From: ICLR.cc/2023/Conference/Paper5443/Authors
title: Response to Reviewer H8X1 - Part 2
comment: **Q2. Will the attack in text classification change the original meaning?**

**A2:** Our attack method can also preserve the original semantic meaning. In our studied dataset, such as IMDb Review, the attacker is only allowed to perturb the sentences by switching some words with their **synonyms**. For example, the word ‘good’ will be replaced with words in its synonym set \{great, fine, satisfactory, excellent, superb, outstanding, wonderful, awesome, fantastic, brilliant,...\} rather than its opposite 'bad'. As a result, the original meaning will not be changed after adversarial perturbation. In appendix A.2 Tables 5 and 6, we provide more samplers for the perturbed sentences in IMDb Review dataset, to further demonstrate our attack will not hurt the original meaning. For other datasets such as IPS and Splice, it is not necessary to consider semantic meaning preserving. For the AG's News dataset, because we only consider the character level perturbation, the semantic meaning can be naturally preserved. 

**Q3: Does our attack assume the features are independent?**

**A3:** In this paper, we never claim that we assume the input features are independent. As we show in the paper, IPS and IMDB are all sequential datasets, and we study LSTM as the victim model. Under these datasets, our method can handle sequential data to successfully find adversarial examples. It suggests that our method does not rely on any assumption about the independence between features.

The fact is, we assume that the process to generate **adversarial perturbation** for different features of a given sample is independent (refer Eq.(3) in the paper). It is different from ''features are independent'' in the victim model. We claim that it is reasonable to assume that adversarial distributions are independent, which actually indicates that one is allowed to perturb features separately. For example, when attacking the sentence 'He likes eating apples', attackers are allowed to replace 'likes' with any word in its synonym set \{loves, adores,...\} and replace 'eating' within its synonym set \{having, chewing,...\} independently. By adding proper constraints following the way in Eq.(3) and Eq.(4), we can constrain the number of perturbed features is limited.

**Q4:More discussions on other papers that the reviewer mentioned.**

**A4:** Thank you for pointing out these important works of NLP attacks. Generally speaking, these suggested methods~[7,8,9] are specially designed for NLP tasks, and they are applications of some general attacking methods that have been included in our discussion and experiments.

* First, the suggested methods in [7,8] both consist of 2 steps. They first determine the set of features to perturb and then iteratively perturb each selected feature to search for effective adversaries. They either use gradients or prediction scores to guide the search for perturbed features and iteratively change each feature, so they can be considered as applications of GA and GGA, included in our discussion and experiments.

* Second, the method in [9] takes three actions into consideration but still adopts an iterative way during the searching process, and it is an application of greedy-search-based method when using properly chosen replacement sets.

In general, these methods can be represented as the GA and GGA methods we studied for general categorical attacks. Our method has significant differences from theirs. Moreover, our general method can be easily adapted for NLP tasks by taking the synonym set as the candidate set to satisfy the additional condition of preserving semantic similarity. 

**References**

[1] Yang et al. Greedy attack and gumbel attack: Generating adversarial examples for discrete data. JMLR 2020

[2] Bao et al. Towards understanding the robustness against evasion attack on categorical data. ICLR 2022

[3] Ebrahimi et al. Hotflip: White-box adversarial examples for text classification.

[4] Sato et al. Interpretable adversarial perturbation in input embedding space for text.

[5] Gong et al. Adversarial texts with gradient methods.

[6] Wang et al. Attackability characterization of adversarial evasion attack on discrete data. SIGKDD 2020


[7] Jin et al. Is BERT really robust? A strong baseline for natural language attack on text classification and entailment. 

[8] Li et al. BERT-ATTACK: adversarial attack against BERT using BERT. EMNLP 2020

[9] Li et al. Contextualized perturbation for textual adversarial attack.

From: ICLR.cc/2023/Conference/Paper5443/Authors
title: Response to Reviewer H8X1 - Part 1
comment: We thank the reviewer for all comments and suggestions. We agree with the reviewer that if our focus was on NLP tasks, other factors, such as preserving semantic meaning should be taken into consideration. **However, we would politely point out that there are severe misunderstandings of the reviewer on the main scope and contribution of this paper.** 

**Our Scope and Contribution.**

Following existing studies [1,2], our main scope is the adversarial attack for general categorical data and not limited to one specific area such as NLP. Correspondingly our paper is to propose a general framework for attacking models with categorical input, whose applications are not limited to NLP areas. As mentioned in the introduction part, many ML applications such as intrusion detection systems, financial transaction systems and biomedicine are representative tasks whose features are categorical. For example, in one dataset Intrusion Prevention System (IPS), we studied in our paper, each data sample contains records for different types of system operations, which are categorical.

**Comparison with NLP attacks.**

More importantly, both the **problem settings** and **methodologies** to do attack in tasks for general categorical data are different from those in the NLP area. 

**In terms of the problem setting,** attacks in the NLP area are usually required to preserve semantic similarity and syntactic similarity while these are not applicable for tasks in other areas such as financial systems where different types of transactions are treated as the same. 

**In terms of the methodologies**, most existing attacking methods in NLP follow two major types of methods:
* One kind of method is trying to solve a discrete optimization problem in the discrete textual space [3] via greedy search. For example, they search the optimal combination of word substitutions for the given sentence which can result in the maximized prediction loss. These methods can be represented by GS Attack and GA Attack which is discussed in our paper. 
* Another kind of method conducts attacks on word embedding space [4,5]. Once they find the adversarial perturbations in the embedding space, they search over the vocabulary to find word substitutions that have the closest distance to the found adversarial example in the embedding space. However, for general categorical data, these methods can not work. It is because the specific assumption in NLP that ''close distance in embedding space means closer semantic meaning'' does not always hold in representation learning for general ML tasks. 

In conclusion, **existing attacking methods in NLP cannot be directly adopted in other categorical data. However, our proposed method can be easily applied to NLP areas.** For example, in the IMDB dataset studied in our paper, we only allow the attacker to perturb a sentence, by replacing the words with their synonyms. In this way, we can also successfully fulfill the goal of adversarial attack in NLP tasks and preserve the semantic meaning. 

In fact, our paper follows the setting of many existing papers in categorical attacks, such as [1,2,6]. These papers also discuss the general categorical attack rather than being limited to one specific area, and our work follows this line.  Compared with these previous papers for the categorical attack, our paper makes significant contributions as (1) we propose an efficient and effective general categorical attacking framework and (2) based on the general attacking framework, many well-established defenses in the continuous domain, such as adversarial training, can be easily adapted to categorical domain.

Next, we will address the reviewer's each specific concern.

**Q1: Why only $l_0$ attack is considered?**

**A1:** For most tasks when the data are categorical, $l_0$ adversarial attack is more intuitive and with board interest. For example, in the transaction system, an attacker aims to attack as few records as possible to make the perturbation insidious; and in Genetics data, only a few nucleotides are mutated. Due to this fact, we study $l_0$ attack in this paper. It is also worth mentioning that many existing works [1,2,6] also only consider $l_0$ attack.

From: ICLR.cc/2023/Conference/Paper5443/Reviewer_H8X1
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: Adversarial attack on discrete (categorical) input is a challenging problem. This paper proposes PCAA, which converts the problem to a continuous optimization problem and solves it with gradient descent. It further shows that using PCAA for adversarial training (PAdvT) can improve the model robustness.

strength_and_weaknesses: Strength:

- The idea of using cross-entropy loss and Lagrangian form is novel and interesting. 

Weaknesses:

- This paper takes an L0 distance as the only constraint to adversarial examples, however it is insufficient for text classification tasks, and may also be flawed on other tasks. For example, replacing the word “good” with “bad” will significantly change the meaning of a sentence by the L0 distance is just 1. 
- The adversarial probability distribution pi assumes that each feature is independent. However, it is not true in natural language and other types of input sequences. 
- Textual adversarial attack requires human evaluation to ensure that the adversarial sentences do not change the meaning (or the true label). That part is missing in this paper. 
- Attacking a text classifier seems to be an important use case for the proposed method. There are numerous works trying to improve the quality of the adversarial sentences. Those methods are not cited and compared in this paper. 
  - Jin et al., Is bert really robust? a strong baseline for natural language attack on text classification and entailment
  - Li et al., Bert-attack: Adversarial attack against bert using bert
  - Li et al., Contextualized perturbation for textual adversarial attack
  - and many other papers. 


clarity,_quality,_novelty_and_reproducibility: The methodology is presented clearly. 

It has some novelty in formulating the attack into an optimization problem. However, there are some issues with the formulation considering practical NLP or sequence classification tasks. There are issues with the experiments as well. 

The source code was not submitted. 

summary_of_the_review: The methodology has some novelty. However, there are still many issues with the problem formulation and experiments. 
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper5443/Reviewer_j3K7
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The paper's objective is to efficiently generate adversarial examples for categorical data stronger than prior search-based methods. To tackle this problem, it proposes a Probabilistic Categorical Adversarial Attack (PCAA) that transfers the adversarial attack in the discrete space to an optimization problem in a continuous probabilistic space. Notably, it assumes that the features of adversarial examples follow a categorical distribution and aims to generate adversarial examples with the corresponding distribution that leads to high loss value and are closer to the clean adversarial examples.

Further, the paper proposes Probabilistic Adversarial Training that utilizes the proposed objective of PCAA for inner maximization in standard adversarial training. The experimental evaluation is conducted on three datasets, where the proposed method achieves a higher success rate with significantly lower running time for higher $\epsilon$ values.

strength_and_weaknesses: ## Strengths
 
* The problem of generating adversarial examples and defending against them is essential, which has yet to gain the required attention in the community. The paper generated categorical adversarial examples by estimating their probabilistic distribution, which is simple and interesting.
* The proposed method is well-written and easy to follow. The time complexity analysis between various attacks is well-described in the paper.
* The attack experiments are conducted over three diverse datasets showing the efficiency of the proposed method.

---

## Weaknesses

### Originality and quality
* The practicality and usefulness of the proposed method need to be clarified. In Table 1, GS obtains the highest success rate for most cases; it is unclear why the adversary would prioritize the attack's efficiency at the cost of having a significantly weaker attack. Further, the reported time is in seconds, except for IPS, the computational cost does not seem too expensive to choose PCAA.
* The paper highlights that PCAA is more efficient for higher budget attacks; however, the choice of higher $ \epsilon $ needs to be better justified. I'd suggest including the comparison between the generated adversary examples for various attacks to motivate the requirement of robustness against higher epsilon for these datasets. 
* While the paper cites Bao et al. [1] in experiments, it lacks a discussion and comparison with this recent paper that analyses the classifier's vulnerability with categorical inputs via an information-theoretic adversarial risk analysis.
* The paper needs to provide confidence intervals for any method for attack and defense evaluation; therefore, it is hard to conclude the improvement provided by PADVt in Figure 3, as PAdvT shows similar or worse performance than ASCC. 


### Clarity 
* The paper is not well-written and requires many changes to improve the readability for the readers:
* The y-axis in Figure 3 and Figure 4 should be mentioned, which makes it difficult to read the plots. 
* The paper also swaps the X-axis and the multi-row in Figure 1 and Figure 2, making it difficult to read these plots.
* For both figures, it would be easier to interpret them when represented as tables instead of statuettes.
* For gradient search SR in LSTM, the SR for 0.32 is lower, is that an exception in that table or was it a typing mistake?
* Instead of having one paragraph for ablation analysis and one subsection, I suggest combining them into one subsection with multiple paragraphs.

### Reproducibility
The code was not released with the submission. I hope the authors can do that in the later version to promote the reproducibility of the attack and defense for future works.

---
[1] Bao et al. Towards Understanding the Robustness Against Evasion Attack on Categorical Data. ICLR 22

clarity,_quality,_novelty_and_reproducibility: The review above elaborates on all these points in detail.
summary_of_the_review: The paper tackles a significant problem; however, the usefulness of the proposed attack is unclear. Further, the proposed defense provides limited or no gains compared to prior works and are limited to NLP data, whereas the objective of the paper was to focus on more categorical tasks. The paper also needs improvement in the clarity of results and overall writing. Due to these reasons, I recommend a weak reject for the current version of the paper.
correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper5443/Reviewer_Vnth
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper proposes a probabilistic categorical adversarial training method, which conduct adversarial attack and adversarial defense on categorical data. The authors transform the discrete optimization problem on categorical data into a continuous problem, which is then solved using PGD. The proposed method can achieve a better balance between time complexity and performance, and the effectiveness is verified by comparisons on multiple datasets and multiple baselines.
strength_and_weaknesses: +Extends PGD based AT to categorical input data.

+Achieve a better balance between time complexity and performance, including categorical adversarial attacks and robustness of adversarial training.

-The technique novelty is somewhat limited, considering that it is only the expansion of PGD based AT on categorical input data.

-The performance is not significant. The results show a better balance between time complexity and adversarial attack or defense robustness. However, compared with the existing technology, the proposed method neither achieves the least time consumption nor achieves the best adversarial attack or defense robustness.

clarity,_quality,_novelty_and_reproducibility: The clarity, quality and reproducibility is ok. The nolvelty is somewhat limited, considering that it is an extension of PGD based AT, and there is no significant performance improvement.
summary_of_the_review: This paper extends PGD-AT well to categorical input data, and the main concern is the inability to achieve better performance than the existing technology.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper5443/Reviewer_qsau
confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This work proposes a new attack for categorical data by probabilistic distribution of non-adversarial samples, named Probabilistic Categorical Adversarial Attack (PCAA). Unlike conventionally expensive attack by greedy search, PCAA can convert the discrete optimization into continuous optimization and apply gradient based approach to generate stronger adversarial samples efficiently. Upon the attack, this work also provides a probabilistic adversarial training method against PCAA and other adversarial attacks.
strength_and_weaknesses: Strength:

1. This work provides an attack/defense pair instead of attack alone to avoid potential risks on adversarial machine learning research.

2. This work does not only provide PCAA theoretically in epsilon/delta language, but it also provides an efficient algorithm with experimental results.

Weakness:

1. Lack of theoretical explanation why PAdvT is also good to attacks other than PCAA.

2. In algorithm 2, the adversarial training only depends on adversarial samples. What about also considering non-adversarial samples? I mean training with mixture of adversarial and non-adversarial samples.
clarity,_quality,_novelty_and_reproducibility: Clarity: All definition, algorithm and writing in this work are clear.

Quality: Both theoretical and applied aspects of attack and defense is good.

Novelty: The attack is novel from probabilistic perspective.
summary_of_the_review: This work is a good work to cooperate theory and application together in the adversarial machine learning area. We should encourage more work like this.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 8: accept, good paper