Venue: NAACL 2021

# Certified Robustness to Word Substitution Attack with

Differential Privacy

 Wenjie Wang, Pengfei Tang, Jian Lou\({}^{*}\) and Li Xiong

Emory University

Atlanta, GA, USA {wang.wenjie, pengfei.tang, jian.lou, lxiong}@emory.edu

###### Abstract

The robustness and security of natural language processing (NLP) models are significantly important in real-world applications. In the context of text classification tasks, adversarial examples can be designed by substituting words with synonyms under certain semantic and syntactic constraints, such that a well-trained model will give a wrong prediction. Therefore, it is crucial to develop techniques to provide a rigorous and provable robustness guarantee against such attacks. In this paper, we propose _WordDP_ to achieve certified robustness against word substitution attacks in text classification via differential privacy (DP). We establish the connection between DP and adversarial robustness for the first time in the text domain and propose a conceptual exponential mechanism-based algorithm to formally achieve the robustness. We further present a practical simulated exponential mechanism that has efficient inference with certified robustness. We not only provide a rigorous analytic derivation of the certified condition but also experimentally compare the utility of _WordDP_ with existing defense algorithms. The results show that _WordDP_ achieves higher accuracy and more than \(30\times\) efficiency improvement over the state-of-the-art certified robustness mechanism in typical text classification tasks.

+
Footnote †: J. Lou is the corresponding author.

## 1 Introduction

Deep neural networks (DNNs) have achieved state-of-the-art performance in many natural language processing (NLP) tasks, such as text classification [25], sentiment analysis [4], and machine translation [7], making the robustness and security of NLP models significantly important. Recent studies have shown that DNNs can be easily fooled by adversarial examples, which are carefully crafted by adding imperceptible perturbations to input examples during inference time [8]. In the context of text classification tasks, adversarial examples can be designed by manipulating the word or characters under certain semantic and syntactic constraints [6, 9, 20, 21]. Among all the attack strategies, word substitution attacks, in which attackers attempt to alter the model output by replacing input words with their synonyms, can maximally maintain the naturalness and semantic similarity of the input. Therefore, in this paper, we consider such word substitution attacks and focus on defending against such attacks. Figure 1 shows an example of the word substitution attack where the clean input text is changed into adversarial text by substituting input words from a synonym list.

Various mechanisms have been developed to defend against adversarial examples in text classification models. Miyato _et al_. miyato applied adversarial training to the text domain that involves adversarial examples in the training stage. Data augmentation in the training phase is another defense approach to improve model robustness. For example, Synonyms Encoding Method (SEM) proposed by wang2019exploring, Dirichlet Neighborhood Ensemble (DNE) proposed by zhou2020exploring, and Robust Encodings (RobEn) proposed by dones2020exploring are different data augmentation methods on either embedding space or word space. How

Figure 1: Word Substitution Attack and Certified Robustness via _WordDP_.



ever, all the above-mentioned works are only evaluated empirically and have no theoretical analysis or guarantee on the robustness of the methods in that they may be broken by other adaptive attacks. Therefore, it is important to provide rigorous and provable certified defense.

There are several attempts to achieve certified robustness for word substitution attacks. Jia _et al._(2019) and Huang _et al._(2019) utilize Interval Bound Propagation (IBP) to compute an upper bound on the model's loss in the forward pass and minimize this bound via backpropagation. Although IBP gives a theoretical bound, it does not provide any certification condition. Another limitation is that it is not applicable to character-level DNNs, because IBP is limited to continuous space so that model input should be the word-level embedding. SAFER Ye et al. (2020) achieves certified robustness with a new randomized smoothing technique. However, its computation of synonym set intersection greatly reduces the computation speed in the inference stage. Besides, SAFER only provides a theoretical certified accuracy and its empirical effectiveness on adversarial examples has not been evaluated.

In this paper, we propose a novel approach _WordDP_ to certified robustness against word substitution attacks in text classification via differential privacy (DP) Dwork (2008). Figure 1 is a high-level illustration. In the inference phase, the input go through a randomized mechanism WordDP. If a clean input satisfies the certification condition of _WordDP_, its adversarial counterpart is guaranteed to predict the same output label. DP is a privacy framework that protects the information of individual record in the database by randomized computations, such that the change of the computation output is bounded when small perturbation is applied on the database. This stable output guarantee is in parallel with the definition of robustness: ensuring that small changes in the input will not result in dramatic shift of its output. The idea of providing robustness certification via DP was originally introduced in PixelDP Lecuyer et al. (2019) which is specifically designed for norm-bounded adversarial examples in the continuous domain for applications like image classification. However, it is challenging to directly apply such an idea against word substitution attack, due to the discrete nature of the text input space. Therefore, in this work, we develop _WordDP_ to achieve the DP and robustness connection in the discrete text space by exploring novel application of the exponential mechanism McSherry and Talwar (2007), conventionally utilized to realize DP for answering discrete queries. To achieve this, we present a conceptual certified robustness algorithm that randomly samples word-substituted sentences according to the probability distribution designated by the exponential mechanism and aggregates their inference result as the final classification for the input.

A fundamental barrier limiting the conceptual algorithm from being applied in practice is that the sampling distribution of the exponential mechanism requires an exhaustive enumeration-based sub-step, which needs to repeat the model inference for every neighboring sentences with word substitutions from the input sentence. To overcome this computational difficulty, we develop a practical _simulated exponential mechanism_ via uniform sampling and re-weighted averaging, which not only lowers the computational overhead but also ensures uncompromising level of certified robustness.

Our contribution can be summarized as follows:

1) We propose _WordDP_ to establish the connection between DP and certified robustness for the first time in text classification domain (Sec.4.1).

2) We leverage conceptual exponential mechanism to achieve _WordDP_ and formally prove an \(L\)-word bounded certified condition for robustness against word substitution attacks (Sec.4.2).

3) We develop a simulated exponential mechanism via uniform sampling and weighted averaging to overcome the computation bottleneck of the conceptual exponential mechanism without compromising the certified robustness guarantee (Sec.4.3).

4) Extensive experiments validate that _WordDP_ outperforms existing defense methods and achieves over 30\(\times\) efficiency improvement in the inference stage than the state-of-the-art certified robustness mechanism (Sec.5).

## 2 Related Work

**Word Substitution Attacks.** Various attacks have been developed to fool DNNs in text classification, including substituting a word with its synonyms Ren et al. (2019); Jin et al. (2019); Zang et al. (2020); Alzantot et al. (2018), manipulating the characters Gao et al. (2018); Ebrahimi et al. (2018), and perturbation on the embedding space Papernot et al. (2016); Liang et al. (2018); Sato et al. (2018); Cheng et al. (2019).

In word substitution attacks, attackers replace 

words in a sentence with their synonyms according to a synonym table, including PWWS (Ren et al., 2019), TEXTFOOLER (Jin et al., 2019), among others (Zang et al., 2020). In particular, PWWS is the most widely used attack algorithm to evaluate defense mechanisms Zhou et al. (2020); Jia et al. (2019); Ye et al. (2020). PWWS uses WordNet to build synonym set and only replaces named entities (NEs) with similar NEs in order to flip the prediction. It incorporates word saliency to determine the replacement order and selects the synonym that can cause the greatest prediction probability change.

**Empirical Defenses to Word Substitution Attacks.** Several existing empirical defenses are effective for adversarial word substitution. Miyato _et al._ (2016) applied adversarial training to the text domain. Wang _et al._ (2019) proposed Synonyms Encoding Method (SEM), which finds a mapping between the words and their synonyms before the input layer. Jones _et al._ (2020) proposed robust encodings (RobEn) that involves an encoding function to map sentences to a smaller, discrete space. Dirichlet Neighborhood Ensemble (DNE) (Zhou et al., 2020) creates virtual sentences by mixing the embedding of the original word with its synonyms' embedding via Dirichlet sampling, which is randomized smoothing based data augmentation.

**Certified Robustness.** Certified robustness has been first studied in image domain, which certifies that a model is robust to adversarial examples when its prediction result is stable when applying small perturbations to the input (Lecuyer et al., 2019; Cohen et al., 2019; Lee et al., 2019). In text domain, Jia _et al._ (2019) and Huang _et al._ (2019) both applied Interval Bound Propagation (IBP) for certification. The intuition is to compute an upper bound on the model's loss through the network in a standard forward pass and minimize this upper bound via backpropagation. One major limitation of IBP certification is that it is not applicable to character-level DNNs, because IBP is limited to continuous space (word-level embedding).

SAFER (Ye et al., 2020) is a certified robust method based on randomized smoothing. The certification is based on the intersection of synonym sets between perturbed examples and clean examples. However, its computation of synonym set intersection greatly reduces the inference efficiency. Besides, it lacks thorough evaluation of empirical effectiveness on adversarial examples.

## 3 Preliminaries

### Adversarial Word Substitution and Certified Robustness

**Adversarial Word Substitution.** Consider a sentence of \(\omega\) words \({\bf X}=(x_{1},x_{2},...,x_{i},...,x_{\omega})\), where each word \(x_{i}\) belongs to a synonym set of \(\kappa(i)\) number of synonyms \({\bf S}(x_{i})=\{x^{1}_{i},x^{2}_{i},...,x^{\kappa(i)}_{i}\}\). Following common practice (Ye et al., 2020), we also assume the synonymous relation is symmetric, such that \(x_{i}\) is in the synonym set of all its synonyms \(x^{2}_{i},...,x^{\kappa(i)}_{i}\) and \({\bf S}(x^{j}_{i})={\bf S}(x^{k}_{i})\) for all \(j,k\in[\kappa(i)]\). The synonym set \({\bf S}(x_{i})\) can be built by following GLOVE (Pennington et al., 2014).

**Definition 3.1**.: (\(L\)-Adversarial Word Substitution Attack) For an input sentence \({\bf X}\), an \(L\)-adversarial word substitution attack perturbs the sentence by selecting at most \(L\) (\(L\leq\omega\)) words \(x_{\tau_{1}},...,x_{\tau_{L}}\) and substitutes each selected word \(x_{\tau_{i}}\) with one of its synonyms \(x^{\prime}_{\tau_{i}}\in{\bf S}(x_{\tau_{i}})\). We denote an attacked sentence by \({\bf X}^{\prime}\) and the set of all possible attacked sentences by \(\mathscr{S}(L)\).

**Certified Robustness.** In general, we say a model is robust to adversarial examples when its prediction result is stable when applying small perturbations to the input.

**Definition 3.2**.: (**Certified Robustness to Word Substitution Attack) Denote a multiclass classification model by \(f({\bf X}):\mathcal{X}\mapsto c\in\mathcal{C}\), where \(c\) is a label in the possible label set \(\mathcal{C}=\{1,...,C\}\). In general, \(f({\bf X})\) outputs a vector of scores \(f^{y}({\bf X})=(f^{y_{1}},...,f^{y_{C}})\in\mathcal{Y}\), where \(\mathcal{Y}=\{{\bf y}:\sum_{i=1}^{C}f^{y_{i}}=1,f^{y_{i}}\in[0,1]\}\), and \(c=\arg\max_{i\in\mathcal{C}}f^{y_{i}}\). A predictive model \(f({\bf X})\) is robust to \(L\)-adversarial word substitution attack on input \({\bf X}\), if for all \({\bf X}^{\prime}\in\mathscr{S}(L)\), it has \(f({\bf X})=f({\bf X}^{\prime})\), which is equivalent to

\[y_{c}({\bf X}^{\prime})>\max_{i\in C::\mu<c}y_{i}({\bf X}^{\prime}). \tag{1}\]

In the following, we refer to the above robustness as \(L\)-certified robustness for short.

### Differential Privacy and Exponential Mechanism

**Differential Privacy.** The concept of DP is to prevent the information leakage of an individual record in the database by introducing randomness into the computation. More specifically, DP guarantees the output of a function over two neighbouring databases are indistinguishable 



**Definition 3.3**: **(Differential Privacy (Dwork et al., 2006))** A randomized mechanism \({\cal A}\) is \(\epsilon\)-differentially private if, for all neighboring datasets \({\bf D}\sim{\bf D^{\prime}}\) that differ in one record or are bounded by certain distance and for all events \({\bf O}\) in the output space \({\cal O}\) of \({\cal A}\), we have

\[\mathbb{P}[{\cal A}({\bf D})\in{\bf O}]\leq e^{\epsilon}\mathbb{ P}[{\cal A}({\bf D^{\prime}})\in{\bf O}]. \tag{2}\]

**Exponential Mechanism.** The exponential mechanism is a commonly utilized DP mechanism in the discrete domain, which consists of the utility score function, sensitivity, and sampling probability distribution as its key ingredients.

**Definition 3.4**: **(Exponential Mechanism (McSherry and Talwar, 2007))** Denote the score function \(u({\bf D},{\bf r}):{\cal D}\times{\cal R}\mapsto\mathbb{R}\), which maps each pair of input dataset \({\bf D}\sim{\cal D}\) and candidate result \({\bf r}\in{\cal R}\) to a real valued score. Denote the sensitivity by \(\Delta_{u}:=\max_{{\bf r}\in{\cal R}}\max_{{\bf D}\sim{\bf D^{\prime}}}|u({\bf D },{\bf r})-u({\bf D^{\prime}},{\bf r})|\). The exponential mechanism \({\cal M}_{E}({\bf D},u,{\cal R})\) selects and outputs an element \({\bf r}\in{\cal R}\) with probability proportional to \(e^{\frac{cu({\bf D},{\bf r})}{2\Delta u}}\). The exponential mechanism is \(\epsilon\)-differentially private.

## 4 Proposed Method

### WordDP for Certified Robustness

_WordDP._ We expand the intuition that DP can be applied to provide certified robustness against textual adversarial examples like word substitution attack by regarding the sentence as a database and each word as a record. If the randomized predictive model satisfies \(\epsilon\)-DP during inference, then the output of a potentially adversarial input \({\bf X^{\prime}}\in{\mathscr S}(L)\) and the output of the original input \({\bf X}\) should be indistinguishable. Thus, our proposed approach is to transform a multiclass classification model's prediction score into a randomized \(\epsilon\)-_WordDP_ score, which is formally defined below.

**Definition 4.1**: **(Word Differential Privacy)** Consider any input sentence \({\bf X}\) and its \(L\)-word substitution sentence set \({\mathscr S}(L)\). For a randomized function \(f_{\mathcal{A}}({\bf X})\), let its prediction score vector be \({\bf y}\in{\cal Y}\). \(f_{\mathcal{A}}({\bf X})\) satisfies \(\epsilon\)-word differential privacy (_WordDP_), if it satisfies \(\epsilon\)-differential privacy for any pair of neighboring sentences \({\bf X}_{1},{\bf X}_{2}\in{\mathscr S}(L)\) and the output space \({\bf y}\in{\cal Y}\).

**Remark 1**: _We stress that WordDP does not seek DP protection for the training dataset as in the conventional privacy area. Instead, it leverages the DP randomness for certified robustness during inference with respect to a testing input._

In practice, for a base model \(f\), a DP mechanism \({\cal A}\) will be introduced to randomize it to \(f_{\mathcal{A}}\). For an \(\epsilon\)-_WordDP_ model \(f_{\mathcal{A}}\), its expected prediction \(\mathbb{E}[f_{\mathcal{A}}({\bf X})]\) is certified robust. Denote the prediction score vector of \(\mathbb{E}[f_{\mathcal{A}}({\bf X})]\) by \(\mathbb{E}[f_{\mathcal{A}}^{y}({\bf X})]=(\mathbb{E}[f_{\mathcal{A}}^{y_{1}}({ \bf X})],...,\mathbb{E}[f_{\mathcal{A}}^{y_{C}}({\bf X})])\in{\cal Y}\). Lemma 4.2 shows \(\mathbb{E}[f_{\mathcal{A}}^{y}({\bf X})]\) satisfies the certified robustness condition in eq.(1), based on Lemma 4.1 that shows each expected prediction score \(\mathbb{E}[f_{\mathcal{A}}^{y_{i}}({\bf X})]\) is stable.

**Lemma 4.1**: _For an \(\epsilon\)-WorldDP model \(f_{\mathcal{A}}\), its prediction score satisfies the relation, \(\forall i\in[C]\),_

\[\mathbb{E}[f_{\mathcal{A}}^{y_{i}}({\bf X}_{1})]\leq e^{\epsilon} \mathbb{E}[f_{\mathcal{A}}^{y_{i}}({\bf X}_{2})],\forall{\bf X}_{1},{\bf X}_{2 }\in{\mathscr L}. \tag{3}\]

From the above property, we can derive the certified robustness condition to adversarial examples.

**Lemma 4.2**: _For an \(\epsilon\)-WordDP model \(f_{\mathcal{A}}\) and an input sentence \({\bf X}\), if there exists a label \(c\) such that:_

\[\mathbb{E}(f_{\mathcal{A}}^{y_{c}}({\bf X}))>e^{2\kappa}\max_{i\neq c}\mathbb{ E}(f_{\mathcal{A}}^{y_{i}}({\bf X})), \tag{4}\]

_then the multiclass classification model \(f_{\mathcal{A}}\) based on the expected label prediction score vector \(\mathbb{E}[f_{\mathcal{A}}^{y}(\cdot)]\) is certified robust to \(L\)-adversary word substitution attack on \({\bf X}\)._

The proofs of the above two lemmas can be adapted from the pixelDP to _WordDP_ context based on _Lemma 1_ and _Proposition 1_ in Lecuyer _et al._ (2019). We relegate the proofs to Appendix A. Our focus is how to design the DP mechanism \({\cal A}\) to achieve _WordDP_ (Subsection 4.2), and how to implement it for efficient inference that still ensures certified robustness (Subsection 4.3).

### WordDP with Exponential Mechanism

In this subsection, we present the conceptual exponential mechanism-based algorithm to achieve _WordDP_ and the certification procedure.

**Exponential Mechanism for _WordDP._ To obtain the DP classifier \(f_{\mathcal{A}}\) given the base model \(f\), we introduce the exponential mechanism \({\cal M}_{E}\) as the randomization mechanism \({\cal A}\) and define \(f_{\mathcal{A}}:=f({\cal M}_{E})\). Given an input example, the mechanism selects and outputs \(L\)-substitution sentences with a probability based on exponential mechanism. It then aggregates the inferences of these samples by an average as the estimated prediction of the input. Figure 2 illustrates the algorithm.

**Definition 4.2**: **(Exponential Mechanism for _WordDP_ and \(L\)-Certified Robustness)** Given the base model \(f\), for any input sentence \({\bf X}\) and potential \(L\)-substitution sentence set \({\mathscr S}(L)\), we define the utility score function as:

\[u({\mathscr S}(L),{\bf X^{\prime}})=e^{-||f^{\mathbf{y}}({\bf X^{\prime}})-f^{\mathbf{y }}({\bf X})||_{1}}, \tag{5}\]

which associates a utility score to a candidate output \({\bf X}^{\prime}\in\mathscr{S}(L)\). The sensitivity of the utility score is \(\Delta_{u}=1-e^{-1}\). Then, the exponential mechanism selects and outputs \({\bf X}^{\prime}\) with probability \(\mathbb{P}_{{\bf X}^{\prime}}\)

\[\mathbb{P}_{{\bf X}^{\prime}}=\frac{1}{\rho}\exp(\frac{\epsilon\cdot u(\mathscr{ S}(L),{\bf X}^{\prime})}{2\Delta u}), \tag{6}\]

where \(\rho=\sum_{i=1}^{|\mathscr{S}({\bf X},L)|}\exp(\frac{\epsilon\cdot u(\mathscr{ S}(L),{\bf X}^{\prime}_{i})}{2\Delta u})\) is the normalization factor.

**Proposition 4.1**.: _The exponential mechanism \(\mathcal{M}(E)\) satisfies \(\epsilon\)-DP. The composition model function \(f_{\mathcal{M}_{E}}({\bf X}):=f(\mathcal{M}_{E}(X))\) is \(\epsilon\)-DP and its prediction score vector \(\mathbb{E}[f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X})]\)-based classification is certified robust to \(L\)-adversary word substitution attack on \({\bf X}\)._

Proof.: To show \(\mathcal{M}_{E}\) is \(\epsilon\)-DP, we prove the sensitivity of the utility score (maximum difference between the utility scores given any two neighboring input) \(\Delta_{u}\) is indeed \(1-e^{-1}\) and the remaining follows the definition of the exponential mechanism (c.f.Definition 3.4). Since \(\lVert f^{\mathbf{y}}({\bf X}^{\prime}_{i})-f^{\mathbf{y}}({\bf X})\rVert_{1}\) is the prediction probability change which is in \([0,1]\), we have \(u(\mathscr{S}(L),{\bf X}^{\prime}_{i})\in[e^{-1},1]\), which leads to \(\Delta_{u}=1-e^{-1}\). Next, since \(\mathcal{M}_{E}({\bf X})\) is \(\epsilon\)-DP, by the post-processing property (i.e., any computation on the output of the DP mechanism remains DP, Proposition 2.1 in (Dwork et al., 2014).), \(f_{\mathcal{M}_{E}}({\bf X})\) is also \(\epsilon\)-DP. Subsequently, by Lemma 4.2, \(\mathbb{E}[f_{\mathcal{M}_{E}}({\bf X})]\) is \(L\)-certified robust on \({\bf X}\). 

**Remark 2**.: 1) The design of the utility function has the intuition that we wish to assign higher probability to sentences that have minimal impact on the prediction score function. 2) The privacy budget \(\epsilon\) influences whether the sampling probability distribution is flat (lower \(\epsilon\)) or peaky (greater \(\epsilon\)). Too small of an \(\epsilon\) value will clearly affect the prediction accuracy. For certification purpose, according to the certified condition Lemma 4.2, too large of an \(\epsilon\) value will result in none certified, so \(\epsilon\) can only be searched within a limited range.

**Certification Condition.** It is a common practice in certified robustness literature to estimate \(\mathbb{E}[f^{\mathbf{Y}}_{\mathcal{M}_{E}}({\bf X})]\) via Monte Carlo estimation Lecuyer et al. (2019); Cohen et al. (2019) in the form of \(\widehat{\mathbb{E}}[f^{\mathbf{Y}}_{\mathcal{M}_{E}}({\bf X})]\). That is, we repeat the exponential mechanism-based inference to draw \(n\) samples of \(f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X}^{\prime}_{r})\), for \(\tau\in[n]\) and let \(\widehat{\mathbb{E}}[f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X})]=\frac{1}{n} \sum_{\tau=1}^{n}f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X}^{\prime}_{\tau})\). The estimation error between \(\widehat{\mathbb{E}}[f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X})]\) and \(\mathbb{E}[f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X})]\) can be bounded based on Hoeffding's inequality with probability \(\eta\), which guarantees that \(\widehat{\mathbb{E}}[f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X})]\in[\mathbb{E} [f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X})]-\sqrt{\frac{1}{2n}ln(\frac{2C}{1- \eta})},\mathbb{E}[f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X})]+\sqrt{\frac{1}{2 n}ln(\frac{2C}{1-\eta})}]:=[\widehat{\mathbb{E}}^{lb}[f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X})], \widehat{\mathbb{E}}^{ub}[f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X})]]\). The next proposition shows that the inference based on the estimated \(\widehat{\mathbb{E}}[f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X})]\) (as versus \(\mathbb{E}[f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X})]\)) can still ensure certified robustness.

**Proposition 4.2**.: _Under the same condition with Proposition 4.1, if there exists a label \(c\) such that_

\[\widehat{\mathbb{E}}^{lb}[f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X})]>e^{z_{c}} \max_{i\neq c}\widehat{\mathbb{E}}^{ub}[f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X })], \tag{7}\]

_the prediction score vector \(\widehat{\mathbb{E}}[f^{\mathbf{y}}_{\mathcal{M}_{E}}({\bf X})]\)-based classification is certified robust with probability \(\eta\) to L-adversary word substitution attack on \({\bf X}\)._

### Simulated Exponential Mechanism

**Simulated Exponential Mechanism.** The conceptual exponential mechanism in Definition 4.2 is computationally impractical. The bottleneck is the need to enumerate the entire \(\mathscr{S}(L)\) in order to calculate the probability distribution of \(\mathbb{P}_{{\bf X}^{\prime}}\) for each \({\bf X}^{\prime}\in\mathscr{S}(L)\) and the normalization factor \(\rho\), which essentially requires us to perform inference for \(\mathscr{S}(L)\gg n\) times (\(n\) is the number of samples) for certifying a single input sentence \({\bf X}\).

In the following, we show that we can significantly reduce the computation cost by sampling via a simulated exponential mechanism, which suffices to sample \(n\) candidate \(L-\)substitution sentences

Figure 2: WordDP with Exponential Mechanism.

 

and calculate only \(n\) times, i.e., the same repetitions as the Monte Carlo estimation. The key insight is based on the different purpose of applying the exponential mechanism between the conventional scenario for achieving DP and our certified robustness scenario. For the former, in order to ensure DP of the final output \(f_{\mathcal{M}_{E}}(\mathbf{X}^{\prime}_{\tau})\), the intermediate \(\mathbf{X}^{\prime}_{\tau}\) is forced to satisfy DP, i.e., drawn from the exact probability distribution designated by the exponential mechanism. For the latter, while the derivation of the certified robustness relied on the randomness of DP and the exponential mechanism, we do not actually require the DP of the intermediate \(\mathbf{X}^{\prime}_{\tau}\). As a result, it allows us to sample \(\mathbf{X}^{\prime}_{\tau}\) from other simpler distributions without calculating the probability distribution of the exponential mechanism, as long as the alternative approach can obtain the equivalent \(\widehat{\mathbb{E}}[f^{Y}_{\mathcal{A}}(\mathbf{X})]\) for robustness certification.

We develop a simulated exponential mechanism via _uniform sampling and re-weighted average prediction score calculation_. Figure 2 shows the simulated mechanism in contrast to the conceptual mechanism. In detail, we sample from \(\mathscr{S}(L)\) with uniform probability, which can be efficiently implemented without generating \(\mathscr{S}(L)\). Denoting a sample by \(\mathbf{X}^{\prime}_{\tau}\), we calculate its scaled exponential mechanism probability by

\[\bar{\mathbb{P}}_{\mathbf{X}^{\prime}_{\tau}=\exp(\frac{\epsilon\cdot u( \mathscr{S}(L),\mathbf{X}^{\prime}_{\tau})}{2\Delta u})}, \tag{8}\]

which can be obtained via a single inference on \(\mathbf{X}^{\prime}_{\tau}\) and the inference on \(\mathbf{X}\) due to the omission of the normalization factor \(\rho\) that requires the entire \(\mathscr{S}(L)\). The inference on \(\mathbf{X}\) only needs to be computed once and shared by all \(n\) Monte Carlo repetitions. Such uniform sampling and scaled probability calculation is repeated for \(n\) times, which requires only \(n+1\) inferences. Finally, we use the following re-weighted average prediction score (weighted by the scaled exponential mechanism probability) for certified robust prediction,

\[\mathbb{E}[f^{Y}_{\mathcal{M}_{E}}(\mathbf{X})]=\sum_{\tau=1}^{ n}\bar{\mathbb{P}}_{\mathbf{X}^{\prime}_{\tau}}\cdot f^{Y}_{\mathcal{M}_{ E}}(\mathbf{X}^{\prime}_{\tau}). \tag{9}\]

The following theorem shows that \(\mathbb{E}[f^{Y}_{\mathcal{M}_{E}}(\mathbf{X})]\)-based prediction guarantees certified robustnessand the conceptual exponential mechanism-based inference in Proposition 4.2 is certified robust provided \(\mathbb{E}[f^{Y}_{\mathcal{M}_{E}}(\mathbf{X})]\) is so.

**Theorem 4.1**.: _For any input \(\mathbf{X}\), let \(\overline{\mathbb{E}}[f^{Y}_{\mathcal{M}_{E}}(\mathbf{X})]\) be calculated by eq.(9). Denote \(\overline{\mathbb{E}}^{lb}[f^{Y}_{\mathcal{M}_{E}}(\mathbf{X})]\) and \(\overline{\mathbb{E}}^{ub}[f^{Y}_{\mathcal{M}_{E}}(\mathbf{X})]\) be \(\eta\)-confidence lower and upper bounds, respectively, i.e., \(\mathbb{E}^{lb}[f^{Y}_{\mathcal{M}_{E}}(\mathbf{X})]=\mathbb{E}[f^{Y}_{ \mathcal{M}_{E}}(\mathbf{X})]-\sqrt{\frac{1}{2n}ln(\frac{2C}{1-\eta})}\) and \(\mathbb{E}^{ub}[f^{Y}_{\mathcal{M}_{E}}(\mathbf{X})]=\mathbb{E}[f^{Y}_{ \mathcal{M}_{E}}(\mathbf{X})]+\sqrt{\frac{1}{2n}ln(\frac{2C}{1-\eta})}\). If there exists a label \(c\) such that_

\[\overline{\mathbb{E}}^{lb}[f^{y_{c}}_{\mathcal{M}_{E}}(\mathbf{ X})]>e^{2\epsilon}\max_{i\neq c}\overline{\mathbb{E}}^{ub}[f^{y_{i}}_{\mathcal{ M}_{E}}(\mathbf{X})], \tag{10}\]

_the prediction score vector \(\overline{\mathbb{E}}[f^{Y}_{\mathcal{M}_{E}}(\mathbf{X})]\)-based classification is certified robust with probability \(\eta\) to \(L\)-adversary word substitution attack on \(\mathbf{X}\)._

The proof of Theorem 4.1 requires the following lemma, which is adapted from Lemma 4.1 from the accurate expectation of \(\mathbb{E}[f^{Y}_{\mathcal{M}_{E}}(\mathbf{X})]\) to the simulated expectation \(\overline{\mathbb{E}}[f^{Y}_{\mathcal{M}_{E}}(\mathbf{X})]\). We stress that during both proofs, we do not use the DP property of \(\overline{\mathbb{E}}[f^{Y}_{\mathcal{M}_{E}}(\cdot)]\), but only its equivalent relation to \(\widehat{\mathbb{E}}[f^{y_{i}}_{\mathcal{M}_{E}}(\cdot)]\).

**Lemma 4.3**.: _For any label \(i\in[C]\) and any \(\mathbf{X}_{1},\mathbf{X}_{2}\in\mathscr{S}(L)\), let \(\overline{\mathbb{E}}[f^{Y}_{\mathcal{M}_{E}}(\mathbf{X})]\) be computed by eq.(9). Then, we have_

\[\mathbb{E}[f^{y_{i}}_{\mathcal{M}_{E}}(\mathbf{X}_{1})]\leq e^{ \epsilon}\mathbb{E}[f^{y_{i}}_{\mathcal{M}_{E}}(\mathbf{X}_{2})]. \tag{11}\]

Proof.: First, we notice that or any \(X^{\prime}\in\mathscr{S}(L)\), it has \(\overline{\mathbb{E}}[f^{y_{i}}_{\mathcal{M}_{E}}(\mathbf{X}^{\prime})]= \frac{\rho}{|\mathscr{S}(L)|}\widehat{\mathbb{E}}[f^{y_{i}}_{\mathcal{M}_{E}}( \mathbf{X}^{\prime})]\) by \(\overline{\mathbb{P}}[\mathbf{X}^{\prime}]=\rho\mathbb{P}[\mathbf{X}^{\prime}]\) and the uniform sampling probability \(\frac{1}{|\mathscr{S}(L)|}\). Second, since \(\widehat{\mathbb{E}}[f^{y_{i}}_{\mathcal{M}_{E}}(\mathbf{X}^{\prime})]\) is \(\epsilon\)-_WordDP_ , we can show that it satisfies Lemma 4.1 by switching \(\mathbb{E}[f^{y_{i}}_{\mathcal{M}_{E}}(\cdot)]\) there to \(\widehat{\mathbb{E}}[f^{y_{i}}_{\mathcal{M}_{E}}(\cdot)]\) here. It follows that:

\[\mathbb{E}[f^{y_{i}}_{\mathcal{M}_{E}}(\mathbf{X}^{\prime})]= \widehat{\mathbb{E}}[f^{y_{i}}_{\mathcal{M}_{E}}(\mathbf{X})]\cdot(\frac{\rho}{| \mathscr{S}(L)|})\] \[\leq e^{\epsilon}\widetilde{\mathbb{E}}[f^{y_{i}}_{\mathcal{M}_{E} }(\mathbf{X}_{2})]\cdot(\frac{\rho}{|\mathscr{S}(L)|})=e^{\epsilon}\mathbb{E}[ f^{y_{i}}_{\mathcal{M}_{E}}(\mathbf{X}_{2})],\]

which proves the lemma. 

Proof.: **(Proof of Theorem 4.1)** For any \(\mathbf{X}^{\prime}\in\mathscr{S}(L)\), by eq.(11), we have

\[e^{\epsilon}\mathbb{E}[f^{y_{c}}_{\mathcal{M}_{E}}(\mathbf{X}^{ \prime})]\geq\mathbb{E}[f^{y_{c}}_{\mathcal{M}_{E}}(\mathbf{X})]\\ >\overline{\mathbb{E}}[f^{y_{u}}_{\mathcal{M}_{E}}(\mathbf{X})]- \sqrt{\frac{1}{2n}ln(\frac{2C}{1-\eta})}=\overline{\mathbb{E}}^{lb}[f^{y_{c}}_{ \mathcal{M}_{E}}(\mathbf{X})];\]

as well as

\[\overline{\mathbb{E}}[f^{y_{i}}_{\mathcal{M}_{E}}(\mathbf{X}^{ \prime})]\leq e^{\epsilon}\mathbb{E}[f^{y_{i}}_{\mathcal{M}_{E}}(\mathbf{X})] \leq e^{\epsilon}\max_{i\neq c}\overline{\mathbb{E}}[f^{y_{i}}_{\mathcal{M}_{E}}( \mathbf{X})]\\ \leq e^{\epsilon}\max_{i\neq c}(\overline{\mathbb{E}}[f^{y_{i}}_{ \mathcal{M}_{E}}(\mathbf{X})]+\sqrt{\frac{1}{2n}ln(\frac{2C}{1-\eta})})\\ =e^{\epsilon}\max_{i\neq c}\overline{\mathbb{E}}^{ub}[f^{y_{i}}_{ \mathcal{M}_{E}}(\mathbf{X})].\]

Equipped with the above two relations, we can prove the claim in Theorem 4.1. We show that \(\overline{\mathbb{E}}[f^{y_{i}}_{\mathcal{M}_{E}}(\mathbf{X})]\) is certified robust for any \(\mathbf{X}^{\prime}\in\mathscr{S}(L)\), as follows,

\[\mathbb{E}[f^{y_{c}}_{\mathcal{M}_{E}}(\mathbf{X}^{\prime})]> \overline{\mathbb{E}}^{lb}[f^{y_{c}}_{\mathcal{M}_{E}}(\mathbf{X})]/e^{\epsilon}\] (12) 

which is \(\mathbb{E}[f^{y_{c}}_{\mathcal{M}_{E}}(\mathbf{X}^{\prime})]>e^{2\epsilon}\max_{i \neq c}\mathbb{E}[f^{y_{i}}_{\mathcal{M}_{E}}(\mathbf{X})].\) For completeness, we can also show that the certified robustness of \(\mathbb{E}[f^{Y}_{\mathcal{A}}(\mathbf{X})]\) implies the certified robustness of \(\widehat{\mathbb{E}}[f^{Y}_{\mathcal{A}}(\mathbf{X})]\):

\[\mathbb{E}[f^{y_{c}}_{\mathcal{M}_{E}}(\mathbf{X}^{\prime})]=( \frac{|\mathscr{S}(L)|}{\rho})\cdot\mathbb{E}[f^{y_{c}}_{\mathcal{M}_{E}}( \mathbf{X}^{\prime})]\] \[>(\frac{|\mathscr{S}(L)|}{\rho})\mathbb{E}^{b_{1}^{t}}[f^{y_{c}}_ {\mathcal{M}_{E}}(\mathbf{X})]/e^{\epsilon}\] \[>(\frac{|\mathscr{S}(L)|}{\rho})e^{\epsilon}\max_{i\neq c} \overline{\mathbb{E}}^{ab}[f^{y_{i}}_{\mathcal{M}_{E}}(\mathbf{X})]\] \[>(\frac{|\mathscr{S}(L)|}{\rho})\max_{i\neq c}\mathbb{E}[f^{y_{ i}}_{\mathcal{M}_{E}}(\mathbf{X}^{\prime})]=\max_{i\neq c}\widehat{\mathbb{ E}}[f^{y_{i}}_{\mathcal{M}_{E}}(\mathbf{X}^{\prime})],\]

which proves \(\widehat{\mathbb{E}}[f^{y_{c}}_{\mathcal{M}_{E}}(\mathbf{X}^{\prime})]>\max_{i \neq c}\widehat{\mathbb{E}}[f^{y_{i}}_{\mathcal{M}_{E}}(\mathbf{X}^{\prime})]\)\(\square\)

**Training procedure.** To achieve a better certification result, we involve randomness in the training stage, which is also adopted by almost all certified robustness approaches. To do so, we use the data augmentation strategy that utilizes the perturbed sentences for training, i.e., \(\mathbf{X}^{\prime}\in\mathscr{S}(L)\setminus\mathbf{X}\) given the original training sample \(\mathbf{X}\). In practice, we first train the model without data augmentation for several epochs to achieve a reasonable performance, followed by training with perturbed \(\mathbf{X}^{\prime}\). For each training data point, we randomly draw one neighbour sentence during training (as opposed to multiple draws during certified inference).

## 5 Experiments

We evaluate _WordDP_ on two classification datasets: Internet Movie Database (IMDB) Maas et al. (2011) and AG News corpus (AGNews) Zhang et al. (2015). IMDB is a binary sentiment classification dataset containing 50000 movie reviews. AGNews includes 30,000 news articles categorized into four classes. The target model architecture we select is a single-layer LSTM model with size of 128. We use Global Vectors for Word Representation (GloVe) Pennington et al. (2014a) for word embedding. The LSTM model achieves 88.4% and 91.8% clean accuracy on IMDB and AGNs, respectively. We use PWWS Ren et al. (2019) to generate adversarial examples on the test dataset. PWWS is a state-of-the-art attack method which uses WordNet to build synonym set and incorporates word saliency to replace selected named entities (NEs) with their synonyms in order to flip the prediction. The details about the datasets, model training and attack algorithm are in Appendix C.

### Evaluation Metrics and Baselines

We use four metrics to evaluate the effectiveness of _WordDP_: certified ratio, certified accuracy, conditional accuracy, and conventional accuracy. **Certified Ratio** represents the fraction of testing set that the prediction satisfies the certification criteria: \(\frac{\sum_{l=1}^{T}certifiedCheck(\mathbf{X}_{t},L,\epsilon)}{T}\), where \(certifiedCheck\) returns 1 if Theorem 4.1 is satisfied and \(T\) is the size of the test dataset. **Certified accuracy (CertAcc)** denotes the fraction of the clean testing set on which the predictions are both correct and satisfy the certification criteria. This is a standard metric to evaluate certified robust model Lecuyer et al. (2019). Formally, it is defined as:

where \(corrClass\) returns 1 if the classification output is correct. When the accuracy of a model is close to 100%, certified accuracy largely reflects certified ratio. **Conventional accuracy (ConvAcc)** is defined as the fraction of testing set that is correctly classified, \(\frac{\sum_{t=1}^{T}corrClass(\mathbf{X}_{t},L,\epsilon)}{T}\), which is a standard metric to evaluate any deep learning systems. Note that the input \(\mathbf{X}_{t}\) can be both adversarial or clean inputs. We use this metric to evaluate how _WordDP_ empirically works on adversarial examples.

Besides the above standard metrics, we introduce a new accuracy metric called **Conditional accuracy (CondAcc)** to evaluate the following: when a clean input \(\mathbf{X}_{t}\) is certified within bound \(L\), whether its corresponding \(L\)-word substitution adversarial example \(\mathbf{X}_{t}^{adv}\) is indeed correctly classified. The CondAcc can be formulated as: \(\frac{\sum_{t=1}^{T}certifiedcheck 

in CertAcc and CondAcc. Besides SAFER, we also compare the ConvAcc on adversarial examples with two state-of-the-art defense methods, i.e., IBP (Jia et al., 2019) and DNE (Zhou et al., 2020), which do not provide certified robustness guarantee. Thus, their defense may be broken by more powerful word substitution attacks in the future.

### Certified Results

**Certified Accuracy.** Figure 3 presents the CertAcc, CondAcc and ConvAcc under different \(\epsilon\) and \(L\), respectively. Each line in the figures represents a certified bound \(L\), which allows \(L\) number of words to be substituted. The first row is the results on IMDB, and the second row is on AGNews.

Figures 3(a) and 3(d) show the certified accuracy on the two datasets. Since the conventional accuracy on the clean examples of our mechanisms is close to 100% (as shown in Figures 3(c) and 3(f)), the certified accuracy mainly reflects the certified ratio (which we skip in the results). As shown, higher \(\epsilon\) can result in lower CertAcc. This is intuitive as the condition in Theorem 4.1 is more difficult to satisfy when given higher epsilon, i.e. weaker requirement of indistinguishability of the output, hence results in lower certified ratio. As illustrated in 3(a), when \(\epsilon\) is around \(1.5\), the mechanism will approach \(0\) certified ratio. This indicates that \(\epsilon\) can only be searched within a limited range.

Comparing each line in 3(a) and 3(d), we note that greater \(L\) results in higher CertAcc in most cases for the AGNews dataset. This can be explained by the fact that a greater \(L\) means more word substitutions and randomness are introduced in _WordDP_, making it easier to ensure the indistinguishability of the output, and hence a higher certified ratio.

**Accuracy on Adversarial Examples.** Figures 3(b), 3(e), 3(c) and 3(f) present CondAcc and ConvAcc of the two datasets on adversarial examples, respectively. Note that we only test the adversarial examples that are within the \(L\) bound. We also show the CondAcc and ConvAcc for both clean and adversarial examples without any defense mechanisms as a reference. In addition, we show ConvAcc of _WordDP_ with varying parameters on clean examples to show the impact of the mechanism on clean examples.

As shown in the figures, _WordDP_ achieves significantly higher accuracy on adversarial examples compared to no defense while maintaining the close to 100% accuracy on clean examples. Conditional

Figure 4: Certified Ration vs. Conditional Accuracy

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & ADV & IBP & DNE & SAFER & _WordDP_ \\ \hline IMDB & 0.172 & 0.722 & 0.823 & 0.727 & **0.972** \\ \hline AGNews & 0.194 & 0.823 & **0.909** & 0.647 & 0.719 \\ \hline \end{tabular}
\end{table}
Table 1: Empirical comparison on accuracy

Figure 3: Certified Accuracy, Conditional Accuracy and Conventional Accuracy on IMDB and AGNews



accuracy is higher than conventional accuracy as expected, since it is computed only on those adversarial examples with a certified counterpart clean example. Besides, we can observe that with higher \(\epsilon\), higher CondAcc on adversarial examples can be achieved. This is because less randomness is introduced in the inference.

In addition, by comparing different \(L\) bound under the same \(\epsilon\), larger \(L\) can yield more accuracy improvement on adversarial examples but less on clean examples. Intuitively, using the aggregated prediction of more distant neighbouring sentences (higher \(L\)) can benefit adversarial examples more than clean examples.

**Trader-off between Certified Ratio and CondAcc.** We can see that \(\epsilon\) has an opposite impact on certified accuracy (certified ratio) and CondAcc, we present the trade-off between the certified ratio and CondAcc of _WordDP_ in Figure 4 in comparison with the baseline method SAFER. Ideally, we want both high certified ratio and high condAcc to contribute to overall high accuracy. The black dot represents the baseline SAFER, since the neighbouring sentence generating method of SAFER does not depend on \(L\) or \(\epsilon\). As illustrated on these two datasets, with \(L=20\) and \(L=40\),_WordDP_ can dominate SAFER and achieve a much better performance in both certified ratio and condAcc.

**Relation between certified bound \(L\) and adversarial attack power \(L_{adv}\).** Figure 5 presents the three accuracy metrics under different attack power and defense power. In Figure 5(a), we fix the attack power \(L_{adv}\) to 40, which means allowing less than 40 word substitutions, and adjust the _WordDP_ defense power by using different certified bound \(L\). As discussed in Section 4, certified bound \(L\) determines the size of neighbouring set. Greater \(L\) leads to higher randomness and thus can benefit the CondAcc and ConvAcc on adversarial examples. On the other hand, greater \(L\) also makes the certified condition more difficult to be satisfied, which result in lower CertAcc.

In Figure 5(b), we fix the certified bound \(L\) to 40, which means using the same power of _WordDP_ to defend against adversarial examples generated by varying attack power \(L_{adv}\). As shown in the figure, the performance increases with higher attack power. This is because the adversarial examples with more word changes (higher \(L_{adv}\)) are more difficult to generate but easier to defend (due to the nature of PWWS attack algorithm).

**Comparison with Empirical Defense.** Besides certified robust method SAFER, we also compare CondAcc of _WordDP_ with baseline empirical defense methods, IBP Jia et al. (2019) and DNE Zhou et al. (2020). Table 1 compares the highest CondAcc achieved by _WordDP_ with the conventional accuracy reported by the baselines (ADV corresponds to no defense). _WordDP_ achieves a much higher accuracy on IMDB dataset compared to IBP, DNE and SAFER. For AGNews, the accuracy of _WordDP_ outperforms SAFER, but is lower than the two empirical defenses. We stress, however, the empirical defense methods do not provide any rigorous certified robustness guarantees and the performance can be significantly dependent on datasets and specific attacks.

**Efficiency Comparison.** We also compare the efficiency of _WordDP_ with SAFER by computing the average time cost for certifying one input and producing the Monte Carlo sampling-based output. It takes _WordDP_ 6.25s and 3.21s on IMDB and AGNews, respectively. As a comparison, it costs SAFER 230.35s and 96.68s. Thus, _WordDP_ achieves more than \(30\times\) efficiency improvement.

## 6 Conclusion

We proposed _WordDP_, a certified robustness method to adversarial word substitution attacks with the exponential mechanism-based algorithm. Compared with previous work, _WordDP_ achieves notable accuracy improvement and 30\(\times\) efficiency improvement. In the future, it would be interesting to expand _WordDPP_ to other kinds of textual adversarial examples, such as character-level attacks. It is also worthwhile to study other certified approaches such as random smoothing.

## Acknowledgement

We sincerely thank all anonymous reviewers for their constructive comments. This work is partially supported by the National Science Foundation (NSF) CNS-1952192, IIS-1838200, and National Institutes of Health (NIH) CTSA Award UL1TR002378.

Figure 5: The trend on accuracy under different defense and attack power





## References

* Alzantot et al. (2018) Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. Generating natural language adversarial examples. In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_, Brussels, Belgium. Association for Computational Linguistics.
* Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. _arXiv preprint arXiv:1409.0473_.
* Bakshi et al. (2016) Rushlene Kaur Bakhsi, Navneet Kaur, Ravneet Kaur, and Gurpreet Kaur. 2016. Opinion mining and sentiment analysis. In _2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom)_, pages 452-455. IEEE.
* Cheng et al. (2019) Yong Cheng, Lu Jiang, and Wolfgang Macherey. 2019. Robust neural machine translation with doubly adversarial inputs. _ACL_.
* Cohen et al. (2019) Jeremy M Cohen, Elan Rosenfeld, and J Zico Kolter. 2019. Certified adversarial robustness via randomized smoothing. _arXiv preprint arXiv:1902.02918_.
* Dwork (2008) Cynthia Dwork. 2008. Differential privacy: A survey of results. In _International conference on theory and applications of models of computation_, pages 1-19. Springer.
* Dwork et al. (2006) Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. 2006. Calibrating noise to sensitivity in private data analysis. In _Theory of cryptography conference_, pages 265-284. Springer.
* Dwork et al. (2014) Cynthia Dwork, Aaron Roth, et al. 2014. The algorithmic foundations of differential privacy. _Foundations and Trends in Theoretical Computer Science_, 9(3-4):211-407.
* Ebrahimi et al. (2018) Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2018. Hotflip: White-box adversarial examples for text classification. _ACL_.
* Gao et al. (2018) Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. 2018. Black-box generation of adversarial text sequences to evade deep learning classifiers. In _2018 IEEE Security and Privacy Workshops (SPW)_, pages 50-56. IEEE.
* Huang et al. (2019) Po-Sen Huang, Robert Stanforth, Johannes Welbl, Chris Dyer, Dani Yogatama, Sven Gowal, Krishnamurthy Divjotham, and Pushmeet Kohli. 2019. Achieving verified robustness to symbol substitutions via interval bound propagation. _EMNLP-IJCNLP_.
* Jia et al. (2019) Robin Jia, Aditi Raghunathan, Kerem Goksel, and Percy Liang. 2019. Certified robustness to adversarial word substitutions. _EMNLP-IJCNLP_.
* Jin et al. (2019) Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2019. Is bert really robust? natural language attack on text classification and entailment. _arXiv preprint arXiv:1907.11932_, 2.
* Jones et al. (2020) Erik Jones, Robin Jia, Aditi Raghuanthan, and Percy Liang. 2020. Robust encodings: A framework for combating adversarial typos. _ACL_.
* Lecuyer et al. (2019) Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. 2019. Certified robustness to adversarial examples with differential privacy. In _2019 IEEE Symposium on Security and Privacy (SP)_, pages 656-672. IEEE.
* Lee et al. (2019) Guang-He Lee, Yang Yuan, Shiyu Chang, and Tommi Jaakkola. 2019. Tight certificates of adversarial robustness for randomly smoothed classifiers. In _Advances in Neural Information Processing Systems_, pages 4910-4921.
* Liang et al. (2018) Bin Liang, Hongcheng Li, Miaoqiang Su, Pan Bian, Xirong Li, and Wenchang Shi. 2018. Deep text classification can be fooled. _IJCAI_.
* Maas et al. (2011) Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In _Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies_, pages 142-150.
* McSherry and Talwar (2007) Frank McSherry and Kunal Talwar. 2007. Mechanism design via differential privacy. In _48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07)_, pages 94-103. IEEE.
* Miyato et al. (2016) Takeru Miyato, Andrew M Dai, and Ian Goodfellow. 2016. Adversarial training methods for semi-supervised text classification. _stat_, 1050:7.
* Papernot et al. (2016) Nicolas Papernot, Patrick McDaniel, Ananthram Swami, and Richard Harang. 2016. Crafting adversarial input sequences for recurrent neural networks. In _MILOM 2016-2016 IEEE Military Communications Conference_, pages 49-54. IEEE.
* Pennington et al. (2014a) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014a. GloVe: Global vectors for word representation. In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 1532-1543, Doha, Qatar. Association for Computational Linguistics.
* Pennington et al. (2014b) Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014b. Glove: Global vectors for word representation. In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_, pages 1532-1543.
* Ren et al. (2019) Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019. Generating natural language adversarial examples through probability weighted word saliency. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics.



* Sato et al. (2018) Motoki Sato, Jun Suzuki, Hiroyuki Shindo, and Yuji Matsumoto. 2018. Interpretable adversarial perturbation in input embedding space for text. _IJCAI_.
* Szegedy et al. (2013) Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. _arXiv preprint arXiv:1312.6199_.
* Wang et al. (2019) Xiaosen Wang, Hao Jin, and Kun He. 2019. Natural language adversarial attacks and defenses in word level. _arXiv preprint arXiv:1909.06723_.
* Ye et al. (2020) Mao Ye, Chengyue Gong, and Qiang Liu. 2020. Safer: A structure-free approach for certified robustness to adversarial word substitutions. _ACL_.
* Zang et al. (2020) Yuan Zang, Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. 2020. Word-level textual adversarial attacking as combinatorial optimization. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 6066-6080.
* Zhang et al. (2015) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In _Advances in neural information processing systems_, pages 649-657.
* Zhou et al. (2020) Yi Zhou, Xiaoqing Zheng, Cho-Jui Hsieh, Kai-wei Chang, and Xuanjing Huang. 2020. Defense against adversarial attacks in nlp via dirichlet neighborhood ensemble. _arXiv preprint arXiv:2006.11627_.