# A Survey of Adversarial Attacks: An Open Issue for Deep Learning Sentiment Analysis Models
## 4 Defense against Adversarial Attacks on Sentiment Analysis
### Adversarial Defense  
In this section, we present the main reviewed defense works proposed against adversarial examples, particularly for sentiment analysis models. These works have been used as a reference for the design of other proposals. Table 2 summarizes the reviewed defenses according to their main characteristics. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Work** & **Defense Strategy** & **Granularity** & **Defense Evaluation** & **DNN Model Defended** \\ \hline \hline
[47] & Modification control & Character & Accuracy in recognizing misspelled words, Sensitivity to adversarial perturbations on a input & BERT fine-tuned \\ \hline
[40] & Modification control & Term & Model robustness & CNN, RNN and BERT \\ \hline
[48] & Modification control & Character/Term & Model robustness & BERT \\
[34] & Modification control & Term & Model robustness & CNN, BiLSTM, RNN \\ \hline
[39] & Data augmentation & Term & Model robustness & CNN, BiLSTM, BERT-Base \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Reviewed adversarial defenses for sentiment analysis models. * [PERSON] et al. [47] proposed designing and implementing a model for word validation and recognition before the classifier. The method is trained to recognize words modified by dropping, adding, and swapping internal characters within words. This defense proposes a two-stage solution to deal with character-level adversarial attacks, placing a word recognition model \(W\) before the downstream classifier \(C\). This recognition model is based on a semi-character architecture of RNNs, introducing various feedback strategies for handling uncommon or unseen words. The proposed defense method is inspired by psycholinguistic studies that proposed a semi-character-based RNN (ScRNN) that processes a sentence of words with misspelled characters and predicts the correct words. ScRNN treats the first and the last characters individually and is agnostic about the ordering of the internal characters. At each sequence step, the training target is the correct corresponding word (output dimension equal to vocabulary size), and the model is optimized using the cross-entropy loss. * [PERSON] et al. [40] proposed a defense mechanism called the Synonym Encoding Method (SEM). This mechanism inserts an encoder before the input layer of the model and then trains the model to remove adversary perturbations. Using an encoder, the defense method substitutes the words in the original sentence with their close synonyms. To define the encoder \(E\) that encodes a set of synonyms, the authors clustered the synonyms in the embedding space and allocated a unique token for each cluster, obtaining the same cluster for different words. * [PERSON] et al. [48] proposed the DIScriminate Perturbations (DISP) mechanism for identifying and adjusting malicious modifications and blocking attacks. To identify adversarial examples, the discriminator validates the probability of a term in the text being modified, based on which it provides a set of potential modifications. For each potential modification, an insertion estimator learns to restore modified terms by selecting a replacement token based on a search of the \(k\) nearest neighbors. The proposed mechanism tries to block adversarial attacks on some models for texts without modifying their structure or the training process. * [PERSON] et al. [34] proposed a general defense mechanism called TextFirewall for different attacks with different strategies. Given a text input \(x\), TextFirewall analyzes and quantifies each word's impact in the text to distinguish the polarity instead of directly detecting the difference between adversarial text and legitimate input (inputs without any modification). With \(x\) feeding the model the _top-\(k\)_ items that impact the classification results, if the _loss_ is greater than zero, then \(y^{*}\) is set to 1; otherwise, \(y^{*}\) is 0. The defense mechanism compares \(y^{*}\) and \(y^{\prime}\) to judge the role of \(x\).