## 4 Experimental Results
### Results
#### 2.2.1 Adversarial Text Detection  
Compared to the image field, research on adversarial attacks in the text field remains to be expanded. In addition, the highly discrete nature of text content is different from the pixels of images, which makes it challenging to transfer image adversarial attacks to the one of texts. Adversarial attacks can expose the fragility of machine learning models, which will help to analyze the robustness, especially the explainability of RNNs [36; 37]. Identifying and deleting adversarial samples can prevent the generation of adversarial patterns and alleviate adversarial attacks. Abstraction models can be used for adversarial text detection. It has been shown in [38] that adversarial text can be automatically generated by perturbating slightly on non-poisoned text. Typical techniques for generating adversarial text include replacing words with synonyms and using machine learning models to translate sentences multiple times (e.g., from English to Spanish and then back to English). Inspired by [PERSON] et al. 's work [25], this work uses Equation (10) to describe the adversarial text detection ratio. Given a sentence \(x\), the adversarial text detection ratio can be defined as follows:  
\[T(x,y)=\frac{P(x,y)}{P(x,\overline{y})} \tag{10}\]  
where \(y\) is the output label predicted by the RNN and \(P(x,y)\) is the probability from input \(x\) to output \(y\) on PFA. \(P(x,\overline{y})\) represents the total probability of reaching outputs other than \(y\) from input \(x\). We use the stochastic model checking tool Prism [35] to compute the probability. We select a threshold \(T_{\theta}\) for comparison with \(T(x,y)\), which can serve as a critical value for distinguishing adversarial text from benign text. If the text's \(T(x,y)\) is less than \(T_{\theta}\), it is considered an adversarial text. **Example 5**. : _Simple Adversarial Text Detection Case._  
Let us consider the sentence and PFA in Example 1. Assuming \(x\) = \"This movie is so beautiful\" and \(y\) = \(P,\overline{y}\) = N. \(T_{\theta}\) is set as 3. The input that passes through the trace to label P is \(s_{1}-s_{3}-s_{3}-s_{4}\). Then, its probability in Equation (10) is \(P(x,y)=tr(s_{1},s_{3})tr(s_{3},s_{3})tr(s_{3},s_{4})\) = 0.2430, where \(tr(s,\)\(s\)\()\) means transition probability from \(s\) to \(s^{\prime}\). \(P(x,\overline{y})=tr(s_{1},s_{3})\cdot tr(s_{3},s_{3})\cdot tr(s_{3},s_{5})=0.0675\). According to Equation (10), \(T(x,y)\) = 3.6. If it is greater than \(T_{\theta}\), then \(x\) is not an adversarial text. Assume there are \(N_{bn}\) benign samples and \(N_{pn}\) samples with poisoning attacks. There are four modes selected for poisoning: (1) insertion, (2) change, (3) delete, and (4) order swapping. The first step of adversarial text detection is to learn PFA from RNNs. Secondly, the poisoned sentence \(x^{\prime}\) is input into PFA, and the probability \(P(x^{\prime},y)\) is calculated based on PFA. Thirdly, a clean sequence \(x\) is input into PFA, and we calculate \(P(x,y)\). Finally, in Equation (10), the sentences that have a lower ratio \(T(x,y)\) than the threshold are identified as adversarial text. For all adversarial samples, if one of them is detected as benign text, it fails to detect real adversarial text. We set the number of samples correctly detected  
Figure 12: Explanation of highlighted keywords in the IMDB. Sentences (**a**–**c**) are text samples in IMDB. The gray words correspond to self-pointed words. The blue words correspond to cross-state words. The red words correspond to cross-cluster words. as \(num_{cd}\), and then detection accuracy can be defined as \(Acc_{ad}=num_{cd}/N_{bn}\). Figure 13 provides an overview of this detection effect evaluation process. The detection accuracy MSE is the average of \(Acc_{ad}\) in ten experiments. The \(Acc_{ad}\) MSE of PSO4 RNN on three datasets is shown in Table 5. Here, LSTM is an RNN with one hidden layer and 508 hidden nodes, and the number of state clusters is 12.