## 4 Experimental Results  
### Experimental Setup  
The cases in the experiment include a regular language and two text classifications. The datasets of the regular language and the movie review are publicly available online (NLP progress sentiment analysis, available online at [[https://nlpprogress.com/english/sentiment_analysis.html](https://nlpprogress.com/english/sentiment_analysis.html)]([https://nlpprogress.com/english/sentiment_analysis.html](https://nlpprogress.com/english/sentiment_analysis.html)) (accessed on 26 December 2023)), while the dataset of the logistics review is based on the logistics reviews of users on Amazon e-commerce platforms (Ni, J. Amazon Review Data. Available online: [[https://nijianmo.github.io/amazon/index.html](https://nijianmo.github.io/amazon/index.html)]([https://nijianmo.github.io/amazon/index.html](https://nijianmo.github.io/amazon/index.html)) (accessed on 28 December 2023)). The first case is the Tomita regular language classification under three artificial grammar rules. These grammar rules are used in automata learning from RNNs [21] to validate learning effectiveness. They consist of three regular languages that are defined over the alphabet \(\{0,1\}\). The grammar rules are listed in Table 1. If the string conforms to the syntax, it is marked as positive. In the experiment, training sets of different lengths for each grammar rules were created, and the length varies from 1-12 to 15-17, and the strings with lengths of 3, 11, and 16 were selected as the test set for each grammar rule. The ratio between the training and test sets is 5:1. The second case is a sentiment classification of DLRV. This dataset is extracted from customer evaluations of product logistics and is collected from comments on logistics services at Amazon e-commerce platforms. The ratio between the training and test sets is 4:1. The third case is the sentiment classification of the movie review dataset in the IMDB. The ratio between the training and testing sets is also set to 4:1.  
We will measure the effectiveness and efficiency of PSO4 RNN with two baseline algorithms. The first one is proposed in [17] (called BSL1 below), which uses an active learning framework to learn DFA from RNNs. The second is proposed in [25] (referred to as BSL2 hereinafter), which learns PFA from RNNs based on the AAlergia algorithm.  
In the experiment, the CPU is an AMD Ryzen 7 5800H, the GPU is an Nvidia RTX 3060 Laptop with 6G video memory and Random Access Memory (RAM) with a capacity of 16G. All cases are run on a Windows 11 operating system, and the Scikit-learn toolkit is used to train RNNs. In this work, the PSO4 RNN framework is implemented using Python 3.7.1. The text length of the input data they process is consistent. Each case is trained on two RNN models: a Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM). We set the number of hidden layers for two RNNs to 1 and the dimension of the hidden state to 508. In the training process, a One-Hot encoding technique is applied to encode  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Grammar** & **Description** \\ \hline Tomita\_1 & 1* \\ Tomita\_2 & (10)* \\ Tomita\_3 & Compliments of ((0 1)* 0)*1(11)*(0(0 1)*1)*0(00)*(1(0 1)*)* \\ \hline \hline \end{tabular} Note: In regular expressions, the asterisk (*) is a quantifier used to specify that the preceding expression can be repeated zero or more times.  
\end{table}
Table 1: Three Tomita artificial grammar rules.  
characters in the Tomita dataset, and word2 vec is used for text classification to convert each word in the real natural language into a 288-dimensional numerical vector.