## 3 Model Abstracting and Explaining Framework
### Learning Abstraction Model for RNNs  
#### 3.1.1 Obtaining the Abstract Traces  
The first step in learning a PFA for RNNs is to extract the hidden states. They play a significant role in revealing the internal causal relationships of RNNs, which include hidden layer weight vectors and output layer weight vectors. In addition to the hidden state of an RNN, they also involve processing the state of the input layer, representing discrete texts as numerical vectors. In the field of NLP, the word embedding technique is feasible and effective for word representation. It converts natural language words into digital form, so we can measure their quantitative properties. If researchers directly obtain all states from RNNs to construct PFA, the construction process will be very cumbersome due to the large observation state table. Therefore, this abstracting technique needs to reduce the number of hidden states continuously. According to the research on RNN model abstraction learning [32; 33; 34], hidden states with similar behaviors have very close positions in their state vector space, so states with similar behaviors can be integrated into the same region. In Step (1) of Stage I, we use \(k\)-means to integrate RNN states, which divide many state vectors into a finite number of clusters to reduce the number of operational state objects in subsequent abstraction learning algorithms. The position vector of the space partition center represents the characteristics of the abstract state. The clustering technique is based on explainable \(k\)-means, so abstract techniques ensure explainability in the clustering process. In addition, clustering divides a large number of complex hidden states into a finite number of partitions. This is in line with the natural cognitive pattern of humans in the process of text classification, which is to  
Figure 3: PSO4 RNN framework. Figure 2: PFA example. classify text words based on their meanings and context, judge the importance of numerous words critical to classification results, and then decide the overall class of the whole text. In the process of abstract states partitioning, appropriate preprocessing (such as word embedding) can reduce the number of input words, the dimensionality of clustering operation input vectors, and the cost of \(k\)-means processing. We use a word2 vec pre-trained model for word embedding. After the state clustering in the original RNN, it is necessary to construct various abstract state traces. Due to the interweaving of nodes during state partitioning, state transitions also change dynamically. The transitions between different states should be independently recorded and processed. Subsequently, each state transition will be added between the corresponding states, creating a complete sequence from the start to the end state. This yields a state trace. Repeat these steps until all state and model behavior samples create corresponding traces. This process corresponds to Step (2) and Step (3) of Stage I, as depicted in Figure 3.  
After sorting out the state traces, the traces will be organized one by one into a frequency prefix tree (FPT). From a practical perspective, an FPT can be considered a special type of PFA, which is a very detailed probabilistic finite automaton. It contains detailed information beyond the user's needs, often overwhelming the messages that humans should mainly focus on. Therefore, it is necessary to trim the FPT further and reduce the number of useless nodes and edges. Such a process is called learning PFA from the FPT. The following example illustrates a brief process of text classification that abstracts traces and organizes them into a frequency prefix tree. This process corresponds to step (4) of Stage I, as depicted in Figure 3.  
**Example 2**. : _An example of trace abstraction for an RNN-based sentiment analysis of movie reviews._  
We use PFA in the sentiment analysis case in Example 1 as a description. Firstly, the learned PFA requires segments from comments and then word embedding.