## 3 Materials and Methods
### Hierarchical Attention Mechanism  
This study builds a basic model of a single classifier in the classifier tree based on the improved multilayer perception mechanism to complete the mapping from the segmented char-embeddings to categories after data preprocessing (Figure 6). From a functional perspective, the model can be divided into three parts. First, a hierarchical attention mechanism layer solves the problem of transitioning from char-level to document-level embeddings [30]. The second layer is the fully connected layer, which converts document-level embeddings into category probabilities. Finally, an activation layer uses the function Argmax to find the target category with the highest probability. Figure 5: Data preprocessing based on XLNet. The hierarchical attention mechanism layer utilizes hierarchical MLP to learn and sequentially obtain (1) attention weights from char-level embeddings to segment-level embeddings mapping and (2) attention weights from segment-level embeddings to document-level embeddings [31]. Using this hierarchical calculation method, vectors that represent the semantic features of documents are obtained. Furthermore, this hierarchical attention mechanism layer can be further divided into a lower-order representation layer and a higher-order representation layer. Of these, the lower-order representation layer represents the semantics of words, whereas the higher-order representation layer represents the semantics in paragraphs. This method of hierarchical attention mechanism can extract document-level embeddings from the text on the one hand and shorten the length of input text in each processing unit through text segmentation and layering on the other hand, thus avoiding the vanishing gradient problem common in long text processing [32]. In the low-order representation layer, the char-embeddings \(s^{{}^{\prime}}_{il}\) of each segment are converted into a hidden layer whose dimension number is half of the char-embeddings size through the linear function. Formally, let \(X\in R^{\text{m}\times\text{n}\times\text{k}\times d}\) be the input char-embeddings, where \(m\) is the amount of PGT, \(n\) is the number of segments in a text, \(k\) is the number of words contained in each segment, and \(d\) is the dimension of the char-embeddings. Moreover, let \(x_{il}\in R^{\text{k}\times\text{d}}\) be the char-embeddings matrix of segment \(l\) in document \(i\). \(A_{il}\in R^{\text{k}\times(\text{d}/2)}\) denotes the activations and is given by  
\[A_{il}=\phi\left(W^{1}x_{il}^{\text{ }T}+b^{1}{}_{l}\right)^{T} \tag{10}\]  
where \(W^{1}{}_{l}\in R^{(d/2)\times d}\) is a weight matrix and \(b^{1}{}_{l}\in R^{(\text{d}/2)\times 1}\) is the bias vector in the low-order layer. Meanwhile, \(\phi\) is a nonlinear activation function, which is taken as the function _Tanh_ in this study, that is,  
\[Tanh(x)=\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}} \tag{11}\]  
Figure 6: Basic transferable classifier model for classifier tree. Let \(O_{il}\in R^{1\times d}\) be the output of the low-order representation layer, given by  
\[O_{il}=\sigma\Big{(}W^{2}{}_{l}A_{il}{}^{T}\Big{)}x_{il} \tag{12}\]  
where \(W^{2}{}_{l}\in R^{1\times(d//2)}\) is a weight matrix that can convert \(A_{il}\) into a vector and \(\sigma\) denotes the function \(softmax(\cdot)\) function, which can calculate the weights of each word for the semantic vectors of a segment. After obtaining segment-level embeddings in the higher-order representation layer, it is necessary to concatenate the embeddings of all segments in the same document in the order of the corresponding segments in the text. Therefore, let \(O_{i}\in R^{n\times d}\) be the segment-level embeddings matrix of text \(i\) and \(\bigcirc\) denote vector concatenation. Then, \(O_{i}\) can be calculated as  
\[O_{i}=O_{i1}\bigodot O_{i2}\bigodot\cdots\bigodot O_{in} \tag{13}\]  
Subsequently, the higher-order representation layer uses a method like the low-level representation layer to calculate document-level embeddings.