## 3 Proposed Method
### Feature Mapping Module  
The feature mapping module processes the statement and generates the feature vector. This process can be divided into three stages: preliminary loading of statements into index vectors, word vector mapping, and traversing the Transformer block. After processing, the original text sentences are transformed into specific vectors for computation. _Phase I: initial load index vector._ At this stage, the input text is segmented according to the length of the sentence and projected into a 1D vector. First, the input text sentence \(x\in R^{Seq}\) will be segmented by the word segmenter according to unit words and will be segmented according to the sentence length or supplementary sentences to generate new sentences. The segmentation process is as follows:  
\[x^{\prime}=Cut(x),\ x^{\prime}\in R^{Seq^{\prime}}, \tag{1}\]  
where \(x^{\prime}\) represents the new sentence after segmentation or supplementation, and \(Seq^{\prime}\) denotes the length of the new sentence. This value is artificially determined. Thereafter, a vocabulary index 1D vector is generated based on each word. An attention mask index 1D vector is also generated, which will map each word \(w_{i},i\in\) 1,2,3, \(\cdots,Seq^{\prime}\) into a specific vector. The index vector of the text is expressed as \(v_{0}\in R^{Seq^{\prime}}\), which is mainly composed of the index of the vocabulary. If one word does not appear in the vocabulary, then its  
Figure 3: The architecture of APTrans. First, the statement will be masked and go through the embedding block before entering the T-PTLM backbone. Second, T-PTLM will hierarchically output text vectors to the hierarchical deeper attention module for further mining of long-distance dependencies. The matrices of different colors in this process in the figure represent the feature relationships mined from the text at different levels. Thereafter, the vector is layered into the connection block, which is mainly composed of the FFCon module to complete the fusion of high-level semantics and low-level location information. The compression connection module is then inputted to complete the pooling of semantic vectors. Finally, the classification header predicts the resulting text category. index is replaced by the index of [_UnK_]. Each attention mask index vector of a sentence is represented as \(m_{0}\in R^{Seq^{\prime}}\). This vector mainly consists of two numbers, 0 and 1, where 1 indicates that the word is a supplementary word and should not be considered for attention calculation, while 0 means that the word is an original word and attention calculation should be performed. _Phase II: vector mapping._ Given that the Transformer layer requires a certain semantic vector input, at this stage, the index vector obtained through processing is used to load the 2D feature vector of each word. Specifically, the text statement is mapped into an exact semantic vector through a dictionary. The projection formula is as follows:  
\[c_{i}=E(v_{i}),\ i\in\text{1,2,3,}\cdots,n, \tag{2}\]  
\[p_{0}=[c_{1},c_{2},c_{3},\cdots,c_{n}], \tag{3}\]  
where \(c_{i}\in\ R^{E}\) represents the vector of each word after projection; \(p_{0}\in R^{Seq^{\prime}\times E}\) represents the matrix containing the word vector, that is, the semantic matrix of the statement; \(Seq^{\prime}\) represents the sentence length; and \(E\) represents the dimension of the vector after mapping each word. This process can be accomplished by pre-training a dictionary of word vectors. _Phase III: traverse transformer block._ The semantic feature vector will go through M Transformer blocks. Each Transformer block will go through dual-stream attention calculations by using the attention mask index vector of the sentence and finally generate an implicit semantic vector.