## 4 Experimental Results
### Implementation Details  
In this study, we completed the experiment according to the following details and steps. In the model architecture, first of all, due to the limitations of the experimental conditions, the backbone network we adopted contains 12 layers of attention parameters from shallow to deep. Therefore, combined with the initial embedding semantic vector, a total of 13 layers of semantic vector information will participate in the subsequent calculation. Secondly, in the hierarchical deepening attention module, the parallel attention computing module has three layers. It is worth mentioning that when the last layer completes the attention computing, the module does not activate the semantic value again, so as to retain the semantic information mined. Finally, the FFCon module transmits a total of 12 layers of semantics and locational information from higher to lower layers, completes feature fusion through effective semantic reduction MLPs, and then participates in classification through pooling connections. The initial weights (in this study, the addresses of the pre-training parameters loaded by the backbone network in all experiments are as follows: English pre-training parameter address: [[https://huggingface.co/xlnet/xlnet-base-cased](https://huggingface.co/xlnet/xlnet-base-cased)]([https://huggingface.co/xlnet/xlnet-base-cased](https://huggingface.co/xlnet/xlnet-base-cased)), accessed on 30 May 2024; Chinese pre-training parameter address: [[https://huggingface.co/hfl/chinese-xlnet-base](https://huggingface.co/hfl/chinese-xlnet-base)]([https://huggingface.co/hfl/chinese-xlnet-base](https://huggingface.co/hfl/chinese-xlnet-base)), accessed on 30 May 2024) of the backbone are loaded with the officially trained English parameters of XLNet [12] and the Chinese parameters trained by [51] based on a Wikipedia dump [10]. The remaining neural network parameters are randomly initialized for training. In terms of training parameter setting, first, the statements entered into the model are cropped to a specified and uniform size to fairly compare the effects. The clipped sentence size hyperparameter is set comprehensively after the maximum sentence length statistics and average sentence length calculation of the dataset to ensure that different sentence length parameters are set for the long-text dataset and the short-text dataset to make the experimental model understand the text semantics scientifically. We trained this model for 100 epochs by using the BERTAdam optimizer with a weight decay of 0.001. For the training process on long-text datasets, the batch size was set as 16, the learning rate \(\alpha\) was 0.00005, and the sentence length was arranged as 512. In terms of short-text training, the batch size was 64, \(\alpha\) was 5 \(\times\) 10\({}^{-5}\), and the sentence length was 128. In particular, when the effect does not improve, the learning rate will drop tenfold, to as low as 5 \(\times\) 10\({}^{-6}\) on short-text datasets and as low as 5 \(\times\) 10\({}^{-7}\) on long-text datasets. This is known as the learning rate multiplier decay strategy. Furthermore, we used an early stopping strategy for all models to avoid model overfitting. The early stop strategy means that when the learning rate reaches the minimum and the model effect on the test set is not improved after a long time of training (in our experiments, we set the time to 500 steps, and 1 step is equal to 10 batches), the program will automatically stop the training process and retain the best model training parameters. In the experiment, in order to ensure the comparability of the baseline model and our proposed method, we used the same hyperparameter settings and training methods for all models on the same dataset. In terms of the selection of specifichyperparameters in the experiment, we set the fixed statement length hyperparameters after several attempts based on the ability of the GPU we have.