## 4 Experiment
### Ablation Study  
We conducted an ablation study on RQ-OSPTrans, focusing on the specific task carried out on the THUCNews dataset. (1) **Influence of the activation function of the feature output layer on the output classification:** The output layer activation function is the first layer in the network to participate in backpropagation. The choice of function determines whether complex features can be mapped to a specific space and divided. In this study, experiments were conducted on the THUCNews dataset by using different prototypes and compositions of activation functions combined with Mish activation. The experimental objects include ReLU, Tanh, Mish, Mish&Tanh, and Mish&ReLU functions (functions on both sides of \"g\" activate \(\mathit{Output}_{\mathit{cls}}\) and \(\mathit{Output}_{\mathit{\ leader}}\); no \"&\" indicates the same mapping for both output types). The experimental results are shown in Figure 5. In the third section, we introduced the sources of the two types of outputs. The learning process to obtain \(\mathit{Output}_{\mathit{cls}}\) is more complex compared with \(\mathit{Output}_{\mathit{\ leader}}\). The Mish function can balance the distribution differences between the two output types while retaining nonlinearity, demonstrating the best performance. By contrast, ReLU can only retain positive weights, neglecting negative ones, resulting in ineffective attenuation of irrelevant information during output weight amplification and suboptimal performance, highlighting the importance of nonlinear mapping in the output. However, excessive nonlinear activations will bring greater classification difficulty. The result of the comparison of the classification performance of Mish, Mish&Tanh, and Tanh activation functions indicates that the accuracy of Tanh activation for \(\mathit{Output}_{\mathit{cls}}\) is lower than that using the Mish activation function, indicating that balanced nonlinear mapping results in superior performance. Therefore, RQ-OSPTrans determines the structure of using the same nonlinear mapping for both output types as the optimal choice. Figure 4: Confusion matrix diagram of the top three models in THUCNews, AG’s News, and arXiv-10 dataset performance. (**a–c**) Confusion matrix diagram of the top three model performance in the THUCNews dataset. (**a**) RQ-OSPTrans in the THUCNews dataset confusion matrix. (**b**) Confusion matrix of RBT6 in the THUCNews dataset. (**c**) Confusion matrix of MacBERT in the THUCNews dataset. (**d–f**) Confusion matrix diagram of the top three model performance in the AG’s News dataset. (**d**) Confusion matrix of RQ-OSPTrans in the AG’s News dataset. (**e**) Confusion matrix of ELECTRA in the AG’s News dataset. (**f**) Confusion matrix of GPT-2 in the AG’s News dataset. (**g–i**) Confusion matrix diagram of the top three model performance in the arXiv-10 dataset. (**g**) Confusion matrix of RQ-OSPTrans in the arXiv-10 dataset. (**h**) Confusion matrix of BERT in the arXiv-10 dataset. (**i**) Confusion matrix of ELECTRA in the arXiv-10 dataset. astro-ph: Astrophysics. cond-mat: Condensed matter. cs: Computer Science. eess: Electrical engineering and systems science. hep-ph: High-energy physics—phenomenology. hep-th: High-energy physics—theory. math: Mathematics. physics: Physics. quant-ph: Quantum physics. stat: Statistics. (2) **Influence of maximum sequence length on classification:**  
We conducted comparative experiments by setting several maximum sequence lengths that have efficiently performed in other text classification tasks. The experimental results are shown in Figure 6.  
Our model is more suitable for shorter text extensions (Figure 6a). Considering the structure of the text, merging news headlines and news content into one text frequently results in only half of the information of the news article being fully included in sequences of 256 and 384 tokens, increasing the ambiguity in the model's understanding of headlines.