## 4 Experiments and Results
### Experimental Results  
In order to assess the performance of the model, we selected the BERT benchmark, the RoBERTa benchmark, the logistic regression algorithm proposed by [9], the Bert GCN proposed by [7], etc., for comparison. The experimental results are as shown in Table 6. For the THUCNews dataset, the experimental data from [PERSON] [38] were used. The experimental results are as shown in Table 7. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & \multicolumn{3}{c}{**FBS**} & \multicolumn{3}{c}{**Our Dataset**} \\ \cline{2-7}  & **Precision** & **Recall** & **F1** & **Precision** & **Recall** & **F1** \\ \hline Bert & 95.25 & 94.84 & 94.97 & 94.68 & 94.60 & 94.56 \\ \hline Bert + Bi-LSTM & 96.19 & 95.94 & 96.05 & 95.22 & 95.12 & 95.09 \\ \hline Bert + Bi-GRU & 95.71 & 95.62 & 95.65 & 95.17 & 94.92 & 94.98 \\ \hline RoBERTa & 97.08 & 96.76 & 96.88 & 96.76 & 96.60 & 96.57 \\ \hline RoBERTa + Bi-LSTM & 97.14 & 96.83 & 94.29 & 96.96 & 96.76 & 96.74 \\ \hline RoBERTa + Bi-GRU & 96.75 & 96.55 & 96.64 & 97.15 & 97.08 & 97.03 \\ \hline Logistic Regression [9] & 94.90 & 94.64 & 94.32 & - & - & - \\ \hline Bert + GCN [7] & 96.62 & 93.56 & 92.68 & - & - & - \\ \hline Bert-MHARC & 96.21 & 95.91 & 96.03 & 95.33 & 95.24 & 95.15 \\ \hline RoBERTa-MHARC (ours) & **97.90** & **97.47** & **97.65** & **98.11** & **98.12** & **98.10** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Table of comparison of various models’ performance on the FBS dataset and our dataset. RoBERTa-MHARC shows strong performance on the three datasets. On the FBS dataset, precision reaches 97.90%, recall reaches 97.47%, and the F1 value reaches 97.65%. On our dataset, precision reaches 98.11%, recall reaches 98.12%, and the F1 value reaches 98.10%. On the THUCNews dataset, precision reaches 94.32%, recall reaches 93.66%, and the F1 value reaches 93.69%. This excellent performance is mainly attributed to the RoBERTa model's use of an extensive pre-training corpus, which enhances the word semantic information and allows it to learn contextual relationships in the text better. This shows that using large-scale pre-trained models can significantly improve the performance in multi-class classification tasks with Chinese text messages. At the same time, multi-head attention is the main body of our model. Adding the multi-head attention mechanism significantly improves the model's performance. The main reason for this is that the introduction of multi-head attention strengthens the model's ability to model the input sequences. Multi-head attention allows the model to focus on different parts of the input simultaneously, and each attention head is able to capture information about specific semantics or locations, thereby improving the model's perception of complex contexts and relationships. By computing multiple attention heads in parallel, the model can more comprehensively understand the characteristics of the input sequence, effectively solving the problems of long-distance dependency and global relationship modeling. thereby allowing it to perform better on tasks. We also compared the impact of different loss functions on the performance of the RoBERTa-MHARC model to understand the performance indicators of these models, thereby providing optimization strategies for practical applications. As can be seen from Figures 4 and 5, the RoBERTa-MHARC model achieved the best experimental results on both datasets by using a combination of the cross-entropy loss function and the inconsistency loss function. We also conducted ablation experiments to verify the role of each module in the model. We designed three experimental groups as part of the ablation study to evaluate the effectiveness of the different components of our model.