# BTCAM: Attention-Based BiLSTM for Imbalanced Classification of Political Inquiry Messages
## 4 Experiment
### Results and Discussion  
#### 4.4.1 Comparison of Baseline Models (RQ1)  
A comparison of the results of BTCAM on POData with six baselines is shown in Table 4. In the table, FastText, DPCNN, TextCNN, TextRNN, BiLSTM_ATT, and BERT represent the performance before data augmentation. TDA\(+\)BiLSTM_ATT, TDA\(+\)DPCNN, TDA\(+\)FastText, TDA\(+\)TextCNN, TDA\(+\)TextRNN, and TDA\(+\)BERT represent the performance after data augmentation using the proposed TDA algorithm. Figure 8 shows the accuracy and loss during the iterative process of the training of each model. Based on a comprehensive analysis of the figures and tables, it is easy to find the following:  
* Without data augmentation, the FastText, DPCNN, and TextCNN models have poor performance on the training dataset and validation dataset. Although TextRNN, BiLSTM_ATT, and BERT improve the accuracy of the model on the training dataset, the recall was only 0.632 and 0.651. * After using the proposed TDA algorithm to increase the number of minority categories, the accuracy of the model on the training dataset is greater than 0.9, and the accuracy and recall on the validation dataset are both greater than 0.8. The performance of the models with TDA has improved greatly. * Compared with the baseline, the model converges faster and achieves the best performance. On the training dataset, the accuracy of BTCAM reached 0.996, and the loss value dropped to 0.03. On the validation dataset, the precision and recall of BTCAM were both greater than 0.93. * To further verify the generalizability of BTCAM, additional experiments were conducted on the public THUCNews dataset, a widely used benchmark for Chinese short-text classification. As shown in Table 5, BTCAM outperforms all baselines by a significant margin, thus confirming its adaptability beyond government-specific data. Table 6 is the classification report of the BTCAM model on the validation dataset. The table shows the precision, recall, and F1 values of different categories. It is easy to find that the model's prediction performance for all categories is above 90%, which is the best performance so far. \begin{table}
\begin{tabular}{l c c c} \hline  & **Precision** & **Recall** & **F1** \\ \hline Labor and social security & 0.912 & 0.956 & 0.932 \\ \hline Environmental protection & 0.983 & 0.913 & 0.943 \\ \hline Transportation & 0.911 & 0.926 & 0.917 \\ \hline Business and tourism & 0.932 & 0.922 & 0.928 \\ \hline Education and sports & 0.930 & 0.924 & 0.926 \\ \hline Urban and rural construction & 0.935 & 0.948 & 0.934 \\ \hline Health and birth control & 0.938 & 0.934 & 0.930 \\ \hline \end{tabular}
\end{table} 
Table 6: Precision, recall, and F1 of different categories on POData. \begin{table}
\begin{tabular}{l c c c c} \hline  & & **Train\_data** & & **Val\_data** \\ \cline{2-5}
**Model** & **Loss** & **Accuracy** & **Accuracy** & **Recall** \\ \hline FastText & 1.102 & 0.745 & 0.732 & 0.509 \\ \hline DPCNN & 0.915 & 0.801 & 0.789 & 0.667 \\ \hline TextCNN & 0.883 & 0.832 & 0.818 & 0.693 \\ \hline TextRNN & 0.508 & 0.865 & 0.841 & 0.746 \\ \hline BiLSTM\_ATT & 0.335 & 0.892 & 0.872 & 0.751 \\ \hline BERT & 0.301 & 0.910 & 0.901 & 0.814 \\ \hline TDA+FastText & 0.295 & 0.915 & 0.892 & 0.875 \\ \hline TDA+DPCNN & 0.228 & 0.928 & 0.910 & 0.892 \\ \hline TDA+TextCNN & 0.098 & 0.948 & 0.925 & 0.914 \\ \hline TDA+TextRNN & 0.051 & 0.961 & 0.938 & 0.927 \\ \hline TDA+BiLSTM\_ATT & 0.048 & 0.963 & 0.946 & 0.935 \\ \hline TDA+BERT & 0.042 & 0.975 & 0.952 & 0.941 \\ \hline BTCAM (ours) & **0.035** & **0.981** & **0.962** & **0.951** \\ \hline \end{tabular}
\end{table} 
Table 5: The performance of different models on THUCNews.