# Heterogeneous Graph Neural Network for Short Text Classification
## 4 Experiment
### Comparative Test  
The results in Table 2 show that the standard deviation of the deep learning-based models in group b is large, and the classification results are not stable. This is because these models are proposed for long text mining and cannot perform well in the face of short texts. The performance of the word embedding-based models in group c is different, which is consistent with the results of the above analysis. The word embedding-based model is limited in application and may not perform so well in all data. While the GNN model in group d is generally better than the other models in groups a, b, and c, which indicates that building graph models on text is effective. In the remaining four datasets, except the MR dataset, while the performance of the GNN model is the best, and GNN can accurately receive the information contained in the text and learn. Overall, PV-DM model and LSTM model perform poorly in prediction accuracy, and LSTM, pre-trained LSTM, and Bi-SLTM predict results with poor stability and high standard deviation. The TWPGCN model proposed in this paper is generally better than the existing models in terms of accuracy and standard deviation. The accuracy rates are improved by \(1.3\%,0.87\%\), \(0.65\%\), \(1.94\%\), and \(0.8\%\) in the five datasets respectively. On the R52 dataset the largest increase. The standard deviation is also more or less improved, only slightly higher in the R52 dataset, and the predicted results in other datasets are relatively stable. The reason is that, since the R52 dataset is used for text classification of 52 categories for short texts, it is normal for the results to sway slightly.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Group** & **Model** & **20 NG** & **R8** & **R52** & **Ohsumed** & **MR** \\ \hline a & TF-IDF + LR & \(0.8319\pm 0.0000\) & \(0.9374\pm 0.0000\) & \(0.8695\pm 0.0000\) & \(0.5466\pm 0.0000\) & \(0.7459\pm 0.0000\) \\ \hline \multirow{4}{*}{b} & CNN-rand & \(0.7693\pm 0.0061\) & \(0.9402\pm 0.0057\) & \(0.8537\pm 0.0047\) & \(0.4387\pm 0.0100\) & \(0.7498\pm 0.0070\) \\  & CNN-non-static & \(0.8215\pm 0.0052\) & \(0.9571\pm 0.0052\) & \(0.8759\pm 0.0048\) & \(0.5844\pm 0.0106\) & \(0.7555\pm 0.0072\) \\  & LSTM & \(0.6571\pm 0.0152\) & \(0.9368\pm 0.0082\) & \(0.8554\pm 0.0113\) & \(0.4113\pm 0.0117\) & \(0.7506\pm 0.0044\) \\  & LSTM (pretrain) & \(0.7543\pm 0.0172\) & \(0.9609\pm 0.0019\) & \(0.9048\pm 0.0086\) & \(0.5110\pm 0.0150\) & \(0.7733\pm 0.0089\) \\  & Bi-LSTM & \(0.7318\pm 0.0185\) & \(0.9631\pm 0.0033\) & \(0.9054\pm 0.0091\) & \(0.4927\pm 0.0107\) & \(0.7768\pm 0.0086\) \\ \hline \multirow{4}{*}{c} & PV-DBOW & \(0.7436\pm 0.0018\) & \(0.8587\pm 0.0010\) & \(0.7829\pm 0.0011\) & \(0.4665\pm 0.0019\) & \(0.6109\pm 0.0010\) \\  & PC-DM & \(0.5114\pm 0.0022\) & \(0.5207\pm 0.0004\) & \(0.4492\pm 0.0005\) & \(0.2950\pm 0.0007\) & \(0.5947\pm 0.0038\) \\ \cline{1-1}  & PTE & \(0.7674\pm 0.0029\) & \(0.9669\pm 0.0013\) & \(0.9071\pm 0.0014\) & \(0.5358\pm 0.0029\) & \(0.7023\pm 0.0036\) \\ \cline{1-1}  & fastText & \(0.7938\pm 0.0030\) & \(0.9613\pm 0.0021\) & \(0.9281\pm 0.0009\) & \(0.5770\pm 0.0049\) & \(0.7514\pm 0.0020\) \\ \cline{1-1}  & fastText (bigrams) & \(0.7967\pm 0.0029\) & \(0.9474\pm 0.0011\) & \(0.9099\pm 0.0005\) & \(0.5569\pm 0.0039\) & \(0.7624\pm 0.0012\) \\ \cline{1-1}  & SWEM & \(0.8516\pm 0.0029\) & \(0.9532\pm 0.0026\) & \(0.9294\pm 0.0024\) & \(0.6312\pm 0.0055\) & \(0.7665\pm 0.0063\) \\ \cline{1-1}  & LEAM & \(0.8191\pm 0.0024\) & \(0.9331\pm 0.0024\) & \(0.9184\pm 0.0023\) & \(0.5858\pm 0.0079\) & \(0.7695\pm 0.0045\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Test accuracy on document classification task. We run all models 10 times and report mean \(\pm\) standard deviation.