# Heterogeneous Graph-Convolution-Network-Based Short-Text Classification
## 4 Experiment and Performance Analysis
### Experimental Results and Analysis  
The effects of the sliding window size and word embedding dimension on the accuracy of the model were tested. With other parameters unchanged, the size of the sliding window was set as 5, 10, 15, 20, 25, or 30. The SHGCN model was used to conduct experiments on the test set of MR And AGNews datasets. The results are shown in Figure 2. With the increase in the size of the sliding window, the accuracy rate increases at first and then \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multirow{2}{*}{\begin{tabular}{c} **Training** \\ **Set** \\ \end{tabular} } & \multirow{2}{*}{**Test Set**} & \multirow{2}{*}{**Category**} & \multicolumn{2}{c}{**Average**} & \multicolumn{2}{c}{**Total**} & \multicolumn{2}{c}{**Average**} & \multirow{2}{*}{\begin{tabular}{c} **Category** \\ **Proportion** \\ \end{tabular} } \\  & & & & **Entities** & & & **Words** & & **Words** \\ \hline AGNews & 120,000 & 7600 & 4 & 5.59 & 23.36 & 302,210 & 37.78 & \
\begin{tabular}{c} world: 0.25 \\ sports: 0.25 \\ business: 0.25 \\ science: 0.25 \\ trade: 0.036 \\ earn: 0.431 \\ jobs: 0.005 \\ ship: 0.016 \\ \end{tabular} \\ MR & 7108 & 3554 & 2 & 1.15 & 20.39 & 224,073 & 21.02 &
\begin{tabular}{c} positive: 0.5 \\ negative: 0.5 \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Summary statistics of datasets. decreases. This is because if the size of the sliding window is too small, the connections between long-distance words will be ignored, resulting in a failure to capture more co-occurrence information between words, and therefore, the performance of the model will be affected. On the contrary, if the size of the sliding window is too large, it will establish relations with some words with weak semantic relevance and generate unnecessary noise, which will eventually affect the classification results. With other parameters unchanged, the embedding dimension was set as 50, 100, 150, 200, 250, or 300 respectively. As shown in Figure 3, when the word embedding dimension is 150, the accuracy rate of SHGCN is the highest. If the word embedding dimensions are too small, the propagation of node information in the graph will be affected. On the contrary, too large word embedding dimensions will reduce the difference between feature words' embedding features, which will reduce the accuracy of SHGCN. To verify the impact of the learning rate on the accuracy of the test set, this paper tested the classification results of the AGNews dataset under different learning rates. As shown in Table 2, when the learning rate decreases, the classification accuracy of the test set increases while the running time of the model increases. Through experimental analysis, the word embedding dimension was set to 150, the learning rate was 0.001, the sliding window size was 20, and the dropout rate was 0.5. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \(\mathbf{10^{-1}}\) & \(\mathbf{10^{-2}}\) & \(\mathbf{10^{-3}}\) & \(\mathbf{10^{-5}}\) \\ \hline AGNews & 83.79 & 86.51 & 88.38 & 88.39 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of accuracy under different learning rates. Figure 3: Accuracies of SHGCN with different embedding dimensions. Figure 2: Accuracies of SHGCN with different sliding window sizes. Classification accuracy was used to measure the performance of all models. Each dataset was subjected to 50 experiments under each model, and the experimental results of seven models on three datasets are shown in Table 3.  
As shown in Table 3, on the AGNews and R52 datasets, the performance of the graph-based models is superior to that of the traditional CNN and BiLSTM models. This is because the graph structure allows information transfer between different types of neighbor nodes, which enables nodes to aggregate more information for feature representation.