# GRNN: Graph-Retraining Neural Network for Semi-Supervised Node Classification
## 5 Experiments and Discussion
### Experimental Settings  
#### 5.3.1 The Citation Network  
A two-layer GRNN model was used on Cora and PubMed for semi-supervised node classification. The first layer was spliced into \(C=8\) heads, and each head had \(p_{1}=16\) features, which means that the first layer had a total of 128 features. In addition, the nonlinear activation function of the first layer was assumed to be an exponential linear unit. The second layer was used as the output layer for classification, which was only a single head that had \(q\) features. It is worth noting that only a one-layer GRNN model was used on Citeseer, and the architectural hyperparameters of GRNN were that the output layer averaged \(C=8\) head outputs and each head had \(q\) features. Finally, the nonlinear activation function of the second layer was assumed to be a softmax activation to normalize  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Attributes** & **Cora** & **Citeseer** & **PubMed** & **Wiki** & **Axiv** & **PPI** \\ \hline Nodes & 2708 & 3327 & 19,717 & 2405 & 169,343 & 56,944 (24 graphs) \\ Edges & 5429 & 4732 & 44,338 & 11,596 & 1,166,243 & 818,716 \\ Features & 1433 & 3703 & 500 & 4973 & 128 & 50 \\ Class & 7 & 6 & 3 & 17 & 40 & 121 (multilabel) \\ Train Nodes & 140 & 120 & 60 & - & 90,941 & 44,906 (20 graphs) \\ Validation Nodes & 500 & 500 & 500 & - & 29,799 & 6514 (2 graphs) \\ Test Nodes & 1000 & 1000 & 1000 & - & 48,603 & 5514 (2 graphs) \\ Label Rate & 0.052 & 0.036 & 0.003 & - & 0.537 & 0.789 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Dataset statistics.  
the results. During training, the Adam SGD optimizer with an initial learning rate of 0.01 as well as \(L_{2}\) regularization with 0.0005 was used. Furthermore, the dropout of nodes with \(\alpha=0.5\) was applied to the input layer, and the dropout of edges with \(\beta=0.5\) was applied to each layer. The architectural hyperparameters of GRNN on PubMed were a little different from the above architectural hyperparameters: The second layer averaged \(C=8\) head outputs and each head had \(q\) features, and \(L_{2}\) regularization was assumed to be 0.001, and the dropout of edges was \(\beta=0\). Last but not least, an early stopping strategy with a patience value of 200 epochs was applied for each iteration that alternated between a learning procedure and an inference procedure. The final accuracy of Cora and Citeseer was achieved on the 20 th iteration, and the final accuracy of PubMed corresponding to the highest validation accuracy was achieved in 20 iterations.  
For GCN+GRNN and GCNII+GRNN, we replaced the benchmark neural network of the GRNN model with GCN and GCNII, respectively, and the hyperparameters settings of GCN and GCNII were consistent with the recommendations in the original paper. For Cora, CiteSeer, and PubMed, the depths of GCN were 2, 2, and 2, respectively, while the depths of GCNII were 64, 32, and 16, respectively. The GCN+GRNN model used the same early stopping strategy as the GRNN model, while the GCNII+GRNN model used the early stopping strategy through which the final accuracy corresponding to the highest validation accuracy was achieved in 20 iterations.  
For the large-scale citation network Arxiv, we used a four-layer network GCN with 16 hidden units and the Adam SGD optimizer with an initial learning rate of 0.01 and \(L_{2}\) regularization with 0. Moreover, the nonlinear activation function of the first layer was assumed as an exponential linear unit. The GCN+GRNN model used the early stopping strategy with which the final accuracy corresponding to the highest validation accuracy was achieved in 10 iterations.