# Heterogeneous Graph Purification Network: Purifying Noisy Heterogeneity without Metapaths
## 5 Experiments
### Ablation Study  
In this section, we remove or replace various novel designs of HGPN to analyze the impact of each component. The study is conducted on OGB-MAG without any additional training skills due to its higher difficulty and larger scale.  
Table 4 presents the results of our ablation study on the HGPN model design, where we consider five variants of HGPN. Three of them remove the heterogeneous structure purification\" module, the adaptive residual update\" module, and the \"cluster-aware loss\" module, respectively. One uses mean-pooling instead of an attention mechanism for intra-subgraph aggregation. The last variant adopts R-GCN style of modeling heterogeneity to use a weight matrix for each relation rather than subgraph decomposition.  
As shown in Table 4, each novel method proposed in HGPN has a positive impact on final performance. Specifically, the cluster-aware loss\" module brings the least increase, which can be explained by the fact that type-specific projection matrices and batch normalization can similarly help maintain distinct node representation. For the cluster-aware  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Methods** & **Test Accuracy** & **Valid Accuracy** & **Training Skills** \\ \hline R-GCN & \(47.37\pm 0.48\) & \(48.35\pm 0.36\) & - \\ HetGNN & \(47.81\pm 0.29\) & \(49.12\pm 0.25\) & - \\ HGT & \(49.82\pm 0.13\) & \(51.24\pm 0.46\) & - \\ R-HGNN & \(52.04\pm 0.26\) & \(53.61\pm 0.22\) & - \\ NARS & \(52.40\pm 0.16\) & \(53.72\pm 0.09\) & - \\ HGPN (Ours) & \(\mathbf{54.47}\pm 0.15\) & \(\mathbf{55.72}\pm 0.17\) & - \\ \hline GAMLP & \(53.96\pm 0.18\) & \(55.48\pm 0.08\) & label input \\ HSAGN & \(53.95\pm 0.14\) & \(55.52\pm 0.16\) & label input \\ HGPN (Ours) & \(\mathbf{55.56}\pm 0.17\) & \(\mathbf{57.43}\pm 0.21\) & label input \\ \hline GAMLP + RLU & \(55.90\pm 0.27\) & \(57.02\pm 0.41\) & label input + multi-stage \\ HSAGN + SLE & \(54.40\pm 0.15\) & \(57.87\pm 0.12\) & label input + multi-stage \\ HGPN (Ours) & \(\mathbf{58.12}\pm 0.11\) & \(\mathbf{59.22}\pm 0.14\) & label input + multi-stage \\ \hline \hline \end{tabular}
\end{table}
Table 3: Overall performances on OGB-MAG dataset. The numbers in bold mean the best results. Vacant positions (“-”) mean that the method does not adopt any additional training skills. We run each method \(10\) times and report the average performances.  
loss\" module and the \"heterogeneous structure purification\" module, we provide detailed analysis of their effects in Section 5.6.  
We also execute experiments on the choices of normalization, the results of which are listed in Table 5. As we can see, our type-specific batch normalization achieves the best scores, layer normalization drops 0.83% while normal batch normalization [42] experiences a significant decrease of up to 10.95%. It demonstrates the significance of maintaining the respective representation distribution of nodes from different types. By comparison, layer normalization [43] does not contribute to this goal, while normal batch normalization is counterproductive to mix the representation of different types of nodes, leading to a serious decrease in the quality of performance.