# Heterogeneous Graph Purification Network: Purifying Noisy Heterogeneity without Metapaths
## 5 Experiments
### Further Analysis  
#### 5.6.1 Parameter Sensitivity Analysis  
In this section, we analyze the sensitivity of different hyperparameters to investigate the robustness of our HGPN model. Experiments are also conducted on OGB-MAG and the results are reported in Figure 2.  
**Number of attention heads:** We fix the dimension of node representation to 512 and vary the number of attention heads from 1 to 16. Figure 2a shows the results. The best performance is achieved when setting the number of heads to 8, which reveals the effect of the multi-head attention mechanism to make training more stable. Thus, we use the same setting of attention heads and representation dimension on all compared baselines.  
\begin{table}
\begin{tabular}{l c c} \hline \hline
**Methods** & **Accuracy** & **Decrease** \\ \hline HGPN & \(54.47\pm 0.15\) & 0 \\ - Structure Purification & \(53.87\pm 0.12\) & \(-0.60\) \\ - Adaptive Residual & \(53.84\pm 0.18\) & \(-0.63\) \\ - Cluster-Aware Loss & \(54.08\pm 0.14\) & \(-0.39\) \\ - Attention Aggregation & \(53.31\pm 0.1\) & \(-1.16\) \\ - Subgraph Decomposition & \(52.74\pm 0.23\) & \(-1.73\) \\ \hline \hline \end{tabular}
\end{table}
Table 4: Results of ablation study on HGPN’s novel design in OGB-MAG dataset. We use normal residual connection and mean-pooling instead of “Adaptive Residual Update” and “Attention Aggregation”. “Subgraph Decomposition” means we adopt R-GCN style of modeling heterogeneity.  
\begin{table}
\begin{tabular}{l c c} \hline \hline
**Methods** & **Accuracy** & **Decrease** \\ \hline Type-specific Batch Norm & \(54.47\pm 0.15\) & 0 \\ Layer Norm & \(53.64\pm 0.12\) & \(-0.83\) \\ Normal Batch Norm & \(43.52\pm 0.35\) & \(-10.95\) \\ \hline \hline \end{tabular}
\end{table}
Table 5: Results of ablation study on normalization choices on OGB-MAG dataset. “Normal Batch Norm” means we use the same batch normalization layer for different types of nodes.  
Figure 2: Parameter Sensitivity Analysis Results on OGB-MAG.  
**Value of weighted coefficient of cluster-aware loss:** We investigate the effect of value of weighted coefficient of cluster-aware loss and the results are reported in Figure 2a. HGPN can obtain the top score when the value of the weighted coefficient is set to 0.3. We also find that the change of the weighted coefficient of cluster-aware loss has a slight influence on the final results. One plausible explanation is that cluster-aware loss is not a very strong constraint for HGPN and it is relatively easy to converge at an early stage.  
**Value of confidence threshold:** We investigated the influence of the confidence threshold value under the \"label input + multi-stage\" training setting and presented the results in Figure 2c. We observed that the optimal confidence threshold value was 0.6. Furthermore, even when the confidence threshold value was set to the worst value, our proposed HGPN still outperformed existing baselines, indicating that the superiority of our model was due to its novel design rather than parameter tuning.  
We did not analyze the number of layers as the limited GPU memory on OGB-MAG dataset allows sample-based models like HGPN to support no more than two layers. Nonetheless, this indicates the superior performance of our model with fewer layers compared to decoupled models such as NARS and GAMLP.