# Heterogeneous Graph Purification Network: Purifying Noisy Heterogeneity without Metapaths
## 5 Experiments
### Further Analysis
#### 5.6.3 Detailed Analysis of Heterogeneous Structure Purification  
In Section 5.5, our ablation study has demonstrated the positive impact of our proposed heterogeneous structure purification mechanism. However, as this module introduces additional parameters and operations, further comparative experiments are necessary to identify the underlying reasons for the observed improvement. Therefore, we have designed three additional tests: \begin{table}
\begin{tabular}{l c c} \hline \hline
**Methods** & **Accuracy** & **Decrease** \\ \hline Ours & \(54.47\pm 0.15\) & \(0\) \\ - stopping mechanism & \(53.84\pm 0.17\) & \(-0.63\) \\ - learnable anchor & \(54.07\pm 0.13\) & \(-0.40\) \\ - Cluster-Aware Loss & \(54.08\pm 0.18\) & \(-0.39\) \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Results of detailed analysis on cluster-aware loss module on OGB-MAG. We use the mean node representation as class centers in the “learnable anchor” variant. Firstly, we aim to investigate the necessity of introducing parallel projection matrices for calculating filter scores. To this end, we have created a variant that reuses the attention score as the filter score. Secondly, we seek to exclude the possibility that the observed improvement is solely due to the introduction of extra model parameters, rather than the purification mechanism. We have achieved this by removing the mask method and simply adding filter scores to attention scores without truncation. Finally, as DropEdge [44] has been proven to be an effective regularization technique in GNNs, we have introduced a variant that randomly masks edges with a rate of 0.3 at each iteration to verify whether our structure purification mechanism outperforms random regularization. All experimental results can be found in Table 7.  
Table 7 shows that our method achieves significantly better results than the other variants. The \"share parameters\" variant also performs similarly well, while the \"only add\" and \"random drop\" variants perform worse compared to the variant without the structure purification mechanism. These results indicate that the observed improvement is indeed due to our proposed structure purification mechanism, which enables HGPN to detect and remove noisy edges in raw graph data. We can also conclude that parallel projection matrices are essential for determining noisy edges and neighbor information aggregation, as they focus on different aspects of the data. Additionally, the random DropEdge method has been found to have a negative effect on training stability. Table 8 presents the statistics of edges that HGPN learns to mask on OGB-MAG under two training settings, and Figure 3 shows the purification rate of each type of edge. From these statistics, we can make the following observations:  
(1) HGPN tends to mask edges such as \"affiliated with\" and its reverse edges, which are less relevant to the task of paper venue prediction and consistent with our prior knowledge. This demonstrates that our proposed structure purification method has the ability to identify useful graph structures with realistic causal significance. The results also explain the drawbacks of DropEdge, which can only mask all types of edges equally and thus may lose more useful information that high-quality edges provide. (2) Under the \"label input\" training setting, the quality of paper nodes' attributes increases due to the additional label information. As we can see, the structure purification method can automatically capture this change by reducing the masking rate of edges related to paper nodes, such as \"cites\", \"writes\", and \"has topic\". This phenomenon reveals that structure noise and feature noise are not independent, and feature quality is positively associated with structure quality. Therefore, our design of HGPN to both purify feature and structure can jointly help to obtain a clean and refined heterogeneous graph.