# Heterogeneous Graph Purification Network: Purifying Noisy Heterogeneity without Metapaths
## 4 Proposed Method
### Inter-Subgraph Aggregation  
After single-relation subgraph decomposition, in the subgraph containing edge type \(\psi(e)\), we assume the target node type is \(\phi(d)\) and the source node type is \(\phi(s)\). In the \(l\)-th layer inner-subgraph aggregation module, firstly we adopt the type-dependent matrices \(W_{\phi(s)}^{l}\) and \(W_{\phi(d)}^{l}\) to project nodes' representation \(X_{s}^{l-1}\) and \(X_{d}^{l-1}\) into the same semantic space for the aggregation calculation, as follows:  
\[\begin{split} X_{s}^{l}&=W_{\phi(s)}^{l}X_{s}^{l-1} \\ X_{d}^{l}&=W_{\phi(d)}^{l}X_{d}^{l-1}\end{split} \tag{2}\]  
In traditional heterogeneous GNNs, the model would indiscriminately aggregate all the source node information \(X_{s}^{l-1}\) to the target node \(X_{d}^{l-1}\), which is likely to cause harm to the downstream task in heterogeneous graph data with high noise structures. To give the model the ability to automatically learn to filter out noisy information and improve robustness, a parallel structure purification mechanism is introduced in the module. Firstly, we use another set of type-dependent projection matrices \(W_{s,\phi(s)}^{l,m}\) and \(W_{d,\phi(d)}^{l,m}\) to project the nodes into another semantic space for determining relevance:  
\[\begin{split} X_{s}^{l,m}&=W_{s\phi(s)}^{l,m}X_{s}^{ l-1}\\ X_{d}^{l,m}&=W_{d,\phi(d)}^{l,m}X_{d}^{l-1}\end{split} \tag{3}\]  
Then, the filter score \(F_{s,d}^{l}\) between each node pair \(X_{s}^{l,m}\) to \(X_{d}^{l,m}\) can be calculated by an additive attention layer, as follows:  
\[F_{s,d}^{l}=tanh(W^{l}LeakyReLU(X_{s}^{l,m}+X_{d}^{l,m}))-\gamma_{\psi(e)} \tag{4}\]where \(\gamma_{\psi(e)}\) is a learnable threshold corresponding to the edge type of subgraph \(\psi(e)\). Each \(\gamma_{\psi(e)}\) is initialized to 0. We introduce the parameter \(\gamma_{\psi(e)}\) because the quality of different edge types often varies significantly in practice. By incorporating this parameter with minimal increase in parameters, our model can learn to differentiate between noisy edge structures, taking into account edge heterogeneity. This allows us to obtain the final mask matrix \(M^{I}\), which ensures that the source node \(s\) will not have any effect on the target node \(d\) if \(F^{I}_{s,d}<0\):  
\[M^{I}_{s,d}=\begin{cases}F^{I}_{s,d}&\text{if }F^{I}_{s,d}\geq 0\\ F^{I}_{s,d}-\infty&\text{if }F^{I}_{s,d}<0\end{cases} \tag{5}\]  
Since the importance of different neighbor nodes cannot be treated equally, we apply attention operators for message aggregation along with the predefined mask matrix \(F^{I}\). Precisely, we calculate the attention score \(A^{I}_{s,d}\) of source node \(s\) to target node \(d\) by:  
\[A^{I}_{s,d}=\frac{exp(a^{T}LeakyReLU(X^{I}_{s}+X^{n}_{d})+M^{I}_{s,d})}{\sum_{s \in N_{\psi(e)}(d)}exp(a^{T}LeakyReLU(X^{I}_{s}+X^{I}_{d})+M^{I}_{s,d})} \tag{6}\]  
where \(N_{\psi(e)(d)}\) denotes the neighbor nodes of target node \(d\) with edge type \(\psi(e)\). Finally, we can compute a weighted average of the transformed features of the neighbor nodes as the aggregated representation \(Z^{I}_{e,d}\), using the normalized attention scores \(A^{I}_{s,d}\):  
\[Z^{I}_{e,d}=\sum_{s\in N_{\psi(e)}(d)}A^{I}_{s,d}X^{I}_{s} \tag{7}\]  
The multi-head attention mechanism [25] can make the training process more stable and allow attention to be integrated and learned in parallel across multiple subspaces, and it has already demonstrated strong performance in models of several domains. Thus, we apply the multi-head attention mechanism here, as shown, where \(||\) means concatenation and \(H\) means the number of attention heads:  
\[Z^{I}_{e,d}=||^{H}_{h=0}Z^{I,h}_{e,d} \tag{8}\]