# MBHAN: Motif-Based Heterogeneous Graph Attention Network
## 4 The Proposed Method
### Node-Level Attention Mechanism  
Before aggregating different motif subgraphs for different aspects of the nodes' representations, we first focus on the different roles played by the nodes' neighbors in each motif subgraph. To that end, we will begin by looking at the significance features aggregated by the neighbors of each node in a specific motif subgraph. MBHAN employs a self-attention mechanism [10] that can learn the weights between different nodes. Given a motif subgraph, \(\mathcal{G}_{m}\), containing a pair of nodes, \((v_{i},v_{j})\), the node-level attention, \(e_{ij}^{m}\), can be defined as follows:  
\[e_{ij}^{m}=att_{node-level}(h_{i},h_{j};m)\,, \tag{1}\]  
where \(e_{ij}\) denotes the importance of node \(v_{j}\) to node \(v_{i}\); \(att_{node-level}\) refers to the deep neural network that performs the node-level attention [16]; and, for a given motif subgraph, \(\mathcal{G}_{m}\), \(att_{node-level}\) is shared for all its node pairs. From Equation (1), it can be seen that the attention level of the node pairs \((v_{i},v_{j})\) within the motif subgraph depends on their features. Unlike the approach adopted in [19], the types of node pairs are not necessarily the same in MBHAN. For example, in the \"\(A-C-B\)\" motif pattern subgraph in Figure 2, a node of type \(A\) is represented by aggregating the features of its first-order neighbors, which are type \(C\) nodes. The feature space transformations for different types of nodes will be presented in Section 4.3. Note also that \(v_{i}\) and \(v_{j}\) are asymmetric. So, the degree of importance of node \(v_{i}\) to node \(v_{j}\) is not necessarily the same as the degree of importance of node \(v_{j}\) to node \(v_{i}\). This is a fundamental property of heterogeneous graphs. Thus, Equation (1) can be more precisely expressed as:  
\[e_{ij}^{m}=\sigma\Big{(}a_{m}^{T}[h_{i}||h_{j}]\Big{)}\,, \tag{2}\]  
where \(h_{i}\) denotes the features vector of node \(v_{i}\); \(\sigma\) denotes the activation function (LeakyReLU) was selected in this case); \(a_{m}\) denotes the node-level attention vector based on the motif subgraph, \(\mathcal{G}_{m}\); and \(||\) denotes the concatenate vector operation. After obtaining the initial attention scores of all the first-order neighbors of node \(v_{i}\), normalization is undertaken using  
Figure 3: The MBHAN framework. The different colors in the heterogeneous graph represent different node types. The different colors in the hierarchical attention mechanism represent different levels of attention. a SoftMax function, and this gives the attention weights, \(\alpha_{ij}\), for the degree of importance, \(e_{ij}\), from node \(v_{j}\) to node \(v_{i}\):  
\[\alpha_{ij}^{m}=\frac{\exp\left(\sigma\big{(}a_{m}^{T}\big{[}h_{i} \big{|}|h_{j}\big{]}\big{)}\right)}{\sum_{n\in\mathcal{N}_{i}^{m}}\exp(\sigma (a_{m}^{T}\big{[}h_{i}\big{|}|h_{n}\big{]}))}, \tag{3}\]  
where \(\mathcal{N}_{i}^{m}\) denotes the neighboring nodes of node \(v_{i}\) in \(\mathcal{G}_{m}\). It should be noted that \(\alpha_{ij}\) is not symmetric, so nodes \(v_{i}\) and \(v_{j}\) do not contribute to each other equally. This is not only because of the order of the vector concatenation on the numerator in Equation (3), but also because they have different neighbors. The embedding vector of node \(v_{i}\) in the motif subgraph \(\mathcal{G}_{m}\) is now an attention-based weighted aggregation of its neighboring nodes' features:  
\[z_{i}^{m}=\sigma\left(\sum_{j\in\mathcal{N}_{i}^{m}}\alpha_{ij}^{m}h_{j} \right), \tag{4}\]  
As the attention weight, \(\alpha_{ij}^{m}\), is based on the specific motif subgraph, \(\mathcal{G}_{m}\), it reflects only the side profile of the node representations in a specific motif pattern. This is also because the non-Euclidean spatial properties of graph data, especially heterogeneous graph data, exhibit high levels of variance [19]. To enrich the ability of the model and stabilize the training process, MBHAN extends the node-level attention to multi-head attention so as to avoid overfitting.