# MSGCN: Multi-Subgraph Based Heterogeneous Graph Convolution Network Embedding
## 5 Application of the Model
### Inductive Learning Task
Then we use the softmax function to normalize all the neighbors:  
\[e_{ij}=a(W\vec{h_{i}},W\vec{h_{j}}) \tag{10}\]  
\[\alpha_{ij}=softmax_{j}(e_{ij})=\frac{exp(LeakyReLU(\vec{a}^{T}[W\vec{h_{i}} ||W\vec{h_{j}}]))}{\sum_{k\in N_{i}}exp(LeakyReLU(\vec{a}^{T}[W\vec{h_{i}} ||W\vec{h_{j}}]))} \tag{11}\]  
Figure 3: Inductive learning task. Here the node embeddings of new nodes are achieved by aggregating existing nodes’ embeddings. (**a**) assumes that all neighbors are of the same role, while (**b**) first distinguishes which nodes are important, thereby assigning greater weights during aggregation. where \(h\) is the node feature, \(N_{i}\) is the neighborhood of node \(v_{i}\) in the graph, \(W\) and \(a\) are trainable parameters. Then a matrix, \(W_{att}\in R^{|V|\times|V|}\), of attention coefficient is achieved. Here we choose \(LeakyReLU\) as the activation function. \(LeakyReLU\) is very similar to \(ReLU\), they can solve the gradient vanishing problem. When the input \(x\) is less than 0, \(ReLU(x)\) will be 0, while \(LeakyReLU(x)\) will be a negative value. The benefit of using \(LeakyReLU\) is that the gradient can also be calculated for the part of the input that is less than 0 during the backpropagation process, avoiding the gradient direction sawtooth problem. So we choose \(LeakyReLU\) as the activation function. For each node, we choose its top \(k\) important neighbors, thus we can get a new matrix \(M\in R^{|V|\times k}\), which records the index of the \(k\) most important neighbors of each node. The value of \(k\) is not suitable to be large, because many nodes have few neighbors and too many unimportant nodes will be taken into account, which is similar to using the degree matrix directly. At the same time, it should not be too small, so that it can include as many important nodes as possible. \[M_{i}=argtpk(W_{att}[i,:]) \tag{12}\]  
A node is important if it appears many times in the matrix as an important neighbor. We count the occurrence times of each node, and the node exceeding a threshold value is regarded as an important node. We set the weight to \(w_{1}\) for important nodes and \(w_{2}\) for unimportant nodes. For a new node \(v_{i}\), as we say before, we sample \(m\) neighbors. Here we have \(m_{1}\) important neighbors and \(m_{2}\) unimportant neighbors, and \(m1+m2=m\). The representation can be achieved in Equation (13):  
\[H_{isample}=\frac{(H[A_{isample}[1]+...H[A_{isample}[m_{1}]])w_{1}+(H[A_{isample }[1]+...H[A_{isample}[m_{2}]])w_{2}}{m_{1}w_{1}+m_{2}w_{2}} \tag{13}\]  
Since the representations of new nodes are obtained based on existing nodes, it is reasonable that the accuracy will not outperform the MSGCN framework. However, it is worth sacrificing a little accuracy for time. Although the accuracy is slightly lower than the direct training of the MSGCN model, in fact, the accuracy is still higher than most models. However, as time goes by, the latest nodes in the future will be neighbors of those nodes with lower accuracy, so the accuracy will further decline. However, the number of new nodes in our experiment is the same as the number of trained nodes in MSGCN, so the effect of the inductive task can still be guaranteed in the short term. However, in order to maintain accuracy, when there are enough new nodes, it is better to use the MSGCN model to train all the data.