# Knowledge Graph Extrapolation Network with Transductive Learning for Recommendation
## 4 The Proposed Method
### Embedding Propagation
Specifically, the entities directly connected to an entity \(e\) is the first-order neighbors of the entity, as \(\mathcal{N}^{1}_{(e)}\). Furthermore, given a triple \((h,r,t)\), the tail entity \(t\) is one of the first-order neighbors of the head entity \(h\). By analogy, the \(l\)-order neighbors set of an entity is defined as \(\mathcal{N}^{l}_{(e)}\) at layer \(l\). The details as follows:  
\[\mathcal{N}^{l}_{(e)}=\{t|(h,r,t)\in\mathcal{G}_{2}\wedge\ h\in\mathcal{N}^{l -1}_{(e)}\}, \tag{3}\]  
where \(\mathcal{N}^{l-1}_{(e)}\) denotes the \((l-1)\)-order neighbors of the entity \(e\) at layer \(l-1\). The triplet set composed of all neighbors at layer \(l\) for entity \(e\) is shown below:  
\[\mathcal{T}^{l}_{(e)}=\{(h,r,t)|(h,r,t)\in\mathcal{G}_{2}\wedge\ h\in\mathcal{ N}^{l-1}_{(e)}\}. \tag{4}\]  
In the real scene, different tail entities have different contributions to feature representation of an entity under different conditions. For example, user \(u_{1}\) prefers a movie named Crouching Tiger, Hidden Dragon. The movie is both a martial arts film and a love story. However, user \(u_{1}\) actually prefers this movie because he likes martial arts films. Therefore, martial arts as the attribute of the film should have a higher weight than love story. Therefore, the attention mechanism is designed to learn the weights of different tail entities. The global embedding of the triple set at the \(l\)-th layer for the entity is defined as follows:  
\[\mathbf{e}^{l}=\sum_{(h,r,t)\in\mathcal{T}^{l}_{(e)}}\pi(h,r)\mathbf{e}^{l}, \tag{5}\]  
where \(\mathbf{e}^{l}\) is the embedding representation of the tail entity \(t\), \(\pi(h,r)\) is a scoring function used to compute the attention weight, which controls how much information from the tail entity is propagated to head entity in this case, where head entity is \(h\) and the relation is \(r\). The \(\pi(h,r)\) is implemented via the attention network, which is formulated as follows:  
\[\pi(h,r)=\sigma(\mathbf{W}_{2}\mathrm{ReLU}(\mathbf{W}_{1}\mathbf{z}_{0}+ \mathbf{b}_{1})+\mathbf{b}_{2}), \tag{6}\]  
\[\mathbf{z}_{0}=\mathrm{ReLU}\Big{(}\mathbf{W}_{0}\Big{(}\mathbf{e}^{h}\| \mathbf{r}\Big{)}+\mathbf{b}_{0}\Big{)}, \tag{7}\]where \(\pi(\cdot)\) is a multi-layer neural network. \(\mathbf{W}_{0}\), \(\mathbf{W}_{1}\) and \(\mathbf{W}_{2}\) are the trainable weight matrices of each layer, respectively. \(\mathbf{b}_{0}\), \(\mathbf{b}_{1}\) and \(\mathbf{b}_{2}\) are the biases. \(\sigma(\cdot)\) is the last nonlinear activation function, which is Sigmoid. ReLU is the activation function except for the last layer. Then, the softmax function is applied to normalize the coefficients across all triples, as follows:  
\[\pi(h,r)=\frac{\exp(\pi(h,r))}{\sum_{(h^{\prime},r^{\prime})\in\mathcal{T}_{e} ^{I}}\exp(\pi(h^{\prime},r^{\prime}))}. \tag{8}\]  
The attention score function is capable of attracting more attention to the neighboring tail entity that has more influence. The method distills more accurate association information. Finally, the representations of entities of other layers are computed according to Equation (5), and the representations set from layer 1 to layer L are obtained for the entity \(e\):  
\[\varphi=\{\mathbf{e}^{(1)},\mathbf{e}^{(2)},\ldots,\mathbf{e}^{(L)}\}, \tag{9}\]  
So, the representation sets of user and item are obtained as \(\varphi_{u}\) and \(\varphi_{v}\). However, there is still an issue to note, which is that the importance of the initial entity set should not be ignored. In particular, the initial entity set of the user is directly related to the user and represents the user's preferred items. It directly reflects user interest and, for items, the initial entity set is a set of collaborative neighborhoods, which contains collaboration information and strong interaction information. Thus, the initial entity sets of users and items are treated as their 0-order neighborhood, respectively.