Paper Review

KGAT: Knowledge Graph Attention Network
M1 Yang Boming

1

Background

Knwoledge Graph

1. Heterogeneous graph: node or
two or more types of edges.
2. node becomes an entity andedge becomes a relationship.
3. Traditional Graph G = (V , E)
Knowledge Graph
G = (V , E, R, T )Nodes, Edges, Node type,Relation type.

https://velog.io/@jbeen2/CS224W-17.-Reasoning-over-Knowledge-Graphs2Abstract

Knowledge Graph Attention Network(KGAT)

1. Provide better recommendation with item side information.
2. KGAT model, which achieves high-order relation modeling in an explicit and end-topend manner under GNN.
3. Conduct extensive experiments to demonstrate the effectiveness pf KGAT and ites
interpretability

3

Task Formulation

Collaborative Knowledge Graph (CKG)

CKG: which encodes user behaviors and item knowledge as a unified relational graph.

http://staff.ustc.edu.cn/~hexn/slides/kdd19-KGAT-poster.pdf

4Introduction

Summary & Limitations of Three-type Works

http://staff.ustc.edu.cn/~hexn/slides/kdd19-KGAT-poster.pdf

5

Methodology

Overview of KGAT

6

MethodologyCKG Embedding Layer

We adopy TransR to parameterize entities andrelations of CKG as vector represetations,considering direct connectivity of each triple(h, r, t):g(h, r, t) =∥ Wr eh + er − Wr et ∥22​

​

​

​

​

LKG = ∑(h,r,t,t′ )∈T − ln σ(g(h, r, t′ ) −g(h, r, t))​​7

Methodology

Attentive Embedding Propagation Layer 1/2

1. Information Propagation: Perform information propagation between an entity h andits neighbors Nh​

eNh = ∑(h,r,t)∈Nh π(h, r, t)et​​​​​where π(h, r, t) controls how much information being propagated from tail entity t tohead entity h conditioned to relation r .
2. Knowledge-aware Attention: Implement π(h, r, t) via relational attention
mechanism, which is formulated as follows:
π(h, r, t) = (Wr et )⊤tanh((Wr eh + er ))
​​

​

​​

8

Methodology

Attentive Embedding Propagation Layer 2/2

3. Information Aggregation: The final phase is to aggregate the entity representation ehand its ego-network representations eN as the new representation of entity h :h​​​(1)eh = LeakyReLU (W1 (eh + eNh )) + LeakyReLU (W2 (eh ⊙ eNh ))
​​​​​

​​

​

​

4. High-order Propagation: Further stack more propagation layers to explore the highorder connectivity information, gathering the information propagated from the higherhop neighbors:(l−1)

eNh(l−1)​​= ∑(h,r,t)∈Nh π(h, r, t)et​

​

​

9

Methodology

Model Prediction

1. After performing L layers, obtain multiple representations for user node u, namely(1)(L)
eu , ⋯ , eu :​​(0)(L) ∗(0)(L)∗eu = eu ∥ ⋯ ∥ eu , ei = ei ∥ ⋯ ∥ ei​

​​​

​

​

2. Finally, conduct inner product of user and item representations, so as to predict theirmatching score:y^(u, i) = e∗u ⊤e∗i
LCF = ∑(u,i,j)∈O − ln σ(y^(u, i) − y^(u, j))
​

​

​

​

​

​

​

10

Methodology

Optimization

Finally, the objective function to learn the loss function of KG and the loss function of CF
joinlty, as follows:

LKGAT = LKG + LCF​​​

11

ResultOverall Performance Comparsion

KGAT consistently yields the best performance on all the datasets.

12

ResultInteraction Sparsity Levels

Sparsity issue usually limits the expressiveness of recommender systems.
(The number on x axis mean the interaction numbers per user)

13

Result

Case Study for Explainable Recommendation

Solid line in left subfigure from u208 to i4293 , has the highest attention score.
The explannation as The Last Colony is recommended since you have watched Old
Man's War written by the same author John Scalzi​​

14Conclusion

Conclusion

High-order connectivity with semantic relations in CKG for knowledge-aware
recommendation.
Core: the attentive embedding propagation layer, which adaptively propagates theembeddings from a node's neighbors to update the node's representation.
Extensive experiments on three real-world datasets demonstrate the rationality andeffectiveness of KGAT.

15

Coding Task

Pull Request

Implements other models
Trains other datasets
Paper review documents update
Dockerfile updates
Bugs and fixes

16