Link prediction based on knowledge graph embedding (KGE) aims to predict new triples to complete knowledge graphs (KGs) automatically . However, recent KGE models tend to improve performance by excessively increasing vector dimensions, which would cause enormous training costs and save storage in practical applications. To address this problem, we first theoretically analyze the capacity of low-dimensional space for KG embeddings based on the principle of minimum entropy. Then , we propose  a novel knowledge distillation framework for knowledge graph embedding, utilizing multiple low-dimensional  KGE models as teachers  . Under a novel iterative distillation strategy, the MulDE modelproduces soft labels according to training epochs and student performance adaptively . The experimental results show that MulDE can effectively improve the performance and training speed of low-dimensional KGE models. The distilled 32-dimensional models are very competitive compared to some of state-or-the-art (SotA) high-dimensional methods on several commonly-used datasets.
Link prediction based on knowledge graph embeddings (KGE) aims to predict new triples to automatically construct knowledge graphs (KGs) . However, recent KGE models achieve performance improvements by excessively increasing the embedding dimensions, which may cause enormous training costs and require more storage space. In this paper, instead of training high-dimensional models , we propose MulDE, a novel knowledge distillation framework , which includes multiple low-dimensional hyperbolic KGE models as teachers and two student components, namely Junior and Senior . Under a novel iterative distillation strategy, the Junior component, a low-dimensional KGE model, asks teachers actively based on its preliminary prediction results, and the Senior component integrates teachers' knowledge adaptively to train the Junior component based on two mechanisms: relation-specific scaling and contrast attention . The experimental results show that MulDE can effectively improve the performance and training speed of low-dimensional KGE models. The distilled 32-dimensional model is competitive compared to the state-of-the-art high-dimensional methods on several widely-used datasets.
R
embedding
embeddings
R
complete
automatically construct
D
automatically
R
tend to improve performance
achieve performance improvements
R
vector
the embedding
R
would
may
R
save storage in practical applications. To address this problem, we first theoretically analyze the capacity of low-dimensional space for KG embeddings based on the principle of minimum entropy. Then
require more storage space. In this paper, instead of training high-dimensional models
A
MulDE,
R
for knowledge graph embedding, utilizing
, which includes
A
hyperbolic
A
and two student components, namely Junior and Senior
R
MulDE modelproduces soft labels according to training epochs and student performance adaptively
Junior component, a low-dimensional KGE model, asks teachers actively based on its preliminary prediction results, and the Senior component integrates teachers' knowledge adaptively to train the Junior component based on two mechanisms: relation-specific scaling and contrast attention
R
models are very
model is
R
some of state-or-the-art (SotA)
the state-of-the-art
R
commonly-used
widely-used