KGR efiner: Knowledge Graph Refinement for Improving Accuracy of Translational Link Prediction Methods (2022)

Abstract: Link Prediction is the task of predicting missing relations between knowledge graph entities (KG). Recent work in link prediction mainly attempted to adapt a model to increase link prediction accuracy by using more layers in neural network architecture, which heavily rely on computational resources. This paper proposes the refinement of knowledge graphs to perform link prediction operations more accurately using relatively fast translational models. Translational link prediction models have significantly less complexity than deep learning approaches; this motivated us to improve their accuracy. Our method uses the ontologies of knowledge graphs to add information as auxiliary nodes to the graph. Then, these auxiliary nodes are connected to ordinary nodes of the KG that contain auxiliary information in their hierarchy. Our experiments show that our method can significantly increase the performance of translational link prediction methods in Hit@10, Mean Rank, and Mean Reciprocal Rank.

## introduction knowledge graphs (kgs) represent a set of interconnected descriptions of entities, including objects, events, or concepts. these graphs are structures by which knowledge is captured in the form of triplets. these triplets consist of three parts: head, relation, and tail. the relation (edge) determines the type of relationship between head and tail nodes. despite many efforts to build kgs, they are far from completeness. one of the developing fields in completing kgs is link prediction (lp). lp tries to embed entities and relations in a small continuous vector space to predict missing links in kgs. in the last few years, deep learning approaches have significantly outperformed other methods in lp, but this accuracy came at the cost of computational complexity. translational lp models, such as transe @xcite , transh @xcite , transd @xcite , rotate @xcite , and hake @xcite , generally use a straightforward function over head and relation vectors to predict the tail based on distance @xcite . one advantage of translational methods over deep learning techniques is that their score function is considerably faster @xcite . since these models are less complex and more efficient, we tried to improve only these translational methods in this work. ontologies are concepts or properties to describe an object foot_0 . wordnet contains hierarchical ontology only for its entities. some work tried to use ontology components of wordnet to boost lp models. for example, grcluster @xcite treated ontology components as paths. it defined path similarity over entities in wordnet and slightly improved lp accuracy. nonetheless, gr-cluster only improved wnnh and wn18, which are not standard lp datasets @xcite . additionally, this work is limited to wordnet. freebase @xcite does not have any hierarchical path for its entity. on the other hand, its relations have a path hierarchy to explain edges. sacn @xcite exploited additional information of fb15k-237 as auxiliary nodes and created fb15k-237-attr. nevertheless, it added numerous nodes to the kg, which makes the method for creating fb15k-237-attr inefficient for more extensive graphs. likewise, this method can only be applied to freebase. translational lp models, such as transe, ro-tate, or transd, when trying to learn the relation between paris and france, neglect that paris is a city and france is a country. we introduce ontology components as auxiliary nodes. these auxiliary nodes are connected to related entities that have these components in their hierarchy. for example, we added an extra node "country" to kg and connected it to all the countries in the kg. our contributions are as follows: firstly, we presented a method for refining kgs that have ontology. our approach adds auxiliary nodes and embeds similar nodes closer in the embedding space, which increases the accuracy of translational link prediction with the same time and space complexity of translational models. secondly, we used state-of-the-art translational models to evaluate our method on two fb15k-237 and wn18rr. the results showed that accuracy in link prediction was significantly increased on h@10, mrr, and mr, especially on wn18rr. ## related work we divided related work into four categories. first, translational models, such as transe @xcite , transh @xcite , transd @xcite , hake @xcite , are distance-based algorithms that use a straightforward operation over head and relation (mainly summation and/or a projection into a secondary space) to measure the distance to the tail entity. some work has been introduced over these fast translational models to improve their performance by using hierarchical information. tkrl @xcite used components of hierarchical structure as a transition to transform kg nodes into secondary space and then performed lp. grcluster @xcite used path similarity over entities in wordnet and slightly improved link prediction accuracy. sacn @xcite proposed fb15k-237_attr that has external resources as triplets (new nodes and edges) to improve the result. grcluster could not improve the wn18rr, and it is limited to kgs that have ontology for their entities. sacn improved fb15k-237 by creating fb15k-237_attr, but it added many nodes and edges. nonetheless, the sacn attribute creator could not be applied to wn18rr. transc @xcite brought similar entities closer in the embedding space and improved lp in yago, but experiment results show no improvements on wordnet or freebase. our work is similar to this category; it is fast and uses translational models as a core. we pushed the limitation of transc to have a better lp result on freebase and wordnet. second, mostly deep models adapt an architec-ture and rarely use anthologies in their main model. for example, conve @xcite used 2d convolution, bert-resnet @xcite and kg-bert @xcite employed bert, sacn @xcite utilized wgcn in its architecture. these models are more accurate but computationally costly. thirdly, kg refinement is a sub-field of kg enhancement. refinement can be done by either adding information to the graph or removing incorrect data @xcite . biokg @xcite worked on medical kgs and has tried to provide a method for removing the inaccurate information in these graphs. in this work, like sacn, we added auxiliary nodes to kgs. these nodes are extracted from ontology hierarchy levels of nodes and edges of kgs. lastly, some works introduced similarities over entities or relations. for example, hrs @xcite presented relation-cluster and subrelations in the scoring function of translational models. it created sub-relations and relationclusters based on clustering results of transe relations; however, it cannot utilize ontology nor improve wn18rr results. for entity similarity, ete @xcite considered that if two entities are embedded closely in the embedding space, they are similar and assigned classes to entities based on closeness. unlike ete, our hypothesis is that if two entities use the same relation type in the graph or have common elements in their hierarchies, they are related. we exploited these affiliations (share hierarchical components) by connecting ordinary nodes to their auxiliary nodes if a node has the auxiliary node in its ontology components. the main distinctions between our work and related work are: first, our method works with any kg with ontology, and it does not matter if it has the hierarchical ontology for nodes or edges. second, it uses translational models; therefore, it has high speed and less time to train these models (see ## kgrefiner in this work, we propose a method that uses ontology as an auxiliary node, which refines the kg and increases lp accuracy. these auxiliary nodes can be obtained from the edges of kg or its nodes. for example, in fb15k-237, we do this refinement by using hierarchies of relations, and in wn18rr, we use hierarchies of entities. we add repetitive com-ponents of hierarchies to kgs as new (auxiliary) nodes. then, we introduce a few new relations to connect these auxiliary nodes to other kg nodes. these auxiliary nodes operate like a magnet for similar entities; they drag similar entities (those entities that share ontology components) together in the embedding space. this closeness of similar entities causes the translational models to prioritize their search for a specific place in the embedding space ( e.g., searching between countries when asked what country's capital city is paris, in evaluation). the rest of this section is dedicated to the proof of this assertion. translational link prediction methods, such as transe, create transition property in their embeddings. for example, in transe, embeddings are made as ⃗ e s + ⃗ r ≈ ⃗ e o . this means the tail entity should be close to the sum of head and relation in embedding space. let us consider n entities share an ontology component o in their hierarchy. if we add o to the kg and connect the o to those n entities, the following optimization will happen in transe: the loss function minimizes the distance between two sides of equations: in the implementation, they are optimized batchwised. also, assume it uses the l1 norm as a distance measure. therefore, the batch loss will be: ## conclusion and future work in this work, we propose kgrefiner, a kg refinement method that alleviates the limitations of translational models by capturing additional information in knowledge graph hierarchies. we used hierarchy components as auxiliary nodes. refined kg comes by connecting these auxiliary nodes to proper entities. our empirical results show that our kgrefiner outperforms other state-of-the-art translational models and data augmentation methods on wn18rr. some models' performance improved on fb15k-237 but was not as good as wn18rr. furthermore, it is the first augmentation method that works with both wordnet and freebase, while old methods only perform only on one dataset. in our work, we had to manually determine the depth cut of hierarchy and minimum repetition for ontology components extraction. in future works, we will automate these two elements, so the model determines each component. additionally, kgrefiner cannot improve the accuracy of deep learning methods; therefore, another study is needed to enhance deep models by using ontological information.