Multilingual Knowledge Graph Completion with Language-Sensitive Multi-Graph Attention (2023)

Abstract: Multilingual Knowledge Graph Completion (KGC) aims to predict missing links with multilingual knowledge graphs. However, existing approaches suffer from two main drawbacks: (a) alignment dependency: the multilingual KGC is always realized with joint entity or relation alignment, which introduces additional alignment models and increases the complexity of the whole framework; (b) training inefficiency: the trained model will only be used for the completion of one target KG, although the data from all KGs are used simultaneously. To address these drawbacks, we propose a novel multilingual KGC framework with language-sensitive multi-graph attention such that the missing links on all given KGs can be inferred by a universal knowledge completion model. Specifically, we first build a relational graph neural network by sharing the embeddings of aligned nodes to transfer language-independent knowledge. Meanwhile, a language-sensitive multi-graph attention (LSMGA) is proposed to deal with the information inconsistency among different KGs. Experimental results show that our model achieves significant improvements on the DBP-5L and E-PKG datasets.

## introduction knowledge graphs (kgs) with plentiful structured semantic information have been widely used in various nlp applications such as question answering @xcite , recommender systems @xcite and information extraction @xcite . due to the well-known incompleteness of kg, the task of kg completion seeks to facilitate the automatic construction of knowledge graphs by predicting missing links @xcite @xcite ; wang et al., 2022a). recently, there has been a lot of interest in improving kg completion by leveraging kgs from different languages. known as multilingual knowledge graph completion (kgc), various attempts have been made to transfer knowledge from one kg to another, such as kens @xcite , alignkgc @xcite , and ss-aga @xcite . and these studies have demonstrated that it is viable to improve the single kgc task by utilizing information from other language-specific kgs. however, the methods for multilingual kgc mainly involve two shortcomings. the first one is referred to be alignment dependency, indicating that in previous frameworks, the multilingual kgc task has to be carried out in conjunction with entity or relation alignment tasks. this leads to a cumbersome framework that always consists of two separate models, one for alignment and the other for completion, and the multilingual kgc task can not be trained alone. in addition, the alignment model will increase extra computational and memory costs, which are usually comparable to the knowledge model. the second shortcoming is called training inefficiency: although multiple kgs are given, the knowledge is only propagated from support kgs (the kgs to provide knowledge of other languages) to one target kg (the kg to be completed). in this way, if we wish to perform the completion task on another kg, we have to re-designate that kg as the target and retrain entirely, thus making the training process far from efficient. as the example shown in figure 1 (a), the french kg is the target kg and two support kgs in english and greek are available. prior methods output the kgc results only for the french kg through a framework including an alignment model and a knowledge model. according to the multilingual machine translation (mt) task @xcite , it is capable of building a unified mt model to translate multiple languages. since multilingual kgc and multilingual mt are both related to knowledge transfer across languages, we attempt to investigate the multilingual kgc in a compact manner without a redundant alignment model while using a universal model to infer the missing facts in all given kgs. like depicted in figure 1 (b), the completion results for all kgs are expected to be obtained by only one knowledge model. motivated by the discussions above, we propose a graph neural network with language-sensitive multi-graph attention (lsmga) for multilingual kgc. first, several separate kgs are connected into a single graph by sharing the embedding of aligned entities. second, we design a languagespecific multi-graph attention to better capture different patterns stored in different graphs. at last, a language-sensitive aggregation module is utilized to integrate the information from multiple sources. experimental results show that our approach achieves better results than previous methods on the multilingual kgc task. it indicates that our framework can take full advantage of all given kgs by using a universal knowledge model. our main contributions are summarized as follows: multilingual knowledge graph completion aims to boost kgc with multiple kgs that are in different languages. for the first time, the mtranse @xcite model extended knowledge graph embeddings from a monolingual scenario to a multilingual scenario, where the information of multiple kgs can be transferred to each other. after then, a lot of work focused on entity alignment task between different kgs @xcite @xcite . on the other hand, @xcite proposes a new framework kens, which improves monolingual kgc by effectively leveraging complementary knowledge of multiple language-specific kgs. alignkgc @xcite performs kgc together with entity alignment and relation alignment on multilingual kgs and improves kgc accuracy, as well as alignment scores. ss-aga (huang et al., 2022) improves the multilingual kgc task by using a relation-aware graph neural network and dynamically generating more potential alignment pairs. however, entity alignment is still the primary focus of the aforementioned methods for multilingual kgc. additionally, the knowledge models in above works are almost built upon the methods for monolingual kgc, while few works address multilingual knowledge transfer directly. in this paper, we are going to address both alignment dependency and training inefficiency. ## methodology the overall architecture is illustrated in figure 2 . four components are included in the whole framework: creating the unified graph. assuming n source kgs: g 1 , g 2 , • • • , g n , we force the aligned entity pairs e i and e j to be representated by the same embedding vector. in this way, n separate kgs are linked into a unified graph g u and the duplicate aligned entities are removed in the unified entity vector set e u . besides, we maintain a unified relation vector set r. by sharing the aligned entities, there is no need to introduce an alignment model in our framework. and since the given kgs are treated equally on the unified graph, we are able to train a model in one go that can be used for the inference on all given kgs. therefore, creating the unified graph is a simple but crucial step to deal with both alignment dependency and training inefficiency. gnn encoder with multi-graph attention. we split the neighbor nodes of the target node into n subgraphs by different sources and encode the target node into n hidden representations by a novel gnn with multi-graph attention. language-sensitive aggregation. with multiple outputs from the gnn encoder, the final representation of the target node is computed by a language-sensitive aggregation module. ## experiments and analysis we have conducted a series of experiments on the multilingual kgc task with our model and the results have been carefully analyzed. ## conclusion in this paper, a multilingual knowledge graph completion model using a graph neural network with language-sensitive multi-graph attention has been proposed. we emphasize that the multilingual kgc could be implemented well without an align-ment model. in addition, language-sensitive multigraph attention allows knowledge transfer among multiple kgs to be carried out in different ways. finally, the experiments on the dbp-5l and e-pkg datasets show that our framework achieves considerable improvement over existing methods.