Learning Inter-Entity-Interaction for Few-Shot Knowledge Graph Completion (2022)

Abstract: Few-shot knowledge graph completion (FKGC) aims to infer unknown fact triples of a relation using its few-shot reference entity pairs. Recent FKGC studies focus on learning semantic representations of entity pairs by separately encoding the neighborhoods of head and tail entities. Such practice, however, ignores the inter-entity interaction, resulting in low-discrimination representations for entity pairs, especially when these entity pairs are associated with 1-to-N, N-to-1, and N-to-N relations. To address this issue, this paper proposes a novel FKGC model, named Cross-Interaction Attention Network (CIAN) to investigate the inter-entity interaction between head and tail entities. Specifically, we first explore the interactions within entities by computing the attention between the task relation and each entity neighbor, and then model the interactions between head and tail entities by letting an entity to attend to the neighborhood of its paired entity. In this way, CIAN can figure out the relevant semantics between head and tail entities, thereby generating more discriminative representations for entity pairs. Extensive experiments on two public datasets show that CIAN outperforms several state-of-the-art methods. The source code is available at https://github.com/cjlyl/FKGC-CIAN .

## introduction knowledge graphs (kgs) like @xcite , freebase @xcite , and wikidata @xcite have been successfully applied to various knowledgedriven applications, such as question answering @xcite , semantic search @xcite , and information retrieval @xcite . a knowledge graph consists of a large number of fact triples, where each triple is represented in the figure 1 : an example to illustrate the importance of the relevant attributes between head and tail entities. intuitively, the political attribute (dashed blue) of head entity georgebush could be more helpful for representing entity pair (georgebush, president). in literature, state-of-the-art kgc models are usually based on knowledge graph embeddings (kge). the key idea behind kge-based methods is to embed entities and relations into lowdimensional vector spaces and measure the plausibility of triples based on their embeddings @xcite . despite their success, existing kge-based models generally require sufficient training triples for all relations. however, in many real-world kgs, a large proportion of (few-shot) relations only contain a limited number of triples @xcite @xcite . the shortage of training triples hinders existing kge-based models from achieving satisfactory performance on these few-shot relations. to address this issue, few-shot knowledge graph completion (fkgc) models have been proposed to predict a tail entity t in a query triple (h, r, ?) to solve this problem, we propose to investigate the inter-entity interaction between head and tail entities, which affects the importance of their semantic attributes to represent entity pairs. the basic idea behind this is that entity pairs are supposed to pay more attention to the relevant semantic attributes between head and tail entities. figure 1 shows an example of the head entity georgebush and its two entity pairs belonging to the task relation hasjobposition. the left neighborhood (dashed blue) describes the political attribute of georgebush as a former president of the us, while the right one (solid green) shows his soldier attribute as a navy pilot. intuitively, the political attribute could be more helpful for representing (georgebush, president), while the soldier attribute could contribute more to representing (georgebush, pilot). hence, exploiting the interactions between head and tail entities is beneficial to characterize entity pairs. to this end, we propose a cross-interaction attention network (cian) for few-shot knowledge graph completion. our contributions can be summarized as follows: ## related work a common approach for kgc is to represent entities and relations as low-dimensional vectors (a.k.a, knowledge graph embeddings, kge) and then learn a well-designed score function to measure the plausibility of triples in the embedding space. existing kge-based models can be roughly classified into two groups: (1) translational distance approaches, which view relations as translation operations and define a distance-based score function accordingly @xcite @xcite , and (2) semantic matching approaches, which establish similarity-based score functions and make predictions by matching latent semantics of entities and relations @xcite @xcite . these kge-based models usually assume that there are sufficient training triples for all relations, and thus are limited in practical few-shot scenarios. recently, much research progress has been made in few-shot kg completion, which solves kgc for unseen relations with a handful of training samples, based on some prior knowledge gained from a large number of similar but different relations. existing fkgc approaches can be divided into two main categories. (1) metric learning-based methods: gmatching @xcite is the first study to solve the one-shot kg completion problem. it learns entity embeddings by encoding their direct neighbors and measures the similarities between the query and reference entity pairs by a multi-step matching processor. fsrl @xcite extends gmatching to a few-shot scenario through a recurrent autoencoder aggregation network and employs a heterogeneous neighbor encoder to improve the quality of entity embeddings. unlike gmatching and fsrl that learn the identical representation for the same entity in various triples, faan @xcite allows for the dynamic properties of entities under different task relations and learns task-aware entity representations. in addition, p-int @xcite leverages the paths from head to tail entities to represent entity pairs and calculates the path interactions between the support and query entity pairs. (2) meta learner-based methods: metar @xcite generates relation-specific meta-information using the embeddings of head and tail entities and updates relation meta via gradient meta. gana @xcite learns relation-meta by incorporating the neighbor information of entities and devises a meta-learning-based transh module to model complex relations. different from the aforementioned methods that treat each entity independently, our work explores the inter-entity interaction and leverages the relevant attributes between entities to represent entity pairs. ## problem formulation a kg can be represented as a set of triples @xmath0, r, t) ∈ e × r × e}, where e and r are the entity set and the relation set, respectively. given any two of three elements within a triple, the kg completion task aims to predict the remaining one. this study focuses on predicting t given (h, r, ?) since our goal is to infer new facts for few-shot relations. following the standard setting in fkgc @xcite , we have access to a background kg g , which is a subset of g with some relations of r. the remaining relations in r are further divided into three disjoint task sets r train , r test and r valid , which are used in the meta-training, meta-testing, and meta-validation phases, respectively. to perform few-shot learning, we adopt the episodic paradigm @xcite to imitate the real test conditions. specifically, in each iteration of the meta-training phase, we sample a task relation r from r train , and construct its support set s @xmath1, t i )} k @xmath2 here, k and b denote the numbers of support and query triples, t j is the ground-truth tail entity for query head entity h about relation r, and c h j ,r is the set of the corresponding candidate entities in g. the candidate entities are constructed based on the entity type constraint @xcite . our goal is to train a model using k support entity pairs in s r such that the model can rank the ground-truth entity t j higher than the corresponding candidate entities in c h j ,r . a few-shot kg completion task can be defined as follows: definition of few-shot kg completion. given a task relation r and its support set s @xmath3, r, t i )} k @xmath4, one task is to predict the missing tail entities of the query triples, where few-shot size k is very small. the task is called k-shot kg completion. after sufficient training with r train , the learned model can be used for the meta-validation and metatesting phases with r valid and r test , which are defined in the same way as r train . ## methodology the framework of the proposed cian is illustrated in figure 2 . cian includes the following stages: ## experiments in this section, we conduct extensive experiments to evaluate the performance of the proposed cian and verify the effectiveness of the key components in cian. ## conclusion in this paper, we investigate the inter-entity interaction and propose a cross-interaction attention network for few-shot kg completion. specifically, a task-aware attention module is designed to extract task-relevant entity attributes, and an entity-pairaware attention module is developed to identify the relevant semantic attributes between head and tail entities. experiments on two public datasets demonstrate that our model outperforms state-ofthe-art models with different few-shot sizes. in the future, we will study how to make use of the side information such as textual descriptions of entities to solve the few-shot kg completion task.