Open Relation Modeling: Learning to Define Relations between Entities (2022)

Abstract: Relations between entities can be represented by different instances, e.g., a sentence containing both entities or a fact in a Knowledge Graph (KG). However, these instances may not well capture the general relations between entities, may be difficult to understand by humans, even may not be found due to the incompleteness of the knowledge source. In this paper, we introduce the Open Relation Modeling problem - given two entities, generate a coherent sentence describing the relation between them. To solve this problem, we propose to teach machines to generate definition-like relation descriptions by letting them learn from defining entities. Specifically, we fine-tune Pre-trained Language Models (PLMs) to produce definitions conditioned on extracted entity pairs. To help PLMs reason between entities and provide additional relational knowledge to PLMs for open relation modeling, we incorporate reasoning paths in KGs and include a reasoning path selection mechanism. Experimental results show that our model can generate concise but informative relation descriptions that capture the representative characteristics of entities.

## introduction people are always interested in relations between entities. to learn about a new concept, people want to know how this concept relates to the ones they are familiar with; when getting two related entities of interest, people ask how exactly they are related. however, although existing systems identify related entities, they do not provide features for exploring relations between entities. for instance, in figure 1 , the top is the sciencedirect topics feature of elsevier, which lists several related terms without any annotation; the bottom is the "see also" feature of wikipedia, where the annotation of deep learning is not specific to the context of natural language processing. users cannot get how deep learning and nlp are related by reading the annotation, while deep learning is used heavily recently for nlp. besides, even relations are represented, they may not be interpretable to humans. there are different ways to represent relations between entities. for example, if two entities co-occur in a sentence, they are possibly related and the relation can be implied by the sentence. from a structured perspective, a relation can be represented as a fact or a multi-hop reasoning path between two entities in a knowledge graph (kg). however, for humans without too much prior knowledge about the entities, it is still difficult to understand the relations by reading them. for example, from sentence "we study data mining and database." or fact "(data mining, facet of, database)", humans can guess data mining and database are related fields, but they cannot know exactly how they are related. besides, due to the limited size of the corpus or the incompleteness of the kg, for many related entities, we may not extract a sentence or a fact containing both entities. based on the above observation, a system for exploring relations between entities needs to meet the following requirements: 1) interpretability: providing interpretable relation descriptions, with which humans can easily understand relations between entities; 2) openness: dealing with a wide range of related entities, including those neither cooccur in a corpus nor be connected in a knowledge graph, where types of relations are not required to be explicitly pre-specified. to achieve a system meeting with the above requirements, we introduce a novel task-open relation modeling, i.e., generating coherent sentences describing general relations between entities, where types of relations do not need to be pre-specified. different from open relation extraction, which aims to extract relational facts between entities from an open-domain corpus @xcite , open relation modeling aims to generate a concise but informative sentence, capturing the representative characteristics of the given entities and their relation. from the perspective of interpretability, compared to open relation extraction whose outputs are phrases with low interpretability, e.g., (data mining methods, to be integrate within, the framework of traditional database systems) by ollie @xcite , open relation modeling improves the interpretability of entity relations. for example, for data mining and database, we want to generate a sentence like "data mining is a process of extracting and discovering patterns in large data sets involving methods at the intersection of machine learning, statistics, and database systems." such a relation description is informative and easy to understand since it contains important and precise information about entities and their relation. to solve the task, we propose to teach machines to learn from defining entities. definitions of entities are highly summarized sentences that capture the most representative characteristics of entities, where the general relations between the defined entity and other entities in the definitions are well captured. therefore, we suggest to find the general relation between two entities by defining one entity in terms of the other entity. to achieve this, we first collect definitions of entities and extract entity pairs from the definitions. then we teach machines to generate definition-like relation descriptions by training a language generation model to produce definitions of entities conditioned on extracted entity pairs. to generate informative relation descriptions, machines need knowledge about entities and relations. therefore, we apply pre-trained language models (plms) @xcite @xcite , which have recently been shown to contain rich relational knowledge of entities @xcite @xcite . to utilize knowledge to describe relations between entities, machines also need to reason between entities. we incorporate reasoning paths in kgs to help plms do multi-hop reasoning and provide additional relational knowledge to plms. we also design a reasoning path selection mechanism by confidence estimation of plms to select interpretable and informative reasoning paths, which are then incorporated by plms for open relation modeling. we conduct both quantitative and qualitative experiments. experimental results show that, after learning from definitions of entities, plms have a great ability to describe relations between entities concisely and informatively. by incorporating reasoning paths and including the reasoning path selection mechanism, machines can often generate relation descriptions well capturing relations between entities, with only minor errors that do not affect the understanding of relations. we also conduct error analysis for the proposed methods and suggest several directions for future work. ## conclusion in this paper, we introduce and study the novel open relation modeling problem-generating coherent sentences describing general relations between entities, where the relations can be multi-hop, even cannot be reasoned in an existing kg. we achieve this by teaching plms to learn from defining entities and select/utilize reasoning paths. we believe this work will open a door for modeling relations between entities. as for future work, we plan to improve our model as discussed in section 4 and apply our methods to downstream applications, e.g., a system for users to explore relations between entities, which can be further applied to explore a taxonomy or ontology. we can also use the generated relation descriptions to help some related tasks, such as relation extraction @xcite , knowledge graph construction and completion @xcite . the trained models can be further fine-tuned for open relation modeling on specific domains.