A Dataset for Hyper-Relational Extraction and a Cube-Filling Approach (2022)

Abstract: Relation extraction has the potential for large-scale knowledge graph construction, but current methods do not consider the qualifier attributes for each relation triplet, such as time, quantity or location. The qualifiers form hyper-relational facts which better capture the rich and complex knowledge graph structure. For example, the relation triplet (Leonard Parker, Educated At, Harvard University) can be factually enriched by including the qualifier (End Time, 1967). Hence, we propose the task of hyper-relational extraction to extract more specific and complete facts from text. To support the task, we construct HyperRED, a large-scale and general-purpose dataset. Existing models cannot perform hyper-relational extraction as it requires a model to consider the interaction between three entities. Hence, we propose CubeRE, a cube-filling model inspired by table-filling approaches and explicitly considers the interaction between relation triplets and qualifiers. To improve model scalability and reduce negative class imbalance, we further propose a cube-pruning method. Our experiments show that CubeRE outperforms strong baselines and reveal possible directions for future research. Our code and data are available at github.com/declare-lab/HyperRED.

## introduction knowledge acquisition is an open challenge in artificial intelligence research @xcite . the standard form of representing the acquired knowledge is a knowledge graph @xcite , which has broad applications such as question answering @xcite and search engines @xcite . relation extraction (re) is a task that has the potential for large-scale and automated knowledge graph construction by extracting facts from natural language text. most relation extraction methods focus on binary relations @xcite which consider the relationship between two entities, forming a relation triplet consisting of the head entity, relation and tail entity respectively. however, knowledge graphs commonly contain hyper-relational facts @xcite which have qualifier attributes for each relational triplet, such as time, quantity, or location. for instance, @xcite found that the freebase knowledge graph contains hyper-relational facts for 30% of entities. hence, extracting relation triplets may be an oversimplification of the rich and complex knowledge graph structure. as shown in figure 1 , a relation triplet can be attributed to one or more qualifiers, where a qualifier is composed of a qualifier label and value entity. for example, the relation triplet (leonard parker, educated at, harvard university) can be factually enriched by specifying the qualifier of (end time, 1967), forming the hyper-relational fact (leonard parker, educated at, harvard university, end time, 1967). hyper-relational facts generally cannot be simplified into the relation triplet format as the qualifiers are attributed to the triplet as a whole and not targeted at a specific entity in the triplet. furthermore, attempting to decompose the hyperrelational structure to an n-ary format would lose the original triplet information and be incompatible with the knowledge graph schema @xcite . on the other hand, hyper-relational facts have practical benefits such as improved fact verification @xcite and representation learning for knowledge graphs @xcite . thus, it is necessary to extract relation triplets together with qualifiers to form hyper-relational facts. in this work, we propose the task of hyperrelational extraction to jointly extract relation triplets with qualifiers from natural language sentences. to support the task, we contribute a generalpurpose and large-scale hyper-relational extraction dataset (hyperred) which is constructed through distant supervision @xcite and partially refined through human annotation. our dataset differs from previous datasets in two distinct ways: (1) compared to existing datasets for binary relation extraction @xcite , hyperred enables richer information extraction as it contains qualifiers for each relation triplet in the sentence. (2) while datasets for n-ary relation extraction @xcite are restricted to the biomedical domain, hyperred covers multiple domains and has a hyper-relational fact structure that is compatible with the knowledge graph schema. unfortunately, to the best of our knowledge, there are no existing models for hyper-relational extraction. currently, a popular end-to-end method for binary relation extraction is to cast it as a tablefilling problem @xcite . generally, a two-dimensional table is used to represent the interaction between any two individual words in a sentence. however, hyper-relational extraction requires the model to consider the interactions between two entities in the relation triplet, as well as the value entity for the qualifier. thus, we extend the table-filling approach to a third dimension, casting it as a cube-filling problem. on the other hand, a naive cube-filling approach faces two issues: (1) computing the full cube representation is computationally expensive and does not scale well to longer sequence lengths. (2) the full cube will be sparsely labeled with a vast majority of entries as negative samples, causing the model to be biased in learning @xcite and hence underperform. to tackle these two issues, we propose a simple yet effective cube-pruning technique that filters the cube entries based on words that are more likely to constitute valid entities. our experiments show that cube-pruning significantly improves the com-putational efficiency and simultaneously improves the extraction performance by reducing the negative class imbalance. in addition to our cube-filling model which we refer to as cubere, we also introduce two strong baseline models which include a two-stage pipeline and a generative sequence-tosequence @xcite ## hyperred: a hyper-relational extraction dataset our goal is to construct a large-scale and generalpurpose dataset for extracting hyper-relational facts from natural language text. however, it is seldom practical to assume to have an ample amount of high-quality labeled samples in real applications, especially for complex tasks such as information extraction. hence, we propose a weakly supervised @xcite data setting which enables us to collect a larger and more diverse training set than would be otherwise possible. to minimize the effect of noisy samples in evaluation, we then perform human annotation for a portion of the collected data and allocate it as the held-out set. in the following sections, we first introduce the process of collecting the distantly supervised data, followed by the human-annotated data portion. ## analysis in this section, we study the effect of cube-pruning and identify directions for future research. further analysis is shown in appendix f. ## conclusions in this work, we propose the hyper-relational extraction task for extracting richer and more complete facts from natural text. to support the task, we introduce hyperred, a large-scale and generalpurpose dataset that is not restricted to any specific domain. as there is no available model for hyper-relational extraction, we propose an end-toend cube-filling approach inspired by table-filling methods for relation extraction. we further propose a cube-pruning method to reduce computational cost and alleviate negative class imbalance during training. experiments on hyperred demonstrate the effectiveness of cubere compared to strong baselines, setting the benchmark for future work. limitations model limitations regarding the cubere model, we propose a cube-pruning method to improve the computational efficiency and reduce the negative class imbalance. the cube-pruning threshold is fixed, although the input can have different sentence lengths. hence, it may result in overly strict pruning if the sentence is extremely long. however, the pruning threshold is similar to the maximum sequence length in most transformerbased models and may need to be tuned according to the specific dataset or application scenario. the optimal cube-pruning threshold is selected based on the analysis in section 5.1. cubere may not work well for overlapping or nested entity spans, which affects 2.11% of the sentences. this can be considered a general limitation of table-filling methods for relation extraction, and future work may need to consider a span-based approach (xu et al., 2021) to address this issue. data limitations regarding the hyperred dataset, the distant supervision method of data collection may not align all valid facts present in the text articles. this is due to the possible incompleteness of the knowledge graph which is an open research challenge @xcite . on the other hand, it is not feasible to manually annotate all possible facts due to constraints in annotation time and cost. furthermore, there are a large number of relation and qualifier labels to consider, resulting in a challenging task for human annotators. a promising and practical method to address the challenges in distant supervision is to adopt a human-in-theloop annotation scheme for re @xcite . the annotation scheme can increase the number of facts in a dataset by training a re model to predict more candidate facts for each text article, which are then reviewed and filtered by humans. however, this model-assisted annotation approach is not applicable to the construction of hyperred as it relies on existing strong re models, whereas there are no suitable models for hyper-relational extraction existing prior to this work.