Query-based Instance Discrimination Network for Relational Triple Extraction (2022)

Abstract: Joint entity and relation extraction has been a core task in the field of information extraction. Recent approaches usually consider the extraction of relational triples from a stereoscopic perspective, either learning a relation-specific tagger or separate classifiers for each relation type. However, they still suffer from error propagation, relation redundancy and lack of high-level connections between triples. To address these issues, we propose a novel query-based approach to construct instance-level representations for relational triples. By metric-based comparison between query embeddings and token embeddings, we can extract all types of triples in one step, thus eliminating the error propagation problem. In addition, we learn the instance-level representation of relational triples via contrastive learning. In this way, relational triples can not only enclose rich class-level semantics but also access to high-order global connections. Experimental results show that our proposed method achieves the state of the art on five widely used benchmarks.

## introduction extracting structured information from open domain texts is a long-standing study topic in nlp. joint entity and relation extraction aims to mine high-quality relational triples (subject, relation, object) from unstructured texts. for example, in the sentence "900 people cross the border between malaysia and singapore", the subject entity 900 people and the object entity border have a physical location relation phys. the current entity and relation extraction methods can be divided into two categories: pipeline methods @xcite and joint methods @xcite . despite the flexibility of pipeline methods, they face the error propagation and the lack of interaction problems (katiyar and prgc first filters out unlikely relations in a sentence, and then learns the relation-specific tagger for potential relations. (b) the gray circle below represents the initial state of query embeddings which are type-agnostic, and the star above represents relation type embeddings. we can break the limitation of type-independence and learn the high-order connections between different types via contrastive learning. @xcite . therefore, many strategies have been proposed to unify the two tasks. @xcite extends tagging schema to tag both entities and relations. however, it cannot handle the triple overlapping problem. to tackle this problem, the generation-based methods @xcite @xcite , the table-filling methods @xcite @xcite and the cascading methods @xcite @xcite have been investigated. recently, an increasing number of studies have come to consider the relational triple extraction task from a stereoscopic perspective. these approaches treat relation type as an important element, and either learn a distinct classifier @xcite for each relation type or construct a relation-specific sequence tagger @xcite @xcite . in this way, these methods can better decouple the task-specific features for relations and entities, and yield promising results. however, they still have some weaknesses. first, error propagation is a well-known problem. @xcite first identifies the subject and then extracts the corresponding triples. however, there are inevitable errors in the first stage. prgc @xcite , as shown in figure 1 (a), first picks the potential relations in the sentence, and later tags the relation-specific entities. the correct relations are then likely to be filtered out in this process. second, these methods mostly suffer from the relation redundancy problem. wei et al. (2020); wang et al. (2020b) assembles triples on all relation types, which generates numerous invalid operations, leading to sparse label and low convergence rate. third, these methods process the triples for each relation type independently, ignoring the high-level global connections between different types of triples. for example, if the triple (maryland, country, u.s.) has been recognized, then u.s. is not supposed to appear in any other family relationship, such as spouse of. to tackle the above problems, we propose a novel query-based instance discrimination network (qidn), in which we leverage instance query to construct instance-level representation for different types of relational triples. as in figure 1 (b), we first use a set of type-agnostic query embeddings to obtain useful contextual information from a pretrained language model. afterwards, by the metricbased comparison between query embeddings and token embeddings, we can extract triples of all types in one step. in this manner, the error propagation problem is eliminated. moreover, since we query the most plausible relation type directly with the query vector, the problem of relation redundancy is well mitigated. in addition, we construct instance-level representations for relational triples by contrastive learning. specifically, as shown in figure 1 (b), we set two training objectives: (1) intra-class instance pairs should get higher similarity than inter-class ones. (2) instance representations should be closer to their corresponding relation type representations. we aim to break the limitation of type-independence in previous methods and establish the global connection between triples. our main contributions are as follows: ## topology of relations to illustrate that our approach establishes connections between different types of relations, we visualize the relation representation learned on nyt. we first perform l2 normalization on the relation embeddings and then use pca @xcite to reduce their dimension. we filter out 5 long-tail relations that appear less than 40 times in the entire training set and then classify them into 4 broad categories based on label prefixes in nyt. as shown in figure 3 , the topology between the relations indicates the semantic connections between them. for example, the two types of sport relations almost overlap, while the six location relations (including country, capital, etc.) are all scattered in the lower left corner. however, we observe an exception on "people", where the 6 subcategories are not spread together spatially. we attribute this to semantic divergence. the three subcategories (place_lived, place_of_birth and place_of_death) semantically express position information, which is distinct from the other subcategories. these all demonstrate that our method can break the limits of type-independence and well consider the highorder connections between different relation types. besides, we conduct a detailed case study to show the connections between queries in appendix c. ## error analysis in this section, we analyze the different error proportions on two tasks: entity classification error (ece) and entity location error (ele) for ner, relation classification error (rce), entity-pair classification error (pce), and entity-pair location error (ple) for re. on the ner task, we can observe from figure 4 that the proportion of classification errors and localization errors for entity queries on ace05 is comparable. but on scierc, the localization errors are more pronounced. we argue that on account of the longer average length of scientific term entities in scierc, the identification of entity boundaries becomes more difficult. as shown in figure 4 , identifying the boundary tokens of the subject and object entities is the most difficult on both ace05 and scierc. we argue that this is due to the complicated patterns of nesting as well as overlapping in boundary recognition. in addition, the proportion of relation classification errors is minor on both datasets, which indicates the ability of queries in categorization. ## conclusion in this paper, we propose a novel query-based instance discrimination network for relational triple extraction. based on instance queries, we can extract all types of triples in one step, thus avoiding the problem of error accumulation and relation redundancy. besides, we set two training objectives for contrastive learning to establish high-order connections between triples while learning rich classlevel semantic information. experimental verifications on five datasets demonstrate that the proposed method reaches state-of-the-art.