A Novel Cascade Binary Tagging Framework for Relational Triple Extraction (2020)

Abstract: Extracting relational triples from unstructured text is crucial for large-scale knowledge graph construction. However, few existing works excel in solving the overlapping triple problem where multiple relational triples in the same sentence share the same entities. In this work, we introduce a fresh perspective to revisit the relational triple extraction task and propose a novel cascade binary tagging framework (CasRel) derived from a principled problem formulation. Instead of treating relations as discrete labels as in previous works, our new framework models relations as functions that map subjects to objects in a sentence, which naturally handles the overlapping problem. Experiments show that the CasRel framework already outperforms state-of-the-art methods even when its encoder module uses a randomly initialized BERT encoder, showing the power of the new tagging framework. It enjoys further performance boost when employing a pre-trained BERT encoder, outperforming the strongest baseline by 17.5 and 30.2 absolute gain in F1-score on two public datasets NYT and WebNLG, respectively. In-depth analysis on different scenarios of overlapping triples shows that the method delivers consistent performance gain across all these scenarios. The source code and data are released online.

## introduction the key ingredient of a knowledge graph is relational facts, most of which consist of two entities connected by a semantic relation. these facts are in the form of (subject, relation, object), or (s, r, o), referred to as relational triples. extracting relational triples from natural language text is a crucial step towards constructing large-scale knowledge graphs. early works in relational triple extraction took a pipeline approach @xcite @xcite . it first recognizes all entities in a sentence and then performs relation classification for each entity pair. such an approach tends to suffer from the error propagation problem since errors in early stages cannot be corrected in later stages. to tackle this problem, subsequent works proposed joint learning of entities and relations, among them are feature-based models @xcite @xcite and, more recently, neural network-based models @xcite @xcite @xcite . by replacing manually constructed features with learned representations, neural network-based models have achieved considerable success in the triple extraction task. however, most existing approaches cannot efficiently handle scenarios in which a sentence contains multiple relational triples that overlap with each other. figure 1 illustrates these scenarios, where triples share one or two entities in a sentence. this overlapping triple problem directly challenges conventional sequence tagging schemes that assume each token bears only one tag @xcite . it also brings significant difficulty to relation classification approaches where an entity pair is assumed to hold at most one relation @xcite . @xcite is among the first to consider the overlapping triple problem in relational triple extraction. they introduced the categories for different overlapping patterns as shown in figure 1 and proposed a sequence-to-sequence (seq2seq) model with copy mechanism to extract triples. based on the seq2seq model, they further investigate the impact of extraction order @xcite and gain considerable improvement with reinforcement learning. @xcite also studied the overlapping triple problem by modeling text as relational graphs with a graph convolutional networks (gcns) based model. despite their success, previous works on extracting overlapping triples still leave much to be desired. specifically, they all treat relations as discrete labels to be assigned to entity pairs. this formulation makes relation classification a hard machine learning problem. first, the class distribution is highly imbalanced. among all pairs of extracted entities, most do not form valid relations, generating too many negative examples. second, the classifier can be confused when the same entity participates in multiple valid relations (overlapping triples). without enough training examples, the classifier can hardly tell which relation the entity participates in. as a result, the extracted triples are usually incomplete and inaccurate. in this work, we start with a principled formulation of relational triple extraction right at the triple level. this gives rise to a general algorithmic framework that handles the overlapping triple problem by design. at the core of the framework is the fresh perspective that instead of treating relations as discrete labels on entity pairs, we can model relations as functions that map subjects to objects. more precisely, instead of learning relation classifiers f (s, o) → r, we learn relation-specific taggers f r (s) → o, each of which recognizes the possible object(s) of a given subject under a specific relation; or returns no object, indicating that there is no triple with the given subject and relation. under this framework, triple extraction is a two-step process: first we identify all possible subjects in a sentence; then for each subject, we apply relationspecific taggers to simultaneously identify all possible relations and the corresponding objects. we implement the above idea in casrel, an end-to-end cascade binary tagging framework. it consists of a bert-based encoder module, a sub-ject tagging module, and a relation-specific object tagging module. empirical experiments show that the proposed framework outperforms state-of-theart methods by a large margin even when the bert encoder is not pre-trained, showing the superiority of the new framework itself. the framework enjoys a further large performance gain after adopting a pre-trained bert encoder, showing the importance of rich prior knowledge in triple extraction task. this work has the following main contributions: ## related work extracting relational triples from unstructured natural language texts is a well-studied task in information extraction (ie). it is also an important step for the construction of large scale knowledge graph (kg) such as dbpedia @xcite , freebase @xcite and knowledge vault @xcite . early works @xcite address the task in a pipelined manner. they extract relational triples in two separate steps: 1) first run named entity recognition (ner) on the input sentence to identify all entities and 2) then run relation classification (rc) on pairs of extracted entities. the pipelined methods usually suffer from the error propagation problem and neglect the relevance between the two steps. to ease these issues, many joint models that aim to learn entities and relations jointly have been proposed. traditional joint models @xcite @xcite are feature-based, which heavily rely on feature engineering and require intensive manual efforts. to reduce manual work, recent studies have investigated neural network-based methods, which deliver state-of-the-art performance. however, most existing neural models like @xcite achieve joint learning of entities and relations only through parameter sharing but not joint decoding. to obtain relational triples, they still have to pipeline the detected entity pairs to a relation classifier for identifying the relation of entities. the separated decoding setting leads to a separated training objective for entity and relation, which brings a drawback that the triple-level dependencies between predicted entities and relations cannot be fully exploited. different from those works, @xcite achieves joint decoding by introducing a unified tagging scheme and convert the task of relational triple extraction to an end-to-end sequence tagging problem without need of ner or rc. the proposed method can directly model relational triples as a whole at the triple level since the information of entities and relations is integrated into the unified tagging scheme. though joint models (with or without joint decoding) have been well studied, most previous works ignore the problem of overlapping relational triples. @xcite introduced three patterns of overlapping triples and try to address the problem via a sequence-to-sequence model with copy mechanism. recently, @xcite also study the problem and propose a graph convolutional networks (gcns) based method. despite their initial success, both methods still treat the relations as discrete labels of entity pairs, making it quite hard for the model to learn overlapping triples. our framework is based on a training objective that is carefully designed to directly model the relational triples as a whole like @xcite , i.e., to learn both entities and relations through joint decoding. moreover, we model the relations as functions that map subjects to objects, which makes it crucially different from previous works. ## the casrel framework the goal of relational triple extraction is to identify all possible (subject, relation, object) triples in a sentence, where some triples may share the same entities as subjects or objects. towards this goal, we directly model the triples and design a training objective right at the triple level. this is in contrast to previous approaches like @xcite where the training objective is defined separately for entities and relations without explicitly modeling their integration at the triple level. formally, given annotated sentence x j from the training set d and a set of potentially overlapping triples t @xmath0, r, o)} in x j , we aim to maximize the data likelihood of the training set d: here we slightly abuse the notation t j . @xmath1. t j |s is the set of triples led by subject s in t j . this formulation provides several benefits. first, since the data likelihood starts at the triple level, optimizing this likelihood corresponds to directly optimizing the final evaluation criteria at the triple level. second, by making no assumption on how multiple triples may share entities in a sentence, it handles the overlapping triple problem by design. third, the decomposition in eq. ( 3 ) inspires a novel tagging scheme for triple extraction: we learn a subject tagger p(s|x j ) that recognizes subject entities in a sentence; and for each relation r, we learn an object tagger p r (o|s, x j ) that recognizes relationspecific objects for a given subject. in this way we can model each relation as a function that maps subjects to objects, as opposed to classifying relations for (subject, object) pairs. indeed, this novel tagging scheme allows us to extract multiple triples at once: we first run the subject tagger to find all possible subjects in the sentence, and then for each subject found, apply relation-specific object taggers to find all relevant relations and the corresponding objects. the key components in the above general framework, i.e., the subject tagger and relation-specific object taggers, can be instantiated in many ways. in this paper, we instantiate them as binary taggers on top of a deep bidirectional transformer bert @xcite . we describe its detail below. ## conclusion in this paper, we introduce a novel cascade binary tagging framework (casrel) derived from a principled problem formulation for relational triple extraction. instead of modeling relations as discrete labels of entity pairs, we model the relations as functions that map subjects to objects, which provides a fresh perspective to revisit the relational triple extraction task. as a consequent, our model can simultaneously extract multiple relational triples from sentences, without suffering from the overlapping problem. we conduct extensive experiments on two widely used datasets to validate the effectiveness of the proposed casrel framework. experimental results show that our model overwhelmingly outperforms state-of-theart baselines over different scenarios, especially on the extraction of overlapping relational triples. fail to form a valid relational triple. in other words, it implies that identifying relations is somehow easier than identifying entities for our model. in contrast to nyt, for webnlg, the performance gap between (e1, e2) and (e1, r, e2) is comparatively larger than that between (e1, r, e2) and (e1, r)/(r, e2). it shows that misidentifying the relations will bring more performance degradation than misidentifying the entities. such observation also indicates that it's more challenging for the proposed casrel model to identify the relations than entities in webnlg, as opposed to what we observed in nyt. we attribute such difference to the different number of relations contained in the two datasets (i.e., 24 in nyt and 246 in webnlg), which makes the identification of relation much harder in webnlg.