Open Information Extraction with Entity Focused Constraints (2023)

Abstract: Open Information Extraction (OIE) is the task of extracting tuples of the form (subject, predicate, object), without any knowledge of the type and lexical form of the predicate, the subject, or the object. In this work, we focus on improving OIE quality by exploiting domain knowledge about the subject and object. More precisely, knowing that the subjects and objects in sentences are oftentimes named entities, we explore how to inject constraints in the extraction through constrained inference and constraint-aware training. Our work leverages the state-of-the-art OpenIE6 platform, which we adapt to our setting. Through a carefully constructed training dataset and constrained training, we obtain a 29.17% F1-score improvement in the CaRB metric and a 24.37% F1-score improvement in the WIRe57 metric. Our technique has important applications – one of them is investigative journalism, where automatically extracting conflict-of-interest between scientists and funding organizations helps understand the type of relations companies engage with the scientists.

## introduction open information extraction (oie) is the task of extracting triples from unstructured corpora in a domain-independent manner. a triple consists of a subject, a relation, and an object. oie has important applications, such as question answering @xcite , or automatically creating or extending knowledge bases @xcite . oie is a challenging task, with the performance of state-ofthe-art models varying from 88.5% f1 score @xcite the work was done when the first author was at inria and institut polytechnique de paris. @xcite to 34% @xcite , depending on the difficulty of the benchmark. when the named entities in a domain are known to be the subject/object of extractions, oie should also identify relations between these entities. an important use case is automatically creating a knowledge base of relations between scientists and companies, i.e. identifying conflict-of-interest between the scientists and funding bodies, where the named entities are the names of scientists and companies, and the relation describes the conflict of interest between them. clustering these relation phrases, such as received a research gift from, received speaker fees or consults for helps analyze the relationships that companies engage with the scientists. these relations are crucial to understanding scientists' positions on health issues @xcite in investigative journalism. however state-of-the-art oie models do not always retain named entities in the extractions, for example, given the sentence "shahrad taheri received funding for research through a grant from cambridge weight plan", an oie tool @xcite returns ⟨shahrad taheri, received, funding for research⟩. while this extraction correctly identifies the subject of the triple, the quality of the predicate and object could be improved as follows: the extraction ⟨shahrad taheri, received funding for research through a grant from, cambridge weight plan⟩ retains the second important entity ( cambridge weight plan) and is precise about the relation. such sentences are frequent in the declarations of conflict of interest that authors add to articles in pubmed, a dataset of scientific articles on life sciences and biomedical topics. in this work, we focus on relation extraction, when the subject and object are named entities. in particular, we would like to significantly improve the performance of oie tools, such that triples as ⟨first entity, predicate, second entity⟩ are not missed or poorly extracted. to achieve this, we leverage deep learning with constraints, i.e. techniques that enforce constraints on the classifier's predictions. constrained learning is very common in sequence-to-sequence tasks, such as relation or entity extraction, where the output should have a specific form. constraint learning has also been successfully used in oie. in our case, we enforce constraints on the subject, object and predicate forms, and we investigate several techniques to achieve the best result, such as constraint-aware training @xcite and constraint inference @xcite . we deployed our technique within openie6 @xcite , a state-of-the-art tool for oie. our salient contributions are: i) we extend the openie6 model with entity-centric constraints; ii) we implement the constraints as penalties in the loss function, and as hard constraints during inference. iii) we show through an extensive evaluation that our method improves over the state-ofthe-art; iv) we perform a large scale evaluation of the system, on conflict of interest declarations from pubmed bibliographical data. ## related work in the literature, the extraction of triples of the form ⟨subject, relation, object⟩ has been studied in several settings. a relation can be expressed using a surface form, i.e., the tokens present in a sentence, or a canonical form, usually introduced in a knowledge base. in the most general setting, we do not enforce any constraints on the types of the three elements, and the task is referred to as open information extraction (oie). in the most restricted setting, the subject and object are entities, and the relation comes from a predefined set of relations. this task is known as relation extraction. finally, open relation extraction, also referred to as relation discovery, refers to approaches that use little training (such as distant supervision, fewshot learning, or semi-supervision) or no training (unsupervised) to classify relations between entities. some inconsistencies arise in the use of the terminology in the literature, e.g., "open relation extraction" has been also used to designate open information extraction, in @xcite . ## problem definition our goal is to extract triples from sentences that respect the guidelines detailed by the carb metric @xcite , i.e., they should be i) complete: all triples should be extracted from a sentence, ii) asserted: the triple should be implied from the sentence iii) informative: the triple should contain maximum relevant information from the sentence and iv) atomic: extraction cannot be split into multiple extractions. given a sentence s containing entities @xmath0, ..., e i , ..., e n }, we denote by ⟨s, r, o⟩ a triple that is extracted from s. the carb rules can be customized to fit our setting as follows: ## entity focused constraints openie6 @xcite receives in input a sentence and outputs a list of extractions of the form ⟨subject, predicate, object⟩. the architecture of the model is a deep neural network that first encodes tokens using bert @xcite , and then iteratively identifies at most m extractions, i.e., calls the same architecture for each extraction for m times (figure 1 ). the embeddings of the labels generated at the end of the 1st iteration are added to the embeddings of the tokens in the second iteration, and so on. this adds context so that a new extraction is generated the next time. each token is assigned a label from {s (subject), r (relationship), o (object) or n (none)}. bert layer to encode tokens openie6 constraint-aware training openie6 uses constraint-aware training to infuse the model with task-related knowledge in the form of constraints. the model learns to satisfy these constraints during training without explicitly enforcing them during the inference, hence these types of constraints are typically referred to in the literature as soft constraints. this is achieved by adding additional penalties in the loss function, as follows: pos coverage (posc). tokens labeled as nouns, verbs, adjectives, or adverbs should be part of at least one extraction. head verb coverage (hvc). verbs that are not light verbs (e.g., do, give, have, make, etc.), referred to as head verbs, should be present in the relation span of a few but not too many extractions. ## results and discussion evaluation. in ## limitations we identify the following limitations affecting our proposed methods: