C o RI : Collective Relation Integration with Data Augmentation for Open Information Extraction (2021)

Abstract: Integrating extracted knowledge from the Web to knowledge graphs (KGs) can facilitate tasks like question answering. We study relation integration that aims to align free-text relations in subject-relation-object extractions to relations in a target KG. To address the challenge that free-text relations are ambiguous, previous methods exploit neighbor entities and relations for additional context. However, the predictions are made independently, which can be mutually inconsistent. We propose a two-stage Collective Relation Integration (CoRI) model, where the first stage independently makes candidate predictions, and the second stage employs a collective model that accesses all candidate predictions to make globally coherent predictions. We further improve the collective model with augmented data from the portion of the target KG that is otherwise unused. Experiment results on two datasets show that CoRI can significantly outperform the baselines, improving AUC from .677 to .748 and from .716 to .780, respectively.

## introduction with its large volume, the web has been a major resource for knowledge extraction. open information extraction @xcite @xcite ) is a prominent approach that harvests subject-relation-object extractions in free text without assuming a predefined set of relations. one way to empower downstream applications like question answering is to integrate those free-text extractions into a knowledge graph (kg), e.g., freebase. relation integration is the first step to integrate those extractions, where their free-text relations (i.e., source relations) are normalized to relations in the target kg (i.e., target relations). only after relation integration can entity linking proceed to resolve the free-text subjects and objects to their canonical entities in the target kg. local approaches. relation integration has been studied by the natural language processing (nlp) community. with exact matching in literal form between entity names in the source graph and target kg, previous methods obtain parallel data, i.e., common entity pairs, between the two graphs as in fig. 1 . features of the entity pairs (e.g., malia-barack) in the source graph and their relations in the target kg (e.g., father) are used to train models to predict target relations for future extractions. a common challenge is the ambiguity of source relations, e.g., "parent" may correspond to father or mother in different contexts. previous methods exploited contextual features including embeddings of seen entities (e.g., "malia"; @xcite , middle relations between (e.g., "parent"; @xcite @xcite @xcite , and neighbor relations around the entity pair (e.g., "gender"; @xcite . assuming rich contexts to address the ambiguity challenge, previous methods may fall short under the evolving and incomplete nature of the source ## preliminaries in this section, we first formulate the task of relation integration, then describe local methods by exemplifying with the state-of-the-art approach openki @xcite . ## collective relation integration as discussed in § 1, the drawback of local methods is that predictions of different entity pairs are independently made. neglecting their dependency may lead to predictions inconsistent with each other. to address the issue, we propose a collective approach cori, which achieves collective relation integration via two stages: candidate generation and collective inference. in this section, we demonstrate the input and output of the two stages, as well as our current implementations. ## data augmentation w/ unmatched kg as in def. 2, the volume of parallel data is limited by the number of shared entity pairs k r ∩ k r of the two graphs. in fig. 1 , the unmatched part of the target kg, containing entity pairs without extraction counterparts (i.e., k r \ k r ) and their target relations, can also indicate common substructures of the target kg, and guide the training of the collective model. to this end, we propose leveraging unmatched kg to generate pseudo parallel data to augment the limited training data. ## experimental results we aim to answer the following questions: (1) is cori superior to local models? (2) is cori robust w.r.t. varying size of training and testing data? (3) is unmatched kg useful for cori? is our parallel data augmentation approach the best choice? ## related work relation integration has been studied by both the database (db) and the nlp communities. the db community formulates it as schema matching that aligns the schemas of two tables, e.g., matching columns of an is in table to those of another subarea of table @xcite @xcite . such the nlp community, on the other hand, investigates the problem at the entity pair level. besides manually designed rules @xcite , most works leverage the link structure between entities and relations. universal schema @xcite learns embeddings of entities and middle relations between entity pairs through decomposing their co-occurrence matrix. however, the entity embeddings make it not generalize to unseen entities. other methods @xcite @xcite also exploit middle relations, but eliminate entity parameters. zhang et al. (2019 ) moves one step further by explicitly considering neighbor relations, leveraging more context from the local link structure. some works @xcite directly minimize the distance between embeddings of relations sharing the same entity pairs. @xcite further leverage compositional representations of entity names instead of using free parameters to deal with unseen entities at test time. there are also works on open ie canonicalization that cluster source relations. some use entity pairs as clustering signals @xcite @xcite , while others use lexical features or side information @xcite . however, the clusters are not finally aligned to relations in target kgs, different from our problem. the two-stage collective inference framework has been explored in other problems like entity linking @xcite @xcite , where candidate entities are generated for each mention independently, and collectively ranked based on their compatibility in the second stage. in machine translation, an effective approach to leverage monolingual corpus in the target language is to back-translate it to the source language to augment the limited parallel corpus @xcite . the above works inspired us to use collective inference for relation integration and leverage the unmatched kg for data augmentation. another approach to perform collective inference is to solve learning problem with constraints, such as integer linear programming @xcite , posterior regularization @xcite , and conditional random fields @xcite . comparing to our approach, these methods usually involve heavy computation, or are hard to optimize. examining the perfor-mance of these methods is an interesting future direction. besides, we also adopted ideas of selecting samples from out-domain data similar to in-domain samples @xcite to select our pseudo parallel data. ## conclusion in this paper, we proposed cori, a collective inference approach to relation integration. to the best of our knowledge, this is the first work exploring this idea. we devised a two-stage framework, where the candidate generation stage employs existing local models to make candidate predictions, and the collective inference stage refines the candidate predictions by enforcing global coherence. observing that the target kg is rich in substructures indicating the desired global coherence, we further proposed exploiting the unmatched kg by selectively synthesizing pseudo parallel data to augment the training of our collective model. our solution significantly outperforms all baselines on two datasets, indicating the effectiveness of our approaches.