OIE @ OIA : an Adaptable and Efficient Open Information Extraction Framework (2022)

Abstract: Different Open Information Extraction (OIE) tasks require different types of information, so the OIE field requires strong adaptability of OIE algorithms to meet different task requirements. This paper discusses the adaptability problem in existing OIE systems and designs a new adaptable and efficient OIE system - OIE@OIA as a solution. OIE@OIA follows the methodology of Open Information eXpression (OIX): parsing a sentence to an Open Information Annotation (OIA) Graph and then adapting the OIA graph to different OIE tasks with simple rules. As the core of our OIE@OIA system, we implement an end-to-end OIA generator by annotating a dataset (we make it open available) and designing an efficient learning algorithm for the complex OIA graph. We easily adapt the OIE@OIA system to accomplish three popular OIE tasks. The experimental show that our OIE@OIA achieves new SOTA performances on these tasks, showing the great adaptability of our OIE@OIA system. Furthermore, compared to other end-to-end OIE baselines that need millions of samples for training, our OIE@OIA needs much fewer training samples (12K), showing a significant advantage in terms of efficiency.

## introduction open information extraction (oie) techniques are gradually attracting more and more attention @xcite @xcite @xcite @xcite as they build a bridge between language to knowledge. oie tasks are generally designed for its information extraction requirements, which vary from verbal relations between entities @xcite , nominal attributes @xcite @xcite , and adverbial components (e.g., time) @xcite . even for the same type of information, the required facts may still differ. the adaptability problem in the oie field has not been well addressed. there are two primary methodologies to obtain an oie system: the rulebased approach and the end-to-end learning-based approach. the rule-based approaches @xcite @xcite @xcite use human-written or bootstraplearned rules to convert linguistic structures of sentences into target facts. the end-to-end learning approaches @xcite @xcite @xcite first build a dataset @xmath0, @xmath1. however, these methodologies develop a specific machine for every single task. when the requirements change, one must rewrite the complex rule system or re-annotate the data and then retrain the model. these methodologies fail to meet the need for strong adaptability in the oie field. recently, a concept called open information expression (oix) was proposed by @xcite to address the adaptability issue of oie algorithms. the idea of oix is to introduce an intermediate layer between the language and oie, which can express the sentence without information loss and be easily adapted to various oie tasks. @xcite proposed a standard, called open information annotation (oia), to implement oix. the oia standard defines an annotation criterion of fact oie2016 re-oie2016 @xmath2, ms. lee, @xmath3, ms. lee, for @xmath4, ms. lee, lily, she is responsible for @xmath5, ms. lee, jimmy, she is responsible for @xmath6, ms. lee, lily and jimmy, she is responsible for @xmath7: facts defined in different oie tasks, for the expression "ms. lee, the headmaster, told lily and jimmy she is responsible for this.". with oia, we can easily adapt to these various oie standards using simple rules. following the methodology of oix/oia, we implement an adaptable and efficient oie system -oie@oia. the framework of oie@oia shown in figure 1 has two components. the first one is the oia generator, which converts a sentence into an oia graph. we annotate a large oia dataset (containing 12,543 training samples, 2,002 development samples, and 2,077 testing samples), develop an efficient learning algorithm to learn and inference the oia graph, and finally build an end-to-end oia graph learner. the second component is a group of adaptors, one for each oie task. we show three popular oie tasks focused on in this paper in the figure. furthermore, one can write new adaptors for new oie tasks, which are very simple, as shown in the following sections. the oie@oia system achieves the sota (or comparable) performance on three oie tasks: oie2016, re-oie2016, and carb, verifying the adaptability of our oie@oia system. furthermore, our oie@oia only needs 12k samples to train, whereas existing end-to-end oie methods typically need millions of training samples. this verifies the efficiency of our oie@oia system. the contribution of this work is as follows: ## oie@oia in this section, we introduce the oie@oia framework, compare it with existing methodologies, and finally discuss its capability and limitation. ## learning the oia graphs converting a sentence into the oia graph is the central operation of the oie@oia framework. we build an oia dataset using active-learning-powered human labeling to implement such an operation. then, we learn equivalent variants of the oia graphs -word-oia graphs and convert them back to oia-graphs, which overcomes the difficulty of learning the structures of the oia graphs. ## experiment i: oia learning the experiment is conducted on the paddlepaddle deep learning platform foot_3 , and the pre-trained bert model is provided by the paddlenlp project foot_4 . following @xcite , the hidden size of mlp (edge) and mlp (label) is set to 500 and 100, respectively. the hidden sizes of mlps used in node attribute predictions are set to 500. the model is trained with the classifier's dropout rate being set to 0.1 and adam optimizer with a learning rate of 10 -5 . α and β in loss function are searched using grid search on dev set and set to 0.2 and 0.4, respectively. evaluation metric. in this experiment, the evaluation metrics for all measurements are based on exact match, that is, score 1 if exactly the same, otherwise 0. for nodes, we compare their node expressions. for a word-oia node, the node expression is the word index in the sentence; for an oia node, the node expression is the phrase based on the word indexes it contains. for edges, we compare the triplets @xmath8, edge_label, end_node_expression>. for graphs, we test whether the two graphs' node sets and edge sets are exactly the same. ## experiment ii: oie@oia we further evaluate the applicability of oia as an intermediate layer between language and oie, i.e., oie@oia, on three tasks: oie2016, re-oie2016, carb. we compared oie@oia with six rulebased systems and five learning-based systems. evaluation metric. the performances of baseline systems on oie2016 are from @xcite while that on re-oie2016 and carb are from @xcite , except that nestie @xcite is implemented using the code from the author. we evaluate the extraction results of the proposed methods, oie based on rule oia and nestie with metric auc and optimal f1 following the setting of the released codes 6 . the performance of our oie@oia system is shown in we think the reason for this phenomenon is three-fold: firstly, compared with the annotation of oia, the annotation of a single oie task is 6 www.github.com/zhanjunlang/span_oie , and www.github.com/youngbin-ro/multi2oie sparse. given a sentence, it will only annotate phrases with interesting relationships. for the other phrases and relationships, it will not annotate. in contrast, in oia, all the phrases and relationships will be annotated. so based on a single sample, the annotation in oia is much more informative than that in any single oie task. secondly, if treating the recognition of a type of facts as a task, learning of oia can be seen as a multi-task learning scenario, and different tasks can augment each other. thirdly, oia is designed as an intermediate layer between language and oie. thus, during the standard design of oia, it has considered the compatibility between oia and oie, which makes it easier to adapt oia to different oie tasks. ## conclusion we introduce an adaptable and efficient open information extract system called oie@oia. to implement oie@oia, we annotate and release an oia dataset containing about 16k sentences, design an efficient learning algorithm, and build an easy-to-implement rule system to adapt oia graphs to different oie tasks. empirical studies on three popular oie tasks show that our oie@oia system can achieve new sota performances on these tasks, using only 12k training sentences. it verifies the great advantage of our system in both effectiveness and efficiency over the previous learning-based baselines, which usually require millions of training samples to achieve comparable performance.