MILIE : Modular & Iterative Multilingual Open Information Extraction (2022)

Abstract: Open Information Extraction (OpenIE) is the task of extracting (subject, predicate, object) triples from natural language sentences. Current OpenIE systems extract all triple slots independently. In contrast, we explore the hypothesis that it may be beneficial to extract triple slots iteratively: first extract easy slots, followed by the difficult ones by conditioning on the easy slots, and therefore achieve a better overall extraction. Based on this hypothesis, we propose a neural OpenIE system, MILIE, that operates in an iterative fashion. Due to the iterative nature, the system is also modularit is possible to seamlessly integrate rule based extraction systems with a neural end-to-end system, thereby allowing rule based systems to supply extraction slots which MILIE can leverage for extracting the remaining slots. We confirm our hypothesis empirically: MILIE outperforms SOTA systems on multiple languages ranging from Chinese to Arabic. Additionally, we are the first to provide an OpenIE test dataset for Arabic and Galician.

## introduction open information extraction (openie) aims to extract structured facts in the form of (subject, relation, object)-triples from natural language sentences @xcite . for example, given a sentence, "barrack obama became the us president in the year 2008", an openie system is expected to extract the following triples: (barrack obama; became; us president) and (barrack obama; became @xcite . we refer to subject, predicate and the object of the triple as slots of a triple. openie extractions are schemafree, human understandable intermediate representations of facts in source texts @xcite . they are useful in a variety of information extraction end tasks such as summarization @xcite , question answering @xcite and automated schema extraction @xcite . the various slots of a triple are dependent on each other and hence an error in one slot renders the entire extraction unusable. we hypothesize that triple extraction errors largely stem from the difficulty of extracting certain slots of a triple and said difficulty may depend on the sentence construction and the language. for example, "barrack obama became the us president in the year 2008" contains two triples (barrack obama; became; us president) and (barrack obama; became us @xcite . extracting the predicate, "became us president in", for the second triple is tricky, because the object of the first triple (us president) overlaps with the predicate of the second triple. but if the extraction system was provided with the object, @xcite , and then asked to extract a triple conditioned on this object, the predicate extraction would be easier. this is precisely the hypothesis we wish to investigate -is it easier to extract certain slots of a triple, say subjects, compared to other slots, such as objects, and is it possible to improve performance by leveraging specific slot extraction orders? given the hypothesis, we propose milie, a modular & iterative multilingual open information extraction system, which iteratively extracts the different slots of a triple. the iterative nature allows for (1) studying the effects of a slot extractions on the remaining extractions, (2) extracting easier triple slots followed by harder ones, (3) aggregating different slot extraction orders as a mixture of experts, and (4) integrating slots supplied by an external rule-based system, resulting in a hybrid system. the latter offers a system that combines the best of neural and rule based systems, e.g. by using a rule-based system to extract high precision slots on which the neural system is conditioned. we empirically confirm our hypothesis: the iterative nature of milie outperforms several sota systems. it proves especially useful for zero-shot multilingual extraction, which we evaluated on five different low resource languages. additionally we show how milie can leverage rule-based slot extraction by conditioning on them to predict the remaining parts of the triple. therefore milie is a boon for existing applications wishing to transition from a rule based information extraction system to a neural one, because milie would allow using the rule-based system to compensate for the lack of exhaustive training data. finally, we perform linguistic analyses that uncovers useful insights on how different languages either make it easy or difficult for openie systems to extract individual elements of the triple. our contributions are summarized as follows: ## milie the backbone of our system is the iterative procedure (section 2.1), which allows us to investigate our hypothesis. the iterative procedure allows us to extract triple slots in various pathway orders, which results in a series of possible aggregation schemes (section 2.2). to create a strong iterative system, the training paradigm (section 2.3 needs to consider two aspects: (1) it needs to prepare incomplete triple extractions which represent incomplete triple extractions the system is expected to predict; (2) it creates negative samples that allow for teaching the system when to not continue with an extraction due to a prior error. with the iterative nature we also integrate rule-based systems (section 2.4) as well as elegantly handle the specific case of n-ary extractions, where more than 3 slots need to be extracted (section 2.5). ## analysis we would like to analyze that the ability of milie to extract triples using different extraction patterns results in improved performance on multilingual data. for this, we compare milie with the water filling aggregation against milie with different extraction pathways.we also compared with a dynamic decoding scheme where milie chooses a decoding pathways based on the sentence. to do this we split a part of the english training set and for each sentence in the split we record the extraction pathway that provides the best f1 score milie as per carb evaluation. we then use this as training data for training another mbert model which classifies each sentence in one of the six classes where each class represents an extraction pathway. additionally, ## related work openie systems largely come in two flavors, (1) unsupervised openie systems that use fine grained rules based on dependency parse trees @xcite @xcite , and (2) supervised neural openie systems, trained end-to-end with large training datasets @xcite @xcite . neural openie systems characterize openie as either a sequence tagging task @xcite , span prediction task or a sequence generation task @xcite . however all these prior approaches extract a triple in a single step, which does not allow us to study the effect of extracting a specific slot and its effect on extracting the rest of the triple. neural generative approaches to openie use sequence-to-sequence models with a copy mechanism for generating triples @xcite . the copy mechanism needs to be learned and is often a source of errors. a series of alternative approaches cast openie as a sequence tagging task where each token is tagged as subject, predicate or object using a bio like tagging scheme @xcite @xcite . in these systems, all triple slots are extracted simultaneously and it is therefore not possible to condition on easier slots. more closely related to our work is spanoie @xcite and multi2oie @xcite , which first extracts the predicate and then all additional arguments. like us, multi2oie @xcite addresses multilinguality by leveraging a pretrained bert model @xcite for transfer learning. in contrast, through our iterative nature, it is possible to enrich the extractions in other languages if rule based models or other models (e.g. ner recognizers) exist to provide input for a triple slot. imojie @xcite iteratively extracts entire triples from a sentence: first a triple is extracted, which is added to the input to extract the next triple. in contrast, our work iteratively extracts the slots of a single triple, which allows us to condition on the easier slots and therefore obtain higher quality triples. @xcite propose openie6, a bert based system, with iterative grid labelling and linguistic constraint based training. such lingusitic constraints with soft penalties cannot be readily ported to other languages since such constraints use head verb based heuristics. consequently oie 6 is evaluated only on english. ## conclusion we introduced milie, a modular & iterative multilingual openie system. we confirmed our hypothesis that it is beneficial to extract triple slots iteratively which allows us to extract easier slots first. our experiments on english as well as five low resource languages uncovered that, with the exception of arabic, triples are easier to extract if the predicate is extracted first followed by the subject and object. more importantly we discovered that extracting triples using multiple extraction pathways is superior than the standard single extractions especially in the multilingual setting. we also demonstrated how milie can be combined seamlessly with rule based systems for improving performance. although our experiments were focused on the openie task, we believe that the insights gained can be translated to other information extraction tasks with coupled extractions. we plan to explore such connections in the future.