LOME : Large Ontology Multilingual Extraction (2021)

Abstract: We present LOME, a system for performing multilingual information extraction. Given a text document as input, our core system identifies spans of textual entity and event mentions with a FrameNet (Baker et al., 1998) parser. It subsequently performs coreference resolution, fine-grained entity typing, and temporal relation prediction between events. By doing so, the system constructs an event and entity focused knowledge graph. We can further apply third-party modules for other types of annotation, like relation extraction. Our (multilingual) first-party modules either outperform or are competitive with the (monolingual) state-of-the-art. We achieve this through the use of multilingual encoders like XLM-R (Conneau et al., 2020) and leveraging multilingual training data. LOME is available as a Docker container on Docker Hub. In addition, a lightweight version of the system is accessible as a web demo.

## introduction as information extraction capabilities continue to improve due to advances in modeling, encoders, and data collection, we can now look (back) toward making richer predictions at the documentlevel, with a large ontology, and across multiple languages. recently, @xcite noted that despite a growth of open-source nlp software in general, there is still a lack of available software for knowledge extraction. we wish to provide a starting point that allows others to build increasingly comprehensive document-level knowledge graphs of events and entities from text in many languages. 1 therefore, we demonstrate lome, a system for multilingual information extraction with large ontologies. figure 1 shows the high-level pipeline by following a multilingual input example. a sentence-level parser identifies both ingestion events and their arguments. to connect these events cross-sententially, the system clusters coreferent mentions and predicts the temporal relations between the events. lome, which supports finegrained entity types, additionally labels entities like the rabbit with living thing/animal. several prior packages have also used advances in state-of-the-art models to build comprehensive information extraction systems. @xcite present an event, relation, and entity extraction and coreference system for three languages: english, russian, and ukrainian. @xcite extend that work to support cross-media documents. however, both of these systems consist of languagespecific models that operate on monolingual documents after first identifying the language. on the other hand, work prioritizing coverage across tens or hundreds of languages is limited in their scope in extraction @xcite . like prior work, lome is focused on extracting entities and events from raw text documents. however, lome is language-agnostic; all components prioritize multilinguality. using xlm-r @xcite as the underlying encoder paves the way for both training on multilingual data (where it exists) and inference in many languages. 2 our pipeline includes a full framenet parser for events and their arguments, neural coreference resolution, an entity typing model over large ontologies, and temporal resolution between events. our system is designed to be modular: each component is trained independently and tuned on task-specific data. to communicate between modules, we use concrete @xcite , a data schema used in other text processing systems @xcite . one advantage of using a stan-figure 1 : architecture of lome. the system processes text documents as input and first uses a framenet parser to detect entities and events. then, a suite of models enrich the entities and events with additional predictions. each individual model can be trained and tuned independently, ensuring modularity of the pipeline. annotations between models are transferred using concrete, a data schema for nlp. ## tasks the overarching application of lome is to extract an entity-and event-centric knowledge graph from a textual document. in particular, we are interested in using these graphs to support a multilingual schema learning task (kairos 3 ) for which data has been annotated by the ldc @xcite . as a result, some parts of lome are designed for compatibility with the kairos event and entity ontology. nonetheless, there is significant overlap with publicly available datasets, which we describe for those tasks. figure 1 presents the architecture of our pipeline. besides the framenet parser, which is run first, the remaining modules can be run in any order, if at all. in addition, our use of a standardized data schema for communication allows for the integration of third-party systems. in this section, we will go into further detail for each task. ## usage we present two methods to interact with the pipeline. the first is a docker container which contains the libraries, code, and trained models of our pipeline. this is intended to run on batches of documents. as a lighter demo of some of the system capabilities, we also have a web demo intended to interactively run on shorter documents. docker our docker image foot_13 consists of the four core modules: framenet parser, coreference resolution, entity typing, and temporal resolution. furthermore, there are two options for entity typing: a fine-grained hierarchical model (with the aida typing ontology) and a coarse-grained model (with the kairos typing ontology). the container and documentation is available on docker hub. as some modules depend on gpu libraries, the image also requires nvidia-docker support. since there is a high start-up (time) cost for using docker and loading models, we recommend using this container for batch processing of documents. further instructions for running can be found on the lome docker hub page. web demo we make a few changes for the web demo. 19 to reduce latency, we preload the models into memory and we do not write the concrete communications to disk. at the cost of modularity, this makes the demo lightweight and fast, allowing us to run it on a single 16gb cpu-only server. to present the predictions, our front-end uses allennlp-demo. 20 in addition, the web demo is currently limited to framenet parsing and coreference resolution, as other models will increase latency and may impede usability. the web demo is intended to highlight only some of the system's capabilities, like its ability to process multilingual documents. ## conclusions to facilitate increased interest in multilingual document-level knowledge extraction with large ontologies, we create and demonstrate lome, a system for event and entity knowledge graph creation. given input text documents, lome runs a full framenet parser, coreference resolution, finegrained entity typing, and temporal relation prediction. furthermore, each component uses xlm-r, allowing our system to support a broader set of languages than previous systems. the pipeline uses a standardized data schema, which invites extending the pipeline with additional modules. by releasing both a docker image and presenting a lightweight web demo, we hope to enable the community to build on top of lome for even more comprehensive information extraction.