O pen UE : An Open Toolkit of Universal Extraction from Text (2020)

Abstract: Natural language processing covers a wide variety of tasks with token-level or sentence-level understandings. In this paper, we provide a simple insight that most tasks can be represented in a single universal extraction format. We introduce a prototype model and provide an open-source and extensible toolkit called OpenUE for various extraction tasks. OpenUE allows developers to train custom models to extract information from the text and supports quick model validation for researchers. Besides, OpenUE provides various functional modules to maintain sufficient modularity and extensibility. Except for the toolkit, we also deploy an online demo with restful APIs to support real-time extraction without training and deploying. Additionally, the online system can extract information in various tasks, including relational triple extraction, slot & intent detection, event extraction, and so on. We release the source code, datasets, and pre-trained models to promote future researches in http://github.com/zjunlp/openue .

## introduction a large number of natural language processing (nlp) tasks exist to analyze various aspects of human language. most of them focus on tokenlevel classification (e.g., named entity recognition, slot filling, and argument role classification) or sentence-level understanding (e.g., relation classification, intent detection, and event classification). previous researchers usually use specifically designed neural network architectures for those tasks. note that most of those tasks share similar encoder and decoder modules @xcite . it is beneficial to achieve a unified model for all diverse information extraction tasks without task-specific architectures. intuitively, we rethink most of the previous tasks and find that most tasks fall in two categories: token-oriented tasks, where the goal is to predict labeled spans (e.g., named entities, slots, aspects) and sentence-oriented tasks, where the goal is to predict labels regarding the semantic understanding of sentences (e.g., relations, intents, sentiments). the commonality of these tasks inspires us whether there is a universal framework. moreover, in the domain of efficient human annotation interfaces, it is already standard to use unified representations for a wide variety of nlp tasks. taking the brat @xcite annotation as an example, this framework has a single unified format which consists of spans (e.g., the span of an entity), and labeled relations between the spans (e.g., "born-in" and "live-in"). motivated by this, we formulate those tasks regarding both token and sentence as universal extraction and design a simple unified model. our prototype model studies the possibility of bridging the gap between tasks with single architecture and providing future insight for unified natural language understanding. furthermore, as there is a lack of practical and stable toolkit to support the implementation, deployment and evaluation of those tasks, we develop a toolkit which is a compliment for existing toolsets such as spacy foot_0 for named entity recognition (ner), tagme @xcite for entity linking (el), openke @xcite for knowledge embedding, stanford openie @xcite for open information extraction, and openner @xcite for relation extraction. to be specific, we develop an open and extensible toolkit named "openue". the toolkit prioritizes operational efficiency based on tensorflow ## application scenarios openue is designed for various tasks, including relational triple extraction, slot filling, intent detection, event extraction, and knowledge extraction from the web, etc. as shown in figure 1 , we give some examples of these application scenarios. ## toolkit design and implementation to implement a single prototype model for all tasks, we introduce our openue approach, as figure 2 shows. we design the prototype implementation with separated sentence classification and sequence labeling modules based the following three empirical observations: 1) joint optimization of sequence labeling and sentence classification requires labor-intensive hyper-parameters fine-tuning @xcite ; 2) sentence classification first can filter out vast amounts of instances which can reduce computation for sequence labeling; 3) sentence labels with additional information (like a machine reading comprehension style) can provide more signals for sequence labeling @xcite . to design the toolkit, we build a unified underlying platform. openue encapsulates various data processing and training strategies, which implies that developers can maximize the reuse of code to avoid redundant and unnecessary model implementations. we design openue based on tensorflow and pytorch, enabling developers to train models on gpus for operational efficiency. we introduce the model and design details in the following sections. ## experiment and evaluation in this section, we evaluate our toolkit openue on several datasets in different tasks. the experimental results illustrate that our implementation with openue can achieve comparable or even better performance compared to some state-of-the-art results. ## online system besides the toolkit, we also release an online system in http://openue.top . as shown in figure 3 , we train models in different scenarios with multilingual support (english and chinese) and deploy the model for online access. the online system can be directly applied to extract structured facts, events, and slots & intents from plain text. we also visualize the graph of relational triples and the probabilities of sentence logits (e.g., relation probabilities) to help to analyze model performance. additionally, we deploy a schema-based never ended learner that can extract factual knowledge from the web. our system has already obtained millions of facts. more details can be shown in the https://openue-docs.readthedocs.io/ en/latest/ . moreover, we provide open restful apis foot_5 for diverse tasks by openue. more tasks, such as aspectbased sentiment analysis, semantic role labeling, and more domains, will be supported in the future. ## conclusion we provide a simple insight that lots of nlp tasks can be represented in a single format. to this end, we provide a prototype model implementation of universal extraction and introduce an open and extensible toolkit, namely, openue. we conduct extensive experiments which demonstrate that the models implemented by openue are efficient, effective, and can achieve comparable performance compared to the state-of-the-art results. furthermore, we also provide an online system with restful apis for meeting real-time extraction without training and deploying. in the future, we plan to utilize the multitask learning or meta-learning algorithms to enhance extraction performance. we will provide long-term maintenance to fix bugs and meet new requests.