A Joint Neural Model for Information Extraction with Global Features (2020)

Abstract: Most existing joint neural models for Information Extraction (IE) use local task-specific classifiers to predict labels for individual instances (e.g., trigger, relation) regardless of their interactions. For example, a victim of a die event is likely to be a victim of an attack event in the same sentence. In order to capture such cross-subtask and cross-instance inter-dependencies, we propose a joint neural framework, OneIE, that aims to extract the globally optimal IE result as a graph from an input sentence. OneIE performs end-to-end IE in four stages: (1) Encoding a given sentence as contextualized word representations; (2) Identifying entity mentions and event triggers as nodes; (3) Computing label scores for all nodes and their pairwise links using local classifiers; (4) Searching for the globally optimal graph with a beam decoder. At the decoding stage, we incorporate global features to capture the cross-subtask and cross-instance interactions. Experiments show that adding global features improves the performance of our model and achieves new state of-the-art on all subtasks. In addition, as OneIE does not use any language-specific feature, we prove it can be easily applied to new languages or trained in a multilingual manner.

## introduction information extraction (ie) aims to extract structured information from unstructured texts. it is a complex task comprised of a wide range of subtasks, such as named, nominal, and pronominal mention extraction, entity linking, entity coreference resolution, relation extraction, event extraction, and event coreference resolution. early efforts typically perform ie in a pipelined fashion, 1 http://blender.cs.illinois.edu/software/ oneie which leads to the error propagation problem and disallows interactions among components in the pipeline. as a solution, some researchers propose joint inference and joint modeling methods to improve local prediction @xcite @xcite @xcite @xcite @xcite . due to the success of deep learning, neural models have been widely applied to various ie subtasks @xcite @xcite . recently, some efforts @xcite revisit global inference approaches by designing neural networks with embedding features to jointly model multiple subtasks. however, these methods use separate local task-specific classifiers in the final layer and do not explicitly model the interdependencies among tasks and instances. figure 1 shows a real example where the local argument role classifier predicts a redundant person edge. the model should be able to avoid such mistakes if it is capable of learning and leveraging the fact that it is unusual for an elect event to have two person arguments. ## task given a sentence, our oneie framework aims to extract an information network representation @xcite , where entity mentions and event triggers are represented as nodes, and relations and event-argument links are represented as edges. in other words, we perform entity, relation, and event extraction within a unified framework. in this sec-tion, we will elaborate these tasks and involved terminologies. entity extraction aims to identify entity mentions in text and classify them into pre-defined entity types. a mention can be a name, nominal, or pronoun. for example, "kashmir region" should be recognized as a location (loc) named entity mention in figure 2 . relation extraction is the task of assigning a relation type to an ordered pair of entity mentions. for example, there is a part-whole relation between "kashmir region" and "india". event extraction entails identifying event triggers (the words or phrases that most clearly express event occurrences) and their arguments (the words or phrases for participants in those events) in unstructured texts and classifying these phrases, respectively, for their types and roles. an argument can be an entity, time expression, or value (e.g., money, job-title, crime). for example, in figure 2 , the word "injured" triggers an injure event and "300" is the victim argument. we formulate the task of extracting information networks as follows. given an input sentence, our goal is to predict a graph @xmath0, e), where v and e are the node and edge sets respectively. each node v @xmath1, b i , l @xmath2, where a and b are the start and end word indices, and l is the node type label. each edge e @xmath3, j, l @xmath4, whereas i and j denote the indices of involved nodes. for example, in figure 2 , the trigger "injured" is represented as 7, 7, injure , the entity mention "kashmir region" is represented as 10, 11, loc , and the event-argument edge between them is 2, 3, place . ## approach as figure 2 illustrates, our oneie framework extracts the information network from a given sentence in four steps: encoding, identification, classification, and decoding. we encode the input sentence using a pre-trained bert encoder @xcite and identify entity mentions and event triggers in the sentence. after that, we compute the type label scores for all nodes and pairwise edges among them. during decoding, we explore possible information networks for the input sentence using beam search and return the one with the highest global score. ## conclusions and future work we propose a joint end-to-end ie framework that incorporates global features to capture the inter-dependency between knowledge elements. experiments show that our framework achieves better or comparable performance compared to the state of the art and prove the effectiveness of global features. our framework is also proved to be languageindependent and can be applied to other languages, and it can benefit from multi-lingual training. in the future, we plan to incorporate more comprehensive event schemas that are automatically induced from multilingual multimedia data and external knowledge to further improve the quality of ie. we also plan to extend our framework to more ie subtasks such as document-level entity coreference resolution and event coreference resolution.