A bstract M eaning R epresentation Guided Graph Encoding and Decoding for Joint Information Extraction (2021)

Abstract: The tasks of Rich Semantic Parsing, such as Abstract Meaning Representation (AMR), share similar goals with Information Extraction (IE) to convert natural language texts into structured semantic representations. To take advantage of such similarity, we propose a novel AMR-guided framework for joint information extraction to discover entities, relations, and events with the help of a pre-trained AMR parser. Our framework consists of two novel components: 1) an AMR based semantic graph aggregator to let the candidate entity and event trigger nodes collect neighborhood information from AMR graph for passing message among related knowledge elements; 2) an AMR guided graph decoder to extract knowledge elements based on the order decided by the hierarchical structures in AMR. Experiments on multiple datasets have shown that the AMR graph encoder and decoder have provided significant gains and our approach has achieved new state-of-the-art performance on all IE subtasks.

## introduction information extraction (ie) aims to extract structured knowledge as an information network @xcite from unstructured natural language texts, while semantic parsing attempts to construct a semantic graph to summarize the meaning of the input text. since both of them focus on extracting the main information from a sentence, the output information networks and semantic graphs have a lot in common in terms of node and edge semantics. in an example shown in figure 1 , many knowledge elements in the information network can be perfectly matched to certain nodes in the semantic graph with similar semantic meanings. moreover, these two types of graphs may also be similar with regard to network topology. specifically, the nodes that are 1 the programs are publicly available for research purpose at https://github.com/zhangzx-uiuc/amr-ie . neighbors or connected via a few hops in the semantic graph are also likely to be close to each other in the corresponding information network. in figure 1 we can see that "scott peterson", which acts as a shared argument for two event triggers "murdering" and "faces", is also directly linked to two main predicates murder-01 and face-01 in the semantic graph. from a global perspective, an information network can be approximately considered as a subgraph of semantic parsing, where the ie nodes are roughly a subset of the nodes in the semantic graph while maintaining similar inter-connections. to further exploit and make use of such similarities for information extraction, we propose an intuitive and effective framework to utilize information from semantic parsing to jointly extract an in-formation network composed of entities, relations, event triggers and their arguments. we adopt abstract meaning representation (amr) @xcite which contains rich semantic structures with fine-grained node and edge types as our input semantic graphs. compared with previous ie models, our proposed model mainly consists of the following two novel components. ## problem formulation we focus on extracting entities, relations, event triggers and their arguments jointly from an input sentence to form an information network. note that the amr graphs in our model are not required to be ground-truth but are generated by pretrained amr parsers. therefore, we do not incorporate additional information and our problem settings are identical to typical joint information extraction approaches such as dygie++ @xcite and oneie @xcite . given an input sentence @xmath0, w 2 , • • • , w n }, we formulate our problem of joint information extraction as follows. entity extraction entity extraction aims to identify word spans as entity mentions and classify them into pre-defined entity types. given the set of entity types e, the entity extraction task is to output a collection e of entity mentions: ## our approach given an input sentence s, we first use a pretrained transformer-based amr parser @xcite to obtain the amr graph for s. we then use roberta @xcite to encode each sentence to identify entity mentions and event triggers as candidate nodes. after that, we map each candidate node to amr nodes and enforce message passing using a gatbased semantic graph aggregator to capture global inter-dependency between candidate nodes. all the candidate nodes and their pairwise edges are then passed through task-specific feed-forward neural networks to calculate score vectors. during decoding, we use the hierarchical structure in each amr graph as a condition to decide the order in beam search and find the best candidate graph with the highest global score. ## related work some recent efforts have incorporated dependency parsing trees into neural networks for event extraction @xcite and relation extraction @xcite . for semantic role labeling (srl), @xcite manages to exploit the similarity between srl and open domain ie by creating a mapping between two tasks. @xcite employ amr as a more concise input format for their ie models, but they decompose each amr into triples to capture the local contextual information between nodes and edges, while the node information is not disseminated in a global graph topology. @xcite proposes a subgraph matching based method to extract biomedical events from amr graphs, while @xcite uses an additional gcn based encoder for obtaining better word representations. besides, graph neural networks are also widely used for event extraction @xcite @xcite and relation and entity extraction @xcite @xcite . graph neural networks also demonstrate effectiveness to encode other types of intrinsic structures of a sentence, such as knowledge graph @xcite , document-level relations @xcite @xcite , and selfconstructed graphs @xcite @xcite . however, all these approaches focus on single ie tasks while can not scale to extracting a joint information network with entities, relations, and events. there are some recent efforts that focus on building joint neural models for performing multiple ie tasks simultaneously, such as joint entity and relation extraction @xcite @xcite @xcite and joint event and entity extraction @xcite . dygie++ @xcite designs a joint model to extract entities, events, and relations based on span graph propagation, while oneie @xcite further makes exploits global features to facilitate the model to capture more global interactions. compared with the flat encoder in oneie, our proposed framework leverages a semantic graph aggregator to incorporate information from fine-grained amr semantics and enforce global interactions in the encoding phase. in addition, instead of a simple left-to-right sequential decoder, we creatively use the amr hierarchy to decide the decoding order of knowledge elements. both the amr-guided graph encoder and decoder are proven highly effective compared to their flat counterparts. ## conclusions and future work amr parsing and ie share the same goal of constructing semantic graphs from unstructured text. ie focuses more on a target ontology, and thus its output can be considered as a subset of amr graph. in this paper, we present two intuitive and effective ways to leverage guidance from amr parsing to improve ie, during both encoding and decoding phases. in the future, we plan to integrate amr graph with entity coreference graph so our ie framework can be extended to document level.