AMR Parsing with Latent Structural Information (2020)

Abstract: Abstract Meaning Representations (AMRs) capture sentence-level semantics structural representations to broad-coverage natural sentences. We investigate parsing AMR with explicit dependency structures and interpretable latent structures. We generate the latent soft structure without additional annotations, and fuse both dependency and latent structure via an extended graph neural networks. The fused structural information helps our experiments results to achieve the best reported results on both AMR 2.0 (77.5% Smatch F1 on LDC2017T10) and AMR 1.0 ((71.8% Smatch F1 on LDC2014T12).

## introduction abstract meaning representations (amrs) @xcite model sentence level semantics as rooted, directed, acyclic graphs. nodes in the graph are concepts which represent the events, objects and features of the input sentence, and edges between nodes represent semantic relations. amr introduces re-entrance relation to depict the node reuse in the graphs. it has been adopted in downstream nlp tasks, including text summarization @xcite , question answering @xcite and machine translation @xcite . amr parsing aims to transform natural language sentences into amr semantic graphs. similar to constituent parsing and dependency parsing @xcite , amr parsers mainly employ two parsing techniques: transitionbased parsing @xcite @xcite @xcite use a sequence of transition actions to incrementally construct the graph, while graphbased parsing @xcite @xcite divides the task into concept identification and relation extraction stages and then generate a full amr graph with decoding algorithms such as greedy and maximum spanning tree (mst). additionally, reinforcement learning @xcite and sequence-to-sequence @xcite have been exploited in amr parsing as well. previous works @xcite shows that structural information can bring benefit to amr parsing. illustrated by figure 1 , for example syntactic dependencies can convey the main predicate-argument structure. however, dependency structural information may be noisy due to the error propagation of external parsers. moreover, amr concentrates on semantic relations, which can be different from syntactic dependencies. for instance, in figure 1 , amr prefers to select the coordination (i.e. "and") as the root, which is different from syntactic dependencies (i.e. "came"). given the above observations, we investigate the effectiveness of latent syntactic dependencies for amr parsing. different from existing work @xcite , which uses a dependency parser to provide explicit syntactic structures, we make use of a two-parameter distribution @xcite to induce latent graphs, which is differentiable under reparameterization @xcite . we thus build a end-to-end model for amr parsing with induced latent dependency structures as a middle layer, which is tuned in amr training and thus can be more aligned to the need of amr structure. for better investigating the correlation between induced and gold syntax, and better combine the strengths, we additionally consider fusing gold and induced structural dependencies into an align-free amr parser @xcite . specifically, we first obtain the input sentence's syntactic dependencies foot_0 and treat the input sentence as prior of the probabilistic graph generator for inferring the latent graph. second, we propose an extended graph neural network (gnn) for encoding above structural information. subsequently we feed the encoded structural information into a two stage align-free amr parser @xcite for promoting amr parsing. to our knowledge, we are the first to incorporate syntactic latent structure in amr parsing. experimented results show that our model achieves 77.5% and 71.8% smatch f1 on standard amr benchmarks ldc2017t10 and ldc2014t12, respectively, outperforming all previous best reported results. beyond that, to some extent, our model can interpret the probabilistic relations between the input words in amr parsing by generating the latent graph 2 . ## baseline: align-free amr parsing we adopt the parser of @xcite as our baseline, which treats amr parsing as sequenceto-graph transduction. ## model the overall structure of our model is shown in figure 2 . first, we use an external dependency parser @xcite to obtain the explicit structural information, and obtain the latent structural information via a probabilistic latent graph generator. we then combine both explicit and latent structural information by encoding the input sentence through an extended graph neural network. finally, we incorporate our model with an alignfree amr parser for parsing amr graphs with the benefit of structural information. ## related work transition-based amr parsers @xcite @xcite @xcite suffer from the lack of annotated alignments between words and concept notes is crucial in these models. @xcite treat the alignments as an latent variable for their probabilistic model, which jointly obtains the concepts, relations and alignments variables. sequence-to-sequence amr parsers transform amr graphs into serialized sequences by external traversal rules, and then restore the generated the amr sequence to avoid aligning issue @xcite . moreover, zhang et al. (2019a) extend a pointer generator @xcite , which can generate a node multiple times without alignment through the copy mechanism. with regards to latent structure, naradowsky et al. (2012) the latent constituent trees are shallower than human annotated, and it can boost the performance of downstream nlp tasks (e.g., text classification). @xcite and @xcite employ self-attention and bi-affine attention mechanism respectively to generate soft connected graphs, and then adopt gnns to encode the soft structure to take advantage from the structural information to their works. gcn and its variants are increasingly applied in embedding syntactic and semantic structures in nlp tasks @xcite @xcite . syntactic-gcn tries to alleviate the error propagation in external parsers with gates mechanism, it encodes both relations and labels with the gates, and filters the output of each gcn layer over the dependencies. @xcite . @xcite encodes amr graphs via gcn to promote the amr-to-text generation task. ## conclusion we investigate latent structure for amr parsing, and we denote that the inferred latent graph can interpret the connection probabilities between input words. experiment results show that the latent structural information improve the best reported parsing performance on both amr 2.0 (ldc2017t10) and amr 1.0 (ldc2014t12). we also propose to incorporate the latent graph into other multi-task learning problems @xcite ., sentence: what advice could you give me? dependency: amr: latent matrix: fused matrix: advice you :arg0 :arg2 root i possible amrunkonwn :arg1-of :arg1 give coul d advice dobj det root you ? what me punct dobj nsubj aux sentence: you could go to the library on saturdays and do a good 8 hours of studying there. dependency: amr: latent matrix: fused matrix: go root nsubj you could to the library on saturdays and do a good 8 hours studying there . of aux nmod nmod cc conj punct case det case dobj det amod nummod acl mark advmod :arg1 root and possible study-01 :op1 :op2 go you librar y dateentity saturday temporalquantity good -02 8 hour :time :weekday :arg0 :arg4 :arg0 :location :duration :arg1-of :quant :unit figure 6: an analysis of the sentence "you could go to the library on saturdays and do a good 8 hours of studying there.".