Linguistic Guidance for Sequence-to-Sequence AMR Parsing (2024)

Abstract: “The Abstract Meaning Representation (AMR) parsing aims at capturing the meaning of a sen-tence in the form of an AMR graph. Sequence-to-sequence (seq2seq)-based methods, utilizingpowerful Encoder-Decoder pre-trained language models (PLMs), have shown promising perfor-mance. Subsequent works have further improved the utilization of AMR graph information forseq2seq models. However, seq2seq models generate output sequence incrementally, and inac-curate subsequence at the beginning can negatively impact final outputs, also the interconnec-tion between other linguistic representation formats and AMR remains an underexplored domainin existing research. To mitigate the issue of error propagation and to investigate the guidinginfluence of other representation formats on PLMs, we propose a novel approach of LinguisticGuidance for Seq2seq AMR parsing (LGSA). Our proposed LGSA incorporates the very limitedinformation of various linguistic representation formats as guidance on the Encoder side, whichcan effectively enhance PLMs to their further potential, and boost AMR parsing. The resultson proverbial benchmark AMR2.0 and AMR3.0 demonstrate the efficacy of LGSA, which canimprove seq2seq AMR parsers without silver AMR data or alignment information. Moreover,we evaluate the generalization of LGSA by conducting experiments on out-of-domain datasets,and the results indicate that LGSA is even effective in such challenging scenarios.”

## introduction abstract meaning representation (amr) @xcite is a semantic representation framework that systematically converts the meaning of a sentence into a rooted, labeled, and directed acyclic graph (dag), where the nodes represent concepts and the edges denote semantic relations. figure 1 depicts an example of an amr graph. amr has gained significant traction in various downstream natural language processing (nlp) tasks, including information extraction @xcite @xcite ), text summarization @xcite , machine translation @xcite , and dialog system @xcite . the methods of amr parsing can be divided into four categories: two-stage parsing @xcite @xcite , transition-based parsing @xcite @xcite @xcite @xcite , graph-based parsing @xcite @xcite and sequence-to-sequence (seq2seq)based parsing @xcite @xcite @xcite @xcite . in contrast to other approaches, seq2seqbased methods view amr parsing as the process of transforming the input sentence into a linearized sequence of amr graph. this allows for the seamless integration of powerful pre-trained seq2seq transformer models, which have shown significant advancements in various nlp tasks. as a result, seq2seq-based methods have achieved remarkable performance, positioning them at the forefront of current research in amr parsing. seq2seq-based works employ various techniques and auxiliary features while the performance is still unsatisfactory. for instance, structure-aware encoding @xcite explicitly models syntax and semantics through complex source sequence linearization of the whole syntactic tree with semantic roles based on pure seq2seq encoder-decoder transformer. more recently, seq2seq-based methods applying pre-trained language models (plms) like bart @xcite , further improve seq2seq-based methods by a large margin. these approaches focus on enhancing amr parsers from various perspectives, such as amr graph linearization @xcite , revised decoder attention @xcite , curriculum learning @xcite , intermediate-task learning @xcite , graph pre-training @xcite , word-to-node alignment @xcite and instructiontuning @xcite . what becomes apparent is that the methods before the adoption of encoderdecoder plms involved complex linearization and the utilization of the other whole linguistic representation formats. they also incorporated a significant amount of structural information that might not have been necessary for the amr task, which could instead result in noise. the subsequent approaches shifted their focus towards better harnessing the inherent structural information and alignments within the amr data based on encoder-decoder pre-trained language models. however, the output amr graph of seq2seq parsers is contingent upon the previously generated subgraph, so the inaccurate sub-graph may lead to error propagation. building upon the above issues, we aim to demonstrate that using the very limited information of different linguistic formats as guidance could still prompt the plms to generate better output tokens and reduce error propagation in amr parsers, further improving their performance. in this paper, we introduce a novel model-agnostic approach of linguistic guidance for seq2seq amr parsing (lgsa). the proposed lgsa leverages the very limited other linguistic representations as guidance on the encoder side and prompts the seq2seq-based parsers to generate more accurate sub-graphs. specifically, we choose three commonly used linguistic representation formats: part-of-speech (pos) that can represent lexical features, dependency parsing (dp) that can represent syntactic features, and semantic dependency parsing (sdp) @xcite that can represent semantic features. we build lgsa on the top of two recent seq2seq amr parsers: spring @xcite and ancestors @xcite . experimental results on proverbial benchmark amr2.0 and amr3.0 demonstrate our proposed lgsa can further enhance the ability of the seq2seq amr parsers. also, ancestors with lgsa of sdp guidance even achieves the comparable performance of amrbart @xcite , which utilizes 200k silver amr data for the post graph pre-training based on bart. we also conduct out-of-domain (ood) experiments on three ood datasets new3, tlp, and bio, which confirm the effectiveness of the proposed lgsa method even in ood situations. amr parsing amr parsing is a fundamental task that transforms a given sentence into a corresponding amr graph. the methods of amr parsing can be divided into four categories: two-stage parsing, c c l 2 0 2 4 root can the girl help me . root help the girl me . possible-01 girl help-01 me arg0 arg0 arg1 arg1 bv nsubj det dobj punct arg0 arg1 arg1 arg0 the girl can help me . sentence pos pron verb aux noun det punct serialization the girl can help me . [det] [aux] [pron] [punct] [noun] [verb] the girl can help me . [root] lgsa-sdp the girl can help me . [root] lgsa-dp lgsa-pos (<@xmath0:arg0 (<@xmath1:arg1 (<@xmath2:@xmath3:arg1 (<@xmath4: serialization of our proposed lgsa, which utilizes the limited information of semantic dependency parsing (sdp), dependency parsing (dp), and part-of-speech (pos) as the linguistic guidance to prompt plms. the obtained linguistic guidance source sequence is input to sequence-to-sequence amr parsers to generate the corresponding linearized amr graph. dependency parsing and semantic dependency parsing dependency parsing (dp) is a fundamental task in natural language processing, aiming at representing the sentence in a format of tree structure. dp hopes to identify the syntactic relationships between words in a sentence. so dp is typically as syntactic features. the 2014 semeval @xcite shared task introduced a new representation of the sentence which focuses on the deep-semantic relations between words. semantic dependency parsing involves converting an input sentence into a directed acyclic graph called a semantic dependency graph (sdg). due to the similarity between sdg and amr graphs, which are both directed acyclic graphs, the structural information of sdg can serve as a supplement to amr. ## conclusions in this paper, we focus on the error propagation of seq2seq parsers and investigate to utilize a limited amount information of from other linguistic representation formats as guidance for seq2seq amr parsing to better utilize pre-trained language models. we propose a novel model-agnostic approach called linguistic guidance for seq2seq amr parsing (lgsa), which leverages part-of-speech (pos) information for lexical features, dependency parsing for syntactic features, and semantic dependency parsing for semantic features. experiments on amr2.0 and amr3.0 demonstrate that lgsa can incorporate additional linguistic information on the encoder side and enhance the ability of amr parsers. furthermore, we also demonstrate the effectiveness of lgsa in out-of-domain (ood) benchmarks including new3, tlp, and bio.