Sequence-to-sequence AMR Parsing with Ancestor Information (2022)

Abstract: AMR parsing is the task that maps a sentence to an AMR semantic graph automatically. The difficulty comes from generating the complex graph structure. The previous state-of-the-art method translates the AMR graph into a sequence, then directly fine-tunes a pretrained sequence-to-sequence Transformer model (BART). However, purely treating the graph as a sequence does not take advantage of structural information about the graph. In this paper, we design several strategies to add the important ancestor information into the Transformer Decoder. Our experiments show that we can improve the performance for both AMR 2.0 and AMR 3.0 dataset and achieve new state-of-the-art results.

## introduction abstract meaning representation (amr) @xcite is a graph that encodes the semantic meaning of a sentence. in figure 1a , we show the amr of the sentence: you told me to wash the dog. amr has been widely used in many nlp tasks @xcite @xcite . amr parsing is the task of mapping a sentence to an amr semantic graph automatically. a graph is a complex data structure which is composed of multiple vertices and edges. there are roughly four types of parsing strategies in previous work: however, purely treating the graph as a sequence may not take advantage of important information about the structure of the graph. when generating the last token dog in figure 1b , for example, the dotproduct attention layer in the transformer decoder attends to all the previous tokens and lets the model learn the weight of these tokens. however, if we can tell the model which tokens are its ancestors, like its parent is wash-01 and its grand-parent is tell-01 (see figure 1a ), it will make this token much easier to generate. adding graph structure has been demonstrated to be useful for the amr-to-text task @xcite @xcite . these approaches added the graph structure to the transformer encoder. therefore, we expect that adding structure in transformer decoder for amr parsing task will also be helpful. in this paper, we base our work on the seq-to-seq model of @xcite with the amr linearized by dfs traversal order. we introduce several strategies to add ancestor information into the transformer decoder layer. we also propose a novel strategy, which consists of setting parameters in the mask matrix for those ancestor tokens and tuning them. we find that this new strategy makes the largest improvement. ## conclusion in this paper, we focus on the dfs linearization and introduce several strategies to add ancestor information into the model. we conduct experiments to show the improvement for both amr 2.0 and amr 3.0 datasets. our method achieves new stateof-the-art performances for the amr parsing task.