Hierarchical Curriculum Learning for AMR Parsing (2022)

Abstract: Abstract Meaning Representation (AMR) parsing aims to translate sentences to semantic representation with a hierarchical structure, and is recently empowered by pretrained sequence-to-sequence models. However, there exists a gap between their flat training objective (i.e., equally treats all output tokens) and the hierarchical AMR structure, which limits the model generalization. To bridge this gap, we propose a Hierarchical Curriculum Learning (HCL) framework with Structure-level (SC) and Instance-level Curricula (IC). SC switches progressively from core to detail AMR semantic elements while IC transits from structure-simple to -complex AMR instances during training. Through these two warming-up processes, HCL reduces the difficulty of learning complex structures, thus the flat model can better adapt to the AMR hierarchy. Extensive experiments on AMR2.0, AMR3.0, structure-complex and out-of-distribution situations verify the effectiveness of HCL.

## introduction abstract meaning representation (amr) @xcite parsing aims to translate a natural sentence into a directed acyclic graph. figure 1(a) illustrates an amr graph where nodes represent concepts, e.g., 'die-01' and 'soldier', and edges represent relations, e.g., ':arg1' and ':quant'. amr has been exploited in the downstream nlp tasks, including information extraction @xcite @xcite ), text summarization @xcite and question answering @xcite . the powerful pretrained encoder-decoder models, e.g., bart @xcite , have been successfully adapted to the amr parsing and became the mainstream and state-of-the-art meth-* equal contribution. sentence: nine of soldiers died . sentence: nine of the twenty soldiers died . ## methodology we formulate amr parsing as a sequence-tosequence transformation. given a sentence @xmath0, ..., x n ), the model aims to generate a linearized amr graph @xmath1, ..., y m ). as shown in figure 1 (a), following @xcite , the amr graph is linearized by the dfs-based linearization method with special tokens to indicate variables and parentheses to mark visit depth. specifically, variables of amr nodes are set to a series of special @xmath2, ..., <@xmath3. in this paper, we propose a hierarchical curriculum learning framework (figure 2 ) with the structureand instance-level curricula to help the flat model progressively adapt to the structured amr graph. ## analysis structure benefit in order to explore the effectiveness of our hcl framework for the structured amr parsing. we divide the fine-grained f1 scores into 2 categories, "structure-dependent" (unlabelled, re-entrancy and srl) and "structureindependen" (the left 5 metrics). please refer to appendix c for the reason for this division. as shown in hard instances benefit figure 4 shows the performances of our hcl and @xcite (spring) at different layers. as is shown, as the number of layers increases, hcl exceeds spring greater, which shows our hcl helps the model better handle hard instances. foot_3 in addition, to some extend, out-of-distribution (ood) instances can be regarded as hard instances, thus we also consider the ood situation. performs @xcite on all 3 ood datasets, which shows our hcl framework can also improve the generalization ability of the model.