Enhancing AMR Parsing with Group Relative Policy Optimization (2025)

Abstract: We investigate the capabilities of the openly available Llama 3.2 1B language model for Abstract Meaning Representation (AMR) parsing through supervised fine-tuning, further enhanced by reinforcement learning via Group Relative Policy Optimization (GRPO). Existing supervised methods for AMR parsing face limitations due to static loss functions and challenges in capturing complex semantic phenomena. To address this, our GRPO-based approach explicitly optimizes fine-grained semantic rewards, including Smatch scores, frame-argument correctness, and structural validity of logical operations. Experimental results show that supervised fine-tuning alone establishes Llama as a capable English AMR parser, and subsequent GRPO fine-tuning further improves its performance. Our final model achieves higher Smatch scores, consistently respects critical low-level semantic constraints, and outperforms existing parsers on high-level semantic evaluation metrics across diverse linguistic phenomena.

## introduction abstract meaning representation has become essential in various natural language processing tasks, such as machine translation @xcite @xcite , question answering @xcite , dialogue understanding @xcite , summarization @xcite @xcite , and fact-checking @xcite @xcite . despite its widespread adoption, amr parsing remains challenging. @xcite recently demonstrated that parsing accuracy has stagnated, highlighting persistent difficulties in capturing complex semantic phenomena, even with advanced models. while large language models (llms) such as the llama models @xcite have demonstrated impressive performance across various language generation tasks, their capability for structured semantic parsing-particularly amr parsing-remains unverified. moreover, it remains unclear whether advanced reinforcement learning (rl) techniques like group relative policy optimization (grpo), introduced by @xcite , can effectively enhance the performance of llms on such structured prediction tasks by directly optimizing for desired graph properties. in this paper, we first examine the baseline capabilities of the openly available llama 3.2 1b model by supervised fine-tuning (sft) on the amr 3.0 dataset @xcite . we refer to this model as llama-sft. we then further fine-tune this model using grpo, incorporating fine-grained reward signals explicitly designed to encourage adherence to critical low-level amr properties, such as frame-argument correctness and structural validity of logical operations (and-or node correctness), alongside smatch and graph parsability. we call this enhanced model llama-grpo. we systematically evaluate our models against publicly available amr parsing model, spring @xcite , using standard metrics (smatch) and the detailed grapes evaluation suite @xcite . our results show that the llama 3.2 1b model, after supervised fine-tuning (llama-sft), achieves amr parsing performance close to open-source amr parser models. critically, when further enhanced through grpo-based reinforcement learning, our model: ## related work early work in amr parsing often relied on transition-based systems @xcite or graph-based approaches @xcite , frequently using specialized features and constrained decoding. the advent of neural sequenceto-sequence models marked a significant shift. many modern parsers treat amr parsing as a translation task from text to a linearized representation of the amr graph @xcite . transformer-based architectures @xcite quickly became dominant. models like spring @xcite , based on bart @xcite , demonstrated strong performance by leveraging pre-training and specialized techniques like graph linearization. spring employs bidirectional pre-training and graph-based regularization during fine-tuning on linearized amr graphs. other parsers, such as those based on t5 or bart @xcite @xcite , have also achieved high results through large-scale pre-training and task-specific adaptations. despite these advances, as highlighted by @xcite , performance has plateaued, suggesting limitations in current supervised approaches. challenges in amr parsing remain, particularly in achieving semantic consistency, cross-lingual adaptability, and structured reasoning. ## methods reinforcement learning has been increasingly used to fine-tune llms for various objectives beyond next-token prediction, such as improving helpfulness, harmlessness, or adherence to specific styles @xcite . techniques such as proximal policy optimization (ppo) @xcite are commonly used but often require training a separate critic model, which can be computationally expensive. grpo @xcite offers a more efficient alternative by using group-based relative ranking, making rl fine-tuning more accessible, especially for complex tasks with non-differentiable or noisy reward signals, as demonstrated in fields like mathematical reasoning @xcite , computer vision @xcite , and speech processing (togootogtokh and klasen, 2025). ## dataset for training, we used the amr 3.0 dataset (ldc2020t02) @xcite , which provides a large collection of human-annotated abstract meaning representation (amr) graphs. we preprocessed the amr graphs the following way. first, we removed wiki tags from the amr graphs. then, we serialized each graph into a single line using a depth-first approach. during serialization, new lines within the original graph notation were replaced with spaces, and consecutive spaces were compressed into a single space. finally, we added spaces around parentheses to ensure consistent tokenization. for evaluation, we used the amr 3.0 test set and the little prince (tlp) corpus test set, which provides a smaller, out-of-domain evaluation with high-quality annotations. we measured the performance of the models using smatch scores @xcite computed with the smatchpp library @xcite . the combination of amr 3.0 and the tlp dataset enables a comprehensive evaluation, balancing broad-domain performance with controlled, high-quality annotations. ## results we compare our llama-sft and llama-grpo models against spring. the results show that llama-sft achieves a competitive smatch score in two epochs, confirming the adaptability of llms to an unseen task. we observe that one epoch of subsequent grpo fine-tuning yields further improvements. llama-grpo achieves a smatch score of 81.92 on amr 3.0, a gain of over 2.3 points over llama-sft, and it outperforms spring. in addition, grpo improves compliance with the targeted low-level constraints. frame-argument correctness improves from 96.5% to over 99% on amr 3.0 and reaches 99.75% on tlp. similarly and-or node correctness jumps from 96.5% to over 99.6% on amr 3.0 and achieves perfect compliance on tlp. this demonstrates the effectiveness of incorporating these specific structural and semantic properties directly into the reward function via grpo. figure 2 illustrates the progression of the overall reward during grpo training. the reward score exhibits a smooth and consistent upward trend throughout the grpo fine-tuning process. this indicates that the model has effectively learned to generate amr structures that better satisfy these constraints, validating the utility of grpo with these specific reward signals. the learning appears stable, without drastic fluctuations, suggesting that grpo provides a reliable optimization process for these objectives. ## conclusion in this work, we investigated the application of the llama 3.2 1b language model to amr parsing, enhanced by group relative policy optimization (grpo). we demonstrated that supervised finetuning establishes llama as a competent baseline amr parser. subsequently, by incorporating finegrained reward signals targeting smatch, graph parsability, frame-argument correctness, and and-or node validity into a grpo fine-tuning stage, we achieved significant improvements. our llama-grpo model not only outperformed its supervised counterpart (llama-sft) in smatch scores, but also showed significantly better performance for crucial low-level semantic and structural constraints. furthermore, evaluation using the grapes suite revealed that llama-grpo generated more complete graph structures (higher prerequisites scores) necessary to capture diverse and complex linguistic phenomena, outperforming both llama-sft and spring while achieving competitive recall performance. these results highlight the potential of combining moderately sized, openly available llms with efficient reinforcement learning techniques like grpo, guided by carefully designed reward functions, to tackle complex structured prediction tasks like amr parsing. this approach allows for direct optimization of desired output properties beyond what is easily achievable with standard supervised learning alone.