LLM s Can Also Do Well! Breaking Barriers in Semantic Role Labeling via Large Language Models (2025)

Abstract: Semantic role labeling (SRL) is a crucial task of natural language processing (NLP). Although generative decoder-based large language models (LLMs) have achieved remarkable success across various NLP tasks, they still lag behind state-of-the-art encoder-decoder (BERT-like) models in SRL. In this work, we seek to bridge this gap by equipping LLMs for SRL with two mechanisms: (a) retrieval-augmented generation and (b) self-correction. The first mechanism enables LLMs to leverage external linguistic knowledge such as predicate and argument structure descriptions, while the second allows LLMs to identify and correct inconsistent SRL outputs. We conduct extensive experiments on three widely-used benchmarks of SRL (CPB1.0, CoNLL-2009, and CoNLL-2012). Results demonstrate that our method achieves state-of-the-art performance in both Chinese and English, marking the first successful application of LLMs to surpass encoder-decoder approaches in SRL.

## introduction semantic role labeling (srl) is a fundamental task in natural language processing (nlp, @xcite , aiming to analyze the semantic relationships between predicates and their corresponding arguments in a sentence @xcite @xcite . srl is crucial for various nlp applications, including information extraction @xcite @xcite , question answering @xcite @xcite , machine translation @xcite @xcite , robot command parsing @xcite @xcite @xcite and etc. besides, we can also use srl to enhance pretrained language models with structured-aware semantic information @xcite . recently, the advent of large language models (llms) has reformulated nlp tasks with promptbased generative solutions @xcite . impressive performance has been achieved across a wide range of tasks @xcite @xcite . the reason behind lies in the powerful capability of llms for general reasoning @xcite . for most nlp tasks, such as information extraction @xcite , sentiment analysis @xcite , question answering @xcite , and (vallina) machine translation @xcite , which require little specialized expertise, llms can handle them easily. thus, it is reasonable to expect that llms can perform well on most of these tasks. for several nlp tasks, this is not the case. unfortunately, srl is one of them, as it requires strong explainability in linguistics @xcite @xcite . as shown in figure 1 , the upper part illustrates a representative baseline of llm performance in resolving srl tasks. without linguistic knowledge, what is the meaning of predicate-argument structures, arg0∼5, and other related concepts? clearly, this is not an easy question that can be answered through general reasoning alone. a solid background in linguistic knowledge is required to provide an explanation. furthermore, the illusion problem in llms is another serious issue, as srl results are highly dependent on the input data. in this work, we propose two novel mechanisms to sufficiently enhance the expertise of llms in addressing the above two problems, respectively. first, we design a retrieval-augmented agent to enable llms to better understand predicates and their semantic arguments. we construct an external knowledge database based on the predicateargument description guidance document from the original dataset. second, to mitigate the illusion outputs of llms, we allow the llm to verify its outputs by evaluating and correcting them autonomously, a process referred to as self-correction. concretely, we design a two-stage conversationbased architecture to perform srl with llms: (1) predicate identification and (2) argument labeling, which are aligned with traditional bert-like encoder-decoder systems. based on this architecture, we integrate the aforementioned two mechanisms. llm reasoning in the two stages is executed in an iterative manner. in addition, llm parameters are also optimized to better suit our srl task. we conduct comprehensive experiments covering both chinese (zh) and english (en) on three widely-used benchmarks: chinese proposition bank 1.0 (cpb1.0) @xcite , @xcite , and @xcite . the results demonstrate that our approach achieves state-of-the-art performance, marking the first time an llm-driven method has surpassed traditional approaches on the complete srl task. this breakthrough highlights not only the effectiveness of leveraging llms through our framework but also the critical role of retrieval-augmented agents in addressing the inherent challenges of srl. in summary, the main contributions of this work can be summarized as follows: ## method in this section, we propose a retrieval-augmented framework for srl, designed to address challenges such as predicate-argument complexity and the need for external knowledge. by reformulating srl as a step-by-step conversational task, the framework integrates retrieval-enhanced agents to incorporate external knowledge and self-correction mechanisms to iteratively refine outputs. the rationale for unifying the tasks is explained in appendix a. as illustrated in figure 2 , our framework employs a two-stage pipeline where each stage is augmented with retrieval-based knowledge and self-correction mechanisms. below, we detail each component along with the mathematical formulations and algorithms involved. a detailed algorithm description and pseudocode can be found in appendix f. ## conclusion in this work, we proposed a novel retrievalaugmented framework aimed at bridging the performance gap between large language models (llms) and traditional encoder-decoder models in semantic role labeling (srl). through the integration of retrieval-augmented agents, which leverage external knowledge about predicate and argument structures, and a self-correction mechanism for iterative refinement, our method addressed key challenges in srl, including the complexity of predicate-argument structures, the need for domain-specific linguistic knowledge, and the illusion issues often associated with llm outputs. experiments conducted on three widely-used benchmarks-cpb1.0, @xcite english and chinese demonstrated that our approach achieved state-of-the-art results. notably, it marked the first instance of an llm-based method outperforming traditional approaches across complete srl tasks. these results highlight the transformative potential of combining llms with structured reasoning and external knowledge for tackling complex linguistic tasks, paving the way for further advancements in semantic understanding.