Rule-Guided Extraction: A Hierarchical Rule Optimization Framework for Document-Level Event Argument Extraction (2025)

Abstract: Document-level event argument extraction (EAE) is a critical task in natural language processing. While most prior approaches rely on supervised training with large labeled datasets or resource-intensive fine-tuning, recent studies explore in-context learning (ICL) with LLMs to reduce data dependence and training costs. However, the performance of ICL-based methods still lags behind fully supervised models.We highlight a key reason for this shortfall: the lack of sufficient extraction rules. In this paper, we conduct a systematic study of using hierarchical rules to enhance LLMs’ ICL capabilities. We first define three types of hierarchical rules and demonstrate their effectiveness in enhancing the performance of LLMs for document-level EAE. Building on this, we further propose an LLM-driven HiErarchical Rule Optimization (HERO) framework that iteratively generates and selects optimal hierarchical rules. Specifically, in each iteration, high-value instances are selected to produce error feedback, which is used to update and expand hierarchical rule sets. This results in multiple candidate hierarchical rule sets, from which the optimal one is selected using a scoring-based mechanism. During inference, prompts are constructed using the optimal hierarchical rules to enhance ICL performance of LLMs. Extensive experiments demonstrate the effectiveness of HERO, surpassing few-shot supervised methods and outperforming state-of-the-art prompting baselines by 3.18% F1 on RAMS, 4.30% F1 on DocEE-N, and 3.17% F1 on DocEE-C.

## introduction document-level event argument extraction (eae) aims to transform unstructured event information from documents into structured formats encapsulating event arguments, facilitating many downstream tasks such as machine reading comprehension @xcite , summarization @xcite , news narrative understanding @xcite and dialogue systems @xcite . most of the previous methods heavily depend on supervised training with large-scale labeled data @xcite @xcite @xcite . recently, there has been a notable surge in applications of large language models (llms) for document-level eae @xcite @xcite @xcite . these methods aim to combine the strengths of slms and llms or fine-tune llms with labeled data. while effective, they still require sufficient labeled data, and fine-tuning llms incurs high training costs. in-context learning (icl) @xcite , a hallmark capability of llms, leverages simple task instructions and a few input-output demonstrations within prompts to circumvent traditional resource-intensive requirements. recent work @xcite advances this paradigm by designing heuristic rules to steer llms in documentlevel eae, effectively bypassing both model tuning and data dependency. however, despite these advantages, the performance of icl-based methods still lags behind supervised models. we underline a critical bottleneck in current icl approaches for document-level eae: the lack of fine-grained extraction rules. this deficiency stems from the inherent complexity of document-level eae, which typically involves hundreds of event types with diverse, fine-grained types of argument roles, especially, most argument roles may have long spans of non-entity arguments. without sufficient extraction rules, llms frequently exhibit two types of errors: to address the aforementioned challenges, in this work, we explore the use of fine-grained hierarchical rules to guide llms for the documentlevel eae via icl. we first define hierarchical rules that mirror human coarse-to-fine reasoning: ## preliminary study in this section, we first define fine-grained hierarchical rules and experimentally verify that our proposed hierarchical rules can effectively guide llms and improve their performance on documentlevel eae. ## conclusion in this paper, we explore guiding llms to perform document-level eae through icl using finegrained hierarchical rules. we begin by defining three types of hierarchical rules and validating their effectiveness. building on this, we propose a hierarchical rule optimization framework that requires only a small amount of labeled data. by iteratively selecting the high-value instances to generate error feedback, the framework refines and selects optimal hierarchical rules without any parameter tuning. experimental results on rams and docee demonstrate the effectiveness of our approach.