LC 4 EE : LLM s as Good Corrector for Event Extraction (2024)

Abstract: Event extraction (EE) is a critical task in natural language processing, yet deploying a practical EE system remains challenging. On one hand, powerful large language models (LLMs) currently show poor performance because EE task is more complex than other tasks. On the other hand, state-of-the-art (SOTA) small language models (SLMs) for EE tasks are typically developed through fine-tuning, lack flexibility, and have considerable room for improvement. We propose an approach, **L**LMs-as-**C**orrector for **E**vent **E**xtraction (**LC4EE**), aiming to leverage the superior extraction capability of SLMs and the instruction-following ability of LLMs to construct a robust and highly available EE system. By utilizing LLMs to identify and correct errors of SLMs predictions based on automatically generated feedback information, EE performances can be improved significantly. Experimental results on the representative datasets ACE2005 and MAVEN-Arg for Event Detection (ED) and EE tasks validated the effectiveness of our method.

## introduction events are basic units of human activities and interactions, containing rich information which is vital for downstream applications @xcite . event extraction is the process of identifying and extracting structured event information from unstructured text. it includes: 1) event detection, which detects event triggers and classifies them into pre-defined event types. 2) event argument extraction, which extracts relevant arguments associated with each event. however, developing a highly practical and flexible ee system remains challenging @xcite . current best performance of ee task on the most commonly used dataset ace 2005 only reaches 57.9% @xcite , far from meeting practical requirements. ## preliminary study in this section, we validated the ability of the llms to correct errors based on the error feedback information under the oracle setting (i.e., only conduct corrections based on incorrect samples) first of all. to thoroughly explore this ability, we designed four granularity levels of feedback information and compared their performance differences. ## lc4ee preliminary study in section 2 has validated that llms possesses strong capabilities for error correction, with significant improvements when detailed error information is available. however, in practical applications, obtaining a large volume of detailed error feedback is not feasible. therefore, in this section, we will explore the ability of llms to provide as fine as possible error feedback automatically and then make corrections based on feedback for incorrect samples. the model lc4ee we designed is illustrated in figure 2 , which consists of two main components: a rules inducer for feedback generation and a llms corrector. rules inducer for feedback generation is used to summarize rules based on incorrect samples for error feedback generation. these rules will be retrieved in llms corrector, then corrections are made based on the generated feedback. we will describe these two components in detail as follows. ## conclusions and future work we proposed the lc4ee model, which combined the superior ee capabilities of slms with the powerful semantic understanding and instructionfollowing abilities of llms. in this way, llms are used to correct prediction errors of slms based on automatically generated feedback information. our model not only effectively improves ee task performances but also establishes a more flexible and practical ee system. we have validated the effectiveness of our method through some experiments. in our future work, we plan to extend lc4ee to more tasks, models, and datasets. in addition, we will increase the number of feedback generation rules and enhance their quality to further improve ee task performance.