D emo SG : Demonstration-enhanced Schema-guided Generation for Low-resource Event Extraction (2023)

Abstract: Most current Event Extraction (EE) methods focus on the high-resource scenario, which requires a large amount of annotated data and can hardly be applied to low-resource domains. To address EE more effectively with limited resources, we propose the Demonstration-enhanced Schema-guided Generation (DemoSG) model, which benefits low-resource EE from two aspects: Firstly, we propose the demonstration-based learning paradigm for EE to fully use the annotated data, which transforms them into demonstrations to illustrate the extraction process and help the model learn effectively. Secondly, we formulate EE as a natural language generation task guided by schema-based prompts, thereby leveraging label semantics and promoting knowledge transfer in low-resource scenarios. We conduct extensive experiments under in-domain and domain adaptation low-resource settings on three datasets, and study the robustness of DemoSG. The results show that DemoSG significantly outperforms current methods in low-resource scenarios.

## introduction event extraction (ee) aims to extract event records from unstructured texts, which typically include a trigger indicating the occurrence of the event and multiple arguments of pre-defined roles @xcite . for instance, the text shown in figure 1 describes two records corresponding to the transport and meet events, respectively. the transport record is triggered by the word "arrived" and consists of 3 arguments:"kelly", "beijing", and "seoul". similarly, the meet record is triggered by "brief " and has two arguments: "kelly" and "yoon". ee plays a crucial role in natural language processing as it provides valuable information for various downstream tasks, including knowledge graph construction @xcite and question answering @xcite . most studies on ee primarily focus on the highresource scenario @xcite @xcite @xcite , which requires a large amount of annotated training data to attain satisfactory performance. however, event annotation is a costly and labor-intensive process, rendering these methods challenging to apply in domains with limited annotated data. thus, there is a growing need to explore ee in low-resource scenarios characterized by a scarcity of training examples, which has garnered recent attention. @xcite models ee as a unified sequenceto-structure generation, facilitating knowledge sharing between the event detection and argument extraction subtasks. building upon @xcite , @xcite introduces various pre-training strategies on large-scale datasets to enhance structure generation and improve low-resource ee performance. @xcite incorporates supplementary information by manually designing a prompt for each event type, encompassing event descriptions and role relations. however, despite their effectiveness, existing methods exhibit cer-tain inadequacies: 1) sequence-to-structure generation ignores the gap between the downstream structure generation and pre-training natural language generation. 2) large-scale pre-training needs a lot of computational resources and excess corpus. 3) manual design of delicate prompts still demands considerable human effort, and the performance of ee is sensitive to prompt design. in this paper, to tackle ee more effectively with limited resources, we propose the demonstrationenhanced schema-guided generation (demosg) model, benefiting from two aspects: 1) improving the efficiency of using annotated data. 2) enhancing the knowledge transfer across different events. to make full use of annotated training data, we consider them not only as signals for supervising model learning, but also as task demonstrations to illustrate the extraction process to the model. specifically, as shown in figure 1 , demosg selects a suitable training example for each event, transforms it into a demonstration in the natural language style, and incorporates the demonstration along with the input text to enhance the extraction. we refer to this paradigm as demonstration-based learning for ee, which draws inspiration from the in-context learning of the gpt series @xcite . to identify appropriate instances for demonstrating, we devise several selecting strategies that can be categorized into two groups: 1) demooriented selection, which aims to choose examples with the best demonstrating features. 2) instanceoriented retrieval, which aims to find the examples that are most semantically similar to the input sentence. with the enhancement of demonstrations, demosg can better understand the ee process and learn effectively with only a few training examples. to further enhance the knowledge transfer capability and improve the effectiveness of demonstrations, we leverage label semantic information by formulating ee to a seq2seq generation task guided by schema-based prompts. in detail, demosg creates a prompt formally similar to the demonstrations for each event, which incorporates the event type and roles specified in the event schema. then, the prompt and demo-enhanced sentence are fed together into the pre-trained language model (plm) to generate a natural sentence describing the event records. finally, demosg employs a rule-based decoding algorithm to decode records from the generated sentences. the schema-guided sequence generation of demosg promotes knowledge transfer in multiple ways: firstly, leveraging label semantics facilitates the extraction and knowledge transfer across different events. for instance, the semantic of the "destination" role hints that its argument is typically a place, and the similarity between "attacker" and "adjudicator" makes it easy to transfer argument extraction knowledge among them, for they both indicate to humans. secondly, natural language generation is consistent with the pre-training task of plm, eliminating the need for excessive pre-training of structural generation methods @xcite . furthermore, unlike classification methods constrained by predefined categories, demosg is more flexible and can readily adapt to new events through their demonstrations without further fine-tuning, which is referred to as the parameter-agnostic domain adaptation capability. our contributions can be summarized as follows: ## methodology in this section, we first elucidate some preliminary concepts of low-resource ee in section 2.1. next, we present the proposed demonstration-enhanced schema-guided generation model in section 2.2. lastly, we provide details regarding the training and inference processes in section 2.3. ## experiments to assess the effectiveness of demosg, we conduct comprehensive experiments under in-domain lowresource, domain adaptation, and high-resource settings. an ablation study is applied to explore the impact of each module and the improvement achieved. furthermore, we also investigate the robustness of applying demonstrations of demosg. ## effectiveness and robustness of demonstration-based learning since demonstrations can influence the lowresource ee performance, we design two variants for demosg with different kinds of damage to the demonstrations to explore the effectiveness and robustness of demonstration-based learning: 1) demonstration perturbation. to analyze the influence of incorrect demonstrations, we sample 40% of the demonstrations and replace the golden triggers and arguments with random spans within the context. test perturbation applies perturbation during the inference phase only, while train-test perturbation applies perturbation during both the training and inference phases. we conduct the above experiments based on de-mosg with rich-role strategy under the 5-shot setting. from figure 3 , we can observe that: to make the demonstration effective for small plms, @xcite proposes a demonstrationbased learning framework for ner, which treats demonstrations as additional contextual information for the sequence tagging model during training. @xcite further studies the robustness of demonstration-based learning in the sequence tagging paradigm for ner. our method differs from these studies from: 1) our demonstrationenhaced paradigm is designed according to the specific characteristics of event extraction. 2) our generative framework enhances the flexibility of demonstration-based learning, providing it with the parameter-agnostic domain-adaptation capability. ## conclusion in this paper, we propose the demonstrationenhanced schema-guided generation (demosg) model for low-resource event extraction, which benefits from two aspects: firstly, we propose the demonstration-based learning paradigm for ee to fully use the annotated data, which transforms them into demonstrations to illustrate the extraction process and help the model learn effectively. secondly, we formulate ee as a natural language generation task guided by schema-based prompts, thereby leveraging label semantics and promoting knowledge transfer in low-resource scenarios. extensive experiments and analyses show that demosg significantly outperforms current methods in various low-resource and domain adaptation scenarios, and demonstrate the effectiveness of our method.