K i PT : Knowledge-injected Prompt Tuning for Event Detection (2022)

Abstract: Event detection aims to detect events from the text by identifying and classifying event triggers (the most representative words). Most of the existing works rely heavily on complex downstream networks and require sufficient training data. Thus, those models may be structurally redundant and perform poorly when data is scarce. Prompt-based models are easy to build and are promising for few-shot tasks. However, current prompt-based methods may suffer from low precision because they have not introduced event-related semantic knowledge (e.g., part of speech, semantic correlation, etc.). To address these problems, this paper proposes a Knowledge-injected Prompt Tuning (KiPT) model. Specifically, the event detection task is formulated into a condition generation task. Then, knowledge-injected prompts are constructed using external knowledge bases, and a prompt tuning strategy is leveraged to optimize the prompts. Extensive experiments indicate that KiPT outperforms strong baselines, especially in few-shot scenarios.

## introduction events describe state changes of participating entities. the event detection (ed) task is one of the essential tasks in the information extraction field. event triggers are the most representative words or phrases in events, and they are usually composed of verbs or nouns. there is a one-to-one correspondence between events and event triggers, so the ed task is equivalent to identifying and classifying event triggers. the ed task has a wide range of applications, providing helpful information for downstream tasks such as text summarization, auto summarization, machine question and answer (qa), etc. meanwhile, with the vigorous development of internet news and social media, ed has become a * weiping li is the corresponding author. practical approach for extracting information from massive texts. therefore, the ed task has attracted increasing attention with great academic and applied value in recent years. most current ed models use a pre-trained language model to build complex downstream networks (including cnn, rnn, gcn, etc.) these methods perform well on public datasets, but they rely heavily on the fine-tuning strategy to train their downstream networks and introduce massive extra parameters. however, due to the scarcity and uneven distribution of the annotated data for ed, these methods may cause severe overfitting problems. further, these methods may perform poorly in data-scarce scenarios because their extra parameters cannot be fully optimized. recently, prompt-based learning methods is a new trend in natural language processing. researchers verified that pre-trained language models already have enough knowledge, so the complex downstream networks are unnecessary in many cases. prompt-based learning methods make full use of the information in pre-trained language models by constructing prompts to guide the language model to solve nlp tasks. specifically, promptbased methods first transform the original input into prompt templates containing the initial input, the prompt tokens, and unfilled slots for output. then, a pre-trained language model is employed to fill the unfilled slots to obtain a final string from which the final output can be derived. prompt learning-based methods eliminate complex downstream networks and massive extra parameters, so they have advantages in data scarcity scenarios. meanwhile, the prompts' quality directly affects the models' performance. however, manually selecting the optimal prompts is timeconsuming and labor-intensive. considering this, prompt tuning strategies have been proposed by introducing continuous virtual tokens as trainable prompts that will be optimized through training. the threat posed by the iraqi dictator justifies a war, which is sure to kill thousands of innocent children. ## related work studies related to our work are mainly discussed from the following three aspects: ## methodology this section first describes the definition of the ed task. then, the proposed kipt model is introduced in detail. ## experiments this section describes our experimental settings and detailed experimental analysis. first, we conduct overall experiments to verify the performance of kipe with sufficient training data. next, we conduct few-shot and zero-shot experiments to verify the performance of kipt in data-scarce scenarios. finally we perform an ablation analysis to explore the influence of each part in our prompts. ## conclusion this paper proposes a prompt-based learning method for ed by introducing knowledge-injected prompt tuning. external knowledge and soft tokens are used to construct knowledge-injected prompts, which can be optimized through training. comprehensive experiments demonstrate that kipt outperforms current prompt-based ed models and strong baselines, especially in data-scarce scenarios. through our method, prompt-based models can introduce task-related knowledge more conveniently and effectively. in the future, we will explore more knowledge injection approaches and their applications in other tasks.