EVIT : Event-Oriented Instruction Tuning for Event Reasoning (2024)

Abstract: Events refer to specific occurrences, incidents, or happenings that take place under a particular background. Event reasoning aims to infer events according to certain relations and predict future events. The cutting-edge techniques for event reasoning play a crucial role in various natural language processing applications. Large language models (LLMs) have made significant advancements in event reasoning owing to their wealth of knowledge and reasoning capabilities. However, smaller instruction-tuned models currently in use do not consistently demonstrate exceptional proficiency in managing these tasks. This discrepancy arises from the absence of explicit modeling of events and the interconnections of them within their instruction data. Consequently, these models face challenges in comprehending event structures and semantics while struggling to bridge the gap between their interpretations and human understanding of events. Additionally, their limitations in grasping event relations lead to constrained event reasoning abilities to effectively deduce and incorporate pertinent event knowledge. In this paper, we propose Event-Oriented Instruction Tuning to train our large language model named EvIT specializing in event reasoning tasks. Specifically, we first propose a novel structure named event quadruple which contains the structure and semantics of events and is complete in the event representation. We then design event-relation learning based on the structures. We encapsulate the learning into the instruction-tuning formulation to better stimulate the event reasoning capacity of our model. To implement our training, we design a heuristic unsupervised method to mine event quadruple from a large-scale corpus. At last, we finetune a Llama model on our Event-Oriented Instruction Tuning. We conduct extensive experiments on event reasoning tasks on several datasets. Automatic and human evaluations demonstrate EvIT achieves competitive performances on event reasoning.

## introduction events are instances or occurrences that form the basic semantic building units encompassing the meanings of activities, accomplishments, achievements, and states @xcite . by employing advanced techniques and models, event reasoning aims to enable machines to comprehend the mechanism of real-world event evolution @xcite . under this ultimate goal, event reasoning consists of several key sub-objectives, including the understanding and reasoning about a diverse range of event inter-relations, and predicting events pertaining to certain relations. reasoning events forms the foundation of sorts of nlp applications such as recommendation systems @xcite , and question answering @xcite . in recent times, substantial research efforts are dedicated to instructing-tuning language models to acquire the abilities for zero-shot inference such as flan-t5 alpaca @xcite , vicuna @xcite , wizardlm @xcite , and dolly @xcite . these models have shown the potential to enhance the language models with versatile instruction-following capabilities through fine-tuning various instruction datasets. nonetheless, in the training of these models, the instruction-tuning data involved did not explicitly model events and their inter-relations. consequently, these models perform inferiorly on most event reasoning tasks. the limitations observed in the instruction-tuned models stem from several fundamental factors. firstly, these models display an inadequate understanding of event structures and semantics and show discrepancies between the model's interpretation and human comprehension of events. secondly, the models exhibit deficiencies in comprehending the relations between events, resulting in insufficient event reasoning capabilities and the inability to effectively infer and integrate relevant event knowledge. based on the perfor-mances, instruction-tuning smaller language models exhibit poorer performance when contrasted with large language models (llms) such as chat-gpt and bloomz-175b @xcite . to address these obstacles, we present evit which is trained on our novel event-oriented instruction tuning. in our method, we incorporate explicit event modeling and event relation comprehension. specifically, to enhance the comprehension of the structure and semantics of events, we first design a novel structure named event quadruple. this event-centric structure contains two events, their relation, and the background information where the fact holds. the event quadruple covering contextualized events and their inter-relation knowledge would improve the model's conceptions of events. based on the event quadruple, we develop an event-relation learning paradigm. we train evit to predict the tail events of event quadruple in both generation and discrimination manners. we further encapsulate this training process into instruction tuning with generated instruction templates. it can better stimulate the model's abilities to conduct event-related reasoning and associate event knowledge. to implement our training, we construct event quadruple from a large-scale textual corpus. we design a heuristic negative events mining algorithm to construct candidate events for discriminative event-relation training. we finetune llama by our event-oriented instruction tuning. we conduct extensive experiments to testify to the effectiveness of evit. we first evaluate the performance of evit across 8 tasks of event reasoning which are not seen during training. among these tasks, four are held-in tasks, involving relations explicitly handled during training, while the remaining four tasks are held-out tasks. results of automatic and human evaluations show that evit outperforms other instruction-tuned models. we summarize our contributions: ## conclusion in this study, we introduce event-oriented instruction tuning to enhance event reasoning capabilities and train our model evit. we first introduce a novel structure called event quadruple as a foundational structure. building upon this, we establish event relation learning through instruction tuning using generated prompts. we create an instructiontuning dataset focused on events, encompassing comprehensive and diversified event data both in syntax and semantics. subsequently, we fine-tune llama to create the evit model. we conduct experiments on both close and open task settings and compare with several strong cutting-edge instruction-tuned models. through extensive experiments on 8 datasets, the outcomes demonstrate the efficacy of our proposed approach.