Explainable Event Detection with Event Trigger Identification as Rationale Extraction (2023)

Abstract: Most event detection methods act at the sentence-level and focus on identifying sentences related to a particular event. However, identifying certain parts of a sentence that act as event triggers is also important and more challenging, especially when dealing with limited training data. Previous event detection attempts have considered these two tasks separately and have developed different methods. We hypothesise that similar to humans, successful sentence-level event detection models rely on event triggers to predict sentence-level labels. By exploring feature attribution methods that assign relevance scores to the inputs to explain model predictions, we study the behaviour of state-of-the-art sentence-level event detection models and show that explanations (i.e. rationales) extracted from these models can indeed be used to detect event triggers. We, therefore, (i) introduce a novel weakly-supervised method for event trigger detection; and (ii) propose to use event triggers as an explainable measure in sentence-level event detection. To the best of our knowledge, this is the first explainable machine learning approach to event trigger identification.

## introduction every day, numerous socio-political protest events occur worldwide, targeting various decisions made by governments or authorities @xcite . these events hold significant importance for political scientists, policymakers, democracy watchdogs, and other stakeholders @xcite due to their potential to provide insights into multiple aspects @xcite . these include analysing the nature, scope, and magnitude of such events, shaping public opinion regarding different causes, assessing the status of freedom and democracy in different nations, and more @xcite . due to the continuous and abundant data flow over time, news media outlets serve as invaluable sources for social and political scientists who seek to establish comprehensive knowledge bases of protest events @xcite . early approaches to creating these knowledge bases relied on manual event detection methods @xcite , which can be expensive and slow. therefore, to cope with the volume of news media, researchers have experimented with automatic event detection methods @xcite . the organisation of the recent shared tasks such as case: challenges and applications of automated extraction of socio-political events from text @xcite has promoted automatic event detection research within the natural language processing (nlp) community. automated event detection tools are designed as pipelines that receive news articles and yield records of events. the first step of these pipelines is discriminating between relevant and irrelevant sentences @xcite . in this research, we refer to this as sentence-level event detection. once event-related sentences are determined, the next task is to extract event information on the token level @xcite ). one such key information is event trigger, defined as the main word that most clearly expresses an event occurrence @xcite . while the sentence-level event detection methods have achieved excellent results recently, the accuracy of word-level predictions still leaves room for improvement. this is partly due to the limited amount of training data, as word-level annotation is timeconsuming and expensive. in this research, we introduce a new weakly-supervised approach to event trigger detection that removes the need for training data at the word-level. to achieve this, we propose addressing event trigger detection as a rationale extraction task @xcite . the domain of explainability encompasses a wide range of techniques focused on explaining the predictions made by machine learning models @xcite . among these techniques, rationale extraction methods aim to identify and select specific portions of the input data that justify the model's output for a given data point. in manual event detection, human perception of sentencelevel annotations is guided by the presence of event triggers @xcite . we hypothesise that sentence-level event detection models also rely on event triggers to make predictions. if that is the case, explanations for sentence-level predictions can be used to detect event triggers, thus removing the need for word-level labelled training data. to extract model explanations, we use post hoc rationale extraction methods @xcite , which try to explain the predictions of a given model. at the same time, by using event triggers as explanations for sentence-level event detection methods, we introduce a new benchmark for evaluating explainability. in opposition to developing different models for sentence-level and token-level, we propose to train a single model for both tasks. our main contributions are: ## data to conduct the experiments, we used the multilingual version of the glocon gold standard dataset @xcite , which was released by case 2021 workshop @xcite , considering its recency, open-availability and coverage. this dataset targeted socio-political events covering demonstrations, industrial actions, group clashes, political violence, armed militancy and electoral mobilisations. it contains data at different levels of granularity, document, sentence and word from multiple news sources covering the languages english, portuguese and spanish. considering the scope of this research, we only utilised sentence and word-level data for our experiments from all available languages. the sentence-level data contained an identifier, sentence text and binary label, which indicates whether that particular sentence describes/contains an event or not, per instance. for simplicity, we will refer to the event-described sentences as event sentences and others as non-event sentences in the below content. the word-level data were in bio (beginning, inside, outside) format @xcite , based on event triggers and arguments (i.e. participant, place, target, organiser, event time and facility name). ## methodology we propose framing weakly-supervised event trigger detection as rationale extraction from sentencelevel event detection models. instead of training a dedicated supervised model for event trigger prediction, we propose deriving word-level scores from a strong sentence-level event detection model by extracting explanations for model predictions (figure 1 ). given a trained sentence-level event detection model and the test data, rationale extraction methods detect the parts of the input that are relevant for model predictions on a sample-by-sample basis. we hypothesise that words with the highest relevance scores should correspond to actual event triggers. our methodology has two main steps; (1) event sentence classification (2) event trigger identification, which we describe in the below sections. ## conclusions in this work, we proposed a new weakly-supervised approach for event trigger detection by exploring feature attribution methods on sentence-level event detection models. our results show that sentencelevel event detection models rely on event triggers to make predictions, and in turn, event triggers can be used as explanations for sentence-level models. we hope this work will encourage further research on improving the efficiency of event trigger detection models with weakly-supervised methods. also, we believe our findings will be useful for social media event detection considering the volume and dynamicity of data @xcite . this work presents numerous avenues for future research, ranging from enhancing achieved outcomes through combining different feature attribution methods to investigating alternative underlying architectures and training objectives at the sentence-level.