MLM odeler5 @ Causal News Corpus 2023: Using R o BERT a for Casual Event Classification (2023)

Abstract: Identifying cause-effect relations plays an integral role in the understanding and interpretation of natural languages. Furthermore, automated mining of causal relations from news and text about socio-political events is a stepping stone in gaining critical insights, including analyzing the scale, frequency and trends across timelines of events, as well as anticipating future ones. The Shared Task 3, part of the 6th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE @ RANLP 2023), involved the task of Event Causality Identification with Causal News Corpus. We describe our approach to Subtask 1, dealing with causal event classification, a supervised binary classification problem to annotate given event sentences with whether they contained any cause-effect relations. To help achieve this task, a BERT based architecture - RoBERTa was implemented. The results of this model are validated on the dataset provided by the organizers of this task.

## introduction the ability to comprehend underlying cause-andeffect associations holds paramount significance across a multitude of disciplines. this knowledge facilitates informed decision making processes. hence, the investigation of causal relationships assumes a pivotal position in the study of understanding the underlying mechanisms that govern the correlation between actions and their consequences. with the uncertainty and complexity that characterizes various domains of inquiry, predictive capabilities empower individuals and organizations to navigate the intricate interplay of variables, facilitating better planning, risk management, and mitigation strategies. the primary objective of this task is to develop a method to classify casual relations and in turn help identify the variable that acts as a reason for another. this will help in anticipating contingencies. the impact of casual analysis is not only seen in the extrapolation of likelihood of events under static conditions but also in events observed in continuously altering conditions, for example, changes induced by treatments or external interventions. @xcite this paper presents our use of the bert based architecture roberta, in subtask 1 casual event classification of case 2023 shared task 3. ## task description the case 2023 shared task 3, event causality identification with causal news corpus @xcite , focuses on the problem of detecting and extracting causal relations in protest event news. a causal relation is defined as a semantic relation between the cause and effect arguments, such that the occurrence of the latter is the subsequent result of the occurrence of the former. for an example to be considered a causal sentence, it has to include at least one cause-effect attribute pair where one argument provides the reason, explanation or justification for the situation described by the other. @xcite subtask 1 -causal event classification @xcite is an event detection task which focuses on the automatic identification of the presence of causality in event sentences, i.e, a supervised classification task to detect causality in a given text. the aim is to develop a model to annotate event sentences with binary labels indicating whether they contained any cause-effect relations. ## dataset description the subtask used the second version of the causal news corpus (cnc), an annotated news corpus comprising 3,767 event sentences extracted from randomly sampled protest event news from varying sources. for the first subtask, the data consisted of event sentences annotated with binary labels indicating the absence or presence of causality within the span of each text, represented by 0 and 1 respectively. the training and development dataset consisted of 3075 and 340 annotated text samples respectively. in the testing phase, the development dataset could be incorporated into the training of the model, and an additional test set of 352 un-annotated event sentences was provided. ## results the final leaderboard for subtask 1 was based on binary f1 score. the baseline binary f1 score was 0.8191. our submission using roberta and dataset 1 achieved a binary f1 score of 0.7475. the organisers also measured precision, recall, accuracy and mcc(matthews correlation coefficient). ## conclusion this paper addresses subtask 1 -casual event classification, which is a binary classification task, with the aim to annotate samples to indicate the presence of causality in a given text. our method includes basic data pre-processing techniques to generate four different datasets, which are used as inputs for the roberta model and to the compare the results across five metrics -f1 score, recall, precision, accuracy and mcc(matthews correlation coefficient). our system achieved a rank of eight based on binary f1 score i.e., 0.74752 achieved using dataset 1. the best binary f1 score value obtained was 0.77347 for dataset 4, post competition. results may be improved by increasing the embedding size to a max length of 500 for roberta. hyperparameter tuning of the roberta model should also be considered for better results.