CSECU - DSG @ Causal News Corpus 2022: Fusion of R o BERT a Transformers Variants for Causal Event Classification (2022)

Abstract: Identifying cause-effect relationships in sentences is one of the formidable tasks to tackle the challenges of inference and understanding of natural language. However, the diversity of word semantics and sentence structure makes it challenging to determine the causal relationship effectively. To address these challenges, CASE-2022 shared task 3 introduced a task focusing on event causality identification with causal news corpus. This paper presents our participation in this task, especially in subtask 1 which is the causal event classification task. To tackle the task challenge, we propose a unified neural model through exploiting two fine-tuned transformer models including RoBERTa and Twitter-RoBERTa. For the score fusion, we combine the prediction scores of each component model using weighted arithmetic mean to generate the probability score for class label identification. The experimental results showed that our proposed method achieved the top performance (ranked 1st) among the participants.

## introduction causality is a fundamental cognitive concept that frequently emerges in various natural language processing (nlp) works. it mostly focuses on the challenges of inference and understanding of the natural language. in general, a causal relation is a semantic relationship between two arguments known as cause and effect, where the occurrence of one (cause argument) incurs the occurrence of the other (effect argument). such causal relation plays an important role in various contemporary nlp tasks including document-summarization, event prediction from text, scene and story generation, question-answering (q/a), product recommendation based on user comments, and other textual entailments @xcite . to address the challenges of event causality identification in texts, tan et al. @xcite introduced a shared task at the case-2022 workshop. the task is composed of two subtasks including a causal event classification task (subtask1) and a cause-effect-signal span detection task (subtask 2). however, we only participated in the causal event classification task (subtask1), where given a text a system needs to determine whether it contains a cause-event meaning or not. to demonstrate a clear view of the task definition, we articulate a few examples from subtask 1 in ## proposed framework in this section, we describe our proposed approach for the event causality identification task. our goal is to exploit the inherent semantics of the sentence to identify whether the event sentence contains any cause-effect meaning. the overview of our proposed framework is depicted in figure 1 . given an input text, we employ two transformer models including roberta @xcite and one of its variants twitter_roberta @xcite to extract the diverse contextual features. such feature representation better captures the inherent semantics of the text. later, a linear feed-forward layer is utilized in each model to estimate the probability score of each class. finally, for the effective fusion of the scores, we take the weighted arithmetic mean of the prediction scores of these models. a class that contains the highest probability scores is considered as the final label. ## conclusion and future directions in this paper, we present an approach to identify the cause-effect relation in texts by exploiting roberta variants with an effective fusion strategy. experimental results demonstrated the efficacy of our fusion strategy of the two sota transformers model which helped us to obtain the best result in subtask 1. in the future, we intend to explore other indicators of textual causal relations for further improvement. especially, a graph-based neural model may exploit complex dependency patterns of causeeffect relations from text more effectively.