ARGUABLY @ Causal News Corpus 2022: Contextually Augmented Language Models for Event Causality Identification (2022)

Abstract: Causal (a cause-effect relationship between two arguments) has become integral to various NLP domains such as question answering, summarization, and event prediction. To understand causality in detail, Event Causality Identification with Causal News Corpus (CASE-2022) has organized shared tasks. This paper defines our participation in Subtask 1, which focuses on classifying event causality. We used sentence-level augmentation based on contextualized word embeddings of distillBERT to construct new data. This data was then trained using two approaches. The first technique used the DeBERTa language model, and the second used the RoBERTa language model in combination with cross-attention. We obtained the second-best F1 score (0.8610) in the competition with the Contextually Augmented DeBERTa model.

## introduction causality is a cause-effect relationship between two arguments, events, processes, states, or objects in which the occurrence of one (cause) is partly responsible for the occurrence of the other (effect) @xcite . a few instances of this cause-effect relationship are illustrated in figure 1 , which were extracted from the causal news corpus (cnc) @xcite . the first instance comprises a causal relation between the phrase "allegedly being involved in the blast" (cause) and "two more youths were arrested later," indicating that the youths were arrested because they were accused of being involved in the bomb blast. the word "for" indicates that this relationship is causal. similarly, other words can be used for indication, as seen in the case of the second instance where "over" is the signal word. there are also cases where the causal relation is explicit and does not have a word to signal the causality, as can be seen in the third instance. for the sentences that do not have causality, they are either missing the effect or two more youths were arrested later for allegedly being involved in the blast. the clash took place over use of the field. two bus drivers were hurt in the attacks. ## task & data description event causality identification shared task aims at tackling inference and understanding by organizing two subtasks: a) causal event classification and b) cause-effect-signal span detection. our team participated in the first subtask, which required the participants to classify the given text into "0" (noncausal) and "1"(causal). the dataset provided in the task was the causal news corpus (cnc) deals with event causality in the news. the cnc dataset builds upon the following datasets: automated extraction of socio-political events from news (ae-spen) in 2020 @xcite and challenges and applications of automated extraction of socio-political events from text (case) in 2021 @xcite . the data in label train dev 0 1322 145 1 1603 178 cnc is based on random samples that have been collected from a total of 869 news documents. the corpus comprises 3,559 data samples, out of which 2925 data points were provided for training, 323 data points were provided for the development set, and the remaining 311 samples were used as the test set. ## methodology this section gives an exhaustive overview of the proposed pipeline. section 3.1 provides the details of the preprocessing performed on the given dataset. section 3.2 describes the augmentation technique used on the data. section 3.3 discusses the transformer models and techniques used to train the data. ## conclusion in this paper we propose contextual embedding augmented deberta and dual cross attention roberta to identify event causality. our approach yielded an f1 score of 0.8610 which was the second best system throughout the shared task. we study the behaviour of both the models in augmented and unaugmented settings to derive proper understanding about the impact of our complete pipeline. in future, experimenting with other language models and extensive hyperparameter tuning through neural architectural search will be an ideal path to follow. the augmentation was successful in maintaining its causality nature. this acts as a fine way of up sampling low resource tasks which lack adequate data.