LTRC @ Causal News Corpus 2022: Extracting and Identifying Causal Elements using Adapters (2022)

Abstract: Causality detection and identification is centered on identifying semantic and cognitive connections in a sentence. In this paper, we describe the effort of team LTRC for Causal News Corpus - Event Causality Shared Task 2022 at the 5th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2022). The shared task consisted of two subtasks: 1) identifying if a sentence contains a causality relation, and 2) identifying spans of text that correspond to cause, effect and signals. We fine-tuned transformer-based models with adapters for both subtasks. Our best-performing models obtained a binary F1 score of 0.853 on held-out data for subtask 1 and a macro F1 score of 0.032 on held-out data for subtask 2. Our approach is ranked third in subtask 1 and fourth in subtask 2. The paper describes our experiments, solutions, and analysis in detail.

## introduction a sizeable amount of text is generated every day due to increase in the amount of news available online from news portals and social media. data available on social, political, and economics has the potential to revolutionise data-driven analysis @xcite . causality identification and span detection @xcite is one such datadriven task. it is one of the many natural language processing (nlp) studies that attempts to address inference and comprehension. a causal relation is a semantic relationship between cause argument and effect argument such that the occurrence of one contributes to the occurrence of the other. cause is a span of text that results in the occurrence of an effect event. an effect is a span of text that is the consequence of the cause event and a signal is a span of text that binds both the cause and effect events. together the study of cause and effect can help in understanding what agents contribute to the causes and the effects they create. causality identification and span detection on climate science domain helps in analysing the rapid climate changes @xcite . similarly analysis on financial domain news @xcite can help in improving trading strategies. further examples include social, economic, and political sciences where the effects created by causes such as a change in policy can be identified over a period of time and analyzed. causal text mining have been shown to be beneficial for downstream tasks like summarization @xcite , question answering and making inferences. task 3 (event causality identification) of case @ emnlp 2022 @xcite aims at automatically identifying sentences that have a cause-effect event and extracting spans of text relating to cause, effect, and signal events. the shared task 3 is divided into two sub-tasks: subtask 1: causal event classification the first subtask identifies if a given event sentence contains any cause-effect. subtask 2: cause-effect-signal span detection this subtask identifies the spans corresponding to cause and effect per sentence. the causal news corpus @xcite ) comprises 3,559 event sentences, extracted from protest event news, that have been annotated with sequence labels on whether it contains causal relations or not. subsequently, causal sentences are annotated with cause, effect, and signal spans. for both tasks, we use a transformer-based model @xcite . we use adapters @xcite , a parameter-efficient fine-tuning method, in conjunction with a pre-trained model with strong language understanding and generation abilities @xcite . recent research has shown that this method is robust to over-fitting in low-resource settings @xcite . in this way, the large pre-trained model roberta, remains frozen, and only small modules the model parameters are optimized. this effectively retains acquired knowledge in the pre-trained language model. the first task was treated as a binary classification task with a single label for the input sentence, while for the second task, label was predicted for each input word of the sentence. ## conclusion and future work with the rapid growth in information from news portals, automated solutions to analyse data and draw inferences from the data play a pivotal role. our solution for the both the subtasks involved adding an adapter layer which improves the performance by avoiding full fine-tuning of the entire model and instead adding additional newly initialized weights at every layer of the transformer which are trained during fine-tuning. though the solutions work well, they could be further improved by using an ensemble model for subtask 1 and by adding an lstm @xcite and crf @xcite @xcite on top of the contextual embeddings layer for proper alignment of tokens and labels for subtask 2. in our experiments on the causal news corpus and on analyzing the misclassified samples we feel that the models for both subtasks can also benefit from having extra syntactic and semantic information. for subtask 1, verbs and signal arguments like conjunctions play a major role in determining if the sentence is causal or non-causal. similarly for subtask 2, having part-of-speech tags information for all the tokens along with contextual embedding from bert might work well. the current models have good contextual representations, but appending them with an extra embedding of the main verbs, conjunctions and parts-of-speech tags might steer the task inference in a better direction.