SPOCK @ Causal News Corpus 2022: Cause-Effect-Signal Span Detection Using Span-Based and Sequence Tagging Models (2022)

Abstract: Understanding causal relationship is an importance part of natural language processing. We address the causal information extraction problem with different neural models built on top of pre-trained transformer-based language models for identifying Cause, Effect and Signal spans, from news data sets. We use the Causal News Corpus subtask 2 training data set to train span-based and sequence tagging models. Our span-based model based on pre-trained BERT base weights achieves an F1 score of 47.48 on the test set with an accuracy score of 36.87 and obtained 3rd place in the Causal News Corpus 2022 shared task.

## introduction subtask 2 of the the causal news corpus shared task at the case-22 (workshop on challenges and applications of automated extraction of sociopolitical events from text) addresses the causal information extraction problem @xcite . the goal of this task is to detect the spans of text in an input sentence that represent cause-effect pairs and, if extant, to also detect the text spans that "signal" this causal relationship. figure 3 shows a sample from the data set. simple examples of such signal spans are result in, lead to, and due to. in other cases, the causal relationship is implicit and it is important to understand the meaning of the whole sentence to detect causality. the causal news corpus data set contains sentences with both implicit and explicit causal relationship, so for this task, language understanding is an essential step. we adopt different pre-trained language models to develop our system owing to their tremendous success in natural language understanding tasks. in this paper, we train and evaluate the performance of span-based and sequence tagging neural network models for the cause-effect-signal span detection task. our team name spock (span and sequence based models for causal knowledge) for the causal news corpus 2022 shared task is inspired by these model architectures. we trained a span-based (eberts and ulges, 2019) causality extraction system foot_0 by fine tuning the bert-base @xcite model. this model resulted in an f1 score of 47.48 and accuracy score of 36.87. this was our best performing model compared to the ensemble of sequence tagging models based on the bio scheme using the bert-base and roberta-large @xcite language models. ## dataset and task we use the data sets from causal news corpus 2022 in our experiments. the sentences in this data set are collected from news sources containing event mentions. there are two subtasks in this challenge: subtask 1 is causal event classification, where the goal is to determine if a sentence expresses a cause-effect relationship; subtask 2 is cause, effect and signal span detection, where the goal is to identify the span of words in a sentence corresponding to a cause, effect, or signal (a span indicating the existence of a causal relation). this paper documents two approaches towards subtask 2. the training and dev set from subtask 2 are used for the training and evaluation of our models. in the final submission to the challenge, the trained models were used to obtain predictions on the test set. this data set contains labels for cause each example in the training and dev sets is labeled with a single pair of cause and effect span. some sentences contain multiple cause-effect pairs; each pair comprises a separate example, so that each example has a single cause and effect pair. not all sentences in the data set contain a signal span. in some examples, the signal span overlaps with the cause or effect span. we show some examples in figure 1 . ## methodology we experimented with two types of neural models for the causal news corpus 2022 challenge. ## conclusion in this paper, we adopt two approaches towards solving the cause-effect-signal detection task for participating in the subtask 2 of the causal news 136 corpus 2022 challenge. the span-based model outperforms the ensemble of sequence tagging models in both the dev set and the blind test set. in future work, we would like to adapt the models to predict multiple cause-effect pairs for a sentence. we will also focus on addressing the lack of large labeled data sets for this tasks by utilizing semi-supervised domain adaptation or generalization techniques.