B osch AI @ Causal News Corpus 2023: Robust Cause-Effect Span Extraction using Multi-Layer Sequence Tagging and Data Augmentation (2023)

Abstract: Understanding causality is a core aspect of intelligence. The Event Causality Identification with Causal News Corpus Shared Task addresses two aspects of this challenge: Subtask 1 aims at detecting causal relationships in texts, and Subtask 2 requires identifying signal words and the spans that refer to the cause or effect, respectively. Our system, which is based on pre-trained transformers, stacked sequence tagging, and synthetic data augmentation, ranks third in Subtask 1 and wins Subtask 2 with an F1 score of 72.8, corresponding to a margin of 13 pp. to the second-best system.

## introduction in this paper, we describe our approach to the event causality identification with causal news corpus shared task @xcite , which took place at the 6th workshop on challenges and applications of automated extraction of socio-political events from text (case 2023). the task, which builds on the 2022 iteration of the same shared task @xcite , but including more labeled data, targets the detection and extraction of causal relationships. in subtask 1, participating systems need to decide whether a sentence contains any causal relationship. subtask 2 requires extracting the spans that denote cause, effect, and trigger words (if any). our system leverages pre-trained transformer encoders and synthetic data augmentation methods, and ranks third in subtask 1. we address subtask 2 using a supervised sequence labeling model, which wins by a margin of 13 percentage points in terms of f1 over the second-best system. we model multiple causal chains per sentence via stacked labels and find that synthetic data augmentation consistently improves performance. our code is publicly available. 1 1 https://github.com/boschresearch/ boschai-cnc-shared-task-ranlp2023 2 dataset and task the causal news corpus @xcite consists of 3767 sentences extracted from news articles. cnc provides annotations of semantic relations of the form "x causes y" that indicate a causal relationship between arguments x and y. the definition of causality follows that of the con-tingency label in the pdtb-3 corpus @xcite , which is used when a statement provides the reason, explanation, or justification for another event. following timeml @xcite , the definition of events includes both actions that happen or occur and states. as illustrated by the example in figure 1 , one event is the immediate effect of another, e.g., the event expressed by "the use of a village field" is the cause of that expressed by "the clash." while following the definition of causal relations of pdtb-3, which focuses on causal relations between sentences or clauses, cnc provides span annotations for causes (arg0), effects (arg1) and signals (sig0) within sentences. spans may comprise one to several words. their boundaries are not restricted to clause or constituent boundaries. signals are expressions such as "has led to" or "causing," but not every causal relation annotation requires a signal. of all annotated relations, 30% do not contain a signal, for example: "[dissatisfied with the package cause ], [workers staged an allnight sit-in effect ]." the average signal length is 1.46 words. @xcite describe the annotation guidelines in detail. the shared task is divided into two subtasks: subtask 1 is a binary classification problem, deciding whether a sentence contains a cause-effect chain or not. subtask 2 deals with the more challenging problem of extracting the correct spans of cause, effect, and signal, where a sentence may contain more than one causal relation. in cnc, the maximum number of causal relations per sentence is four. spans are annotated using xml-like tags: 〈arg0〉 refers to causes, 〈arg1〉 to effects, and 〈sig0〉 to signals. ## modeling and augmentation in this section, we describe the neural architectures that we use to solve the two subtasks. to produce contextualized embeddings of the input sentences, we use bert-large @xcite and roberta-large @xcite . ## experimental evaluation this section describes our experimental results for both subtasks. evaluation of subtask 2 is performed using faireval foot_1 , which implements a relaxation of traditional hard-matching span evaluation metrics on sentences marked as containing a causal relation in the gold standard only. we train our on all samples of the train split, including those without causal relations. ## conclusion and outlook in this paper, we have described our modeling approach to the "event causality identification with causal news corpus" shared task (case 2023). we have proposed a multi-layer sequence tagging model that aims at identifying causal relations within news-related sentences. our approach significantly outperforms all participating systems in subtask 2. furthermore, we have shown that synthetic data augmentation methods are beneficial for this task. our results indicate that careful modeling, more advanced data augmentation, and leveraging larger language models may be fruitful directions for further improvements.