The Financial Document Causality Detection Shared Task ( F in C ausal 2025) (2025)

Abstract: We present the Financial Document Causality Detection Task (FinCausal 2025), a multilingual challenge to identify causal relationships within financial texts. This task comprises English and Spanish subtasks, with datasets compiled from British and Spanish annual reports. Participants were tasked with identifying and generating answers to questions about causes or effects within specific text segments. The dataset combines extractive and generative question-answering (QA) methods, with abstractly formulated questions and directly extracted answers from the text. Systems performance is evaluated using exact matching and semantic similarity metrics. The challenge attracted submissions from 10 teams for the English subtask and 10 teams for the Spanish subtask. FinCausal 2025 is part of the 6th Financial Narrative Processing Workshop (FNP 2025), hosted at COLING 2025 in Abu Dhabi.

## introduction financial analysis relies on factual data to provide a clear view of current conditions, but it also needs deeper insights to understand how and why these facts have come to be. the ultimate goal of fin-causal 2025 is to determine, regarding a given context, which events or chain of events can cause a financial object to be modified or an event to occur. historically, extracting cause-effect relationships has been primarily extractive, as demonstrated in previous iterations of the fincausal task @xcite @xcite . however, 2025 task is framed as a question-answering task, requiring systems to respond to causality-focused questions, with their answers assessed through exact matching and similarity metrics. the task comprises two subtasks, one in english and one in spanish. participants were required to provide the answer for each question using any method of their choice. both datasets were created from annual reports, making them suitable for testing of multilingual models. annual reports detail a company's economic, financial, and operational performance during the year, including management insights, corporate governance, and social responsibility. for this task, we focus solely on the narrative sections, excluding the financial statements. ## the dataset in both subtasks, causality was described as a relationship in which two events are connected, with one event, occurring earlier in time, acting as the trigger for the other. causes and their effects may be represented by agents or facts. there are two primary types of causes: to create the dataset, a question was formulated for each context asking for either the cause or the effect, followed by a corresponding answer. each context contains a cause-effect relationship, though not every sentence in the sample is case of causality. a maximum of two questions per context were allowed in cases involving complex causal relationships, such as a chain of three or more elements or non-linear relationships. contexts lacking a clear or complete causal relationship, or those express-ing conditions, purposes, or concessions, were excluded. this exclusion was based solely on the provided context, without drawing inferences from any external knowledge. the dataset comprises three key components: in english, the context ranged from 9 to 191 words, with an average of 43 words. in spanish, it spanned 4 to 255 words, averaging 46 words. each context has its own id. sequential ids were given when two questions were formulated for a single context (with letters xx.a, xx.b, and xx.c, etc.) and when the context was divided into multiple parts (with numbers xx.xx.1, xx.xx.2, xx.xx.3...). there was an emphasis on not inserting external data or superfluous details. questions were framed in third person or impersonally if the source text used the first person. when multiple text chains were possible answers, the option with the greatest level of detail was selected. in contexts with two questions, one answer could partially or fully match the other one. both the english and spanish dataset sizes are shown in ## evaluation metrics semantic answer similarity (sas), as introduced in @xcite , is the primary metric used to measure how similar two texts are based on their semantic meaning rather than just word-forword matching. it is particularly useful in evaluating responses in tasks like abstractive questionanswering. sas utilizes pre-trained language models like bert @xcite or sentence transformers @xcite to generate text embeddings and then computes cosine similarity between these embeddings to assess how closely two pieces of text align in meaning, even if they use different words or structures. this allows for more accurate evaluation of content that conveys the same idea but is expressed differently. we chose to include sas as a metric because, in fincausal 2023, the majority of the participating models were generative prompting-based models (based on gpt), and a traditional metric such as exact match (em) alone proved inadequate for accu-rately evaluating their outputs. for fincausal 2025, we have used the paraphrase multilingual mpnet base v2 model foot_2 using a sentence transformer architecture built on a pre-trained xlm-roberta model @xcite to give support to the spanish and english subtasks, converting text into 768-dimensional vectors. additionally, we used exact match (em) as a secondary metric. it measures the accuracy by checking whether the model's generated answer matches the reference answer exactly, word by word. both metrics, sas and em, are averages over the individual values of the examples to which they are applied. ## conclusions after several editions dedicated to the extraction of cause-effect segments in financial annual reports, fincausal 2025 has been approached as a qa task. the challenge includes both english and spanish subtasks, each supported by datasets containing 2,500 samples. this year's edition incorporated the sas metric alongside the em metric for a more comprehensive evaluation of participants' responses. in fact, the sas metric was suggested by participants of the previous fincausal 2023. in the english subtask, team nirvanatear achieved top performance by fine-tuning gpt4omini on targeted datasets, while the tu graz data team employed multilingual models with lorabased fine-tuning and bilingual datasets. team sarang showcased the potential of lightweight approaches without external databases. the laith system employs a traditional extractive framework based on the multilingual xlm-roberta-large model. the model has been fine-tuned on both english and spanish fincausal datasets, without external databases. oragenai introduced kulfi, a framework automating prompt optimization through teacher-student interactions. many teams also used these systems in the spanish subtask, demonstrating the adaptability of their models. notably, team lenguajenatural.ai highlighted the importance of language-specific resources.