CLRG @ F in C ausal2025: Cause-Effect Extraction in Finance Domain (2025)

Abstract: This paper presents our work on Cause-Effect information extraction specifically in the financial domain. Cause and effect information is very much needed for expert decision making. Particularly, in the financial domain, the fund managers, financial analysts, etc. need to have the information on cause-effects for their works. Natural Language Processing (NLP) techniques help in the automatic extraction of cause and effect from a given text. In this work, we build various cause-effect text span detection models using pre-trained transformer-based language models and fine tune these models using the data provided by FinCausal 2025 task organizers. We have only used FinCausal 2025 data sets to train our models. No other external data is used. Our ensemble of sequence tagging models based on the Fine-tuned RoBERTa-Large language model achieves SAS score of 0.9604 and Exact match score of 0.7214 for English. Similarly for Spanish we obtain SAS score of 0.9607 and Exact match score of 0.7166. This is our first time participation in the FinCausal 2025 Task.

## introduction domain-specific causal information is very important for an informed decision making, particularly in expert decision-making processes. for example, financial organizations collect historical data of stock price movements and their causes to develop effective trading strategies. financial institutes collect and store causality information in english and other languages to understand early stock price fluctuation. the required information is published in different forms in different languages and magazines. all these information needs to be processed in real time for it to be useful for any decision making. therefore, there is a need to develop automatic cause-effect information extraction systems. the fincausal2025 shared task at the financial narrative processing workshop (fnp) addresses this step by providing annotated data in english and spanish. this paper further describes our work on the participation in this fincausal 2025 shared task where we have developed span based models by fine tuning pre-trained large language models for our purpose. ## related work the goal of the fin causal 2025 shared work @xcite was to identify causation in financial records. it was headed by antonio moreno sandoval, blanca carbajo coronado, jordiportazamorano, yanco amor torteroloorta, and doaasamy. this version analyzed datasets selected from english and spanish annual reports, signaling a move away from extractive approaches and toward question-answering (qa)focused strategies. semantic answer similarity (sas) and exact match (em), two assessment measures, were highlighted in the challenge, along with abstractive question design. advanced transformer-based models were utilized by the participants, and performance was improved by strategies such multilingual datasets and lora fine-tuning. dominique mariko, mahmoud el-haj, and his team lead the fin causal 2023 shared task, which provided improved english and spanish datasets with complex causal structures, including multieffect causes and multi-cause effects. robust system assessment was achieved by using evaluation criteria such as token-level f1 scores and exact match. innovative techniques including retrieval-augmented generation and chain-ofthought prompting, together with state-of-the-art models like roberta, span bert, and gpt-4based architectures, were used by teams to push the limits of causality identification in multilingual environments. building on previous iterations, the fin causal 2022 joint effort, headed by dominique mariko, kim trottier, and mahmoud el-haj, concentrated solely on causality detection. financial news from 2019 and excerpts from sec filings were added to the dataset. with the goal of identifying causes and effects in financial texts, participants made significant progress in detecting causality. team spock outperformed the other contestants in the use of ensemble sequence tagging models with roberta-large and the bio scheme. other noteworthy contributions were ilab's graph-based embeddings and expert neurons' clever preprocessing algorithms, which demonstrated a variety of approaches to successfully address causality extraction. by supplementing the dataset with more instances from financial news, the fin causal 2021 shared task-which was managed by dominique mariko, hanna abi-akl, estelle labidurie, stephanedurfort, hugues de mazancourt, and mahmoud el-haj-further improved causality extraction. nus-ids, the victorious team, used a bert-crf in conjunction with a viterbi decoder, using dependency graphs for token categorization. to get high accuracy in identifying causal sequences, other groups tried ensemble learning, sequence labeling, and graph neural networks. even with improvements, there were still several difficulties, such as forecasting intricate causal networks, which highlights the need for more research. the topic of causality identification in financial narratives has grown as a result of these common objectives, showing how methods have developed from straightforward extraction to complex, context-aware generative models and multilayered analytical frameworks. ## system description our model makes use of the xlm-roberta architecture, which is ideal for multilingual question-answering tasks since it uses selfattention methods to record contextual dependencies. the fundamental concept behind improving the model is to apply it directly to the span-based answer prediction problem, which entails guessing the beginning and ending locations of a response in the context. in order to comprehend and interpret the context effectively, this transformer network-based model framework functions inside a strong self-attention mechanism @xcite . ## results and discussion using their respective development datasets, the english and spanish datasets underwent independent fine-tuning procedures. by taking use of the unique traits and subtleties of the english and spanish environments, this guarantees that the models were tuned separately for each language. the outcomes repeatedly show that model performance is much improved by fine-tuning. in every measure and language, fine-tuned models perform better than their non-fine-tuned counterparts for the practice and development datasets. interestingly, em scores demonstrate significant increases, especially in spanish datasets, with gains of more than 50 percentage points in certain cases, while sas scores for fine-tuned models routinely above 0.90 in the majority of setups. fine-tuned xlm-roberta-large demonstrates its outstanding ability to comprehend semantics by achieving the highest sas score of 0.96 on the practice dataset in english datasets. the large model consistently demonstrates its capacity to generalize between phases on the development dataset, attaining an em score of 0.61 and an sas score of 0.91. the base model receives comparable scores, with an em of 0.70 and an sas of 0.94 on the development dataset, although trailing the large model by a little margin in sas. while the base model offers a compromise between semantic comprehension and accuracy in some contexts, our results highlight the large model's superiority in managing semantic complexity. spanish datasets show that fine-tuning has a major effect, especially on exact match scores. after fine-tuning, for example, the em of the base model on the practice dataset increases from 0.13 to 0.73. with the em score increasing from 0.17 to 0.71 on the development dataset, the refined base model displays a comparable pattern. the fine-tuned large model achieved a peak sas of 0.96 on the practice dataset, and similarly, the fine-tuned models' sas scores above 0.95 on both datasets. these findings show that the models can successfully adjust to multilingual data, particularly in spanish and highlight the significance of fine-tuning in improving performance across both sas and em measures. these findings provide several insights: a) making adjustments to language-specific the significance of adapting the models to the language and contextual peculiarities of english and spanish is shown in the necessity of development datasets for optimizing sas and em scores. b) the base model's success in em demonstrates its computational economy, while the xlm-roberta-large model's superiority in sas qualifies it for semantically rich jobs. spanish datasets highlight the difficulty of multilingual adaptation by relying more on finetuning for better performance.