P resi U niv at F in C ausal 2025 Shared Task: Applying Fine-tuned Language Models to Explain Financial Cause and Effect with Zero-shot Learning (2025)

Abstract: Transformer-based multilingual question-answering models are used to detect causality in financial text data. This study employs BERT (CITATION) for English text and XLM-RoBERTa (CITATION) for Spanish data, which were fine-tuned on the SQuAD datasets (CITATION) (CITATION). These pre-trained models are used to extract answers to the targeted questions. We design a system using these pre-trained models to answer questions, based on the given context. The results validate the effectiveness of the systems in understanding nuanced financial language and offers a tool for multi-lingual text analysis. Our system is able to achieve SAS scores of 0.75 in Spanish and 0.82 in English.

## introduction as the growing connectivity of global markets and the rising use of multiple languages in communication continue, the need for a model that can interpret text data has become increasingly important. question answering (qa) is a key component in extracting or identifying relevant data across domains. traditionally, qa models have been trained separately for individual languages, resulting in fragmented systems that are costly to maintain and difficult to scale. although some multilingual models such as typologically diverse question answering (tydiqa) @xcite and multilingual knowledge questions and answers (mkqa) @xcite have been introduced in recent years, they often struggle with maintaining accuracy in non-english languages or processing large datasets efficiently. these limitations underscore the gap between current technologies and the demands of modern multilingual applications @xcite . in light of this, we decided to participate in the 2025 fincausal shared task. the goal of the shared task is to create a strong and effective multilingual system for english and spanish that can cater to international markets. ## problem statement the fincausal 2025 shared task foot_0 focuses on the extraction of causal relationships from financial reports @xcite . the task involves processing financial reports to identify explicit and implicit causal relationships between financial events, entities, or market factors. participants are required to develop models that can accurately detect these causal links, taking into account the complex, and often ambiguous nature of the financial language. this task expands on earlier work in extracting causal relationships, which has been studied in areas like event extraction @xcite and causal inference in news data. unlike prior editions of the shared task, this edition challenges participants to handle diverse financial contexts with increased accuracy and scalability from financial reports @xcite . the aim of the shared task is to advance the field of financial event analysis by providing robust, scalable methods for causal extraction in real-world financial data @xcite . ## related work research on multilingual question-answering has advanced significantly, frequently as a result of shared assignments that address many aspects of the qa pipeline. the question answering for machine reading evaluation (qa4mre) @xcite tasks which were organized at clef from 2011 to 2013, focused on machine reading comprehension across languages, was one of the first multilingual qa challenges. the best-performing systems used hybrid strategies to enhance their reasoning abilities across multilingual texts by fusing rule-based techniques with machine learning models. the advent of datasets like tydi qa @xcite ) marked a turning point for multilingual qa by emphasizing typological diversity. this dataset aimed to provide a benchmark for systems handling typologically distinct languages, such as swahili and finnish. participants in shared tasks built on tydi qa used techniques ranging from fine-tuned transformer-based models, such as multilingual bert @xcite and xlm-roberta @xcite to multi-task learning for better performance on low-resource languages. in 2020, the mkqa shared task @xcite emphasized the evaluation of systems on a translated version of the natural questions dataset. the challenge revealed that translation-based evaluation often introduces biases, as noted by the top participants. these teams leveraged cosine similarity measures and context-sensitive embeddings from pre-trained models to tackle semantic drift during translation @xcite . the semeval 2022 multilingual news article similarity shared task required systems to handle domain-specific and multilingual inquiries. in order to enhance performance in a variety of settings, winning entries combined cross-lingual retrieval models with retrieval-augmented generation (rag) frameworks @xcite ). an effective technique for solving contextual ambiguity and improving substitute generation in multilingual contexts is prompt engineering on large-language models @xcite . these shared tasks and their evolving methodologies have significantly shaped the development of efficient qa systems, demonstrating the interaction between dataset design, evaluation strategies, and model capabilities in advancing multilingual nlp. ## dataset the dataset used in the shared task has 2 tracks for 2 different languages -english and spanishconsisting of data from financial annual reports in those languages. further details of the dataset can be found in @xcite . 2 the shared tasks organisers provided three sets of data for both languages. the reference and training datasets have 4 columns namely "id", "text", "question" and "answer". the "id" column is an identifier for each instance of the data. the "text" column contained the context which has both, the cause and the effect. the "question" column was the question that was asked, and the "answer" column is the expected answer. the testing dataset had the first 3 columns as the training dataset, and the shared task was to predict the answer. questions in the dataset required participants to use the given text data to either identify the cause(s) given the effect(s) or vice versa for the financial data. all the columns are delimited by a semicolon (;). ## system in this section, we describe our system. ## results and analysis in this section, we report and analyze our results. ## conclusion and future work our transformer models demonstrated the capability to extract and predict cause-effect relationships from financial data. this system not only enhances the analytical process of complex multilingual financial documents, but also fosters datadriven decision-making to promote economic stability. while the model did not achieve the best overall performance, it exhibited a strong semantic understanding of the data. however, further refinements and fine-tuning would help us achieve better verbatim matching and a better understanding with domain-specific nuances in diverse datasets. in the future, we plan to enhance our model by incorporating an explainability module to provide human-readable explanations for causal predictions, thereby improving user trust and interpretability. we also plan to explore the model's multilingual capabilities by including additional languages and implementing cross-lingual transfer learning to address linguistic nuances more effectively.