Exploring the Effectiveness of Multilingual and Generative Large Language Models for Question Answering in Financial Texts (2025)

Abstract: This paper investigates the use of large language models (LLMs) for financial causality detection in the FinCausal 2025 shared task, focusing on generative and multilingual question answering (QA) tasks. Our study employed both generative and discriminative approaches, utilizing GPT-4o for generative QA and BERT-base-multilingual-cased, XLM-RoBerta-large, and XLM-RoBerta-base for multilingual QA across English and Spanish datasets. The datasets consist of financial disclosures where questions reflect causal relationships, paired with extractive answers derived directly from the text. Evaluation was conducted using Semantic Answer Similarity (SAS) and Exact Match (EM) metrics. While the discriminative XLM-RoBerta-large model achieved the best overall performance, ranking 5th in English (SAS: 0.9598, EM: 0.7615) and 4th in Spanish (SAS: 0.9756, EM: 0.8084) among 11 team submissions, our results also highlight the effectiveness of the generative GPT-4o approach. Notably, GPT-4o achieved promising results in few-shot settings, with SAS scores approaching those of fine-tuned discriminative models, demonstrating that the generative approach can provide competitive performance despite lacking task-specific fine-tuning. This comparison underscores the potential of generative LLMs as robust, versatile alternatives for complex QA tasks like financial causality detection.

## introduction the increasing complexity of financial documents necessitates advanced methodologies to extract and analyze causality within such texts. the fincausal 2025 shared task introduced a hybrid questionanswering (qa) framework for detecting causal relationships in financial disclosures across english and spanish languages. the task required participants to address a combination of extractive and generative qa challenges. questions were formulated abstractly, focusing on either the cause or the effect of a relationship, while answers were required to be extracted directly from the provided financial texts. evaluation of the task was based on two metrics: exact match, which measures the strict correctness of answers, and semantic answer similarity (sas), which evaluates the semantic alignment between predicted answers and ground truths. the multilingual nature of the task, combined with the hybrid qa format, offered a unique opportunity to test the performance of state-of-the-art models in addressing causality detection across different linguistic contexts. this paper outlines our approach to the task, which involved experimenting with multiple pretrained large language models (llms), including gpt-4o, xlm-roberta (base and large), and bert-base-multilingual-cased. the results demonstrate the effectiveness of xlm-roberta-large, which achieved the best performance among the tested models, securing a 5th-place rank in english and 4th-place rank in spanish. these findings highlight the importance of leveraging multilingual large language models for nuanced tasks like financial causality detection. the code and dataset are available in github: https://github.com/ yemen2016/fincausal-2025 ## related work the task of causal relationship detection in financial texts has garnered significant attention in recent years, particularly with the rise of advanced natural language processing (nlp) models @xcite . early approaches in this domain often relied on rule-based systems and traditional machine learning methods, such as support vector machines (svms) and decision trees @xcite , to detect causal patterns in financial reports and news articles. these models, however, required extensive feature engineering and often struggled to capture the complex nuances of causal relationships in the language of finance. in recent years, the advent of deep learning and transformer-based models, such as bert and its multilingual variants, has revolutionized this field by providing models capable of understanding and extracting contextual information with little to no manual feature extraction @xcite . a significant body of work in causal relationship extraction from financial text has focused on the use of pre-trained large language models like bert and its multilingual variants @xcite . researchers have fine-tuned these models on domain-specific datasets, achieving state-of-theart results in both causal relationship extraction and other financial text analysis tasks, such as sentiment analysis and event extraction @xcite . for instance, studies have shown that xlm-r and xlm-roberta models, which are trained on a diverse set of multilingual corpora, can generalize well to a variety of languages, including english and spanish, making them ideal for multilingual financial text analysis tasks @xcite . fine-tuned plms have been demonstrated to achieve competitive performance, outperforming traditional machine learning approaches, particularly when working with large and complex datasets like financial reports @xcite . alongside fine-tuned models, there has been growing interest in leveraging generative models, such as gpt-3 and gpt-4, for causal relationship detection @xcite . unlike extractive models, which pull information directly from the text, generative models produce new text based on the input provided, offering more flexibility in handling abstract and complex questions. while gpt-3 and gpt-4 have been primarily used in conversational ai, recent studies have explored their potential in tasks like question answering (qa) @xcite @xcite , euphemism detection @xcite . research has shown that generative models can be particularly useful in scenarios where few-shot learning is beneficial, as they can adapt to new tasks with minimal training data. however, while generative models show promise, they often require careful prompt engineering to achieve optimal results, as their performance can vary depending on the context and number of examples provided @xcite . ## discussion of results the experimental results highlight the comparative performance of fine-tuned pre-trained language models (plms) and gpt-4o prompting techniques for detecting causal relationships in financial texts across english and spanish. for both semantic answer similarity (sas) and exact match (em), finetuned models demonstrated superior performance, with xlm-roberta-large emerging as the bestperforming model. it achieved the highest sas scores (0.96 for english and 0.98 for spanish) and em scores (0.762 for english and 0.808 for spanish), showcasing its capability to handle complex extractive question-answering tasks. these results underscore the strength of leveraging large-scale multilingual plms for tasks requiring precision and contextual understanding. among the fine-tuned models, xlm-roberta-base also performed strongly, particularly in spanish, where it achieved a high sas of 0.97 and an em of 0.739. bert-base-multilingual-cased, while slightly behind, still delivered competitive results, particularly in english, with an em of 0.517. this demonstrates that even smaller, less computationally intensive models can perform effectively, particularly when fine-tuned on specific tasks. in contrast, gpt-4o, while initially less effective in zero-shot configurations (sas: 0.77 and em: 0.002 for both english and spanish), showed significant improvement under few-shot settings. by incorporating up to eight random examples during prompting, gpt-4o achieved sas scores of 0.94 for both languages, with corresponding em scores of 0.515 for english and 0.487 for spanish. these results illustrate gpt-4o's adaptability and potential in resource-constrained environments where extensive fine-tuning of large plms is not feasible. however, the relatively lower em scores in comparison to fine-tuned plms suggest that gpt-4o, while versatile, may not yet match the precision offered by task-specific models in exact-match scenarios. the disparity in performance between english and spanish, particularly for fine-tuned models, further underscores the influence of language-specific characteristics on model effectiveness. spanish financial texts consistently yielded higher sas and em scores, suggesting better alignment between the models and linguistic nuances of spanish financial disclosures. this finding highlights the need for tailored approaches and datasets to ensure optimal performance in multilingual environments. in summary, the results demonstrate the complementary strengths of fine-tuned plms and generative models. fine-tuned models excel in accuracy and task-specificity, while gpt-4o offers a flexible alternative, particularly when fine-tuning is infeasi-ble. future research could explore hybrid methodologies that combine the robustness of fine-tuned models with the adaptability of generative techniques, potentially enhancing performance across diverse tasks and languages. ## conclusion this study investigated the effectiveness of finetuned pre-trained language models (plms) and generative prompting techniques for causal relationship detection in financial disclosures in english and spanish. the results underscore the complementary strengths of both approaches in addressing this challenging task. the gpt-4o generative model showcased impressive adaptability, particularly in few-shot configurations, where its sas scores approached those of fine-tuned plms. despite lower em scores, gpt-4o's ability to perform competitively without extensive fine-tuning makes it a valuable alternative in scenarios with limited resources or time constraints. these results reinforce the versatility of generative language models, particularly when used with targeted prompting techniques. on the other hand, fine-tuned plms, particularly xlm-roberta-large, demonstrated superior performance, achieving the highest scores in both semantic answer similarity (sas) and exact match (em) metrics. these results highlight the advantages of leveraging large-scale multilingual plms for tasks requiring high precision and contextual understanding. the performance of smaller models, such as xlm-roberta-base and bert-base-multilingual-cased, also underscores the potential of fine-tuned plms to deliver strong results even with reduced computational demands. notably, the consistently higher performance on spanish financial texts highlights the impact of language-specific nuances in financial disclosures and emphasizes the importance of tailored datasets and approaches in multilingual contexts. overall, this work demonstrates the value of using fine-tuned plms and generative approaches for extractive question answering tasks. future research could focus on hybrid methodologies, integrating the precision of fine-tuned models with the adaptability of generative models, to further enhance causal relationship detection in financial texts across diverse languages and domains.