Sarang at F in C ausal 2025: Contextual QA for Financial Causality Detection Combining Extractive and Generative Models (2025)

Abstract: This paper describes our approach for the FinCausal 2025 English Shared Task, aimed at detecting and extracting causal relationships from the financial text. The task involved answering context-driven questions to identify causes or effects within specified text segments. Our method utilized a consciousAI RoBERTa-base encoder model, fine-tuned on the SQuADx dataset. We further fine-tuned it using the FinCausal 2025 development set. To enhance the quality and contextual relevance of the answers, we passed outputs from the extractive model through Gemma2-9B, a generative large language model, for answer refinement. This hybrid approach effectively addressed the task’s requirements, showcasing the strength of combining extractive and generative models. We (Team name: Sarang) achieved outstanding results, securing 3rd rank with a Semantic Answer Similarity (SAS) score of 96.74% and an Exact Match (EM) score of 70.14%.

## introduction causality within financial documents is necessary for understanding financial markets and making informed decisions. manually extracting causal relationships from financial data is both tedious and time-consuming. automating this process enhances efficiency and enables the analysis of large volumes of data that would be impractical to handle manually. the fincausal 2025 shared task @xcite , part of the financial narrative processing workshop, focuses on advancing methods for detecting causal relationships in financial texts. the task involves identifying and extracting causes and effects within given segments from financial annual reports, with datasets provided in both english and spanish. this year's edition introduces a shift from traditional extractive methods to a generative ai framework. participants must answer abstractive questions about causes or effects, with evaluations based on ex-act matching and semantic similarity metrics. we started with prompt engineering with zero-shot and few-shot prompting to efficiently explore various llms, namely llama3.2-1b-instruct, llama-3.2-3b-q8, llama-3.1-8b-instruct-q8_0,mistralins-7b-q4, gemma-2-2b-it, gemma-2-9b-it, gemma-2-27b, etc. our best-performing system is finetuning + refinement using gemma2-9b. the rest of the paper is as follows. section 2 contains related work, section 3 describes the dataset, section 4 describes our methodology, section 5 contains experimental results, section 6 describes strengths and weaknesses, section 7 provides feedback on the dataset, and section 8 includes conclusions and future work. ## related work the necessity for precise identification of causeeffect links in domain-specific situations has made the extraction of causal relationships in financial documents a critical task in natural language processing. the fincausal shared tasks, conducted between 2020 and 2022, have significantly contributed to the advancement of research in financial text analysis by establishing benchmarks for detecting and extracting causal relationships within financial texts. with each successive edition of the event, it introduced more complex datasets and refined evaluation metrics, driving progress and innovation in this domain. the 2020 shared task @xcite laid the groundwork by offering a foundational dataset and benchmarks for causal extraction. subsequent editions in 2021 @xcite and 2022 @xcite introduced increasingly intricate causal chains, highlighting the limitations of purely extractive approaches and promoting the adoption of hybrid architectures for enhanced performance. in recent years, hybrid methods that integrate extractive and generative models have demonstrated potential in overcoming these challenges. authors in @xcite proposed a method where an extractive step selects relevant information, which is then summarized and used to condition a transformer language model for text generation. further, a systematic comparison of generative and extractive readers by @xcite highlighted that extractive readers often outperform generative ones in short-context qa tasks and exhibit better out-of-domain generalization. neurips 2020 efficientqa competition @xcite highlights the balance between efficiency and accuracy in qa systems. the competition demonstrated that well-tuned lightweight extractive models can deliver performance close to state-of-the-art performance while avoiding the high computational costs of larger generative models. these findings are especially valuable for scaling hybrid architectures in practical financial applications. expanding on previous research, our method utilizes roberta @xcite , fine-tuned on the squadx and fin-causal 2025 datasets, to accurately identify causal links. to enhance contextual relevance and semantic coherence, we integrate a generative refinement step powered by gemma2-9b. this hybrid approach effectively combines extractive and generative strategies, achieving high scores in both semantic similarity and exact match evaluations. ## dataset for fincausal2025 in the provided development set, we could load 1996 rows, excluding a few bad entries. we prepared two variants of this data, one as it is, i.e. 1,996 samples and another cleaned version of 1,985 samples, which contains only those entries where the answer is a sub-string of context. the development sets are provided in a csv file format with the following headers: id, text, question and answer, separated by semicolons (;). the evaluation dataset includes only the id, context and question fields; we are supposed to extract an answer. ## experimental results inspired from @xcite , we tried prompt tuning and fine-tuning of gemma-2-2b-it, but in both cases, i.e. prompt tuning and fine-tuning using quantized low-rank adaptation (qlora) @xcite , gemma-2-2b-it was not behaving as expected, so we dropped the idea of prompt tuning, which we strongly believe could have given much better result provided we prompt tune either 9b or 27b variant of gemma2. next, we fine-tuned roberta-base on the development set, using a 90:10 train/validation split. this achieved a sas score of 0.465 and an exact match score of 0.071 on validation set. later we changed the model checkpoint to consciousai/question-answering-roberta-bases 5 which is encoder-only (roberta-base) @xcite with questionanswering lm head, finetuned on squadx @xcite . we tried fine-tuning the consciousai checkpoint as well and observed a further performance boost (baseline-3 as in ## conclusions and future work we have described all our experimented approaches: zero-shot, few-shot, dynamic few-shot prompting on various llms, parent-child llm, fine-tuning and a combination of fine-tuning + answer enhancement using llm. our best submission achieved an sas of 0.9674 and an exact match score of 0.7014, outperforming initial baselines. in addition, we performed a comparative analysis of the gap in the exact match. future work will focus on resource constraints to fully explore the prompt tuning of larger models. also, it will be interesting to explore data augmentation to fine-tune the consciousai checkpoint. in addition, trying llm agents can not be ruled out. jonathan pilault, raymond li, sandeep subramanian, and chris pal. 2020. on extractive and abstractive neural document summarization with transformer language models. in proceedings of the 2020 conference on empirical methods in natural language processing (emnlp), pages 9308-9319, online. association for computational linguistics. pranav rajpurkar, jian zhang, konstantin lopyrev, and percy liang. 2016. squad: 100,000+ questions for machine comprehension of text. in proceedings of the 2016 conference on empirical methods in natural language processing, pages 2383-2392, austin, texas. association for computational linguistics.