Addressing Hallucination in Causal Q & A : The Efficacy of Fine-tuning over Prompting in LLM s (2025)

Abstract: This paper presents our approach and findings for participating in the FinCausal 2025 competition, which addresses causal question answering derived from financial documents, specifically English and Spanish annual reports. We investigate the effectiveness of generative models, such as Llama, in contrast to common extractive methods like BERT-based token classification. While prompt optimization and few-shot learning offer some improvements, they were insufficient for consistently outperforming extractive methods in FinCausal, suffering from hallucinations. In contrast, fine-tuning generative models was shown to be essential for minimizing hallucinations and achieving superior performance. Using our fine-tuned multilingual model for both tasks, we outperform our extractive and monolingual approaches, achieving top results for Spanish and second-best for English in the competition. Our findings indicate that fine-tuned large language models are well-suited for causal Q&A from complex financial narratives, offering robust multilingual capabilities and effectively mitigating hallucinations.

## introduction causality extraction from financial documents is vital for knowledge-driven decisionmaking @xcite . financial analysts must identify the various factors that influence performance, including economic shifts, market trends, and regulatory policies. detecting causality enables models to interpret cause-effect relationships in complex financial events, enhancing insights into financial risks, investment opportunities, and strategic decisions. the fincausal shared tasks have progressively advanced causality detection in finance, evolving from span-based detection in 2020 to addressing implicit causality in 2021 and multi-step reasoning in 2022. the focus of the 2025 task transitions to generative models for causality extraction, requiring models to answer open-ended questions about causes and effects through interpretative and abstractive methods. fincausal 2025 aims for models to interpret both explicit and implicit causal relationships, moving beyond token-level accuracy to provide coherent, contextually relevant answers. input. question: a natural language question posed by a user, e.g., "what is the main reason why the group trading continues to be subdued?"; context passage: a passage of text that contains the answer to the question, e.g., "overall, group trading continues to be subdued in large part due to legacy issues." output. extractive answer: a contiguous span of text from the passage that directly answers the question, e.g., "legacy issues". evaluation metrics. exact match (em): the percentage of questions for which the extracted answer exactly matches the gold-standard answer. ## related work over the years, fincausal tasks have progressed from extractive to generative approaches. in 2020 and 2021, models like bert @xcite and roberta @xcite used token classification and bio tagging to identify cause-effect spans, achieving high token-level accuracy @xcite . in 2022, methods such as by @xcite combined pre-trained models with post-processing heuristics, improving exact match (em) and semantic answer similarity (sas) scores @xcite . ensemble techniques with models like sec-bert enhanced implicit causality detection but struggled with abstract responses. ## method for this research, two types of extractive q&a methods have been investigated. first, token classification using bert-based models detailed in section 3.1, and second, generative models comparing a variety of pre-trained llms in a few-shot setting with fine-tuning llama 3.1 with a multi-lingual dataset. ## results and discussion the results demonstrate a clear advantage of finetuned generative models over fine-tuned extractive models for the open-ended causal extraction tasks in fincausal 2025. extractive models such as bert performed moderately well in identifying explicit causal links where linguistic markers (e.g., "due to," "as a result of") were present. ## conclusion generative methods can outperform common extractive methods in extractive q&a tasks, provided that hallucinations are minimized. however, prompt engineering alone is not sufficient to achieve this. while few-shot learning represents an improvement, it also falls short of consistently achieving better results than extractive methods. in contrast, fine-tuning provides the necessary control to remove nearly all hallucinations in these tasks. moreover, fine-tuned llms demonstrate remarkable adaptability to tasks in a language not encountered during fine-tuning, offering excellent multilingual capabilities. using an additional model as a tiebreaker further enhances performance and suggests promising potential for a future mixture of expert solutions tailored to extractive q&a tasks.