KULF i Framework: Knowledge Utilization for Optimizing Large Language Models for Financial Causal Reasoning (2025)

Abstract: This paper presents our contribution to the Financial Document Causality Detection (FinCausal) task 2025. The FinCausal challenge centers on the extraction of cause-and-effect relationships from financial texts written in both English and Spanish. We introduce KULFi, a novel Knowledge Utilization framework designed to augment the capabilities of Large Language Models (LLMs) by leveraging the expertise of more advanced reasoning models. Through the utilization of Teacher LLMs to generate task-specific instructions, KULFi optimizes the performance of Student LLMs via automated prompt optimization. We evaluate the efficacy of KULFi on the Financial Document Causality Detection Task, where Student LLM achieves a similarity score comparable to human-guided prompt optimization for the same LLM, demonstrating significant improvements in causal reasoning performance. Our results demonstrate that KULFi enables effective knowledge transfer from more robust models to less capable ones, as well as efficient learning from training data, minimizing the need for human input in prompt design and enabling more precise causal analysis in financial contexts. Our system attained SAS and Exact Match scores of 0.92 and 0.35 on the English dataset, and 0.92 and 0.09 on the Spanish dataset, respectively. This framework has far-reaching implications, with potential applications in enhancing decision-making across complex financial environments.

## introduction the financial document causality detection task @xcite focuses on determining the causes of changes in the financial environment to generate concise financial narrative summaries. it evaluates how events or chains of events lead to transformations in financial objects within specific contexts. participants were tasked with identifying either the cause or effect for particular segments of text. the task consists of two subtasks, one in english and one in spanish, using datasets from uk and spanish financial annual reports to test the performance of multilingual models. different from earlier editions @xcite that used extractive methods, the 2025 task redefines the challenge as a generative ai problem, where systems generate cause-effect responses, assessed through exact match and similarity metrics. recently, the potential of llms to identify causal relationships and perform reasoning within natural language contexts has garnered significant attention (section 2). existing work @xcite analyzes the approach of distinguishing between causal relationships (@xmath0. while this approach is effective for many taskspecific models trained on input-output pairs, continued task-specific training may be impractical or prohibitively expensive for these general-purpose llms. in the era of large language models (llms), knowledge distillation (kd) @xcite is pivotal for transferring advanced capabilities from powerful models to weaker models on specific domains or tasks. this process mimics a skilled teacher imparting knowledge to a student, enhancing the performance of weaker models through the expertise of stronger ones. in this work, we present knowledge utilization framework, kulfi, where a model with limited reasoning ability learns from a more capable reasoning model, specifically targeting financial causal reasoning. although not yet evaluated, this framework has the potential to be generalized to a wide range of tasks where prompt optimization or knowledge transfer is required to enhance performance. ## kulfi framework while human-guided prompt engineering improves llm performance, it requires domain-specific expertise, making it labor-intensive, dataset-specific. fine-tuning llms on the given training data requires substantial computational resources, which can be a significant barrier for smaller teams and limited budgets. fine-tuned models also risk limited adaptability to new information and may suffer from catastrophic forgetting @xcite . an alternative approach could be automatic prompt optimization using training data, which reduces both cost of training llm and human involvement in designing prompts. our preliminary analysis shows that some llms possess inherently stronger reasoning abilities than others. we present kulfi-knowledge utilization for optimizing llms, an automated framework (figure 1) that employs prompt optimization using a teacher-student model. the teacher refines prompts based on the student's performance, iteratively enhancing output quality. its functions as follows: • the student llm harnesses the reasoning abilities of the teacher llm via chain-of-thought (cot) generation, (auto cot transfer). optimized prompt instructions were generated as outlined in the following sections and added to the default prompt (section 4.1) for the student llm. ## experiment setup we utilized the llama3.1-405b 2 and cohere command r+ 3 models, available as oci genai services offerings 4 . for both models, the temperature and frequency penalty were set to 0.0, and the topp value was set to 0.95, with all other parameters 2 https://ai.meta.com/blog/meta-llama-3-1/ 3 https://docs.cohere.com/v2/docs/command-r-plus 4 https://www.oracle.com/in/artificialintelligence/generative-ai/generative-aiservice/features/#models left at their default values. llama3.1-405b demonstrated superior performance with default prompts ( ## results discussion and error analysis using the kulfi framework, the performance of the student llm, command r+, consistently outperformed the default prompt and matched the performance of human-guided prompts ( with a similarity score of approximately 92%, the system exhibits robust performance, with errors primarily concentrated in specific cases. a detailed