One More Question is Enough, Expert Question Decomposition ( EQD ) Model for Domain Quantitative Reasoning (2025)

Abstract: Domain-specific quantitative reasoning remains a major challenge for large language models (LLMs), especially in fields requiring expert knowledge and complex question answering (QA). In this work, we propose Expert Question Decomposition (EQD), an approach designed to balance the use of domain knowledge with computational efficiency. EQD is built on a two-step fine-tuning framework and guided by a reward function that measures the effectiveness of generated sub-questions in improving QA outcomes. It requires only a few thousand training examples and a single A100 GPU for fine-tuning, with inference time comparable to zero-shot prompting. Beyond its efficiency, EQD outperforms state-of-the-art domain-tuned models and advanced prompting strategies. We evaluate EQD in the financial domain, characterized by specialized knowledge and complex quantitative reasoning, across four benchmark datasets. Our method consistently improves QA performance by 0.6% to 10.5% across different LLMs. Our analysis reveals an important insight: in domain-specific QA, a single supporting question often provides greater benefit than detailed guidance steps.

## introduction the performance of llms may significantly degrade in specialized domains @xcite . even advanced llms, such as gpt-4o @xcite and llama3 @xcite , exhibit substantial gaps compared to human experts in domain-specific question answering (qa), particularly in tasks involving quantitative reasoning, like financial analysis @xcite . this performance gap stems from the complex terminology and specialized knowledge inherent in these domains, which are often underrepresented in the pretraining corpora used for general-purpose llms. recent research addresses domain quantitative reasoning challenges through two main approaches: context (c): undesignated hedges was $ 41.2 million and … decomposed questions from gpt (dq1): 1. how do we calculate cash flow hedges? 2. how do we determine cash flow hedges for 2010? 3. how do we determine cash flow hedges for 2011? 4. what was the cash flow hedges amount for 2010? 5. what was the cash flow hedges amount for 2011? 6. how do we calculate the percentage change in cash flow hedges between 2010 and 2011? original question (q): what is the percentage change in cash flow hedges in 2011 compared to the 2010? figure 1: a practical example comparing different qa processes. general llms struggle to give correct answers directly. the cot method attempts to simplify the question, but often decompose the query into overly detailed steps, introducing confusion. in contrast, our eqd model adds a single sub-question that effectively guides the llm toward the correct answer. two often-overlooked aspects are potential to mitigate these limitations. first, complex domain knowledge can often be decomposed into simpler, more general components. for instance, a financial question like "what is the roi of the investment" can be transformed into basic arithmetic questions about "the initial investment, returns, and percentage change". this suggests that much of the domain-specific knowledge encoded in finetuned models may be redundant. second, given llms' inherent strong reasoning abilities, detailed, step-by-step guidance may be unnecessary, or even detrimental. overly verbose reasoning chains can introduce noise or distract from the core problem, suggesting that targeted questions focusing on key challenges tend to be more effective. based on these insights, we develop an expert question decomposition (eqd) model that generates concise and effective supporting questions to guide llms in domain-specific reasoning tasks. we select the financial domain, which is characterized by specialized knowledge and quantitative requirements, as the testbed for domain quantitative reasoning. as illustrated in figure 1 , a challenging financial question that remains unsolvable using standard decompositions becomes solvable with a single, supporting question from our eqd model. we consistently observe that such questions significantly improve qa performance. eqd is developed through a two-step process: domain fine-tuning and qa expert alignment. in the first step, we fine-tune llama 3.1-8b-instruct model using step-by-step question data from financial dialogues. unlike prior approaches that aim to inject broad domain knowledge into llms, our method focuses specifically on fine-tuning the model to decompose domain questions into simpler sub-questions. in the second step, we use a rewardbased alignment process. we design a novel reward function that measures the impact of supporting questions by comparing qa performance with and without decomposition. this reward guides the model through reinforcement learning to optimize the quality of generated sub-questions. our approach balances computational efficiency and domain knowledge integration. it requires only a small decomposition dataset and a representative domain qa dataset, substantially less than domainspecific llm fine-tuning. eqd is also compatible with both open-and closed-source llms, unlike domain-specific models such as finma @xcite and investlm @xcite , which are tied to outdated base models. during inference, eqd typically add only a single supporting question, incurring minimal additional processing overhead and preserving response time comparable to zero-shot prompting. beyond its efficiency, eqd demonstrates strong performance in improving domain qa. across four financial qa benchmarks, it achieves performance gains of 0.6% to 10.5% across multiple llms, outperforming advanced domain-adapted models and prompting techniques. these results challenge the conventional emphasis on comprehensive and stepby-step cot prompting, revealing that concise and supporting questions can lead to better llm reasoning in specialized domains. in summary, our key contributions include: 1 we propose a two-stage training framework for expert question decomposition, which integrates domain knowledge efficiently while maintaining inference time comparable to zero-shot prompting. it requires only a few thousand examples and one gpu for training. we have made our code publicly available at: eqd github repository. 2 we introduce a novel answer comparison reward that guides eqd to generate concise and effective supporting questions. experiments on four financial qa datasets show that our method consistently improves the performance of various llms by 0.6% to 10.5%, and achieve at least a 5% improvement over existing qa approaches. 3 our results and analysis reveal that concise and supporting questions are more effective than extensive reasoning steps, providing new insights into llm reasoning mechanisms. ## related work llms have shown progress in quantitative reasoning @xcite , but continue to face challenges in incorporating specialized domain knowledge @xcite . the financial domain, with its combination of technical terminology and numerical reasoning, has emerged as a key testbed for evaluating domain-specific reasoning capabilities. nlp techniques play an important role in advancing financial applications @xcite , and several financial benchmark datasets have been introduced to evaluate different aspects of domain quantitative reasoning. finqa @xcite and convfinqa @xcite focus on multi-20356 llm fine-tuning step 2: qa expert alignment step 1: domain fine-tuning domain question decomposition data original question: q: what percentage did the balance increase from 2007 to 2009? dialogue questions: q1: what was the net change in the value of the balance from 2007 to 2009? q2: what was the 2009 value? q3: what is the percent change? qd model qd model c q c q dqs qa model qa model direct qa answer -2 +1 -1 +2 proximal policy optimization qd model question decomposition model c context question q dqs decomposed questions qd-qa answer step mathematical reasoning, while tatqa @xcite addresses the challenge of processing diverse input formats, including structured tables and unstructured text in question answering. llms have prompted the development of various methods to enhance domain reasoning, including specialized qa models @xcite , prompting techniques @xcite , and reasoning planning frameworks @xcite . however, most existing methods primarily depend on manually designed prompts to optimize llms' domain performance @xcite @xcite , without fully considering the differences between human reasoning and model behavior. some recent studies have explored using llms' own errors as optimization signals @xcite , but their reward functions remain relatively simple. in this work, we propose a four-level reward function that guides llms to improve the generation of supporting questions by leveraging qa model outputs. this design aligns with qa models' inherent challenges, thereby enhancing llm performance. despite these advances, a persistent challenge across these approaches lies in balancing domain knowledge integration with computational efficiency. our eqd method addresses this trade-off by offering a lightweight yet effective question decomposition model. rather than relying on extensive domain fine-tuning or verbose prompting, eqd generates concise, supporting questions that enhance llm reasoning with minimal computational overhead. ## method we propose a two-step training approach for developing the expert question decomposition (eqd) model, as illustrated in figure 2 . the first step involves instruction fine-tuning to integrate domain knowledge into the model. unlike existing meth-ods that require extensive domain corpora for llm adaptation, we propose to use a small financial conversation dataset to develop a domain-specific question decomposition (qd) model. the second step uses reinforcement learning with proximal policy optimization (ppo; @xcite to align the qd model with the qa process, optimizing the effectiveness of generated decomposed questions (dqs) in enhancing qa performance. we introduce a novel answer comparison reward function for this end. ## results and discussions we evaluate our eqd method from four perspectives, presenting comprehensive experimental re-1 https://www.anthropic.com/news/claude-3-5sonnet sults to support our findings. in section 5.1, we compare the performance of various llms on four datasets, both in direct qa and with eqd support, demonstrating that our approach consistently enhances llm performance on domain-specific quantitative reasoning tasks. section 5.2 focuses on the two most challenging datasets, finqa and tat-qa, to compare different reasoning support methods, further validating the effectiveness of eqd. section 5.3 presents ablation studies, comparing our eqd model with other llms for question decomposition, and analyzing the impact of each finetuning step. finally, section 5.4 evaluates the computational efficiency of our eqd methods through inference time and generation length analysis. ## conclusions this paper introduces a novel two-step fine-tuning method, including domain fine-tuning and qa expert alignment, to develop an expert question decomposition (eqd) model. our eqd model demonstrates significant effectiveness and efficiency in supporting domain quantitative reasoning. experimental results across four financial datasets show consistent improvements in qa performance across various llms, maintaining computational efficiency comparable to zero-shot prompting methods. furthermore, our analysis reveals that a single critical supporting question is more beneficial to the domain qa process than detailed step-by-step guidance, providing novel insights into llms' reasoning capabilities in specialized domains.