Fine-tuning Smaller Language Models for Question Answering over Financial Documents (2024)

Abstract: Recent research has shown that smaller language models can acquire substantial reasoning abilities when fine-tuned with reasoning exemplars crafted by a significantly larger teacher model. We explore this paradigm for the financial domain, focusing on the challenge of answering questions that require multi-hop numerical reasoning over financial texts. We assess the performance of several smaller models that have been fine-tuned to generate programs that encode the required financial reasoning and calculations. Our findings demonstrate that these fine-tuned smaller models approach the performance of the teacher model.To provide a granular analysis of model performance, we propose an approach to investigate the specific student model capabilities that are enhanced by fine-tuning. Our empirical analysis indicates that fine-tuning refines the student models ability to express and apply the required financial concepts along with adapting the entity extraction for the specific data format. In addition, we hypothesize and demonstrate that comparable financial reasoning capability can be induced using relatively smaller datasets.

## introduction in recent years, the development of large language models (llms) has achieved significant advances in natural language processing (nlp). scaling up the size of these models has not only improved sampling efficiency and performance, @xcite but also introduced reasoning capabilities @xcite . llms have been shown to perform well on tasks requiring reasoning capabilities in various domains, including code writing @xcite , math problem solving @xcite , dialogue @xcite , common sense reasoning @xcite and symbolic reasoning @xcite . a major drawback of these methods is their reliance on extremely large models with hundreds of billions of parameters. these models can be costly to deploy at scale due to high computational requirements and inference costs. to overcome these limitations, recent research has focused on inducing reasoning in smaller models. a common approach is to use large models to generate training samples with demonstrations of reasoning on one or more tasks that are then used to fine tune smaller models @xcite . while these methods have shown promising results on various tasks including arithmetic, symbolic and common-sense reasoning, the applicability, and effectiveness of these methods in specific domains such as finance need further exploration. question answering in the finance domain poses a unique set of challenges, requiring the understanding of financial concepts along with the ability to perform numerical reasoning. this complexity introduces a significant challenge that is distinct from classical question answering problems @xcite in this paper, we present an empirical study that provides experimental evidence supporting the effectiveness of fine-tuning small language models for financial question answering. our research is guided by several critical questions: rq1: to what degree does fine-tuning small language models improve their performance on financial question answering tasks? rq2: what are the intrinsic characteristics of the base language model that contribute to its performance prior to fine-tuning? rq3: which aspects of question answering benefit directly from the pre-trained knowledge, and what specific improvements are enabled by finetuning? rq4: what are the fine-tuning data requirements to achieve these improvements? to address these questions, we adapt previous ap-proaches on inducing reasoning in smaller models for the financial question answering task. in our experimental setup, we employ gpt-4 as the teacher model, building upon its documented success in the realm of financial question answering @xcite . for the student models, we explore a suite of state-of-the-art, yet relatively smaller, language models including phi-3 variants (3.5b and 14b parameters) @xcite , mistral 7b, and orca-2 configurations (7b and 13b parameters) @xcite . our methodology involves training the student model using python programs generated by the teacher model. the teacher generated code systematically delineates the steps for financial reasoning, including question comprehension, formula identification and entity extraction. at inference time, the language models are tasked with producing python code, which is then executed by an external interpreter. this code generation and external execution strategy has been demonstrated to be more effective than methods that rely on the language model to internally perform the calculations. @xcite @xcite . our main contributions are summarized as follows: combining gpt-4's automated assessment with manual evaluation yields new insights into the distinct competencies that are enhanced in the student model during fine-tuning. our experimental findings reveal that smaller language models fine-tuned for financial reasoning, can achieve performance that rivals that of the larger teacher model. moreover, our empirical analysis suggests that the fine-tuning helps refine concept understanding and enables consistent reasoning with those concepts. in addition, the entity extraction abilities get honed on the specific format of the financial dataset. ## background pre-trained large language models are shown to perform well on tasks requiring reasoning when used with certain techniques. @xcite proposed prompting the llms to solve the problem step-by-step by providing a few exemplars. @xcite proposed a few-shot prompt to produce a program which is then executed externally. however, methods relying on pre-trained llms can be costly to deploy at scale. recent efforts attempt to replicate these reasoning capabilities in small language models. one of the common approaches is following a teacherstudent setup where a pre-trained llm acts as a teacher, generating training data which is used to teach a small language model, the student. @xcite aim to train models to exhibit generic reasoning abilities. they utilize gpt-3.5 turbo and gpt-4 as teacher models to generate training data with carefully crafted prompts. on the other hand, @xcite @xcite train task specific small language models with cot based explanation from pre-trained llms. specifically for problems involving mathematical reasoning, @xcite @xcite propose to generate programs from the pre-trained llms and train the small language models. in contrast, we focus on fine-tuning small language models for financial question answering problems @xcite @xcite . solving these problems requires financial domain knowledge and complex reasoning compared to the math word problems addressed in previous studies. prior works have studied the use of pre-trained llms for financial question answering. @xcite perform a detailed comparison of the performance of pre-trained llms on financial datasets with various prompting techniques. they also introduce a novel prompting technique optimized for semi-structured documents. @xcite developed zero-shot prompt templates for question answering over financial documents that guide the llms to encode reasoning into a python program. these efforts do not consider fine-tuning to specialize language models for the financial domain. @xcite explore task-wise integration of external tools, calculator and sql engine, with fine-tuned small language models for tabular data analysis in finance. however, their approach utilizes predetermined prompt templates for data generation rather than a teacherstudent approach for fine-tuning. in our approach, we generate structured python programs using pretrained llms and train small language models with supervised fine-tuning. furthermore, we examine in detail the nature of the alignment achieved by the fine-tuning for the financial question answering, an area that is largely unexplored in previous research. ## fine-tuning for financial question answering we adopt the approach of using very large-scale models as reasoning teachers, and fine-tuning relatively small-scale student models from the promptcompletion pairs generated using the teacher model @xcite . specifically, the large model is used to generate python code with comments that encapsulates the reasoning required to answer the financial question. furthermore, the programs are generated with a specific structure that facilitates subsequent performance analysis of the fine-tuned model, as discussed in detail in sec. 5. the fine-tuning task is performed in three steps as shown in figure 1 . ## performance analysis we examine the evolution of model capabilities during fine-tuning, seeking to identify specific model enhancements. to this end, we define, and measure three key capabilities (1) concept understanding measured by the ability to correctly identify the required calculation (2) entity extraction measured by the ability to extract all required entities and (3) generation of executable code. our method relies on comparing the output of the student models with that of the teacher generated codes. therefore, we remove all samples with incorrect teacher generated codes from further analysis. ## conclusion we explored the performance of small language models fine-tuned for financial question answering, using exemplars generated by a very large teacher model. the small models achieved accuracy com-parable to that of the teacher model, driven primarily by improved ability to apply financial concepts as well as entity extraction. we showed that smaller datasets can yield similar results, suggesting that small language models can be efficiently fine-tuned for complex financial domain problems.