F ine P rompt: Unveiling the Role of Finetuned Inductive Bias on Compositional Reasoning in GPT -4 (2023)

Abstract: Compositional reasoning across texts has been a long-standing challenge in natural language processing. With large language models like GPT-4 taking over the field, prompting techniques such as chain-of-thought (CoT) were proposed to unlock compositional, multi-step reasoning capabilities of LLMs. Despite their success, the prompts demand significant human effort to discover and validate them. Our work draws attention to the idea of transferring task-specific inductive biases from finetuned models to prompts, as a way of improving GPT-4’s compositional reasoning capabilities. To leverage these inductive biases, we formulate prompt templates to ease the transfer of inductive biases. The experimental results on multi-hop question answering and numerical reasoning over text show that our proposed prompt scheme shows competitive zero-shot and few-shot performances compared to existing prompts on complicated reasoning tasks, highlighting the importance of adopting the validated biases of the previous paradigm.

## introduction large language models (llm) such as @xcite have demonstrated impressive capability to solve textual understanding problems at a level parallel to or surpassing state-of-the-art taskspecific models @xcite . however, one of the characteristic pitfalls of llms is that they exhibit poor zero-shot and few-shot performance in tasks such as multihop reasoning @xcite and numerical reasoning over text @xcite , both of which involve compositional, multistep reasoning across multiple referents in text. to overcome such limitation of llms, previous works proposed various elicitive prompting strategies such as chain-of-thought (cot) @xcite , self-ask @xcite and leastto-most prompting @xcite . these prompting techniques have effectively unlocked the compositional, multi-step reasoning capabilities of llms by generating step-by-step rationales or breaking down an end task into a series of subproblems. regardless of their efficacy in improving llm reasoning, these prompting techniques still entail (i) significant amount of human effort to discover the right prompting strategy, and (ii) lack task specificity that takes into account the characteristic differences between end tasks. prior to llms and prompt learning, many taskspecific finetuned lms proposed a novel set of inductive biases to improve the compositional reasoning capabilities of finetuned lms @xcite @xcite @xcite @xcite on tasks like multi-hop question answering (mhqa) @xcite @xcite and numerical reasoning over text @xcite . for example, numnet @xcite injected the strict inequality inductive bias into lms to significantly improve its performance on drop @xcite , while de-comprc @xcite divided the multi-hop (a) attribute-infused prompt you will be given a document preceded by "document:" and a question … attributes that can be useful for the given task (b) pipeline-infused prompt you will be given a set of evidence documents, a multi-hop question … (c) graph-infused prompt you will be given a set of evidence documents, a multi-hop question … question: which team scored more points, texans or eagles? answer: texans the sentences are prefixed with paragraph and sentence numbers. the prefixes can connect sentences. there are three connection types: 1) "question": … 2) "intra": … 3) "inter": … nodes question type: bridging answer: clifton college, london numbers have specific relationships ... where the "<" symbol represents a is less than b, the ">" symbol represents a is greater than b … document: .. and their 45 points were the most in franchise history until week 4 of the 2017 season, when they again defeated the titans 57-14.. ## fineprompt we propose to transfer validated finetuned features into prompts (hence, the name fineprompt) to investigate whether (i) the finetuned features, in forms of prompts, have the same effect of improving the performance of gpt-4 on textual compositional reasoning tasks, and (ii) how the various models/approaches can be effectively transferred to structured prompts. to transfer the features into prompts, we divide models by their properties from sections 2.1 to 2.3 as shown in figure 2 . in each section, we describe which characteristic of a finetuned model aligns with one of the three prompt-infusion strategies; attribute-infused ( §2.1), pipeline-infused ( §2.2) and graph-infused prompts ( §2.3). note that inductive biases of finetuned models can be manifested in various forms, because they can be derived from one of the following strategies: (i) integrating additional, taskrelated features into the models through training (e.g., learning to solve basic arithmetic problems prior to solving complex textual numeric problems @xcite ), (ii) formulating a pipelined process that decomposes a complex reasoning task into a sequential set of sub-problems, and (iii) incorporating external graph structures to leverage the connected, structural inductive bias. our work, which aims to transfer the central inductive biases into prompts, directly adopts the previous works @xcite @xcite to minimize the human effort of extracting the features previously leveraged in these models. for example, as shown in figure 2 , while we have to manually construct the task-specific instruction and finetuned instruction chunks in the prompts, we can simply adopt the code bases of previous models to extract the necessary features used in in-context samples. ## related works task-specific models on tasks such as musique @xcite and drop @xcite , numerous models have been proposed to enable multi-step, compositional reasoning @xcite @xcite @xcite @xcite . these models, prior to the llm prompt learning paradigm took their place, proposed various novel ideas on this matter. prompt learning with prompting techniques like cot @xcite , self-ask @xcite and least-to-most prompting @xcite having shown to improve llms' compositional reasoning ability, our work explores how these prompts compare against the fineprompt scheme. we do not deal with least-to-most prompt-ing as it does not deal with compositional reasoning in a textual understanding setting. ## conclusion this work studies the transfer of validated inductive biases from finetuned models to prompts and their effectiveness on compositional reasoning of gpt-4. our empirical results suggest that (i) end task-related attributes and graphs help elicit robust multi-step reasoning capability from llms, and (ii) previous finetuned model pipelines, if they involve decomposing a task into a smaller sub-problems, are also effective for prompting. our work suggests that more can be exploited from the previous pretrain-then-finetuned models and our proposed template can incorporate those features seemlessly into llm prompting paradigm. we hope future works can explore further in this direction and easily leverage the power of llms with fineprompt.