INFORM : Information e N tropy based multi-step reasoning FOR large language Models (2023)

Abstract: Large language models (LLMs) have demonstrated exceptional performance in reasoning tasks with dedicated Chain-of-Thought (CoT) prompts. Further enhancing CoT prompts with exquisite exemplars can significantly improve reasoning performance.However, the effectiveness of CoT prompts may fluctuate dramatically with different choices of in-context examples. Additionally, manual construction of rationale steps can be time-consuming, presenting challenges for the widespread adoption of CoT prompting. In this work, we propose a novel approach by introducing information entropy (IE) as a criteria on for CoT prompt selection. We extend this criterion to the CoT generation and inference stages, automatically generating CoT prompts with higher information entropy scores and adaptively determining the number of samples. These three stages together form our proposed information- entropy-based multi-step reasoning for large language models, named INFORM. Our experiments across seven reasoning benchmarks utilizing two language models(GPT-3.5-Turbo and text-davinci-003) demonstrate the superiority of INFORM both in performance and efficiency.

## introduction large language models (llms) @xcite @xcite @xcite have achieved great success in recent years. these models are commonly employed through in-context learning @xcite , where instructions and exemplars are provided to enhance language understanding and generation. however, the widely-used in-context learning methods might perform poorly for complex reasoning tasks @xcite . recent studies @xcite have highlighted the importance of elaborating the reasoning steps in exemplars, leading to the emergence of chainof-thought (cot) prompting. cot has shown promising results in improving the reasoning abilities of llms. various strategies like self-notes @xcite , progressive-hint prompting @xcite and least-to-most prompting @xcite , have been proposed to enhance cot further. while these works have focused on leveraging the inherent capabilities of models to augment the original prompts, recent research has shown that the performance of cot heavily relies on the choice of exemplars @xcite @xcite @xcite . this has led to investigations into identifying exemplars that maximize llms' reasoning abilities. notably, complexity-based prompting @xcite has introduced a selection strategy based on the number of rationale steps in annotated data. they have observed that more complex cot prompts effectively stimulate llms' multi-step reasoning capabilities. however, this approach is limited by its dependency on carefully annotated data. in the absence of annotations, it degrades to a querylenbased strategy, which is more fragile to input noise. this limitation highlights a fundamental drawback of cot prompting, i.e., the heavy reliance on human engineering and the scarcity of annotated datasets, which are time-consuming to create. to address this, previous works such as zero-shotcot @xcite and auto-cot @xcite attempted to alleviate the reliance on human effort in constructing cot prompts. however, they faced challenges such as low-quality generation and high computational cost. to overcome the aforementioned issues, we proposed a comprehensive cot prompting process that bypasses the need for extensive human effort while improving the effectiveness of cot. our first and foremost objective is to identify a pre-selection strategy that achieves comparable effectiveness to complexity-based prompting while remaining applicable in all scenarios. through extensive experiments, we have discovered that information entropy can serve as an effective and accurate criterion. figure 1 depicts the similar trends between performance and the number of rationale steps and information entropy scores of queries, respectively, on the gsm8k dataset using gpt-3.5-turbo and text-davinci-003. we consider information entropy to be a suitable criterion for several reasons: (1) information entropy can be applied to queries, making it applicable to all datasets. (2) the information entropy score of a query provides insights into its complexity, which is related to the information contained in the annotated or generated prompt. (3) information entropy is well-suited for both the generation and inference stages, enabling a coherent and unified framework, namely inform. our experiments further validate the effectiveness of information entropy. besides, we combined our information entropy criteria with self-consistency @xcite , extending our strategy from the input space (question selection) to the output space (cot generation and ie inference). in a nutshell, the contributions of our work are: ## inform the overall schematic of our proposed inform are illustrated in figure 2 . our approach is developed based on information entropy criteria, which we integrate into three different stages of standard question: the man had spent all day collecting old appliance…… answer: let's think step by step. ## analysis we conducted additional experiments to assess the effectiveness of inform and analyze the contributions of each component of inform. due to the high cost associated with using the text-davinci-003 api, we primarily utilized gpt-3.5-turbo for the additional experiments. ## conclusion this paper presents inform, a novel framework for cot prompting that consists of a comprehensive process encompassing query selection, cot generation, and ie-sc inference. inform is designed to be adaptable to various datasets and models, effectively improving the performance of cot prompts and mitigating the requirement for additional human involvement. our experimental results demonstrate the efficacy and versatility of inform, showcasing its ability to significantly enhance cot performance in various scenarios.