Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models (2024)

Abstract: Large language models (LLMs) can achieve impressive performance on various reasoning tasks by incorporating chain-of-thought (CoT) prompting, where step-by-step reasoning is provided to guide LLMs to generate answers to questions, and the question-rationale-answer triplets are utilized as demonstration exemplars. However, the reasoning chains of demonstrations generated by LLMs are observed to be prone to errors, which can subsequently lead to incorrect reasoning during inference. Furthermore, inappropriate exemplars, e.g., overly simplistic or complex exemplars depending on the question’s difficulty level, can affect the LLM’s performance. To address these issues, we introduce Iter-CoT (Iterative bootstrapping in Chain-of-Thoughts prompting). Iter-CoT has two advantages: (1) it adopts iterative bootstrapping that enables LLMs to rectify errors autonomously, resulting in more precise and comprehensive reasoning chains. (2) it selects exemplars of challenging yet answerable (i.e., the LLM has the potential to answer correctly) questions, enhancing the LLMs’ generalizability to answer questions with varying difficulty levels. Experimental results exhibit Iter-CoT superior performance on three distinct reasoning tasks on ten datasets.

## introduction chain-of-thought (cot) @xcite prompting is a technique to enhance the reasoning abilities of large language models (llms) by generating a series of reasoning steps to obtain the answer, and the reasoning chains are utilized as exemplars to demonstrate the task and provide in-context learning (icl) @xcite to llms. recently, llms @xcite ; thoppilan et @xcite @xcite with cot prompting have demonstrated remarkable performance in complex reasoning tasks, including arithmetic @xcite @xcite @xcite @xcite @xcite , and symbolic reasoning @xcite . existing studies on cot prompting can be classified into two categories. the first category is manually constructed cot prompting( @xcite @xcite @xcite @xcite ), where human annotators manually craft a collection of question-rationale-answer exemplars to guide the model's reasoning process. however, human annotations' inherent subjectivity and limitations make these approaches costly, sub-optimal, and highly inconsistent. the second category is automatically generated cot prompting( @xcite @xcite ), where llm-generated cot is utilized. in practice, reasoning chains generated by llms have demonstrated superior performance compared with human annotations. however, three issues remain under-explored in the literature. (1) difficulty of questions. it is regarded that questions of mediate difficulty level can best guide the llms @xcite . as shown in figure 1 , it is observed that simplistic examples (simple-cot) perform poorly in solving complex questions of more hops 1 , while excessively complex exemplars @xcite ) are unsatisfying on simpler questions. (2) correctness of demonstration. reasoning chains of demonstrations generated by llms are prone to errors @xcite , which can significantly reduce overall performance. as shown in figure 2 , accuracy on various datasets decreases as incorrect exemplars increase. (3) missing contextual information. previous works merely combine the question and the "let's think step by step" prompt @xcite during the generation of demonstrations without incorporating contextual information (such as incorrect reasoning chains and feedback answers generated by llms). missing contextual information limits the llm's capability to learn from previous reasoning errors and avoid making similar errors. in order to address the issues above, we propose iter-cot (iterative bootstrapping in chainof-thoughts prompting). iter-cot allows llms to self-correct and summarize the more precise and comprehensive reasoning chains, which identify challenging yet answerable (i.e., llm has the potential to answer correctly) questions as demonstrations in order to enhance the llms' generalizability to answer questions with varying difficulty levels. we evaluate iter-cot on three distinct rea-1 following @xcite , we measure the question's difficulty by the number of hops in the rationale, with fewer hops indicating simpler questions and more hops indicating more complex questions. our contributions are summarized as follows: (1) we propose a new paradigm for cot, which generates precise and comprehensive reasoning chains during interaction with llms. to our knowledge, our work is the first to illustrate the importance of iterative interaction with the llms to generate high-quality demonstrations. (2) we propose iter-cot, an approach that generates self-corrected and summarized reasoning chains on exemplars with intermediate difficulty levels, which are utilized as demonstrations to enhance the llms' performance. (3) we implement iter-cot under both labeled and unlabeled conditions, achieving stateof-the-art (sota) results in both scenarios across ten datasets within three distinct tasks. ## motivation we propose iter-cot, which enhances llms' reasoning performance by integrating iterative bootstrapping to self-correct the reasoning chains in demonstrations. ## conclusion this paper proposes iter-cot, an iterative bootstrapping in chain-of-thoughts prompting for llm reasoning. unlike previous work, our method prompts llms to self-correct their errors in reasoning chains by leveraging iterative bootstrapping and obtaining more precise and comprehensive reasoning chains. experimental results on ten reasoning datasets among three different tasks demonstrate that our approach significantly outperforms the previous methods, achieving new state-of-the-art.