Let’s Be Self-generated via Step by Step: A Curriculum Learning Approach to Automated Reasoning with Large Language Models (2025)

Abstract: While Chain of Thought (CoT) prompting approaches have significantly consolidated the reasoning capabilities of large language models (LLMs), they still face limitations that require extensive human effort or have performance needs to be improved. Existing endeavors have focused on bridging these gaps; however, these approaches either hinge on external data and cannot completely eliminate manual effort, or they fall short in effectively directing LLMs to generate high-quality exemplary prompts. To address the said pitfalls, we propose a novel prompt approach for automatic reasoning named LBS3 , inspired by curriculum learning which better reflects human learning habits. Specifically, LBS3 initially steers LLMs to recall easy-to-hard proxy queries that are pertinent to the target query. Following this, it invokes a progressive strategy that utilizes exemplary prompts stemmed from easy-proxy queries to direct LLMs in solving hard-proxy queries, enabling the high-quality of the proxy solutions. Finally, our extensive experiments in various reasoning-intensive tasks with varying open- and closed-source LLMs show that LBS3 achieves strongly competitive performance compared to the SOTA baselines.

## introduction with super-sized training corpora and computational cluster resources, large language models (llms) have demonstrated emergent capabilities, thus enabling state-of-the-art performance in a wide range of natural language tasks @xcite @xcite @xcite ; wei * corresponding author † corresponding author @xcite . however, directly applying llms to complex reasoning tasks (e.g., mathematical reasoning, commonsense reasoning, etc.) in a naive manner presents significant challenges @xcite @xcite . for instance, the performance may be inadequate when simply feeding queries or using few-shot query-answer pairs in incontext learning (icl) approaches for these kinds of tasks. recent studies have shed light on that prompting llms to generate multiple reasoning steps (i.e., rationale) can markedly enhance their ability to reason, resulting in the development of the chain-of-thought (cot) prompting @xcite @xcite @xcite @xcite @xcite . current cot prompting approaches fall broadly into two categories, namely few-shot cot (fs-cot) @xcite and zero-shot cot(zs-cot) @xcite . among them, fs-cot involves providing llms with few task-specific context exemplars of query-rationale-answer triplets tied to the target query to prompt the generation of reasoning steps; zs-cot instead stimulates llms' reasoning capabilities by furnishing general trigger instructions (such as "let's think step by step") appended to the target query. despite their considerable success, obstacles persist in the field of prompt engineering research that plague real-world applications. fs-cot, for example, delivers well-crafted exemplary prompts but at the cost of labor-intensive manual annotations. to mitigate this, some efforts have been made to enhance the quality of exemplary prompts by retrieving the most relevant, complex and diverse existing queries or exemplars for the target task, which is achieved by tapping into external sources related to the task at hand, such as datasets or corpora, and employing various pre-defined similarity figure 1 : the illustrative comparison between lbs3 and existing representative approaches (including self-icl, auto-icl and analogical prompting) regarding proxy queries generated on top of qwen1.5-14b-chat. given a mathematical query, i.e., "q: lee used • • • ", self-icl, auto-icl and analogical prompting purely explore new, diverse and relevant proxy queries. in contrast, lbs3 investigates that from easy to hard. note that the implementation of analogical prompting follows the original paper, and we break down the results for ease of illustration. to overcome the aforementioned issues, recent initiatives (e.g., self-icl @xcite , auto-icl @xcite and analogical prompting @xcite ) work on bootstrapping llms to self-generate few-shot new proxy queries that are relevant and diverse to the target query, along with constructing corresponding exemplary prompts of triplets, thereby augmenting their capabilities to tackle reasoning tasks. essentially, these methods draw parallels with the concept of analogical reasoning in psychology, where individuals rely on past related experiences to approach new problems @xcite . the underlying insight behind them is that pre-trained llms (such as llama3, gpt-3.5 and gpt-4.0, etc.) have amassed a wealth of knowledge that equips them to fulfill various reasoning tasks. however, we observed that merely prompting llms to recall experiences related to the target queries may lead to the generation of proxy queries that are as difficult as the target queries themselves, along with corresponding incorrect proxy solutions, misguiding the resolution of the target queries, as exemplified in fig. 1 . see the related works in appendix a for more details. the issues mentioned above motivate us to propose a novel automatic reasoning prompt approach, coined lbs3, which is inspired by curriculum learning that mirrors the progressive nature of human learning styles. the idea of curriculum learning @xcite has been widely applied in the field of artificial intelligence, emulating the human learning process from easy to hard tasks @xcite @xcite @xcite @xcite . thus, lbs3 critically aims to 1) guide llms to generate easy-(or hard-) proxy queries related to the target query and 2) enhance the effectiveness of the {[𝑞 𝑖 ]} 𝑖=1 ## preliminaries in this paper, we work on scenarios wherein we address a given query (e.g., a math problem, multiplechoice question, or true/false assessment, etc.) without any additional information. given a query @xcite , the objective is to produce a solution consisting of the rationale (i.e., multiple reasoning steps) and the final answer, denoted by @xcite . a prompt template, represented by t , is designed for solving @xcite . note that multiple sub-prompt templates are assembled to form pipeline templates in certain specific prompt approaches. a prevalent prompting approach aims to integrate t with @xcite , resulting in @xmath0, [q]), which is then fed to an llm to elicit the corresponding solution [r, a] = llm (h). listed below are the existing prompting approaches that are most pertinent to our work. of note, the one-pass generation mode employed in this approach necessitates that the llm possesses robust capabilities for 1 https://openai.com/api/ both following instructions and generating responses. notably, compared to analogical prompting, self-icl/auto-icl, despite incurring additional computation and costs due to multiple interactions with the llm, offer greater flexibility and more closely mirror human cognitive processes. also, they place more modest demands on the llm's ability to follow instructions and generate responses. we aim to tailor a prompt approach that enables an llm to self-generate high-quality proxy exemplars, improving the accuracy of the solution it produces for a given query @xcite . ## approach in this section, we elaborate on our approach, lbs3, which draws inspiration from the concept of curriculum learning. lbs3 empowers an llm to self-generate few-shot exemplars that are pertinent to the target query (a.k.a. problem), ranging from simple to complex. figure 2 illustrates the reasoning pipeline of lbs3 in a modular fashion, which contains three key modules: the simple problem generation (spg) module, the analogous problem generation (apg) module and the rationale and answer generation (rag) module. remarkably, the rag consists of two sub-modules: one that solves the given query using the zs-cot manner and the other utilizing the fs-cot manner, denoted as rag-z and rag-f, respectively. thereafter, we introduce lbs3 from two perspectives: firstly, how it bootstraps an llm to generate proxy queries related to the given query in increasing order of difficulty, and secondly, how it effectively addresses the more challenging proxy queries. ## conclusion in this paper, we introduce a novel automatic reasoning prompt approach, dubbed as lbs3, drawing inspiration from the concept of curriculum learning. concretely, lbs3 initially ushers llms to recall easy-to-hard proxy queries that are pertinent to the target query. following this, it implements a progressive strategy that utilizes exemplary prompts stemmed from easy-proxy queries to direct llms in solving hard-proxy queries, enabling the highquality of the proxy solutions. at last, we validate the effectiveness of lbs3 with extensive experiments on several state-of-the-art open-and closedsource llms and reasoning benchmarks.