Can Language Models Learn to Skip Steps?

Authors: tengxiao  liu
qipeng  guo
xiangkun  hu
♢  cheng
yue  zhang
xipeng  qiu
zheng  zhang
♢ ♠ fudan
santa  barbara
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 11:40:30

Abstract: Trained on vast corpora of human language, language models demonstrate emergent human-like reasoning abilities. Yet they are still far from true intelligence, which opens up intriguing opportunities to explore the parallels of humans and model behaviors. In this work, we study the ability to skip steps in reasoning—a hallmark of human expertise developed through practice. Unlike humans, who may skip steps to enhance efficiency or to reduce cognitive load, models do not inherently possess such motivations to minimize reasoning steps. To address this, we introduce a controlled framework that stimulates step-skipping behavior by iteratively refining models to generate shorter and accurate reasoning paths. Empirical results indicate that models can develop the step skipping ability under our guidance. Moreover, after fine-tuning on expanded datasets that include both complete and skipped reasoning sequences, the models can not only resolve tasks with increased efficiency without sacrificing accuracy, but also exhibit comparable and even enhanced generalization capabilities in out-of-domain scenarios. Our work presents the first exploration into human-like step-skipping ability and provides fresh perspectives on how such cognitive abilities can benefit AI models.

Main paper: 
 introduction: 
 The pursuit of Artificial General Intelligence (AGI) is profoundly influenced and inspired by human intelligence [35, 6] . Trained extensively on human language, language models not only excel in various tasks, but also begin to exhibit emergent human-like abilities that are not explicitly engineered into them [24] . Among these, reasoning stands out as a core human-like cognitive ability, and has demonstrated great potential in a wide range of problem solving scenarios [47, 11, 30, 37, 28, 34] . Despite their advances in displaying human-like cognitive activities, huge gaps remain in how models and humans actually behave [22, 46, 20] . These differences bring up interesting questions regarding the exploration and development of similar capabilities between models and humans.We aim to investigate whether the models exhibit any reasoning abilities unique to human experts, and whether they can evolve from beginners to reasoning experts. When humans learn to reason, beginners typically start with detailed, step-by-step solutions to imitate the gradual process of problem solving. As practice makes perfect, human experts not only solve problems more swiftly but also utilize shorter mental pathways, often skipping steps in their reasoning process [36] . This particular ability helps them speed up the reasoning and saves cognitive load for more challenging steps [44] . As demonstrated in Figure 1 , the step-skipping behavior illustrated on the right side is commonly adopted by human experts during equation simplification. Step skipping in equation simplification. We use the specified number of steps in the input as a stimulation to induce the model to perform skipping by using fewer steps.In this work, we are curious whether models exhibit mature human-like reasoning ability -skipping steps, and how such abilities can influence the model's reasoning behaviors. Unlike humans, models do not inherently possess the intrinsic motivation like time limit or skill maturity that naturally drives efficiency in cognitive tasks. To induce the skipping step behavior in models, we introduce a controlled training environment where models are instructed to generate reasoning sequences within a specified number of steps. Our method includes two phases: initialization and iteration. We begin with a dataset that contains complete stepwise reasoning processes for the questions. In initialization, models are first trained to solve the tasks comprehensively, adhering to the full sequence of reasoning steps. In Figure 1 , the illustration on the left demonstrates how models are trained to follow a specified number of steps. Then in the iteration phase, the models are prompted to produce shorter answers based on the original training data (Figure 1 right ). We then select the shorter reasoning paths that still achieve correct answers and mix them with the full-step reasoning paths. This expanded dataset is used to train a new model to have advanced step-skipping capabilities. Each iteration refines the model's ability to identify how steps can be skipped without sacrificing accuracy. Finally, we fine-tune the models using these iteratively generated datasets, including data instances that demonstrate successful step-skipping during each iteration.We conduct experiments with three different reasoning datasets, each characterized by clear internal reasoning steps, to evaluate model behaviors. Empirical results demonstrate that models exhibit and develop the ability of skipping steps in our framework -not only solving tasks effectively but also actively omitting steps to enhance efficiency. Further analysis of model behaviors indicate that these skipped reasoning paths act as beneficial enhancements rather than mere biased shortcuts, as evidenced by their maintenance or even improvement of out-of-distribution (OOD) performance across various tasks. To the best of our knowledge, this work is the first investigation into the humanlike ability of step-skipping in language models, providing empirical evidence that models can indeed skip steps. These preliminary findings provide a fresh perspective on easy-to-hard generalizationtraining models on simpler data comprising both comprehensive and skipped reasoning steps can enhance their ability to generalize to more complex scenarios. ¶  
related work: 
 Human-like Abilities in Language Models Many of the capabilities widely used in current models are inspired by human intelligence. For instance, in-context learning enables models to address problems by mimicking the patterns demonstrated in examples [5] . In reasoning tasks, models benefit from progressively answer derivations and step-by-step chain-of-thought processes [47] and their humanlike enhancements, such as planning [18] , task decomposition [50] , and refinement [32, 38] . Another series of studies explore from the perspectives of cognitive science and psychology [10, 2, 12, 9] . Kosinski [24] reveal that current large language models have demonstrated a certain level of Theory-of-Mind (ToM) abilities by testing their performance to impute another's mental states and perspectives. Further studies [21] provide preliminary evidence of a correlation between the embeddings in LLMs and human brain neurons during ToM tasks, while Ma et al. [31] highlights the limitation of current ToM evaluations as they target narrow and inadequate aspects of ToM. Apart from these cognitive abilities, our work draws inspiration from human problem solving [23, 42, 3, 44] and evaluates language models on these unique step skipping behaviors. Additionally, our work aligns with an expanding field exploring the correlation between System 1 and System 2 reasoning mechanisms [14, 15, 49] . Rather than removing all reasoning trajectories, our work explores gradual shortening to provide a smoother transition that mirrors natural cognitive processing.𝐷 !"!# 𝑀 $ 𝐷 % ′ 𝐷 $ = 𝐷 % + 𝐷 % ′ 𝑀 & 𝐷 $ ′ 𝐷 & = 𝐷 % + 𝐷 $ ′ …… Solve itCompositional Generalization Challenges Transformers have shown limitations in complex compositional generalization scenarios [17, 39] . Previous work also indicates that models may develop biased shortcut, negatively impacting their OOD performance [27, 25, 16] . A growing body of research focuses on easy-to-hard generalization [4, 7, 19, 41, 48] , where models improve their generalization ability by learning from easy tasks, without requiring intensive supervision on harder ones. Following this line, our work encourages the model to learn from self generated skipping paths, which has been empirically shown to maintain and even enhance OOD generalization capabilities. 
method: 
 Humans develop the ability to skip steps for several reasons. With practice in specific tasks, they evolve from novices to experts, optimizing lengthy thought processes into quicker, more efficient reasoning. Additionally, factors such as time constraints or the desire to conserve cognitive resources can also prompt humans to skip steps [13] . In contrast, models lack an inherent cognitive signal that would drive them to minimize reasoning steps. Rather than attempting to replicate these human-like signals, we design a training approach to directly control the number of steps in their reasoning processes. By restricting the steps in model responses, we can guide the model to self-generate data including skipped steps. Our framework has two phases: initialization and iteration.We begin with a training dataset D 0 , which contains detailed full-step reasoning answers to the questions. Our goal is to train a model that can generate answers by following the specified number of steps in the input question. Depending on the characteristics of different tasks, there are two design choices to initialize the framework: cold start and warm start.In the cold start approach, we directly fine-tune the model on the original full-step training data, i.e., D init = D 0 . The trained model is expected to not only learn to solve the problems, but also adhere to the specified number of steps in the input instructions.Training exclusively with full steps does not always guarantee the ability of controlling the number of steps, especially for the challenging tasks. Therefore, we manually create answers that contain skipped steps based on human expertise. Optionally, we can also randomly merge adjacent steps or simply omit random steps within the rationales to create such skipped-step data. In either way, we can expand the original training set with additional data D skip that can better help models learn how to solve the problems with fewer steps. Thus, the data for warm start initialization can be describes asD init = D 0 + D skip .Using the prepared data, we fine-tune the model to generate the answers with the given number of steps. For each QA pair in D init , the question q is concatenated with the instruction I n which indicates that the reasoning process a (n) should be completed in n steps. Therefore, the resulting model in the initialization phase, M 0 , is described as:EQUATIONwhere the instruction I n stands for "Solve it in n steps".After the initialization, the model is expected to have learned to solve the problems with detailed steps using the specified number of steps in the input. Leveraging this particular ability, we can encourage the model to actively engage in step skipping behavior. At the beginning of each iteration k, the model M k-1 is prompted to solve the same problems in the training set using fewer steps than the full number. Responses that are both correct and meet the reduced step criterion are filtered and composed into a new dataset D ′ k . These reasoning answers are generated solely by the model itself, reflecting its understanding after training on the initialized data and demonstrating its active preferences when reducing steps.We define the dataset used for current iteration asD k = D 0 ∪D ′ k-1, where the original training set D 0 includes full reasoning steps and the filtered dataset D ′ k-1 contains new responses that successfully utilized fewer steps. This ensures that the model has access to both the original complete reasoning processes and examples of effective step-skipping generated by the model itself. To finalize current iteration, the model M k is trained onD k : M k = (q,a (n) )∈D k P (a (n) |q, I n ).The iterative training process described above requires specifying the number of steps in the input, which is impractical in real-world applications because it can be difficult to determine the exact number of steps needed for a given question. To be more applicable, we aim to understand how models learn from the generated skipped data and what benefits they can derive from it. Therefore, for each intermediate resulting dataset D k , we train a new model using a standard QA finetuning setting without specifying the number of steps in the input:M standard k = (q,a (n) )∈D k P (a (n) |q).(EQUATIONThis phase aims to solidify the model's skipping behavior, simulating a more advanced form of cognitive processing akin to human reasoning. 
datasets: 
 We design three tasks to investigate the model's step skipping behavior (Figure 3 ). In each task, the intermediate steps needed to solve these problems are explicitly detailed and well-defined, facilitating a clear analysis of the model's predictions. When creating skipped data for warm start, we either omit certain steps or heuristically merge two adjacent steps. Details on data creation can be found in Appendix B.1.Analog of Algebra Following Blessing and Anderson [3] , we create an analog of algebra by replacing the variables and operators with different symbols. As shown in Figure 3 , each variable and standard operator is mapped to a unique, unrelated symbol. The desired result is to isolate the symbol ♥ (i.e., x) on the left side of the symbol ↔ (i.e., =). This task is entirely new for the model, making it an ideal scenario to understand how models develop problem-solving abilities from scratch. We use a heuristic script to generate the questions along with the stepwise solutions. After generating the QA pairs, we filter the data based on the number of variables involved in the question and the steps required to solve it. The training and in-domain test data contains questions with up to 7 variables and requiring no more than 5 steps. In addition, we create two out-of-domain datasets of varying difficulties to evaluate generalization performance: OOD-easy includes variables unseen during training, with 8 and 9 variables, no limit on steps. OOD-hard is the most challenging setting, including 10 -14 variables and ≥ 9 steps to solve. Both OOD sets contain unseen variables.Q: ♋ ❀ ♙ ↔ ♏ ✾ ❤ (A -B = C * x) Answer: 1. ♏ ✾ ❤ ↔ ♋ ❀ ♙ (C * x = A -B) 2. ❤ ↔ ♋ ♪ ♏ ❀ ♙ ♪ ♏ (x = A / C -BMulti-digit Addition As a basic arithmetic task, multi-digit addition naturally involves detailed stepwise reasoning processes, serving as a suitable task for studying model behaviors in compositionality generalization [45, 26, 40] . We utilize step-by-step reasoning processes to perform addition operation digit by digit, as illustrated in Figure 3 . For training and in-domain test data, we only consider additions involving numbers with up to 3 digits. We introduce two out-of-domain datasets depending on the number of digits involved in the addition: OOD-easy includes one number with up to 3 digits and another with 4-7 digits. OOD-hard contains two numbers, both with 4-7 digits.Directional Reasoning We additionally consider long-form symbolic directional reasoning, which poses a challenge for direct solution and necessitates continuous reasoning steps to arrive at the answer. This task provides an initial direction and a list of turning actions. The desired answer is the final facing direction. For training and in-domain test set, we consider questions that contain ≤ 10 actions. OOD-easy includes questions with 11-20 actions and OOD-hard includes questions with 21-30 actions. The detailed statistics of three datasets can be found in Table 1 . For all our experiments, we use Llama 2 (7B parameters) [43] and phi-3-mini (3.8B parameters, with context length of 4K) [1] as our base model. We train the model using a learning rate of 5e-6 for 2 epochs with the AdamW optimizer [29] . During inference, we employ greedy decoding. We run our experiments with three different random seeds and report the average and standard deviation. All experiments are conducted on eight V100 GPUs each with 32GB memory. The total training time required to complete one full cycle of five iterations is under six hours. 
can models learn to skip steps?: 
 To make sure our framework can proceed to iterations smoothly, one crucial factor is the initialized model's ability to adhere to the specified number of steps in the input. In the cold start setting, we Step number following ability of the initialized Llama2 models across different tasks. "# Skipping" represents the number of instances where n -i > 0. "Step Consistency" quantifies the match between the actual number of steps taken and the number indicated in the input. "Answer Accuracy" calculates the percentage of correct final answers out of the "# Skipping" cases. "AverageStep" reflects the mean number of steps across all predictions within the dataset.Analogy of Algebra Multi-digit Addition Directional Reasoning As shown in Table 2 , the results demonstrate that the fine-tuned model exhibits good step-number following ability on the Analog of Algebra -over 99 % of the answers follow the given number of steps. Additionally, when prompted to generate condensed answers with fewer steps, the model can produce some correct answers in the specified number of steps, achieving accuracies of 8.14% and 2.77% respectively. Despite this relatively low accuracy, these small amount of correct data can still assist the model in gradually developing step skipping ability through iterations. Ultimately, the model manages to produce over 90% of correct skipping data. The trend of the data quantity change can be found in Appendix B.2.i = 1 i = 2 i = 1 i = 2 i = 1 i = 2 # Skipping 5,However, this ability varies across different tasks. For the other two tasks, models do not naturally develop the capability for active step skipping, leading to near zero step consistency when required to provide answers in fewer steps. To address this issue, we employ the warm start setting for these tasks. Table 2 presents the results of Multi-digit Addition and Directional Reasoning under the warm start setting, indicating that this approach enhances the models' proficiency with step skipping.Ideally, we aim for models to be initialized through cold start. The benefits of this approach are obvious -it allows the model to spontaneously develop step skipping behavior, giving it sufficient freedom to decide and control which steps to skip. However, our experiments have revealed that it can be challenging for models to develop such capability in all scenarios. In contrast, the warm start offers an alternative design choice by providing human-created skipped data. This data includes intuitive and valid skipping steps derived from human expertise, making it more natural and helping models develop human-understandable behaviors. However, it might also introduce human biases that constrain the model's independent exploration of step skipping. This influence can be mitigated in the subsequent iteration phase, where the model is given full freedom to develop and amplify its own step-skipping behavior.Based on this new mixed data including both complete and skipped answers at each iteration, we train the standard models to analyze the change of model's performance -what models can learn from the behavior of skipping steps.Models learn to solve problems more effectively with fewer steps. We evaluate the standard models on both in-domain and OOD data, with the results presented in Table 3 . Detailed results from each iteration of the evaluation can be found in Appendix B.3. Given the simplicity of the tasks, the model is able to overfit on in-domain data, achieving nearly perfect performance. Further iterations of skipping steps manage to guide the model to use fewer steps while maintaining the performance. In two OOD scenarios, we find that the model trained with mixed data performs comparably to the model trained with complete steps on the OOD test sets, and even exhibits superior generalization abilities. Specifically, in Analog of Algebra, Llama2 models of iteration 5 achieves 4.76% gain on OOD-easy, while phi-3-mini achieves 7.08% gain on OOD-hard set. In the Multi-digit Addition task, the Llama2 model demonstrates a 13.91% improvement in OOD-easy performance Models achieve near perfect performance on in-domain data but diverge on lengthy OOD data. and a 4.75% increase in OOD-hard performance. In the OOD-hard dataset for Directional Reasoning, Llama2's performance improvs by 9.2%. These results suggest that not only is the model unaffected by potential shortcut bias from the skipping steps, but it actually benefits from the mixed training data to gain enhanced task solving abilities. The ablation analysis on various data-mixing approaches are provided in Appendix B.5. Furthermore, we observe that the model uses fewer steps, thereby increasing problem-solving efficiency.Figure 4 (a) presents the performance of Llama2 models across various iterations in the Analog of Algebra task, differentiated by the number of steps required in the complete answers. The solid lines represent the accuracy of final answers. We perform uniform evaluation on the union of all in-domain and OOD test sets. Initially, all models maintain high accuracy for in-domain problems with up to five steps, after which a significant drop is observed as the complexity increases. As the model undergoes iterations, there is a noticeable improvement in its ability to handle longer step lengths (green solid line), particularly in the range of 6 to 10 steps where other models show significant weaknesses. The dashed lines illustrate the proportion of data exhibiting step-skipping in model predictions. The blue dashed line indicates models initially adopt step-skipping as problems extend in length. After iterations, the green dashed line indicates the models consistently employ step skipping in shorter questions, thereby improving the reasoning efficiency. We observe that the artificial skipped data has minimal impact on the model, with negligible differences between the cold start and warm start phases. Upon entering the iterative phase, the model's performance notably declines during the first iteration, particularly in handling longer problems. This downturn may reflect the model's adjustment from manually injected skipped data to its own step skipping ability. Subsequent iterations show that the model benefits more significantly from data generated during the iteration process, as evidenced by the results in Iteration 5. The model maintains consistency with the baseline in both in-domain and out-of-domain performances, and exhibits a slight advantage in solving longer problems. Similar to the previous task, the Iteration 5 Ratio curve (dashed green line) also shows a significant increase in step-skipping behavior, suggesting an evolved efficiency in reasoning as the model opts to bypass steps while maintaining or even improving accuracy.In Figure 5 , we show a finer-grained evaluation of multi-digit addition tasks on Llama2. The horizontal and vertical axes of the matrices represent the number of digits in the two addends for each question in the test set (both in-domain and OOD test data). We utilize the following three metrics: Question-level accuracy assesses whether the final answer is correct for additions involving different numbers of digits.Step-level distribution illustrates the distribution of the digit lengths used in each individual step of the model's stepwise reasoning process.Step-level accuracy measures the accuracy of the single step calculations involving different numbers of digits.In Figure 5 (a), as iterations progress, the model demonstrates improved generalization performance across all test datasets. When initialized with a cold start, the model can only learn from the training data involving single-digit addition steps, resulting in overfitting to in-domain test data (digit ≤ 3). When augmented with manually created skipped data for a warm start, the model begins to incorporate multi-digit additions with skipped steps. However, the inconsistency between the manually injected data and the model's inherent behavior does not significantly enhance the question-level accuracy.As the model is encouraged to explore during the iteration phase, it undertakes broader and bolder attempts-often combining additions across more digits in skipped steps. With the integration of these data, the model trained on this expanded iterative dataset also shows a more pronounced ability to solve OOD problems. As seen in Figure 5 (b), the model increasingly employs multi-digit additions in single-step operations. Furthermore, as illustrated in Figure 5 (c), there is an improvement in the accuracy of these skipped single-step operations. We believe this may be due to the model-generated data during self-iterations, which are more conducive to enhancing its capability to skip steps, thereby benefiting from this process.Figure 6 shows the step skipping behavior and accuracy of the standard models at each iteration on the Analog of Algebra task using Llama-2. The Skipping Ratio measures how often the model skips steps in the test set, while Accuracy reflects the correctness of these skipping answers.We observe that in the beginning models inherently struggle in OOD scenarios, often producing reasoning steps that are incomplete or shorter than the problem complexity requires. In "cold start" settings, where the model is trained solely with complete steps, it performs well with in-domain questions but fails to maintain complete reasoning steps and tends to generate shorter responses on OOD sets. Due to its limited generalizability, these skipping or missing steps negatively impact the performance. However, as the model progressively adapts to step skipping over iterations, the accuracy of the shorter responses improves, suggesting it gradually develops a more reliable ability to skip steps when appropriate. Analysis across all tasks can be found in Appendix B.4. Throughout the iterations, as the model progressively generates more successful step skipping data, the size and the quality of the resulting dataset also gradually increases. This can be considered as a special form of augmentation for answer diversity. To investigate whether the performance improvements are primarily due to the model learning from more training steps, we increase the number of training epochs during the initialization phase to match the data volume after iterations. The comparison results shown in Table 4 reveal that increasing the number of training epochs does not always lead to performance enhancements; instead, it may cause a performance decline due to overfitting. In contrast, mixing skip-step data from the iterative process not only maintains or improves performance in in-domain and OOD-easy tasks but also achieves consistent gains in OODhard setting. When the total number of training steps is similar, the integration of skipping data yields better performance. In this section, we extend the iterative process to allow the model to skip up to 4 steps, rather than restricting it to less than 2 steps on Analog of Algebra. The process is continued for a total of 9 iterations, and the results are shown in Figure 7 . The model continues to benefit from additional iterations beyond Iteration 5, which serves as the default cutoff in our main results. Specifically, the accuracy on the OOD-hard set improves steadily, reaching over 18% by the ninth iteration. This increase suggests that even with a greater allowance for step-skipping, the model's ability to generalize to harder out-of-domain samples is enhanced with continued training.Simultaneously, the average number of steps taken decreases across all test sets as iterations progress, suggesting that the model is converging towards fewer steps and becoming increasingly efficient. By the ninth iteration, the step count appears to plateau, indicating that the model has likely reached a stable balance between accuracy and efficiency. We hope our work provides a fresh perspective on exploring the connection between System 2 slow reasoning and System 1 fast thinking, and on facilitating their transformation, paving the way for future research in this direction.  
conclusion: 
 In this work, we explore the human-like ability of step skipping in language models, providing initial empirical evidence that models can skip steps and benefit from such cognitive behaviors. Addressing the absence of intrinsic motivation for step skipping in models, we design an approach that not only enables models to spontaneously develop the ability but also iteratively encourages models to actively adopt and enhance this behavior. Through experiments on three tasks, we demonstrate that models equipped with step-skipping capabilities can solve tasks more efficiently in fewer steps, without sacrificing accuracy. Further empirical results suggest that training on easy data containing both full steps and skipped reasoning steps can potentially help models generalize to harder scenarios. We hope this work offers insights into the relationship and transition between System 1 and System 2 thinking and contributes to advancing easy-to-hard generalization in language model reasoning.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: We describe all the details of datasets and implementations in Section 4.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: We report the average results and the standard deviation in our experiments 3.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA] Justification: Our work investigates the human-like behaviors of language models. There is no necessary societal impact of this work that should be specified.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Justification: We mainly evaluate model behaviors on simple and well developed reasoning tasks. This work does not pose such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We use Llama2 models: https://ai.meta.com/llama/license/ Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: Our work serves as a preliminary exploration of human-like step skipping capabilities in models, focusing solely on the expansion of problem types in terms of length and compositional complexity, without extending to advanced problem difficulty generalization. We also recognize that ideally there should be a clear criterion for determining when to terminate iterations. We observe that the model can also perform better in intermediate rounds, which suggests the need for further adjustment of this hyperparameter. Additionally, for the convenience in evaluation, our investigations were confined to three simple yet representative tasks. While our designed method can be applied to practical tasks, we leave the exploration of scalability to complex reasoning scenarios as future work.B.1 Details of data creationFor the Analog of Algebra task, we ensure the quality of the auto-generated dataset by creating full-step reasoning data using standard algebraic rules applied to operators. To further verify the validity and consistency of the intermediate steps, we utilize the SymPy [33] library. Specifically, we perform SymPy simplification for each intermediate step and ensure that the resulting equation remains algebraically equivalent to the final simplified answer.For the Multi-digit Addition task, the internal results are generated using Python's built-in calculation modules, ensuring accurate computations.For the Directional Reasoning task, the clarity of the question formulation guarantees that all intermediate steps are 100% correct. Each step is derived through rule-based decomposition, ensuring the correctness of the intermediate steps.We define several heuristic rules to create skipping data for warm start initialization. For the multidigit addition task, we randomly merge two single-digit addition steps to form a two-digit addition step. For the directional reasoning task, we incorporate more human expertise by skipping steps that involve two adjacent directions that result in no change. For example, adjacent actions such as "right-left", "left-right", and "around-around" will not alter the final facing direction, so we manually skip these steps. We only manually create one skipped step within a single data. From Figure 8 , the number of correct skipping data keeps increasing as the iterations progress. Higher accuracy results in more valid data involved in the mixed dataset. This iterative approach allows the model to gradually develop the step skipping ability and produce more valid data with fewer steps.Table 5 and Table 6 show the detailed performance of standard finetuned models from each iteration on Llama2-7B and phi-3-mini respectively. We report the average performance and the standard deviation across three runs with different random seeds.Analyzing the results from each iteration, we find that the final iteration does not consistently yield the best performance, highlighting the importance of identifying an optimal stopping point as a direction for future work. Additionally, significant fluctuations are observed in the test results, particularly in the OOD settings. Therefore, developing a more stable approach for OOD generalization tasks is another potential area for further exploration. Cold start vs. warm start In the Multi-digit Addition task, we observe that phi-3-mini achieves satisfactory results with cold start training alone, allowing the model to enter the iteration phase without relying on manually provided skipping data. Table 7 shows the model's performance when initialized with a cold start in Multi-digit Addition. Compared to the results in Table 6 , where the model begins with a warm start, the cold start approach enables the model to independently explore and develop its skipping behaviors. This leads to a more pronounced improvement in the OOD settings, with accuracy of 25.06% versus 14.98% in Iteration 5 on OOD-hard. Additionally, we observe that while warm start enables a more immediate reduction in steps, cold start shows a more gradual decrease in the number of steps taken.In this section, we provide the ratio and the accuracy of the skipping responses across three tasks using both base models. The results are shown in Figure 9 and Figure 10 . In general, the models demonstrate a progressively enhanced step skipping capability across various test settings for all tasks. In most cases, the model increasingly favors adopting more skipped reasoning steps over iterations, with the accuracy of skipped responses also improving correspondingly. However, we observe that the proportion of skipped responses fluctuates across different stages of iteration, rather than following a strictly monotonic trend. Given that the model autonomously decides whether to employ skipping, this pattern may indicate the model's attempt to find a balance between using step In this section, we analyze the role of data mixture in iterative training and its effect on the performance of standard models M standard . Specifically, we examine how the inclusion of both cold-start data and generated skipping data enhances the model's generalization ability and comprehension of complex reasoning paths. Table 8 presents an ablation study comparing different data mixing strategies with phi-3-mini model on the Analog of Algebra task. The "Skipping" setting utilizes only the generated skipping data D ′ k-1 for training the standard model M k , while "w/ Cold Start" incorporates both the original cold-start data and the skipping data, which serves as the default configuration in our experiments. The analysis is based on data from Iteration 5, and we report average performance across three runs with different random seeds. Our findings suggest that relying solely on skipping data may limit the model's capacity to address OOD scenarios. Although skipping data provides shorter average steps, it lacks the complete reasoning steps essential for a comprehensive understanding of the task, potentially leading the model to depend on shortcuts that harm generalization. By incorporating a mixture of cold-start and skipping data, the model is able to learn from both complete and skipped reasoning chains, which enables a more robust understanding, supporting stronger generalization capabilities.Table 9 : Cross-domain generalization of step-skipping capability in the phi-3-mini model. In the specified "Withheld Task" setting, step-skipping data is excluded from one specific task, while the "All" setting includes only full-step data across three tasks. Figure 10 : Skipping ratio and accuracy at each iteration on phi-3-mini. On Multi-digit Addition, we illustrate the analysis of the model that is initialized from Cold start.To investigate the cross-domain generalization of step-skipping capabilities, we conduct a controlled experiment to assess the impact of step-skipping training data from one task on the model's performance in others. Specifically, we sampled 2,000 training examples per dataset, including 1,600 step-skipping answers in which exactly one step was successfully skipped from these samples, all from Iteration 5. This setup ensures an equal balance of full-step and step-skipping data across all three tasks.We use the phi-3-mini model across three tasks, with the "withheld task" representing the task that lacks step-skipping data during training. The "All" setting contains only full-step answers for all tasks, with no step-skipping data included. The configurations are as follows:• All setting: task1-full + task2-full + task3-full• Withheld setting: task1-full + task1-skip + task2-full + task2-skip + task3-full Table 9 summarizes the model's performance on each evaluation task. The withheld task's results are compared to those from the "All" setting, where all tasks are trained with only full-step answers.Our findings reveal that step-skipping data in one or more tasks positively affects the performance of the withheld task. In most cases, models trained with step-skipping data from other tasks exhibit improved accuracy and step-skipping performance across datasets, maintaining a comparable number of steps to the "All" setting. For example, in the Analog of Algebra task, the average steps remain similar, yet accuracy improvements are observed in OOD settings, indicating that training with step-skipping data promotes a transferable ability to reason efficiently across domains. The overall accuracy increase suggests that inclusion of step-skipping data in some tasks enables the model to generalize this ability, even when explicit step-skipping examples are unavailable in the target task. These results suggest that the step-skipping capability learned in one domain can generalize across different tasks, underscoring the potential for enhancing model efficiency through strategic data composition.In addition to the synthetic datasets analyzed in the main body of the paper, we conduct experiments on GSM8K [8] to evaluate the applicability of our method to more complicated tasks. To create a controlled experimental setting, we classify data requiring no more than 4 steps in the annotated answers as in-domain data and the remaining as out-of-domain data. Table 10 provides an overview of the dataset splits. Full-step answer: 1. 0 + 1 (plus carry 0) = 1 (write down 1, no carry over). 2. 9 + 4 (plus carry 0) = 13 (write down 3, carry over 1).3. 3 + 6 (plus carry 1) = 10 (write down 0, carry over 1). 4. Carry the final carry over 1 to the next place.Thus, the answer is 1031.Skipped-step answer: 1. 9 + 1 (plus carry 0) = 10 (write down 0, carry over 1). 2. 90 + 64 (plus carry 1) = 155 (write down 55, carry over 1). 3. 3 + 0 (plus carry 1) = 4 (write down 4, no carry over).Thus, the answer is 4550. Justification: While currently we have not released our code and data, we will make them publicly available in online repository upon acceptance of this work.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.¶ Code and data are publicly available at: https://github.com/tengxiaoliu/LM_skip. 


Decision: In this work, the authors explore how to guide LLM's training so they can skip unnecessary reasoning steps like humans do. Concretely, they propose a framework that iteratively refining models to generate shorter reasoning paths, at every iteration, they keep the generated shorter reasoning paths that lead to the correct answers to fine-tune the model in the next iteration. By doing so, they show that on three dataset, a llama2-based model could solve tasks with less steps but not sacrificing accuracy. 

This work has many merits. All reviewers agree that the idea is interesting and intuitive, shortening reasoning steps could be very useful practically; the proposed method is technically sound; the paper is clearly written, the structure is well organized; the three tasks, despite synthetic, are well designed to show how and why the proposed method could work, the experimental results support well the authors' claim; the authors provide detailed analyses that helped readers to understand the training pipeline; the idea is generally well connected to the paradigm of easy-to-hard generalization.

On the negative side, reviewers' concerns were mainly around 1) clarity on some details; 2) the relatively simple datasets/tasks; and 3) the authors failed to demonstrate the method with more than one LLM backbones. 

To address the concerns, during the author-reviewer discussion period, the authors provided the following additional information:
- The authors added experimental results with Phi-3-mini, in addition to the Llama2-7B in their submission. Results provide more evidence on generalizability of the method on other backbone models.
- The authors provided an analysis on the skipping ratio (the proportion of answers where steps were skipped in the entire test set) at every iteration. This shows that the model indeed learns to skip more steps (while accuracy keeps unchanged or even improved) with more training iterations.
- The authors provided an additional experiment testing the method's generalizability across different domains. They show that training the model with such step-skipping behavior could generalize to unseen tasks.
- The authors clarified some points that were unclear before.

The authors agreed to conduct experiments on more realistic reasoning datasets such as GSM8K, MATH, or coding. However, due to the limited time, they fail to finish that set of experiments (which is understandable). Because of this, Reviewer vxGY remains their negative score of 4, which makes sense, I also understand. 

In general, the authors did a great job addressing the reviewers' concerns and answering their questions. Reviewer DBKK increased their score from 4 to 6. During the reviewer-AC discussion period, both Reviewer DBKK and rzDm involved in the discussion, they both express that while acknowledging that the work could be much stronger if they could include the more realistic datasets, they believe the current version (initial submission + authors' response) gives them enough confidence that the work could be well received by the NeurIPS community. 

Personally, I agree with all reviewers, I think this work provides some interesting insights on how models could be guided to evolve from system 1 to system 2 thinking. At the same time, I also agree with Reviewer vxGY that more complex / non-synthetic datasets could be crucial in better justifying the method. I tentatively suggest an accept.

Reviews: review: paper_summary: paper_summary: The paper proposes a method for training an LLM to solve reasoning problems using fewer verbalized reasoning steps than it is naturally encouraged to by a fixed training dataset. The resulting model is shown to maintian or improve performance on in-distribution data and OOD data testing extrapolation w.r.t. length or compositionality, while using fewer reasoning steps at inference time. Analysis shows that performance gains are concentrated around problems requiring an intermediate number of reasoning steps, rather than very few reasoning steps. Experiments are conducted with Llama-2-7b on three synthetic datasets. The method itself works by using warm-start data with mixed length reasoning demonstrations, followed by bootstrapped training data created by controlling model generations with control codes (instructions) combined with filtering model generations for correctness to create new gold data.
strength_weakness: strength_weakness: - Important: What are the instructions at inference time? Do you require a ground-truth number of reasoning steps to run the model at inference time? If so, this important detail is missing from the paper and could make the method difficult to use in practice for problems if it is not know how difficult they are in advance. Would the method be robust to misspecified instructions at inference time?
- Important: I find it a little confusing to reconcile the results of Sec. 5.1 with Sec. 5.2. Sec. 5.1 makes it look like using fewer steps greatly hurts model performance, while Sec. 5.2 makes it seem like using fewer steps does not hurt performance (specifically the Warm start rows, relative to Cold start baselines).
- Of some importance: The data is a little artifical. There are existing reasoning and compositional reasoning benchmarks that could be appropriate for this work (though they could require stronger models), including SCAN (https://arxiv.org/abs/1711.00350), GSM8k, StrategyQA, and MATH datasets. However, this is not a major weakness as using clean, controlled datasets is advantageous for studying these kinds of phenomena and they enable automatic construction of warm start data.
questions: questions: - Why keep the cold start data in the training data if the bootstrapped data is good or better? Do you have ablations that suggest what mixture of the data is best?
- Suggested experiment: if you could have two models that are similar except for one being better at long-context reasoning, it would be interesting to see how your method affects each model. The reason for this is that compressing reasoning length could be beneficial by virtue of reducing the context length, rather than some other inherent benefit like allowing the model to spend more computation on harder steps. Such an experiment would help disambiguate if the improvement comes from shortening context length or from using fewer steps.
- Note L.68-69 is heavily disputed by follow work on ToM, e.g. https://arxiv.org/pdf/2310.19619
- Just so you’re aware, some highly related work has appeared contemporaneously: (1) https://arxiv.org/pdf/2405.14838, (2) https://arxiv.org/pdf/2407.06023
- L.34: use an em-dash rather than single dash here
- L.221: Fig7(a) should read Fig4(a)
limitations: limitations: Discussion was adequate, but it could be worth mentioning that experiments were conducted with only one model and three synthetic datasets.
paper_summary: paper_summary: This paper proposes to teach LLMs to deliberately skip steps when doing complex tasks involving multi-step reasoning. The authors use self-generated inference paths with fewer steps to fine-tune the models, which is similar to self-distillation. The authors conduct experiments on a few controlled tasks show that the proposed approach can effectively reduces the reasoning steps while maintain performance.
strength_weakness: strength_weakness: 1. The experiments are not solid because the tasks considered in the experiments are very artificial and not representative for real-world reasoning tasks. The paper could be made much stronger by conducting tasks/datasets such as GSM8K/MATH or coding tasks, instead of simple reasoning tasks. Without the empirical study on realistic tasks, it is hard to confirm the contribution and usefulness of the proposed metric.
questions: questions: N/A
limitations: limitations: Yes
paper_summary: paper_summary: This paper proposes an iterative training method that helps sequence models learn to skip steps. The method starts from a training set with full-length solutions or mixed with some skipped-length solutions. At each stage a model learns these solutions with the instruction “Solve it in n steps” and is prompted to generate shorter answers. Correct shorter answers are added to the training set. The effect of this approach is tested using LlaMa-7B on three tasks, including algebraic evaluations, multi-digit addition, and a direction inference task.
strength_weakness: strength_weakness: My main concern is the generalizability of this method. As shown in the paper, the model largely benefited from the warm start setup that includes some skipped problems in the first training set, and has trouble generalizing to problems requiring more steps. One interesting generalization test would be to train on a mixture of all three tasks, but withhold adding skipped step instances for one task, and see if the model can generalize skipping steps on the withheld task.

The shorter answers generated during the iterative process also don't seem quite “generated by the model itself”, as filtering out correct answers would require oracle knowledge. Is it assumed that correct answers are any exact subset of the full-length solution?
questions: questions: The accuracy metric measures final answer accuracy, are intermediate steps correct?

I'm unsure if it makes sense to read too much into the average step metric when accuracy is low.

In figure 4, what does the accuracy look like for only problems where the model skipped steps?

What do you think make multi-digit addition and directional inference more difficult than the algebra task? Especially that accuracy and average step for OOD problems are still pretty bad for multi-digit addition even with the warm start and a few iterations in.

What is the change in the ratio between D_init and D_skipped over the iteration process?

What’s the range of i (in n-i) of the added skip-step instances in D_skipped under warm start?

Line 221 has incorrect figure reference.
limitations: limitations: Noted in the paper.
paper_summary: paper_summary: The paper explores the ability of language models to skip steps in their reasoning processes. The authors introduce a controlled framework to stimulate step-skipping behavior by iteratively refining models to generate shorter and accurate reasoning paths. The study demonstrates that models can develop this ability under guidance, leading to increased efficiency without sacrificing accuracy. The paper presents empirical results showing enhanced generalization capabilities in out-of-domain scenarios after fine-tuning on expanded datasets that include both complete and skipped reasoning sequences.
strength_weakness: strength_weakness: - Only one backbone model is considered. Experiments across model families and model sizes should be considered to show the generalization ability of the proposed methods.
- The OOD test is actually the harder in-domain test. I am curious about the across domain effect of the proposed method For example, how's the effect of training on "Analog of Algebra" and test on  "Multi-digit Addition", given that the skip ability should be a general ability across different domains?
- In methodology: "We begin with a training dataset D0, which contains detailed full-step reasoning answers to the questions." -> How the full-step reasoning data created?
questions: questions: see weakness
limitations: limitations: yes

Comments: title: Respose to rebuttal
comment: Thank you so much for your recognition and support of our work! 

We have conducted additional experiments to evaluate generalization across different domains. Our findings indicate that step-skipping data from other tasks positively impacts the step-skipping performance on the withheld task. We kindly invite you to review the detailed results in our response to Reviewer DBKK and Reviewer E6Qj. 

We will make sure to incorporate these valuable suggestions into our revision. Once again, we sincerely appreciate the time and effort you’ve taken to thoroughly review our work.
We are grateful for your recognition of our work! We here address the remaining concern as follows.

**[W1] Generalization across different domains**

We appreciate the insightful question regarding the generalization of step-skipping ability across different tasks. To address this concern, we conduct a controlled experiment. Specifically, we sample 2000 training examples from each dataset, along with 1600 step-skipping answers where exactly one step is successfully skipped, drawn from these 2000 questions in Iteration 5. This approach ensures that all three tasks have an equal amount of full-step and step-skipping data.

We utilize the Phi-3-mini model across all tasks. In the table below, the "withheld task" refers to the task that does not have step-skipping data in the training phase. The "All" setting includes only full-step answers across all three tasks without any step-skipping data.

To clarify the settings: 
* All setting = task1-full + task2-full + task3-full
* Withheld setting = task1-full + *task1-skip* + task2-full + *task2-skip* + task3-full

*Table 1: Evaluation task - Analog of Algebra*

| Withheld task       | In-domain | Avg steps | OOD-easy | Avg steps | OOD-hard | Avg steps |
|---------------------|-----------|-----------|----------|-----------|----------|-----------|
| All                 | 51.3      | 2.65      | 44.5     | 5.58      | 1.9      | 10.68     |
| Analog of Algebra   | 53.9      | 2.71      | 56.9     | 5.74      | 7.1      | 10.97     |

*Table 2: Evaluation task - Multi-digit Addition*

| Withheld task       | In-domain | Avg steps | OOD-easy | Avg steps | OOD-hard | Avg steps |
|---------------------|-----------|-----------|----------|-----------|----------|-----------|
| All                 | 100.0     | 2.86      | 22.4     | 4.71      | 4.2      | 5.39      |
| Multi-digit Addition| 95.7      | 2.59      | 34.3     | 4.75      | 2.4      | 5.35      |

*Table 3: Evaluation task - Directional Reasoning*

| Withheld task          | In-domain | Avg steps | OOD-easy | Avg steps | OOD-hard | Avg steps |
|------------------------|-----------|-----------|----------|-----------|----------|-----------|
| All                    | 100.0     | 7.01      | 96.0     | 15.46     | 75.8     | 25.03     |
| Directional Reasoning  | 97.8      | 6.98      | 96.2     | 15.42     | 80.0     | 24.92     |

The results show that the presence of step-skipping data in the training of other tasks positively impacts the performance of the withheld task. Compared to the "All" setting, models trained with step-skipping data in other tasks demonstrate improved step-skipping performance across all datasets with a comparable number of steps. Since the model is explicitly instructed to solve problems with fewer steps, the average steps in the withheld task remain similar to the "All" setting. However, the observed boost in accuracy suggests that the model benefits from the generalized step-skipping ability acquired from other tasks. This confirms the generalizability of the step-skipping ability across different domains. 

We appreciate your insightful questions and believe that these findings will further strengthen our work. We will make sure to incorporate these valuable suggestions into our revision. Thank you again for taking the time to review our work thoroughly.
Thank you for your valuable feedback! We address the remaining concerns as follows.

### **[W2] Generalization across different domains**

We appreciate the insightful questions regarding the generalization of step-skipping ability across different tasks. To address these concerns, we conduct a controlled experiment. Specifically, we sample 2000 training examples from each dataset, along with 1600 step-skipping answers where exactly one step is successfully skipped, drawn from these 2000 questions in Iteration 5. This approach ensures that all three tasks have an equal amount of full-step and step-skipping data.

We focus on evaluating a model that requires the number of steps as input. During testing, we instruct the model to solve each problem in $n−1$ steps, where $n$ is the full step count for that problem. This setting allows us to directly assess the step-skipping behavior by measuring accuracy under this evaluation setting. If the accuracy increases under this setting, it indicates improved step-skipping ability.

We utilize the Phi-3-mini model across all tasks. In the table below, the "withheld task" refers to the task that does not have step-skipping data in the training phase. The "All" setting includes only full-step answers across all three tasks without any step-skipping data.

To clarify the settings: 
* All setting = task1-full + task2-full + task3-full
* Withheld setting = task1-full + *task1-skip* + task2-full + *task2-skip* + task3-full

*Table 1: Evaluation task - Analog of Algebra*

| Withheld task       | In-domain | Avg steps | OOD-easy | Avg steps | OOD-hard | Avg steps |
|---------------------|-----------|-----------|----------|-----------|----------|-----------|
| All                 | 51.3      | 2.65      | 44.5     | 5.58      | 1.9      | 10.68     |
| Analog of Algebra   | 53.9      | 2.71      | 56.9     | 5.74      | 7.1      | 10.97     |

*Table 2: Evaluation task - Multi-digit Addition*

| Withheld task       | In-domain | Avg steps | OOD-easy | Avg steps | OOD-hard | Avg steps |
|---------------------|-----------|-----------|----------|-----------|----------|-----------|
| All                 | 100.0     | 2.86      | 22.4     | 4.71      | 4.2      | 5.39      |
| Multi-digit Addition| 95.7      | 2.59      | 34.3     | 4.75      | 2.4      | 5.35      |

*Table 3: Evaluation task - Directional Reasoning*

| Withheld task          | In-domain | Avg steps | OOD-easy | Avg steps | OOD-hard | Avg steps |
|------------------------|-----------|-----------|----------|-----------|----------|-----------|
| All                    | 100.0     | 7.01      | 96.0     | 15.46     | 75.8     | 25.03     |
| Directional Reasoning  | 97.8      | 6.98      | 96.2     | 15.42     | 80.0     | 24.92     |

The results show that the presence of step-skipping data in the training of other tasks positively impacts the performance of the withheld task. Compared to the "All" setting, models trained with step-skipping data in other tasks demonstrate improved step-skipping performance across all datasets with a comparable number of steps. Since the model is explicitly instructed to solve problems with fewer steps, the average steps in the withheld task remain similar to the "All" setting. However, the observed boost in accuracy suggests that the model benefits from the generalized step-skipping ability acquired from other tasks. This confirms the generalizability of the step-skipping ability across different domains. 

We appreciate the reviewers' insightful questions and believe that these findings will further strengthen our work. We will incorporate these findings into our revision.

### **[Q] How to ensure the quality of the auto-generated dataset?**

Thank you for your additional question!

For the Analog of Algebra task, we ensure the quality of the auto-generated dataset by creating full-step reasoning data using standard algebraic rules applied to operators. To further verify the validity and consistency of the intermediate steps, we utilize the SymPy library. Specifically, we perform SymPy simplification for each intermediate step and ensure that the resulting equation remains algebraically equivalent to the final simplified answer.

For the Multi-digit Addition task, the internal results are generated using Python's built-in calculation modules, ensuring accurate computations.

For the Directional Reasoning task, the clarity of the question formulation guarantees that all intermediate steps are 100% correct. Each step is derived through rule-based decomposition, ensuring the correctness of the intermediate steps.

All the data is guaranteed to be correct due to the heuristic nature of the creation process. We will make sure that the data creation and quality control processes are further clarified in the revision.
Thank you for the responses, given the authors' response, the W2 has not been resolved.  In addition, I am curious about how the authors ensure the quality of the auto-generated dataset. Is there any human evaluation of that?
Thank you for the thorough responses, they all make sense, and I appreciate the additional concrete evidence. I think adding/adjusting the metrics in your response to Q1 and Q3 on all tasks could really enhance the paper, and have raised my score accordingly.
>We only require the step number as input when generating the skipped data for the training set during the iteration phase. At inference time…

Thanks for the clarification! This highlights to me that there may not be strong control over how much step skipping the model does. Rather, the model learns to skip some portion of the time based on the training data. I think this is fine. I think the strength of this paper is interesting results around step skipping and model generalization, and not a new method for controlling how many steps a model takes when solving a problem.

>We apologize for the confusion regarding the results in Sections 5.1 and 5.2…

Thanks this makes sense! 

>We are actively working on additional experiments using these datasets

Great, please do upload these if they finish in time.

---

Based on the above discussion, I plan on keeping my score at 7. If the results were especially interesting on additional, harder datasets, I could increase it, but I think the core contribution of the paper is already solid based on the three datasets used.