Self-Reasoning Language Models: Unfold Hidden Reasoning Chains with Few Reasoning Catalyst (2025)

Abstract: Inference-time scaling has attracted much attention which significantly enhance the performance of Large Language Models (LLMs) in complex reasoning tasks by increasing the length of Chain-of-Thought. These longer intermediate reasoning rationales embody various meta-reasoning skills in human cognition such as reflection and decomposition, being difficult to create and acquire. In this work, we introduce Self-Reasoning Language Model (SRLM), where the model itself can synthesize longer CoT data and iteratively improve performance through self-training. By incorporating a few demonstration examples (i.e., 1,000 samples) on how to unfold hidden reasoning chains from existing responses, which act as a reasoning catalyst, we demonstrate that SRLM not only enhances the model’s initial performance but also ensures more stable and consistent improvements in subsequent iterations. Our proposed SRLM achieves an average absolute improvement of more than +2.5 points across five reasoning tasks: MMLU, GSM8K, ARC-C, HellaSwag, and BBH on two backbone models. Moreover, it brings more improvements with more times of sampling during inference, such as absolute +7.89 average improvement with 64 sampling times, revealing the in-depth, diverse and creative reasoning paths in SRLM against the strong baseline .

## introduction recent studies have demonstrated that inferencetime scaling @xcite effectively improves performance of large language models (llms) in various reasoning tasks, such as mathematics and complex question answering, by extending the length of chain-of-thought (cot) @xcite reasoning rationales @xcite . with longer cots during inference, these llms could explore more creative and diverse reasoning rationales while assembling various meta-reasoning skills observed in human cognition @xcite @xcite , such as reflection and decomposition @xcite , resulting in better performance. however, the scarcity of these longer cot data remains a significant obstacle to advancing inference-time scaling, especially for instructions without verifiable answers in the general instruction-tuning data. several prior studies have explored various approaches to get better cot tuning datasets, where most of them utilize either the llm itself @xcite @xcite or external reasoning models @xcite to generate (or refine) new (or existing) responses in the instruction-tuning dataset, leading to great improvements at various downstream tasks. despite the effectiveness of these proposed methods, they still face several limitations. on the one hand, most of them focus on questions with verifiable answer such as math and code @xcite , being infeasible for general instructiontuning dataset. on the other hand, another line of work typically assumes access to more powerful models to refine each sample iteratively @xcite . such methods suffer from performance plateaus or even degradation @xcite and are inherently constrained by the capability ceiling of the powerful model. to this end, we present self-reasoning language models (srlm), which is capable to selfunfolding its own reasoning rationales and iteratively optimize itself, leading to enhanced overall capability. specifically, we first create only few reasoning catalyst data that compose the demonstrations of how to enrich shorter cot rationales into more longer and comprehensive cot with the augmentation of various meta-reasoning skills. after incorporating the reasoning catalyst data with the original instruction-tuning data, the tuned model not only inherit the basic reasoning capabilities from the instruction-tuning dataset, but also learn how to refine reasoning simultaneously, resulting in self-reasoning language models. consequently, the srlm can refine its own reasoning rationales at each iteration with the processing of reasoning expansion and selection. during this process, the model generates enriched reasoning rationale candidates for the same instructions in the original instruction-tuning dataset. these rationale pairs are then filtered and selected using three proposed selectors without any prior assumption about the instruction and answer. finally, the newly selected instruction-tuning dataset is combined with the reasoning catalyst data to create the training data for the next iteration of srlm, which is initialized from the same base model. to conclude, our contributions can be summarized as follows: in addition to constructing datasets for sft purpose, there are also efforts in constructing cot related datasets for evaluation purposes, such as for multi-modal llms @xcite and for attributed question answering @xcite . ## conclusion in this paper, we present self-reasoning language models (srlms), which are capable of generating responses and unfolding hidden reasoning chains at the same time, leading to self-evolved models that enhance performance across various reasoning benchmarks. while there are many avenues left unexplored, we believe this is exciting and promising since the srlms are better able to discover diverse and creative reasoning paths in future iterations for improving instruction following -a kind of virtuous circle.