T w T : Thinking without Tokens by Habitual Reasoning Distillation with Multi-Teachers’ Guidance (2025)

Abstract: Large Language Models (LLMs) have made significant strides in problem-solving by incorporating reasoning processes. However, this enhanced reasoning capability results in an increased number of output tokens during inference, leading to higher computational costs. To address this challenge, we propose TwT (Thinking without Tokens), a method that reduces inference-time costs through habitual reasoning distillation with multi-teachers’ guidance, while maintaining high performance. Our approach introduces a Habitual Reasoning Distillation method, which internalizes explicit reasoning into the model’s habitual behavior through a Teacher-Guided compression strategy inspired by human cognition. Additionally, we propose Dual-Criteria Rejection Sampling (DCRS), a technique that generates a high-quality and diverse distillation dataset using multiple teacher models, making our method suitable for unsupervised scenarios. Experimental results demonstrate that TwT effectively reduces inference costs while preserving superior performance, achieving up to a 13.6% improvement in accuracy with fewer output tokens compared to other distillation methods, offering a highly practical solution for efficient LLM deployment.

## introduction large language models (llms) have demonstrated remarkable improvements in problemsolving by incorporating reasoning process @xcite @xcite . it enhances the reasoning capability of llms by breaking down complex tasks into intermediate steps, leading to better performance. however, reasoning capability comes at a significant cost: the reasoning process substantially increases the number of output tokens * work during internship at microsoft. recent research has primarily explored two key approaches to addressing this issue: reducing the cost per token and reducing the number of reasoning tokens as illustrated in figure 1 @xcite @xcite . a common strategy for reducing the cost per token is to replace large models with smaller, more efficient ones. however, directly using smaller models often results in a significant drop in performance, particularly on complex reasoning tasks. to mitigate this problem, knowledge distillation @xcite has emerged as an effective solution, enabling student models to mimic the performance of larger teacher models @xcite . however, traditional knowledge distillation typically relies on expensive human-labeled data, limiting its practical applicability. additionally, most distillation methods employ only a single teacher model, restricting the diversity of knowledge that could be leveraged from multiple teacher models. to overcome these limitations, we propose dual-criteria rejection sampling (dcrs), a method that first utilizes a multiteacher strategy to generate pseudo-labels and then applies a two-stage selection process-quality selection and diversity selection-to construct a highquality and diverse distillation dataset. this approach not only enhances the efficiency of knowledge transfer but also enables our model to adapt effectively to unsupervised settings. for the other issue, the number of reasoning tokens can be reduced by shrinking intermediate reasoning paths @xcite . while this effectively lowers token usage, it often comes at the cost of degraded model performance. therefore, it is essential to develop a method that balances computational efficiency and model accuracy. consider a real-world learning scenario where a teacher possesses a deep and comprehensive understanding of a concept, while a student may struggle to grasp the material fully. to bridge this gap, the teacher distills knowledge, extracting only the most essential and concise information to help the student learn more effectively. over time, the student internalizes this reasoning process, enabling them to generate answers instantly upon encountering a question, without requiring explicit intermediate reasoning steps. inspired by this human cognitive process, we propose habitual reasoning distillation (hard), a method that internalizes explicit reasoning into the model's habitual behavior through a multi-stage distillation process, thereby reducing the need for explicit reasoning during inference. hard follows a three-stage distillation strategy: (a) full reasoning distillation, where the student learns reasoning patterns from complete reasoning paths generated by teacher models; (b) reasoning-compressed distillation, where the reasoning process is progressively compressed, with teachers refining their outputs based on the student's responses to create reasoning paths aligned with the student's capabilities; and (c) reasoning-free distillation, where the student is trained without explicit reasoning steps, relying only on final labels, allowing it to generate high-quality answers directly. this process shifts the computational burden from inference to training, enabling both high performance and low inference cost. in this work, we propose twt (thinking without tokens), a method that achieves an optimal balance between inference-time computational cost and performance. twt follows a two-step process. first, dcrs utilizes multi-teacher llms to generate pseudo-labels, enabling the model to adapt to unsupervised settings. then, hard applies a multi-stage distillation approach to progressively internalize explicit reasoning abilities into the student model as inherent capabilities. our key contributions are summarized as follows: twt, a novel framework that aims to reduce inference-time computational cost through habitual reasoning distillation with multiteachers' guidance while preserving high performance. unlabeled dataset prompt llm t1 llm t2 llm t3 labelling 1.dual-criteria rejection sampling score threshold: ≥0.95 ## method in this section, we provide a detailed explanation of the implementation of our twt as illustrated in fig. 2 . first, we propose a dual-criteria rejection sampling (dcrs) strategy to obtain high-quality and diverse distillation samples (section 3.1). then, we design a habitual reasoning distillation (hard) strategy that progressively internalizes the reasoning ability at each distillation stage, allowing the reasoning capabilities of the teacher models to be gradually internalized into the student model (section 3.2). this approach ensures a high-quality and diverse distillation dataset, enabling our method to effectively adapt to unsupervised scenarios. ## conclusion and future work we introduced twt, a novel distillation framework that internalizes reasoning abilities into a student model under multi-teachers' guidance. it incorporates a dual-criteria rejection sampling stage to obtain a high-quality and diverse distillation dataset and a habitual reasoning distillation strategy to gradually integrate reasoning capabilities into the student model. twt achieves high performance with low inference cost without relying on labeled data or an explicit reasoning process. in future work, we will continue to explore whether further subdividing the distillation stages can enhance our framework