Fast Quiet- ST a R : Thinking Without Thought Tokens (2025)

Abstract: Large Language Models (LLMs) have achieved impressive performance across a range of natural language processing tasks. However, recent advances demonstrate that further gains—particularly in complex reasoning tasks—require more than merely scaling up model sizes or training data. One promising direction is to enable models to “think” during the reasoning process. Recently, Quiet-STaR significantly improves reasoning by generating token-level thought traces, but incurs substantial inference overhead. In this work, we propose Fast Quiet-STaR, a more efficient reasoning framework that preserves the benefits of token-level reasoning while reducing computational cost. Our method introduces a curriculum-learning-based training strategy that gradually reduces the number of thought tokens, enabling the model to internalize more abstract and concise reasoning processes. We further extend this approach to the standard Next Token Prediction (NTP) setting through reinforcement learning-based fine-tuning, resulting in Fast Quiet-STaR NTP, which eliminates the need for explicit thought token generation during inference. Experiments on four benchmark datasets with Mistral 7B and Qwen2.5 7B demonstrate that Fast Quiet-STaR consistently outperforms Quiet-STaR in terms of average accuracy under the same inference time budget. Notably, Fast Quiet-STaR NTP achieves an average accuracy improvement of 9% on Mistral 7B and 5.7% on Qwen2.5 7B, while maintaining the same inference latency.

## introduction artificial intelligence has made remarkable progress in recent years @xcite , particularly with large language models (llms) ( @xcite through pre-training models with billions of parameters on massive datasets. however, merely scaling up model size or increasing the amount of training data is insufficient for enabling strong performance on tasks that require complex reasoning or longterm planning. to further enhance model capabilities, one promising direction is to enable models to engage in autonomous "thinking" before producing final answers. recently, a growing body of research has explored this paradigm to strengthen the reasoning abilities of llms. notably, models such as openai o1 @xcite , @xcite , qwq @xcite , and kimi-1.5 @xcite have demonstrated impressive performance across a variety of challenging tasks, such as mathematical competition problems @xcite . recently, quiet-star (quiet self-taught rea-soner) @xcite has been proposed as a novel reasoning paradigm that shifts the thinking process from the problem level to a finer, token-level granularity. in quiet-star, before predicting the next token, the model first generates a sequence of intermediate thought trace (represented @xmath0,thought_token1, thought_token2, ... 
), based on which the model predicts the next token. compared to other approaches, quiet-star can significantly enhance the model's reasoning ability through a lightweight unsupervised training process. for instance, it achieves a 10% performance gain on commonsenseqa @xcite for the mistral 7b @xcite by continue pre-training with only 0.2m tokens, demonstrating remarkable improvements through efficient training. although quiet-star significantly enhances the model's reasoning capabilities, it substantially increases inference overhead due to the requirement of generating a thought trace for every token. as shown in figure 1 , even when using only 8 thought tokens, the average time-to-first-token (ttft) of quiet-star remains over 10 times higher than that of conventional next token prediction (ntp) models. despite the high inference costs, these thought tokens cannot be directly reduced or eliminated as they are the main contributor to performance improvements. for example, as shown in figure 1 , halving the number of thought tokens from 16 to 8 leads to a 4.7% accuracy drop. such dilemma on efficiency severely undermines the practical value of quiet-star. prior work has shown that llms are capable of skipping reasoning steps by omitting nonessential steps without sacrificing overall reasoning performance @xcite . inspired by this, we believe that within the quiet-star reasoning paradigm, the model can maintain its strong reasoning abilities obtained from long thoughttrace training by compressing the number of thought tokens and keeping only a more abstract thought trace. to improve the efficiency of the quiet-star reasoning paradigm with minimal performance degradation, we propose fast quiet-star. we employ a multi-stage training strategy that progresses from easy to hard. that is, we gradually guide the model from generating a detailed thought trace using more thought tokens to generating a concise thought trace using fewer thought tokens. to further accelerate the quiet-star inference paradigm to nlp-level efficiency, we employ a reinforcement learning-based finetuning strategy for fast quiet-star model under the ntp setting. the resulting fast quiet-star ntp model preserves the original thinking abilities of fast quiet-star while eliminating reliance on generating explicit thought trace during inference. we evaluate our method on two open-source models, mistral 7b @xcite and qwen2.5 7b @xcite across four public datasets. extensive experiments show that, given the same number of thought tokens(same inference time), fast quiet-star achieves substantial performance gains over quiet-star. furthermore, under equivalent inference time, fast quiet-star ntp improves the average accuracy by 9% on mistral 7b and 5.7% on qwen2.5 7b compared to the original pre-trained models. we summarize our contribution as follows: ## methodology the training procedure of fast quiet-star is illustrated in figure 2 . building upon quiet-star, we propose a progressive, multi-stage training framework inspired by the principles of curriculum learning. this approach facilitates a gradual transition from easy to hard reasoning paradigms. in particular, during the final stage of training, we incorporate reinforcement learning to transition the reasoning paradigm of quiet-star to the the standard ntp paradigm. ## conclusion in this paper, we proposed fast quiet-star, an efficient extension of the quiet-star reasoning paradigm that maintains the core benefits of finegrained token-level reasoning while significantly reducing inference overhead. by leveraging a curriculum learning-based training strategy that progressively reduces the number of thought tokens, fast quiet-star enables models to develop compact yet effective reasoning abilities. furthermore, through reinforcement learning-based fine-tuning, we extend this paradigm to the standard next token prediction setting, eliminating the need for explicit thought-token generation during inference. experiments on mistral 7b and qwen2.5 7b across four benchmark datasets show that fast quiet-star achieves substantial gains over quiet-star under the same number of thought tokens, and fast quiet-star ntp outperforms the pre-trained model and performs on par with quiet-star. these results highlight fast quiet-star as a practical solution for enhancing reasoning capabilities in llms.