Thinker: Learning to Think Fast and Slow
Authors: name: Stephen Chung; name: Wenyu Du; name: Jie Fu
Published: 2025-05-27 12:22:46

Abstract: Recent studies show that the reasoning capabilities of Large Language Models
(LLMs) can be improved by applying Reinforcement Learning (RL) to
question-answering (QA) tasks in areas such as math and coding. With a long
context length, LLMs may learn to perform search, as indicated by the
self-correction behavior observed in DeepSeek R1. However, this search behavior
is often imprecise and lacks confidence, resulting in long, redundant responses
and highlighting deficiencies in intuition and verification. Inspired by the
Dual Process Theory in psychology, we introduce a simple modification to the QA
task that includes four stages: Fast Thinking, where the LLM must answer within
a strict token budget; Verification, where the model evaluates its initial
response; Slow Thinking, where it refines the initial response with more
deliberation; and Summarization, where it distills the refinement from the
previous stage into precise steps. Our proposed task improves average accuracy
from 24.9% to 27.9% for Qwen2.5-1.5B, and from 45.9% to 49.8% for
DeepSeek-R1-Qwen-1.5B. Notably, for Qwen2.5-1.5B, the Fast Thinking mode alone
achieves 26.8% accuracy using fewer than 1000 tokens, demonstrating substantial
inference efficiency gains. These findings suggest that intuition and
deliberative reasoning are distinct, complementary systems benefiting from
targeted training.