Title: Fast-Slow Thinking GRPO for Large Vision-Language Model Reasoning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Large Vision-Language Model

Fast-Slow Thinking

Reasoning
Abstract: When applying reinforcement learning—typically through GRPO—to large vision-language model reasoning struggles to effectively scale reasoning length or generates verbose outputs across all tasks with only marginal gains in accuracy.
To address this issue, we present FAST-GRPO, a variant of GRPO that dynamically adapts reasoning depth based on question characteristics. 
Through empirical analysis, we establish the feasibility of fast-slow thinking in LVLMs by investigating how response length and data distribution affect performance.
Inspired by these observations, we introduce two complementary metrics to estimate the difficulty of the questions, guiding the model to determine when fast or slow thinking is more appropriate. 
Next, we incorporate adaptive length-based rewards and difficulty-aware KL divergence into the GRPO algorithm.
Experiments across seven reasoning benchmarks demonstrate that FAST achieves state-of-the-art accuracy with over 10% relative improvement compared to the base model, while reducing token usage by 32.7-67.3% compared to previous slow-thinking approaches, effectively balancing reasoning length and accuracy.


Peer reviews:
comments: SUMMARY:
The paper introduces FAST-GRPO, a reinforcement-learning framework that explicitly balances “fast” (concise) and “slow” (detailed) reasoning in large vision-language models (LVLMs). It couples four ideas—(i) difficulty estimation, (ii) adaptive length reward, (iii) difficulty-aware KL regularisation, and (iv) a Slow-to-Fast curriculum sampler—to encourage short yet accurate answers. On seven math-heavy vision benchmarks (MathVision, MathVista, MM-Math, etc.), FAST-GRPO lifts average accuracy by 10 + pp while cutting answer length by 32.7 – 67.3 %. Remarkably, it reaches or surpasses strong baselines with only 18 k RL samples and no SFT fine-tuning.

STRENGTHS AND WEAKNESSES:
Strengths
1、The authors are first to combine difficulty-aware length rewards, a dynamic KL schedule, and a Slow-to-Fast sampler within GRPO for LVLMs, giving explicit control over “fast vs. slow” reasoning.
2、On seven public benchmarks, accuracy rises by double digits while answers shorten substantially; ablation studies show each component is necessary.

Weakness:
1、Experiments stop at 7 B parameters; scalability to ≥ 70 B remains untested (noted in Limitations).
2、Evaluation focuses on math/diagram reasoning; generalisation to open-domain VQA, captioning, or dialogue is uncertain.

QUESTIONS:
Further validating the framework’s scalability to larger models, improving the reliability of the difficulty estimation, and adding cross-task results would greatly enhance the paper’s credibility and practical value.

LIMITATIONS:
see above.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n1. The authors identify a key problem in current Large Vision-Language Models (LVLMs): \"slow-thinking\" models often \"overthink\" by generating overly verbose responses for all questions, regardless of their actual difficulty. This can be inefficient and even hurt accuracy on simpler problems. To address this, the authors propose FAST-GRPO- a method that enables the model to adapt its reasoning depth. The core of the approach is to first use two metrics to estimate the difficulty of a question. This difficulty estimation then guides GRPO process in two ways: adaptive length-based rewards to encourage concise answers for easy questions and more detailed reasoning for difficult ones. It uses a difficulty-aware KL divergence to control how much the model's policy can change, allowing for more exploration on complex problems.\n2. The proposed approach forces the model to learn when to apply \"fast thinking\" (short answers) and when to use \"slow thinking\" (detailed reasoning), leading to higher accuracy while significantly reducing the average response length.\n3. Two difficulty metrics are introduced - (a) Extrinsic Difficulty: Based on model's pass@k rate (how often it answers correctly), (b) Intrinsic Difficulty: Based on image complexity, using GLCM entropy and ViT entropy.\n4. A \"Slow-to-Fast\" curriculum is used—early training focuses on hard questions to build depth, then shifts to easier ones to promote concise thinking.\n5. The reward function combines - Accuracy reward, Format reward and Length reward: Encourages brevity on easy questions and thoroughness on hard ones. he KL coefficient is dynamically adjusted—looser for hard questions to allow exploration, tighter for easy ones to prevent overthinking.\nThe model is trained on 18K multimodal reasoning questions (filtered and difficulty-stratified), and evaluated on 7 benchmarks including MathVista, MathVerse, and MM-Vet.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths-\n1. Introduces a dynamic RL-based method (FAST-GRPO) that intelligently adapts reasoning depth based on question difficulty—filling a key gap in LVLM reasoning literature.\n2. The paper proposes innovative, complementary difficulty metrics (extrinsic and intrinsic), including a practical image complexity estimation using GLCM and ViT entropy.\n3. The empirical results show that on multiple LVLM reasoning benchmarks, the proposed approach provides SOTA accuracy while significantly reducing token usage compared to existing slow-thinking methods.\n4. The insights presented in the prelimnary investigation are useful.\n\nAreas of Improvement-\n1. The paper's method for calculating complexity, using the formula $S_{complexity} = S_{difficulty} \\cdot H_{img}$, contains a potential flaw in its logic. The issue arises when a problem is very difficult for the model to solve (resulting in a high $S_{difficulty}$) but is paired with a visually simple image (leading to a very low $H_{img}$). Because of the multiplication, the final complexity score for this problem would be close to zero. According to the framework's design, a low complexity score signals the model to use \"fast thinking\" and rewards shorter, more concise answers. This creates a direct contradiction: the model is encouraged to provide a brief solution for a problem that it finds very hard and likely needs a more detailed, step-by-step reasoning process (\"slow thinking\") to solve correctly. This conflicting reward signal could hinder the model's ability to learn how to tackle such challenging problems effectively.\n2. The authors start with ~500K questions and, after two filtering stages, arrive at a final training set of just 18K questions. However, the exact details of filtering strategy is not laid out well.\n3. The current curriculum learning approach from slow-fast has a binary threshold in terms of epochs, however, the authors do not experiment with a graded approach of switching from slow to fast thinking.\n4. All experiments are conducted on models up to 7B parameters; it’s unclear whether FAST-GRPO would maintain gains on larger-scale LVLMs (e.g., 70B+).\n5. The paper lacks sufficient qualitative analysis and error breakdown. While it reports strong quantitative improvements, it provides only a single example of reasoning inconsistency and does not include comparative qualitative outputs that illustrate how FAST-GRPO adapts reasoning depth in practice. Additionally, there is no systematic error analysis to identify failure modes across question types or reasoning stages, which limits interpretability and diagnostic insights into model behavior.\n\nQUESTIONS:\n1. Could you provide a more robust justification for the multiplicative formulation of the complexity score (S_complexity)? Have you experimented with alternative formulations, such as a weighted sum?\n2. The \"Slow-to-Fast\" curriculum is a binary split. Have you considered a more continuous curriculum where the probability of sampling \"Easy\" questions gradually increases over the course of training?\n3. The ablation study (Table 5) shows that removing Data Sampling causes the largest performance drop. Given the aggressiveness of the filtering, to what extent are the final gains attributable to the highly curated nature of the 18K dataset versus the FAST-GRPO algorithm itself?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n* This paper proposes FAST-GRPO, which is a variant of the GRPO reinforcement learning method that balances fast and slow reasoning for LVLMs through incorporating adaptive length-based rewards and dynamic regularization determined by question characteristics.\n* To determine the difficulty of questions, they propose 2 metrics — extrinsic and intrinsic difficulty — which provides accurate feedback to guide the model to choose between fast and slow thinking.\n* They improve the GRPO algorithm through including adaptive length-based rewards and difficulty-aware KL divergence.\n  * Empirical results on 7 reasoning benchmarks show the effectiveness of the proposed FAST-GRPO framework in achieving both efficiency in reasoning length and reasoning accuracy.\n\nSTRENGTHS AND WEAKNESSES:\n### **Strengths**\n* The paper is clearly written with good presentation.\n* The research question is very well-motivated. The authors first conducted pilot experiments to find out limitations of the current GRPO method on LVLMs, the observation which motivates them to explore whether determining and adaptively providing different reward feedback to multimodal questions with different levels of difficulty.\n* The proposed FAST-GRPO method is effective in balancing fast and slow reasoning, achieving effective reasoning length and accuracy performance.\n  * The authors proved this through extensive experiments across reasoning benchmarks and models, as well as conducted ablation experiments in 5.3 to prove the contribution of each method component.\n  * In 5.4, the authors conduct additional analysis to provide more empirical insights.\n\n### **Weakness**\n* Limited scope of reasoning tasks. As discussed in lines 239-245, 6 of the 7 benchmarks evaluated are limited to the domain of math reasoning. This draws questions to the generalizability of the proposed FAST-GRPO method on reasoning tasks in other domains.\n* Section 5.4 could be restructured to provide more clear insights to readers.\n  * For instance, “validation of image complexity”, “difficulty threshold analysis”, and “extrinsic difficulty analysis” are all discussing insights on the difficulty of multimodal question data. These could be organized into a sub-subsection with some higher level summarizations.\n\nQUESTIONS:\n* How does FAST-GRPO perform on reasoning tasks from a broader scope of domains, such as physics and social reasoning?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}