Orionnnn/hub-stats huggingface dataset:
When Do LLMs Admit Their Mistakes? Understanding the Role of Model
  Belief in Retraction
Authors: name: Yuqing Yang, name: Robin Jia
Published: 2025-05-22 03:16:00

Can large language models (LLMs) admit their mistakes when they should know
better? In this work, we define the behavior of acknowledging errors in
previously generated answers as "retraction" and aim to understand when and why
LLMs choose to retract. We first construct model-specific datasets to evaluate
whether a model will retract an incorrect answer that contradicts its own
parametric knowledge. While LLMs are capable of retraction, they do so only
infrequently. We demonstrate that retraction is closely tied to previously
identified indicators of models' internal belief: models fail to retract wrong
answers that they "believe" to be factually correct. Steering experiments
further demonstrate that internal belief causally influences model retraction.
In particular, when the model does not believe its answer, this not only
encourages the model to attempt to verify the answer, but also alters attention
behavior during self-verification. Finally, we demonstrate that simple
supervised fine-tuning significantly improves retraction performance by helping
the model learn more accurate internal beliefs. Code and datasets are available
on https://github.com/ayyyq/llm-retraction.
SafeKey: Amplifying Aha-Moment Insights for Safety Reasoning
Authors: name: Kaiwen Zhou, name: Xuandong Zhao, name: Gaowen Liu, name: Jayanth Srinivasa, name: Aosong Feng, name: Dawn Song, name: Xin Eric Wang
Published: 2025-05-22 03:46:03

Large Reasoning Models (LRMs) introduce a new generation paradigm of
explicitly reasoning before answering, leading to remarkable improvements in
complex tasks. However, they pose great safety risks against harmful queries
and adversarial attacks. While recent mainstream safety efforts on LRMs,
supervised fine-tuning (SFT), improve safety performance, we find that
SFT-aligned models struggle to generalize to unseen jailbreak prompts. After
thorough investigation of LRMs' generation, we identify a safety aha moment
that can activate safety reasoning and lead to a safe response. This aha moment
typically appears in the `key sentence', which follows models' query
understanding process and can indicate whether the model will proceed safely.
Based on these insights, we propose SafeKey, including two complementary
objectives to better activate the safety aha moment in the key sentence: (1) a
Dual-Path Safety Head to enhance the safety signal in the model's internal
representations before the key sentence, and (2) a Query-Mask Modeling
objective to improve the models' attention on its query understanding, which
has important safety hints. Experiments across multiple safety benchmarks
demonstrate that our methods significantly improve safety generalization to a
wide range of jailbreak attacks and out-of-distribution harmful prompts,
lowering the average harmfulness rate by 9.6\%, while maintaining general
abilities. Our analysis reveals how SafeKey enhances safety by reshaping
internal attention and improving the quality of hidden representations.
Mind the Gap: Bridging Thought Leap for Improved Chain-of-Thought Tuning
Authors: name: Haolei Xu, name: Yuchen Yan, name: Yongliang Shen, name: Wenqi Zhang, name: Guiyang Hou, name: Shengpei Jiang, name: Kaitao Song, name: Weiming Lu, name: Jun Xiao, name: Yueting Zhuang
Published: 2025-05-20 17:59:31

Large language models (LLMs) have achieved remarkable progress on
mathematical tasks through Chain-of-Thought (CoT) reasoning. However, existing
mathematical CoT datasets often suffer from Thought Leaps due to experts
omitting intermediate steps, which negatively impacts model learning and
generalization. We propose the CoT Thought Leap Bridge Task, which aims to
automatically detect leaps and generate missing intermediate reasoning steps to
restore the completeness and coherence of CoT. To facilitate this, we
constructed a specialized training dataset called ScaleQM+, based on the
structured ScaleQuestMath dataset, and trained CoT-Bridge to bridge thought
leaps. Through comprehensive experiments on mathematical reasoning benchmarks,
we demonstrate that models fine-tuned on bridged datasets consistently
outperform those trained on original datasets, with improvements of up to
+5.87% on NuminaMath. Our approach effectively enhances distilled data (+3.02%)
and provides better starting points for reinforcement learning (+3.1%),
functioning as a plug-and-play module compatible with existing optimization
techniques. Furthermore, CoT-Bridge demonstrate improved generalization to
out-of-domain logical reasoning tasks, confirming that enhancing reasoning
completeness yields broadly applicable benefits.
Robo2VLM: Visual Question Answering from Large-Scale In-the-Wild Robot
  Manipulation Datasets
Authors: name: Kaiyuan Chen, name: Shuangyu Xie, name: Zehan Ma, name: Ken Goldberg
Published: 2025-05-21 13:42:52

Vision-Language Models (VLMs) acquire real-world knowledge and general
reasoning ability through Internet-scale image-text corpora. They can augment
robotic systems with scene understanding and task planning, and assist
visuomotor policies that are trained on robot trajectory data. We explore the
reverse paradigm - using rich, real, multi-modal robot trajectory data to
enhance and evaluate VLMs. In this paper, we present Robo2VLM, a Visual
Question Answering (VQA) dataset generation framework for VLMs. Given a human
tele-operated robot trajectory, Robo2VLM derives ground-truth from non-visual
and non-descriptive sensory modalities, such as end-effector pose, gripper
aperture, and force sensing. Based on these modalities, it segments the robot
trajectory into a sequence of manipulation phases. At each phase, Robo2VLM uses
scene and interaction understanding to identify 3D properties of the robot,
task goal, and the target object. The properties are used to generate
representative VQA queries - images with textural multiple-choice questions -
based on spatial, goal-conditioned, and interaction reasoning question
templates. We curate Robo2VLM-1, a large-scale in-the-wild dataset with 684,710
questions covering 463 distinct scenes and 3,396 robotic manipulation tasks
from 176k real robot trajectories. Results suggest that Robo2VLM-1 can
benchmark and improve VLM capabilities in spatial and interaction reasoning.
Let LLMs Break Free from Overthinking via Self-Braking Tuning
Authors: name: Haoran Zhao, name: Yuchen Yan, name: Yongliang Shen, name: Haolei Xu, name: Wenqi Zhang, name: Kaitao Song, name: Jian Shao, name: Weiming Lu, name: Jun Xiao, name: Yueting Zhuang
Published: 2025-05-20 16:53:40

Large reasoning models (LRMs), such as OpenAI o1 and DeepSeek-R1, have
significantly enhanced their reasoning capabilities by generating longer chains
of thought, demonstrating outstanding performance across a variety of tasks.
However, this performance gain comes at the cost of a substantial increase in
redundant reasoning during the generation process, leading to high
computational overhead and exacerbating the issue of overthinking. Although
numerous existing approaches aim to address the problem of overthinking, they
often rely on external interventions. In this paper, we propose a novel
framework, Self-Braking Tuning (SBT), which tackles overthinking from the
perspective of allowing the model to regulate its own reasoning process, thus
eliminating the reliance on external control mechanisms. We construct a set of
overthinking identification metrics based on standard answers and design a
systematic method to detect redundant reasoning. This method accurately
identifies unnecessary steps within the reasoning trajectory and generates
training signals for learning self-regulation behaviors. Building on this
foundation, we develop a complete strategy for constructing data with adaptive
reasoning lengths and introduce an innovative braking prompt mechanism that
enables the model to naturally learn when to terminate reasoning at an
appropriate point. Experiments across mathematical benchmarks (AIME, AMC,
MATH500, GSM8K) demonstrate that our method reduces token consumption by up to
60% while maintaining comparable accuracy to unconstrained models.
GoT-R1: Unleashing Reasoning Capability of MLLM for Visual Generation
  with Reinforcement Learning
Authors: name: Chengqi Duan, name: Rongyao Fang, name: Yuqing Wang, name: Kun Wang, name: Linjiang Huang, name: Xingyu Zeng, name: Hongsheng Li, name: Xihui Liu
Published: 2025-05-22 17:59:58

Visual generation models have made remarkable progress in creating realistic
images from text prompts, yet struggle with complex prompts that specify
multiple objects with precise spatial relationships and attributes. Effective
handling of such prompts requires explicit reasoning about the semantic content
and spatial layout. We present GoT-R1, a framework that applies reinforcement
learning to enhance semantic-spatial reasoning in visual generation. Building
upon the Generation Chain-of-Thought approach, GoT-R1 enables models to
autonomously discover effective reasoning strategies beyond predefined
templates through carefully designed reinforcement learning. To achieve this,
we propose a dual-stage multi-dimensional reward framework that leverages MLLMs
to evaluate both the reasoning process and final output, enabling effective
supervision across the entire generation pipeline. The reward system assesses
semantic alignment, spatial accuracy, and visual quality in a unified approach.
Experimental results demonstrate significant improvements on T2I-CompBench
benchmark, particularly in compositional tasks involving precise spatial
relationships and attribute binding. GoT-R1 advances the state-of-the-art in
image generation by successfully transferring sophisticated reasoning
capabilities to the visual generation domain. To facilitate future research, we
make our code and pretrained models publicly available at
https://github.com/gogoduan/GoT-R1.
GRIT: Teaching MLLMs to Think with Images
Authors: name: Yue Fan, name: Xuehai He, name: Diji Yang, name: Kaizhi Zheng, name: Ching-Chen Kuo, name: Yuting Zheng, name: Sravana Jyothi Narayanaraju, name: Xinze Guan, name: Xin Eric Wang
Published: 2025-05-21 17:54:49

Recent studies have demonstrated the efficacy of using Reinforcement Learning
(RL) in building reasoning models that articulate chains of thoughts prior to
producing final answers. However, despite ongoing advances that aim at enabling
reasoning for vision-language tasks, existing open-source visual reasoning
models typically generate reasoning content with pure natural language, lacking
explicit integration of visual information. This limits their ability to
produce clearly articulated and visually grounded reasoning chains. To this
end, we propose Grounded Reasoning with Images and Texts (GRIT), a novel method
for training MLLMs to think with images. GRIT introduces a grounded reasoning
paradigm, in which models generate reasoning chains that interleave natural
language and explicit bounding box coordinates. These coordinates point to
regions of the input image that the model consults during its reasoning
process. Additionally, GRIT is equipped with a reinforcement learning approach,
GRPO-GR, built upon the GRPO algorithm. GRPO-GR employs robust rewards focused
on the final answer accuracy and format of the grounded reasoning output, which
eliminates the need for data with reasoning chain annotations or explicit
bounding box labels. As a result, GRIT achieves exceptional data efficiency,
requiring as few as 20 image-question-answer triplets from existing datasets.
Comprehensive evaluations demonstrate that GRIT effectively trains MLLMs to
produce coherent and visually grounded reasoning chains, showing a successful
unification of reasoning and grounding abilities.
VideoGameQA-Bench: Evaluating Vision-Language Models for Video Game
  Quality Assurance
Authors: name: Mohammad Reza Taesiri, name: Abhijay Ghildyal, name: Saman Zadtootaghaj, name: Nabajeet Barman, name: Cor-Paul Bezemer
Published: 2025-05-21 19:08:38

With video games now generating the highest revenues in the entertainment
industry, optimizing game development workflows has become essential for the
sector's sustained growth. Recent advancements in Vision-Language Models (VLMs)
offer considerable potential to automate and enhance various aspects of game
development, particularly Quality Assurance (QA), which remains one of the
industry's most labor-intensive processes with limited automation options. To
accurately evaluate the performance of VLMs in video game QA tasks and
determine their effectiveness in handling real-world scenarios, there is a
clear need for standardized benchmarks, as existing benchmarks are insufficient
to address the specific requirements of this domain. To bridge this gap, we
introduce VideoGameQA-Bench, a comprehensive benchmark that covers a wide array
of game QA activities, including visual unit testing, visual regression
testing, needle-in-a-haystack tasks, glitch detection, and bug report
generation for both images and videos of various games. Code and data are
available at: https://asgaardlab.github.io/videogameqa-bench/
Pixel Reasoner: Incentivizing Pixel-Space Reasoning with
  Curiosity-Driven Reinforcement Learning
Authors: name: Alex Su, name: Haozhe Wang, name: Weimin Ren, name: Fangzhen Lin, name: Wenhu Chen
Published: 2025-05-21 19:35:08

Chain-of-thought reasoning has significantly improved the performance of
Large Language Models (LLMs) across various domains. However, this reasoning
process has been confined exclusively to textual space, limiting its
effectiveness in visually intensive tasks. To address this limitation, we
introduce the concept of reasoning in the pixel-space. Within this novel
framework, Vision-Language Models (VLMs) are equipped with a suite of visual
reasoning operations, such as zoom-in and select-frame. These operations enable
VLMs to directly inspect, interrogate, and infer from visual evidences, thereby
enhancing reasoning fidelity for visual tasks. Cultivating such pixel-space
reasoning capabilities in VLMs presents notable challenges, including the
model's initially imbalanced competence and its reluctance to adopt the newly
introduced pixel-space operations. We address these challenges through a
two-phase training approach. The first phase employs instruction tuning on
synthesized reasoning traces to familiarize the model with the novel visual
operations. Following this, a reinforcement learning (RL) phase leverages a
curiosity-driven reward scheme to balance exploration between pixel-space
reasoning and textual reasoning. With these visual operations, VLMs can
interact with complex visual inputs, such as information-rich images or videos
to proactively gather necessary information. We demonstrate that this approach
significantly improves VLM performance across diverse visual reasoning
benchmarks. Our 7B model, \model, achieves 84\% on V* bench, 74\% on
TallyQA-Complex, and 84\% on InfographicsVQA, marking the highest accuracy
achieved by any open-source model to date. These results highlight the
importance of pixel-space reasoning and the effectiveness of our framework.
AceReason-Nemotron: Advancing Math and Code Reasoning through
  Reinforcement Learning
Authors: name: Yang Chen, name: Zhuolin Yang, name: Zihan Liu, name: Chankyu Lee, name: Peng Xu, name: Mohammad Shoeybi, name: Bryan Catanzaro, name: Wei Ping
Published: 2025-05-22 08:50:47

Despite recent progress in large-scale reinforcement learning (RL) for
reasoning, the training recipe for building high-performing reasoning models
remains elusive. Key implementation details of frontier models, such as
DeepSeek-R1, including data curation strategies and RL training recipe, are
often omitted. Moreover, recent research indicates distillation remains more
effective than RL for smaller models. In this work, we demonstrate that
large-scale RL can significantly enhance the reasoning capabilities of strong,
small- and mid-sized models, achieving results that surpass those of
state-of-the-art distillation-based models. We systematically study the RL
training process through extensive ablations and propose a simple yet effective
approach: first training on math-only prompts, then on code-only prompts.
Notably, we find that math-only RL not only significantly enhances the
performance of strong distilled models on math benchmarks (e.g., +14.6% /
+17.2% on AIME 2025 for the 7B / 14B models), but also code reasoning tasks
(e.g., +6.8% / +5.8% on LiveCodeBench for the 7B / 14B models). In addition,
extended code-only RL iterations further improve performance on code benchmarks
with minimal or no degradation in math results. We develop a robust data
curation pipeline to collect challenging prompts with high-quality, verifiable
answers and test cases to enable verification-based RL across both domains.
Finally, we identify key experimental insights, including curriculum learning
with progressively increasing response lengths and the stabilizing effect of
on-policy parameter updates. We find that RL not only elicits the foundational
reasoning capabilities acquired during pretraining and supervised fine-tuning
(e.g., distillation), but also pushes the limits of the model's reasoning
ability, enabling it to solve problems that were previously unsolvable.
SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Reward
Authors: name: Kaixuan Fan, name: Kaituo Feng, name: Haoming Lyu, name: Dongzhan Zhou, name: Xiangyu Yue
Published: 2025-05-22 17:59:53

Recent advances have shown success in eliciting strong reasoning abilities in
multimodal large language models (MLLMs) through rule-based reinforcement
learning (RL) with outcome rewards. However, this paradigm typically lacks
supervision over the thinking process leading to the final outcome.As a result,
the model may learn sub-optimal reasoning strategies, which can hinder its
generalization ability. In light of this, we propose SophiaVL-R1, as an attempt
to add reward signals for the thinking process in this paradigm. To achieve
this, we first train a thinking reward model that evaluates the quality of the
entire thinking process. Given that the thinking reward may be unreliable for
certain samples due to reward hacking, we propose the Trust-GRPO method, which
assigns a trustworthiness weight to the thinking reward during training. This
weight is computed based on the thinking reward comparison of responses leading
to correct answers versus incorrect answers, helping to mitigate the impact of
potentially unreliable thinking rewards. Moreover, we design an annealing
training strategy that gradually reduces the thinking reward over time,
allowing the model to rely more on the accurate rule-based outcome reward in
later training stages. Experiments show that our SophiaVL-R1 surpasses a series
of reasoning MLLMs on various benchmarks (e.g., MathVisita, MMMU),
demonstrating strong reasoning and generalization capabilities. Notably, our
SophiaVL-R1-7B even outperforms LLaVA-OneVision-72B on most benchmarks, despite
the latter having 10 times more parameters. All code, models, and datasets are
made publicly available at https://github.com/kxfan2002/SophiaVL-R1.
LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning
Authors: name: Zebin You, name: Shen Nie, name: Xiaolu Zhang, name: Jun Hu, name: Jun Zhou, name: Zhiwu Lu, name: Ji-Rong Wen, name: Chongxuan Li
Published: 2025-05-22 17:23:26

In this work, we introduce LLaDA-V, a purely diffusion-based Multimodal Large
Language Model (MLLM) that integrates visual instruction tuning with masked
diffusion models, representing a departure from the autoregressive paradigms
dominant in current multimodal approaches. Built upon LLaDA, a representative
large language diffusion model, LLaDA-V incorporates a vision encoder and MLP
connector that projects visual features into the language embedding space,
enabling effective multimodal alignment. Our empirical investigation reveals
several intriguing results: First, LLaDA-V demonstrates promising multimodal
performance despite its language model being weaker on purely textual tasks
than counterparts like LLaMA3-8B and Qwen2-7B. When trained on the same
instruction data, LLaDA-V is highly competitive to LLaMA3-V across multimodal
tasks with better data scalability. It also narrows the performance gap to
Qwen2-VL, suggesting the effectiveness of its architecture for multimodal
tasks. Second, LLaDA-V achieves state-of-the-art performance in multimodal
understanding compared to existing hybrid autoregressive-diffusion and purely
diffusion-based MLLMs. Our findings suggest that large language diffusion
models show promise in multimodal contexts and warrant further investigation in
future research. Project page and codes:
https://ml-gsai.github.io/LLaDA-V-demo/.
NovelSeek: When Agent Becomes the Scientist -- Building Closed-Loop
  System from Hypothesis to Verification
Authors: name: NovelSeek Team, name: Bo Zhang, name: Shiyang Feng, name: Xiangchao Yan, name: Jiakang Yuan, name: Zhiyin Yu, name: Xiaohan He, name: Songtao Huang, name: Shaowei Hou, name: Zheng Nie, name: Zhilong Wang, name: Jinyao Liu, name: Runmin Ma, name: Tianshuo Peng, name: Peng Ye, name: Dongzhan Zhou, name: Shufei Zhang, name: Xiaosong Wang, name: Yilan Zhang, name: Meng Li, name: Zhongying Tu, name: Xiangyu Yue, name: Wangli Ouyang, name: Bowen Zhou, name: Lei Bai
Published: 2025-05-22 17:27:43

Artificial Intelligence (AI) is accelerating the transformation of scientific
research paradigms, not only enhancing research efficiency but also driving
innovation. We introduce NovelSeek, a unified closed-loop multi-agent framework
to conduct Autonomous Scientific Research (ASR) across various scientific
research fields, enabling researchers to tackle complicated problems in these
fields with unprecedented speed and precision. NovelSeek highlights three key
advantages: 1) Scalability: NovelSeek has demonstrated its versatility across
12 scientific research tasks, capable of generating innovative ideas to enhance
the performance of baseline code. 2) Interactivity: NovelSeek provides an
interface for human expert feedback and multi-agent interaction in automated
end-to-end processes, allowing for the seamless integration of domain expert
knowledge. 3) Efficiency: NovelSeek has achieved promising performance gains in
several scientific fields with significantly less time cost compared to human
efforts. For instance, in reaction yield prediction, it increased from 27.6% to
35.4% in just 12 hours; in enhancer activity prediction, accuracy rose from
0.52 to 0.79 with only 4 hours of processing; and in 2D semantic segmentation,
precision advanced from 78.8% to 81.0% in a mere 30 hours.