Orionnnn/hub-stats huggingface dataset:
Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective
  Reinforcement Learning for LLM Reasoning
Authors: name: Shenzhi Wang, name: Le Yu, name: Chang Gao, name: Chujie Zheng, name: Shixuan Liu, name: Rui Lu, name: Kai Dang, name: Xionghui Chen, name: Jianxin Yang, name: Zhenru Zhang, name: Yuqiong Liu, name: An Yang, name: Andrew Zhao, name: Yang Yue, name: Shiji Song, name: Bowen Yu, name: Gao Huang, name: Junyang Lin
Published: 2025-06-02 17:54:39

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a
powerful approach to enhancing the reasoning capabilities of Large Language
Models (LLMs), while its mechanisms are not yet well understood. In this work,
we undertake a pioneering exploration of RLVR through the novel perspective of
token entropy patterns, comprehensively analyzing how different tokens
influence reasoning performance. By examining token entropy patterns in
Chain-of-Thought (CoT) reasoning, we observe that only a small fraction of
tokens exhibit high entropy, and these tokens act as critical forks that steer
the model toward diverse reasoning pathways. Furthermore, studying how entropy
patterns evolve during RLVR training reveals that RLVR largely adheres to the
base model's entropy patterns, primarily adjusting the entropy of high-entropy
tokens. These findings highlight the significance of high-entropy tokens (i.e.,
forking tokens) to RLVR. We ultimately improve RLVR by restricting policy
gradient updates to forking tokens and uncover a finding even beyond the 80/20
rule: utilizing only 20% of the tokens while maintaining performance comparable
to full-gradient updates on the Qwen3-8B base model and significantly
surpassing full-gradient updates on the Qwen3-32B (+11.04 on AIME'25 and +7.71
on AIME'24) and Qwen3-14B (+4.79 on AIME'25 and +5.21 on AIME'24) base models,
highlighting a strong scaling trend. In contrast, training exclusively on the
80% lowest-entropy tokens leads to a marked decline in performance. These
findings indicate that the efficacy of RLVR primarily arises from optimizing
the high-entropy tokens that decide reasoning directions. Collectively, our
results highlight the potential to understand RLVR through a token-entropy
perspective and optimize RLVR by leveraging high-entropy minority tokens to
further improve LLM reasoning.
ARIA: Training Language Agents with Intention-Driven Reward Aggregation
Authors: name: Ruihan Yang, name: Yikai Zhang, name: Aili Chen, name: Xintao Wang, name: Siyu Yuan, name: Jiangjie Chen, name: Deqing Yang, name: Yanghua Xiao
Published: 2025-05-31 12:54:49

Large language models (LLMs) have enabled agents to perform complex reasoning
and decision-making through free-form language interactions. However, in
open-ended language action environments (e.g., negotiation or question-asking
games), the action space can be formulated as a joint distribution over tokens,
resulting in an exponentially large action space. Sampling actions in such a
space can lead to extreme reward sparsity, which brings large reward variance,
hindering effective reinforcement learning (RL). To address this, we propose
ARIA, a method that Aggregates Rewards in Intention space to enable efficient
and effective language Agents training. ARIA aims to project natural language
actions from the high-dimensional joint token distribution space into a
low-dimensional intention space, where semantically similar actions are
clustered and assigned shared rewards. This intention-aware reward aggregation
reduces reward variance by densifying reward signals, fostering better policy
optimization. Extensive experiments demonstrate that ARIA not only
significantly reduces policy gradient variance, but also delivers substantial
performance gains of an average of 9.95% across four downstream tasks,
consistently outperforming offline and online RL baselines.
Incentivizing Reasoning for Advanced Instruction-Following of Large
  Language Models
Authors: name: Yulei Qin, name: Gang Li, name: Zongyi Li, name: Zihan Xu, name: Yuchen Shi, name: Zhekai Lin, name: Xiao Cui, name: Ke Li, name: Xing Sun
Published: 2025-06-02 08:11:44

Existing large language models (LLMs) face challenges of following complex
instructions, especially when multiple constraints are present and organized in
paralleling, chaining, and branching structures. One intuitive solution, namely
chain-of-thought (CoT), is expected to universally improve capabilities of
LLMs. However, we find that the vanilla CoT exerts a negative impact on
performance due to its superficial reasoning pattern of simply paraphrasing the
instructions. It fails to peel back the compositions of constraints for
identifying their relationship across hierarchies of types and dimensions. To
this end, we propose a systematic method to boost LLMs in dealing with complex
instructions via incentivizing reasoning for test-time compute scaling. First,
we stem from the decomposition of complex instructions under existing
taxonomies and propose a reproducible data acquisition method. Second, we
exploit reinforcement learning (RL) with verifiable rule-centric reward signals
to cultivate reasoning specifically for instruction following. We address the
shallow, non-essential nature of reasoning under complex instructions via
sample-wise contrast for superior CoT enforcement. We also exploit behavior
cloning of experts to facilitate steady distribution shift from fast-thinking
LLMs to skillful reasoners. Extensive evaluations on seven comprehensive
benchmarks confirm the validity of the proposed method, where a 1.5B LLM
achieves 11.74% gains with performance comparable to a 8B LLM. Codes and data
are available at https://github.com/yuleiqin/RAIF.
REASONING GYM: Reasoning Environments for Reinforcement Learning with
  Verifiable Rewards
Authors: name: Zafir Stojanovski, name: Oliver Stanley, name: Joe Sharratt, name: Richard Jones, name: Abdulhakeem Adefioye, name: Jean Kaddour, name: Andreas Köpf
Published: 2025-05-30 16:20:18

We introduce Reasoning Gym (RG), a library of reasoning environments for
reinforcement learning with verifiable rewards. It provides over 100 data
generators and verifiers spanning multiple domains including algebra,
arithmetic, computation, cognition, geometry, graph theory, logic, and various
common games. Its key innovation is the ability to generate virtually infinite
training data with adjustable complexity, unlike most previous reasoning
datasets, which are typically fixed. This procedural generation approach allows
for continuous evaluation across varying difficulty levels. Our experimental
results demonstrate the efficacy of RG in both evaluating and reinforcement
learning of reasoning models.
Learning Video Generation for Robotic Manipulation with Collaborative
  Trajectory Control
Authors: name: Xiao Fu, name: Xintao Wang, name: Xian Liu, name: Jianhong Bai, name: Runsen Xu, name: Pengfei Wan, name: Di Zhang, name: Dahua Lin
Published: 2025-06-02 17:57:06

Recent advances in video diffusion models have demonstrated strong potential
for generating robotic decision-making data, with trajectory conditions further
enabling fine-grained control. However, existing trajectory-based methods
primarily focus on individual object motion and struggle to capture
multi-object interaction crucial in complex robotic manipulation. This
limitation arises from multi-feature entanglement in overlapping regions, which
leads to degraded visual fidelity. To address this, we present RoboMaster, a
novel framework that models inter-object dynamics through a collaborative
trajectory formulation. Unlike prior methods that decompose objects, our core
is to decompose the interaction process into three sub-stages: pre-interaction,
interaction, and post-interaction. Each stage is modeled using the feature of
the dominant object, specifically the robotic arm in the pre- and
post-interaction phases and the manipulated object during interaction, thereby
mitigating the drawback of multi-object feature fusion present during
interaction in prior work. To further ensure subject semantic consistency
throughout the video, we incorporate appearance- and shape-aware latent
representations for objects. Extensive experiments on the challenging Bridge V2
dataset, as well as in-the-wild evaluation, demonstrate that our method
outperforms existing approaches, establishing new state-of-the-art performance
in trajectory-controlled video generation for robotic manipulation.
Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with
  Jigsaw Puzzles
Authors: name: Zifu Wang, name: Junyi Zhu, name: Bo Tang, name: Zhiyu Li, name: Feiyu Xiong, name: Jiaqian Yu, name: Matthew B. Blaschko
Published: 2025-05-29 16:01:22

The application of rule-based reinforcement learning (RL) to multimodal large
language models (MLLMs) introduces unique challenges and potential deviations
from findings in text-only domains, particularly for perception-heavy tasks.
This paper provides a comprehensive study of rule-based visual RL, using jigsaw
puzzles as a structured experimental framework. Jigsaw puzzles offer inherent
ground truth, adjustable difficulty, and demand complex decision-making, making
them ideal for this study. Our research reveals several key findings:
Firstly, we find that MLLMs, initially performing near to random
guessing on the simplest jigsaw puzzles, achieve near-perfect accuracy and
generalize to complex, unseen configurations through fine-tuning.
Secondly, training on jigsaw puzzles can induce generalization to
other visual tasks, with effectiveness tied to specific task configurations.
Thirdly, MLLMs can learn and generalize with or without explicit
reasoning, though open-source models often favor direct answering.
Consequently, even when trained for step-by-step reasoning, they can ignore the
thinking process in deriving the final answer. Fourthly, we observe
that complex reasoning patterns appear to be pre-existing rather than emergent,
with their frequency increasing alongside training and task difficulty.
Finally, our results demonstrate that RL exhibits more effective
generalization than Supervised Fine-Tuning (SFT), and an initial SFT cold start
phase can hinder subsequent RL optimization. Although these observations are
based on jigsaw puzzles and may vary across other visual tasks, this research
contributes a valuable piece of jigsaw to the larger puzzle of collective
understanding rule-based visual RL and its potential in multimodal learning.
The code is available at: https://github.com/zifuwanggg/Jigsaw-R1.
Cora: Correspondence-aware image editing using few step diffusion
Authors: name: Amirhossein Almohammadi, name: Aryan Mikaeili, name: Sauradip Nag, name: Negar Hassanpour, name: Andrea Tagliasacchi, name: Ali Mahdavi-Amiri
Published: 2025-05-29 18:00:56

Image editing is an important task in computer graphics, vision, and VFX,
with recent diffusion-based methods achieving fast and high-quality results.
However, edits requiring significant structural changes, such as non-rigid
deformations, object modifications, or content generation, remain challenging.
Existing few step editing approaches produce artifacts such as irrelevant
texture or struggle to preserve key attributes of the source image (e.g.,
pose). We introduce Cora, a novel editing framework that addresses these
limitations by introducing correspondence-aware noise correction and
interpolated attention maps. Our method aligns textures and structures between
the source and target images through semantic correspondence, enabling accurate
texture transfer while generating new content when necessary. Cora offers
control over the balance between content generation and preservation. Extensive
experiments demonstrate that, quantitatively and qualitatively, Cora excels in
maintaining structure, textures, and identity across diverse edits, including
pose changes, object addition, and texture refinements. User studies confirm
that Cora delivers superior results, outperforming alternatives.
From Token to Action: State Machine Reasoning to Mitigate Overthinking
  in Information Retrieval
Authors: name: Dohyeon Lee, name: Yeonseok Jeong, name: Seung-won Hwang
Published: 2025-05-29 04:04:25

Chain-of-Thought (CoT) prompting enables complex reasoning in large language
models (LLMs), including applications in information retrieval (IR). However,
it often leads to overthinking, where models produce excessively long and
semantically redundant traces with little or no benefit. We identify two key
challenges in IR: redundant trajectories that revisit similar states and
misguided reasoning that diverges from user intent. To address these, we
propose State Machine Reasoning (SMR), a transition-based reasoning framework
composed of discrete actions (Refine, Rerank, Stop) that support early stopping
and fine-grained control. Experiments on the BEIR and BRIGHT benchmarks show
that SMR improves retrieval performance (nDCG@10) by 3.4% while reducing token
usage by 74.4%. It generalizes across LLMs and retrievers without requiring
task-specific tuning, offering a practical alternative to conventional CoT
reasoning. The code and details are available at https://github.com/ldilab/SMR.
OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and
  Cleaning
Authors: name: Yifan Peng, name: Shakeel Muhammad, name: Yui Sudo, name: William Chen, name: Jinchuan Tian, name: Chyi-Jiunn Lin, name: Shinji Watanabe
Published: 2025-05-31 01:44:44

The Open Whisper-style Speech Models (OWSM) project has developed a series of
fully open speech foundation models using academic-scale resources, but their
training data remains insufficient. This work enhances OWSM by integrating
YODAS, a large-scale web-crawled dataset with a Creative Commons license.
However, incorporating YODAS is nontrivial due to its wild nature, which
introduces challenges such as incorrect language labels and audio-text
misalignments. To address this, we develop a scalable data-cleaning pipeline
using public toolkits, yielding a dataset with 166,000 hours of speech across
75 languages. Our new series of OWSM v4 models, trained on this curated dataset
alongside existing OWSM data, significantly outperform previous versions on
multilingual benchmarks. Our models even match or surpass frontier industrial
models like Whisper and MMS in multiple scenarios. We will publicly release the
cleaned YODAS data, pre-trained models, and all associated scripts via the
ESPnet toolkit.
WHEN TO ACT, WHEN TO WAIT: Modeling Structural Trajectories for Intent
  Triggerability in Task-Oriented Dialogue
Authors: name: Yaoyao Qian, name: Jindan Huang, name: Yuanli Wang, name: Simon Yu, name: Kyrie Zhixuan Zhou, name: Jiayuan Mao, name: Mingfu Liang, name: Hanhan Zhou
Published: 2025-06-02 17:11:10

Task-oriented dialogue systems often face difficulties when user utterances
seem semantically complete but lack necessary structural information for
appropriate system action. This arises because users frequently do not fully
understand their own needs, while systems require precise intent definitions.
Current LLM-based agents cannot effectively distinguish between linguistically
complete and contextually triggerable expressions, lacking frameworks for
collaborative intent formation. We present STORM, a framework modeling
asymmetric information dynamics through conversations between UserLLM (full
internal access) and AgentLLM (observable behavior only). STORM produces
annotated corpora capturing expression trajectories and latent cognitive
transitions, enabling systematic analysis of collaborative understanding
development. Our contributions include: (1) formalizing asymmetric information
processing in dialogue systems; (2) modeling intent formation tracking
collaborative understanding evolution; and (3) evaluation metrics measuring
internal cognitive improvements alongside task performance. Experiments across
four language models reveal that moderate uncertainty (40-60%) can outperform
complete transparency in certain scenarios, with model-specific patterns
suggesting reconsideration of optimal information completeness in human-AI
collaboration. These findings contribute to understanding asymmetric reasoning
dynamics and inform uncertainty-calibrated dialogue system design.
Reasoning Like an Economist: Post-Training on Economic Problems Induces
  Strategic Generalization in LLMs
Authors: name: Yufa Zhou, name: Shaobo Wang, name: Xingyu Dong, name: Xiangqi Jin, name: Yifang Chen, name: Yue Min, name: Kexin Yang, name: Xingzhang Ren, name: Dayiheng Liu, name: Linfeng Zhang
Published: 2025-05-31 14:22:40

Directly training Large Language Models (LLMs) for Multi-Agent Systems (MAS)
remains challenging due to intricate reward modeling, dynamic agent
interactions, and demanding generalization requirements. This paper explores
whether post-training techniques, specifically Supervised Fine-Tuning (SFT) and
Reinforcement Learning with Verifiable Rewards (RLVR), can effectively
generalize to multi-agent scenarios. We use economic reasoning as a
testbed, leveraging its strong foundations in mathematics and game theory, its
demand for structured analytical reasoning, and its relevance to real-world
applications such as market design, resource allocation, and policy analysis.
We introduce Recon (Reasoning like an
ECONomist), a 7B-parameter open-source LLM post-trained on a
hand-curated dataset of 2,100 high-quality economic reasoning problems.
Comprehensive evaluation on economic reasoning benchmarks and multi-agent games
reveals clear improvements in structured reasoning and economic rationality.
These results underscore the promise of domain-aligned post-training for
enhancing reasoning and agent alignment, shedding light on the roles of SFT and
RL in shaping model behavior. Code is available at
https://github.com/MasterZhou1/Recon .
SATA-BENCH: Select All That Apply Benchmark for Multiple Choice
  Questions
Authors: name: Weijie Xu, name: Shixian Cui, name: Xi Fang, name: Chi Xue, name: Stephanie Eckman, name: Chandan Reddy
Published: 2025-05-31 17:14:21

Large language models (LLMs) are increasingly evaluated on single-answer
multiple-choice tasks, yet many real-world problems require identifying all
correct answers from a set of options. This capability remains underexplored.
We introduce SATA-BENCH, the first dedicated benchmark for evaluating LLMs on
Select All That Apply (SATA) questions across diverse domains, including
reading comprehension, law, and biomedicine. Our evaluation of 27 open-source
and proprietary models reveals a significant gap: even the strongest model
achieves only 41.8% exact match, exposing LLMs' inability to reliably identify
all correct answers. We find that this weakness stems from two core challenges:
selection bias - models favor certain choices regardless of content, and count
bias - models fail to predict the correct number of answers. To address these
issues, we propose Choice Funnel, a decoding strategy that combines token
debiasing with adaptive thresholding to guide models toward complete and
accurate selections. Choice Funnel achieves up to 29% higher exact match than
competitive baselines while reducing inference cost by over 64%. Our findings
expose fundamental limitations in current LLMs and introduce a new framework
for diagnosing and improving multi-answer reasoning. We release SATA-BENCH and
Choice Funnel to promote LLM development for robust decision-making in
realistic, multi-answer applications.
VisualSphinx: Large-Scale Synthetic Vision Logic Puzzles for RL
Authors: name: Yichen Feng, name: Zhangchen Xu, name: Fengqing Jiang, name: Yuetai Li, name: Bhaskar Ramasubramanian, name: Luyao Niu, name: Bill Yuchen Lin, name: Radha Poovendran
Published: 2025-05-29 20:08:36

Vision language models (VLMs) are expected to perform effective multimodal
reasoning and make logically coherent decisions, which is critical to tasks
such as diagram understanding and spatial problem solving. However, current VLM
reasoning lacks large-scale and well-structured training datasets. To bridge
this gap, we propose VisualSphinx, a first-of-its-kind large-scale synthetic
visual logical reasoning training data. To tackle the challenge of image
synthesis with grounding answers, we propose a rule-to-image synthesis
pipeline, which extracts and expands puzzle rules from seed questions and
generates the code of grounding synthesis image synthesis for puzzle sample
assembly. Experiments demonstrate that VLM trained using GRPO on VisualSphinx
benefit from logical coherence and readability of our dataset and exhibit
improved performance on logical reasoning tasks. The enhanced reasoning
capabilities developed from VisualSphinx also benefit other reasoning tasks
such as algebraic reasoning, arithmetic reasoning and geometry reasoning.
Taming LLMs by Scaling Learning Rates with Gradient Grouping
Authors: name: Siyuan Li, name: Juanxi Tian, name: Zedong Wang, name: Xin Jin, name: Zicheng Liu, name: Wentao Zhang, name: Dan Xu
Published: 2025-06-01 15:30:37

Training large language models (LLMs) poses challenges due to their massive
scale and heterogeneous architectures. While adaptive optimizers like AdamW
help address gradient variations, they still struggle with efficient and
effective parameter-wise learning rate estimation, resulting in training
instability, slow convergence, and poor compatibility with parameter-efficient
fine-tuning (PEFT) techniques. This work introduces Scaling with Gradient
Grouping (SGG), an optimizer wrapper that improves adaptive learning rate
estimation by dynamic grouping and group-specific scaling. SGG first groups
gradient statistics in each layer into clusters and then applies
cluster-specific scaling to calibrate learning rates for each parameter, thus
imposing collective group-wise constraints while maintaining precise
per-parameter adaptation. Experiments on diverse (M)LLM benchmarks show that
SGG integrates seamlessly with existing optimizers, and offers consistent gains
and faster convergence over baselines, with various model sizes. Its stability
across varying batch sizes and learning rates establishes SGG as a robust
choice for LLM optimization.
Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision
  Geometry Priors
Authors: name: Duo Zheng, name: Shijia Huang, name: Yanyang Li, name: Liwei Wang
Published: 2025-05-30 14:16:41

Previous research has investigated the application of Multimodal Large
Language Models (MLLMs) in understanding 3D scenes by interpreting them as
videos. These approaches generally depend on comprehensive 3D data inputs, such
as point clouds or reconstructed Bird's-Eye View (BEV) maps. In our research,
we advance this field by enhancing the capability of MLLMs to understand and
reason in 3D spaces directly from video data, without the need for additional
3D input. We propose a novel and efficient method, the Video-3D Geometry Large
Language Model (VG LLM). Our approach employs a 3D visual geometry encoder that
extracts 3D prior information from video sequences. This information is
integrated with visual tokens and fed into the MLLM. Extensive experiments have
shown that our method has achieved substantial improvements in various tasks
related to 3D scene understanding and spatial reasoning, all directly learned
from video sources. Impressively, our 4B model, which does not rely on explicit
3D data inputs, achieves competitive results compared to existing
state-of-the-art methods, and even surpasses the Gemini-1.5-Pro in the
VSI-Bench evaluations.