Orionnnn/hub-stats huggingface dataset:
Reward Models Enable Scalable Code Verification by Trading Accuracy for
  Throughput
Authors: name: Gabriel Orlanski, name: Nicholas Roberts, name: Aws Albarghouthi, name: Frederic Sala
Published: 2025-06-11 17:58:21

The standard paradigm for solving coding tasks via large language models
(LLMs) is to generate-then-rank programs, where the latter step uses a verifier
in the ranking process. The growing consensus is that a comprehensive verifier
(e.g., a full test suite) should be prioritized over an outcome reward model
(ORM) whenever possible, with little consideration given to the trade-offs
involved. We aim to challenge this assumption by systematically exploring the
tradeoff between speed and accuracy. We find that ORMs play a crucial role in
scaling verification through trading accuracy for speed, even when a
comprehensive verifier is available. Their value becomes especially apparent
when used in a generate-prune-then-rank approach, where a faster but less
accurate verifier removes incorrect solutions prior to ranking -- leading to a
system that is 11.65x faster while only being 8.33% less accurate than the full
test suite. We analyze the generate-prune-then-rank approach and show that it
works by filtering out incorrect but highly ranked solutions. These findings
enable the design of scalable and accurate program ranking systems.
Mirage-1: Augmenting and Updating GUI Agent with Hierarchical Multimodal
  Skills
Authors: name: Yuquan Xie, name: Zaijing Li, name: Rui Shao, name: Gongwei Chen, name: Kaiwen Zhou, name: Yinchuan Li, name: Dongmei Jiang, name: Liqiang Nie
Published: 2025-06-12 06:21:19

Recent efforts to leverage the Multi-modal Large Language Model (MLLM) as GUI
agents have yielded promising outcomes. However, these agents still struggle
with long-horizon tasks in online environments, primarily due to insufficient
knowledge and the inherent gap between offline and online domains. In this
paper, inspired by how humans generalize knowledge in open-ended environments,
we propose a Hierarchical Multimodal Skills (HMS) module to tackle the issue of
insufficient knowledge. It progressively abstracts trajectories into execution
skills, core skills, and ultimately meta-skills, providing a hierarchical
knowledge structure for long-horizon task planning. To bridge the domain gap,
we propose the Skill-Augmented Monte Carlo Tree Search (SA-MCTS) algorithm,
which efficiently leverages skills acquired in offline environments to reduce
the action search space during online tree exploration. Building on HMS, we
propose Mirage-1, a multimodal, cross-platform, plug-and-play GUI agent. To
validate the performance of Mirage-1 in real-world long-horizon scenarios, we
constructed a new benchmark, AndroidLH. Experimental results show that Mirage-1
outperforms previous agents by 32\%, 19\%, 15\%, and 79\% on AndroidWorld,
MobileMiniWob++, Mind2Web-Live, and AndroidLH, respectively. Project page:
https://cybertronagent.github.io/Mirage-1.github.io/
Feedback Friction: LLMs Struggle to Fully Incorporate External Feedback
Authors: name: Dongwei Jiang, name: Alvin Zhang, name: Andrew Wang, name: Nicholas Andrews, name: Daniel Khashabi
Published: 2025-06-13 16:31:51

Recent studies have shown LLMs possess some ability to improve their
responses when given external feedback. However, it remains unclear how
effectively and thoroughly these models can incorporate extrinsic feedback. In
an ideal scenario, if LLMs receive near-perfect and complete feedback, we would
expect them to fully integrate the feedback and change their incorrect answers
to correct ones. In this paper, we systematically investigate LLMs' ability to
incorporate feedback by designing a controlled experimental environment. For
each problem, a solver model attempts a solution, then a feedback generator
with access to near-complete ground-truth answers produces targeted feedback,
after which the solver tries again. We evaluate this pipeline across a diverse
range of tasks, including math reasoning, knowledge reasoning, scientific
reasoning, and general multi-domain evaluations with state-of-the-art language
models including Claude 3.7 (with and without extended thinking). Surprisingly,
even under these near-ideal conditions, solver models consistently show
resistance to feedback, a limitation that we term FEEDBACK FRICTION. To
mitigate this limitation, we experiment with sampling-based strategies like
progressive temperature increases and explicit rejection of previously
attempted incorrect answers, which yield improvements but still fail to help
models achieve target performance. We also perform a rigorous exploration of
potential causes of FEEDBACK FRICTION, ruling out factors such as model
overconfidence and data familiarity. We hope that highlighting this issue in
LLMs and ruling out several apparent causes will help future research in
self-improvement.
Med-PRM: Medical Reasoning Models with Stepwise, Guideline-verified
  Process Rewards
Authors: name: Jaehoon Yun, name: Jiwoong Sohn, name: Jungwoo Park, name: Hyunjae Kim, name: Xiangru Tang, name: Yanjun Shao, name: Yonghoe Koo, name: Minhyeok Ko, name: Qingyu Chen, name: Mark Gerstein, name: Michael Moor, name: Jaewoo Kang
Published: 2025-06-13 05:36:30

Large language models have shown promise in clinical decision making, but
current approaches struggle to localize and correct errors at specific steps of
the reasoning process. This limitation is critical in medicine, where
identifying and addressing reasoning errors is essential for accurate diagnosis
and effective patient care. We introduce Med-PRM, a process reward modeling
framework that leverages retrieval-augmented generation to verify each
reasoning step against established medical knowledge bases. By verifying
intermediate reasoning steps with evidence retrieved from clinical guidelines
and literature, our model can precisely assess the reasoning quality in a
fine-grained manner. Evaluations on five medical QA benchmarks and two
open-ended diagnostic tasks demonstrate that Med-PRM achieves state-of-the-art
performance, with improving the performance of base models by up to 13.50%
using Med-PRM. Moreover, we demonstrate the generality of Med-PRM by
integrating it in a plug-and-play fashion with strong policy models such as
Meerkat, achieving over 80\% accuracy on MedQA for the first time using
small-scale models of 8 billion parameters. Our code and data are available at:
https://med-prm.github.io/
Prompt Candidates, then Distill: A Teacher-Student Framework for
  LLM-driven Data Annotation
Authors: name: Mingxuan Xia, name: Haobo Wang, name: Yixuan Li, name: Zewei Yu, name: Jindong Wang, name: Junbo Zhao, name: Runze Wu
Published: 2025-06-04 11:42:37

Recently, Large Language Models (LLMs) have demonstrated significant
potential for data annotation, markedly reducing the labor costs associated
with downstream applications. However, existing methods mostly adopt an
aggressive strategy by prompting LLM to determine a single gold label for each
unlabeled sample. Due to the inherent uncertainty within LLMs, they often
produce incorrect labels for difficult samples, severely compromising the data
quality for downstream applications. Motivated by ambiguity aversion in human
behaviors, we propose a novel candidate annotation paradigm wherein large
language models are encouraged to output all possible labels when incurring
uncertainty. To ensure unique labels are provided for downstream tasks, we
develop a teacher-student framework CanDist that distills candidate annotations
with a Small Language Model (SLM). We further provide a rigorous justification
demonstrating that distilling candidate annotations from the teacher LLM offers
superior theoretical guarantees compared to directly using single annotations.
Extensive experiments across six text classification tasks validate the
effectiveness of our proposed method. The source code is available at
https://github.com/MingxuanXia/CanDist.
Infinity Instruct: Scaling Instruction Selection and Synthesis to
  Enhance Language Models
Authors: name: Jijie Li, name: Li Du, name: Hanyu Zhao, name: Bo-wen Zhang, name: Liangdong Wang, name: Boyan Gao, name: Guang Liu, name: Yonghua Lin
Published: 2025-06-09 06:37:15

Large Language Models (LLMs) demonstrate strong performance in real-world
applications, yet existing open-source instruction datasets often concentrate
on narrow domains, such as mathematics or coding, limiting generalization and
widening the gap with proprietary models. To bridge this gap, we introduce
Infinity-Instruct, a high-quality instruction dataset designed to enhance both
foundational and chat capabilities of LLMs through a two-phase pipeline. In
Phase 1, we curate 7.4M high-quality foundational instructions
(InfInstruct-F-7.4M) from over 100M samples using hybrid data selection
techniques. In Phase 2, we synthesize 1.5M high-quality chat instructions
(InfInstruct-G-1.5M) through a two-stage process involving instruction
selection, evolution, and diagnostic filtering. We empirically evaluate
Infinity-Instruct by fine-tuning several open-source models, including Mistral,
LLaMA, Qwen, and Yi, and observe substantial performance gains across both
foundational and instruction following benchmarks, consistently surpassing
official instruction-tuned counterparts. Notably, InfInstruct-LLaMA3.1-70B
outperforms GPT-4-0314 by 8.6\% on instruction following tasks while achieving
comparable foundational performance. These results underscore the synergy
between foundational and chat training and offer new insights into holistic LLM
development. Our
datasethttps://huggingface.co/datasets/BAAI/Infinity-Instruct and
codeshttps://gitee.com/li-touch/infinity-instruct have been publicly
released.
LoRA-Edit: Controllable First-Frame-Guided Video Editing via Mask-Aware
  LoRA Fine-Tuning
Authors: name: Chenjian Gao, name: Lihe Ding, name: Xin Cai, name: Zhanpeng Huang, name: Zibin Wang, name: Tianfan Xue
Published: 2025-06-11 18:03:55

Video editing using diffusion models has achieved remarkable results in
generating high-quality edits for videos. However, current methods often rely
on large-scale pretraining, limiting flexibility for specific edits.
First-frame-guided editing provides control over the first frame, but lacks
flexibility over subsequent frames. To address this, we propose a mask-based
LoRA (Low-Rank Adaptation) tuning method that adapts pretrained Image-to-Video
(I2V) models for flexible video editing. Our approach preserves background
regions while enabling controllable edits propagation. This solution offers
efficient and adaptable video editing without altering the model architecture.
To better steer this process, we incorporate additional references, such as
alternate viewpoints or representative scene states, which serve as visual
anchors for how content should unfold. We address the control challenge using a
mask-driven LoRA tuning strategy that adapts a pre-trained image-to-video model
to the editing context. The model must learn from two distinct sources: the
input video provides spatial structure and motion cues, while reference images
offer appearance guidance. A spatial mask enables region-specific learning by
dynamically modulating what the model attends to, ensuring that each area draws
from the appropriate source. Experimental results show our method achieves
superior video editing performance compared to state-of-the-art methods.
Learning a Continue-Thinking Token for Enhanced Test-Time Scaling
Authors: name: Liran Ringel, name: Elad Tolochinsky, name: Yaniv Romano
Published: 2025-06-12 20:28:54

Test-time scaling has emerged as an effective approach for improving language
model performance by utilizing additional compute at inference time. Recent
studies have shown that overriding end-of-thinking tokens (e.g., replacing
"</think>" with "Wait") can extend reasoning steps and improve accuracy. In
this work, we explore whether a dedicated continue-thinking token can be
learned to trigger extended reasoning. We augment a distilled version of
DeepSeek-R1 with a single learned "<|continue-thinking|>" token, training only
its embedding via reinforcement learning while keeping the model weights
frozen. Our experiments show that this learned token achieves improved accuracy
on standard math benchmarks compared to both the baseline model and a test-time
scaling approach that uses a fixed token (e.g., "Wait") for budget forcing. In
particular, we observe that in cases where the fixed-token approach enhances
the base model's accuracy, our method achieves a markedly greater improvement.
For example, on the GSM8K benchmark, the fixed-token approach yields a 1.3%
absolute improvement in accuracy, whereas our learned-token method achieves a
4.2% improvement over the base model that does not use budget forcing.
JAFAR: Jack up Any Feature at Any Resolution
Authors: name: Paul Couairon, name: Loick Chambon, name: Louis Serrano, name: Jean-Emmanuel Haugeard, name: Matthieu Cord, name: Nicolas Thome
Published: 2025-06-10 20:53:12

Foundation Vision Encoders have become essential for a wide range of dense
vision tasks. However, their low-resolution spatial feature outputs necessitate
feature upsampling to produce the high-resolution modalities required for
downstream tasks. In this work, we introduce JAFAR, a lightweight and flexible
feature upsampler that enhances the spatial resolution of visual features from
any Foundation Vision Encoder to an arbitrary target resolution. JAFAR employs
an attention-based module designed to promote semantic alignment between
high-resolution queries, derived from low-level image features, and
semantically enriched low-resolution keys, using Spatial Feature Transform
(SFT) modulation. Notably, despite the absence of high-resolution supervision,
we demonstrate that learning at low upsampling ratios and resolutions
generalizes remarkably well to significantly higher output scales. Extensive
experiments show that JAFAR effectively recovers fine-grained spatial details
and consistently outperforms existing feature upsampling methods across a
diverse set of downstream tasks. Project page at
https://jafar-upsampler.github.io
SwS: Self-aware Weakness-driven Problem Synthesis in Reinforcement
  Learning for LLM Reasoning
Authors: name: Xiao Liang, name: Zhong-Zhi Li, name: Yeyun Gong, name: Yang Wang, name: Hengyuan Zhang, name: Yelong Shen, name: Ying Nian Wu, name: Weizhu Chen
Published: 2025-06-10 17:02:00

Reinforcement Learning with Verifiable Rewards (RLVR) has proven effective
for training large language models (LLMs) on complex reasoning tasks, such as
mathematical problem solving. A prerequisite for the scalability of RLVR is a
high-quality problem set with precise and verifiable answers. However, the
scarcity of well-crafted human-labeled math problems and limited-verification
answers in existing distillation-oriented synthetic datasets limit their
effectiveness in RL. Additionally, most problem synthesis strategies
indiscriminately expand the problem set without considering the model's
capabilities, leading to low efficiency in generating useful questions. To
mitigate this issue, we introduce a Self-aware Weakness-driven problem
Synthesis framework (SwS) that systematically identifies model deficiencies and
leverages them for problem augmentation. Specifically, we define weaknesses as
questions that the model consistently fails to learn through its iterative
sampling during RL training. We then extract the core concepts from these
failure cases and synthesize new problems to strengthen the model's weak areas
in subsequent augmented training, enabling it to focus on and gradually
overcome its weaknesses. Without relying on external knowledge distillation,
our framework enables robust generalization byempowering the model to
self-identify and address its weaknesses in RL, yielding average performance
gains of 10.0% and 7.7% on 7B and 32B models across eight mainstream reasoning
benchmarks.
Beyond Homogeneous Attention: Memory-Efficient LLMs via
  Fourier-Approximated KV Cache
Authors: name: Xiaoran Liu, name: Siyang He, name: Qiqi Wang, name: Ruixiao Li, name: Yuerong Song, name: Zhigeng Liu, name: Linlin Li, name: Qun Liu, name: Zengfeng Huang, name: Qipeng Guo, name: Ziwei He, name: Xipeng Qiu
Published: 2025-06-13 15:35:54

Large Language Models struggle with memory demands from the growing Key-Value
(KV) cache as context lengths increase. Existing compression methods homogenize
head dimensions or rely on attention-guided token pruning, often sacrificing
accuracy or introducing computational overhead. We propose FourierAttention, a
training-free framework that exploits the heterogeneous roles of transformer
head dimensions: lower dimensions prioritize local context, while upper ones
capture long-range dependencies. By projecting the long-context-insensitive
dimensions onto orthogonal Fourier bases, FourierAttention approximates their
temporal evolution with fixed-length spectral coefficients. Evaluations on
LLaMA models show that FourierAttention achieves the best long-context accuracy
on LongBench and Needle-In-A-Haystack (NIAH). Besides, a custom Triton kernel,
FlashFourierAttention, is designed to optimize memory via streamlined
read-write operations, enabling efficient deployment without performance
compromise.
ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual
  Perception in VLMs
Authors: name: Xiyao Wang, name: Zhengyuan Yang, name: Chao Feng, name: Yongyuan Liang, name: Yuhang Zhou, name: Xiaoyu Liu, name: Ziyi Zang, name: Ming Li, name: Chung-Ching Lin, name: Kevin Lin, name: Linjie Li, name: Furong Huang, name: Lijuan Wang
Published: 2025-06-11 19:16:54

Reinforcement learning (RL) has shown great effectiveness for fine-tuning
large language models (LLMs) using tasks that are challenging yet easily
verifiable, such as math reasoning or code generation. However, extending this
success to visual perception in vision-language models (VLMs) has been impeded
by the scarcity of vision-centric tasks that are simultaneously challenging and
unambiguously verifiable. To this end, we introduce ViCrit (Visual Caption
Hallucination Critic), an RL proxy task that trains VLMs to localize a subtle,
synthetic visual hallucination injected into paragraphs of human-written image
captions. Starting from a 200-word captions, we inject a single, subtle visual
description error-altering a few words on objects, attributes, counts, or
spatial relations-and task the model to pinpoint the corrupted span given the
image and the modified caption. This formulation preserves the full perceptual
difficulty while providing a binary, exact-match reward that is easy to compute
and unambiguous. Models trained with the ViCrit Task exhibit substantial gains
across a variety of VL benchmarks. Crucially, the improvements transfer beyond
natural-image training data to abstract image reasoning and visual math,
showing promises of learning to perceive rather than barely memorizing seen
objects. To facilitate evaluation, we further introduce ViCrit-Bench, a
category-balanced diagnostic benchmark that systematically probes perception
errors across diverse image domains and error types. Together, our results
demonstrate that fine-grained hallucination criticism is an effective and
generalizable objective for enhancing visual perception in VLMs.
Inherently Faithful Attention Maps for Vision Transformers
Authors: name: Ananthu Aniraj, name: Cassio F. Dantas, name: Dino Ienco, name: Diego Marcos
Published: 2025-06-10 15:41:22

We introduce an attention-based method that uses learned binary attention
masks to ensure that only attended image regions influence the prediction.
Context can strongly affect object perception, sometimes leading to biased
representations, particularly when objects appear in out-of-distribution
backgrounds. At the same time, many image-level object-centric tasks require
identifying relevant regions, often requiring context. To address this
conundrum, we propose a two-stage framework: stage 1 processes the full image
to discover object parts and identify task-relevant regions, while stage 2
leverages input attention masking to restrict its receptive field to these
regions, enabling a focused analysis while filtering out potentially spurious
information. Both stages are trained jointly, allowing stage 2 to refine stage
1. Extensive experiments across diverse benchmarks demonstrate that our
approach significantly improves robustness against spurious correlations and
out-of-distribution backgrounds.
A Self-Refining Framework for Enhancing ASR Using TTS-Synthesized Data
Authors: name: Cheng Kang Chou, name: Chan-Jan Hsu, name: Ho-Lam Chung, name: Liang-Hsuan Tseng, name: Hsi-Chun Cheng, name: Yu-Kuan Fu, name: Kuan Po Huang, name: Hung-Yi Lee
Published: 2025-06-10 17:30:32

We propose a self-refining framework that enhances ASR performance with only
unlabeled datasets. The process starts with an existing ASR model generating
pseudo-labels on unannotated speech, which are then used to train a
high-fidelity text-to-speech (TTS) system. Then, synthesized speech text pairs
are bootstrapped into the original ASR system, completing the closed-loop
self-improvement cycle. We demonstrated the effectiveness of the framework on
Taiwanese Mandarin speech. Leveraging 6,000 hours of unlabeled speech, a
moderate amount of text data, and synthetic content from the AI models, we
adapt Whisper-large-v2 into a specialized model, Twister. Twister reduces error
rates by up to 20% on Mandarin and 50% on Mandarin-English code-switching
benchmarks compared to Whisper. Results highlight the framework as a compelling
alternative to pseudo-labeling self-distillation approaches and provides a
practical pathway for improving ASR performance in low-resource or
domain-specific settings.
DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware
  Regressive GRPO
Authors: name: Jinyoung Park, name: Jeehye Na, name: Jinyoung Kim, name: Hyunwoo J. Kim
Published: 2025-06-09 06:15:54

Recent works have demonstrated the effectiveness of reinforcement learning
(RL)-based post-training in enhancing the reasoning capabilities of large
language models (LLMs). In particular, Group Relative Policy Optimization
(GRPO) has shown impressive success by employing a PPO-style reinforcement
algorithm with group-based normalized rewards. However, the application of GRPO
to Video Large Language Models (Video LLMs) has been less studied. In this
paper, we explore GRPO for video LLMs and identify two primary issues that
impede its effective learning: (1) reliance on safeguards, and (2) the
vanishing advantage problem. To mitigate these challenges, we propose
DeepVideo-R1, a video large language model trained with our proposed Reg-GRPO
(Regressive GRPO) and difficulty-aware data augmentation strategy. Reg-GRPO
reformulates the GRPO objective as a regression task, directly predicting the
advantage in GRPO. This design eliminates the need for safeguards like clipping
and min functions, thereby facilitating more direct policy guidance by aligning
the model with the advantage values. We also design the difficulty-aware data
augmentation strategy that dynamically augments training samples at solvable
difficulty levels, fostering diverse and informative reward signals. Our
comprehensive experiments show that DeepVideo-R1 significantly improves video
reasoning performance across multiple video reasoning benchmarks.