Orionnnn/hub-stats huggingface dataset:
Speechless: Speech Instruction Training Without Speech for Low Resource
  Languages
Authors: name: Alan Dao, name: Dinh Bach Vu, name: Huy Hoang Ha, name: Tuan Le Duc Anh, name: Shreyas Gopal, name: Yue Heng Yeo, name: Warren Keng Hoong Low, name: Eng Siong Chng, name: Jia Qi Yip
Published: 2025-05-23 03:05:47

The rapid growth of voice assistants powered by large language models (LLM)
has highlighted a need for speech instruction data to train these systems.
Despite the abundance of speech recognition data, there is a notable scarcity
of speech instruction data, which is essential for fine-tuning models to
understand and execute spoken commands. Generating high-quality synthetic
speech requires a good text-to-speech (TTS) model, which may not be available
to low resource languages. Our novel approach addresses this challenge by
halting synthesis at the semantic representation level, bypassing the need for
TTS. We achieve this by aligning synthetic semantic representations with the
pre-trained Whisper encoder, enabling an LLM to be fine-tuned on text
instructions while maintaining the ability to understand spoken instructions
during inference. This simplified training process is a promising approach to
building voice assistant for low-resource languages.
One RL to See Them All: Visual Triple Unified Reinforcement Learning
Authors: name: Yan Ma, name: Linge Du, name: Xuyang Shen, name: Shaoxiang Chen, name: Pengfei Li, name: Qibing Ren, name: Lizhuang Ma, name: Yuchao Dai, name: Pengfei Liu, name: Junjie Yan
Published: 2025-05-23 17:41:14

Reinforcement learning (RL) has significantly advanced the reasoning
capabilities of vision-language models (VLMs). However, the use of RL beyond
reasoning tasks remains largely unexplored, especially for perceptionintensive
tasks like object detection and grounding. We propose V-Triune, a Visual Triple
Unified Reinforcement Learning system that enables VLMs to jointly learn visual
reasoning and perception tasks within a single training pipeline. V-Triune
comprises triple complementary components: Sample-Level Data Formatting (to
unify diverse task inputs), Verifier-Level Reward Computation (to deliver
custom rewards via specialized verifiers) , and Source-Level Metric Monitoring
(to diagnose problems at the data-source level). We further introduce a novel
Dynamic IoU reward, which provides adaptive, progressive, and definite feedback
for perception tasks handled by V-Triune. Our approach is instantiated within
off-the-shelf RL training framework using open-source 7B and 32B backbone
models. The resulting model, dubbed Orsta (One RL to See Them All),
demonstrates consistent improvements across both reasoning and perception
tasks. This broad capability is significantly shaped by its training on a
diverse dataset, constructed around four representative visual reasoning tasks
(Math, Puzzle, Chart, and Science) and four visual perception tasks (Grounding,
Detection, Counting, and OCR). Subsequently, Orsta achieves substantial gains
on MEGA-Bench Core, with improvements ranging from +2.1 to an impressive +14.1
across its various 7B and 32B model variants, with performance benefits
extending to a wide range of downstream tasks. These results highlight the
effectiveness and scalability of our unified RL approach for VLMs. The V-Triune
system, along with the Orsta models, is publicly available at
https://github.com/MiniMax-AI.
Transformer Copilot: Learning from The Mistake Log in LLM Fine-tuning
Authors: name: Jiaru Zou, name: Yikun Ban, name: Zihao Li, name: Yunzhe Qi, name: Ruizhong Qiu, name: Ling Yang, name: Jingrui He
Published: 2025-05-22 06:00:45

Large language models are typically adapted to downstream tasks through
supervised fine-tuning on domain-specific data. While standard fine-tuning
focuses on minimizing generation loss to optimize model parameters, we take a
deeper step by retaining and leveraging the model's own learning signals,
analogous to how human learners reflect on past mistakes to improve future
performance. We first introduce the concept of Mistake Log to systematically
track the model's learning behavior and recurring errors throughout
fine-tuning. Treating the original transformer-based model as the Pilot, we
correspondingly design a Copilot model to refine the Pilot's inference
performance via logits rectification. We name the overall Pilot-Copilot
framework the Transformer Copilot, which introduces (i) a novel Copilot model
design, (ii) a joint training paradigm where the Copilot continuously learns
from the evolving Mistake Log alongside the Pilot, and (iii) a fused inference
paradigm where the Copilot rectifies the Pilot's logits for enhanced
generation. We provide both theoretical and empirical analyses on our new
learning framework. Experiments on 12 benchmarks spanning commonsense,
arithmetic, and recommendation tasks demonstrate that Transformer Copilot
consistently improves performance by up to 34.5%, while introducing marginal
computational overhead to Pilot models and exhibiting strong scalability and
transferability.
Trinity-RFT: A General-Purpose and Unified Framework for Reinforcement
  Fine-Tuning of Large Language Models
Authors: name: Xuchen Pan, name: Yanxi Chen, name: Yushuo Chen, name: Yuchang Sun, name: Daoyuan Chen, name: Wenhao Zhang, name: Yuexiang Xie, name: Yilun Huang, name: Yilei Zhang, name: Dawei Gao, name: Yaliang Li, name: Bolin Ding, name: Jingren Zhou
Published: 2025-05-23 12:41:09

Trinity-RFT is a general-purpose, flexible and scalable framework designed
for reinforcement fine-tuning (RFT) of large language models. It is built with
a decoupled design, consisting of (1) an RFT-core that unifies and generalizes
synchronous/asynchronous, on-policy/off-policy, and online/offline modes of
RFT, (2) seamless integration for agent-environment interaction with high
efficiency and robustness, and (3) systematic data pipelines optimized for RFT.
Trinity-RFT can be easily adapted for diverse application scenarios, and serves
as a unified platform for exploring advanced reinforcement learning paradigms.
This technical report outlines the vision, features, design and implementations
of Trinity-RFT, accompanied by extensive examples demonstrating the utility and
user-friendliness of the proposed framework.
On the Design of KL-Regularized Policy Gradient Algorithms for LLM
  Reasoning
Authors: name: Yifan Zhang, name: Yifeng Liu, name: Huizhuo Yuan, name: Yang Yuan, name: Quanquan Gu, name: Andrew C Yao
Published: 2025-05-23 06:01:21

Policy gradient algorithms have been successfully applied to enhance the
reasoning capabilities of large language models (LLMs). Despite the widespread
use of Kullback-Leibler (KL) regularization in policy gradient algorithms to
stabilize training, the systematic exploration of how different KL divergence
formulations can be estimated and integrated into surrogate loss functions for
online reinforcement learning (RL) presents a nuanced and systematically
explorable design space. In this paper, we propose regularized policy gradient
(RPG), a systematic framework for deriving and analyzing KL-regularized policy
gradient methods in the online RL setting. We derive policy gradients and
corresponding surrogate loss functions for objectives regularized by both
forward and reverse KL divergences, considering both normalized and
unnormalized policy distributions. Furthermore, we present derivations for
fully differentiable loss functions as well as REINFORCE-style gradient
estimators, accommodating diverse algorithmic needs. We conduct extensive
experiments on RL for LLM reasoning using these methods, showing improved or
competitive results in terms of training stability and performance compared to
strong baselines such as GRPO, REINFORCE++, and DAPO. The code is available at
https://github.com/complex-reasoning/RPG.
Teaching Large Language Models to Maintain Contextual Faithfulness via
  Synthetic Tasks and Reinforcement Learning
Authors: name: Shuzheng Si, name: Haozhe Zhao, name: Cheng Gao, name: Yuzhuo Bai, name: Zhitong Wang, name: Bofei Gao, name: Kangyang Luo, name: Wenhao Li, name: Yufei Huang, name: Gang Chen, name: Fanchao Qi, name: Minjia Zhang, name: Baobao Chang, name: Maosong Sun
Published: 2025-05-22 10:10:07

Teaching large language models (LLMs) to be faithful in the provided context
is crucial for building reliable information-seeking systems. Therefore, we
propose a systematic framework, CANOE, to improve the faithfulness of LLMs in
both short-form and long-form generation tasks without human annotations.
Specifically, we first synthesize short-form question-answering (QA) data with
four diverse tasks to construct high-quality and easily verifiable training
data without human annotation. Also, we propose Dual-GRPO, a rule-based
reinforcement learning method that includes three tailored rule-based rewards
derived from synthesized short-form QA data, while simultaneously optimizing
both short-form and long-form response generation. Notably, Dual-GRPO
eliminates the need to manually label preference data to train reward models
and avoids over-optimizing short-form generation when relying only on the
synthesized short-form QA data. Experimental results show that CANOE greatly
improves the faithfulness of LLMs across 11 different downstream tasks, even
outperforming the most advanced LLMs, e.g., GPT-4o and OpenAI o1.
VeriThinker: Learning to Verify Makes Reasoning Model Efficient
Authors: name: Zigeng Chen, name: Xinyin Ma, name: Gongfan Fang, name: Ruonan Yu, name: Xinchao Wang
Published: 2025-05-23 14:17:56

Large Reasoning Models (LRMs) excel at complex tasks using Chain-of-Thought
(CoT) reasoning. However, their tendency to overthinking leads to unnecessarily
lengthy reasoning chains, dramatically increasing inference costs. To mitigate
this issue, we introduce VeriThinker, a novel approach for CoT compression.
Unlike conventional methods that fine-tune LRMs directly on the original
reasoning task using synthetic concise CoT data, we innovatively fine-tune the
model solely through an auxiliary verification task. By training LRMs to
accurately verify the correctness of CoT solutions, the LRMs inherently become
more discerning about the necessity of subsequent self-reflection steps,
thereby effectively suppressing overthinking. Extensive experiments validate
that VeriThinker substantially reduces reasoning chain lengths while
maintaining or even slightly improving accuracy. When applied to
DeepSeek-R1-Distill-Qwen-7B, our approach reduces reasoning tokens on MATH500
from 3790 to 2125 while improving accuracy by 0.8% (94.0% to 94.8%), and on
AIME25, tokens decrease from 14321 to 10287 with a 2.1% accuracy gain (38.7% to
40.8%). Additionally, our experiments demonstrate that VeriThinker can also be
zero-shot generalized to speculative reasoning. Code is available at
https://github.com/czg1225/VeriThinker
Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark
  Study
Authors: name: DongGeon Lee, name: Joonwon Jang, name: Jihae Jeong, name: Hwanjo Yu
Published: 2025-05-21 11:26:40

Rapid deployment of vision-language models (VLMs) magnifies safety risks, yet
most evaluations rely on artificial images. This study asks: How safe are
current VLMs when confronted with meme images that ordinary users share? To
investigate this question, we introduce MemeSafetyBench, a 50,430-instance
benchmark pairing real meme images with both harmful and benign instructions.
Using a comprehensive safety taxonomy and LLM-based instruction generation, we
assess multiple VLMs across single and multi-turn interactions. We investigate
how real-world memes influence harmful outputs, the mitigating effects of
conversational context, and the relationship between model scale and safety
metrics. Our findings demonstrate that VLMs show greater vulnerability to
meme-based harmful prompts than to synthetic or typographic images. Memes
significantly increase harmful responses and decrease refusals compared to
text-only inputs. Though multi-turn interactions provide partial mitigation,
elevated vulnerability persists. These results highlight the need for
ecologically valid evaluations and stronger safety mechanisms.
Model Already Knows the Best Noise: Bayesian Active Noise Selection via
  Attention in Video Diffusion Model
Authors: name: Kwanyoung Kim, name: Sanghyun Kim
Published: 2025-05-23 07:09:10

The choice of initial noise significantly affects the quality and prompt
alignment of video diffusion models, where different noise seeds for the same
prompt can lead to drastically different generations. While recent methods rely
on externally designed priors such as frequency filters or inter-frame
smoothing, they often overlook internal model signals that indicate which noise
seeds are inherently preferable. To address this, we propose ANSE (Active Noise
Selection for Generation), a model-aware framework that selects high-quality
noise seeds by quantifying attention-based uncertainty. At its core is BANSA
(Bayesian Active Noise Selection via Attention), an acquisition function that
measures entropy disagreement across multiple stochastic attention samples to
estimate model confidence and consistency. For efficient inference-time
deployment, we introduce a Bernoulli-masked approximation of BANSA that enables
score estimation using a single diffusion step and a subset of attention
layers. Experiments on CogVideoX-2B and 5B demonstrate that ANSE improves video
quality and temporal coherence with only an 8% and 13% increase in inference
time, respectively, providing a principled and generalizable approach to noise
selection in video diffusion. See our project page:
https://anse-project.github.io/anse-project/
Large Language Models Implicitly Learn to See and Hear Just By Reading
Authors: name: Prateek Verma, name: Mert Pilanci
Published: 2025-05-20 22:20:16

This paper presents a fascinating find: By training an auto-regressive LLM
model on text tokens, the text model inherently develops internally an ability
to understand images and audio, thereby developing the ability to see and hear
just by reading. Popular audio and visual LLM models fine-tune text LLM models
to give text output conditioned on images and audio embeddings. On the other
hand, our architecture takes in patches of images, audio waveforms or tokens as
input. It gives us the embeddings or category labels typical of a
classification pipeline. We show the generality of text weights in aiding audio
classification for datasets FSD-50K and GTZAN. Further, we show this working
for image classification on CIFAR-10 and Fashion-MNIST, as well on image
patches. This pushes the notion of text-LLMs learning powerful internal
circuits that can be utilized by activating necessary connections for various
applications rather than training models from scratch every single time.
Distilling LLM Agent into Small Models with Retrieval and Code Tools
Authors: name: Minki Kang, name: Jongwon Jeong, name: Seanie Lee, name: Jaewoong Cho, name: Sung Ju Hwang
Published: 2025-05-23 08:20:15

Large language models (LLMs) excel at complex reasoning tasks but remain
computationally expensive, limiting their practical deployment. To address
this, recent works have focused on distilling reasoning capabilities into
smaller language models (sLMs) using chain-of-thought (CoT) traces from teacher
LLMs. However, this approach struggles in scenarios requiring rare factual
knowledge or precise computation, where sLMs often hallucinate due to limited
capability. In this work, we propose Agent Distillation, a framework for
transferring not only reasoning capability but full task-solving behavior from
LLM-based agents into sLMs with retrieval and code tools. We improve agent
distillation along two complementary axes: (1) we introduce a prompting method
called first-thought prefix to enhance the quality of teacher-generated
trajectories; and (2) we propose a self-consistent action generation for
improving test-time robustness of small agents. We evaluate our method on eight
reasoning tasks across factual and mathematical domains, covering both
in-domain and out-of-domain generalization. Our results show that sLMs as small
as 0.5B, 1.5B, 3B parameters can achieve performance competitive with next-tier
larger 1.5B, 3B, 7B models fine-tuned using CoT distillation, demonstrating the
potential of agent distillation for building practical, tool-using small
agents. Our code is available at https://github.com/Nardien/agent-distillation.
Teaching with Lies: Curriculum DPO on Synthetic Negatives for
  Hallucination Detection
Authors: name: Shrey Pandit, name: Ashwin Vinod, name: Liu Leqi, name: Ying Ding
Published: 2025-05-23 07:05:09

Aligning large language models (LLMs) to accurately detect hallucinations
remains a significant challenge due to the sophisticated nature of hallucinated
text. Recognizing that hallucinated samples typically exhibit higher deceptive
quality than traditional negative samples, we use these carefully engineered
hallucinations as negative examples in the DPO alignment procedure. Our method
incorporates a curriculum learning strategy, gradually transitioning the
training from easier samples, identified based on the greatest reduction in
probability scores from independent fact checking models, to progressively
harder ones. This structured difficulty scaling ensures stable and incremental
learning. Experimental evaluation demonstrates that our HaluCheck models,
trained with curriculum DPO approach and high quality negative samples,
significantly improves model performance across various metrics, achieving
improvements of upto 24% on difficult benchmarks like MedHallu and HaluEval.
Additionally, HaluCheck models demonstrate robustness in zero-shot settings,
significantly outperforming larger state-of-the-art models across various
benchmarks.
Reasoning Model is Stubborn: Diagnosing Instruction Overriding in
  Reasoning Models
Authors: name: Doohyuk Jang, name: Yoonjeon Kim, name: Chanjae Park, name: Hyun Ryu, name: Eunho Yang
Published: 2025-05-22 19:00:01

Large language models have demonstrated remarkable proficiency in long and
complex reasoning tasks. However, they frequently exhibit a problematic
reliance on familiar reasoning patterns, a phenomenon we term reasoning
rigidity. Despite explicit instructions from users, these models often
override clearly stated conditions and default to habitual reasoning
trajectories, leading to incorrect conclusions. This behavior presents
significant challenges, particularly in domains such as mathematics and logic
puzzle, where precise adherence to specified constraints is critical. To
systematically investigate reasoning rigidity, a behavior largely unexplored in
prior work, we introduce a expert-curated diagnostic set, . Our
dataset includes specially modified variants of existing mathematical
benchmarks, namely AIME and MATH500, as well as well-known puzzles deliberately
redesigned to require deviation from familiar reasoning strategies. Using this
dataset, we identify recurring contamination patterns that occur when models
default to ingrained reasoning. Specifically, we categorize this contamination
into three distinctive modes: (i) Interpretation Overload, (ii) Input Distrust,
and (iii) Partial Instruction Attention, each causing models to ignore or
distort provided instructions. We publicly release our diagnostic set to
facilitate future research on mitigating reasoning rigidity in language models.
gen2seg: Generative Models Enable Generalizable Instance Segmentation
Authors: name: Om Khangaonkar, name: Hamed Pirsiavash
Published: 2025-05-21 08:42:05

By pretraining to synthesize coherent images from perturbed inputs,
generative models inherently learn to understand object boundaries and scene
compositions. How can we repurpose these generative representations for
general-purpose perceptual organization? We finetune Stable Diffusion and MAE
(encoder+decoder) for category-agnostic instance segmentation using our
instance coloring loss exclusively on a narrow set of object types (indoor
furnishings and cars). Surprisingly, our models exhibit strong zero-shot
generalization, accurately segmenting objects of types and styles unseen in
finetuning (and in many cases, MAE's ImageNet-1K pretraining too). Our
best-performing models closely approach the heavily supervised SAM when
evaluated on unseen object types and styles, and outperform it when segmenting
fine structures and ambiguous boundaries. In contrast, existing promptable
segmentation architectures or discriminatively pretrained models fail to
generalize. This suggests that generative models learn an inherent grouping
mechanism that transfers across categories and domains, even without
internet-scale pretraining. Code, pretrained models, and demos are available on
our website.
RoPECraft: Training-Free Motion Transfer with Trajectory-Guided RoPE
  Optimization on Diffusion Transformers
Authors: name: Ahmet Berke Gokmen, name: Yigit Ekin, name: Bahri Batuhan Bilecen, name: Aysegul Dundar
Published: 2025-05-19 16:50:26

We propose RoPECraft, a training-free video motion transfer method for
diffusion transformers that operates solely by modifying their rotary
positional embeddings (RoPE). We first extract dense optical flow from a
reference video, and utilize the resulting motion offsets to warp the
complex-exponential tensors of RoPE, effectively encoding motion into the
generation process. These embeddings are then further optimized during
denoising time steps via trajectory alignment between the predicted and target
velocities using a flow-matching objective. To keep the output faithful to the
text prompt and prevent duplicate generations, we incorporate a regularization
term based on the phase components of the reference video's Fourier transform,
projecting the phase angles onto a smooth manifold to suppress high-frequency
artifacts. Experiments on benchmarks reveal that RoPECraft outperforms all
recently published methods, both qualitatively and quantitatively.
Fixing Data That Hurts Performance: Cascading LLMs to Relabel Hard
  Negatives for Robust Information Retrieval
Authors: name: Nandan Thakur, name: Crystina Zhang, name: Xueguang Ma, name: Jimmy Lin
Published: 2025-05-22 17:47:57

Training robust retrieval and reranker models typically relies on large-scale
retrieval datasets; for example, the BGE collection contains 1.6 million
query-passage pairs sourced from various data sources. However, we find that
certain datasets can negatively impact model effectiveness -- pruning 8 out of
15 datasets from the BGE collection reduces the training set size by
2.35times and increases nDCG@10 on BEIR by 1.0 point. This motivates a
deeper examination of training data quality, with a particular focus on "false
negatives", where relevant passages are incorrectly labeled as irrelevant. We
propose a simple, cost-effective approach using cascading LLM prompts to
identify and relabel hard negatives. Experimental results show that relabeling
false negatives with true positives improves both E5 (base) and Qwen2.5-7B
retrieval models by 0.7-1.4 nDCG@10 on BEIR and by 1.7-1.8 nDCG@10 on zero-shot
AIR-Bench evaluation. Similar gains are observed for rerankers fine-tuned on
the relabeled data, such as Qwen2.5-3B on BEIR. The reliability of the
cascading design is further supported by human annotation results, where we
find judgment by GPT-4o shows much higher agreement with humans than
GPT-4o-mini.