Orionnnn/hub-stats huggingface dataset:
R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model
Authors: name: Hengguang Zhou, name: Xirui Li, name: Ruochen Wang, name: Minhao Cheng, name: Tianyi Zhou, name: Cho-Jui Hsieh
Published: 2025-03-07 04:21:47

Recently DeepSeek R1 demonstrated how reinforcement learning with simple
rule-based incentives can enable autonomous development of complex reasoning in
large language models, characterized by the "aha moment", in which the model
manifest self-reflection and increased response length during training.
However, attempts to extend this success to multimodal reasoning often failed
to reproduce these key characteristics. In this report, we present the first
successful replication of these emergent characteristics for multimodal
reasoning on only a non-SFT 2B model. Starting with Qwen2-VL-2B and applying
reinforcement learning directly on the SAT dataset, our model achieves 59.47%
accuracy on CVBench, outperforming the base model by approximately ~30% and
exceeding both SFT setting by ~2%. In addition, we share our failed attempts
and insights in attempting to achieve R1-like reasoning using RL with instruct
models. aiming to shed light on the challenges involved. Our key observations
include: (1) applying RL on instruct model often results in trivial reasoning
trajectories, and (2) naive length reward are ineffective in eliciting
reasoning capabilities. The project code is available at
https://github.com/turningpoint-ai/VisualThinker-R1-Zero
TinyR1-32B-Preview: Boosting Accuracy with Branch-Merge Distillation
Authors: name: Lin Sun, name: Guangxiang Zhao, name: Xiaoqi Jian, name: Yuhan Wu, name: Weihong Lin, name: Yongfu Zhu, name: Change Jia, name: Linglin Zhang, name: Jinzhu Wu, name: Junfeng Ran, name: Sai-er Hu, name: Zihan Jiang, name: Junting Zhou, name: Wenrui Liu, name: Bin Cui, name: Tong Yang, name: Xiangzheng Zhang
Published: 2025-03-06 16:25:53

The challenge of reducing the size of Large Language Models (LLMs) while
maintaining their performance has gained significant attention. However,
existing methods, such as model distillation and transfer learning, often fail
to achieve high accuracy. To address this limitation, we introduce the
Branch-Merge distillation approach, which enhances model compression through
two phases: (1) the Branch Phase, where knowledge from a large teacher model is
selectively distilled into specialized student models via
domain-specific supervised fine-tuning (SFT); And (2) the Merge Phase, where
these student models are merged to enable cross-domain knowledge transfer and
improve generalization. We validate our distillation approach using DeepSeek-R1
as the teacher and DeepSeek-R1-Distill-Qwen-32B as the student. The resulting
merged model, TinyR1-32B-Preview, outperforms its counterpart
DeepSeek-R1-Distill-Qwen-32B across multiple benchmarks, including Mathematics
(+5.5 points), Coding (+4.4 points) and Science (+2.9 points), while achieving
near-equal performance to DeepSeek-R1 on AIME 2024. The Branch-Merge
distillation approach provides a scalable solution for creating smaller,
high-performing LLMs with reduced computational cost and time.
Learning from Failures in Multi-Attempt Reinforcement Learning
Authors: name: Stephen Chung, name: Wenyu Du, name: Jie Fu
Published: 2025-03-04 02:53:39

Recent advancements in reinforcement learning (RL) for large language models
(LLMs), exemplified by DeepSeek R1, have shown that even a simple
question-answering task can substantially improve an LLM's reasoning
capabilities. In this work, we extend this approach by modifying the task into
a multi-attempt setting. Instead of generating a single response per question,
the model is given multiple attempts, with feedback provided after incorrect
responses. The multi-attempt task encourages the model to refine its previous
attempts and improve search efficiency. Experimental results show that even a
small LLM trained on a multi-attempt task achieves significantly higher
accuracy when evaluated with more attempts, improving from 45.6% with 1 attempt
to 52.5% with 2 attempts on the math benchmark. In contrast, the same LLM
trained on a standard single-turn task exhibits only a marginal improvement,
increasing from 42.3% to 43.2% when given more attempts during evaluation. The
results indicate that, compared to the standard single-turn task, an LLM
trained on a multi-attempt task achieves slightly better performance on math
benchmarks while also learning to refine its responses more effectively based
on user feedback. Full code is available at
https://github.com/DualityRL/multi-attempt
BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation
  for Everyday Household Activities
Authors: name: Yunfan Jiang, name: Ruohan Zhang, name: Josiah Wong, name: Chen Wang, name: Yanjie Ze, name: Hang Yin, name: Cem Gokmen, name: Shuran Song, name: Jiajun Wu, name: Li Fei-Fei
Published: 2025-03-07 18:15:21

Real-world household tasks present significant challenges for mobile
manipulation robots. An analysis of existing robotics benchmarks reveals that
successful task performance hinges on three key whole-body control
capabilities: bimanual coordination, stable and precise navigation, and
extensive end-effector reachability. Achieving these capabilities requires
careful hardware design, but the resulting system complexity further
complicates visuomotor policy learning. To address these challenges, we
introduce the BEHAVIOR Robot Suite (BRS), a comprehensive framework for
whole-body manipulation in diverse household tasks. Built on a bimanual,
wheeled robot with a 4-DoF torso, BRS integrates a cost-effective whole-body
teleoperation interface for data collection and a novel algorithm for learning
whole-body visuomotor policies. We evaluate BRS on five challenging household
tasks that not only emphasize the three core capabilities but also introduce
additional complexities, such as long-range navigation, interaction with
articulated and deformable objects, and manipulation in confined spaces. We
believe that BRS's integrated robotic embodiment, data collection interface,
and learning framework mark a significant step toward enabling real-world
whole-body manipulation for everyday household tasks. BRS is open-sourced at
https://behavior-robot-suite.github.io/
Sketch-of-Thought: Efficient LLM Reasoning with Adaptive
  Cognitive-Inspired Sketching
Authors: name: Simon A. Aytes, name: Jinheon Baek, name: Sung Ju Hwang
Published: 2025-03-07 06:57:17

Recent advances in large language models have demonstrated remarkable
reasoning capabilities through Chain of Thought (CoT) prompting, but often at
the cost of excessive verbosity in their intermediate outputs, which increases
computational overhead. We introduce Sketch-of-Thought (SoT), a novel prompting
framework that combines cognitive-inspired reasoning paradigms with linguistic
constraints to minimize token usage while preserving reasoning accuracy. SoT is
designed as a flexible framework that can incorporate any custom reasoning
paradigms based on cognitive science, and we instantiate it with three such
paradigms - Conceptual Chaining, Chunked Symbolism, and Expert Lexicons - each
tailored to different reasoning tasks and selected dynamically via a
lightweight routing model. Through comprehensive evaluation across 15 reasoning
datasets with multiple languages and multimodal scenarios, we demonstrate that
SoT achieves token reductions of 76% with negligible accuracy impact. In
certain domains like mathematical and multi-hop reasoning, it even improves
accuracy while using significantly fewer tokens. Our code is publicly
available: https://www.github.com/SimonAytes/SoT.
Unified Reward Model for Multimodal Understanding and Generation
Authors: name: Yibin Wang, name: Yuhang Zang, name: Hao Li, name: Cheng Jin, name: Jiaqi Wang
Published: 2025-03-07 08:36:05

Recent advances in human preference alignment have significantly enhanced
multimodal generation and understanding. A key approach is training reward
models to guide preference optimization. However, existing models are often
task-specific, limiting their adaptability across diverse visual applications.
We also argue that jointly learning to assess multiple tasks may foster a
synergistic effect, where improved image understanding enhances image
generation assessment, and refined image evaluation benefits video assessment
through better frame analysis. To this end, this paper proposes UnifiedReward,
the first unified reward model for multimodal understanding and generation
assessment, enabling both pairwise ranking and pointwise scoring, which can be
employed for vision model preference alignment. Specifically, (1) we first
develop UnifiedReward on our constructed large-scale human preference dataset,
including both image and video generation/understanding tasks. (2) Then, it is
utilized to automatically construct high-quality preference pair data based on
the vision models, fine-gradually filtering their outputs through pair ranking
and point sifting. (3) Finally, these data are used for their preference
alignment through Direct Preference Optimization (DPO). Experimental results
demonstrate that joint learning to assess diverse visual tasks can lead to
substantial mutual benefits and we apply our pipeline to both image and video
understanding/generation tasks, significantly improving the performance in each
domain.
Forgetting Transformer: Softmax Attention with a Forget Gate
Authors: name: Zhixuan Lin, name: Evgenii Nikishin, name: Xu Owen He, name: Aaron Courville
Published: 2025-03-03 23:35:23

An essential component of modern recurrent sequence models is the forget
gate. While Transformers do not have an explicit recurrent form, we show that a
forget gate can be naturally incorporated into Transformers by down-weighting
the unnormalized attention scores in a data-dependent way. We name this
attention mechanism the Forgetting Attention and the resulting model the
Forgetting Transformer (FoX). We show that FoX outperforms the Transformer on
long-context language modeling, length extrapolation, and short-context
downstream tasks, while performing on par with the Transformer on long-context
downstream tasks. Moreover, it is compatible with the FlashAttention algorithm
and does not require any positional embeddings. Several analyses, including the
needle-in-the-haystack test, show that FoX also retains the Transformer's
superior long-context capabilities over recurrent sequence models such as
Mamba-2, HGRN2, and DeltaNet. We also introduce a "Pro" block design that
incorporates some common architectural components in recurrent sequence models
and find it significantly improves the performance of both FoX and the
Transformer. Our code is available at
https://github.com/zhixuan-lin/forgetting-transformer.
On the Acquisition of Shared Grammatical Representations in Bilingual
  Language Models
Authors: name: Catherine Arnett, name: Tyler A. Chang, name: James A. Michaelov, name: Benjamin K. Bergen
Published: 2025-03-05 23:27:58

While crosslingual transfer is crucial to contemporary language models'
multilingual capabilities, how it occurs is not well understood. In this paper,
we ask what happens to a monolingual language model when it begins to be
trained on a second language. Specifically, we train small bilingual models for
which we control the amount of data for each language and the order of language
exposure. To find evidence of shared multilingual representations, we turn to
structural priming, a method used to study grammatical representations in
humans. We first replicate previous crosslingual structural priming results and
find that after controlling for training data quantity and language exposure,
there are asymmetrical effects across language pairs and directions. We argue
that this asymmetry may shape hypotheses about human structural priming
effects. We also find that structural priming effects are less robust for less
similar language pairs, highlighting potential limitations of crosslingual
transfer learning and shared representations for typologically diverse
languages.
How to Steer LLM Latents for Hallucination Detection?
Authors: name: Seongheon Park, name: Xuefeng Du, name: Min-Hsuan Yeh, name: Haobo Wang, name: Yixuan Li
Published: 2025-03-01 19:19:34

Hallucinations in LLMs pose a significant concern to their safe deployment in
real-world applications. Recent approaches have leveraged the latent space of
LLMs for hallucination detection, but their embeddings, optimized for
linguistic coherence rather than factual accuracy, often fail to clearly
separate truthful and hallucinated content. To this end, we propose the
Truthfulness Separator Vector (TSV), a lightweight and flexible steering vector
that reshapes the LLM's representation space during inference to enhance the
separation between truthful and hallucinated outputs, without altering model
parameters. Our two-stage framework first trains TSV on a small set of labeled
exemplars to form compact and well-separated clusters. It then augments the
exemplar set with unlabeled LLM generations, employing an optimal
transport-based algorithm for pseudo-labeling combined with a confidence-based
filtering process. Extensive experiments demonstrate that TSV achieves
state-of-the-art performance with minimal labeled data, exhibiting strong
generalization across datasets and providing a practical solution for
real-world LLM applications.
LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLM
Authors: name: Sambal Shikhar, name: Mohammed Irfan Kurpath, name: Sahal Shaji Mullappilly, name: Jean Lahoud, name: Fahad Khan, name: Rao Muhammad Anwer, name: Salman Khan, name: Hisham Cholakkal
Published: 2025-03-06 18:59:38

Recent advancements in speech-to-speech dialogue systems leverage LLMs for
multimodal interactions, yet they remain hindered by fine-tuning requirements,
high computational overhead, and text-speech misalignment. Existing
speech-enabled LLMs often degrade conversational quality by modifying the LLM,
thereby compromising its linguistic capabilities. In contrast, we propose
LLMVoX, a lightweight 30M-parameter, LLM-agnostic, autoregressive streaming TTS
system that generates high-quality speech with low latency, while fully
preserving the capabilities of the base LLM. Our approach achieves a
significantly lower Word Error Rate compared to speech-enabled LLMs, while
operating at comparable latency and UTMOS score. By decoupling speech synthesis
from LLM processing via a multi-queue token streaming system, LLMVoX supports
seamless, infinite-length dialogues. Its plug-and-play design also facilitates
extension to various tasks with different backbones. Furthermore, LLMVoX
generalizes to new languages with only dataset adaptation, attaining a low
Character Error Rate on an Arabic speech task. Additionally, we have integrated
LLMVoX with a Vision-Language Model to create an omni-model with speech, text,
and vision capabilities, without requiring additional multimodal training. Our
code base and project page is available at https://mbzuai-oryx.github.io/LLMVoX .
Union of Experts: Adapting Hierarchical Routing to Equivalently
  Decomposed Transformer
Authors: name: Yujiao Yang, name: Jing Lian, name: Linhui Li
Published: 2025-03-04 11:01:25

Mixture-of-Experts (MoE) enhances model performance while maintaining
computational efficiency, making it well-suited for large-scale applications.
However, expert in exist MoE paradigm works as an individual, thereby lacking
high-quality expert interactions. Moreover, they have not been effectively
extended to attention block, which constrains further efficiency improvements.
To tackle these issues, we propose Union-of-Experts (UoE), which decomposes
transformer into an equitant group of experts, and then implement dynamic
routing on input data and experts. Our approach advances MoE design with three
key innovations: (1) We conducted equitant expert decomposition on both MLP
blocks and attention blocks based on matrix partition in tensor parallelism.
(2) We developed two routing paradigms: patch wise data selection and expert
selection, to apply routing across different levels. (3) We design the
architecture of UoE model, including Selective Multi-Head Attention (SMHA) and
Union-of-MLP-Experts (UoME). (4) We develop parallel implementation of UoE's
routing and computation operation, and optimize efficiency based on the
hardware processing analysis. The experiments demonstrate that the model
employed with UoE surpass Full Attention, state-of-art MoEs and efficient
transformers in several tasks across image and natural language domains. The
source codes are available at https://github.com/YujiaoYang-work/UoE.
Lost in Literalism: How Supervised Training Shapes Translationese in
  LLMs
Authors: name: Yafu Li, name: Ronghao Zhang, name: Zhilin Wang, name: Huajian Zhang, name: Leyang Cui, name: Yongjing Yin, name: Tong Xiao, name: Yue Zhang
Published: 2025-03-06 12:14:45

Large language models (LLMs) have achieved remarkable success in machine
translation, demonstrating impressive performance across diverse languages.
However, translationese, characterized by overly literal and unnatural
translations, remains a persistent challenge in LLM-based translation systems.
Despite their pre-training on vast corpora of natural utterances, LLMs exhibit
translationese errors and generate unexpected unnatural translations, stemming
from biases introduced during supervised fine-tuning (SFT). In this work, we
systematically evaluate the prevalence of translationese in LLM-generated
translations and investigate its roots during supervised training. We introduce
methods to mitigate these biases, including polishing golden references and
filtering unnatural training instances. Empirical evaluations demonstrate that
these approaches significantly reduce translationese while improving
translation naturalness, validated by human evaluations and automatic metrics.
Our findings highlight the need for training-aware adjustments to optimize LLM
translation outputs, paving the way for more fluent and
target-language-consistent translations. We release the data and code at
https://github.com/yafuly/LLM_Translationese.
LINGOLY-TOO: Disentangling Memorisation from Reasoning with Linguistic
  Templatisation and Orthographic Obfuscation
Authors: name: Jude Khouja, name: Karolina Korgul, name: Simi Hellsten, name: Lingyi Yang, name: Vlad Neacs, name: Harry Mayne, name: Ryan Kearns, name: Andrew Bean, name: Adam Mahdi
Published: 2025-03-04 19:57:47

Effective evaluation of the reasoning capabilities of large language models
(LLMs) are susceptible to overestimation due to data exposure of evaluation
benchmarks. We introduce a framework for producing linguistic reasoning
problems that reduces the effect of memorisation in model performance estimates
and apply this framework to develop LINGOLY-TOO, a challenging evaluation
benchmark for linguistic reasoning. By developing orthographic templates, we
dynamically obfuscate the writing systems of real languages to generate
numerous question variations. These variations preserve the reasoning steps
required for each solution while reducing the likelihood of specific problem
instances appearing in model training data. Our experiments demonstrate that
frontier models, including OpenAI o1-preview and DeepSeem R1, struggle with
advanced reasoning. Our analysis also shows that LLMs exhibit noticeable
variance in accuracy across permutations of the same problem, and on average
perform better on questions appearing in their original orthography. Our
findings highlight the opaque nature of response generation in LLMs and provide
evidence that prior data exposure contributes to overestimating the reasoning
capabilities of frontier models.
IFIR: A Comprehensive Benchmark for Evaluating Instruction-Following in
  Expert-Domain Information Retrieval
Authors: name: Tingyu Song, name: Guo Gan, name: Mingsheng Shang, name: Yilun Zhao
Published: 2025-03-06 17:32:22

We introduce IFIR, the first comprehensive benchmark designed to evaluate
instruction-following information retrieval (IR) in expert domains. IFIR
includes 2,426 high-quality examples and covers eight subsets across four
specialized domains: finance, law, healthcare, and science literature. Each
subset addresses one or more domain-specific retrieval tasks, replicating
real-world scenarios where customized instructions are critical. IFIR enables a
detailed analysis of instruction-following retrieval capabilities by
incorporating instructions at different levels of complexity. We also propose a
novel LLM-based evaluation method to provide a more precise and reliable
assessment of model performance in following instructions. Through extensive
experiments on 15 frontier retrieval models, including those based on LLMs, our
results reveal that current models face significant challenges in effectively
following complex, domain-specific instructions. We further provide in-depth
analyses to highlight these limitations, offering valuable insights to guide
future advancements in retriever development.
Identifying Sensitive Weights via Post-quantization Integral
Authors: name: Yuezhou Hu, name: Weiyu Huang, name: Zichen Liang, name: Chang Chen, name: Jintao Zhang, name: Jun Zhu, name: Jianfei Chen
Published: 2025-02-28 07:04:19

Serving Large Language Models (LLMs) is costly. However, post-training weight
quantization can address this problem by both compressing their sizes for
limited memory and saving bandwidth for acceleration. As not all weight
dimensions are equally important, those methods typically rely on a sensitivity
metric, which indicates the element-wise influence of weights on loss function
and is used to preprocess original weights for better quantization. In this
work, we conduct an empirical study on the accuracy of the sensitivity metric,
and find that existing gradient and Hessian based metrics are very inaccurate:
they underestimate quantization's impact on the loss function by orders of
magnitude, mainly due to the small convergence radius of local 2nd order
approximation, \ie, gradient and Hessian term in Taylor's formula. To tackle
this problem, we propose Post-quantization Integral (PQI), an accurate metric
to estimate posterior sensitivity in a fine-grained manner. To leverage this
accurate metric, we further propose ReQuant, a simple yet powerful framework
that mainly consists of two Dense-and-Sparse detach components: self-adaptive
outlier selection and step-wise significant weights detach. Results show that
ReQuant boosts state-of-the-art post-training quantization methods, with a
pronounced improvement of 2.66 perplexity gain on Llama 3.2 1B with QTIP.
Combining Flow Matching and Transformers for Efficient Solution of
  Bayesian Inverse Problems
Authors: name: Daniil Sherki, name: Ivan Oseledets, name: Ekaterina Muravleva
Published: 2025-03-03 10:17:56

Solving Bayesian inverse problems efficiently remains a significant challenge
due to the complexity of posterior distributions and the computational cost of
traditional sampling methods. Given a series of observations and the forward
model, we want to recover the distribution of the parameters, conditioned on
observed experimental data. We show, that combining Conditional Flow Mathching
(CFM) with transformer-based architecture, we can efficiently sample from such
kind of distribution, conditioned on variable number of observations.