Orionnnn/hub-stats huggingface dataset:
DiSA: Diffusion Step Annealing in Autoregressive Image Generation
Authors: name: Qinyu Zhao, name: Jaskirat Singh, name: Ming Xu, name: Akshay Asthana, name: Stephen Gould, name: Liang Zheng
Published: 2025-05-26 17:59:57

An increasing number of autoregressive models, such as MAR, FlowAR, xAR, and
Harmon adopt diffusion sampling to improve the quality of image generation.
However, this strategy leads to low inference efficiency, because it usually
takes 50 to 100 steps for diffusion to sample a token. This paper explores how
to effectively address this issue. Our key motivation is that as more tokens
are generated during the autoregressive process, subsequent tokens follow more
constrained distributions and are easier to sample. To intuitively explain, if
a model has generated part of a dog, the remaining tokens must complete the dog
and thus are more constrained. Empirical evidence supports our motivation: at
later generation stages, the next tokens can be well predicted by a multilayer
perceptron, exhibit low variance, and follow closer-to-straight-line denoising
paths from noise to tokens. Based on our finding, we introduce diffusion step
annealing (DiSA), a training-free method which gradually uses fewer diffusion
steps as more tokens are generated, e.g., using 50 steps at the beginning and
gradually decreasing to 5 steps at later stages. Because DiSA is derived from
our finding specific to diffusion in autoregressive models, it is complementary
to existing acceleration methods designed for diffusion alone. DiSA can be
implemented in only a few lines of code on existing models, and albeit simple,
achieves 5-10times faster inference for MAR and Harmon and 1.4-2.5times
for FlowAR and xAR, while maintaining the generation quality.
Textual Steering Vectors Can Improve Visual Understanding in Multimodal
  Large Language Models
Authors: name: Woody Haosheng Gan, name: Deqing Fu, name: Julian Asilis, name: Ollie Liu, name: Dani Yogatama, name: Vatsal Sharan, name: Robin Jia, name: Willie Neiswanger
Published: 2025-05-20 08:23:08

Steering methods have emerged as effective and targeted tools for guiding
large language models' (LLMs) behavior without modifying their parameters.
Multimodal large language models (MLLMs), however, do not currently enjoy the
same suite of techniques, due in part to their recency and architectural
diversity. Inspired by this gap, we investigate whether MLLMs can be steered
using vectors derived from their text-only LLM backbone, via sparse
autoencoders (SAEs), mean shift, and linear probing. We find that text-derived
steering consistently enhances multimodal accuracy across diverse MLLM
architectures and visual tasks. In particular, mean shift boosts spatial
relationship accuracy on CV-Bench by up to +7.3% and counting accuracy by up to
+3.3%, outperforming prompting and exhibiting strong generalization to
out-of-distribution datasets. These results highlight textual steering vectors
as a powerful, efficient mechanism for enhancing grounding in MLLMs with
minimal additional data collection and computational overhead.
Bridging Supervised Learning and Reinforcement Learning in Math
  Reasoning
Authors: name: Huayu Chen, name: Kaiwen Zheng, name: Qinsheng Zhang, name: Ganqu Cui, name: Yin Cui, name: Haotian Ye, name: Tsung-Yi Lin, name: Ming-Yu Liu, name: Jun Zhu, name: Haoxiang Wang
Published: 2025-05-23 17:17:40

Reinforcement Learning (RL) has played a central role in the recent surge of
LLMs' math abilities by enabling self-improvement through binary verifier
signals. In contrast, Supervised Learning (SL) is rarely considered for such
verification-driven training, largely due to its heavy reliance on reference
answers and inability to reflect on mistakes. In this work, we challenge the
prevailing notion that self-improvement is exclusive to RL and propose
Negative-aware Fine-Tuning (NFT) -- a supervised approach that enables LLMs to
reflect on their failures and improve autonomously with no external teachers.
In online training, instead of throwing away self-generated negative answers,
NFT constructs an implicit negative policy to model them. This implicit policy
is parameterized with the same positive LLM we target to optimize on positive
data, enabling direct policy optimization on all LLMs' generations. We conduct
experiments on 7B and 32B models in math reasoning tasks. Results consistently
show that through the additional leverage of negative feedback, NFT
significantly improves over SL baselines like Rejection sampling Fine-Tuning,
matching or even surpassing leading RL algorithms like GRPO and DAPO.
Furthermore, we demonstrate that NFT and GRPO are actually equivalent in
strict-on-policy training, even though they originate from entirely different
theoretical foundations. Our experiments and theoretical findings bridge the
gap between SL and RL methods in binary-feedback learning systems.
MMIG-Bench: Towards Comprehensive and Explainable Evaluation of
  Multi-Modal Image Generation Models
Authors: name: Hang Hua, name: Ziyun Zeng, name: Yizhi Song, name: Yunlong Tang, name: Liu He, name: Daniel Aliaga, name: Wei Xiong, name: Jiebo Luo
Published: 2025-05-26 02:07:24

Recent multimodal image generators such as GPT-4o, Gemini 2.0 Flash, and
Gemini 2.5 Pro excel at following complex instructions, editing images and
maintaining concept consistency. However, they are still evaluated by disjoint
toolkits: text-to-image (T2I) benchmarks that lacks multi-modal conditioning,
and customized image generation benchmarks that overlook compositional
semantics and common knowledge. We propose MMIG-Bench, a comprehensive
Multi-Modal Image Generation Benchmark that unifies these tasks by pairing
4,850 richly annotated text prompts with 1,750 multi-view reference images
across 380 subjects, spanning humans, animals, objects, and artistic styles.
MMIG-Bench is equipped with a three-level evaluation framework: (1) low-level
metrics for visual artifacts and identity preservation of objects; (2) novel
Aspect Matching Score (AMS): a VQA-based mid-level metric that delivers
fine-grained prompt-image alignment and shows strong correlation with human
judgments; and (3) high-level metrics for aesthetics and human preference.
Using MMIG-Bench, we benchmark 17 state-of-the-art models, including Gemini 2.5
Pro, FLUX, DreamBooth, and IP-Adapter, and validate our metrics with 32k human
ratings, yielding in-depth insights into architecture and data design. We will
release the dataset and evaluation code to foster rigorous, unified evaluation
and accelerate future innovations in multi-modal image generation.
Hybrid Latent Reasoning via Reinforcement Learning
Authors: name: Zhenrui Yue, name: Bowen Jin, name: Huimin Zeng, name: Honglei Zhuang, name: Zhen Qin, name: Jinsung Yoon, name: Lanyu Shang, name: Jiawei Han, name: Dong Wang
Published: 2025-05-24 01:26:16

Recent advances in large language models (LLMs) have introduced latent
reasoning as a promising alternative to autoregressive reasoning. By performing
internal computation with hidden states from previous steps, latent reasoning
benefit from more informative features rather than sampling a discrete
chain-of-thought (CoT) path. Yet latent reasoning approaches are often
incompatible with LLMs, as their continuous paradigm conflicts with the
discrete nature of autoregressive generation. Moreover, these methods rely on
CoT traces for training and thus fail to exploit the inherent reasoning
patterns of LLMs. In this work, we explore latent reasoning by leveraging the
intrinsic capabilities of LLMs via reinforcement learning (RL). To this end, we
introduce hybrid reasoning policy optimization (HRPO), an RL-based hybrid
latent reasoning approach that (1) integrates prior hidden states into sampled
tokens with a learnable gating mechanism, and (2) initializes training with
predominantly token embeddings while progressively incorporating more hidden
features. This design maintains LLMs' generative capabilities and incentivizes
hybrid reasoning using both discrete and continuous representations. In
addition, the hybrid HRPO introduces stochasticity into latent reasoning via
token sampling, thereby enabling RL-based optimization without requiring CoT
trajectories. Extensive evaluations across diverse benchmarks show that HRPO
outperforms prior methods in both knowledge- and reasoning-intensive tasks.
Furthermore, HRPO-trained LLMs remain interpretable and exhibit intriguing
behaviors like cross-lingual patterns and shorter completion lengths,
highlighting the potential of our RL-based approach and offer insights for
future work in latent reasoning.
Hybrid Neural-MPM for Interactive Fluid Simulations in Real-Time
Authors: name: Jingxuan Xu, name: Hong Huang, name: Chuhang Zou, name: Manolis Savva, name: Yunchao Wei, name: Wuyang Chen
Published: 2025-05-25 01:27:18

We propose a neural physics system for real-time, interactive fluid
simulations. Traditional physics-based methods, while accurate, are
computationally intensive and suffer from latency issues. Recent
machine-learning methods reduce computational costs while preserving fidelity;
yet most still fail to satisfy the latency constraints for real-time use and
lack support for interactive applications. To bridge this gap, we introduce a
novel hybrid method that integrates numerical simulation, neural physics, and
generative control. Our neural physics jointly pursues low-latency simulation
and high physical fidelity by employing a fallback safeguard to classical
numerical solvers. Furthermore, we develop a diffusion-based controller that is
trained using a reverse modeling strategy to generate external dynamic force
fields for fluid manipulation. Our system demonstrates robust performance
across diverse 2D/3D scenarios, material types, and obstacle interactions,
achieving real-time simulations at high frame rates (11~29% latency) while
enabling fluid control guided by user-friendly freehand sketches. We present a
significant step towards practical, controllable, and physically plausible
fluid simulations for real-time interactive applications. We promise to release
both models and data upon acceptance.
InstructPart: Task-Oriented Part Segmentation with Instruction Reasoning
Authors: name: Zifu Wan, name: Yaqi Xie, name: Ce Zhang, name: Zhiqiu Lin, name: Zihan Wang, name: Simon Stepputtis, name: Deva Ramanan, name: Katia Sycara
Published: 2025-05-23 18:36:13

Large multimodal foundation models, particularly in the domains of language
and vision, have significantly advanced various tasks, including robotics,
autonomous driving, information retrieval, and grounding. However, many of
these models perceive objects as indivisible, overlooking the components that
constitute them. Understanding these components and their associated
affordances provides valuable insights into an object's functionality, which is
fundamental for performing a wide range of tasks. In this work, we introduce a
novel real-world benchmark, InstructPart, comprising hand-labeled part
segmentation annotations and task-oriented instructions to evaluate the
performance of current models in understanding and executing part-level tasks
within everyday contexts. Through our experiments, we demonstrate that
task-oriented part segmentation remains a challenging problem, even for
state-of-the-art Vision-Language Models (VLMs). In addition to our benchmark,
we introduce a simple baseline that achieves a twofold performance improvement
through fine-tuning with our dataset. With our dataset and benchmark, we aim to
facilitate research on task-oriented part segmentation and enhance the
applicability of VLMs across various domains, including robotics, virtual
reality, information retrieval, and other related fields. Project website:
https://zifuwan.github.io/InstructPart/.
WHISTRESS: Enriching Transcriptions with Sentence Stress Detection
Authors: name: Iddo Yosha, name: Dorin Shteyman, name: Yossi Adi
Published: 2025-05-25 11:45:08

Spoken language conveys meaning not only through words but also through
intonation, emotion, and emphasis. Sentence stress, the emphasis placed on
specific words within a sentence, is crucial for conveying speaker intent and
has been extensively studied in linguistics. In this work, we introduce
WHISTRESS, an alignment-free approach for enhancing transcription systems with
sentence stress detection. To support this task, we propose TINYSTRESS-15K, a
scalable, synthetic training data for the task of sentence stress detection
which resulted from a fully automated dataset creation process. We train
WHISTRESS on TINYSTRESS-15K and evaluate it against several competitive
baselines. Our results show that WHISTRESS outperforms existing methods while
requiring no additional input priors during training or inference. Notably,
despite being trained on synthetic data, WHISTRESS demonstrates strong
zero-shot generalization across diverse benchmarks. Project page:
https://pages.cs.huji.ac.il/adiyoss-lab/whistress.
Force Prompting: Video Generation Models Can Learn and Generalize
  Physics-based Control Signals
Authors: name: Nate Gillman, name: Charles Herrmann, name: Michael Freeman, name: Daksh Aggarwal, name: Evan Luo, name: Deqing Sun, name: Chen Sun
Published: 2025-05-26 01:04:02

Recent advances in video generation models have sparked interest in world
models capable of simulating realistic environments. While navigation has been
well-explored, physically meaningful interactions that mimic real-world forces
remain largely understudied. In this work, we investigate using physical forces
as a control signal for video generation and propose force prompts which enable
users to interact with images through both localized point forces, such as
poking a plant, and global wind force fields, such as wind blowing on fabric.
We demonstrate that these force prompts can enable videos to respond
realistically to physical control signals by leveraging the visual and motion
prior in the original pretrained model, without using any 3D asset or physics
simulator at inference. The primary challenge of force prompting is the
difficulty in obtaining high quality paired force-video training data, both in
the real world due to the difficulty of obtaining force signals, and in
synthetic data due to limitations in the visual quality and domain diversity of
physics simulators. Our key finding is that video generation models can
generalize remarkably well when adapted to follow physical force conditioning
from videos synthesized by Blender, even with limited demonstrations of few
objects. Our method can generate videos which simulate forces across diverse
geometries, settings, and materials. We also try to understand the source of
this generalization and perform ablations that reveal two key elements: visual
diversity and the use of specific text keywords during training. Our approach
is trained on only around 15k training examples for a single day on four A100
GPUs, and outperforms existing methods on force adherence and physics realism,
bringing world models closer to real-world physics interactions. We release all
datasets, code, weights, and interactive video demos at our project page.
FLAME-MoE: A Transparent End-to-End Research Platform for
  Mixture-of-Experts Language Models
Authors: name: Hao Kang, name: Zichun Yu, name: Chenyan Xiong
Published: 2025-05-26 17:06:25

Recent large language models such as Gemini-1.5, DeepSeek-V3, and Llama-4
increasingly adopt Mixture-of-Experts (MoE) architectures, which offer strong
efficiency-performance trade-offs by activating only a fraction of the model
per token. Yet academic researchers still lack a fully open, end-to-end MoE
platform for investigating scaling, routing, and expert behavior. We release
FLAME-MoE, a completely open-source research suite composed of seven
decoder-only models, ranging from 38M to 1.7B active parameters, whose
architecture--64 experts with top-8 gating and 2 shared experts--closely
reflects modern production LLMs. All training data pipelines, scripts, logs,
and checkpoints are publicly available to enable reproducible experimentation.
Across six evaluation tasks, FLAME-MoE improves average accuracy by up to 3.4
points over dense baselines trained with identical FLOPs. Leveraging full
training trace transparency, we present initial analyses showing that (i)
experts increasingly specialize on distinct token subsets, (ii) co-activation
matrices remain sparse, reflecting diverse expert usage, and (iii) routing
behavior stabilizes early in training. All code, training logs, and model
checkpoints are available at https://github.com/cmu-flame/FLAME-MoE.
EgoZero: Robot Learning from Smart Glasses
Authors: name: Vincent Liu, name: Ademi Adeniji, name: Haotian Zhan, name: Raunaq Bhirangi, name: Pieter Abbeel, name: Lerrel Pinto
Published: 2025-05-26 17:59:17

Despite recent progress in general purpose robotics, robot policies still lag
far behind basic human capabilities in the real world. Humans interact
constantly with the physical world, yet this rich data resource remains largely
untapped in robot learning. We propose EgoZero, a minimal system that learns
robust manipulation policies from human demonstrations captured with Project
Aria smart glasses, and zero robot data. EgoZero enables: (1)
extraction of complete, robot-executable actions from in-the-wild, egocentric,
human demonstrations, (2) compression of human visual observations into
morphology-agnostic state representations, and (3) closed-loop policy learning
that generalizes morphologically, spatially, and semantically. We deploy
EgoZero policies on a gripper Franka Panda robot and demonstrate zero-shot
transfer with 70% success rate over 7 manipulation tasks and only 20 minutes of
data collection per task. Our results suggest that in-the-wild human data can
serve as a scalable foundation for real-world robot learning - paving the way
toward a future of abundant, diverse, and naturalistic training data for
robots. Code and videos are available at https://egozero-robot.github.io.
REARANK: Reasoning Re-ranking Agent via Reinforcement Learning
Authors: name: Le Zhang, name: Bo Wang, name: Xipeng Qiu, name: Siva Reddy, name: Aishwarya Agrawal
Published: 2025-05-26 14:31:48

We present REARANK, a large language model (LLM)-based listwise reasoning
reranking agent. REARANK explicitly reasons before reranking, significantly
improving both performance and interpretability. Leveraging reinforcement
learning and data augmentation, REARANK achieves substantial improvements over
baseline models across popular information retrieval benchmarks, notably
requiring only 179 annotated samples. Built on top of Qwen2.5-7B, our
REARANK-7B demonstrates performance comparable to GPT-4 on both in-domain and
out-of-domain benchmarks and even surpasses GPT-4 on reasoning-intensive BRIGHT
benchmarks. These results underscore the effectiveness of our approach and
highlight how reinforcement learning can enhance LLM reasoning capabilities in
reranking.
UFT: Unifying Supervised and Reinforcement Fine-Tuning
Authors: name: Mingyang Liu, name: Gabriele Farina, name: Asuman Ozdaglar
Published: 2025-05-22 17:53:57

Post-training has demonstrated its importance in enhancing the reasoning
capabilities of large language models (LLMs). The primary post-training methods
can be categorized into supervised fine-tuning (SFT) and reinforcement
fine-tuning (RFT). SFT is efficient and well-suited for small language models,
but it may lead to overfitting and limit the reasoning abilities of larger
models. In contrast, RFT generally yields better generalization but depends
heavily on the strength of the base model. To address the limitations of SFT
and RFT, we propose Unified Fine-Tuning (UFT), a novel post-training paradigm
that unifies SFT and RFT into a single, integrated process. UFT enables the
model to effectively explore solutions while incorporating informative
supervision signals, bridging the gap between memorizing and thinking
underlying existing methods. Notably, UFT outperforms both SFT and RFT in
general, regardless of model sizes. Furthermore, we theoretically prove that
UFT breaks RFT's inherent exponential sample complexity bottleneck, showing for
the first time that unified training can exponentially accelerate convergence
on long-horizon reasoning tasks.
The Birth of Knowledge: Emergent Features across Time, Space, and Scale
  in Large Language Models
Authors: name: Shashata Sawmya, name: Micah Adler, name: Nir Shavit
Published: 2025-05-26 02:59:54

This paper studies the emergence of interpretable categorical features within
large language models (LLMs), analyzing their behavior across training
checkpoints (time), transformer layers (space), and varying model sizes
(scale). Using sparse autoencoders for mechanistic interpretability, we
identify when and where specific semantic concepts emerge within neural
activations. Results indicate clear temporal and scale-specific thresholds for
feature emergence across multiple domains. Notably, spatial analysis reveals
unexpected semantic reactivation, with early-layer features re-emerging at
later layers, challenging standard assumptions about representational dynamics
in transformer models.
Don't "Overthink" Passage Reranking: Is Reasoning Truly Necessary?
Authors: name: Nour Jedidi, name: Yung-Sung Chuang, name: James Glass, name: Jimmy Lin
Published: 2025-05-22 16:41:37

With the growing success of reasoning models across complex natural language
tasks, researchers in the Information Retrieval (IR) community have begun
exploring how similar reasoning capabilities can be integrated into passage
rerankers built on Large Language Models (LLMs). These methods typically employ
an LLM to produce an explicit, step-by-step reasoning process before arriving
at a final relevance prediction. But, does reasoning actually improve reranking
accuracy? In this paper, we dive deeper into this question, studying the impact
of the reasoning process by comparing reasoning-based pointwise rerankers
(ReasonRR) to standard, non-reasoning pointwise rerankers (StandardRR) under
identical training conditions, and observe that StandardRR generally
outperforms ReasonRR. Building on this observation, we then study the
importance of reasoning to ReasonRR by disabling its reasoning process
(ReasonRR-NoReason), and find that ReasonRR-NoReason is surprisingly more
effective than ReasonRR. Examining the cause of this result, our findings
reveal that reasoning-based rerankers are limited by the LLM's reasoning
process, which pushes it toward polarized relevance scores and thus fails to
consider the partial relevance of passages, a key factor for the accuracy of
pointwise rerankers.
STAR-R1: Spatial TrAnsformation Reasoning by Reinforcing Multimodal LLMs
Authors: name: Zongzhao Li, name: Zongyang Ma, name: Mingze Li, name: Songyou Li, name: Yu Rong, name: Tingyang Xu, name: Ziqi Zhang, name: Deli Zhao, name: Wenbing Huang
Published: 2025-05-21 17:57:38

Multimodal Large Language Models (MLLMs) have demonstrated remarkable
capabilities across diverse tasks, yet they lag significantly behind humans in
spatial reasoning. We investigate this gap through Transformation-Driven Visual
Reasoning (TVR), a challenging task requiring identification of object
transformations across images under varying viewpoints. While traditional
Supervised Fine-Tuning (SFT) fails to generate coherent reasoning paths in
cross-view settings, sparse-reward Reinforcement Learning (RL) suffers from
inefficient exploration and slow convergence. To address these limitations, we
propose STAR-R1, a novel framework that integrates a single-stage RL paradigm
with a fine-grained reward mechanism tailored for TVR. Specifically, STAR-R1
rewards partial correctness while penalizing excessive enumeration and passive
inaction, enabling efficient exploration and precise reasoning. Comprehensive
evaluations demonstrate that STAR-R1 achieves state-of-the-art performance
across all 11 metrics, outperforming SFT by 23% in cross-view scenarios.
Further analysis reveals STAR-R1's anthropomorphic behavior and highlights its
unique ability to compare all objects for improving spatial reasoning. Our work
provides critical insights in advancing the research of MLLMs and reasoning
models. The codes, model weights, and data will be publicly available at
https://github.com/zongzhao23/STAR-R1.