Orionnnn/hub-stats huggingface dataset:
Fostering Video Reasoning via Next-Event Prediction
Authors: name: Haonan Wang, name: Hongfu Liu, name: Xiangyan Liu, name: Chao Du, name: Kenji Kawaguchi, name: Ye Wang, name: Tianyu Pang
Published: 2025-05-28 15:13:34

Next-token prediction serves as the foundational learning task enabling
reasoning in LLMs. But what should the learning task be when aiming to equip
MLLMs with temporal reasoning capabilities over video inputs? Existing tasks
such as video question answering often rely on annotations from humans or much
stronger MLLMs, while video captioning tends to entangle temporal reasoning
with spatial information. To address this gap, we propose next-event prediction
(NEP), a learning task that harnesses future video segments as a rich,
self-supervised signal to foster temporal reasoning. We segment each video into
past and future frames: the MLLM takes the past frames as input and predicts a
summary of events derived from the future frames, thereby encouraging the model
to reason temporally in order to complete the task. To support this task, we
curate V1-33K, a dataset comprising 33,000 automatically extracted video
segments spanning diverse real-world scenarios. We further explore a range of
video instruction-tuning strategies to study their effects on temporal
reasoning. To evaluate progress, we introduce FutureBench to assess coherence
in predicting unseen future events. Experiments validate that NEP offers a
scalable and effective training paradigm for fostering temporal reasoning in
MLLMs.
Chain-of-Zoom: Extreme Super-Resolution via Scale Autoregression and
  Preference Alignment
Authors: name: Bryan Sangwoo Kim, name: Jeongsol Kim, name: Jong Chul Ye
Published: 2025-05-24 08:50:08

Modern single-image super-resolution (SISR) models deliver photo-realistic
results at the scale factors on which they are trained, but collapse when asked
to magnify far beyond that regime. We address this scalability bottleneck with
Chain-of-Zoom (CoZ), a model-agnostic framework that factorizes SISR into an
autoregressive chain of intermediate scale-states with multi-scale-aware
prompts. CoZ repeatedly re-uses a backbone SR model, decomposing the
conditional probability into tractable sub-problems to achieve extreme
resolutions without additional training. Because visual cues diminish at high
magnifications, we augment each zoom step with multi-scale-aware text prompts
generated by a vision-language model (VLM). The prompt extractor itself is
fine-tuned using Generalized Reward Policy Optimization (GRPO) with a critic
VLM, aligning text guidance towards human preference. Experiments show that a
standard 4x diffusion SR model wrapped in CoZ attains beyond 256x enlargement
with high perceptual quality and fidelity. Project Page:
https://bryanswkim.github.io/chain-of-zoom/ .
Universal Reasoner: A Single, Composable Plug-and-Play Reasoner for
  Frozen LLMs
Authors: name: Jaemin Kim, name: Hangeol Chang, name: Hyunmin Hwang, name: Choonghan Kim, name: Jong Chul Ye
Published: 2025-05-25 10:19:10

Large Language Models (LLMs) have demonstrated remarkable general
capabilities, but enhancing skills such as reasoning often demands substantial
computational resources and may compromise their generalization. While
Parameter-Efficient Fine-Tuning (PEFT) methods offer a more resource-conscious
alternative, they typically requires retraining for each LLM backbone due to
architectural dependencies. To address these challenges, here we propose
Universal Reasoner (UniR) - a single, lightweight, composable, and
plug-and-play reasoning module that can be used with any frozen LLM to endow it
with specialized reasoning capabilities. Specifically, UniR decomposes the
reward into a standalone reasoning module that is trained independently using
predefined rewards, effectively translating trajectory-level signals into
token-level guidance. Once trained, UniR can be combined with any frozen LLM at
inference time by simply adding its output logits to those of the LLM backbone.
This additive structure naturally enables modular composition: multiple UniR
modules trained for different tasks can be jointly applied by summing their
logits, enabling complex reasoning via composition. Experimental results on
mathematical reasoning and machine translation tasks show that UniR
significantly outperforms existing baseline fine-tuning methods using the
Llama3.2 model. Furthermore, UniR demonstrates strong weak-to-strong
generalization: reasoning modules trained on smaller models effectively guide
much larger LLMs. This makes UniR a cost-efficient, adaptable, and robust
solution for enhancing reasoning in LLMs without compromising their core
capabilities. Code is open-sourced at https://github.com/hangeol/UniR
Just as Humans Need Vaccines, So Do Models: Model Immunization to Combat
  Falsehoods
Authors: name: Shaina Raza, name: Rizwan Qureshi, name: Marcelo Lotif, name: Aman Chadha, name: Deval Pandya, name: Christos Emmanouilidis
Published: 2025-05-23 13:20:23

Generative AI models often learn and reproduce false information present in
their training corpora. This position paper argues that, analogous to
biological immunization, where controlled exposure to a weakened pathogen
builds immunity, AI models should be fine tuned on small, quarantined sets of
explicitly labeled falsehoods as a "vaccine" against misinformation. These
curated false examples are periodically injected during finetuning,
strengthening the model ability to recognize and reject misleading claims while
preserving accuracy on truthful inputs. An illustrative case study shows that
immunized models generate substantially less misinformation than baselines. To
our knowledge, this is the first training framework that treats fact checked
falsehoods themselves as a supervised vaccine, rather than relying on input
perturbations or generic human feedback signals, to harden models against
future misinformation. We also outline ethical safeguards and governance
controls to ensure the safe use of false data. Model immunization offers a
proactive paradigm for aligning AI systems with factuality.
Let's Predict Sentence by Sentence
Authors: name: Hyeonbin Hwang, name: Byeongguk Jeon, name: Seungone Kim, name: Jiyeon Kim, name: Hoyeon Chang, name: Sohee Yang, name: Seungpil Won, name: Dohaeng Lee, name: Youbin Ahn, name: Minjoon Seo
Published: 2025-05-28 10:28:35

Autoregressive language models (LMs) generate one token at a time, yet human
reasoning operates over higher-level abstractions - sentences, propositions,
and concepts. This contrast raises a central question- Can LMs likewise learn
to reason over structured semantic units rather than raw token sequences? In
this work, we investigate whether pretrained LMs can be lifted into such
abstract reasoning spaces by building on their learned representations. We
present a framework that adapts a pretrained token-level LM to operate in
sentence space by autoregressively predicting continuous embeddings of next
sentences. We explore two embedding paradigms inspired by classical
representation learning: 1) semantic embeddings, learned via autoencoding to
preserve surface meaning; and 2) contextual embeddings, trained via
next-sentence prediction to encode anticipatory structure. We evaluate both
under two inference regimes: Discretized, which decodes each predicted
embedding into text before re-encoding; and Continuous, which reasons entirely
in embedding space for improved efficiency. Across four domains - mathematics,
logic, commonsense, and planning - contextual embeddings under continuous
inference show competitive performance with Chain-of-Thought (CoT) while
reducing inference-time FLOPs on average by half. We also present early signs
of scalability and modular adaptation. Finally, to visualize latent
trajectories, we introduce SentenceLens, a diagnostic tool that decodes
intermediate model states into interpretable sentences. Together, our results
indicate that pretrained LMs can effectively transition to abstract, structured
reasoning within latent embedding spaces.
MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware
  Multi-Segment Grounding
Authors: name: Fuwen Luo, name: Shengfeng Lou, name: Chi Chen, name: Ziyue Wang, name: Chenliang Li, name: Weizhou Shen, name: Jiyue Guo, name: Peng Li, name: Ming Yan, name: Ji Zhang, name: Fei Huang, name: Yang Liu
Published: 2025-05-27 04:50:07

Video temporal understanding is crucial for multimodal large language models
(MLLMs) to reason over events in videos. Despite recent advances in general
video understanding, current MLLMs still struggle with fine-grained temporal
reasoning. While reinforcement learning (RL) has been explored to address this
issue recently, existing RL approaches remain limited in effectiveness. In this
work, we propose MUSEG, a novel RL-based method that enhances temporal
understanding by introducing timestamp-aware multi-segment grounding. MUSEG
enables MLLMs to align queries with multiple relevant video segments, promoting
more comprehensive temporal reasoning. To facilitate effective learning, we
design a customized RL training recipe with phased rewards that progressively
guides the model toward temporally grounded reasoning. Extensive experiments on
temporal grounding and time-sensitive video QA tasks demonstrate that MUSEG
significantly outperforms existing methods and generalizes well across diverse
temporal understanding scenarios. View our project at
https://github.com/THUNLP-MT/MUSEG.
VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich
  Information Understanding via Iterative Reasoning with Reinforcement Learning
Authors: name: Qiuchen Wang, name: Ruixue Ding, name: Yu Zeng, name: Zehui Chen, name: Lin Chen, name: Shihang Wang, name: Pengjun Xie, name: Fei Huang, name: Feng Zhao
Published: 2025-05-28 06:30:51

Effectively retrieving, reasoning and understanding visually rich information
remains a challenge for RAG methods. Traditional text-based methods cannot
handle visual-related information. On the other hand, current vision-based RAG
approaches are often limited by fixed pipelines and frequently struggle to
reason effectively due to the insufficient activation of the fundamental
capabilities of models. As RL has been proven to be beneficial for model
reasoning, we introduce VRAG-RL, a novel RL framework tailored for complex
reasoning across visually rich information. With this framework, VLMs interact
with search engines, autonomously sampling single-turn or multi-turn reasoning
trajectories with the help of visual perception tokens and undergoing continual
optimization based on these samples. Our approach highlights key limitations of
RL in RAG domains: (i) Prior Multi-modal RAG approaches tend to merely
incorporate images into the context, leading to insufficient reasoning token
allocation and neglecting visual-specific perception; and (ii) When models
interact with search engines, their queries often fail to retrieve relevant
information due to the inability to articulate requirements, thereby leading to
suboptimal performance. To address these challenges, we define an action space
tailored for visually rich inputs, with actions including cropping and scaling,
allowing the model to gather information from a coarse-to-fine perspective.
Furthermore, to bridge the gap between users' original inquiries and the
retriever, we employ a simple yet effective reward that integrates query
rewriting and retrieval performance with a model-based reward. Our VRAG-RL
optimizes VLMs for RAG tasks using specially designed RL strategies, aligning
the model with real-world applications. The code is available at
https://github.com/Alibaba-NLP/VRAG{https://github.com/Alibaba-NLP/VRAG}.
Unveiling Instruction-Specific Neurons & Experts: An Analytical
  Framework for LLM's Instruction-Following Capabilities
Authors: name: Junyan Zhang, name: Yubo Gao, name: Yibo Yan, name: Jungang Li, name: Zhaorui Hou, name: Sicheng Tao, name: Shuliang Liu, name: Song Dai, name: Yonghua Hei, name: Junzhuo Li, name: Xuming Hu
Published: 2025-05-27 13:40:28

The finetuning of Large Language Models (LLMs) has significantly advanced
their instruction-following capabilities, yet the underlying computational
mechanisms driving these improvements remain poorly understood. This study
systematically examines how fine-tuning reconfigures LLM computations by
isolating and analyzing instruction-specific sparse components, i.e., neurons
in dense models and both neurons and experts in Mixture-of-Experts (MoE)
architectures. In particular, we introduce HexaInst, a carefully curated and
balanced instructional dataset spanning six distinct categories, and propose
SPARCOM, a novel analytical framework comprising three key contributions: (1) a
method for identifying these sparse components, (2) an evaluation of their
functional generality and uniqueness, and (3) a systematic comparison of their
alterations. Through experiments, we demonstrate functional generality,
uniqueness, and the critical role of these components in instruction execution.
By elucidating the relationship between fine-tuning-induced adaptations and
sparse computational substrates, this work provides deeper insights into how
LLMs internalize instruction-following behavior for the trustworthy LLM
community.
SVRPBench: A Realistic Benchmark for Stochastic Vehicle Routing Problem
Authors: name: Ahmed Heakl, name: Yahia Salaheldin Shaaban, name: Martin Takac, name: Salem Lahlou, name: Zangir Iklassov
Published: 2025-05-28 02:03:31

Robust routing under uncertainty is central to real-world logistics, yet most
benchmarks assume static, idealized settings. We present SVRPBench, the first
open benchmark to capture high-fidelity stochastic dynamics in vehicle routing
at urban scale. Spanning more than 500 instances with up to 1000 customers, it
simulates realistic delivery conditions: time-dependent congestion, log-normal
delays, probabilistic accidents, and empirically grounded time windows for
residential and commercial clients. Our pipeline generates diverse,
constraint-rich scenarios, including multi-depot and multi-vehicle setups.
Benchmarking reveals that state-of-the-art RL solvers like POMO and AM degrade
by over 20% under distributional shift, while classical and metaheuristic
methods remain robust. To enable reproducible research, we release the dataset
and evaluation suite. SVRPBench challenges the community to design solvers that
generalize beyond synthetic assumptions and adapt to real-world uncertainty.
One-Way Ticket:Time-Independent Unified Encoder for Distilling
  Text-to-Image Diffusion Models
Authors: name: Senmao Li, name: Lei Wang, name: Kai Wang, name: Tao Liu, name: Jiehang Xie, name: Joost van de Weijer, name: Fahad Shahbaz Khan, name: Shiqi Yang, name: Yaxing Wang, name: Jian Yang
Published: 2025-05-28 04:23:22

Text-to-Image (T2I) diffusion models have made remarkable advancements in
generative modeling; however, they face a trade-off between inference speed and
image quality, posing challenges for efficient deployment. Existing distilled
T2I models can generate high-fidelity images with fewer sampling steps, but
often struggle with diversity and quality, especially in one-step models. From
our analysis, we observe redundant computations in the UNet encoders. Our
findings suggest that, for T2I diffusion models, decoders are more adept at
capturing richer and more explicit semantic information, while encoders can be
effectively shared across decoders from diverse time steps. Based on these
observations, we introduce the first Time-independent Unified Encoder TiUE for
the student model UNet architecture, which is a loop-free image generation
approach for distilling T2I diffusion models. Using a one-pass scheme, TiUE
shares encoder features across multiple decoder time steps, enabling parallel
sampling and significantly reducing inference time complexity. In addition, we
incorporate a KL divergence term to regularize noise prediction, which enhances
the perceptual realism and diversity of the generated images. Experimental
results demonstrate that TiUE outperforms state-of-the-art methods, including
LCM, SD-Turbo, and SwiftBrushv2, producing more diverse and realistic results
while maintaining the computational efficiency.
What Makes for Text to 360-degree Panorama Generation with Stable
  Diffusion?
Authors: name: Jinhong Ni, name: Chang-Bin Zhang, name: Qiang Zhang, name: Jing Zhang
Published: 2025-05-28 08:54:04

Recent prosperity of text-to-image diffusion models, e.g. Stable Diffusion,
has stimulated research to adapt them to 360-degree panorama generation. Prior
work has demonstrated the feasibility of using conventional low-rank adaptation
techniques on pre-trained diffusion models to generate panoramic images.
However, the substantial domain gap between perspective and panoramic images
raises questions about the underlying mechanisms enabling this empirical
success. We hypothesize and examine that the trainable counterparts exhibit
distinct behaviors when fine-tuned on panoramic data, and such an adaptation
conceals some intrinsic mechanism to leverage the prior knowledge within the
pre-trained diffusion models. Our analysis reveals the following: 1) the query
and key matrices in the attention modules are responsible for common
information that can be shared between the panoramic and perspective domains,
thus are less relevant to panorama generation; and 2) the value and output
weight matrices specialize in adapting pre-trained knowledge to the panoramic
domain, playing a more critical role during fine-tuning for panorama
generation. We empirically verify these insights by introducing a simple
framework called UniPano, with the objective of establishing an elegant
baseline for future research. UniPano not only outperforms existing methods but
also significantly reduces memory usage and training time compared to prior
dual-branch approaches, making it scalable for end-to-end panorama generation
with higher resolution. The code will be released.
Pitfalls of Rule- and Model-based Verifiers -- A Case Study on
  Mathematical Reasoning
Authors: name: Yuzhen Huang, name: Weihao Zeng, name: Xingshan Zeng, name: Qi Zhu, name: Junxian He
Published: 2025-05-28 10:28:41

Trustworthy verifiers are essential for the success of reinforcement learning
with verifiable reward (RLVR), which is the core methodology behind various
large reasoning models such as DeepSeek-R1. In complex domains like
mathematical reasoning, rule-based verifiers have been widely adopted in
previous works to train strong reasoning models. However, the reliability of
these verifiers and their impact on the RL training process remain poorly
understood. In this work, we take mathematical reasoning as a case study and
conduct a comprehensive analysis of various verifiers in both static evaluation
and RL training scenarios. First, we find that current open-source rule-based
verifiers often fail to recognize equivalent answers presented in different
formats across multiple commonly used mathematical datasets, resulting in
non-negligible false negative rates. This limitation adversely affects RL
training performance and becomes more pronounced as the policy model gets
stronger. Subsequently, we investigate model-based verifiers as a potential
solution to address these limitations. While the static evaluation shows that
model-based verifiers achieve significantly higher verification accuracy,
further analysis and RL training results imply that they are highly susceptible
to hacking, where they misclassify certain patterns in responses as correct
(i.e., false positives). This vulnerability is exploited during policy model
optimization, leading to artificially inflated rewards. Our findings underscore
the unique risks inherent to both rule-based and model-based verifiers, aiming
to offer valuable insights to develop more robust reward systems in
reinforcement learning.
Characterizing Bias: Benchmarking Large Language Models in Simplified
  versus Traditional Chinese
Authors: name: Hanjia Lyu, name: Jiebo Luo, name: Jian Kang, name: Allison Koenecke
Published: 2025-05-28 17:56:49

While the capabilities of Large Language Models (LLMs) have been studied in
both Simplified and Traditional Chinese, it is yet unclear whether LLMs exhibit
differential performance when prompted in these two variants of written
Chinese. This understanding is critical, as disparities in the quality of LLM
responses can perpetuate representational harms by ignoring the different
cultural contexts underlying Simplified versus Traditional Chinese, and can
exacerbate downstream harms in LLM-facilitated decision-making in domains such
as education or hiring. To investigate potential LLM performance disparities,
we design two benchmark tasks that reflect real-world scenarios: regional term
choice (prompting the LLM to name a described item which is referred to
differently in Mainland China and Taiwan), and regional name choice (prompting
the LLM to choose who to hire from a list of names in both Simplified and
Traditional Chinese). For both tasks, we audit the performance of 11 leading
commercial LLM services and open-sourced models -- spanning those primarily
trained on English, Simplified Chinese, or Traditional Chinese. Our analyses
indicate that biases in LLM responses are dependent on both the task and
prompting language: while most LLMs disproportionately favored Simplified
Chinese responses in the regional term choice task, they surprisingly favored
Traditional Chinese names in the regional name choice task. We find that these
disparities may arise from differences in training data representation, written
character preferences, and tokenization of Simplified and Traditional Chinese.
These findings highlight the need for further analysis of LLM biases; as such,
we provide an open-sourced benchmark dataset to foster reproducible evaluations
of future LLM behavior across Chinese language variants
(https://github.com/brucelyu17/SC-TC-Bench).
Thinking with Generated Images
Authors: name: Ethan Chern, name: Zhulin Hu, name: Steffi Chern, name: Siqi Kou, name: Jiadi Su, name: Yan Ma, name: Zhijie Deng, name: Pengfei Liu
Published: 2025-05-28 16:12:45

We present Thinking with Generated Images, a novel paradigm that
fundamentally transforms how large multimodal models (LMMs) engage with visual
reasoning by enabling them to natively think across text and vision modalities
through spontaneous generation of intermediate visual thinking steps. Current
visual reasoning with LMMs is constrained to either processing fixed
user-provided images or reasoning solely through text-based chain-of-thought
(CoT). Thinking with Generated Images unlocks a new dimension of cognitive
capability where models can actively construct intermediate visual thoughts,
critique their own visual hypotheses, and refine them as integral components of
their reasoning process. We demonstrate the effectiveness of our approach
through two complementary mechanisms: (1) vision generation with intermediate
visual subgoals, where models decompose complex visual tasks into manageable
components that are generated and integrated progressively, and (2) vision
generation with self-critique, where models generate an initial visual
hypothesis, analyze its shortcomings through textual reasoning, and produce
refined outputs based on their own critiques. Our experiments on vision
generation benchmarks show substantial improvements over baseline approaches,
with our models achieving up to 50% (from 38% to 57%) relative improvement in
handling complex multi-object scenarios. From biochemists exploring novel
protein structures, and architects iterating on spatial designs, to forensic
analysts reconstructing crime scenes, and basketball players envisioning
strategic plays, our approach enables AI models to engage in the kind of visual
imagination and iterative refinement that characterizes human creative,
analytical, and strategic thinking. We release our open-source suite at
https://github.com/GAIR-NLP/thinking-with-generated-images.