Orionnnn/hub-stats huggingface dataset:
Training-Free Efficient Video Generation via Dynamic Token Carving
Authors: name: Yuechen Zhang, name: Jinbo Xing, name: Bin Xia, name: Shaoteng Liu, name: Bohao Peng, name: Xin Tao, name: Pengfei Wan, name: Eric Lo, name: Jiaya Jia
Published: 2025-05-22 16:21:32

Despite the remarkable generation quality of video Diffusion Transformer
(DiT) models, their practical deployment is severely hindered by extensive
computational requirements. This inefficiency stems from two key challenges:
the quadratic complexity of self-attention with respect to token length and the
multi-step nature of diffusion models. To address these limitations, we present
Jenga, a novel inference pipeline that combines dynamic attention carving with
progressive resolution generation. Our approach leverages two key insights: (1)
early denoising steps do not require high-resolution latents, and (2) later
steps do not require dense attention. Jenga introduces a block-wise attention
mechanism that dynamically selects relevant token interactions using 3D
space-filling curves, alongside a progressive resolution strategy that
gradually increases latent resolution during generation. Experimental results
demonstrate that Jenga achieves substantial speedups across multiple
state-of-the-art video diffusion models while maintaining comparable generation
quality (8.83times speedup with 0.01\% performance drop on VBench). As a
plug-and-play solution, Jenga enables practical, high-quality video generation
on modern hardware by reducing inference time from minutes to seconds --
without requiring model retraining. Code:
https://github.com/dvlab-research/Jenga
SpatialScore: Towards Unified Evaluation for Multimodal Spatial
  Understanding
Authors: name: Haoning Wu, name: Xiao Huang, name: Yaohui Chen, name: Ya Zhang, name: Yanfeng Wang, name: Weidi Xie
Published: 2025-05-22 17:59:03

Multimodal large language models (MLLMs) have achieved impressive success in
question-answering tasks, yet their capabilities for spatial understanding are
less explored. This work investigates a critical question: do existing MLLMs
possess 3D spatial perception and understanding abilities? Concretely, we make
the following contributions in this paper: (i) we introduce VGBench, a
benchmark specifically designed to assess MLLMs for visual geometry perception,
e.g., camera pose and motion estimation; (ii) we propose SpatialScore, the most
comprehensive and diverse multimodal spatial understanding benchmark to date,
integrating VGBench with relevant data from the other 11 existing datasets.
This benchmark comprises 28K samples across various spatial understanding
tasks, modalities, and QA formats, along with a carefully curated challenging
subset, SpatialScore-Hard; (iii) we develop SpatialAgent, a novel multi-agent
system incorporating 9 specialized tools for spatial understanding, supporting
both Plan-Execute and ReAct reasoning paradigms; (iv) we conduct extensive
evaluations to reveal persistent challenges in spatial reasoning while
demonstrating the effectiveness of SpatialAgent. We believe SpatialScore will
offer valuable insights and serve as a rigorous benchmark for the next
evolution of MLLMs.
Think or Not? Selective Reasoning via Reinforcement Learning for
  Vision-Language Models
Authors: name: Jiaqi Wang, name: Kevin Qinghong Lin, name: James Cheng, name: Mike Zheng Shou
Published: 2025-05-22 16:13:29

Reinforcement Learning (RL) has proven to be an effective post-training
strategy for enhancing reasoning in vision-language models (VLMs). Group
Relative Policy Optimization (GRPO) is a recent prominent method that
encourages models to generate complete reasoning traces before answering,
leading to increased token usage and computational cost. Inspired by the
human-like thinking process-where people skip reasoning for easy questions but
think carefully when needed-we explore how to enable VLMs to first decide when
reasoning is necessary. To realize this, we propose TON, a two-stage training
strategy: (i) a supervised fine-tuning (SFT) stage with a simple yet effective
'thought dropout' operation, where reasoning traces are randomly replaced with
empty thoughts. This introduces a think-or-not format that serves as a cold
start for selective reasoning; (ii) a GRPO stage that enables the model to
freely explore when to think or not, while maximizing task-aware outcome
rewards. Experimental results show that TON can reduce the completion length by
up to 90% compared to vanilla GRPO, without sacrificing performance or even
improving it. Further evaluations across diverse vision-language tasks-covering
a range of reasoning difficulties under both 3B and 7B models-consistently
reveal that the model progressively learns to bypass unnecessary reasoning
steps as training advances. These findings shed light on the path toward
human-like reasoning patterns in reinforcement learning approaches. Our code is
available at https://github.com/kokolerk/TON.
Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel
  Decoding
Authors: name: Runpeng Yu, name: Xinyin Ma, name: Xinchao Wang
Published: 2025-05-22 17:55:04

In this work, we propose Dimple, the first Discrete Diffusion Multimodal
Large Language Model (DMLLM). We observe that training with a purely discrete
diffusion approach leads to significant training instability, suboptimal
performance, and severe length bias issues. To address these challenges, we
design a novel training paradigm that combines an initial autoregressive phase
with a subsequent diffusion phase. This approach yields the Dimple-7B model,
trained on the same dataset and using a similar training pipeline as
LLaVA-NEXT. Dimple-7B ultimately surpasses LLaVA-NEXT in performance by 3.9%,
demonstrating that DMLLM can achieve performance comparable to that of
autoregressive models. To improve inference efficiency, we propose a decoding
strategy termed confident decoding, which dynamically adjusts the number of
tokens generated at each step, significantly reducing the number of generation
iterations. In autoregressive models, the number of forward iterations during
generation equals the response length. With confident decoding, however, the
number of iterations needed by Dimple is even only text{response
length}{3}. We also re-implement the prefilling technique in autoregressive
models and demonstrate that it does not significantly impact performance on
most benchmark evaluations, while offering a speedup of 1.5x to 7x.
Additionally, we explore Dimple's capability to precisely control its response
using structure priors. These priors enable structured responses in a manner
distinct from instruction-based or chain-of-thought prompting, and allow
fine-grained control over response format and length, which is difficult to
achieve in autoregressive models. Overall, this work validates the feasibility
and advantages of DMLLM and enhances its inference efficiency and
controllability. Code and models are available at
https://github.com/yu-rp/Dimple.
Understanding Generative AI Capabilities in Everyday Image Editing Tasks
Authors: name: Mohammad Reza Taesiri, name: Brandon Collins, name: Logan Bolton, name: Viet Dac Lai, name: Franck Dernoncourt, name: Trung Bui, name: Anh Totti Nguyen
Published: 2025-05-22 03:35:15

Generative AI (GenAI) holds significant promise for automating everyday image
editing tasks, especially following the recent release of GPT-4o on March 25,
2025. However, what subjects do people most often want edited? What kinds of
editing actions do they want to perform (e.g., removing or stylizing the
subject)? Do people prefer precise edits with predictable outcomes or highly
creative ones? By understanding the characteristics of real-world requests and
the corresponding edits made by freelance photo-editing wizards, can we draw
lessons for improving AI-based editors and determine which types of requests
can currently be handled successfully by AI editors? In this paper, we present
a unique study addressing these questions by analyzing 83k requests from the
past 12 years (2013-2025) on the Reddit community, which collected 305k
PSR-wizard edits. According to human ratings, approximately only 33% of
requests can be fulfilled by the best AI editors (including GPT-4o,
Gemini-2.0-Flash, SeedEdit). Interestingly, AI editors perform worse on
low-creativity requests that require precise editing than on more open-ended
tasks. They often struggle to preserve the identity of people and animals, and
frequently make non-requested touch-ups. On the other side of the table, VLM
judges (e.g., o1) perform differently from human judges and may prefer AI edits
more than human edits. Code and qualitative examples are available at:
https://psrdataset.github.io
KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Models
Authors: name: Yongliang Wu, name: Zonghui Li, name: Xinting Hu, name: Xinyu Ye, name: Xianfang Zeng, name: Gang Yu, name: Wenbo Zhu, name: Bernt Schiele, name: Ming-Hsuan Yang, name: Xu Yang
Published: 2025-05-22 14:08:59

Recent advances in multi-modal generative models have enabled significant
progress in instruction-based image editing. However, while these models
produce visually plausible outputs, their capacity for knowledge-based
reasoning editing tasks remains under-explored. In this paper, we introduce
KRIS-Bench (Knowledge-based Reasoning in Image-editing Systems Benchmark), a
diagnostic benchmark designed to assess models through a cognitively informed
lens. Drawing from educational theory, KRIS-Bench categorizes editing tasks
across three foundational knowledge types: Factual, Conceptual, and Procedural.
Based on this taxonomy, we design 22 representative tasks spanning 7 reasoning
dimensions and release 1,267 high-quality annotated editing instances. To
support fine-grained evaluation, we propose a comprehensive protocol that
incorporates a novel Knowledge Plausibility metric, enhanced by knowledge hints
and calibrated through human studies. Empirical results on 10 state-of-the-art
models reveal significant gaps in reasoning performance, highlighting the need
for knowledge-centric benchmarks to advance the development of intelligent
image editing systems.
Scaling Diffusion Transformers Efficiently via μP
Authors: name: Chenyu Zheng, name: Xinyu Zhang, name: Rongzhen Wang, name: Wei Huang, name: Zhi Tian, name: Weilin Huang, name: Jun Zhu, name: Chongxuan Li
Published: 2025-05-21 08:49:03

Diffusion Transformers have emerged as the foundation for vision generative
models, but their scalability is limited by the high cost of hyperparameter
(HP) tuning at large scales. Recently, Maximal Update Parametrization (muP)
was proposed for vanilla Transformers, which enables stable HP transfer from
small to large language models, and dramatically reduces tuning costs. However,
it remains unclear whether muP of vanilla Transformers extends to diffusion
Transformers, which differ architecturally and objectively. In this work, we
generalize standard muP to diffusion Transformers and validate its
effectiveness through large-scale experiments. First, we rigorously prove that
muP of mainstream diffusion Transformers, including DiT, U-ViT,
PixArt-alpha, and MMDiT, aligns with that of the vanilla Transformer,
enabling the direct application of existing muP methodologies. Leveraging
this result, we systematically demonstrate that DiT-muP enjoys robust HP
transferability. Notably, DiT-XL-2-muP with transferred learning rate
achieves 2.9 times faster convergence than the original DiT-XL-2. Finally, we
validate the effectiveness of muP on text-to-image generation by scaling
PixArt-alpha from 0.04B to 0.61B and MMDiT from 0.18B to 18B. In both cases,
models under muP outperform their respective baselines while requiring small
tuning cost, only 5.5% of one training run for PixArt-alpha and 3% of
consumption by human experts for MMDiT-18B. These results establish muP as a
principled and efficient framework for scaling diffusion Transformers.
Backdoor Cleaning without External Guidance in MLLM Fine-tuning
Authors: name: Xuankun Rong, name: Wenke Huang, name: Jian Liang, name: Jinhe Bi, name: Xun Xiao, name: Yiming Li, name: Bo Du, name: Mang Ye
Published: 2025-05-22 17:11:58

Multimodal Large Language Models (MLLMs) are increasingly deployed in
fine-tuning-as-a-service (FTaaS) settings, where user-submitted datasets adapt
general-purpose models to downstream tasks. This flexibility, however,
introduces serious security risks, as malicious fine-tuning can implant
backdoors into MLLMs with minimal effort. In this paper, we observe that
backdoor triggers systematically disrupt cross-modal processing by causing
abnormal attention concentration on non-semantic regions--a phenomenon we term
attention collapse. Based on this insight, we propose Believe Your Eyes (BYE),
a data filtering framework that leverages attention entropy patterns as
self-supervised signals to identify and filter backdoor samples. BYE operates
via a three-stage pipeline: (1) extracting attention maps using the fine-tuned
model, (2) computing entropy scores and profiling sensitive layers via bimodal
separation, and (3) performing unsupervised clustering to remove suspicious
samples. Unlike prior defenses, BYE equires no clean supervision, auxiliary
labels, or model modifications. Extensive experiments across various datasets,
models, and diverse trigger types validate BYE's effectiveness: it achieves
near-zero attack success rates while maintaining clean-task performance,
offering a robust and generalizable solution against backdoor threats in MLLMs.
OViP: Online Vision-Language Preference Learning
Authors: name: Shujun Liu, name: Siyuan Wang, name: Zejun Li, name: Jianxiang Wang, name: Cheng Zeng, name: Zhongyu Wei
Published: 2025-05-21 19:26:09

Large vision-language models (LVLMs) remain vulnerable to hallucination,
often generating content misaligned with visual inputs. While recent approaches
advance multi-modal Direct Preference Optimization (DPO) to mitigate
hallucination, they typically rely on predefined or randomly edited negative
samples that fail to reflect actual model errors, limiting training efficacy.
In this work, we propose an Online Vision-language Preference Learning (OViP)
framework that dynamically constructs contrastive training data based on the
model's own hallucinated outputs. By identifying semantic differences between
sampled response pairs and synthesizing negative images using a diffusion
model, OViP generates more relevant supervision signals in real time. This
failure-driven training enables adaptive alignment of both textual and visual
preferences. Moreover, we refine existing evaluation protocols to better
capture the trade-off between hallucination suppression and expressiveness.
Experiments on hallucination and general benchmarks demonstrate that OViP
effectively reduces hallucinations while preserving core multi-modal
capabilities.
TinyV: Reducing False Negatives in Verification Improves RL for LLM
  Reasoning
Authors: name: Zhangchen Xu, name: Yuetai Li, name: Fengqing Jiang, name: Bhaskar Ramasubramanian, name: Luyao Niu, name: Bill Yuchen Lin, name: Radha Poovendran
Published: 2025-05-20 17:16:44

Reinforcement Learning (RL) has become a powerful tool for enhancing the
reasoning abilities of large language models (LLMs) by optimizing their
policies with reward signals. Yet, RL's success relies on the reliability of
rewards, which are provided by verifiers. In this paper, we expose and analyze
a widespread problem--false negatives--where verifiers wrongly reject correct
model outputs. Our in-depth study of the Big-Math-RL-Verified dataset reveals
that over 38% of model-generated responses suffer from false negatives, where
the verifier fails to recognize correct answers. We show, both empirically and
theoretically, that these false negatives severely impair RL training by
depriving the model of informative gradient signals and slowing convergence. To
mitigate this, we propose tinyV, a lightweight LLM-based verifier that augments
existing rule-based methods, which dynamically identifies potential false
negatives and recovers valid responses to produce more accurate reward
estimates. Across multiple math-reasoning benchmarks, integrating TinyV boosts
pass rates by up to 10% and accelerates convergence relative to the baseline.
Our findings highlight the critical importance of addressing verifier false
negatives and offer a practical approach to improve RL-based fine-tuning of
LLMs. Our code is available at https://github.com/uw-nsl/TinyV.
Scaling Reasoning, Losing Control: Evaluating Instruction Following in
  Large Reasoning Models
Authors: name: Tingchen Fu, name: Jiawei Gu, name: Yafu Li, name: Xiaoye Qu, name: Yu Cheng
Published: 2025-05-20 18:18:01

Instruction-following is essential for aligning large language models (LLMs)
with user intent. While recent reasoning-oriented models exhibit impressive
performance on complex mathematical problems, their ability to adhere to
natural language instructions remains underexplored. In this work, we introduce
MathIF, a dedicated benchmark for evaluating instruction-following in
mathematical reasoning tasks. Our empirical analysis reveals a consistent
tension between scaling up reasoning capacity and maintaining controllability,
as models that reason more effectively often struggle to comply with user
directives. We find that models tuned on distilled long chains-of-thought or
trained with reasoning-oriented reinforcement learning often degrade in
instruction adherence, especially when generation length increases.
Furthermore, we show that even simple interventions can partially recover
obedience, though at the cost of reasoning performance. These findings
highlight a fundamental tension in current LLM training paradigms and motivate
the need for more instruction-aware reasoning models. We release the code and
data at https://github.com/TingchenFu/MathIF.
QuickVideo: Real-Time Long Video Understanding with System Algorithm
  Co-Design
Authors: name: Benjamin Schneider, name: Dongfu Jiang, name: Chao Du, name: Tianyu Pang, name: Wenhu Chen
Published: 2025-05-22 03:26:50

Long-video understanding has emerged as a crucial capability in real-world
applications such as video surveillance, meeting summarization, educational
lecture analysis, and sports broadcasting. However, it remains computationally
prohibitive for VideoLLMs, primarily due to two bottlenecks: 1) sequential
video decoding, the process of converting the raw bit stream to RGB frames can
take up to a minute for hour-long video inputs, and 2) costly prefilling of up
to several million tokens for LLM inference, resulting in high latency and
memory use. To address these challenges, we propose QuickVideo, a
system-algorithm co-design that substantially accelerates long-video
understanding to support real-time downstream applications. It comprises three
key innovations: QuickDecoder, a parallelized CPU-based video decoder that
achieves 2-3 times speedup by splitting videos into keyframe-aligned intervals
processed concurrently; QuickPrefill, a memory-efficient prefilling method
using KV-cache pruning to support more frames with less GPU memory; and an
overlapping scheme that overlaps CPU video decoding with GPU inference.
Together, these components infernece time reduce by a minute on long video
inputs, enabling scalable, high-quality video understanding even on limited
hardware. Experiments show that QuickVideo generalizes across durations and
sampling rates, making long video processing feasible in practice.
Training-Free Reasoning and Reflection in MLLMs
Authors: name: Hongchen Wei, name: Zhenzhong Chen
Published: 2025-05-22 02:51:12

Recent advances in Reasoning LLMs (e.g., DeepSeek-R1 and OpenAI-o1) have
showcased impressive reasoning capabilities via reinforcement learning.
However, extending these capabilities to Multimodal LLMs (MLLMs) is hampered by
the prohibitive costs of retraining and the scarcity of high-quality,
verifiable multimodal reasoning datasets. This paper introduces FRANK Model, a
training-FRee ANd r1-liKe MLLM that imbues off-the-shelf MLLMs with reasoning
and reflection abilities, without any gradient updates or extra supervision.
Our key insight is to decouple perception and reasoning across MLLM decoder
layers. Specifically, we observe that compared to the deeper decoder layers,
the shallow decoder layers allocate more attention to visual tokens, while the
deeper decoder layers concentrate on textual semantics. This observation
motivates a hierarchical weight merging approach that combines a
visual-pretrained MLLM with a reasoning-specialized LLM. To this end, we
propose a layer-wise, Taylor-derived closed-form fusion mechanism that
integrates reasoning capacity into deep decoder layers while preserving visual
grounding in shallow decoder layers. Extensive experiments on challenging
multimodal reasoning benchmarks demonstrate the effectiveness of our approach.
On the MMMU benchmark, our model FRANK-38B achieves an accuracy of 69.2,
outperforming the strongest baseline InternVL2.5-38B by +5.3, and even
surpasses the proprietary GPT-4o model. Our project homepage is at:
http://iip.whu.edu.cn/frank/index.html
Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement
  Learning
Authors: name: Guanting Dong, name: Yifei Chen, name: Xiaoxi Li, name: Jiajie Jin, name: Hongjin Qian, name: Yutao Zhu, name: Hangyu Mao, name: Guorui Zhou, name: Zhicheng Dou, name: Ji-Rong Wen
Published: 2025-05-22 09:00:19

Recently, large language models (LLMs) have shown remarkable reasoning
capabilities via large-scale reinforcement learning (RL). However, leveraging
the RL algorithm to empower effective multi-tool collaborative reasoning in
LLMs remains an open challenge. In this paper, we introduce Tool-Star, an
RL-based framework designed to empower LLMs to autonomously invoke multiple
external tools during stepwise reasoning. Tool-Star integrates six types of
tools and incorporates systematic designs in both data synthesis and training.
To address the scarcity of tool-use data, we propose a general tool-integrated
reasoning data synthesis pipeline, which combines tool-integrated prompting
with hint-based sampling to automatically and scalably generate tool-use
trajectories. A subsequent quality normalization and difficulty-aware
classification process filters out low-quality samples and organizes the
dataset from easy to hard. Furthermore, we propose a two-stage training
framework to enhance multi-tool collaborative reasoning by: (1) cold-start
fine-tuning, which guides LLMs to explore reasoning patterns via
tool-invocation feedback; and (2) a multi-tool self-critic RL algorithm with
hierarchical reward design, which reinforces reward understanding and promotes
effective tool collaboration. Experimental analyses on over 10 challenging
reasoning benchmarks highlight the effectiveness and efficiency of Tool-Star.
The code is available at https://github.com/dongguanting/Tool-Star.