Orionnnn/hub-stats huggingface dataset:
Minute-Long Videos with Dual Parallelisms
Authors: name: Zeqing Wang, name: Bowen Zheng, name: Xingyi Yang, name: Yuecong Xu, name: Xinchao Wang
Published: 2025-05-27 11:55:22

Diffusion Transformer (DiT)-based video diffusion models generate
high-quality videos at scale but incur prohibitive processing latency and
memory costs for long videos. To address this, we propose a novel distributed
inference strategy, termed DualParal. The core idea is that, instead of
generating an entire video on a single GPU, we parallelize both temporal frames
and model layers across GPUs. However, a naive implementation of this division
faces a key limitation: since diffusion models require synchronized noise
levels across frames, this implementation leads to the serialization of
original parallelisms. We leverage a block-wise denoising scheme to handle
this. Namely, we process a sequence of frame blocks through the pipeline with
progressively decreasing noise levels. Each GPU handles a specific block and
layer subset while passing previous results to the next GPU, enabling
asynchronous computation and communication. To further optimize performance, we
incorporate two key enhancements. Firstly, a feature cache is implemented on
each GPU to store and reuse features from the prior block as context,
minimizing inter-GPU communication and redundant computation. Secondly, we
employ a coordinated noise initialization strategy, ensuring globally
consistent temporal dynamics by sharing initial noise patterns across GPUs
without extra resource costs. Together, these enable fast, artifact-free, and
infinitely long video generation. Applied to the latest diffusion transformer
video generator, our method efficiently produces 1,025-frame videos with up to
6.54times lower latency and 1.48times lower memory cost on 8timesRTX
4090 GPUs.
Sci-Fi: Symmetric Constraint for Frame Inbetweening
Authors: name: Liuhan Chen, name: Xiaodong Cun, name: Xiaoyu Li, name: Xianyi He, name: Shenghai Yuan, name: Jie Chen, name: Ying Shan, name: Li Yuan
Published: 2025-05-27 13:53:50

Frame inbetweening aims to synthesize intermediate video sequences
conditioned on the given start and end frames. Current state-of-the-art methods
mainly extend large-scale pre-trained Image-to-Video Diffusion models (I2V-DMs)
by incorporating end-frame constraints via directly fine-tuning or omitting
training. We identify a critical limitation in their design: Their injections
of the end-frame constraint usually utilize the same mechanism that originally
imposed the start-frame (single image) constraint. However, since the original
I2V-DMs are adequately trained for the start-frame condition in advance,
naively introducing the end-frame constraint by the same mechanism with much
less (even zero) specialized training probably can't make the end frame have a
strong enough impact on the intermediate content like the start frame. This
asymmetric control strength of the two frames over the intermediate content
likely leads to inconsistent motion or appearance collapse in generated frames.
To efficiently achieve symmetric constraints of start and end frames, we
propose a novel framework, termed Sci-Fi, which applies a stronger injection
for the constraint of a smaller training scale. Specifically, it deals with the
start-frame constraint as before, while introducing the end-frame constraint by
an improved mechanism. The new mechanism is based on a well-designed
lightweight module, named EF-Net, which encodes only the end frame and expands
it into temporally adaptive frame-wise features injected into the I2V-DM. This
makes the end-frame constraint as strong as the start-frame constraint,
enabling our Sci-Fi to produce more harmonious transitions in various
scenarios. Extensive experiments prove the superiority of our Sci-Fi compared
with other baselines.
rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale
  Verified Dataset
Authors: name: Yifei Liu, name: Li Lyna Zhang, name: Yi Zhu, name: Bingcheng Dong, name: Xudong Zhou, name: Ning Shang, name: Fan Yang, name: Mao Yang
Published: 2025-05-27 15:00:57

Advancing code reasoning in large language models (LLMs) is fundamentally
limited by the scarcity of high-difficulty datasets, especially those with
verifiable input-output test cases necessary for rigorous solution validation
at scale. We introduce rStar-Coder, which significantly improves LLM code
reasoning capabilities by constructing a large-scale, verified dataset of 418K
competition-level code problems, 580K long-reasoning solutions along with rich
test cases of varying difficulty. This is achieved through three core
contributions: (1) we curate competitive programming code problems and oracle
solutions to synthesize new, solvable problems; (2) we introduce a reliable
input-output test case synthesis pipeline that decouples the generation into a
three-step input generation method and a mutual verification mechanism for
effective output labeling; (3) we augment problems with high-quality,
test-case-verified long-reasoning solutions. Extensive experiments on Qwen
models (1.5B-14B) across various code reasoning benchmarks demonstrate the
superiority of rStar-Coder dataset, achieving leading performance comparable to
frontier reasoning LLMs with much smaller model sizes. On LiveCodeBench,
rStar-Coder improves Qwen2.5-7B from 17.4% to an impressive 57.3%, and
Qwen2.5-14B from 23.3% to 62.5%, surpassing o3-mini (low) by3.1%. On the more
challenging USA Computing Olympiad, our 7B model achieves an average pass@1
accuracy of 16.15%, outperforming the frontier-level QWQ-32B. Code and the
dataset will be released at https://github.com/microsoft/rStar.
ImgEdit: A Unified Image Editing Dataset and Benchmark
Authors: name: Yang Ye, name: Xianyi He, name: Zongjian Li, name: Bin Lin, name: Shenghai Yuan, name: Zhiyuan Yan, name: Bohan Hou, name: Li Yuan
Published: 2025-05-26 17:53:33

Recent advancements in generative models have enabled high-fidelity
text-to-image generation. However, open-source image-editing models still lag
behind their proprietary counterparts, primarily due to limited high-quality
data and insufficient benchmarks. To overcome these limitations, we introduce
ImgEdit, a large-scale, high-quality image-editing dataset comprising 1.2
million carefully curated edit pairs, which contain both novel and complex
single-turn edits, as well as challenging multi-turn tasks. To ensure the data
quality, we employ a multi-stage pipeline that integrates a cutting-edge
vision-language model, a detection model, a segmentation model, alongside
task-specific in-painting procedures and strict post-processing. ImgEdit
surpasses existing datasets in both task novelty and data quality. Using
ImgEdit, we train ImgEdit-E1, an editing model using Vision Language Model to
process the reference image and editing prompt, which outperforms existing
open-source models on multiple tasks, highlighting the value of ImgEdit and
model design. For comprehensive evaluation, we introduce ImgEdit-Bench, a
benchmark designed to evaluate image editing performance in terms of
instruction adherence, editing quality, and detail preservation. It includes a
basic testsuite, a challenging single-turn suite, and a dedicated multi-turn
suite. We evaluate both open-source and proprietary models, as well as
ImgEdit-E1, providing deep analysis and actionable insights into the current
behavior of image-editing models. The source data are publicly available on
https://github.com/PKU-YuanGroup/ImgEdit.
OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for
  Subject-to-Video Generation
Authors: name: Shenghai Yuan, name: Xianyi He, name: Yufan Deng, name: Yang Ye, name: Jinfa Huang, name: Bin Lin, name: Chongyang Ma, name: Jiebo Luo, name: Li Yuan
Published: 2025-05-26 17:59:46

Subject-to-Video (S2V) generation aims to create videos that faithfully
incorporate reference content, providing enhanced flexibility in the production
of videos. To establish the infrastructure for S2V generation, we propose
OpenS2V-Nexus, consisting of (i) OpenS2V-Eval, a fine-grained benchmark, and
(ii) OpenS2V-5M, a million-scale dataset. In contrast to existing S2V
benchmarks inherited from VBench that focus on global and coarse-grained
assessment of generated videos, OpenS2V-Eval focuses on the model's ability to
generate subject-consistent videos with natural subject appearance and identity
fidelity. For these purposes, OpenS2V-Eval introduces 180 prompts from seven
major categories of S2V, which incorporate both real and synthetic test data.
Furthermore, to accurately align human preferences with S2V benchmarks, we
propose three automatic metrics, NexusScore, NaturalScore and GmeScore, to
separately quantify subject consistency, naturalness, and text relevance in
generated videos. Building on this, we conduct a comprehensive evaluation of 16
representative S2V models, highlighting their strengths and weaknesses across
different content. Moreover, we create the first open-source large-scale S2V
generation dataset OpenS2V-5M, which consists of five million high-quality 720P
subject-text-video triples. Specifically, we ensure subject-information
diversity in our dataset by (1) segmenting subjects and building pairing
information via cross-video associations and (2) prompting GPT-Image-1 on raw
frames to synthesize multi-view representations. Through OpenS2V-Nexus, we
deliver a robust infrastructure to accelerate future S2V generation research.
Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering
  Target Atoms
Authors: name: Mengru Wang, name: Ziwen Xu, name: Shengyu Mao, name: Shumin Deng, name: Zhaopeng Tu, name: Huajun Chen, name: Ningyu Zhang
Published: 2025-05-23 17:59:18

Precise control over language model generation is vital for ensuring both
safety and reliability. Although prompt engineering and steering are commonly
used to intervene in model behaviors, the vast number of parameters in models
often results in highly intertwined internal representations. This
interdependency can limit control precision and sometimes lead to unintended
side effects. Recent research has explored the use of sparse autoencoders (SAE)
to disentangle knowledge in high-dimensional spaces for steering. However,
these applications have been limited to toy tasks owing to the nontrivial issue
of locating atomic knowledge components. In this paper, we propose Steering
Target Atoms (STA), a novel method that isolates and manipulates disentangled
knowledge components to enhance safety. Comprehensive experiments demonstrate
the effectiveness of our approach. Further analysis reveals that steering
exhibits superior robustness and flexibility, particularly in adversarial
scenarios. We also apply the steering strategy to the large reasoning model,
confirming its effectiveness in precise reasoning control.
OmniConsistency: Learning Style-Agnostic Consistency from Paired
  Stylization Data
Authors: name: Yiren Song, name: Cheng Liu, name: Mike Zheng Shou
Published: 2025-05-24 01:00:20

Diffusion models have advanced image stylization significantly, yet two core
challenges persist: (1) maintaining consistent stylization in complex scenes,
particularly identity, composition, and fine details, and (2) preventing style
degradation in image-to-image pipelines with style LoRAs. GPT-4o's exceptional
stylization consistency highlights the performance gap between open-source
methods and proprietary models. To bridge this gap, we propose
OmniConsistency, a universal consistency plugin leveraging large-scale
Diffusion Transformers (DiTs). OmniConsistency contributes: (1) an in-context
consistency learning framework trained on aligned image pairs for robust
generalization; (2) a two-stage progressive learning strategy decoupling style
learning from consistency preservation to mitigate style degradation; and (3) a
fully plug-and-play design compatible with arbitrary style LoRAs under the Flux
framework. Extensive experiments show that OmniConsistency significantly
enhances visual coherence and aesthetic quality, achieving performance
comparable to commercial state-of-the-art model GPT-4o.
Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via
  Semantic-Aware Permutation
Authors: name: Shuo Yang, name: Haocheng Xi, name: Yilong Zhao, name: Muyang Li, name: Jintao Zhang, name: Han Cai, name: Yujun Lin, name: Xiuyu Li, name: Chenfeng Xu, name: Kelly Peng, name: Jianfei Chen, name: Song Han, name: Kurt Keutzer, name: Ion Stoica
Published: 2025-05-24 21:30:29

Diffusion Transformers (DiTs) are essential for video generation but suffer
from significant latency due to the quadratic complexity of attention. By
computing only critical tokens, sparse attention reduces computational costs
and offers a promising acceleration approach. However, we identify that
existing methods fail to approach optimal generation quality under the same
computation budget for two reasons: (1) Inaccurate critical token
identification: current methods cluster tokens based on position rather than
semantics, leading to imprecise aggregated representations. (2) Excessive
computation waste: critical tokens are scattered among non-critical ones,
leading to wasted computation on GPUs, which are optimized for processing
contiguous tokens. In this paper, we propose SVG2, a training-free framework
that maximizes identification accuracy and minimizes computation waste,
achieving a Pareto frontier trade-off between generation quality and
efficiency. The core of SVG2 is semantic-aware permutation, which clusters and
reorders tokens based on semantic similarity using k-means. This approach
ensures both a precise cluster representation, improving identification
accuracy, and a densified layout of critical tokens, enabling efficient
computation without padding. Additionally, SVG2 integrates top-p dynamic budget
control and customized kernel implementations, achieving up to 2.30x and 1.89x
speedup while maintaining a PSNR of up to 30 and 26 on HunyuanVideo and Wan
2.1, respectively.
Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?
Authors: name: Junhao Cheng, name: Yuying Ge, name: Teng Wang, name: Yixiao Ge, name: Jing Liao, name: Ying Shan
Published: 2025-05-27 16:05:01

Recent advances in CoT reasoning and RL post-training have been reported to
enhance video reasoning capabilities of MLLMs. This progress naturally raises a
question: can these models perform complex video reasoning in a manner
comparable to human experts? However, existing video benchmarks primarily
evaluate visual perception and grounding abilities, with questions that can be
answered based on explicit prompts or isolated visual cues. Such benchmarks do
not fully capture the intricacies of real-world reasoning, where humans must
actively search for, integrate, and analyze multiple clues before reaching a
conclusion. To address this issue, we present Video-Holmes, a benchmark
inspired by the reasoning process of Sherlock Holmes, designed to evaluate the
complex video reasoning capabilities of MLLMs. Video-Holmes consists of 1,837
questions derived from 270 manually annotated suspense short films, which spans
seven carefully designed tasks. Each task is constructed by first identifying
key events and causal relationships within films, and then designing questions
that require models to actively locate and connect multiple relevant visual
clues scattered across different video segments. Our comprehensive evaluation
of state-of-the-art MLLMs reveals that, while these models generally excel at
visual perception, they encounter substantial difficulties with integrating
information and often miss critical clues. For example, the best-performing
model, Gemini-2.5-Pro, achieves an accuracy of only 45%, with most models
scoring below 40%. We aim that Video-Holmes can serve as a "Holmes-test" for
multimodal reasoning, motivating models to reason more like humans and
emphasizing the ongoing challenges in this field. The benchmark is released in
https://github.com/TencentARC/Video-Holmes.
MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent
  Systems
Authors: name: Xuanming Zhang, name: Yuxuan Chen, name: Min-Hsuan Yeh, name: Yixuan Li
Published: 2025-05-25 02:32:57

Human social interactions depend on the ability to infer others' unspoken
intentions, emotions, and beliefs-a cognitive skill grounded in the
psychological concept of Theory of Mind (ToM). While large language models
(LLMs) excel in semantic understanding tasks, they struggle with the ambiguity
and contextual nuance inherent in human communication. To bridge this gap, we
introduce MetaMind, a multi-agent framework inspired by psychological theories
of metacognition, designed to emulate human-like social reasoning. MetaMind
decomposes social understanding into three collaborative stages: (1) a
Theory-of-Mind Agent generates hypotheses user mental states (e.g., intent,
emotion), (2) a Domain Agent refines these hypotheses using cultural norms and
ethical constraints, and (3) a Response Agent generates contextually
appropriate responses while validating alignment with inferred intent. Our
framework achieves state-of-the-art performance across three challenging
benchmarks, with 35.7% improvement in real-world social scenarios and 6.2% gain
in ToM reasoning. Notably, it enables LLMs to match human-level performance on
key ToM tasks for the first time. Ablation studies confirm the necessity of all
components, which showcase the framework's ability to balance contextual
plausibility, social appropriateness, and user adaptation. This work advances
AI systems toward human-like social intelligence, with applications in
empathetic dialogue and culturally sensitive interactions. Code is available at
https://github.com/XMZhangAI/MetaMind.
MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in
  Video Scenarios
Authors: name: Yang Shi, name: Huanqian Wang, name: Wulin Xie, name: Huanyao Zhang, name: Lijie Zhao, name: Yi-Fan Zhang, name: Xinfeng Li, name: Chaoyou Fu, name: Zhuoer Wen, name: Wenting Liu, name: Zhuoran Zhang, name: Xinlong Chen, name: Bohan Zeng, name: Sihan Yang, name: Yuanxing Zhang, name: Pengfei Wan, name: Haotian Wang, name: Wenjing Yang
Published: 2025-05-27 15:27:46

Multimodal Large Language Models (MLLMs) have achieved considerable accuracy
in Optical Character Recognition (OCR) from static images. However, their
efficacy in video OCR is significantly diminished due to factors such as motion
blur, temporal variations, and visual effects inherent in video content. To
provide clearer guidance for training practical MLLMs, we introduce the
MME-VideoOCR benchmark, which encompasses a comprehensive range of video OCR
application scenarios. MME-VideoOCR features 10 task categories comprising 25
individual tasks and spans 44 diverse scenarios. These tasks extend beyond text
recognition to incorporate deeper comprehension and reasoning of textual
content within videos. The benchmark consists of 1,464 videos with varying
resolutions, aspect ratios, and durations, along with 2,000 meticulously
curated, manually annotated question-answer pairs. We evaluate 18
state-of-the-art MLLMs on MME-VideoOCR, revealing that even the best-performing
model (Gemini-2.5 Pro) achieves an accuracy of only 73.7%. Fine-grained
analysis indicates that while existing MLLMs demonstrate strong performance on
tasks where relevant texts are contained within a single or few frames, they
exhibit limited capability in effectively handling tasks that demand holistic
video comprehension. These limitations are especially evident in scenarios that
require spatio-temporal reasoning, cross-frame information integration, or
resistance to language prior bias. Our findings also highlight the importance
of high-resolution visual input and sufficient temporal coverage for reliable
OCR in dynamic video scenarios.
Seeing is Believing, but How Much? A Comprehensive Analysis of
  Verbalized Calibration in Vision-Language Models
Authors: name: Weihao Xuan, name: Qingcheng Zeng, name: Heli Qi, name: Junjue Wang, name: Naoto Yokoya
Published: 2025-05-26 17:16:36

Uncertainty quantification is essential for assessing the reliability and
trustworthiness of modern AI systems. Among existing approaches, verbalized
uncertainty, where models express their confidence through natural language,
has emerged as a lightweight and interpretable solution in large language
models (LLMs). However, its effectiveness in vision-language models (VLMs)
remains insufficiently studied. In this work, we conduct a comprehensive
evaluation of verbalized confidence in VLMs, spanning three model categories,
four task domains, and three evaluation scenarios. Our results show that
current VLMs often display notable miscalibration across diverse tasks and
settings. Notably, visual reasoning models (i.e., thinking with images)
consistently exhibit better calibration, suggesting that modality-specific
reasoning is critical for reliable uncertainty estimation. To further address
calibration challenges, we introduce Visual Confidence-Aware Prompting, a
two-stage prompting strategy that improves confidence alignment in multimodal
settings. Overall, our study highlights the inherent miscalibration in VLMs
across modalities. More broadly, our findings underscore the fundamental
importance of modality alignment and model faithfulness in advancing reliable
multimodal systems.
The Pragmatic Mind of Machines: Tracing the Emergence of Pragmatic
  Competence in Large Language Models
Authors: name: Kefan Yu, name: Qingcheng Zeng, name: Weihao Xuan, name: Wanxin Li, name: Jingyi Wu, name: Rob Voigt
Published: 2025-05-24 04:24:59

Current large language models (LLMs) have demonstrated emerging capabilities
in social intelligence tasks, including implicature resolution (Sravanthi et
al. (2024)) and theory-of-mind reasoning (Shapira et al. (2024)), both of which
require substantial pragmatic understanding. However, how LLMs acquire this
competence throughout the training process remains poorly understood. In this
work, we introduce ALTPRAG, a dataset grounded in the pragmatic concept of
alternatives, designed to evaluate whether LLMs at different training stages
can accurately infer nuanced speaker intentions. Each instance pairs two
contextually appropriate but pragmatically distinct continuations, enabling
fine-grained assessment of both pragmatic interpretation and contrastive
reasoning. We systematically evaluate 22 LLMs across key training stages:
pre-training, supervised fine-tuning (SFT), and preference optimization, to
examine the development of pragmatic competence. Our results show that even
base models exhibit notable sensitivity to pragmatic cues, which improves
consistently with increases in model and data scale. Additionally, SFT and RLHF
contribute further gains, particularly in cognitive-pragmatic reasoning. These
findings highlight pragmatic competence as an emergent and compositional
property of LLM training and offer new insights for aligning models with human
communicative norms.