Orionnnn/hub-stats huggingface dataset:
InterActHuman: Multi-Concept Human Animation with Layout-Aligned Audio
  Conditions
Authors: name: Zhenzhi Wang, name: Jiaqi Yang, name: Jianwen Jiang, name: Chao Liang, name: Gaojie Lin, name: Zerong Zheng, name: Ceyuan Yang, name: Dahua Lin
Published: 2025-06-11 17:57:09

End-to-end human animation with rich multi-modal conditions, e.g., text,
image and audio has achieved remarkable advancements in recent years. However,
most existing methods could only animate a single subject and inject conditions
in a global manner, ignoring scenarios that multiple concepts could appears in
the same video with rich human-human interactions and human-object
interactions. Such global assumption prevents precise and per-identity control
of multiple concepts including humans and objects, therefore hinders
applications. In this work, we discard the single-entity assumption and
introduce a novel framework that enforces strong, region-specific binding of
conditions from modalities to each identity's spatiotemporal footprint. Given
reference images of multiple concepts, our method could automatically infer
layout information by leveraging a mask predictor to match appearance cues
between the denoised video and each reference appearance. Furthermore, we
inject local audio condition into its corresponding region to ensure
layout-aligned modality matching in a iterative manner. This design enables the
high-quality generation of controllable multi-concept human-centric videos.
Empirical results and ablation studies validate the effectiveness of our
explicit layout control for multi-modal conditions compared to implicit
counterparts and other existing methods.
Branched Schrödinger Bridge Matching
Authors: name: Sophia Tang, name: Yinuo Zhang, name: Alexander Tong, name: Pranam Chatterjee
Published: 2025-06-10 17:29:48

Predicting the intermediate trajectories between an initial and target
distribution is a central problem in generative modeling. Existing approaches,
such as flow matching and Schr\"odinger Bridge Matching, effectively learn
mappings between two distributions by modeling a single stochastic path.
However, these methods are inherently limited to unimodal transitions and
cannot capture branched or divergent evolution from a common origin to multiple
distinct outcomes. To address this, we introduce Branched Schr\"odinger Bridge
Matching (BranchSBM), a novel framework that learns branched Schr\"odinger
bridges. BranchSBM parameterizes multiple time-dependent velocity fields and
growth processes, enabling the representation of population-level divergence
into multiple terminal distributions. We show that BranchSBM is not only more
expressive but also essential for tasks involving multi-path surface
navigation, modeling cell fate bifurcations from homogeneous progenitor states,
and simulating diverging cellular responses to perturbations.
Seedance 1.0: Exploring the Boundaries of Video Generation Models
Authors: name: Yu Gao, name: Haoyuan Guo, name: Tuyen Hoang, name: Weilin Huang, name: Lu Jiang, name: Fangyuan Kong, name: Huixia Li, name: Jiashi Li, name: Liang Li, name: Xiaojie Li, name: Xunsong Li, name: Yifu Li, name: Shanchuan Lin, name: Zhijie Lin, name: Jiawei Liu, name: Shu Liu, name: Xiaonan Nie, name: Zhiwu Qing, name: Yuxi Ren, name: Li Sun, name: Zhi Tian, name: Rui Wang, name: Sen Wang, name: Guoqiang Wei, name: Guohong Wu, name: Jie Wu, name: Ruiqi Xia, name: Fei Xiao, name: Xuefeng Xiao, name: Jiangqiao Yan, name: Ceyuan Yang, name: Jianchao Yang, name: Runkai Yang, name: Tao Yang, name: Yihang Yang, name: Zilyu Ye, name: Xuejiao Zeng, name: Yan Zeng, name: Heng Zhang, name: Yang Zhao, name: Xiaozheng Zheng, name: Peihao Zhu, name: Jiaxin Zou, name: Feilong Zuo
Published: 2025-06-10 17:56:11

Notable breakthroughs in diffusion modeling have propelled rapid improvements
in video generation, yet current foundational model still face critical
challenges in simultaneously balancing prompt following, motion plausibility,
and visual quality. In this report, we introduce Seedance 1.0, a
high-performance and inference-efficient video foundation generation model that
integrates several core technical improvements: (i) multi-source data curation
augmented with precision and meaningful video captioning, enabling
comprehensive learning across diverse scenarios; (ii) an efficient architecture
design with proposed training paradigm, which allows for natively supporting
multi-shot generation and jointly learning of both text-to-video and
image-to-video tasks. (iii) carefully-optimized post-training approaches
leveraging fine-grained supervised fine-tuning, and video-specific RLHF with
multi-dimensional reward mechanisms for comprehensive performance improvements;
(iv) excellent model acceleration achieving ~10x inference speedup through
multi-stage distillation strategies and system-level optimizations. Seedance
1.0 can generate a 5-second video at 1080p resolution only with 41.4 seconds
(NVIDIA-L20). Compared to state-of-the-art video generation models, Seedance
1.0 stands out with high-quality and fast video generation having superior
spatiotemporal fluidity with structural stability, precise instruction
adherence in complex multi-subject contexts, native multi-shot narrative
coherence with consistent subject representation.
SAFE: Multitask Failure Detection for Vision-Language-Action Models
Authors: name: Qiao Gu, name: Yuanliang Ju, name: Shengxiang Sun, name: Igor Gilitschenski, name: Haruki Nishimura, name: Masha Itkina, name: Florian Shkurti
Published: 2025-06-11 16:59:13

While vision-language-action models (VLAs) have shown promising robotic
behaviors across a diverse set of manipulation tasks, they achieve limited
success rates when deployed on novel tasks out-of-the-box. To allow these
policies to safely interact with their environments, we need a failure detector
that gives a timely alert such that the robot can stop, backtrack, or ask for
help. However, existing failure detectors are trained and tested only on one or
a few specific tasks, while VLAs require the detector to generalize and detect
failures also in unseen tasks and novel environments. In this paper, we
introduce the multitask failure detection problem and propose SAFE, a failure
detector for generalist robot policies such as VLAs. We analyze the VLA feature
space and find that VLAs have sufficient high-level knowledge about task
success and failure, which is generic across different tasks. Based on this
insight, we design SAFE to learn from VLA internal features and predict a
single scalar indicating the likelihood of task failure. SAFE is trained on
both successful and failed rollouts, and is evaluated on unseen tasks. SAFE is
compatible with different policy architectures. We test it on OpenVLA, pi_0,
and pi_0-FAST in both simulated and real-world environments extensively. We
compare SAFE with diverse baselines and show that SAFE achieves
state-of-the-art failure detection performance and the best trade-off between
accuracy and detection time using conformal prediction. More qualitative
results can be found at https://vla-safe.github.io/.
SeerAttention-R: Sparse Attention Adaptation for Long Reasoning
Authors: name: Yizhao Gao, name: Shuming Guo, name: Shijie Cao, name: Yuqing Xia, name: Yu Cheng, name: Lei Wang, name: Lingxiao Ma, name: Yutao Sun, name: Tianzhu Ye, name: Li Dong, name: Hayden Kwok-Hay So, name: Yu Hua, name: Ting Cao, name: Fan Yang, name: Mao Yang
Published: 2025-06-10 15:17:26

We introduce SeerAttention-R, a sparse attention framework specifically
tailored for the long decoding of reasoning models. Extended from
SeerAttention, SeerAttention-R retains the design of learning attention
sparsity through a self-distilled gating mechanism, while removing query
pooling to accommodate auto-regressive decoding. With a lightweight plug-in
gating, SeerAttention-R is flexible and can be easily integrated into existing
pretrained model without modifying the original parameters. We demonstrate that
SeerAttention-R, trained on just 0.4B tokens, maintains near-lossless reasoning
accuracy with 4K token budget in AIME benchmark under large sparse attention
block sizes (64/128). Using TileLang, we develop a highly optimized sparse
decoding kernel that achieves near-theoretical speedups of up to 9x over
FlashAttention-3 on H100 GPU at 90% sparsity. Code is available at:
https://github.com/microsoft/SeerAttention.
PlayerOne: Egocentric World Simulator
Authors: name: Yuanpeng Tu, name: Hao Luo, name: Xi Chen, name: Xiang Bai, name: Fan Wang, name: Hengshuang Zhao
Published: 2025-06-11 17:59:53

We introduce PlayerOne, the first egocentric realistic world simulator,
facilitating immersive and unrestricted exploration within vividly dynamic
environments. Given an egocentric scene image from the user, PlayerOne can
accurately construct the corresponding world and generate egocentric videos
that are strictly aligned with the real scene human motion of the user captured
by an exocentric camera. PlayerOne is trained in a coarse-to-fine pipeline that
first performs pretraining on large-scale egocentric text-video pairs for
coarse-level egocentric understanding, followed by finetuning on synchronous
motion-video data extracted from egocentric-exocentric video datasets with our
automatic construction pipeline. Besides, considering the varying importance of
different components, we design a part-disentangled motion injection scheme,
enabling precise control of part-level movements. In addition, we devise a
joint reconstruction framework that progressively models both the 4D scene and
video frames, ensuring scene consistency in the long-form video generation.
Experimental results demonstrate its great generalization ability in precise
control of varying human movements and worldconsistent modeling of diverse
scenarios. It marks the first endeavor into egocentric real-world simulation
and can pave the way for the community to delve into fresh frontiers of world
modeling and its diverse applications.
ComfyUI-R1: Exploring Reasoning Models for Workflow Generation
Authors: name: Zhenran Xu, name: Yiyu Wang, name: Xue Yang, name: Longyue Wang, name: Weihua Luo, name: Kaifu Zhang, name: Baotian Hu, name: Min Zhang
Published: 2025-06-11 14:35:15

AI-generated content has evolved from monolithic models to modular workflows,
particularly on platforms like ComfyUI, enabling customization in creative
pipelines. However, crafting effective workflows requires great expertise to
orchestrate numerous specialized components, presenting a steep learning curve
for users. To address this challenge, we introduce ComfyUI-R1, the first large
reasoning model for automated workflow generation. Starting with our curated
dataset of 4K workflows, we construct long chain-of-thought (CoT) reasoning
data, including node selection, workflow planning, and code-level workflow
representation. ComfyUI-R1 is trained through a two-stage framework: (1) CoT
fine-tuning for cold start, adapting models to the ComfyUI domain; (2)
reinforcement learning for incentivizing reasoning capability, guided by a
fine-grained rule-metric hybrid reward, ensuring format validity, structural
integrity, and node-level fidelity. Experiments show that our 7B-parameter
model achieves a 97\% format validity rate, along with high pass rate,
node-level and graph-level F1 scores, significantly surpassing prior
state-of-the-art methods that employ leading closed-source models such as
GPT-4o and Claude series. Further analysis highlights the critical role of the
reasoning process and the advantage of transforming workflows into code.
Qualitative comparison reveals our strength in synthesizing intricate workflows
with diverse nodes, underscoring the potential of long CoT reasoning in AI art
creation.
SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner
Authors: name: Lei Zhang, name: Jiaxi Yang, name: Min Yang, name: Jian Yang, name: Mouxiang Chen, name: Jiajun Zhang, name: Zeyu Cui, name: Binyuan Hui, name: Junyang Lin
Published: 2025-06-10 17:23:33

We introduce **SWE-Flow**, a novel data synthesis framework grounded in
Test-Driven Development (TDD). Unlike existing software engineering data that
rely on human-submitted issues, **SWE-Flow** automatically infers incremental
development steps directly from unit tests, which inherently encapsulate
high-level requirements. The core of **SWE-Flow** is the construction of a
Runtime Dependency Graph (RDG), which precisely captures function interactions,
enabling the generation of a structured, step-by-step *development schedule*.
At each step, **SWE-Flow** produces a partial codebase, the corresponding unit
tests, and the necessary code modifications, resulting in fully verifiable TDD
tasks. With this approach, we generated 16,061 training instances and 2,020
test instances from real-world GitHub projects, creating the **SWE-Flow-Eval**
benchmark. Our experiments show that fine-tuning open model on this dataset
significantly improves performance in TDD-based coding. To facilitate further
research, we release all code, datasets, models, and Docker images at
[Github](https://github.com/Hambaobao/SWE-Flow).
Seeing Voices: Generating A-Roll Video from Audio with Mirage
Authors: name: Aditi Sundararaman, name: Amogh Adishesha, name: Andrew Jaegle, name: Dan Bigioi, name: Hyoung-Kyu Song, name: Jon Kyl, name: Justin Mao, name: Kevin Lan, name: Mojtaba Komeili, name: ShahRukh Athar, name: Sheila Babayan, name: Stanislau Beliasau, name: William Buchwalter
Published: 2025-06-09 22:56:02

From professional filmmaking to user-generated content, creators and
consumers have long recognized that the power of video depends on the
harmonious integration of what we hear (the video's audio track) with what we
see (the video's image sequence). Current approaches to video generation either
ignore sound to focus on general-purpose but silent image sequence generation
or address both visual and audio elements but focus on restricted application
domains such as re-dubbing. We introduce Mirage, an audio-to-video foundation
model that excels at generating realistic, expressive output imagery from
scratch given an audio input. When integrated with existing methods for speech
synthesis (text-to-speech, or TTS), Mirage results in compelling multimodal
video. When trained on audio-video footage of people talking (A-roll) and
conditioned on audio containing speech, Mirage generates video of people
delivering a believable interpretation of the performance implicit in input
audio. Our central technical contribution is a unified method for training
self-attention-based audio-to-video generation models, either from scratch or
given existing weights. This methodology allows Mirage to retain generality as
an approach to audio-to-video generation while producing outputs of superior
subjective quality to methods that incorporate audio-specific architectures or
loss components specific to people, speech, or details of how images or audio
are captured. We encourage readers to watch and listen to the results of Mirage
for themselves (see paper and comments for links).
DRAGged into Conflicts: Detecting and Addressing Conflicting Sources in
  Search-Augmented LLMs
Authors: name: Arie Cattan, name: Alon Jacovi, name: Ori Ram, name: Jonathan Herzig, name: Roee Aharoni, name: Sasha Goldshtein, name: Eran Ofek, name: Idan Szpektor, name: Avi Caciularu
Published: 2025-06-10 06:52:57

Retrieval Augmented Generation (RAG) is a commonly used approach for
enhancing large language models (LLMs) with relevant and up-to-date
information. However, the retrieved sources can often contain conflicting
information and it remains unclear how models should address such
discrepancies. In this work, we first propose a novel taxonomy of knowledge
conflict types in RAG, along with the desired model behavior for each type. We
then introduce CONFLICTS, a high-quality benchmark with expert annotations of
conflict types in a realistic RAG setting. CONFLICTS is the first benchmark
that enables tracking progress on how models address a wide range of knowledge
conflicts. We conduct extensive experiments on this benchmark, showing that
LLMs often struggle to appropriately resolve conflicts between sources. While
prompting LLMs to explicitly reason about the potential conflict in the
retrieved documents significantly improves the quality and appropriateness of
their responses, substantial room for improvement in future research remains.
MMRefine: Unveiling the Obstacles to Robust Refinement in Multimodal
  Large Language Models
Authors: name: Gio Paik, name: Geewook Kim, name: Jinbae Im
Published: 2025-06-05 07:11:36

This paper introduces MMRefine, a MultiModal Refinement benchmark designed to
evaluate the error refinement capabilities of Multimodal Large Language Models
(MLLMs). As the emphasis shifts toward enhancing reasoning during inference,
MMRefine provides a framework that evaluates MLLMs' abilities to detect and
correct errors across six distinct scenarios beyond just comparing final
accuracy before and after refinement. Furthermore, the benchmark analyzes the
refinement performance by categorizing errors into six error types. Experiments
with various open and closed MLLMs reveal bottlenecks and factors impeding
refinement performance, highlighting areas for improvement in effective
reasoning enhancement. Our code and dataset are publicly available at
https://github.com/naver-ai/MMRefine.
Interpretable and Reliable Detection of AI-Generated Images via Grounded
  Reasoning in MLLMs
Authors: name: Yikun Ji, name: Hong Yan, name: Jun Lan, name: Huijia Zhu, name: Weiqiang Wang, name: Qi Fan, name: Liqing Zhang, name: Jianfu Zhang
Published: 2025-06-08 08:47:44

The rapid advancement of image generation technologies intensifies the demand
for interpretable and robust detection methods. Although existing approaches
often attain high accuracy, they typically operate as black boxes without
providing human-understandable justifications. Multi-modal Large Language
Models (MLLMs), while not originally intended for forgery detection, exhibit
strong analytical and reasoning capabilities. When properly fine-tuned, they
can effectively identify AI-generated images and offer meaningful explanations.
However, existing MLLMs still struggle with hallucination and often fail to
align their visual interpretations with actual image content and human
reasoning. To bridge this gap, we construct a dataset of AI-generated images
annotated with bounding boxes and descriptive captions that highlight synthesis
artifacts, establishing a foundation for human-aligned visual-textual grounded
reasoning. We then finetune MLLMs through a multi-stage optimization strategy
that progressively balances the objectives of accurate detection, visual
localization, and coherent textual explanation. The resulting model achieves
superior performance in both detecting AI-generated images and localizing
visual flaws, significantly outperforming baseline methods.
QQSUM: A Novel Task and Model of Quantitative Query-Focused
  Summarization for Review-based Product Question Answering
Authors: name: An Quang Tang, name: Xiuzhen Zhang, name: Minh Ngoc Dinh, name: Zhuang Li
Published: 2025-06-04 14:50:32

Review-based Product Question Answering (PQA) allows e-commerce platforms to
automatically address customer queries by leveraging insights from user
reviews. However, existing PQA systems generate answers with only a single
perspective, failing to capture the diversity of customer opinions. In this
paper we introduce a novel task Quantitative Query-Focused Summarization
(QQSUM), which aims to summarize diverse customer opinions into representative
Key Points (KPs) and quantify their prevalence to effectively answer user
queries. While Retrieval-Augmented Generation (RAG) shows promise for PQA, its
generated answers still fall short of capturing the full diversity of
viewpoints. To tackle this challenge, our model QQSUM-RAG, which extends RAG,
employs few-shot learning to jointly train a KP-oriented retriever and a KP
summary generator, enabling KP-based summaries that capture diverse and
representative opinions. Experimental results demonstrate that QQSUM-RAG
achieves superior performance compared to state-of-the-art RAG baselines in
both textual quality and quantification accuracy of opinions. Our source code
is available at: https://github.com/antangrocket1312/QQSUMM
Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction
Authors: name: Junhong Shen, name: Hao Bai, name: Lunjun Zhang, name: Yifei Zhou, name: Amrith Setlur, name: Shengbang Tong, name: Diego Caples, name: Nan Jiang, name: Tong Zhang, name: Ameet Talwalkar, name: Aviral Kumar
Published: 2025-06-09 17:50:02

The current paradigm of test-time scaling relies on generating long reasoning
traces ("thinking" more) before producing a response. In agent problems that
require interaction, this can be done by generating thinking traces before
acting in the world. However, this process does not allow agents to acquire new
information from the environment or adapt their behavior over time. In this
work, we propose to scale test-time interaction, an untapped dimension of
test-time scaling that increases the agent's interaction horizon to enable
running rich behaviors such as exploration, backtracking, and dynamic
re-planning within a single rollout. To demonstrate the promise of this scaling
dimension, we study the domain of web agents. We first show that even
prompting-based interaction scaling without any training can improve task
success on web benchmarks non-trivially. Building on this, we introduce TTI
(Test-Time Interaction), a curriculum-based online reinforcement learning (RL)
approach that trains agents by adaptively adjusting their rollout lengths.
Using a Gemma 3 12B model, TTI produces state-of-the-art open-source, open-data
web agents on WebVoyager and WebArena benchmarks. We further show that TTI
enables agents to balance exploration and exploitation adaptively. Our results
establish interaction scaling as a powerful, complementary axis to scaling
per-step compute, offering new avenues for training adaptive agents.
Institutional Books 1.0: A 242B token dataset from Harvard Library's
  collections, refined for accuracy and usability
Authors: name: Matteo Cargnelutti, name: Catherine Brobston, name: John Hess, name: Jack Cushman, name: Kristi Mukk, name: Aristana Scourtas, name: Kyle Courtney, name: Greg Leppert, name: Amanda Watson, name: Martha Whitehead, name: Jonathan Zittrain
Published: 2025-06-10 00:11:30

Large language models (LLMs) use data to learn about the world in order to
produce meaningful correlations and predictions. As such, the nature, scale,
quality, and diversity of the datasets used to train these models, or to
support their work at inference time, have a direct impact on their quality.
The rapid development and adoption of LLMs of varying quality has brought into
focus the scarcity of publicly available, high-quality training data and
revealed an urgent need to ground the stewardship of these datasets in
sustainable practices with clear provenance chains. To that end, this technical
report introduces Institutional Books 1.0, a large collection of public domain
books originally digitized through Harvard Library's participation in the
Google Books project, beginning in 2006. Working with Harvard Library, we
extracted, analyzed, and processed these volumes into an extensively-documented
dataset of historic texts. This analysis covers the entirety of Harvard
Library's collection scanned as part of that project, originally spanning
1,075,899 volumes written in over 250 different languages for a total of
approximately 250 billion tokens. As part of this initial release, the
OCR-extracted text (original and post-processed) as well as the metadata
(bibliographic, source, and generated) of the 983,004 volumes, or 242B tokens,
identified as being in the public domain have been made available. This report
describes this project's goals and methods as well as the results of the
analyses we performed, all in service of making this historical collection more
accessible and easier for humans and machines alike to filter, read and use.