Orionnnn/hub-stats huggingface dataset:
From Intention to Execution: Probing the Generalization Boundaries of
  Vision-Language-Action Models
Authors: name: Irving Fang, name: Juexiao Zhang, name: Shengbang Tong, name: Chen Feng
Published: 2025-06-11 16:52:18

One promise that Vision-Language-Action (VLA) models hold over traditional
imitation learning for robotics is to leverage the broad generalization
capabilities of large Vision-Language Models (VLMs) to produce versatile,
"generalist" robot policies. However, current evaluations of VLAs remain
insufficient. Traditional imitation learning benchmarks are unsuitable due to
the lack of language instructions. Emerging benchmarks for VLAs that
incorporate language often come with limited evaluation tasks and do not intend
to investigate how much VLM pretraining truly contributes to the generalization
capabilities of the downstream robotic policy. Meanwhile, much research relies
on real-world robot setups designed in isolation by different institutions,
which creates a barrier for reproducibility and accessibility. To address this
gap, we introduce a unified probing suite of 50 simulation-based tasks across
10 subcategories spanning language instruction, vision, and objects. We
systematically evaluate several state-of-the-art VLA architectures on this
suite to understand their generalization capability. Our results show that
while VLM backbones endow VLAs with robust perceptual understanding and high
level planning, which we refer to as good intentions, this does not reliably
translate into precise motor execution: when faced with out-of-distribution
observations, policies often exhibit coherent intentions, but falter in action
execution. Moreover, finetuning on action data can erode the original VLM's
generalist reasoning abilities. We release our task suite and evaluation code
to serve as a standardized benchmark for future VLAs and to drive research on
closing the perception-to-action gap. More information, including the source
code, can be found at https://ai4ce.github.io/INT-ACT/
Optimizing Multilingual Text-To-Speech with Accents & Emotions
Authors: name: Pranav Pawar, name: Akshansh Dwivedi, name: Jenish Boricha, name: Himanshu Gohil, name: Aditya Dubey
Published: 2025-06-19 13:35:05

State-of-the-art text-to-speech (TTS) systems realize high naturalness in
monolingual environments, synthesizing speech with correct multilingual accents
(especially for Indic languages) and context-relevant emotions still poses
difficulty owing to cultural nuance discrepancies in current frameworks. This
paper introduces a new TTS architecture integrating accent along with
preserving transliteration with multi-scale emotion modelling, in particularly
tuned for Hindi and Indian English accent. Our approach extends the Parler-TTS
model by integrating A language-specific phoneme alignment hybrid
encoder-decoder architecture, and culture-sensitive emotion embedding layers
trained on native speaker corpora, as well as incorporating a dynamic accent
code switching with residual vector quantization. Quantitative tests
demonstrate 23.7% improvement in accent accuracy (Word Error Rate reduction
from 15.4% to 11.8%) and 85.3% emotion recognition accuracy from native
listeners, surpassing METTS and VECL-TTS baselines. The novelty of the system
is that it can mix code in real time - generating statements such as "Namaste,
let's talk about <Hindi phrase>" with uninterrupted accent shifts while
preserving emotional consistency. Subjective evaluation with 200 users reported
a mean opinion score (MOS) of 4.2/5 for cultural correctness, much better than
existing multilingual systems (p<0.01). This research makes cross-lingual
synthesis more feasible by showcasing scalable accent-emotion disentanglement,
with direct application in South Asian EdTech and accessibility software.
UniFork: Exploring Modality Alignment for Unified Multimodal
  Understanding and Generation
Authors: name: Teng Li, name: Quanfeng Lu, name: Lirui Zhao, name: Hao Li, name: Xizhou Zhu, name: Yu Qiao, name: Jun Zhang, name: Wenqi Shao
Published: 2025-06-20 17:52:31

Unified image understanding and generation has emerged as a promising
paradigm in multimodal artificial intelligence. Despite recent progress, the
optimal architectural design for such unified models remains an open challenge.
In this work, we start by analyzing the modality alignment behaviors of
task-specific expert models for understanding and generation, as well as
current unified models. Our analysis reveals a crucial observation:
understanding tasks benefit from a progressively increasing modality alignment
across network depth, which helps build up semantic information for better
comprehension; In contrast, generation tasks follow a different trend: modality
alignment increases in the early layers but decreases in the deep layers to
recover spatial details. These divergent alignment patterns create a
fundamental conflict in fully shared Transformer backbones, where a uniform
representational flow often leads to performance compromises across two tasks.
Motivated by this finding, we introduce UniFork, a novel Y-shaped architecture
that shares the shallow layers for cross-task representation learning, while
employing task-specific branches in deeper layers to avoid task interference.
This design effectively balances shared learning and task specialization.
Through extensive ablation experiments, we demonstrate that Unifork
consistently outperforms conventional fully shared Transformer architectures,
and achieves performance on par with or better than task-specific models.
Long-term Traffic Simulation with Interleaved Autoregressive Motion and
  Scenario Generation
Authors: name: Xiuyu Yang, name: Shuhan Tan, name: Philipp Krähenbühl
Published: 2025-06-20 17:59:21

An ideal traffic simulator replicates the realistic long-term point-to-point
trip that a self-driving system experiences during deployment. Prior models and
benchmarks focus on closed-loop motion simulation for initial agents in a
scene. This is problematic for long-term simulation. Agents enter and exit the
scene as the ego vehicle enters new regions. We propose InfGen, a unified
next-token prediction model that performs interleaved closed-loop motion
simulation and scene generation. InfGen automatically switches between
closed-loop motion simulation and scene generation mode. It enables stable
long-term rollout simulation. InfGen performs at the state-of-the-art in
short-term (9s) traffic simulation, and significantly outperforms all other
methods in long-term (30s) simulation. The code and model of InfGen will be
released at https://orangesodahub.github.io/InfGen
Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with
  Production-Ready PBR Material
Authors: name: Team Hunyuan3D, name: Shuhui Yang, name: Mingxin Yang, name: Yifei Feng, name: Xin Huang, name: Sheng Zhang, name: Zebin He, name: Di Luo, name: Haolin Liu, name: Yunfei Zhao, name: Qingxiang Lin, name: Zeqiang Lai, name: Xianghui Yang, name: Huiwen Shi, name: Zibo Zhao, name: Bowen Zhang, name: Hongyu Yan, name: Lifu Wang, name: Sicong Liu, name: Jihong Zhang, name: Meng Chen, name: Liang Dong, name: Yiwen Jia, name: Yulin Cai, name: Jiaao Yu, name: Yixuan Tang, name: Dongyuan Guo, name: Junlin Yu, name: Hao Zhang, name: Zheng Ye, name: Peng He, name: Runzhou Wu, name: Shida Wei, name: Chao Zhang, name: Yonghao Tan, name: Yifu Sun, name: Lin Niu, name: Shirui Huang, name: Bojian Zheng, name: Shu Liu, name: Shilin Chen, name: Xiang Yuan, name: Xiaofeng Yang, name: Kai Liu, name: Jianchen Zhu, name: Peng Chen, name: Tian Liu, name: Di Wang, name: Yuhong Liu, name: Linus, name: Jie Jiang, name: Jingwei Huang, name: Chunchao Guo
Published: 2025-06-18 13:14:46

3D AI-generated content (AIGC) is a passionate field that has significantly
accelerated the creation of 3D models in gaming, film, and design. Despite the
development of several groundbreaking models that have revolutionized 3D
generation, the field remains largely accessible only to researchers,
developers, and designers due to the complexities involved in collecting,
processing, and training 3D models. To address these challenges, we introduce
Hunyuan3D 2.1 as a case study in this tutorial. This tutorial offers a
comprehensive, step-by-step guide on processing 3D data, training a 3D
generative model, and evaluating its performance using Hunyuan3D 2.1, an
advanced system for producing high-resolution, textured 3D assets. The system
comprises two core components: the Hunyuan3D-DiT for shape generation and the
Hunyuan3D-Paint for texture synthesis. We will explore the entire workflow,
including data preparation, model architecture, training strategies, evaluation
metrics, and deployment. By the conclusion of this tutorial, you will have the
knowledge to finetune or develop a robust 3D generative model suitable for
applications in gaming, virtual reality, and industrial design.
InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video
  Understanding
Authors: name: Minsoo Kim, name: Kyuhong Shim, name: Jungwook Choi, name: Simyung Chang
Published: 2025-06-18 02:22:14

Modern multimodal large language models (MLLMs) can reason over hour-long
video, yet their key-value (KV) cache grows linearly with time--quickly
exceeding the fixed memory of phones, AR glasses, and edge robots. Prior
compression schemes either assume the whole video and user query are available
offline or must first build the full cache, so memory still scales with stream
length. InfiniPot-V is the first training-free, query-agnostic framework that
enforces a hard, length-independent memory cap for streaming video
understanding. During video encoding it monitors the cache and, once a user-set
threshold is reached, runs a lightweight compression pass that (i) removes
temporally redundant tokens via Temporal-axis Redundancy (TaR) metric and (ii)
keeps semantically significant tokens via Value-Norm (VaN) ranking. Across four
open-source MLLMs and four long-video and two streaming-video benchmarks,
InfiniPot-V cuts peak GPU memory by up to 94%, sustains real-time generation,
and matches or surpasses full-cache accuracy--even in multi-turn dialogues. By
dissolving the KV cache bottleneck without retraining or query knowledge,
InfiniPot-V closes the gap for on-device streaming video assistants.
PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and
  Quantized Attention in Visual Generation Models
Authors: name: Tianchen Zhao, name: Ke Hong, name: Xinhao Yang, name: Xuefeng Xiao, name: Huixia Li, name: Feng Ling, name: Ruiqi Xie, name: Siqi Chen, name: Hongyu Zhu, name: Yichong Zhang, name: Yu Wang
Published: 2025-06-19 06:25:02

In visual generation, the quadratic complexity of attention mechanisms
results in high memory and computational costs, especially for longer token
sequences required in high-resolution image or multi-frame video generation. To
address this, prior research has explored techniques such as sparsification and
quantization. However, these techniques face significant challenges under low
density and reduced bitwidths. Through systematic analysis, we identify that
the core difficulty stems from the dispersed and irregular characteristics of
visual attention patterns. Therefore, instead of introducing specialized
sparsification and quantization design to accommodate such patterns, we propose
an alternative strategy: *reorganizing* the attention pattern to alleviate the
challenges. Inspired by the local aggregation nature of visual feature
extraction, we design a novel **Pattern-Aware token ReOrdering (PARO)**
technique, which unifies the diverse attention patterns into a
hardware-friendly block-wise pattern. This unification substantially simplifies
and enhances both sparsification and quantization. We evaluate the
performance-efficiency trade-offs of various design choices and finalize a
methodology tailored for the unified pattern. Our approach, **PAROAttention**,
achieves video and image generation with lossless metrics, and nearly identical
results from full-precision (FP) baselines, while operating at notably lower
density (~20%-30%) and bitwidth (**INT8/INT4**), achieving a **1.9x** to
**2.7x** end-to-end latency speedup.
Vision-Guided Chunking Is All You Need: Enhancing RAG with Multimodal
  Document Understanding
Authors: name: Vishesh Tripathi, name: Tanmay Odapally, name: Indraneel Das, name: Uday Allu, name: Biddwan Ahmed
Published: 2025-06-19 05:11:43

Retrieval-Augmented Generation (RAG) systems have revolutionized information
retrieval and question answering, but traditional text-based chunking methods
struggle with complex document structures, multi-page tables, embedded figures,
and contextual dependencies across page boundaries. We present a novel
multimodal document chunking approach that leverages Large Multimodal Models
(LMMs) to process PDF documents in batches while maintaining semantic coherence
and structural integrity. Our method processes documents in configurable page
batches with cross-batch context preservation, enabling accurate handling of
tables spanning multiple pages, embedded visual elements, and procedural
content. We evaluate our approach on a curated dataset of PDF documents with
manually crafted queries, demonstrating improvements in chunk quality and
downstream RAG performance. Our vision-guided approach achieves better accuracy
compared to traditional vanilla RAG systems, with qualitative analysis showing
superior preservation of document structure and semantic coherence.
Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weights
Authors: name: Zhiyuan Liang, name: Dongwen Tang, name: Yuhao Zhou, name: Xuanlei Zhao, name: Mingjia Shi, name: Wangbo Zhao, name: Zekai Li, name: Peihao Wang, name: Konstantin Schürholt, name: Damian Borth, name: Michael M. Bronstein, name: Yang You, name: Zhangyang Wang, name: Kai Wang
Published: 2025-06-19 15:38:21

Modern Parameter-Efficient Fine-Tuning (PEFT) methods such as low-rank
adaptation (LoRA) reduce the cost of customizing large language models (LLMs),
yet still require a separate optimization run for every downstream dataset. We
introduce Drag-and-Drop LLMs (\textit{DnD)}, a prompt-conditioned
parameter generator that eliminates per-task training by mapping a handful of
unlabeled task prompts directly to LoRA weight updates. A lightweight text
encoder distills each prompt batch into condition embeddings, which are then
transformed by a cascaded hyper-convolutional decoder into the full set of LoRA
matrices. Once trained in a diverse collection of prompt-checkpoint pairs, DnD
produces task-specific parameters in seconds, yielding i) up to
12,000times lower overhead than full fine-tuning, ii) average gains
up to 30\% in performance over the strongest training LoRAs on unseen
common-sense reasoning, math, coding, and multimodal benchmarks, and iii)
robust cross-domain generalization despite never seeing the target data or
labels. Our results demonstrate that prompt-conditioned parameter generation is
a viable alternative to gradient-based adaptation for rapidly specializing
LLMs. Our project is available at
https://jerryliang24.github.io/DnD{https://jerryliang24.github.io/DnD}.
Reranking-based Generation for Unbiased Perspective Summarization
Authors: name: Narutatsu Ri, name: Nicholas Deas, name: Kathleen McKeown
Published: 2025-06-19 00:01:43

Generating unbiased summaries in real-world settings such as political
perspective summarization remains a crucial application of Large Language
Models (LLMs). Yet, existing evaluation frameworks rely on traditional metrics
for measuring key attributes such as coverage and faithfulness without
verifying their applicability, and efforts to develop improved summarizers are
still nascent. We address these gaps by (1) identifying reliable metrics for
measuring perspective summary quality, and (2) investigating the efficacy of
LLM-based methods beyond zero-shot inference. Namely, we build a test set for
benchmarking metric reliability using human annotations and show that
traditional metrics underperform compared to language model-based metrics,
which prove to be strong evaluators. Using these metrics, we show that
reranking-based methods yield strong results, and preference tuning with
synthetically generated and reranking-labeled data further boosts performance.
Our findings aim to contribute to the reliable evaluation and development of
perspective summarization methods.
DreamCube: 3D Panorama Generation via Multi-plane Synchronization
Authors: name: Yukun Huang, name: Yanning Zhou, name: Jianan Wang, name: Kaiyi Huang, name: Xihui Liu
Published: 2025-06-20 17:55:06

3D panorama synthesis is a promising yet challenging task that demands
high-quality and diverse visual appearance and geometry of the generated
omnidirectional content. Existing methods leverage rich image priors from
pre-trained 2D foundation models to circumvent the scarcity of 3D panoramic
data, but the incompatibility between 3D panoramas and 2D single views limits
their effectiveness. In this work, we demonstrate that by applying multi-plane
synchronization to the operators from 2D foundation models, their capabilities
can be seamlessly extended to the omnidirectional domain. Based on this design,
we further introduce DreamCube, a multi-plane RGB-D diffusion model for 3D
panorama generation, which maximizes the reuse of 2D foundation model priors to
achieve diverse appearances and accurate geometry while maintaining multi-view
consistency. Extensive experiments demonstrate the effectiveness of our
approach in panoramic image generation, panoramic depth estimation, and 3D
scene generation.
Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with
  Hybrid History Condition
Authors: name: Jiaqi Li, name: Junshu Tang, name: Zhiyong Xu, name: Longhuang Wu, name: Yuan Zhou, name: Shuai Shao, name: Tianbao Yu, name: Zhiguo Cao, name: Qinglin Lu
Published: 2025-06-20 17:50:37

Recent advances in diffusion-based and controllable video generation have
enabled high-quality and temporally coherent video synthesis, laying the
groundwork for immersive interactive gaming experiences. However, current
methods face limitations in dynamics, generality, long-term consistency, and
efficiency, which limit the ability to create various gameplay videos. To
address these gaps, we introduce Hunyuan-GameCraft, a novel framework for
high-dynamic interactive video generation in game environments. To achieve
fine-grained action control, we unify standard keyboard and mouse inputs into a
shared camera representation space, facilitating smooth interpolation between
various camera and movement operations. Then we propose a hybrid
history-conditioned training strategy that extends video sequences
autoregressively while preserving game scene information. Additionally, to
enhance inference efficiency and playability, we achieve model distillation to
reduce computational overhead while maintaining consistency across long
temporal sequences, making it suitable for real-time deployment in complex
interactive environments. The model is trained on a large-scale dataset
comprising over one million gameplay recordings across over 100 AAA games,
ensuring broad coverage and diversity, then fine-tuned on a carefully annotated
synthetic dataset to enhance precision and control. The curated game scene data
significantly improves the visual fidelity, realism and action controllability.
Extensive experiments demonstrate that Hunyuan-GameCraft significantly
outperforms existing models, advancing the realism and playability of
interactive game video generation.
VIKI-R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement
  Learning
Authors: name: Li Kang, name: Xiufeng Song, name: Heng Zhou, name: Yiran Qin, name: Jie Yang, name: Xiaohong Liu, name: Philip Torr, name: Lei Bai, name: Zhenfei Yin
Published: 2025-06-10 17:59:44

Coordinating multiple embodied agents in dynamic environments remains a core
challenge in artificial intelligence, requiring both perception-driven
reasoning and scalable cooperation strategies. While recent works have
leveraged large language models (LLMs) for multi-agent planning, a few have
begun to explore vision-language models (VLMs) for visual reasoning. However,
these VLM-based approaches remain limited in their support for diverse
embodiment types. In this work, we introduce VIKI-Bench, the first hierarchical
benchmark tailored for embodied multi-agent cooperation, featuring three
structured levels: agent activation, task planning, and trajectory perception.
VIKI-Bench includes diverse robot embodiments, multi-view visual observations,
and structured supervision signals to evaluate reasoning grounded in visual
inputs. To demonstrate the utility of VIKI-Bench, we propose VIKI-R, a
two-stage framework that fine-tunes a pretrained vision-language model (VLM)
using Chain-of-Thought annotated demonstrations, followed by reinforcement
learning under multi-level reward signals. Our extensive experiments show that
VIKI-R significantly outperforms baselines method across all task levels.
Furthermore, we show that reinforcement learning enables the emergence of
compositional cooperation patterns among heterogeneous agents. Together,
VIKI-Bench and VIKI-R offer a unified testbed and method for advancing
multi-agent, visual-driven cooperation in embodied AI systems.
Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate
  Details
Authors: name: Zeqiang Lai, name: Yunfei Zhao, name: Haolin Liu, name: Zibo Zhao, name: Qingxiang Lin, name: Huiwen Shi, name: Xianghui Yang, name: Mingxin Yang, name: Shuhui Yang, name: Yifei Feng, name: Sheng Zhang, name: Xin Huang, name: Di Luo, name: Fan Yang, name: Fang Yang, name: Lifu Wang, name: Sicong Liu, name: Yixuan Tang, name: Yulin Cai, name: Zebin He, name: Tian Liu, name: Yuhong Liu, name: Jie Jiang, name: Linus, name: Jingwei Huang, name: Chunchao Guo
Published: 2025-06-19 17:57:40

In this report, we present Hunyuan3D 2.5, a robust suite of 3D diffusion
models aimed at generating high-fidelity and detailed textured 3D assets.
Hunyuan3D 2.5 follows two-stages pipeline of its previous version Hunyuan3D
2.0, while demonstrating substantial advancements in both shape and texture
generation. In terms of shape generation, we introduce a new shape foundation
model -- LATTICE, which is trained with scaled high-quality datasets,
model-size, and compute. Our largest model reaches 10B parameters and generates
sharp and detailed 3D shape with precise image-3D following while keeping mesh
surface clean and smooth, significantly closing the gap between generated and
handcrafted 3D shapes. In terms of texture generation, it is upgraded with
phyiscal-based rendering (PBR) via a novel multi-view architecture extended
from Hunyuan3D 2.0 Paint model. Our extensive evaluation shows that Hunyuan3D
2.5 significantly outperforms previous methods in both shape and end-to-end
texture generation.
Show-o2: Improved Native Unified Multimodal Models
Authors: name: Jinheng Xie, name: Zhenheng Yang, name: Mike Zheng Shou
Published: 2025-06-18 15:39:15

This paper presents improved native unified multimodal models, i.e.,
Show-o2, that leverage autoregressive modeling and flow matching. Built upon a
3D causal variational autoencoder space, unified visual representations are
constructed through a dual-path of spatial (-temporal) fusion, enabling
scalability across image and video modalities while ensuring effective
multimodal understanding and generation. Based on a language model,
autoregressive modeling and flow matching are natively applied to the language
head and flow head, respectively, to facilitate text token prediction and
image/video generation. A two-stage training recipe is designed to effectively
learn and scale to larger models. The resulting Show-o2 models demonstrate
versatility in handling a wide range of multimodal understanding and generation
tasks across diverse modalities, including text, images, and videos. Code and
models are released at https://github.com/showlab/Show-o.