Orionnnn/hub-stats huggingface dataset:
A Survey on Vision-Language-Action Models: An Action Tokenization
  Perspective
Authors: name: Yifan Zhong, name: Fengshuo Bai, name: Shaofei Cai, name: Xuchuan Huang, name: Zhang Chen, name: Xiaowei Zhang, name: Yuanfei Wang, name: Shaoyang Guo, name: Tianrui Guan, name: Ka Nam Lui, name: Zhiquan Qi, name: Yitao Liang, name: Yuanpei Chen, name: Yaodong Yang
Published: 2025-07-02 17:34:52

The remarkable advancements of vision and language foundation models in
multimodal understanding, reasoning, and generation has sparked growing efforts
to extend such intelligence to the physical world, fueling the flourishing of
vision-language-action (VLA) models. Despite seemingly diverse approaches, we
observe that current VLA models can be unified under a single framework: vision
and language inputs are processed by a series of VLA modules, producing a chain
of action tokens that progressively encode more grounded and
actionable information, ultimately generating executable actions. We further
determine that the primary design choice distinguishing VLA models lies in how
action tokens are formulated, which can be categorized into language
description, code, affordance, trajectory, goal state, latent representation,
raw action, and reasoning. However, there remains a lack of comprehensive
understanding regarding action tokens, significantly impeding effective VLA
development and obscuring future directions. Therefore, this survey aims to
categorize and interpret existing VLA research through the lens of action
tokenization, distill the strengths and limitations of each token type, and
identify areas for improvement. Through this systematic review and analysis, we
offer a synthesized outlook on the broader evolution of VLA models, highlight
underexplored yet promising directions, and contribute guidance for future
research, hoping to bring the field closer to general-purpose intelligence.
FreeMorph: Tuning-Free Generalized Image Morphing with Diffusion Model
Authors: name: Yukang Cao, name: Chenyang Si, name: Jinghao Wang, name: Ziwei Liu
Published: 2025-07-02 17:58:20

We present FreeMorph, the first tuning-free method for image morphing that
accommodates inputs with different semantics or layouts. Unlike existing
methods that rely on finetuning pre-trained diffusion models and are limited by
time constraints and semantic/layout discrepancies, FreeMorph delivers
high-fidelity image morphing without requiring per-instance training. Despite
their efficiency and potential, tuning-free methods face challenges in
maintaining high-quality results due to the non-linear nature of the multi-step
denoising process and biases inherited from the pre-trained diffusion model. In
this paper, we introduce FreeMorph to address these challenges by integrating
two key innovations. 1) We first propose a guidance-aware spherical
interpolation design that incorporates explicit guidance from the input images
by modifying the self-attention modules, thereby addressing identity loss and
ensuring directional transitions throughout the generated sequence. 2) We
further introduce a step-oriented variation trend that blends self-attention
modules derived from each input image to achieve controlled and consistent
transitions that respect both inputs. Our extensive evaluations demonstrate
that FreeMorph outperforms existing methods, being 10x ~ 50x faster and
establishing a new state-of-the-art for image morphing.
LongAnimation: Long Animation Generation with Dynamic Global-Local
  Memory
Authors: name: Nan Chen, name: Mengqi Huang, name: Yihao Meng, name: Zhendong Mao
Published: 2025-07-02 17:55:50

Animation colorization is a crucial part of real animation industry
production. Long animation colorization has high labor costs. Therefore,
automated long animation colorization based on the video generation model has
significant research value. Existing studies are limited to short-term
colorization. These studies adopt a local paradigm, fusing overlapping features
to achieve smooth transitions between local segments. However, the local
paradigm neglects global information, failing to maintain long-term color
consistency. In this study, we argue that ideal long-term color consistency can
be achieved through a dynamic global-local paradigm, i.e., dynamically
extracting global color-consistent features relevant to the current generation.
Specifically, we propose LongAnimation, a novel framework, which mainly
includes a SketchDiT, a Dynamic Global-Local Memory (DGLM), and a Color
Consistency Reward. The SketchDiT captures hybrid reference features to support
the DGLM module. The DGLM module employs a long video understanding model to
dynamically compress global historical features and adaptively fuse them with
the current generation features. To refine the color consistency, we introduce
a Color Consistency Reward. During inference, we propose a color consistency
fusion to smooth the video segment transition. Extensive experiments on both
short-term (14 frames) and long-term (average 500 frames) animations show the
effectiveness of LongAnimation in maintaining short-term and long-term color
consistency for open-domain animation colorization task. The code can be found
at https://cn-makers.github.io/long_animation_web/.
Kwai Keye-VL Technical Report
Authors: name: Kwai Keye Team, name: Biao Yang, name: Bin Wen, name: Changyi Liu, name: Chenglong Chu, name: Chengru Song, name: Chongling Rao, name: Chuan Yi, name: Da Li, name: Dunju Zang, name: Fan Yang, name: Guorui Zhou, name: Hao Peng, name: Haojie Ding, name: Jiaming Huang, name: Jiangxia Cao, name: Jiankang Chen, name: Jingyun Hua, name: Jin Ouyang, name: Kaibing Chen, name: Kaiyu Jiang, name: Kaiyu Tang, name: Kun Gai, name: Shengnan Zhang, name: Siyang Mao, name: Sui Huang, name: Tianke Zhang, name: Tingting Gao, name: Wei Chen, name: Wei Yuan, name: Xiangyu Wu, name: Xiao Hu, name: Xingyu Lu, name: Yang Zhou, name: Yi-Fan Zhang, name: Yiping Yang, name: Yulong Chen, name: Zhenhua Wu, name: Zhenyu Li, name: Zhixin Ling, name: Ziming Li, name: Dehua Ma, name: Di Xu, name: Haixuan Gao, name: Hang Li, name: Jiawei Guo, name: Jing Wang, name: Lejian Ren, name: Muhao Wei, name: Qianqian Wang, name: Qigen Hu, name: Shiyao Wang, name: Tao Yu, name: Xinchen Luo, name: Yan Li, name: Yiming Liang, name: Yuhang Hu, name: Zeyi Lu, name: Zhuoran Yang, name: Zixing Zhang
Published: 2025-07-02 17:57:28

While Multimodal Large Language Models (MLLMs) demonstrate remarkable
capabilities on static images, they often fall short in comprehending dynamic,
information-dense short-form videos, a dominant medium in today's digital
landscape. To bridge this gap, we introduce Kwai Keye-VL, an
8-billion-parameter multimodal foundation model engineered for leading-edge
performance in short-video understanding while maintaining robust
general-purpose vision-language abilities. The development of Keye-VL rests on
two core pillars: a massive, high-quality dataset exceeding 600 billion tokens
with a strong emphasis on video, and an innovative training recipe. This recipe
features a four-stage pre-training process for solid vision-language alignment,
followed by a meticulous two-phase post-training process. The first
post-training stage enhances foundational capabilities like instruction
following, while the second phase focuses on stimulating advanced reasoning. In
this second phase, a key innovation is our five-mode ``cold-start'' data
mixture, which includes ``thinking'', ``non-thinking'', ``auto-think'', ``think
with image'', and high-quality video data. This mixture teaches the model to
decide when and how to reason. Subsequent reinforcement learning (RL) and
alignment steps further enhance these reasoning capabilities and correct
abnormal model behaviors, such as repetitive outputs. To validate our approach,
we conduct extensive evaluations, showing that Keye-VL achieves
state-of-the-art results on public video benchmarks and remains highly
competitive on general image-based tasks (Figure 1). Furthermore, we develop
and release the KC-MMBench, a new benchmark tailored for real-world
short-video scenarios, where Keye-VL shows a significant advantage.
Depth Anything at Any Condition
Authors: name: Boyuan Sun, name: Modi Jin, name: Bowen Yin, name: Qibin Hou
Published: 2025-07-02 12:05:57

We present Depth Anything at Any Condition (DepthAnything-AC), a foundation
monocular depth estimation (MDE) model capable of handling diverse
environmental conditions. Previous foundation MDE models achieve impressive
performance across general scenes but not perform well in complex open-world
environments that involve challenging conditions, such as illumination
variations, adverse weather, and sensor-induced distortions. To overcome the
challenges of data scarcity and the inability of generating high-quality
pseudo-labels from corrupted images, we propose an unsupervised consistency
regularization finetuning paradigm that requires only a relatively small amount
of unlabeled data. Furthermore, we propose the Spatial Distance Constraint to
explicitly enforce the model to learn patch-level relative relationships,
resulting in clearer semantic boundaries and more accurate details.
Experimental results demonstrate the zero-shot capabilities of DepthAnything-AC
across diverse benchmarks, including real-world adverse weather benchmarks,
synthetic corruption benchmarks, and general benchmarks.
  Project Page: https://ghost233lism.github.io/depthanything-AC-page
  Code: https://github.com/HVision-NKU/DepthAnythingAC
JAM-Flow: Joint Audio-Motion Synthesis with Flow Matching
Authors: name: Mingi Kwon, name: Joonghyuk Shin, name: Jaeseok Jung, name: Jaesik Park, name: Youngjung Uh
Published: 2025-06-30 06:51:40

The intrinsic link between facial motion and speech is often overlooked in
generative modeling, where talking head synthesis and text-to-speech (TTS) are
typically addressed as separate tasks. This paper introduces JAM-Flow, a
unified framework to simultaneously synthesize and condition on both facial
motion and speech. Our approach leverages flow matching and a novel Multi-Modal
Diffusion Transformer (MM-DiT) architecture, integrating specialized Motion-DiT
and Audio-DiT modules. These are coupled via selective joint attention layers
and incorporate key architectural choices, such as temporally aligned
positional embeddings and localized joint attention masking, to enable
effective cross-modal interaction while preserving modality-specific strengths.
Trained with an inpainting-style objective, JAM-Flow supports a wide array of
conditioning inputs-including text, reference audio, and reference
motion-facilitating tasks such as synchronized talking head generation from
text, audio-driven animation, and much more, within a single, coherent model.
JAM-Flow significantly advances multi-modal generative modeling by providing a
practical solution for holistic audio-visual synthesis. project page:
https://joonghyuk.com/jamflow-web
Ella: Embodied Social Agents with Lifelong Memory
Authors: name: Hongxin Zhang, name: Zheyuan Zhang, name: Zeyuan Wang, name: Zunzhe Zhang, name: Lixing Fang, name: Qinhong Zhou, name: Chuang Gan
Published: 2025-06-30 16:22:51

We introduce Ella, an embodied social agent capable of lifelong learning
within a community in a 3D open world, where agents accumulate experiences and
acquire knowledge through everyday visual observations and social interactions.
At the core of Ella's capabilities is a structured, long-term multimodal memory
system that stores, updates, and retrieves information effectively. It consists
of a name-centric semantic memory for organizing acquired knowledge and a
spatiotemporal episodic memory for capturing multimodal experiences. By
integrating this lifelong memory system with foundation models, Ella retrieves
relevant information for decision-making, plans daily activities, builds social
relationships, and evolves autonomously while coexisting with other intelligent
beings in the open world. We conduct capability-oriented evaluations in a
dynamic 3D open world where 15 agents engage in social activities for days and
are assessed with a suite of unseen controlled evaluations. Experimental
results show that Ella can influence, lead, and cooperate with other agents
well to achieve goals, showcasing its ability to learn effectively through
observation and social interaction. Our findings highlight the transformative
potential of combining structured memory systems with foundation models for
advancing embodied intelligence. More videos can be found at
https://umass-embodied-agi.github.io/Ella/.
MusiXQA: Advancing Visual Music Understanding in Multimodal Large
  Language Models
Authors: name: Jian Chen, name: Wenye Ma, name: Penghang Liu, name: Wei Wang, name: Tengwei Song, name: Ming Li, name: Chenguang Wang, name: Ruiyi Zhang, name: Changyou Chen
Published: 2025-06-28 20:46:47

Multimodal Large Language Models (MLLMs) have achieved remarkable visual
reasoning abilities in natural images, text-rich documents, and graphic
designs. However, their ability to interpret music sheets remains
underexplored. To bridge this gap, we introduce MusiXQA, the first
comprehensive dataset for evaluating and advancing MLLMs in music sheet
understanding. MusiXQA features high-quality synthetic music sheets generated
via MusiXTeX, with structured annotations covering note pitch and duration,
chords, clefs, key/time signatures, and text, enabling diverse visual QA tasks.
Through extensive evaluations, we reveal significant limitations of current
state-of-the-art MLLMs in this domain. Beyond benchmarking, we developed
Phi-3-MusiX, an MLLM fine-tuned on our dataset, achieving significant
performance gains over GPT-based methods. The proposed dataset and model
establish a foundation for future advances in MLLMs for music sheet
understanding. Code, data, and model will be released upon acceptance.
FreNBRDF: A Frequency-Rectified Neural Material Representation
Authors: name: Chenliang Zhou, name: Zheyuan Hu, name: Cengiz Oztireli
Published: 2025-07-01 06:48:50

Accurate material modeling is crucial for achieving photorealistic rendering,
bridging the gap between computer-generated imagery and real-world photographs.
While traditional approaches rely on tabulated BRDF data, recent work has
shifted towards implicit neural representations, which offer compact and
flexible frameworks for a range of tasks. However, their behavior in the
frequency domain remains poorly understood. To address this, we introduce
FreNBRDF, a frequency-rectified neural material representation. By leveraging
spherical harmonics, we integrate frequency-domain considerations into neural
BRDF modeling. We propose a novel frequency-rectified loss, derived from a
frequency analysis of neural materials, and incorporate it into a generalizable
and adaptive reconstruction and editing pipeline. This framework enhances
fidelity, adaptability, and efficiency. Extensive experiments demonstrate that
\ours improves the accuracy and robustness of material appearance
reconstruction and editing compared to state-of-the-art baselines, enabling
more structured and interpretable downstream tasks and applications.
Mixture of Reasonings: Teach Large Language Models to Reason with
  Adaptive Strategies
Authors: name: Tao Xiong, name: Xavier Hu, name: Wenyan Fan, name: Shengyu Zhang
Published: 2025-07-01 09:39:04

Large language models (LLMs) excel in complex tasks through advanced
prompting techniques like Chain-of-Thought (CoT) and Tree-of-Thought (ToT), but
their reliance on manually crafted, task-specific prompts limits adaptability
and efficiency. We introduce Mixture of Reasoning (MoR), a training framework
that embeds diverse reasoning strategies into LLMs for autonomous,
task-adaptive reasoning without external prompt engineering. MoR has two
phases: Thought Generation, creating reasoning chain templates with models like
GPT-4o, and SFT Dataset Construction, pairing templates with benchmark datasets
for supervised fine-tuning.Our experiments show that MoR significantly enhances
performance, with MoR150 achieving 0.730 (2.2% improvement) using CoT prompting
and 0.734 (13.5% improvement) compared to baselines. MoR eliminates the need
for task-specific prompts, offering a generalizable solution for robust
reasoning across diverse tasks.
Training for X-Ray Vision: Amodal Segmentation, Amodal Content
  Completion, and View-Invariant Object Representation from Multi-Camera Video
Authors: name: Alexander Moore, name: Amar Saini, name: Kylie Cancilla, name: Doug Poland, name: Carmen Carrano
Published: 2025-07-01 00:36:56

Amodal segmentation and amodal content completion require using object priors
to estimate occluded masks and features of objects in complex scenes. Until
now, no data has provided an additional dimension for object context: the
possibility of multiple cameras sharing a view of a scene. We introduce
MOVi-MC-AC: Multiple Object Video with Multi-Cameras and Amodal Content, the
largest amodal segmentation and first amodal content dataset to date. Cluttered
scenes of generic household objects are simulated in multi-camera video.
MOVi-MC-AC contributes to the growing literature of object detection, tracking,
and segmentation by including two new contributions to the deep learning for
computer vision world. Multiple Camera (MC) settings where objects can be
identified and tracked between various unique camera perspectives are rare in
both synthetic and real-world video. We introduce a new complexity to synthetic
video by providing consistent object ids for detections and segmentations
between both frames and multiple cameras each with unique features and motion
patterns on a single scene. Amodal Content (AC) is a reconstructive task in
which models predict the appearance of target objects through occlusions. In
the amodal segmentation literature, some datasets have been released with
amodal detection, tracking, and segmentation labels. While other methods rely
on slow cut-and-paste schemes to generate amodal content pseudo-labels, they do
not account for natural occlusions present in the modal masks. MOVi-MC-AC
provides labels for ~5.8 million object instances, setting a new maximum in the
amodal dataset literature, along with being the first to provide ground-truth
amodal content. The full dataset is available at
https://huggingface.co/datasets/Amar-S/MOVi-MC-AC ,
Confident Splatting: Confidence-Based Compression of 3D Gaussian
  Splatting via Learnable Beta Distributions
Authors: name: AmirHossein Naghi Razlighi, name: Elaheh Badali Golezani, name: Shohreh Kasaei
Published: 2025-06-28 18:11:30

3D Gaussian Splatting enables high-quality real-time rendering but often
produces millions of splats, resulting in excessive storage and computational
overhead. We propose a novel lossy compression method based on learnable
confidence scores modeled as Beta distributions. Each splat's confidence is
optimized through reconstruction-aware losses, enabling pruning of
low-confidence splats while preserving visual fidelity. The proposed approach
is architecture-agnostic and can be applied to any Gaussian Splatting variant.
In addition, the average confidence values serve as a new metric to assess the
quality of the scene. Extensive experiments demonstrate favorable trade-offs
between compression and fidelity compared to prior work. Our code and data are
publicly available at
https://github.com/amirhossein-razlighi/Confident-Splatting
IR3D-Bench: Evaluating Vision-Language Model Scene Understanding as
  Agentic Inverse Rendering
Authors: name: Parker Liu, name: Chenxin Li, name: Zhengxin Li, name: Yipeng Wu, name: Wuyang Li, name: Zhiqin Yang, name: Zhenyuan Zhang, name: Yunlong Lin, name: Sirui Han, name: Brandon Y. Feng
Published: 2025-06-29 17:02:57

Vision-language models (VLMs) excel at descriptive tasks, but whether they
truly understand scenes from visual observations remains uncertain. We
introduce IR3D-Bench, a benchmark challenging VLMs to demonstrate understanding
through active creation rather than passive recognition. Grounded in the
analysis-by-synthesis paradigm, IR3D-Bench tasks Vision-Language Agents (VLAs)
with actively using programming and rendering tools to recreate the underlying
3D structure of an input image, achieving agentic inverse rendering through
tool use. This "understanding-by-creating" approach probes the tool-using
generative capacity of VLAs, moving beyond the descriptive or conversational
capacity measured by traditional scene understanding benchmarks. We provide a
comprehensive suite of metrics to evaluate geometric accuracy, spatial
relations, appearance attributes, and overall plausibility. Initial experiments
on agentic inverse rendering powered by various state-of-the-art VLMs highlight
current limitations, particularly in visual precision rather than basic tool
usage. IR3D-Bench, including data and evaluation protocols, is released to
facilitate systematic study and development of tool-using VLAs towards genuine
scene understanding by creating.
FreeLong++: Training-Free Long Video Generation via Multi-band
  SpectralFusion
Authors: name: Yu Lu, name: Yi Yang
Published: 2025-06-30 18:11:21

Recent advances in video generation models have enabled high-quality short
video generation from text prompts. However, extending these models to longer
videos remains a significant challenge, primarily due to degraded temporal
consistency and visual fidelity. Our preliminary observations show that naively
applying short-video generation models to longer sequences leads to noticeable
quality degradation. Further analysis identifies a systematic trend where
high-frequency components become increasingly distorted as video length grows,
an issue we term high-frequency distortion. To address this, we propose
FreeLong, a training-free framework designed to balance the frequency
distribution of long video features during the denoising process. FreeLong
achieves this by blending global low-frequency features, which capture holistic
semantics across the full video, with local high-frequency features extracted
from short temporal windows to preserve fine details. Building on this,
FreeLong++ extends FreeLong dual-branch design into a multi-branch architecture
with multiple attention branches, each operating at a distinct temporal scale.
By arranging multiple window sizes from global to local, FreeLong++ enables
multi-band frequency fusion from low to high frequencies, ensuring both
semantic continuity and fine-grained motion dynamics across longer video
sequences. Without any additional training, FreeLong++ can be plugged into
existing video generation models (e.g. Wan2.1 and LTX-Video) to produce longer
videos with substantially improved temporal consistency and visual fidelity. We
demonstrate that our approach outperforms previous methods on longer video
generation tasks (e.g. 4x and 8x of native length). It also supports coherent
multi-prompt video generation with smooth scene transitions and enables
controllable video generation using long depth or pose sequences.