Orionnnn/hub-stats huggingface dataset:
LHM: Large Animatable Human Reconstruction Model from a Single Image in
  Seconds
Authors: name: Lingteng Qiu, name: Xiaodong Gu, name: Peihao Li, name: Qi Zuo, name: Weichao Shen, name: Junfei Zhang, name: Kejie Qiu, name: Weihao Yuan, name: Guanying Chen, name: Zilong Dong, name: Liefeng Bo
Published: 2025-03-13 17:59:21

Animatable 3D human reconstruction from a single image is a challenging
problem due to the ambiguity in decoupling geometry, appearance, and
deformation. Recent advances in 3D human reconstruction mainly focus on static
human modeling, and the reliance of using synthetic 3D scans for training
limits their generalization ability. Conversely, optimization-based video
methods achieve higher fidelity but demand controlled capture conditions and
computationally intensive refinement processes. Motivated by the emergence of
large reconstruction models for efficient static reconstruction, we propose LHM
(Large Animatable Human Reconstruction Model) to infer high-fidelity avatars
represented as 3D Gaussian splatting in a feed-forward pass. Our model
leverages a multimodal transformer architecture to effectively encode the human
body positional features and image features with attention mechanism, enabling
detailed preservation of clothing geometry and texture. To further boost the
face identity preservation and fine detail recovery, we propose a head feature
pyramid encoding scheme to aggregate multi-scale features of the head regions.
Extensive experiments demonstrate that our LHM generates plausible animatable
human in seconds without post-processing for face and hands, outperforming
existing methods in both reconstruction accuracy and generalization ability.
SALT: Singular Value Adaptation with Low-Rank Transformation
Authors: name: Abdelrahman Elsayed, name: Sarim Hashmi, name: Mohammed Elseiagy, name: Hu Wang, name: Mohammad Yaqub, name: Ibrahim Almakky
Published: 2025-03-20 11:42:41

The complex nature of medical image segmentation calls for models that are
specifically designed to capture detailed, domain-specific features. Large
foundation models offer considerable flexibility, yet the cost of fine-tuning
these models remains a significant barrier. Parameter-Efficient Fine-Tuning
(PEFT) methods, such as Low-Rank Adaptation (LoRA), efficiently update model
weights with low-rank matrices but may suffer from underfitting when the chosen
rank is insufficient to capture domain-specific nuances. Conversely, full-rank
Singular Value Decomposition (SVD) based methods provide comprehensive updates
by modifying all singular values, yet they often lack flexibility and exhibit
variable performance across datasets. We propose SALT (Singular Value
Adaptation with Low-Rank Transformation), a method that selectively adapts the
most influential singular values using trainable scale and shift parameters
while complementing this with a low-rank update for the remaining subspace.
This hybrid approach harnesses the advantages of both LoRA and SVD, enabling
effective adaptation without relying on increasing model size or depth.
Evaluated on 5 challenging medical datasets, ranging from as few as 20 samples
to 1000, SALT outperforms state-of-the-art PEFT (LoRA and SVD) by 2% to 5% in
Dice with only 3.9% trainable parameters, demonstrating robust adaptation even
in low-resource settings. The code for SALT is available at:
https://github.com/BioMedIA-MBZUAI/SALT
MotionStreamer: Streaming Motion Generation via Diffusion-based
  Autoregressive Model in Causal Latent Space
Authors: name: Lixing Xiao, name: Shunlin Lu, name: Huaijin Pi, name: Ke Fan, name: Liang Pan, name: Yueer Zhou, name: Ziyong Feng, name: Xiaowei Zhou, name: Sida Peng, name: Jingbo Wang
Published: 2025-03-19 17:32:24

This paper addresses the challenge of text-conditioned streaming motion
generation, which requires us to predict the next-step human pose based on
variable-length historical motions and incoming texts. Existing methods
struggle to achieve streaming motion generation, e.g., diffusion models are
constrained by pre-defined motion lengths, while GPT-based methods suffer from
delayed response and error accumulation problem due to discretized non-causal
tokenization. To solve these problems, we propose MotionStreamer, a novel
framework that incorporates a continuous causal latent space into a
probabilistic autoregressive model. The continuous latents mitigate information
loss caused by discretization and effectively reduce error accumulation during
long-term autoregressive generation. In addition, by establishing temporal
causal dependencies between current and historical motion latents, our model
fully utilizes the available information to achieve accurate online motion
decoding. Experiments show that our method outperforms existing approaches
while offering more applications, including multi-round generation, long-term
generation, and dynamic motion composition. Project Page:
https://zju3dv.github.io/MotionStreamer/
Expert Race: A Flexible Routing Strategy for Scaling Diffusion
  Transformer with Mixture of Experts
Authors: name: Yike Yuan, name: Ziyu Wang, name: Zihao Huang, name: Defa Zhu, name: Xun Zhou, name: Jingyi Yu, name: Qiyang Min
Published: 2025-03-20 11:45:08

Diffusion models have emerged as mainstream framework in visual generation.
Building upon this success, the integration of Mixture of Experts (MoE) methods
has shown promise in enhancing model scalability and performance. In this
paper, we introduce Race-DiT, a novel MoE model for diffusion transformers with
a flexible routing strategy, Expert Race. By allowing tokens and experts to
compete together and select the top candidates, the model learns to dynamically
assign experts to critical tokens. Additionally, we propose per-layer
regularization to address challenges in shallow layer learning, and router
similarity loss to prevent mode collapse, ensuring better expert utilization.
Extensive experiments on ImageNet validate the effectiveness of our approach,
showcasing significant performance gains while promising scaling properties.
Zero-1-to-A: Zero-Shot One Image to Animatable Head Avatars Using Video
  Diffusion
Authors: name: Zhou Zhenglin, name: Ma Fan, name: Fan Hehe, name: Chua Tat-Seng
Published: 2025-03-20 05:07:46

Animatable head avatar generation typically requires extensive data for
training. To reduce the data requirements, a natural solution is to leverage
existing data-free static avatar generation methods, such as pre-trained
diffusion models with score distillation sampling (SDS), which align avatars
with pseudo ground-truth outputs from the diffusion model. However, directly
distilling 4D avatars from video diffusion often leads to over-smooth results
due to spatial and temporal inconsistencies in the generated video. To address
this issue, we propose Zero-1-to-A, a robust method that synthesizes a spatial
and temporal consistency dataset for 4D avatar reconstruction using the video
diffusion model. Specifically, Zero-1-to-A iteratively constructs video
datasets and optimizes animatable avatars in a progressive manner, ensuring
that avatar quality increases smoothly and consistently throughout the learning
process. This progressive learning involves two stages: (1) Spatial Consistency
Learning fixes expressions and learns from front-to-side views, and (2)
Temporal Consistency Learning fixes views and learns from relaxed to
exaggerated expressions, generating 4D avatars in a simple-to-complex manner.
Extensive experiments demonstrate that Zero-1-to-A improves fidelity, animation
quality, and rendering speed compared to existing diffusion-based methods,
providing a solution for lifelike avatar creation. Code is publicly available
at: https://github.com/ZhenglinZhou/Zero-1-to-A.
See-Saw Modality Balance: See Gradient, and Sew Impaired Vision-Language
  Balance to Mitigate Dominant Modality Bias
Authors: name: JuneHyoung Kwon, name: MiHyeon Kim, name: Eunju Lee, name: Juhwan Choi, name: YoungBin Kim
Published: 2025-03-18 02:17:41

Vision-language (VL) models have demonstrated strong performance across
various tasks. However, these models often rely on a specific modality for
predictions, leading to "dominant modality bias.'' This bias significantly
hurts performance, especially when one modality is impaired. In this study, we
analyze model behavior under dominant modality bias and theoretically show that
unaligned gradients or differences in gradient magnitudes prevent balanced
convergence of the loss. Based on these findings, we propose a novel framework,
BalGrad to mitigate dominant modality bias. Our approach includes
inter-modality gradient reweighting, adjusting the gradient of KL divergence
based on each modality's contribution, and inter-task gradient projection to
align task directions in a non-conflicting manner. Experiments on UPMC
Food-101, Hateful Memes, and MM-IMDb datasets confirm that BalGrad effectively
alleviates over-reliance on specific modalities when making predictions.
CLS-RL: Image Classification with Rule-Based Reinforcement Learning
Authors: name: Ming Li, name: Shitian Zhao, name: Jike Zhong, name: Yuxiang Lai, name: Kaipeng Zhang
Published: 2025-03-20 14:37:45

Classification is a core task in machine learning. Recent research has shown
that although Multimodal Large Language Models (MLLMs) are initially poor at
image classification, fine-tuning them with an adequate amount of data can
significantly enhance their performance, making them comparable to SOTA
classification models. However, acquiring large-scale labeled data is
expensive. In this paper, we explore few-shot MLLM classification fine-tuning.
We found that SFT can cause severe overfitting issues and may even degrade
performance over the zero-shot approach. To address this challenge, inspired by
the recent successes in rule-based reinforcement learning, we propose CLS-RL,
which uses verifiable signals as reward to fine-tune MLLMs. We discovered that
CLS-RL outperforms SFT in most datasets and has a much higher average accuracy
on both base-to-new and few-shot learning setting. Moreover, we observed a
free-lunch phenomenon for CLS-RL; when models are fine-tuned on a particular
dataset, their performance on other distinct datasets may also improve over
zero-shot models, even if those datasets differ in distribution and class
names. This suggests that RL-based methods effectively teach models the
fundamentals of classification. Lastly, inspired by recent works in inference
time thinking, we re-examine the `thinking process' during fine-tuning, a
critical aspect of RL-based methods, in the context of visual classification.
We question whether such tasks require extensive thinking process during
fine-tuning, proposing that this may actually detract from performance. Based
on this premise, we introduce the No-Thinking-CLS-RL method, which minimizes
thinking processes during training by setting an equality accuracy reward. Our
findings indicate that, with much less fine-tuning time, No-Thinking-CLS-RL
method achieves superior in-domain performance and generalization capabilities
than CLS-RL.
Improving Autoregressive Image Generation through Coarse-to-Fine Token
  Prediction
Authors: name: Ziyao Guo, name: Kaipeng Zhang, name: Michael Qizhe Shieh
Published: 2025-03-20 14:41:29

Autoregressive models have shown remarkable success in image generation by
adapting sequential prediction techniques from language modeling. However,
applying these approaches to images requires discretizing continuous pixel data
through vector quantization methods like VQ-VAE. To alleviate the quantization
errors that existed in VQ-VAE, recent works tend to use larger codebooks.
However, this will accordingly expand vocabulary size, complicating the
autoregressive modeling task. This paper aims to find a way to enjoy the
benefits of large codebooks without making autoregressive modeling more
difficult. Through empirical investigation, we discover that tokens with
similar codeword representations produce similar effects on the final generated
image, revealing significant redundancy in large codebooks. Based on this
insight, we propose to predict tokens from coarse to fine (CTF), realized by
assigning the same coarse label for similar tokens. Our framework consists of
two stages: (1) an autoregressive model that sequentially predicts coarse
labels for each token in the sequence, and (2) an auxiliary model that
simultaneously predicts fine-grained labels for all tokens conditioned on their
coarse labels. Experiments on ImageNet demonstrate our method's superior
performance, achieving an average improvement of 59 points in Inception Score
compared to baselines. Notably, despite adding an inference step, our approach
achieves faster sampling speeds.
InfiniteYou: Flexible Photo Recrafting While Preserving Your Identity
Authors: name: Liming Jiang, name: Qing Yan, name: Yumin Jia, name: Zichuan Liu, name: Hao Kang, name: Xin Lu
Published: 2025-03-20 17:59:34

Achieving flexible and high-fidelity identity-preserved image generation
remains formidable, particularly with advanced Diffusion Transformers (DiTs)
like FLUX. We introduce InfiniteYou (InfU), one of the earliest robust
frameworks leveraging DiTs for this task. InfU addresses significant issues of
existing methods, such as insufficient identity similarity, poor text-image
alignment, and low generation quality and aesthetics. Central to InfU is
InfuseNet, a component that injects identity features into the DiT base model
via residual connections, enhancing identity similarity while maintaining
generation capabilities. A multi-stage training strategy, including pretraining
and supervised fine-tuning (SFT) with synthetic single-person-multiple-sample
(SPMS) data, further improves text-image alignment, ameliorates image quality,
and alleviates face copy-pasting. Extensive experiments demonstrate that InfU
achieves state-of-the-art performance, surpassing existing baselines. In
addition, the plug-and-play design of InfU ensures compatibility with various
existing methods, offering a valuable contribution to the broader community.
Ultra-Resolution Adaptation with Ease
Authors: name: Ruonan Yu, name: Songhua Liu, name: Zhenxiong Tan, name: Xinchao Wang
Published: 2025-03-20 16:44:43

Text-to-image diffusion models have achieved remarkable progress in recent
years. However, training models for high-resolution image generation remains
challenging, particularly when training data and computational resources are
limited. In this paper, we explore this practical problem from two key
perspectives: data and parameter efficiency, and propose a set of key
guidelines for ultra-resolution adaptation termed URAE. For data
efficiency, we theoretically and empirically demonstrate that synthetic data
generated by some teacher models can significantly promote training
convergence. For parameter efficiency, we find that tuning minor components of
the weight matrices outperforms widely-used low-rank adapters when synthetic
data are unavailable, offering substantial performance gains while maintaining
efficiency. Additionally, for models leveraging guidance distillation, such as
FLUX, we show that disabling classifier-free guidance, i.e., setting
the guidance scale to 1 during adaptation, is crucial for satisfactory
performance. Extensive experiments validate that URAE achieves comparable
2K-generation performance to state-of-the-art closed-source models like FLUX1.1
[Pro] Ultra with only 3K samples and 2K iterations, while setting new
benchmarks for 4K-resolution generation. Codes are available
https://github.com/Huage001/URAE{here}.
MagicID: Hybrid Preference Optimization for ID-Consistent and
  Dynamic-Preserved Video Customization
Authors: name: Hengjia Li, name: Lifan Jiang, name: Xi Xiao, name: Tianyang Wang, name: Hongwei Yi, name: Boxi Wu, name: Deng Cai
Published: 2025-03-16 23:15:09

Video identity customization seeks to produce high-fidelity videos that
maintain consistent identity and exhibit significant dynamics based on users'
reference images. However, existing approaches face two key challenges:
identity degradation over extended video length and reduced dynamics during
training, primarily due to their reliance on traditional self-reconstruction
training with static images. To address these issues, we introduce
MagicID, a novel framework designed to directly promote the
generation of identity-consistent and dynamically rich videos tailored to user
preferences. Specifically, we propose constructing pairwise preference video
data with explicit identity and dynamic rewards for preference learning,
instead of sticking to the traditional self-reconstruction. To address the
constraints of customized preference data, we introduce a hybrid sampling
strategy. This approach first prioritizes identity preservation by leveraging
static videos derived from reference images, then enhances dynamic motion
quality in the generated videos using a Frontier-based sampling method. By
utilizing these hybrid preference pairs, we optimize the model to align with
the reward differences between pairs of customized preferences. Extensive
experiments show that MagicID successfully achieves consistent identity and
natural dynamics, surpassing existing methods across various metrics.
Unleashing Vecset Diffusion Model for Fast Shape Generation
Authors: name: Zeqiang Lai, name: Yunfei Zhao, name: Zibo Zhao, name: Haolin Liu, name: Fuyun Wang, name: Huiwen Shi, name: Xianghui Yang, name: Qinxiang Lin, name: Jinwei Huang, name: Yuhong Liu, name: Jie Jiang, name: Chunchao Guo, name: Xiangyu Yue
Published: 2025-03-20 16:23:44

3D shape generation has greatly flourished through the development of
so-called "native" 3D diffusion, particularly through the Vecset Diffusion
Model (VDM). While recent advancements have shown promising results in
generating high-resolution 3D shapes, VDM still struggles with high-speed
generation. Challenges exist because of difficulties not only in accelerating
diffusion sampling but also VAE decoding in VDM, areas under-explored in
previous works. To address these challenges, we present FlashVDM, a systematic
framework for accelerating both VAE and DiT in VDM. For DiT, FlashVDM enables
flexible diffusion sampling with as few as 5 inference steps and comparable
quality, which is made possible by stabilizing consistency distillation with
our newly introduced Progressive Flow Distillation. For VAE, we introduce a
lightning vecset decoder equipped with Adaptive KV Selection, Hierarchical
Volume Decoding, and Efficient Network Design. By exploiting the locality of
the vecset and the sparsity of shape surface in the volume, our decoder
drastically lowers FLOPs, minimizing the overall decoding overhead. We apply
FlashVDM to Hunyuan3D-2 to obtain Hunyuan3D-2 Turbo. Through systematic
evaluation, we show that our model significantly outperforms existing fast 3D
generation methods, achieving comparable performance to the state-of-the-art
while reducing inference time by over 45x for reconstruction and 32x for
generation. Code and models are available at
https://github.com/Tencent/FlashVDM.
JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play
  Visual Games with Keyboards and Mouse
Authors: name: Muyao Li, name: Zihao Wang, name: Kaichen He, name: Xiaojian Ma, name: Yitao Liang
Published: 2025-03-20 17:21:58

Recently, action-based decision-making in open-world environments has gained
significant attention. Visual Language Action (VLA) models, pretrained on
large-scale web datasets, have shown promise in decision-making tasks. However,
previous work has primarily focused on action post-training, often neglecting
enhancements to the foundational model itself. In response, we introduce a
novel approach, Act from Visual Language Post-Training, which refines Visual
Language Models (VLMs) through visual and linguistic guidance in a
self-supervised manner. This enhancement improves the models' capabilities in
world knowledge, visual recognition, and spatial grounding in open-world
environments. Following the above post-training paradigms, we obtain the first
VLA models in Minecraft that can follow human instructions on over 1k different
atomic tasks, including crafting, smelting, cooking, mining, and killing. Our
experiments demonstrate that post-training on non-trajectory tasks leads to a
significant 40% improvement over the best agent baseline on a diverse set of
atomic tasks. Furthermore, we demonstrate that our approach surpasses
traditional imitation learning-based policies in Minecraft, achieving
state-of-the-art performance. We have open-sourced the code, models, and
datasets to foster further research. The project page can be found in
https://craftjarvis.github.io/JarvisVLA.
Stop Overthinking: A Survey on Efficient Reasoning for Large Language
  Models
Authors: name: Yang Sui, name: Yu-Neng Chuang, name: Guanchu Wang, name: Jiamu Zhang, name: Tianyi Zhang, name: Jiayi Yuan, name: Hongyi Liu, name: Andrew Wen, name: Shaochen, name: Zhong, name: Hanjie Chen, name: Xia Hu
Published: 2025-03-20 17:59:38

Large Language Models (LLMs) have demonstrated remarkable capabilities in
complex tasks. Recent advancements in Large Reasoning Models (LRMs), such as
OpenAI o1 and DeepSeek-R1, have further improved performance in System-2
reasoning domains like mathematics and programming by harnessing supervised
fine-tuning (SFT) and reinforcement learning (RL) techniques to enhance the
Chain-of-Thought (CoT) reasoning. However, while longer CoT reasoning sequences
improve performance, they also introduce significant computational overhead due
to verbose and redundant outputs, known as the "overthinking phenomenon". In
this paper, we provide the first structured survey to systematically
investigate and explore the current progress toward achieving efficient
reasoning in LLMs. Overall, relying on the inherent mechanism of LLMs, we
categorize existing works into several key directions: (1) model-based
efficient reasoning, which considers optimizing full-length reasoning models
into more concise reasoning models or directly training efficient reasoning
models; (2) reasoning output-based efficient reasoning, which aims to
dynamically reduce reasoning steps and length during inference; (3) input
prompts-based efficient reasoning, which seeks to enhance reasoning efficiency
based on input prompt properties such as difficulty or length control.
Additionally, we introduce the use of efficient data for training reasoning
models, explore the reasoning capabilities of small language models, and
discuss evaluation methods and benchmarking.
XAttention: Block Sparse Attention with Antidiagonal Scoring
Authors: name: Ruyi Xu, name: Guangxuan Xiao, name: Haofeng Huang, name: Junxian Guo, name: Song Han
Published: 2025-03-20 17:59:58

Long-Context Transformer Models (LCTMs) are vital for real-world applications
but suffer high computational costs due to attention's quadratic complexity.
Block-sparse attention mitigates this by focusing computation on critical
regions, yet existing methods struggle with balancing accuracy and efficiency
due to costly block importance measurements. In this paper, we introduce
XAttention, a plug-and-play framework that dramatically accelerates
long-context inference in Transformers models using sparse attention.
XAttention's key innovation is the insight that the sum of antidiagonal values
(i.e., from the lower-left to upper-right) in the attention matrix provides a
powerful proxy for block importance. This allows for precise identification and
pruning of non-essential blocks, resulting in high sparsity and dramatically
accelerated inference. Across comprehensive evaluations on demanding
long-context benchmarks-including RULER and LongBench for language, VideoMME
for video understanding, and VBench for video generation. XAttention achieves
accuracy comparable to full attention while delivering substantial
computational gains. We demonstrate up to 13.5x acceleration in attention
computation. These results underscore XAttention's ability to unlock the
practical potential of block sparse attention, paving the way for scalable and
efficient deployment of LCTMs in real-world applications. Code is available at
https://github.com/mit-han-lab/x-attention.