Unleashing Vecset Diffusion Model for Fast Shape Generation
Authors: name: Zeqiang Lai, name: Yunfei Zhao, name: Zibo Zhao, name: Haolin Liu, name: Fuyun Wang, name: Huiwen Shi, name: Xianghui Yang, name: Qinxiang Lin, name: Jinwei Huang, name: Yuhong Liu, name: Jie Jiang, name: Chunchao Guo, name: Xiangyu Yue

3D shape generation has greatly flourished through the development of
so-called "native" 3D diffusion, particularly through the Vecset Diffusion
Model (VDM). While recent advancements have shown promising results in
generating high-resolution 3D shapes, VDM still struggles with high-speed
generation. Challenges exist because of difficulties not only in accelerating
diffusion sampling but also VAE decoding in VDM, areas under-explored in
previous works. To address these challenges, we present FlashVDM, a systematic
framework for accelerating both VAE and DiT in VDM. For DiT, FlashVDM enables
flexible diffusion sampling with as few as 5 inference steps and comparable
quality, which is made possible by stabilizing consistency distillation with
our newly introduced Progressive Flow Distillation. For VAE, we introduce a
lightning vecset decoder equipped with Adaptive KV Selection, Hierarchical
Volume Decoding, and Efficient Network Design. By exploiting the locality of
the vecset and the sparsity of shape surface in the volume, our decoder
drastically lowers FLOPs, minimizing the overall decoding overhead. We apply
FlashVDM to Hunyuan3D-2 to obtain Hunyuan3D-2 Turbo. Through systematic
evaluation, we show that our model significantly outperforms existing fast 3D
generation methods, achieving comparable performance to the state-of-the-art
while reducing inference time by over 45x for reconstruction and 32x for
generation. Code and models are available at
https://github.com/Tencent/FlashVDM.