Uni-3DAR: Unified 3D Generation and Understanding via Autoregression on
  Compressed Spatial Tokens
Authors: name: Shuqi Lu; name: Haowei Lin; name: Lin Yao; name: Zhifeng Gao; name: Xiaohong Ji; name: Weinan E; name: Linfeng Zhang; name: Guolin Ke
Published: 2025-03-20 16:07:04

Abstract: Recent advancements in large language models and their multi-modal extensions
have demonstrated the effectiveness of unifying generation and understanding
through autoregressive next-token prediction. However, despite the critical
role of 3D structural generation and understanding ({3D GU}) in AI for science,
these tasks have largely evolved independently, with autoregressive methods
remaining underexplored. To bridge this gap, we introduce Uni-3DAR, a unified
framework that seamlessly integrates {3D GU} tasks via autoregressive
prediction. At its core, Uni-3DAR employs a novel hierarchical tokenization
that compresses 3D space using an octree, leveraging the inherent sparsity of
3D structures. It then applies an additional tokenization for fine-grained
structural details, capturing key attributes such as atom types and precise
spatial coordinates in microscopic 3D structures. We further propose two
optimizations to enhance efficiency and effectiveness. The first is a two-level
subtree compression strategy, which reduces the octree token sequence by up to
8x. The second is a masked next-token prediction mechanism tailored for
dynamically varying token positions, significantly boosting model performance.
By combining these strategies, Uni-3DAR successfully unifies diverse {3D GU}
tasks within a single autoregressive framework. Extensive experiments across
multiple microscopic {3D GU} tasks, including molecules, proteins, polymers,
and crystals, validate its effectiveness and versatility. Notably, Uni-3DAR
surpasses previous state-of-the-art diffusion models by a substantial margin,
achieving up to 256\% relative improvement while delivering inference speeds up
to 21.8x faster. The code is publicly available at
https://github.com/dptech-corp/Uni-3DAR.