AToken: A Unified Tokenizer for Vision
Authors: name: Jiasen Lu; name: Liangchen Song; name: Mingze Xu; name: Byeongjoo Ahn; name: Yanjun Wang; name: Chen Chen; name: Afshin Dehghan; name: Yinfei Yang
Published: 2025-09-17 23:11:18

Abstract: We present AToken, the first unified visual tokenizer that achieves both
high-fidelity reconstruction and semantic understanding across images, videos,
and 3D assets. Unlike existing tokenizers that specialize in either
reconstruction or understanding for single modalities, AToken encodes these
diverse visual inputs into a shared 4D latent space, unifying both tasks and
modalities in a single framework. Specifically, we introduce a pure transformer
architecture with 4D rotary position embeddings to process visual inputs of
arbitrary resolutions and temporal durations. To ensure stable training, we
introduce an adversarial-free training objective that combines perceptual and
Gram matrix losses, achieving state-of-the-art reconstruction quality. By
employing a progressive training curriculum, AToken gradually expands from
single images, videos, and 3D, and supports both continuous and discrete latent
tokens. AToken achieves 0.21 rFID with 82.2% ImageNet accuracy for images, 3.01
rFVD with 32.6% MSRVTT retrieval for videos, and 28.19 PSNR with 90.9%
classification accuracy for 3D. In downstream applications, AToken enables both
visual generation tasks (e.g., image generation with continuous and discrete
tokens, text-to-video generation, image-to-3D synthesis) and understanding
tasks (e.g., multimodal LLMs), achieving competitive performance across all
benchmarks. These results shed light on the next-generation multimodal AI
systems built upon unified visual tokenization.