CODA: Repurposing Continuous VAEs for Discrete Tokenization
Authors: name: Zeyu Liu; name: Zanlin Ni; name: Yeguo Hua; name: Xin Deng; name: Xiao Ma; name: Cheng Zhong; name: Gao Huang
Published: 2025-03-22 12:59:00

Abstract: Discrete visual tokenizers transform images into a sequence of tokens,
enabling token-based visual generation akin to language models. However, this
process is inherently challenging, as it requires both compressing visual
signals into a compact representation and discretizing them into a fixed set of
codes. Traditional discrete tokenizers typically learn the two tasks jointly,
often leading to unstable training, low codebook utilization, and limited
reconstruction quality. In this paper, we introduce
CODA(COntinuous-to-Discrete Adaptation), a
framework that decouples compression and discretization. Instead of training
discrete tokenizers from scratch, CODA adapts off-the-shelf continuous VAEs --
already optimized for perceptual compression -- into discrete tokenizers via a
carefully designed discretization process. By primarily focusing on
discretization, CODA ensures stable and efficient training while retaining the
strong visual fidelity of continuous VAEs. Empirically, with 6
times less training budget than standard VQGAN, our approach achieves a
remarkable codebook utilization of 100% and notable reconstruction FID (rFID)
of 0.43 and 1.34 for 8 times and 16 times
compression on ImageNet 256times 256 benchmark.