ε-VAE: Denoising as Visual Decoding
Authors: name: Long Zhao; name: Sanghyun Woo; name: Ziyu Wan; name: Yandong Li; name: Han Zhang; name: Boqing Gong; name: Hartwig Adam; name: Xuhui Jia; name: Ting Liu
Published: 2024-10-05 08:27:53

Abstract: In generative modeling, tokenization simplifies complex data into compact,
structured representations, creating a more efficient, learnable space. For
high-dimensional visual data, it reduces redundancy and emphasizes key features
for high-quality generation. Current visual tokenization methods rely on a
traditional autoencoder framework, where the encoder compresses data into
latent representations, and the decoder reconstructs the original input. In
this work, we offer a new perspective by proposing denoising as decoding,
shifting from single-step reconstruction to iterative refinement. Specifically,
we replace the decoder with a diffusion process that iteratively refines noise
to recover the original image, guided by the latents provided by the encoder.
We evaluate our approach by assessing both reconstruction (rFID) and generation
quality (FID), comparing it to state-of-the-art autoencoding approach. We hope
this work offers new insights into integrating iterative generation and
autoencoding for improved compression and generation.