English
Autoregressive
Tokenizer
IBQ-Tokenizer-8192 / README.md
shifengyuan's picture
Update README.md
4a7d125 verified
metadata
license: apache-2.0
language:
  - en
tags:
  - Autoregressive
  - Tokenizer

Taming Scalable Visual Tokenizer for Autoregressive Image Generation

Code: https://github.com/TencentARC/SEED-Voken

Paper: https://arxiv.org/abs/2412.02692

This repo is used for hosting IBQ’s checkpoints.

Introduction

We propose Index Backpropagation Quantization (IBQ), a new vector quantization method for the joint optimization of all codebook embeddings and the visual encoder, ensuring the consistent latent space.  IBQ enables scalable training of visual tokenizers and, for the first time, achieves a large-scale codebook (2^18) with high dimension (256) and high utilization.