WeTok: Powerful Discrete Tokenization for High-Fidelity Visual Reconstruction
Authors: Shaobin Zhuang
Yiwei Guo
Fangyikang Wang
Canmiao Fu
Zhipeng Huang
Zeyue Tian
Xiaohui Li
Ying Zhang
Chen Li
Yali Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Image Reconstruction
Image Generation
Primary area: foundation or frontier models, including LLMs
TL;DR: A family of powerful discrete visual tokenizers designed to resolve the long-standing conflict between compression efficiency and reconstruction fidelity.

Abstract: Visual tokenizer is a critical component for vision generation. However, the existing tokenizers often face unsatisfactory trade-off between compression ratios and reconstruction fidelity. To fill this gap, we introduce a powerful and concise WeTok tokenizer, which surpasses the previous leading tokenizers via two core innovations. (1) Group-wise lookup-free Quantization (GQ). We partition the latent features into groups, and perform lookup-free quantization for each group. As a result, GQ can efficiently overcome memory and computation limitations of prior tokenizers, while achieving a reconstruction breakthrough with more scalable codebooks. (2) Generative Decoding (GD). Different from prior tokenizers, we introduce a generative decoder with a prior of extra noise variable. In this case, GD can probabilistically model the distribution of visual data conditioned on discrete tokens, allowing WeTok to reconstruct visual details, especially at high compression ratios. On the ImageNet 50k validation set, at a high-fidelity setting, WeTok achieves a record-low zero-shot rFID of 0.12, outperforming leading continuous tokenizers like FLUX-VAE (0.18) and SD-VAE 3.5 (0.19) with 400% compression ratio. Furthermore, in a high-compression regime, WeTok achieves a zero-shot rFID of 3.49 at a 768× compression ratio, substantially surpassing Cosmos, which scores 4.57 at only 50% our compression ratio.

WETOK: POWERFUL DISCRETE TOKENIZATION FOR HIGH-FIDELITY VISUAL RECONSTRUCTION
Shaobin Zhuang!. Yiwei Guo
Fangyikang Wang* Ying Zhang² WeChat Vision, Tencent Inc.
Canmiao Fu²
Zhipeng Huang² Zeyue Tian S.
Xiaohui Li
Chen Li Yali Wang3...
Shanghai Jiao Tong University
Shenzhen Key Lab of Computer Vision and Pattern Recognition, Shenzhen Institutes of Zhejiang University
Advanced Technology, Chinese Academy of Sciences 5 Hong Kong University of Science and Technology
6 Shanghai Al Laboratory
SWING PARIDE 1948
Mashed
Soalabk Codakask
Senerative Recae.
GALE STORM
0.15
Encoder
Escodes
Escode
Loolog-Frre Fre
x-X
0.10
Deceda
Dasoder
Decaderr
(Dun))
SWING PARADE
SWING of
SD-VAE 3.5
LlansGenn
Solor
Open-MAGGVI2
Cosres
SOXL-VAR
HhiTTck
SD-VAE 2
GALE SURMN
100
200
3001
400
500
600
700
800
Compraacion Bario Sozz Comprosnnd
Compuctoon
PQYAR
Open M4CV77
WeTWi (Chon)
Quantitative comparison with other methods
(b) Qualitative comparison with other methods
Figure I: Zero-shot reconstruction comparison with state-of-the-art tokenizers. (a) Our WeTok establishes new state-of th--th--rtt trade-off between compression and reconstruction performance among the compared methods. (b) WeTok achieves significant improvement reconstruction quality over previous discrete tokenizers such as VQVAE and Open-MAGVIT2.
ABSTRACT
Visual tokenizer is a critical component for vision generation. However, the ex- isting tokenizers often face unsatisfactory trade-off between compression ratios and reconstruction fidelity. To fill this gap, we introduce powerful and concise WeTok tokenizer, which surpasses the previous leading tokenizers via two core innovations. (1) Group-wise lookup-free Quantization (GQ). We partition the la- features into groups, and perform lookup-free quantization for each group. As a result, GQ can efficiently overcome memory and computation limitations of prior tokenizers, while achieving a reconstruction breakthrough with more scal- able codebooks. (2) Generative Decoder (GD). Different from prior tokenizers, we introduce a generative decoder with prior of extra noise variable. this case, GD can probabilistically model the distribution of visual data conditioned on discrete tokens, allowing WeTok to reconstruct visual details, especially at high compression ratio. On the ImageNet 50k validation set, at a high-fidelity setting, WeTok achieves a record-low zero-shot rFID 0.12, outperforming leading con- tinuous tokenizers like FLUX-VAE (0.18) and SD- VAE 3.5 (0.19) with 400% compression ratio. Furthermore, zero-shot rFID of 3.49 768x compression ratio, substantially surpassing Cosmos, which scores 4.57 at only 50% our compression ratio. Code and models are available: http://gttabbccm//uuugghhhhitt
high-compression regime, We'Tok achieves
"Corresponding author.
Work done interss WeChat Vision, Tencent Inc.

---

INTRODUCTION
In visual generation, the high computational cost of pixel-based data is central challenge CChen jt al. 2020b Rombach al. 2022b). Visual tokenizer is a key solution that uses an encoder to compress image into a compact latent representation and a decoder to reconstruct it (Kingma & Welling 2013; Rezende et al. 2014), allowing generative models to operate efficiently in latent space (Rombach et al. (022a). These tokenizers are broadly divided into two categories: continuous (Kingma & Welling 2013) and discrete (Van Den Oord et al., 2017). Continuous tokenizers map images continuous latent space, while discrete tokenizers employ a quantizer to produce finite set of codes. This architectural difference introduces a critical trade-off. Discrete tokenizers can achieve higher compression ratio, but this efficiency often comes the cost of lower reconstruction fidelity compared to continuous methods. This leads to a natural question: Can we build a discrete tokenizer that can maintain high compression as well as achieve high-fidelity reconstruction?
To achieve this goal, two critical issues must be resolved. (1) Scalable Codebook. To minimize quantization error of discrete tokenizers, the existing methods attempt to enlarge the codebook (Yu
itt al. 2024al Zhao et al. 2024c [un et al. (2024). In particular, the Lookup-Free Quantization (LFQ) Yuu et al. 2024a) quantizes the latent features directly, which largely increases the codebook size for better reconstruction. However, the substantial memory and computational overhead required to manage such large codebook during training hinders further scalability. (2) Generative Modeling. Discrete tokenizers are inherently deterministic. Rather than modeling data distribution of images, decoder trained to reconstruct the expected value of images (Esser et al.) 2020), corresponding to the latent codes from encoder. Such manner is limited to capture rich diversity and fine details in the original images, leading to unsatisfactory reconstruction, particularly at high compression ratios.
To fill the gap, we introduce powerful discrete tokenizer, WeTok, which consists of two concise designs to solve the issues above. First, we develop a Group-Wist Lookup-Free Quantization (GQ), which groups the latents and employs LFQ for each group. As shown in Tab. and Fig. 3 GQ
addresses the challenge in LFQ where entropy loss (Chang et al.] 2022) Jansen et al.) 2019) causes memory usage to grow with the codebook, while yielding superior reconstruction performance. Furthermore, we analyzed that the theoretical error caused by GQ is strictly smaller than that of BSQ Second, we introduce the GAN-style generator into the decoder (GD). In Fig. 77] GD effectively models data distribution of images, allowing to reconstruct visual details at high compression ratios.
Finally, we conduct extensive experiments on mainstream benchmarks, via scaling WeTok across group size, model size, and training data size. Moreover, we pre-train our We'Tok on a 400M general-domain dataset across multiple compression ratios. As illustrated in Fig. [ (a), WeTok consistently outperforms the state-of-ther the-art continuous and discrete tokenizers with a 400% com- pression ratio, e.g., rFID on ImageNet 50k validation set: WeTok: 0.12 vs. FLUX- PLUX-VAE: 0.18
(Batifol et al. 2025) vs. SD-VAE 3.5: 0.19 (Esser et al. (002aa) Furthermore, our highest com- pression model also achieves the superior reconstruction performance, e.g., rFID on ImageNet 50k validation set: WeTok: 3.59 vs. Cosmos; 4.57 (Agarwal et al. 2025)) while Cosmos (384) only has 50 % compression ratio of our WeToo (768), showing effectiveness and efficiency of WeTokk
2 RELATED WORK
VQVAE Van Den Oord et al.) 2017) and VQGAN (Esser et al.) 2021) employ vector-quannizatioon (VQ) to transform visual input into discrete tokens. But they both suffer from low reconstruction quality caused by instability of the codebook utilization. To overcome these drawbacks, one line of work introduces optimization strategies or modules to improve performance (Lee et al. 20226
Shi et al. 2024; Zhu et al. 2024 Yu et al. 2024c). Another line of work focuses on scaling up the
codebook size by grouping codebooks (Ma et al. 2025 Jia et al. [2025] Zhang et al. 2025 Bai et al.) 2024 ImageFolder (Li et al. 2024). DualToken (Song et al. (025)) and TokenF-low (Qu al. [024] use multiple codebooks to assist in optimizing model understanding and generation capabilities.
However, VQ-based tokenizers still introduce additional costs due to the lookup operation (Yu et al.)
202 Lee al. 2022b Fang et al. 2025). MAGVIT-v2 (Yu et al. 2024a) introduces Lookup- Free Quantization (LFQ) to address extra cost and proposes the entropy loss (Chang et al. 2022]
Jansen et al.) 2019) to ensure the utilization of the codebook. BSQ (Zhao et al., 2024b) assumes independence between the bits of the binary code to eliminate unbearable computational overhead from entropy loss, while this assumption leads to performance degradation. In contrast, WeTook does
2

---

Group- Lookap-Fry Free Quantzation
Generative Decoderr
Recenatectes
Paneppal
1ask
Lookup-Ttee Convers Grouped
Tikera into Indesos
Grouped Token Loss
Groupal Codehookk
Gruupal Caicbook Lows
Coenel Erarpy
Counit
Gveeil
Ladio
Tekan Erasey Loss
Figure 2: WeTok with Group- Lookup-Free Quantization and Generative Decoder.
not rely on explicit codebooks, and eliminate the memory usage caused by entropy loss while having better performance than LFQ. More related works could be found in Sup. B
DiTo (Chen et al. 2025b), consistency decoder (Betker et al.) and e- e-VEE Zhao et al.
2024a intro-
duce diffusion into decoder help visual reconstruction of continuous image tokenizer. Our We'Tok
the first to introduce a generative decoder into discrete tokenizer, overcoming the instability of generative training caused by the stopping gradient estimation operation. Compare diffusion modeling, Compared to diffusion modeling, our GAN-based WeTTok achieves more efficient single- step sampling while also achieving state-of-the--e reconstruction performance.
METHOD
In this section, we first establish the necessary preliminaries for discrete tokenization. We then in- troduce the Group-Wiss Wise Lookup-Free Quantization (GQ) to unify Lookup-Free Quantization (LFQ) YYu al. 2024a) and Binary Spherical Quantization (BSQ) (Zhao et al. (024c). Finally, we present Generative Decoder (GD) specifically engineered for high-compression scenarios to reconstruct high-fidelity outputs from the compact representations generated by our GQ.
3.1 PRELIMINARIES
Vector Quantized Variational Autoencoder (Esser et al. 2020). VQVAE first compresses image #x*
I E R W x3 into latent feature E Mx through the encoder E. Then it is quantized
into latent codes by searching the nearest neighbor in the codebook CER xdd_d [c1, C2...., cxl
ji arg min [[ cll2,
(1)
cEC
Notice that here we define Q to have a backward gradient only with respect to C, and introduce Ug = 2/ + sg Q LI] as variable that shares the same value Q but has backward gradient only with respect to U. Here, sg[-] denotes the stop-gradient operation.
Finally, the intermediate quantized result Q is reconstructed into image space Ý = G(U@) through the decoder G. The loss function of VQ-VAE consists of the following five parts,
LVQUAR
- 11 |ll +a IM sgll +3 LIPIPS (I, (I, +y (o)N(
(2)
Recon. Loss
Codehook Loss
Commitment Loss
Perceptual Loss
GAN Loss
where perceptual loss (Zhang al. 2018) and GAN loss introduced for better visual quality.
Lookup-Free Quantization et book CLRO = { 1, 11d to perform lookup- free quantization on each channel of latent feature,
2024a). . LFQ introduces an implicit and learning free code- et al.
Q sign(L((i,
(3)
3

---

Since the codebook in LFQ is fixed, there is need for codebook loss during LFQ training. To address the issue of codebook utilization collapse VQVAE, LFQ introduces the following entropy loss in replace of the commitment loss in equation 2
1 ΣE i=
Entropy (L)
H (g(el((5))) -
hw (c(
(4)
hw i=j==1
Token Entropy Loss
Codebook Entropy
where (X) = - -x X,logX; {(c(U[(;,))) denote the conditional distribution of given 3] and refers to the weight of codebook entropy loss.
Binary Spherical Quantization (Zhao et al. 2024c). When increasing the codebook size, i.e. increasing d, the (.)) calculation in the Listrop of LFQ leads to significant memory consumption. To alleviate this issue, BSQ processes each binary of LFQ seperately. Firstly, rewrite the token en- 1. E Ej= Ei ((q
tropy loss as how
k))). Next, the codebook entropy loss could
be transformed into E H(gs(cik) Aw Ti Σl Ei= ((c/[/(i. ()) R I the variable space of the H--) calculation from {- to the linear combination of d 1-1.1) 1), significantly decreasing memory consumption. However, as the approximation for the codebook entropy loss would introduce additional errors, which leads to performance degradation.
Σ i= Eド= Ji J. k))) by assuming the approximation Both operations reduce
Aw E Σド= q (c (i. (.
3.2 GROUP- WISE LOOKUP- FREE QUANTIZATION
As shown in Eq. the computational cost of H(-) increases linearly with the codebook. To address and the optimization error of BSQ, we group the latents and perform LFQ. As shown in Fig. group the latent features in channel dimension, reshape U into xx where d=gd and and d' represent the number and channel of groups. For k-th group, there is a non-lcarnable grouped codebook Coo.==(-1,1)) The conditional probability can be reformulated as
a(clU(i,))) = II ga (c& (,
(5)
where с₁ refers to the k-th latent code after c is divided into 9 parts, i.e., c₁ Considering the additivity of entropy and Eq. 5 we can rewrite the token entropy loss term as
cl(k- 1)dd +1 kd").
Lioken Entropy Loss
lww
H(g(cU((.)))
= hw
H ((oc(c (6)
==
We first transform the token entropy loss from the 1)1, space into a linear combination of {-1, 13dd spaces. This change eliminates the token entropy loss as the memory bottleneck.
For the codebook entropy loss, the H :) operation prevents us decompose the { 1)dd space into a linear combination of multiple subspaces. We propose the assumption that
ΣV q(cU(i,))) = 2 (ccc,
IΣ
Σ
(7)
Thus, we could transform the codebook entropy loss into
h H( hw
h har ΣΣ ca
LCadehook Entropy Loss
((clU()))))))
=
(8)
Similar to the token entropy loss, the codebook entropy loss is converted from {-1, 13 space
into grouped form, which eliminates the memory bottleneck caused by this term. GQ provides tunable trade-off between approximation accuracy and memory cost by g. Experiments in Sec. 4.1
demonstrate that by selecting appropriate g, GQ significantly reduces memory overhead while
4

---

063
0.18
0,22
22.00
0.123
0.113
0.14
0.54
21.25
0.103
21.00
0.52
30.75
Training Skepe
Training Saggs
Training Steps
Figure 3: Quantization method ablation. GQ and LFQ significantly better than BSQ.
0.36
500
(285)
0.14
12
3.4
0.5
20.5
0-tafti
0-0
250
Viliaiion
SSIM
0.38
200
0.28
0.8
(-R15511)
(1
150
0.54
1008
1804
200L
2004
100k
1.00
2004
1006
1966
30%
250c
Training Staps
Training Stags
Training Staga
Figure 4: Number of group ablation. G refers to the number of group. The reconstruction perfor- mance of the model increases significantly with the increase of G.
8.120
0:22
262
0.8955
CH1S
Dnd
C-25,
0.115
0.71
26.0
5.100
68255
21
8.100
2900
1.2
D162
0.875
5.00
067
0.05
0.00
230
Trining Steps
Trining
Trinnig Stepe
Figure 5: Model architecture ablation. C and B refer to the number of base channel and residual block respectively. C' = 256 and B = 4 achieve the best reconstruction performance.
introducing minimal optimization error. This allows GQ to surpass the LFQ and BSQ. Furthermore, this tunable design provides the flexibility scale the codebook to a virtually unlimited size.
Proposition 3.1. For any choice of group G. the codebook entropy approximation error (as in Eq. 7 of our GQ method is smaller than that of the BSQ method.
Remark 3.2. The detailed proof and definition of the approximation error of the Proposition 3.1 is deferred to Sup. A We mainly adopt the order theory in abstract algebra to derive such conclusion.
3.3 GENERATIVE DECODER
Unlike the continuous tokenizer, the decoders in previous discrete (Esser et al., 2021 Yu et al. 2024a) tokenizers fit deterministic transformations. Although GAN loss is employed during training as shown in Eq. 2 and its specific form is as follows
(u)) log(1 D(G(U)))),
where D the discriminator: However, the GAN loss only serves assist in improving the percep- tual quality. In high compression ratio scenarios, the correspondence between Ug and the ground truth likely not unique. In such cases, what we need a generative decoder. As shown in Fig. 2 we randomly sample E (O, concatenate it with Ug along the channel dimension, and then feed the result into the decoder. In this way, the GAN loss is subsequently reformulated as
LaAN(M2) EEzEN (log(1 (o)()
(10)
5

---

0.900
24
6120
0.72
25.
24
0.15
0.70
25.6
Valldion
0.044
000
0.90
1.1
0.100
23.0
1.4
0160
22.5
Training Skeps
Thining Stups
Training
Figure 6: Training data and learning rate schedule ablation. GD refers to general general-domain data. The model trained on general-domal domain data not as good as the model trained on in in-distribution data in terms of distribution fitting metrics, but has better generalization of consistent learning schedule is significant compared with warm up + cosine decay.
PSNR and SSIM. The effect
Table I: Memory usage ablation. We set Table Generative decoder modeling ablation.
d'=8 in GQ. 0OM refers to out of memory. Stage2 refers to generative decoder modeling.
Method
d=8
d=16
24
d 32
Stagel
Stage2
rFI
LPIPS
SSIM
PSNR
LRQ
10.5 GB
G8
00M
00M
00M
5.37
0.17
0.54
20.53
BSQ
10.5
GB
10.6
60
10.5
G8.6
GB
G8
10.6
/
3,90
0.16
0.55
20.72
where Ug serves as condition for the generation process. The development from Eq. 9 to Eq. not merely concatenating the input of the decoder. More importantly. the decoder shifts to modeling the transformation from Gaussian noise conditioned on to the ground truth distribution. In previous discrete tokenizer, the decoder is trained to minimize a reconstruction loss like L2 or LPIPS. When a single, highly compressed discrete token Ue could correspond to multiple ground- truth images (e-g., different textures of fur, different patterns leaves), the decoder learns to output the conditional expectation or the "average" of all these possibilities. Our GD, by incorporating z as an additional input, transforms the decoder into a conditional generative model. no longer learns one-to-one mapping but instead models the conditional distribution. The noise vector allows the decoder to sample one specific, plausible instance from this distribution. This single sample can contain coherent, high-frequenee details (e.g.. a specific fur texture), making the output appear much more realistic and less blurry than the "average" image.
10
To ensure training stability, we employ two-stage training strategy. In first stage, we train WeTok with the reconstruction loss, i.e. Eq. [ and In the second stage, we adapt the model for generative tasks. Specifically, we expand the channel dimension of the conv_in layer in decoder and employ zero-initialization to new channel to accept z as additional input. This strategy ensures that at the beginning of the second stage, the decoder's behavior is identical its pre-trained state.
4 EXPERIMENTS
Datasets. We perform large-scale training on two datasets: (i) 1.2M ImageNet (Russakovsky al., 2014) training set; (ii) 400M general-domani dataset. For comparison, we evaluate WeTok perfor- mance ImageNet 50k validation set and MS-COCO 2017 validation set (Lin et al. 2014). Unless otherwise stated, we conduct a series of ablation studies on the ImageNet training set. Besides, we train the taas-to- model on the ImageNet training set and test it the validation set.
Settings. WeTok adopts the architecture proposed in Open-MAGVIT2 (Luo et al. 2024). employing the CNN architecture for encoder, decoder, and discriminator. Images randomly cropped to 256 x 256 for training. For ablation study, all models are trained for 250K steps with Adam (Kingmal & Ba 2014) and consistent set of hyperparameters. For large-scale training, hyperparameters are individually tuned for each model to achieve optimal performance, For class-to-image generation, we adopt the transformer architecture in LlamaGen (Sun al. 2024). More details Sup. E
4.1 ABLATION STUDY
We conducted a comprehensive ablation study to validate the key components of WeTok. We first verify the effectiveness of our proposed GQ and GD. Then ablate the performance improvements

---

Ground Trush
wio Generative Decoder
w/ Generative Decoder
Greund Truth
wel Genemtive Doedeer
to Gerenative Decoder
*
Figure 7: Qualitative ablation of GD on MS-COCO val2017. The images reconstructed by the model with GD are obviously more fidelity and natural.
Ope-MAGGVTZ-AR
WeTok-AR (Ours)
FID (L Lower is Better)
IS (1 Higher is Better)
Precision (1 Higher is Better)
Recall (f Higher is Better)
0,500
0.55
3.5
370
08:5
0.54
5.00
2.5
260
0.44
0.30
2.30
230
0355
0.50
240
0.550
0.51
Model Perameters
Modkl Panmuturs (M)
Model Parmuturss (M)
Model Panematirn
Figure 8: Parameter ablation experiments of the Autoregressive model with WeTok. The per- formance of the AR model with WeTok improves more significantly with the increase of parameters.
various dimensions, including the number of groups, model architecture, training data, and learn- ing rate schedule. In addition, we conduct parameter ablation experiments the AR model based on WeTok. More details in Sup. [E. We employ validation loss, rFID (Heusel et al.) 2017). LPIPS Zhang et al. 2018), SSIM (Wang al. 2004), and PSNR to evaluate the quality of ablation study.
Quantization method. We ablate quantization methods under the same compression ratio, i.e., d=gd'=16. As shown in Tab. [ GQ does not introduce additional memory usage BSQ. In Fig. 3 we set the same compression ratio for different quantization methods (LFQ: g=1, d6=16; BSQ: 16 d1=1; GQ: g-2, d' 8),, GQ performs better than LFQ and far exceeds BSQ.
Generative decoder. When the performance of the model saturated after the first stage of recon- struction training. we conduct the second stage of generative training. As shown in Tab. 2 the results show that GD continues to improve the reconstruction performance of the model, especially in rFID. We present qualitative ablation results in Fig. 7 After converting the decoder a generative model, the reconstructed images are more realistic, demonstrating the effectiveness of our GD.
Number of groups in GQ. We increase the number of groups by a power of 2. As shown in Fig. 4 the results show that as g increases, the reconstruction performance of the model continues to increase significantly and does not encounter the memory bottleneck like LFQ.
Model architecture. We ablate the number of base channels and residual blocks in the encoder and decoder to scale up WeTok. As shown in Fig. 5 across different settings, configuration with 256 base channels and 4 residual blocks achieves the best reconstruction performance. The encoder and decoder for this optimal architecture contain 198M and M parameters, respectively.
Training data. As shown in Fig. 6. we ablated models trained on ImageNet versus large 400M general-domain dataset. The model trained on the loss and higher SSIM and PSNR, but performs worse rFID and LPIPS. We attribute to the dis- tribution gap between the general-doman data and the in-distribution ImageNet dataset. This result highlights trade-off between generalization and performance in-distribution evaluation metrics.
data achieves a lower validation
Learning rate schedule. The warm-up and cosine decay learning rate schedule widely adopted in training. However, this convention may be suboptimal for the training of discrete tokenizers. In Fig. 6 the model trained with a constant learning rate demonstrates better performance. Based on this, adopt the constant learning rate schedule for all subsequent large-scale training in WeTok.

---

Ground Truth
SD-VAE 1.5
SDXL-VAE
WeTok (Ours)
RACECENTRE
WHYSH
CALTTER
RAIE EXECECERRE
WETEER
PA
PA
PA
PA
Figure 9: Qualitative comparison of 512 x 512 image reconstruction on TokBench.
Table Reconstruction evaluation on 256 256 ImageNet 50K validation set. All models trained on ImageNet. WeTok achieves SOTA results on different downsampling rates. specifies that it is obtained through testing.
Token
Train Resolution
Codebook Size
Codebook
Method
Tokens
Ratio
rFID]
PSNR
Type
Usagef
VQGAN Esser ct al.
2020
x 16
16
256 x 256
1024
8.30
19.51
VQGAN Hsser ct
2020
x 16
16
x 256
16384
4.99
20.00
SD-VQGAN Rombach ct
2022h
x 16
16
256 x 256
16384
5.15
MaskGIT Chang et al 2022
2D
16 x 16
16
256 x 256
1024
2.28
ReVQ Zhang ct 2025
2D
16 x 16
16
256 x 256
65536
2.57
21.69
LlamaGen (Sun ct 2024
2D
16 x 16
16
256 x 256
32768
2.26
20.59
85%
LlamaGen Sum m
2124
2D
16 x 16
16
256 x 256
16384
2.19
20.79
97%
ReVQ Zhang et al 2025
2D
16 x 16
16
256 x 256
248
2.05
21.96
Sweetlok Yu et a 2024b
1D
256
16
256 x 256
10481
0.73
TYTok* Yu al. 20244b
ID
256
16
256 x 256
4096
1.66
20.01
100%
FlexTokk [Bachaann
2025
1D
256
256 x 256
64000
1.45
18.53
VAR Tian 2024)
16
16
4096
21.30
97%
IBQ Shi 2025
16
16384
22.35
96%
Opcn-MAGVV12 [Luo
2024
16
x
2
1.17
22.64
100%
IBQ et 2025
16
16
x 256
218
1.00
20.30
84%
FlowMo-Lo (Sargent et al.
2025
ID
256
x 256
218
0.95
22.07
VFMTok [Zheng et al. 2025
ID
256
x 256
16384
0.89
100%
Giga Xiong el al. 2025
1D
256
256 x 256
16384
0.79
21.65
AliTok Wi el ml. 2025c
ID
273
256 x 256
4096
0.84
MGVQ Dia el 2025
2D
16 x 16
16
256 x 256
252
0.64
23.71
100%
WeTok (Ours)
2D
16 x 16
16
256 x 256
2
0.61
24.50
100
ViT- Yu et al.
2021a
2D
32 x 32
8
256 x 256
8192
1.28
OmiTokenizer VQ Wang
2024a
2D
32 32
256 x 256
8192
1.11
Jamaaenn (Sun
2024
2D
32 x 32
256 x 256
16384
0.59
24.45
Open-MAGVITE (Luo
2024
2D
32 32
128 128
218
0.34
27.02
100%
BSQ Zhao al.
2024b
1024
256 256
2
1.14
25.36
100%
FlowMo-Hi Sargent
2025
1024
x
2
0.56
24.93
Selftnk WWng al. 2025
1024
213
0.54
BSQ ZZao et al. 20246
1024
x 256
236
0.45
28.14
100%
SweetTok Yu et al. 2024b
1021
x 256
10481
0.37
MGVQ Jia at. 2025
32 x 32
256 x 256
252
0.31
28.42
100%
WeTok (Ours)
2D
32 x 32
8
256 x 256
242
0.19
29.69
100%
Parameter of autoregressive model. As shown in Fig. BeTTo-basee AR model and compared it with Open-MAGVIT2-1 AR models. The results show that the performance of the WeTok-based AR model is slightly inferior than the Open-
ablate the parameter size of the
8

---

Table Zero-shot reconstruction comparison on ImageNet and MS-COCO va12017 validation set. Our WeTokk achieves the best performance on both resolution settings.
Tokenizer
Training
Compression
MS-COCO 2017
Imagenet-1k
Method
Ratio
Type
Data
Ratio!
IID
PSNR
SSIM
IIDD
PSNR
SSIM
Resize 230 256
WeTok (Ours)
Discrete
400M
32
768
8.94
20.31
0.55
3.49
20.77
0.55
Cosmos Agarwal et ol.
2025
Discrete
16
384
11.97
19.22
0.48
4.57
19.93
0.49
Show-o ef
Discrete
35M
9.26
20.90
0.59
3.50
21.34
Open-MAGVIT2-R PP Luo
2024
Discrete
103M
16
439
7.93
22.21
0.62
2.55
22.21
0.62
LlamaGen Sun et al
2021
Discrete
70M
16
139
8.40
20.28
0.55
2.47
20.65
0.54
WeTok (Ours)
Discrete
400M
16
384
6.55
21,99
0.63
1.58
22.38
0.62
DALL-E dVAE (Ramesh et al.
2021a
Discrete
103M
18
118
48.60
26.97
0.08
32.63
27.31
0.79
BSQ Zhho et 20240
1B
219
3.81
24.12
QU JP..P
72in et al
Zhin
Discrete
18
219
3.21
23.16
0.63
QLIP-L
elar ai A72
Diserete
18
168
1,46
25.36
0.69
SD-VAF
(Rombach
2022a
Discrete
IB
110
5.75
24.17
0.70
1.13
24.48
0.69
Wellok (Ours)
Discrete
400M
16
192
4.41
24.44
0.74
0.60
24.77
0.73
SD-VAIE
[Rombach
Continuous
1B
5.94
0.69
23.54
QLIP-B Zhao al MMAS
Disercte
18
55
0.70
26.79
0.79
SD-VAE Romaacc et 2022a
Continuous
6B
24
4.26
26.62
0.77
0.70
26,90
0.76
SDXL-VAE (Podell
Continnous
26B
24
3.93
27.08
0.80
0.67
27.37
0.78
UniTok
Ma er 21125
Discrete
18
16
64
0.41
<AAE-D ZZao al
1024a
Continouus
3.65
26.01
0.86
29.49
WeTak (Ours)
Discrete
400M
48
2,18
29.49
0.89
0.20
29.63
0.88
SD-VAE 3.5
Esser et al.
2024b
Continuous
1.66
31.08
0.90
0.19
31.19
0.90
FLUX- VAB
Continnous
1.35
32.32
0.93
0.18
32.74
0.92
WeTok (Ours)
Diserete
400M
24
1.43
32.00
0.93
0.12
32,06
0.93
Original Resolution
WeTok (Ours)
Discrete
400M
32
768
8.94
20.31
0.55
3,49
20.77
0.55
Cosnos (Agarwal et al. Open-MAGVITZ-FPI
2025
Discrete
16
384
7.23
20.45
0.53
2.52
20.49
0.52
2024
Discrete
100M
16
6.65
0.57
21.74
WeTok (Ours)
Discrete
400M
16
384
5,30
21.94
0.59
0.81
21.99
0.58
DALL-E dVAE (Ramesh et al
10
Discrete
103M
18
118
55.07
25.15
0.75
36.84
25.46
0.74
SD-VAE Lx Rombach
Discrete
IB
8
110
6.07
22.54
0.65
1.23
22.82
0.64
WeTok: (Ours)
Discrete
400M
16
192
3,80
23.70
0.67
0.40
23.75
0.67
SD-VAE
Rombach
Continouus
1B
5.94
0.64
21.99
SD-VAE 2xx
Kombaahh d N7775
Continuous
6B
8
24
4.63
24.82
0,72
0.78
25.08
0.71
SDXL-VAB
Podall et 2023
Continnous
2-68
24
4.23
25.11
0.74
0.72
25.38
0.73
WeTok (Ours)
Discrete
400M
48
2.09
27.50
0.82
0.18
27.54
0.82
SD-VAE 3.5
Esser ct al. 2024b
Continuous
8
6
1.64
28.35
0.86
0.24
28. 39
0.86
WeTok (Ours)
Discrete
400M
1.46
0.85
29.51
MAGVIT2-based AR model when the parameter size is small. However, as the parameter size increases, WeTok-based AR model surpasses the Open-MAGVVIT2-bsel AR model in all metrics.
4.2 COMPARISON WITH STATE-OF-TIIE-ART
Visual Reconstruction. We first evaluate WeTok's performance in in-distribution setting on Ima- geNet. As shown in Tab. 3 WeTTok outperforms existing methods across different downsampling ratios. Subsequently, we evaluate WeTok's performance in general-domain setting. As shown in Tab. WeTok shows the state-of-the-arr reconstruction performance in wide range of compres- sion ratio scenarios. It not only shows the strongest performance among discrete tokenizers, but also even surpasses the current strongest continuous tokenizers, LLUX-VAE (Labs) 2024) and SD-VAE 3.5 (Esser et (004a)). As shown in Fig. [4] we present qualitative comparison results on the TokBench (Wu et al. 2025b), our WeTok outperforms the widely used SDXL- VAE (Podell et al.) 2023 and SD-VAE 1.5 (Rombach et al. (022b) under the same compression ratio.
Iterative Visual Reconstruction. The use of a tokenizer for iterative image compression- decompression is promising approach for information transmission (Joshi et al. 22000. As shown in Fig. 10 we compare WeTok (192 compression ratio) with SOTA tokenizers (Labs) 2024: Esser We surprisingly find that while recon- structions from leading models like FLUX-VAE and SD- VAE 3.5 collapse after iterations, WeTok's outputs remarkably robust and converge to fixed value. More details in Sup. C
al. 2024b) iterative image wemppeesioondecompesseee
Visual Generation. To evaluate WeTok's capabilities for visual generation, we modifys LlamaGen like OpVn-MAGVIT2. We employ the in-distribution WeTok with 16x downsampling in Tab. 3 as

---

Image / Dogradation over 200 Iteraioo
Iieraion
Herztion 50
Itentionn 100
SD-VAE 3.5 FLUX-VAE WoTek (Ours)
Isunge
671
ILUSANE
Figure 10: Qualitative and quantitative comparison of iterative image reconstruction.
Table Class-conditional generation on 256 256 ImageNet. * specifies the generated images are 384 x 384 and are resized to 256x256 for evaluation.
Type
Model
#Para.
FID
IS1
Precision
Recall
ADM
Dhariwal & Nichol
2021
554M
10.94
101.0
0.69
0.63
CDM Ho at. 202T
4.88
158.7
Diffusion
LDM-4
Rommach al.
2022b
400M
3.60
247.7
DiT-XL
(Pecllss & Xic
2022
675M
2.27
278.2
0.83
0.57
SiT-XL/2
Ma ct al.
2024
675M
2.06
277.50
0.83
0.59
VQGAN Esser ct al.
2020
227M
18.65
80.4
0.78
0.26
VQGAN Esser et al. 2020
1,4B
15.78
74.3
VQGAN-re Esser
2020
1.4B
5.20
280.3
ViT-VQGAN Yu et al.. 2021a
1.7B
4.17
175.1
ViT-VQGAN-re ro Yu ct al.
2021
1.7B
3.04
227.4
RQTran (Le ct al. 2022a
3.8B
7.55
134.0
RQTran.-re Lee el al. '2022a
3.8B
3.80
323.7
LlamaGen-L (Sn et al.
2024
343M
3.07
256.06
0.83
0.52
LlamaGen-XL (Sun al.
2024
775M
2.62
244.08
0.80
0.57
AR
LlamaGen-XXL (Sun al.
2024
1.4B
2.34
253.90
0.80
0.59
LlamaGen-L Sun cf al. (024]
343M
3.80
248.28
0.83
0,51
LlamaGen-XI. (Sun (Sun et
al. 2024
775M
3.39
227.08
0.81
0.54
(Sn LlamaGen-XXI.
2024
1.4B
3.09
253.61
0.83
0.53
UniToo (Ma ct al. 2025
1.4B
2.51
216.7
0.82
0.57
MAGVIIZ-AR-1 (Luo et al.
2024
343M
3.08
258.26
0.85
0.51
Been-MAGVII--ARS
(Luo al.
2024
804M
2.51
271.70
0.84
0.54
Open-MAGVIT2-AR-XX
(Luo el al.
2024
1.5B
2.33
271.77
0.54
WeTok-AR-XI. (Ours)
1.5B
2.31
276.55
0.84
0.55
tokenizer. More details in Sup. H.2 shown in Tab. 5 our WeTok-based AR model achieves state- of-the-art performance on the ImageNet 50K validation set. This result demonstrates that We'Tok is effective tokenizer not only for image reconstruction but also for high-fidelity visual generation. As shown in Fig. 15 we show the realistic and diverse image generation results of our WR-XL-
CONCLUSION
In this paper, we introduce WeTok, family of powerful discrete visual tokenizer designed to resolve the conflict between compression ratio and reconstruction. We propose GQ to provides a scalable and memory-efficient solution for codebooks, and GD that excels at producing high-fidelity images even from highly compressed representations. Through extensive experiments, we demonstrated that WeTook consistently outperforms existing state-of- -ftt--------rtt discrete and continuous tokenizers in both in-distribution and zero-shot reconstruction tasks across a wide range of compression ratios. Furthermore, by integrating WeTok into autoregressive framework, performance in class-conditional image generation, confirming that its learned tokens are highly ef- fective for downstream generative tasks. WeTok proves that discrete tokenizers can achieve superior reconstruction quality without compromising their inherent advantage in compression.
achieved vate-of-the-an
10

---

ACKNOWLEDGMENTS
This work was supported by Guangdong Science and Technology 2024TQ08X365).
Program (Grant No.
ETHICS STATEMENT
Our WeTok is family powerful discrete visual tokenizer designed to resolve the conflict between compression ratio and reconstruction. To ensure ethical compliance, our training data is carefully selected from public sources and take deliberate measures to minimize potential biases, fully align- ing with universal ethical guidelines. We explicitly emphasize that this framework is not intended for misuse in achieving harmful purposes; downstream users are encouraged to adhere to ethical principles when applying the technology. Additionally, all authors declare conflicts of interest related to this work.
REPEATABILITY STATEMENT
To ensure full reproducibility, we will publicly release all code and data necessary to replicate our experiments. Comprehensive implementation details, including model architecture, hyperparame- ters, and training methodology. are provided in this paper and its appendix. We are committed to open-soureing all essential resources to ensure that our findings can be fully verified and built upon by the research community.
REFERENCES
Nvidia Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, Daniel Dworakowski, Jiaojiao Fan, Michele Fenzi, Francesco Ferroni, Sanja Fidler, Dieter Fox, Songwei Ge, Yunhao Ge, Jinwei Gu, Sid- dharth Gururani, Ethan He, Jiahui Huang, Jacob Samuel Huffmman, Pooya Jannaty, Jingyi Jin, Seung Wook Kim, Gergely KI'ar, Grace Lam, Shiyi Lan, Laura Leal-Taixé, Anqi Li, Zhaoshuo Li, Chen-Hsuan Lin, Tsung-Y. Yi Lin, Huan Ling, Ming- Yu Liu, Xian Liu, Alice Luo. Qianli Ma, Hanzi Mao, Kaichun Mo, Arsalan Mousavian, Seungjun Nah, Sriharsha Niverty, David Page, Despoina Paschalidou, Zeeshan Patel, Lindsey Pavao, Morteza Ramezanali, Fitsum A. Reda, Xiao-Shuai Ren, Vasanth Rao Naik Sabavat, Ed Schmerling, Stella Shi, Bartosz Stefaniak, Shitao Tang, Lyne P. Tchapmi, Przemek Tredak, Wei-Cheng Tseng, Jibin Rajan Varghese, Hao Wang, Haoxiang Wang. Hengyi Wang. Tingwei Wang. Fangyin Wei, Xinyue Wei, Jay Zhangjie Wu, Jiashu Xu, Wei Yang, Lin Yen-Chen, Xiaohui Zeng, Yuan Zeng, Jing Zhang, Qinsheng Zhang, Yuxuan Zhang, Qingqing Zhao, and Artur Zolkowski. Cosmos world foundation model platform for physical al. ArXiv, abs/2501.03577 2025.
Roman Bachmann, Jesse Allardice, David Mizrahi, Enrico Fini, Oguzhan Fatih Kar, Elmira Amir- loo, Alaaeldin El-Nouby, Amir Zamir, and Afshin Dehghan. Flextok: Resampling images into token sequences of flexible length. In Forty-second International Conference on Machine Learning. 2025.
Zechen Bai, Jianxiong Gao, Ziteng Gao, Pichao Wang, Zheng Zhang, Tong He, and Mike Zheng Shou. Factorized visual tokenization and generation. ArXiv, 1..2411116661,, 2024. URL
https://api. www.ntiisccoolla.....oomc org/Corpps 12272881440
Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dock- horn, Jack English, Zion English, Patrick Esser, Sumith Kulal, et al. Flux. kontext: Flow match- ing for in-context image generation and editing in latent space. arXiv e-prints, pp. arXiv-2506. 2025.
James Betker, Gabriel Goh, Li Jing, + TimBrooks, Jianfeng Wang. Linjie Li, LongOuyang, JuntangZhuang, t JoyceLee, & YufeiGuo, WesamManassra, & PrafullaDarriwal, CaseyChu, YunxinJiao, and Aditya Ramesh. Improving image generation with better captions. URL https: 7/api. semant cachol a ott..////////pppsS11044440002000

---

Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv: 1096, 2018.
Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T. Freeman. Maskgit: Masked generative image transformer. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022.
(CVPR), pp.
Hao Chen, Yujin Han, Fangyi Chen, Xiang Li, Yidong Wang, Jindong Wang, Ze Wang, Zicheng Liu, Difan Zou, and Bhiksha Raj. Masked autoencoders effective tokenizers for diffusion models. In Forty-second International Conference on Machine Learning, 2025a.
Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and lya Sutskever. Generative pretraining from pixels. In International conference on machine learning, pp- 1691 1703. PMLR, 2020a.
Mark Chen, Alec Radford, Jeff Wu, Heewoo Jun, Prafulla Dhariwal, David Luan, and Ilya Sutskever. Generative pretraining from pixels. In International Conference Machine Learning. 2020b.
Yinbo Chen, Rohit Girdhar, Xiaolong Wang, Sai Saketh Rambhatla, and Ishan Misra. Diffusion autoencoders are scalable image tokenizers. ArXiv, hbwp/5511.1859) 2025b. URL https://
api. semanticscho.in http://cho.aaaooooooooo/oooosaaa
Google DeepMind. Gemini 2.5 flash, 2025. URL https::tttttttttt natps://reninni.gogee..oo
Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis. ArXiv, abs/2105.05233, 2021.
Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, al. Cogview: Mastering text-to-image generation via transformers. Advances in neural information processing systems, 19822-199835, 2021.
Patrick Esser, Robin Rombach, and Bjöm Ommer. Taming transformers for high-resolution image synthesis. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12868-12877 2020.
Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogni- fion, pp. 12873-12883, 2021.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, 2024a.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In International Conference on Machine Learning, 2024b.
Xianghong Fang, Litao Guo, Hengchao Chen, Yuxuan Zhang, Dingjie Song, Yexin Liu, Hao Wang, Harry Yang. Yuan Yuan, Qiang Sun, et al. Enhancing vector quantization with distributional matching: A theoretical and empirical study. arXiv preprint 2rXiv:2506.1502 2025.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge local nash equilibrium. In Neural Information Processing Systems, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances inxneural information processing systems, 33:6840-6855 2020.
Jonathan Ho, Chitwan Saharia, William Chan, David J. Fleet, Mohammad Norouzi. and Tim Sali- mans. Cascaded diffusion models for high fidelity image generation. J. Mach. Leamn Res., 23: 47:1-47:32 2021.
12

---

Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Lcage-scale pre- training for text-to-videe video generation via transformers. arXiv preprint arXiv:2205. 2022.
Aren Jansen, Daniel P. W. Ellis, Shawn Hershey, R. Channing Moore, Manoj Plakal, Ashok Popat, and Rif A. Saurous. Coincidence, categorization, and consolidation: Learning to recognize sounds with minimal supervision. ICASSP 2020 2020 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 121-125, 2019.
Mingkai Jia, Wei Yin, Xiaotao Hu, Jiaxin Guo, Xiaoyang Guo, Qian Zhang, Xiao-Xiao Long, and Ping Tan. Mgvq: Could vq-vae beat vae? generalizable tokenizer with multi-group quantization. arXiv preprint arXiv:2507.07997, 2025.
Rajan L. Joshi, Majid Rabbani, and Margaret A. Lepley, Comparison of multiple compression cycle performance for jpeg and jpeg 2000. In SPIE Optics + Photonics, 2000.
Minguk Kang, Jun-Yar Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scaling up gans for rext-to-image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. (0124-10134) 2023.
Diederik P. Kingma and Jimmy Ba. abs/14 (bbs/141..998,, 2014.
Adam: A method for stochastic optimization. CoRR,
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. CoRR, abs/1312.6114 2013.
Dan Kondratyuk, Lijun Yu, Xiuye Gu, José Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vighnesh Birodkar, Jimmy Yan, Ming-Chang Chiu, et al. Videopoet: large language model for zero-shot video generation. arXiv preprint arXiv:2312.1422222 2023.
Black Forest Labs. Flux, 2024. /ttpattsss///tttubb.oomm
forest Labs flux
Doyup Lee, Chiheon Kim, Sachoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11513-11522, 2022a.
Doyup Lee, Chiheon Kim, Sachoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1 11523-11533 2022b.
Xiang Li, Kai Qiu, Hao Chen, Jason Kuen, Jiuxiang Gu, Bhiksha Raj, and Zhe Lin. Image- folder: Autoregressive image generation with folded tokens. ArXiv, abs/2410.01756, 2024. URL
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. In European Conference on Computer Vision, 2014. URL https::/ api. swmanticscholee......e..eoccc oorgo
Corpus ID: ID: 4113767
Dongyang Liu, Shitian Zhao, Le Zhuo, Weifeng Lin, Yi Xin, Xinyue Li, Qi Qin, Yu Qiao, Hong- sheng Li, and Peng Gao. Lumina-mgpt: Illuminate flexible photorealistic text-to-image genera- tion with multimodal generative pretraining. arXiv preprint arXiv:2408.0265: 02657,, 2024.
Dongyang Liu, Shitian Zhao, Zhuo, Weifeng Lin, Xin, Xinyue Li, Qi Qin, Yu Qiao, Hong- sheng Li, and Peng Gao. Lumina-mgpt: Illuminate flexible photorealistic text-to-image genera- tion with multimodal generative pretraining, 2025.
Zhuoyan Luo, Fengyuan Shi, Yixiao Ge, Yujiu Yang, Limin Wang, and Ying Shan. Open- magvit2: An open-source project toward democratizing auto-regressive visual generation. ArXiv, abs/2409.044100 2024.
Chuofan Ma, Yi Jiang. Junfeng Wu, Jihan Yang, Xin Yu, Zehuan Yuan, Bingyue Peng, and Xiaojuan Qi. Unitok: A unified tokenizer for visual generation and understanding. ArXiv, abs/2502.20321, 2025.
13

---

Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden and Sain- ing Xie. Sit: Exploring flow and diffusion-basee generative models with scalable interpolant transformers. In European Conference on Computer Vision, pp. 23-40. Springer, 2024.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4195-4205, 2023.
William S. Peebles and Saining Xie. Scalable diffusion models with transformers. 2023 IEEE/CVE International Conference Computer Vision, pp. 4172-4182, 2022. URL https://api,
semant wrt.CoopooooooiI/2.... 224854438
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhom, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXitv:307.001922 2023.
Liao Qu, Huichao Zhang, Yiheng Liu, Xu Wang. Yi Jiang, Yiming Gao, Hu Ye, Daniel K. Du, Zehuan Yuan, and Xinglong Wu. Tokenflow: Unified image tokenizer for multimodal under- standing and generation. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recog- URL https: https////ap..smmttissssseieo www.sseeennnnooo......... .org/
nition (CVPR), pp. 2545-2555, 2024.
CorpusID:274455
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. ArXiv, abs/2102. 12092, 2021a.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. International conference on machine learning, pp. 8821-8831. Pmlr, 2021b.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and ap- proximate inference in deep generative models. In International Conference on Machine Learn- ing. 2014.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- on computer vision and pattern recognition, pp.
2022a.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022b.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei- Fei. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115:211 -
252, 2014.
Kyle Sargent, Kyle Hsu, Justin Johnson, Li Fei-Fei, and Jiajun Wu. Flow the mode: Mode-seeking diffusion autoencoders for state-of-the-art image tokenization. arXiv preprinl arXiv:2503. 1056, 2025.
Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets. In ACM SIGGRAPH 2022 conference proceedings, pp. 1-10, 2022.
Fengyuan Shi, Zhuoyan Luo, Yixiao Ge, Yujiu Yang. Ying Shan, and Limin Wang. Scalable image tokenization with index backpropagation quantization. arXiv preprini arXiv:2412.02692. 2024.
Fengyuan Shi, Zhuoyan Luo, Yixiao Ge, Yujiu Yang, Ying Shan, and Limin Wang. Scalable image tokenization with index backpropagation quantization, 2025.
Wei Song, Yuran Wang, Zijia Song, Yadong Li, Haoze Sun, Weipeng Chen, Zenan Zhou, Jian- hua Xu, Jiaqi Wang, and Kaicheng yu. Dualtoken: Towards unifying visual understanding and generation with dual visual vocabularies. ArXiv, 0ix/2503.143244 2025. URL https:
semant www.hcaccaa.oooooo htwp://ccooo.....oooooooooooooo.
14

---

Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng. Ping Luo, and Zehuan Yuan. Au- toregressive model beats diffusion: Llama for scalable image generation. ArXiv, abs/2406.06525, 2024.
Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv:2405.098/1 2024.
arXiv preprint
Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. ArXiv, abs/2404.02905. 2024.
Aaron Van Den Oord, Oriol Vinyals, al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- lion processing systems, 30, 2017.
Bohan Wang. Zhongqi Yue, Fengda Zhang, Shuo Chen, Li'an Bi, Junzhe Zhang, Xue Song, Ken- nard Yanting Chan, Jiachun Pan, Weijia Wu, et al. Selftok: Discrete visual tokens of autoregres- sion, by diffusion, and for reasoning. arXiv preprint arXiv:2505.07552 2025.
Junke Wang, Yi Jiang, Zehuan Yuan, Binyue Peng, Zuxuan Wu, and Yu-Gang Jiang. Omnitokenizer: A joint -mage-video tokenizer for visual generation. ArXiv, abs/2406.09399. 2024a.
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409. 18869, 2024b.
Zhou Wang, Alan Conrad Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assess- ment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13: 600-612, 2004.
Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 2296611997, 2025a.
Junfeng Wu, Dongliang Luo, Weizhi Zhao, Zhihao Xie, Yuanhao Wang, Junyi Li, Xudong Xie, Yuliang Liu, and Xiang Bai. Tokbench: Evaluating your visual tokenizer before visual generation. arXiv preprint arXiv:2505.1811 2025b.
Pingyu WU, Kai Zhu, Yu Liu, Longxiang Tang. Jian Yang, Yansong Peng, Wei Zhai, Yang Cao, and Zheng-Jun Jun Zha. Alitok: Towards sequence modeling alignment between tokenizer and autore- gressive model. arXiv preprini arXiv:2506.05288 2025c.
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. ArXiv, abs/2408. 12528, 2024.
Tianwei Xiong, Jun Hao Liew, Zilong Huang, Jiashi Feng, and Xihui Liu. Gigatok: Scaling visual tokenizers to 3 billion parameters for autoregressive image generation, 2025.
Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang. Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu. Vector-quantized image modeling with improved vqgan. ArXiv, abs/21 10.04627, 2021a.
Jiahui Yu, Xin Li, Jing Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu. Vector-quantized image modeling with improved vqgan. arXiv preprint
0tt::2110004627. 2021b.
Lijun Yu, José Lezama, Nitesh B. Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, Alexander G. Hauptmann, Boqing Gong, Ming- Yang, Irfan Essa, David A. Ross, and Lu Jiang. Language model beats diffusion tokenizer is key to visual generation, 2024a.
15

---

Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang- Chen. An image is worth 32 tokens for reconstruction and generation. ArXiv, abx/2406.075555 2024b. htttht//p/appi ottp://assa..cccoo..oooooooooooa
Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang-Chieh Chen. An image is worth 32 tokens for reconstruction and generation. Advances in Neural Information Processing Systems, 37:128940-12896. 2024c.
Kaiwen Zha, Lijun Yu, Alireza Fathi, David A Ross, Cordelia Schmid, Dina Katabi, and Xiuye Gu. Language- Language-guided guided image tokenization for generation. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 15713-15722, 2025.
Borui Zhang, Qihang Rao, Wenzhao Zheng. Jie Zhou, and Jiwen Lu. Quantize-then-reatig Effi cient vq-vae vae training. arXiv preprint arXiv 2025.
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 586-595, 2018.
Long Zhao, Sanghyun Woo, Ziyu Wan, Yandong Li, Han Zhang, Boqing Gong, Hartwig Adam,
Xuhui Jia, and Ting Liu. Epsilon-vact Denoising visual decoding. 2024a. URL https:
/
Yue Zhao, Yuanjun Xiong, and Philipp Krahenbuhl. Image and video tokenization with binary spherical quantization. ArXiv, abs/2406.07548, 2024b.
Yue Zhao, Yuanjun Xiong. and Philipp Krähenbühl. Image and video tokenization with binary spherical quantization. arXiv preprint arXiv:2406.07548 2024c.
Yue Zhao, Fuzhao Xue, Scott Reed, Linxi Fan, Yuke Zhu, Jan Kautz, Zhiding Yu, Philipp Krahen- buhl, and De-An Huang. Qlip: Text-aligned visual tokenization unifies auto-regressive multi- modal understanding and generation. ArXiv, abs/2502.05173 2025.
Anlin Zheng, Xin Wen, Xuanyang Zhang, Chuofan Ma, Tiancai Wang, Gang Yu, Xiangyu Zhang, and Xiaojuan Qi. Vision foundation models as effective visual tokenizers for autoregressive image generation. arXiv preprint arXiv:2507.0844) 2025.
Lei Zhu, Fangyun Wei, Yanye Lu, and Dong Chen. Scaling the codebook size of vq-gan to 100,000 with a utilization rate of 99%. Advances in Neural Information Processing Systems, 37:12612- 12635, 2024.
Shaobin Zhuang, Kunchang Li, Xinyuan Chen, Yaohui Wang, Ziwei Liu, Yu Qiao, and Yali Wang. Vlogger: Make your dream vlog. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 8806-8817, 8817, 2024.
Shaobin Zhuang, Zhipeng Huang, Ying Zhang, Fangyikang Wang, Canmiao Fu, Binxin Yang, Chong Sun, Chen Li, and Yali Wang. Video-gpt next clip diffusion. arXiv preprint N.Ni::2005./2489 2025.
16

---

A THEORETICAL ANALYSIS ON APPROXIMATION ERRORS
A.1 ABSTRACT ALGEBRA MODELING THE GROUPING THE GQ METHOD
To rigorously analyze the relationships between different grouping strategies within the Group Quantization (GQ) framework, we employ concepts from abstract algebra, specifically lattice the-
ory. This formalization allows us to create structured understanding of how different groupings relate to one another in terms of granularity. We begin by defining the set of all possible grouping configurations.
Let the latent feature be represented by sequence of d elements, indexed by the set S = dI...
Definition A.1. (The Set of Groupings G) A grouping E G of the set of indices S is collection of non-empty, disjoint, and contiguous blocks whose union S. block set of consecutive integers. For any 9 E G, if we order its blocks B1, .,,, B3 such that for any < J. every element in B, smaller than every element in Bj, then this grouping is unique.
Remark A.2. The cardinality of the grouping sel G is 2d-1 This is because we can think of placing divider in any of the d 1 spaces between the elements {1, 2). .2.3)..... {dd 1.d]. For each space, we can either place a divider or not, leading to 2d possible groupings.
Having established the universe of all possible groupings, we now introduce relation to formally compare them based on their level of subdivision.
Definition A.3. (The Refinement Ordering Relation >) Let yı and g2 be two groupings in G. We
say that 91 1 92 if 92 is a refinement of g1. This means that every block in 92 is a subset of some block in 91-
Remark A.4. For example, let 91 = {(1,2,3), {4}} and g2 = (1),,123)) (4)). Here, {1} S (1,2,3), (2,3) C (1, and [4) C (4). Therefore, 91 92-
This refinement relation imposes formal structure on the set G. We first demonstrate that this structure satisfies the fundamental properties of a partially ordered set.
Lemma A.5. (G, 3) a partially ordered set.
Proof.
A partially ordered set must satisfy three properties: reflexivity, antisymmetry, and transitiv-
ity.
Reflexivity: For any g E G. 3 This is true because every block in g is a subset of itself.
Antisymmetry: For any 91, J2 E G, if 91 I 92 and g2 A 91, then 91 = G2- If 91 I 92.
every block of 92 is subset of block of 91- If 92 3 91, every block of Φ1 subset of
a block of y2- Let B2 be a block in 92. Then B2 C B, for some block B, in gi- Also,
Bi C B2 for some block B/ in 92. Thus, B2 B1 C B/. Since the blocks of grouping
are disjoint, we must have B3 = By. This implies B Bg. Since this holds for all blocks, 91 and 92 consist of the same blocks, so 91 92-
For any 91, 12. 93 E G. if 9i 3 92 and g2 - 9a. then 91 I ga- If 9i I g2. every block in g2 is subset of some block in 91. If g2 - gs. every block in g3 is subset
Transitivity:
of some block in 92. Let B3 be block in Then B3 C Bg for some block B2 in g2-
And B2 S B1 for some block B, in g1. Therefore, B, C By. Since this is true for every block in gs. it follows that 91 83.
17

---

The poset (G, =) possesses groupings, we can always find a unique common coarser grouping (meet) and unique common finer grouping (join), which establishes the structure lattice.
stronger and useful structure. We show that for any two
Lemma A.6. (G, 3) is a Laltice.
Proof. To prove that it is lattice, we must show that for every pair of elements 91.92 92 G, there exists a unique greatest lower bound (GLB or meet) and a unique least upper bound (LUB or join).
The Greatest Lower Bound (Meet)
Let 91-92 92 E G. Their meet, denoted Ø1 A 92, is the finest grouping that is still coarser than both 9i and 92- The meet is constructed as follows: the blocks of g1 A 92 are formed by taking unions of blocks from and that overlap.
Formally, can define an equivalence relation on the set of indices S [l,... d}. where ~ j if and only if and j belong to the same block in both and G2 after transi- tively closing the relation (i.e., i and & in the same block of I1. and l and j are in the same block of 92, then ~ j). The equivalence classes of this relation form the blocks of new grouping, which is 91 A 92-
By its construction, lower bound (g' 3 91 and g' 3 92) will be coarser than 9I 92, meaning g' 91 A 92- Thus, 91 A 92 the unique greatest lower bound.
A
91 and A 92 92- Any other grouping g' that is also
The Least Upper Bound (Join)
The join of gi and y2, denoted gi g2. is the coarsest grouping that is refinement of both 91 and J2- The blocks of the join are formed by the non-empty intersections of the blocks from 91 and 92-
Formally, for each block Bı E 91 and each block B2, g2. we form a new block B1 B2.j. The set of all such non-empty intersections forms the grouping 91 V 92-
By its construction, every block in 91 V Ga is a subset of block in and a block in g2. so 91 3 91 V Vg and 92 91 V Vg2. Any other grouping g" that is also upper bound (91 3 g" and G2 N g²) will be a finer partition than 9i V 92, meaning 91 V g2 = g". Thus, 91 V 92 is the unique least upper bound.
Since unique meet and join exist for any pair of elements in G, (G, 3) is lattice.
Finally, we consider the extremal elements of this lattice. These correspond the LFQ and BSQ quantization strategies.
Proposition A.7. (G, 3) is bounded lattice, and its universal least element is the case of k = 1 (corresponding to the case of LFQ). Its universal greatest element is the case of d (corre- sponding to the case of BSQ).
Proof. obvious.
A.2 ANALYSIS ON THE APPROXIMATION ERROR
Having established the lattice structure of the grouping strategies, we now analyze how the choice of specific grouping affects defined approximation error e(g). This error measures the difference between a sum of products and a product of sums of spatially interpreted measure of the statistical coupling across the spatial dimensions.
probabilities, which can be
18

---

Definition A.8. For any 9 E G, let its approximation error be defined as:
h < = ΣΣ 1=
lol
lgl h
II k=1 ==1
(11)
We now propose that this approximation error behaves monotonically with respect to the refinement ordering relation 3. Specifically, finer grouping (one with more blocks) will result in a larger approximation eπror.
Proposition A.9. Given two groupings 91.92 G such that I (i.e., 91 is a refinement of 92). then under the assumption that the variables associated with each block are independent, the approximation error is non-decreasing with refinement:
c(g2) S c(g1)
(12)
Proof. To simplify the proof, let's introduce more compact notation.
Let the spatial index represent the pair (i,,),, where ranges from to N hw. Let x,, (c(c [all, k]) denote the non-negative term for block & of grouping 9 at spatial location s. Where
the context clear, we write y, Let m [g] be the number of blocks in grouping g.
The eπrоr can now be written as:
YN
m
((g)
ΣI
II
X
X a
The product of sums can be expanded as:
II 3.b x
ΣE
Σ
(X 22..2... (e.......m.. Xmm.m)
81=1
This summation is over all combinations of spatial indices (s1,..., sm). We can split this sum into two parts: where all indices are identical (s1 not all identical.
Sm 8), and where the indices
m II k=
X Σ 8=1
(X Xmm.m
==1
(St...,
not all equal
Rearranging this gives:
m
IE X
ΣII k=
X
Σ (8) (5......mm)) not all equal
(Xs,,1.. ... .X.,....Xxmm.i Xmm.m)
Since (-(-) is the output of softmax function, each term Xx, non non-negative. Therefore, the sum on the right-hand side is also non-negative. This allows us to drop the absolute value bars.
N
((g)
-= X. k=1
ΣII X., 20
The relation 82 3 means that is obtained from by sequence of one or more block subdivi- sions. It is sufficient prove that the error increases after single subdivision, as the general result follows by repeated application.
Let 1g2l m and Ig1| m + 1. Per the hint that a finer group has more terms, we model refinement by introducing an additional multiplicative set of variables (X,,m+1ke 1. into the error calculation. Let A.. = e(gz) and Am+1 e(g)).
m+1 N
NN
m+11
A Am+1
N
II
X
==1
s=1
19

---

We separate the (m 1)-th term:
m N
N
N
m
Am+1 =
II
Σ
II
Xam++
From the definition of Am, we substitute
I
Es Xark
Am E, ITk=
Xo
Am+1
A m
X
Σ
X X
Σ
X Xem+1
W=1
Distributing the terms yields the recurrence relation:
N
N
NV
Xammta )
Am+1 = Am
Σ X
X.kkk
Σ
X
Xs,m+1
Σ
II 1=1
X
sm
k=1
The expression is of two components.
1. The first component is Am (Eam Xam++) Since An 2 0 and Xa,m+ 2 0, this term
is non-negative.
2. The second component, in brackets, is the form -)),,,, (A, where A, = III Xe, and B, Xam+1 are non-negative. As shown in Step 2, this form always non-negative.
Thus, Am+1 is the sum of non-negative quantities. This leads to the inequality Am+1 Z Am (IN X, xm,m+))- To ensure that Am+1 Am, rely the reasonable assumption that for any block k, the total probability mass over the large spatial domain is not contractive, i.c., Ee Xa. 1. With this assumption, we have:
Am+1 2 Am
We have shown that for single refinement step that increases the number of blocks from to m + I, the error is non-decreasing, Since any refinement 9i of corresponds to a sequence of such steps, the error for the finer grouping 9i will be greater than or equal the error for the coarser grouping
e(g1) 2 e(g2)
This proves the proposition.
Then, the Proposition 3.1 in the main paper is direct result of Proposition A.7 and A.9
B
MORE RELATED WORK
B. CONTINUOUS TOKENIZER
Generative modeling the pixel space typically requires extensive compute resources (Chen al. 2020a Ho et al. 2020). Subsequent works (Rombach et al., (202b Podell et al. 2023 Peebles & Xie| 2023 Esser et al. 2024b Batifol al. [2225] [ et al. 2025 2024 Esser al.) 2024a Zha et al. 2025 Chen et al. 2025a) adopt VAE (Kingma & Welling 2013). which projects visual content from pixels to latent features, achieving efficient and photo-realistic visual generation at high resolution. FLUX-VAI (Batifol et al. 2025) shows the state-of-the-art performance in both reconstruction quality and generalization ability across all continuous tokenizers. However, contin- uous tokenizer is criticized for its low compression rate, because latent features usually stored and calculated float 32 or bfloat16. 6. Therefore, discrete tokenizers that can store data in i nt bool to be more promising in terms of compression capabilities.
20

---

B.2 DISCRETE TOKENIZER
VQVAE (Van Den Oord et al. 2017) and VQGAN (Esser et al. (02I) employ vector-quantization (VQ) to transform visual input into discrete tokens. But they both suffer from low reconstruction quality caused by instability of the codebook utilization. To overcome these drawbacks, one line of work introduces specific optimization strategies modules improve performance (Lee al. (Si et al. 2024:220 Zhu et al. 2024; Yu et al. 2202cc) Another line of work focuses on mitigating the training instability when scaling up the codebook size by using grouped codebooks (Ma al., 2025 Jia et al. 2025 Zhang et al., 2025). These methods split the input feature into groups along the channel dimension, where each group then looked up using sub-codebook. However, VQ- based tokenizers still introduce additional inference and training costs due to the lookup operation Yu et al. 202 Lee et al., 2022b Fang al. 2025). MAGVIT-v2 (Yu et al. 2024a) introduces Lookup- Free Quantization (LFQ) to address this extra cost and proposes the entropy loss (Chang) ctt al. 2022; Jansen et al., 2019) to ensure the utilization of the codebook. However, the entropy loss causes unaffordable memory cost it scales linearly with the codebook, limiting the further expansion of the codebook. BSQ (Zhao et al.] 2024b) is proposed to mitigate this issue by assuming independence between the bits of the binary code, while this strong assumption leads to performance degradation. In contrast, WeTok does not rely on explicit codebooks, and eliminate the memory usage caused by entropy loss while having better performance than L.Q..
2022b
B.3 AUTOREGRESSIVE VISUAL GENERATION
The autoregressive (AR) modeling paradigm, which underpins modern Large Language Models (LLMs) Vassaani et al.
2017). has been successfully adapted for visual generation (Chen et al. 2020a). where models learn to predict sequences of discrete tokens for images (Ramesh et al. 222
Ding et al. 2021 Liu al.
2024) and videos (Hong et al. 2022 Kondratyuk et al.) 2023). Recent
AR models (Sun et al. 2024 Team 2024 Wu et al.) 2025a Wang et al. 2024b Liu et al., 2025) achieve remarkable image quality, highlighting the potential of this paradigm. Notably, the AR mod- els is critically dependent on the visual tokenizer. Therefore,
adopt WeTok to the AR (Sun) et al. 2024) framework to enable high-fidelity autoregressive generation. This shows that We'Tok is not only capable of compression, but its compressed features also suitable for generative models.
C ITERATION INVARIANCE OF WEТOK
common use case in practice involves compressing images for transmission and subsequently de- compressing them upon reception. However, this process is often iterative, where an image may undergo multiple cycles of compression and decompression (Joshi et al. 2000). While modern tok- enizers have proven effective for image compression and even show potential compression algorithms, an unexpected issue arises. As illustrated in Fig. IT and [2] serve that when state-of-the-art continuous tokenizers (Esser et al.] 2024b) Labs 2024) used for compression-decomprsstt ation. Accordingly, we proceeded to quantitatively analyze the cosine similarity between the latent representation of the original image and that of the image after undergoing multiple compression- decompression iterations. As illustrated in Fig. [3] we selected WeTok trained on general- domain data with a compression rate of 192 for our evaluation. The results demonstrate that after multiple iterations, the image processed by WeTTok gradually converges to a stable state. In contrast, the images processed by FLUX-VAB and SD-VAE 3.5 collapse.
next-generation
we ob-
iterations, the image quality progressively degrades with each iter-
D MORE RESULT
D. COMPARISON WITH STATE-OF-THE-ART
Visual Reconstruction. As shown in Fig.
Visual Generation. As shown in Fig. 15 and Tab. 6
21

---

Iteration 1
Iteration 25
Iteration 50
Iteration 100
SD-VAR
FLUX-VAE
WeTok (Ouns)
Figure 11: Qualitative comparison of image I on compresion-deecomprssio iteration.
Iteration
Iteration 25
Iteration 50
Iteration 100
SD.VAR
FLUX-VAE
WeTok (Ours)
Figure 12: Qualitative comparison of image 2 on compresion-decomppssiio iteration.
Imaage? Degrad
Hooh
FLUGANL
FLUSOAA
Figure 13: Quantitative comparison of image_ I and image 2 copprssionddecoppesiin iteration. After multiple iterations, the images processed by WeTok gradually converged to stable value, while the images processed by FLUX-VAE and SD-VAE 3.5 collapse.
22

---

Ground Truth
SD-VAE 1.5
SDXL-VAL VAE
WeTok (Ours)
Produced UIDSLEYY PARSONS HARRY ROMM
Geedond
Produced
FLODIUS POEDIUE AIMM EN00
LIAEELLY MATNAL BARRT BAMR
LINDLLEY PARSONS HARRY ROMM
SWING PARK
Directed by Phil Karlson
SWIIGG PAR
Osoand to OmN Deviron
SWING PARK
Dwouind by PlUl Rorlson
SWING PARKE
Directed Phil Karison
THE STORBES
STORBES
STORBES
THE STOREES
SOMNEE BUSWELL
GOONEE BOSWOLL
COMNEE BUSWELL
COMEE BISWELL
DODA
WIL PER
W00AA
WIL
URANA
201M
Figure 14: More qualitative comparison 512 x 512 image reconstruction on TonBench.
Aloakan Malamate
Founain
Space Shutie
Conil Reaf
Volemoo
Scaoeed
Maraw
Balkson
Figure 15: More WeTok-AR-XI, generated samples at 256 x 256 resolution.
23

---

Table Class-conditional generation 256 256 ImageNet. specifies the generated images
are 384 x 384 and are resized to 256x256 for evaluation.
Type
Model
#Para.
FID
IS1
Precision"
Recall
BigGAN (Brock et al. 2018
112M
6.95
224.5
0.89
0,38
GAN
GigaGAN (Kang al.
2023
569M
3.45
225.5
0.84
0.61
StyleGan-XIi (Suuer
2022
166M
2.30
265.1
0.78
0.53
ADM (hhriiall & Nichol 2021
554M
10.94
101.0
0.69
0.63
CDM
al. 2021
4.88
158.7
Diffusion
LDM-4 Rombach et al.
2022b
400M
3.60
247.7
DiT-XI/2 (Peehles & X
2022
675M
2.27
278.2
0.83
0.57
SiT-XT/2
Ma 2024
675M
2.06
277.50
0.83
0.59
MaskGIT CChng et al. 2022]
227M
6.18
182.1
0.80
0.51
Mask.
MaskGIT-re (Chag
2022
227M
4.02
355.6
VAR-d16
(Tan al. 2024
310M
3.30
274.4
0.84
0.51
VAR-d20
Tian ct al.
2024
600M
2.57
302.6
0.83
0,56
VAR
VAR-d24
Tian ct al.
2024
1.0B
2.09
312.9
0.82
0.59
VAR-d30
Tian
2024
2.0B
1.92
323.1
0.82
0.59
VQGAN Esser et al.
2020
227M
18.65
80.4
0.78
0.26
VQGAN EEser
2020
1.4B
15.78
74.3
VQGAN-re [Esser ct al. 2020
1.4B
5.20
280.3
ViT-VQGAN Yu ct al. 2021a
1.7B
4.17
175.1
VIT-VQGAN- re Yu al. 2021a
1.7B
3.04
227.4
RQTran. (Lee et al. 2022a
3.8B
7.55
134.0
RQTran,-re [Lee et al. 2202a
3.8B
3.80
323.7
LlamaGen-L" (Sun
2024
343M
3.07
256.06
0.83
0.52
LlamaGen-X0 (Sun al.
2024
775M
2.62
244.08
0.80
0.57
AR
LlamaGen-XXL (Sun cf
2024
1.4B
2.34
253.90
0.80
0.59
LlamaGen-L. Sun et al. 2024
343M
3.80
248.28
0.83
0.51
LlamaGen-XI. (Sun (
2024
775M
3.39
227.08
0.81
0.54
LlamaGen- Sun et al.
2024
1.4B
3.09
253.61
0.83
0.53
UniToo (Ma et al. 2025
1.4B
2.51
216.7
0.82
0.57
Open- Seen-MAGVIIZ-ALE
Luoo et al.
2024
343M
3.08
258.26
0.85
0.51
Open-MAGVIT22
(Lu al.
2024
804M
2.51
271.70
0.54
Open-MAGVVIT-ARR-2
(Tuo el al.
2014
1.5B
2.33
271.77
0.84
0.54
Wellok-AR-XL (Ours)
1.5B
2.31
276.55
0.84
0.55
MORE IMPLEMENTATION DETAILS
E.
ABLATION STUDY
Quantization method. As shown in Tab. and
Generative decoder. shown Tab. and
Number of group in GQ. As shown in Tab.
16 and
Model architecture, As shown in Tab.
20
Training data. As shown in Tab. 27 and 28
Learning rate schedule. As shown Tab.
29
Parameter of autoregressive model. As shown in Tab.
30
31 and
32
E.2 COMPARISON WITH STATE-OF-THE-ARY
Visual Reconstruction. The settings of in in-distribution comparison shown in Tab.
35
and
The settings of general-domain comparison are shown in Tab. 35
36
37 and 39
Visual Generation. As shown in Tab. 22
24

---

Table GQ training setting.
Table 8: LFQ training setting.
Table 9: BSQ training setting,
config
GQ
config
LFQ
config
BSQ
daia
IN-IK training
training data
IN-IK training (256, 256]
training data image
IN-IK training [256, 256]
image
(256, 256]
image
data augmentation
rundom crop
datu augmentation
rundom crop
data augmentation downsample
rundom crop 16
downsample
16
downsample
16
cma
True
cma
Truc
cma
True
fgroup number)
(group nomber)
fgroup number) of (group channel)
16
d' (group channel)
(group channel)
optimizer
Adam 0,50.5 0.9
optimizer
Adam Pr.
optimizer
Adum S1. 0,55 0.9
optimizer momentum
optimizer momentum
0250
optimizer momentum
weight decay
weight decay
weight decay
learning rate schedale
consistint le-4
learning rate schedulle leaming
consistent 1e-4
learning rate schedale leamingg
consistent le-4
leaming
warmup steps
warnup stops
warnup steps
cos decay end ratio
cos decay end ratio
1
cos decay end rañio
total steps
2502.50
steps channel mult
250250
total steps channel mult
2502.50
channel mult
[1.1,2.2.4]
[1.1.2.2.4]
[1.1,2.2.4]
channel
128
channel
128
channel
128
.lockks
mum_res,
generative decoder
False
generative decoder
False
generative decoder
False
GPU batchsize
16
per GPU batchsize
16
GPU batchsize
16
global batchsize
128
global barchsize
global batchsize
128
GPU number
H20
GPU number
8 H20
GPU number
H20
Table 10: Stage-1 training setting.
Table 11: Stage-2 training setting.
config
Stage-1
config
Stage-2
training data
general domain daraset |256, 256] random crop
training data image size data augmentation downsample
general domain dataset [256, 256] random crop 32 x 32 True
image size
data augmentation downsample
32 x 32 True 4
ema
ema g (group number) d' (group channel)
(group number)
d' (group channel)
8
optimizer
Adam
optimizer
Adam
optimizer momentum weight decay learning rate schedule
81, 01.52005,00. 0.9
optimizer momentum weight decay learning rate schedule learning rate warmup steps cos decay end ratio
81, S, 1,,5,,50500. 0.9
0 consistent le-4 0
consistent le-4
learning rate
warmup steps
0
cos decay end ratio
1
total steps
550550
total steps
550550
channel.mult
[1,1,2,2,4.8]
channel_muit
[1.1,2,2.4.8]
channel
256
channel num blocks
256
num_res.d blocks
4
.uu_res.bllock
4
generative decoder
False
generative decoder
True
GPU batchsize
6
per GPU batchsize
6
global batchsize
1056
global batchsize
1056
GPU number
176 H20
GPU number
176 H20
25

---

Table 12: 1 group training set- ting.
Table 13: group training set- ting.
Table 14: 2 group training set- ting.
config
group IN-TK training 256]
config
group
config
group
training datn
training datn
IN-TK training set
training data image
IN-IK training
image
image
(256,
256] random crop
data augmentation
random crop
dataa sugmentation
random crop
data augmentation dowusample
dowusample
16 16
downsample
16 True
16 16
ema
ema
(group numher)
(group numher)
(group number) d' (group channel)
d' (group channel)
(group channel)
optimizer
Adam Ar 8y=0.5, 0.0.0...... 0.9
optimizer
Adam P. 44=05,05..00 0.9
optimizer
Adam
optimizer momentum
optimizer momeatum
optimizer momentum
8 0,-05, 0.8 0.9
weight decay
weight learning rate schedule learning rate
weight decay
Jeamning rate schedale
consistent le-4
consistent le.4
learning rate schedale
consistent le-4
leaming rate
leaming rate
wammup steps
warmup stops
wammup steps
cos decay end ratio
cos decay end ratio
cos decay end radio
total steps
250250
total steps
250250
total steps
250250
[1,1,222.4]
[1,1,2,2,4]
[1,1,222.4]
channel
128
channel
128
channel
128
blocks
blocks
4
blocks
generative decoder
False
generative decoder
False
generative decoder
False
GPU batchsize
16
per GPU batchsize
16
GPU batchsize
16
glohal batchsize
128
glohal batchsize
128
glohal batchsize
128
GPU number
GPU number
1120
GPU number
Table 15: 8 group training set- ting.
Table 16: 16 group training Table 17: 32 group training setting.
setting.
config
8 group training (256, 256]
config training image size data migmentation
16 group IN-TK training set (256, 256]
config training image size data augmentation downsample
32 group maaningg
training
image size
[256, 256]
data augmentation
middam crop 16
middam crop 16 Thue
crop
downsample
dowusample
16 16 Thic
ema
Thue
ema y fgroup number) d' (group channel)
y tgroup number)
(group uumber)
a' (group channel)
(group channel)
optimizer
Adam P1. 51-05,
optimizer
Adam A P..Pp-0.5
optimizer
Adam 91,8,-03 0.9
optimizer momentum
optimizer momentum
optimizer momentum
weight decay
weight decay
weight decay
leaming schedule
consistent 16-4
leamingg rate schedule
consistent 16-4
leamning schedule
consistent 1c-4
Jeanning
learning, rate
leanning
warmup stops
0
warmup steps
0
warmup steps
decay ratio
decay ratio
total steps
250250
total steps
250250
total steps
250250
channel_me mmult
[1,1,222,4]
channel_mn
[1.1,2,2.4]
chamnel_mm
[1,1,2,2.4]
channel
channel
mumures blocks
numures blacks
mumures. blocks
generative decoder
False
generative decoder
False
generative decoodr
False
per GPU batchsize
batchsize
global batchsize
128
global batchsize
128
global batchsize
128
GPU number
8 $1200
GPU number
8 1120
GPU number
H120
26

---

Table 18: 128 channel 4 block training sefting.
Table 19: 192 channel 4 block training setting.
Table 20: 256 channel 4 block training setting.
condig
128 channel 4 block
config
192 channel block
condig
256 channel 4 block
training daia
IN-IK traning (256, 256]
traininng data
IN-IK traning (256, 256]
training daía
IN-IK traning (256, 256]
image
image size
image size data augmentation downsample
data augmentation
random crop 16 16 True
data augmentation downsample
random crop 16 x 16 Trme
random crop
downsamppe
16 16 True
a dgroup number)
g (groopp mumber)
a (group number) d' (group chanmel)
d' (group channel)
d' (group channel)
S Adam Av 025
optimieer
Adam Br. 0.5005
optimiier
Adam
optimizer momentum
optimizor momentum
optimizer momentum
Br. 02.55.005500
weight docay
weight docay
weight docay
learning rate schedule
consistent le-4
learning rate schedule
consistent lc-4
learning rate schedule
consistent le-4
learning rute
learning
learning rute
warmup steps
warmup steps cos decay end mito
warmup steps cos deeay end ratio
cos deeay end ratio
rotal steps
250250
total steps chammel_mult
250250
rotal steps
250250
Lmalt
[1.1.222,4]
[1.1.2.2.4
chaunel
[1.1,2,244
channel
128
channel
192
channel
256
num res blocks
-
num res. blocks
44
num res blocks
.f
gencrative decoder
Falsc
generative decendr
False
generative decoder
Falsc
per batchsizo
per GPU batchsize global batchsize GPU number
per GPU batchsize globall batchnizee GPU number
global batchsize GPU number
128
128
128
8 H20
16 H20
32 H20
Table 21: 128 channel 8 block training setting.
Table 22: 128 channel 16 block training setting.
Table 23: 192 channel 8 block training setting.
confg
128 channel & block
config
128 channel 16 block
confi
192 channel & black
training data image size
IN-IK training set [256, 256)
training data
IN-TK training [256, 256]
training data image size data augmentation downsamppe
IN-IK trainin set [256, 256) random crop 16 True
image
data augmentation downsample
random crop
duta sugmentation downsample
random crop 16 x 16 True
ema
True
ema mimber)
(group nimber)
(group number)
d' (group channel)
(group channel)
d' (group channel)
optimizer
Adam
optimizer
Adam
optimizer
Adam
optimizer momenturn
Pr./2-00.5 (.5.2.5..... 0.9
optimizer momentam
optimizer momenturn
P2.52-0.5.033 0.9
weight docay
weight decay
weight docay
leurning rate schedule
consistent le-4
learnnng scholule leaming rale
consistent Ic-4
leurning rate schedule
consistent le-4
learning rate
learning rate
warmup steps
0
warmup steps
warmup steps
0
decay end ratio
pos docay end miio
decay end ratio
rotal steps
250250
stops channel muult
250250
total steps
250250
channel malt
[1.1,222,4
[1,1,222.4]
channel maltt
(1.1.222,4)
chonnel
128
channel
128
channel
192
num.res blocks
num res blocks
16
numlres blocks
generative decoder
False $
decoder
False
generative decuder
False
GPU batchsizo
per GPU butchsize global batchaize GPU nunner
GPU batchsize
4
global batchsize
128
global batchsize GPU number
128
GPU number
H20
Table 24: 256 channel 2 block training setting.
Table 25: 384 channel 4 block training setting.
Table 26: 256 channel 8 block training setting.
config
256 channel 2 block
contig training data image size
384 channel block
config training data
256 channel & block
traiming data
IN- traning [256, 256]
IN-IK traniin [256, 256]
IN-IK traning [256, 256]
image
image
data augmentation downsample
random crop 16 10
data augmentation dow asample
random crop
data augmentation downsample
random crop 16 18 True
16 x 16 Trie
True
g (group nimber)
g Groupp nimber)
g (group number) a (group channel) optimiier
d' (group channel)
(group channel)
8
optimieer
Adam
optimiier
Adam
Adam
optimizer
Pr. P...2.0.5,00 0.9
optimizer momentum
Pr. 0.55
optimizer
Pr: P2.05 0.9
weight docay
weight docay
O
weight docay learning rate schodule learning rute warmup steps cas decay end ratio
lcarning rate schodule
consistent lc-4
learning rate schodule
consisient Ic-4
consisient lc-4
learning rate
learning
warmup steps
warmup sieps cas decay end ratio
cas decay end ratio
rotal steps
250250
total steps channel_malt
250250
rotal steps channel mallt channel num_res, blocks
250250
channel malt
(.2,244)
[1.1.222.4
[.2224))
channel
2356
channel
384
255
num_res, blocks
num jes, blocks
8
generative decoder
False
generative deceder
False
generative decoder
Falsc
per GPU hatchsize
per GPU hatchsize
per GPU hatchsize
global batchsize
128
global batchsize GPU number
128
global batchsize GPU number
128
GPU number
16 H20
64 H2U
64 H2U
27

---

Table 27: ImageNet training setting.
Table 28: General-domain training setting.
config
1.2M
config
400M
training data
IN-IK training set
training data image size data augmentation downsample
general domain dataset
image size
[256, 256] random crop 16 16
[256. 256] random crop
data augmentation downsample
16 x 16
True
ema (group number)
True
9 (group number) d' (group channel) optimizer
4
8 Adam
d' (group channel) optimizer
8
Adam
optimizer momentum
81 0.5, 0.9
optimizer momentum
31, 82 -.5, 0.9
weight decay
weight decay learning rate schedule
0 consistent lc-4
learning rate schedule
consistent le-4
learning rale
learning rate
warmup steps
0
warmup steps
0
cos decay end ratio
decay end ratio
1
total steps
550550
total steps channel
550550
channel_mult
[1,1.2,2,4]
[1,1,2,2,4]
channel
128
channel
128
num res blocks
num res blocks
generative decoder
False
generative decoder
False
per GPU batchsize
16
per GPU batchsize
16
global batchsize
global batchsize GPU number
128
GPU number
8 H20
8 H20
Table 29: Warm up cosine decay learning rate schedule training setting.
config
warm up cosine decay
training data
general domain dataset
image size
[256, 256] random crop 16 16
data augmentation
downsample
ema
Truc
(group number)
4
d' (group channel)
8
optimizer
Adam
optimizer momentum weight decay
31 0.5,
0
learning rate schedule learning rate warmup steps cos decay end ratio
warm up cosine decay
le-4
10000
0.01
total steps channel mult channel
550550
[1,1,2,2,4]
128
num_res_bloo blocks
4
generative decoder
False
per GPU batchsize
16
global batchsize
128
GPU number
8 H20
28

---

Table 30: LlamaGen Base training setting.
Table 31: LlamaGen Large
Table 32: LlamaGen X-Large training setting.
training setting.
config
Llanstiin Bese
coafig Training data sugmentation
LamaCion Large
config training data sugmentation downsimple
TamaCion X-1 a-Large
Training data
IN-TK triining
IN-IK triming sel [256,
IN-TK triining 256] candom crop False
256]
sugmenaation
candom crop 16
random crop
downsimple
downssmppl
ema
False
ema
Falsx
ema
optimizer
AdamW
optimizer
AdamW
optimizer
AdamW
optiiieer momentum
3,-11111 0.95
optimizer momentuum
81.92-0.9.
optimizer momentuum
9,md9,99 0:95
weight deeay
56-2
weiight deeay
Se-2
weight deeay
56-2
leaming schedule
leamning schedule
up lincar decay
leaming schedule
up
laming
loaming
3e-4
leaming
wammup spochs
wammup spoclts
warmup spochs
linear decay end maño
linear docay end radio
0.1
linear decay end rañio
total epochs
1000
total epochs
1000
total epochs
1000
dim
1024
dim
1280
dim
1536
num head
16
num head
20
num head
24
Jayers
Irans Jayers
36
Jayerss cond cim DacterizedLaayrr
48
cond dim
1280
15:30
Dayyrs
factorized..k
factorized.k
dlmp
token,dro drop residual drop
0.1
loken_dmpp
residual drop
0.1
residual drop
per GPU baschsine
64
per GPU baschsioe
32
per GPU baschsize
15
global batchsize
3072
glahal batchsize
3072
global batchsize
3072
GPU number
48 F120
GPU number
96 H20
GPU number
192 FE20
Table 33: Large scale training on Ima-
Table 34: Large scale training on Ima-
geNet training set at 16 downsample ratio.
get training set at 8 downsample ratio.
config
IN-IK 16x SOTA
config
IN-1K 8x SOTA
training data
IN-1K training set
training data
IN-IK training set
image size
[256, 256] random crop 16 16
image size
[256, 256] random crop 8
data augmentation downsample
data augmentation downsample
ema
True 4
ema g (group number) d' (group channel) optimizer
True 4
g (group number)
d' (group channel) optimizer
8
Adam
Adam
optimizer momentum weight decay learning rate schedule
Pr. 0.5, 0.9
optimizer momentum weight decay learning rate schedule
31 0.50 0.9
0 consistent
0 consistent le-4
learning rate
le-4
learning rate warmup steps cos decay end ratio
warmup steps
0
0
cos decay end ratio
1
total steps
400400
total steps channel mult channel
350350
channeL_mult
[1.1,2,2,4]
[1,2,2.4]
channel
256
256
num res blocks
4
num reo_klocks
4
generative decoder
False
generative decoder per GPU batchsize
False
per GPU batchsize
global batchsize
1024
global batchsize GPU number
1024
GPU number
128 H20
128 H20
29

---

Table 35: Large scale training Table 36: Large scale training on general-domain dataset at general-domain dataset at 768 compression ratio.
Table 37: Large scale training on gemeral-domain dataset at 192 compression ratio.
384 compression ratio.
suidi
768 compecainn SOTA
suufig traning data
compaooiin raio SOTA general domais danaet [356, 256] raralen cmp
sundg
192 compeesiin SOTA general domais datase: [256, 256] raraken cmp
traiiing data
general domais datase: [256, 256] raraken cmp
traiiing data
image tie
image sixs
image sine
engmostaiion
dats sugmentation downsumppe
angmonattin
downsmmple
downsmmple
ema
Thme
(group channel)
o (proep numbert
Egropp channel)
optiniser
optimiser
Adum
optinizer
Adan Pg. 11.00.550000 5,0.
optinteer momentum
Pg
Pg-.3-10.33 5,0.9
uptintenc mumentom
Pai 15..2-0..,50 0,5, 0
uptintiec momentum
weight dlaay
seighht doeay
wiight daaay
lassnng schalule
cxmoisicai
leasning nice scholule
comsisicnt Te-5
lessning schaiule
cxmoisicni 1es
leszing
1e-4
le-4
1e-5
leasning rice
le-4
Je-b
lessning
1e-4
le-b
warmup sops
warmin saps
warmup sups
on decsy and
me deesy and nitlo lotal steps channel muit channel
ros decsy and niii
und steps
550550
$50550
330330
550350
200200 50050
bood steps
470470
90090 10020
charnel muit
[1.1.222.4.5)
[1.12222444
channel muit
[1.1,22244
charnel
236
236
channel
256
min minuesNove Nocks
nm mom_jes.hog hicks
4 Falie
nim Jes Nocks
generative deorder
True
Tiue
generntive decrder
Falue
generative deorder
Filie
False
per butchsize
bunchsiae
per butchiize
glcbal batchilne
GPU batchilee
glcbal batzhiize
176 H20
128 H20
Table 38: Large scale training on general-
Table 39: Large scale training on general- domain dataset at 24 compression ratio.
domain dataset at 48 compression ratio.
config
48 compression ratio SOTA general domain dataset
config training data image
24 compression ratio SOTA
training data
general domain dataset
image size
[256, 256] random crop
[256, 256] random crop
data augmentation downsample
data augmentation
8 x True
downsample
8 x True
ema
ema
g (group number)
g (group number) a' (group channel)
d' (group channel)
8 Adam
optimizer
optimizer
Adam 91, f5-00,50000
optimizer momentum
B1, 3,55, 0.9
optimizer momentum
weight decay
0 consistent le-5
weight decay learning rate schedule
lcarning rate schedule
consistent lc-5
learning rute
1e-4
1e-6
learning rate warmup steps cos decay end ratio
lc-4
lc-5
warmup steps
cos decay end ratio
total steps
200200
200200
20020
total steps channel_mule
300300
50050
400400
channel_mule
[1,2,2,4]
[1.2.2.4]
channel
256
channel
256
num_res_hlowe Jhlckks
4
num blocks
4
generative decoder
False
generative decoder
False
per GPU batchsize
8
per GPU batchsize
global batchsize
1024
global batchsize
1024
1024
4096
GPU number
128 H20
GPU number
128 H20
128 H20
512 H20
30

---

F DECLARATION OF USE OF LARGE LANGUAGE MODELS (LLM)
We affirm that this paper was primarily written by the authors. Large Language Models (LLMs) were utilized solely general-purpose assistive tools for language refinement, grammar correction, and stylistic improvements during the writing process. Specifically, Gemini 2.5 Pro (DeepMind) 2025 was employed for minor text polishing and rephrasing enhance clarity and readability. No LLM was used for conceptual ideation, experimental design, data analysis, or generating any substantive content of the research.
31


Reviews:
[{"review_id": "0CFGnkvRa2", "reviewer": "Reviewer_YMro", "summary": "This paper introduces WeTok, a new family of discrete visual tokenizers that aims to resolve the long-standing trade-off between compression ratio and reconstruction fidelity in latent-space visual generation. WeTok combines two main innovations: Group-wise Lookup-Free Quantization (GQ) and Generative Decoder (GD). Extensive experiments on ImageNet-50k and MS-COCO show that WeTok achieves state-of-the-art (SOTA) reconstruction metrics and strong performance in zero-shot and class-conditional image generation.", "questions": "Please refer to the weakness section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed GQ formulation provides a mathematically grounded way to reduce the entropy-loss memory bottleneck in LFQ and BSQ, with a provably smaller approximation error.\n2. The paper includes large-scale ablations (quantization types, group numbers, architectures, learning schedules) and comparisons across both high-fidelity and high-compression regimes.\n3. The proposed method achieves strong performance on both image reconstruction and AR-based generation results, even surpassing continuous tokenizers at similar compression ratios.", "weaknesses": "1. Diffusion-based decoder for visual reconstruction has been studied in previous literatures[1][2], it would be better to cite these work and further discuss the differences with them.\n2. In the ablation study section, it's interesting to see that after converting the decoder to a generative model, the reconstructed images are more realistic. It would be better to include some further discussion or analysis.\n\n[1] Epsilon-VAE: Denoising as Visual Decoding\n[2] Diffusion Autoencoders are Scalable Image Tokenizers", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "U7GW0wBFGL", "reviewer": "Reviewer_bHBm", "summary": "This paper presents WeTok, a novel tokenizer that effectively balances high compression efficiency with high-fidelity image reconstruction, achieving state-of-the-art reconstruction performance.", "questions": "NA", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is well written.\n\nThe reconstruction results achieve SOTA performance among existing discrete tokenizer, demonstrating the effectiveness of the proposed framework.", "weaknesses": "## Fairness of Comparison\n\nThe comparison in Table 3 appears unfair.  The strong baseline MGVQ is a VQ-based tokenizer, whereas WeTok adopts LSQ, which has already been shown to be more efficient than VQ.  To ensure a fair evaluation, the authors should compare WeTok with an LSQ-based version of MGVQ. \nFurthermore, the MGVQ codebook size is only 8192 × 4, but its effective capacity is actually $2^{52}$, not limited by the nominal codebook size.\n\n## Lack of Novelty\nThe proposed method shows limited novelty.  Overall, the approach seems like a combination of existing components rather than a fundamentally new idea.  Specifically:\n- LSQ has been widely explored in prior works.  \n- Group-wise quantization is not new.  \n- The Generative Decoder design has already been introduced in previous studies.\n\nThe authors should better clarify what unique contribution or new insight WeTok introduces beyond these known elements.\n\n## Missing Discussion on Semantic Tokenizers\n\nThe paper lacks discussion and comparison with **semantic tokenizers**, which have proven to be powerful for visual understanding and generation.  Several recent works are relevant and should be considered:\n\n[1] ImageFolder: Autoregressive Image Generation with Folded Tokens. https://arxiv.org/pdf/2410.01756\n\n[2] Factorized Visual Tokenization and Generation. https://arxiv.org/pdf/2411.16681\n\n[3] DualToken: Towards Unifying Visual Understanding and Generation with Dual Visual Vocabularies. https://arxiv.org/pdf/2503.14324\n\n[4] TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation. https://arxiv.org/pdf/2412.03069\n\n## Missing High-Compression Evaluation\n\nThe authors claim that **WeTok** achieves a **768× compression ratio**, which is indeed a significant advantage. However, there is no **quantitative or qualitative evidence** showing the generation quality under this extreme compression rate. It is recommended that the authors provide **generation results** and **gFID metrics** at the claimed **768× compression level** to substantiate this important claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yG6V8COjsI", "reviewer": "Reviewer_Dztj", "summary": "The paper presents WeTok, which is a discrete tokenizer for improving the tradeoff between compression ratio and reconstruction fidelity. Discrete tokenizer has the benefit of having a higher compression ratio, but long has the problem of low reconstruction fidelity.  \n\nWeTok proposes two key techniques, Group-wise lookup-free Quantization (GQ) and Generative Decoder (GD). \n\nGQ tries to address the memory and computational cost of Lookup-free-quantization (LFP). It partitions the codebook into groups and performs quantization on each group independently to eliminate token entropy loss as the memory bottleneck.\n\nGD is a generative decoder, instead of the traditional one-step deterministic decoder used in prior methods. \n\nQualitative results in Table 3 and Table 4 are relatively strong comparing to the prior state-of-the-art methods", "questions": "- \"We surprisingly find that while reconstructions from leading models like FLUX-VAE and SD-VAE 3.5 collapse after iterations, WeTok’s\noutputs are remarkably robust and converge to a fixed value.\" Is there any explanation on why?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(+) The results presented in the tables are relatively strong, achieving a much larger codebook size and good rFID and PSNR\n\n(+) The proposed two methods (GQ and GD) make sense and are well motivated. GQ seems to be a practical and effective solution for solving the bottleneck of the CE loss\n\n(+) The evaluation compared against many methods in Tables 3 and 4", "weaknesses": "(-) the GD method is not very novel. It is known to the community such diffusion decoder can work, dating back to OpenAI's \"Consistency Decoder\". In addition, such generative decoder does not come with no cost. First, the decoding time increases, which can limit some of the real-time or latency-sensitive applications. Second, as it is a generative model, the decoder could also hallucinate \n\n(-) lack of comparison with more state-of-the-art autoencoders. For example, infinity tokenizer (https://cvpr.thecvf.com/virtual/2025/poster/34414). Both work claim to increase codebook size and claim to be SotA in the field, though taking pretty different approaches. Therefore a comparison is worthy for the community. However the paper primarily focuses on older VAEs such as VQGAN and SD-VAE\n\n(-) For generation tasks, such as the ones presented in Supp Mat Table 6, shows the proposed method is only marginally improving against SotA Open-MAGVIT2-AR-XL (Luo et al., 2024) 2.33 vs WeTok-AR-XL (Ours) 2.31 in FID\n\n(-) The model is trained in two stages: \"In first stage, we train our WeTok with the reconstruction loss, i.e., Eq. 2, 6 and 8. In the second stage, we adapt the model for generative tasks\". This is atypically and lacks of explanation on why this is needed. Traditional VAE are trained with a single stage and its generation task quality is on-par with the proposed method (see the point above)\n\n(-) seems datasets play a role in terms of the model quality (which is expected), as evidenced in Figure 6. However, in results section, all models are trained with different datasets, which complicates the analysis of whether the proposed method is effective or the 400M GD data is more suitable for Coco and ImageNet", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jI0jR6rHq2", "reviewer": "Reviewer_2k14", "summary": "The authors propose Group-wise Lookup-Free Quantization (GQ) to reduce computational and memory costs while maintaining high-fidelity image reconstruction. This design enables flexible scaling of the codebook to an effectively unlimited size. Furthermore, WeTok introduces a generative decoder that integrates adversarial training, using Gaussian noise as input and quantized tokens as conditional guidance, to mitigate the reconstruction loss typically caused by high compression ratios. Experimental results demonstrate that WeTok achieves superior reconstruction and generation performance even under extremely high compression settings.", "questions": "1. There is a minor concern regarding GQ. The approximation of the entropy loss is theoretically lower-bounded by BSQ, which performs worse than LFQ. However, Figure 4 in the ablation study shows a consistent improvement as the number of groups increases. It remains unclear where the performance begins to degrade — what is the optimal balance point for the number of groups?\n\n2. There are several existing 1D tokenizer models, such as TiTok [1] and SweetTok [2]. The authors are encouraged to include comparisons with these methods to strengthen the experimental validation and make the results more comprehensive.\n\n[1] An image is worth 32 tokens for reconstruction and generation. NIPS 2024\n\n[2] SweetTok: Semantic-Aware Spatial-Temporal Tokenizer for Compact Video Discretization. ICCV 2025", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The authors propose Group-wise Lookup-Free Quantization (GQ) to overcome the computational bottleneck, thereby achieving higher image reconstruction fidelity.\n\n2. The experiments yield promising results, and the ablation studies are comprehensive.", "weaknesses": "1. I am mainly concerned about the compression ratio. As shown in Table 4, when the compression ratio reaches 768×, the number of image tokens is only 8 × 8 = 64. However, the resulting rFID of 8.94 is considerably higher than that reported by TiTok [1], indicating a notable degradation in reconstruction quality under such high compression.\n\n2. The main contribution, Group-wise Quantization (GQ), appears to primarily alleviate computational overhead. However, the paper does not provide experiments demonstrating how this design directly facilitates higher compression ratios. For instance, conducting experiments with a downsampling ratio of 32×32 and a hidden channel size of 64 (8×8) could better illustrate how GQ contributes to improving compression efficiency.\n\n3. The Generative Decoder (GD) introduces Gaussian noise as the generator’s input, which may lead to training instability. To address this issue, the authors propose a two-stage training scheme; however, this approach appears overly complex and potentially difficult to reproduce. It would be helpful to report the performance when training all losses jointly from scratch, to evaluate whether the two-stage procedure is truly necessary.\n\n[1] An image is worth 32 tokens for reconstruction and generation. NIPS 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)