Switchable Token-Specific Codebook Quantization For Face Image Compression
Authors: Yongbo Wang
Haonan Wang
Guodong Mu
Ruixin Zhang
Chen Jiaqi
Jingyun Zhang
Jun Wang
Yuan Xie
zhizhong zhang
Shouhong Ding
Venue: NeurIPS 2025 poster ()
Keywords: image compression
image tokenization
Primary area: deep_learning

Abstract: With the ever-increasing volume of visual data, the efficient and lossless transmission, along with its subsequent interpretation and understanding, has become a critical bottleneck in modern information systems. The emerged codebook-based solution 
utilize a globally shared codebook to quantize and dequantize each token, controlling the bpp by adjusting the number of tokens or the codebook size. 
However, for facial images—which are rich in attributes—such global codebook strategies overlook both the category-specific correlations within images and the semantic differences among tokens, resulting in suboptimal performance, especially at low bpp. Motivated by these observations,  we propose a Switchable Token-Specific Codebook Quantization for face image compression, which learns distinct codebook groups for different image categories and assigns an independent codebook to each token. 
By recording the codebook group to which each token belongs with a small number of bits, our method can reduce the loss incurred when decreasing the size of each codebook group. This enables a larger total number of codebooks under a lower overall bpp, thereby enhancing the expressive capability and improving reconstruction performance. Owing to its generalizable design, our method can be integrated into any existing codebook-based representation learning approach and has demonstrated its effectiveness on face recognition datasets, achieving an average accuracy of 93.51\% for reconstructed images at 0.05 bpp.

Switchable Token-Specific Codebook Quantization For Face Image Compression
Yongbo Wang"
Haonan Wang"
East China Normal University Shanghai, China
Tencent Youtu Lab Shanghai, China
Guodong Mu
Ruixin Zhang
Jiaqi Chen
Tencent Youtu Lab Shanghai, China
Tencent Youtu Lab Shanghai, China
East China Normal University Shanghai, China
Jingyun Zhang Tencent WeChat Pay Lab33 Shenzhen, China
Jun Wang
Yuan Xie
Tencent WeChat Pay Lab33 Shenzhen, China
East China Normal University Shanghai, China
Zhizhong Zhang
Shouhong Ding
East China Normal University Shanghai, China
Tencent Youtu Lab Shanghai, China
(51166990110055527590113333 stu.eesssseecoooo ww..eeduduccc. (yxie,zzzhang @es..enn.eeeee ecn..edu..n gugiiahnwaanggeooommumuuiiniaan
J teencent.com
Abstract
With the ever-increasing volume of visual data, the efficient and lossless transmis- sion, along with its subsequent interpretation and understanding, has become a critical bottleneck in modern information systems. The emerged codebook-bas based solution utilize globally shared codebook to quantize and dequantize each to- ken, controlling the hpp by adjusting the number of tokens or the codebook size. However, for facial images which are rich in attributes such global codebook strategies overlook both the category-specifer correlations within images and the semantic differences among tokens, resulting in suboptimal performance, espe- cially low bpp. Motivated by these observations, we propose Switchable Token-Specific Codebook Quantization for face image compression. which learns distinct codebook groups for different image categories and assigns an independent codebook to each token. By recording the codebook group which each token belongs with a small number bits, our method can reduce the loss incurred when decreasing the size of each codebook group. This enables larger total number of codebooks under a lower overall bpp, thereby enhancing the expressive capability and improving reconstruction performance. Owing to its generalizable design, representation learn-
method can be integrated into any existing codebook- ing approach and has demonstrated its effectiveness face recognition datasets, achieving an average accuracy of 93.51 % for reconstructed images at 0.05 hpp.
"Equal contribution. This work done by Yongbo Wang during internship Tencent Youtu Lab. "Corresoonding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

TSC #1
TSC #1
7
TSC #M
[log2N| || x T bits
[log2 M| + [log2K] x T bits (b)
(a)
Figure 1: Storage cost comparison between previous latent space models and our method. (a)
Previous latent space model: Global-shared codebook requiring storage cost of [og, N] bits. (h) Our method: Token-specific codebook selection reduces storage to 7 floog2 K'] + [log2 M] bits.
Introduction
The volume of image data produced daily by smart device has skyrocketed. However, due to bandwidth limitations and storage costs, such data are typically stored and transferred in lossy compressed formats instead of raw RGB data. While lossy compression can greatly reduce storage requirements, it inevitably leads to a drop in visual quality and significantly impairs the performance of certain machine perception tasks, such as face recognition (1.).
In recent years, numerous solutions have been proposed with the aim of achieving high-fidelity image reconstruction and maintaining recognition capabilities with extremely low bpp. For instance, traditional compression techniques such as JPEG (3) GIF (4) and WebP (5) remain widely adopted due their strong compatibility to trade off computational complexity, storage size, and visual quality. Mcanwhile, with ad vannemeenss in deep learning, neural network-bae
compression methods have gained increasing traction. VQ-VAE (6) transforms images into discrete indices mapped codebook, enabling the compression of 2D pixel spaces into compact latent spaces. Despite their expressive feature representation, these methods are constrained by the requirement maintain 2D structural correspondence, which prevents them from fully leveraging spatial redundancy in images and limits further reduction of bpp. To address this limitation, TiTok (7) introduced latent code framework that encodes images into a ID latent space, achieving high-efficiency compression of 256 x 256 images with only 32 tokens.
However, when targeting even lower bpp, these methods primarily rely on reducing the number of tokens or decreasing the size of the codebook. Unfortunately, both strategies result in severe degradation of visual quality for compressed images and significant drops in machine recognition performance. Through detailed analysis, we identify a pivotal bottleneck in existing VQ-VAE -tyytlle methods: all tokens share a single global codebook. To ensure that the codebook accommodates the diverse features of all images, must be sufficiently large. Consequently. reducing the size of the codebook leads to drastic performance decline. This raises important question: Can we reorganize the codebook simplify the problem into smaller, more manageable subproblems?
Taking face images as an example, variations in attributes such as gender, age, and ethnicity suggest that images with similar attributes often share similar features. Therefore, the global shared codebook can be replaced with multiple codebooks, each designed for images with specific attributes. By enabling images selectively use a suitable codebook. the complexity of each codebook's 's task can be reduced. Furthermore, within single image, regardless of the feature extraction architecture used (i.e., CNN or ViT), different tokens explicitly or implicitly represent semantic information pertaining to different aspects of the image. For example, some tokens may correspond to facial regions, while others may be associated with the image's category. Forcing all tokens to share same codebook increases learning difficulty. To this end, we propose token-specific codebook quantization mechanism, where each token assigned its own unique sub-codebook, significantly reducing the capacity requirements of individual sub-codebooks.

---

Based on the above analysis, we propose a switchable token-speciffs codebook quantization mech- anism that combines image-level and token-level segmentation and has been verified on multiple face datasets. In our method, codebook routing module determines which codebook within the codehook pool is appropriate for a given image. Within the selected codebook, each token is assigned a sub-codebook tailored for its specific characteristics. In this way, the entire codebook pool can offer greater capacity, enabling improved compression and reconstruction performance. Additionally, as illustrated in Figure D due to our hierarchically dynamic structure, the actual storage overhead decreases from T x [logg N to T x [log2 K| + [log2 M 1, enabling stronger feature representations under the same or even lower bpp. This ultimately results in substantial performance improvements.
We summarize our main contributions follows:
1. We first introduce a switchable codebook quantization mechanism. By adjusting the bit width of the routing module and the size of the codebooks, our method supports flexible bpp configurations and increases total codebook capacity under the approximation of bpp. thereby enhancing overall performance.
2. We analyze intra-image token characteristics and propose a token-specific codebook quantization mechanism, thereby reducing the complexity of each codebook and improving overall performance.
We propose a hierarchically dynamic codebook structure that incorporates both image-level and token-level codebook partitioning. This module is blug-and-play and can be seamlessly integrated with ofate-of-the-art codebook-basee compression methods.
4. We evaluate our method on the face recognition task and demonstrate its effectiveness with extensive experiments. Compared to the state-of-the-art method (TTTok), our approach achieves higher accuracy at the same bpp (e.g., improving accuracy from 87.56% to 91.66% at 0.0234 bpp) or reduces bpp at the same accuracy (e.g., from 0.0234 0.0157 87% accuracy).
2
Related Works
2.1 Lossy Image Compression
Traditional lossy image compression frameworks typically employ manually crafted pipelines, as exemplified by standards such as JPEG (3), JPEG2000 (8), HEVC (9). and VVC (10). However, isolated module optimizations prevent partial improvements from translating into global performance gains, inherently limiting the evolvability of such frameworks. Building upon advances in neu- ral networks, Ballé et al. ( pioneered convolutional neural networks (CNN)-based end-to-end optimized nonlinear transform coding framework, in which the analysis/synthesis transforms and entropy models are jointly trained to outperform traditional codecs in rate-distortion performance, and further extended this approach using a variational autoencoder formulation (12). Subsequent studies have advanced neural image compression by exploring improvements in network archi- tectures (13 14 15), quantization methods (16) 17), entropy modeling techniques (18 19), and
optimization objectives (20 21).
Beyond neural image compression, Agustsson et al. (22) introduced generative image compression to address blurred reconstructions at low bitrates inherent in prior methods, leveraging perceptual loss optimization for realistic synthesis. While early generative image compression frameworks primarily employed generative adversarial networks (GANs)
(22 23 24), recent work has explored text/sketch-guidee diffusion models (25), non-binary discriminator with quantized conditioning (26), and VQ- VAE-based latent-space transform coding (27). achieving high-fidelity and high-realism reconstructions under ultra-low bitrate constraints (28 29). In addition, some specialized image compression frameworks have emerged for domain-specific tasks. For facial image compression, studies (30, 31 investigate racial bias induced by lossy compression in face recognition. Others develop frameworks tailored for facial images by utilizing edge maps (32) or semantic priors (33). However, these approaches either struggle to achieve effective compression under ultra-low bitrate constraints, or overlook the critical role of identity information facial recognition tasks, resulting in insufficient exploration of recognition accuracy and identity consistency.
3

---

Latent Encoder
Latent Decoder
DKSS
2
Nearest Neighbor
STSC
Selection
Image-level
(a)
Token-level
Token-level
TSC #1
14
Router
TSC #4
TSC #M
(b)
(c)
(d)
Figure 2: (a) Overview of the proposed architecture. (b) Dynamic switching mechanism for token- specific codebook selection. (c) Composition of the i-th token-specific codebook. (d) Token-specific quantization and dequantization process for the j-th token. Sample face images are from the FFHQ dataset (4).
2.2 Latent Space Model
Latent space model, initially developed for visual generation tasks, compresses high-dimensional raw pixels into compact latent representations for image synthesis. While variational autoencoders (VABs) (34) map images into a continuous latent space, vector-quantized VAEs (VQ-VAES (6) 35 learn discrete latent representations via codebook learning, offering enhanced controllability and compression capability. Extending this framework, VQGAN (36) integrates perceptual loss and adversarial training to maintain high perceptual quality elevated compression rates, further bridging generative modeling with discrete latent space compression. Recent advances have demonstrated the potential of latent space model: Rombach et al. (37) implement high-resolution image synthesis by performing diffusion processes in the latent space of VQGAN; Yu et al. (7) break away from conventional 2D latent grids by learning ID token sequences for more flexible latent representations; and Shi et al. (38) pioneer scalable training paradigms to enable large-scale high-dimensional codebooks, significantly improving the utilization of large-scale codebooks.
Latent space modeling has also shown significant promise facial processing tasks. Wang et al. (39)achieve high-fidelity and generalizable talking face generation by leveraging pre-trained codebook to encode target faces. Tan et al. (40) further advance this domain by designing a unified codebook capable of representing diverse identities and expressions. Similarly, works (41 42 43
employ VQGAN for blind face restoration, using codebooks pretrained on high-quality facial images priors to guide degraded image reconstruction. However, prior research predominantly focuses on spatial token compression within fixed codebook frameworks, where further reduction of token counts has reached diminishing returns. Our work pioneers codebook-centric methodology by proposing a universal switchable token-specific codebook quantization. This innovation enhances compression efficiency through dynamic codebook specialization while maintaining reconstruction fidelity.
3 Method
3.1
Preliminary
The key idea of the latent space model lies in learning discrete latent representations to establish compressed semantic space for image compression and generation. typical latent space model comprises three fundamental components: encoder Enc, vector quantizer Quant, and a decoder

---

Dec. The encoder hierarchically compresses high-resolution images into compact latent vectors. The quantizer, rather important in compression, maintains an embedding codebook C E R gxd with N learnable latent vectors that defines a discrete projection space. Through vector quantization, the continuous latent vectors produced by the encoder are discretized into codebook indices, thereby converting the image into a sequence of symbolic tokens. Specifically, given an input image x: x
XW x3, the corresponding discrete feature map can be formulated as follows:
2, = Enc(2),
(1)
2q = Quant(z.) = arg min IZ:- - -el
(2)
eEC
where Ze, Rg E R.xxxxd The quantizer vias substituting continuous latent vectors with their nearest neighbors in learnable codebook, thereby reformulating the input image as a compressed index sequence. This transformation achieves significant storage efficiency while preserving critical visual fidelity. For example, for the codebook C E R RNxd xd, each token requires only [log2 N bits. Thus, bpp is determined by both the codebook size and the token count.
The decoder operates on these discrete latent embeddings to reconstruct the input image i = (2z).. The training objective of the latent space model harmonizes three critical aspects: reconstructing error minimization, quantization error minimization, and perceptual quality preservation. The loss function can be formulated as:
LvQ
|| 212 ([sg(ze) 2g 12 |1)((2)) zell2 + Xp
where Eper indicates the perceptual loss, and sg(-) refers to the stop- hoopggrdieett operation. As illustrated in Figure 2 our method adheres to the latent space model but introduces a critical innovation: replacing conventional static codebooks with switchable token-specific codebooks, achieving superior rate-distortion performance compared to prior latent space models.
3.2 Switchable Codebook Quantization
The above analysis illustrates that the bpp is influenced by both the number of tokens and the bit- of token indices. Since the number of tokens corresponds to the model architecture, a viable strategy is to reduce the bit-width of token indices. The bit-width of token indices L is determined by the codebook size N, following the relation = [log, N However, simply decreasing the codebook size will negatively impact the reconstruction performance by limiting the diversity of codes available. Moreover, even halving the codebook size only reduces each index by one bit, which significantly diminishes the representational capacity of the latent space.
The inherent variations in facial attributes (e.g., gender, age, ethnicity) suggest that samples sharing common attributes exhibit analogous feature distributions. To mitigate the diminished codebook diversity caused by codebook compression, we propose Switchable Codebook Quantization (SCQ). Given an original codebook Corig Xd we replace it with M learnable codebooks {C E R 1
fi=1, where s S ... This design ensures that code diversity remains comparable to or exceeds that of Corig while allowing storage compression.
Specifically, the original latent space model requires x b bits (for n tokens with b-bit width), whereas SCQ reduces per-token bit-width by and introduces only log, M additional bits to switch codebook.
The multiplicative reduction bit-width (x n(b s)) dominates the additive overhead (+ log, M). For instance, when an image is represented by 256 tokens, replacing the original 4096-entr codebook with 256 codebooks (each containing 256 entries) reduces total bit allocation from 3072 bits to 2056 bits, achieving a 33% reduction in hpp.
During quantization, shown in Figure each image selects codebook via router G that maps encoded features ze to their optimal codebook partition. The selected codebook then quantizes z, into discrete indices through the nearest-neighbor search, preserving constrained bpp.
k =G(z).
(4)
2g Quant 2.) arg min ||z - ee12-
(5)
ecci
5

---

3.3 Codebook Routing
Increasing the number of codebooks introduces corresponding selection challenge, as becomes necessary to determine which codebook is most appropriate for given input context. To minimize quantization error, the most straightforward routing strategy to compute quantization errors across all codebooks and select the one with minimal error:
arg min ||z. Quant(z.)153 (E)
(6)
Ci
However, direct error-based routing may lead preferential optimization of better-optimized code- books, thereby compromising codebook diversity. Inspired by Mixture-of-Expert (45) 46) (MoE), we design differentiable routing network Ga composed of probabilistic sub-routers:
Go(ze) = arg max go (ze).
iE (E(......M
where d computes the selection probability for the i-th codebook. To ensure full codebook utilization, we introduce three loss functions:
M
Cent
Σ
To Ze log To (ze).
(8)
Ldxe
1
Σ gh (ze) log o (ze);
(9)
M
Lqua
Σ
gh (z+) sg(11z, Quant(z.))l a).
(10)
where (2e) denotes the average probability of the current codebook group across samples in the batch, and 1 Li=1 ||z Quant (z.) )13 feot maximizes the entropy of selection distribution to enforce balanced utilization across all codebooks, preventing preferential collapse to dominant codebooks. This promotes full parameter space exploration during training. Las reduces prediction ambiguity by concentrating probability mass on the optimal codebook index. Lqu guides the router towards codebooks producing below-average reconstruction errors. The composite loss function becomes:
Lyouter Lquu + fent Sace
(11)
Although the learnable router Go provides adaptive codebook selection during training, its stochastic nature cannot guarantee persistent global optimality in routing decisions. This limitation stems from the exploration-exxloitttion dilemma inherent entropy-regularized optimization. Consequently, we only employ Gg for training. For inference, we only employ Gnaive ensure quantization fidelity through guaranteed minimal-error codebook assignment.
3.4 Token-Specific Codebook Quantization
Previous vector quantization methods employ global codebook Cyiobal E k xd where all tokens share the same quantization space. However, within individual facial images, local regions (c.g.. ocular vs. nasal areas) exhibit significant feature-space divergence due to comain-specific texture and geometric aariations.Thereffo leading to incomplete utilization of the full codebook. Besides, token features may partially overlap, perfect alignment rarely occurs. As shown Figure ([c), we propose decomposing the global codebook into token-specific sub-codebooks:
individual tokens cannot effectively span the entire feature space,
Css [C1 +
... C. e Txxxd
(12)
where denotes the number of tokens, each sub-codebook C, E Kx independently learns the distribution of the t-th token. The token-specific codebook quantization becomes:

---

OASI
TSDQ
Latent Decoder
SC
STSC
STSC
initialization
(b)
Figure 3: Illustration of the training strategy. (a) Training stage 1: Optimization of the switchable sokend codebook. (b) Training stage 2: Optimization of switchable token Seeensspecific codebooks, initialized from the Stage token-shared codebook. (c) Training stage 3: Exclusive latent decoder optimization with frozen switchable token-specifie codebooks. Sample face images are from the FFHQ dataset (44).
2/ Quant 21)) arg min || - el|2. C.
(13)
eEC,
Despite increased total codebook size (T x K vs K), per-token bit-width remains b = [log, K identical previous methods. Besides, by dedicating individualized sub-codebooks to model token- specific feature distributions, our method achieves higher sampling density within each token's characteristic subspace, which directly translates to improved reconstruction fidelity.
3.5 Training Strategy
Building upon our foundational innovations in Switchable Codebook and Token- Token-Speciin Specific Codebook, we synergistically combine these components to formulate STSCQ - novel architecture that leverages multiple token- specifcc codebooks to simultaneously reduce per-token index bit-width and mitigate quantization errors. To maximize synergistic advantages, shown Figure we introduce three-stage progressive training paradigm.
Firstly, initialize model with pre-traine pre-trained latent space model and implement Switchable Codebook Quantization - replacing the original single codebook with multiple learnable codebooks. After routing, all tokens within image share unified codebook for quantization. During this stage, we freeze all parameters except codebooks and routing network, focusing optimization on:
Ssugel
||zee Quant(z.))13 + Lrouter where = Go (Ze)-
(14)
e
The first stage crucially establishes codebook diversity and routing policy initialization for subsequent token-specific adaptation. Building upon this foundation, the second stage implements progressive codebook refinement by leveraging the pre-trained codebooks as initialization vectors for token- specific codebooks. Following the same parameter freezing protocol as Stage 1, we exclusively update token- specific codebooks and routing network. The training objective during codebook refinement is:
T
Ssaage2
Σ ||z QQuan(zz))13 Crouter, where i Gg(Ze).
(15)
C
As the codebook's s feature space evolves through training iterations, the pre-trained decoder becomes suboptimally aligned with the updated codebook representations. To maintain precise latent-to to-xelll space mapping, we perform decoder fine-tuning on the original training dataset, ensuring accurate image reconstruction from the transformed latent features. Furthermore, given the critical requirement for high-fidelity preservation of facial attributes compression systems, we integrate an identity conservation mechanism during decoder refinement. Specifically, we leverage the widely-adopted AreFace (47) loss to impose semantic consistency berween original and reconstructed faces. The decoder can be supervised by the image-level loss:

---

Liugee =110 - 8112 + Xphrer ds Stuce
(16)
where Eper denotes perception loss, Liace denotes face loss.
4 Experiments
4.1 Setups
Dataset. We train our models on the CASIA-WebFasd dataset (88), a large-scale face image dataset widely used in the field of face recognition research, which contains approximately 500K images of 10,575 individuals, collected from the Internet. In the test stage, we evaluate the reconstruction quality of our method on five face recognition datasets: LFW (2) CFP-FP (49). AgeDB (50), CPLFW (5I), CALFW (52). They all have 6-7K pairs of images that are used determine whether they belong to the same person. Note that all images for training and evaluating are resized to 256 x 256, and data augmentation strategies such as random cropping and random flipping are applied during training.
Training Details. We adopt our Switchable -eeecf-S Codebook Quantization on both CNN- based and ViT-based VQ-tokenizers. In our training pipeline, the encoder remains fixed throughout the process. During stage 1 and stage 2, only the codebook is learnable, with its initial size set to 4096. In stage 3, only the decoder is trained adapt the quantized representations produced by the updated codebook. For the training dataset CASIA -WeeFeee, we train 100K steps for stage 1, 400K steps for stage 2, and 100K steps for stage 3. Our models are optimized by Adam W with the initial learning rate of le 4. Our methods are implemented on eight NVIDIA V100 GPUs with nearly 2 days for training.
Evaluation Metrics. We evaluate the level of image compression using bits per pixel (bpp), and assess the impact of compression on facial images using a pre-trained face recognition model. Specifically, we compute the Mean Accuracy (MeanAcc) and the Identity Similarity (IDS). MeanAcc refers the average recognition accuracy across five face recognition benchmark datasets after compression and reconstruction, while IDS measures the cosine similarity between the features of the original and reconstructed images.
4.2 Main Results
We evaluate our proposed method on two representative baselines, TYTok 7 and VQGAN (36). For TYTok, we conduct experiments under two different scales, where each image is represented by either 128 or 32 discrete indices. For VQGAN, we follow the experimental setup of MASKGIT (53). We conduct comparisons with variety of state-of-the-art methods, encompassing both traditional com- pression algorithms (e.g.. JPEG2000 ( and codebook-bard learning approaches (e.g., MaskGIT and TiTok), as shown in Table I comparison with other methods, our proposed approach pre- serves outstanding recognition effectiveness for compressed facial images, maintaining recognition accuracy of around 70% even at extremely low bit rates (bpp 0.01). Specifically, compared with traditional compression algorithms, our approach achieves higher recognition accuracy and IDS at approximately half the bit rate. However, at compression rate of 0.01 hpp. JPEG2000 exhibits considerable limitations, as its capacity to retain essential facial details for reliable recognition is substantially reduced. For codebook learning-baat standing performance. By learning token-specific codebooks for each token, we significantly enhance the representational capacity of the latent space. For example, on MaskGi our method achieves a recognition accuracy of 93.51% and the IDS of 0.6659, while on TiTok-s, obtain the accuracy of 91.66% and the IDS of 0.6120. These results represent a substantial improvement over the baseline methods.
methods, our approach also demonstrates out-
4.3 Ablation Studies
Generalization Ability Our Method. With the help of proposed Switchable - Codebook Quantization, we can flexibly adjust the compression rate by reducing the size of codebooks while maximizing the retention of its latent space encoding capacity. To show generalizable design of our methods, we conduct generalization experiments on three baseline models, each evaluated
8

---

Table 1: Quantitative comparison with state-of-the-ard methods.
Method
Model Type
# Tokens
MeanAcc(%)
IDS
bpp
JPEG 2000
56.98
0.0312
0.0100
JPEG 2000
8
/
85.64
0.3551
0,0500
CodeFormer
(41
2D
256
89.99
0.6210
0.0390
MaskGit- VQGAN 53
2D
256
90.70
0.6314
0.0469
TiTok-S
ID
128
87.56
0.5764
0.0234
TīTok-L. 7
ID
32
65.07
0.1812
0.0059
Oass(aasGGit-VQGANN
2D
256
93.51 (42.81f)
0.6659
0.0469
Ours(TTTok-S)
ID
128
91.66 (+4.10f)
0.6120
0.0234
(Trs(TYook-L)
ID
32
73.13 (+8.061)
0.2583
0.0059
Accuracy vs BPP
IDS vs BPP
95
90
85
0.5
Accuracy (%)
80
0.4
IDS
75
$128 Baseline
70
L32 Baseline
JPEG
0.2
Durs
60
Ours-L32
0.1
Ours: MaskGit
0.01
0.02
0.03
0.04
0.05
0.01
0.02
0.03
0.04
0.05
BPP
BPP
Figure 4: Comparisons of different baselines and our methods.
under three different hpp configurations. The results shown in Table 2 and Figure Our method demonstrates superior rate-distortion performance compared to conventional baselines across both 1D and 2D latent-space modeling frameworks. For 1D modeling (TiTok-S), 32.9% (from 0.0234 - 0.0157 bpp) while maintaining comparable recognition accuracy (87.56% 87.60%). In 2D latent-space architectures (MaskGit-VQGAND our approach achieves 16.6% bitrate reduction (0.0469 - 0.0391 bpp) coupled with a 1.53% absolute accuracy improvement (90.70% 92. 23%). validating that codebook specialization simultaneously enhances compression efficiency and feature representation fidelity.
reduce the bitrate by
To better evaluate the efficiency of our method, evaluate the inference latency and storage overhead introduced by our method, as shown Table 2 The expansion of the overall codebook size does indeed incur additional storage costs, along with slight increase in inference latency. However, explore a routing- based inference: only the codebook group selected by the router needs to be loaded, rather than searching with the minimum error across all codebooks. With this improvement, both inference latency and storage overhead are substantially alleviated, while the performance remains competitive with the baselines.
Effectiveness of Switchable Codebook Quantization. We conduct ablation studies about our proposed method with the codebook size of 1024 and show results in Table 3 where Idx0 indicates the original single codebook without any modifications, and NN means the nearest-neighhor search. The results for Idx0 and Idx1 indicate that employing routing mechanism to select among multiple codebooks can further enhance the representational capacity of the codebooks under fixed bpp, as evidenced by improvement in recognition accuracy from 88.11% to 88.24%.
Effectiveness of Codebook Routing and Token-Specific Codebook.
shown Table the results for Idx2 and Idx4 suggest that the presence of multiple codebooks, together with the routing mechanism, can further alleviate the learning difficulty in the feature quantization process and improve model performance. In addition, the comparison between Idx1 I and Idx4 demonstrates that

---

Table 2: Switchable Seken-Specific Codebook Quantization different baselines. #Tks and # indicates the number of tokens and codebooks respectively. Nes refers to the size of each codebook. 'SP" means selection policy, where 'NN' means Nearest-Neighhor search, 'CR' means proposed
Codebook Routing search. linf indicates the inference time per image tested on V100.
Backbone
#Tke
#Cbs
Nca
hpp
SP
McanAce(%)
IDS
fuu (s)
Storage (MB)
256
4096
0.0469
NN
93.51
0.6659
0.1554
1122.70
MaskGit- VQGAAN X
256
16
1024
0.0391
NN
92.23
0.6264
0.1771
4194,70
256
16
1024
0.0391
CR
92.22
0.6253
0.1544
354.70
128
4096
0.0234
91.66
0.6120
0.1437
122.68
T
128
256
256
0.0157
0.5180
482.71
128
256
256
0.0157
87.54
0.5125
0.1450
100.21
32
4096
0.0059
NN
73.13
0.2583
0.1744
1163.47
TTTok-L. 7
32
256
256
0.0040
NN
66.02
0.1885
0,1750
1253.50
32
256
256
0.0040
CR
65.65
0.1864
0.1741
1157.87
Table 3: Ablation studies.
Idx
Tok-shared
Tok-specific
NN
CR
McanAco(%%
IDS
bpp
0
88.11
0.5361
0.0195
1
v
88.24
0.5412
0.0196
2
/
89.28
0.5701
0.0196
3
89.89
0.5740
0.0196
learning specific codebook for each token enhances the representational capacity of each token in the latent space, as evidenced by improvements in recognition accuracy. Furthermore, token-specific codebook quantization able solve the uneven distribution of codebook utilization due to the original strategy of using a global-shared codebook across all tokens. As shown in Table 4 the proposed approach enables more effective utilization of the codebook, with an average increase of approximately 20% in per-token codebook usage, thereby reducing quantization errors caused by codebook utilization imbalance.
Table 4: Codebook utilization rates (%) per token LFW dataset.
Method
bpp
Min
Max
Mean
STD
Global-shared
0.0234
3.49
78.12
54.17
14.71
Ours 0.0234
17.90
83.89
74.02
9.14
Conclusion and Limitation
In this paper, propose switchable token-specific codebook quantization mechanism. Specifically, we design a codebook routing algorithm that assigns each image to its own small codebook, and further allocate an independent codebook to each token within the image. Our approach supports flexible bpp (bits- (bits-per- cess-per-pae pir--ieelrpl) settings and enables the codebook to better exploit its representational capacity under the same bpp configuration. We validate the effectiveness. of our method face recognition tasks, demonstrating that facial images can maintain competitive recognition accuracy when compressed to extremely low hpp.
However, our method still has certain limitations. Since our approach focuses on flexible configuration of codebook size without introducing special designs for the encoder decoder, its performance is highly dependent on the underlying autoencoder used as baseline. We leave these extensions for future work.
10

---

Acknowledgements. This work is supported by the National Natural Science Foundation of China No. 62476090, 62302167, U23A20343, 62222602, 62176092, 72192821; Shanghai Sailing Program 23YF14 10500; Young Elite Scientists Sponsorship Program by CAST YESS20240780: Natural Science Foundation of Chongqing CSTI2023NSCQU0X0000, CSSTE223SSCC--MSX0113 CCF- Tencent RAGR20240122 the Open Research Fund of Key Laboratory of Advanced Theory and Application Statistics and Data Science-MOE, ECNU.
References
[1] W. Zhao, R. Chellappa, J. Phillips, and A. Rosenfeld, "Face recognition: A literature survey," ACM computing surveys (CSUR), vol. 35, no. pp. 399-458, 2003.
[2] G. Huang. M. Mattar, T. Berg, and E. Learned-Miller, "Labeled faces the wild: A database forstudying face recognition in unconstrained environments," in Workshop on faces in Real-Life Images: detection, alignment, and recognition, 2008
[3] G. K. Wallace, "The jpeg still picture compression standard," Communications the ACM, vol. 34, no. pp. 1.300-4 44, 1991.
[4] "Graphics interchange format," http:///wwwwww.ooooooooo/oootana 2020.
[5] "Webp." http:///weeeeeeeeeeeee////////ee studdeeepppppdyyyyeeeee 2024.
[6] A. Van Den Oord, O. Vinyals er al., "Neural discrete representation leamiin." Advances in neural information processing systems, vol. 30, 2017.
[7] Q. Yu, M. Weber, X. Deng, X. Shen, D. Cremers, and L.-C. Chen, "An image is worth 32 tokens for reconstruction and generation," Advances in Neural Information Processing Systems, vol. 37, pp. 128 940- 128 966, 2024.
[8] D. Lee, "Jpeg 2000: Retrospoctive and new developments," Proceedings of the IEEE, vol. no. pp. 32-41, 2005.
[9] G. J. Sullivan, Ohm, W.-J Han, and T. Wiegand, "Overview of the high efficiency video coding (heve) standard," IEEE Transactions on circuits and systems for video technology, vol. 22, no. 12, pp. 1649-1668, 2012.
[10] J.RR. Ohm
Sullivan, "Versatile video coding-towarte the next generation video compression." in Picture Coding Symposium, vol. 2018, 2018.
[11] J. Ballé, V. Laparra, and E. P. Simoncelli, "End-to-end optimized image compression," in International Conference on Learning Representations, 2017.
[12] Ballé, D. Minnen, S. Singh. S. Hwang, and Johnston, "Variational image compression with scale hyperprior," in International Conference on Learning Representations, 2018.
[13] H. Li, S. Li, W. Dai, C. Li, J. Zou, and H. Xiong, "Frequency-awar transformer for learned image compression," in The Twelfth International Conference Learning Representations, 2024. [Online]. Available:
[14] Z. Cheng, H. Sun, M. Takeuchi, and Katto, "Leamed image compression with discretized gaussian mixture likelihoods and attention modules," in Proceedings of the IEEE/CVE conference on computer vision and puttern recognition, 2020, pp. 7939-7948.
[15] Liu, H. Sun, and Katto, "Learred image compression with mixed transformer-enn architectures," in Proceedings of the JEFF/CVF conference on computer vision and pattern recognition, 2023, pp. 14 388- 14 197.
[16] L. Theis, W. Shi, A. Cunningham, and F. Huszár. "Lossy image compression with compressive autoencoders." in International Conference Learning Representations, 2017. [Online]. Available: htns://oonneeewwww///oom//dddddd
[17] E. Agustsson, F. Mentzer, M. Tschannen, L. Cavigelli, R. Timofte, L. Benini, and L. V. Gool, "Soft-to-hand hard vector quantization for end-to to-end learning compressible representations." Advances in neural information processing systems, vol. 30, 2017.

---

[18] Y. Qian. X. Sun, M. Lin. Z. Tan, and R. Jin, "Entroformer: A transformer- based entropy model for learned image compression." in Interaational Conference Learning Representations, 2022. [Online]. Available: https://openeeviie..ee///oomoo vrrpa///oooooooooooooooroonoaa
[19] W. Jiang J. Yang. Y. Zhai, P. Ning, F. Gao, and R. Wang, "Mlic: Multi-reference entropy model for learned image compression." in Proceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 761 7618-7627.
[20] G. Zhang. J. Qian, J. Chen, and Khisti, "Universal rate- distortion percetioon representations for lossy compression." Advances in Neural Information Processing Systems, vol. 34, pp. 11 517-7-11 529, 2021.
[21] Zhao, B. J. Li, R. Xiong, and Y. Lu, "A universal optimization framework for learning- based image codee," ACM Transactions on Multimedia Computing, Communications and Applications, vol. 20, no. 1, pp. 1-19, 2023.
[22] F. Agustsson, M. Tschannen, F. Mentzer, Timofte, and I., V. Gool, "Generative adversarial networks for extreme learned image compression," in Proceedings of the IEEF/CVF international conference on computer vision, 2019, pp. 221-231.
[23] F. Mentzer, G. D. Toderici, M. Tschannen, and E, Agustsson, "High-fidelity generative image compression," Advances in neural information processing systems, vol. 33, pp. 11 913-11 924, 2020.
[24] E. Agustsson, D. Minnen, G. Toderici, and F. Mentzer, "Multi-realism image compression with conditional generator," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 22 324-22333 333.
[25] E. Lei, Y. B. Uslu, H. Hassani, and S. Saeedi Bidokhti, "Text+ sketch: Image compression at ultra low rates," in ICML Workshop on Neural Compression. ICML, 2023.
[26] M. J. Muckley, A. El-Nouby. K. Ulirich, H. Jégou, and J. Verbeek, "Improving statistical fidelity for neural image compression with implicit local likelihood models," in International Conference Machine Learning. PMLR, 2023, pp. 426 443.
[27] Z. Jia, Li. B. Li, H. Li. and Y. La, "Generative latent coding for ultra-low bitrate image compression." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 26 088 098.
[28] M. Careil, M. J. Muckley, Verbeek, and S. Lathuilière, "Towards image compression with perfect realism at ultra-low bitrates," in The Twelfth International Conference on Learning Representations, 2023.
[29] C. Li, Lu, D. Feng, Wu, Z. Zhang, Liu, Zhai, W. Lin, and W. Zhang, "Misc: Ultra-low bitrate image semantic compression driven by large multimodal model," IEEE Transactions on Image Processing. 2024.
[30] S. Yucer, M. Poyser, N. Al Moubayed, and T. P. Breekon, "Does lossy image compression affect racial bias within face recognition?" in 2022 IEEE International Joint Conference on Biometrics (IJCB). IEEE, 2022, pp. 1-10.
[31] T. Qiu, A. Nichani, R. Tadayontahmaschi, and H. Jeong, "Gone with the bits: Revealing racial bias in low-rate neural compression for facial images," arXiv preprint arXiv:2505.0000 2025.
[32] S. Yang, Y. Hu, W. Yang, L.-Y. Duan, and J. Liu, "Towards coding for human and machine vision: Scalable face image coding," IEEE Transactions on Multimedia, vol. 23, pp. 2957-2971, 2021.
[33] Y. Zhang, C. Jia, J. Chang, and S. Ma. "Machine perception- -riienn facial image compression: A layered generative approach," IEEE Transactions on Circuits and Systems for Video Technology, 2024.
[34] D. P. Kingma and M. Welling, "Auto-encoding variational bayes," arXiv e-prints, pp. MeXiv-1312. 2013.
[35] A. Razavi, A. Van den Oord, and O. Vinyals, "Generating diverse high-fidelity images with vq-vae-2," Advances in neural information processing systems, vol. 32, 2019.
[36] P. Esser, R. Rombach, and B. Ommer, "Taming transformers for high-resolution image synthesis," in Proceedings of the IEEE/CVF conference on computer vision and puttern recognition, 2021, pp. 12 873- 12 883.
[37] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, latent diffusion models." in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp.
image synthesis with
12

---

[38] F. Shi, Z. Luo, Y. Ge. Y. Yang. Y. Shan, and L. Wang. "Taming scalable visual tokenizer for autoregressive image generation." " arXiv preprint arXiv:2412.02692. 2024.
[39] J. Wang, K. Zhao, S. Zhang. Y. Zhang. Y. Shen, D. Zhao, and J. Zhou, "Lipformer: High-fidelity and generalizable talking face generation with a pre-leamed facial codebook." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. pp. 844-13 853.
[40] S. Tan, B. Ji, and Y. Pan, "Flowvgtalker: High-quality emotional talking face generation through normaliz- ing flow and quantization." in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 26 36317-28 2277
[41] S. Zhou, K. Chan, C. Li, and C. Loy, "Towards robust blind face restoration with codebook lookup transformer." Advances in Neura Information Processing Systems, vol. 35. pp. 30 599-30 611, 2022.
[42] Z. Wang, J. Zhang. R. Chen, W. Wang, and Luo. "Restoreformer: High-quality blind face restoration from undegraded key-value pairs." in Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, 2022, pp. 17 52512-11 521.
[43] Y. Gu, X. Wang. L. Xie, C. Dong. G. Li, Y. Shan, and M.-M. Cheng, "Vqir: Blind face restoration with vector quatti-ua dictionary and parallel decoder," in European Conference on Computer Vision. Springer, 2022, pp. 126-143.
[44] T. Karras, S. Laine, and T. Aila, "A style-based generator architecture for generative adversarial networks." in Proceedings of the IEEE/CVF conference computer vision and pattern recognition, 2019, pp. 4401-4410.
[45] R. A. Jacobs, M. I. Jordan, S. Nowlan, and G. E. Hinton, "Adaptive mixtures of local experts." Neural computation, vol. 3, no. 1. pp. 79-87, 1991.
[46] N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, "Outragcously large neural networks: The sparsely-gater gated of layer," arXiv preprint arXiv: 066538 2017.
[47] J. Deng, Guo, N. Xue, and S. Zafeiriou, "Arcface: Additive angular margin loss for deep face recognition." in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019. pp. 4690- 4699.
[48] D. Yi, Z. Lei, S. Liao, and S. Li, "Learning face representation from scratch." arXiv preprint arXiv:1411.791 1411.993, 2014.
[49] S. Sengupta, J.-C. Chen, C. Castillo, V.M. Patel, R. Chellappa, D. Jacobs, "Frontal profile face verification in the wild," in 2016 IEEE winter conference on applications of computer vision (WACV). IEEE, 2016, pp. 1-9.
[50] S. Moschoglou. A. Papaioannou, C. Sagonas, J. Deng, 1. Kotsia, and S. Zafeiriou, "Agedb: the first manually collected, in-the-wild age databasc," in proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 51-59.
[51] T. Zheng and W. Deng, "Cross-pose 1fw: A database for studying cross poss-ppse face recognition in uncon- strained environments," Beijing University of Posts and Telecommunications, Tech. Rep, vol. 5, no. 7, p. 5, 2018.
[52] T. Zheng, W. Deng, and J. Hu, "Cross-age Ifw: A database for studying cross-age facc recognition in unconstrained environments," arXiv preprint arXiv:1708.00 170001778..8197, 2017.
[53] H. Chang, H. Zhang, L. Jiang, C. Liu, and W. T. Freeman, "Maskgit: Masked generative image transformer." in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 11 315- 325.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our approach is to further enhance the face compression by our proposed switchable token-specific codebook.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have already written the limitation of our method in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have thoroughly introduced our method in the Method section and detailed our training process in the Experiments section. Additionally, we plan to release our code and checkpoints.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
15

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We will submit our codes.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips //ipss.cc/pill wwtpcccpppppppppeeeeeeeeeeeeeeee httaa//gggiiee//eeuuebiisissssss /Cc////oddiiisssseeeeeee ssioiolicyyyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: The training and testing protocols are comprehensively detailed the Experi- ments section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: The paper does not include error bars the presented results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have already provide information on the computer resources in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer:
[Yes]
Justification:
Our research conducted in the paper conform with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- cration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have already discussed societal impacts of the work performed in Section 1.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our work does not have such risks because it is based on open source datasets and works.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have already cite the original paper that produced the code package or dataset in our work.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our work does involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
19

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLM has no influence on the core methodology, scientific rigor or originality of this study,
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
20

---

A Technical Appendices and Supplementary Material
A.1 Image-level and Token-level Analysis
Due to the unique characteristics of facial images, images sharing the same attributes (such as ethnicity, gender, etc.) often exhibit many common features at the image level. At the token level, tokens corresponding to different facial regions also tend to have distinct feature representations. To address the attribute distribution characteristics at the image level, we design multiple groups of codebooks with routing mechanism capture both the differences and commonalities among image attributes. As illustrated in Figure[5a] we visualize the activation patterns of 16 codebook groups with respect the ethnicity attribute. It can be observed that, for African faces, the 5th and 14th codebook groups are frequently activated, whereas for Asian faces, the 9th and 3rd codebook groups are more likely be activated. This observation supports our hypothesis that designing separate codebooks for different attributes is beneficial.
At the token level, we conducted statistical analysis comparing two approaches: sharing single codebook among all tokens versus learning a separate codebook for each token, as shown in Figure 55b] The results indicate that when all tokens share single codebook, tokens different positions are unable to fully exploit the representational capacity of the codebook's latent space. This is reflected in the low utilization rate and large standard deviation for individual tokens. In contrast, with our proposed token-specific codehook approach, the utilization of the codebook by each token is significantly improved, with consistently high utilization rates across all tokens.
Divergence Analyals
Teeeristefff: pon1 Tbva [Closs]
<m 2114 001
ane 603 AX) AM 0.0 5.0
676 6/6 039 Kets mm 112
0.0
0.2
0.4
0.5
0.0
1.0
(a) Visualization analysis of codebook routing different ethnics.
(b) Visualization analysis of codebook utilization of different tokens.
Figure 5: Visualization analysis from image-level and token-level.
A.2 Alleviating the Trade-off between Compression and Recognition Accuracy
Directly reducing the codebook size to lower the hpp often leads to significant performance degra- dation. In contrast, our method effectively mitigates the loss in recognition performance associated with decreasing bpp. As shown in Figure 6 we conducted experiments on the TiTok-s128 baseline under three different bpp settings. The results demonstrate that our approach consistently improves recognition performance across all bpp levels. Moreover, as bpp decreases, our method better pre- serves recognition accuracy. For example, bpp 0.0235, method achieves improvements of 0.97% in recognition accuracy and 0.019 in IDS compared to the baseline. At an even lower bpp of 0.0157, the improvements increase to 1.32% in accuracy and 0.043 in IDS. These results confirm the robustness of our method to changes in hpp.
A.3 Impact of the Number of Routing Codebooks
Our method employs multiple small codebooks and utilizes a routing mechanism to assign each image to its respective codebook. This approach enables reduction in codebook size while maximally preserving the representational capacity of the codebooks' latent space. investigate the impact of the number of codebooks on final recognition performance, conducted further experiments. As
21

---

Accuracy vs BPP
IDS vs BPP
+0.97
Ours-5128
5128 Baseline
0.00
+0.019
S128 Baseline
Ours-5128 Improvement Area
91
0.58
Accuracy (%6
90
0.56
ID5
88
0.52
0.50
87
+0.043
+1.32
0.48
0.024 0.023 0.022 0.021 0.020 0.018 0.017 0.016 0.015
0.024 0.023 0.022 0.021 0.020 0.019 0.018 0.017 0.016 0.015
BPP
BPP
Figure 6: Comparison of recognition performance degradation with decreasing hpp between our method and the bascline.
shown in Figure 7 with the codebook size fixed at 1024, increasing the number of codebooks from 1 to leads to noticeable improvement recognition accuracy. This suggests that increasing the number of codebooks can indeed mitigate the performance degradation caused by directly reducing codebook size. However, we also observed that when the number of codebooks is further increased to 128, recognition performance begins decline. We speculate that this is due to the increased learning difficulty associated with a large number of codebooks. Fully leveraging the vast representational capacity of such large latent space is beyond the scope of this work.
Accuracy vs # Codebooks
IDS vs # Codebooks
90.00
Ours-S128
0.573
Accuracy (%)
0.572
IDS
0,570
0.569
16
64
128
16
64
128
Nums of Codebooks
Nums Codebooks
Figure 7: Effect routing codebook quantity on recognition accuracy.
22


Reviews:
[{"review_id": "CXUEiTDWII", "reviewer": "Reviewer_bPDd", "summary": "This paper proposes a switchable token-specific codebook quantization framework for face image compression. To address the capacity limitations of conventional codebook-based image compression methods, it reorganizes the codebook at multiple levels to decompose the problem into smaller subproblems by integrating both an image-level switchable codebook and a token-level sub-codebook quantization mechanism. To enable effective training of this hierarchical structure, the authors further design a novel three-stage training strategy. The idea has practical significance and, as one of the first attempts to realize such a concept, demonstrates a clear degree of originality.", "questions": "a) However, the paper has several weaknesses. It does not include visual examples of the reconstructed compressed images, so the perceptual quality of the images is hard to assess. Furthermore, it remains unclear whether decreasing the single codebook size might damage the ability to preserve token diversity, thus potentially leading to information loss.\n\nb) Also, the MoE-based routing design may lead to multiple codebooks learning redundant representations, but the paper does not provide explicit evidence to ensure codebook diversity. Additionally, the inference latency and memory overhead introduced by the multiple routing paths and token-specific codebooks are not analyzed, nor is the training resource requirement compared to other baselines.\n\nc) The method has only been validated on face image datasets, so it remains unclear whether the token-specific strategy would generalize well to other domains such as natural scenes or anime images, where the data distribution could be more complex. In addition, the approach heavily relies on the quality of the pretrained autoencoder, and the impact of different encoder-decoder backbones has not been explored. It would also be helpful to clarify how well this design could transfer to real-world edge devices with limited computational or memory resources.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The concept of using both image-level and token-level codebook quantization mechanisms is novel and timely, and the joint optimization approach is new in the context of codebook compression. During training, the integration of the MoE strategy and the loss functions is well designed. Moreover, with extensive benchmarks and detailed ablation studies, the experimental results are comprehensive and convincing. The paper is well written with clear figures and formulas. Finally, the proposed approach provides a novel framework for ultra-low bitrate face image compression, which can serve as a valuable resource for future research.", "was_revised": true, "final_justification": "The reviewer went through the discussion and made the final remarks. This is the final decision.", "rebuttal": "**Answer (a):**\n\nThank you for your valuable suggestion. To better assess perceptual quality, we have included visualization examples in the supplementary material. However, due to policy limitations, we regret that we are unable to incorporate additional visual examples at this stage. To further validate the image generation quality of our method, we have conducted quantitative evaluations of image quality metrics, as presented in the table below. We are committed to providing more comprehensive visualizations in future iterations to further demonstrate the efficacy of our approach.\n\n**Table: Quantitative comparison about image quality with baseline methods.**\n|Method|Model Type|Tokens|bpp|SSIM ↑|PSNR ↑|\n|-|-|-|-|-|-|\n|TiTok-S|1D|128|0.0234|0.7799|27.21|\n|TiTok-L|1D|32|0.0059|0.6868|23.25|\n|Ours(TiTok-S)|1D|128|0.0234|0.7823|27.39|\n|Ours(TiTok-L)|1D|32|0.0059|0.6931|23.33|\n\nRegarding \"decreasing the single codebook size might damage the ability to preserve token diversity\", in fact, directly reducing the size of a single codebook can indeed weaken its representational capacity. However, our method addresses this issue in two perspectives.\n\nFirst, we argue that a more diverse or larger codebook feature space is not always optimistic solution; rather, it should be well-matched to the latent space of the encoder outputs. If the codebook covers a very broad space but the tokens cannot effectively utilize it, this will lead to low codebook utilization and wasted resources. \n\nSecond, We observed that the original strategy of using a global-shared codebook across all tokens leads to an uneven distribution of codebook utilization. As shown in the table below, different tokens exploit the shared codebook but with varying degrees. To address this, we propose a token-specific codebook strategy, where each token learns its own codebook to maximize the preservation of its visual characteristics. With our approach, the issue of uneven codebook utilization among tokens is significantly alleviated. This demonstrates that our codebook is better able to fit the feature space of the encoder outputs. As a result, the feature information of each token can be more effectively quantized by the current codebook, thereby reducing quantization errors caused by codebook utilization imbalance.\n\n**Table: Codebook utilization rates (%) per token on LFW dataset.**\n\n|Method|bpp|Min|Max|Mean|STD|\n|-|-|-|-|-|-|\n|Global-shared|0.0234|3.49|78.12|54.17|14.71|\n|Ours|0.0234|17.90|83.89|74.02|9.14|\n\nFinally, by introducing multiple groups at the image level and employing a routing strategy to assign the most suitable group to each image, our approach effectively increases the overall codebook capacity, even if the size of each individual codebook is reduced. Moreover, each codebook becomes more specialized in capturing the common features of a particular category of images, which helps maintain the diversity of the codebook representation space.\n\n**Table: Total codebook utilization rates (%) under five face recognition datasets.**\n\n|bpp|Tokens|Codebooks|Codebook Size|LFW|CPLFW|CFPFP|AgeDB|CALFW|Mean Utilization|\n|-|-|-|-|-|-|-|-|-|-|\n|0.0234|128|1|4096|74.01|74.24|60.57|61.44|67.07|**67.47**|\n|0.0196|128|16|1024|90.39|94.68|87.53|86.19|87.93|**89.34**|\n|0.0157|128|256|256|93.15|96.45|94.46|91.89|92.00|**93.59**|\n\n**Answer (b):**\n\nTo maintain the representational capacity of the codebook latent space as individual codebook size decreases, our method employs multiple groups of codebooks and effectively utilizes them.\nTo better evaluate the utilization of these multiple codebook groups, we used the TiTok-S baseline as an example. We tested their codebook utilization under three different bpps, and the results are presented in the table below.\n\n\nAs shown in the table, the overall codebook utilization increases as the number of codebook groups grows. This result supports our hypothesis that, by organizing codebooks into groups, we can use smaller codebooks to capture more representative features. Consequently, this approach allows us to maintain the diversity of the codebook representation space while reducing the bpp during compression.\n\nWe have also evaluated the inference latency and storage overhead introduced by our method on a V100 GPU, as summarized in the table below. The table reports the inference latency and required storage for three baselines under different bpp (bits per pixel) settings. As shown, the expansion of the overall codebook size does indeed incur additional storage costs, along with a slight increase in inference latency. Since the number and dimensionality of tokens in the MaskGIT baseline are both significantly higher than those in the TiTok baseline (256 vs. 128 tokens, 256 vs. 12 dimensions), the storage overhead introduced by MaskGIT is substantially greater than that of TiTok. However, we would like to emphasize that in practical applications, our primary concern is the storage and transmission overhead of the images themselves. Our goal is to minimize image storage costs while preserving image quality as much as possible. Under the same bpp, our method achieves better recognition performance and image quality. Additionally, we have also explored approaches to reduce the model storage overhead. We explored a routing-based inference: only the codebook group selected by the router needs to be loaded, rather than searching with the minimum error across all codebooks. With this improvement, both inference latency and storage overhead are substantially alleviated, while the performance remains competitive with the baselines. There are also several engineering approaches to mitigate storage overhead. For example, we suggest performing quantization on the encoding side so that both the features and the codebook are matched within the int8 range. After obtaining the index sequence, the original fp32 codebook can be used on the cloud side for reconstruction and decoding.\n\n**Table: Inference latency and memory overhead statistics. For ours method, 'NN' and 'CR' refer to Nearest-Neighbor search and Codebook Routing search respectively.**\n\n|Backbone|bpp|Token-specific|Codebooks|Infer Time (s / img)|GPU Mem (MB)|Storage (MB)|FLOPs|\n|-|-|-|-|-|-|-|-|\n|Maskgit-VQGAN|0.0469|-|1|0.1495|474.14|94.69|158.14 G|\n|Ours (NN)|0.0469|✓|1|0.1554|3283.39|1122.70|158.14 G|\n|Ours (NN)|0.0391|✓|16|0.1771|12508.17|4194.70|158.14 G|\n|Ours (CR)|0.0391|✓|16|0.1544|5116.15|354.70|158.14 G|\n|TiTok-S|0.0234|-|1|0.1433|586.03|98.68|113.57 G|\n|Ours (NN)|0.0234|✓|1|0.1437|609.84|122.68|113.57 G|\n|Ours (NN)|0.0157|✓|256|0.1496|1894.54|482.71|113.57 G|\n|Ours (CR)|0.0157|✓|256|0.1450|1157.62|100.21|113.57 G|\n|TiTok-L|0.0059|-|1|0.1728|2709.24|1157.61|269.62 G|\n|Ours (NN)|0.0059|✓|1|0.1744|2716.80|1163.47|269.62 G|\n|Ours (NN)|0.0040|✓|256|0.1750|2865.43|1253.50|269.62 G|\n|Ours (CR)|0.0040|✓|256|0.1741|2804.02|1157.87|269.62 G|\n\n**Table: Quantitative comparison between different codebook selection policies during inference.**\n\n|Backbone|Codebook Selection|bpp|MeanAcc(%)|IDS|\n|-|-|-|-|-|\n|Maskgit-VQGAN|NN|0.0391|92.23|0.6264|\n|Maskgit-VQGAN|CR|0.0391|92.22|0.6253|\n|TiTok-S|NN|0.0157|87.60|0.5180|\n|TiTok-S|CR|0.0157|87.54|0.5125|\n|TiTok-L|NN|0.0040|66.02|0.1885|\n|TiTok-L|CR|0.0040|65.65|0.1864|\n\nRegarding the comparison of training resources, the original paper reports that training TiTok-S requires 32 A100-40G GPUs for 50 hours. In contrast, our method primarily focuses on training the codebook, while the encoder is directly adopted from pretrained weights and the decoder is correspondingly fine-tuned. As a result, our training cost is 8 V100 GPUs for nearly 40 hours. We hope this clarification helps you better understand the differences in training resource requirements.\n\n**Answer (c)**\n\nOur proposed token-specific strategy can be easily adapted to images from other domains as well. To better evaluate the generalizability of our method, we also conducted experiments on the larger and more diverse ImageNet-1K dataset, with the results presented in the table below. As shown, our approach achieves significant improvements over the original baseline across various metrics. For example, the FID score is reduced from 3.91 to 2.27, and the LPIPS score is improved from 0.3889 to 0.3564. These results demonstrate the necessity and effectiveness of learning a separate codebook for each token.\n\n**Table: Evaluation of reconstruction performance on the ImageNet-1K 256 × 256.**\n\n|Method|bpp|FID ↓|LPIPS ↓|SSIM ↑|PSNR ↑|\n|-|-|-|-|-|-|\n|TiTok-S|0.0234|3.91|0.3889|17.48|0.4034|\n|Ours|0.0234|2.27|0.3564|17.67|0.4139|\n\nAs you mentioned, our method relies on the performance of the pre-trained autoencoder baseline, but it can be conveniently integrated into any existing codebook-based image compression framework. Here, we present updated results based on the IBQ baseline published at ICCV 2025 [1], where our approach also demonstrates its effectiveness, as shown in the table below.\n\n**Table: Evaluation on state-of-the-art IBQ.**\n\n|Method|Model Type|Tokens|MeanAcc(%)|IDS|bpp|\n|-|-|-|-|-|-|\n|IBQ|2D|256|95.73|0.8403|0.0508|\n|Ours(IBQ)|2D|256|96.13|0.8545|0.0508|\n\nThe goal of image compression is to balance the trade-off between compression rate and distortion, aiming to minimize storage costs while preserving as much information from the original image as possible. In practical applications, we recommend deploying a lightweight encoder on edge devices to obtain high-quality compressed representations at lower bpp using our method, which can then be decoded or further processed by a decoder deployed in the cloud. Since our approach remains effective even at extremely low bpp (<0.01), it can significantly reduce the cost of image transmission and storage.\n\nReference: [1] Shi F, Luo Z, Ge Y, et al. Taming scalable visual tokenizer for autoregressive image generation. ICCV 2025."}, {"review_id": "xNj9pIZ57H", "reviewer": "Reviewer_whZv", "summary": "The authors propose a new vector quantization approach for low-bitrate image compression named Switchable Token-Specific Codebook Quantization (STSCQ). The approach addresses a key limitation of existing VQ-based compression models, where a globally shared codebook suffers at low bitrates due to reduced size. STSCQ introduces both image-level switchable codebooks (trained via routing) and token-specific sub-codebooks to better adapt to spatial and identity-specific variations in facial images. The method is evaluated on top of TiTok (Yu et al., 2024) and VQGAN (Esser et al., 2021), demonstrating improved performance on facial image reconstruction and recognition tasks at very low bpp.", "questions": "- In eq. 8, what is $p_i( \\cdot )$? The entropy loss seems incorrectly formulated, should it be $\\sum_i p_i \\log p_i$?\n- In eq. 12, the second $C_1$ seems a typo, should it be $C_2$? Also, does this imply that each of the $M$ image-level codebooks contains $T$ token-specific codebooks, for a total of $M \\times T$ sub-codebooks, each being the same as $C_{global}$? \n- While the motivation for token-specific codebooks is reasonable, would assigning a separate codebook to each token lead to redundancy? Have you measured codebook utilization for each $C^j_i$? Are they all being used effectively?\n- For the main results (Table 1), how are the codebook size $K$ and the number of image-level codebooks $M$ selected?   \n- In token-specific quantization, is only nearest-neighbor search (eq. 6) used? Why not extend the routing mechanism to this level as well to improve diversity? \n- The authors mention that the router is used only during training due to exploration-exploitation tradeoffs. Have you compared the posterior codebook utilization between learned routing and nearest-neighbor selection? This could further clarify the value and limitations of using a learned router.", "limitations": "The authors have discussed limitations.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths: \n- The idea of introducing switchable codebooks at the image-level and token-level to improve quantization capacity is well motivated.\n- The proposed training strategy appears compatible with standard latent-space models, making the approach plug-and-play.\n\nWeaknesses:\n- While STSCQ maintains a similar bit allocation to global-codebook methods (with negligible overhead for routing), the decoder must store and access a large number of codebooks at inference time (up to $M \\times T$). This additional memory/storage overhead is not reported or discussed.  \n- The ablation study in Section A.4 shows performance degradation when increasing the number of routing codebooks, suggesting potential limitations of the method on more complex or diverse datasets. \n- The paper’ presentation is not clear enough. See detailed questions below.", "was_revised": true, "final_justification": "The rebuttal has addressed most of my concerns. While the proposed method introduces additional overhead, the authors have presented an alternative inference approach that maintains similar task performance with reduced overhead. Thus I decide to increase my scores. Additionally, I hope the authors can further improve the presentation and the clarity of their method in the revised manuscript.", "rebuttal": "**Answer 1:**\n\nWe have provided a detailed analysis of the required storage overhead and inference latency on a V100 GPU, as shown in the table below. Admittedly, selecting the codebook group based on the minimum error during inference does introduce additional overhead, which is particularly noticeable when using MaskGIT-VQGAN with 256 tokens as the backbone. To address this, we also experimented with directly utilizing the routing results during inference. In this approach, the model only needs to load the codebook group indicated by the routing decision, rather than maintaining all codebook groups on the decoder side. This strategy offers clear advantages, significantly reducing both storage overhead and inference latency. In the future, we also plan to further mitigate this issue through optimization techniques such as model quantization or pruning.\n\n**Table: Inference latency and memory overhead statistics. For ours method, 'NN' and 'CR' refer to Nearest-Neighbor search and Codebook Routing search respectively.**\n\n|Backbone|bpp|Token-specific|Codebooks|Infer Time (s / img)|GPU Mem (MB)|Storage (MB)|FLOPs|\n|-|-|-|-|-|-|-|-|\n|Maskgit-VQGAN|0.0469|-|1|0.1495|474.14|94.69|158.14 G|\n|Ours (NN)|0.0469|✓|1|0.1554|3283.39|1122.70|158.14 G|\n|Ours (NN)|0.0391|✓|16|0.1771|12508.17|4194.70|158.14 G|\n|Ours (CR)|0.0391|✓|16|0.1544|5116.15|354.70|158.14 G|\n|TiTok-S|0.0234|-|1|0.1433|586.03|98.68|113.57 G|\n|Ours (NN)|0.0234|✓|1|0.1437|609.84|122.68|113.57 G|\n|Ours (NN)|0.0157|✓|256|0.1496|1894.54|482.71|113.57 G|\n|Ours (CR)|0.0157|✓|256|0.1450|1157.62|100.21|113.57 G|\n|TiTok-L|0.0059|-|1|0.1728|2709.24|1157.61|269.62 G|\n|Ours (NN)|0.0059|✓|1|0.1744|2716.80|1163.47|269.62 G|\n|Ours (NN)|0.0040|✓|256|0.1750|2865.43|1253.50|269.62 G|\n|Ours (CR)|0.0040|✓|256|0.1741|2804.02|1157.87|269.62 G|\n\n**Answer 2:**\n\nOur method can be readily transferred to images from other domains and is equally applicable to more complex and diverse datasets. For example, we report experimental results on ImageNet-1K, which contains more than 1M images from different categories. As shown in the table below, our approach achieves significant improvements over the original baseline across various metrics. Specifically, the FID score is reduced from 3.91 to 2.27, and the LPIPS score is improved from 0.3889 to 0.3564. \n\n**Table: Evaluation of reconstruction performance on the ImageNet-1K 256 × 256.**\n\n|Method|bpp|FID ↓|LPIPS ↓|SSIM ↑|PSNR ↑|\n|-|-|-|-|-|-|\n|TiTok-S|0.0234|3.91|0.3889|17.48|0.4034|\n|Ours|0.0234|2.27|0.3564|17.67|0.4139|\n\nAs discussed in Section A.4 of the supplementary material, we did observe that when increasing the number of routing groups while keeping the size of each codebook fixed, the evaluation metrics first improved and then declined. We attribute this to the limited size of our training dataset CASIA-WebFace, which is insufficient to fully utilize the large codebook space. Additionally, increasing the number of groups raises the routing complexity, resulting in under-trained embeddings within each codebook group. These factors together negatively impact the evaluation metrics. We plan to address this issue in our future work.\n\n**Answer for Q1:**\n\nWe apologize for the oversight—there is an error in Eq. 8 in the manuscript. The correct form is $\\mathcal{L}\\_{\\text{ent}} = \\sum\\_{i=1}^M \\bar{g\\_{\\theta}}(\\mathbf{z}\\_e) \\log \\bar{g\\_{\\theta}}(\\mathbf{z}\\_e)$, where $\\bar{g\\_{\\theta}}(\\mathbf{z}\\_e)$ denotes the average probability of the current codebook group across samples in the batch. This loss is to encourage the router to assign different codebook groups to samples within the same batch, thereby making full use of the expanded codebook latent space.\n\n**Answer for Q2:**\n\nWe appreciate you highlighting this oversight. You are correct—this was an editorial error, and the second $C_1$ should indeed be $C_2$. Regarding the codebook quantization in our method, when both switchable codebook quantization and token-specific codebook quantization are used in combination, the total number of sub-codebooks is indeed $M \\times T$. Furthermore, $C_{global}$ refers to the globally shared codebook in previous methods, where all $T$ tokens share a common set of $K$ embeddings. We hope this clarification addresses your concern. \n\n**Answer for Q3:**\n\nWe investigated the utilization of sub-codebooks when assigning separate codebooks to individual tokens. Our findings reveal that these sub-codebooks exhibit no sign of redundancy. Conversely, the token-specific codebook strategy not only further enhances the codebook utilization per token but also mitigates the imbalanced utilization allocation observed when using a single, shared codebook across all tokens, as shown in the table below. For those tokens with low utilization, we believe this may be due to their limited information content, which is related to the performance of the encoder. In the future, it would be reasonable to assign codebooks of varying sizes to different tokens to further reduce redundancy.\n\n**Table: Codebook utilization rates (%) per token on LFW dataset.**\n\n|Method|bpp|Min|Max|Mean|STD|\n|-|-|-|-|-|-|\n|Global-shared|0.0234|3.49|78.12|54.17|14.71|\n|Ours|0.0234|17.90|83.89|74.02|9.14|\n\n**Answer for Q4:**\n\nWe have now supplemented the main results in Table 1 by adding the corresponding number of codebook groups ($M$) and the size of individual codebooks ($K$). The updated results are shown below.\n\nIt is important to note that the main results presented here solely utilize the token-specific strategy. Consequently, the switchable codebook strategy—specifically designed to reduce the bits per pixel (bpp)—is not included in this main results table. Instead, it can be found in Table 2 within the main body of the paper.\n\n**Table: Detailed quantitative comparison with state-of-the-art methods.**\n\n| Method| Model Type | # Tokens | # Codebooks | Codebook Size | MeanAcc(%) | IDS| bpp|\n|-|-|-|-|-|-|-|-|\n| MaskGit-VQGAN| 2D| 256| 1| 4096| 90.70| 0.6314 | 0.0469 |\n| TiTok-S| 1D| 128| 1| 4096| 87.56| 0.5764 | 0.0234 |\n| TiTok-L| 1D| 32| 1| 4096| 65.07| 0.1812 | 0.0059 |\n| **Ours(MaskGit-VQGAN)** | 2D| 256| 1| 4096| 93.51| 0.6659 | 0.0469 |\n| **Ours(TiTok-s)**| 1D| 128| 1| 4096| 91.66| 0.6120 | 0.0234 |\n| **Ours(TiTok-l)**| 1D| 32| 1| 4096| 73.13| 0.2583 | 0.0059 |\n\n**Answer for Q5:**\n\nWhen quantizing each token, we employ the nearest neighbor search strategy, not the routing mechanism used at the image level. The reason for this design choice is that, if we were to continue using the routing selection mechanism for each token, it would indeed enhance the diversity of token representations. However, this approach would require storing an additional group index for every token, introducing extra overhead. This contradicts our goal of reducing bpp while increasing the overall codebook capacity. Specifically, assuming that image-level codebook routing is not considered, an image is represented by 256 tokens. If each token-specific codebook is divided into $M$ groups, each containing $K$ entries, then the number of bits required to represent an image is $256 \\times (\\log M + \\log K)$. For example, given an original codebook size of 4096, the required storage is $256 \\times \\log 4096 = 3072$ bits. However, if we partition the codebook into 256 groups, each of size 256, the storage increases to $256 \\times (\\log 256 + \\log 256) = 4096$ bits. \n\n**Answer for Q6:**\n\nYes, we have compared the codebook utilization between nearest neighbor search and our routing selection mechanism, and the results are shown in the table below. When using nearest neighbor search, both codebook utilization and evaluation metrics drop significantly—especially codebook utilization, which falls to only 19.89\\%. This clearly contradicts our original motivation for introducing multiple codebook groups. We believe this is because, with nearest neighbor search, the model tends to favor codebooks that have been more extensively optimized, leading to redundancy in other codebooks. Therefore, we introduce the routing selection mechanism to encourage the model to make full use of as many codebooks as possible, thereby leveraging the diversity of the codebook latent space. \n\n**Table: Comparisons of different codebook selection strategies used while training.**\n\n| Method| Codebook Selection | # Tokens | # Codebooks | Codebook Size | MeanAcc(%) | IDS| Codebook Mean Utilization(%) |\n|-|-|-|-|-|-|-|-|\n| Ours(TiTok-S)  | NN | 128| 16| 1024| 89.28| 0.5701 | 19.89|\n| Ours(TiTok-S)  | Routing| 128| 16| 1024| 89.89| 0.5740 | 89.34|"}, {"review_id": "ZKAg7POyXE", "reviewer": "Reviewer_GXvd", "summary": "This paper proposes a switchable token-specific codebook quantization method which dynamically specifies a codebook for each image using router and uses specified sub-codebooks for each token in the codebook. The hierarchically dynamic structure significantly reduces the required bpp while achieving high accuracy on face recognition datasets.", "questions": "1.From the title of the paper, it appears to be a paper in the field of facial image encoding. But from the content of the article, it seems more like a downstream task in the field of facial image encoding. I recommend supplementing the experimental results according to the weakness on experiment mentioned in the previous text.\n2.As the point mentioned in the weakness on method,  in Equation (8) and  in Equation (9) are essentially the same. The effectiveness of these two losses should be clearly discussed in the paper. Moreover, what are the consequences of using different routing strategies during training and inference due to the optimization dilemmas?\nIt would be more convincing if the authors could address the issues related to their methods", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Paper Strengths: \n1.The paper addresses an important challenge in the low bpp facial image compression by proposing a dynamic method for specifying codebooks.\n2.The paper is generally easy to follow, and the mathematical formulations are logically consistent.\n3.Quantitative experiments demonstrate the advantages of the proposed method over existing baselines.\n\nMajor Weaknesses:\nWeakness on paper writing:\n\n1.In Equation (12), the subscripts of the first two codebooks are both 1, which should be modified to 1 and 2.\n2.In line 284-285, the original text in the paper is: “our approach achieves a 16.6% bitrate reduction (0.0469 → 0.0391 bpp) coupled with a 1.53% absolute accuracy improvement (90.70% → 92.23%).” The accuracy “90.70%” is inconsistent with the data “93.51%” provided in Table 3. Please check the validity of the data.\n\nWeakness on method:\n1.In order to achieve dynamic routing of the codebooks, the paper introduces a router to calculate the probability distribution of each codebook. However, the paper points out that the routing strategy is inconsistent during training and inference due to the exploration-exploitation dilemma inherent in entropy-regularized optimization, which is equivalent to the router only working during training. The paper lacks theoretical analysis of the dilemma. It would be beneficial to discuss the reason of the dilemma.\n2.In Equation (8) and Equation (9), two losses designed for training routers are introduced, which are calculated based on the probability distribution output by the router. However, due to the lack of description of  in the text, based on the known information, these two losses are actually equivalent to the information entropy of selection distributions with different weights.\n\nWeakness on experiments:\n1.For evaluating compression performance, the paper only relies on Mean Accuracy and IDS, ignoring commonly used evaluation metrics in facial image compression, such as PSNR and LPIPS. It is recommended to complete experimental results to obtain a more comprehensive evaluation.\n2.The paper mentions that five facial datasets were used for evaluation in the baseline model comparison experiment. However, the paper did not provide specific results on each dataset. At the same time, it is recommended to expand the number of baseline models, which helps to more accurately evaluate the effectiveness of the method\n3.The paper should compare with some facial compression methods such as [1][2][3].\n\nReference:\n[1] Qi Mao et al. Scalable Face Image Coding via StyleGAN Prior: Towards Compression for Human-Machine Collaborative Vision. TIP.2023\n[2] Yuefeng Zhang et al. Machine Perception-Driven Facial Image Compression: A Layered Generative Approach. TCSVT.2024.\n[3] Shuai Yang et al. Towards Coding for Human and Machine Vision:: Scalable Face Image Coding.TMM.", "was_revised": true, "final_justification": "I maintain my original score. The image quality achieved by the proposed method does not appear to show significant improvement compared to the baseline. I understand that the scope of this work is focused on face recognition; however, I suggest revising the paper’s title to be more specific, as “face image compression” implies a substantially different concept in the compression domain. Given that the title explicitly claims “face image compression,” a thorough comparison with a broad range of existing methods in this area is expected. Currently, such comparisons are too limited—I have listed a few examples, but ideally, many more should be included. Moreover, the experiments primarily emphasize the low-bpp setting, without addressing whether the method can effectively compress high-resolution images. The papers cited mostly operate at very low bpp, and some are specifically designed for ID feature preservation, which can lead to particularly strong results in those conditions. As it stands, the compression application scenario targeted by this work appears very narrow. If this is positioned as a compression paper, from the perspective of the compression community, I do not consider it to fully meet that standard.", "rebuttal": "**Answer for paper writing:**\n\nThank you for pointing out the editorial error; we will correct it in future revisions. Specifically, regarding lines 284-285, our intention was to convey that our method is able to maintain—or even surpass—the baseline performance as the bpp decreases. For example, when bpp = 0.0469, the baseline achieves a MeanAcc of 90.70\\%, whereas our method achieves a MeanAcc of 92.23\\% at a lower bpp of 0.0391. We hope this clarification addresses your concerns.\n\n**Answer for method 1:**\n\nSelecting the most appropriate codebook for the model poses a significant challenge. During training, we experimented with directly using a nearest neighbor strategy for codebook selection. However, we observed that this approach resulted in a notably low codebook utilization rate of only 19.89\\%, indicating that the multiple codebooks were not being fully exploited. We attribute this to the nearest neighbor strategy’s tendency to favor codebooks that have been more thoroughly optimized, leading to a collapse where multiple codebooks degenerate into a single codebook.\n\n**Table: Comparisons of different codebook selection strategies used while training.**\n\n| Method| Codebook Selection | # Tokens | # Codebooks | Codebook Size | MeanAcc(%) | IDS    | Codebook Mean Utilization(%) |\n|-|-|-|-|-|-|-|-|\n| Ours(TiTok-S) | NN| 128| 16| 1024| 89.28| 0.5701 | 19.89|\n| Ours(TiTok-S) | Routing| 128| 16| 1024| 89.89| 0.5740 | 89.34|\n\nTo address this issue, we designed a routing strategy based on entropy-regularized optimization during training, which encourages the model to select a more diverse set of codebooks and mitigates codebook overfitting, thereby promoting more comprehensive codebook training. During inference, we adopt the nearest neighbor strategy based on the minimum loss principle, as the codebook with the smallest loss corresponds to the upper bound of our model’s performance. We also experimented with using the routing strategy during inference and found that its performance was comparable to that of the nearest neighbor strategy, while offering a more efficient inference process. Specifically, it significantly reduced both inference latency and storage overhead. For example, under the TiTok-S architecture, the inference latency per image using the routing strategy was 0.0103 seconds, while remaining the MeanAcc of 87.54\\%, compared to 0.0154 seconds with the nearest neighbor strategy for the MeanAcc of 87.60\\%. \n\n**Table: Quantitative comparison between different codebook selection policies during inference.**\n\n|Backbone|Codebook Selection|bpp|MeanAcc(%)|IDS|\n|-|-|-|-|-|\n|Maskgit-VQGAN|NN|0.0391|92.23|0.6264|\n|Maskgit-VQGAN|CR|0.0391|92.22|0.6253|\n|TiTok-S|NN|0.0157|87.60|0.5180|\n|TiTok-S|CR|0.0157|87.54|0.5125|\n|TiTok-L|NN|0.0040|66.02|0.1885|\n|TiTok-L|CR|0.0040|65.65|0.1864|\n\n**Answer for method 2:**\n\nWe apologize for the oversight—there is an error in Equation 8 in the manuscript. The correct form is $\\mathcal{L}\\_{\\text{ent}} = \\sum\\_{i=1}^M \\bar{g\\_{\\theta}}(\\mathbf{z}\\_e) \\log \\bar{g\\_{\\theta}}(\\mathbf{z}\\_e)$, where $\\bar{g\\_{\\theta}}(\\mathbf{z}\\_e)$ denotes the average probability of the current class across samples in the batch. Equation 8 is designed to encourage the router to select diverse codebook groups for different samples within the same batch, thereby fully exploring and utilizing the expanded codebook latent space. In contrast, Equation 9 measures the entropy of the predicted distribution for each individual sample, aiming to suppress excessive uncertainty in single-sample predictions. Together with Equation 10, these three objectives not only promote comprehensive utilization of the codebook space, but also effectively balance inter-sample diversity with the certainty of individual predictions.\n\n**Answer for experiment 1:**\n\nYou are absolutely right that including image quality metrics is essential for a comprehensive evaluation. In response, we have additionally assessed both 1D and 2D baselines, using three widely adopted metrics: SSIM, PSNR, and LPIPS. The results are summarized in the table below, where IBQ refers to the state-of-the-art 2D baseline published at ICCV 2025. As shown, our method achieves significant improvements in image quality, further demonstrating its effectiveness. Moreover, we observe that the LPIPS metric fluctuates across different baselines. We hypothesize that this is because the LPIPS score is computed using a network pre-trained on ImageNet to assess perceptual similarity between images, while there are substantial differences between facial data and ImageNet data. Therefore, we prefer to use MeanAcc and IDS as more reliable indicators of machine perception for facial compression tasks. We will incorporate these latest results in the revised version of our paper. \n\n**Table: Quantitative comparison about image quality with baseline methods.**\n| Method| Model Type | # Tokens | bpp    | SSIM ↑  | PSNR ↑ | LPIPS ↓ |\n|-|-|-|-|-|-|-|\n| IBQ            | 2D         | 256      | 0.0508 | 0.8520  | 29.79  | 0.1880  |\n| MaskGIT        | 2D         | 256      | 0.0469 | 0.7908  | 27.22  | 0.2204  |\n| TiTok-S        | 1D         | 128      | 0.0234 | 0.7799  | 27.21  | 0.2323  |\n| TiTok-L        | 1D         | 32       | 0.0059 | 0.6868  | 23.25  | 0.2959  |\n| Ours(IBQ)      | 2D         | 256      | 0.0508 | 0.8600  | 30.21  | 0.1765  |\n| Ours(MaskGIT)  | 2D         | 256      | 0.0469 | 0.7802  | 27.27  | 0.2395  |\n| Ours(TiTok-S)  | 1D         | 128      | 0.0234 | 0.7823  | 27.39  | 0.2354  |\n| Ours(TiTok-L)  | 1D         | 32       | 0.0059 | 0.6931  | 23.33  | 0.2968  |\n\n**Answer for experiment 2:**\n\nWe have provided the detailed recognition accuracy and IDS metrics on five face recognition datasets, as shown in the table below. \n\n**Table: Recognition accuracy(%) on five facial datasets.**\n| Method         | Model Type | bpp    | LFW   | CFP_FP | AgeDB | CPLFW | CALFW | Mean   |\n|----------------|------------|--------|-------|--------|-------|-------|-------|--------|\n| MaskGIT        | 2D         | 0.0469 | 98.33 | 89.11  | 88.03 | 88.33 | 89.67 | 90.70  |\n| TiTok-S        | 1D         | 0.0234 | 96.51 | 85.29  | 83.41 | 86.15 | 86.45 | 87.56  |\n| TiTok-L        | 1D         | 0.0059 | 77.84 | 63.07  | 57.29 | 62.19 | 64.97 | 65.07  |\n| Ours(MaskGIT)  | 2D         | 0.0469 | 99.46 | 92.40  | 91.09 | 91.82 | 92.79 | 93.51  |\n| Ours(TiTok-S)  | 1D         | 0.0234 | 98.68 | 90.21  | 88.77 | 89.12 | 91.52 | 91.66  |\n| Ours(TiTok-L)  | 1D         | 0.0059 | 87.34 | 67.07  | 65.74 | 71.82 | 73.66 | 73.13  |\n\n**Table: IDS on five facial datasets.**\n\n| Method         | Model Type | bpp    | LFW    | CFP_FP | AgeDB  | CPLFW  | CALFW  | Mean   |\n|----------------|------------|--------|--------|--------|--------|--------|--------|--------|\n| MaskGIT        | 2D         | 0.0469 | 0.6481 | 0.5989 | 0.6198 | 0.6643 | 0.6259 | 0.6314 |\n| TiTok-S        | 1D         | 0.0234 | 0.5883 | 0.5558 | 0.5570 | 0.6076 | 0.5733 | 0.5764 |\n| TiTok-L        | 1D         | 0.0059 | 0.1719 | 0.1682 | 0.1743 | 0.2215 | 0.1696 | 0.1812 |\n| Ours(MaskGIT)  | 2D         | 0.0469 | 0.6751 | 0.6428 | 0.6765 | 0.6654 | 0.6698 | 0.6659 |\n| Ours(TiTok-S)  | 1D         | 0.0234 | 0.6187 | 0.5967 | 0.6180 | 0.6072 | 0.6195 | 0.6120 |\n| Ours(TiTok-L)  | 1D         | 0.0059 | 0.2563 | 0.2381 | 0.2405 | 0.2963 | 0.2604 | 0.2583 |\n\nRegarding your suggestion to compare with face compression methods, we have carefully reviewed the three representative papers you mentioned. Unfortunately, none of these works have released their code, and their experimental settings for the face compression task differ from ours. Specifically, these studies focus on preserving image details in high-resolution data. For example, [1] trains on the high-resolution FFHQ dataset and evaluates image quality on CelebA-HQ, which is not directly comparable to our setting, where the goal is to compress images from face recognition datasets while preserving identity features.\n\nNevertheless, we attempted to use screenshots of the visual results in Fig.6 from [2] and computed the IDS value between the original images and compressed images to assess the effectiveness in preserving facial identity. As shown in the table, our method demonstrates superior performance in identity preservation. In future versions, we will also include visual comparisons with these face compression methods.\n\n**Table: Comparison of IDS between other facial compression methods and ours.**\n\n| Method         | bpp    | IDS ↑   |\n|----------------|--------|---------|\n| MPDFIC[2]      | 0.246  | 0.4788  |\n| MPDFIC[2]      | 0.092  | 0.3571  |\n| MPDFIC[2]      | 0.045  | 0.1866  |\n| Ours(TiTok-S)  | 0.0234 | 0.6357  |\n| Ours(TiTok-S)  | 0.0157 | 0.5626  |"}, {"review_id": "afXuss7cVm", "reviewer": "Reviewer_twLH", "summary": "This paper proposes a novel method for compressing facial images using a Switchable Token-Specific Codebook Quantization (STSCQ) framework. Traditional VQ-based image compression methods degrade under ultra-low bitrate (bpp) due to reliance on a single global codebook. To address this, the authors introduce a two-level codebook structure: (i) image-level routing to select a specialized codebook and (ii) token-level codebook decomposition to assign each token its own quantizer. The architecture is trained in a three-stage progressive manner, incorporating ArcFace loss for identity preservation. Extensive experiments across five face recognition benchmarks show improved rate-distortion trade-offs and recognition accuracy over prior methods like TiTok and MaskGIT. The method demonstrates superior compression efficiency and recognition fidelity, particularly at low bpp (e.g., 66.02% accuracy at 0.004 bpp).", "questions": "1. Can you rerun the full evaluation over ≥ 3 random seeds and report mean ± standard deviation (or 95 % confidence intervals) for all metrics in Tables 1 and 2?\n2. What are the training-time and inference-time costs, FLOPs, and latency, relative to TiTok-s and MaskGIT?", "limitations": "No. The paper lists technical limitations (dependency on the underlying auto-encoder) and briefly claims positive societal value, but it omits two key areas:\n\n1. Supplementary Fig. 2 shows that routing groups correlate with ethnicity; however, the authors provide no demographic parity analysis or mitigation strategy. A brief experiment on RFW or a similar dataset, along with a discussion of safeguards (e.g., bias-aware training or post-compression calibration), is necessary.\n \n2. Ultra-low-bpp face compression can lower the cost of large-scale surveillance. The paper should acknowledge this and outline responsible-release measures (e.g., gating access to pretrained weights, adding watermarking, or providing usage guidelines).", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The paper offers a compelling, though not flawless, contribution. Its chief strength lies in the originality of a two-level quantization design that marries switchable image-level routing with token-specific sub-codebooks, an architectural twist that demonstrably boosts face-recognition accuracy at ultra-low bitrates and thus promises real utility for storage-constrained biometric pipelines. \n\nMethodologically, the authors conduct sensible ablations, test the idea on both 1-D and 2-D latent tokenizers, and document a clear three-stage training schedule. Well-designed figures also aid reader comprehension. \n\nHowever, important weaknesses remain: all quantitative results lack error bars or confidence intervals, so the statistical robustness of the reported gains is uncertain. Furthermore, the novelty of image-level routing is somewhat incremental given prior work on mixture-of-experts and product-quantization, and the paper omits a discussion of computational overhead, which is crucial for edge deployment. \n\nIf the authors add statistical significance reporting and quantify the computational cost, the work would merit acceptance; without these corrections, the contribution falls just short of the bar for a premier venue.", "was_revised": true, "final_justification": "The authors’ rebuttal offers a generally adequate and structured response to the raised concerns, addressing each key issue with new experimental evidence and clarifications. For statistical robustness, they reran experiments over three random seeds and reported mean ± standard deviation for the TiTok-S baseline. While this partially satisfies the request, the omission of multi-seed evaluations for other backbones limits completeness, though they reasonably justify this with resource constraints. \n\nThe concern regarding computational overhead is fully addressed: detailed tables report inference time, GPU memory, storage, and FLOPs across configurations, comparing their method with TiTok and MaskGIT and demonstrating significant improvements, particularly under the Codebook Routing strategy. \n\nOn demographic fairness, the authors provide per-ethnicity accuracy evaluations using the RFW dataset and argue that the high number of codebooks dilutes any strong alignment with specific groups. While no direct mitigation is implemented, the authors propose future work involving group-specific decoder fine-tuning, which reasonably addresses the concern. \n\nEthical risks around ultra-low-bpp face compression are now acknowledged, with promised countermeasures such as watermarking and usage guidelines to be included in the final version. Although the authors do not explicitly expand on the originality concern, they reiterate the architectural innovations of their token-specific quantization and routing strategy. \n\nOverall, the rebuttal resolves the primary issues of evaluation rigor, computational efficiency, and ethical responsibility. The rebuttal justifies an upgrade to an accept.", "rebuttal": "**Answer 1:**\n\nWe re-evaluated our method using three different random seeds, and the results are presented in the table below. Due to time and resource constraints, we are only able to report results based on the TiTok-S baseline. As shown in the table, the impact of random seed variation is not significant, which can be attributed to the fact that the core of our method lies in training the codebook, while the encoder and decoder are initialized with pre-trained weights. We plan to update the re-evaluation results for other baselines in future versions.\n\n**Table: Switchable Token-Specific Codebook Quantization on different baselines.**\n\n| Backbone  | # Tokens | # Codebooks | Codebook Size | MeanAcc(%)         | IDS                | Storage(bits) | bpp    |\n|-----------|----------|-------------|---------------|--------------------|--------------------|--------------|--------|\n| TiTok-s   | 128      | 1           | 4096          | 91.65±0.02         | 0.6130±0.0020      | 1536         | 0.0234 |\n| TiTok-s   | 128      | 16          | 1024          | 89.75±0.14         | 0.5681±0.0059      | 1284         | 0.0196 |\n| TiTok-s   | 128      | 256         | 256           | 87.63±0.03         | 0.5179±0.0002      | 1032         | 0.0157 |\n\n**Answer 2:**\n\nWe have reported the FLOPs of the model, the required storage overhead and the inference time per image on a V100 GPU in the table below. As shown, our method does introduce additional storage and inference latency, and this effect becomes more pronounced as the number of codebook groups and the size of the codebook latent space increase. We attribute this to the need to maintain all codebook groups during inference when selecting the nearest codebook based on the minimum error criterion. To address this, we experimented with directly using the routing results during inference, so that the decoder only needs to access the codebook corresponding to the routing index. The benefits of this approach are clearly reflected in the table: for example, with the MaskGIT backbone, storage overhead is reduced by approximately 60\\% and inference latency by about 40\\%, while remaining the performance of MeanAcc and IDS. \n\n**Table: Inference latency and memory overhead statistics. For ours method, 'NN' and 'CR' refer to Nearest-Neighbor search and Codebook Routing search respectively.**\n\n|Backbone|bpp|Token-specific|Codebooks|Infer Time (s / img)|GPU Mem (MB)|Storage (MB)|FLOPs|\n|-|-|-|-|-|-|-|-|\n|Maskgit-VQGAN|0.0469|-|1|0.1495|474.14|94.69|158.14 G|\n|Ours (NN)|0.0469|✓|1|0.1554|3283.39|1122.70|158.14 G|\n|Ours (NN)|0.0391|✓|16|0.1771|12508.17|4194.70|158.14 G|\n|Ours (CR)|0.0391|✓|16|0.1544|5116.15|354.70|158.14 G|\n|TiTok-S|0.0234|-|1|0.1433|586.03|98.68|113.57 G|\n|Ours (NN)|0.0234|✓|1|0.1437|609.84|122.68|113.57 G|\n|Ours (NN)|0.0157|✓|256|0.1496|1894.54|482.71|113.57 G|\n|Ours (CR)|0.0157|✓|256|0.1450|1157.62|100.21|113.57 G|\n|TiTok-L|0.0059|-|1|0.1728|2709.24|1157.61|269.62 G|\n|Ours (NN)|0.0059|✓|1|0.1744|2716.80|1163.47|269.62 G|\n|Ours (NN)|0.0040|✓|256|0.1750|2865.43|1253.50|269.62 G|\n|Ours (CR)|0.0040|✓|256|0.1741|2804.02|1157.87|269.62 G|\n\n**Table: Quantitative comparison between different codebook selection policies during inference.**\n\n|Backbone|Codebook Selection|bpp|MeanAcc(%)|IDS|\n|-|-|-|-|-|\n|Maskgit-VQGAN|NN|0.0391|92.23|0.6264|\n|Maskgit-VQGAN|CR|0.0391|92.22|0.6253|\n|TiTok-S|NN|0.0157|87.60|0.5180|\n|TiTok-S|CR|0.0157|87.54|0.5125|\n|TiTok-L|NN|0.0040|66.02|0.1885|\n|TiTok-L|CR|0.0040|65.65|0.1864|\n\n**Answer for limitations 1:**\n\nWe conducted experiments on the four ethnic groups in the RFW dataset, and the results are shown in the table below. Our findings indicate that the introduction of multiple codebooks does not lead to significant changes in ethnic bias. Although our supplementary materials reveal that the codebook groupings are related to ethnicity, the number of codebook groups is much larger than the number of ethnic categories, so each group is not strongly associated with any single ethnicity. We suggest that future work could explore fine-tuning the decoder for each ethnic group in our stage 3 to better balance recognition performance across different groups. We will include a discussion of these limitations in future versions.\n\n**Table: Quantitative comparison between different codebook selection policies during inference.**\n\n| Backbone | bpp    | # Codebooks | African | Asian  | Caucasian | Indian | MeanAcc(%)        | IDS    |\n|----------|--------|-------------|---------|--------|-----------|--------|-------------------|--------|\n| MaskGIT  | 0.0469 | 1           | 83.30   | 83.12  | 90.22     | 85.13  | 85.44±2.87        | 0.6932 |\n| MaskGIT  | 0.0391 | 16          | 80.35   | 79.27  | 87.72     | 84.70  | 83.01±3.39        | 0.6592 |\n| TiTok-S  | 0.0234 | 1           | 80.13   | 80.00  | 87.88     | 87.65  | 82.42±3.22        | 0.6428 |\n| TiTok-S  | 0.0157 | 256         | 71.97   | 76.52  | 79.25     | 75.78  | 75.88±2.60        | 0.5438 |\n\n**Answer for limitations 2:**\n\nWe acknowledge that ultra-low-bpp face compression carries the risk of facilitating large-scale surveillance and potential misuse. To mitigate such concerns, we will provide clear usage guidelines and ethical instructions upon the open-sourcing of our project, and incorporate watermarking into the compressed images to help prevent improper use. We will include a statement addressing these issues in the limitations section of a future version."}]

Author rebuttal:

Meta-review: In this paper the aurhors introduce a switchable, token-specific codebook quantization framework for face image compression. Unlike conventional codebook-based methods, which are often constrained by limited capacity, the proposed approach restructures the codebook at multiple levels. It combines an image-level switchable codebook with a token-level sub-codebook quantization mechanism. This breaks down the overall task into smaller, more manageable subproblems. To effectively train this hierarchical design, the authors devise a novel three-stage training strategy. As one of the first implementations of this idea, the method shows both practical significance and a notable degree of originality.
Strenghts include (i) novel idea of combining image-level and token-level codebook, (ii) novel idea of joint optimization approach for this problem, (iii) solid experimental evaluation, (iv) sensible ablation study, (v) good results, (vi) well written. 
Weaknesses include (i) lack of visual illustrations, (ii) only tested on face images, (iii) risk of redundant representations, (iv) high memory requirements for the decoder.
The paper got mixed reviews before the rebuttal, but after the rebuttal and the following thorough discussions, the reviewers where predominantly positive with scores 4,4,3,5.
Decision: Accept (poster)