Concept-Centric Token Interpretation for Vector-Quantized Generative Models
Authors: Tianze Yang
Yucheng Shi
Mengnan Du
Xuansheng Wu
Qiaoyu Tan
Jin Sun
Ninghao Liu
Venue: ICML 2025 poster (2025)
Keywords: Vector-Quantized Generative Model
Explainability
Information Bottleneck
Primary area: deep_learning->everything_else

Abstract: Vector-Quantized Generative Models (VQGMs) have emerged as powerful tools for image generation. However, the key component of VQGMs---the codebook of discrete tokens---is still not well understood, e.g., which tokens are critical to generate an image of a certain concept?
This paper introduces Concept-Oriented Token Explanation (CORTEX), a novel approach for interpreting VQGMs by identifying concept-specific token combinations. Our framework employs two methods: (1) a sample-level explanation method that analyzes token importance scores in individual images, and (2) a codebook-level explanation method that explores the entire codebook to find globally relevant tokens. Experimental results demonstrate CORTEX's efficacy in providing clear explanations of token usage in the generative process, outperforming baselines across multiple pretrained VQGMs. Besides enhancing VQGMs transparency, CORTEX is useful in applications such as targeted image editing and shortcut feature detection. 
Our code is available at https://github.com/YangTianze009/CORTEX.

Concept-Centric Token Interpretation for Vector-Quantized Generative Models
Tianze Yang"
Yucheng Shi"
Mengnan Du Xuansheng Wu
Qiaoyu Tan 3 Jin Sun
Ninghao Liu'
Abstract
white doctor the hospital"
Vector- cesto-Quuattized Generative Models (VQGMs) have emerged powerful tools for image genera- tion. However, the key component of VQGMs- the codebook of discrete tokens-is still not well understood, e.g., which tokens are critical to gen- erate an image of certain concept? This paper introduces Corcept-Oriented Token Explanation (CORTEX), a novel approach for interpreting VQGMs by identifying concept-specific token combinations. Our framework employs two methods: (1) a sample-level explanation method that analyzes token importance scores in indi- vidual images, and (2) a codebook-level expla- nation method that explores the entire code- book to find globally relevant tokens. Experi- mental results demonstrate CORTEX's efficacy in providing clear explanations of token usage in the generative process, outperforming base- lines across multiple pretrained VQGMs. Be- sides enhancing VQGMs transparency, CORTEX is useful in applications such as targeted im- age editing and shortcut feature detection. Our code is available at https: /ttls:://gttttttt github. com/ YangTi anze CORTEX.
Explanation
White dectoor reloted takans: [228] '9tor, '11927" '31.3" 18663 ND283 90514 '22857 '1139/ '5401')
black doctor the hospital"
Explanation
Block doctor related tolens: P1125; 16628 11337Y, 16147. 18485) 11.024 4285;; 15647" 1660e', (1811))
"A doctor the hospital"
Average per image:
White doctor: 8.55
Black doctor: 2.32
Token-based analysis reveals racial bias in generative models. While the model can generate both white and black doc- tors when explicitly prompted (top, middle). neutral prompts (bot- tom) show a systematic hias. CORTEX quantifics this bias through token analysis, showing white dostor-associated tokens appear nearly times more frequently (8.55 2.32 tokens per image).
Figure 1:
ponent of these models is the codebook (Esser et al., 2021), which acts learned dictionary of visual elements. This codebook stores finite set of discrete tokens, each repre- senting various patterns or features within an image. How- ever, not all tokens contribute equally to the generation of a particular concept (e.g., object categories or visual attributes), leading to the need for methods that can dis- tinguish between concept-relevant and background tokens. Improving the interpretability of these token-concept rela- tionships can help identify potential biases in the model's s representations and enable precise control through targeted image editing.
1. Introduction
Vector- -uroo-Quanizzed Generative Models (VQGMs) have be- come powerful tools for high-quality image generation using discrete latent space representations (Ramesh et al., 2021; Esser et al., 2021; Yu et al., 2021; Jin et al., 2023; Tian et al., 2024). Despite their success, these models often exhibit con- cerning behaviors (e.g., showing demographic disparities in professional representation, Fig. I demonstrates), Un- derstanding these issues requires interpreting how VQGMs internally represent and process concepts. A critical com-
Given any user-interested concept, our goal is to select to- kens from codebook whose combination can best repre- sent it. A straightforward approach to token interpretation is select tokens that frequently appear in images gener- ated for specific concept (Blei al., 2003). However, this method often selects tokens that represent contextual or background elements, resulting in explanations cluttered with irrelevant information. This inability to differentiate between essential and non-essential tokens hinders a clear understanding of how the model represents concepts.
'Equal contribution 'School of Computing, University of Geor- gia 'Department of Data Science, New Jersey Institute of Tech- nology 'Department of Computer Science, New York University. Correspondence to: Ninghao Liu <ninghao. uuga.ddu>
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Concept-Centric Token Interpretation for Vector-Quantized Generative Models
To address this issue, we draw on the Information Bottleneck principle (Tishby al., 2000), which focuses on compress- ing imput data while retaining the most relevant information for a given label. In the context of VQGMs, we apply this principle to train an Information Extractor, a module that maps image tokens back to semantic labels reversing the information flow of generative models which typically map semantic descriptions to images.
2. Problem Statement
2.1. Vector-Quantized Generative Models
Vector-Quantize Generative Models (VQGM) (Ramesh et al., 2021; Esser et al., 2021; Yu et al., 2021; Jin et al., 2023) generate images via decoding from discrete tokens. These models are typically designed for conditional genera- tion and are capable of generating images based on given text descriptions or class labels. During the image genera- tion process, these models typically consist of three parts: a codebook that stores token information, a transformer- based predictor that predicts tokens based on the codebook and the concepts, and a decoder that decodes tokens into images.
Based on this, we propose CORTEX (Concept-Oriee) Oriented Token Explanation) to interpret VQGMs, which utilizes the Information Extractor to understand the relation among the codebook, generated images, and visual concepts. CORTEX comprises two complementary methods: a sample-level explanation method that analyzes individual token impor- tance in generated images, and a codebook-level explanation method that explores how codebook tokens are combined to represent specific concepts. By systematically identify- ing the most critical tokens and filtering out non-essential information through the Information Extractor, CORTEX provides clear, interpretable explanations of how VQGMs represent and generate specific concepts.
Let G be VQGM with a codebook C RKxd [lo...., (x-., where K is the total number of unique tokens, and f, E Rd is a d-dimensional vector represent- ing the token i. The codebook is pretrained on a large volume of image data to encode a diverse set of visual ele- ments (Esser al., 2021). The codebook maps continuous -ish-iimennional visual features into discrete tokens by serving look-up table. When generating image, the model first uses its transformer predictor to predict se- quence of m² tokens according to the input concept. It then looks up each token's 's corresponding vector in the codebook. These vectors are arranged into a new matrix E E Rdxxxxx xm which are called token-based embeddings. More details on how these token-based embeddings are extracted from the codebook can be found in Appendix A. Finally, the decoder maps this embedding E into H x H image.
Experimental results demonstrate the effectiveness of COR- TEX through comprehensive evaluation using various pre- trained classification models, including ResNet and Vision Transformer variants. Our sample-level explanation method reveals consistent patterns of concept-relevant tokens in gen- crated images, while the codebook-leve explanation method extends this understanding by discovering fundamental to- ken combinations that characterize each concept. Together, these methods enhance the transparency and controllability of VQGMs, providing valuable insights into the model's in- ternal representations and offering practical tools for down- stream generative tasks such as precise image editing, as well as identifying potential biases in model representations (e.g., revealing how tokens associated with white doctors appear more frequently than those of black doctors even with a neutral prompt, as shown in Figure 1). Our main contributions are summarized as follows:
2.2. Problem Definition
Given a pretrained VQGM with a codebook C E Rxxd Io., tt.....,--.]) -k-1] tions T = -T T5 ...., Tx)) for a set of concepts of inter- est Y = {1I, 92, ..., }. where each T C C corresponds to y. Each concept y; E Y represents a certain aspect of visual content. These concepts are specified by users, which can be contrastive (e.g.. male vs. female in gender) or par- allel (e.g., different object categories like "cat", "dog", and "bird"). For each concept yt, its corresponding explanation T,' is a combination of tokens drawn from the codebook that characterizes how the VQGM represents the concept in its generative process: T," C (o....,,k-11.
our goal is to find token-based explana-
We develop sample-level explanation method that identifies concept-relevant token combinations across generated images, providing initial insights into how VQGMs represent visual concepts.
We further introduce an codebook-level explanation method that extends our analysis to the entire code- book space, utilizing the same Information Extractor to discover fundamental token combinations that char- acterize specific concepts.
The key challenge in explaining generative models lies in identifying truly concept-relevant tokens while filter- ing out background or irrelevant information. During im- age generation, VQGM constructs token-based embeddings E E Rd using high- nigh-dimensional features that are not intuitively comprehensible to humans. For instance, when explaining the concept "dog", we need to identify tokens that capture the dog's 's distinctive features while excluding those representing contextual elements like sky or terrain.
Our experiments validate the efffectiveness of our framework in enhancing VQGMs interpretability and enabling applications such as precise image editing and bias identification.
2

---

Concept-Centrie Token Interpretation for Vector-Quantized Generative Models
3.2 Train the Information Extractor
3.3 Sample-level Explanation
"Indigo Bird"
Bird
Bird 2
Bird 3
token-baset
Pretrained Codebook
embedding
C
Transformer
Predict
Train
Information
49
SB
5
12
67
67
9B
62
VQGM Decoder
Decode
Step 0: Image/Token Generated from VQGMs
VQGM
Input
Information Extractor
Concept
3.4 Codebook-level Explanation
Token Visualization
Token Selection
Concept
y;
Token Selection Matrix
Step 1: 1:Timage Image taken set selection
ba Pd
57
pa
12
P1(K-1)
Information Extractor
Optimization
Decode
(Select top-n takens each Image
maximum TiS)
Pmr1
(=
VQGM Decoder
Gumbel- Softmax
Multiplication
Pretrained
Codebook
Eodebookk
Vimegee [Visualization original images) Step T conceuu, selection
Sm
0
Output
Arbitrary token
embeddings
B5 55
Optim ize tokens for concept V/
57
|Select too-k (k=3) frequently tokens Tingoss
Figure 2: CORTEX has three components: Information Extractor module and explanation methods. The sample-level explanation method identifies concept-releved tokens each individual image, and the codebook-level explanation method optimizes token combinations that represent the concept.
To address this challenge of extracting concept-specific kens, we develop information extractor in Section 3.2.
produce images based textual descriptions and concepts, the IEM works in reverse, mapping from images (code- book tokens) to concepts. Given token-based embedding E E [dxmmx of an image, the trained IEM f generates probability distribution over all concepts:
3. Methodology
3.1. Overview
p f (E) [0.
(1)
Our framework of concept-specific token explanation for VQGMs is shown in Figure 2. We start by developing an information extractor which maps codebook tokens de-
where p = [p,, P2, Pw. is probability distribution over the concepts Y {y11 32 ... Vn. with each p; representing the probability that concept yi is present in the embedding E. This formulation allows us quantify how strongly each concept is represented in any given token-based embedding.
concepts. Based the information extractor, velop two complemenaary methods to explain how VQGMs generate images with codebook tokens. The first method, called sample-level explanation, reveals how tokens form concepts in actual generated images. The second method, called codebook level explanation, interprets how VQGMs represent concepts codebook space. These explanations illuminate how can identify potential weaknesses in VQGMs and make targeted edits generated images.
To train the IEM, we first use VQGM to generate a training dataset by using each concept E Y as either text prompts or class conditions to generate images. For each image, we also obtain its corresponding token-based embedding E. The IEM is trained supervised classifier map beddings to their respective concepts, learning to recognize concept-specific token patterns within VQGM's output.
3.2. Information Extractor
The design of our IEM inherently aligns with the Infor- mation Bottleneck (IB) principle (Tishby et al., 2000). To accurately predict the probabilities for each concept, the
The Information Extractor Module (IEM) serves as the foun- dation of our framework. Unlike generative models that
3

---

Concept-Centric Token Interpretation for Veclor-Quantized Generative Models
IEM must identify and extract the most informative features while discarding irrelevant ones. This natural requirement for discriminative feature extraction mirrors the core idea of the IB principle, which seeks to compress input infor- mation while preserving task-relevant features. Previous studies have shown that such compression leads to more interpretable representations (Bang et al., 2021; Yang et al., 2022), as it forces the model to focus on the essential fea- tures of each concept. By leveraging this property, we can extract explanations from the IEM by contrastively ana- lyzing how different tokens represent the given concepts. Given the trained IEM f and concepts Y. we aim to find the optimal token combinations T = ; . T..,. that explain each concept:
After calculating the TISs, we identify relevant token com- binations for each concept yı:
Timuge. = Top-n(ts i 1....,,,,,, m key = TIS v:)),
Teoncopt.1 = Top-k(
U
Tiaage key = Freq).
sampled images
(5) Here, Trmage. represents the Top-n tokens with the high- est TIS for each specific image with respect to concept yt- providing an image-specific explanation. It identifies the most distinguishable tokens for generating the target image. T concept. aggregates these image-specific sets across all sam- pled images related to concept yi and selects the k most frequent tokens, offering a concept-specifie explanation.
T off Y),
3.4. Codebook-level Explanation
where á represents either our sample-level explanation method that requires input embeddings E to analyze to- ken importance in specific images, or our codebook-level explanation method that explores the entire codebook space without needing the Embedding E.
While sample-level explanation reveals how tokens are uti- lized in specific generated images, it only examines existing token patterns in generated data. To better understand the semantic meaning of model components within VQGMs, we propose an appimization-basee method that directly ex- plores the entire codebook space. For any concept y: Y, this approach directly searches for token combinations that best characterize the concept.
3.3. Sample-level Explanation
We first propose a sample-level explanation method based on token importance analysis. Formally, given token- based embedding E E RRdxmx generated by the VQGM, we compute the saliency score S; (Simonyan, 2013; Chefer et al., 2021; Selvaraju et al., 2020; Smilkov et al., 2017) of each token with respect to each concept y as:
Given concept ys. we aim to find a token selection matrix P E Rm* where m is the total number of token posi- tions in the embedding E, and K is the size of the codebook. Each row of P contains probability distribution over the K possible tokens. We employ a binary mask Mausk to specify the image regions where the tokens are optimized.
S
VEly, (E+6) +2),
Since the direct selection of discrete tokens from the codebook is not differentiable, we employ Gumbel- Softmax (Jang et al., 2016) for differentiable token selection. This technique transforms the discrete selection process into continuous, differentiable operation, enabling the use of gradient- based optimization algorithms to find the optimal token combinations.
where fg. (E) represents the prediction probability of con- cept y/ (i.e., pi), N is the number of samples, and 61 ~ N(0, o²I) with o = a (max((E)- - mi)( The resulting S, I Rdxmxx has the same dimensions as E.
We then calculate the Token Importance Score (TIS) for each token& i2 in the x m grid with respect to each con- cept yı. The TIS serves as a measure of the importance or relevance of the token to the prediction of concept ys, with higher values indicating higher importance. TIS(t,, () is computed by taking the maximum value across all d chan- nels of the gradient at the specific position corresponding to the token ty as follows:
= GumbelSSffmax (P, x C,
(6)
where GumbelSoftmax((Pr) matrix representing the selected tokens, is the temperature parameter, and C E R2 xd is the codebook matrix (details in Appendix A.4). The optimization of P is conducted by:
{0, 1), 1 is one-hot
TIS(t,, v:)) = max (k.
(4)
1sk5d
Pk+1 1 Pk (V O Mmask), C(P) = ff. EE) +
where P5 represents the position of token Lj in the m x grid, and (k) denotes the saliency score at position Py in the k-th channel for concept y:- This operation reduces the d-dimensional gradient vector at each token's 's position to scalar, representing the token's relevance to concept yt-
where a is a regularization parameter and is the learning rate. After optimization converges, we obtain the final se- lection matrix P The optimal token combination T," for
4

---

Concept-Centric Token Interpretation for Vector-Quantized Generative Models
concept ys is derived from pa for the unmasked positions:
Table 1: Comparison of IEMs prediction accuracy (%).
Teodebook, = {t& : k = arg max Pij, vZ where Mmusk,1 = 1}.
Model
Top-1
Top-3
Top-5
Top-10
j
CNN-based Extractor
53.07
71.37
77.73
84.65
(8)
ResNet-based Extractor
51.43
69.23
76.00
83.12
Transformer-base Extractor
48.71
66.74
73.63
81.43
By applying this process to each concept in Y, we can obtain the complete set of token-based explanation T Tendebook {T Fooeebook.11 1: codebookk .... .....Toooo Tcodebook,, n that shows how VQGMs represent different concepts at the codebook level.
most crucial for concept representation, evaluate method through a counterfactual approach: if these tokens are truly critical, masking them should significantly im- pact the model's ability to recognize the target concept. To ensure the robustness of our evaluation, we employ four well-established pretrained classification models as bench- marks. The selected pretrained models include variants of ResNet (He et al., 2016) (ResNet18 and ResNet50) and Vision Transformer (ViT) (Dosovitskiy, 2020) (ViT-B/16 and ViT-B/32). These models represent state-of-the-ar ap- proaches in image recognition. To elucidate the selected T from our proposed explanation method 6 of VQGMs, we use a synthetic data set generated by VQGAN (Esser et al., 2021), which encompasses the same categories as ImageNet (synthetics dataset details in Appendix A.2).
Together, these complementary approaches provide both sample and global perspectives on how VQGMs encode conceptual information in their token-based representations.
3.5. Applications of Token-based Explanations
Our framework's token-level explanations offer several prac- tical applications:
Shorteut Feature Detection. Using our sample-level quantitatively detect if short- cut features or concepts are involved in image generation. For instance, given demographic concepts and y2 (c.g., "white doctor" and "black doctor"), we can measure bias by comparing the frequencies of their corresponding to- ken sets Temncop.1 and Tromcopr,2 images generated from neutral prompts (e.g.. "doctor""). The systematic differ- ences in these frequencies provide a statistical measure of the model's inherent biases.
explanation Teancept
In experiment, the IEM f is trained as image classi- fier with 1, 000 ImageNet categories. We train 3 IEMs with different architectures: (1) CNN-based Extractor (CE), (2) ResNet-based Extractor (RE), and (3) Transformer- based Extractor (TE). IEMs' architectures can be found in the Appendix A.3. All IEMs take token-based embedding E 256 16 input and output probability distribu- tions over 1000 ImageNet labels. Based on Table 1, the per- formance of our models, with Top-1 accuracies of 53.07%, 14%, and 48.71% for CNN-based, ResNet-based, and Transformer-basee architectures, respectively, is significant considering the potential inaccuracies introduced by VQ- GAN image generation process. Despite this challenge, the high Top-5 accuracies (exceeding 73%) demonstrate that our IEMs effectively capture the relationship between token patterns and image labels.
Targeted Image Editing. Our codebook-level explana- tion method enables precise concept manipulation by op- timizing token selection matrix P within a specified mask region Mmakk Given a target concept y/, we optimize the tokens in the masked region to maximize fy, (E), effec- tively steering the image representation towards concept Yi while preserving tokens outside M The optimized token set Te then provides a controlled way edit local image regions according to the target concept while maintaining the integrity of surrounding content.
4. Experiments
4.2. Sample-level Explanation Evaluation
Our proposed framework aims explain cppcept- information in VQGMs on a diverse range of concepts. The experiments are designed to verify that the token combina- tions T selected by CORTEX are indeed the most relevant to the concepts Y being explained.
Experimental Design. In this part, our experiments aim to validate that the token combinations Tiagge and Texncepp iden- tified by our sample-level explanation method are indeed the most relevant and representative of specific concepts. In our image-specific analysis, given each concept yi Y, we define Timage.. as the set of the Top-n tokens with the highest TIS in images containing concept yt, where n ranges from 5 to 50 in increments of 5. For each token in Tinge. mask the corresponding regions in these images and mea- sure the change in softmax probability for concept across four pretrained models: ViT-B/16, ViT-B/32, ResNet18, 8, and ResNet50. As a baseline, we randomly select n tokens and mask the associated regions in the same images to compare
4.1. Experimental Setup
To validate our proposed explanation method, we adopt the methodology of treating each category in the ImageNet chal- lenge dataset as distinct concept y: to be explained (Chefer et al., 2021; Binder et al., 2016; Simonyan, 2013; Deng et al., 2009). To validate that our selected tokens are indeed the
5

---

Concept-Centric Token Interpretation for Vector- Vector-Quantized Generative Models
CE
CE
Average Probabilities
0.45
RE
0.40
RE
0.40
TE
Maaaas Probabilic
0.35
TE
Baseline
0.30
Baseline
0.35
0.25
30.20
0.15
0.30
0.25
0.05
10
20
30
40
50
10
20
30
40
50
Number of Masked Takens
Number of Masked Tokens
ViT-B/16
ViT-B/32
Figure 4: Image-specific sample-level explanation method evalua- tion results: average probabilities vs. number of masked tokens.
correspond to specific, concrete visual features within each concept. For instance, the consistent highlighting of eyes, red crowns, and other distinctive features across multiple images of the same class indicate that these tokens can effec- tively represent meaningful, class-specific characteristics.
Quantitatively, Figure 4 shows the average change in soft- probability for specific labels high-TIS tokens. Across two SOTA pretrained ViT mod- els, our method consistently leads to steeper decline in probability compared to random selection, demonstrating its effectiveness in identifying tokens crucial to concept rep- resentation. Notably, both CNN-based and ResNet-based IEMs exhibit similar declining trends, suggesting that dif- ferent models attend to similar tokens for specific concepts, while the Transformer-basee IEM shows relatively lower performance. This lower performance of the Transformer-
mask from to 50
Figure Token visualization for different concepts using our sample-level explanation method. Each row shows images of distinct concept. Red boxes highlight high-TIS tokens, revealing consistent identification of class-specifee features (e.g.. eyes, neck, red crowns, flame) across images.
the effect with masking tokens in Timuge.i"
In our concept-specific analysis. given each concept yi. we first identify the Top-n (n 20) highest-TIS tokens in each training image containing concept yı to form individ- ual Timage,i i sets. From the union of these sets across all training images containing the concept y/i, we select the Top-k (k 100) most frequent tokens to form concept. We compare this with frequency-band based baseline of selecting the Top-100 most frequent tokens in all images containing the concept Yi without using our IEM f. We then mask the corresponding patches of selected tokens in test im- ages containing the concept yı and measure the change in the probability of the target concept. Our analyses use the 1000 ImageNet classes as our set of concepts of interest Y = y1 1/2 y1000. allowing us to evaluate our method's effectiveness across diverse range of visual concepts.
based IEM can be attributed to its weaker classification capabilities, which may result in less accurate information learning and token importance estimation. These results validate our sample-level explanation method's ability to identify label-relevant features and provide interpretable insights into VQGMs.
Concept-specifie Evaluation Results. Table 2 presents the concept-specific sample-level evaluation results across different pretrained models. When masking tokens based our Teoncops, we mask fewer tokens on average (n1 42. 176, n2 = 40.629, and 733 45.552 for the CNN-based, ResNet-based and Transformer-basee extractors, respec- tively) compared to the baseline (n, 64.166). Despite masking smaller regions, our CNN-based and ResNet-based methods achieve superior performance- for instance, with ResNet50, they decrease accuracy by 46.6% and 46.8% and probability by 0.403 and 0.404, respectively, compared to baseline's 's 41.9% and 0.365. While our Transformer-basee extractor shows comparable performance to the baseline (e.g., 42.4% vs 42.2% accuracy drop for ResNer18) 18), it also achieves this with fewer masked tokens. These significant changes in prediction accuracy with fewer masked tokens indicate that our Toonoep successfully identifies regions con-
Image-specific Evaluation Results. Figure 3 demon- strates the efficacy our sample-level explanation method in identifying concept-related features across multiple im- ages (more results can be found in Appendix A.7). Each row in the figure represents a distinct label. For each la- bel, we present 4 different images. Within each image, we highlight a specific token in Tiiger Image, using a red bounding box. These visualizations demonstrate our sample-level explanation method's s ability to focus on tokens that often
6

---

Concept-Centric Token Interpretation for Vector-Quantized Generative Models
Table 2: concept-speciffe sample-lever explanation evaluation results (Ace: prediction accuracy, P: probability, n: number masked tokens, AA: change in accuracy, A: change in probability).
Pretrained Model
CE
RE
TE
Baseline
Acc
P
A.A
AP 4
AA 4
APL
n
AA 4
AP 4
AA
AP J
ResNet18
55.6%
0.452
-45,4%
-0.381
-45.5 %
-0.382
-42.4%
-0.358
-42.2%
-0.356
ResNet50
56.1%
0.472
-46.6%
-0.403
-46.8%
-0.404
-43.1%
-0.374
-41.9%
-0.365
42.176
40.629
45.552
64.166
VIT-B/16
59.0%
0.472
-9.50%
-0.112
-9.60%
-0.113
-7.01%
-0.089
-7.30%
-0.090
VIT-B/32
58.0%
0.442
-36.0%
-0.289
-36.0%
-0.289
-32.7%
-0,260
-33.2%
-0.264
taining erucial information for concept yi- This demon- strates that our IEMs effectively fulfill their role in the Infor- mation Bottleneck Tramework-identinne iddntiffing and preserving the most essential concept-specific tokens while filtering out irrelevant ones, regardless of the architecture used.
For color bias analysis, we used the prompts "A black doctor in the hospital" and "A white doctor in the hos- pital". For gender bias anal- ysis, similar prompts were used to generate images rep- resenting male and female doctors. We generated 10,000 images for each prompt, di- viding them into training (8,000), validation (1,000), and test (1,000) sets. These images were used to train our IEM as a binary classifier, achieving high classification accuracy, as shown in Table 3. Using the trained IEMs, we applied our sample-level explanation method to identify tokens that characterize each concept (color & gender). This provided us with a set of tokens that could serve as indicators of potential biases in the model's representations.
Table 3: IEM Classification Accuracy
Bias Type
Accuracy (%)
Gender Color
99.7
99,95
4.3. Discussion
Our counterfactual masking experiments demonstrate that removing the tokens identified by CORTEX substantially degrades pretrained classification models' performance, in- dicating that these tokens indeed capture the most discrim- inative features for each target concept. By contrast, ran- domly masking the same number of tokens leads to rela- tively smaller performance drops. This comparative analysis confirms the effectiveness of our sample-level explanation method in pinpointing concept-critical tokens, thereby val- idating the interpretability and reliability of CORTEX's
token selections.
Table 4: Comparison of Mean Frequency (per image) and Cliff's 5 for different attributes.
5. Application of Explanations
Mean Frequency (per image)
Cliff's 6
This section demonstrates practical applications of our COR- TEX framework through two key use cases. First, we show how CORTEX can be used to detect and quantify biases in text-to-image models by analyzing concept-specific token distributions. Second, we illustrate CORTEX's capability in targeted image editing by optimizing specific concept- relevant tokens.
Top-5
Top-10
Top-20
Top-5
Top-10
Top-20
Color
White
4.80
6.61
8.55
0.311
0.418
0.456
Black
2.35
2.18
2.32
Gender Male
7.37
8.27
5.65
0.359
0.394
0.264
Feraae
3.27
3.43
3.45
5.1. Bias Detection in Text-to-lmage Models
Bias analysis with the Neutral Prompt. In the second phase, we generated 2, 000 images using the neutral prompt "A doctor in the hospital" to investigate potential biases. For each demographic attribute, we analyzed the frequency of tokens from their concept-specific token sets (T comeee
To demonstrate CORTEX's 's capability in detecting biases within a specific text-to-image model, Dalle-mini (Dayma et al., 2021), we conducted experiments focusing two common types of biases; gender and color representation in professional settings. We specifically examined the case of doctor representations, as professional representation bias has been a significant concern in generative models.
Teomcept. femeeet.femmtee femate. T comncett. and Teimeept ). Following Equation 5, we identify the Top-n tokens (n 5, 10. 20) with highest TIS in each training image, then select Top-k (k 10) most frequent tokens to form each concept-specific set. As shown in Table 4, using Cliff's 's 8 (detailed in Ap- pendix A.6), tokens from T concept,whitee appear nearly four times more frequently than Teoncepu.bacck (8.55 vs 2.32 to- kens per image when n = 20, 8 = 0.456). For gender, tokens from T Te appear more than twice as fre-
Experimental Setup. Our experiment consisted of two phases. In the first phase, we generated training data us- ing specific prompts to establish baseline representations.
7

---

Concept-Centric Token Interpretation for Vector-Quantized Generative Models
Original Image
Step 1
Step 500
Step 1000
Step 3000
Target Example
Figure 5: Codebook- -lvell optimization-bast based image editing process.
quently Tconccpt. female (8.27 3.43 tokens per image when n = 10, 8 = 0.394). This systematic bias in token fre- quencies directly corresponds to observable patterns in the model's outputs, outputs,, when given the neutral prompt "a doctor in the hospital", the model consistently generates images of white male doctors. These results demonstrate COR- TEX's effectiveness detecting and quantifying biases in text-to-image models.
(Van Den Oord et al., 2017) pioneered the of discrete latent codes for efficient image reconstruction, overcom- ing "posterior collapse" issues in VAEs. DALL-E (Ramesh et al., 2021) extended this to text-to-imaae generation, while VQGAN (Esser et al., 2021) and ViT-VQGAN (Yu et al., 2021) enhanced image quality through perceptual and adver- sarial objectives. In video generation, MAGVIT (Yu et al., 2023), VideoPoet (Kondratyuk al., 2023), and Va (Jin et al., 2024; 2023) applied vector quantization for spatial- temporal modeling and multimodal learning. FSQ (Mentzer et al., 2023) simplifies vector quantization with efficient scalar quantization, while MAGE (Li et al., 2023) combines masked token modeling with generative objectives to im- prove image representation learning. Our work builds upon these VQGMs, offering novel approach to interpreting dis- crete tokens and providing insights into visual information encoding and utilization.
5.2. Image Editing
Figure shows the visualization of the image editing pro- cess using our token selection codebook-level explanation method proposed in section 3.4. By applying M to con- strain optimization to only the tokens corresponding to the bird's head region, these sequences demonstrate the gradual transformation of one bird species into another while pre- serving other regions. Using the target bird species as con- cept y/, our method optimizes T within the masked region, resulting in progressive changes to features like beak shape and color. This controlled transformation illustrates our method's ability to identify and manipulate concept- specific tokens effectively.
Vision model explainability. Traditional approaches to explaining vision models primarily fall into two categories: heatmap-based methods (Sundararajan et al., 2017; Sel- varaju et al., 2020; Binder et al., 2016; Gandelsman et al., 2023; Chefer al., 2021; Wang et al., 2020; 2024; Bog- gust et al., 2023; Jetley et al., 2016; Gupta et al., 2022; Du et al., 2018), which highlight influential image regions, and optimization- han et al., 2009; Yosinski et al., 2015; Nguyen et al., 2015; Simonyan, 2013; Nguyen et al., 2016a; FEL et al., 2023;
5.3. Discussion
Our sample-level explanation detects shortcut features or biases by analyzing how frequently certain concept tokens appear under neutral prompts. Meanwhile, the codebook- level explanation enables localized image editing by selee- tively refining tokens in a chosen region, providing both interpretability and precise control in generative modeling.
methods (Nguyen et al., 2016b; Er-
Goh et al., 2021; Zimmermann et al., 2021), which generate synthetic inputs to maximize specific activations. While insightful, these pixel-level approaches are limited in ex- plaining VQGMs. Our CORTEX approach extends this to the token level, providing concept-specific explanations of discrete latent representations. This offers deeper insights into VQGMs' generative processes, bridging traditional and modern explainability.
6. Related Work
Vector quantization in computer vision. Vector quanti- zation has been instrumental in advancing image generative models (Gray, 1984; Nasrabadi & Feng, 1988). VQ-VAB
8

---

Concept-Centric Token Interpretation for Vector-Quantized Generative Models
7. Conclusion
and Samek, W. Layer-wise relevance propagation for neural networks with local renormalization layers. In Ar- tificial Neural Networks and Machine Leaming-ICANN 2016: 25th International Conference on Artificial Neu- ral Networks, Barcelona, Spain, September 6-9, 2016, Proceedings, Part 25, pp. 63-71. Springer, 2016.
In this paper, we introduced CORTEX, a framework that interprets VQGMs through concept-specific token analy- sis guided by the Information Bottleneck principle. Our experiments show that CORTEX effectively identifies to- kens critical to concept representation, enabling not only fine-grained image editing but also systematic bias detec- tion by examining token distributions for sensitive attributes. By revealing how codebook tokens capture high-level con- cepts, CORTEX enhances transparency and controllability in generative models. Future work includes extending COR- TEX to a broader range of VQGMs and vision-language models (Shi et al., 2025), such as video-based models, and further exploring its potential usability (Wu et al., 2024) in model improvement.
Blei, D. M., Ng. A. Y., and Jordan, M. I. Latent dirichlet allocation. Journal of machine Learning research, 3(Ja)): 1022, 2003.
Boggust, A., Suresh, H., Strobelt, H., Guttag, J., and Satya- narayan, A. Saliency cards: A framework to characterize and compare saliency methods. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pp. 285-296, 2023.
Chefer, H.,, Gur, S., and Wolf, L. Transformer interpretabil- ity beyond attention visualization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 782-791, 2021.
Acknowledgements
The work in part, supported by NSF (#I1S-2223768). The views and conclusions in this paper are those of the authors and should not be interpreted as representing any funding agencies.
Dayma, B., Patil, S., Cuenca, P., Saifullah, K., Abraham, T., Lê Khe, P., Melas, L.,, and Ghosh, R. Dall-e mini, 7 2021. URL URL https: /ttpt/gghhh//////// con commboriise borrisdama// da dalle-mini.
Impact Statement
Deng. J., Dong, W., Socher, R., Li, L.-J,, Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vi- sion and Pattern Recognition, pp. 248-255, 255, 2009. doi: 10. 10.1109/CVPR.2009552222 5206848.
Our research has significant implications for understanding and detecting biases in AI systems, particularly in generative models. Our experimental results revealed concerning demo- graphic disparities in image generation, specifically show- ing systematic biases in professional representation where tokens associated with certain demographics appeared sig- nificantly more frequently than others in neutral prompts. These findings highlight the importance of developing ro- bust methods for detecting and quantifying such biases in AI systems.
Dosovitskiy, A. An image is worth 16x16 words: Trans- formers for image recognition scale. arXiv preprint arXiv:2010.1 11999, 2020.
Du, M., Liu, N., Song, Q., and Hu, X. Towards explanation of dnn-based prediction with guided feature inversion. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp- 1358-1367, 2018
While our work contributes to model interpretability and bias detection, we acknowledge that identifying biases is only the first step toward addressing broader issues of fair- ness and representation in AI systems. We encourage future work build upon these findings to develop more equi- table generative models, while remaining mindful of both the capabilities and limitations of interpretability tools in addressing societal challenges.
Erhan, D., Bengio, Y., Courville, A., and Vincent, P. Visual- izing higher-layer features of a deep network. Technical Report, Univeristé de Montréal, 01 2009.
Esser, P., Rombach, R., and Ommer, B. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12873-12883, 2021.
References
vision
Bang, S., Xie, P., Lee, H., Wu, W., and Xing, E. Explaining a black-box by using a deep variational information bot- tleneck approach. In Proceedings of the AAA/ conference on artificial intelligence, volume 35, pp. 11396-11404, 2021.
FEL. T., Boissin, T., Boutin, V., PICARD, A., Novello, P., Colin, J., Linsley, D., ROUSSEAU, T., Cadene, R., Goetschalckx, L., et al. Unlocking feature visualization for deep network with magnitude constrained optimiza- tion. Advances in Neural Information Processing Systems, 36:37813-37826, 2023.
Binder, A., Montavon, G., Lapuschkin, S., Müller, K.-R.,
9

---

Concept-Centric Token Interpretation for Vector-Quantized Generative Models
Gandelsman, Y., Efros, A. A., and Steinhardt, J. Interpreting clip's image representation via text-based decomposition. arXiv preprint arXiv:2310.0591 2023.
Nasrabadi and Feng. Vector quantization of images based upon the kohonen self-organizing feature maps. In 1988 International Conference on Neural Networks, pp. 101- IEEE, 1988.
IEEE
Goh, G., Cammarata, N., Voss, C., Carter, S., Petrov, M., Schubert, L., Radford, A., and Olah, C. Multimodal neurons in artificial neural networks. Distill, 6(3):e30, 2021.
Nguyen, A., Yosinski, J., and Clune, J. Deep neural net- works are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pp. 427-436, 2015.
Gray, R. Vector quantization. IEEE Assp Magazine, 1(2): 4-29, 1984.
Nguyen, A., Dosovitskiy, A., Yosinski, J., Brox, T., and Clune, J. Synthesizing the preferred inputs for neurons in neural networks via deep generator networks. Advances in neural information processing systems, 29, 2016a.
Gupta, A., Saunshi, N., Yu, D., Lyu, K., and Arora, S. New definitions and evaluations for saliency methods: Stay- ing intrinsic, complete and sound. Advances in Neural Information Processing Systems, 35:33120-3314 2022.
Nguyen, A., Yosinski, J., and Clune, Multifaceted feature visualization: Uncovering the different types of features learned by each neuron in deep neural networks. arXiv preprint arXiv: 1w0.iv::600... 060.00002066666 2016b.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE conference computer vision and puttern recognition, pp. 770-778, 2016.
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Rad- ford, A., Chen, M., and Sutskever, I. Zero-shot text-to- image generation. In International conference on mu- chine learning, pp. 8821-883) Pmlr, 2021.
Jang. E., Gu, S., and Poole, B. Categorical repa- rameterization with gumbel-softmax. arXiv preprint arXiv:1611.0114. 144, 2016.
Jetley, S., Murray, N., and Vig, E. End-to-end saliency mapping via probability distribution prediction. In Pro- ceedings of the IEEE conference on computer vision and pattern recognition, pp. 5753-5761, 2016
Romano, J. and Kromrey, J. Appropriate statisties for or- dinal level data: Should we really be using t-test and cohen's d for evaluating group differences on the nsse and other surveys? 01 2006.
Selvaraju. R. R., Cogswell, M.. Das, A., Vedantam, R., Parikh, D., and Batra, D. Grad-cam: visual explana- tions from deep networks via gradient- based localization. International journal of computer vision, 128:336-359, 2020.
Jin, Y., Xu, K., Xu, K., Chen, L., Liao, C., Tan, J., Mu, Y., et al. Unified language-vision pretraining in Ilm with dynamic discrete visual tokenization. arXiv preprint arXiv:2309.0400 04669, 2023.
Jin, Y., Sun, Z. Xu, K., Chen, L., Jiang, H., Huang, Q., Song, C., Liu, Y., Zhang, D., Song, Y., et al. Video-lavit: Uni- fied video-language pre-training with decoupled visual- motional tokenization. arXiv preprint 0r22v1240..03161, 2024.
Shi, Y., Li, Q., Sun, J., Li, X., and Liu, N. Enhancing cognition and explainability of multimodal foundation models with self-synthesized data. In The Thirteenth International Conference on Learning Representations, 2025.
Kondratyuk, D., Yu, I.., Gu, X., Lezama, J., Huang, J., Hormung, R., Adam, H., Akbari, H., Alon, Y., Birodkar, V., et al. Videopoet: A large language model for zero- shot video generation. arXiv preprini arXiv:2312. 14125, 2023.
Simonyan, K. Deep inside convolutional networks: Visu- alising image classification models and saliency maps. arXiv preprint arXiv: 6034, 2013.
Smilkov, D., Thorat, N., Kim, B., Viégas, F,, and Watten- berg, M. Smoothgrad: removing noise by adding noise. 2017.
Li, T., Chang, H., Mishra, S., Zhang, H., Katabi, D., and Krishnan, D. Mage: Masked generative encoder unify representation learning and image synthesis. In Proceed- ings of the IEEE/CVF Conference Computer Vision 2023.
arXiv preprint arXiv:1706.0388
Sundararajan, M., Taly, A., and Yan, Q. Axiomatic attribu- tion for deep networks. International conference on machine learning, pp. 3319-3328. PMLR, 2017.
and Pattern Recognition, pp.
Mentzer, F., Minnen, D., Agustsson, E.,, and Tschannen, M. Finite scalar quantization: Vq-vae made simple. arXiv preprint Xr099 15505, 2023.
Tian, K., Jiang, Y., Yuan, Z., Peng, B., and Wang, L. Visual autoregressive modeling: Scalable image generation via next-scale prediction. 2024.
10

---

Concept-Centric Token Interpretation for Vector-Quantized Generative Models
Tishby, N., Pereira, F. C., and Bialek, W. The informa- tion bottleneck method. arXiv preprini physics/0004557, 2000.
Van Den Oord, A., Vinyals, O., et Neural discrete rep- resentation learning. Advances in neural information processing systems, 30, 2017.
Wang, H., Wang, Z., Du, M., Yang, F., Zhang, Z., Ding, S., Mardziel, P., and Hu, Score-cam: weighted visual explanations
convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 24-25, 2020.
Wang, H., Tan, S., and Wang. H. Probabilistic conceptual ex- plainers: Trustworthy conceptual explanations for vision foundation models. arXiv preprint arXiv:2406.12649, 2024.
Wu, X., Zhao, H., Zhu, Y., Shi, Y.. Yang. F., Liu, T., Zhai, X., Yao, W., Li, J., Du, M., Usable xai: 10 strategies towards exploiting explainability in the Ilm era. arXiv preprint 2rXiv:2403.08946, 2024.
Yang. T., Wang. Y., Lu, Y., and Zheng, N. Visual concepts tokenization, Advances in Neural Information Processing Systems, 35:31571-31555 2022.
Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., and Lipson, H. Understanding neural networks through deep visual- ization. arXiv preprint 150.006599 2015.
Yu, J., Li, X., Koh, J. Y., Zhang, H., Pang, R., Qin, J., Ku, A., Xu, Y., Baldridge, J., and Wu, Y. Vector-quantized image modeling with improved vqgan. arXiv preprint arXiv:21 10. 10.04627, 2021.
Yu, L., Cheng, Y., Sohn, K., Lezama, J., Zhang, H., Chang, H., Hauptmann, A. G., Yang, M.-H., Hao, Y., Essa, I., et al. Magvit: Masked generative video transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Putlern Recognition, pp. 10459-10469, 2023.
Zimmermann, R. S., Borowski, J., Geirhos, R., Bethge, M., Wallis, T., and Brendel, W. How well do feature visual- izations support causal understanding of enn activations? Advances in Neural Information Processing Systems, 34: 11730-11744, 2021.

---

Concept-Centric Token Interpretation for Vector-Quantized Generative Models
A. Appendix
A.1. Codebook Extraction in VQGMs
(ector-Quantized Generative Models (VQGMs) utilize codebook as a learned dictionary of discrete tokens to represent high-ding nigh-dimensional visual features. During the encoding process, input images are passed through an encoder to produce continuous latent representations, which then quantized by mapping each vector its nearest neighbor in the codebook using vector quantization. The resulting sequence of discrete token indices forms the compressed representation of the image.
Formally, let z E Idd be a latent vector produced by the encoder, and C = {to. tt....,tt_]] tx-1] be the codebook with K codebook entries, where each t; E TRd. The quantized output is obtained via:
arg min I EC
t
This process is repeated for each location in the latent grid (c.g.. 16 x 16), resulting in grid of token indices that compactly represent the input image. These tokens are then used as input to the decoder for image reconstruction or generation.
A.2. Symthetic Dataset
To evaluate our proposed methods, we utilize synthetic dataset generated by VQGAN (Esser et al., 2021). Using synthetic dataset rather than real-world images offers two key advantages: (1) it provides direct access to the token-based embeddings E during the generation process, enabling us to analyze how the model encodes concepts at the token level, and (2) ensures that the data perfectly matches the token distribution learned by the generative model, eliminating potential distribution gaps between training and evaluation. This synthetic dataset encompasses all ImageNet categories, allowing us directly examine how the generative model utilizes tokens encode concept-specific information.
The dataset consists of:
1,000,000 training images
300,000 validation images
50,000 test images
The images are evenly distributed across all ImageNet categories, resulting in 1,000, 300, and 50 images per category in the training, validation, and test sets, respectively. Each generated image has resolution of 256 x 256 pixels and is represented by 16 x 16 grid of tokens. During the generation process, we obtain both the generated image and its corresponding token-bacce based embedding E, enabling direct analysis of the relationship between tokens and visual concepts.
A.3. Information Extractor
This appendix provides details on the structure of two Information Extractor: CNN-based model and Resnet-based model.
1. CE: CNN-BASED EXTRACTOR
The CNN-based Extractor (CE) is a convolutional neural network designed for image classification. The model comprises two main blocks, each containing four convolutional layers (conv to conv 1_4 and conv2_1 to conv2.4). Each convolutional layer utilizes 512 filters with 3 x kernel size, stride of 1, and padding of 1, followed by batch normalization and ReLU U activation. Max pooling (2 2 kemel, stride 2) is applied after each block. The network concludes with three fully connected layers: the first transforms 512 x 4 input features 4096 output features, the second maintains 4096 features, and the final layer maps to the number of classes. Additional features include batch normalization and ReLU activation after the first two fully connected layers, with dropout (0.5) applied after the first fully connected layer.
A.3.2. RE: RESNET-BASED EXTRACTOR
The ResNet-based Extractor (RE) an advanced model incorporating residual connections and Squeeze-and an--xxctatioion (SE) blocks. The network consists of two main layers, each containing 3 residual blocks. Each residual block comprises two
12

---

Concept-Centric Token Interpretation for Veclor-Quantized Generative Models
convolutional layers (3 kernel, maintaining channel size) with batch normalization and ReLU activation, shorteut connection, and SE block for channel-wise attention. The SE block employs global average pooling followed by two fully connected layers with reduction and expansion, using sigmoid activation for generating attention weights. The model concludes with global average pooling to reduce spatial dimensions, followed by two fully connected layers: 512 to 2048 features, and 2048 to the number of classes. Batch normalization and ReI.U activation are applied after the first fully connected layer, with dropout (0.5) implemented.
A.3.3. TE: TRANSFORMER- EXTRACTOR
The Transformer-based Extractor (TE) processes input features [B, 256, 16, 16] by reshaping them into 256 sequences of 16x x 16 patches. These patches are projected to 512 dimensions through linear layer with LayerNorm and GELU activation. A learnable class token is prepended, and learnable positional embeddings are added after scaling by V512. The backbone comprises 12 encoder layers with Pre-LN and dual-stream normalization. Each encoder contains 8-head self-attention with dropout (0.2) and a feed-forward network (512- + 2048 - 512) with GELU activation. Both attention and feed-forward outputs undergo BatchNorm before residual connections. For classification, the class token passes through LayerNorm and a two-layer MLP (512- + 1024 with LayerNorm, GELU, and dropout), followed by projection to class logits.
Both CE, RE, and TE are designed process input tensors with 256 channels and spatial dimensions of 16 x 16 tokens. While the Transformer-base deliberate design choice to maintain lightweight model structure. Given IEM's requirement for computational efficiency, we implemented a relatively shallow Transformer architecture, which may constrain its potential to surpass CNN based models this specific token pattern recognition task. Nevertheless, the overall performance demonstrates both architectures' capability to effectively learn meaningful representations from token-based input embeddings E.
architecture demonstrates marginally lower performance metrics, this outcome aligns with
A.3.4. TRAINING SETTINGS
Training setting for CE and RE. These information extractors were trained using a batch size of 256 for 80 epochs, with the task involving classification across 1000 classes. We employed the Adam optimizer with initial learning rate of 0.001 and weight decay of le 4. To adjust the learning rate during training, we implemented a StepL.R scheduler, which decreased the learning rate by a factor of 0.1 every 20 epochs. The loss function used for training was Cross Entropy Loss. Our experimental setup allowed for the training of multiple model architectures under consistent conditions, enabling fair comparison of their performance.
Training setting for TE. Due to the distinct characteristics of transformer-based architectures, we adopted a specialized training strategy different from CNN and ResNet approaches. Specifically, we employed Adam W optimizer with weight decay le-4, combined with a hybrid learning rate schedule consisting of linear warmup phase (10% of total iterations) followed by cosine annealing. The initial learning rate was to le-3. For training stability and efficiency, we implemented mixed-precision training using automatic mixed precision (AMP) with gradient scaling. The model was trained for 80 epochs with batch size of 256. This configuration addresses the optimization challenges specific to transformer architectures while maintaining stable gradient flow throughout the training process.
A.4. Gumbel-Softmax Technique for Token Selection Optimization
In our implementation, we employ the Gumbel-Softmax technique to optimize the selection of tokens from the codebook. This method enables differentiable sampling from a discrete distribution, which is essential for our gradient-based optimiza- tion process. The core of our approach involves a matrix P of shape (256, 16384), where each row represents a probability distribution over the codebook tokens.
The Gumbel-Softmax approximation operates by adding Gumbel noise the logits (log probabilities) derived from at each optimization step. The Gumbel-Max trick states that for a categorical distribution with class probabilities pi. sampling can be performed as:
(log(), + gi)
where g are i.i.d. samples from Gumbel(O, 1) distribution.
13

---

Concept-Centric Token Interpretation for Vector-Quantized Generative Models
We then apply softmax function with temperature parameter т to these noisy logits:
exp((log(p)) + 9:)//))
yi Σ, exp((log(p)) 0;)/r)
(10)
In the "hard" version of this technique, we convert this soft distribution a one-hot vector by selecting the maximum value:
Shard onchot(argmax,
(1I)
The final output is then:
stop crop--raddent(/amn y)
(12)
This process allows us sample discrete tokens while maintaining differentiability, thereby enabling backpropagation through the sampling process.
A key feature of the Gumbel-Softmin is the temperature parameter т, which controls the discreteness of the samples. As approaches zero, the samples become more discrete, closely approximating one-hot vectors. Conversely, increases, the distribution becomes smoother and more uniform.
Throughout the optimization process, we update the matrix based on the gradients computed through this differentiable sampling procedure. The gradient of the Gumbel-Softuax estimator with respect to the logits is:
Oy: i llg(Pk)
(13)
where & is the Kronecker delta.
By utilizing this approach, we optimize the selection of discrete tokens from the codebook in a manner compatible with based optimization methods. This compatibility is crucial for our objective of maximizing the activation of target labels in our classification model.
The Gumbel-Softmax technique thus serves as bridge between the discrete nature our token selection problem and the continuous optimization landscape required for effective gradient- based learning. It allows us backpropagate through the discrete sampling operation, enabling end-to-end training of our model while maintaining the ability to produce discrete outputs during inference.
A.5. Codebook-level Explanation Evaluation
Experimental Design. To validate that codebook-level explanation method effectively identify concept-specific token combinations Tecdebook- provide comprehensive quantitative experimental results. Due to the computational intensity of multi-step optimization in codebook-level explanation, we conduct our evaluation using CNN-based (CE) and ResNet-basse based (RE) IEMs, as the Transformer-batt prohibitively long optimization times. We conduct experiments using 10 bird categories (500 images in total) from the synthetic dataset. These bird images can be generated by VQGAN in high quality, and the 4 pretrained models achieve high prediction accuracies on these 500 images, ranging from 84. 6% 90.2% (refer to Acconig Table 5).
IEM (TE) has significantly more parameters and would require
We pair 10 bird categories into groups, each category serving both original and target labels. For each category, we utilize 50 images from the test set. In experimental design, when images from a category paired category becomes the target label, and vice versa. For instance, in the pair (Goldfinch, Water Ouzel), when Goldfinch images are being optimized, Water Ouzel serves as the target label, and conversely, when Water Ouzel images are used as original images, Goldfinch becomes the target label. This reciprocal design is applied consistently across all five pairs depicted in Figure 6. Every image in our dataset undergoes optimization as an original image, with its paired category serving as the target label. this experiment, each concept yi is a specific bird category.
the original images, its
14

---

Concept-Centric Token Interpretation for Vector-Quantized Generative Models
Goldfinch
House Finch
Snowbird
Indigo Bird
Robin
Water Ouzel
Chickadee
Magpie
Jay
Bulbul
Figure 6: codebook-level explanation method experiment design
The optimization process begins with coken-based embeddings E from the original label images, optimizing towards the target label, which is the concept to be explained. We focus on fixed 4 x 4 region within the 16 x 16 token grid, limiting i to 16 tokens (only 1/16 total 256 tokens). We evaluate the changes in softmax probabilities for both the original and target labels across four pretrained models.
We explore two optimization methods, both aiming to maximize the activation of a target bird label (the concept to be explained): 1) Token selection optimization (our method): We optimize token selection matrix, which represents the probability of selecting each token from the codebook for specific positions in the target region. 2) Embedding optimization (baseline): We directly optimize the d-dimensional embedding in the target region. After optimization, quantization (Gray, 1984) to map each optimized embedding vector to the nearest token in the codebook, minimizing the Euclidean distance. Optimized embeddings were decoded via VQGAN to generate images. To measure the effectiveness of each method optimization method, we calculate the change probabilities for the original and target labels before and after optimization:
apply vector
A Porig =
Ponig (optimized) - Pong (initial).
(14)
Prgg = Piarg (optimized) Pragg (initial).
Evaluation Results. Table 5 shows the results of our optimization methods across different models. The Token Selection method consistently outperforms the Embedding Optimization baseline by both reducing the original label probability (APong) and increasing the target label probability (APrurg). For instance, in the ResNet18 model with CNN-based extractor, our method decreases the probability the original label by 36.5% (from 0. 0.795 to 0.505) and increases the probability of the target label about times (from 0.016 to 0.178) compared the initial probability. In contrast, the Embedding Optimization baseline achieves 28.2% decrease in the original label probability and 7.9-fold increase in the target label probability. This shows that method surpasses the baseline by achieving greater reduction in the original label and more significant increase in the probability of the target label. Similar improvements are observed in another information extractor. In the ViT-B/16 model with ResNet-basee extractor, our Token Selection method reduces the probability of the original label by 31.7% and increases the probability of the target label by over 40 times, significantly outperforming the baseline. These results indicate that our Token Selection method effectively identifies the most important tokens contributing to the target label. By directly optimizing the token selection matrix end-to-end, it finds the token combination that
15

---

Concept-Centric Token Interpretation for Vector-Quantized Generative Models
maximally activates our information extractor /. In contrast, the Embedding Optimization method optimizes embeddings and then maps them back to the nearest tokens in the codebook, which may result in suboptimal token combinations due the lack of end-to-end optimization.
These results validate the effectiveness of our codebook-level explanation method in identifying and manipulating class- relevant features within VQGMs. The substantial increases in target label probabilities, often by more than an order of magnitude, demonstrate the method's potential for enhancing model interpretability and its applicability targeted image editing tasks
A.6. Cliff's 8
Cliff's is a non-parametric effect size measure that quantifies the degree of overlap between two groups of observations. For two groups X and Y with sizes N₂ and Ny. it is calculated as:
N [T vU) + +0.5T(r, yU)] N.N, 6 Σ E
(15)
N₂
where (-) is the indicator function. The value ranges from -1 to 1, with larger absolute values indicating stronger effect sizes. Following Romano et al. (Romano & Kromrey, 2006), the effect size can be interpreted as: negligible for [6] < 0.147, small for [6] < 0.33, medium for [ō] < 0.474, and large otherwise. In our analysis, X and Y represent the occurrence frequencies of specific concepts (T*) across different images. positive 8 indicates significant difference the activation frequencies between two groups, suggesting potential bias in concept representation.
A.7. More Visualizatoin Results
Figure presents additional sample-level explanation visualization results, complementing the analysis provided in Sec- tion 4.2. The figure is structured into two distinct sets of four rows each, each focusing on specific token for a particular category. This approach demonstrates the efficacy of sample-level explanation method in identifying concept-speciffs features across multiple images. The first four rows showcase visualizations related to the "black grouse" category, high- lighting a single, consistently meaningful token across different images of this bird species. Similarly, the subsequent four rows are dedicated to visualizations of the "candle" category, emphasizing the same token across various candle images. In each image, we highlight the token exhibiting a high TIS using a red bounding box. These visualizations illustrate our sample-level explanation method's ability to focus on tokens that frequently correspond to specific, concrete visual features within each concept. For instance, the consistent highlighting of particular features (such as the red crown for the black grouse or flame for candles) across multiple images of the same class indicates that these tokens effectively represent meaningful, class- -pasispe characteristics. By consistently focusing the same token within each category, demonstrate our method's 's ability to extract and emphasize stable, category-relevan features across diverse visual representations.
Table codehook-level explanation method evaluation results.
P \ AP f
Model
Accorig
P Orig
P Targ
Embedding Optimization
Token Selection
CE
RE
CE
RE
ResNet18
86.8%
0.795
0.016
-0.224 / 0.127
-0.202 / 0.097
-0.290 / 0.162
-0.284 / 0.
ResNet50
84.6%
0.780
0.011
-0.217 / -0.122
-0.206 / 0.102
-0,282 0.165
-0.277 / 0.155
ViT-B/16
89.0%
0.766
0.003
-0.193
0.095
-0.174 0.072
-0.237 0.128
-0.243 / 0.121
VIT-B/32
90.2%
0.758
0.003
-0.196 0.090
-0.190 / 0.080
-0.239 0.112
-0.245 / 0.120
16

---

Concept-Centric Token Interpretation for Vector-Quantized Generative Models
Figure 7: Token visualization in different categories: red box in each represents the token
17


Reviews:
[{"review_id": "uwxRMVLNq2", "reviewer": "Reviewer_1ugt", "summary": "* This paper presents CORTEX (Concept-Oriented Token Explanation), a novel framework for interpreting Vector-Quantized Generative Models (VQGMs). VQGMs have become powerful in image generation, but the role of their codebook tokens in representing concepts remains unclear.\n\n\n* The authors identify the problem of differentiating concept - relevant tokens from background ones in VQGMs. To address this, they draw on the Information Bottleneck principle to develop an Information Extractor. This extractor maps codebook tokens to concepts, calculating Token Importance Scores (TIS) to find optimal token combinations for each concept.\nCORTEX consists of two complementary methods. The sample - level explanation method analyzes token importance scores in individual images. It computes saliency scores for tokens, uses Gumbel - Softmax for differentiable token selection, and optimizes a token selection matrix to identify concept - relevant tokens in generated images. The codebook - level explanation method explores the entire codebook space. It directly searches for token combinations that best represent specific concepts, optimizing a token selection matrix within a specified mask region.\n\n\n* The authors conduct extensive experiments using various pretrained classification models and a synthetic dataset generated by VQGAN. They train three Information Extractor Models (IEMs) with different architectures and evaluate the performance of CORTEX through counterfactual evaluation. Results show that CORTEX effectively identifies tokens critical to concept representation. The sample - level explanation method can highlight concept - related features in images, and the codebook - level explanation method outperforms the baseline in identifying and manipulating class - relevant features.\n\n\n* The framework has practical applications. It can be used for shortcut feature detection, such as detecting biases in text - to - image models by analyzing concept - specific token distributions.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review. We address your concerns as follows.\n\n**Limited technical contribution to VQGMs:**  \nOur method provides a **quantifiable** approach to **detecting bias** in VQGMs and pinpointing the **specific tokens responsible**, enabling targeted debiasing and image editing. This interpretability can be used to improve fairness, which is a key aspect of generative model performance, by facilitating both the detection of bias and the identification of its underlying sources.\n\n**Unclear benchmark evaluation:**  \nHere, we construct a comprehensive evaluation protocol to measure whether CORTEX-selected tokens are truly relevant to the target concept. It is worth noting that no off-the-shelf benchmark exists for evaluating token-level interpretability in VQGMs. Specifically, we conduct three evaluations:\n\n1. **Sample-level evaluation** (Section 4.2 Figure 4 and Table 2):  \n   - **Dataset:** VQGAN-generated ImageNet (comprehensive visual concepts coverage).  \n   - **Evaluation Metric:** Drop in pretrained classifier accuracy (ViT/ResNet) after masking identified tokens; greater drops indicate higher importance of selected tokens.  \n   - **Results:** Masking CORTEX-identified tokens causes significantly larger accuracy drops compared to randomly or frequency-based selected tokens, confirming that CORTEX effectively identifies concept-critical tokens.\n\n2. **Codebook-level evaluation** (Appendix A.4 Table 5):  \n   - **Dataset:** VQGAN-generated 10 bird categories (chosen because pretrained classifiers achieve high accuracy on these categories, ensuring reliable evaluation metrics).  \n   - **Evaluation Metric:** Increase in target label probability ($\\Delta P_{\\text{Targ}}$) when selecting small token regions; larger $\\Delta P_{\\text{Targ}}$ indicates stronger association between selected tokens and target concept.  \n   - **Results:** CORTEX consistently achieves higher $\\Delta P_{\\text{Targ}}$ values compared to baseline methods, indicating that CORTEX selects tokens that are more aligned with the target concept.\n\n3. **DALLE-mini bias detection** (Section 5.1 Table 4):  \n   - **Dataset:** Images generated by DALLE-mini using neutral prompts (\"a doctor in the hospital\").  \n   - **Evaluation Metric:** Frequency comparison of tokens associated with \"white doctor\" vs. \"black doctor.\"  \n   - **Results:** Tokens representing white doctors occur four times more frequently than those representing black doctors, demonstrating clear bias in DALLE-mini’s generated outputs.\n\nWe will provide a more detailed description of our benchmark and evaluation in the revision.\n\n**Comparisons with other tokenizers:**  \nOur proposed CORTEX can be applied to any vector-quantized generative model. We chose VQGAN as our backbone since it is a highly representative method in the field.\n\nAlso, following your suggestions, we further validate CORTEX on VAR [1], a SOTA VQGM. Specifically, we trained an Information Extractor Model (IEM) using token-based embeddings generated by VAR. We mask the Top-10, 20, and 30 tokens selected by CORTEX and compare the drop in prediction probability against randomly selected tokens. Evaluation using 10,000 VAR-generated images confirms that CORTEX effectively identifies concept-critical tokens.\n\n| Pretrained Model | Top-10 (ours) | Top-10 (random) | Top-20 (ours) | Top-20 (random) | Top-30 (ours) | Top-30 (random) |\n| ---------------- | ------------- | --------------- | ------------- | --------------- | ------------- | --------------- |\n| ViT-B/32         | **3.9**       | 1.3             | **9.4**       | 3.2             | **15.8**      | 6.7             |\n| ResNet50         | **11.8**      | 4.5             | **31.3**      | 22.5            | **49.4**      | 46.5            |\n\n*Table: Prediction probability drop after masking tokens from CORTEX on VAR-generated images; Larger drops indicate that the masked tokens are more important.*\n\n**T2I model application:**  \nWe would like to clarify that our method has been applied to a T2I model, as shown in Figure 1 and Table 4 using DALL·E.  \nInstead of using category IDs, we directly use the text prompt (e.g., “a black doctor”) as the concept. CORTEX identifies visual tokens that are strongly associated with concept words in the prompt, allowing us to analyze token-concept relationships without requiring predefined labels.\n\n**Innovation in interpretability methods:**  \nFirst, traditional methods like CAM or Grad-CAM interpret models in the pixel space, while our method operates in **discrete token space**.\nSecond, traditional methods aim to explain the classifier itself, whereas we leverage the Information Bottleneck principle to train a **classification model that interprets large generative models**.\n\n[1] Tian, et al. \"Visual autoregressive modeling: Scalable image generation via next-scale prediction.\" NeurIPS, 2024.", "claims_and_evidence": "The claims made in the submission are clear", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria in the paper generally make sense for the problem or application at hand. The experiments are primarily conducted on VQGAN models trained on the ImageNet dataset to verify the relationship between tokens and categories. However, the benchmark used for validating the token and category relationship is not particularly authoritative, and there is a notable lack of comparison with other existing methods.", "theoretical_claims": "I have a good understanding of the derivations presented in the Sample-level Explanation and Codebook-level Explanation sections. The mathematical formulations and explanations provided in these two sections are relatively clear and well-structured, making it easier to follow the authors' reasoning and methodologies.", "experimental_designs_or_analyses": "Please refer to the section of Methods And Evaluation Criteria", "relation_to_broader_scientific_literature": "This paper focuses on interpretability-related work and its broader research value is somewhat limited.", "essential_references_not_discussed": "The method focuses on the interpretability of VQ-tokenizer tokens, but the paper lacks analysis and discussion of other state-of-the-art tokenizers. For instance, it does not compare or analyze methods such as Residual Quantization, VAR, and MAGVIT2, which are considered excellent tokenizers. Including such comparisons and discussions would provide a more comprehensive understanding of the context and contributions of the proposed method.", "other_strengths_and_weaknesses": "This paper primarily focuses on the interpretability analysis of tokens and their corresponding semantic categories within image tokenizers. The methodology involves using a pretrained generative model (e.g., VQGAN) to generate samples, then training an IEM based on these generated samples and their corresponding categories. The IEM is subsequently used to calculate the relationship between tokens and categories in the image. Frankly, I find this framework quite interesting, but I have the following concerns:\n\n1. I believe the interpretability of tokens offers limited technical contribution to the enhancement of generative model performance and the optimization of future tokenizers.\n2. I feel that the benchmark constructed in this paper is not very clear. How to evaluate the effectiveness of this work on the benchmark itself is not clearly explained.\n3. This work lacks comparisons with other image tokenizers. It is uncertain whether different tokenizers will exhibit various forms of performance.\n4. Additionally, there are no experimental validations on T2I models. If this experiment were conducted on T2I models, considering the absence of category IDs, how would the association between tokens and text be calculated?\n5. Additionally, I found that the interpretability methods themselves lack significant innovation; these methods are mostly an application of previously established interpretability techniques from the visual recognition field (refer to CAM or Grad-CAM) applied to tokenizer recognition.", "other_comments_or_suggestions": "Considering these issues, my inclination is to give a weak reject. I believe the authors need to address these questions. Of course, if they can provide convincing answers, I would consider raising my score.", "questions_for_authors": "please refer to \"Other Strengths And Weaknesses\"", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ALon2FDJXu", "reviewer": "Reviewer_EKNf", "summary": "The paper introduces Concept-Oriented Token Explanation (CORTEX), a framework for interpreting Vector-Quantized Generative Models (VQGMs). CORTEX employs sample-level and codebook-level explanation methods to identify concept-specific tokens, enhancing the transparency of how VQGMs generate images. By using an Information Extractor Model based on the Information Bottleneck principle, CORTEX effectively distinguishes critical tokens from background elements. Experimental results demonstrate its superiority over baseline methods in explaining token usage, with practical applications in detecting biases and enabling targeted image editing. CORTEX thus provides valuable insights into the internal workings of VQGMs, paving the way for more interpretable and controllable generative models.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive assessment of our work. We address your questions below:\n\n### Parameters n and k:\n**n=20** represents the top tokens with the highest Token Importance Scores (TIS) selected from each image, while **k=100** represents the most frequently occurring tokens across all sample images. Both parameters face similar trade-offs:\n\nIf the values are too small, we might miss tokens that significantly contribute to the concept or fail to capture the full diversity of concept representations.  \nOn the other hand, if the values are too large, we risk including irrelevant tokens that dilute the concept representation or include more background information.\n\nFigure 4 in the main paper demonstrates how performance changes as n varies from 5 to 50, showing that as the number of masked tokens increases, prediction probability continuously decreases. However, the rate of this decreasing gradually diminishes, indicating that the initial few tokens contain the core information about the concept, while including more tokens begins to incorporate less relevant information.\n\n### Latest SOTA models\nOur proposed **CORTEX** can be applied to any type of vector-quantized generative model.  \nFollowing your suggestions, we further validate CORTEX on the latest SOTA VQGM, **VAR [1]** published in NeurIPS 2024. Specifically, we trained an Information Extractor Model (IEM) using token-based embeddings generated by VAR. We mask the Top-10, Top-20, and Top-30 tokens selected by CORTEX and compare the drop in prediction probability of pretrained ViT and ResNet against randomly selected tokens. Evaluation using 10,000 VAR-generated images confirms that CORTEX effectively identifies concept-critical tokens.\n\n| Pretrained Model | Top-10 (ours) | Top-10 (random) | Top-20 (ours) | Top-20 (random) | Top-30 (ours) | Top-30 (random) |\n| ---------------- | ------------- | --------------- | ------------- | --------------- | ------------- | --------------- |\n| ViT-B/32         | **3.9**       | 1.3             | **9.4**       | 3.2             | **15.8**      | 6.7             |\n| ResNet50         | **11.8**      | 4.5             | **31.3**      | 22.5            | **49.4**      | 46.5            |\n\n**Table:** Prediction probability drop after masking tokens from CORTEX on VAR-generated images; Larger drops indicate that the masked tokens are more important.\n\nWe will also include a discussion of models such as **MAGE** and **FSQ** in the revised version.\n\n[1] Tian, Jiang, et al. \"Visual autoregressive modeling: Scalable image generation via next-scale prediction.\" *NeurIPS*, 2024.", "claims_and_evidence": "Claim 1: CORTEX Enhances VQGM Interpretability\n\nEvidence: Comprehensive evaluations show CORTEX outperforms baseline methods in explaining token usage, revealing consistent patterns of concept-relevant tokens in generated images.\n\nClaim 2: CORTEX Identifies Concept-Specific Tokens\n\nEvidence: The Information Extractor Model (IEM) effectively captures the relationship between token patterns and image labels, with high Top-5 accuracies demonstrating accurate identification of relevant tokens.\n\nClaim 3: CORTEX Detects Biases in Generative Models\n\nEvidence: Experiments with neutral prompts reveal systematic biases, with tokens associated with certain demographics appearing more frequently, quantified using statistical measures like Cliff’s $\\delta$.\n\nClaim 4: CORTEX Enables Targeted Image Editing\n\nEvidence: Visualizations and quantitative comparisons show CORTEX can precisely manipulate image content by optimizing concept-relevant tokens, leading to significant changes in target label probabilities.", "methods_and_evaluation_criteria": "The methods used in the paper include the development of the Concept-Oriented Token Explanation (CORTEX) framework, which consists of two main approaches: sample-level explanation and codebook-level explanation. The sample-level explanation method analyzes token importance scores in individual images to identify concept-specific tokens, while the codebook-level explanation method explores the entire codebook to find globally relevant tokens using an Information Extractor Model (IEM) based on the Information Bottleneck principle. The evaluation criteria involve comprehensive experimental validation using various pretrained classification models to assess the effectiveness of CORTEX in providing clear explanations of token usage, detecting biases by analyzing token frequencies in neutral prompts, and enabling precise image editing by optimizing concept-relevant tokens. The performance is measured through changes in softmax probabilities for original and target labels, comparison of token importance scores, and statistical measures like Cliff’s $\\delta$ to quantify biases.", "theoretical_claims": "The CORTEX framework significantly enhances the interpretability of Vector-Quantized Generative Models (VQGMs) by providing detailed, concept-specific explanations of how these models represent and generate images. By identifying and analyzing the importance of discrete tokens, CORTEX offers insights into the internal mechanisms of VQGMs, making them more transparent and understandable.", "experimental_designs_or_analyses": "This work identifies the Top-n (n = 20) highest-TIS tokens and Top-k (k = 100) most frequent tokens. How do the parameters n and k affect the performance?\nThis work is only based on the data generated by VQGAN. What is the effect of the data generated by the latest SOTA model such as MAGE and FSQ?", "relation_to_broader_scientific_literature": "CORTEX extends these explainability techniques to the token level, offering concept-specific explanations for discrete latent representations in VQGMs. This provides a deeper understanding of the generative processes in these models, bridging the gap between traditional pixel-level explanations and modern token-based approaches.", "essential_references_not_discussed": "All key references have been cited.", "other_strengths_and_weaknesses": "CORTEX provides a comprehensive framework that includes both sample-level and codebook-level explanations. This dual approach allows for detailed analysis at both the individual image level and the broader codebook level, offering a more complete understanding of how VQGMs generate images.", "other_comments_or_suggestions": "No other comments.", "questions_for_authors": "No other questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yhinCz9wFL", "reviewer": "Reviewer_4bpB", "summary": "This paper introduces CORTEX, a method for interpreting Vector-Quantized Generative Models (VQGMs) with concept-oriented token explanations. CORTEX employs sample-level and codebook-level explanations. CORTEX is useful for shortcut detection (i.e. biases) and image editing. For the evaluation, they CORTEX employs a synthetic dataset generated by VQGAN with ImageNet categories. Experiments show that CORTEX enhances transparency and controllability in generative models.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed review. We address your concerns and questions below:\n\n**Limited evaluation setting:**  \nOur evaluation is not limited to VQGAN or ImageNet classes. We also include a text-to-image VQGM, DALL·E in section 5.1, where concepts are defined by prompts such as “male/female/black/white doctor”. These experiments demonstrate that CORTEX can explain arbitrary, user-defined concepts beyond fixed class labels. We also include additional results on the SOTA VQGM, VAR [1]. Evaluation using 10,000 VAR-generated images confirms that CORTEX effectively identifies concept-critical tokens.  \n\n| Pretrained Model | Top-10 (ours) | Top-10 (random) | Top-20 (ours) | Top-20 (random) | Top-30 (ours) | Top-30 (random) |\n| ---------------- | ------------- | --------------- | ------------- | --------------- | ------------- | --------------- |\n| ViT-B/32         | **3.9**       | 1.3             | **9.4**       | 3.2             | **15.8**      | 6.7             |\n| ResNet50         | **11.8**      | 4.5             | **31.3**      | 22.5            | **49.4**      | 46.5            |\n\n*Table: Prediction probability drop after masking tokens from CORTEX on VAR-generated images; Larger drops indicate that the masked tokens are more important.*\n\n**Lack of comparisons:**  \nAs the first work to explain VQGMs in their token space, we have limited baseline options for comparison. However, we also established non-trivial baselines except for the random ones, including the frequency-based baseline in Table 2 and the embedding optimization baseline in Table 5. We evaluate CORTEX with different IEM architectures and find it consistently identifies concept-related tokens, demonstrating architecture-independent interpretability.\n\n**Unclear methodology aspects:**  \nWe will improve figures and notation in the revision. Specifically:\n\n1. **Less frequently activated tokens:**  \n   Less frequently activated tokens represent elements that contribute minimally to a specific concept. These typically correspond to **background information** like sky or grass that lack the distinctive characteristics of the concept.  \n\n2. **Token-based embeddings:**  \n   In Figure 3, the colored grids represent the token-based embedding. Each position contains a token from the codebook. We will include the notation for **E** in Figure 2 in the revision.\n\n3. **Multiple embeddings per class:**  \n   Yes, there can be multiple possible token-based embeddings for a class like \"Indigo Bird.\" This is why we train our IEM to identify the token combination that **best** represents the concept.\n\n4. **Equation 3 clarification:**  \n   Equation (3) is inspired by SmoothGrad [2], which adds noise to the input multiple times and averages the resulting gradients to reduce the model’s sensitivity to small perturbations, yielding a more stable saliency score.\n\n5. **Max in Equation 4:**  \n   Since each channel captures different aspects of feature importance, we use the maximum across channels to identify the most discriminative feature at each position, following the max-selection approach from Section 3.1 of [3]. To validate this, we compare max and average strategies by measuring the drop in prediction probability after masking top-ranked tokens. The max operation consistently causes greater drops, indicating it better captures concept-relevant tokens.\n\n| Pretrained Model | Top-10 (max) | Top-10 (average) | Top-20 (max) | Top-20 (average) | Top-30 (max) | Top-30 (average) |\n| ---------------- | ------------ | ---------------- | ------------ | ---------------- | ------------ | ---------------- |\n| ViT-B/32         | **12.3**     | 11.7             | **23.6**     | 22.7             | **31.2**     | 29.8             |\n| ResNet50         | **25.3**     | 23.7             | **36.3**     | 35.1             | **41.4**     | 40.5             |\n\n*Table: Performance comparison across average and max operation on VQGAN-generated images.*\n\n6. **Letter 'k' is used multiple times:**  \n   Thank you for your suggestion. We will improve our notations in the revision.\n\n7. **Different metrics for bias:**  \n   Cliff's δ provides a standardized measurement that accounts for distribution differences. Other metrics like Jensen-Shannon divergence could be used, but Cliff's δ offers clear interpretability with established thresholds for effect size (small/medium/large).\n\n8. **Concept label space:**  \n   Yes, they are the same: the concept labels we want to explain are exactly the same as the labels (or text prompts) used to condition the VQGM.\n\n**Extraction of codebook:**  \nWe will add a section in the Appendix to provide more details.  \n\n[1] Tian, Jiang, et al. \"Visual autoregressive modeling: Scalable image generation via next-scale prediction.\" NeurIPS, 2024.  \n[2] Smilkov, et al. \"Smoothgrad: removing noise by adding noise.\" ICML, 2017.  \n[3] Simonyan, et al. \"Deep inside convolutional networks: Visualising image classification models and saliency maps.\" arXiv preprint.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes, however the experimental setting is limited (see weaknesses below).", "theoretical_claims": "This is not a theoretical paper, however, it relies on the Information Bottleneck principle in compression. They use an Information Extractor network to filter out background and non-essential tokens. The visualizations in the paper seem to support this principle.", "experimental_designs_or_analyses": "Yes, the VQGMs setting is legit.", "relation_to_broader_scientific_literature": "Neural networks are very well known black box models affected by different biases. This work contributes to interpretability methods for VQGMs.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "### Strengths\n\n- The proposed approach identifies the most important token used by VQGMs, enhancing their transparency.\n\n- The paper is relevant to the community, especially for shortcut detection.\n\n### Weaknesses\n\n- The evaluation setting is limited: only a VQGM is used (GAN-based), concepts are limited to ImageNet classes.\n\n- Lack of comparisons to other methods. Results with different kinds of information extraction networks are not sufficient. The authors should explain why their method, for example, better identifies biases than others.\n\n- Some aspects in the proposed methodology are unclear. The figures and notation for example should be improved. See questions below.\n\n- More details on how the extraction of codebook works in VQGM should be explained better; an additional section in the Appendix could benefit the paper's readability.", "other_comments_or_suggestions": "No", "questions_for_authors": "- What are the implications of tokens that activate less frequently? Have the underrepresented concepts generated images, for example, lower quality?\n\n- In Figure 3, which are the token-based embeddings? The coloured squares? The figures need additional notation used in the paper.\n\n- Given a class, i.e. \"Indigo Bird\", are there multiple possible token-based embeddings $E$?\n\n- Equation 3 is unclear. What is changing in the summation with different $l$s? $E$? Maybe an index notation on $E$ would be clearer. Why add noise $\\epsilon_l$?\n\n- Equation 4, why max? What about taking the average of the channels instead?\n\n- The letter 'k' is used multiple times in different contexts (codebook, top-k etc.). A different letter for different things is recommended to improve readability.\n\n- Are there more meaningful metrics other than frequency or cliff to identify biases?\n\n- Does the concept label space $Y$ need to be the same as the label-conditioned space in the VQGM?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AZA1Y3a6Hg", "reviewer": "Reviewer_YRHi", "summary": "This paper introduces CORTEX, an approach for interpreting Vector-Quantized Generative Models (VQGMs) by identifying concept-specific token combinations from their codebooks. The authors develop two complementary methods: a sample-level explanation that analyzes token importance in individual images and a codebook-level explanation that searches the entire codebook for globally relevant tokens representing specific concepts. Experimental evaluations show CORTEX's ability to provide clear explanations of token usage in the generative process while also enabling applications such as targeted image editing and bias detection.\n\n## update after rebuttal\n\nThe authors have addressed my main questions and concerns. For me, this is a self-consistent and complete work. I have also carefully read and am aware of the other reviewers' concerns. Overall, I maintain my original borderline accept score, but I would not be surprised if it were rejected.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedback. We address your concerns:\n\n### Weak baseline comparison:\nOur comparison against both random selection and the frequency-based baseline (Table 1) highlights CORTEX’s effectiveness in identifying concept-relevant tokens. While the frequency-based method offers a stronger baseline than random selection, our method achieves a greater reduction in concept probability using fewer masked tokens. This suggests that CORTEX can better filter out contextual information, whereas frequency-based selection tends to pick background tokens that are not strongly tied to the concept itself.\n\n### Threshold-based approach:\nThank you for the suggestion. While threshold-based selection can serve a similar role as the top-n or top-k strategy by selecting the most important tokens based on their importance scores, it requires careful tuning for each VQGM and IEM. In contrast, our top-n and top-k strategies provide a consistent and architecture-independent evaluation across models.\n\n### Gradient-based attribution issues:\nTo reduce the impact of noisy gradients, we adopt the SmoothGrad [1] principle by adding noise to the input embeddings and averaging the resulting gradients, thus producing more stable token attributions. Furthermore, our sample-level explanation $\\mathcal{T}^*_{\\text{concept}}$ also aggregates highly activated tokens across multiple images via Equation 5, effectively mitigating the issue of gradient saturation that may occur in individual samples.\n\n### Convergence guarantees:\nWe acknowledge that this issue may arise with certain concepts, but we have quantitatively analyzed the performance of codebook-level explanations in Table 5 of our appendix. This analysis demonstrates that convergence to adversarial patterns is not the dominant case.\n\n### Token interdependencies:\nOur codebook-level explanation can explain the interdependencies between tokens because this method optimizes tokens within a region simultaneously, obtaining the token combination that best represents this concept. We don't optimize tokens one by one but rather optimize the token selection matrix, and the optimization process considers the interdependencies between tokens.\n\n### Multi-concept tokens:\nTokens can indeed be relevant to multiple concepts. The token importance scores (TIS) are concept-specific, allowing us to map the same token to different concepts with varying importance levels. Although a single token may be associated with multiple concepts, it can represent different concepts when combined with different other tokens.\n\n### \"Concept-centric\" terminology:\nOur method can explain abstract concepts, not just entities or objects. For example, in Section 5, we demonstrate how CORTEX explains relatively abstract concepts like \"male\" and \"female\" which go beyond simple visual objects. Furthermore, in text-to-image generative models like Dalle, our approach can explain any concept by identifying the most relevant token combinations when concepts such as \"happy\" or \"dangerous\" are used as prompts to generate images. We thank the reviewer for this thoughtful suggestion and will consider using more precise terminology in the final version of our paper.\n\n[1] Smilkov, Daniel, et al. \"Smoothgrad: removing noise by adding noise.\" ICML 2017.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "There are no theoretical claims.", "experimental_designs_or_analyses": "The sample-level explanation evaluation compares against random token selection, which is a weak baseline. A fairer comparison would include other attribution methods adapted to the token space.", "relation_to_broader_scientific_literature": "This work is related to vector quantization, information bottleneck principles, and bias detection in generative models.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "## pros\n\n- Addresses an interesting interpretability gap in VQGMs.\n- Visualizations illustrate the concept-token relationships identified by the method.\n- The paper is well written and easy to follow.\n\n## cons\n\n- See the questions below.", "other_comments_or_suggestions": "- Instead of using top-n and top-k token selection in the sample-level explanation, the authors could consider using a threshold-based approach that better adapts to the importance distribution.", "questions_for_authors": "- The sample-level explanation uses gradient-based attribution, which has known issues like gradient saturation and noisy gradients. How does this affect the reliability of token importance scores?\n- About the codebook-level explanation, what guarantees the process consistently converges to meaningful token combinations rather than adversarial patterns that simply maximize class probability?\n- The main evaluation relies on masking tokens and measuring probability changes, but this assumes tokens have independent effects. How does the method account for the interdependencies between tokens where combinations matter more than individual tokens?\n- How does CORTEX handle tokens that might be relevant to multiple concepts simultaneously? The framework assumes a clear concept-to-token mapping that may not exist in practice.\n- The term \"concept\" may not be suitable. The proposed method primarily focuses on entities or objects. How well does CORTEX generalize to more abstract concepts like \"happy\" or \"dangerous\" that don't have clear visual correspondences? If not suitable for these abstract concepts, I suggest using \"object-centric\" instead of \"concept-centric\".", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces a method for interpreting the tokens generated by VQ models and demonstrates its potential usage across several applications.

Following the rebuttal, two reviewers responded that their main concerns had been addressed, with one increasing their score to a weak accept. Another reviewer, who had initially recommended acceptance, did not participate in the discussion phase. The remaining reviewer, Reviewer 1ugt, acknowledged the paper’s significance and described their stance as “borderline, leaning towards acceptance.” The remaining concern of `Reviewer 1ugt` is the limited technical contribution, such as how interpretability contributes to improving the quality and performance of the VQ model.

Ultimately, none of the four reviewers opposed acceptance, and no major outstanding issues were raised. The Area Chair agrees with the overall consensus and recommends accepting this submission.
Decision: Accept (poster)