ULTra: Unveiling Latent Token Interpretability in Transformer-Based Understanding and Segmentation
Authors: Hesam Hosseini
Ghazal Hosseini Mighan
Amirabbas Afzali
Sajjad Amini
Amir Houmansadr
Venue: Accepted by TMLR ()

Abstract: Transformers have revolutionized Computer Vision (CV) through self-attention mechanisms. However, their complexity makes latent token representations difficult to interpret. We introduce ULTra, a framework for interpreting Transformer embeddings and uncovering meaningful semantic patterns within them. ULTra enables unsupervised semantic segmentation using pre-trained models without requiring fine-tuning. Additionally, we propose a self-supervised training approach that refines segmentation performance by learning an external transformation matrix without modifying the underlying model. Our method achieves state-of-the-art performance in unsupervised semantic segmentation, outperforming existing segmentation methods. Furthermore, we validate ULTra  for model interpretation on both synthetic and real-world scenarios, including Object Selection and interpretable text summarization using LLMs, demonstrating its broad applicability in explaining the semantic structure of latent token representations.

Published in Transactions on Machine Learning Research (01/2026)
ULTra: Unveiling Latent Token Interpretability in Bansformer-Based Understanding and Segmentation
Hesam Hosseini
hesomshosseimm@@maall.cmm
Sharif University of Technology
Ghazal Hosscini Mighan Sharif University of Technology
ghuzaldesu@@mmiil.cmm
Amirabbas Afzali Sharif University of Technology
amir8nfaall@@maell.co
Sajjad Amini
aminni@shrrif.ee edu, samini@umass. edu
Sharif University of Technology
Amir Houmansadr
amir umiisscs.mmss.coo wmas. edu
University of Massachusetts Amherst
Reviewed on OpenReview: https: openreview. net/ pet/fore forum? id- Ldp3mmjjG
Abstract
Transformers have revolutionized Computer Vision (CV) through self-attention mechanisms. However, their complexity makes latent token representations difficult to interpret. We in- troduce ULTra, a framework for interpreting Transformer embeddings and uncovering mcan- ingful semantic patterns within them. ULTra enables unsupervised semantic segmentation using pre-trained models without requiring fine-tuning. Additionally, we propose self- supervised training approach that refines segmentation performance by learning an exter- nal transformation matrix without modifying the underlying model. Our method achieves seate-of-thearte performance in unsupervised semantic segmentation, outperforming existing segmentation methods. Furthermore, we validate ULTra for model interpretation in both synthetic and real-world scenarios, including Object Selection and interpretable text sum- marization using LLMs, demonstrating its broad applicability in explaining the semantic structure latent token representations.
1 Introduction
In recent years, the Transformer architocture and foundation models, which leverage self-attention mech- anisms to capture complex dependencies, have transformed Natural Language Processing (NLP) bench- marks (Vaswani et al., 2017; Touvron et al., 2023; Team et al., 2024). Similarly, Vision Transformers (ViTs) (Dosovitskiy et al., 2020) have been adapted in Computer Vision (CV) and now serve as the back- bone for various tasks such as segmentation and object detection (Thisanke et al., 2023; Liu et al., 2021). Despite their success, understanding the interpretability of Transformers remains a challenge due to the complexity of their latent token representations.
Several methods have been developed enhance the interpretability of CNN-based models (Simonyan al., 2014; Zeiler & Fergus, 2014; Selvaraju et al., 2017). While some of these be extended to Transformer architectures, they do not fully leverage the unique attention mechanisms inherent to Transformers. Recent
*This work conducted during visit CICS, UMass Amherst. The codebase available https://github. com//CocoAixs/UU.rrr

---

Published in Transactions on Machine Learning Research (01/2026)
rescarch has introduced interpretability methods specifically designed for Transformers (Chefer al., 2021b; Abnar & Zuidema, 2020; Vig & Belinkov, 2019). However, these approaches primarily aim to explain the final outputs of a model and cannot be directly applied to interpret latent tokens, offering only limited insight into the intermediate processes that drive prodictions. This gap naturally raises a fundamental question:
Do transformer models exhibit semantic awareness within their latent representations?
To address this, we propose Unweiling Lalent Token Interprelabilll in Transformer-Base Based Underslanding (ULTra), framework for interpreting latent tokens in Transformers. At a high level, ULTra interprets latent tokens by tracing how input information flows through the Transformer's attention layers to individual token representations. Given a pre-trained Transformer and target layer, select latent token and define a scalar function of its embedding. We then backpropagate this signal through the attention probability matrices, constructing layer-wise contribution maps that are aggregated across layers to produce a token- specific explanation map in the input space. These explanation maps reveal which input regions most strongly influence each latent token, exposing their semantic specialization without modifying the model or roquiring aligned modalitics.
Recent work with similar objectives (Chen et al., 2024) interprets latent tokens by projecting them into CLIP's multi-modal embedding space. This is achieved by disabling the self-attention mechanism and then using an external text encoder to associate tokens with semantic descriptions. In contrast, ULTra (i) offers a direct interpretation of latent tokens without relying on any aligned modality, and (ii) preserves the model architecture and behavior at inference time, thereby avoiding modifications that could compromise faithfulness.
By examining the semantie information encoded in latent tokens, we demonstrate that Transformers in- herently capture the semantic structure of their input as collection of distinct concepts. For instance, in Figure 2, tokens clearly separate semantic entities, with individual tokens specializing in the dog, the cat, the background, or even fine-grained attributes such as the cat's head. This observation naturally leads to a second question:
Can such alignment be exploited for unsupervised semantic segmentation (USS)?
we cluster token-level explanation maps and aggregate them to form pixel-wise semantic regions, thereby achieving unsupervised semantic segmentation. Unlike prior unsupervised segmentation methods that re- quire additional training (Sick al., 2024; Hamilton et al., 2022; Li et al., 2023), ULTra leverages the intrinsic knowledge of pre-trained models on tasks other than semantic segmentation to achieve stato-of-the- art performance on benchmark dalasets without the need for fine-tuning, To further enhance segmentation performance, we introduce a self celf-cooni approach that learns an external transformation matrix in a self-supervised manner, refining segmentation without modifying the underlying model. Additionally, we validate our interpretability framework on transformer- based LLMs through qualitative analyses text summarization, remonstrating its broad applicability across modalities.
Our main contributions are as follows:
We propose a framework for interpreting latent tokens in Transformers, uncovering the stored se- mantic knowledge in each latent token. To the best of our knowledge, we are the first to investigate latent token interpretability directly.
We extend ULTra to several designed synthetic and real-world tasks, showcasing its versatility and applicability, including Object Selection and interpretation of LLMs in text summarization tasks.
By aggregating explanation maps generated from latent tokens, our method enables unsupervised se- mantic segmentation using pre-trained ViTs without requiring any additional training. This strategy outperforms existing stale-of-the-art approaches that rely on fine-tuning or supervision, highlighting the effectiveness of leveraging the semantic structure embedded in large-scale models.
To further enhance segmentation performance, we introduce ULTraw, a lightweight learnable exten- sion that optimizes self-consistency loss. This loss encourages stable token representations under
2

---

Published in Transactions on Machine Learning Research (01/2026)
imput perturbations and yields transformation matrix that projects tokens onto more informative subspace while keeping the backbone model unchanged.
2 Related Work
Interpretable Deep Learning. Model interpretability is critical aspect of deep learning, particularly for complex architectures like Transformers. Traditional methods such as saliency maps (Simonyan et al., 2014), Grad-CAM (Selvaraju et al., 2017), LIME (Ribciro et al., 2016), and SHAP (Lundberg, 2017) have been effective for CNNs but do not fully exploit Transformers' self-atlention mechanisms. growing body of research focuses on transformer-specfff. interpretability techniques that leverage attention mechanisms as intrinsic explanations (Vig & Belinkov, 2019; Abnar & Zuidema, 2020; Chefer et al., 2021b; Jain & Wallace, 2019; Wu et al., 2024). These methods are largely post-hoc, providing explanations after model training. In contrast, ante-hoc approaches such as TA-VIT (Qiang et al., 2023) and Mechanistie Interpretability (Rai et al., 2024) seek to make Transformers inberently interpretable. Additionally, techniques like LeGrad (Bousselham et al., 2024) and IA-RED* (Pan et al., 2021) improve interprotability by analyzing feature formation and reducing redundancy in self-attention. Despite these advancements in model interpretability, understanding latent tokens in ViTs remains underexplored, underscoring the need for methods that explicitly interpret these latent tokens. A related study (Chen et al., 2024) explores latent token interpretation in CLIP by modifying self-attention mechanisms.
Semantic Segmentation. Unsupervised semantic segmentation has progressed through self-supervised learning and clustering techniques. Early methods such as IIC Ji et al. (2019) and PICIE Cho et al. (2021) leveraged mutual information and consistency principles to enhance feature representations. Transformer- based approaches like DINO Caron et al. (2021) and STEGO Hamilton et al. (2022) further improved seg- mentation by capturing mcaningful structures through self-attention. Other techniques, including MaskCon- trast Van Gansbeke et al. (2021), Leopart Ziegler & Asano (2022), and ACSeg Li et al. (2023), refined seg- mentation through clustering and adaptive conceptualization. More recent methods, such as DepthG Sick et al. (2024), incorporate depth-guided correlations, while U2SEG Niu et al. (2024) utilizes pseudo-labeling, Additionally, SmooSeg Lan et al. (2023) enforces smoothness priors, and HSG Ke et al. (2022) applies hierar- chical segmentation via multiview clustering Transformers. In the context of semantic segmentation, Weakly Supervised Semantic Segmentation (WSSS) aims to generate segmentation masks using only image-level labels. These methods often rely on saliency maps from interpretabllity tochniques, such as Class Activation Maps (CAMs), to localize objects Choe & Shim (2019); Yin et al. (2020); Chen et al. (2023). Typically, they depend class logits to guide segmentation, refining masks based predicted class scores.
Latent Embedding. The high dimensionality and complex distribution of latent embeddings in deep models pose significant challenges for interpretation and manipulation. Methods like GroupViT Xu et al. (2022) utilize hicrarchical grouping to facilitate more mcaningful representation learning, enabling seman- tie segmentation. Other approachess, such as Lee et al. (2024); Bolya et al. (2023); Liang al. (2022), improve computational efficiency by eliminating redundant tokens, while register-basee ViTs Darcet et al. (2024) address artifacts in feature maps caused by nurl tokens through the introduction of reg- ister tokens. However, these techniques primarily emphasize computational efficiency and representation structuring rather than the interpretability of latent embeddings.
3 Methodology
In this section, we introduce our approach for interpreting latent representations in Transformers. We start by outlining the essential preliminaries, followed by a detailed explanation of the ULTra framework for analyzing latent tokens. Additional details on the Transformer architecture are provided in Appendix 1.
3.1 Preliminaries
Previous research on attention-baser model interpretability (Abnar & Zuidema, 2020; Chefer et al., 2021b;a; Wu et al., 2024) has primarily focused on analyzing the semantic flow from input tokens to class logits.
3

---

Published in Transactions on Machine Learning Research (01/2026)
Backward path
5(0
c,
f(2 0h
5,
s(0)
B/
A
layer
layer b
layer I
Forward path
Figure 1: The overall architecture of the ULTra framework. The framework consists of forward path, where the imput data x is fed into the model, and backward path, starting from the target layer 4, where we compute the gradient of a scalar function of the i-th latent token, f(z("), with respect the attention probability matrix of the middle layer b. Next, we compute the corresponding contribution map ((b.)) for all middle layers. Finally, we construct the explanation map 5("), select its i-th row, and transform it to the imput size. As an example, on the left, we observe that the token corresponding to the middle window assigns considerable attention to the left window, suggesting an underlying semantic understanding.
This is typically achieved by adding the attention probability matrix of each layer to an identity matrix and aggregating the results across attention heads through weighted average. The weights are derived from the gradients of the class logits with respect to the attention probabilities. Specifically, to assess the contribution of each class logit using attention information, the contribution map for layer is defined as:
C I E, (v. xo A
(1)
S. = C!I)) c 2 C(1)
(2)
S. 5, 1 :]}
(3)
where @ denotes the Hadamard product, and (-)+ is the operator that retains only positive values, I represent the identity matrix, which reflects the contributions of skip connections and A (6) (1+++))(x+1)
denotes the attention probability matrix of head h and layer b. The term V (N)((c = pp(cl is the partial
aA
derivative of the attention map with respect to the predicted probability for class c, and E&, denotes the mean over multiple attention heads. Equation 2 represents matrix multiplication, which accounts for aggregating contributions from all layers. Each element. of Sa(i.]] represents the influence the j-th input token on the i-th output token. Thus, each element of ScDi quantifies the influence the j-th input token on class c, with token corresponding the CLS token. This raises new question: Can we expect similar semantically meaning for latent token? ? Is there similar approach to interpret latent tokens in Transformers?
4

---

Finally, we define the correspouding explanation map for the latent token z denoted as s!") E ER, where Published in Transactions on Machine Learning Research (01/2026)
3.2 ULTra Framework
Inspired by Equation 1, in this work we aim to measure the contribution of latent token z to the input space using the underlying attention probabilities, where {0, [0. denotes the token index and represents the layer. Specifically, to quantify the influence of the attention map at layer b on the latent token (1) the contribution map c,000) is defined as follows:
z;
c.u.) = =I+ En ((va))e())* oA))
(4)
Intuitively, this approach traces the most influential input information contributing to a latent token z
where the notion of inffuence is defined based on the impact an imput token has on the function f : R² + R refer to the degree to which an imput token affects f(z,") of the target token. Specifically, by inffuence, reflecting its relative importance in the representation. Since tokens are high-dimensional vectors, assuming equal importance across all components can obscure critical contributions. Moreover, different aggregation strategies can assign varying importance to individual dimensions. To address this, we evaluated three functions: (i) a simple summation over all components, (ii) the vector norm (treating all components equally), and (iii) our proposed ULTraw, which learns weighting function. Empirically, ULTraw generally yields stronger downstream performance.
each element represents the influence of input token on (1) The explanation map is computed as:
5 c((..) c.(.))..(..))) CH-LD
(5)
where represents matrix multiplication, which aggregates contributions from all layers to the target latent token. Transformer skip connections cause most contributions to concentrate on fi hindering token- level analysis. To mitigate this, we replace it with the maximun of the other elements, better capturing token contributions. Moreover, in some experiments, e.g., semantic segmentation, we reshape and upsample the explanation map using bilinear or cubic interpolation to match the input resolution, producing 5(!) The overall framework of ULTra is shown in Figure 1.
Additionally, As we illustrate in section 5, in our experiments we utilize three approaches for designing f in ULTra framework:
(i) fa(()))= = (z.", 1) In this formulation, 1 represents an all-ones vector, meaning that the function simply computes the sum of all elements in z!" This approach is denoted by ULTras. The underlying intuition is that this approach treats all elements of the token equally.
(ii) [e(24))= (z ()) This approach is based on the energy (or norm) of a given token, a concept that has been previously explored in the literature Darcet et al. (2024). By leveraging token energy, this method captures the magnitude of the token's latent representation. We refer to this variant as ULTrag.
(iii) fo fw (z (z.", w:): Beyond the previous approaches, we introduce a learnable vector w; to project each token representation (.. Geometrically, the function fa in ULTrag can be viewed as mapping all tokens onto a shared projection basis within the token space. In contrast, learning w; enables the construction of more informative, robust projection basis. We refer to this variant as ULTraw. Unlike ULTraş and ULTrag, which are entirely training-free free, ULTraw requires light training to optimize the w; vectors. In practice, we train these vectors using only a limited number of batches, yet this additional flexibility improves segmentation performance. In Section 4.1, we discuss this idea more in detail by introducing a self-supervised strategy for learning w,, which relies on another ULTra variants for an initial segmentation.
For both ULTras and ULTrag, our framework remains fully tranning-free. In semantic segmentation tasks, experimental results show that these variants not only approach but often surpass baselines that rely on fine-tuning, underscoring both their effectiveness and efficiency.
5

---

Published in Transactions on Machine Learning Research (01/2026)
(a) Original Image
(b) Latent Token's explanation Map
(c) Predicted Binary Mask
Figure An example token interpretation by our model and its predicted binary mask. (a) Original image. (b) Overlay of ((13) the original image for different i, where the location of the i-th token is indicated by the purple square. (c) The binary mask M(13) for each corresponding explanation map in (b). We can generally observe that tokens clearly separate semantic entities, attending to the dog, the cat, the background, or even fine-grained attributes like the cat's or dog's head.
4 Tailoring ULTra for Different Applications
In this section, we aim to examine ULTra's capability to adapt to various tasks involving semantic knowledge.
4.1 Unsupervised Semantic Segmentation
Explanation maps are defined for each latent token at fixed layer, resulting in as many maps as there are latent tokens. For segmentation, we use hierarchical clustering to group these maps. This clustering method naturally matches how vision transformers process images, eventually aggregating similar concepts at multiple scales of fineness. the number of clusters beforehand. In Appendix Figure also provides a schematic of this process. In our experiments, we set a predefined number of clusters l for the algorithm which setup the threshold corresponding to that number of classes, however threshold & gives flexible control over segmentation detail; lower values show fine details and higher values create broader regions. To ensure fair representation, we apply Min-Max scaling to prevent larger objects from dominating the clustering process.
finds the structure in token explanations without needing us to guess
discuses in the appendix G segmenting using only the
After clustering, distinct concepts are defined by aggregat- ing explanation maps. The aggregated explanation map for cluster e is:
[1.0
Σ EE
5 [ vl.
(6)
where d(c)= = (i: (las)(5(()) cf represents the grouping of label assignments. Class labels are assigned to input pixels using the explanation map of the 1-th layer as follows:
Class|æ, " argmax S(0) v). cE(l....k)
Some examples illustrating our segmentation method are pre- sented in Figure 3.
Figure 3: ULTra segmentation results on sam- ple images. The top row displays the orig- inal images, the middle row shows true an- notations, and the bollom row presents our model's predictions.
ULTraw- As previously mentioned, to further enhance seg- mentation performance, we propose ULTraw, which utilizes a learnable transformation matrix to obtain a more informa- tive projection basis for the token space. For simplicity of notation, we fix the layer and omit the layer superscript in the following derivations. Consider 2,, c Zi where e, is the j-th standard basis vector. The gradients then be rewritten as:

---

Published in Transactions on Machine Learning Research (01/2026)
VAph(zi)= Σ VAgau. Vaglw(zi) = AR.
In other words, the gradient of [w(2,) is a weighted version of the gradient of f2(2)). From this perspeclive, by manipulating the weighting vector Wi, we can prioritize the more informative dimensions of the token space. As illustrated in Section 5, this leads to improved performance of ULTra compared to variants that rely naive choice of f.
Here, crucial question arises: How do we characterize the amount of information in the token space to achieve a good projection basis for the loken space? In other words, we need to learn a matrix W = (W1...,,,,, to perform more effective segmentation. To this end, we introduce a perturbation on the input image and, by optimizing W, aim to minimize the divergence of specific tokens from their unperturbed versions. This motivation is supported by prior findings showing that transformers remain robust under noisy inputs Zhou et al. (2022), suggesting that they encode information in noise-resislanl representations.
Let x be an input image and x its perturbed version corresponding to segmentation class c, which can be computed as:
x
(9)
where Pø² is a perturbation strategy that applies perturbations to the region that are not predicted for class c C using the initial segmentation. Moreover, 6 quantifies the amount of perturbation, and for O, we have x =.
Now, optimize the parameters W, we formulate the problem as:
min ERWD W
Σ
Σ
dw(z;, 2i)
s.t. || Ak E (1,",,n)
(10)
OEC (i)x)==)))
where dw, 2.) denotes the divergence between the two vectors based on the projection basis w., |I- lll is the Euclidean norm, and x, and z, represent the i-th input token and latent token, respectively. In particular, we do not perturb the regions recognized by ULTraw that contain localized information. This encourages the model to become more aware of self-attended regions, producing sharper and more focused heatmaps over positively contributing areas. Consequently, the model's moccisionmmkiing becomes more precise in these critical regions. Qualitative examples and additional discussion of ULTraw are provided in Appendix C.
In our experiments, we parameterize cach projection vector as 0x/1|0xl22 Hence, based on the for- mulation in Equation 10, to optimize the parameters Ok, we define the Self-Consislency loss as follows:
Lad θ) Σ Σ
W (.a-2,]²
(11)
(EI x
where w, 0./110,1, 1 la and θ [ ... 0T1 We use the term 'Self" because leverage the model's own segmentation results, obtained using UL.Traw, to enhance its performance through systematic weighting approach. Experimentally, we found that lightly optimizing O on a limited number of samples improves the model's segmentation performance.
4.2 Latent Token Interpretability Assessment
A key challenge in interprelability is the lack of universal evaluation metric Chefer et al. (2021b); Wu et al. (2024). In supervised settings, explainability is typically evaluated indirectly via downstream tasks (c.g., semantic segmentation) or perturbation tests. Assuming strong explanation maps, performance should improve or coherent output changes should occur under perturbation. Since tokens in ULTra naturally have no labels, we extend this principle to the unsupervised domain. As we already discussed how unsupervised semantic segmentation serves as an interpretability evaluation approach for ULTra, in this section we evaluate ULTra using additional introduced metrics. Specifically, designed two tasks: (i) Perturbation Test and
2The perturbation strategy defined based the model parameters since the model involved in distinguishing the perturbation using segmentation.

---

Published in Transactions on Machine Learning Research (01/2026)
(a) Original Image
(b) Explanation Relevency Map
Example 1: Elephant and zebra scene.
Example Jungle with multiple animals.
Figure 4: Two examples illustrating the model's decision-making across layers. Columns correspond to progressively deeper layers. The first row for each image shows the CLS token, while the subsequent rows show three selected tokens (highlighted by red squares). Deeper layers capture richer semantics: in Example 1, elephant, zebra, and background tokens become increasingly distinct as go through layers. The CLS token represents both animals but does not differentiate background types (sky, ground, water), whereas token-level representations do. In Example 2, which contains multiple animals, the CLS token captures only the tiger and part of the elephant, while other tokens represent additional objects. Interestingly, the model confuses the parrot with part the rainbow due to similar colors.
(ii) Object Selection. Additionally, in Section 5.3, propose quantitative metrics to evaluate our method on these tasks to provide deeper insights into the interpretability of ULTra by identifying influential regions and behaviors within the model's decision-making process.
Perturbation Test. The perturbation test is a widely used metric for evaluating explainability methods Chefer al. (2021b); Wu et al. (2024). This approach involves perturbing parts of an image based on the explanation map generated by the method and measuring the resulting change in the predicted class probability. Inspired by this technique, instead of using class probability, leverage the change in the

---

Published in Transactions on Machine Learning Research (01/2026)
embedding space design perturbation-basee validation test. This allows us to assess the reliability of the explanation maps generated for the latent tokens.
Object Selection. In this task, we convert the upsampled explanation map S into a bi- nary segmentation mask using threshold 7, where the binary mask M, is defined as:
Method
Model
Training
ACC
U. mloU
IIC
RI8+FPN
21.8
6.7
PICIE
RI8+FPN
48.1
13.8
VIT-S/8
28.7
11.3
M [r. vl
o. if 5 otherwise.
(12)
DINO
VTT-S/16
22.0
VIT-B/8
30.5
9.6
ACSeg
VIT-S/16
16.4
TransFGU
VIT-S/8
52.7
17.5
Here, 51 [z, y] represents the relevance value at position (z. y] in 51)) with the thresh- old. When M, [z, v] = 1, it indicates that the position [z. y/ belongs to the object region.
VIT-S/8
48.3
24.5
STEGO
VTT-S/16
52.5
23.7
VIT-B/8
56.9
28.2
VIT-S/8
57.2
24.6
STEGO HHP
VIT-S/16
54.5
24.3
Our findings indicate that as tokens propa- gate through the network, they refine their ob- ject representation while retaining the seman- tic meaning of their associated image patches, performing Object Selection. Figure 4 visually illustrates this process. Deeper layers show richer semantic awareness, with tokens grad- ually capturing entire objects like the clophant zebra, or the background, while the CLS token captures both the zebra and elephant. For given patch token X,, the object it most strongly represents is denoted as class k, The latent token z generates explanation map that highlights areas with higher values associ- ated with class k; in the whole image, includ- ing x, After applying threshold, this map becomes a binary sogmentation mask expected exhibit a high Intersection over Union (IoU) with the corresponding class k; region in the image. An illustrative example is shown in Fig- ure 2.
VIT-S/8
56.3
25.6
DepthG
VIT-B/8
58.6
29.0
U2Sog
R50
63.9
30.2
VTT-B/32
60.6
34.1
ULTra
CLIP
VIT-B/16
63.8
34.0
VIT-L//14
67.9
38.2
VTT-B/32
X
60.8
34.6
Uliha
CLIP
VTT-B/16
63.0
33.2
VIT-L,/14
66.5
37.5
VIT-B/32
59.5
32.6
ULTra
CLIP
VIT-B/16
53.6
26.6
VIT-L/14
59.0
31.7
VIT-S/16
67.2
34.4
ULTraw
VIT-B/16
67.4
37.7
ULTra
DINO
VIT-S/16
66.4
33.3
VIT-B/16
X
67.3
35.6
ULTra
VIT-S/16
63.4
31.6
VIT-B/16
63.0
31.3
Table 1: Comparison of unsupervised segmentation meth- ods the COCO-Stuff dataset.
4.3 Interpreting LLMs in Text Summarization
In this section, we examine how our interpretability framework can be applied to text summarization tasks, taking steps toward uncovering the underlying intent of I.L.Ms. In Section 5.4, we qualitatively evaluate ULTras using two samples by visualizing the regions of the input context that an LLM prioritizes while interpreting given TL;DR summary. This analysis reveals key input regions shaping the model's decisions, aiding in understanding how concise and relevant summaries are generated.
For this task, we concatenate the context and the summary with a separator token. After feeding this input into the model, we compute the relevance scores the TL;DR, tokens with respect to the context tokens.
We then average these scores for each token in obtain scalar value, referred to as the Token Contribution Score, { E RR,, which highlights the contribution each context token interpreting the summary y within the given context. Accordingly, x(0 is computed as:

---

Published in Transactions on Machine Learning Research (01/2026)
= Σ
50 w s4
, ,2/3,
{1,
(13)
where
denotes the number of tokens in the text.
5 Experiments & Results
5.1
Experimental Setup
Datasets. In experiments, evaluate model performance several unsupervised semantic segmenta- tion benchmarks, focusing vision-related tasks. We conducted experiments four datasets: COCO-Stuff 27 Caesar et al. (2018), PASCAL VOC 2012 Everingham & Winn (2011), Potsdam-3 ISPRS (2018), and Cityscapes Cordis et. al. (2016).
For our qualitative analysis of LLM interpretation in the task of text summarization, as described Section 4.3, we utilized the TL;DR. datasot (Stiennon et al., 2022). This dataset contains summary comparisons with human feedback collected by OpenAL.
Models. For all experiments in the vision tasks, used different protrained versions of CLIP's image encoder (Radford et al., 2021) well DINO ViT-S/16 and ViT-B/16 (Caron el al., 2021). For interpreting text summarization, as described in Section 4.3, we used the Llama-2-78 language model (Touvron et al., 2023). All experiments were run on 4 NVIDIA A100-80GB GPUs.
Further details of the datasets and models used are provided in Appendix
Method
Model
Training
U. mloU
5.2 Semantic Segmentation
IIC
R18+FPN
9.8
To evaluate the effectiveness of our approach, we use the Unsupervised mean Intersection over Union (U. mloU) and Unsupervised Pixel Accu- racy (U. ACC) metries. The experimental results the COCO-Stuff, PASCAL VOC, Potsdam, and Cityscapes datasets reported in Tables 1, 3, 4, and 2, respectively. In these lables, the Training column indicates whether any additional training is required. Morcover we present the re- sult of using the threshold Ç instead predefined number of class l in the appendix along with its sensitivity analysis.
MaskContrast
R50
/
35.0
Leopart
VIT-S/16
41.7
TransFGU
VIT-S/8
37.2
MaskDistill
VIT-S/16 + R50
42.0
ACSeg
VIT-S/16
47.1
VIT-B/32
51.2
ULTra
CLIP
ViT-B/16
50.9
VIT-L/14
49.1
VIT-B/32
X
49.2
ULTra CLIP GLPP
ViT-B/16
X 48.3
We benchmarked the segmentation performance of our approach against several state-of-the-art (SOTA) methods in unsupervised segmentation. The projection matrix W in ULTraw was opti- mized using the ADAM optimizer with a learning of 0.01 and a batch size of 32, over total of 256 training samples and 10 epochs. This com- pact training setup suffices, W implements linear mapping with a parameter count equal to the embedding dimensionality.
VIT-L/14
X
48.7
ViT-B/32
X 50.0
UL.Tra
CLIP
VIT-B/16
X 40.0
VIT-L/14
X 45.2
ULTray
DINO
VIT-S/16
/ 48.9
VIT-B/16
50.5
ULTra
DINO
VIT-S/16
X 47.9
VIT-B/16
X 50.0
ULTraa
DINO
VIT-S/16
X
46.9
VIT-B/16
50.0 X
Among the proposed variants of ULTra, ULTraw achieves the highest performance, requiring only small number of training samples. Notably, even when training data is available, ULTrag still
Table Comparison on the PASCAL VOC 2012
dataset. The Training column Indicates fino-tuming required
10

---

Published in Transactions on Machine Learning Research (01/2026)
Method
Model
Training
U. mloU
Method
Model
Training
U. ACC
IIC
R18+FPN
IC
RI8+FPN
65.1
PICIE
R18+FPN
12.3
DINO
VIT-S/8
71.3
STEGO
VIT-B/8
21.0
STEGO
VIT-S/8
/
77.0
VIT-S/8
STEGO +HP
ViT-B/8
18.4
DepthG
VIT-S/8
80.4
DepthG
VTT-B/8
23.1
ViT-B/32
V
78.7
ULTraw
CLIP
VIT-B/16
80.9
ViT-B/32
17.6
VIT-L/14
82.4
ULTra
CLIP
VIT-B/16
24.8
VIT-L/14
25.1
ViT-B/32
X
78.3
Ul
VTT-B/16
X
80.9
ViT-B/32
ViT-L/14
82.8
ULTma CLP
VIT-B/16
21.2
ViT-L/14
21.2
VIT-B/32
X
78.1
ULitag
CLAP
VIT-B/16
X
70.4
VIT-B/32
20.4
VIT-L/14
X
75.5
ULTrag CLIP
VIT-B/16
20.9
VIT-L/14
23.0
ULTraw
ViT-S/16
79.0
VIT-B/16
V
80.7
ULTray
Dino
VIT-B/16
VIT-S/16
26.5
25.8
ULTrag
Dino
VIT-S/16
77.4
VTT-B/16
X
80.8
ULiTaa Dino
VTT-B/16
VTT-S/16
24.2
25.7
ULTrag
Dino
VIT-S/16
75.7
VIT-B/16
X
76.8
ULT Dino
ViT-S/16
22.9
VIT-B/16
23.0
Table 4: Comparison of different unsupervised seg-
mentation methods on the Potsdam dalaset. The Training column Indicates if fine-tuning is required
Table 3: Comparison of different unsupervised seg- mentation methods on the Cityscapes dataset.
achieves state-of-the-art results several bench- marks.
For some models, such as ViT-L/14, no existing baseline is available for direct comparison, highlighting the versatility of ULTra across different architectures. Furthermore, we conducted an ablation study related to the model depth Appendix A.
We hypothesize that ULTra's strong training training-free performance arises from two key factors: (i) Comprehensive token utilization: leveraging information from all latent tokens yields richer representations than relying solely the final CLS token; and (ii) Information-preeerviii aggregation: projecting embeddings into the input space before aggregation reduces information loss compared to methods that aggregate first. As shown in Figure 4, the CLS token loses substantial information that remains accessible through individual tokens.
Despite being zero-shot, our method is computationally intensive for semantic segmentation, since generating explanation maps requires computing multiple gradients per token (sce Appendix B). Future work could explore approximation or more efficient gradient strategies (see Section 6).
5.3 Interpretability Evaluations
Perturbation Test. To assess the reliability of the explanation maps, conduct perturbation test by selectively altering image regions based on the explanation map s(t) for each token The perturbation is applied the patch level while ensuring the total perturbed area remains consistent across all cases.
We consider two types of perturbations:
(i) Posétive Perturbation: Removing highly relevant regions, which should significantly affect the token's representation.
(ii) Negative Perturbation: Removing less relevant regions, which should have minimal impact.
Two types of modifications are introduced. In the masking perturbation, selected patches are replaced with zeros, effectively removing the corresponding visual information. This is applied to both highly relevant
11

---

Published in Transactions on Machine Learning Research (01/2026)
VOC
Potsdam
COCO
Cityscapes
Model
Perturb.
Neg.
Pos. f
Neg. +
Pos.
Neg.
Pos.
Neg.
Pos. f
Mask
13.16
15.51
12.35
14.02
12.62
15.21
9.95
14.42
ViT-B/32
Noise
6.97
10.23
9.39
11.44
6.83
10.26
5.99
11.13
Mask
14.68
17.5
13.91
15.51
14.16
17.39
11.33
16.27
ViT-B/16
Noise
9.87
14.03
13.5
14.45
10.14
14.12
9.46
14.22
Mask 6.66
9.43
6.61
8.99
6.38
9.31
5.36
8.86
VIT-L/14
Noise
4.45
7.88
4.95
7.96
4.27
7.74
3.89
7.77
Table 5: Average loken vector differences for ViT models under mask and noise perturbation tests across multiple datasets, highlighting the impact of positive and negative perturbations based on the relevancy map on token representations.
regions (positive masking) and less relevant areas (negative masking). In contrast, the noise perturbation introduces Gaussian noise to the same sets of patches, adding controlled randomness to test the robustness of token representations.
The noise follows a standard normal distribution with standard deviation of 0.3. Figure 5 provides detailed visualization of the perturbation test conducted on sample image from the PASCAL VOC dataset using the CLIP ViT-B/32 model. It illustrates the model'ss explanation map (d), and compares the effects of both positive (b, c) and negative (e, f) perturbations applied through masking and Gaussian noise, demonstrating how altering semantically relevant regions leads to more significant changes in the model's internal representations.
To ensure fairness, the initial token's patch remains unchanged in both per- turbations, as its information is directly propagated to the larget loken through skip connections, Given a perturbed rep- resentation z(1), we measure the devia- tion from the original token representa- (!) tion z; using the Euclidean distance, we then compute the average deviation over the entire dataset by selecting k random tokens per image and aggregating the dis- tances:
(a) Original
(b) Pos. Mask
(c) Pos. Noise
dguc
1911
j=1
(14) where M the number of images, and le tokens are randomly selected from each. For our results, set 10. The per- turbation test results are presented in Ta- ble 5
(d) Explain. Map
(c) NeN. Mask
(f) Neg. Noisc
Figure 5: Effect of perturbations on sample image from the PASCAL VOC dataset, where the model used is CLIP ViT-B/32.
Positive pert perturrations result in greater deviations compared to negative ones, confirming that the explanation maps effectively highlight influential regions. The consistency of these results across multiple datasets further validates the reliability of our interprotability framework. Similar results also hold for cosine similarity. However, due redundancy and the fact that cosine similarity is bounded between and 1, which results in weaker rasts, do not report these results.
12

---

Published in Transactions on Machine Learning Research (01/2026)
How do 1 0 M| stop feeling bad about myself for having bo relationahip experience at POST just seems like everyone know bad a thing with someone by this point| ||| ve made out with a girl once ( who later told me that was mistake ] and I feel like girls always reject only starting to get ups et that 1 ve never had any kind of relationship got rejected by girl who thought was into me and ve been feeling
I need help about those Eeclings POST 11 am B M She Isa 7 F We relationship which started
got troubles year ago I made mist akers she made scenario is explained in here: Now I Feel like I bate her |I used to adm ire her a lot, but I m really disappoinl ed with her and with her Character just realized still like
pure friendship last situation
friend| Which is perfectly acceptable but just
hate her And just after that bad situation happened realized she also had that feeling Well now we both hate and love each other What to do What think What feel
So well yeah
bad ever since I just don It know what's wrong with me I guess
little bit skin ny work reqularly though ), but show every day
imional into today our friend asked
for
dress pretty well all that stuff
help with some calculations and I made a jo ke about our physics teacher
She laughed and smiled at me just like one year ago but after she realized that. she soemed kind gr ouch
(a) TL;DR: I've had very bad luck with girls my whole life and I don't know how to get my confidence up.
(b) TL;DR: I still like her but my rational side says 'no, she is trash person'.
Visualization of Token Contribution Scores (A highlighting the relevance of context tokens in interpreting the summary. Each token is colored proportionally to its X value. These visualizations demonstrate the model's ability to identify key semantic elements in the context for generating relevant summaries. Further analysis and examples are provided in Appendix H.
Figure 6:
Object Selection. To quantily aligument, we compute the IoU by converting the explanation map s/D into a binary mask M, and comparing with the ground-truth mask. We propose the Initial Token IoU (ITloU) metrie, which measures how well the explanation maps of input tokens align with their respective class masks. The ITloU is calculated as:
c
ITlou
LoU(M,)) G.), X,ET.
Σ
(15)
T
where C' denotes the number of classes, T. represents the set of tokens associated with class i, M is the binary segmentation mask for token xj within class i, and G; is the ground-truth mask for class i in image x. The inner sum averages the loU for tokens in T, for each class, and the outer sum then averages across all classes. Using a threshold of 0.2, our ITIoU metric achieves an average score of 37 37.84% on the COCO-Stuff validation dataset and 39.51% on the PASCAL VOC dataset. A more detailed analysis of ITIoU is provided in Appendix F.
5.4 Interpretable Text Summarization
In this experiment, we used a Supervised Fine-Tuned (SFT) version of Llama-2-7B trained on the UI- traFeedback Binarized (UFB) dataset (Cui et al., 2024). Additionally, we aligned the model to the text summarization task on the TL;DR datasct (Stiennon et al., 2022) using the Direct Preference Optimization (DPO) method (Rafailov et al., 2024) for 1,000 iterations, with learning rate of 5 x 10-6 and 3 = 0.5. To validate our framework, we selected the preferred response (TL;DR) of each sample in the dataset, denoted by y, and used it as the summary of the context x. The result can be seen in Figure 6
In example (a), semantically significant words such as 'relationship', 'experience', 'rejection', and 'never' are prominently highlighted, reflecting the model's interpretation of the person's 's struggles with relationships and feelings of rejection. Additionally, the highlighting of the question at the beginning of the context 'How do I stop feeling bad...' suggests the model recognizes the presence of uncertainty and request for guidance, which is encapsulated in the summary as 'I don't know.'
In example (b), x scores reveal the model's focus on words such as 'feelings', 'hate', 'disappoint', 'love, and 'like', which correspond to the person's mixed emotions toward their girlfriend, as described in the summary. The apparent contradiction between 'love' and 'trashness' in the summary appears to be derived from these highlighted terms, suggesting the model understands the conflicting emotions present in the text. Furthermore, the focus on 'character' reflects the summary's judgmental tone, suggesting the model associates this term with a personality assessment.
Quantitative Validations. While the qualitative visualizations in Figure 6 illustrate ULTra's ability to highlight semantically relevant context tokens, further assess the faithfulness of these attributions
13

---

Published in Transactions on Machine Learning Research (01/2026)
using the Comprehensiveness metric (DeYoung et al., 2020). This metric measures how much the model's confidence in producing the gold summary decreases when important context tokens are removed.
Following Eq. 13, we compute token-level contribution scores x!" and select the top-k% tokens the rationale set Rg. Using the pretrained language model π, define the summary support score the log- probability of the gold summary y given context x:
(z. v) = log ( x)) = Σ
log (1s
x, Mer),
(16)
and compute the Comprehensineness score:
Compy(e, v) S(z. v) S(x \ Rx.y)
(17)
S(x,
where larger values indicate that the removed tokens were more necessary for maintaining model confidence.
As shown in Figure 7, ULTra consistently achieves higher Comp& values than the random bascline, which selects rationale sets Rs uniformly at random. At small budgets (k < 15%), the performance of both mot is similar; however, as & increases, ULTra exhibits steady improvements, highlighting its ability to identify context regions that causally influence model predictions.
This quantitative analysis further confirms that UL- Tra's highlighted tokens are not only interpretable but also faithfully reflect the model's internal rea- soning process. We also present additional qualita- tive examples and a detailed discussion of the metric in Appendix H.
ULTra
Randem :lc
standard deviation
0.3
Comp4
0.1
These language- based experiments are not intended benchmark for interpretable summarization, es- pecially given the absence of standardized evalu- alion metries, but rather to demonstrate the ver- satility of ULTra across modalities. These exper- iments also highlight an important direction for future research: applying post-hoe interpretability techniques to better understand and align large lan- guage models within frameworks such as RLHF (Christiano et al., 2023; Stiennon et al., 2022; Ouyang et al., 2022) and direct preference optimiza-
-0.
Selection rate (%)
Figure Average Comprehensivennss score Comp; on the validation set across different selection rates k for ULTra and random baseline. UL.Tra shows consistent increase, indicating that its selected tokens are critical for preserving model confidence in the summary.
tion (Rafailov et al., 2024; Azar et al., 2023; L'thayarahh et al., 2024), where understanding model behavior and intent is critical.
6 Concluding Remarks and Limitations
Summary. We introduced a framework for interpreting latent tokens in Transformers, providing new insights into the semantic information encoded within them. Our method achieves state-of-the-art performance unsupervised semantic segmentation across multiple datasets and settings, notably without requiring any additional training. Beyond segmentation, we validated the approach through perturbation tests and object selection, highlighting its broader applicability for probing Transformer behavior at the layer level.
Discussion. A central challenge in interprotability is its evaluation. Traditional methods often roly on class logits as the primary signal, employing strategies such as supervised semantic segmentation or perturbation tests to measure changes in these logits. However, even for explanations final predictions, it remains unclear whether such approaches faithfully reflect the quality of interpretability. In our setting, many of these metrics are not directly applicable due fundamental methodological differences. To address this, we evaluated our framework through the lens of unsupervised semantic segmentation. While baseline methods were
14

---

Published in Transactions on Machine Learning Research (01/2026)
specifically designed for this task, approach achieved superior results by aggregating explanation maps, demonstrrating both the potential of latent token interpretability and the importance of effective aggregation strategies. For instance, clustering- based techniques could further improve performance, especially at smaller patch sizes; when the patch size is 8, the large number of tokens makes aggregation particularly challenging.
Limitations and Future Work. Despite operating in a zero-shot setting, our method incurs notable compu- tational cost for semantic segmentation. High inference times result from computing multiple gradients for each token when generating explanation maps. We provide FLOPs and estimateddruntime analysis (batch size 64) in Appendix B. Future work could alleviate this limitation by exploring more efficient strategies, including: (i) approxinations in the attention mechanism (e.g., sparse attention or token pruning), (ii) gra- dient approximations (e.g,, using fewer layers), and (iii) optimized computations such as parallel processing or alternative interpretability signals. In particular, we examines layer selection effect in Appendix A, where for some models, using middle layers instead of the last layer improves both accuracy and computational efficiency, whereas in others, a trade-off between performance and computational cost remains.
7 Broader Impact Statement
Our work contribules lo the interpretability large language models (LLMs) by identifying and analyzing latent token structures. While interpretability can enhance transparency, accountability, and trust in LLM- based systems, it also introduces potential risks. In particular, deeper insights into model internals could be misused for adversarial purposes, such crafting more effective jailbreak prompts, reverse engineering proprietary models, or developing targeted manipulations that reduce model safety. Furthermore, inter- pretability methods risk conveying false sense of security if stakeholders overestimate the completeness or reliability of the provided explanations.
Despite these risks, we believe that developing interpretability methods is ultimately beneficial for safe and responsible deployment of LLMs. By making the limitations and inner workings of these systems more visible, our work can help practitioners identify failure modes, mitigate biases, and design more robust safeguards. Careful communication of the scope and limitations of our method will be essential to minimize potential misuse and prevent overconfidence in the explanations it provides.
References
Samira Abnar and Willem Zuidema. arXiv:2005.0000 2020. 2, 3
Quantifying attention flow in transformers. arXiv preprint
Mohammad Gheshlaghi Azar, Mark Rowland, Bilal Piot, Daniel Guo, Daniele Calandriello, Michal Valko, and Rémi Munos. A general theoretical paradigm to understand learning from human preferences, 2023. URL http:///axvvvvooe.oeeee
1220000.0023........... 12036. 14
Daniel Bolya, Jian Wang, Ambrish Pujari, Ross Wightman, and Alexander Kirillov. Token merging: Your vit but faster. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3838 3848, 2023.
Walid Bousselham, Angic Boggust, Sofian Chaybouti, Hondrik Strobelt, and Hilde Kuchne. Legrad: An explainability method for vision transformers via feature formation sensitivity. arXiv preprinl arXiv:2404.005 03211, 2024. 3
Holger Caesar, Jasper Uijlings, and Vittorio Ferrari. Coco-stuff: Thing and stuff classes in context. In Computer vision and pattern recognition (CVPR), 2018 IEEE conference on. IEEE, 2018. 10
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9650-9660, 2021. 3, 10, 23
Hila Chefer, Shir Gur, and Lior Wolf. Generic attention-modd explainability for interpreting bi-modal and encoder-decoder transformers. In 2021 IREF/CVF International Conference on Computer Vision (ICCV),
15

---

Published in Transactions on Machine Learning Research (01/2026)
pp. 387-396. IEEE, October 2021a. doi: 10.1009/2cc::89222222..0000 URL http://dx.doi.orgg doi. 1109/ICCV48922. 2021. .0005.. 3
Hila Chefer, Shir Gur, and Lior Wolf. Transformer interpretability beyond attention visualization. Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 782-79777777777 2021b. 2,
3, 7,8
Haozhe Chen, Junfeng Yang, Carl Vondrick, and Chengzhi Mao. Invite: Interpret and control vision vision-language models with text explanations. In The Twelfth International Conference on Learning Representations, 2024. 2,
Tao Chen, Yazhou Yao, Lei Zhang, Qiong Wang, Guo-Sen Xie, and Fumin Shen. Saliency guided inter- and intra-class relation constraints for weakly supervised semantic segmentation. IEEE Transactions on Multimedia, 25:1727-1737, 2023. ISSN 1941-0077. doi: 10.1109/tmm.2222.3357455 202. 3157788. URL http:
//dx. wwt.:oi.o///..... 1tt/:/...1111//1 /MM09/TMM.... 2022.3157444
Jang Hyun Cho, Utkarsh Mall, Kavita Bala, and Bharath Hariharan. Picie: Unsupervised semantic seg- mentation using invariance and equivariance in clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Paftern Recognition, pp. 16794-16804, 2021.
Jaegul Choe and Hyunjung Shim. Attention-based dropout layer for weakly supervised object localization. In Proceedings of the LEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2219 2228, 2019. 3
Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforce- ment learning from human preferences, 2023. URL
httplaawww.....raaaa////eeeeeeee 177110377611..... 14
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understand- ing. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognation (CVPR), 2016. 10, 23
Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang Hc, Wei Zhu, Yuan Ni, Guotong Xic, Ruobing Xie, Yankai Lin, Zhiyuan Liu, and Maosong Sun. Ultrafeedback: Boosting language models with scaled ai feedback, 2024. URL atttp:://arxss org/ ats//31s/231000 01377. 13
Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. In The Twelfth International Conference on Learning Representations, 2024. URL /ttpps///openrevverr net/forum?id
3,
Jay De Sarthak Jain, Nazneen Fatemna Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C. Wallace. ERASER: benchmark to evaluate rationalized NLP models. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4443-4458. Online, July 2020. Association for Computational Linguis- ties. doi: (18653/w112020..o/ m08...08............ mainimaaamaa URL tttps://aclanthologg org/2020..c.... acl-main. aecl-main.408.. 408/.. 14, 26
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohna Zhai, Thomas Un- terthiner, Mostafa. Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x16 words:
Transformers for image recognition scale. arXiv preprint arXic:2010. 1 11929, 2020.
Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model alignment as prospect thcoretic optimization, 2024. URL https://arxiv/ wttp:/ii.org/ab/////eae 2402. 01306. 14
Mark Everingham and John Winn. The pascal visual object classes challenge 2011 (voc2011) development kit. Pattern Analysis, Statistical Modelling and Computational Learning, Tech. Rep, 8, 2011. 10, 23
Mark Hamilton, Zhoutong Zhang, Bharath Hariharan, Noah Snavely, and William T Freeman. Unsupervised semantic segmentation by distilling feature correspondences. arXiv preprint ar N2w:2203.08/ 14, 2022. 2, 3
16

---

Published in Transactions on Machine Learning Research (01/2026)
ISPRS. 2d semantic labeling contest potsdam. hawpaneeenaa/ddd--e--aaeeellllle
aspx, 2018. 10
Sarthak Jain and Byron C Wallace. Attention is not explanation. arXiv preprint arx:v11902 10186, 2019. 3
Xu Ji, Joao Henriques, and Andrea Vedaldi. Invariant information clustering for unsupervised image classification and segmentation. In Proceedings of the LEEE/CVF international conference on computer vision, pp. 9865-9874, 2019. 3
Tsung- Wei Ke, Jyh-Jing Hwang, Yunhui Guo, Xudong Wang, and Stella X Yu. Unsupervised hicrarchi- cal semantic segmentation with multiview cosegmentation and clustering transformers. orv preprinl 11Xiv:2904,11432, 2022. 3
Mengcheng Lan, Xinjiang Wang, Yiping Ke, Jiaxing Xu, Litong Feng, and Wayne Zhang. Smooseg: smooth- ness prior for unsupervised semantic segmentation. Advances in Neural Information Processing Systems, 36:11333-11333 2023. 3
Seungju Lee, Kyumin Cho, Eunji Kwon, Sejin Park, Seojeong Kim, and Seokhyeong Kang. Vit-togo: Vision transformer accelerator with grouped token pruning. In 2024 Design, Automation & Test Europe Conference & Exhibition (DATE), pp. 1-6. IEEE, March 2024. doi: 0TT0:///DDDEEE040022220100155 URL attp://dx.doi: doi. www./../............ 222.29 2024. 10546804.
Kehan Li, Zhennan Wang, Zesen Cheng, Runyi Yu, Yian Zhao, Guoli Song, Chang Liu, Li Yuan, and Jie Chen. Acseg: Adaptive conceptualization for unsupervised semantic sogmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7162 7172, 2023. 2, 3
Youwei Liang, Chongjian GE, Zhan Tong, Yibing Song, Jue Wang, and Pengtao Xie. EVit: Expediting vision transformers via token reorganizations. In International Conference on Learning Representations, 2022. URL https:// http:ppewreeiieeeeeooreerrrneeee het.://orrrmm..ccByyymx.....
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014. 23
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hicrarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference compuler vision, pp. 10012 10022, 2021.
Scott Lundberg. A unified approach to interpreting model predictions. arXiv preprint arXiv: 1exxx:1775.... 705.07874. 2017.
Dantong Niu, Xudong Wang, Xinyang Han, Long Lian, Roei Herzig, and Trevor Darrell. Unsupervised universal image sogmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognilion, pp. 22744 22754, 2024. 3
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddic Simons, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models follow instructions with human feedback, 2022. URL https://arxx: wwwaaiioirgaaaallaaaaaaaaaaaaaae abg/333/2003. 02155. 14
Bingyi Pan, Dejia Zeng. Zhengping Shou, Zhiyu Yan, Lin Zhang, Gang Yuan, Houqiang Li, and Shuicheng Yan. la-red?: Interpretability-wwaa redundancy reduction for vision transformers. In Advances in Neural Information Processing Systems, pp. 13845-13856, 2021.
Yao Qiang, Chengyin Li, Prashant Khanduri, and Dongxiao Zhu. Interpretability-aaarr vision transformer. arXiv preprint ar Xic:23309.00 08035, 2023.
17

---

Published in Transactions on Machine Learning Research (01/2026)
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748 8763. PMLR, 2021. 10, 23
Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model, 2024. URL https: //arxiv.orgp www.ooggg//bb/oooooeee 18290. 13, 14
Daking Rai, Yilum Zhou, Shi Feng, Abulhair Saparov, and Ziyu Yao. A practical review of mechanistic interpretability for transformer-based language models. arXiv preprint arXiv2207.006446 2024. 3
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. ' why should trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016. 3
Ramprasaath Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient- based localization. Pro- ceedingg of the IEEE international conference on compater vision, pp. 618 (8866,,,660 2017. 1,
Leon Sick, Dominik Engel, Pedro Hermosilla, and Timo Ropinski. Unsupervised semantic segmentation
through depth-guided feature correlation and sampling. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp.
2024. 2, 3
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising
image classification models and saliency maps. arXiv preprint arXin:3112.6034, 2014. 1,
Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. Learning to summarize from human foedback, 2022. URL https://arxiv: org/abs/2009 01325. 10, 13, 14
Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Téonard Hussenot, Pier Giuseppe Sessa, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Ambrose Slone, Amélic Héliou, Andrea Tacchetti, Anna Bulanova, Antonia Paterson, Beth Tsai, Bobak Shahriari, Charline Le Lan, Christopher A. Choquetle-Choo, Clément Crepy, Daniel Cer, Daphne Ippolito, David Reid, Elena Buchatskaya, Eric Ni, Eric Noland, Geng Yan, George Tucker, Choorg--Chisttee raru, Grigory Rozhdestvenskiy, Henryk Michalewski, Ian Tenney, Ivan Grishchenko, Jacob Austin, James Keeling, Jane Labanowski, Jean- -ran-Baptiste Lespiau, Jeff Stanway, Jenny Brennan, Jeremy Chen, Johan Ferret, Justin Chiu, Justin Mao-Jones, Katherine Lee, Kathy Yu, Katie Millican, Lars Lowe Sjoesund, Lisa Lee, Lucas Dixon, Machel Reid, Maciej Mikula, Mateo Wirth, Michael Sharman, Nikolai Chinaev, Nithum Thain, Olivier Bachem, Oscar Chang, Oscar Wahltinez, Paige Bailey, Paul Michel, Petko Yolov, Rahma Chaabouni, Ramona Comanescu, Reena Jana, Rohan Anil, Ross Mcllroy, Ruibo Liu, Ryan Mullins, Samuel L Smith, Sebastian Borgcaud, Sertan Girgin, Sholto Douglas, Shree Pandya, Siamak Shakeri, So- ham De, Ted Klimenko, Tom Hennigan, Vlad Feinberg, Wojciech Stokowiec, Yu hui Chen, Zafarali Ahmed, Zhitao Gong, Tris Warkentin, Ludovic Peran, Minh Giang, Clément Farabet, Oriol Vinyals, Jeff Dean, Koray Kavukcuoglu, Demis Hassabis, Zoubin Ghahramani, Douglas Eck, Joelle Barral, Fernando Pereira, Eli Collins, Armand Joulin, Noah Fiedel, Evan Senter, Alck Andreev, and Kathleen Kencaly. Gemma: Open models based gemini research and technology, 2024. URL https: //arziv.////i..ee/ http://awwv..oggbb//////a wtt./a/////.../////// 08295.
Mu-
1
Hans Thisanke, Chamli Deshan, Kavindu Chamith, Sachith Seneviratne, Rajith Vidanaarachchi, and Damayanthi Herath. Semantic segmentation using vision transformers: A survey, Engineering Appli- cations of Artificial Intelligence, 126:106669, 2023. 1
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bash- lykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Fer- rer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,
18

---

Published in Transactions on Machine Learning Research (01/2026)
Cynthia Gao, Vedamuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Tsabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinct, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungla, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subra- manian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Augela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023. URL attpss///axxe ceepiiiv.or//ieeeee abs/2307 09288. 1, 10
Wouter Van Gansbeke, Simon Vandenhende, Stamatios Georgoulis, and Luc Van Gool. Unsupervised seman- tic segmentation by contrasting object mask proposals. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10052-10022, 2021. 3
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Usckoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Newral Information Processing Systems (NewrIPS), 2017.
Jesse Vig and Yonatan Belinkov. Analyzing the structure of attention in a transformer language model. arXiv preprint arxiv: 04284, 2019. 2, 3
Junyi Wu, Bin Duan, Weitai Kang, Hao Tang, and Yan Yan. Token Transformation Matters: Towards Faithful Post-Hoc Explanation for Vision Transformer In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Los Alamitos, CA, USA, June 2024. IEEE Computer Society.
3, 7,
Jingkang Xu, Yifan Jiang, Zhenda Wang, Vijay Vasudevan, Xiaolong Liu, Tsung- Yi Lin, Song Bai, Luc Van Gool, and Qi Dai. Groupvit: Semantic segmentation emerges from text supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 18134 18144, 2022.
Guangwei Yin, Xiaopeng Zhang, Lihui Liu, Jie Shao, Zhengming Chen, and Jianbo Wang. Dual-attention guided dropblock module for weakly supervised object localization. arXiv preprint. 3Xii:2003.04711 2020. 3
Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. arXie preprint arXiv:1311.290 2014.
Daquan Zhou, Zhiding Yu, Euze Xie, Chaowei Xiao, Auimashree Anandkumar, Jiashi Feng, and Jose Alvarez. Understanding the robustness in vision transformers. In International conference on machine learning, pp. 27378 27394. PMLR, 2022.
Adrian Ziegler and Yuki M Asano. Self-supervised learning of object parts for semantic segmentation. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 14502-14511 2022.7 3
Appendix
A Effect of Layer Depth in ViT Token Understanding
In this section, we analyze the impact of depth on our model's interpretability and sogmentation performance, providing insights into the contribution of each layer. For smaller models such as ViT-B/32 in Figure 9, deeper layers generally carry more semantic significance. However, the contribution diminishes in the final layers, suggesting that a depth of around 13 layers might be more than sufficient for the ViT effectively comprehend image content. This finding implies that even fewer layers might achieve comparable results, potentially reducing computational costs without compromising performance.
19

---

Published in Transactions on Machine Learning Research (01/2026)
Accuracy
Mesn loU
36
64-
Accuracy (56
Moon ToU (96)
62-
60-
27
58-
3
10
15
20
Layer Number
Figure 8: The segmentation performance of the CLIP ViT-L/14 model in an unsupervised setting, measured by accuracy and loU, changes across different layers. As layer depth increases, both metries improve at first, reaching their highest point in the mid-depth layers. However, performance slightly declines deeper layers. This suggests that while deeper layers capture richer semantic details, they also introduce complexity that does not always improve segmentation. Additionally, the limitations of labeled datasets, especially the ambiguity in object definitions, further restrict the model's ability to achieve better segmentation, despite its strong interpretability.
Mon
Meon loll (95)
Armmary
U. Autrag
16
Lager
(a) COCO-Staff
(b) PASCAL VOC
Figure 9: Ablation study on two evaluation metrics across layers ViT-B/32. These plots demonstrate a progressive improvement in semantic segmentation performance in the deeper layers of the transformer model. This enhancement is altributed to latent tokens capluring more meaningful segment structures, resulting increasingly accurate and refined semantic representations.
We observe an intriguing behavior in the initial layers, where performance initially declines before improving. This phenomenon is also visually evident in Figure 4, where the attention maps in the first. layer appear to focus on the entire image. This suggests that, initially, the token examines the image as a whole before selectively gathering information from tokens with similar characteristics.
In the CLIP ViT-I./14 model, deeper layers capture more refined and detailed feature representations. Unlike shallower models, where sogmentation performance stabilizes carly, VIT-L/14 benefits from its depth by gradually extracting richer hierarchical features. As shown in Figure 8, accuracy and Mean IoU improve layers deepen, with segmentation performance peaking around the mid-depth layers. However, in deeper layers (beyond layer 20), segmentation quality slightly declines. This suggests that while the model gains a better understanding of high-level semantics, it loses some spatial precision. This trade-off occurs because later layers prioritize semantic raction over detailed spatial structures.
While deeper models like VIT-L/14 offer improved feature extraction, more layers do not always lead better segmentation. Instead, an optimal balance between depth and spatial representation is necessary for effective segmentation.
20

---

Published in Transactions on Machine Learning Research (01/2026)
We note that, for fixed model, the overall performance remains relatively consis- tent across different datasets, as illustrated in Figure 4. This observation suggests the presence of a sweet spot region where the model achieves optimal performance. A similar pattern can be observed in Fig- ure 8, where flat region the curve represents this stable performance zone. Interestingly, the location of this sweet spot tends to be similar across datasets for the same model. In practice, it can be identified experimentally by evaluat- ing the model's performance on small validation subset and then verifying its consistency on other datascts.
Original
ULTras ULTraw
B Complexity Analysis of ULTra for Segmenetation
In this section, we analyze the time com- plexity of ULTra for the segmentation lask. To compute the final segmentation mask, our method requires the heatmaps of all tokens from fixed layer. As result, the computational complexity is significantly higher than that of meth- ods that rely solely on & forward pass and are explicitly designed for segmen- tation. In our case, We perform a single forward pass and then backward passes (one backward pass per token, where n denotes the number of tokens). The backward passes constitute the primary source of computational overhead.
To quantify this cost, we report the to- tal number of FLOPs needed for both the forward and backward passes fixed layer. The results presented in Table 6. I-t is important to note that, although our segmentation approach is computationally intensive, ULTra is pri- marily designed as an interpretability framework. Segmentation is just one ap- plication that benefits from the generated heatmaps. While the framework was not specifically developed for segmentation, it nonetheless achieves state-of-the-art performance in unsupervised semantic segmentation and notably, without any training. Moreover, segmentation is also included part of our evaluation.
Figure 10: Visual comparison of segmentation quality between ULiras and ULTraw several examples from the COCO-Stuff (CLIP ViT-B/32). Each row corresponds to a different example, and the columns show the original image, the heatmap generated by ULTras, and the refined heatmap produced by ULTraw-
C Qualitative Analysis of f.r
In this section, we visually present the improvement of ULTraw over ULTras when ULTras is used as the initial segmentation. Figure 10 provides several examples illustrating how this improvement captured during the learning of fw. Recall that ULTraw trained by perturbing the negative regions in the image and learning a representation that robust to such perturbations. This process encourages the model to develop features that are less sensitive to the negative regions and more focused on the positive regions. Consequently, the resulting heatmaps become centered the target object that previously captured by ULTras..
21

---

Published in Transactions on Machine Learning Research (01/2026)
Model
Forward (FLOPs)
Layer
Layer
Layer
Layer 9
Layer 11
Layer 13
ViT-B/32
5.7 x 1011
5.6 x 1012
2.0 x 10t3
4.4 x 10th
7.7 x 1013
1.1 x 1014
1.7 x 1014
Ratio
1x
10x
35x
77x
135x
193x
298x
ViT-B/16
2.3 x 10
9.2 1013
3.3 10
7.3 x 10
1.2 x 10
1.9 x 10
2.8 x 10
Ratio
1x
40x
143x
317x
521x
826x
1217x
Table 6: FLOPs required to compute forward pass and to generate gradient- traseent-based explanation maps targeting different layers for batch size of 64. Ratios show the multiple relative to one forward pass.
BRO
This figure illustrates challenging samples from the COCO-Stuff 27, Potsdam-3, and PASCAL VOC that contribute to low performance due to poor dataset labeling quality. The first row displays the original images, the second row shows the ground truth labels and the third row presents our predicted segmentation.
Figure 11:
It important to note that though ULTraw depends on the initial segmentation, we do not expect it perform worse than the initial results. This is because the target token itself always lies within the positive region, and surrounding areas with positive values tend to be reinforced. In essence, ULTraw sharpens the heatmap around these positive regions. As shown in Figure 10 ULTraw further captures the regions identified by ULTras.
D Datasets Limitations
A significant challenge in deep learning is obtaining labeled datasets, which are essential for the success of deep learning mothods. However, labeling can ambiguous, as objects or attributes may be labeled separately or combined as a single entity. Our method demonstrates how ViT interprets images a zero-shot setting, capturing fine-grained or general representations. Notably, our method's predictions often exhibit logical consistency that surpasses the ground truth, effectively identifying relationships between objects.
However, since our approach does not involve supervised training, it cannot adapt to detect only the specific objects labeled in the dataset. As a result, while the logical quality of the predictions is high, the numerical metrics may decline due to mismatches with the dataset's s ground truth labels. Figure 11 illustrates this phenomenon with examples, showing instances where our method successfully captures unlabeled objects that are omitted the dataset annotations.
E
Datasets & Models
E.1 Datasets
We utilize a combination of datasets provide diverse testing ground for evaluating our method across both standard and challenging perspectives in semantic segmentation and interpretability evaluations.
22

---

Published in Transactions on Machine Learning Research (01/2026)
COCO-Stuff 27 Lin et al. (2014): A subset of the COCO dataset, featuring complex real-world scenes with pixel-level annotations across various object categories.
PASCAL VOC 2012 Everingham & Winn (2011): A widely used benchmark containing pixel-level annotations for foreground objects in structured scenes.
Potsdam-3: A high-resolution aerial-view dataset capturing urban landscapes, including buildings, roads, and vegetation, presenting additional challenges due to its large-scale top-down perspective.
Cityscapes Cordts et al. (2016): An urban street scene dataset with fine-grained pixel-level anno- tations, enabling the evaluation of segmentation performance in structured environments.
These datasets allow for a comprehensive evaluation of our approach across different environments, ensuring robustness across diverse segmentation challenges.
E.2 Models
We employ various Vision Transformers (ViTs) pre-trained on large-scale datasets. These models process images as non-overlapping patches and employ self-allention mechanisms across multiple layers to capture long-range dependencies.
Transformer Architectures
CLIP VIT Radford et al. (2021): A vision transformer trained using contrastive learning on 400 million imago-text pairs. It encodes images into shared embedding space with text prompts. CLIP variants include:
ViT-B/16: Consists of 12 Iransformer layers, a hidden size of 768, and processes images with 16x 16 patch resolution.
Similar to VIT-B/16 but with a 32x32 patch resolution, reducing computational cost at the expense of finer details.
ViT-B/32:
VIT-L/14: A larger model with 24 transformer layers, a hidden size of 1024, and a 14x14 patch resolution, providing enhanced feature extraction.
DINO VIT Caron et al. (2021): A self-supervised vision transformer trained using knowledge distillation without labeled data. It learns image reprosentations by maximizing similarity between different augmented views. Evaluated variants:
VIT-S/16: smaller model with 12 transformer layers, a hidden size 384, and a patch size of 16x16.
ViT-B/16: larger model with 12 layers, a hidden size of 768, and a patch size of 16x16, providing stronger feature representation.
F ITloU Analysis
This section evaluates the effectiveness of ITIoU in asscssing the performance of our object selection process. As expected, the final layers exhibit. superior performance compared to the earlier layers, consistent with the results illustrated Figure 13. This improvement highlights the increasing relevance of features in deeper layers for accurate object selection. Additionally, we obscrve the existence of an optimal threshold, r, which significantly influences segmentation performance. This phenomenon depicted in Figure 12, where performance trends analyzed across different threshold values.
23

---

Published in Transactions on Machine Learning Research (01/2026)
1.00
1132
0.22
0.23
11.34
0.23
1.20
0.35
0.00
0.00
0.20
0.30
0.40
0.30
0.50
5.00
0.50
Loger
Thresheld
Figure 13: Layer-wise ITIoU analysis for COCO- Stuff 27 dataset. Final layers perform significantly better than earlier ones due to their ability to cap- ture high-level semantic features. This progression is evident in the increasing ITToU values.
Figure 12: Impact of varying the threshold т on ITIoU performance for COCO-Stuff 27 dataset. The plot demonstrates the existence of an optimal T, where segmentation performance is maximized.
Original Image
Segmentation Results
Figure 14: Hicrarchical clustering tree showing the grouping of token explanation maps for all tokens in a lalent layer of the Vision Transformer, not limited to the CLS token. Each leaf node represents a single token explanation map, while higher-level nodes show aggregated clusters based on a clustering threshold ç, which controls the level of detail. Lower Ç values reveal finer details, while higher values create broader, more general clusters.
G Threshold-Based Segmentation
Hierarchical clustering is used to segment the token explanation maps, where the threshold parameter Ç controls the level of granularity. Lower values of Ç produce fine-grained segmentations, while higher values merge similar clusters, leading to broader groupings.
Figure 14 illustrates the hierarchical clustering tree, where each leaf node represents a single token explanation map. As Ç increases, multiple explanation maps are grouped into larger clusters, reducing segmentation gramularity. The rightmost part of the figure shows how different values affect the final segmentation output.
Figure 15 further quantifies the impact of Ç on segmentation performance for the COCO-Stuff 27 dataset. Accuracy and Mean loU metrics are plotted as a function of (, demonstrating that while different values of Ç yield varying levels of segmentation detail, our method remains robust across a range of threshold values.
The results confirm that selecting an appropriate Ç enables the model to segment objects at different levels of abstraction, demonstrating the adaptability of pre-trained Vision Transformers in unsupervised semantic
24

---

Published in Transactions on Machine Learning Research (01/2026)
Meni
65
Avemany (86)
61-
(96)
15
62-
60
55-
0.20
0.25
0.50
0.35
0.45
0.50
0.53
Thrshold
Figure 15: Impact of the clustering threshold Ç on segmentation performance for the COCO-Stuff 27 dataset. Accuracy and Mean loU metrics show how different values of Ç affect segmentation quality.
Dataset
U. ACC
mloU
COCO-Stuff 27
67.2
32.9
PASCAL VOC
51.9
Potsdam
74.6
Table 7: ULTra results across different datasets using the same threshold Ç - 0.4. Accuracy (U. ACC) and Mean loU (U. mloU) are reported where available.
segmentation. Table 7 summarizes the segmentation performance of our method across multiple datasets using a fixed threshold 0.4. However, discussed in Section D, not all threshold values may be equally suited for standard datasets.
H Further Analysis of Text Summarization
H.1 Additional Examples
To complement the qualitative results presented in Section 5.4, provide additional visualization examples of the TL,DR summarization task in Figure 16. These samples further validate the interpretability of the ULIra framework by demonstrating its ability to consistently highlight semantically salient tokens across diverse contexts. The extended examples illustrate how token-level relevance scores align with the distilled summaries.
Explanations:
Semantically significant. words such as Score deeply," "depression," " "culling, and "upset" are prominently highlighted, reflecting the model's interpretation of the partner's struggles and the narrator's emotional reaction. The focus on "love" " and "don'l know" further shows the model capturing both the romantic attachment and uncertainty, which are condensed into the summary.
(a):
(b): The model highlights tokens like "team," " "emotional," " "underdog win," " and "ery/teary," " focus- ing the parts about bonding with team and the emotional reaction to underdog victories. This attention shows the model captures the link between shared struggles and the narrator's tears, which is distilled into the TL.DR.
(c): The model highlights tokens such as "pretty girl," " "father," " "shy." " "well with the dad, " and "unsure," " which emphasize both the attraction and the social barriers around the situation. Strong attention is also given to the repeated question "How should I proceed how should go about asking out, showing the model identifies the narrator's uncertainty the key theme. By balancing the highlighted
25

---

Published in Transactions on Machine Learning Research (01/2026)
references to family dynamics and the narrator's hesitation, the model distills the post into the TL;DR: asking the girl out without misunderstandings awkwardness.
(d): The model highlights tokens such as "mid year evaluation," " "pay," "amount of units," "24 units away from gradualing," " and "working full time over the sumsner, which directly connect to the question of salary adjustment. Attention is also given to phrases about reflecting the "current amount of units" " taken, showing the model identifies the link between academic progress and pay. These focused regions explain why the TL;DR condenses the post into the core concern of aligning mid-year pay with completed units. By contrast, the final sentence ("I called HR... finals coming up ") receives lower scores, as it conveys anxiety rather than information directly relevant to the pay issue.
(e): Here, the model highlights tokens such as "younger brother died, " "suddenly," " "impacting him heavily," "support him," and "help him through this grief process, " which directly connect to the main concern of coping with sudden loss. Broader attention is placed on sentences describing the death and its emotional toll, showing the model captures the central event and the narrator's intent to provide support.
(f): The model highlights tokens such as "pulled over " "red light," " "ran through, " and "verbal warning," " which directly capture the core sequence of events leading to the summary. Broader attention is given to the description being stopped and told not to repeat the behavior, showing the model identifies this as the essential outcome.
H.2 Quantitative Validations
To quantitatively assess the faithfulness of the highlighted tokens produced by ULTra, we adopt the Com- prehensiveness metric introduced by DeYoung et al. (2020) for our task, mcasuring the impact of removing important context tokens on the model's behavior.
First, as in Eq. 13, we compute token-level contribution scores X IR+ denotes the contribution of the i-th context token to interpreting the summary y at fixed layer I.
Based on these scores, we select the top-&% of context tokens to form the rationale set R3. We then evaluate how the removal of these tokens affects the model's ability to support the gold summary using the same pretrained model.
To evaluate the effect of removing important tokens, we define a scoring function S(a, that quantifies how well the model supports the gold summary y given the context x. Let the language model æ define the conditional distribution:
M
(y x) = II
T(m x. He:),
(18)
where denotes the i-th token of the summary and yes the prefix tokens preceding it. We use the log- probability of the gold summary given the context as the score:
l
S(x. v) = log ( x) = Σ log (ys x. yex).
(19)
Intuitively, v) reflects the model's confidence in generating the reference summary when conditioned the provided context. Let be the original context and x \ Rg be the context with the selected tokens removed. The Comprehensivenes score at selection rate k is then defined as:
Comp S(x, y) S(x) \ Ra.y)
(20)
S(I,
where larger values indicate that removing the selected tokens leads to a larger drop in entailment, implying that these tokens are more necessory for preserving the semantic content of the summary.
26

---

Published in Transactions on Machine Learning Research (01/2020)
Is it fair to tell someone you can't 't be with them unless they seek outside help for their issues ? POST This account is just becoming Secret Time but anyways il was wondering if anyone been in this sort of situation I Im seeing someond I care deeply about E sure I the Love thing ) and they ire going through some tough times related to loss and just general depression ( put ting I etc Di Rocendy the cutting has started up again ( it had only happened for a short period time before and I found out pretry soon afterwards lat the and of it anded ) and don know makes me upset I and is draining and think sort of evidently altering how fool about them I frmm antically speaking which think in turn just making it
(a) TL:DR: I'm seeing someone 1 care about get cut again and they's 'r getting depressed, and I'm kind of in love with them and don't 't know what I should do.
of Reddit! What makes you cry Sports Edition ( Story inside POST: So maybe 's just me maybe not I ''ve never been an emotional kid growing up I rarely cried the usual tear ket movies or other emotional experiences but ever since playing soccer all throughout high school and going through thick and thin with my team It always gets me emotional to see the under dog win in just about any sport I and get little te ary Is this normal or am just the odd one out R If you ve ever cried after playing or watching any sports event whetter it is the Super b awl I the Wimbledon or the World Table Tennis Championships shane your story please
(b) TL.DR: I cry when 1 see an underdog win in just about any sports, does anyone else? Share your story!
[21 I M like a pretty girl I 4122 F J. but she always with her Eather How should proceed ? POST ! ge to certain shop occasionally and the cashier is really pretty girl who socms about age I I den it talk to her nuch II think she 's shy because her dad the owner of the shog always standing behind her II get on well with the dad better than his daughter i who is very quiet in my presence ). I ''m quite sure he thinks well of me and I think he knows I might like his dauughter i but I ''m unsure How should I go about asking out his daughter withoout offending him or making the llaughter
uncomleriable Even so / daughter might noi even like me in that way so Is a tricky situntion
(c) TL:DR: How do go about asking the girl out without any misunderstandinng /aw kwardness?
ceiving raise as an intern POST: ; So my mid year evaluation is cominn this Friday The pay is dhrectly correlated with the amount of units you have taken When I was hired! I was junior and my pay was set at 8 thr and Il gladly accepted I Now I am 2 4 units away from graduating and will be working full time over the summner | The average pay for senior is 2 Phr II was going to say if there was anyway my pay sould refleet the surrent amount of units I have taken. called HR and they said I should be fine. . but still need my bosses approval am getting bit auxious and with finals coming up not great Any suggestioos? ?
(d) TL:DR: How do 1 get my mid-year pay to reflect the increase in units 1 have taken.
5 F with my SO [2 MJ j of 4 years how to help him cope with sudden loss POST: This is a brief and casy one Two days ago my SO 's younger brother died gathor brunally and ( viously ) suddenly I have only mot this man once in the 4 years we 'v been together but obviously this is impacting him heavily He has been with family out of lown for the last couple days and is his way home me and daughter now ( ( had to stay home and
continue work and take our daughter to school ,) How do I best support him througli this I will obviously be attending the service with him in å couple weeks| I Im doing my best to make sure that the house is clean and life is as normal as possible when he gets home so he has very little to stress about But what can do for him now 2 How can best help him through this grief process? ?
(e) TL:DR: Boyfriend's brother died suddenly and it's impacting him very hard. What can I do to help him through this?
police officet ever becn Good Guy Greg to you Describe it POST I was once pulled over coming home from friends Halloween party for running sod light II went througli tight as it changed but ran through nonetheless I The office pulled me over and did the usual deal | Licence and registration do you know why pulled you over I told him i knew that had ran the light and my reasoning was that was just dog tired, as it was 8 a 9 AM and I just wanted to get home I Hi asked me if there had been any Brinking or drugs at the party and I hurried by said no as I waa only 1 the time He went back his cruiser and tan my licence and plates and then came back and informed me that was going to get verball warning, i le don do again kind of thing, and then be escorted me home, which was about five miles away.
(f) TL:DR: pulled over for running a red light, was told not to do it again.
Figure 16: Comparison of six different summarization examples. Each subfigure illustrates how the model interprets and condenses the input lext, highlighting variation across samples.
In Figure 17, we compare Comp, of ULTra, i.e., selection is performed based on the ULTra method as defined in Eq. 13, with the same-sized random selection as a baseline, denoted as Random. We report the results across multiple selection rales, averaged over 300 samples randomly drawn from the validation set of the TL;DR dataset. The results demonstrate that ULTra consistently achieves higher Comp, values than the Random bascline, particularly the selection budget increases. At lower selection rates, ULTra is only slightly better than Random, rellecting the difficulty of pinpointing influential tokens with very limited budgets. However, as more tokens are selected, ULTra identifies context regions that have a clearer causal impact on the model's output, while the Random baseline exhibits negative or unstable behavior, especially at smaller budgets.
Limitations. While the proposed Comprehensiveness metric offers an intuitive way to assess ULTra's effectiveness in identifying influential context tokens for summarization, it does not provide a complete evaluation of interpretability. This metric focuses only the causal impact of token removal and does not
27

---

Published in Transactions on Machine Learning Research (01/2026)
0.4
ULIra +1
Random +1a
standard deviation
Compk
0.2
0.1
-0.1
Selection rate k (%)
Figure 17: Average Comprehensiveness score Comp; on the validation set across different selection rates k (%) for ULTra and random baseline. At small budgets (A: s 15), UL.rra performs only marginally better than Random; as increases, exhibits steadier gains, whereas Random shows negative values at low & and non-monotonie behavior beyond 20%.
capture other important aspects such as sufficiency, stability under paraphrasing, or alignment with human rationales. Tt is also sensitive to the choice of language model and may reflect model-specific behaviors. A more comprehensive benchmark and evaluation strategy for interpretable text summarization should be developed in future work, combining multiple faithfulness mcasures, human evaluations, and robustness analyses.
Transformer Architecture
The architecture of a typical Transformer can be formulated as follows: the input X split into n tokens {x,) After tokenization, token emboddings {c: are computed, where eg corresponds to the CLS token. Positional encodings PE, added the i-th token embedding to incorporate spatial information, resulting in the latent token representation z; (1) e, + PE,, Here, z, represents a latent token, where denotes the layer index with 1 E (1...., L} and L the total number of layers in the Transformer, and represents the i-th token within the 1-th layer.
For each head h E f H} in the multi-head attention mechanism, the queries, keys, and values corre- sponding to the i-th token are obtained via lincar transformations, projecting the latent token of dimension d into dimension For vI E (.....,
2 (z) (111) = (W/1)) z (1-1)
(1) 1-1) (W Whikk IT (1-1) K (z)
(1) (z (1-1)) (w/ p.p)) zi
(1-1)
(21)
h.g. b.ke h,n E Bdxk The attention weights for cach token pair (i,5) layer and head are where W-(D) Wil mi(())
computed
(Q) (2) (z)
K (2,
(
softmax,
(22)
V
The i-th token is then updated by summing over the weighted values across all heads:
n
Σ ()
Σ
a
ahi V
(
(z,, (2)(-1)),
(23)
j=1
28

---

Published in Transactions on Machine Learning Research (01/2026)
where Wa, Rexd The updated token representation u, after the attention layer is computed as:
= (ayerNorm(z (1-1) + n (
(24)
Each token then passes through ferd-forward network:
n.),
(25)
1 LayerNorm(u; )
(26)
Here, W E Rdxm l Rxxd
29


Reviews:
[{"review_id": "QxjBpKb6Kr", "reviewer": "Reviewer_iUCf", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors have provided a thoughtful and largely comprehensive response to the initial review. They have acknowledged the key points and have outlined concrete plans for revision, which is commendable.\n\nHowever, while the response addresses the \"what\" of the revisions, several critical aspects concerning the \"how\" and \"why\" still require deeper clarification and substantiation in the manuscript itself.\n\nBased on the authors' responses, the following points need to be explicitly and clearly addressed in the revised manuscript to ensure clarity, reproducibility, and robustness of the claims.\n\n1. The authors' response mentions training the projection vectors $w_i$ with \"a limited number of batches.\" For reproducibility, the manuscript must specify: 1) The exact optimizer (e.g., Adam, SGD) and learning rate used; 2) The number of epochs and the batch size; 3) A brief justification or sensitivity analysis for why the chosen number of samples (256) is sufficient and robust across different models (ViT-B, L) and datasets.\n\n\n2. The authors' response provides a high-level rationale (leveraging all tokens, mapping before aggregation). This discussion must be expanded and empirically grounded in the manuscript: 1) Can you provide a quantitative or qualitative analysis (e.g., in an appendix) that demonstrates the \"lower information loss\" compared to methods that aggregate first? For instance, visualizing the explanation maps from an aggregated token representation vs. ULTra's approach could be very illuminating; 2) The claim that latent tokens contain richer information than the final CLS token is central to the work. A direct ablation or comparison highlighting this would significantly strengthen the argument.\n\n3. While the authors acknowledge the high cost in the response and Appendix B.2, this is a major practical limitation that deserves a more prominent discussion: 1) The main text (e.g., the conclusion or the segmentation experiment section) should explicitly state this limitation; 2) The promised discussion on \"mitigation strategies\" (sparse attention, token pruning) should be included. It should be clear whether these were tried in the current work (even if only preliminarily) or are purely speculative for future work. Providing an estimate of the inference time or FLOPs relative to a baseline forward pass would be highly valuable for practitioners.\n\n4. The promised discussion on \"mitigation strategies\" (sparse attention, token pruning) should be included. It should be clear whether these were tried in the current work (even if only preliminarily) or are purely speculative for future work. Providing an estimate of the inference time or FLOPs relative to a baseline forward pass would be highly valuable for practitioners: 1) Could the authors design a simple human evaluation? For example, asking annotators to identify the most important sentences in a context and measuring the correlation with the top-k tokens highlighted by ULTra? 2) Alternatively, is there a way to leverage existing faithfulness metrics for text explanations, even if adapted, to move beyond pure qualitative demonstration?\n\n5. The authors' defense for using Euclidean distance is reasonable given the class-agnostic nature of the tokens. However, to further bolster this choice: 1) Have the authors experimented with other distance metrics like cosine distance in the embedding space? A brief note on this, even if results were similar, would demonstrate thoroughness; 2) It would be helpful to include a few visual examples in the appendix showing the \"positive\" and \"negative\" perturbation regions for a sample of tokens, making the test and its results more concrete for the reader.\n\n6. The authors have committed to grammatical revisions. I strongly advise a thorough proofreading pass, potentially with the help of a native English speaker or professional editing service, to ensure the final manuscript is polished and meets the high presentation standards of TMLR. Please ensure consistent notation (e.g., ULTraWCLIP vs. $ULTra\\(_\\mathcal{W}^{\\text{CLIP}}\\)$) throughout the text and figures.", "claims_and_evidence_yn": "Yes", "claims_explanation": "See Summary Of Contributions.", "audience_yn": "Yes", "audience_explanation": "See Summary Of Contributions.", "requested_changes": "See Summary Of Contributions.", "broader_impact_concerns": "See Summary Of Contributions", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yTepokPiON", "reviewer": "Reviewer_v9J2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces ULTra, a framework to describe latent tokens in Transformer models in terms of the semantic contribution of individual tokens. Although designed as a general interpretability tool, its first use and result are towards unsupervised semantic segmentation(USS). The framework is applied to pre-trained models without requiring fine-tuning and is tested on multiple vision datasets, with an additional qualitative test on text summarization.\n\nStrengths\n\n1.\tLeverages attention-based interpretability and logically extends it to latent token analysis is an interesting idea.\n\n2.\tThe proposed method achieves state-of-the-art performance on standard unsupervised semantic segmentation benchmarks, showcasing the existence of rich semantic structure in latent representations without fine-tuning.\n\nWeaknesses\n\n1.\tVague Goal: The main goal of the paper is vague. It varies from being a method for latent token interpretability to being a method for unsupervised semantic segmentation. Both the abstract and the introduction place the work within interpretability, yet the experiments and results concentrate primarily on demonstrating USS performance. This is unclear in terms of the main contribution of the paper. There are a few things that remain unanswered: \n\n(1) How do we measure the success of interpreting a latent token? The paper proposes a framework to measure token contribution, but we have no idea what this contribution is to, and how we can check whether it is correct or not. \n\n(2) The paper introduces a function f(z_i^(l)) to yield a scalar output from a token embedding so that it can be backpropagated, yet the rationale for the specific choices (sum, energy, or weighted projection) is performance on downstream tasks, not fidelity to the token's \"true\" meaning.\n\n(3) What is the metric for determining interpretability itself? The indirect and insufficient use of USS performance as a proxy for the quality of interpretation.\n\n2.\tUnpersuasive Interpretability Assessment. Qualitative results used to demonstrate interpretability are not quite convincing. The visualizations shown in Figures 2 and 4 show heatmaps corresponding to objects, but this is normal behavior that gives a good segmentation. The heatmaps do not reveal much new insight into the model's reasoning from what is already gained through the final segmentation mask. It does not seem clear what specific new insight is gained through these visualizations.\n\n3.\tThe Unsupervised Problem. The work points out its \"unsupervised\" nature, but this actually makes interpretability assessment basically difficult. There is no ground truth in an unsupervised setup on what a latent concept should be. Without this, any statement on finding semantically significant patterns is human and can't be rigidly confirmed. The work does not mention this inherent limitation, thus making the interpretability claims appear unproven.\n\nMinors: Typo in Section 3.2, \"desiging\" -> \"designing\".", "claims_and_evidence_yn": "Yes", "claims_explanation": "The paper convincingly supports its USS performance claims (solid zero-shot results on standard benchmarks; no fine-tuning) but does not provide clear, convincing, and falsifiable evidence for its interpretability claims.", "audience_yn": "Yes", "audience_explanation": "Even if the interpretability evidence is not fully convincing, I believe several segments of TMLR’s audience would still find the paper’s findings and methodology interesting", "requested_changes": "Please refer to the weakness section.", "broader_impact_concerns": "None", "additional_comments": "None", "extra_scores": {}, "extra_text": {}}, {"review_id": "eKGrOK5cMt", "reviewer": "Reviewer_T9SA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes Ultra, a method designed to provide interpretability to tokens in Visual Transformer architectures. While the method was initially developed for general use, it focuses specifically on unsupervised semantic segmentation (USS). The method is evaluated on multiple datasets (Pascal VOC, COCO-Stuff, Cityscapes, Potsdam) and also reports results in text analysis.\n\nSide note: The paper is stated to be of standard length (12 pages), meaning the appendices are not considered mandatory material. If they are intended to be treated as part of the paper, they should be moved into the main text.", "claims_and_evidence_yn": "No", "claims_explanation": "The paper does not make clear claims and does not present clear (accompanied by detailed explanation) results to round claims.", "audience_yn": "Yes", "audience_explanation": "There two strengths:\n- the unsupervised semantic segmentation, where the paper shows strong results.\n- the interpretability of token in ViT", "requested_changes": "**Strengths**:\n\n    - The paper is technically correct: I have checked the mathematics and found no major errors.\n    - The results are strong: I have reviewed multiple papers on unsupervised semantic segmentation, and the results presented here are indeed competitive. The paper could stand on its own as a contribution to USS.\n\n**Weaknesses**:\n\n    - The paper is not particularly clear in its purpose. It oscillates between “interpretability of the token” and unsupervised semantic segmentation. From the abstract and introduction, I initially assumed the focus was on the former, yet the evaluation is centered on USS, with some addition on text. Several aspects remain unclear:\n\n    - How is the success of “measuring the contribution of latent tokens” evaluated? Since the paper does not explicitly state its main objective, it is unclear: contribution of tokens to what?\n   - The paper states that it measures the influence weight of a token in a specific representation (before Eq. (5)). This is fine, but why is it relevant? Why is it more important than simply aggregating the weights between the two?\n\n   - What loss function is used for USS training? This is related to the previous point—what is expected from token interpretability? I believe the paper should start with a hand-drawn example illustrating what is expected.\n   - The same issue applies to the text analysis section.\n  -  The interpretability results are not compelling. The figures do not communicate much. We see heatmaps (saliency maps) in Figures 1, 3, 4, and 6, but it is unclear what we are supposed to observe and how this demonstrates successful token interpretability. For instance, in Figure 3, the token heatmap focuses on a dog, and then a segmentation mask around the dog is produced. However, the segmentation mask already conveys all the necessary information.\n\n**Concluding weakness**: The paper emphasizes “unsupervised” settings, but interpretability in unsupervised contexts is problematic—since there are no ground truths, it is difficult to judge whether the results are good or not. As a result, the paper lacks a metric for evaluating token contribution or interpretability in representation. To me, the work currently feels like a work in progress rather than a finished piece.\n\n**Minor issues**:\n\n   - text : \"experimetns\"\n    - Figure 2 comes in text after figure 3", "broader_impact_concerns": "There are no concerns here.", "additional_comments": "Followin discussion with authors, I believe that if changes assumed in the comment are made, the paper moves in the positive area.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)