Reading Images Like Texts: Sequential Image Understanding in Vision-Language Models
Authors: Yueyan Li
Chenggong Zhao
Zeyuan Zang
Caixia Yuan
Xiaojie Wang
Venue: ICLR 2026 Poster (2026)
Keywords: multimodal language models
interpretability
spatial reasoning
Primary area: interpretability and explainable AI

Abstract: Vision-Language Models (VLMs) have demonstrated remarkable performance across a variety of real-world tasks. However, existing VLMs typically process visual information by serializing images, a method that diverges significantly from the parallel nature of human vision. Moreover, their opaque internal mechanisms hinder both deeper understanding and architectural innovation. Inspired by the dual-stream hypothesis of human vision, which distinguishes the "what" and "where" pathways, we deconstruct the visual processing in VLMs into object recognition and spatial perception for separate study. For object recognition, we convert images into text token maps and find that the model's perception of image content unfolds as a two-stage process from shallow to deep layers, beginning with attribute recognition and culminating in semantic disambiguation. For spatial perception, we theoretically derive and empirically verify the geometric structure underlying the positional representation in VLMs. Based on these findings, we introduce an instruction-agnostic token compression algorithm based on a plug-and-play visual decoder to improve decoding efficiency, and a RoPE scaling technique to enhance spatial reasoning. Through rigorous experiments, our work validates these analyses, offering a deeper understanding of VLM internals and providing clear principles for designing more capable future architectures.

Published as a conference paper at ICLR 2026
READING IMAGES LIKE TEXTS: SEQUENTIAL IMAGE UNDERSTANDING IN VISION-LANGUAGE MODELS
Yueyan Li, Chenggong Zhao, Zeyuan Zang, Caixia Yuan, Xiaojie Wang
Beijing University of Posts and Telecommunications fsiriuslala zeggoon, zangzeyuan, yuancx, yuancx,x/wanggJebupe. (bupt.
ABSTRACT
Vision-Language Models (VLMs) have demonstrated remarkable performance across a variety of real-world tasks. However, existing VLMs typically process visual information by serializing images, a method that diverges significantly from the parallel nature of human vision. Moreover, their opaque internal mech- anisms hinder both deeper understanding and architectural innovation. Inspired by the dual-stream hypothesis of human vision, which distinguishes the "what" and "where" pathways, we deconstruct the visual processing VLMs into ob- ject recognition and spatial perception for separate study. For object recognition, we convert images into text token maps and find that the model's perception image content unfolds as two-stage process from shallow deep layers, begin- ning with attribute recognition and culminating in semantic disambiguation. For spatial perception, we theoretically derive and empirically verify the geometric structure underlying the positional representation in VLMs. Based on these find- ings, we introduce an instooction-agnostic token compression algorithm based on a plug-and-play visual decoder to improve decoding efficiency, and RoPE scal- ing technique to enhance spatial reasoning. Through rigorous experiments, our work validates these analyses, offering deeper understanding of VLM internals and providing clear principles for designing more capable future architectures.
INTRODUCTION
Generative vision-language models (VLMs) take images and texts input and generate texts as output. Existing VLMS are typically based on the Transformer architecture (Vaswani et al.; (017) and have achieved strong performance to significant hallucination issues, such as incorrectly describing objects or misjudging the spatial relationships between them. These problems only present challenges for model improvement but also highlight the need for interpretability research, motivating deeper investigation into the underlying mechanisms behind a model to foster a more comprehensive understanding of
numerous real-world tasks. However, VLMs are prone
The existing research on the interpretability of multimodal models is very limited. Some studies focus on the internal representations of model, finding the neurons or components (e.g., attention
heads) corresponding to the concepts in the real world (Goh et al. [021] Gandelsman et al. 2023
2024). or extracting semantic information using logit lens (Neo et al. 20224) Sonia Joseph 2024
Jiang et al. 2024) sparse autoencoders (Hugofry) 2024 Daujotas, 2024 Rao al. 2024). An- other line of work employs causal tracing to investigate information storage and transfer (Rajaram et al. 2024 Palit et al. 2023 Basu et al. 2024 or uses blocking- based interventions to study the information flow (Zhang et al. 2024b 2025) in VLMs. However, current research often overlooks fundamental distinction between neural networks and human cognition. Specifically, the visual encoder in a VLM is typically a Vision Transformer (ViT) (Dosovisskiy et al.] 2020) that adopts the Transformer architecture originally designed for inherently sequential text. When processing an image, a ViT employs raster scan: it partitions the image into patches and flattens them into a one-dimensional sequence. The adjacent patches belonging to the same object may be scattered in different and discontinuous positions in that sequence. In contrast, human visual perception exhibits
"Corresponding author: yuancx@@hpp..coo Cooee is available at htt.:/e.///thhib.cc///iiiiaaaaar

---

Published as a conference paper at ICLR 2026
characteristics of Gestalt cognition (Koffka 2013), where the brain actively organizes and integrates discontinuous visual signals upon reception, viewing them as a whole. This discrepancy raises crit- ical questions: How do VLMs leverage ID image sequence to understand complex 2D concepts and perform tasks like spatial reasoning? And does this cognitive gap between machine and human processing adversely affect VLM performance?
The dual-stream hypothesis (Goodale & Milner 1992) of the human brain posits that visual pro- cessing is divided into ventral pathway for object recognition (the "what pathway") and a dorsal pathway for spatial perception (the "where pathway"). Inspired by this theory, our work focuses on these two aspects and poses two primary questions: (1) How do VLMs associate positionally discontinuous tokens belonging to the same object within a flattened sequence to predict the object's category? (2) How do VLMs infer 2D spatial relationships between objects from ID sequence?
To address question (1), we conduct an investigation into the visual information processing within the VLM's visual encoder, combining quantitative and visualization methods. Specifically, we focus on the encoder's representations layer by layer, employing analyses based on logit lens. We find that the visual encoder exhibits a two-stage process from shallow to deep layers: it first performs attribute recognition, identifying local features such color and texture, and then leverages the attention mechanism for semantic disambiguation to assign these features to a specific object category. This two-stage process collectively resembles pattern of Gestalt cognition. For question (2), we begin by analyzing the properties of learnable 1D absolute position embeddings, such as those in LLaVA (Liu et al. 2023b). We then focus on 2D Rotary Position Embeddings (RoPH) (Su et al. 2024). the most common method for achieving dynamic resolution. We theoretically analyze how a 2D RoPE-based visual encoder represents positional information, and empirically visualize the geometric properties of the resulting positional representation, confirming the validity of our theoretical analysis.
To further validate the efficacy and practical value of the aforementioned findings, we propose two corresponding model improvements. First, based on our discoveries in object recognition, we in- troduce a novel mnstruction-agnostic token compression algorithm. We first distill a visual decoder that takes visual embeddings as input and outputs the logits for visual tokens, and the latter can be mapped to text tokens. Then we compress the visual embeddings at the pre-filling stage using run- length encoding to merge the similar visual tokens, making it more efficient for inference. Second, informed by our analysis of spatial perception, we propose the RoPE scaling algorithm to address the issue of indistinct positional representation in RoPE-based visual encoders. This method en- hances the model's 's spatial reasoning capabilities by adaptively amplifying positional information in low-frequency regions. RoPE scaling demonstrates strong performance in both training-free and fine-tuning- based experiments. In summary, our main contributions are as follows:
For the "what" pathway in a VLM, we present an in-depth analysis of the dynamic visual information processing in the VLM's visual encoder based on logit lens and visualization, uncovering a two-stage processing pattern analogous to Gestalt cognition (Section 3
For the "where" pathway in VLM, we provide theoretical analysis of the mechanism be- hind spatial perception for RoPE-based visual encoders, and reveal the geometric structure of the representation of spatial relationships through empirical studies (Section 4
Based on the visual processing characteristics of VLMs, we propose instruction-agnose token compression method that reduces image sequence length during the decoding phase while limiting performance loss to acceptable range (Section [.I]]
Based on the spatial perception characteristics of VLMs, we introduce RoPE scaling, method that improves the spatial reasoning capabilities of VLMs based on 2D RoPE while preserving their general capabilities (Section 5.2)
2 BACKGROUND
2.1 VLM ARCHITECTURE
As shown in Figure I a VLM typically consists of a visual encoder fr(-), a modality connector fc(-), and language model f.). Suppose the input to VLM is image X and an instruction
XT. The instruction is tokenized and mapped to a sequence of text embeddings T (t,,..,,twy (Nr) e NTD where Ny is the number of text tokens and D is the dimension of the language model.
2

---

Published as a conference paper at ICLR 2026
Byity rahuxo,
12
VIT Block 4
LLM Block&
Unembedding
x
Logit
2
Connector
1
Softmax
VIT Block
LLM Block 1 W
X Deseribe the imugo.
A chhll skingg
P ae
['Desenbe,,
imuge"
Te T
Figure 1: The typical architecture of the vision-language model. It consists of an visual encoder, modality connector and a decoder-only language model. Notations are detailed in Section 2.1
For the image input, the visual encoder is a VIT with bidirectional attention. First, the image is partitioned into a sequence of patches / = (pI, .... PNy) R where Ny is the number of patches and Dy is the dimension of the visual encoder. Then the sequence goes through Ly layers of VIT blocks, and the output of layer 1(1 [1, Lv]) is H
................ =NN) ) E R xDv The modality connector is a projection (e.g. MLP). It maps the output of the visual encoder H4v to the semantic space of texts, which results in the visual embeddings V = (UI,.., ONv) E R Finally, the embeddings of the image and the instruction
concatenated as a multimodal input H (Ul,..., Uy t .... (Ny) (Nv N of the language model for response generation.
2.2 POSITION EMBEDDING
Position embedding is introduced to add positional information to input, and thus the computation of the attention weights in Transformer is assition-aware It can be categorized into absolute position embedding and relative position embedding. The absolute position embedding is directly added to the input embedding before being passed into the Transformer blocks. Let = (ø1, .e.,,,.,,,)) E Rn be a d-dimensional input embedding with = (e1....,e,, Cx) E Raxd being its absolute position embedding. Then the input with absolute position embedding is:
X E (x1 C1,...,,
Fnn + C,) E IRax
In practice, the absolute position embedding could he either pre-calculated by a sinusoidal function (Vaswani al.] 2017) set of learnable vectors (Radford al., 2018 Devlin al.) 2019). In- stead of assigning a unique embedding to each position, relative position embedding focuses on the relative distances between different positions. The most commonly used one RoPE (Su et al. 2024), which encodes the absolute position with rotation matrix and incorporates the ex- plicit relative position dependency in self-attention formulation in each layer. Considering query
qm = W@am = (0) + iqm and a key k, = W WK = (0(0) + (1) (m # n) in a 2-dimensional input X E Rax2 at two different positions m and n, RoPE applies a transformation ff to them and the inner product between the query and the key is calculated as follows:
< (qm; m), I (kn, n1) >= Re(yme im09. (nn))""))) = Re(qm ( (mm-
(2)
where 0 is a preset constant. In equation 2 the relative distance m n is introduced during the calculation of the attention weights. For 2D RoPE in VLMs, the position IDs corresponding to the width and the height of a patch are both introduced in RoPE. Given two positions (m1. m2) and (n,, na) for the query and the key, the query is at least 4D and can be written as 4m = x 4m) (0)
(3) ), where X and Y denote the components along the width and the height directions, respectively. The same applies to the key as well. Then the inner product is:
(gh1 + (1) 9m + igm'
< fggn, m1, m2), (kin 01, n2) > Re (gxeeime -. imz²) ((Ri) cini9)' (bi) e
(3)
= Re Re(gii xx n + g X. e cl(me-n2)8
LY
In this work, the VLMs we use include LLaVA-1.5 (Liu et al. 2024a). Qwen2 / 2.5-VL (aag et al.
2024| Bai et al. 2025) and InternV--2.. (Chen et al. 2024b). The Qwen2 / 2 series of models
use 2D RoPE in the visual encoders, while others use learnable ID absolute position embedding in
VIT. Detailed information about RoPE and the models can be found in Appendix C D
3

---

Published as a conference paper at ICLR 2026
3 INVESTIGATING OBJECT RECOGNITION IN VLMS
In this section, our goal is to understand how model, from the shallower to deeper layers of the visual encoder ! in a VLM, dynamically associates and combines tokens that belong to the same object but are positionally discontinuous in the image sequence, as illustrated in Figure I Let us consider an image as input and its output HV = (zf...., INyy) E x (1 E [1. [v]) at each layer of the visual encoder, and assume there are M main objects (01, 0M)) ) in the image.
3.1 METHOD: LOGIT LENS FOR VISUAL TOKENS
It is good way to visualize the geometry of visual representations via representation similarity, while relying solely on the metric in the linear space is insufficient. This because the semantics within an individual visual token becomes increasingly complex in deeper ViT layers, due to the interaction between visual tokens via self-attention (see Figure X and in Appendix [E.]). Thus, the semantics of visual tokens cannot simply be investigated using metrics such as cosine similarity. Logit lens introduced to study the model behaviors from the view of activations (nostalgebraist, 2020). It applies the unembedding matrix Wo E Dx to an activation in order to extract semantic information from it, where V is the vocabulary of the language model. In VLM, we can apply We directly to the image representations and get the text tokens corresponding to the image patches:
W = (w ,..,, xx,) = arg max (Softmaz (Wu [(o ...,, II)
(4)
WEV
where w. is the decoded text token for image patch Ps. and H" (of o is visual part in the output of the 1'tth layer in the LLM. Neo et al. (2024); Jiang et al. (2024) found that the text tokens contain rich semantic information related to their corresponding image areas. To interpret image processing in details, we inspect ViT layer by layer. For the 1-th ViT layer, we (1) delete the ViT layers after the (-th ViT layer and (2) apply logit lens to the outputs of the I'tth LLM layer. In practice, the value of L' is 25 in VLAVA--15-7B and 32 in Dwen2.55VL-7B. as the emergence of the meaningful tokens is the most significant in these layers (see Appendix E.2). Thus, we actually create family of functions F {fi that map the image patches P their corresponding text tokens (fi P WK) and reflect the dynamic process of object detection via natural language.
To visualize the dynamic process, we propose two con- cepts: token map and segmentation map. The idea orig- inates from emoji logit lens (Sonia Joseph| (224) that maps image patches to emojis for visualization. Here a token map is an image where each grid is filled with its corresponding text token. An illustration of the token map is shown in Figure 2 Examples of the token maps in VA- and Qwen2.3-VL. are shown in Figure 14 and 17] respectively. As for the segmentation map, we create a set of keywords including the names of the main elements in object (e.g. for bear: ("bear", "head", "eye", "nose", "paw")) for each of the main objects in the image. We fill in each grid with the color of object on if the text token in that grid
rodk
rodk
Ing
uster
oater
log
Water
water
water
Figure 2: An illustration of token map.
falls into the keywords set of object Om- Details of this process shown in Algorithm [ The segmentation maps of LLa VA-1.5 are shown in Figure 3 See Appendix E.3 for more examples.
3.2 EXPERIMENTAL RESULTS AND ANALYSES
We use the samples from the GQA dataset (Husson & Manning 2019). As shown in Figure 3 the geometric structure in segmentation maps gradually approaches the shapes of the objects in the original image. We also observe that in the first few token maps of the earlier layers in ViT, tokens with no practical semantic meaning, such as punctuation or white spaces, account for the majority. From shallow to middle layers, attribute words for common local features (e.g. "fur", "yellow") begin to appear. While from middle to deep layers, the attribute words gradually disappear and the representative words (object labels) for specific global object (c.g. "bear", "rock") begin to
'The visual encoders in VLMs are typically based on the ViT architecture, so unless otherwise specified in the following text, we consider visual encoders and ViT as one thing.
4

---

Published as a conference paper at ICLR 2026
Original
VIT layer
ViT layer 10
VIT layer 15
VIT layer 20
VIT layer
Figure 3:
The segmentation maps of the image representations in ViT layers of LLLVA--15-7B. From shallow to deep layers, VIT gradually performs semantic disambiguation.
emerge. To quantify this phenomenon, we create a of attribute words W4..om and representative words Will.om for each object On in the image, and define the ratio of attribute words as TA = Nv 2m=1 E Ewewa.om count (w:), where count(-) is the number of occurrences of token. The ratio of representative words rn is defined in the same way.
We sample batch of 50 images from GQA and compute F'A and on in each layer in LLaVA-1.5-7B. Results in Figure 4 indicate that the ratio of the attribute words starts to increase from layer 5, reaches its maximum at around layer 15, and then suddenly decreases. While at the same time, the repre- sentative words begin to appear in place of the attribute words. Therefore, the object recognition in VIT could be divided into two stages: attribute recognition (shallow to middle layers) and semantic disambiguation (middle to deep layers). In the first stage, attribute recognition is responsible for the detec- tion of local low-level features such as color or texture, which could be shared among different objects. In the second stage, the model performs semantic disambiguation to integrate the common low-level features into specific high-level concepts.
agec eDriucaT wods
6.2
Figure 4: The change in the ratio of attribute words and representative words with ViT layers in LA.
To further prove this, we evaluate the hallucination of LLaVA-1.5-7B on the POPE (Li et al. 2023 dataset, in which the questions are all in the form of "Is there a/an [object] ...?". We directly check layer by layer if any of the words in the token maps matches the [object] word. We set the answer to "yes" any of the text tokens can be found the synonyms set the [object] from Lu et al. (2018), and "no" if not. Experimental settings are detailed in Appendix E.2] Results show that the accuracy hovers around 50% or less (random guess) until starting to increase at around layer 12, implying that the model has confidence in what it until reaching middle layers. It first perceives discrete low-level features, and then integrate them together and view them as a whole, behaving in a way that follows the Gestalt principles of perceptual (Koffkal 2013) (Appendix E..4 By analogy, the model learns to link similar low-level features together via dot-product self-attention (Principle of Similarity and Proximity), and automatically fill in the "gaps" between discontinuous visual tokens via prior knowledge to view them as a whole object (Principle of Closure).
should be noted that during pre- pre-training. the parameters of the modality connector were adapted to the final layer of the ViT. Therefore, more rigorous approach would be to train dedicated modality connector for each VIT layer, though it laborious to do this for each layer in each model. However, with the existence of the residual stream, every layer in Transformer contains the output of all previous layers. This means that the last layer current layer's output and the outputs of all previous layers. During VLM pre-training, level, the output from the last layer of the VIT is aligned with the language space via the modality connector. While from micro perspective, the modality connector actually processes the last layer of the ViT, along with the output of all preceding layers. Therefore, it should have developed the ability to handle early ViT layers, and thus the reasonableness of our method is guaranteed.
ViT is essentially linear superposition of the macro
5

---

Published as a conference paper at ICLR 2026
(a) PE in LLaVA (row)
(b) PE in LLavA (col) (c) PE in In (row) (d) PE in IntemVL (col)
Figure 5: The geometry of the ID absolute position embedding in LLaVA-115-78 and Intern 2.5-8B (w/o [CLS] token). The visualization is performed via t-SNE for dimensionality reduction.
The labels are set to row IDs
(5a 5c and column IDs (5b 5d), respectively.
4 INVESTIGATING SPATIAL PERCEPTION IN VLMS
4.1 GEOMETRY STRUCTURE OF ID ABSOLUTE POSITIONAL EMBEDDING
In this section, we explore how positional information is represented. Continuing the discussion in Section 2.2] we first analyze the learnable ID absolute position embedding. This type of position embedding requires that the processed images have fixed size. In LLaVA-1.5-7B, the position embedding E' E DN (N = 577, DV 1024) has a length of 577, corresponding to the size of patchified 24x24 image (plus one [CLS] token). After pre-training the position embedding for each patch should encode the unique row and column coordinate information for that position. To verify this, we apply t-SNE to reduce the dimensionality of the position embedding from LLaVA-1.5-7B and Intern 2L-2.5-8B to two dimensions. As visualized in Figure 5 the geometric structure of the absolute position embedding exhibits distinct rows and columns, which confirms our hypothesis.
However, we are more interested in 2D RoPE because it enables image processing with dynamic resolution and possesses better scalability. Unlike absolute position embedding, the image sequences processed by 2D RoPE are of variable length, making it impossible assign unique and fixed representation to each position. In this scenario, the position information can only be established upon the representations of objects and manifested through interactions between them. Therefore, is an abstract and high-level feature, and we begin our analysis from theoretical perspective.
4.2 THEORETICAL ANALYSES ON 2D ROPE
To simplify the discussion, we only consider two objects: A and B, and four spatial relationships between them: R = {'left" "right", "front", "behind"). ). We simplify each object into a single patch, reduce the ViT dimensionality to 4 (minimum required for 2D RoPF), and ignore all other content in the image. Thus the shape the image sequence is 2 x 4. As spatial relationships are relative, use object B as a frame of reference and establish coordinate system in the image where is the origin, and the X and Y axes correspond to the directions of the image's 's width and height, respectively, shown in Figure [bb] When considering the position of object A relative object B, A and B can be termed the "satellite" and "nucleus," and the positions of object A corresponding to the four aforementioned relationships are (-m, 0), (m, 0), (0, n) and (0, -n), respectively.
We now investigate the representation of object A in these different positional relationships. Since RoPE is applied to queries and keys, we only consider the self-attention in ViT and ignore the scale factor, the Softmax function as well as the output projection for the convenience of discussion. When object A is the left of object B, the attention output object
layer can be written as:
h =< f(gs, m. 0), f(kA: m, 0) VA+ < f f(g,, -m, 0), f(B (kB,0,0) 0,0) > UB
(5)
Relax xx + +9x xxxxx* JoA + Relg k- e²(l <((=)0) + 9x xxx 10B
where (()) kc) and 04-1 denote the query, key and value, respectively, and all other notations are derived from Section 2.2 Similarly, we can obtain the representations for objects A and B across all four spatial relationships, (KA) and (hE)reR (Equation 24 31) It is obvious that they are weighted sums of U and UB, where the weighting coefficients its influence. Thus, we focus on the constituent components these coefficients. First, by com- paring the four representations of object A, (rE)) find that the coefficients of U are iden-
precisely where RoPE exerts

---

Published as a conference paper at ICLR 2026
"hchinef"
(0, nn)
of
( m.
(m,,
(0, front of"
(a)
(b)
Figure 6: (a): Examples in the What's Up B dataset. the top row are original images, while the bottom row shows how we view each image. (b) The coordinate system in our theoretical analysis.
tical across all of them. The only difference lies in the X-axis related component of the un co- efficient. Comparing / and h right we find that the X-axis components of the vB coefficient, Relg k. ef(=m()] and Re[q k cd(mm)), formally possess pair of conjugate symmetric terms. When written in the valued form (Equation 32 (3). they reveals pair of collinear and opposing (1) vectors: + =[( KB qA k )sin(mm)]us: This determines why "left" and "right" are opposites
A
(0)
in the model's 's visual geometry. Similarly, comparing h//ft and hiehind, we find that the X X-axis and -axis components of their respective uB coefficients carry the relative positional information for the horizontal and vertical directions via different dimensions (g)) k ci(-m0) vs. q ky c((-n))). This implies the orthogonality of "left-right" and ffront-backe in the model's view, and we will verify this through empirical studies.
Therefore, when the model is asked, "In which direction is A relative to B?", the components related object B within object A's representation becomes the basis for determining the direction. Thus, the model should be able to determine the spatial relationship using only object A's representation, which will be verified later. However, recent studies on information flow in VLMs (Zhang et al., 2025) indicate that the LLM first attends to specific image regions based on the objects mentioned the instruction, then aggregates the information into the last token for next-token prediction. Therefore, we further consider the interaction between objects A and B and propose the concept of the direction vector: v" h/.g h6 (r E R), where oş and oN are the satellite and nucleus, respectively. The subtraction is used for two reasons: (1) Though attention uses weighted sum, subtraction maintains linearity and does not affect our conclusions when comparing different spatial relationships later on. (2) It removes common terms from the two object representations, amplifying their differences and facilitating subsequent experimental analysis. We first compute the direction vectors for "left" and "right" (Eq. 34 35). By comparison, gleft and & oright can be written as C1UA+ C2UB (A + CDUB) ) and Ci C2 + (CAVA +CUUB), where the terms for distinguishing (0)
(0) (1) (1) ( (mo) and ca (4. x 9'A x.10 Isin (m0) (Eq. 36 37
(1),
positions C (gB A 4B R
The opposing vectors in o feet and right indicate the collinearity of their encoded directions. While
for behind the C3 and Cg terms in it are both zero, gets the terms (4/ k/4 9B k
(3) (2) )sin(n)) and
(qa x2) TA KB
(2) (3) (3) (2) ) )sin(n0), which are absent in fleft. When comparing /e and behind (Eq. 38
39 we find that although they are not strictly orthogonal, their directional information is encoded in different dimensions in the orthogonal subspaces of the and axes. Thus, the model supposed to separate them easily, which will be verified later. However, the key terms for spatial reasoning (e.g. C3, C.) are relatively small compared with the common terms, which could potentially limit the model's 's ability for spatial reasoning, and possible solution is presented in Section 5.2
To verify the above analyses, we use the subset B of the What's Up dataset (Kamath et al. 2023) (Figure ba) and Qwen2.5-VL-7B First, we conduct an experiment where we "erase" objects from an image, i.e., we replace the embedding of object with another and check the change the model's prediction of directions. Experimental results are shown in Table E] We observe that when the embedding of B is replaced with that of A (or vice versa), the token probability of the right answer only drops little bit (0.909 0.858 on average). This indicates that object's visual embedding contains sufficient information to determine its spatial relationships with other objects. Then, we compute the direction vectors. For each direction in R, randomly select the visual tokens from the embeddings of A and B and compute has and hox via mean-poss mean-pooling for 100 times. Details can be found in Appendix F3 We visualized the 400 vectors via PCA. Figure demonstrrass the
7

---

Published as a conference paper at ICLR 2026
(a)
(b)
(c)
(d)
Figure 7: (a) and (b): The PCA results of the direction vectors for left of / right of in front of / behind w and w/o position embedding. (c) and (d): Results of the intervention in visual embeddings demonstrate the collinearity of "left" & "right" and orthogonality of "left" vs. "in front of behind"..
collinearity and orthogonality of the direction vectors discussed before, while Figure shoss the geometry of them is broken without 2D RoPE. Further, we intervene in the visual embeddings of A and B (where the relationship ="left") with those in r' ="right" "behind" via: [V]]' (1 a <a))VV + a Mean(V.; 1N, where ar is the intervention intensity, No is the number of patches in object o, 1 1,, is an all-ones vector in NR.N. After this intervention, the direction vector v" will
shows that
become (1 - a:)e" + aaa"" (see derivations in Equation Results in Figure 7c 7d show that the intervention from Vrightt causes obvious performance drop as a increases, while Voehind does not. This further verifies the geometry of positional information discussed before.
5 APPLICATIONS
5.1 TOKEN COMPRESSION BASED ON THE TOKEN MAP
Reducing the number of input visual tokens is an effective way for VLM inference acceleration. Cur- rent approaches are often based on similarity, which is insufficient (see Section 3.1), or are based on attention scores to measure instruction relevance, which is incompatible with efficient attention implementations. We focus on the inherent redundancy in an image and propose an instruction- agnostic method for token compression. Specifically, when the token map of an image is flattened into a ID sequence of text tokens, we can observe many identical and positionally contiguous to- kens within the sequence. Therefore, natural idea is to compress the visual embeddings using the concept of run-length encoding (RLE) (Golomb) 1966). Our algorithm consists of two steps: (1) First, we count the consecutively repeated tokens and their repetition counts in the text token quence of the visual embedding. (2) Second, for each run of consecutive tokens, we compress their corresponding embeddings into single embedding, thereby shortening the image sequence length.
In practice. we introduce an additional "visual decoder" & between the modality connector and the LLM. takes the visual embeddings as input and outputs its logits, which can be decoded into text tokens. It is initialized from the unembedding matrix Wy in the LLM and trained via knowledge distillation (KD) (Hinton al., 2015). Given a visual embedding V (Vi,..., UNv. it is processed by an L-layrr LLM in the original model, resulting in the final logits logits tim
UN.,....)))) ).
Wo Wol(uf. .... o ) E IR Nv xIV The logits from the visual decoder is logits = =p((11 .... UNv)l )1
N To make the distribution of logits closer to logits!im, we adopt the loss for KD:
C= + aa)ard =
+ ( (1=a)H(X,Q)
(6)
where is the temperature, Qr Softmar ('ogii= and Pr Softmar (oggis/mm are the smoothed distributions for the student and the teacher, respectively. Q Softmar(logia (Po the actual output of the visual decoder yp, and Y is the ground-truth labels derived from Softmar((ogits/mm) The coefficient a controls the weight of the soft loss (KL divergence) and the hard loss (cross-entropy) for the learning of the distribution and the hard labels, respectively.
We test our method on LLa and Qwen2.5-VL-7B. For the training of the visual decoder, we use the images from the train set of GQA and TextVQA (Singh et al. 2019), containing 148k and 22k samples, respectively. Training such visual decoder is quite simple. As detailed in Appendix
G it takes approximately 5 hours on an A40 GPU and is highly data efficient. For LLaVA-1.5-78
its visual decoder only requires training for 6k steps with batch size of 16, using less than 100k: of
8

---

Published as a conference paper at ICLR 2026
Table 1: Evaluation results of the token compression method based on run-length encoding. The reduction rate is computed as the average reduction ratio on all samples during evaluation.
Methods & Datasets
VQA
GQA
ScienceQA
TextVQA
MMBench
POPE
Reduction rate (%)
Original Decoding
LLaVA-1.5-7B
48.93
60.50
48.46
45.01
53.73
85,96
/
Qwen2.5-VL-7B
79.04
61.20
80,55
78.21
84.68
89.23
method1: Runlength compression
LLaVA-1.5-BB
49.24
61.32
47.37
43.16
51.40
86,00
27.83
Qwen2.5-VL-7B
77.91
60.80
80.71
76.33
84.27
87,46
16.19
method2: Runlength compression Remove punctuation (Look Top1 token)
LLaVA-1.5-7B
48.59
60.14
46.59
35.04
47.33
85.66
58.35
Qwen2.5-VI-7B
77.24
58.69
79.70
71.80
83.39
85,70
45,00
method3: Runlength compression + Remove punctuation (Look at Top2 tokens)
LLaVA-1.5-7B
49.12
61.06
47.18
38.53
50.45
85,90
48.55
Qwen2,5-VL-7B
77.38
60.03
80.48
74.78
83,58
86.83
32.09
unlabeled data. For evaluation, we use GQA, Te VQA (Antol et al., 2013) and ScienceQA (Lu al. 2022) for visual question answering. We also use MMBench (Liu et 2024b)) and POPE for the test of the general capabilities and hallucination. As shown in Table L we implement three methods: (1) method 1 the original algorithm; (2) method 2 deletes all the visual embeddings that are decoded into punctuation marks with a highest probability among all tokens in the vocabulary; (3) method 3 is based on method 2, while it only remove those with top-2 text tokens both being
punctuation, which are more likely to be meaningless visual tokens. Details of these choices are shown in Algorithm 2 Further details of training and evaluation are presented in Appendix G
Results show that our method could not only reduce sequence length for faster inference, but also control performance loss within an acceptable range. Compared with previous methods that are based on instruction relevance (Chen et al. 2024a Zhang et al., 2024c; Lin et al. 2025 Zhu et al. 2024). though they seem to be better in terms of compression ratio and task performance, they are impractical in real-world scenarios because they cannot use FlashAttention (Dao et al., 2022) and require re-computing the relevance to the user prompt for every inference. Our method is more similar the methods that focus solely on the image itself. The most representative is ToME (Bolya) et al. 2022). While they measure the similarity between visual tokens via the keys from the ViT's attention, our method is more refined because converts an image to natural language and merges similar tokens directly via text strings. Thus it has lower compression ratio than ToME, but better performance on downstream tasks. More comparisons are provided in Appendix G
5.2 ENHANCING SPATIAL REASONING WITH RECTIFIED ROPE
According the discussions in Section 4.2 the component for positional discrimination (c.g., +[(g k/ 4A k (1),,()) )inn(m0))))))) has a relatively small magnitude compared with the common terms shared between the object representations of pair of spatial relationships. This is because the values of trigonometric functions, which carry relative distance information, are always less than To verify this, we focus on attention weights that RoPE directly influences. We split the dot-product attention into two parts: q k (qxx -q" "K- X = nx (I-X) + q (kY )", and compute the contribution to attention scores from the two axes by concatenating the two parts and applying Softmax to them together. For the Xaxxis and Yaxiss
(1)
separately select their "attention scores" from the "satellite" to the "nucleus" and average them across all attention heads and tokens (defined in Equation 53 54p. Details of this process are shown in Appendix H The test is conducted on What's ' Up B and Qwen2-VL-2B. Results in Figure 23 show that for "left" and "right" (in the X X-axis), attention from Y -axis is about 1.5 times larger than the X-axis, while the latter contains the key information for the determination of the positional relationship between two objects.
On the other hand, as the RoPE frequency 0, = b b-2i/d decays rapidly when the dimension group index i increases, the dimensions corresponding to large values of i become highly insensitive to positional changes. For example, when i = 2 and RoPE base = 10000 (in Qwen2-VL), for relative distances of -50 and 50, the difference in their sine function values, 2sin(500./a)) is merely around 0.01. These issues can all lead to critical information being "drowned out" by other irrelevant information when the model determines relative positional relationships between objects.

---

Published as a conference paper at ICLR 2026
Table 2: Experimental results on spatial reasoning benchmarks for RoPE scaling and basclines.
COCO- spatial
COCO-
GQA- spatial
GQA-
Methods & Datasets
What's Up A What's Up B
VSR
spatial 2
spatial 2
Qwen2- VL-2B
74.61
53.16
73.30
49.84
58.18
76.61
76.98
Qwen2-VL-22 VL-2B + RoPE scaling
77.27
58.25
73.75
50.24
58.24
78.22
78.03
Qwen2-VL-22 + SFT
78.54
61.52
73.98
58.08
66.59
81.29
83.05
Qwen2-VL-22B + RoPE scaling
79.42
63.48
74.17
59.03
66.59
82.24
83.33
Qwen2-VI-775
98.06
87.84
77.77
88.79
70.45
92.84
84.54
Qwen2- VL,-B + RoPE scaling
98.86
88.97
78.09
89.27
72.05
94.31
85.65
Qwen2- VL-7B + SFT
98.98
90.00
80,96
89.05
74.55
96.72
95.42
Qwcn2-VL-7B + SFT RoPE scaling
99.03
90.44
81.14
89.67
75.45
96,98
94.5
To address the above issues, we propose RoPE scaling, which adaptively scales the relative dis- tances via a modification: (0.)² 0, g(i), where
( = ++ (2i/d)"
(7)
In Eq. 7 d is the dimension of an attention head, a serves as the scaling magnitude, while p is used to control the scaling factor g(i) such that applies minimal scaling for small and significant scaling for large i (see Figure 24b for illustration). This compensates for the loss of positional information caused by frequency decay. We test our method (training-
fine-tune on 60k samples from the GQA training set) on the Qwen2-VL VL series of models and evaluate on What's Up. VSR (Liu et al.) 2023a), and COCO/GQA-spatial (Kamath et al.) (023). The selection of a and p, along with the training and evaluation setups, are presented in Appendix H Results in Table 2 show that RoPE scaling achieves competitive performance across multiple benchmarks. We also lest our method on MMBench and find that the model performance in general capabilities is even better with RoPE scaling, demonstrating its potential as a trick to enhance spatial reasoning ability. However, it serves only localized remedy for potential limitations in RoPE. We expect developments in positional encoding that can more effectively capture relative spatial relationships in the future.
RELATED WORK
Interpretabilityy of VLMs A line of work aims to interpret model components. For example, Goh
al. (202T); Gandelsman et al. 2023 2024) find the neurons and attention heads in CLIP that correspond to real-world concepts. Elhage et al. (2022) demonstrate that neurons are polysemantic
due to the existence of superposition, and sparse autoencoders (SAFs) are used to address this issue in LLMs (Bricken et al. 2023). Similarly, Huuofryy (2024); Daujotas (2024); |ao et al. (0024)
extract features in CLIP using SAEs. Similar to our work, Neo et al. (2024); Sonia Joseph (024)
interpret activations via logit lens. [Sonia Joseph (024) directly maps visual tokens to labels via the classification head in VIT. Neo et al. (2024) maps the visual tokens to texts via the language head.
Our work is based on theirs, while go step further to systematically investigate VLMs layer by layer, and provide a detailed analysis of the dynamic visual information processing in a VLM.
Another line of work employs the idea of causal tracing originally developed for interpreting LLMs (Meng et al. 2022). Rajaram et al. (2024); Palit et al. (2023): y Basu et al. (2024) investigate in- formation storage and transfer in CLIP, BLIP and other VLMS via visual circuits. Recently, Neo ett al. (0024); Zhang et al. 20024b 2025) use blocking-base based interventions to study the information flow VLMs. Other approaches provide visualizations for the model internals. Some work pro- vides heatmaps based on gradients (Selvaraju et al. 2017/Chefer
Chefer et al. 2021 Z0angnaang et al. 2024b),
attention (Ben Melech Stan et al.) 2024) or causal inference (Li et al. 2025).
7 CONCLUSION
We investigated image understanding in VLMs from the view of object recognition (the "what" way) and spatial perception (the "where" way). Based on the findings, we proposed an instruction- agnostic token compression method for inference acceleration, and RoPE scaling to improve the spatial reasoning capabilities of VLMs based on 2D RoPE. Our work offers new insights for both better understanding the mechanisms of visual information processing in VLMs and designing su- perior algorithms or model architectures. Limitations of this work are detailed in Appendix A
10

---

Published as a conference paper at ICLR 2026
8 ACKNOWLEDGEMENT
We thank the anonymous reviewers their insightful comments. The research is supported by the Natural Science Foundation of Beijing, China (Grant No. L247010) and National Key R&D Program of China (Grant No. 2024YFF0907003).
REPRODUCIBILITY STATEMENT
To ensure reproducibility of this work, key supporting materials distributed in the main text, appendix, and supplements. The anonymous source code for our proposed model/algorithm is avail- able in Supplementary Material. For theoretical analysis for spatial perception, key assumptions and results are presented in SectionFoff the main text, with derivation process detailed in Appendix и.
Dataset details- sources, preprocessing, and splitting-are overviewed in the main text and detailed in the Appendix and Supplementary Material. These materials enable accurate replication our results. Finally, LLMs are used to aid our writing of this paper.
REFERENCES
Saeed Ranjbar Alvar, Gursimran Singh, Mohammad Akbari, and Yong Zhang, Divprune: Diversity- based visual token pruning for large multimodal models. Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 1392-94011 2025.
Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zit- nick, and Devi Parikh. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision, pp. 2425-2433, 2015.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Siho Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025.
Samyadeep Basu, Martin Grayson, Cecily Morrison, Besmira Nushi, Soheil Feizi, and Daniela Mas- siceti. Understanding information storage and transfer multi-modal large language models. Advances in Neural Information Processing Systems, 37:7400-7426, 2024.
Gabriela Ben Melech Stan, Estelle Aflalo, Raanan Yehezkel Rohekar, Anahita Bhiwandiwalla, Shao-
Yen Tseng, Matthew Lyle Olson, Yaniv Gurwicz, Chenfei Wu, Nan Duan, and Vasudev Lal. Lvlm-intrepret: An interpretability tool for large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8182-8 1182-8187, 2024.
Daniel Bolya, Cheng- Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your vit but faster. arXiv preprint arXiv:2210.0946/1 2022.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Con- crly, Nick Tumner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodde Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023. https://transformer- htw.:///w.22...ooommmmmeeeeeeeee
Hila Chefer, Shir Gur, and Lior Wolf. Transformer interpretability beyond attention visualization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp- 782-791, 2021.
Liang Chen, Haozhe Zhao, Tianyu Liu, Shuai Bai, Junyang Lin, Chang Zhou, and Baobao Chang An image is worth 1/2 tokens after layer Plug-and-play inference acceleration for large vision- language models. In European Conference on Computer Vision, pp. 19-35. Springer, 2024a.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, al. Expanding performance boundaries of soun-s multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.0527. 2024b.
11

---

Published as a conference paper at ICLR 2026
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. Advances in neural information processing systems, 35:16344-16359, 2022.
Gytis Daujotas. Case study: Interpreting, manipulating, and controlling clip with sparse autoen- coders. LessWrong, 2024. http://wwwwwwwwooooooo////////// himmppoosiiuuiZBBBBogggs////e/e eneepreting-manipulatnggg aagaallliingaaaddooooooiiniiiiii
Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers), pp. 4171-4186, 2019.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010.1 19929, 2020.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfiel-Doodss, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei. Martin Wattenberg, and Christopher Olah. Toy models of superposition. Transformer Circuits Thread, 2022. https:/trrassfrmmer htwp:p/ww22002oooooooo////eeeeee htmll.
Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. Interpreting clip's image representation via text-based decomposition. arXiv preprint (uXiv:231000591 2023.
Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. Interpreting the second-order effects of neurons in clip. arXiv preprint arXiv:2406. 04341, 2024.
Gabriel Goh, Nick Cammarata t. Chelsea Voss t. Shan Carter, Michael Petrov, Ludwig Schubert, Alec Radford, and Chris Olah. Multimodal neurons in artificial neural networks. Distill, 2021. doi:
http://tiiilllpub/////mmiiiiiiii
Solomon Golomb. Run-length encodings (corresp.). IEEE transactions on information theory, 12 (3):399-401, 1966.
Melvyn Goodale and David Milner. Separate visual pathways for perception and action. Trends in neurosciences, 15(1):20-2) 25, 1992.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge neural network. arXiv preprint arXiv:1503.0253: 2015.
Drew Hudson and Christopher Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 6700-6709, 2019.
Hugofry. Towards multimodal interpretability: Learning sparse interpretable features in vision trans- http://wwwggooo///sss//b/bbttt WWAA////CCCC.tWaAA//ADDDDR////aa
formers. LessWrong, 2024. mullinoda--ieerrcccaiii--eeengnn
Ahmadreza Jeddi, Negin Baghbanzadeh, Elham Dolatabadi, and Babak Taati. Similarity- aware token pruning: Your vlm but faster. arXiv preprint arXiv:2503./1549, 2025.
Nick Jiang, Anish Kachinthaya, Suzie Petryk, and Yossi Gandelsman. Interpreting and editing vision-language representations mitigate hallucinations. arXiv preprint 10.02.24.0.02762, 2024.
Amita Kamath, Jack Hessel, and Kai-Wei Chang. What's" up" with vision-language models? in- vestigating their struggle with spatial reasoning. arXiv preprint arXiv:2310. 2023.
Kurt Koffka. Principles of Gestalt psychology. routledge, 2013. 2013.
12

---

Published as a conference paper at ICLR 2026
Yi Li, Hualiang Wang, Xinpeng Ding, Haonan Wang, and Xiaomeng Li. Token activation map to visually explain multimodal llms. arXiv preprint arXiv 2025.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305. 10355, 2023.
Zhihang Lin, Mingbao Lin, Luxi Lin, and Rongrong Ji. Boosting multimodal large language models with visual tokens withdrawal for rapid inference. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 5334-5342 2025.
Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning. Transactions of the Associ- ation for Computational Linguistics, 11:635-651, 2023a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, Advances in neural information processing systems, 36:34892-34916, 2023b.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 26296-26300 2024a.
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, al. Mmbench: your multi-modal model an all-around player? In European conference computer vision, pp. 216-233. Springer, 2024b.
Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. Neural baby talk. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7219-7228, 2018.
Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai- Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507-2521, 2022.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in neural information processing systems, 35:17359-17372 2022.
Clement Neo, Luke Ong, Philip Torr, Mor Geva, David Krueger, and Fazl Barez. Towards interpret- ing visual information processing in vision-language models. arXiv preprint arXiv:2410.07149, 2024.
nostalgebraist. Interpreting gpt: The logit lens, 2020. URL http:: www. wewwwweng....................... com/ post s/ Actt://wwwpaa//vo/t//ttttttrrrre
ght-thh-----.. oogit lens
Vedant Palit, Rohan Pandey, Aryaman Arora, and Paul Pu Liang. Towards vision-language mech- anistic interpretability: causal tracing tool for blip. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 2856-28666888 0866-286,220 2023.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language under- standing by generative pre-training. 2018
Achyuta Rajaram, Neil Chowdhury, Antonio Torralba, Jacob Andreas, and Sarah Schwettmann. Automatic discovery of visual circuits. arXiv preprint arXiv:2404.1 24.::240444449, 2024.
Sukrut Rao, Sweta Mahajan, Moritz Böhle, and Bernt Schiele. Discover-then-name: Task-agnostic concept bottlenecks via automated concept discovery. In European Conference on Computer Vision, pp. 444-461. Springer, 2024.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEFE international conference on computer vision, pp. 618-626, 2017.
Oriane Siméoni, Huy V Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Mare Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, et al. Dinov3. arXiv preprint 2008 10104, 2025.
13

---

Published as a conference paper at ICLR 2026
Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8317-8326, 2019.
Neel Nanda Sonia Joseph.
Laying the foundations for vision and multimodal mechanistic interpretability & open problems. Al Alignment Forum, 2024.
https//www.alignmenffooo http://wwww..aooooooooooooooo/// sabio--nddmmlimmodal-mmeeee mechanistic.
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: En-
hanced transformer with rotary position embedding. Neurocomputing, 167:12, 2024.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- tion processing systems, 30, 2017.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et Qwen2-vl: Enhancing lsion-language model's s perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024.
Zichen Wen, Yifeng Gao, Weijia Li, Conghui He, and Linfeng Zhang. Token pruning in multimodal large language models: Are we solving the right problem? arXiv preprint arXiv:2502. 11501, 2025a.
Zichen Wen, Yifeng Gao, Shaobo Wang, Junyuan Zhang, Qintong Zhang, Weijia Li, Conghui He, and Linfeng Zhang. Stop looking for important tokens in multimodal language models: Duplica- tion matters more. arXiv preprint 1xxXi:25022.1999 2025b.
Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, et al. Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy reduction. arXiv preprint arXiv:2410. 17247, 2024.
Qizhe Zhang, Aosong Cheng, Ming Lu, Zhiyong Zhuo, Minqi Wang, Jiajun Cao, Shaobo Guo, Qi She, and Shanghang Zhang. [cls] attention is all you need for training-free visual token prun- ing: Make vlm inference faster. arXiv e-prints, pp. arXiv-2412, 2024a.
Xiaofeng Zhang, Yihao Quan, Chen Shen, Xiaosong Yuan, Shaotian Yan, Liang Xie, Wenxiao Wang, Chaochen Gu, Hao Tang, and Jieping Ye. From redundancy to relevance: Information 2024b.
flow in Ivims across reasoning tasks. arXiv preprint
Yuan Zhang, Chun-Kai Fan, Junpeng Ma, Wenzhao Zheng, Tao Huang, Kuan Cheng, Denis Gu- dovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, et al. Sparsevlm: Visual token sparsifi- cation for efficient vision- languae model inference. arXiv preprint 24X1v:2410.04417, 2024c.
Zhi Zhang, Srishti Yadav, Fengze Han, and Ekaterina Shutova. Cross-modal information flow in multimodal large language models. In Proceedings of the Computer Vision and Pattern Recogni- tion Conference, pp. 19781-19791, 2025.
Yuke Zhu, Chi Xie, Shuang Liang, Bo Zheng, and Sheng Guo. Focusllava: A coarse-to-fine ap- proach for efficient and effective visual token compression. arXiv preprint arXiv:2411.14228. 2024.
14

---

Published as a conference paper at ICLR 2026
A LIMITATIONS
To facilitate the future advancement of our work, we list some limitations here for discussion:
(For object recognition in VLMs) In Section while studying the object recognition process VLMs, we directly extracted the output of given ViT layer and passed through the modality connector and language model, using logit lens to examine the content represented by that layer. We discovered two-stage process from attribute recognition VIT layers deepen. However, it is possible that during pre-traine training, the parameters of the modality connector were specifically adapted to the final layer of the ViT. Directly passing outputs from other VTT layers through this connector may not fully reveal the content represented by those layers.
semantic disambiguation as the
Therefore, we believe a more rigorous approach would to add a dedicated modality connector for each VIT layer, fine-tune it on a large-scale dataset, and then observe the layer's representations. Though it is more rigorous, it would incur more computational costs and more time-consuming and labor-intensive than our method. We thus leave this more rigorous approach for future work.
Section our study of spatial perception in VLMs only considered the four basic relationships of front, behind, left, and right. We did not (1) investigate the difference between "bchind/above" and "in front of/below," nor did we (2) study more complex spatial relationships (c.g.. top-left, bottom-right). Regarding (1), we believe more complex struc- ture exists within the model to distinguish between "behind" and "above" in a 2D image, which corresponds to 3D space. For instance, the model might observe the relative sizes of two objects and combine this with prior knowledge to infer depth properties and thus their spatial relationship.
(For spatial perception in VLMs)
Regarding (2), believe that based on our theoretical analysis in Section 4.2 more complex lationships like "top-left" could also be conveniently described by formulas and verified using the empirical methods from Section 4.2 We leave these investigations for future exploration.
B THE USE OF LARGE LANGUAGE MODELS
LLMs are used to aid our writing of this paper, primarily for checking vocabulary, correcting gram- mar, and polishing the prose.
C DETAILS FOR THE ROTARY POSITION EMBEDDING
C.1 ID ROPE LANGUAGE MODELS
Rotary Position Embedding (RoPE) was originally introduced in language models to represent the relative distances between tokens (Su al. 2024). Due to its excellent extrapolation capabilities, it is also adopted in vision-language models for position encoding in image sequences, enabling variable-resolution image processing. Here, we detail the working principle of RoPE and its practi- cal implementation details.
First, we consider the 1D RoPE used language models. The core idea of RoPE is to multiply each query and key by its positional information before calculating the attention scores. When query and a key undergo a dot product, their positional information interacts and is converted into the relative distance between them. Specifically, RoPE can be expressed in both complex and vector forms. Suppose we have a two-dimensional query qm. at position index m,, with vector form of [gmn qm and complex form of (0) +igm. RoPE first applies position-dependee transformation f (gm. m) to this vector to imbue it with positional information, as shown Equation
[gm: m) am 2m0
(0) + iq (cos(m0) + isen(m0))
(8)
(0 cos(mo) - a sin(m0)] + il sin(m0) + q cmccosmm))
(0)
15

---

Published as a conference paper at ICLR 2026
Based on the result of Equation 8 can express this transformation matrix multiplication, as shown in Equation 9
cos(mo)) - sin(mo)
(0) gm
f(mm m)
cos(m)))
9m
qm cos(mo)) qm sinnsinss (mo) (0) 9m sin(mo) + 9m cos (m0))
As can be seen, the transformation RoPE applies to the query fm is effectively multiplication by rotation matrix that carries its positional information. The frequency is typically 9 = b-2%/d, where b the RoPE base, d is the model's dimension, and is the dimension group index (in this example, i only takes the value of 1). Now, we consider the dot product between query at position m and key at position n. First, we illustrate this process in Equation using matrix multiplication:
< f(mm m), fknn n.
=f =fmm m) n))
cos(m sin(mm)
sin(maf)
(cos(n0) sin(n0)
-si((n)) cos(n0)
(1)
qm
cos(m0))
k
(10)
(0) qm
(1)
cos(
()))
[m]] sin((m m20)
kin
qm
sin (m n))0
cos( (mm m)))
= -4 (0)))
kn 1
(eos)))ccmmmmm n)) (g k
a!!h ) n)0)
As can be seen, the dot product between the query and key effectively results in the fusion of their corresponding rotation matrices. The resulting rotation matrix contains their position difference, m n, thereby modeling the relative distance. Similarly, based on the matrix multiplication form, we can express this in the form of complex multiplication, as shown in Equation II
(4m, m). [Kin nt)
[{mms m)j (km, nt)
Relgmme e0m0 imo
(kn))))))))
(11)
=Re K (mm-n)8]
=(g!0)/0) k.(0)
( (oos)(((m n)0) (
(in n)))
As can be seen, when the query and key are expressed in complex form, their dot product is the real part of the multiplication between the transformed query and the conjugate of the transformed key. For the convenience of subsequent discussion, we will primarily use the complex form represent the dot product process.
C.2
2D ROPE IN VISION-LANGUAGE MODELS
Due to its favorable extrapolation properties, RoPE has played a positive role in long-text pro- cessing. To enable variable-resolutionn image processing in Vision-Language Models, |Wan et al. (2024) introduced 2D RoPE Similar to the basic form of ID ROPF, 2D RoPE introduces two- dimensional positional information to separately represent information along the width and height directions for each patch in an image. Specifically, 2D RoPE divides the model's dimensions into two equal halves, used for processing information on the X-axis (width) and Y-axis (height), spectively. Suppose we have a four-dimensiona query 4m at coordinates (m1.m²). Its vector form
ia I 1qm 4m 9m (3)1, where qith 1 4m (1) dix
(1)
[g(?) (3) Its complex form
is X XX = Ig.0 + iqm [2) + iqm
(3) where 9,t qm igmh dt. =qm + iqm The
(2)
transformation f(am, ml, m2) that 2D RoPE applies to this vector is shown Equation 12
co)
sin(m11)
(0))
(in)))
cos(m,0)
(())
[gm, Mi, T
(12)
cos(m22) sin(m20)
sin((m2))
g(3)
0
cos(m20)
(33)
16

---

Published as a conference paper at ICLR 2026
When written in the complex form:
fams my, m2), f (kn; R1, n2) > = Re [qm elmo e elmett ((ki e (Ky )*)
Re Im Ki
X x* cfmi-ni]0 y x Bm e clmm--m2)0
(13)
clear that have effectively constructed two separate rotation matrices, using gm qm and qm qm to represent positional information on the X and Y axes, respectively.
C.3 ENGINEERING DETAILS OF RoPE
In practical implementations, there are two main styles for RoPE: GPT-J style and GPT-NeoX style. The GPT-J style pairs adjacent dimensions of the model serve the two dimensions for rotation matrix, as shown in Equation
x0())
cos(mbo)
-x(1)
sin(mBu) sin(mbo)
x(1))
cos(mbo)
x11)
SGPT-J(Am; =
+
(14)
xld-2)
cos(m////-1)
(d(-11)
sin sin(m0aa/211 sin(mBa/22
(old-1)
cos(mba d/2
(dd22)
In contrast, the GPT-NeoX style, for computational efficiency, pairs (i O,..., (d 1)/2) with ((d+1)/2+5) through "rotate half" operation, as shown Equation 15 Qm
fGPT-NeoX (mm, m)
2 (0)
x((++1//2) - ((d-3))12)
cos(mbo) cos(mo))
sin(mal) siu(mnn)
(1)
(0(( ()/3)
cos(mm c0s(m00/222)
(dd-22) 2)
sin(mo
2x(d (d-1)/2)
cos cos0mma/222 cos(mlu) cos(mB2)
(ld-1)
sin sin(mbaa/2-
(15)
x (1d+1)/22
(0)
x: (d433//2)
(11)
sin(mo,)
: x (1d-3)/2) x (d-1)/2)
(d (d-1)
cos (mm m cos (m0 a/20)/2-1)
sin sin(mbar222) sin(m0a/2211
Most of the models nowadays (eg. Qwen2-VL) adopt the GPT-NeoX style for the implementation of RoPE, while we use the GPT-J style in our theoretical analysis for convenience of discussion.
17

---

Published as a conference paper at ICLR 2026
D DETAILS FOR THE MODELS
In this work, we primarily use three models: LLaVA-1.5-7B, Qwen2 2.5-VLL77 and Intern 2.5-8B. We list the key parameters of these models relevant to our work in Table
Table Detailed information of the models used in this work.
Model Info
-LaVA-1,5-7B
Qwen2.5-VLL VL-7B
Intern VL-2.588B
dynamic image size
False
True
True
image size patch size
336
448
14
14
14
num hidee.aadsss
24
32
24
VIT
hidden size
1024
1280
1024
attention head head_dim
16
16
16
80
position embedding
ID learnable (1+24*24)
2D RoPE
ID learnable (1+32*32)
Connector
downsampling rate
0.5
0.5
num haydennsaeerrss hidden size vocab_sin sizc
28
32
LLM
4096
3584
4096
32064
152064
92553
18

---

Published as a conference paper at ICLR 2026
E ANALYSIS BASED ON LOGIT LENS
E. ANALYSIS ON THE REPRESENTATION SIMILARITY
For the analysis of the object recognition process in ViT, a natural idea to use similarity- based methods for visualization. Therefore, for the output representation H -21, INyy E RVvxDV (1 [1, Lv]) of each ViT layer I, first t-SNE reduce its dimensionality to three, and then perform clustering via K-means, The number of clusters. K, is set to the number of prin- cipal objects in the image (as provided in the GQA dataset), We employ two clustering strategies:
1. First perform standardization, then apply K <-mans clustering based on Euclidean distance (the conventional approach).
2. First perform L. normalization on the representation each image patch, then apply K- means clustering based Euclidean distance.
The essence the second clustering method that it is actually based on the cosine similarity be- tween data points. Specifically, after L2 normalization, the representation of each patch in the image sequence is a unit vector. At this point, the squared Euclidean distance between the representations of any two image patches can be written as: ||<; xy ||>>|| 2r,ær + ||ær,|12 = 2(1 x; As can be seen, clustering based on Euclidean distance at this stage is effectively equivalent to clus- tering based cosine similarity, since the term 2,z is equal to the cosine similarity | between unit vectors 2; and zz.
2,2,
Original
VIT layer
ViT layer
VIT layer 10
VIT layer 15
VIT layer 20
Figure 8: The visualization of the image representations VIT layers LLaVA-1.5-7B, based on dimensionality reduction (t-SNE) and clustering (K-means method 1).
The visualization results for the two clustering methods are shown in Figure and respectively. is evident that as the VIT layers deepen, the clustering quality deteriorates. This is because in the shallower layers of the VIT. the image representations are still low-level visual features extracted during the initial patchifying process, resulting in clustering visualizations that resemble the original image. However, as the layers deepen, the features of the image patches undergo frequent interaction through the attention mechanism, and the representation at one position becomes mixed with repre- sentations from other positions. Furthermore, deeper representations often correspond high-level semantic information, which is difficult to measure simply with linear metrics.
Recently, Siméoni et al. (2025) used a technique DINOv3 called Gram anchoring to mitigate feature map degradation during self-supervised training by reducing the discrepancy between the student network's 's Gram matrix and that an earlier model, thereby maintaining patch-level consis- tency. However, this method has not yet been validated in the training of VLMs. Therefore, studying model representations using metrics based on linear spaces, such as similarity, is often insufficient. We need to find more reliable metrics, such the multimodal logit lens detailed in Section 3
E.2 DETAILS ABOUT THE TOKEN MAP
In Section 3.1
we provide an illustration the token map and briefly describe the way of drawing segmentation map based the token map of an image. Here we discuss in detail about these.
19

---

Published as a conference paper at ICLR 2026
Original
VIT layer
layer?
VIT layer 10
VIT layer
layer
Figure 9: The visualization of the image representations in VIT layers of LLaVA-1.5-7BB based on dimensionality reduction (t-SNE) and clustering (K-means method 2).
The first step is decide which layer's output from the VLM's language model should be used as the "logits" for the logit lens. Intuitively, the final layer might seem opti- mal, as it has been processed by more Transformer blocks and should thus contain richer information. However, this not always the case. To make more rigorous deci- sion, we determine which layer's representation to use based the emergence of representative words. Specifi- cally, we first randomly select 500 samples from the GQA dataset. Each image in GQA is annotated with the names and bounding boxes of its principal objects, yielding an object name W {w :..., wpm) for each image, where w is the name or the "representative word" of object o₁. Then these samples enter the VLM, and it- erate through each layer of the language model. For each layer, we pass output through the language model head obtain the its logits, which are then decoded to generate token map. For every text token in this map, if it falls within the object name set W. we classify as a representative word. After iterating through calculate the emergence rate of representative words as n
0.25
0.20
0.15
0.10
0.55
Figure 10: The emergence of represen- tative words in LLaVA-1.5.
entire token map, we Ny where n is the count of representative
words and Ny is the length of the image sequence.
We compute the average emergence rate for each layer of the language model, and the results for LLaVA-1.5-7B are presented in Figure I0 As can be seen, the 25th layer of LLaVA-1.5-78 shows significant emergence. Therefore, select the representations from this layer for our subsequent experiments. Similarly, for Qwen2.5-VI.-7B, we select the final layer.
In Figure 14 15 16 and
present the token maps derived from LLaVA-1.5-7B and Qwen2.5- VL-7B, corresponding to the three images shown in Figure [1] The token maps are the results of the original model, without any intervention described Section 3.1 In token maps
18 19
#include <iostream> using namespace std;
int main()
{
cout << "Hello World"; return 0;
(a) A bear on log.
(b) child skiing.
(c) C++ code.
Figure 11: Example images in this section.
20

---

Published as a conference paper at ICLR 2026
0.02
648
Lowir
(a) Accuracy
(b) Percision
(c) Recall
Figure 12: Experimental results of LLaVA-1.5-7BB on the POPE random set. The hallucination rate is tested directly using the token map corresponding to each ViT layer. From shallow to middle layers, the model is state of random guessing. While from middle to deep layers, the model gradually figure out the exact categories of the objects in the image (semantic disambiguation).
from LLaVA-1.5, we can observe a lot of meaningful text tokens that describe the image content in the corresponding image areas. Thus we provide a way of readiing images like texts". In token maps from Qwen2.5-Vi. there are also lots of meaningful text tokens in both English and Chinese, since the model is pretrained on multilingual data with Chinese accounting for large part of the corpus in the data. Besides, we also observe some meaningless tokens like the punctuation and whitespaces, or the "in" token which frequently appears in the token maps from V.L.VA- We regard this kind of tokens as meaningless tokens that might be redundant for representing the image, which would be verified via our token compression method in Section 5.1
we observe a tendency that the visual encoder exhibits a two-stage process from shallow to deep layers: first performs attribute recognition from shallow to middle layers, and then performs semantic disambiguation from middle to deep layers figure out the specific object categories in an image. To further demonstrate this, we test the model's hallucination layer by layer directly based on the token maps. Specifically, we use the POPE benchmark, in which the input to a model is an image together with question asking whether a certain object is in that image or not. We use the POPE random test set and input images and questions into the VLM. Then we perform layer-wise intervention on the ViT (visual encoder): we iteratively process up to each layer, discard all subsequent ViT layers, and then obtain the token map.
In Section 3 3.2
In the POPE dataset, the answer to each question is either "yes" or "no," indicating whether an object in the question is present in the image. We determine the final answer by checking if the token map contains the name of the target object. Specifically, the POPE dataset is built upon the MSCOCO dataset, and et al. (2018) provide a synonym map for objects in MSCOCO. This allows us to treat semantically identical nouns as the same class (e.g., truck, pickup, lorry, hauler, firetruck all belong to the "truck" class). Therefore, we can iterate through all text tokens in the token map. If token matches the target object's 's name or its synonym, the answer is set to "yes", otherwise is "no".
Through this blocking intervention on the VIT layers, we obtain the accuracy, precision, and recall for each layer, as shown in Figure T2 Note that "strict" means that we only look at the Top-1 token (with the maximum probability), ), while "loose" means we look for the target object names in the Top- 3 tokens. We observe that the VLM is essentially in state of random guessing from the shallow to the middle layers of the ViT (up to layer 11). Starting around layer 11, the model's 's accuracy steadily improves. This indicates that the process of semantic disambiguation in the VLM truly begins in the middle layers, while the preceding layers are primarily processing low-level visual features, which corresponds to what we term the attribute recognition phase. Furthermore, the final hallucination rate measured via our token map approach (in terms of the three aforementioned metrics) is almost identical to the rate obtained from testing the model normally. This proves that the token map can faithfully represent the model's 's object recognition process.
E.3 DETAILS ABOUT THE SEGMENTATION MAP
As for the segmentation map, it is bulit on the token map. The core idea is to identify the label (object category) of a text token in an image patch and fill the patch with the color of that label. The detailed process is shown in Algorithm T It generates a segmentation map through a two-phase process centered context-aware token refinement. In the beginning, we write set of keywords
21

---

Published as a conference paper at ICLR 2026
Original
VIT layer
ViT layer
VIT layer 15
VIT layer 20
VIT layer
Figure 13: The segmentation maps of the image representations in VIT layers of LLaVA-1.5-7B. From shallow to deep layers, VIT gradually performs semantic disambiguation.
Wo. = [w": wf" w for each object o, of the main objects in an image. For example, for the object "bear" in Figure Ia its keywords set is ("bear", "head", "eye", "nose", "paw"), where "bear" is what we called the "representative word (w?t)" )" or the "label" of this object, while other keywords are either the aliases of the object, or the names of the elements in the object. Note that "background" and "others" are added as additional objects.
The first phase iteratively constructs label map by evaluating each token from the initial token map. If token is already a recognized keyword for a specific object, its corresponding object label is directly assigned to the map. For unclassified tokens, the algorithm initiates local refinement procedure. This procedure collects all meaningful neighboring tokens within a one-step Manhattan distance, filtering any meaningless tokens such as punctuation or whitespaces. In the second phase, this completed label map is translated into the final visual output by assigning a predefined color to each position the segmentation map according its object label.
The core of the refinement lies in a voting mechanism used to determine the token's 's new identity. first identifies the most frequent token(s) among the meaningful neighbors. If a unique winner found by frequency, it is selected. In the event of a multi-way tie in frequency, the algorithm arbi- trates by selecting the token whose occurrences have the minimum cumulative Manhattan distance to the central position, thereby prioritizing spatial proximity. The winning token's 's corresponding object label then assigned label is set to "others". If token has no meaningful neighbors, it is designated as "background".
map. the winner not a keyword for any known object, the
E.4 FURTHER DISCUSSIONS ABOUT THE GESTALT RECOGNITION
we discuss about the similarity between the object detection process in VLMs and human recognition. Here we put details about some of the Gestalt principles of perceptual, and discuss about the corresponding mechanisms VLMs.
In Section 3.2
Principle of Similarity: We tend to group together elements that share similar visual characteris- ties, such as shape, color, or size. In VLMs, the scaled dot-product attention is related to represen- tation (cosine) similarity. Thus the attention mechanism encourages the interaction between visual tokens, and allocates more attention to similar tokens.
Principle of Proximity: We tend perceive elements that are close to each other a single group. In VLMs, especially those using 2D RoPE, tokens that are positionally closer tend to have higher attention scores, and thus they are more likely to be in the same object in the view of VLMs.
Principle of Closure: Our minds automatically fill in gaps in an incomplete figure to perceive a whole, complete object. In VLMs, due to the raster scan operation, the image patches that are adjacent to each other in 2D space may become positionally discontinuous in 1D sequence. Nonetheless, the model is able to first associate them together as discussed in the aforementioned two principles, and then call prior knowledge stored in its weights to determine whether they belong to the same object. For example, though the feet of the child is not adjacent to his body in the image sequence in Figure the model is still able to put them togther.
22

---

Published as a conference paper at ICLR 2026
Algorithm Segmentation Map Generation via Token Refinement
Require: Object-keyworde map {o W"); Object-color map {o + c): Initial token map Map foken, Set of meaningless tokens M..
Ensure: Final segmentation map Map"c9_
1: function RESOLVEUNCCASSSHHDTOOOKKNO j. Map, {o - W.
2: N +-1]
Collect meaningful neighbors and their Manhattan distances
3:
for di {--,00, and E (- 0] do if (di, dj) # (0, 0) and (i+ di df) is within map bounds then neighbor + Mapli + di, di]
4:
5:
6:
neighbor M then
7:
Append (neighbor, |di] + dji) to N
8:
end if end if end for
9:
11:
if N is empty then return "background"
12:
end
13:
F T- CoumtFrequencies((N)) C +- FindMosiCommon(F)
14:
mar- +- C[O] count
15:
TiedTokens okens (tg. token for t4 in C if tg.count m-ammarccmm coc.cc
16:
if [ieedTokens] then
b If there a unique winner by frequency
winner TiedTToken(()
18:
else
Resolve multi-way ties by finding the one with minimum distance winner FindTokknWithhiDDittaeeeee (Ted AN)
20:
end if
winner winner-object +- GerOOhietNammeFooooeeennn
fo + Wo])
22:
return winner _object if winmer winner_object + null else "others"
23:
end function
24:
procedure GENERATTESECNNPPMMOpp troken {o Wr) fo- M)
25:
Map tahel +- NewMap(h, w)
Phase 1: Refine to create direct Label Map
26:
for = 1 to do
27:
for j = 1 to do
28:
object + (etDhhetthamfffoorbaaaaaaaaaaaa (i. (o Wo))
object o null then
30:
Map tabel [i + object
>> Directly assign the label if token is a keyword
else
32:
Map label [. + ResolveUnclasffiefTloke/// Map fo + W..M)
33:
end if
34:
end for
end for
36:
Map sgg <- New.mage(h, w)
Phase 2: Draw Segmentation Map
37:
for = to do
38:
for j = to do
39:
Map [i. + GetColor(MMppaher - ct)
40:
end for end for
41:
return Map
43:
end procedure
23

---

Published as a conference paper at ICLR 2026
Mindows
Val
103
A01
pear
Hiwww:
mdll
store
boxt
box
200
ssumne
walk
1n
2807
Bear
bear
bon
bear
noct
mec
Will
Stecs
zone
beat
bear
baar
2887
deep
roda
rocts
Will Nonated
Ca
Bear
hAy
Dee
bear
Iabe
mec
Wi1
0/16
MPp
log
loge
Igg
200
part
welt
WS1
Ind
lige
In
lngt
loge
borr
and
See
roct
logs
200
lgg
mall
logs
loga
Ingr
wit
lige
in
logs
Inge
bpp
kop
250
logo
mole
log:
in
In
In
lage
in
In
water
Nater Water Weter
WIW
1gg
pod
wattr
relotton
loge
tn
In
age
ltg
Iege
poct
waeer Ietertid Kater pell
In back
WWW
Intactoo
Infectos
kgp
150
colar
p0ir
Wall
wall
coler
pole
infaction
water
polar
poia
21
water
mar
polar
Figure 14: The token map of Figure 1a from LLaVA-1.5-7BB
24

---

Published as a conference paper at ICLR 2026
My
doud
doad
ceuds
Biue
0ced
dddd
dlud
doud
cool
by
clud
sky
dead
Viea
de
p36
douce
dbad
dbud
24K
15OW
2R0W
STD0
Swet
00W
di
mix
mise
90
SHOW
50ex
of
jazt
64
IroW
ed
64
inow
srow
bou
badkg Surddd
3700 snow
41
m0y
in
one
Beit
mide
kg
SHOW
170M
d5ga
Fide
84
170
84
moe
deid
lodee
KYOW
2000
pth
number
2000
1706
41
64
1n
66
rhis
900W
SHOW
pmb
SROW
370W
1009
9156
100W
2r0w
32
80
150W
2700
3026
3700
200W
show
S9OW
Figure 15: The token map of Figure
11b from LLaVA-1.55
25

---

Published as a conference paper at ICLR 2026
Gode
Bodos
Bolevt
Herr Bolox:
sode
plas
pla
doc
Aolat
AbBa
Ade
Adiat
bods
rnde
mide
Inde
mode
cnde
oode
Stvam
Tame
NATK:
sode
tode
codk
NAME
lona
cade
2ode
man
code
NAME
code
Sodoul
hollo
ricc
code
hollo
nnle
Iedc
Indu
morl
warls
World
Mar
lnd
web
Wale
mnd:
VICE
cade
world
Deum
cade
mode
cide
rode
3nth
cAde
PM
Figure 16: The token map of Figure 1c from LLaVA-1.5-7B The original image is resized to 336*336 px, which results a 24*24 sized token map.
26

---

Published as a conference paper at ICLR 2026
<im
*b
a
*6
afs
enclosure
exhibit
enclosure
enclosure
walls
exhibit
exhibit
peper long
polar
large creamy enclosur
bear
formations
large
on
bear
and bear and cre
10:88 S2
9R
on bears bear eyes
water In pool pro
long cabin enclosure
on frost on and near its nose R looking poler *
echibit -ike
with log of
0.b E
betweer den simming
and
trunk
of immendd head
Im_end black
*
per
dipping
enclosure
edge large
swimming
grain
huge
enclosure water
light photo
large
pool A 200 neur water blue
blue
and
large
across
large
over
edge
blue
blue
pool
dark
light
body
edge
on
use
edge
dark
endlasurv
BR
rock
polar
Figure 17:
The token map of Figure
la from Qwen2.5-VL. 7B. Some of the meaningful Chinese
characters in this token map: ")""
"E [iii" "ground", "iithen "wall", "si "rock". "Fi"
"stone". "the "block", " e% "crack", "=E" = "fur" "*"" = "rice" "o = "ice", "Ei""
"stripe". "At" "bear", "f"" "body", "'"""" "paw", " " "nose". "iH" "stretch". "ast.
"climb", "x"" = "waterr", "iat"" = "deep", " "log", "B"- "round".
27

---

Published as a conference paper at ICLR 2026
im_end
box
im
ing
dressed
poles
poles
snowy
sky
ing
B
#
ia
resort poles
resort
landscape one
ski
X%
Junior
and
tracks plant
B
winter
anle
#
and < im end
j
run
prssineekggouud
*
#:
7
1
1
ing
show
lines
tracks
TE:25
skingg
left
#:
#
polos- im, m_ennd
snow
bahind grip tracks
red
FHR
#
ski
68
Verb
gripping top
im_end
black
M sports ler
very
learnin
right
Ing
poles
gripiin
child
holding one winter
right still lttle
red I red on snow snow Is snow sitting on
on on on blue blue pristine taking pristinocantrolled 4 ing be
powder
and thild blue
ski
5
sk
and d+8k feet snow be < immdend >x sk sk
Abomic
US
blue
tracks
blue
snow
on
uneven
and
child
deep
blue < im and
with foor
11b from Qwen2.5-VL-7B. Some of the meaningful Chinese characters in this token map: "i "blue", "(z)) / "# "cloud". "T "sunny". "ati""
Figure 18: The token map of Figure
"snow", "['" "white" "ol ~ "cover", "[lI" "mountain", "Iæ" "peak", " = "'blique",
"* 4" "weather", "Ah" "sit", "Hf == "hat", "=E" "fur". "Rrm" "keep warm". wit "hand", "HH"" = "ski", "#T" = "pole", "=" "hold", "# "knit", " f "black", " "stripe", "M""" "down", "Bh" = "pedal", ">/"7"" "child".
28

---

Published as a conference paper at ICLR 2026
ream
mimerdd Airddd
The
Zhe
intude
include
dostrom
end
38
remppaneesos
<immendpp
pinclde
outpur
inoond-man main
progrom cussut
beckg
blue
end
Ibrary
endipinpin
<imm endiom end end
im.end <<
The
Std
Hell
ell
0
cass
Fandard
a
cout 0ut
hello
<lin_essom conssmnsseeee m enuond
cott out
in enditn end
Help
font
World
sring
<ie endrom end
retur
retum
Im www.nndymm.om Lnd 188t
#incude
induce osrream outpux At
in.ondiencond
cobr
<
<im_ond Lnndddd std
prispet
This
the
nemezpade
when
8M
Figure 19: The token map of Figure
Icc from Qwen2.5-VL-7B. Some of the meaningful Chinese "this segment".
characters in this token map: "J Fx "segment", "ix FM
F DETAILS OF THE ANALYSIS ON SPATIAL PERCEPTION
F.1 THEORETICAL ANALYSIS
M4(0,
(- B(0.0)
X
A(m,
B(0,
X
B(0,
X
A(0,m)
helationship
"left
"right of"
"behind!
Input sequence
(g)
(Xa.3u)
(Ka.
The idealized computational model for the theoretical analysis spatial perception in VLMs. We choose four directions A relative to The "Input sequence" refers to an input to certain layer in the visual encoder. The order of A and B in the input sequence is the result of raster scan when the 2D image is flattened to a 1D sequence.
Figure 20:
In Section we theoretically analyzed how VLM using 2D RoPE distinguishes between differ- ent opposing directions. Due to space constraints in the main text, we present some of the formulas and the detailed derivations here.
We consider two objects, A and B, simplifying each to a single image patch while ignoring the back- ground and other content. The resulting input sequence has a length of 2, as illustrated Figure 20 Considering specific layer in the visual encoder, we first calculate the output of its attention module. In the attention calculation, we disregard the scaling factor, the Softmax operation, and the output projection, considering only the dot product between queries and keys and the weighted sum- mation over values. We consider four directions R = "left", "right", "front", "behind"). When the direction of A relative to B is r(r E R), denote the outputs for the positions of object A and
29

---

Published as a conference paper at ICLR 2026
object B as h" and hg. respectively. They can be expressed in simple and clear form, shown in
Equations 16 23:
f h A
f(ga, m. ((k,, -m, > Ea+ fl,,, 0,0) > uB
(16)
-m,,) left =< (,, > V < 01, 0,0) "B
(17)
=<
(18)
rrghtt
=< [(,n..00000,(((a,000))) > (000000)>>00000000000000000//010 <|0gg v4
(19)
bahind A
=< - m 4x4 0n), f 0,0) >
(20)
hB
behind
=<
-n) > A사+ < 0lg 0,0) > UB (21)
bronnt=
=< flea. f(k,, 0,0) ug+ flqa, 0,
> VA
(22)
h hfront =< f(g.. fkB, 0,0) ug+ f(gg, 0, 0), f (k. 0, > VA
(23)
When we expand the dot product form into the complex form (see Equation 13). they can be written as shown in Equation 24 31
Naft Rela X XX* + xx lUA + Relax ((- + 9 kig JuB n
(24)
Aeftt = Relggh X xx* e imo + k JA + Re K + k JUB
(25)
h hright = Relgx
X Ky + + JUA + Re x cim + + "
(26)
h hright = Reiq X e (((mm)) + - UUA + Rela
k
kD + 1 "|uB
(27)
h
behind Relg KA Relqg kx
+ 4
y k l0A k
Relg + q k ed )(
(28)
hB
behind
+ q
y xxx"" e000 ua + Relg bo q 1 Y JuB
(29)
A front
Front Relgx X
k
+ q x Jox + Relgx + g e ei
(30)
hg
Rela X + v < + Relg x + k Jun
xx*
(31)
As discussed in Section 4.2 by comparing the four representations of object A, (MA)ERR, we find that the coefficients of dA are identical across all of them. The only difference lies in the X-xxss related component of the Uy coefficient. Comparing jefet and rrghtt we find that the X-axis components of the ug coefficient, Re(qx X k e ed(-m0) and Re (t kg e((m)), formally possess pair of conjugate symmetric terms, as shown in Equation 32 and 33
RelgX X K e ((-mo)) lug
=Re[(g]) (0) + ig A (k (0)
(1)
ikk (1)
(cos - iisi((m))]
(32)
-Rel
(0) #B + GA k - i(ga kg - -9 (1) ) (cos(m6t) - is))(mm))) uB
(1)
(0)
= =[(9A] (0) (0))) + QA )c)((mm)))) (g
(1) (((()))
1 ke) q ) uB
Relgx W ((mo)) JBB
=Re((g/
(0) igg (1) )1(0 ) (cos(mB) + iisi(mm))]
(1)
(33)
=Re[[(g) (0) k + (1)((1) -(g
(1)
10) k1) 2A (1kk))
(1) (0) )] (cos(mB)) + isin(mm))] un
= =[(0)] (1))0)))) m + LA )oo((ml) + (g
(1)
(0) k LA k )sin(mf)]up
(1)x
30

---

Published as a conference paper at ICLR 2026
As for the direction vectors, we first compare Pleft and rright, shown in 34 and 35
hlets -1 Vieft
feft =Relg x + t o ny em
98 A Juat
x
(34)
RelgA " e((-m0) + + - 4B kg - 1 JUB
Dright = hright
nrght
Relgx hx² + +xxxx 4B k ei(-m0) - k Teat
BB
(35)
Relgx kxt* cim0 + + k on kn - k '00
Through comparison, it can be seen that Steft and tright share a lot of common terms: Relg ky + 2x k2x o k |u and Re Re(y k T k-x kaki lug. They can be written as C + C2UB- The differences are underlined in Equation 36 37 which can be written as (C3 + C1UB) and (CaUA + CLUB), respectively. It is obviously that fteft and Bright share pair of opposing vectors,
implying the spatial geometry of the two directions.
Pleft =Relyx kx + +x A lus Rel-qThh xx eime
Relax kn edc ([[-mm)]u) uB + Relg ki
n -
4B k 1
Relgx kx*
+ qx x - ki JUA -(18 Ky
(0)
(1) )cos(mm))A
+ 44 KA
(36)
(9
(0) k())) 4n k ) ( (0) (1) (1)
(1)
KB + + KB )cos
(9) (0) k qA KR )sin(mB)uB + RelgX kY q k 4 kb 1 JuB
-IUA + +2UB - (cSUA + +CUBB
-RelgX xX* + g k Uu + Re(- xxx* ci(-m0) Aaat
Oright
Relqx X KB imi
UB + Re y* -
1 Y JUB
= =Re(gā kx + +9 kxx -/ JUA (9p Kt + qB )eos(mO)VA
(1) (1)
(37)
+ + (0) k()) - on
(1) (0) )sin(m0) + (g ( (0) + qA ccos(m0)6B
(1) (1)
+(gA (0) (()) (1)
9A k )sin(m0)uB Relgakk X T K - - "
=C + C2UB + (cJeA + C&UB)
As for the direction vectors of "left" and "behind", we analyze them in the same way before, as shown in Equation 38 and 39 The underlined parts in the two equations represent the terms they do not contain relative to each other. To make more clear, we re-write fteft and Ubehind into Equation and 41 We observe that the term (K UA + Ky UB) in Ubehind is absent in Uteft, while the term (kg C + kn)) in Vieft is also absent in Ubehind- We regard these terms as the key terms for determine the relationship between objects.
Cleft =RelgA KX + qa IuA + Rel-ggA bix iim - Y x JJAA
Rel-oø X X- - lun + Relga kB x* ((-m() + g ki Y QB
JUB
Re(uy X A:x + +x JuA + Rel-u 1 1.x
Y X "
[ (9 (0) (0) + 44 KA )cos(m8)
(1) (1)
(1p
(2) (2)
+ 4B R juA
(3) (3)
(38)
[- (0 (0) (())) - 4B 1 Jsin(mo)
(1)
vAt
(9 kB + 4A kg (cos(mo)) + (9 (2) (2))
(0)
(1)
(3) 33)
9 kg Fgt
[ (g00 (() q kB
(1) (0)
)sin(mo)) UB
31

---

Published as a conference paper at ICLR 2026
Ubehind =Relqx + k Ra 2 hh cdmte)uaa
Rel- 1 1 JUB Relgx kg KB
+ t e"t-nolJBB
Relgx + UUA k
"
[ - (4n (0) Kx + +0B (1)) - (1/ (2) (1) + tB (cos(n0) vat
(3) (3)
(39)
[(9 (2)) k - 9B (2)
(2) (nn)
[(0/)] "B + 41 BB (41
(0) (1) (1)
(2) (2)
+ + "B [oos(nn)))mn))
(3)
[9 (2) (8) 9A B
(3) Jsin (n0) UB
Veett pommmon (kg Un - kj va)cosÓ + (1:) un ky VA)
(40)
(kg VA k2 uB sind + (K UΛ k vB)
Ufront Voommon + un Un + (k) vB - Ko A coss
(41)
(Kg Un kx un) (K/ va + kx un )snd
In Equation 40 and 41
Vcommon Relgx 'i
+xxxX"10x 1UA
Uu + Rel-
k Y
lUB
(42)
1X (0) (0)
4B (1 + 48 (1) (1)
(1)
(43)
KXX (0) (0) + 9 (1) x0)
(44)
K. = In (0)(11) 41 1
(45)
K 4A A 9A 1
(0)
(46)
KY = 9B K + 44 (3)
(2)
(47)
K. 9A
(2) (2) + 11 (3)
(48)
KY 9B ( (3) (3) (2)
9B KA
(49)
KY GA KB
(2) (3)
LA K/
(50)
F.2 EMPIRICAL ANALYSIS ON THE REPRESENTATION OF SINGLE OBJECT
In Section 4.2, we first hypothesized that in an image sequence processed by the visual encoder, the representation of single object contains sufficient information for the model to determine its spatial relationship with other objects. We then designed an experiment called "erasing objects" to test this hypothesis. The details are as follows.
"Frasing" an object in an image involves replacing the embeddings at positions corresponding to that object with the embeddings of another object within the image embedding V E NDD, which has been processed by the visual encoder and modality connector. Specifically, following the simplifying assumption in Section 4.2, we consider an image containing only three elements: a satellite os, a nucleus ON and a background og, with corresponding position ID sets Is. IN and Ig ([Is +IZN/+ [Tn Nv) in the sequence of length Ny. For an image where the satellite's 's position relative to the nucleus is we define the embeddings for these three objects as V3 (v,) (ET), V = (Mz)jezı and Vg = (vy Jey When we state that we "crase the nucleus by replacing it with the satellite." we replace all embeddings at positions IN in the image embedding V with the mean of the embeddings Is positions Is, which is I Σラ Vš.z Other replacement cases are handled similarly and not detailed here.
32

---

Published as a conference paper at ICLR 2026
Table 4: Experimental results of "crasing objects". The notation "A + B" means erasing object A by replacing it with B. Results show that the representation within single object is sufficient for the determination of the spatial relationship between the satellite and the nucleus.
Qwen2.5-VL-7B
Methods & Models
Left of
Right of
Behind
In front of
Overall
Original
0.959
0.899
0.939
0.840
0.909
Satellite background 0.905
0.848
0.934
0.676
0.841
Nucleus + background
0.949
0.897
0.889
0.782
0.878
Satellite nucleus
0.907
0.850
0.924
0.640
0.830
nucleus + Satellite
0.943
0,886
0.837
0.764
0,858
Satellite + background
0.897
0.850
0.884
0.583
0.804
Nucleus background
In the experiment, the instruction we input to the VLM is: "Is the {obj in front offbehind/to the left of/to the right of the {obj nucleus?? Please choose the best answer from the four options: [In front of. Behind, Left, Right], and reply with only one word. Your answer is: To better observe changes in model performance, we calculate the sum of probabilities for the tokens corresponding to the correct answer in the output of the VLM. For example, if the correct answer is "left," we compute p("left"(z) p("Left"(zz) as the model's confidence the correct answer, where is the model's input. The experimental results are shown in Table 4 Evidently, the performance of the model does not suffer a significant loss when we apply various "erase objects" methods. This validates our initial theoretical analysis and hypothesis.
F.3 EMPIRICAL ANALYSIS ON THE DIRECTION VECTORS
we formally defined the direction vector; here, we detail calculation method. Con- trary to our theoretical assumption, in real image, object does not correspond to just one image patch. As shown in Figure 6a single object can comprise tens to over a hundred image patches. In this case, we define the direction vector v" h-- h Mean VV) Mean (r R), where Mean represents mean pooling, and the definitions of V and V are detailed Appendix F2 In actual calculations, better validate did not fix the representation h of an object o to a single deterministic value Mean(Vy) Instead, randomly sample n (n [V=')) patch embeddings from V and then perform mean pooling.
In Section 4.2
theory and visualize the direction vectors,
In the experiment, for a given spatial relationship r, we perform random sampling within the image patches of the satellite and nucleus compute his and hiy- respectively. We then subtract them obtain the direction vector 8" IR,,. This process repeated 100 times yield 100 direction vectors for that relationship. The number of sampled image patches, n, is determined follows: for objects with fewer than 20 patches, we do not perform sampling; for objects with more than 20 patches, set to half the number of patches for that object.
Furthermore, Section 4.2 we conducted an intervention experiment by using object representa- tions from other spatial relationships intervene on the object representations in the current rela- tionship: [VT]' (1 avV? Mean (V,') iN,, where is the current relationship, r' is another relationship, a is the intervention strength, and IN. is ones vector of length N, (i.e., VV In the experiment, we apply this intervention both the satellite and the nucleus. The direction vector corresponding to the intervened image embedding is:
(ury' Mcan (V', 1',''' l') Mcan(VV.,))
(1 a) Mean( (V ) + Mean(VV' - {(1 - ax) Mean(V + a (iMMen(VV')))
(51)
(1 a) a)Menn (Vr.) Mean(V., ))] + [Mean(Vrr) (lean(Vr')
(1 a)v" +
33

---

Published as a conference paper at ICLR 2026
G THE TOKEN COMPRESSION ALGORITHM BASED ON RUN-LENGTH ENCODING (RLE)
In Section 5.1
wc propose a token compression method based on Run-Length Encoding (RLE). Prior methods primarily include those based on the similarity between visual tokens (Wen et al.
20256
Alvar et al.
2025) and those based on instruction relevance (Chen et al. 2025 Jeddi et al.
2024a Zhang et al., 2024c| Xing et al. 2024). Similarity- based approaches compress visual tokens
according to their similarity. Such methods depend on the chosen similarity metric, and commonly used metries like cosine similarity are not universally applicable. As discussed in Appendix [.. in the deeper layers of a ViT, the representations of visual tokens often contain information from other tokens, and this entanglement becomes increasingly complex with depth. This makes it difficult to compute the similarity between visual tokens using simple metrics in a linear space.
As for methods based on instruction relevance, they measure the correlation between visual and text tokens based on attention scores and compress the visual tokens accordingly. However, this approach has significant practical drawbacks. First, modern models often rely on frameworks like FlashAttention (Dao et al., 2022) for fast inference, in which case the full attention score matrix is inaccessible, making such token compression methods difficult to implement. Second. /Zhan et al. (2024a); Wen et al. (2025a) have pointed out that the existence of the "attention shift" phenomenon introduces bias when these methods measure relevance. More importantly, token compression meth- ods guided by user instructions often only applicable to the simplest question-answw answering datasets (e.g.. VQA), and their utility in practical multi-turn dialogue scenarios is difficult to guarantee.
Therefore, building upon interpretability analysis of VLM object recognition in Section 3.1 propose an cnaructiio-agnostti token compression method. The core of this method is the RLE concept, which reduces the length of the image sequence by compressing the visual embeddings (after the ViT and modality connector) before it enters the language model of the VL.. Specifi- cally, we use the visual decoder proposed in Section 5.1 to decode the visual embeddings into text tokens, thereby obtaining token map of the image. Then, we can read images like texts by treating consecutive and identical text tokens in the sequence as group, which is a "run" in RLE. The cor- responding image embeddings for this group of text tokens are then compressed into an embedding of length one via mean pooling or other methods. The specific algorithm detailed in Algorithm 2
For the training data, we randomly selected the first 148K samples from the GQA training set and added all 22K samples from the TextVQA training set to ensure the visual decoder has a certain degree of OCR capability. From this data, we randomly sampled 1,000 entries as validation set. Here, we discuss the training details of the visual decoder. As shown in Equation 6 we train it via knowledge distillation, with the specific loss definition detailed in Section 5.1 During training,
siphaw0 1emp-2.5
sloba-0 temp-2.5
stphaw tempud.0
sleba-d.. tempud.0
tamp-5.0
4.6
aish2-0.6
temp
temp-6.0
1.0
Val dodtn 1055
Taan 1065
4.4
0.8
4.3
4.1
2000
4000
8005
8003
10000
12000
2003
4200
5000
2000
20000
12003
Staps
Staps
(a) Training loss
(b) Validation loss
Figure 21: The training and validation losses during the training of the visual decoder. Through com- parison, observe that higher temperature leads to lower training and validation losses. When the temperature rises to around 5.0, the loss does not decrease significantly. Note that the validation loss only contains the hard loss to reflect the model's performance in practical applications.
34

---

Published as a conference paper at ICLR 2026
temp-3.3
lamp-5.
Mehaao0 temp=5.0
temp=50
alphanD temp-5.0
temp-5.0
12
1.0
Vo Logs
44
7ith
0.-
2000
4000
$000
9000
10000
12000
2000
4209
60:00
8000
10509
12009
Steps
Steps
(a) Training loss
(h) Validation loss
Figure 22: The training and validation losses during the training of the visual decoder. We observe that a higher loss scaling factor leads lower training loss but higher validation loss. To strike balance between ensuring the student model learns from the teacher's ' knowledge and maintaining its practical performance, we a to 0.8 when training the visual decoder for Qwen2.5-VL-7B.
need to balance the soft loss (KL divergence between the target and original distributions) and the hard loss (cross-entropy loss between the target and original results) by adjusting the loss scaling factor Concurrentyy, for the soft loss, also need to adjust the distillation temperature т to enable the student model to learn richer knowledge from the teacher model. We experimented with different hyperparameter combinations. For the temperature т, tried values such as 2.5, 4.0, 5.0, and 6.0. A higher temperature results in a smoother probability distribution. We found that, with other conditions held constant, the loss gradually decreases the temperature increases, but reaches a lower bound around a temperature of 5.0, as shown by the experimental results on Qwen2.5-VLL77B Figure 21 For the final temperature setting, based on our experience, we use 2.5 for LLaVA-1.5-7B and 5.0 for Qwen2.5-VL-7B.
For the loss scaling factor a, larger value means greater proportion of the soft loss. Compared to I.L 1.5--B, - has a much larger vocabulary (152,064 versus 32,064), which means the supervision signal provided by the limited training data is sparser. Therefore, for Qwen, we appropriately increased to help the model better learn the data distribution. As shown in Figure 22 we tried three values: 0.7, 0.8, and 0.9. During training, a larger a better helps the training loss to decrease. However, results on the validation set show that an overly large leads to mediocre actual performance, as our validation set only considers the cross-entropy loss to reflect the model's 's performance in practical applications. Consequently, for the final setting of a, 0.7 for LLaVA- and 0.8 for Qwen2.5-VV. VLL-BB.
Regarding other training parameters, for both models, we used learning rate warmup strategy (warmup steps=1000) and a cosine annealing schedule. For LLLVA--1.57BB, the learning rate was le-4 with batch size of 16. For Qwen2.5-VL-7BB the learning rate was 2e-4 with a batch size of 32. Additionally, we employed an early stopping strategy, ending the training when the loss on the validation set did not decrease for 5 consecutive evaluations. V.aVA--,5-7B was ultimately trained for 6,000 steps, and Qwen2.5-VL-77 for 9,000 steps.
In practical deployment, we adopted three approaches. The first is compress all visual tokens. Since we observed many meaningless text tokens such as punctuation and spaces in the token map, we speculated that the semantics of these tokens are not very clear and may interfere with the model's prediction. Thus, in the second method, we remove all such tokens. The third method builds on the second, but requires that for a visual token to be considered meaningless, the top two most probable tokens decoded by the visual decoder must both be meaningless punctuation or spaces, rather than only considering the single most probable text token. As for the compression for a run tokens in RLE, we randomly selected one visual embedding the compression result, since we found that the performance of mean pooling is worse than random selection. The reason may be that the operation of mean pooling could hurt the original structure of visual embeddings.
35

---

Published as a conference paper at ICLR 2026
For evaluation, we test on the benchmarks mentioned in Section 5.1 The evaluation for GQA is conducted by repeatedly sampling 1,000 samples from the test set 10 times and calculating the average of the accuracy scores. Furthermore, the truncation rate is defined the average reduction ratio of the image sequence length during token compression, with the final ratio being the average result across all benchmarks.
For inference efficiency, we provide here a theoretical derivation for the compression ratio of our method. Since only add extra visual decoder after the modality connector compared the original model, we ignore the computational cost of the ViT and the modality connector, and only consider the LLM. Furthermore. we focus only on the main overhead the matrix multiplications within the Attention and MLP functions, and the final language head.
ignoring the text embedding, Softmax, activation
Assume the LLM has L layers, the model dimension is d and the intermediate dimension of the MLP is d'. The visual decoder &o has a size of [dd where V is the vocabulary. We consider the input of single image, where the visual sequence length is n, and we ignore the length of the instruction. The computational cost of the original model is:
C = -8nd2 + 4n + J. (FLOPs)
When the compression ratio is the computational cost of our method is:
C' [8n r)d + 2 r)² d 4n - r)dd" + 2nd (FLOPs)
Therefore, measure inference efficiency by C¹ For LLaVA-115-7B, the compression ratio of our method can reach 0.58, corresponding to C
C 43.82%, which is highly efficient.
36

---

Published as a conference paper at ICLR 2026
Algorithm 2 Token compression based on RLE (during the pre-filling stage)
Input: an image X an instruction X", VLM consisting of an visual encoder fi(-), modality connector fc(+), and language model fr(:), pretrained visual decoder yo.
2: -/ Get input embeddings and the text tokens for visual tokens
3: Get the visual embeddings fc(fr(XV) (,,, UNy) E NvD and text embeddings
T (41,...,
IR:rxD;
4: Get the output of the visual decoder: logits p0((1, UNV)] NvVVV
5: Get the Top-1 text tokens: WY = (w), W WNv.1) arg max (Softmaz(logita) E RR
WEV
6: Get the Top-2 text tokens W (Wi,2),.. WKy, 2) probability, where w//2 = arg max (Softmar (Softmaz(logits (logits(i::)) ww.)
RN according to the second highest
7: // - Stage I: Run-Length Encoding on all tokens
8: Define a of meaningless tokens M (e.g., punctuation, whitespaces).
9: Initialize a list of runs R < [], start index <- 1, run length / + 1.
for 2 to Nv do
11:
if w i 1 w then
Extend the current run
13:
else
14:
Append (s, 1, w. w/-12) to R. sti, f- 1.
>> Save: (start, length, Topl token, Top2 token) Start
15:
16:
end
end for
18:
Append the last run (s, w 11 w Ny to R.
Save the final run
19: // - Stage 2: Filtering and Compression (Choose one method)
Initialize the compressed visual embeddings list V' + []. Let Compress(U,, ... Us+1-1) be
function that returns single embedding (e.g., via random selection or mean pooling).
22: Method 1: Perform RLE-based compression on all visual tokens.
23: for each run (8, wi, W2) A
24: + Compress (Us Ugy Ve41-1).
25: Append V"
26: end for
27:
28: Method 2: Based on Method 1, but delete meaningless tokens (Top-1 based).
29: for each run (, Wi, W2) in R do
30: if' wi & M then
31:
- ---Coorss(s,....... 1s+1-1).
Append V/
33: end if
end for
35:
36: Method 3: Based on Method 2, but use Top-2 tokens for meaninglessness check.
37: for each run (s, 1, w)) in R do
38: if not (wi E M and W2 E M) then
- Comppessssss....... Us+1-1). 40: Append to V"
39:
41: end if
42:
end for
44: Output: The input of the language model: 4 = (uf, U'yr ti.... txr) B(M++N))x (M Ny), which is the combination of the compressed visual embeddings 1 (v/...,, u'r) and the embeddings X
37

---

Published as a conference paper at ICLR 2026
H ROPE SCALING
In Section 4.2, we identify a potential deficiency in the spatial perception process of visual encoders based on 2D RoPE: the magnitude of the term responsible for distinguishing spatial relationships is often smaller than other terms. For a VLM, isolating this position-related representation from the multiple components of the image representation may be a non-trivial challenge. Furthermore, as the dimension group index i of RoPE increases, its corresponding rotation angle 8, decays exponentially, as shown in Figure 24a leading to reduced sensitivity to positional changes.
To verify that the magnitude of the component responsible for representing spatial information is small relative other components, we decompose the attention score calculation into two parts based the X and Y axes. Assume the image sequence has length n and the dimension of an attention head is d, where dimensions I to carry positional information for the X-axis, and mensions d + 1 to d carry positional information for the Y -axis. The matrix multiplication of the query and key can be represented in Equation 52
Q1
Lid
QK
:
On1
qud
Kid
kind/
111
kn1
:
Q11
0ig
(1(g+1)
qid
kid
kin
k((+1)
knn $ +1
(52)
9nl
n
Unth+ +1)
(nd)
:
kid
knd
Kx = =(xx Qn) = =Q*K* +Q"K"
=M*+M²
Here, Q* and Q correspond to the parts of the query that carry positional information for the X-axis and Y Yaxiss respectively. The same applies to the key. We then concatenate the dot product result from the X-axis dimensions, M and the result from the Y-axis dimensions, MM to obtain a dot product matrix (X,Y) of shape R N X2N We pass (X,) through a Softmax operation to get the "attention score matrix" (XXY)) By separating the parts corresponding to the and Y axes, obtain the "attention scores" contributed by the X-axis dimensions, A and the Yaxxis dimensions, A'. The "attention scores" we calculate reflect the relative contributions of the X-axis and Y -ais dimensions of the image representation to the final attention scores.
For two objects in an image, a satellite og and a nucleus ON we focus on the "attention scores" from the satellite to the nucleus. Assuming the position IDs for these two objects in the sequence of length N are Is and IN, we calculate the "attention scores" for the X-axis and -xis parts from the satellite to the nucleus as follows:
Σ 1=1 E
A/
n2 [IN| n1 [s]
(53)
JEISJJEN
717221
Y -
Σ カ= E A
n1 [sl, n2 [IN|
(54)
71
In Equation 53 and 54 nu is the number of attention heads, and A A.X and h.Y are the "attention scores" from query qi to key ky at the l-th attention, corresponding to the dimensions from the -axiis and Y -axis, respectively. In practice, we conduct experiments using the What's Up subset B. We perform this calculation for each layer of the ViT on all samples to obtain the layer-wise X.1 "attention scores" aS-- * >K as-NN. and as-.N $ >K ds-.N. (for / E [1, Lv),, where K is the number of samples in the dataset. Finally, for each ViT layer I, we normlize a
x.1
Y.!
Y!
Y and N [0, 1). Results QL--2-VIL-BB are shown in Figure 23 In Figure 23a Baaadd 23b the
38

---

Published as a conference paper at ICLR 2026
(a) Left
(b) Right
(c) front of
(d) Behind
Figure 23: The contribution to attention scores from the dimensions in the X- and Y-axis.
8
0.4
0.2
0.2
0.0
Demension
(a)
(b)
Figure 24: (a)The decay of RoPE 0. (b)The compensation for the decay of RoPE 0 via RoPE scaling.
difference in spatial relationships between the two objects lies in the X-axis, while the contribution to the final attention scores from the X-axis is smaller than that of the Yaxxis. Similar phenomenon can be found in Figure 23c and
23d
Therefore, we propose RoPE scaling. The core idea to compensate for the decay of 0g by ampli- fying the distance information in the low-frequency dimensions (corresponding to larger dimension group indices i). In other words, we want all dimensions to effectively represent positional informa- tion. In our experiments, we first conducted training-free experiments, in which we directly changed the original 2D RoPE to the RoPE scaling form in Equation 7 The training-fiee free results shown in Table 2 are the best results obtained by tuning the hyperparameters and for each benchmark.
To more accurately reflect the effect of RoPE scaling, we also fine-tuned both the original model and the model with RoPE scaling applied. The fine-tuning data consists of 60K samples related to spatial reasoning, randomly sampled from the GQA training set. We require that either the question or the answer should contain at least one word in the keyword list for spatial reasoning: ["leftt", "right", "top". "bottom", "in front of", "behind", "above". "below", "next to",, "beside", "between", "on top of", "under", "over"]. During fine-tuning, all training settings, including batch size, learning rate, and training data, were kept identical to ensure a fair comparison, with the only exception being the RoPE scaling parameters. Specifically, for Qwen2-VL.. VL-2B, we performed full fine-tuning with a batch size of 32 and a learning rate of 3e-6. For Qwen2-VL-7B, we used LoRA fine-tuning with batch size of 32, a learning rate of 3e-6, and a LoRA rank and alpha of 16 and 32, respectively. All models were trained for one epoch. For the models using RoPE scaling, we searched for the hyperparameters a and p. and the best results are shown in Table 2 The corresponding a and p are 99 and 8 for Qwen2-VL-
and 49 and 8 for Qwen2- -L-77B
We evaluated the fine-tuned models on the evaluation sets described in Section 5.2 and found that RoPE scaling demonstrates excellent spatial reasoning performance across multiple test sets, as shown in Table To ensure the practical utility of RoPE scaling, we also evaluated the general ca- pabilities on MMBench (English). The original performance (accuracy) of Qwen2-VL-7B on MM- Bench is 84.82%. We found that for the fine-tuned Qwen2-VL-7B, the accuracy of the model with RoPE scaling was 85.56%, which is even higher than the model without RoPE scaling (85.45%). Therefore, RoPE scaling, as a method for enhancing the spatial awareness of VLMs, can maintain the general capabilities of the model without degradation.
39


Reviews:
[{"review_id": "xDP8Ii7lM9", "reviewer": "Reviewer_fYWu", "summary": "First, they investigate _how_ the vision encoder of the VLM encodes what an object is, and they do so by skipping ViT layers and using logit lens on the LLM to see what the LLM processes. They show that early layers encode high-level attributes and later layers encode specific repesentations of the objects. Then, they study at how VLMs perceive spatial relation through a theoretical and empirical analysis. Lastly, they propose two applications based on their findings: token compression and enhancing spatial reasoning.", "questions": "* What was the method used to come up with the keyword set? If it was what I mentioned, is there some more principled way to do it?\n* For the token compression application, the tokens embeddings only become text-like in the later layers of a LM. I assume you are compressing the tokens at the input to the LM (i.e. replacing or truncating the visual embeddings.) How are you compressing the tokens, or what are you replacing the visual embeddings with? I understand it to be text tokens from the logit lens. If so, have you tried merely replacing all visual embeddings with logit lens tokens? (I'm not sure if this is what \"original decoding\" refers to, or whether original decoding is referring to merely using the original visual embeddings)", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "* They use logit lens on the LLM to study how the Vision Encoder works: this is a particularly insightful idea that I had not previously considered. And it seems to work extremely well, especially with further validation from POPE polling.\n* It is timely to study how VLMs do spatial perception as they are becoming better at it, and the paper does a sound, comprehensive study of it. While the theoretical study is limited to only two objects, I find this to be a necessary constraint.", "weaknesses": "* The segmentation map keywords seem to be chosen by the authors. This may mean that the results could be due to human-confirmation bias here, if the authors first saw the logit lens results then came up with the keywords.\n* The applications and results don't seem very strong, but nevertheless serve as further empirical validation of their findings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cmHPXCsUKh", "reviewer": "Reviewer_ywbf", "summary": "This paper provides a deep interpretability analysis of Vision-Language Models (VLMs) through the lens of the brain’s dual-stream hypothesis, examining how models achieve both object recognition (“what”) and spatial perception (“where”). For the “what” pathway, logit lens decoding reveals a two-stage process in token representations: early detection of low-level features and later semantic refinement. For the “where” pathway, theoretical and empirical investigations of 2D RoPE demonstrate that spatial relations are geometrically encoded in attention maps. Leveraging these findings, the authors introduce (1) an instruction-agnostic token compression algorithm using run-length encoding, and (2) RoPE scaling to enhance positional signals and spatial reasoning. Experiments on LLaVA and Qwen2.5-VL validate both the insights and practical effectiveness of the proposed methods.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper offers a systematic and mechanistic analysis of visual processing in VLMs. The application of the logit lens to visual tokens is particularly effective, converting otherwise opaque embeddings into interpretable token maps and enabling fine-grained examination of object recognition dynamics.\n\n2. The paper provides a rigorous theoretical analysis of how 2D RoPE encodes spatial relationships, with empirical validation via PCA, object erasure, and intervention experiments. The findings on collinearity of “left/right” and orthogonality of “left/behind” directions convincingly illustrate the geometric structure in the learned representations.\n\n3. The paper features extensive ablation studies, comprehensive training protocols, and thoughtful handling of edge cases, such as top-1 versus top-2 token filtering for compression. Evaluation across multiple models (LLaVA, Qwen2.5-VL, InternVL) and datasets (GQA, POPE, What’s Up) further enhances the generalizability of the findings.", "weaknesses": "1. The study's focus on four basic directional relationships (“left,” “right,” “front,” “behind”) excludes more complex spatial configurations (e.g., “top-left,” “surrounding,” “partially occluded”), thereby limiting the scope of the “where” pathway analysis. \n\n2. The logit lens approach assumes that visual representations can be linearly decoded into semantic tokens using the language model’s unembedding matrix. However, this assumption may not hold for earlier ViT layers, whose features are not yet aligned with the final modality connector, as noted in Appendix A. Employing per-layer projection heads could offer a more robust alternative.\n\n3. Although the compression method is instruction-agnostic, it necessitates training a visual decoder through knowledge distillation, which introduces additional engineering overhead. Furthermore, the assertion that \"random selection outperforms mean pooling\" is made without theoretical justification, indicating a need for more thorough investigation in future work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "h1g8jkAETs", "reviewer": "Reviewer_RfD1", "summary": "This paper investigates how Vision-Language Models (VLMs) process visual information by decomposing their mechanisms into object recognition and spatial perception, inspired by the dual-stream hypothesis of human vision. The study reveals that VLMs recognize image content in two stages—progressing from low-level attribute detection to high-level semantic understanding—and uncovers the geometric structure underlying their positional  representation. Building on these insights, the authors propose a token compression algorithm and a RoPE scaling technique to enhance decoding efficiency and spatial reasoning. Overall, the work deepens understanding of VLM internals and offers principled guidance for future model design.", "questions": "Please refer to the Weaknesses part.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* This paper conducted comprehensive theoretical and experimental analyses. \n* The paper connects theoretical analysis with applications. Based on findings on the visual processing and spatial perception characteristics of VLMs, this paper introduces an instruction-agnostic token compression method, which reduces image sequence length during decoding, and RoPE scaling, which enhances the spatial reasoning capabilities of VLMs.", "weaknesses": "* The paper does not quantitatively compare the proposed token compression method with other existing compression techniques, such as [1] and [2], in terms of inference efficiency and training cost. The proposed method also requires additional training of the visual decoder, which may limit its application scenarios.\n* In Table 1, methods 2 and 3 exhibit a more significant performance decline on TextVQA than on other tasks. What accounts for this difference? \n* There are several instances in the paper where LaTeX quotation marks are incorrectly formatted. For example, ”In which direction is A relative to B?” in line 316.\n\n[1] Lin, Zhihang, et al. \"Boosting multimodal large language models with visual tokens withdrawal for rapid inference.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 5. 2025.\n\n[2] Zhu, Yuke, et al. \"Focusllava: A coarse-to-fine approach for efficient and effective visual token compression.\" arXiv preprint arXiv:2411.14228 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SDC9sjJGhK", "reviewer": "Reviewer_fdDb", "summary": "The paper examines how vision-language models (VLMs) encode and reason with visual information within a one-dimensional token sequence. It addresses two core questions: how VLMs associate positionally discontinuous tokens representing the same object to identify its category (\"what\"), and how they infer 2D spatial relationships between objects (\"where\"). The authors use logit lens and visualizations to uncover a two-stage visual processing pattern. They theoretically and empirically analyze how RoPE-based encoders represent spatial relationships, revealing their underlying geometric structure. Building on these insights, the paper proposes two practical methods: a token compression approach that shortens image sequences with minimal performance loss, and a RoPE scaling technique that enhances spatial reasoning while maintaining overall model performance.", "questions": "My only requests/questions on top of the previously mentioned points are about newer architectures and higher-dimensional RoPEs:\n\n* Will this analysis transfer to models that use DeepStack [2], like Qwen3? \n\n* Can you say something similar about 4D RoPEs in videos (i.e., improving the \"where\" pathway for video frames, or extending it to a \"when\" pathway)?\n\n[2] Meng et al., DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMs, NeurIPS 2024", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "* The analysis section is thorough and compelling, offering valuable insights into how visual encoders and visual tokens are processed within VLMs. This contributes to a deeper understanding of the inner mechanisms driving object and spatial reasoning.\n\n* The proposed applications are well grounded in the paper’s analytical findings and serve as effective validations of the authors’ observations, demonstrating both conceptual soundness and practical relevance.\n\n* The theoretical and empirical analysis of RoPE-based spatial processing highlights a real limitation in the current VLM architecture. By identifying and addressing this limitation, the work provides a meaningful direction for future research and improvements in spatial reasoning within VLMs.", "weaknesses": "__Paper structure:__\n\nThe paper’s structure could be improved. Much of the core analysis is deferred to the appendix, making it difficult to follow the main findings from the main text alone. At the same time, several sections—particularly the Related Work—contain substantial repetition of content already covered in the Introduction. Streamlining the exposition and integrating key analyses into the main body would make the paper more cohesive and readable.\n\n__Comparison to prior work__:\n\nThe paper lacks a sufficient comparison to other recent token compression methods, such as [1]. Since prior methods already identify salient tokens and achieve significant token reduction, it is unclear whether (1) the proposed method identifies similar or distinct subsets of tokens, and (2) whether it provides any advantage in compression efficiency or downstream task performance. A quantitative or qualitative comparison with these approaches would strengthen the paper’s claims.\n\n__Training-free RoPE scaling experiment:__\n\nIn the training-free RoPE scaling experiment, the authors report the best results achieved by tuning the hyperparameters \\alpha and p. It is unclear whether these parameters were optimized using the test set. If so, this constitutes data leakage and invalidates the experiment. The authors should clarify the procedure used to select these parameters (e.g., validation split, held-out set) and ensure that the evaluation remains unbiased.\n\n\n[1] Kaduri et al., What's in the Image? A Deep-Dive into the Vision of Vision Language Models, CVPR 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)