How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decoding
Authors: Zhuoran Yu
Yong Jae Lee
Venue: COLM 2025 (2025)
Keywords: Multimodal LLM
Interpretability
TL;DR: We propose a probing framework to analyze how multimodal LLMs process visual and textual inputs across layers

Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong performance across a wide range of vision-language tasks, yet their internal processing dynamics remain underexplored. In this work, we introduce a probing framework to systematically analyze how MLLMs process visual and textual inputs across layers. We train linear classifiers to predict fine-grained visual categories (e.g., dog breeds) from token embeddings extracted at each layer, using a standardized anchor question. To uncover the functional roles of different layers, we evaluate these probes under three types of controlled prompt variations: (1) lexical variants that test sensitivity to surface-level changes, (2) semantic negation variants that flip the expected answer by modifying the visual concept in the prompt, and (3) output format variants that preserve reasoning but alter the answer format. Applying our framework to LLaVA-1.5, LLaVA-Next-LLaMA-3, and Qwen2-VL, we identify a consistent stage-wise structure in which early layers perform visual grounding, middle layers support lexical integration and semantic reasoning, and final layers prepare task-specific outputs. We further show that while the overall stage-wise structure remains stable across variations in visual tokenization, instruction tuning data, and pretraining corpus, the specific layer allocation to each stage shifts notably with changes in the base LLM architecture. Our findings provide a unified perspective on the layer-wise organization of MLLMs and offer a lightweight, model-agnostic approach for analyzing multimodal representation dynamics.

Published as a conference paper at COLM 2025
How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decoding
Zhuoran Yu University of Wisconsin-Madison zhuoran. Mnuoran.y@@vicc.ee yu@wisc.
Yong Jae Lee
University of yongjaelee@css. wisc.edu
Abstract
Multimodal Large Language Models (MLLMs) have demonstrated strong performance across wide range of vision-language tasks, yet their internal processing dynamics remain underexplored. In this work, we introduce a probing framework to systematically analyze how MLLMs process vi- sual and textual inputs across layers. We train linear classifiers to predict fine-grained visual categories (e.g,, dog breeds) from token embeddings extracted at each layer, using a standardized anchor question. To uncover the functional roles of different layers, we evaluate these probes under three types of controlled prompt variations: (1) lexical variants that test sensitivity to surface-level changes, (2) semantic negation variants that flip the expected answer by modifying the visual concept in the prompt, and (3) output format variants that preserve reasoning but alter the answer format. Applying our framework LLaV LLaVA-Nex--LLaMA- and Qwen2-VI, we identify consistent stage-wise structure in which early layers perform visual grounding, middle layers support lexical in- tegration and semantic reasoning, and final layers prepare task-specific outputs. We further show that while the overall stage-wise structure re- mains stable across variations in visual tokenization, instruction tuning data, and pretraining corpus, the specific layer allocation to each stage shifts notably with changes in the base LLM architecture. Our findings provide a unified perspective on the layer-wise organization of MLLMs and offer lightweight, model-agnostic approach for analyzing multimodal representation dynamics.
1 Introduction
Multimodal Large Language Models (MLLMs) (Liu et al., 2024b; Bai et al., 2023; Wang et al., 2024; Zhu et al., 2023; Li et al., 2023) have recently demonstrated remarkable capabilities in integrating visual and textual inputs to perform tasks such image captioning (Chen et al., 2015; Agrawal et al., 2019), visual question answering (Goyal et al., 2017; Marino et al., 2019; Hudson & Manning, 2019), and open-ended instruction following (Hendrycks al., 2021; Zhong et al., 2023). While these models exhibit strong performance across benchmarks, their internal processing
paaticcs-prtticlaari how visual and linguistic information interact across layers- are not yet fully understood. Understanding this internal structure important not only for interpretability but also for gaining deeper insights into model behavior and limitations.
Several recent studies have begun to explore the internal mechanisms of MLLLMs using tools such causal tracing (Basu et al., 2024; Neo et al., 2024), logit lens (Huo et al., 2024), and concept-level perturbation (Golovanevsky et al., 2024). These methods provide valuable insights into token-level attribution, attention behavior, and concept encoding. However, they often require custom instrumentation or focus on isolated components, making it difficult to obtain high-level understanding of how information flows through the model.
In this work, we introduce a probing-basee based framework that examines how prompt variations affect layer-wise representations during multimodal inference. We treat each transformer
1

---

Published as a conference paper at COLM 2025
Grounding
Lexical
Reasoning
Decoding
Qwen2-VL
LLaVA-1,5
5
13
17
21
25
29
Layer
Figure 1: Layer-wise stage comparison between LLaVA-1.5 and Qwen2-VL. Each colored segment corresponds to a distinct functional phase identified by our probing analysis: Grounding (visual input encoding), Lexical Integration (alignment between image and prompt phrasing), Semantic Reasoning (internal answer formulation), and Decoding (output generation). While both models exhibit the same stage-wise structure, Qwen2- VL allocates fewer layers to grounding and more to reasoning, reflecting architectural differences in how multimodal understanding is distributed across layers.
layer / in the language model feature extractor, and train linear classifier f()) Rd
[I,...,N] .] to predict visual class labels from the final token representation h E Rd. Training is conducted on set of N image classes paired with a fixed textual prompt-referred to as the anchor prompt-which is held constant across all training instances.
At test time, we evaluate the same probes classes, each paired with systematically perturbed versions of the anchor prompt. Each prompt variant is designed to target a specific aspect of the input-such as lexical form, semantic content, or output format
held-out set of images from the same N
preserving the correct visual label. The core assumption is that different layers of the transformer of computation. Therefore, the extent to which probe accuracy is affected by each variant reflects the sensitivity of given layer to that type of perturbation, revealing its functional role. By aggregating these effects across prompt types and layers, we derive a structured view of how MLLMs process and integrate multimodal information.
specialized for different types
We design each variant to target a distinct stage of the model's internal computation, en- abling us localize where different processing stages occur. Lexical variants test where the model begins aligning visual information with specific prompt phrasing- involved in grounding should be sensitive even small wording changes. Semantic negation helps identify where the model begins to commit to an answer, as representations at these layers should reflect changes in the predicted outcome. However, because negation also changes the underlying visual concept being reasoned about, we introduce output format variants to decouple reasoning from decoding. These variants keep the visual input and inter- pretation fixed while altering how the answer is expected to be expressed (e.g., "yes or no" vs. "1 or 0"), allowing us to test whether the model's ''s internal representations encode the decision itself or just the tokens used to communicate it. Together, these controlled variations enable a fine-gra grainedd,,d, layer-wise map of how multimodal information flows and transforms within the model.
Applying our framework to LLaVA-155 (Liu et al., 2023), we uncover a four-stage processing hierarchy: early layers (1-4) perform visual grounding, middle layers (5-13) integrate lexical information with visual features, and deeper layers transition into answer decoding. Notably, the output format variant further subdivides the middle-to-late stages, revealing that layers 12-15 are responsible for semantic reasoning -encoiinggg the model's internal decision-while layers beyond 15 shift toward formatting that decision into specific output tokens. We find this structure to be preserved in ALaVA-Next-LLMMA- (Liu et al., 2024a), despite differences visual tokenization, instruction tuning data, and the pretraining corpus of the base LLM suggesiing these factors do not fundamentally alter processing dynamics. In contrast, Qwen2-VL (Wang et al., 2024), which is built on a different LLM architecture (i.e, Qwen-LLM (Bai et al., 2023)), exhibits the same stage-wise structure but reallocates layer depth across stages, using fewer layers for grounding and more for reasoning. This highlights the critical role of L.L architecture in determining how and where multimodal
2

---

Published as a conference paper at COLM 2025
Linear Prob Training
Projector
Decoder Layer
Decoder Layer
Decoder Layer k
Vision Encoder
Training image
Anchor Question
Tokenizer
Linear
CE Loss
Linear Prob Testing
Projector
Decoder Layer 1
Decoder Layer
Decoder Layer k
Vision Encoder
Testing image
Prompt Variant
Tokenizer
Linear
Accuracy
Figure 2: Linear probing at decoder layer k. Training (top): We pass training images with an anchor question through the frozen multimodal LLM, extract the sequence's last-token representation at layer k, and fit a linear probe on the ground-truth labels (in this example, for dog breed classification). Testing (bottom): we keep the probe fixed and evaluate it on last-token features from testing images under prompt variants (e.g., lexical rewrites, semantic negation), reporting classification accuracy. We repeat this for every decoder layer to obtain the layer-wise profile used in our analysis.
integration and decision formulation unfold within the model. This comparison is visualized in Figure 1, which highlights how different models distribute computational stages across depth.
2 Methodology
Our study investigates how multimodal large language models process information within the LLM across different layers by analyzing their token embeddings under varying prompt conditions. We employ linear probing layer-wise embeddings to understand how representations evolve when the prompt question changes with constant visual inputs. An illustration of our probing framework can be found in Figure 2.
2.1 Experiment Setup
Dataset and Anchor Question We use subset of ImageNet containing exclusively dog breed classes with the class-agnostic anchor question: "Does this image show an animal? The answer must be always yes or no." " Before extracting embeddings, we first verify model compliance by running the anchor prompt through the MLLM for all images. We discard any images that do not receive confident "yes" response. This filtering step removes cases where the model fails to follow instructions or misclassifies the image content, ensuring the model properly attends to the visual content and strictly maintains the "always yes" condition. The filtering process is applied when conducting evaluation with various prompt questions (details in Section 2.2), therefore, ImageNet training split to the validation set. This allows us to retain enough examples even after discarding cases where the model fails to produce the expected answer.
move 300 images per class from the original
We choose fine-grained visual classes (i.e., different dog breeds) rather than coarse categories to ensure that the classification task remains non-trivial. If the visual classes were too

---

Published as a conference paper at COLM 2025
coarse (e.g. "animal" vs. "non-animal"), the decision boundaries would be too robust to reveal small changes in internal representations caused by prompt variation, making probe sensitivity difficult to interpret. Additionally, we single anchor prompt shared across all training instances to avoid introducing prompt-textual cues that correlate with class labels. Using class-specific prompts would allow probes to memorize prompt features rather than genuinely reflect changes in how the model integrates visual and textual information.
Embedding Extraction For the filtered dataset, we extract embeddings using each model's default prompt template. For example, VL-15 uses the following structure:
USER: <image>
(Does this image show animal? The answer must be always yes no.)
ASSISTANT:
Other models employ their own standardized formats. For each model and image x, we
collect the embedding ( of the last token at layer I, which captures the model-specific multimodal representation combining both visual and textual information each processing
stage.
Probing Model Training We train linear classifiers f()) at each layer to predict dog breeds E f1,..., from embeddings e miniiiizin the cross-entropy loss:
(
1
Σ (x
log
el) (ex (1)
B
where B denotes a batch of training samples.
We train each probing model using the Adam optimizer with a learning rate of 0.001 and batch size of 16. Training is performed independently for each layer until convergence.
2.2 Evaluation Protocol
The core objective of our evaluation is to understand how different layers of multimodal LLMs encode and process information, using carefully designed prompt variations. After training probing models
embeddings generated from the standardized anchor question, we systematically evaluate them using three categories modified prompts, each targeting distinct aspect of the model's internal computation.
Lexical variants introduce minor surface-level changes (e.g. "image" - "picture") without altering the semantic meaning or expected answer. They help identify where the model begins aligning visual features to textual input: layers involved in this alignment should exhibit changes in representation- phrasing shifts. These variants localize the onset of visual- risual-linguistc integration, showing where prompt wording begins to influence image-conditioned representations.
thus drops in probing accuracy- even for subtle
Semantic negation variants alter the core visual concept in the prompt (e.g, "animal" + "plane"), flipping the expected answer from "yes" to "no" while preserving syntactic structure. These variants help identify layers where the model begins encoding its internal decision: if representations are entangled with the answer label, they should diverge from the anchor despite identical images. A key challenge, however, is that semantic negation also changes the visual concept the model must reason about-introducch intrruucini variation in the reasoning path and making it harder to isolate where answer commitment occurs.
To address this, we introduce the output format variant, which the question remains fixed but the expected answer is altered in form only-for example, "yes/no /no" becomes "1/0." Since the model must arrive at the same semantic answer regardless of the output token, this variant allows us to decouple reasoning from decoding. If model has already completed its reasoning at a given layer, the representation should remain consistent across output formats, and any changes should appear only in later decoding layers. Thus, this
4

---

Published as a conference paper at COLM 2025
variant isolates the transition point between reasoning about the input and preparing to generate the final answer.
Together, these variants allow to characterize the full trajectory of multimodal process- from ing-from early visual-text alignment, to semantic reasoning, to final answer generation. For each variant type, we evaluate probing accuracy layer-by-layer and compare it to the anchor baseline. We include only examples where the model produces the expected answer to both anchor and variant prompts, ensuring that observed differences reflect changes in internal representations rather than model failure.
Results
We begin by analyzing layer-wise processing patterns in LLaVA-1.5 (Liu al., 2023) as representative open-source multimodal LLM, and then extend this analysis to other popular models. All models evaluated in this section are of relatively modest scale (7B-8B parameters) due to limited compute resources.
3.1 Probing Analysis for VLaVA-1.5
Our probing framework trains layer-wise linear classifiers f()) on last-token embeddings extracted using a standardized anchor question: Does this image show an animal? The answer must be always yes or no. (see Section 2). LLaVA-1.5 (Liu et al., 2023) uses Vicuna-7B (Chiang et al., 2023) as its base LLM, which consists of 32 transformer layers with an embedding dimension of 4096. Accordingly, we train 32 separate probing models, one for each layer, using the corresponding embeddings.
We first evaluate these probing models using embeddings generated from two categories of prompt variations: Lexical Variants, which modify the surface form of the anchor question while preserving its meaning, and Semantic Variants, which introduce semantic negation by altering the visual concept referenced in the prompt.
For lexical variants, we implement two specific edits: (1) substituting image-referencing nouns (e.g,, image + picture), and (2) varying verbs (e.g., show + feature). For semantic variants, we replace the semantic keyword animal with an unrelated concept, e-g,, plane, thereby changing the expected answer from yes to no. All evaluations are performed on embeddings from the same set of test images, after filtering to ensure that the model produces the expected responses for both anchor and variant prompts.
Early and Middle Layers: Grounding and Integration As shown in Figure 3, even at very early layers (1-2), subtle but consistent increase in accuracy is observed for the anchor question alone (i.e., without prompt variation), suggesting the emergence of initial visual grounding. When lexical and semantic variants are introduced at test time, early layers (1-4) continue to exhibit relatively stable accuracy, indicating that these layers primarily encode visual information into the last-token embeddings without yet reflecting changes in the textual prompt. In contrast, middle layers (5-13) show substantial sensitivity to lexical changes; even minor modifications, such as altering the verb (show - feature), lead to an accuracy drop of approximately 40% at layer 13. These observations indicate a clear internal shift in VA-1:51 while early layers (1-4) focus on visual grounding regardless of the prompt content, the alignment and integration of textual and visual information predominantly occurs in the middle layers (5-13).
Late Layers: Reasoning vs. Answer Decoding A critical behavioral transition occurs around layer 14, where the accuracy trajectories of lexical and semantic variants begin to diverge significantly. Specifically, accuracy for lexical variants recovers in deeper layers and becomes closely aligned with the anchor prompt, while accuracy for semantic variants remains consistently low. This contrast reflects a key difference in how the two variant types are constructed: lexical variants preserve the expected answer (yes), whereas semantic negation variants alter both the reasoning target (e.g., "animal" + "plane") and the expected answer (no). As a result, the persistent accuracy drop for semantic variants likely reflects not only sensitivity to conceptual shift but also commitment to different output. This

---

Published as a conference paper at COLM 2025
Lexical Variant
Semantic Negation Variant
1.0
0.9
0.8
Accuracy
0.8
Accuracy
0.7
0.4
0.6
Anchor Question Image picture show >2 feature
0.2
0.5
animal plane
13
21
29
32
17
29
32
Layer
Layer
Figure 3: Probing accuracy across layers for lexical and semantic negation variants. Left: Lexical variants preserve the expected answer while modifying surface phrasing (e.g,, image + picture, show + feature). Accuracy drops sharply in middle layers, revealing where visual-textual alignment occurs, but recovers deeper layers the model converges on the correct decision. Right: Semantic negation introduces shift in both visual reasoning and expected answer (e-g,, animal plane), causing a persistent accuracy drop deeper layers, indicating divergence in the model's internal decision process.
makes interpretation more challenging- probing accuracy alone cannot disentangle whether deeper layers reflect internal decision-making the formatting of that decision into specific answer tokens. To resolve this ambiguity, we introduce output format variants in the next section, which modify only the expression of the answer while keeping the reasoning path and expected outcome fixed.
3.2 Decoupling Reasoning Paths from Answer Decoding
As observed in Section 3.1, deeper layers (around layer 14 and beyond) begin diverge in how they represent prompts with different semantic content and expected answers. However, in the case of se- mantic variants (e.g,, changing animal to plane), this divergence reflects two coupled changes: a shift in the model's internal reasoning target and a shift in the expected output. Specifically, switching from a positive (yes) prompt to negative (no) one requires the model to realign visual features with a different semantic category, fundamentally altering its reason- ing trajectory. As a result, the observed decline probing accuracy for semantic variants cannot be attributed solely to output decoding but may also reflect deeper semantic reprocessing. To disentangle these factors, we introduce output format variants, which modify only the expected answer format (e.g., yes/no + 1/0) while keeping the reasoning path and semantic content identical to the anchor question. This setup enables us localize the transition point between semantic reasoning and output formatting: since the visual input and question remain fixed, any observed differences in internal representations must stem from how the model prepares to generate the answer in a specific output format, rather than from differences in the reasoning process itself.
Output Format Variant
0.8
Accuracy
0.6
0.2
3.2
3.
Figure Probing accuracy when al- tering only the answer output for- mat from textual (yes/no) to numeri- cal (1/0).
Figure 4 shows the probing accuracy across layers under this controlled evaluation. Early layers (before layer 12) show accuracy patterns similar to those under lexical variations, reflecting sensitívity to minor wording changes in the prompt. Notably, accuracy under the output format variant rises from layers 12 to 15 before declining in deeper layers. We hypothesize that this trend highlights separation between semantic reasoning and output

---

Published as a conference paper at COLM 2025
formatting: the initial increase occurs because the output variant and the anchor question share the same reasoning path, allowing the probe to generalize. The subsequent decline reflects divergence in answer format (yes/no vs. 1/0), deeper layers increasingly encode the specific form of the model's final response. These results help localize the transition from reasoning to decoding within the model's hierarchy.
Summary. Our layer-wise probing analysis reveals four distinct phases in LLaVA-1.3's multimodal processing pipeline:
Visual Grounding (Layers 1-4): Early layers primarily encode visual input, produc- ing representations that are largely invariant to both lexical and semantic prompt variations. This indicates that the model has not yet begun integrating language context into its processing.
Lexical Integration (Layers 5-11): this stage, representations become sensitive to surface-level phrasing changes in the prompt. This suggests that the model is aligning visual features with linguistic cues, marking the onset of multimodal interaction.
Semantic Reasoning (Layers 12-15): These upper-middle layers reflect the model's internal decision-making process. Representations here diverge depending on the semantic content of the prompt, indicating that the model has committed to specific interpretation of the input.
Answer Formatting (Layers 16+): Late layers focus on shaping the final output. Representations are increasingly influenced by the desired response format rather than the input content itself, as confirmed by output format variant experiments.
3.3 Determinants of Multimodal Processing Mechanisms
Building our layer-wise probing analysis of LLa (Section 3.1), we now investigate which factors govern the internal processing behavior of multimodal LLMs. Contemporary MLLMs differ across three key dimensions: (1) visual
LLLv----1 uses fixed 576-token representation, while LLaVA-News (Liu et al., 2024a) increases this by 4x through multi-resolution chunking, and Qwen2-VL (Wang et al., 2024) adopts dynaaii-rreoouutinn encoding (Dehghani et al., 2023); (2) instruction tuning data, which varies in both scale and content across models; and (3) base LLM characleristics, including both architectural design (e.g,, LLaMA-variations (Touvron et al., 2023b; Chiang et al., 2023) vs. Qwen (Bai et al., 2023)) and pretraining corpus (e.g., Vicuna (Chiang al., 2023) vs. LLaMA-3 (Dubey al., 2024)). (Liu et al., 2023),
(Liu et al., 2024a), and Qwen2-VL-7B (Wang et al., 2024), we isolate which aspects of the processing hierarchy remain stable and which are sensitive to architectural or training choices.
By comparing
LVA--Net-LLLMA--
3.3.1 Processing Dynamics Are Invariant Tokenization and Training Data
To test whether components such as visual tokenization, instruction tuning data, or the base LLM's pretraining corpus fundamentally alter the processing behavior of multimodal LLMs, we compare
to LLaVA-1.5 using our probing framework. These models differ along several key dimensions: LLa VA-Next adopts denser visual tokenization scheme (4 more tokens via finer image chunking), is trained on a larger and more diverse instruction tuning corpus, and uses base LLM (LLaMA-3) that shares architecture with LLaVA-1.5's Vicuna but differs substantially in its pretraining dala.
As shown in Figure 5a, LLaVA-Next-L those of LLa (Figure 3) under both lexical and semantic variants: early layers show rising accuracy due to visual grounding, followed by a sharp drop from lexical variation in the middle layers. In deeper layers, accuracy for lexical variants that preserve the expected answer recovers and aligns with the anchor prompt, while semantic negation variants remain consistently low. Similarly, under output format variation (Figure 6a), LLaVA- Next-LLaMA-3 shows an accuracy peak from layers 10-15, corresponding to semantic reasoning-ape matching the behavior observed in L.a These results suggest that
exhibits probing trends that closely mirror
7

---

Published as a conference paper at COLM 2025
Lexical Variant
Semantic Negation Variant
1.0
Accuracy
0.9
Accuracy
0.8
0.6
Anchor Quection
animal plane
0.4
Anchor Question Image picture
0.7
0.2
show > feature
5
13
21
29
32
17 Layer
29
32
Layer
(a) LLaVA-Next-LLamm-- (Liu al., 2024a)
Lexical Variant
Semantic Negation Variant
1.0
0.8
Accuracy
Accuracy
0.7
0.6
Question animal > plane
0.6
0.4
0.5
Anchor Question
picture
0.2
show
13
21
28
28
Layer
Layer
(b) VL (Wang et al., 2024)
Figure 5: Probing analysis with Lexical Variant and Semantic Negation Variant for LLaVA- Next-LLaMA-3-BP (Liu et al., 2024a) and Qwen2-VL. (Wang et al., 2024). LLaVA-Next exhibits trends consistent with LLaVA-1.5, indicating that visual tokenization, instruction tuning data, and pretraining corpus have minimal impact on the overall processing hierarchy. Qwen2-VI. shows a similar stage-wise trend but allocates fewer layers to visual grounding, reflecting differences in base L.M architecture.
processing dynamics are robust to changes in visual tokenization, instruction tuning data, and the base L.M''s pretraining corpus, long the underlying architecture remains fixed.
3.3.2 Base LLM Architecture Shifts Processing Depth, Not Structure
While visual tokenization, instruction tuning data, and the base LLM's pretraining corpus do not substantially impact processing trends, the architecture of the base LLM itself does. To investigate this, we compare Qwen2-VL with LLaVA-1.5 using our probing framework. Unlike VA-1.5, which is built on Vicuna (a LLaMA-based architecture), Qwen2-VL is based on the Qwen family of LLMMs an entirely different transformer design, trained with its own tokenizer, rotary embedding scheme, and pretraining corpus.
As shown in Figure 5b, Qwen2-VL maintains high probing accuracy under lexical variants only in the first layer, with accuracy dropping sharply afterward and remaining low through layer 10. This indicates that integration between image and text tokens in Qwen2-Ve begins as early as layer 2 and continues through layer 10. This range is broadly comparable to LLaVA, where lexical variant accuracy also declines across the first 11 layers. Thus, both models use similar number of early layers for multimodal integration (visual grounding and lexical alignment).
Beyond the early layers, the two models begin to diverge more noticeably in how they allocate depth to later processing stages. Qwen2-VL extends the semantic reasoning stage over longer range. As shown in Figure 6b, output format accuracy rises steadily from
8

---

Published as a conference paper at COLM 2025
Output Format Variant
Output Format Variant
1.0
0.8
Ancher Question
0.8
Accuracy
0.7
Accuracy
0.6
0.6
0.5
0.2
17
25
Layer
Layer
(a) LLaVA-Nextt-lama---55
(b) Qwen2-VL-7B
Figure 6: Probing accuracy on format-instruction variants (yes/no 1/0). Layer-wise accuracy initially increases (layers 12-15), highlighting semantic reasoning. Deeper layers show accuracy drops, indicating a transition to answer-decoding stages, especially prominent with stronger base LLMs like LLaMA-3. Qwen2-VL exhibits similar trend but allocates broader range of layers to reasoning, with more gradual transition into decoding, reflecting architectural differences in how decision formulation and output formatting are handled.
layer 10 to 20, indicating a prolonged period during which internal representations are refined toward task-specific decisions. By contrast, VL-15 exhibits a shorter reasoning window-typically confined to layers 12 to 15-after which decoding behavior dominates. This implies that Qwen2-VI LL leverages a deeper portion of its network for reasoning rather than output formatting, potentially enabling more flexible decision encoding before token generation.
In the final layers of Qwen2-VL (layer 21-28), the accuracy for lexical variant recovers to the level of anchor questions, whereas the accuracy under semantic negation remains low, indicating those layers primarily responsible for answer decoding. By contrast, LLaVA allocates larger portion of late layers for decoding, with lexical recovery appearing later and spanning more layers.
In summary, LLaVA and Qwen2-VL follow a common internal processing mechanism to solve image dasks-vistue grounding, lexical integration, task reasoning and output decoding-yer the depth allocated to each stage and the transition points differ by model. In other words, the choice of different base LLM controls layer allocation across stages, not the internal pattern of processing.
4 Related Work
Large Language Models Large Language Models (LLMs) have demonstrated strong lan- guage understanding and generation capabilities, enabled by scaling transformer architec- tures and training on large text сorpora (Brown et al., 2020; Touvron al., 2023a; Zhang et al., 2022). Building on this foundation, recent work extends LLMs to vision-language tasks by integrating pretrained vision encoders with frozen or lightly finetuned language backbones (Liu et al., 2024b; Li et al., 2023; Zhu et al., 2023; Bai et al., 2023; Wang et al., 2024). These multimodal LLMs (MLLMs) align image features to the LLM input space via projection or adapter, followed by instruction tuning on limited multimodal data. Rather than optimizing performance, our goal is to understand how MILMss process visual and textual inputs internally. We introduce a layer-wise probing framework that analyzes token embeddings under controlled prompt variations, revealing a consistent stage-wise structure: early layers handle visual grounding, middle layers support reasoning, and later layers perform decoding modulated by architectural choices.
Interpretability of Large Language Models. A growing body of work has developed techniques to interpret the internal mechanisms of large language models (LLMs), including causal tracing (Meng et al., 2022; Vig et al., 2020; Heimersheim & Nanda, 2024), logit

---

Published as a conference paper at COLM 2025
lens (Nanda, 2023), sparse autoencoders (Lan et al., 2024; He et al., 2024; Gao et al., 2024; Dunefsky et al., 2024), and probing (Hewitt & Manning, 2019; Wang et al., 2023; Yeo et al., 2024). These tools have recently been extended to multimodal LLMs (MLLMs), with causal tracing emerging as especially useful. For example, Basu et al. (2024) and Neo et al. (2024) apply it to LLaVA (Liu et al., 2024b), showing that early layers encode contextual cues, while middle layers focus on object-level features. Golovanevsky et al. (2024) compare cross-attention in BLIP-2 and LLaVA, finding that BLIP-2 performs both object detection and suppression, whereas LLaVA primarily suppresses outliers. Complementary to these efforts, Huo et al. (2024) use the logit lens show that image embeddings often encode mixtures of semantic concepts rather than single-token alignments.
While existing interpretability methods provide valuable insights, they often require model- specific instrumentation or intervention. In contrast, we use linear probing lightweight and architecture-agnosic approach that operates directly on intermediate embeddings. By training layer-wise classifiers under controlled prompt variations, we identify when MLLMs perform visual grounding, reasoning, and or relying on custom tracing infrastructure.
modifying model behavior
Limitations
Although our study targets the LLaVA family-in which a frozen vision encoder passes projected image tokens directly into the language model-mullimode architectures vary widely in how and where they blend modalities. Pre-LLM fusion systems such as the BLIP series et al., 2023; Dai et al., 2023) insert a joint transformer before the decoder, while early- fusion designs like Chameleon (Chameleon Team, 2024) interleave image and text tokens from the first layer. These choices, together with differences visual-token granularity, co-training versus freezing of the vision tower, and task objectives, can redistribute or blur the grounding, lexical integration, reasoning, and decoding stages we identify for LLaVA. Extending layer-wise probing framework to these alternative designs and verifying whether the same stage pattern still emerges remains an important direction for future work.
Conclusion
We present a probing-based framework for analyzing the layer-wise processing dynamics of multimodal large language models. Applying our method to LLaVA-1.5, we identify three-stage hierarchy: early layers (1-4) of the LLM perform visual grounding, middle layers (5-13) integrate lexical and visual information, and deeper layers transition into answer decoding. Output format variations further reveal that layers 12-15 encode semantic decisions, while layers beyond 15 shift toward formatting those decisions into specific output tokens. We observe this structure is preserved in LLa LLaVA--Net-LLMM..A in visual tokenization, instruction tuning data, and pretraining corpus indicating these factors do not substantially alter the model's 's internal processing flow. In contrast, Qwen2- VL reallocates layer depth across stages, with shallower grounding and extended reasoning, highlighting how base LLM architecture shapes multimodal integration across depth. We hope this analysis framework can serve as foundation for future interpretability studies and facilitate more principled comparisons across multimodal model designs.
despite differences
Acknowledgment
This work was supported in part by NSF IIS2404180, Microsoft Accelerate Foundation Mod- els Research Program, and Institute of Information & communications Technology Planning & Evaluation (IIPP) grants funded by the Korea government (MSIT) (No. 2022-0-00871, Development of AI Autonomy and Knowledge Enhancement for AI Agent Collaboration) and (No. RS-2022-00187238, Development of Large Korean Language Model Technology for Efficient Pre-training).
We also thank Xueyan Zou for insightful discussions and helpful feedback.
10

---

Published as a conference paper at COLM 2025
References
Harsh Agrawal, Karan Desai, Yufei Wang, Xinlei Chen, Devi Parikh, and Dhruv Batra. nocaps: novel object captioning at scale. In Proceedings of the TEEE International Conference on Computer Vision (ICCV), pp. 8948-8957, 2019
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint ar (xXi:2230911600) 2023.
Samyadeep Basu, Martin Grayson, Cecily Morrison, Besmira Nushi, Soheil Feizi, and Daniela Massiceti. Understanding information storage and transfer in multi-modal large language models. ar preprint ar 0rXiv22466002255 2024.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advinces in neural information processing systems, 33:1877-1901, 2020.
Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. ar preprint arXit:2405.09811, 2024.
Xinlei Chen, Hao Fang, Tsung-Y Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco captions: Data collection and evaluation server. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2015.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, lon Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90% e chatgpt quality, March 2023. 00000-0033---00 v3-
URL httt::://lmsss..og/bllooo.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards general-purpose vision- language models with instruction tuning. Advances in neural information processing systems, 36:49250-49267 2023.
Mostafa Dehghani, Basil Mustafa, Josip Djolonga, Jonathan Heek, Matthias Minderer, Mathilde Caron, Andreas Steiner, Joan Puigcerver, Robert Geirhos, Ibrahim M Alabdul- mohsin, et al. Patch n' napa::: Navit, vision transformer for any aspect ratio and resolution. Advances in Neural Information Processing Systems, 36:2252-222 -2774, 2023.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama herd of models. Xiv preprint ar Xr::::2407.2177 217883 2024.
Jacob Dunefsky, Philippe Chlenski, and Neel Nanda. Transcoders find interpretable Ilm feature circuits. ar Xiv preprint ar XrXin:2406.11944 2024.
Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. arv preprint ar Xiv::20060444 04093,, 2024.
Michal Golovanevsky, William Rudman, Vedant Palit, Ritambhara Singh, and Carsten Eickhoff. What do vlms notice? mechanistic interpretability pipeline for gaussian-noise- free text-image corruption and evaluation. arviv preprint #rx.:22406.16320, 2024.
Yash Goyal, Tejas Khot, Douglas Summers-Stay Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6904-6913, 2017.
Zhengfu He, Wentao Shu, Xuyang Ge, Lingje Chen, Junxuan Wang, Yunhua Zhou, Frances Liu, Qipeng Guo, Xuanjing Huang, Zuxuan Wu, et al. Llama scope: Extracting millions of features from 11a8b-3.1-8b with sparse autoencoders. arv preprint ar arXiv:2410.20526, 2024.
11

---

Published as a conference paper at COLM 2025
Stefan Heimersheim and Neel Nanda. How to use and interpret activation patching. ar Xiv preprint ar #rXiv:2404.15253 2024.
Dan Hendrycks, Collin Burns, Eric Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. International Conference on Learning Representations (ICLR), 2021.
John Hewitt and Christopher D Manning. A structural probe for finding syntax in word representations. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and
Short Papers), pp. 4129-4138, 2019.
Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVE conference on computer vision and patlern recognition, pp. 6700-6709, 2019.
Jiahao Huo, Yibo Yan, Boren Hu, Yutao Yue, and Xuming Hu. Mmneuron: Discovering neuron-level dopaii- interpretation in multimodal large language model. Xiv preprint ar Xix::2406.111933 2024.
Michael Lan, Philip Torr, Austin Meek, Ashkan Khakzar, David Krueger, and Fazl Barez. Sparse autoencoders reveal universal feature spaces across large language models. arviv preprint ar Xiw:24100006811, 2024.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip- Bootstrapping language- image pre-training with frozen image encoders and large language models. In International conference on machine learning, pp. 19730-19742. PMLR, 2023.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024a. URL https://llava-vll gi thub. hooblogg/024// 01-22---1-3--110 -1----11----aeaeeeeeeee-- llava-next/.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, 2024b.
Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowledge. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3195-3204, 2019.
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Adoances in neural information processing systems, 35:17359-17377 2022.
Neel Nanda. Interpreting gpt: The logit lens. https: /twps://www... www.wwong...... com/ hose://ccheeoaaloovvvvieeerrrerd www. www.Jesswrong.comm/ppsttss///// http://ww.o..oo...//o////aaaaaar gpt- topt-the-lgii--eee Ac-
2023.
URL https:// //
cessed: 2025-03-26.
Clement Neo, Luke Ong, Philip Torr, Mor Geva, David Krueger, and Fazl Barez. Towards interpreting visual information processing in vision-language models. arXio preprint arXiv:2410.07149, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models, arXiv preprint arXio:2302.13971 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. Xr preprint Xiv::2300 .09000, 2023b.
12

---

Published as a conference paper at COLM 2025
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analysis. Adoances in neural information processing systemss 33:12388-12244 2020.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint urXiv:2409.12191, 2024.
Xintong Wang, Xiaoyu Li, Xingshan Li, and Chris Biemann. Probing large language models from human behavioral perspective. arXiv preprint arXie:2310.05216, 2023.
Wei Jie Yeo, Ranjan Satapathy, Rick Siow Mong Goh, and Erik Cambria. How interpretable are reasoning explanations from prompting large language models? arXiv preprint arXiv:2402.11863, 2024.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:220.001068 2022.
Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. Agieval: A human-centric benchmark for evaluating foundation models. Xiv preprint arXiv:2304.06364 2023.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint ar Aiv::2330 10592, 2023.
13


Reviews:
[{"review_id": "05sp8Gs2KI", "reviewer": "Reviewer_oi97", "summary": "This paper investigates how recent multimodal large language models (MLLMs) process information across their layers, aiming to determine whether there is a hierarchical structure in how these models handle different types of information. The main objective of this work is to build a probing classifier for each layer's representations to assess whether MLLM layers are robust to certain properities such as visual grounding, lexical-visual integration, and task-specific information processing. To perform this, the authors consider three popular MLLMs LLaVA-1.5, LLaVA-Next-LLaMA-3, and Qwen2-VL. The results show that across all MLLMs: early layers encode visual grounding, middle layers support semantic reasoning, while later layers focus on task-specific outputs.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Probing MLLMs is a valuable direction, especially given the growing popularity of instruction-tuned MLLMs for zero-shot and general-purpose tasks. Understanding how information is processed at each layer can offer deeper insights into the hierarchical structure of these models.\n2. The authors evaluate multiple MLLMs, and the consistency of results across models suggests that the findings may generalize well beyond a single architecture.\n3. Exploring different prompt structure variants adds to the robustness of the analysis, showing how these models handle variations in input format and maintain stable processing across prompts.", "reasons_to_reject": "1. Unlike previous probing studies in language models, such as [Conneau et al. 2018],[Tenney et al. 2018],[Jawahar et al. 2019], which use a wide range of probing tasks to understand the type of information encoded at each layer, this study relies on only three probing tasks. As a result, it does not offer a comprehensive view of how information is processed across layers in MLLMs. Additionally, the use of just three prompt variants limits the strength of the conclusion that these models follow a hierarchical structure of information processing. Incorporating a broader set of probing tasks-similar to those used in earlier language model studies-would provide a more thorough and interpretable analysis of these models.\n\n\n[Conneau et al. 2018] What you can cram into a single \\$ \\&!\\#* vector: Probing sentence embeddings for linguistic properties, ACL-2018\n\n[Tenney et al. 2018] What do you learn from context? Probing for sentence structure in contextualized word representations, ICLR-2018\n\n[Jawahar et al. 2019] What does BERT learn about the structure of language?, ACL-2019\n\n2. The Limited structure variations in input prompts (Image->Picture, Animal->plane) are not sufficient to generalize claims about how these models process information. Also, if an MLLM has been pretrained on datasets like ImageNet, it is possible that the some of the observed robustness inherits from prior knwowledge rather than true generalization. Further, in typical probing studies, tasks are designed to be out-of-domain so that the model has no direct knowledge of the tasks themselves. Each probing task is meant to isolate and test specific properties, ensuring that the results reflect what the model has truly learned, rather than what it has memorized.\n\n3. The results presented in the paper are more observational based on Figs, lacking deeper implications or strong conclusions drawn from each finding.", "questions_to_authors": "The authors could explain why they chose only two structured prompt variants for lexical changes (Image->Picture, Show->Feature), but only one prompt variant for semantic negation (Animal->plane). It is unclear how these specific variants were selected and what criteria were used to define them as representative. The paper would benefit from either referencing prior literature that supports the use of these variants or explaining whether alternative variants were considered. Additionally, it would be helpful to know if the authors explored other commonly used approaches from past studies that examine model robustness through diverse prompt structures.", "extra_scores": {}}, {"review_id": "jRC7SkxB35", "reviewer": "Reviewer_nNBQ", "summary": "This paper introduce a probing framework to systematically analyze how MLLMs process visual and textual inputs across layers.\nSpecificaly, the authors trained probing linear network on the last token of each layer of the LLM in MLLMs when answering visual classification questions. Then the author introduce several input variants of MLLMs and test the probing network on these variants and ues the classification accuracy drop to show the discrepency of LLM embeddings when changeing the MLLM input.\n\nBy systemmatical experiment, the authors find a consistent stage-wise structure in early layers perform visual grounding, middle layers support semantic reasoning, and later layers prepare task-specific outputs. The authors further show that such pattern is robust to training data and visual tokenization but shift with changes of base LLM.", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The experiment is well-designed and provide insightful interpretation to how MLLMs interact with multimodal information.\n\nThis paper also provide a useful protocol for future research when people want to understand MLLM behavior in more models and more tasks.", "reasons_to_reject": "Some experiment conclusions need to be further justified.\nFor example, the authors claim that \n> When lexical and semantic variants are introduced at test time, early layers (1–4) continue to exhibit relatively stable accuracy, indicating that these layers primarily encode visual information into the last-token embeddings without yet reflecting changes in the textual prompt.\n\nIt could be better if image variation is also included and see if the performance drop significantly when image is changed.\n\nOnly several simple visual question is studied, it's possible that the pattern observed in this paper might shift significantly when the model is answering a different type of visual question, such as OCR. \n\nThe writing needs to be substantially improved and the paper needs better organization, for example section 2.2 only mentioned Lexical variant, Semantic negation variants and output format variant, prompt structure variants is never mentioned. In addition prompt structure variants is introduced two times in section 3 (line 191 and 249).", "questions_to_authors": "Figure 3 is redundant with Figure 5, it is suggested to carefully check the paper before submission.\n\nIn Figure 2, the image -> picture shows much less accuracy drop comparing with show -> feature, I'm wondering if the authors have any insights about this.", "extra_scores": {}}, {"review_id": "1UCtfZzjw1", "reviewer": "Reviewer_C29N", "summary": "This work investigates how MLLMs process visual and textual information across different layers using a probing-based approach. Specifically, the authors train linear classifiers on the hidden states of the final token at each layer to track how representations evolve under varying prompts, thereby uncovering the functional roles of individual layers.\nUsing the anchor question “Does this image show an animal? The answer must be always yes or no.”, they introduce three categories of prompt variants: (a) surface-level edits (e.g., “image” → “picture”), (b) semantic negation (e.g., “animal” → “plane”), and (c) output format changes (e.g., “yes or no” → “1 or 0”). \nTheir findings include: (a) middle layers (layers 5–13) are highly sensitive to surface-level changes; (b) late layers (after layer 14) show a significant drop in probe accuracy under semantic negation; and (c) a rebound in accuracy for semantic negation variants are observed in the final layers (30–32).", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is clearly written. \n2. The authors applied the analysis to multiple models (LLaVA-1.5, LLaVA-Next-LLaMA-3, and Qwen2-VL), thus offering valuable comparative insights in how multimodal information is processed.\n3. A probing-based framework that reveals the internal processing dynamics of multimodal LLMs across layers.", "reasons_to_reject": "1) In the experimental setup, it seems incongruent to train the probing classifier to predict dog breeds, as this effectively poses the question “What is this dog?”, whereas the actual prompt given to the model is “Does this image show an animal?”. This mismatch introduces a disconnect between the probing task and the model’s intended behaviour. A more appropriate approach might be to extract hidden states from both positive and negative examples and train the probe to directly answer the binary question “Is this an animal?”, thereby aligning the probe’s objective with the original task.\n \n2) The probe is trained using the hidden states of the last token at each layer. However, information propagation in transformer models is not confined to the final token. Prior studies [1,2] have shown that in early layers, important information is often distributed across earlier tokens. As such, relying solely on the last-token representation may overlook critical dynamics in the processing flow, thereby raising concerns about the completeness and faithfulness of the conclusions drawn from the probe’s performance.\n \n3) The experimental design involving output format variants (e.g., changing “yes or no” to “1 or 0”) appears problematic and may not robustly support the authors’ conclusions. Since the semantic content remains unchanged while only the output token format is altered, it is unclear whether the observed differences in probing accuracy genuinely reflect a transition from semantic reasoning to answer formatting. Without stronger justification or complementary analyses, this setting may not provide reliable evidence for localizing the decoding stage.\n4) Semantic negation involves replacing “animal” with another noun like “plane.” This does not only flip the expected answer; it changes the semantic category being reasoned about, introducing semantic drift. This makes interpretation ambiguous.\n \n[1] Dissecting Recall of Factual Associations in Auto-Regressive Language Models https://aclanthology.org/2023.emnlp-main.751.pdf \n[2] A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysis https://aclanthology.org/2023.emnlp-main.435.pdf", "questions_to_authors": "1. Wouldn't it be more appropriate to train the probing classifier to answer the binary question “Is this an animal?” in line with the model's actual prompt, rather than predicting dog breeds, which introduces a mismatch between the probing task and the model’s intended behaviour?\n2. Does replacing “animal” with a different noun like “plane” truly constitute semantic negation, or does it instead introduce semantic drift by altering the category being reasoned about making interpretation ambiguous? \n3. Could the controlled environment oversimplify multimodal reasoning, limit generalisability, introduce artificial prompt constraints, and ambiguity in prompt interpretation?", "extra_scores": {}}, {"review_id": "ogkPrnnrI9", "reviewer": "Reviewer_RU2Z", "summary": "This paper investigates how multimodal LLMs process visual and textual input across layers using a probing framework. To this end, linear classifiers are trained to predict fine-grained visual categories (dog breeds) from token embeddings extracted at each layer, using a class-agnostic prompt: “Does this image show an animal?” as an anchor question (answer is always “yes”) and baseline for the experiments. Three types of prompt variations are evaluated to investigate the role of different layers: lexical variants, semantic negation and output format variants. The results show a robust structure in MLLMs, with early layers performing visual grounding, middle layers supporting semantic reasoning and later layers preparing the task-specific output.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- the paper is clearly structured and well-written, the methodology is clearly explained.\n- this research provides interesting insights in how visual and linguistic information interacts across layers in multimodal LLMs, using a straightforward approach that operates directly on intermediate embeddings. As such, this research is an interesting contribution to interpretability research for MLLMs.", "reasons_to_reject": "- some concepts need a more detailed explanation/definition, e.g. “semantic reasoning”\n- I understand the authors want to create a completely controlled experimental environment, but I would have preferred a more exhaustive set of experiments with different examples per  variant. If we take the lexical variants, for instance, you replace \"image\" > \"picture\", and \"show\" > \"feature\". As feature is also polysemous (even with distinct part-of-speech categories), it would be interesting to compare the results with other replacements where the verb is replace with another non-ambiguous verb. I think this study is very useful to give some first insights, but it could be expanded with a more elaborate test bed leading to deeper and more robust insights.\n- It would be interesting to relate your results to findings in other interpretability research for LLMs, and more specifically, research investigating which information is encoded in the various layers. This is briefly mentioned in the related work section, but deserves a more elaborate discussion. This could better motivate why your approach, that uses linear probing as an architecture-agnostic approach,  is a more straightforward but viable method to investigate the inner workings of MLLMs.", "questions_to_authors": "See remarks above.", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents a thoughtful study on the internal workings of Multimodal LLMs using a novel probing framework. I find the research question timely and the approach clever. The reviewers raised several valid points during the initial review phase. They questioned the limited scope and specific design choices, which could have weakened the paper's conclusions. These initial concerns were reflected in a wide spread of scores, indicating a clear need for clarification from the authors.

The authors' rebuttal was compelling and directly addressed the main criticisms. I was particularly convinced by their argument that their goal required a different, more controlled probing setup than prior work in unimodal NLP. Their careful explanations satisfied most reviewers, leading to two score increases (C29N was not able to update the score but verbally confirmed) and a strong consensus for acceptance among those who engaged in the discussion. While one reviewer remained negative and did not respond, their primary concerns were substantially resolved by the authors' clarifications. In my judgment, the paper makes a solid contribution to MLLM interpretability.

The AC considered the review, the author response and the discussion and decided to recommend Accept.
Decision: Accept