SAE-V: Interpreting Multimodal Models for Enhanced Alignment
Authors: Hantao Lou
Changye Li
Jiaming Ji
Yaodong Yang
Venue: ICML 2025 poster (2025)
Keywords: interpretability
alignment
multimodal large language model
Primary area: social_aspects->accountability_transparency_and_interpretability
TL;DR: We propose SAE-V, a mechanistic interpretability framework for multimodal large language models to analyze multimodal feature and enhance their alignment.

Abstract: With the integration of image modality, the semantic space of multimodal large language models (MLLMs) is more complex than text-only models, making their interpretability more challenging and their alignment less stable, particularly susceptible to low-quality data, which can lead to inconsistencies between modalities, hallucinations, and biased outputs. As a result, developing interpretability methods for MLLMs is crucial for improving alignment quality and efficiency. In text-only LLMs, Sparse Autoencoders (SAEs) have gained attention for their ability to interpret latent representations. However, extending SAEs to multimodal settings presents new challenges due to modality fusion and the difficulty of isolating cross-modal representations. To address these challenges, we introduce SAE-V, a mechanistic interpretability framework that extends the SAE paradigm to MLLMs. By identifying and analyzing interpretable features along with their corresponding data, SAE-V enables fine-grained interpretation of both model behavior and data quality, facilitating a deeper understanding of cross-modal interactions and alignment dynamics. Moreover, by utilizing cross-modal feature weighting, SAE-V provides an intrinsic data filtering mechanism to enhance model alignment without requiring additional models. Specifically, when applied to the alignment process of MLLMs, SAE-V-based data filtering methods could achieve more than 110% performance with less than 50% data. Our results highlight SAE-V’s ability to enhance interpretability and alignment in MLLMs, providing insights into their internal mechanisms.

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
Hantao Lou 12 Changye Li 12 Jiaming Ji
Yaodong Yang
Abstract
Large Modaley Cup
Landsaape
Modelly Foned
With the integration of image modality, the seman- tic space of multimodal large language models (MLLMs) is more complex than text-only mod- cls, making their interpretability more challeng- ing and their alignment less stable, particularly susceptible to low-quality data, which can lead inconsistencies between modalities, hallucina- tions, and biased outputs. As result, developing interpretability methods for MLLMs is crucial for improving alignment quality and efficiency. In text-only LLMs, Sparse Autoencoders (SAFs) have gained attention for their ability to interpret latent representations. However, extending SAEs multimodal settings presents new challenges due modality fusion and the difficulty of isolat- ing cross-modal representations. To address these challenges, we introduce SAE-V, a mechanistic interpretability framework that extends the SAE paradigm to MLLMs. By identifying and analyz- ing interpretable features along with their corre- sponding data, SAE-V enables fine-graing inter- pretation of both model behavior and data quality, facilitating a deeper understanding of cross-modal interactions and alignment dynamics. Moreover, by utilizing cross-moda feature weighting, SAE- V provides an intrinsic data filtering mechanism to enhance model alignment without requiring ad- ditional models. Specifically, when applied the alignment process of MLLMs, SAE- V-based data filtering methods could achieve more than 100% performance with less than 50% data. Our results highlight SAE-Y's ability to enhance interpretabil- ity and alignment in MLLMs, providing insights into their internal mechanisms.
Multimodal LLM
SAE-V
Daliett
Dataen
Interpetabble
CDES0589ES
Pestine
coosocleees
Quality High Quality
Fitured
Figure I. Operational Dynamics SAE-V Based Data Filter- ing Method. SAE-V encodes and interprets the representation inside MLLM during alignment and inference time. Based on this representation, could reveal the modality gap within the data, and improve the alignment process through the selection of modality- high-quality data. This pipeline performs data filtering without requiring additional models, relying instead on MLLM itself to prioritize high-value value data effectively.
1. Introduction
With the development and success of large language mod- els (LLMs) (Dubey et al., 2024; Achiam et al., 2023), re- searchers have begun to introduce visual understanding to these models, thereby extending their operational scope from language to a mix of vision and language, resulting in the creation of powerful multimodal large language models (MLLMs) (Alayrac et al., 2022; Liu et al., 2024a; Team et al., 2024; Team, 2024). To enhance the multimodal under- standing capabilities of MLLMs, the research community has explored various architectures, including using individ- ual image/text encoders to encode cross-modal information into a joint representation space (Zhang et al., 2023; Liu et al., 2024a; Zhu et al., 2024; Wu al., 2024c) and lever- aging image tokenizers to transform all inputs into unified token sequence (Team, 2024; Xie et al., 2024; Wu et al., 2024a; Wang et al., 2024). Despite the difference in the ar- chitectures of these models, their essential goal is the same: Fuse the text and image representation space into a joint multimodal semantic space.
"qqual contribution 'Institute for AI, Poking University, Beijing, China State Key Laboratory of General Artificial Intelligence, In- stitute for AI, Peking University, Beijing, China. Correspondence to: Hantao Lou <hantaolou.htlou http:///.ooooooo@ml.ooooooo com,, Yaodong Yang <yaodong. yang @ ekpu.cdu.cn...
Proceedings of the 4dnd International Conference on Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
OUr codehase and model released Githuh and Hugging- face.
As MILLMs continue scale up in both size and capability,

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
their interpretability and controllability remain significant challenge (Zhang & Zhu, 2018; Ben Melech Stan et al., 2024). Currently, mechanistic interpretability techniques such as circuit analysis (Olsson et al., 2022) and dictio- nary learning with sparse autoencoders (Huben et al., 2024) are the most widely recognized approaches to interpreting LLMs. However, their application to MLLMs, especially in the context of cross-modal integration. has been limited. There is a pressing need for specialized tools and frame- works that can unravel the intricate workings of MLLMs.
Top-k Pretunes #12015 ei0000
Tocenizze
Encoder
Reconstructdd
Decodar
LLM Layer
BOOOO
No.
LLM Layer
Facedaet
Soore 5
0.M5
Asthvation
Activation
SAI
LLM Layer
Figure 2. The interpretability and data filtering pipeline of SAE- V. SAE-V is trained to encode MLLM activations into sparse, interpretable features. We first acquire the gross-modal weight of these features via SAE-V models, then reversely score the given data by the weighted average of each feature's score. this way. we provide an intrinsic data filtering tool by eliciting MLLM's latent representation these data.
Moreover, current interpretability efforts are focused mainly on interpreting models, rather than applying this inter- pretability to real alignment situations, which also makes difficult to evaluate these methods effectively. Top-down approaches, such as Representation Engineering (Zou al., 2023) and activation steering (Turner et al., 2023; Rim- sky et al., 2024), can directly evaluate the control effects of interpretability methods through control or unlearning techniques. However, for bottom-up methods like circuit analysis (Wang et al., 2023), sparse autoencoders (SAEs) (Huben et al., 2024), and cross-coders, effective evaluation methods beyond loss reduction are limited. Based on the previous discussion, can we propose a bottom-up multi- modal interpretability approach that can directly enhance the alignment process?
performance on multimodal understanding tasks.
Filtering metric to improve alignment Based on the previous investigation with SAE-V, we developed a metric to filter multimodal datasets and acquire high- quality data, therefore improving aligmment quality and efficiency. Experiments demonstrate that our filtering tool achieves more than 110% performance compared the full dataset while using 50% less data, under- scoring the efficiency and effectiveness of SAE-V.
In this work, we developed SAE-V, a mechanistic inter- pretability framework for MLLMs that extends the SAE paradigm to MLLMs. These tools are then applied to in- terpret the training process of transitioning from LLMs to MLLMs, as well the process of enhancing the multimodal capabilities of MLLMs. Furthermore, utilizing the inter- pretable features of SAE-V models and their relationship to MILM capabilities, we designed a data filtering metric based on SAE-V. This metric can filter out data that hinder the development of multimodal understanding, achieving stronger alignment with a smaller dataset. Overall, our work makes the following contributions:
2. Methodology
In this section, we present our method to train, evaluate, and apply SAE-V to interpret MILMS and multimodal data.
2.1. Preliminary: Sparse Autoencoder Paradigm
We adopt SAE-V (denoted as Sq) architecture from the methodology proposed in (Bricken et al., 2023), which com- prises an encoder and a feature dictionary Fe: {ff as decoder. Let the input be denoted as H E R where I donates the number of input tokens and m donates the shape of hidden state token, the hidden state of a specific layer of a MILMM Mg. The SAE-V encoding operation Sp(+) is defined as
Multimodal interpretability tool We developed mech- anistic interpretability tools for MLLMs based on pre- vious attempts on LLMs and trained corresponding SAE-V models. We demonstrated that SAE-V models trained on MLLMs can effectively extract interpretable features, and SAE-V V models can be transferred to the corresponding LLMs. Specifically, the reconstruction loss of our SAE-V models trained on MLLMs is 38.3% and 50.6% compared the SAE model when applied to MLLMs and LLMs, respectively.
Z = ReLU(H x Wenc + bene),
(1)
where 7 E I is the feature activation of the input. The reconstruction loss of Sg donates as
C = || Z x (f1, f2. ... fn) ....................))
(2)
Interpreting Multimodal Alignment Process We uti- lized SAE-V V to study the feature distribution through- out the alignment process. We discovered that the fea- ture distribution of SAE-V corresponds to the MIL.M's
The training loss is defined by
C - LR + XC1.
(3)
2

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
Algorithm 1 Cosine similarity score Ranking
by MLLM forward and Equation 1, i.e.
Require: Text token vocabulary: T: vision token vocab-
H4 Ma(dd)
(4)
ulary: V: multimodal dataset D = {d, ) MLLM pa- rameterized by Ma; SAE-V V model Sy; features of SAE-V model Fa: {f activation bound: ó; cosine similar- ity function C (Equation 7) Ensure: Ranked data DR
7, So (Hi):
(5)
2] = ej Zi:
(6)
where e, = (0, 0....
,,..,. ,0.
j-th position
Stage 1: Collect Feature Activation Token Initialize activated token set of features Ar +- 2 D& + Sample(D)
We define a SAE-V feature fx is activated on zj if z,ı > 8, where 8 is activation bound. Correspondingly, we state that fk. is activated on d: if 2j Z,, s.t. 2j activates fx-
for each d, E "D. do
H₂ + MM(d.)
Our algorithm I consists of three stages: (1) Collecting feature activation tokens from dataset, (2) Computing cross-
Z₁ + So(H.) HH))
for each fi E Fg do
modal weight of SAE-V V features, and (3) Ranking dataset by cross-modal weight.
A&+ c A& (hyi hy E H,, = e,Zi, > 8}
end for
end for
1. Feature Activation Token Collecting We first sample a small subset D3 of the training dataset D and input these samples into the MLLM to obtain hidden states H. These hidden states are then fed into the SAE-V V encoder to extract feature activations defined as Equa- tion 4. For each feature, we collect its hidden state tokens thereby obtaining a sample of feature activation tokens across the dataset.
Stage 2: Compute Cross-modal Weight
Initialize cross-modal weight of features wk + 0
for each fx E Fo do
We < -C(Top((AA O T), Top (D)
end for
Stage 3: Rank Dataset by Cross-modal Weight Initialize cross-modal score of data s; <- 0 for each d: E 'D do
2. SAE-V Feature Weighting For each feature fx. we identify its top-K hidden state text tokens = [Ap n T) and top-K hidden state vision tokens v (AA n V), where the top- is ranked by the activation value Zjk of the token. We then compute the cosine similarity between the two lists of tokens, donating the cross-modal weight of feature fx as
Z; + (o))) F - [s:: 3 3z E Zi, s.t. 2j activates Ju}
8; - ELEF, Wk
end for
DR + SDrt(D, 181)1=)
t; 0.
Cosine(t, v) = Essj ||t,|||l,,|l
(7)
where C, = ||%|| adds a sparsity constraint to the learned features and À is a hyperparameter controlling the level of sparsity. The training results could also quantized by incorporating an additional sparsity constraint via Lo || which counts the number of nonzero elements in the leamedd features Z.
which represents the capability of the feature to capture multimodal information within data.
3. Data Ranking Using the weighted features of SAE-V model, we score the entire training dataset. The cosine similarity score of each piece of data is defined the sum of the cosine similarity scores of its activating features. We rank the data set by the score and the resulting cosine similarity score order allows us to filter data that are better aligned with the structures of multimodal semantic information.
2.2. Interpreting Multimodal Data with SAE-V
It has been previously demonstrated (Gao et al., 2025; Huben et al., 2024) that SAE can be employed to inter- pret how LLMs encode semantic information from these models. This feature motivates us to apply SAE-V V to as- sess the quality of the data and thus facilitate data filter for alignment.
We present our experiments and results in Section 4, demon- strating that our cosine similarity score ranking method can effectively filter high-quality data from the training data set.
We adopt cosine similarity score ranking algorithm for data filtering (shown in Algorithm 1). Let the multimodal
training dataset be donated as D= = {/ where d, = (u,, U2.... 44), Uj E T V Uj E V.j = 1,2,... I, is a sequence of tokens from text vocabulary T and tokens from
3. Interpretability Analysis with SAE-V
In this section, we conduct experiments on the SAE-V paradigm, aiming to demonstrate the capability and transfer-
vision vocabulary V. We acquire feature activation token zj
3

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
Veature H440031
Doberman Dogs
Fenture #11105
Symmetry
skaniffel
Cakonil lasies come in mary soo.
Diqpit grod runerbeniing ssality und Smereey
His cnded whea the Biteerns liteed her had exured, coorgglt assghh...
they-201 Sie kapom fxtr Serreeyy
Baleaass will
compatiive
(A) Activation
(h) Acthoattnn
(c) Activation
D0 Activation
(e) AlAtsttttion
Activation
Activatias
Activatios
(a)
(b)
Figure 3. Examples of interpretable features discovered by SAE-V. We presented examples of interpretable SAF-V V features on LLaVA-NeXT-7B that demonstrate cross-mos modal semantic consistency. (a) Feature #44031 exhibits strong activation for 'Doberman dogs' across both text and image modalities, showing SAE-V's ability identify specific concepts with concrete physical meanings. (b) Feature #11105 captures the abstract concept of "Symmetry" across different modalities, activating for various symmetry patterns including left-right, top-bottom, and central symmetry, with activation regions precisely aligned with symmetrical elements in images. These examples illustrate that SAE-V can discover features representing both concrete entities and abstract concepts that maintain consistent semantic meaning across modalities, going beyond what traditional probing methods on raw activations can achieve.
ability of SAE-V V model. We also performed experiment to prove the effectiveness of our SAE-V-based data interpreting tool from the inference side.
the base LLM of LLaVA-NeXT-7B."
Evaluation Metrics To evaluate the performance of SAE- V models, we use two key metrics: Co = ||||| where defined Equation 6 and reconstruction loss LR in Equa- tion 2. Lo quantifies the number of activated features, re- flecting the method's ability to extract interpretable features, while reconstruction loss measures the method's activation reconstruction capability compared with the model output, indicating the method's s accuracy in giving interpretations.
3.1. Training and Evaluating SAE-V Model
We trained a series of SAE and SAE-V models on MLLMS and their base LLMs. We evaluated the performance of these models, and the results demonstrated that SAE-V model is capable of interpreting MILMSs and that SAE-V model trained on MLLM can be effectively transferred to its origi- nal LLM.
3.1.2. EXPERIMENT RESULT
3.1.1. EXPERIMENT SETUP
14
Datasets For text-only and multimodal situations, we se- lected the Pile (Gao et al., 2020) and Obelics (Laurençon et al., 2023) datasets separately. Specifically, 100K data from each dataset as the train set and 10K data as the test set. The Pile is diverse language modeling dataset for LLM pretraining, and Obelics is a massive inter- leaved image-text dataset for MLLM pretraining. These two datasets are widely recognized in various pretraining and interpretability works (Black al., 2022; Biderman al., 2023; Huben et al., 2024; Team, 2024).
SAE-V
Original
sampled
Reconstrationn Laws
HLAWNNNNT Mistni-78
LANA-NEX- LEANA-NEXT
Chamclonn-7B
Amole-7B
Vissmm7B
Vicum-13R
Models We selected two generic MILMs, LLaVA-NeXT models (including
Figure 4. Reconstruction capability of SAE and SAE-V models. Each section compares the metrics of zero (set all activations as zero), SAF model, SAE-V model, and the Original reference state. SAE-V V model consistently demonstrates superior reconstruction performance actoss all tested models. In Chameleon-7B and Anole- 7B, SAE performs worse than the zero baseline, which indicates that SAE trained in text data fails to capture interpretable features in these MLLMs.
LLaVA- VeXT--icuna-7B and LLaVA-NeXT-Vicmme 13B) (Liu et al., 2024a) and Chameleon-7B (Team, 2024), as our target models. These models represent two distinct architectures, and testing our method on them can demonstrate that our method is applicable to different architectures.
Additionally, also studied Anole- (Cher et al., 2024) and Mistral-7B (Jiang et al., 2023) to compare the be- havior of SAE and SAE-V models before and after fine- tuning, specifically the transitioning fine-tuning from LLM to MLLM. Anole-7B is a variant of Chameleon-PP with its image generation capability unlocked, while Mistral-7B is
Capability of SAE-V Model We compare the perfor- mance SAE-V and SAE on different multimodal models.
We present the detailed training setup and parameras in Appendix

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
Model
Method
Co
Zem SAE Mistril
SAE SAE-V
24757.6
Liava SAB-V
Chameleon-7B
50.1
Revonstruction Loss
Original
SAE
62.1
Anole-7B
SAE-V
50.1
4-
SAE
94.5
LLaVA-NexT-Miistral-BB
SAE-V
192.5
+
SAE 3162.96
LLaVA-NexT-Vicunn-TB
SAE-V
585.64
LLaVA-NeXT--icunn--
SAE 128.56
Mistral-7B, Text
LLaVA-NEXT-7B, Text
LLaVA-NEXT-7B, Vision
SAE-V
193.63
Table 1. The La metric of SAE and SAE-V models. Co indi- cates the sparsity cost (average activated feature number). The results vary significantly across models due their architectural differences. For Anolc-7B and Chameleon-7B, SAE-V V models maintain lower Lo, suggesting more efficient feature utilization. However, L.aVA-NeXT models shows a contrary pattern with SAE-V requiring higher feature activation than SAE. We propose that extra activated features of SAE-V V model are introduced by ex- tra vision tokens in multimodal data, Notably, Chameleon-7B and LLaVA-NeXTA V-iuunaa- with SAE model exhibits unusually high sparsity cost, attributed to multiple unseen vision tokens in the inference stage.
Figure 5. Reconstruction performance of SAE and SAE-V. The x-axis shows different models and task configurations, text indi- cates text-only task, and text & vision indicates multimodal task. The colored bars represent five experiment groups (zero activation, SAE of Mistral-7B, SAE of LLa VA-NeXT-7B trained with the Pile, SAE of LLLVA-NeXT-7B trained with Obliecs, original perfor- mance). Across various settings, SAE-V consistently demonstrates superior transfcrability compared to SAE and achieves reconstruc- tion loss close to the original performance, the maximum relative gap being 67.28%. SAE based on Mistral-7B and SAE' based on LLa JLaVA-NeXT-7B achieves nearly the same loss in all tasks and models, indicating the equivalence of training SAE with MILLM and its hase I.I.M.
The Co (shown in Table 1) varies significantly across the three models. For LLaVA-NeXT-7B, the Lo of SAE-V is much higher than that of SAE. For Chameleon-74 SAE-V performs normally, whereas the Lo of SAE is abnormally high, indicating that SAE fails to extract sparse features. We suppose that the failure is attributed to a large num- ber of unseen vision tokens for SAE during the inference stage. For Anole-7B, the Lo of SAE and SAE-V are nearly identical. The reconstruction loss (shown in Figure 4) of SAE-V is lower than SAE and is closer to the original acti- vation, demonstrating that SAE-V behaves much better at reconstructing original activation than SAE across all three models. The results indicate that SAE-V outperforms SAE in terms of capability.
both MLLM and LLM, SAE model exhibits nearly identical reconstruction loss values, showing its robust transfcrability.
These results highlight that training SAE-V model for MILMs with multimodal data is effective for interpreting MLLMs, and even LLMs, as SAE model trained solely on textual data fail to extract and disentangle the hidden repre- sentations of MLLMs effectively. Moreover, SAE-V V model demonstrates superior capability in reconstructing the rea- soning features of MLLMs compared to the standard SAE model.
3.2. Apply SAE- V Model on Multimodal Data
In this section, we conduct an image classification task on the ImageNet dataset (Russakovsky et al., 2015) to investi- gate whether SAE-V capture the key information within images and validate the effectiveness of the methods pro- posed in Section 2.2 on multimodal data. We apply 4 meth- ods, namely Lu. L1. oo-occurring La. and cosine similarity score, where co-occurring Cn is defined as the number of features activated on at least one text and image token. The cosine similarity score defined as the sum of cross-modal weights of features, consistent with Algorithm 1. We adopt these metries to filter image patches, thus obtaining images that preserve 75%, 50%, and 25% patches, respectively. 3
Transferability of SAE-V Model The transferability of SAEs between foundation models and instruction-tuned models has been extensively investigated in text-only con- texts (Kissane et al., 2024; Kutsyk et al., 2024; Gallifant et al., 2025), as it demonstrates whether SAES can capture universal semantic features within LLMs. Similarly, the transferability from MLLMs corresponding LLMs serves as a critical metric for the quality of features learned by SAE-V. We compared the reconstruction performance of SAE-V model trained on I.LaV--NeXT-BB and SAE model to prove that SAE-V V model trained on MLLMs can general- ize to its base LLM. The findings (shown in Figure 5) indi- cate that across different settings, SAE-V V model consistently achieves the best performance. Moreover, when trained on
We present complete algorithms of 4 methods in Ap-
pendix C.I.
5

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
significantly surpass the accuracy obtained by the random preservation method. These results demonstrate that SAE-V can accurately capture critical information in images and that the methods proposed in Section 2.2 effectively utilize SAE-V features during inference."
(a) Original Image
(b) Top 75% Patch
4. Alignment Experiment
In this section, we adopt cosine similarity score rank- ing al- gorithm as data filter (as shown in Algorithm 1) to acquire high-quality data for model alignment.
4.1. Experiment Setup
(c) Top 50% Patch
(d) Top 25% Patch
Dataset and Model Consistent with Section 3.1.1, we se- lected LLaVA-NeXT-77 (Liu et al., 2024a) and Chameleon- 7B (Team, 2024) for our alignment experiment. Since the LLaVA-NeXT-7B model is rather powerful in multi- modal capabilities, we selected the AlignnAnything (Ji et al., 2024) eex-iinagett--text dataset for our experiment. Align- Anything is 400K multimodal preference dataset contain- ing fine- -ee-ggraidd annotated multimodal input-output prefer- ence data, and we used the 40K subset of text-image input and text output in our experiment.
Figure 6. Case analysis of image patch filtering using Lo met- ric. rank and filter image patches according to the number of features activated on them. The top row shows the original image (a) and its reduced-patee patch versions retaining 75% (b), 50% (c), and 25% (d). In this dog image, the patches are filterd out from edge to the middle and preserved almost only dog patches, suggesting that SAE-V V model is preserving the main semantic information of the image.
Lons Value
Algorithm We adopt the cosine similarity score ranking algorithm (shown in Algorithm 1) as a filter exclude data with low scores. In addition, we also adopt two algorithms, the Lo ranking, and the co-occurrence ranking. 6
0.5
1B
65
Quistlaly
Randor
100%
15%
Dats Percentage
Evaluation To evaluate the efficiency of our methods, we applied Direct Preference Optimization (DPO) to the model using the filtered datasets. We then evaluate the multimodal capabilities of the model using LLaVA-Benn (Liu et al., 2024a) benchmarks.
Figure? 7. The classification performance on ImageNet. We com- pare the classification accuracy after filtering the image patches with Lo method, Li method, co-occurrence Lo method, cosine similarity score method, and the random baseline. All methods achieve high accuracy when preserving 75% or 50% patches and Lo method, Ln method, and cosine similarity score method main- tains high accuracy even in the least patches. The result shows that SAE-V is efficient in capturing critical information from images.
4.2. Experiment Results
We performed SAE-V -based data filter with different fil- tering ratios on the LLaVA-NeXT-7B model and Align- Anything dataset. The filtered datasets were then used to fine-tune MLLMs, which were evaluated on LLaVA-Bench. The results (shown in Figure 9) demonstrate that our SAE- V-based filtering method effectively enhances the alignment of LLa VA- even with reduced data. Since most of the data in Biith-nn contribute positively to model alignment, the performance of the model is higher than the base model without any fine-tuning in most cases. At any data filter proportion, the SAE-V-based data filtering method
Case Analysis Figure 6 illustrates the image patches when using Co metric for filtering. Even when employing the simplest Lo metrie, SAE-V is still able to effectively capture the critical semantic information of the image."
Quantized Results The quantized results are presented in Figure 7, where we observe that, for all methods, preserv- ing 75% or 50% of the patches achieves an accuracy close to that obtained using the full image. In the challenging scenario where only 25% of the patches are retained, Lo method, L1 method, and cosine similarity score method maintain accuracy levels close to 70,, and all methods
'More experiments and their quantative analysis are presented in Appendix C.3
Detailedd descriptions of these ablation algorithms and their corresponding hyperparameters are provided in Appendix B. "We present detailed training parameters in Appendix B.2.
Moor cases and analyses are presented in Appendix C.2.
6

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
Sodee Conearee
Looks Gouurnie
Kandes
Daaa Jaturage
Totb Parsaagaa
(a) Performance on Anythngg dataset
(b) Performance on MMInstruct datasct
(c) Performance on RLAIE-V dataset
Figure Ablation study SAE-V V-based data filter method on Chameleon-7B model and different datasets. We replicate SAE- V-based data filter Chameleon-7B model and three distinct datasets. The results show that SAE- V-based data filter method is still effective on different datasets and different model architectures and image encoding methods other than CLIP. (a) On Align-nnything dataset, both cosine similarity and co-oceurennce filters perform better than random filters at almost every data percentage, and the cosine similarity filter achieved score of 52.1 I (120% of the full dataset's performance) with 70% data. (b) On MMInstruct, the cosine similarity filter outperforms the random filter baseline every data proportions. Specifically, the cosine similarity filter achieved score of 61.2 (117% of the full dataset's performance) with 60% data. (c) On RLAIE-V dataset, the cosine similarity filter achieves the highest score of 54.2 (125% of the full dataset's performance) with only 10% data, demonstrating its supreme efficiency.
L5
Couine ConEEMETO
Itrformaee 166
Parfmraaee Score
30
50
20
200
Data Pescenlage (56)
Avargo Conine Similarity
Figure 9. The performance of SAE-V-basel V-bsee data filter method We evaluated SAE- V-based data filter method on LLaVA-NeXT. 7B model, Align-Anything dataset, and LLaVA-Bench benchmark. The results show that all SAE- V-based methods significantly out- perform the random selection baseline, while the cosine similarity filter achieved 108% of the full dataset's performance with only 20% data, and co-occurrence filter peaked at 50% data, reaching score of 108.17. As more straightforward uti- lization of SAE-V model, the filter also generally outperforms the random selection baseline.
Figure 10. The relationship between average cosine similarity score and MILM performance. We measure the average cosine similarity score of models in Section 4.2, and fit linear relation- ship between model performance and average cosine similarity score. The correlation coefficient of the correlation reaches 0.84, suggesting higher similarity scores on SAE- features corre- spond to enhanced MLLM performance.
model training. To further investigate the relationship be- tween model performance and cross-modal similarity, as measured by cosine similarity of SAE-V features, further measure the average cosine similarity score of these models. Given dataset, we apply the cosine similarity score rank- ing algorithm (shown in Algorithm 1) to the MI.LM, and we define the MLLM's average cosine similarity score as the mean score of non-zero cross-modal weight SAE-V features.
outperforms the random selection baseline, with the best result being 108.17 17 (115 1155 of the full dataset's performance) achieved using 50% filtered data from the cooceurrence filter, and 104.20 (108% of the full dataset's performance)
achieved using 20% filtered data from the cosine similarity filter. However, as the dataset inevitably contains some low- quality data, the performance is optimal with a moderate data proportion and shows downward trend as the data proportion increases.
We caleulated the average cosine similarity scores for the models discussed in Section 4.2. The result (shown in Figure 10) revealed a positive correlation between the average co- sine similarity score of SAE-V V feature and the performance of MLLM, suggesting that higher similarity scores of SAE-V V features correspond to enhanced MILLM performance.
4.3. Relationship between MILMM Capability and SAE-V Features
In the previous section, we demonstrated the effectiveness of utilizing the cosine similarity score for data filters in

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
Method
0%
10% %
20%
30 %
40 %
50 %
60%
70%
80%
90%
100 %
SAE-V 94.2
98.2
106.8
114.9
114.5
114.8
112.9
112.3
111.0
109.5
98.5
Random
94.2
96.5
97.0
98.3
95.3
93.7
96.5
98.8
98.2
96.8
98.5
Table 2. The performance of SAE-V-basee data filter method on 1/20 Align-Anything dataset The performance of SAE-V-based data filter method on 1/20 Aign--nything dataset closely resembles that Figure 9, confirming that alignment metries on a small validation set resemble the distribution on the complete datasct, cnabling efficient hyperparameter selection.
Performance
Figure 8 (b) and Figure 8 (c)) further confirm that SAE- V- based data filter is working across different datasets. More- over. on RLAIF-V, the cosine similarity filter could achieve a score of 54.23 (125% of the full dataset's performance) by using only 10% of the data, and on MMInstruct, the cosine similarity filter could achieve score of 61.2 (117% of the full dataset's performance) with 60% data, demonstrating exceptional efficiency.
Method
0%
20%
40%
60%
80%
100%
SAE-V
104.6
105.8
116.7
112.3
112.0
111.2
Random
104.6
105.3
105.8
107.2
110.4
111.2
Table The performance of V-bsed data filter method on LLLaA--NeT--VE Vicunaa-11 We replicate SAE-V-hasel data filter on -eNV----NT---uuini 13B and Align-Anything dataset. The result shows that SAE-V- V-base data filter outperforms the random selection baseline, and reached the highest performance of 116.67 with 40% data. This further indicates that --E-ed data filter could work regardless of model architecture and sizes.
Ablation on Dataset Sizes On real deployment scenarios, due to the large size of datasets, it is often difficult to use the SAE-V-based data filter method to search for filtering ratio parameters on the complete dataset. Therefore, we reproduced the SAE-V --a-ed data filter method on a 1/20 subset of the Align-Anything dataset to demonstrate that in practical deployment situations, it is feasible to search for filter ratios on small-scale dataset subsets for application to the complete dataset. The results (shown in Table 2) indicate that the performance of S-E-V- data filter the small-scale dataset exhibits extremely similar trends with respect to filtered data ratios as those observed on the complete dataset (shown in Figure 9), which demonstrates that searching for filtered data ratios through small-scale dataset subsets is feasible.
4.4. Ablation Study
Ablation Models To prove that SAE-V- data filter method could generalize to distinct model architectures, we replicate SAE-V-based data filter on Chameleon-7B and VIVA-NNTT-Viiunea 13B models with the Align- dataset. The result (shown in Figure (a) and Table 3) demonstrates that SAE- V-based filter method still shows its effectiveness regardless of model architecture, sizes, and image encoding methods.
When using a smaller data proportion, the performance is strongly correlated with the data quantity, and thus, the dif- ferences between methods are minimal. However, with a larger data proportion, the SAE-V-based filter method signif- icantly surpasses the random filter, achieving a peak of 52.13 (120% of the performance on full dataset) with 70% data on Chameleon-7B and 116.67 (104% of the performance full dataset) with 40% data VA ---------------- Vicuna- 13B. The largest performance gap is observed in the 40-80% data range, while the differences converge again as the data pro- portion approaches 100%. This proves that SAE-V-based data filter is effective on architectures other than CLIP-based MLLM, and shows its potential to generalize across wide range of models.
Randen
Cosise
LFD
303-
Perfermece Score
300-
38
36-
20
20
30
40
50
50
30
90
100
Data Percentage (7)
Figure 11. The performance of SAE-V- data filter and IFD metrics We replicate the IFD metric on MILMS and compare the result with our cosine similarity filter. The results show that although the peak performance of SAE-V- babede data filtering is little lower compared to IFD metric, our method could achieve generally similar performance compared to IFD metric without introducing additional models and training process.
Ablation on Datasets We also performed an ablation study the datasets to be filtered. Since the NeXTT 7B model is highly capable. most datasets fail to further enhance its multimodal abilities. Therefore, we selected the RLAIF-V (Yu et al., 2024) and MMInstruct (Liu et al., 2024b) datasets with the relatively weaker Chameleon-7B model for the dataset ablation study. The results (shown in
Comparation with Other Filtering Methods To validate the effectiveness of our SAE- V-bseed data filtering method,
8

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
conducted ablation study comparing it with other similar data filtering approaches. Since there are currently no widely recognized data filtering methods specifically designed for multimodal data, we adapted the IFD metric (Li et al., 2024a) method to the multimodal setting. The result (shown in Figure 11) suggests that our data filter method achieves a performance comparable to the IFD met- ric. However, considering that the IFD metrie needs to train an additional cherry model, our SAE- V-based data filter could directly fit various datasets, demonstrating greater generalizability and efficiency.
firstly attempted apply SAE to analyze the open-semantic features of MILMS.
In this paper, we extended the scope of SAE to MLLMs, thereby building SAE-V. We further demonstrated SAE-V'S V's capability and transferability on MLLMs, and built data filter tool based on SAE-V to enhance multimodal alignment.
Data Filter in Alignment Data filtering ensures that only relevant high-quality data are used during the alignment of LLMs or MILMMs, thus reducing the quantity of data while achieving greater performance (Zhou et al., 2023; Chen et al., 2024; Du et al., 2023; Li et al., 2024b;a; Tu et al., 2025). For example, LIMA (Zhou et al., 2023), ALPAGA- SUS (Chen et al., 2024), and IFD (Li et al., 2024a) use human annotation, API annotation and train a new model for annotation to score data separately. Our method, SAE- V-based data filter, provides self-guided and interpretable metric to evaluate the similarity of multimodal data, which indicates their qualities. The method is stable and efficient for models of different architectures.
5. Related Work
Multimodal Large Language Model MILMM is a type of LLM integrated with multimodal modules that incorpo- rate multimodal information to deal with multimodal tasks. Based on the method of integrating vision features into the model, most MLLMs can be categorized into three types:
-I.e--based MILMMs These models encode images with CLIP (Radford et al., 2021) and use MLP to project visual features. Examples include LLaVA (Liu et al., 2024a) series and NEXT-GPT (Wu et al., 2024b).
6. Conclusion
This work introduced SAE-V, a framework that extends SAE to MLLMs and improves their alignment. Through experiments on LLaVA-NeXT-7B and Chameleon-7B, we demonstrated that SAÆ-V V model demonstrates excellent ca- pability and transferability in interpreting MLLM and mul- timodal data, and SAE- V-based data filtering methods could achieve more than 10% performance with less than 50% data. These results highlight SAE-V's V's potential to enhance multimodal model interpretability and alignment efficiently.
Early-Fusion MLLMs: These models directly tokenize visual features for input. Examples include Chameleon (Team, 2024) and Janus (Wu et al., 2024a) series.
Q-Former-based MLLMs: These models use a structure similar to Q-Former (Li et al., 2023) extract visual representations, represented by Qwen- VI. (Bai et al., 2023) and MiniGPT-4 (Zhu et al., 2024).
Limitation While SAE-V V introduces significant advance- ments in interpreting multimodal models and enhancing alignment through mechanistic analysis, several limitations remain unaddressed and warrant further exploration: (I) Although SAE-V V demonstrates superior interpretability and data filtering efficiency compared to SAF, the theory behind SAE-V, especially the mathematical relationship between image-text similarity metrics, cross-modal co-occurrence features, and model performance, is not fully revealed. (2) Due to resource constraints, SAE- V is primarily evaluated on text and vision modalities, leaving its effectiveness on other modalities such as audio, video, and embodied AI systems unexplored. Our future work will focus on estab- lishing a comprehensive theoretical foundation for SAE-V V and extending its application to additional modalities, such as audio, video, and embodied AI systems, to broaden its utility and impact.
Our study focuses on the CL.P--baedd and early-fusion MLLMs. Specifically, we select LLaVA-NeXT-7B and Chameleon-7B as the target models.
Mechanistic interpretability with Sparse Autoencoder Mechanistic interpretability seeks to uncover and explain the internal mechanisms that enable models understand input data and generate responses (Rai et al., 2024). Specif- ically, most current mechanistic interpretability methods focus on analyzing features, smaller units that contribute to performing explainable semantic tasks, within models (Olah et al., 2020).
Sparse Autoencoder (SAE) aims to learn sparse and inter- pretable features from polysemantic model representations (Yun et al., 2021; Bricken et al., 2023; Sharkey et al., 2022; Peigné, 2023; Elhage et al., 2022). By introducing sparsity constraints, the activation values in the hidden layers of SAE are mostly zero, allowing SAE to encode polysemantic features in LLM to monosemantic ones. Zhang et al. (2024)
9

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
Impact Statement
Black, S., Biderman, S., Hallahan, E., Anthony, Q. Gao. L., Golding, I.., He, H., Leahy, C.,, McDonell, K., Phang, J., Pieler, M., Prashanth, U. S., Purohit, S., Reynolds, L., Tow, J., Wang, B., and Weinbach, S. GPT-NeoX-2000 An open-source autoregressive language model. In Fan, A., Ilie, S., Wolf, T., and Gallé, M. (eds.), Proceedings of BigScience Episode #5 Workshop on Challenges Perspectives in Creating Large Language Models, pp. 95- 136, virtual+Dublin, May 2022. Association for Computa- tional Linguistics. doi: 10. 10..865//1220222issiieeeee 1.9, URL https::// aac///claaao secience-1.9/
The source code and checkpoints of SAE-V mentioned in this paper will be released under the CC BY-NC 4.0 li- cense. This research has several potential risks that must be considered. The interpretability tools introduced in this work, while beneficial for alignment, could also be lever- aged to manipulate or reverse-engineer model behaviors in unintended ways. Additionally, while SAE-V V provides a self- gelddd filtering mechanism, remains dependent on the initial dataset quality, meaning biases in the dataset could still propagate into the final model. We strongly condemn any malicious of the SAE-V V code and checkpoints and advocate for its responsible and ethical use.
Bricken, T., Templeton, A., Batson, J.,, Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., et al. Towards monosemanticity. Decomposing language models with dictionary learning. Transformer Circuits Thread, 2, 2023.
Acknowledgement
This work is sponsored by National Natural Science Foun- dation of China (62376013, 623B2003) and the Natural Science Foundation of Beijing (QY25124). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the funding agencies.
Chen, L., Li, S., Yan, J., Wang, H., Gunaratna. K., Yadav, V., Tang, Z., Srinivasan, V., Zhou, T., Huang, H., and Jin, H. Alpagasus: Training a better alpaca with fewer data. In The Twelfth International Conference on Learning Representations, 2024, URL https: openreview meet/forumidd=ad/e
References
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I,, Aleman, F. L.,, Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.0879 08077474.. 2023.
Chern E., Su, J.. Ma, Y., and Liu, P. Anole: An open, autore- gressive, native large multimodal models for interleaved image-text generation. arXiv preprint urXiv:2407.06133. 2024.
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I,, Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. Flamingo: a visual language model for few- shot learning. Advances in neural information processing systems, 1723776-237666 2022.
Du, Q. Zong, C., and Zhang. J. Mods: Model-oriented data selection for instruction tuning. arXiv preprint arXiv:2311.1565, 2023.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21780 2024.
Bai, J. Bai, S., Yang, S., Wang. S., Tan, S., Wang. P., Lin, J., Zhou, C., and Zhou, J. Qwen-vl: A versatile vision- language model for understanding, localization, text read- ing, and beyond. arXiv preprint 2r0iv:2308. 12966, 2023.
Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds. Z., Lasenby, R., Drain, D., Chen, C., et al. Toy models of superposition. arXiv preprint arXiv:2209.16653 2022.
Ben Melech Stan, G., Aflalo, E.,, Rohekar, Y., Bhiwandi- walla, A., Tseng, S.-Y., Olson, M. I.., Gurwiez, Y., Wu, C.. Duan, N., and Lal, V. Lvim-intrepret: An interpretabil- ity tool for large rision-languse models. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 8182-8187, June 2024.
Gallifant, J., Chen, S., Sasse, K., Aerts, H., Hartvigsen, T., and Bitterman, D. S. Sparse autoencoder features for classifications and transferability. arXiv preprint arXiv:2502. 11367, 2025.
Biderman, S., Schoelkopf, H., Anthony, Q. G., Bradley, H., O' Brienn, K., Hallahan, E., Khan, M. A., Purohit, S., Prashanth, U. S., Raff, E., et al. Pythia: A suite for ana- lyzing large language models across training and scaling. In International Conference on Machine Learning, pp. 2397-2430. PMLR, 2023.
Gao, I.., Biderman, S., Black, S., Golding, I.., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., et al. The Pile: An 800GB dataset of diverse text for language modeling. arXiv preprint arXiv:2101.000. 2020.
10

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
Gao, L., la Tour, T. D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., and Wu, J. Scaling and evaluating sparse autoencoders. In The Thirteenth International Conference on Learning Representations, 2025. URL. https:/ /ttps//oppeeeee opp.reveeeeee.eeo/ooee net/ forum #id=tcs2t922KK
One-shot learning as instruction data prospector for large language models. In Ku, I..-W., Martins, A., and Sriku- mar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 4586-4601, Bangkok, Thailand, Au- gust 2024b. Association for Computational Linguistics. URL https: walc-llongg 252/.
doi: 10. 10.186653/1122224ac---oooog org. www.://thhhooooy.....//o2./ //aclanthology
Huben, R., Cunningham, H., Smith, L.. R., Ewart, A., and Sharkey, L. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representations, 2024. URL net/ www.teeet//oiiieeeo Fd=
Liu, H., Li, C., Wu, Q. and and Lee, Y. Visual instruction tun- ing. Advances in neural information processing systems, 36, 2024a.
wRSLeK.
Liu, Y., Cao. Y., Gao, Z., Wang, W., Chen, Z., Wang, W., Tian, H., Lu, L.. Zhu, X., Lu, T., et al. Mminstruct: A high-quality multi-modal instruction tuning dataset with extensive diversity. Science China Information Sciences, 67(11):1-16, 2024b.
Ji, J., Zhou, J., Lou, H., Chen, B., Hong, D., Wang. X., Chen, W., Wang. K., Pan, R., Li, J., et al. Align anything: Training all-modality models to follow instructions with language feedback. arXiv preprint UrXv:22412.15838 2024.
Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., and Carter, S. 7oom in: An introduction circuits. Distill, 5()):c000240011, 2020.
Jiang, A. Q.. Sablayrolles, A., Mensch, A., Bamford, C.,, Chaplot, D. S.,, Casas, D. d. I., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. Mistral 7b. arXiv preprint 10.00:2310.06825, 2023.
Olsson, C.,, Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y., Chen, A., et al. In-context learning and induction heads. arXiv preprint arXiv:2209. 1895, 2022.
Kissane, C., Krzyzanowski, R., Conmy, A., and Nanda, N. Saes (usually) transfer between base and chat models. In Al Alignment Forum, 2024.
Peigné, P. Taking features out of superposition with sparse autoencoders more quickly with informed initialization, Sep 2023, URL https: www. lesswrong. com/ post s /ttp://DDAgg//HHHH////////////// res out of wuperpssitisooo..o withoonwwiitt-hm spares
Kutsyk, T., Mencattini, T., and Florea, C. Do sparse autoen- coders (saes) transfer across base and finetuned language models? In Al Alignment Forum, 2024.
Laurençon, H., Saulnier, I.., Tronchon, I.., Bekman, S., Singh, A., Lozhkov, A., Wang. T., Karamcheti, S., Rush, A. M., Kiela, D., Cord, M., and Sanh, V. Obelics: an open web-scale filtered dataset of interleaved image-text documents. NIPS '23, Red Hook, NY, USA, 2023. Curran Associates Inc.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PMLR, 2021.
J., Li, D., Savarese, S., and Hoi, S. 2lip-2: Bootstrapping Lnguuae-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pp. 19730 19742. PMLR, 2023.
Rai, D., Zhou, Y., Feng, S., Saparov, A., and Yao, Z. A practical review of mechanistic interpretability for wansformer-basee language models. arXiv preprint arXiv:2407.02644 2024.
Li, M. Zhang, Y., He, S., Li, Z., Zhao, H., Wang, J., Cheng, N., and Zhou, T. Superfilterig: Weak-to-strone ttronn data filtering for fast instruction-tuning Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), pp. 14255-14273, Bangkok, Thailand, August 2024a. As- sociation for Computational Linguisties. URL https: //aclanthology 202/ 1ollllloggg 769.
Rimsky, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., and Turner, A. Steering Ilama 2 via contrastive activa- tion addition. In Ku, L.-W., Martins, A., and Srikumar, V. (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 15504-15522 Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 10.186533w1/20044xcl-ooom 828. URL https: acl- long 828/
In Ku, L.-W.,
Li, Y., Hui, B., Xia, X., Yang. J., Yang, M., Zhang, L., Si, S., Chen, L., Liu, J., Liu, T., Huang, F., and Li, Y.
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheosh, S., Ma, S., Huang, Z.,, Karpathy, A., Khosla, A., Bernstein,

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
M., Berg, A. C., and Fei-Fei, L.. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211-252, 2015. doi: 10.1007/s1 1233-015-08161-1
Wu, S., Fei, H., Qu, L., Ji, W., and Chua, T.-S NExT- GPT: Any-to-any multimodal LLM. In Proceedings of the International Conference Machine Learning, pp. 53366-53397, 2024b.
Sharkey. L., Braun, D., and Millidge. B. Taking features out of superposition with sparse autoencoders, Dec 2022.
Wu, S., Fei, H., Qu, L., Ji, W., and Chua, T.-S. NEXt-GPT: Any-to-any multimodal L.M.. In Forty-first International Conference on Machine Learning, 2024c. URL https:
URL http:///wwwwll.gmeenffruu... ost2://60OOtttaa2XXooo//ttteteee
/ opoeereeeeew. net/ forum? LituummiddNNKOusNN..
rch- rch-report-taking wwwpkkiiing-euueeeeeeeeee features of-s uperposition. [Interim research report..
Xie, J., Mao. W., Bai, Z., Zhang, D. J., Wang. W., Lin, K. Q., Gu, Y., Chen, Z., Yang. Z., and Shou, M. Z. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408. /2528, 2024.
Team, C. Chameleon: Mixed-modal early- early-fusion foundation models. arXiv preprint arXiv:2405.0000 2024.
Yu, T., Zhang, H., Yao, Y., Dang, Y., Chen, D., Lu, X., Cui, G., He, T., Liu, Z., Chua, T.-S,, and Sun, M. Rlaif-v: Aligning milms through open-source ai feed- back for super gpt-4v trustworthiness. arXiv preprint arXiv:2405 102.0.1.72200 17220, 2024.
Team, G., Reid, M., Savinov, N.. Teplyashin, D., Dmitry, L., Lillicrap, T., Alayrac, J., Soricut, R., Lazaridou, A., Firat, O., et al. Gemini 1.5: Unlocking multimodal understand- ing across millions of tokens of context, arxiv [es. cl]. arxiv, 2024.
Yun, Z., Chen, Y., Olshausen, B., and LeCun, Y. Trans- former visualization via dictionary learning: contextual- ized embedding as linear superposition of transformer factors. In Agirre, E.,, Apidianaki, M., and Vulić, I. (eds.), Proceedings of Deep Learning Inside Out (DeeLIO): The 2nd Workshop on Knowledge Extraction and Integra- tion for Deep Learning Architectures, pp. 1-10, Online, June 2021. Association for Computational Linguistics. doi: 10. 10.18653/v1/2021 http:///1221111eeoooooo deelio- 1.1. URL https: //aclantholsgee org/ dwwlio-e.ele...e
Tu, Z. Meng, X., He, Y., Yao, Z., Qi, T., Liu, J., and Li, M. ResoFilter: Fine- aine-grained synthetic data filter- ing for large language models through parameer resonance analysis. In Chiruzzo, L., Ritter, A., and Wang, L. (eds.), Findings of the Association for Com- putational Linguistics: NAACL 2025, pp. 5414-5428, Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8-89176-195-77 URL https::t/// http:/alaanntttt www.annnnooooooorg///00000 ndings wtw.-aaac....... 299/.
Zhang, H., Li, X., and Bing, L. Video-LLaMA: An instruction-tun -tnned aidio-visual language model for video understanding. In Feng, Y. and Lefever, E. (eds.), Pro- ceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 543-553, Singapore, December 2023, Association for Computational Linguistics. doi: 10. 00.18663/v1/2223eemm 1p-demo.49. URL ht https: /tt//allaaa hology org /2023. www.mmnneeeeeeeeeem dem........ 49/.
Turner, A. M., Thiergart, L., Leech, G., Udell, D., Vazquez, J. J., Mini, U., and MacDiarmid, M. Steering lan- guage models with activation engineering. arXiv preprint arXiv:2308. 10248, 2023.
Wang, K. R., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, J. Interpretability in the wild: circuit for indi- rect object identification in GPT-2 small. In The Eleventh International Conference on Learning Representations, 2023. URL hwwps:ttp..............tt openreview net forum ?id NpVSNosVSN64411.
Zhang, K., Shen, Y., Li, B., and Liu, Z. Large multi-modal models can interpret features in large multi-modal models, 2024. URL https: arxiv. org/ abs/ /211..11.11 4982.
Wang, X., Zhang, X., Luo, Z., Sun, Q., Cui, Y., Wang. J., Zhang, F., Wang, Y., Li, Z., Yu, Q., Zhao, Y., Ao, Y., Min, X., Li, T., Wu, B., Zhao, B., Zhang, B., Wang, L., Liu, G., He, Z., Yang, X.. Liu, J., Lin, Y., Huang, T., and Wang, Z. Emu3: Next-token prediction is all you need. CoRR, abs/2409.
Zhang, Q.-.. and Zhu, S.-C. Visual interpretability for deep learning: a survey. Frontiers of Information Technology & Electronic Engineering, 19(1):27-39, 2018
2024. URL https: 2409. 18499.1889., 1869.
wrrgor////// /1///1..10 10. 08555///5iiii./ .20.ii..44.....
Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., YU, L., Zhang, S., Ghosh, G., Lewis, M., Zettlemoyer, L., and Levy, O. LIMA: Less is more for alignment. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https: / /pppreevivveec ew. net/ forum? d=KMMOKM2hhe..
Wu, C., Chen, X., Wu, Z., Ma, Y., Liu, X., Pan. Z., Liu, W., Xie, Z. Yu, X., Ruan, C., et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410. 13848, 2024a.
12

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
Zhu, D., Chen, J., Shen, X., Li, X., and Elhoseiny, M. MiniGPT-4: Enhancing vision-language understanding with advanced large language models. In The Twelfth International Conference on Learning Representations, 2024. URL. https::t///..........tt wwwp/eeeeeeee.eee///ee net/ forum ?id-1t2bq88E27.
Zou, A., Phan, L., Chen, S., Campbell, J., Guo, P., Ren, R., Pan, A., Yin, X., Mazeika, M., Dombrowski, A.-K., et al. Representation engineering: A top-down approach to ai transparency. arXiv preprint arX00:3310001005, 2023.
13

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
A. Details of Interpretability Experiment
Ranking takes an initial step toward cross-modal consid- eration by only counting features that are activated across both modalities. Building upon these algorithms, we further developed the cosine similarity score Ranking approach.
A.1. Hyperparameter of SAE and SAE-V Models Training
SAE and SAE-V of MLaVA-NoT/MMistrll
SAE and SAE-V of Chameloo//Anole
Hyper-parameters
Algorithm 2 Lo-based Ranking
Training Parameters
Require: multimodal dataset DD = [d MILM Ma: SAE-V So; features of SAE-V; activation bound: 5;
total training steps
30000
30000
batch size
4096
4096
5e-5
Fo: {
LR warmup steps
1500
1500
decay steps
6000
6000
Ensure: Ranked data DR
adam betal
0.9
0.9
for each d; E DD do
adam beta2
0.999
0.999
LR scheduler name
constant
constant
Zi - So(Mo(d.))
LR coefficient
5
F; - {fk: 22 E Zi, s.t. 2j activates f&)
seed
42
dtype
32
floar32
Co.; f |Fi
buffer batches num
32
end for
store batch size prompts
16
leature sumpling window
1000
1000
DR + Sort {fo,)?=
dead feature window
1000
1000
dead feature threshold
le-4
le-4
SAE and SAB-V Parameters
book layer
16
input dimension
4096
4096
Algorithm 3 Co-ocurring Co-based Ranking
expansion factor
16
feature number
65536
131072
Require: Text token vocabulary: T; vision token vocab-
context size
4096
2048
ulary: V; multimodal dataset D = [d.)]= MLLM Ma:
SAE-V Sy; features of SAE-V Fe: (fs 1a 1= activation
Table Hyperparameters of training SAE and SAE-V models.
bound: d;
The hyperparameters of the training SAE and SAÆ-V are shown in Table 4. The differences in training parameters arise because the LLa LLaVA-NeXT-7B model requires more GPU memory handle vision input, fewer batches can be cached. For the SAE and SAE-V V parameters, we set dif- ferent hook layers and context sizes based on the distinct architectures of the two models. We also experimented with different feature numbers on both models, but found that only around 30,000 features are actually activated during training. All training runs were conducted until convergence. All SAE and SAE-V training is performed on 8xA800 GPUs and each training typically takes around 21 hours. We en- sured that the variations in the parameters did not affect the experiment results.
Ensure: Ranked data DR
Initialize coocurrence feature set of data F; + Ø for each d, E D do
Initialize activated token set of features As + Ø
H4 + (M))
Z, + SH,
for each fü E Fo do
A + A U (hy h2 E =
Zzb > 26}
if A T + 2 xA A NV+ Ø then
end if F.U
end for end for DR + SDri(D, (111))))
B. Details of alignment experiment
Hyperparameters of Algorithms 1,2,3 The hyperparam- eters of Algorithms 1,2.3 are shown in Table 5. We ensure that all parameters are the same to ensure a fair comparison between the algorithms.
We present details of alignment experiment in this section, including algorithms and hyperparameters of algorithms and model training.
B.I. Algorithms in alignment experiment
Hyper-parameters top
Cosine similarity
Concurrence
Co
The complete algorithm of Co-based Ranking and co- occurring based Ranking are shown in Algorithm 2 and Algorithm 3. These two algorithms serve as ablation vari- ants of the cosine similarity score Ranking (shown in Al- gorithm 1). The Lo based Ranking represents a straightfor- ward algorithm that selects data by directly computing the sum of Lo for each data point. The co-occurring Lo based
5
text token vocabolary size vision token vocabulary size activation bound sample data size
32000
32000
32000
64
64
64
1000
1000
1000
Table 5. Hyper-parameters of Algorithm 1,2,3.
14

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
B.2. Hyperparameter of Model Training
Algorithm 5 Li patch filter
Require: Vision token vocabulary: V; image V; fixed
In this section, we list out the hyperparameters used for model training through SFT and DPO (shown in Table 6). All SAE training is performed on 8xA800 GPUs. To en- sure fair comparison between algorithms, we maintained consistent parameter settings across all experiments.
Prompt T; MLLM Me; SAE-V Sg; features of SAE-V
Fo: ffx Ri= activation bound ò; mask rate 7
Ensure: Filtered image V"
Initialize score of each patch p. + 0
H < MM(MTTV]
Z < (H) for each h; E H do
Hyper-puaramelrs
SET
DPO
if h4 E V then
max length
4096
4096
device train batch size per device eval batch size
8
Pi (z())) end if end for K + [a74
8
gradient accumulation steps
IR scheduler type
cosine
cnssiee
le-6
lc-6
warmup stops
10
10
eval steps
50
50
V + TopK(u; E V) sorted by Pa
epochs
3
val size bl16
0.1
0.1
True
Table Hyperparameters of SFT training and DPO training.
These algorithms take images as input and produce masked images, where the masking proportion is determined by the mask rate y. All algorithms utilize the activation patterns of SAE-V features for patch filtering, with their primary distinctions lying in their methods of computing fcature activation (Co, L1) and measuring cross-modal similarity (co-occurring Co, cosine similarity score).
C. Details of Applying SAE-V on Multimodal Data
In this section, we present implementation details of the SAE-V application experiments. We enumerate 4 image patch selection algorithms employed in this study and pro- vide additional case analyses. These comprehensive results further demonstrate the robust inference capabilities of SAF- V.
Algorithm 6 Co-occuring Lo patch filter
C.1. Algorithm
Require: Text token vocabulary T; vision token vocabu-
lary: V; image V; fixed Prompt T:; MLLM Ma; SAE-V
The complete algorithms of Lo. Li, co-occurring Lo, and cosine similarity score methods are shown in Algorithm 4, Algorithm 5, Algorithm 6 and Algorithm 7.
Sg; features of SAE-V V ( activation bound o; mask rate 7
Ensure: Filtered image V
Initialize score of each patch p; < 0, co-occuring feature set F Ø and activated token set of features A₁ + Ø
Algorithm 4 Lo patch filter
H < Ma (T
Require: Vision token vocabulary: V; image V; fixed Prompt T: MLLM Mg; SAE-V Sg; features of SAE-V Fg: } activation bound 6; mask rate y;
Z < Sa(H)
for each f, E Fg do
A < A₁ U (hę h E H. = e, > 8)
Ensure: Filtered image V
A n T # Ø A A + Ø then
Initialize score of each patch p; +- 0
F F+ (f,)
H + Ma(T, V)
end if end for for each h, E H do
Z (H) for each h, E H do
if h, E V then
if h4 E V then
P: = Σj 1(zi > 8)
P: E, 1a2 >8 2 S3 FF
end if
end if end for
end for
K= + |>|4|
K |W4
V TopK(u; E V) sorted by p.
V' f- TopK(u: V) sorted by pt
15

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
Algorithm 7 Cosine similarity score patch filter
We present 5 cases filtered with the cosine similarity score method in Figure 13. The results show that SAÆ-V model performs excellently in capturing critical patches in images.
Require: Text token vocabulary Т; vision token vocabu- lary: V; image V; fixed Prompt Т;; MLLM Ma: SAE-V So; features of SAE-V For jfffy"yii activation bound 8; mask rate 7:cosine similarity weight {wy)
Ensure: Filtered image V';
Initialize score each patch P; < 0, co-occuring feature - 2 and activated token of features Aj f-
H MMa(T
Z ())
for each fj E Fo do
A₁ + A, U (hę h, E = > 6}
A T + Ø A NV 2 then
F FU
end if end for for each h2 E H do
if h; E V then
= Σj (zz > 8 A f, F)
end if
end for K + [
V' TopK(u, E V) sorted by p.
C.2. Case Analysis
Original
Top
Top Top 2.5%
We present 4 cases in Figure 12, corresponding to each of our metric in Section 3.2. The cases intuitively show that Co method and cosine similarity score method are more ca- pable identifying significant patches in images compared to other methods, which aligns with the quantized results shown in Figure 7.
Figure 13. Case Analysis of cosine similarity score method in
Section 3.2. 5 cases filtered with cosine similarity method are shown the Figure. Each case contains contains 4 images as original image, preserving top 75% patches, top 50% patches and top 25% patches.
C.3. Quantative Analysis
Masking (%)
0
25
50
75
Text Accuracy (%)
80.2
77.7
66.5
53.3
(c) E mothod
Image Accuracy (%)
80.2
78.8
78.4
70.7
Table 7. Applying SAE-V on VQA tasks We apply SAE-V-based patch filtering experiment the text and image of VQA task sep- arately, using A-OKVQA validation set with -LaVA-NeXT-P model. The experiment shows that image information demon- strates a lower compression rate (more redundancy) than text. Moreover, text masking shows roughly linear relation of accu- racy and masked token, while image filter maintains performance until 50% with more significant drop only appearing 75%, suggesting that the information of text is more evenly distributed compared to image.
(al Original image
(d) Co-occuriin 4, method
Ep method
(e) Cosine similartty score method
Figure 12. Case Analysis of all image patch filtering methods in Scction 3.2. We present the original image (a) and 4 case for methods, Lo (b), Li (c), co-occurring Co (d) and cosine similarity score (c). Each case contains 3 images preserving top 75% patches, top 50% patches and top 25% patches.
We performed additional experiments on SAE-V-based patch filtering. To be specific, we use SAE- based patch filter on the text and image part of each VQA question sepa-
16

---

SAE-V: Interpreting Multimodal Models for Enhanced Alignment
rately, and test the accuracy when part of the information is masked. The results show that image information demon- strates lower compression rate (more redundancy) than text. Moreover, text masking shows roughly linear relation of accuracy and masked token, while image filter maintains performance until 50%, with a more significant drop only appearing at 75%, suggesting that the information of text is more evenly distributed compared image.
17


Reviews:
[{"review_id": "keJurBmkNI", "reviewer": "Reviewer_b89F", "summary": "This paper introduces SAE-V, a framework that extends Sparse Autoencoders (SAEs) to multimodal large language models (MLLMs). The authors argue that MLLMs present unique interpretability challenges due to the complex semantic space created by integrating visual modalities with text. SAE-V aims to address these challenges by identifying and analyzing interpretable features in MLLMs, focusing on cross-modal interactions and alignment dynamics. The authors demonstrate that SAE-V can be used to filter high-quality data for model alignment, achieving comparable or better performance with significantly less data. They conduct experiments on multiple MLLM architectures (LLaVA-NeXT-7B and Chameleon-7B) and datasets (Align-Anything and RLAIF-V) to validate their approach.\n\n## update after rebuttal\nThe authors' response has resolved my concerns. After reading other reviewers' comments, I think this paper is above the threshold of the acceptance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# We deeply appreciate your thoughtful insights that will significantly strengthen our paper's overall presentation.\n\nIn the rebuttal period, we used all available resources and devoted efforts to conduct additional experiments. We addressed all your negative comments below and will add them into the revision. **If this rebuttal addresses your concerns, we earnestly and kindly ask you to consider raising the score and supporting us for acceptance.**\n\n## W1: We defined these concepts with specific metrics, and we acknowledge that more rigorous definitions would strengthen our presentation.\n\nWe define a feature as `interpretable` when it activates for semantically related inputs across modalities. This definition is aligned with the automated interpretability score[1], but since there are currently no benchmarks for multimodal interpretability, we would display an example instead. (e.g., [Rebuttal Fig. 1](https://github.com/saev-2025/Imagebed/blob/main/rebuttal_figure_1.pdf) shows an `interpretable` feature with its semantic meaning being consistent.) \n\nAs for `quality`, a feature is of `high quality` when it activates text and image patches that are semantically similar (given by Eq. 7), focusing more on the similarity across modalities instead of overall consistency. The feature in the example above would also be of `high quality`.\n\n\n## W2: We provided examples for the failure modes, as well as statistical evidence supporting that these failure modes are scarce.\n\nWe agree that there are failure cases where SAE-V didn't behave well on multimodal data, and here is an example: [Rebuttal Fig. 2](https://github.com/saev-2025/Imagebed/blob/main/rebuttal_figure_2.pdf). In this example, SAE-V fails to capture the most informative patches because of its similarity with the background.\n\nHowever, we need to mention that SAE-V is effective in most cases. Shown in Fig. 6, all SAE-V-based methods achieve high accuracy when preserving 75% or 50% patches, and cosine similarity score method maintains high accuracy even when only 25% patches are preserved. For more baselines and ablations, see the rebuttal of **Reviewer bBFh Weakness 3** and **Reviewer o9Vp Claim 2**.\n\n## W3&W5: We tested our method on additional models to prove that our method could generalize and scale up.\n\nTo prove that SAE-V and its data filtering paradigm could generalize to other multimodal models and scale up to larger models, we replicated SAE-V and its data filtering method on LLaVA-NeXT-Vicuna-13B and LLaVA-NeXT-Vicuna-7B (in the paper, we used LLaVA-NeXT-Mistral-7B). Unfortunately, due to rebuttal time constraints and compute limitations, we were unable to test our method on larger models and more architectures, and we only tested our data filter method in a 5-fold manner rather than 10-fold in the paper.\n\nThe interpretability metrics of SAE and SAE-V on both models are shown in the table below:\n\nModel|Method|L0|\n-|-|-\nLLaVA-NeXT-Vicuna-13B|SAE|128.56|\n||SAE-V|193.63|\nLLaVA-NeXT-Vicuna-7B|SAE| 3162.96|\n||SAE-V|585.64|\n\nModel|Method|Reconst.\n-|-|-\nLLaVA-NeXT-Vicuna-13B|Zero|10.37\n||SAE| 3.170\n||SAE-V|**2.954**\n||Original|2.868\nLLaVA-NeXT-Vicuna-7B|Zero|10.37\n||SAE| 8.126\n||SAE-V|**7.957**\n||Original| 7.479\n\n\nIn **all** MLLMs and metrics, SAE-V consistently outperforms SAE, demonstrating its superior capability across different architectures, sizes, and semantic complexity. \n\nThe alignment experiment results of SAE-V are shown in the table below:\n\nLLaVA-NeXT-Vicuna-13B:\n\nFilter Method|LLaVA Bench Performance @ 0%|20%|40%|60%|80%|100%\n-|-|-|-|-|-|-\nSAE-V Cosine Similarity|104.60|105.77|**116.67**|112.27|111.96|111.20\nRandom|104.60|105.27|105.77|107.20|110.40|111.20\n\nThe results show that SAE-V-based data filter outperforms the random selection baseline, and reached the highest performance of `116.67` with `40%` data.\n\nWe believe that our experiments across 7B, 13B models and three different architectures (Chameleon, LLaVA-NeXT-Mistral, and LLaVA-NeXT-Vicuna) provide sufficient evidence to prove SAE-V's potential to generalize across architectures, scale to larger models, and handle more complex semantic spaces. We commit to adding experiments on at least one 30B-scale model in future versions of our work.\n\n## W4: We reported our training time and computation cost, and demonstrated the effectiveness of SAE-V.\n\nOur SAE-V training was completed on 8\\*A800 GPUs. Using 100k multimodal data samples, each training typically takes around `21` hours, which is comparable to training an SAE on 7B model using the same amount of data. \n\nThe effectiveness of SAE-V lies in its `generalization capabilities`: as reported in Section 3.1.2 and Fig. 4, SAE-V trained on MLLMs demonstrates strong generalization capability to the corresponding LLMs. Therefore, a single training can yield an SAE-V that is applicable to both multimodal models and text-only models.\n\n\n## Reference\n\n[1] Bills et al., \"Language models can explain neurons in language models\", 2023.", "claims_and_evidence": "The main claims of the paper are:\n1. The authors demonstrate this through reconstruction loss metrics, showing that SAE-V outperforms standard SAE models when applied to MLLMs.\n2. The authors show that SAE-V models trained on MLLMs can be effectively applied to their base LLMs.\n3. Through image patch filtering experiments, the authors demonstrate that SAE-V can identify the most important parts of an image.\n4. The authors show that data filtered using SAE-V features can achieve better performance with less data compared to random selection or using the full dataset.\n\nThe evidence presented includes quantitative metrics (reconstruction loss, L0 sparsity, model performance on benchmarks) and qualitative analyses (case studies of image patch filtering).", "methods_and_evaluation_criteria": "SAE-V (Sparse Autoencoder for Multimodal Models) is proposed for interpretability and data filtering in MLLMs. It contains:\n1. Sparse Autoencoders (SAEs) extract interpretable multimodal features.\n2. Cosine similarity ranks data quality for filtering.\n3. Filtered data improves model alignment efficiency.\n\nThe paper adopts several benchmarks for evaluation:\n1. Align-Anything\n2. RLAIF-V\n3. ImageNet", "theoretical_claims": "The main claims of the paper are:\\\n1. The authors demonstrate this through reconstruction loss metrics, showing that SAE-V outperforms standard SAE models when applied to MLLMs.\n2. The authors show that SAE-V models trained on MLLMs can be effectively applied to their base LLMs.\n3. Through image patch filtering experiments, the authors demonstrate that SAE-V can identify the most important parts of an image.\n4. The authors show that data filtered using SAE-V features can achieve better performance with less data compared to random selection or using the full dataset.", "experimental_designs_or_analyses": "The paper includes several experimental components:\\\n1. Comparing reconstruction capabilities of SAE-V versus standard SAE on multiple models.\n2. Testing various metrics derived from SAE-V (L0, L1, co-occurring L0, cosine similarity) to identify important image patches.\n3. Using SAE-V features to filter high-quality data for model alignment, comparing against random selection and IFD metric.\n4. Examining the relationship between average cosine similarity scores and model performance.", "relation_to_broader_scientific_literature": "The paper positions itself at the intersection of mechanistic interpretability and multimodal model alignment. It builds upon previous work in sparse autoencoders for LLM interpretability and extends these approaches to multimodal settings. The authors compare their approach to recent data filtering methods like IFD, showing comparable results without requiring additional models.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "strengths:\\\n1. The paper presents a clear contribution by extending SAE techniques to multimodal models, which is valuable given the growing importance of MLLMs.\n2. The authors demonstrate a concrete application of their interpretability method (data filtering) that improves model alignment, connecting theoretical interpretability to practical benefits.\n3. The experiments cover multiple models, datasets, and evaluation metrics, providing strong evidence for the effectiveness of the proposed approach.\n\n\nweaknesses:\\\n1. The process of determining which features are considered \"interpretable\" or \"high-quality\" is somewhat subjective and could benefit from more rigorous definition.\n2. The image patch filtering experiments may be subject to confirmation bias - examples are chosen where the method works well, but it's unclear how often the method fails to identify important regions.\n3. The experiments are limited to 7B parameter models. It's unclear if the findings would generalize to larger models where the semantic spaces may be even more complex.\n4. SAE-V claims to be efficient, but training times and computational costs are not reported.\n5. The results might not generalize to other multimodal models since only two models are tested.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JcNjzG8up3", "reviewer": "Reviewer_bBFh", "summary": "This work aims to improve the vision language alignment performance of multimodal foundation models by finetuning data selection and filtering using interpretable tools, i.e., improved SAE. Specifically, it uses the alignment scores between selected topK vision-language tokens determined by SAE to select the finetuning dataset subset, and then perform a series of investigations. In the experimental results, this paper investigates the reconstruction error trend w.r.t. dataset size, and different data filtering strategies and metrics. The results show that the proposed approach can improve performance w.r.t. entire-dataset baseline and minorly improve performance compared with the previous filtering approach.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Despite some misunderstandings, we conducted more experiments to clarify your valuable concerns.\n\nWe conducted additional experiments, addressed your comments, and would add them into revision. **If this rebuttal addresses your concerns, we kindly ask you to consider raising the score.**\n\n## Methods & Evaluation: We added additional paragraph to clarify this potential confusion.\n\nIn Eq. 1, we define the operation $S_{\\theta}(\\cdot)$ as the encoding operation of the SAE-V, whereas $Z_i \\in R^{\\ l\\times n}$ is the feature activation, $l$ is the length of the input to SAE-V, and $n$ denotes the number of features in the SAE-V. Each $z_j \\in R^{1\\times n}$ represents the activation of a specific token (index $j$) across all features in the SAE-V.\n\n## Theoretical Claims: We sincerely apologize for the mistake.\n\nThanks for pointing this out! Based on our definitions, Eq. 1 should be: $Z = ReLU(H\\times W_{enc} + b_{enc})$. We updated the paper accordingly.\n\n## Broader Literature: We perform experiments to show the scalability of SAE-V empirically.\n\nRegarding [1], we acknowledge that reduced reconst. loss alone cannot guarantee meaningful features. However, our work empirically validates SAE-V through image patch filtering (Section 3.2), and data filter & alignment (Section 4). \n\nWhile the theoretical analysis of underlying mechanism remains an open question for future work, we conducted experiments to show that SAE-V has the potential to scale up to larger models and datasets. For more details, see the rebuttal of **Reviewer b89F W3&W5** and **Reviewer k2no point \\#4**.\n\n## References: We acknowledge the novelty of [2], though we are making distinct contributions.\n\nWe acknowledge that [2] is a pioneering effort in applying SAE for mechanistic interpretability of VLMs, and we modified our claims accordingly and added [2] into the related works subsection. Compared with [2], our work provides additional insights and extends the practical applications of mech interp. For more details, see the rebuttal of **Reviewer k2no point \\#3**.\n\n## W1&2: We prune the overall presentation of the paper to reduce repetitive expressions and add necessary backgrounds.\n\nThanks for your suggestions! We have modified the paper accordingly. For some decent examples, see the rebuttal of **Reviewer o9Vp Claim 1**.\n\n## W3: We conducted deeper analysis of the underlying mechanisms.\n\nWe replicated the patch filtering experiments in Section 3.2 on VQA tasks (using A-OKVQA val. set with LLaVA-NeXT-7B), examining both text tokens and image patches:\n\nMasking (%) |0|25|50|75\n-|-|-|-|-\nText Acc. (%)|80.2|77.7|66.5|53.3\nImage Acc. (%)|80.2|78.8|78.4|70.7\n\nKey Findings\n\n- **Compression rate**: Image information demonstrates lower compression rate (more redundancy) than text. \n- **Accuracy scaling w/ reduced tokens**: Text masking shows a roughly linear relation of accuracy and masked token, while image filter maintains performance until 50%, with a more significant drop only appearing at 75%, suggesting that the information of text is more evenly distributed compared to image.\n\nFor interpretability beyond reconst. score, see the rebuttal of **Reviewer b89F W1**.\n\nFor more datasets, see the rebuttal of **Reviewer k2no point \\#4**.\n\n## W4: We presents adaptive parameter selection strategies to make our methods effective and pratical.\n\nThank you for raising this consideration! We tested hyperparameter selection using a 1/20 subset of Align-Anything dataset:\n\nFilter Method|LLaVA-Bench Performance @ 0%|10%|20%|30%|40%|50%|60%|70%|80%|90%|100%\n-|-|-|-|-|-|-|-|-|-|-|-\nSAE-V Cosine Similarity|94.2|98.2|106.8|114.9|114.5|114.8|112.9|112.3|111.0|109.5|98.5\nRandom|94.2|96.5|97.0|98.3|95.3|93.7|96.5|98.83|98.2|96.8|98.5\n\nThe overall trend closely resembles Fig. 7, confirming that alignment metrics on a small val. set resemble the distribution on the complete dataset, enabling efficient hyperparameter selection.\n\nAdditionally, Section 4 have showed that our method outperforms using the complete dataset across most hyperparameter settings, making more refined parameter tuning (such as naive elbow) helpful but not strictly necessary.\n\n## W5\n\nWhile we don't provide formal guarantees, our method is grounded in established principles of dictionary learning that have been validated by the community. We also performed comprehensive ablation study and open-sourced our code to confirm replicability. (see rebuttals above).\n\n## Question\n\nThanks for pointing out! As is shown in the supp. material (code/SAELens-V/scripts/cosimilarity.py, line 179-189), we actually used point-to-point cosine. We modified Eq. 7, and we verified that pair-wise consine performs similarly as the point-to-point in actual dataset selection:\n\nFiltered Top(%)|25|50|75\n-|-|-|-\nIoU|0.71|0.77|0.85\n\n## Reference\n\n[1] Heap et al. Sparse Autoencoders Can Interpret Randomly Initialized Transformers, 2025.\n\n[2] Zhang et al. Large Multi-modal Models Can Interpret Features in Large Multi-modal Models, 2024.", "claims_and_evidence": "The *interpreting* in the title is a little concerned since most interpretability experiments done are reconstruction probing and limited case studies. I would suggest authors either rephrase the title or add additional explainability experiments to support it. This title, at my first glance, tells me this paper must include a comprehensive list of vision-language qualitative explainable alignment results.", "methods_and_evaluation_criteria": "The method presentation is unclear. What is the function of $\\mathcal{S}_\\theta$ in its representation? Is it an SAE encoder or decoder or entire network? Then what is the $Z_i$ in its representation with its shape undefined.", "theoretical_claims": "Some minor mistakes:\n\n- Eq 1, I think the matrix multiplication order is wrong.", "experimental_designs_or_analyses": "All related questions are included in Other Strengths And Weaknesses.", "relation_to_broader_scientific_literature": "I understand the ICML reviewing policy on concurrent works. However, considering the whole community is moving so fast, to the most updated information, if thinking in a broader sense, I consider the significance of this work might be affected by this recent paper:\n\n*Sparse Autoencoders Can Interpret Randomly Initialized Transformers,* which points out that reduced reconstruction loss cannot guarantee the meaningfulness of learned patterns or features.\n\nOn the other hand, the improved alignment performance has been validated which turns out the effectiveness of SAE-based approach. However, an in-depth investigation of the underlying working mechanism is lacking.\n\nThe proposed SAE-based data filtering approach is purely empirical-driven. It is understandable that the experiments is not that large-scale due to the high computational cost. Whether and how will this method generalize to larger-scale models and datasets is unclear with its underlying working mechanism left as a mystery.", "essential_references_not_discussed": "Missing a reference:* Large Multi-modal Models Can Interpret Features in Large Multi-modal Models*, which is the first mechanistic interpretability work using SAE on the VL foundation models. I think this may relate to the significance of the first claimed contribution in this paper. However, it is still acceptable to ignore, since it is not formally published.", "other_strengths_and_weaknesses": "strengths:\n\n- The overall method is novel in some senses, intuitive and simple, and effective.\n- The results are strong to support the effectiveness of the proposed approach.\n\nweaknesses:\n\n- This paper may not be that easy-to-follow for those who are unfamiliar with the mechanistic interpretability due to lack of introduction of motivation behind the operations or experimental settings, e.g., I think it is necessary to introduce the motivation and benefits of studying model transferrability of SAE (line 245) in the multimodal cases, or at least give some references, instead of repetitive descriptions of phenomenon (line 245-253). I suppose the readers coming from intersectional fields (both VL alignment and mechanistic interpretability).\n- The overall presentation of this paper still needs to be refined. As mentioned above, some expressions are not informative and even repetitive to readers. Please try to condense the sentences.\n- Some experiments only present findings, results, and speculations, and there is a lack of in-depth investigation and analysis beyond unveiling the trend: For example, in section 3.2, how would the classification accuracy vary scaling with reduced tokens? Comparing with compressing at token level and image level, how much is the gap? How much interpretable is the approach beyond the reconstruction error? Experiments on more finetuning datasets are appreciated.\n- Some questions related to the methodology: I find the variations of performance w.r.t. the data percentage is significant, how to adaptively select the hyperparameter threshold $\\eta$ and percentage in practice when the fine-tuning dataset is very large (original dataset is 400K). Do we need a hold-out validation set to train multiple rounds to search the parameters? We typically want to avoid that since our original goal is to train on only a small subset of FT dataset. Besides, when we search for optimal or near-optimal hparams, a reasonable strategy is to look ahead only say $k$ steps of the proportion (since we want to avoid exhaustive search), in this case, how would your findings guide the practice, e.g., naive elbow algorithm?\n- Lack some theoretical insights, foundations, or guarantees supporting this method, which I think can be fine.\n\nquestion:\n\n- Why eq 7 is pair-wise cosine not point-to-point cosine (fully connected bi-graph)?", "other_comments_or_suggestions": "None.", "questions_for_authors": "I will raise scores if some of my key questions are addressed.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7SiSDl6F0D", "reviewer": "Reviewer_k2no", "summary": "- This paper straightforwardly extends the SAE framework to MLLMs, calling it the SAE-V framework.\n- The authors introduce he cosine-sim scores as the cosine-sim b/w the TopK activated image and text features for a given input based on SAE activations.\n- Based on the cosine-sim scores, the authors filter training datasets for MLLMs and find a correlation between the the performance and avg cosine-sim score for a filtered dataset.\n- Using filtering they find only a fraction of data can boost performance.\n- The SAE-V extends to LLMs well.\n\n## update after rebuttal\n\nI will keep my rating as \"weak accept\" due to:\n\n- the authors test only on LLaVA-Bench and MME, which are not great benchmarks. I would have liked to see more benchmarks like MMStar, POPE, etc.\n- Pretraining a 7B llava-next model does not take 100s of GPU days since you use to the pretrained LLM, so you only do the multimodal PT and IFT which should take no more than a week for the 7B model, I would have liked to see that experiment.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Thanks for your valuable suggestion!\n\nDuring rebuttal period, we used all available resource and devoted efforts to conduct additional experiments. We addressed all your negative comments below and will add them into the revision. **If this rebuttal addresses your concerns, we earnestly and kindly ask you to consider raising the score and supporting us for acceptance.**\n\n## point 1\n\n> I am unsure what benchmarks authors use to report the MLLM performance, so I would like clarification. \n\nWe used LLaVA-Bench[1] to report the MLLM performance, and we also test our method on MME benchmark. Here is the result on LLaVA-NeXT-7B and Align Anything dataset:\n\nMME:\nFilter Method|LLaVA-Bench Performance @ 0%|10%|20%|30%|40%|50%|60%|70%|80%|90%|100%\n-|-|-|-|-|-|-|-|-|-|-|-\nSAE-V Cosine Similarity|1233.87|1088.48|1115.41|1161.13|1211.46|**1360.43**|1246.26|1231.91|1276.10|1148.70|1246.26\nRandom|1233.87|1069.52|\t1132.25|1072.34|1156.12|1220.76|1289.27|1292.73|1198.32|1217.40|1246.26\n\nThis result demonstrates that the superiority of SAE-V is not effected by the selection of benchmarks.\n\n## point 2\n\n> Why do they only try to fine-tune the LLaVA-NeXT-7B model and not pretrain and fine-tune the model from scratch? This is important to know how does the dataset filtering affect different training stages.\n\nWe appreciate the reviewer's suggestion about pretraining from scratch. While we fully agree this would provide valuable insights into how our filtering method affects different training stages, the computational resources required for pretraining MLLMs from scratch are prohibitively expensive for academic research team like us.\n\nPretraining even a 7B parameter model requires hundreds of GPU-days on A100/H100 GPUs, which is unfortunately beyond our current resource constraints. Instead, we focused our experiments on fine-tuning existing models, which still demonstrates the effectiveness of our approach while being computationally feasible (for more details supporting this claim, see the rebuttal of **Reviewer b89F W3&W5** and **Reviewer k2no point \\#4**.). We commit to conduct at least one experiment at pretrain stage in the future versions of our paper.\n\n## point 3\n\n> Extending SAE to MLLMs is nothing technically innovative.\n\nWhile extending SAE to MLLMs may appear straightforward, our work contributes several key innovations:\n\n- **Cross-modal feature analysis:** We've developed novel methods to identify and analyze features that capture cross-modal interactions (Section2.2).\n- **Interpreting Multimodal Alignment:** Our framework provides unique insights into how MLLMs integrate information across modalities during the alignment process. As shown in Section 3.1.2, SAE-V reveals patterns in feature distribution that directly correspond to model performance on multimodal understanding tasks.\n- **Self-guided data filtering:** Our paper made the first attempt to use mechanistic interpretability methods to perform multimodal data filter using the model's own representations.\n\nOur work provides additional insights and extends the practical applications of multimodal interpretability methods, which is also the main contribution of our paper.\n\n## point 4\n\n> The findings about the cosine-score and performance are interesting, but I'd have liked to see more models and more datasets being used for analysis.\n\nWe conducted experiments on larger models (LLaVA-NeXT-Vicuna-13B) and datasets (MMInstruct) during rebuttal period. \n\nFor experiment on larger models, see the rebuttal of **Reviewer b89F W3&W5**.\n\nFor experiments on larger datasets, we selected MMInstruct[2], which contain 200k data pieces, and is 4 times bigger than Align Anything dataset. Based on MMInstruct, we applied SAE-V-based data filter and alignment on Chameleon-7B, and the results are shown below:\n\nFilter Method|LLaVA-Bench Performance @ 0%|20%|40%|60%|80%|100%\n-|-|-|-|-|-|-\nSAE-V Cosine Similarity|42.6|48.1|57.6|**61.2**|54.7|52.3\nRandom|42.6|47.4|46.8|51.2|54.8|52.3\n\nIt demonstrates that SAE-V paradigm could scale up to larger models and datasets, while maintaining its performance.\n\n## Reference\n\n[1] Liu et al. Visual instruction tuning, 2023.\n[2] Liu et al. Mminstruct: A high-quality multi-modal instruction tuning dataset with extensive diversity, 2024.", "claims_and_evidence": "- The authors claim SAE-V can identify important semantic patches in the image, which is validated.\n- The data filtration technique is also shown to work well.", "methods_and_evaluation_criteria": "- Yes, using LLaVA-NeXt and Chameleon for experiments makes sense.\n- I am unsure what benchmarks authors use to report the MLLM performance, so I would like clarification.", "theoretical_claims": "- N/A", "experimental_designs_or_analyses": "- Yes, the training and evaluation of SAE-V models seem okay.\n- The dataset used to train the MLLM also seem fine.\n- One question I have for the authors is: why do they only try to fine-tune the LLaVA-NeXT-7B model and not pretrain and fine-tune the model from scratch? This is important to know how does the dataset filtering affect different training stages.", "relation_to_broader_scientific_literature": "SAEs is an established technique in LLMs, and extending it to MLLM is only natural and of interest to the community.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "The paper doesn't seem to have any major issues except the fact that the scope seems quite limited:\n\n- Extending SAE to MLLMs is nothing technically innovative.\n- The findings about the cosine-score and performance are interesting, but I'd have liked to see more models and more datasets being used for analysis.\n\nStill, I believe it's a good paper that deserves a weak accept but a more thorough analysis section of how the findings can be used by the community would make it a very good paper.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DQBCr0nJWL", "reviewer": "Reviewer_o9Vp", "summary": "The paper proposes SAE-V, a framework that utilizes SAEs trained on top of multimodal large language models (MLLMs) to measure image-text alignment. Specifically, for a given SAE feature, it retrieves the top activating tokens and image patches and computes their cosine similarity score, which produces an alignment metric for a single dataset sample. The paper evaluates this metric for applications like image patch filtering and dataset filtering.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Your suggestions are insightful and will enhance the completeness of our paper!\n\nWe used all available resource and devoted efforts to conduct additional experiments. We addressed all your negative comments below and will add them into the revision. **If this rebuttal addresses your concerns, we earnestly ask you to consider raising the score and supporting us for acceptance.**\n\n## Claim 1 & Weakness: We prune the overall presentation of the paper to reduce repetitive expressions and add necessary backgrounds.\n\nLimited by the length of rebuttal, we are not able to list all the changes we made, so only key examples are provided. E.g.:\n\n- line 245-253: \n\n> The transferability of SAEs between foundation models and instruction-tuned models has been extensively investigated in text-only contexts[3][4][5], as it demonstrates whether SAEs can capture universal semantic features within LLMs. Similarly, the transferability from MLLMs to corresponding LLMs serves as a critical metric for the quality of features learned by SAE-V. \n> (Followed by original content)\n\n- Fig. 7 caption:\n\n> We evaluated SAE-V data filter method on LLaVA-NeXT-7B model, Align Anything dataset, and LLaVA-Bench benchmark. The result show that all SAE-V-based methods significantly outperforms the random selection baseline, while the cosine similarity filter achieved 108% of the full dataset’s performance with only 20% of the data, and the co-occurrence filter peaked at 50% of the data, reaching a score of 108.17.\n\nTo address the specific concerns:\n\n- SAE only accepts textual tokens, while SAE-V is designed for both text and image tokens. When evaluating with multimodal input, SAE can only reconstruct text features, whereas SAE-V reconstructs both text and image features, achieving lower reconst. loss (Fig. 4, Col. 3). Moreover, even when evaluating on text tokens, SAE-V surpasses SAE in reconstruction capability (Fig. 4, Col. 1-2), showing effective cross-modal generalization.\n- Regarding the \"Original\" bar in Fig. 3: The reconst. loss measures how well the model predicts the next token compared to ground truth. The original model's predictions are also probability distributions, not exact predictions, which is why the \"Original\" has non-zero reconst. loss.\n\n## Claim 2: We provided additional details, and conducted extra ablation study to compare SAE-V with non-training baseline.\n\nRegarding details of the experiment in Section 3.2:\n\n- **Pipeline:** Given the 1000 val. set of ImageNet, We score the 24\\*24 image patches according to the metrics of SAE-V features (Fig. 6 legend). We filter out the patches according to the score and the given ratio (x-axis in Fig. 6). We use LLaVA-NeXT-7B to classify the filtered image, and calculate the accuracy (Not loss value) of LLaVA-NeXT-7B output as y-axis in Fig. 6.\n- **Target:** This experiment is designed to support the claim that SAE-V could preserve the key info from images, and the higher accuracy is, the more key info is preserved through the leftover patches. E.g., in Fig. 5, SAE-V could preserve the most relevant information (the dog) in the image.\n\nWe added an additional baseline to this experiment, using attention score of image patches and the original text token to filter the patches. Results:\n\nMasking (%)|0|25|50|75\n-|-|-|-|-\nAttn. Score|0.9020|0.8770|0.8200|0.6930\nSAE-V Cos.|0.9020|0.8670|0.8110|0.6630\n\nSAE-V could achieve relatively similar performance using **reconstruction** instead of **original activation** of MLLMs.\n\n## Claim 3: We updated the presentation and added additional baselines.\n\nSorry for the ambiguous figure. To clarify, we used LLaVA-Bench[2] to report the MLLM performance, and the y-axis is not percentage, but the exact score on LLaVA-Bench. To demonstrate that our method is superior regardless of benchmark selection, we did an ablation study of benchmarks. For more details see the rebuttal of **Reviewer k2no point 1**.\n\nAs for data selection, we included the CLIP baseline and added a paragraph discussing the related work [1]. The experiment results are as follows:\n\nFilter Method|LLaVA-Bench @ 0%|20%|40%|60%|80%|100%\n-|-|-|-|-|-|-\nCLIP|94.2|99.3|102.9|102.6|**103.8**|95.8\nSAE-V Cosine Similarity|94.2|**104.1**|103.8|100.4|101.1|95.8\nRandom|94.2|99.6|98.4|97.6|93.5|95.8\n\nThe experiment shows that although the peak performance is close, CLIP achieve the peak performance using 4 times data than SAE-V, showing the effectiveness of SAE-V.\n\n\n\n## Reference\n\n[1] Gadre et al., DATACOMP: In search of the next generation of multimodal datasets, 2023.\n\n[2] Liu et al., Visual instruction tuning, 2023.\n\n[3] Kissane et al., Saes (usually) transfer between base and chat models, 2024.\n\n[4] Taras et al., Do Sparse Autoencoders (SAEs) transfer across base and finetuned language models?, 2024.\n\n[5] Gallifant et al., Sparse autoencoder features for classifications and transferability, 2025.", "claims_and_evidence": "- Claim 1. SAE-V is superior to SAE in reconstruction quality.\n    - The presentation of this section should be improved; it is hard to evaluate the validity of the claim without further explanation.\n    - Figure 3 is missing a description of SAE vs. SAE-V. Is it that SAE is trained only on text features while SAE-V is trained jointly on text and image features? Is SAE evaluated on both text and image features, or only on text features? I don’t understand “Original” — shouldn’t the original feature achieve a  reconstruction loss of 0?\n- Claim 2. SAE-V can be used for image patch filtering.\n    - This experiment is quite novel and interesting, but more details could be provided.\n    - In L265, how is this evaluation performed? What is the size of the ImageNet test set, what MLLM is being evaluated here, how is it being evaluated (is it a comparison between the MLLM output vs. the ground truth ImageNet class)? In Figure 6, the y-axis is labeled as “loss value” but the caption states that it is “classification accuracy” — what is being plotted here? How is the masking performed; are the features of those image patches just zeroed out?\n    - Claim 2 and 3 are also missing a comparison against the non-SAE baseline. For example, you could compute the alignment metric by computing the cosine similarity of the original text token and image patch features, without any SAE projection. This would better illustrate why SAE-V training is necessary, beyond a simple training-free baseline.\n- Claim 3. SAE-V can be used for dataset filtering.\n    - The experiment is well motivated and interesting, but the presentation is confusing.\n    - In Figure 7, what is the performance score metric exactly — is it some classification accuracy or a loss value? While it is trained on Align-Anything, is it also evaluated on a held out subset of Align-Anything, and what is the size of this subset? The figure caption states that the y-axis is scaled according to the “full dataset’s performance” — why at the 100% data percentage is the performance score ~96%, not 100%?\n    - For the comparison in L371, beyond IFD I would recommend the paper also include a CLIP baseline (i.e., taking the top percentage of samples based on highest CLIP scores). This CLIP baseline is already explored in prior work such as in [1]. To this end, I would also disagree with L375, which states there are “no widely recognized data filtering methods specifically designed for multimodal data.”\n\n[1] Gadre et. al., 2023. DATACOMP: In search of the next generation of multimodal datasets. NeurIPS 2023.", "methods_and_evaluation_criteria": "See “Claims And Evidence” above.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "See “Claims And Evidence” above.", "relation_to_broader_scientific_literature": "The paper proposes a novel use case for SAEs for measuring image-text alignment in multimodal models. To this end, they also introduce image patch filtering and data filtering as evaluation tasks. Both the method and evaluation tasks have not been explored in the context of MLLMs.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths\n\n- The proposed alignment score is well-motivated and interesting.\n- The proposal of image patch filtering as an evaluation task is very unique and compelling. Future work in multimodal alignment would also benefit from this framework.\n- The dataset filtering evaluation is sensible and a practical example of how SAE-V is useful.\n\nWeaknesses\n\n- The presentation of the experiments needs a lot of work. Many of the figures and experimental setup are unclear and missing key details; also see “Claims And Evidence” above.\n\nOverall, I like the premise of the paper but the presentation is poor. I am open to revising my score if the authors are able to address my questions and clarify details regarding the experiments.", "other_comments_or_suggestions": "- L160 typo, “donated” should be “denoted”", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces SAE-V, a sparse autoencoder-based framework that extends mechanistic interpretability methods to multimodal large language models (MLLMs). The key contribution lies in using cross-modal feature similarity to score and filter alignment data, enabling improved performance with reduced supervision. The reviewers generally agree that the method is intuitive and the empirical results are promising. However, there are notable concerns. First, the assumption that high cross-modal similarity indicates high-quality data may not always hold—low similarity might reflect model limitations rather than poor data quality. Second, the evaluation is limited to a few benchmarks (e.g., LLaVA-Bench, MME), lacking broader validation on widely adopted datasets such as POPE or MMStar.

Nonetheless, the authors addressed most reviewer concerns in the rebuttal and presented new experiments on larger models and datasets. While the core idea is not fundamentally novel, its practical implications are valuable to the community. Given the relevance to alignment and interpretability in MLLMs, I lean toward a very weak acceptance.
Decision: Accept (poster)