Title: Explaining Similarity in Vision-Language Encoders with Weighted Banzhaf Interactions
Venue: NeurIPS 2025
Accepted: yes
Keywords: explainability

feature interactions

Shapley interactions

Banzhaf value

CLIP
Abstract: Language-image pre-training (LIP) enables the development of vision-language models capable of zero-shot classification, localization, multimodal retrieval, and semantic understanding. Various explanation methods have been proposed to visualize the importance of input image-text pairs on the model's similarity outputs. However, popular saliency maps are limited by capturing only first-order attributions, overlooking the complex cross-modal interactions intrinsic to such encoders. We introduce faithful interaction explanations of LIP models (FIxLIP) as a unified approach to decomposing the similarity in vision-language encoders. FIxLIP is rooted in game theory, where we analyze how using the weighted Banzhaf interaction index offers greater flexibility and improves computational efficiency over the Shapley interaction quantification framework. From a practical perspective, we propose how to naturally extend explanation evaluation metrics, such as the pointing game and area between the insertion/deletion curves, to second-order interaction explanations. Experiments on the MS COCO and ImageNet-1k benchmarks validate that second-order methods, such as FIxLIP, outperform first-order attribution methods. Beyond delivering high-quality explanations, we demonstrate the utility of FIxLIP in comparing different models, e.g. CLIP vs. SigLIP-2.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper points out that the current explanation methods with saliency maps are limited by the unimodal view of the model’s output. The vision-language encoders should be explained by considering the cross-modal interactions between image-text pairs. Therefore, the paper proposes an explanation method to measure the importance of each token in the image and text encoders via weighted faithful Banzhaf interactions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Very comprehensive algorithm demonstration.\n- It's the first work to use game theory to generate explanations for cross-modal interaction in the vision-language models.\n\nWeaknesses:\n- The clarity of the paper needs to be improved. Concretely, please see the questions.\n\nQUESTIONS:\n- It is confusing that the explanation for the two-modality interaction is called second-order. If there is a definition, it should mention the reference.\n- It said that this is the first work using weighted Banzhaf interactions to overcome the out-of-distribution problem apparent in removal-based explainability in line 46. How to understand the out-of-distribution? It’s better to have a clear demonstration in the introduction. \n- I know the attribution-based methods like Grad-ECLIP and GAME are totally different from the methods with game theory. Then, what’s the main difference and the advantages of using Banzhaf compared with Shapley to estimate the token importance? It will be clearer to have a figure to show the comparison of the two kinds of algorithms.\n- For the explanation graph in Figure 1, why do the tokens of object parts have a negative contribution to the similarity, such as the two dog patches and the three hydrant patches? More demonstrations about the results in Figure 1 should be added. It’s hard to understand.\n\nAbout experiments:\n- In Figure 2, negative y-axis values can be obtained with the Sharpley and FIXLIP methods, while the minimum values for the attention heat map methods (Grad-ECLIP, GAME, and XCLIP) are zero. This will greatly increase the performance gap. The reason given in the paper is “negative values denote that the model is predicting the inputs are unsimilar.” What does that mean? Another image is also used in the deletion/insertion experiments? I'm still confused about why the similarity drop can be negative. \n- What is the concrete process in the PGR metric and sanity check in Section 5.2? I know that the point game is that a hit shows when the maximum value on the heat map is located on the correct object. So, what’s the operation when multiple objects are involved? And what is the sanity check with a pointing game? \n\nI may increase my rating after my concerns are solved.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

comments: SUMMARY:
This paper proposes FIXLIP for interpreting similarity scores in vision–language encoders through second-order interaction attributions grounded in weighted Banzhaf interaction indices. The authors address the limitations of existing first-order saliency methods by modeling cross- and intra-modal token interactions, thus capturing richer and more faithful explanation structures. Experiments show that FIXLIP produces more faithful and informative explanations than prior state-of-the-art attribution methods.

STRENGTHS AND WEAKNESSES:
Strengths:
1. Authors extend explainability to second-order interactions using weighted Banzhaf indices, offering more expressive and faithful attribution than first-order approaches.
2. Strong theoretical formulation.
3. The experimental results show the superiority of the proposed second-order estimator,

Weakness:
1. What is WLS in Figure 1?
2. Simply masking the patches and subtokens could be a bad choice as the final representations of patches and subtokens are entangled with each other. Also I assume masking subtokens means remove them from the sentence. But what does masking mean for patches? Do authors mask them as a whole black or white patch?
3. Although second-order interactions are insightful, the method does not scale to third-order or beyond, which might limit expressivity in complex VLMs. A simple case could be just have the third-order established by edge transformation.
4. It would be interesting to see how FlixLIP compared to previous works given the same budget. 
5. How does the subset selection affect performance (Sec. 3.3)? 
6. What are the computational overhead for different p settings in Table 1? Also what is the budget for other methods.

QUESTIONS:
See weakness.

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an explanation method for image-text dual encoder models that is based on interactions between image patch features and text token features. In practice, the method works by sampling many masks for the image patch and text token positions, computing for each match the image-text similarity according to the model, and fitting a least-squares regression that predicts the image-text similarity from the weighted sum of interactions between pairs of image and text features. The method is evaluated in several ways: an insertion/deletion setting, a pointing game setting, and a qualitative analysis. The method generally performs better than first-order attribution methods. The experiments also include an analysis of how the computational requirements of the method scale with the number of masks sampled.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The experiments section is thorough. There are several applications or ways of evaluating the method, and confidence intervals/error bars are provided for at least some of the quantitative evaluations. In addition, a running-time analysis is provided.\n- The method does provide value over first-order attribution methods according to the insertion/deletion and pointing-game evaluations.\nWeaknesses:\n- Due to the heavy use of notation, the method section is quite challenging to understand and is overwhelming.\n- Isn’t Definition 2 circular, since we use b_i to define b?\n- What does it mean for a mask to be high or low similarity (line 202)?\n- Explanation of the insertion/deletion evaluation could be dramatically improved\n- Line 308 needs ref (“consistent with prior work”)\n\nQUESTIONS:\n- Can you explain the method with less notation?\n- In the insertion/deletion evaluation, the proposed method is not uniformly better than the Shapley baseline for all choices of p. Are there hyperparams that could be tuned in the Shapley case and if so, is the reported Shapley line the best of those?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 1, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}