MM - SHAP : A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks (2023)

Abstract: Vision and language models (VL) are known to exploit unrobust indicators in individual modalities (e.g., introduced by distributional biases) instead of focusing on relevant information in each modality. That a unimodal model achieves similar accuracy on a VL task to a multimodal one, indicates that so-called unimodal collapse occurred. However, accuracy-based tests fail to detect e.g., when the model prediction is wrong, while the model used relevant information from a modality. Instead, we propose MM-SHAP, a performance-agnostic multimodality score based on Shapley values that reliably quantifies in which proportions a multimodal model uses individual modalities. We apply MM-SHAP in two ways: (1) to compare models for their average degree of multimodality, and (2) to measure for individual models the contribution of individual modalities for different tasks and datasets. Experiments with six VL models – LXMERT, CLIP and four ALBEF variants – on four VL tasks highlight that unimodal collapse can occur to different degrees and in different directions, contradicting the wide-spread assumption that unimodal collapse is one-sided. Based on our results, we recommend MM-SHAP for analysing multimodal tasks, to diagnose and guide progress towards multimodal integration. Code available at https://github.com/Heidelberg-NLP/MM-SHAP .

## introduction vision and language (vl) tasks are dominated by general-purpose pretrained transformer-based vl models @xcite @xcite @xcite . but we are only starting to understand why these multimodal (mm) models work so well, and how they utilise and fuse image and text modalities @xcite . even worse, these highly parametrised neural vl models, pretrained on large amounts of data, tend to exploit artefacts and statistical correlations in the data @xcite , showing little to no evidence of detailed linguistic or visual understanding @xcite @xcite . statistical biases towards indicators in one modality -to the detriment of others -can cause unimodal collapse @xcite , where seemingly mm models exploit one modality that exhibits biases, meaning that the mm system effectively reduces to a unimodal model @xcite )e.g., if a model answers "how many...?" questions with "two" -the most frequent answer seen in training @xcite . unimodal collapse is severe, as it leads to loss of system reliability. it also shows that multimodal fusion is far from being solved. hence the importance of measuring multimodal degree -the degree to which modalities are used in model predictions -with reliable metrics. to test for unimodal collapse, research has so far focused on performance tests: a vl model is evaluated on a mm task, but one modality crucial for solving it correctly is missing, corrupted @xcite or permuted @xcite . these tests are indicative of unimodal collapse, but we argue that they are not appropriate to reliably measure the contribution of each modality. clearly, accuracy reflects whether a model prediction is (in)correct, but it may detect illicit cases where the model prediction is wrong, although it does use crucial indicators in a given modality. conversely, a prediction might be correct, but may be derived from unrobust indicators. fig. 1 shows very different shap-based contribution patterns of image regions and text tokens leading to model responses of different image-sentence alignment (isa) scores (e.g., albef caption vs. foil), while yielding same isa accuracy since both scores surpass the 0.5 classification threshold. as an alternative to accuracy-based methods, we propose mm-shap, a performance-agnostic metric to quantify and interpret the contribution of individual modalities in vl models. mm-shap is based on shapley values @xcite , a theoretically well-founded interpretability method from cooperative game theory. we apply mm-shap to quantify the contribution of specific parts of the input towards model predictions. our main contributions are: lxmert, clip and four albef variantson four vl tasks: image-sentence alignment, vqa, gqa and on the more fine-grained valse vl benchmark. iv) we identify vl models that are balanced in their usage of two modalities (clip), models that show a higher visual degree (lxmert) or a stronger textual degree (albef). v) we show that 1) fine-tuning a model can affect its mm degree and that 2) current vl models do not all collapse towards the same modality, as reported in recent work @xcite , but that directions can differ from model to model. ## related work testing for unimodal collapse strong prediction indicators in either modality can cause mm models to ignore weaker indicators in another modality. prior work has proposed ways to identify (and remove) such biases from data @xcite . foiling approaches introduce mistakes in image descriptions and test whether vl models notice the discrepancy between image and captions @xcite , finding that models are surprisingly insensitive to such foils. @xcite , in a similar vein, exchange images with other images or captions with other captions, expecting that inputs with misleading information in one modality incur a decrease in model accuracy. they use an observed decrease in task accuracy to calculate a perceptual score as a measure of the mm degree of a model. their findings suggest that across their tested vl models, textual input consistently matters more than visual input. ablation methods remove information from either modality and test whether the model can still solve the task. here, @xcite find that the visual modality matters more than text: vl models suffer from image parts removal when predicting masked text, but can predict masked visual inputs when text input is ablated. this contradicts @xcite 's finding, but their investigations have only a single model in common, namely lxmert. hence, the literature agrees that vl models are not as cross-modal as expected -but disagree on whether models rely more on the textual @xcite or on the visual modality @xcite . we argue that a reason for this discrepancy is that prior work computes mm scores based on model performance. in our work we argue that methods for measuring a model's mm degree should not rely on accuracy (see §3.1 for motivation). instead, we propose an accuracy-agnostic method to measure the mm degree of vl models, using the shap (lundberg and lee, 2017) interpretability method that is theoretically suitable to define an mm score. interpretability methods for explaining predictions of neural models can be classified into two categories: white-box methods, which require access to specific components of neural architectures and black-box methods, which are model-agnostic, requiring only access to model inputs and outputs. notable white-box methods are: attention-based methods, which correlate high attention weights with high feature importance. but the equivalence of importance score and attention is debated and must be considered with care (jain and wallace, 2019; wiegreffe and pinter, 2019) (see app. d for a detailed discussion on why attention is inappropriate for defining an mm score). layer-wise relevance propagation @xcite or gradientbased methods e.g., grad-cam @xcite can also be used to determine the importance of inputs, but can be deceived by small changes in inputs (adversarial attacks). notable black-box methods are: lime @xcite and its multimodal adaptation dime @xcite approximate the vicinity of the input with a linear function that is interpretable. but depending on the choice of the size of the vicinity, lime can lead to very disparate results. methods like rise @xcite and shap (lundberg and lee, 2017) compute importance scores by randomly masking parts of the input and determining the effect this has on the output. shap exhibits great theoretical properties that enable us to define a mm score, as we will motivate in §3.4. ## multimodal contributions across models and datasets we use mm-shap to study mm contributions for different i) model types, ii) datasets and iii) tasks. in doing so we iv) re-evaluate prior findings on visual vs. textual unimodal collapse and v) showcase mm-shap's abilities for interpreting predictions for individual samples, for error analysis. we evaluate pretrained vl models with mm-shap and complement our analysis by measuring the model's task accuracy. we compare mm-shap to a 50% t-shap -50% v-shap baseline and gauge how much the model tends towards the textual or visual modality. we hypothesise that in average, v&l should contribute equally when the model predicts whether the contents of the modalities are aligned (image-sentence alignment). we test on matching image-captions, but also on cases with discrepancies between modalities. we break down our incongruity tests into high discrepancy (cases of completely mismatching imagecaptions, tab. 1), and low discrepancy (cases where a single word or phrase incurs a mismatch, tab. 2). ## conclusions and future work we present mm-shap, a performance-agnostic metric that measures the mm degree of vl models at dataset and sample level. our results show that on the same task, dataset, and on specific instances, different types of models rely on modalities to different degrees and in different directions. using mm-shap we are the first to quantify changes in a model's mm degree through fine-tuning. our analyses show that degrees of mm contributions can be orthogonal to task performance, supporting the need for performance-agnostic metrics. mm-shap is applicable to further modalities. it enables model-based data cleaning and thus, dataset bias removal. finally, it can serve as a diagnostic tool for improving mm fusion methods. mm-shap can be used for testing true model understanding at a global and at instance level, and whether a model is giving the right answer for the right reasons, at corpus -and instance-levelwhich is not guaranteed for performance-dependent metrics. it can help us track mm contributions during (pre-)training and towards assessing and eventually predicting how much a model needs to rely on how many and which modalities in a given task or instance case -and how to explain this. we hence believe that many future research questions will profit from our mm score as an unbiased mm contribution metric, with ai research advancing to include more and more modalities beyond vision and language @xcite : acoustics, haptics, emotion, and more (cf. @xcite ).