Benchmarking Robustness under Distribution Shift of Multimodal Image-Text Models
Abstract
Multimodal image-text models have shown remarkable performance in the past few years.
However, the robustness of such foundation models against distribution shifts is crucial in downstream applications.
In this paper, we investigate their robustness under image and text perturbations.
We first build several multimodal benchmark datasets by applying 17 image perturbation and 16 text perturbation techniques.
Then we extensively study the robustness of 6 widely adopted models on 3 downstream tasks (image-text retrieval, visual reasoning, and visual entailment).
We observe that these powerful multimodal models are sensitive to image/text perturbations, especially to image perturbations.
For text, character-level perturbations have shown higher adversarial impact than word-level and sentence-level perturbations.
We also observe that models trained by generative objectives tend to be more robust.
Our findings in terms of robustness study could facilitate the development of large image-text models, as well as their deployment for real-world applications.
Introduction and Related Work
Multimodal learning has drawn increasing attention in the past few years [9, 22, 47, 48, 94, 64, 41, 45, 44, 88, 15, 65, 79, 1, 64, 91].
Many datasets and models are collected and proposed to accelerate research in this field.
However, despite the extraordinary performance and exciting potential, multimodal models might be vulnerable under distribution shifts.
In Figure 1, we show an example of CLIP [64] model’s performance on image-text retrieval under image or text perturbations.
When pixelation is applied to the original image for image-to-text retrieval, the perturbed image retrieves less relevant or even wrong texts.
For text perturbation, we replace words with their synonym and delete words for text-to-image retrieval.
We find the retrieved images changed dramatically even though the semantics of the sentence didn’t change.
Similar findings have also been observed in previous works [21, 20, 26, 62].
There is a sizable literature on robustness evaluation of unimodal vision models [89, 95, 16, 13, 27, 63, 3, 55, 57, 2, 96] or unimodal language models [81, 7, 80, 68, 24, 73, 14, 28, 56, 78].
However, robustness evaluation of multimodal image-text models under distribution shift has rarely been studied [25, 12].
(More related work can be found in Appendix 5.8).
To our best knowledge, there is currently no benchmark dataset nor a comprehensive study of how the perturbed data can affect their performance.
In this work, we would like to (1) build robustness evaluation benchmarks for multimodal image-text models, and (2) investigate these models’ robustness under image or text perturbations in downstream applications.
Our contributions can be summarized as follows:
• We build multimodal robustness evaluation benchmarks by leveraging existing datasets and tasks, e g., image-text retrieval (Flicker30K, COCO), visual reasoning (NLVR2), and visual entailment (SNLI-VE).
We design 17 image perturbation and 16 text perturbation strategies to extend them to multimodal evaluation settings.
• We observe that multimodal image-text models are more sensitive to image perturbations than text perturbations, while for text perturbations, character-level perturbations showed higher impact than word-level and sentence-level perturbations.
• We introduce a new metric, termed MMI (MultiModal Impact score), to account for the relative performance drop under distribution shift in downstream applications.
Multimodal Robustness Benchmark under Distribution Shift
To evaluate the robustness of large pretrained multimodal models under distribution shift, we start by building several evaluation benchmark datasets, by perturbing the original image-text pairs on either image side or text side.
Image Perturbation
In this work, we adopt the perturbation strategies from ImageNet-C [33] and
Stylize-ImageNet [23, 58].
The reason we include Stylize-ImageNet is because it is an effective method to perturb the original image by breaking its shape and texture [23].
The perturbations are drawn into five categories: Noise, Blur, Weather, Digital, and Stylize.
Specifically, we have 17 image pertubation techniques (1) Noise: Gaussian Noise, Shot Noise, Impulse Noise, Speckle Noise; (2)
Blur: Defocus Blur, Frosted Glass Blur, Motion Blur, Zoom Blur; (3) Weather:
Snow, Frost,
Fog, Brightness; (4) Digital: Contrast, Elastic, Pixelate, JPEG Compression; and (5) Stylize .
Note that real-world corruptions can manifest themselves at varying intensities, we thus introduce variation for each corruption following [33, 23, 58].
In our evaluation setting, each category has five levels of severity [1 , 2 , 3 , 4 , 5] , resulting in 85 perturbation methods in total.
More details can be found in the Appendix 5.2.
These strategies are commonly considered as synthetic distribution shifts, and can serve as a good starting point since they are precisely defined and easy to apply.
Examples of perturbed images from COCO dataset [50] are shown in the Appendix 5.4.
Text Perturbation To simulate the real-world distribution shift in language, we design the text perturbation into three categories: character-level, word-level and sentence-level.
In detail, for character-level perturbation, we adopt 6 strategies from [54], including Keyboard , OCR , Character
Insert (CI) , Character Replace (CR) , Character Swap (CS) , Character Delete (CD) .
These perturbations can be considered as simulating real-world typos or mistakes during typing.
For word-level perturbation, we adopt 5 strategies from EDA and AEDA [83, 40], including Synonym
Replacement (SR) , Word Insertion (WR) , Word Swap (WS) , Word Deletion (WD) , and Insert punctuation (IP) .
These perturbations aim to simulate different writing habits that people may replace, delete, or add words to express the same meaning.
For sentence-level perturbation, (1) we first adopt the style transformation strategies from [42, 18, 70, 69], i.e., transferring text style into formal , casual , passive , and active ; (2) we also adopt the back translation method from [54].
These perturbations will focus more on language semantics, due to the differences of speaking/writing styles, or translation error.
For strategies within character-level and word-level, we apply 5 perturbation levels [0 . 15 , 0 . 20 , 0 .
35] , while for strategies within character-level, there is only one level.
This leads to 60 text perturbation methods in total.
Examples of the text perturbation of captions in
Flickr30K dataset [90] and more details about each text perturbation strategy can be found in the
Appendix 5.3 and Appendix 5.5.
Evaluation Tasks and Datasets We select three widely adopted downstream tasks for a comprehensive evaluation on the robustness of multimodal image-text models, including image-text retrieval, visual reasoning (VR), and visual entailment (VE).
For each task, we perturbed the corresponding datasets, i.e., Flickr30K [90] and COCO [50] for image-text retrieval, and NLVR2 [74] for visual reasoning, SNLI-VE [86, 87] for visual entailment, using the image perturbation (IP) and text perturbation (TP) methods introduced above, which results in: (1) Flickr30K-IP, Flickr30K-TP, COCO-IP, and COCO-TP for image-text retrieval evaluation; (2) NLVR2-IP and NLVR2-TP for visual reasoning evaluation; and (3) SNLI-VE-IP and SNLI-VE-TP for visual entailment evaluation.
Experiments and Results
For evaluation, we select six representative large pretrained multimodal models which publicly released their pretrained models 1 , including CLIP [64], ViLT [41], ALBEF [45], BLIP [44], TCL [88], and METER [15].
To qualitatively analyze the multimodal image-text models’ robustness under perturbations, we propose a new impact score MMI (multimodal impact score) to calculate the averaged performance ("ave") drop compared with the non-perturbed performance ("clean"), which is defined as: MMI =
( s c − s p )
/s c , where s p is the perturbed score and s c is the clean score.
More experimental settings can be found in the Appendix 5.6.
[Table]
[Table]
Discussion To emphasize the important findings, we provide a summary of the experiments.
(More discussion can be found in the Appendix 5.
[Table]
According to our impact score, overall, both image and text perturbation methods can effectively attack the current multimodal image-text models, for image-text retrieval, visual reasoning
[Table]
and visual entailment tasks.
In general, models are more sensitive to image perturbations than text perturbations.
Considering the characteristic of models, generative models may be more robust than discriminative models.
In addition, different models’ sensitivity to perturbation methods is also very different.
To combine the similarities, we found that
Zoom Blur shows a consistently high impact in three downstream tasks across different models as an effective image perturbation method.
In contrast, Glass Blur and Brightness are less effective in attacking models.
From text perturbation results, Keyboard and CR could be the two powerful perturbation methods, while sentence-level perturbation methods along with IP (Insert Punctuation) seem to be "soft" perturbation methods that rarely have a significant impact on models’ performance.
[Table] [Table]
[Table] [Table]
Conclusion
In the study [Table] we investigate the robustness of large multimodal pretrained image-text models.
We introduce several evaluation benchmarks under distribution shift by applying 17 image perturbation and 16 text perturbation strategies.
We select three downstream tasks, including image-text retrieval, visual reasoning
[Table]
and visual entailment, to evaluate 6 popular models.
Our developed multimodal perturbation datasets could serve as robustness evaluation benchmarks for image-text models.
We hope our findings could provide inspiration on how to develop and deploy more robust models for real-world applications.
Appendix 5.1 Multimodal Robustness Benchmark under Distribution Shift
Distribution shift is one of the significant problems of applying models in real-world scenarios [75, 52].
It is caused by scarcity of data, i.e., the models cannot be trained on all possible data in p ( x , y ) , where p ( x , y ) is considered as the real-world data distribution.
In other words, the training set contains the collected data that fits a certain distribution p tr ( x | y ) , but the test set usually has a different distribution p te ( x | y ) ̸
= p tr ( x | y ) .
5.2 Image Perturbation
In Table 7 and Table
[Table]
we show more details about the image and text perturbations, respectively
[Table]
[Table]
[Table]
5.3 Text Perturbation
Fidelity To build a convincing benchmark, we need to ensure the perturbed text remains the same semantics as the original one.
Otherwise, for image-text pairs in multimodal learning, the perturbed text won’t be a matching pair to the original image.
In this work, we use paraphrases from pretrained sentence-transformers [67] to evaluate the semantic similarity between the original and perturbed sentences.
Specifically, “paraphrase-mpnet-base-v2” is used to extract the original and perturbed sentence embeddings for computing similarity score α s .
Given a predefined tolerance threshold α 0 , a higher score α s > α 0 means the perturbed text still has similar semantics.
However, if α s < αmeans their semantics are different, we will perturb the sentence again until the semantic similarity score meets the requirement, in a reasonable looping time.
For example, we set number of loops to
[Table] [Table]
N max = 100 .
Beyond N max , we will just remove this text sample from our robustness benchmark.
This procedure guarantees semantic closeness and ensures our benchmark is valid for evaluation.
5.4 Examples of Image Perturbation
Examples of perturbed images from COCO dataset [50] are shown in Figure 2.
5.5 Examples of Text Perturbation
Examples of the text perturbation of captions in Flickr30K dataset [90] are shown in Table 9.
[Table]
5.6 More Experimental Setting
By building the robustness benchmark datasets
[Table]
we would like to answer the following questions: (1)
How robust can multimodal pretrained image-text models be under distribution shift?
What is the sensitivity of each model under different perturbation methods?
(3) Which model architecture or loss objectives might be more robust under image or text perturbations?
(4) Are there any particular image/text perturbation methods that can consistently show significant influence?
Evaluation Tasks We select three widely adopted downstream tasks for a comprehensive evaluation on the robustness of multimodal image-text models, including image-text retrieval, visual reasoning (VR)
[Table]
and visual entailment (VE).
Image-text retrieval includes two subtasks: (1) retrieve images with given text (Image Retrieval) and (2) retrieve text with given images (Text Retrieval) [6, 31].
Visual Reasoning (VR) requires the model to determine whether a textual statement describes a pair of images.
Visual Entailment (VE) is a visual reasoning task to predict whether the relationship between an image and text is entailment, neutral, or contradictory [86, 87].
Evaluation Models We select six representative large pretrained multimodal models, which have publicly released their pretrained models 2 , including CLIP [64], ViLT [41], ALBEF [45], BLIP [44],
TCL [88], and METER [15].
In order to provide a fair comparison, we adopt the model weights provided by their official repositories 3 for either zero-shot prediction or fine-tuned results.
We only perform the tasks of each model that have been studied in its original work, where their reported scores are marked as “clean” in our Tables.
Evaluation Metric We adopt standard evaluation metrics for each task.
To be specific, for imagetext retrieval, we use recall and RSUM [85].
Here, recall is defined as K (R@K) metric, where K = { 1 , 5 , 10 } , and RSUM is defined as the sum of recall metrics at K = { 1 , 5 , 10 } of both image and text retrieval tasks.
As for visual reasoning and visual entailment tasks, we use prediction accuracy as the evaluation metric.
However, there is no appropriate metric that could be used for robustness evaluation under distribution shift.
Inspired by an example in [75], given a clean dataset d 1 and its perturbed dataset d 2 , model mshould be considered more robust than model m 2 if m 1 ’s performance drop is less significant than mfrom d 1 to d 2 , even though m 2 ’s absolute accuracy/recall on d 2 is higher than m 1 ’s.
Thus we believe robustness should be evaluated relatively when there are distribution shifts.
To qualitatively analyze the multimodal image-text models, we introduce a new evaluation metric, termed MultiModal Impact score (MMI).
We compute MMI as the averaged performance drop compared with the non-perturbed performance (“clean”), i.e., MMI =
( s c − s p ) /s
c where s p is the perturbed score and s c is the clean score.
In the following experiments, we report both standard performance scores, i.e., recall, RSUM, accuracy, as well as our MMI.
Our Benchmark Datasets For each task, we perturb the corresponding datasets i.e., Flickr30K [90] and COCO [50] , NLVR2 [74], SNLI-VE [86, 87], using the image perturbation (IP) and text perturbation (TP) methods introduced in Section 2.
This leads to our 8 benchmark datasets: (1)
Flickr30K-IP, Flickr30K-TP, COCO-IP, and COCO-TP for image-text retrieval robustness evaluation; (2) NLVR2-IP and NLVR2-TP for visual reasoning robustness evaluation; and (3) SNLI-VE-IP and
SNLI-VE-TP for visual entailment robustness evaluation.
• For image-text retrieval, the Flickr30K dataset contains 1,000 images, and each of them has 5 corresponding captions, while the COCO dataset contains 5,000 images, and each of them also has 5 corresponding captions.
We report the RSUM score averaged on five perturbation levels under each perturbation method to reveal the overall performance.
More detailed results, including the recall at K (R@K) metric, K = { 1 , 5 , 10 } , can be found in the Appendix 5.9.
For CLIP and TCL, we provide the evaluation results for both zero-shot (ZS) and fine-tuned (FT) settings, while for ALBEF and BLIP, we follow their original settings and report the fine-tuned (FT) results.
• For visual reasoning, the NLVR2 dev set contains 2,018 unique sentences and 6,982 samples, while the test-P set contains 1,995 unique sentences and 6,967 samples.
We report the accuracy of both the dev set and test-P set of the NLVR2 dataset under image and text perturbations.
We evaluate the robustness of ALBEF, ViLT, TCL, BLIP, and METER.
• For visual entailment, the SNLI-VE val set contains 1,000 images and 6,
576 sentences, while the test set contains 1,000 images and 6,592 sentences.
We evaluate the accuracy of both the dev set and test set of the SNLI-VE dataset under image and text perturbations.
We report the results of ALBEF, TCL, and METER.
5.7 Experimental Results and Discussion
Image-text Retrieval Results and Observation
The averaged RSUM results of different methods under five perturbation levels are shown in Table 1 and Table 2, for image perturbation and text perturbation, respectively.
Through Table 1, we found that all the models’ performance dropped under image perturbation.
According to the impact score, overall, the CLIP model is, in general, more robust than other models, which we hypothesized might be due to the large datasets that
CLIP was trained upon, where large data may lead to better performance, as also noted by [75].
Due to the generative loss objective, the BLIP model also shows good robustness performance.
We think the generative loss objective can help to learn better data distribution, and we observe a recent paradigm change from using discriminative contrastive loss, i.e., CLIP, ALBEF [64, 45], to using generative loss, i.e., BLIP, CoCa, SimVLM, PaLI, Unified-IO, OFA [44, 91, 82, 8, 53, 79].
In detail, we also found that different image perturbation methods have different impact levels on the model’s performance, and the methods that have the biggest impact also vary among different models and datasets.
CLIP-FT, TCL-ZS, ALBEF, and BLIP seem to be more sensitive to Zoom Blur perturbation, while ViLT and TCL-FT are more sensitive to pixelation perturbation.
Glass blur and brightness are the two "soft" perturbation methods, where all the models are very robust under these settings.
Besides, fine-tuning may also help to improve robustness, i.e., TCL-FT shows robustness improvements compared with TCL-ZS on both Flickr30K and COCO datasets.
As in Table 2, we found that all the models’ performance dropped under text perturbation.
BLIP overall shows the best robustness performance, which may bring the idea that the generative loss objective is useful.
In addition, we found that character-level perturbations show much more influence than word-level and sentence-level perturbations, especially Keyboard and CR (Character Replace) methods consistently show high impact in attacking the model’s performance.
IP (Insert Punctuation), Formal, and Active are the three least effective text perturbation methods across different models.
Visual Reasoning Results and Observation
The averaged accuracy results of different methods under five perturbation levels are shown in Table 3 and Table 4, for image perturbation and text perturbation, respectively.
From Table 3, we can find that all the models’ performance dropped under image perturbation, especially ALBEF.
TCL shows better performance than ALBEF, where TCL introduced an intra-modal contrastive objective based on the ALBEF architecture, which may be helpful in improving the model’s performance.
In detail, Zoom Blur consistently shows the most effective impact on attacking all the models’ performance for both the dev set and test-P set.
In contrast, Glass Blur seems to be one of the least effective perturbation methods, while Gaussian
Noise, Defocus Blur, Fog, and JPEG Compression can also be not effective in attacking the model’s performance.
Besides, as shown in Table 4, all the models’ performance also drooped under text perturbation.
In detail, character-level perturbation still shows a much stronger influence than wordlevel and sentence-level perturbations for the visual reasoning task, and different models seem to be sensitive to different character-level perturbations.
The sensitivities of different models also vary, where Keyboard, OCR, CI, CR, CS, and CD show different impacts.
However, IP (Insert Punctuation) seems to be one of the least effective ones in attacking in the visual reasoning task, while SR, Formal,
Active, Back_trans can also be stable methods in different evaluation models.
Visual Entailment Results and Observation
The averaged accuracy results of different methods under five perturbation levels are shown in Table 5 and Table 6, for image perturbation and text perturbation, respectively.
Similar to the results in image-text retrieval and the visual reasoning tasks, the performance of all the models dropped under both image perturbation and text perturbation settings.
In detail, Zoom Blur still serves as the most powerful image perturbation method, and Brightness is the least effective one, as shown in Table 5.
In addition, as shown in Table 6, character-level perturbation also shows a much stronger influence than word-level and sentence-level perturbations for the visual entailment task, where IP, Formal, Casual, Passive, and Active can be stable perturbation strategies.
Unlike the results in image-text retrieval and the visual reasoning tasks, the performance drop seems insignificant in the VE task, which may be due to VE being a relatively easy task, so different model variations are not shown explicitly.
Limitation Given our work is one of the early efforts in this direction, there are several limitations and promising future work.
First, we only adopt synthetic image and text perturbation strategies in our benchmark.
However, there are other perturbation methods that could be explored for further robustness evaluation, e.g., real distribution shift [75, 84].
Second, we only study three downstream tasks, while there are more interesting ones, such as visual question answering, image captioning and text-to-image generation.
For those generation tasks, new evaluation metrics might be needed to properly evaluate the model’s robustness.
Third, we only evaluate these image-text models but the more important question is, how to improve their robustness.
Data augmentation is a common technique to improve unimodal models’ robustness [35, 32], which we could also explore for multimodal setting [31].
5.8 Related Work
Robustness of unimodal vision models is a longstanding and challenging goal of computer vision [89].
Stable training, adversarial robustness, out-of-distribution and transfer performance, and many other aspects have been studied by previous works in deep learning era [95, 16, 13, 27].
Recently,
Vision Transformer (ViT) has shown improved robustness compared with previous models, i.e., the comparison between ViT and ResNet for robustness against common corruptions and perturbations [3], robustness under distribution shifts and natural adversarial setting [63], robustness against different Lp-based adversarial attacks [55], adversarial examples [57], and adaptive attacks [2].
In terms of benchmark, [33] proposed ImageNet-C and ImageNet-P benchmarks for classifier’s robustness to common perturbations.
[36] proposed ImageNet-A and ImageNet-O benchmarks for adversarial filtration and out-of-distribution detection.
[66] proposed ImageNet-V2 for evaluating distribution shift.
[23] proposed Stylized-ImageNet by removing local texture cues in ImageNet while retaining global shape information on natural images via AdaIN style transfer.
Recently, [29] built the GRIT benchmark to evaluate the performance, robustness, and calibration of a vision system across a variety of image tasks, concepts, and data sources.
Robustness of unimodal language models under distribution shift or adversarial attack has also been explored by many previous works, i.e., [7, 81] provided reviews of how to define, measure and improve robustness of NLP systems, [80] proposed controlled adversarial text generation to improve robustness, [24] unified four standard evaluation paradigms, [73] proposed a search and semantically replace strategy, [14] studied robustness against word substitutions, [56] formalised the concept of semantic robustness, etc.
In terms of benchmark, [34] systematically examined and measured the out-of-distribution (OOD) generalization for seven NLP datasets.
[11] built a large benchmark and analyzed the impact of robustness on the performance of distribution shifts, calibration, out-of-distribution detection, fairness, privacy leakage, smoothness, and transferability.
Recently, [60] presented empirical results achieved with a comprehensive set of non-adversarial perturbation methods for testing the robustness of NLP systems on non-synthetic text.
[28] proposed a multilingual robustness evaluation platform that incorporates universal text transformation, taskspecific transformation, adversarial attack, and subpopulation to provide comprehensive robustness analysis.
[78] proposed a benchmark to evaluate the vulnerabilities of modern large-scale language models under adversarial attacks.
Multimodal learning has advanced quickly in recent years with appealing applications in different fields [49, 39, 59, 4, 37, 97, 17, 30, 51], i.e., embodied autonomous agents, image/video understanding, multimedia and healthcare.
Thanks to the larger datasets [64, 92, 72, 71] and larger transformer models [93, 8, 5, 10], many powerful multimodal image-text models have been developed and shown great capability.
However, unlike unimodal models, the robustness study of these multimodal models under distribution shift has rarely been explored.
Robustness of multimodal models is essential to study before deploying these amazing foundation models to real applications.
Previous works [21, 20, 26, 62] have unsystematically tested some pre-trained models, i.e., CLIP [64], by attacking with text patches and adversarial pixel perturbations. [77] measured the robustness of multimodal learning by fusing the input modalities and adversarial attack.
[19] found that diverse training distribution is the main cause for robustness gains.
[76, 43] investigated the audio-visual model robustness under multimodal attacks.
For benchmarks, [46] collected an Adversarial VQA dataset to evaluate the robustness of VQA models.
A concurrent work [69] studied the robustness of video-text models under perturbations, but their models, tasks, and datasets are different from ours.
In this work, we focus on studying robustness under distribution shifts for multimodal image-text models.
We introduce new datasets and metrics, and extensively evaluate recent multimodal models.
5.9 Detailed Image-Text Retrieval Results
In the appendix, we provide the detailed robustness evaluation results for the image-text retrieval task, where the evaluation datasets are Flickr30K and COCO.
In the following tables, we report the recall at K (R@K) metric, K = { 1 , 5 , 10 } , where Mean is the averaged recall results for either text retrieval or image retrieval, RSUM is defined as the sum of recall metrics at K = { 1 , 5 , 10 } of both image and text retrieval tasks.
5.9.1 Image Perturbations
[Table]
[Table] [Table]
Motion
[Table] [Table] [Table] [Table]
[Table]
Zoom
[Table]
Weather Snow
[Table]
Brightness
[Table]
Digital Contrast
[Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table] [Table]