Scalable Performance Analysis for Vision-Language Models (2023)

Abstract: Joint vision-language models have shown great performance over a diverse set of tasks. However, little is known about their limitations, as the high dimensional space learned by these models makes it difficult to identify semantic errors. Recent work has addressed this problem by designing highly controlled probing task benchmarks. Our paper introduces a more scalable solution that relies on already annotated benchmarks. Our method consists of extracting a large set of diverse features from a vision-language benchmark and measuring their correlation with the output of the target model. We confirm previous findings that CLIP behaves like a bag of words model and performs better with nouns and verbs; we also uncover novel insights such as CLIP getting confused by concrete words. Our framework is available at https://github.com/MichiganNLP/Scalable-VLM-Probing and can be used with other multimodal models and benchmarks.

## introduction recent years have witnessed an explosion of visionlanguage models @xcite @xcite @xcite . these models have shown great performance in a variety of tasks, such as image/video classification and text-image/video retrieval @xcite , even without leveraging task-specific or in-domain training. in addition, these models have shown to be practical when leveraged as underlying models for textto-image generation such as dall-e 2 @xcite and image captioning such as clip-cap @xcite . little is however known about the limitations of these models. recent work, such we test clip on the benchmark, extract a diverse set of semantic features from the data, and measure the correlation between each feature and the clip score (p , n , or d). features with positive correlation (e.g., female, plant) impact positively the model performance, while features with negative correlation (e.g., animal) impact negatively the model performance. as winoground @xcite , @xcite , or valse @xcite , have designed benchmark probing tasks by annotating data to follow specific properties (i.e., object color, location, size, swapping word order, replacing words). this line of research led to valuable insights into the limitations of current state-of-the-art multi-modal models such as clip @xcite and vilbert @xcite . an important limitation of current work is the reliance on time-consuming data annotation procedures, making it unscalable and limited in scope. as a complementary solution, we propose a method to probe vision-language models by relying on existing data, without requiring extra annotations. the method consists of extracting a large set of candidate features from a vision-language benchmark and testing their correlation with respect to the output of the target models on the given benchmark. by applying our method on clip @xcite , a widely used state-of-the-art multi-modal model, using the svo-probes (hendricks and nematzadeh, 2021) benchmark, we confirm the findings of @xcite of clip behaving like a bag of words model and that of @xcite of clip performing better with nouns and verbs. we also find that clip gets confused by concrete words and that it surprisingly improves in performance for more ambiguous words while noting little change from the word frequencies. to the best of our knowledge, we are the first to conduct an in-depth analysis of how language semantic properties influence clip's performance. we summarize our contributions as follows. first, we propose a scalable way of measuring the limitations of vision-language models. second, we test our method using a state-of-the-art visionlanguage model (clip) and a popular benchmark (svo-probes), validate known challenges, and uncover new ones. third, our work opens up avenues for future models to focus on solving the newly discovered challenges. ## related work recently, an increasing number of benchmarks have been created for the evaluation of visionlanguage model abilities to perform various multimodal tasks. hendricks and nematzadeh (2021) evaluate state-of-the-art vision-language models by building svo-probes, a probing benchmark focused on verb understanding. they show that image-language transformers fail to distinguish fine-grained differences between images and find that they are worse at verb understanding compared to subjects or objects. in our work, we continue their proposed future work direction by analyzing model performance on fine-grained verb categories. other work focuses on testing more precise capabilities of vision-language models using other probing techniques. in valse, @xcite demonstrate that vision-language models have difficulty in counting objects and in correctly classifying spatial relations between objects. @xcite ; @xcite show that, although state-of-the-art vision-language models can grasp color, they do not fully understand more difficult concepts such as object size and position in the image. in winoground, @xcite designed adversarial examples that require differentiating between a similar image and text, where the text pairs only differ in their word order. their results show that state-of-the-art vision-language models lack compositional reasoning abilities. several other works build benchmarks on probing visionlanguage on compositional reasoning @xcite @xcite @xcite ) find that they behave like a bag-of-words model -i.e., have poor relational understanding and a severe lack of word order sensitivity. in contrast, our work focuses not on creating new probing tasks for vision-language models, but on using current benchmarks to learn additional, more fine-grained features that can be discovered using simple correlation methods. to the best of our knowledge, we are the first to analyze the performance of clip on a diverse set of semantic features and use correlation methods to draw insights about what concepts are challenging for the model. ## methodology to probe clip given a benchmark, we measure how a visionlanguage model performs on a variety of semantic concepts. our aim is to quantify which concepts are the most and the least challenging for the model. our setting is illustrated in figure 1 , and can be described in three main steps. first, we use clip @xcite ) to compute scores for instances from the svo-probes (hendricks and nematzadeh, 2021) dataset and obtain two corresponding alignment scores for each sentence and its corresponding positive and negative image. next, we extract and process a diverse set of semantic features from svo-probes. finally, we compute the correlation coefficients between each feature and the clip score. the features with the highest coefficients will represent concepts that clip performs well on, while features with the lowest coefficients will represent challenging concepts for clip. ## results our main observations and takeaways from this evaluation are the following: this finding is also supported by the fact that many features from words in common contribute to increasing both the positive (p ) and the negative scores (n ): e.g., hypernym_food.n.02 increases p by 0.042 and n by 0.050; liwc "money" increases p by 0.036, and n by 0.032. as described in section 3.5, we measure the importance of each feature as the difference of means between the clip scores when the feature is present and when is not. we observed that many of the features for the words in common appeared to influence similarly both p and n , confirming this hypothesis. figure 5 shows how the clip scores are distributed for the different number of words, showing for example that there is a great overlap between the similarity scores between texts of length 6 and a negative image, and the similarity scores between texts of length 3 and a positive image. this implies clip is more likely to select the wrong text when comparing an image with a short correct text and one with long incorrect text. ## conclusion in this work, we proposed a simple and effective method to probe vision-language models. our method is scalable, as it does not require data annotation and makes use of existing datasets. with our method, we analyzed the performance of clip, a popular state-of-the-art multi-modal model, on the svo-probes benchmark. we confirmed the recent findings of @xcite of clip behaving like a bag of words model and that of @xcite of clip performing better with nouns and verbs. we also uncovered novel findings, for instance, that clip gets confused by concrete words, surprisingly improves performance for more ambiguous terms, or that the frequency of words does not significantly change the behavior of clip. we hope our work contributes to ongoing efforts to discover the limitations of multi-modal models and help build more robust and reliable systems. our framework can be easily used to analyze other benchmarks, features, and multi-modal models, and it is publicly available at https://github.com/michigannlp/ scalable-vlm-probing .