An Explainable Toolbox for Evaluating Pre-trained Vision-Language Models (2022)

Abstract: We introduce VL-CheckList, a toolbox for evaluating Vision-Language Pretraining (VLP) models, including the preliminary datasets that deepen the image-texting ability of a VLP model. Most existing VLP works evaluated their systems by comparing the fine-tuned downstream task performance. However, only average downstream task accuracy provides little information about the pros and cons of each VLP method. In this paper, we demonstrate how minor input changes in language and vision will affect the prediction outputs. Then, we describe the detailed user guidelines to utilize and contribute to the community. We show new findings on one of the representative VLP models to provide an example analysis. The data/code is available at https://github.com/om-ai-lab/VL-CheckList

## introduction the ability to quickly iterate various methods and obtain insightful feedback is crucial for successful research. for production machine learning (ml) system, comprehensive testing before deployment is crucial for reliable user experience. therefore, explainable ml evaluation has emerged to complement benchmark evaluation @xcite @xcite , which strives to provide an interpretable evaluation of a ml systems and analyze the system from a number of disentangled aspects @xcite . the advantages of explainable evaluation vs. typical benchmark evaluation include: (1) downstream task performance only provides a black box score and it is difficult to obtain insights for improving a system. (2) typical dataset is not designed to test models' robustness against extreme corner cases, which are however crucial for many real-world tasks, e.g. autonomous driving. given the importance of explainable ml evaluation, this paper concerns about vision-language pretraining (vlp) models. vlp models have rapidly improved @xcite @xcite , thanks to the emergence of multimodal transformers @xcite and the availability of large paired image-text dataset @xcite . many proposed vlp models have aided in achieving the state-of-the-art performance of a variety of downstream multimodal tasks, ranging from visual qa @xcite , multimodal retrieval @xcite to visual grounding @xcite and many others. on the other hand, the current defacto method to evaluate a vlp model is based on the fine-tuned downstream tasks performance, which is insufficient due to the limitations of benchmark evaluation. to address this challenge, this paper introduces vl-checklist, an explainable framework that comprehensively evaluates vlp models, facilitates deeper understanding, and inspires new ideas for improvement. the core principle of vl-checklist are: (1) evaluate a vlp model's fundamental capabilities instead of its performance on applications (2) disentangle capabilities into relatively independent variables that are easier to analyze. specifically, we choose image-text-matching (itm) as the main target of evaluation since it is perhaps the most universal pretraining objective that appear in all vlp methods @xcite @xcite . we then propose a taxonomy that divides the capabilities of vlp systems into three categories: object, attribute and relation. each aspect is then further divided into more fine-grained variables, e.g. attribute is composed of color, material, and size, etc. then, a linguisticaware negative sample sampling strategy is proposed to create "hard negative" that challenges a vlp model's discriminative power against small changes in the input space. lastly, vl-checklist is implemented as a toolbox that allows the research community to plug into their evaluation pipeline. ## related work benchmark evaluation is a common method to compare different ml models in previous research @xcite @xcite . however, researchers have reported several limitations of the existing vlp benchmark. 1) the objects of interest have a biased distribution of size and location, i.e., tend to be large objects that appeared in the center region. 2) benchmark evaluation returns only a plain score instead of fine-grained details on the taxonomy. therefore, it is difficult to understand the strengths and weaknesses of a model without a comprehensive analysis. recent studies show even the state-of-the-art systems that achieved better scores than humans, may still be insufficient in real-world applications @xcite . thus, researchers have attempted to evaluate ml models with more fine-grained details and avoid bias on the test set. one of the successful tools for the qualitative analysis of natural language processing (nlp) is checklist @xcite which evaluates general linguistic capabilities and revealed weaknesses in several state-of-the-art nlp models and commercial applications. in computer vision, the vision checklist was proposed to help system designers to understand model capabilities @xcite . they offered a set of transformation operations to generate diverse test samples of different test types, such as rotation, replacing image patches, blur, and shuffle. however, target objects in the transformed images are unchanged, still center and large. the idea of the checklist has also been applied other fields, e.g. evaluating reinforcement learning (rl) agents @xcite , dynabench @xcite was proposed to generate dynamic benchmark datasets. it over-comes the problem that the existing benchmark fails to cover fundamental linguistic challenges. tide @xcite ) is a tool to analyze the errors of object detection. it defines critical error types and shows a comprehensive analysis. ## vl-checklist an intuitive approach to evaluate multi-modal systems is to check if a model correctly predicts alignment between different modalities. we choose image-text matching (itm) to check the alignment between vision and language for the following reasons. specifically, itm is defined as the function that outputs the probability of an image i is matched to a sentence t. the itm task is used as an effective and universal pretraining objective in almost all vlp models @xcite . the itm task is also model agnostic and applies to all multimodal fusion architectures. thus, we exploit the itm to fairly compare the vlp models without finetuning them on downstream tasks. the basic principle of the vl-checklist is to probe the model's robustness on the negative examples. a robust vlp model should be able to return a higher itm score for the positive image-text pair than the negative example on the itm head. we perturb the one-side modality to manipulate them and compare the score with original samples. lv-checklist offers both language-side and vision-side variations. ## detailed user guideline this section describes a guideline for researchers to use and contribute to the vl-checklist project. first, users can install from github foot_0 or from pip install vl-checklist. we further provide a huggingface demo for people to try out different vlp models foot_1 . then the following is a step-by-step guideline to use vl-checklist. 1) define corpus: a user defines a corpus in the yaml config file. we provide four initial pre-processed corpora using the semistructured dataset such as vg @xcite , swig @xcite , vaw @xcite and hake @xcite . we build a benchmark dataset for each capability test in the proposed framework. we provide the pre-processed datasets in the corpus folder of our github page. an example of the corpus config yaml file is as follows: anno_path: " a t t r / a c t i o n . j s o n " img_root: " v g / " type: " t u p l e _ j s o n " anno_path is the specific json file path that includes positive and negative captions and the specific image path. the data type is tuple_json. we converted the corpus into list of image path and captions(positive and negative), in the format of a list of [[{image_path:str, "pos":pos_captions:list, "neg": neg_captions:list}] . . . ] dir: " o u t p u t / c l i p " the "max_num" is the maximum number of data points and the "model_name" needs to be specified. appropriate "batch_size" should be input based on the gpu resources. the "task" can be either "itc" or "itm". the "itc" score compares models' scores on both positive and negative captions. it counts as a true positive when the score on the original is higher than the negatively transformed one. the "itm" is predicting each image and a caption. it is called the true positive when a softmax score on a positive example on the image is higher than the threshold of 0.5. the data tag consists of types and test_data. the types is the storage paths of the "ymal_files". in the top-level directory, we can divide it into three categories: object, relation, and attribute. for swig, vg, etc., there are multiple data subsets, so the data subset type should be filled in the test_data. we can specify the evaluation data, output directory, and format as the example above. after defining a config file, users can simply start the evaluation as follows:. from engine import model from vl_checklist import evaluate if __name__ = = ' __main__ ': @xmath0. ckpt ') @xmath1. yaml " , @xmath2. start () ## experimental settings in this section, we profile one of the most representative vlp models, clip @xcite by testing its ability to understand an object, attribute, and relationship between a text prompt and a given image for language variations. metric: we return the model output scores between the text description and the generated negative samples. if the model score on the original text description is higher than the score on the generated negative samples, we regard it as positive output. we obtain the accuracy with the following equation. ## results and analysis in general, the ability of clip to understand object changes is promising when the object is center and large (see the prefix-o at figure 3 ). we hypothesize that the clip model pays more attention to the central region and focus on salient objects, similar to the perspective of human observation. on the other hand, clip's ability on recognizing attribute and relationrelated variants is surprisingly low, especially for relation-spatial variations (figure 3 ). then, we investigate whether performance can be improved by cropping the regions of interest (roi) first and then encoding the cropped rois via clip. we extract text descriptions on each bounding box on the vg dataset to form a new image-text pair (image local ,text), and construct new datasets for vg: local subj , local obj . results on local subj and local obj show that region clip outperforms the original clip (whole image encoding) by 3.9% and 5.7% respectively ( and text spans. thus the understanding of minor objects in the image for clip is still challenging and explore more fine-grained regionto-text multimodal alignment is a promising direction @xcite . for vision variations, we synthesize images by changing an object's size and location. in figure 2 , the image on the left is a big apple in the center, while the image on the right is a small apple in the corner. the text prompt we input is "an apple on the grass" and "a dog on the grass". the accuracy of the left image with a big and center apple is nearly 1.00, while the right image with a small and corner apple only obtains 0.127 of accuracy. the location and size of the object in the image can significantly affect the judgment of the model. thus experimental results indicate that the current benchmark evaluation reveals a gap of performance for real applications. clip mostly focus on objects that appeared in the center of the image and the size of the objects should be large. this limits its performance if the target objects are minor in the marginal regions for real-world applications. ## conclusion this paper introduces vl-checklist to analyze vlp models from language and vision variations. for language variance, we evaluated from three aspects: object, attribute and relation. for vision variance, we generated synthesized images using cropped target objects and background. we found limitations of the clip model: 1) limited understanding for small objects in the corner 2) incompetence for recognizing relations and attributes. in the future, we plan to include more fine-grained taxonomies and synthesizing strategies into vl-checklist and also improve existing vlp methods under the guidance of vl-checklist report.