What Are We Measuring When We Evaluate Large Vision-Language Models? An Analysis of Latent Factors and Biases (2024)

Abstract: Vision-language (VL) models, pretrained on colossal image-text datasets, have attained broad VL competence that is difficult to evaluate. A common belief is that a small number of VL skills underlie the variety of VL tests. In this paper, we perform a large-scale transfer learning experiment aimed at discovering latent VL skills from data. We reveal interesting characteristics that have important implications for test suite design. First, generation tasks suffer from a length bias, suggesting benchmarks should balance tasks with varying output lengths. Second, we demonstrate that factor analysis successfully identifies reasonable yet surprising VL skill factors, suggesting benchmarks could leverage similar analyses for task selection.Finally, we present a new dataset, OLIVE ^1 , which simulates user instructions in the wild and presents challenges dissimilar to all datasets we tested. Our findings contribute to the design of balanced and broad-coverage vision-language evaluation methods. ^1 https://github.com/jq-zh/olive-dataset

## introduction benefiting from enormous training data, large model sizes, and pretrained large language models, the current generation of vision-language models (vlms) (e.g., @xcite @xcite @xcite ) demonstrate competence in a wide range of tasks, including visual questionanswering, optical character recognition, and spatial relation recognition. however, their broad competence pose a new challenge to the design of evaluation benchmarks, as many previous work focuses on only assessing one or a few capabilities, relying on data from a single distribution and annotation pipeline. this approach can result in test data that fails to fully represent all potential user inputs, lead-ing to discrepancies between benchmark scores and real-world user experiences. a prevailing evaluation strategy involves testing across a range of tasks and computing an average score @xcite @xcite @xcite . this type of benchmarks usually involves a manual categorization of test tasks, as a benchmark that covers more categories is often considered more comprehensive and capable of measuring broad competence. for example, touchstone @xcite sorts tasks into five skills, ranging from visual recognition to visual storytelling. however, such categorizations are based on human intuition and lack support from empirical evidence. in this paper, we promote an alternative approach that identifies vision-language (vl) capabilities underlying various tests directly from data. inspired by the distributional hypothesis @xcite , we characterize test tasks using neighborhood structures inferred from transfer learning. that is, transfer learning between datasets that follow similar distributions and require similar vl capabilities will likely yield high performance. by analyzing transfer performance between a large number of source and target tasks, we can observe dataset similarity, infer shared vl capabilities, and gain insights into the vl benchmarks. specifically, we fine-tune four popular vlms with different strengths, blip-2 @xcite , mini-gpt4 @xcite , llava @xcite , and mplug-owl @xcite , on 23 training (source) tasks and evaluate them on 29 test (target) tasks. together with the original model performance before any finetuning (referred to as zero-shot performance), we obtain 2,784 performance measurements. after that, we examine the patterns and conduct exploratory factor analysis, discovering six interpretable latent factors underlying the measurements. the analyses reveal a few surprising findings. first, we find that a surface-form property, the average output length, has a surprisingly strong influence on transfer performance. this suggests current evaluation results may be affected by this length bias. second, factor analysis is capable of discovering unexpected yet reasonable factors that explain model performance. for example, we identify factors that differentiate reading text off images from multi-hop reasoning. these findings have important implications for the design of unbiased and comprehensive vl benchmarks. finally, to simulate real-world user instructions, we present a new vision-language dataset, openworld language instruction for visual-language evaluation (olive). olive consists of 9,450 images, 30,120 unique instructions, and 47,250 responses. empirically, we show that olive has a transfer profile distinct from all other datasets we tested and hence provides a test complementary to existing tasks. we summarize our contributions as follows: ## analysis techniques the transfer performance from n source (training) tasks to k target (test) tasks on model m is stored as a matrix a (m) ∈ r n ×k . the performance numbers of different tasks cannot be compared directly due to differences in the scales of the evaluation metrics. therefore, we first normalize the data. subsequently, we apply singular value decomposition and factor analysis. both techniques can be understood as decompositions of the matrices a (m) , albeit with different mathematical formulations. ## source and target tasks we gather 27 publicly available vl datasets and create variations, yielding 23 source tasks and 29 target tasks. we show the full list of tasks in tab. 1 and describe them below. the performance metrics used are auc for hateful memes, cider @xcite for opencqa, olive, and all captioning datasets, rouge-l @xcite for more and new yorker explanation, and accuracy for the remaining tasks. to focus on end-to-end per-formance, we do not perform any separate optical character recognition. image captioning. image captioning is one of the most popular image-text tasks and is commonly used as a pretraining task for vlms @xcite . here we select two classic datasets: coco caption @xcite and flickr30k @xcite . in addition, we include textcaps @xcite , which involves the description of textual content in images. we also include as a source task web capfilt, a set of synthetic image captions covering a wide variety of web images. web capfilt was generated by blip @xcite for self-training. we hypothesize that its diversity could be beneficial in transfer learning. ## conclusions in this work, we aim to empirically uncover latent factors and biases that contribute to the performance of various vlms on vl tasks. using efa, we identify six highly interpretable factors and biases that affect performance. we also contribute a new dataset, olive. we hope this research will lead to the creation of vl benchmarks with more balanced and complete coverage of vl capabilities.