Title: Vision Language Models are Biased
Primary area: datasets and benchmarks
Keywords: vision language models, multimodal reasoning, benchmark, bias

Abstract: Large language models (LLMs) memorize a vast amount of prior knowledge from the Internet that helps them on downstream tasks but also may notoriously sway their outputs towards wrong or biased answers. In this work, we test how the knowledge about popular subjects hurt the accuracy of vision language models (VLMs) on standard, objective visual tasks of counting and identification. We find that state-of-the-art VLMs are strongly biased (e.g., unable to recognize the 4th stripe has been added to a 3-stripe Adidas logo) scoring an average of 17.05% accuracy in counting (e.g., counting stripes in an Adidas-like logo) across 7 diverse domains from animals, logos, chess, board games, optical illusions, to patterned grids. Our work presents an interesting failure mode in VLMs and a human-supervised automated framework for testing VLM biases.

TL;DR: A benchmark to demonstrate that VLMs primarily rely on prior knowledge while ignoring visual input.