言語処理学会 第29回年次大会 発表論文集 (2023年3月)

On the Bias of CLIP for Object-Attribute Recognition
Yutaro Yamada1∗ , Yingtian Tang2∗ , Ilker Yildirim1
1 Yale University
2 University of Pennsylvania{yutaro.yamada,ilker.yildirim}@yale.edu yingtian@seas.upenn.edu

概要
Large-scale vision-language models such as CLIP haveshown impressive performance on zero-shot image classification and image-to-text retrieval. However, such zeroshot performance of CLIP-based models does not realize intasks that require a finer-grained correspondence between
vision and language, such as Visual Question Answering(VQA). We investigate why this is the case, and report aninteresting phenomenon of CLIP, which we call the Concept Association Bias (CAB), as a potential cause of the
difficulty of applying CLIP to VQA and similar tasks. CAB
is especially apparent when two concepts are present in thegiven image while a text prompt only contains a single concept. In such a case, we find that CLIP tends to treat input
as a bag of concepts and attempts to fill in the other missing
concept crossmodally, leading to an unexpected zero-shot
prediction.

1

Introduction

Recent large-scale vision-language models such as CLIP[1] and ALIGN [2] have shown remarkable performanceon zero-shot classification and text-image retrieval tasks.
These models are trained via cross-modal contrastive
learning on web-scale image-text pairs and obtain powerfulmultimodal representations. Encouraged by these strong
zero-shot capabilities, several recent papers explored CLIP
for more complicated vision-language tasks. The initial attempt made by [3] reports near chance accuracy for zeroshot performance of CLIP on VQA-v2 [4], a common visual question answering benchmark. However, their approach simply uses “question: [question text] answer: [answer text]” as text input for the text encoder of CLIP,
which makes the prediction harder than it should be. A
subsequent work [5] proposes a better prompt generation
method. They first convert a question into a masked prompt

CLIP: "In this picture, the color of the lemon is purple."

図 1 When we ask CLIP the color of the lemon in the above
image, CLIP answers “purple”. The text prompt we use is “In thispicture, the color of the lemon is [mask]”, where CLIP picks one
from [red, green, yellow, orange, purple].

(e.g. “What’s in the bowl behind the cake” becomes “The[mask] is in the bowl behind the cake”), and filter impossible answers using a language model, which improves
CLIP’s zero-shot performance on VQA-v2.
However, the zero-shot performance of CLIP on VQAv2 is still not state-of-the-art, which is achieved by taskspecific models [3]. While investigating what makes CLIP
hard to adapt to VQA, we discover an interesting phenomenon, which we call the Concept Association Bias
(CAB).
To describe this phenomenon, we present a simple image containing a “lemon” and an “eggplant” to CLIP, andask what color the lemon is, as shown in Figure 1. Surprisingly, CLIP predicts “purple” with high confidence. When
we instead ask for the color of the eggplant, CLIP answers“yellow”. To cross-check this phenomenon, we formulate abinary zero-shot image classification task on the same image where the two labels are “yellow lemon” and “purple
lemon”, and find that CLIP predicts “purple lemon” withhigh confidence.
We hypothesize that this phenomenon comes from thediscrepancy between what is described in the image andtext input, where CLIP attempts to fill in the missing con-

― 2295 ―This work is licensed by the author(s) under CC BY 4.0(https://creativecommons.org/licenses/by/4.0/).

cept. The association between “purple” and “eggplant” isfact, image generation models that do not use CLIP such as

strong, so when asked to fill in the mask in “[mask] lemon”,

Imagen and Parti are known to be better at generating im-predicting “purple” instead of “yellow” makes more sense

ages that require compositional reasoning [18, 19]. How-

for CLIP, because the text description of “purple lemon” isever, few works go into depth to analyze the behavior ofaligned with the image that contains both a lemon and an

CLIP in zero-shot image classification and visual question

eggplant more faithfully than “yellow lemon”, which only
describes the lemon in the image. In fact, when we ran-answering. Our analysis based on CAB offers a new perspective on the weakness of CLIP-based models for com-

domize the color of the lemon and eggplant (e.g. “red”

positional reasoning.

for lemon and “green” for eggplant), we find that this bias

3

disappears, and CLIP picks the color almost randomly between the two.

The Concept Association Bias
The zero-shot image classification of CLIP is remark-Vision-language models such as CLIP are being de-able for images that contain a single concept. However,ployed for increasingly broad range of downstream appli-

when there are multiple concepts in the image but the text

cations [6, 7, 8, 9]. However, the concept association bias

input does not cover all of them, the zero-shot classifica-suggests caution in such efforts.

tion of CLIP can be significantly biased towards the miss-2

ing concept(s). We call this bias the Concept Association

Related Work

Bias (CAB). We first showcase this bias using color recog-

Vulnerability of vision and language models

nition tasks.1） For this analysis, we use the Natural-Color

There are a number of papers that study the robustness of

Dataset (NCD) [20], which is a dataset of vegetables andvision and language models. Some prior work [10] shows

fruits with a white background. We take the following ob-that Transformer trained via Masked Language Modelingjects: banana, brinjal, broccoli, carrot, cherry, corn, cu-[11] is insensitive to word orders, suggesting that the suc-

cumber, lemon, orange, plum, pomegranate, strawberry,cess of BERT largely depends on learning higher-order

tomato. We then randomly sample two images with differ-word co-occurrence rather than learning syntactic and se-ent vegetable types and place the two objects side-by-side,

mantic abstractions. Many benchmarks are proposed toresulting in 494 images in total. Examples are shown inevaluate robustness of ImageNet models towards various

Figure 2.

perturbations including common corruption [12], imagestyle change [13], and different viewpoints [14]. Our work
differs from these studies that are purely based on language
or vision, because CAB is a cross-modal phenomenon,
which occurs when both image and language data are used.
[15] tests compositional generalization of vision and language models. [16] introduced a probing dataset called
Winoground, which evaluates visuo-linguistic compositionality of vision and language models. They evaluate adiverse range of state-of-the-art vision and language mod-

図 2 Example images from Natural-Color Dataset (NCD) [20],modified for our color recognition tasks so that each image contains two different objects.

For zero-shot transfer from CLIP to our color recogni-

els, including CLIP, but all of them perform close to ortion task, we ask for the color of one of the objects in the

below random chance. Our work also reveals brittleness ofimage. The labels we use are “red”, “yellow”, “purple”,

CLIP through the lens of CAB, which has been overlooked“green”, and “orange”, so it is a 5-way color recognitionin the past.
In the image generation com-

task. When there is a single object in the image, we use thefollowing text prompt: “In this picture, the color of the ob-munity, it has been reported that state-of-the-art models

ject is [mask].” When there are two objects in the image,such as DALL·E 2 [6] struggle with composionality [17].

we specify one of these objects in the prompt. For exam-

Peculiarities of CLIP

One of the potential causes of such failure has been attributed to the use of CLIP-based image encoder [6]. In

1） For all experiments in the main text, we use the ResNet50-x4 backbone for CLIP.

― 2296 ―This work is licensed by the author(s) under CC BY 4.0(https://creativecommons.org/licenses/by/4.0/).

ple, if there is a lemon and another object in the image,

Image
Lemon
Lemon

the prompt takes the following format: “In this picture, thecolor of the lemon is [mask].”

Text

Eggplant
Eggplant

Yellow
Yellow-

The color of the lemonis [mask].

Eggplant=

PurplePurplePurple(a) Natural color
Image---------------------

Lemon
Lemon

Text

Eggplant
Eggplant

Yellow
Yellow-

The color of the lemon
is purple.

=

PurplePurpleCAB gap(b) Natural color ([mask] = purple)
Text

Image
Red

Lemon

図 3 Zero-shot performance of CLIP on color recognition tasks
using NCD [20]. CLIP achieves almost perfect accuracy when
there is a single object in the image, but the accuracy significantly
drops when there are two objects. “Two object*” refer to the casein which we instead measure the accuracy of predicting the colorof the object B when it is asked for the color of the object A, where
we see 80% zero-shot accuracy. We claim this gap between Two
objects and Two objects* is a result of the Concept Association
Bias (CAB).

The results are shown in Figure 3. We first note that thezero-shot performance of CLIP on our color recognition
task is almost perfect when there is a single object per image (“Single object” in Figure 3). However, the classification performance considerably degrades to below chancewhen there are two objects per image (“Two objects” inFigure 3).
How does this happen? We suggest that CLIP does nothave a mechanism that stores object-centric representationthat correctly binds the object’s name and its attribute. In

Eggplant
Eggplant-

Eggplant

The color of the lemonis [mask].

=

Green

Red

Green

(c) Unnatural color

図 4 The concept binding diagram. Two variables per object
represent the object name and its attribute (e.g. color), respectively. We suggest that the text prompt and the image are represented as two separate “bags of concepts” in CLIP. When a pairof object-attribute concepts are naturally associated with eachother, then both concepts can be accounted for by including in
the prompt either of the object or the attribute. When only someof the concepts in the image are included in the text, this leaves
other concepts in the image unaccounted for.

resents the object’s name in the image and the color attribute of the object, as shown in Figure 4. When the colors
are natural (Figure 4 (a)), both the object “lemon” and its
attribute “yellow” in the image are fully explained by theword “lemon” in the text prompt, resulting in the conceptof the eggplant remaining. When CLIP performs zero-shotcolor recognition, we see that placing the color “purple” inthe prompt can most faithfully explain the remaining concept of the eggplant in the image (Figure 4 (b)).

another words, CLIP processes its input as a “bag of concepts”.
To inspect this possibility, we look at what kind of mistakes CLIP makes when there are two objects A and B. We
find that many mistakes are derived from a common source.
That is, when asked for the color of object A, CLIP often
predicts the color of object B in the image. In fact, when wemeasure the accuracy of predicting the color of the object B

図 5 Examples from UNCD. Single object (Top) and Two objects per image (Bottom).

The above explanation suggests that there is a strong as-

when in reality it is asked to predict the color of the object
A, we see that the zero-shot transfer performance of CLIP

sociation between the color “purple” and the object “egg-is much higher (“Two objects*” in Figure 3), approachingplant” in CLIP to the point where “purple” can partiallythe single object accuracy.

explain the concept of the eggplant. What if we break thisTo understand this phenomenon, we find it helpful toconsider two variables per object, where each variable rep-strong association? Does the gap between Two objects andTwo objects* disappear?

― 2297 ―This work is licensed by the author(s) under CC BY 4.0(https://creativecommons.org/licenses/by/4.0/).

Zero-shot transfer from CLIP to unnatural color recognition
1.00

pears in red. Likewise, for the Two objects* task (in which
the correct prediction is defined as the color of object B
when asked for object A), CLIP essentially randomly picks
one of the two colors present in the image, despite the fact

0.75

Accuracythat each object has their own very distinct color.
0.50

Prediction accuracy of different spatial arrangements
Two objects

Two objects*

0.25
left & right

0.00
Single object

Two objects

Two objects*

up & down

図 6 Zero-transfer performance of CLIP to color recognition on
UNCD, where we assign non-associated color to each vegetable.
CLIP achieves 80% accuracy when there is a single object in theimage. While the accuracy drops for Two objects, the drop is notas significant as the NCD case. Furthermore, the gap between
Two objects and Two objects* vanishes, compared to the NCD
case.

To test this, we create a version of NCD, which wecall UNnatural-Color Dataset (UNCD), where we artificially change the color of each fruit and vegetable to nonassociated color. Examples are shown in Figure 5. We repeat the same experiment on UNCD. The results are shown

upper-left &down-rightlarge & small
0.00

up the color attribute even if the color is not strongly associated with the object itself. However, for the two object

0.50

0.75

1.00

Accuracy

図 7 The Concept Association Bias (CAB) remains regardlessof the spatial configurations such as “left & right”, “up & down”,“upper-left & down-right”, and “large & small”. We use the same
subset of NCD as in Figure 3.

3.1

in Figure 6. We see that the zero-shot performance for asingle object is still high, suggesting that CLIP can pick

0.25

The spatial arrangement has almost
no effect on CAB

In our earlier experiments on NCD and UNCD, two objects are positioned side-by-side. To see if CAB is robust to
the positioning of objects, we vary the spatial arrangementcases, we see that there is almost no difference betweenof the two objects in the image. Concretely, we test theTwo objects and Two objects* tasks. In other words, CLIP
predicts the two non-associated colors in the image withand upper-left & down-right. We also vary the size of thealmost equal chance.
Why does the CAB gap disappear when objects are
paired with random attributes in images? This result arises
from a common mechanism that impacts both the Two objects and Two objects* tasks. To see this, we go back to
our diagram in Figure 4 (c). When the colors are unnatural(e.g., a lemon in red color and an eggplant in green color),then the remaining bag of concepts that are yet to be explained by the text include “red”, “green”, and “eggplant”.
This is because the color “red” is not associated with theconcept of “lemon”, and therefore the word “lemon” in the
text prompt cannot explain the color “red”, unlike the casethat uses natural color. As a result, CLIP can choose either “red” or “green” for color recognition. And indeed,surprisingly, CLIP randomly chooses between the two –it does not associate the concept of “red“ with the lemon,

following spatial configurations: left & right, up & down,
two objects for left & right, which is denoted as “large &small”. As Figure 7 shows, CAB is not affected by either
spatial arrangements or the object size.

4

Conclusion
Every object has a set of concepts that are roughly asso-

ciated with it. For instance, the object “lemon” can be associated with “yellow”, “fruit”, and so on. Such concept association is automatically learned in vision-language models such as CLIP, to the point where the word “yellow” canpartially explain the object “lemon” in certain cases. We
establish that the Concept Association Bias (CAB) exists
for CLIP through a series of experiments. CLIP is increasingly popular in both computer vision and natural language
processing. We hope our work raises awareness of the brittleness of CLIP as we develop new models on top of CLIP.

even though in the image the lemon unambiguously ap-― 2298 ―This work is licensed by the author(s) under CC BY 4.0(https://creativecommons.org/licenses/by/4.0/).

Acknowlegments[11]

Computational resource of AI Bridging Cloud Infrastructure (ABCI) provided by National Institute of Advanced Industrial Science and Technology (AIST) wasused for the experiments in this paper.

参考文献

[12]

[1] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya
Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry,
Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen
Krueger, and Ilya Sutskever. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Conference on
Machine Learning, pp. 8748–8763. PMLR.
[2] Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana
Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li,
and Tom Duerig. Scaling Up Visual and Vision-Language
Representation Learning With Noisy Text Supervision. In
Proceedings of the 38th International Conference
on Machine Learning, pp. 4904–4916. PMLR.
[3] Sheng Shen, Liunian Harold Li, Hao Tan, Mohit Bansal,
Anna Rohrbach, Kai-Wei Chang, Zhewei Yao, and Kurt
Keutzer. How Much Can CLIP Benefit Vision-andLanguage Tasks?
[4] Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv
Batra, and Devi Parikh. Making the v in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering. pp. 6904–6913.
[5] Haoyu Song, Li Dong, Weinan Zhang, Ting Liu, and Furu
Wei. CLIP Models are Few-Shot Learners: Empirical
Studies on VQA and Visual Entailment. In Proceedingsof the 60th Annual Meeting of the Association
for Computational Linguistics (Volume 1: Long
Papers), pp. 6088–6100. Association for Computational
Linguistics.
[6] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey
Chu, and Mark Chen. Hierarchical Text-Conditional Image Generation with CLIP Latents.
[7] Renrui Zhang, Ziyu Guo, Wei Zhang, Kunchang Li, Xupeng Miao, Bin Cui, Yu Qiao, Peng Gao, and Hongsheng
Li. PointCLIP: Point Cloud Understanding by CLIP. pp.
8552–8562.
[8] Aditya Sanghi, Hang Chu, Joseph G. Lambourne,
Ye Wang, Chin-Yi Cheng, Marco Fumero, and Kamal Rahimi Malekshan. CLIP-Forge: Towards Zero-Shot
Text-To-Shape Generation. pp. 18603–18613.
[9] Can Wang, Menglei Chai, Mingming He, Dongdong Chen,
and Jing Liao. CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields. pp. 3835–3844.
[10] Koustuv Sinha, Robin Jia, Dieuwke Hupkes, Joelle Pineau,
Adina Williams, and Douwe Kiela. Masked Language
Modeling and the Distributional Hypothesis: Order Word
Matters Pre-training for Little. In Proceedings of the2021 Conference on Empirical Methods in Natural
Language Processing, pp. 2888–2913. Association for

[13]

[14]

[15]

[16]

[17]
[18]

[19]

[20]

― 2299 ―Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina
Toutanova. BERT: Pre-training of Deep Bidirectional
Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American
Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1(Long and Short Papers), pp. 4171–4186. Association
for Computational Linguistics.
Dan Hendrycks and Thomas Dietterich. Benchmarking
Neural Network Robustness to Common Corruptions andPerturbations.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu,
Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt,
and Justin Gilmer. The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization.
Andrei Barbu, David Mayo, Julian Alverio, William Luo,
Christopher Wang, Dan Gutfreund, Josh Tenenbaum, andBoris Katz. ObjectNet: A large-scale bias-controlled
dataset for pushing the limits of object recognition models. In Advances in Neural Information Processing
Systems, Vol. 32, pp. 9453–9463.
Ben Bogin, Shivanshu Gupta, Matt Gardner, and Jonathan
Berant. COVR: A Test-Bed for Visually Grounded Compositional Generalization with Real Images. In Proceedings of the 2021 Conference on Empirical Methods
in Natural Language Processing, pp. 9824–9846. Association for Computational Linguistics.
Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet
Singh, Adina Williams, Douwe Kiela, and Candace Ross.
Winoground: Probing Vision and Language Models for
Visio-Linguistic Compositionality. pp. 5238–5248.
Royi Rassin, Shauli Ravfogel, and Yoav Goldberg.
DALLE-2 is Seeing Double: Flaws in Word-to-Concept
Mapping in Text2Image Models.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala
Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed
Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi,Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho,
David J. Fleet, and Mohammad Norouzi. Photorealistic
Text-to-Image Diffusion Models with Deep Language Understanding.
Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong,
Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander
Ku, Yinfei Yang, Burcu Karagol Ayan, Ben Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason
Baldridge, and Yonghui Wu. Scaling Autoregressive Models for Content-Rich Text-to-Image Generation.
Saeed Anwar, Muhammad Tahir, Chongyi Li, Ajmal Mian,
Fahad Shahbaz Khan, and Abdul Wahab Muzaffar. Image
Colorization: A Survey and Dataset.

This work is licensed by the author(s) under CC BY 4.0(https://creativecommons.org/licenses/by/4.0/).