Language-biased image classification: evaluation based on semantic representations
Authors: Yoann Lemesle
Masataka Sawayama
Guillermo Valle-Perez
Maxime Adolphe
Hélène Sauzéon
Pierre-Yves Oudeyer
Venue: ICLR 2022 Poster (2022)
Keywords: interpretation of learned representations
language and visual processing
language-biased image classification
cognitive science

Abstract: Humans show language-biased image recognition for a word-embedded image, known as picture-word interference. Such interference depends on hierarchical semantic categories and reflects that human language processing highly interacts with visual processing. Similar to humans, recent artificial models jointly trained on texts and images, e.g., OpenAI CLIP, show language-biased image classification. Exploring whether the bias leads to interference similar to those observed in humans can contribute to understanding how much the model acquires hierarchical semantic representations from joint learning of language and vision. The present study introduces methodological tools from the cognitive science literature to assess the biases of artificial models. Specifically, we introduce a benchmark task to test whether words superimposed on images can distort the image classification across different category levels and, if it can, whether the perturbation is due to the shared semantic representation between language and vision. Our dataset is a set of word-embedded images and consists of a mixture of natural image datasets and hierarchical word labels with superordinate/basic category levels. Using this benchmark test, we evaluate the CLIP model. We show that presenting words distorts the image classification by the model across different category levels, but the effect does not depend on the semantic relationship between images and embedded words. This suggests that the semantic word representation in the CLIP visual processing is not shared with the image representation, although the word representation strongly dominates for word-embedded images.

Published as a conference paper at ICLR 2022
LANGUAGE-BIASED IMAGE CLASSIFICATION: EVALUATION BASED ON SEMANTIC REPRESENTATIONS
Yoann Lemesle*
Masataka Sawayama
Guillermo Valle-Perez
INRIA, France
INRIA
INRIA
ENS Rennes, France
France
France
Maxime Adolphe
Hélène Sauzéon
Pierre-Yve Oudeyer
INRIA
INRIA, France
INRIA, France Microsoft Research Montreal
France
Université de Bordeaux, France
ABSTRACT
Humans show language-biase image recognition for word-embedded image. known as ficture-word interference. Such interference depends on hierarchical semantic categories and reflects that human language processing highly interacts with visual processing. Similar to humans, recent artificial models jointly trained on texts and images, e.g., OpenAI CLIP, show language- image classifica- tion. Exploring whether the bias leads to interference similar to those observed in humans can contribute to understanding how much the model acquires hierar- chical semantic representations from joint learning of language and vision. The present study introduces methodological tools from the cognitive science literature introduce benchmark
assess the biases of artificial models. Specifically, task to test whether words superimposed on images can distort the image clas- sification across differrntt category levels and, if can, whether the perturbation is due to the shared semantic representation between language and vision. Our dataset is set of word-embedded images and consists of a mixture natural im- age datasets and hierarchical word labels with superordiinte/basic category levels. Using this benchmark test, we evaluate the CLIP model. We show that presenting words distorts the image classification by the model across different category lev- els, but the effect does not depend on the semantic relationship between images and embedded words. This suggests that the semantic word representation in the CLIP visual processing is not shared with the image representation, although the word representation strongly dominates for word-embedded images.
INTRODUCTION
Language is not only fundamental tool enabling humans to communicate with each other: also shapes other aspects of perception and cognition, ranging from influencing visual processing to en- abling abstract thinking (Vygotsky 1934 Miller 1951 Waxman & Markow 1995] Colas et al, 2021p. Analogous to this human ability, recent artificial intelligence technology has also utilized language to drive the training of visual skills in machines. For instance, the OpenAl team intro- duced CLIP (Contrastive Language-Image Pre-training), consisting of joint learning of language and vision (Radford et al., 2021). The CLIP model efficiently learns visual concepts from natural language supervision and can be applied to various visual tasks in zero-shot manner. Joint learning of language and vision has also been utilized in the Visual Question Answering (VQA) literature. where artificial models are asked to answer questions about visual content (Antol al. 2015
et al. 2016 Fukui et al. 2016)).
"equal contribution
masataka.a www.aaaaaaaaaaaaaaaaaaaa..a inria.fr; sawayama@mswyy.comm

---

Published as a conference paper at ICLR 2022
Classification task type
What the superordinate category of the image?
What basic category of the image?
What the superordinate category of the image?
What the basic category of the image?
Superimposed words
Superordinate word
Basic word
(S/S)
(B/S)
(S/B)
(B/B)
electronic
electronic
laptop
laptop
New prediction electronic | 99 90% Original prediction. animal 72% 0.19%
New prediction dogl 50.97% Original prediction. dogl 60. 80.78% 50.97%
New prediction electronic 60.00% Original prediction animal 45. 72%% 14.94%
New prediction laptop | 85.38% Original prediction dog | 60. 78% 4.48%
Figure I: Overview of our benchmark test. Our dataset is a set of word-superimposee images con- sisting of the natural image datasets and hierarchical word labels. The classification tasks are divided into four conditions: superordinate image classification for the image with the superordinate cate- gory word (S/S), superordinate image classification for the image with the basic word (S/B), basic image classification for the image with the superordinate category word (B/S), and basic classifica- tion for the image with basic category word (B/B).The classification examples by the CLIP model are shown in Appendix AI
While joint training with language effectively fosters the acquisition of general abilities for visual recogaitiio/understandingg its abstraction can produce biased recognition for either humans ma- chines. The Stroop effect known in human cognitive science literature is one of such biases (Stroop) I935). When human participants observe a mismatched word, where the word "red" is printed with the "blue" color, and are asked to answer the "color" of the word, their response to the word can be slower than when they see the word "red" with the "red" color. In addition to the color-word in- terference, language also interferes with human image category recognition, known as picture-wort interference (Rosinski 1977 Lupker 1979). When a participant observes an image coupled with an incongruent category word (e.g., Figure D]. the image categorization speed can be delayed. Similar to the effect in humans, recent deep learning systems jointly trained on language and vision, e.g., CLIP, show object recognition interference when words are embedded in images (Goh et al. 2021 Specifically, by simply adding a handwritten word to an image, the object recognition can be biased to the superimposed word.
A critical aspect of picture- word interference in humans is that it depends on the semantic relation- ship between images and superimposed words. For instance, the reaction time the image classifi- cation for word-superimposed images is slower when the image category is semantically similar to the word one (Rosinski] (977). Based the accumulated findings, previous works have commonly suggested that at least two different processes mediate the effect in humans, as introduced in detail in Section 2.2. Specifically, when participant observes word-superimposed image, an activation process synthesizes semantic representations corresponding to the superimposed word and image (Figure [ activation process). Based on the representations, it has been assumed that a selection process decides which possible activation is the answer for the current task. (Figure 2 selection pro- cess). The activated representation is shared across visual word forms and images, and the dual activation by words and images confuses the decision in the selection process.
These human cognitive findings raise the question whether interference artificial models have some similarities with those observed in humans. Indeed, some artificial models show language- biased image classification, but one possibility is that the bias might emerge irrespective of the semantic interaction between words and images. When we consider model with text and image encoders for picture-word interference (Figure 2). we can regard the functional role of the image encoder synthesizing semantic representations for word-superimposed images, Although picture-
2

---

Published as a conference paper at ICLR 2022
Classification task type
Superordinate Basic label
animal furniture electronic
dog giraffe cat
Selection process
photo (label)
Text encoder
Image encoder
electronis
L I.
Ii T:
Ii T
I: Ts
Activation process
a photo (animal) 0.19%
photo
(electronic)
99 99.00%
Image biased answer
Language biased answer
Figure 2: Process diagram of the CLIP model for our task. This figure is created based on Radford al. (202I). The list superordinate or basic word labels was used for the classification depending on the task type. We regard the image/text encoder the selection/activation process in human picture-word interference, as discussed in the main text.
word interference humans depends on shared semantic representations written words and im- ages, this is the only mechanism that can produce the interference. For instance, the image encoder of the model may assign the written word "dog" a different semantic representation than the visual image "dog" and show strong preferential activation written words while ignoring im- age contents. Since joint learning of language and vision is expected to enable machines to acquire highly interacted representations of texts and images solve various general tasks, observing such independent representations and bias for one of them can be problematic and show limit of the joint training process.
In this study, we import methodological tools from the cognitive science literature to assess biases of artificial models image classification word-superimposed images. Specifically, we intro- duce paradigm to evaluate picture-word interference, in which agent has to predict the image superordinate/basic category for images with superordinate/basic words (Figure D. Our benchmark dataset is set of images with superimposed words and consists of mixture of natural image datasets (Cichy et al. 2016; Mohsenzadeh et al.) 2019) and hierarchical word labels (Lin et al,) 2014 Krizhevsky et 2009). Our benchmark task aims test artificial models with text encoder decisions and an image encoder (Figure 2 The task enables us to test 1) whether language-biase
happen across superordinate/basic object category levels and 2) the extent to which picture-word in- terference in artificial models depends on the semantic similarity between superimposed words and images. Furthermore, by importing methodological tool from the neuroscience literature, called representational similarity analysis (RSA) (Kriegeskorte et al., 2008a), we can also test 3) whether the image encoder has common semantic representation for superimposed words and images. Out of various joint learning models of language and vision, the present study evaluates the CLIP model using the dataset because the model consists of dual processing of visual and text encoders and developed be applied to various general visual tasks without fine-tuning the dataset. To anticipate the result, we show that 1) presenting words disturbs the CLIP image classification even across dif- ferent category levels, 2) the effect does not depend the semantic relationship between images and words, and 3) the superimposed word representation in the CLIP image encoder is not shared with the image representation. The present study provides open-source code to reproduce the results and the full dataset http::/whhbb..oooooeeeeeee/////e
2 RELATED WORKS
Joint learning of language and vision in artificial models: Some recent works in machine learning have reported language-biased image recognition for artificial models trained with joint learning of
3

---

Published as a conference paper at ICLR 2022
language and vision. One example is the CLIP model (Radford et al. 2021 The CLIP architecture consists of an image encoder and a text encoder (Figure 2). The image encoder is based on ResNet (He et al., 2016) or Vision Transformer (Dosovitskiy et al. 2020), and the text encoder is Language Transformer model (Vaswani et al. 2017). The model is trained on a large dataset of image-text pairs with contrastive objectives, where it learns to align text and image representations for each pair. The pre-trained model can be used for various visual tasks in a zero-shot manner. For instance, when the model is applied for the image classification task, the learned text encoder synthesizes a zero-shot linear classifier for the descriptions, including classification labels, e.g., "a photograph of [label]" (Figure ( The learned image encoder outputs the representation for an input image, and the text classifier decides which label representation is the closest to the image representation. Language- biased image classification is reported in the CLIP image classification task as a typographic attack (Goh et al., (02T). As shown in Figure T when a written word is superimposed on an image, the CLIP classifies the image category as the superimposed word.
Another example of language-biased image classification is reported in the context of the Visual Question Answering (VQA) task (Antol et al. 2015; Lu et al. 2016 Fukui et al. 2016). This task's goal is to answer questions about visual content, e.g., a text question "What color is the dog?" for a dog image. The task also needs joint learning of language and vision, i... an image encoder to recognize what the visual content is and text encoder to understand the question. However, joint learning tends to be biased by superficial correlations between texts and images in the training data and ignore visual input to solve task (Agrawal et al., 2016). For instance, even when model sees green (untypical) banana and is asked about its color, the model tends to answer "the color yellow" while ignoring the actual image content. Based on the observations, recent VQA models have attempted to remove the biases (Agrawal et al. 2018 Ramakrishnan et al. 2018).
Picture-word interference in humans: It is essential to understand how the language-biasse biased pro- cessing in artificial models is related to the human mechanisms discussed in the cognitive science literature. This subsection reviews the functional connection between text/image encoders in arti- ficial models and picture-word interference in humans (Figure 2). Picture-word interference refers to language- biased recognition for word-superimposed images, and previous works in cogmitive sci- ence have investigated this effect using various experimental paradigms (reviewed by Bürki et al. (2020)) Here, we focus on interference when a participant observes a word- sopr--upprimppsed image and answers the image category for the image compare the underlying cognitive mechanisms with artificial models (Rosinski 1977).
The discussion about what type of mechanisms mediate picture-word interference is still contro- versial in the cognitive science literature, especially the stage at which the interference emerges. However, we can summarize common functional mechanisms shared across different theoretical views and ask whether artificial models acquire these fundamental components in the current study. First, most views assume that visual words and images activate common semantic representations, i.e., a written word "dog" activates similar semantic representations to an image "dog." Second, they commonly assume that interference emerges due to some types of selection process. The type of selection process is extensively debated and differs depending on the views. One is the lexical selection hypothesis, and the selection works to compare the semantic representation competitively (Levelt et al.. 1999; Roelofs] 1992). This selection relies on relative activation in semantic represen- tations, and picture-word word interference emerges because the activation by words induces misselcction when it is semantically close to the activation by images. However, the lexical selection is incon- sistent with some behavioral findings, e.g., the activation amplitude does not always correspond to the effect size of interference. Another theoretical view assumes the response selection, which is closer to the decision stage than the lexical selection (Finkbeiner & Caramazza 2006). Since the re- sponse selection assumes additional processing for relative activated representations, it can explain the inconsistency of the phenomenon with the lexical selection assumption.
We first ask whether one observes patterns of interference at the behavioral level that are similar to humans and then study more precisely the learned representations to see if the mechanisms proposed to explain human interference could also be at play the artificial model. To investigate machine mechanisms in terms of the cognitive science findings, we regard an image encoder in the artificial model as the processing to activate semantic representations from images (Figure 2 We also con- sider text encoder the selection process, which tries to find the answers according to the task objectives (superordinate/basic selection).
4

---

Published as a conference paper at ICLR 2022
3 LANGUAGE-BIASED IMAGE CLASSIFICATION TEST
Our dataset is set of word-superinposed images. We use the two image datasets from the cog- nitive neuroscience literature on object recognition (Cichy et al. 2016 Mohsenzadeh et al. 2019). which include 188 and 156 images with natural object categories. Previous works recorded the spa- tiotemporal neural activity using fMRI and MEG measurements for the images. The first dataset was annotated with 118 basic categories (Cichy et al. (016) and the second one with five super- ordinate categories (faces, bodies, animals, objects, and scenes) (Mohsenzadeh et al.) 2019) For the word labels, extracted the superordinate category words from the MS-COCO dataset and the basic category words from the MS-COCO and CIFAR-I00 datasets (Lin et al., 2014 Krizhevsky et al. 2009). The MS-COCO dataset had the "person" label for superordinate and basic categories. We used this label only for the superordinate category because it was a higher category than other person-related basic labels in CIFAR- We removed multi-word labels and duplicates (including the plural of an already existing label) when compounding the MS-COCO and CIFAR-100 basic word label sets. The numbers of superordinate and basic word labels were 12 and 138, respectively. We superimposed all labels on each image, and thus, our word- porddssperrmposee image dataset included 41,100 images in total. The color of superimposed words was red and bordered by white lines to enhance the word visibility considering future usage in human experiments. Since the image dataset supposed observe objects the foveal vision, we put the words in the center of the image with a size that largely does not hide the original object.
The classification task was to answer the image label for word-superimposed images. The classi- fication label belonged either to the superordinate or the basic category. The set of labels in the superordinate and basic categories were the same as the words used to make the word-superimposed images. Therefore, the stimulus conditions were divided into four, corresponding to two classifi- cation types and two superimposed word categories (Figure D.. The former index of "S/S" in Figure indicates the classification task level (superordinate basic), and the latter shows the cat- egory level of the word superimposed on the image (superordinate or basic). In addition to these conditions, we used no-word images evaluate the original prediction by the model.
EXPERIMENTS AND RESULTS
4.1 EXPERIMENTAL SETUP
Performing image classification with the pre- pre-trained CLIP model needs the input of list of candi- date classification labels (or a textual representation of them) for the text encoder and an image for the image encoder (Figure 2). We used the sentence of "a photo of [label]" for the text encoder. The two encoders output the representations of texts and the image respectively. We then computed the similarities of text and image representations for set of labels, passed them through the softmax function to obtain the probability of each label for given input image, and extracted the label with the highest value for the final classification (Radford et al. 2021). We present the results of the pre-trained CLIP image encoder using a Vision Transformer (VIT-B/32) in the main text and those of other image encoders Appendix C
4.2 LABEL SWITCHING RATE
To test whether superimposing the word affects the classification, we first evaluated the label switch- ing rate. We defined a label-switched image as a word-superimposed image with a different im- age classification than the corresponding no-word image. The label switching rate refers to the ratio between the number of label-switched images and the total number of images under each classficationsuupripppeedd word condition.
Table shows the results of the label switching rates for each condition. When the classification task type is consistent with the superimposed word category (i.c., the S/S and B/B conditions), the choices of the classification labels include the same word as that superimposed on the image. For these conditions, the CLIP model switched the original classification with high rates. These findings suggest that the model can detect the visual word form on the image and match it with the classification label. These are consistent with the previous work reporting the language-biase image classification of the CLIP model (Goh et al. 2021
5

---

Published as a conference paper at ICLR 2022
Superordinate prediction
Basic prediction
Superordinate word
(S/S) 92.77 %
(B/S) 41.24
Basic word
(S/B) 28.29 %
(B/B) 73.97 %
Table CLIP's label switching rate for cord-superimposed images.
(a) Semantic similarity
(b) Spelling similarity
(S/S)
(B/S)
(S/S)
(B/S)
SPELLING SINELARITYY
SPELLING ENVOLAIIY
8 &
Unswitched
Unswitched
Lobel-switched
Unswitched
Label-switched
Unswitched
Label-switched
S\/B
(B/B)
(S/B)
(B/B)
10
1o
SHOLLO SOMELLLAAL LOUTTY
SPILLING SIPILARITY
Unswitched
sabel-wlichhdd
Unswitched
Lobel-switched
Unswltched
Label-oun Lobel-owitched
Unswitched
Label-switched
Figure 3: Results of the (a) semantic and (b) spelling similarity analyses.
In contrast, when the classification type is different from the superimposed word category (i.e., the S/B and B/S conditions), the classification labels do not include the same word as the superim- posed one. Even under the conditions, the CLIP model showed label-switching for some of the word-superimposed images. Furthermore, some of the switched new labels were related to the su- perimposed words (Appendix G). The findings suggest that the model can match the meaning of the written word in the image with the label category across different levels. However, it remains un- clear whether the image encoder acquires common semantic representation for the images and the visual word forms. There also a possibility that the superimposed words are separately represented from the images in the image encoder. (e.g., superimposed word "dog" is differently represented from an image of dog). Even so, the label switching across different categories can emerge when the label activation of the text encoder can match with the representation of the superimposed word. To understand the semantic relationship, we evaluated the following semantic similarity between images and superimposed words for the switche and unswitched images.
4.3 SEMANTIC SIMILARITY
We evaluated the semantic similarity between the original prediction labels for no-word images and the superimposed words test the semantic dependency picture-word interference. Specifi- cally, we calculated the vector representations of these labels using a pre-trained Word2Vec model (Mikolov al. 2013b a), and then measured the cosine similarity between the two vector represen- tations. If picture-wort word interference in CLIP depends on the semantic similarity between superim- posed words and images, we would expect the similarity the label-switched condition would be higher than that in the unswitched condition. Figure 33 shows the semantic similarity distribution for the label-switched and unswitched images the CLIP model. The results show that for all task type and word category conditions, the semantic similarity distributions the label-switched con- dition were similar to those on the unswitched condition. This finding suggests that picture-wor word interference in the CLIP model does not depend on the semantic relationship between images and superimposed words, suggesting that the CLIP image encoder does not acquire a common semantic representation across superimposed words and images.

---

Published as a conference paper at ICLR 2022
VGG-19
ResNet-152
CLIP (VIT-B/32)
20m1
000000
foce
anamaal
food
animal
incoor
booy
human
animal
pbent
08
person
12
brze
Indo
Tuman
anma
1003)
100
Figure 4: Results of the representational similarity analysis. See also Appendices B1 and
H1
4.4 SPELLING SIMILARITY
In addition to the semantic similarity, we evaluated the spelling similarity using the Jaro- Winkler metric (Winkler 1990) test how the superimposed word is related the image category. the CLIP image encoder acquires common representation of superimposed words and images based on the edit distance, this metric should change with the labelsswichhe//nssiichhe condition. Figure shows the result of the CLIP model on the spelling similarity for the label-switched and unswitched images. As in the semantic similarity, the distribution median for the task conditions was always similar between the label-switched and unswitched conditions.
4.5 REPRESENTATIONAL SIMILARITY ANALYSIS (RSA)
We also used methodological tool from the cognitive neuroscience literature, called representa- tional similarity analysis (RSA) (Kriegeskorte et al., 2008a), to understand what is represented in the image encoder for word-superimposed images. RSA refers to an image-by-image similarity assessment of intermediate representations in brain or model. By computing representational dis- similarity matrices (RDMs) for no-word images and word-superimposed images, how the representation for the no-word images in the CLIP image encoder is affected by adding words on the image. We input one of the two datasets (156 images) (Mohsenzadeh al.) 2019) for this analysis because it consisted of the previously defined superordinate categories. In addition the CLIP model, we evaluated the two ferd-forward convolutional networks (i.e., VGG- 19 and ResNet- 152) trained for object category classification using ImageNet without joint learning of lan- guage and vision to confirm whether presenting words as image manipulation can affect the image encoder without joint learning.
evaluate
Figure shows the results of the RSA. Each matrix indicates the srage-by-imann cosine dissimi- larity between intermediate features for the paired images. For the features, used the last fully connected layer for middle, and bottom rows show the results for the original images, the word-superimposed images
and the visual encoder output for the CLIP model. The top.

---

Published as a conference paper at ICLR 2022
prompt
B/S
S/B
B/B
"a red word label over a picture of a X"
87,69 %
44.16 %
29.64 %
66.23 %
"a word printed in a red font over a picture of X"
91.34
44.86 %
35,29 %
69.72
"a photo of word written in red font over picture of X"
86.53 %
43.07 %
30.78 %
62.69 %
text that says
96.78 %
52,50 %
54.58 %
88.22
"a word of a X printed in red font over a picture"
95.48 %
51,65 %
46.28 %
78.02 %
"a photo of the word written in a rod font over a picture"
96.22
53,58 %
42.91 %
85.05
"a photo of the word Y written in red font over picture of x"
34.34 %
35.33 %
31.04 %
34.92 %
Table 2: CLIP's label switching rate for word-superimposed images on different prompts. The red-colored texts are the prompts that specify that the image content what should be predicted. The blue-colored texts are the prompts that specify that the superimposed word is what should be predicted. The X means the label to be answered by the model. The orange-colored text is the condition in which a prompt varies according to the superimposed word on each image.
with the "animal" label, and the word-superimposed images with the "person" label, respectively. Each left image is one example in the 156 images. The results of other word-embeddings are shown in Appendices B1 and HI
For VGG-19 and 152, we observed some representational clusters according to the super- ordinate categories (i.e., the squared clusters around the diagonal line in each panel with lower dis- similarity values) (Figure top. VGG-19 and ResNet-152)) It has been known that higher cortical areas in humans and monkeys also show such categorical clusters for RDMs of natural scene images (Kriegekkorte et al., 2008b, Cichy et al. 2014). Similar to these feed-forward models, the CLIP im- age encoder also showed representational clusters depending on the superordinate categories (Figure top, CLIP), suggesting that the encoder has hierarchical structure for visual images.
In contrast, the results of the CLIP model for word- superimprdd images showed different repre- sentations from that of original images (Figure 4 middle and bottom, CLIP). First, the dissimilarity values for word-superinposed images were generally lower than those for original images (i.e., the representations of all images are similar to each other). The results are consistent with the strong language bias of the CLIP model. As in the other analyses, presenting words on the image shift the CLIP image classification to the direction of the superimposed word. This means that when we put a single word on the different images, their representations can be similar.
Most importantly, our analysis showed that adding words to images did not produce categorically selective effects on the representation of the image encoder. Specifically, we observed category- related squares similar to the original one around the diagonal line of each representational matrix for word-superimposed images (Figure 4 middle and bottom, CLIP). In addition, even when we put different word labels, "person" and "animal," these representational matrices remained similar. If the superimposed words were similarly represented in the image encoder to the image category, presenting words on the image should selectively affect the original image category representation and depend on what words were superimposed. The current results are inconsistent with the hy- pothesis and suggest that the CLIP image encoder has different representations of word forms (i.c., superimposed words) from visual image representations.
4.6 PROMPT DEPENDENCY
The CLIP model is not trained for image classification but just for matching between images and texts. One may consider that the language-biased classification emerges because the task objective for the model, which is defined by the prompt "a photo of X," is ambiguous. The prompt can be both photo of the written word" and "a photo of the image content." We tested various prompts to explore the effect of prompts. First, we created three prompts that specify that the image content is what should be predicted by the CLIP model (red in Table 2 and three prompts that specify that the superimposed word is what should be predicted by the CLIP model (blue in Table 2). The label switching rate refers to the ratio of switching ford-superimposed images with a different image classification than the corresponding wo-word images. In addition to these fixed prompts, a prompt that varies according to the superimposed word on each image. We used a prompt "a photo of the word Y written in red font over picture of X," where X is the label to be answered by
created
8

---

Published as a conference paper at ICLR 2022
the model and Y is the same variable word superimposed on each image (orange in Table 2). For instance, we used a prompt "a photo of the word electric in a red font over a picture of a X" when the superimposed word is "electrie," and prompt "a photo of the word dog in red font over a picture of a X" when the superimposed word is "dog."
Table 2 shows the results of the label-switching rate for different prompts. The results showed that the rate changed with the different prompts, i.e., the focus of the task (image content or superimposed word). When the prompt was focused on the image content (red in Table 2 the switching rates were lower than when it was focused on the superimposed word (blue in Table 2 However, the language-biased classification was high for conditions. The results on the variable prompt (orange in Table 2 showed that the label switching rate for the S/S and B/B conditions decreased more than the prompt conditions that specify the image content (red in Table 2 The finding suggests that describing the superimposed word in the prompt is effective in decreasing the language-biased classification. Even so, the label-switching rates were still high, more than 30% for all conditions. In addition, we analyzed the semantic similarity between the image and superimposed words for label- switched and unswitched images. The results showed that, in any prompts, the semantic relationship did not explain the label-switching effect (Appendix Figure DI).
5 DISCUSSION AND CONCLUSION
The present study developed benchmark test for language biases in image classification. Experiments we presented show that adding printed words over images leads to high classification switching rates across different categories of superimposed words and different classification tasks. The finding suggests that the CLJP image encoder can recognize the visual word form as a word, and the model can match the visual word representation with the classification label representations across different category levels. However, the semantic similarity analysis showed that the language-biased image classification in the CLIP model was irrespective of the semantic/spelling relationship between image and superimposed word categories. This finding different from human findings and suggests that the CLIP image encoder does not acquire the semantic word representation shared with the image representation. The RSA results supported the hypothesis by showing that presenting words on images did not affect the no-word image representation selectively to specific categories.
visual models to evaluate the
We consider multiple steps for an artificial model with joint learning of language and vision to acquire semantic compositionality for visual word and image categories. First, an image encoder needs to recognize the visual word form as a word. Next, the semantic representation for words and images needs to be shared in the image encoder. That is, the written word "dog" needs to have the same semantic representation as the image "dog." The present study showed that the CLIP model acquires the first step, but the second step remains challenging. Our language-biased image classification test enables researchers to evaluate step-by-step semantic interactions in joint learning of language and vision.
A remaining question is how the CLIP model acquires the biased representations in the image en- coder. The presented results may be improved by further studying them as a function of potential biases in the training data of CLIP itself: for example, it would be useful to understand in the first place how CLIP acquired the ability to visually recognize superimposed words and why we observed bias favoring the superimposed words over the whole image. In particular, we can ask whether the written words are in the training data and if they are, whether there are adversarial examples in the training data like our word-supert hord-superimposed images.
Picture-word interference in humans phenomenon showing a language-biased effect in terms of image classification. However, this can also be considered as a developmental product humans acquire, where such bias could have functionalities. Some human studies have shown that picture- word interference is inextricably linked to facilitation effects of language priming for image category recognition (Finkbeiner& & Caramazza 2006). The finding reminds us that acquiring human- bi- ased representations for artificial models can have a functional meaning to process world information efficiently in natural environments. The current efforts to manage language-bil -biased image recogni- tion in artificial models are mainly aimed remove the biases (Agrawal et al., [2018] Ramakrishnan et al. 2018). However, one direction might be to make the biases more human-like by controlling joint learning curricula of language and vision (c.f., Sigaud al. (20TT).

---

Published as a conference paper at ICLR 2022
6 REPRODUCIBILITY STATEMENT
All the source codes are available at our GitHub repository (https://githan) com/ flowersteam/ petteamm/iccuureewerd--neeee interfenneeeeeecce to create the word-superimposed images from the image datasets (Cichy et al. [016] Mohsenzadeh et al.) (019) and to reproduce the current analysis results, which are also shared supplementary materials of the ICLR submission. The analysis relies on the pre-trained models shared in the previous studies (e.g., CLIP).
REFERENCES
Aishwarya Agrawal, Dhruv Batra, and Devi Parikh. Analyzing the behavior of visual question answering models. arXiv preprint 1606.00360. 2016.
Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Aniruddha Kembhavi. Don't just assume; look and answer: Overcoming priors for visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1971-49948 2018.
Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zit- nick, and Devi Parikh. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision, pp. 2223,,433, 2015.
Audrey Bürki, Shereen Elbuy, Sylvain Madec, and Shravan Vasishth. What did we learn from forty years of research on semantic interference? a bayesian meta-analysis. Journal of Memory and Language, 114:104,25, 2020.
Radoslaw Martin Cichy, Dimitrios Pantazis, and Aude Oliva. Resolving human object recognition in space and time. Nature neuroscience, 17(3):455 462, 2014.
Radoslaw Martin Cichy, Dimitrios Pantazis, and Aude Oliva, Similarity- fusion of meg and fmri reveals spatio-temporal dynamics human cortex during visual object recognition. Cerebral Cortex, 2668)33563-3579, 2016.
Cédric Colas, Tristan Karch, Clément Moulin-Frier, and Pierre- Yves Oudeyer. Language as cognitive tool: Dall-e, humans and vygotskian rl agents, March 2021. URL httpp::/hal archives ouvertee........................
-33159978
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010.11929, 2020.
Matthew Finkbeiner and Alfonso Caramazza. Now you see it, now you don't: On turning semantic interference into facilitation a stroop-like task. Cortex, 42(6):790-799 2006.
Akira Fukui, Dong Huk Park, Daylen Yang, Anna Rohrbach, Trevor Darrell, and Marcus Rohrbach. Multimodal compact bilinear pooling for visual question answering and visual grounding. arXiv preprint ar 1701v:1606.0/880 2016.
Gabriel Goh, Nick Cammarata i, Chelsea Voss t. Shan Carter, Michael Petrov, Ludwig Schubert, Alec Radford, and Chris Olah. Multimodal neurons in artificial neural networks. Distill, 2021.
doi: 10.2391 5/distill.00030
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Emmanuel Keuleers and Mare Brysbaert. Wuggy: multilingual pseudoword generator. Behavior research methods, 4233),627-633, 2010.
Nikolaus Kriegeskorte, Maricke Mur, and Peter A Bandettini. Representational similarity analysis- connecting the branches of systems neuroscience. Frontiers in systems neuroscience, 2:4, 2008a.
10

---

Published as a conference paper at ICLR 2022
Nikolaus Kriegeskorte, Maricke Mur, Douglas A Ruff, Roozbeh Kiani, Jerzy Bodurka, Hossein Esteky, Keiji Tanaka, and Peter A Bandettini. Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron, (166):126-11144 2008b.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images.
2009.
Willem JM Levelt, Ardi Roclofs, and Antje S Meyer. A theory of lexical access in speech production. Behavioral and brain sciences, 22(1):1-38, 1999.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. Hierarchical question-image co-attention for visual question answering. Advances in neural information processing systems, 29:289-2977 2016.
Stephen Lupker.
The semantic nature of response competition in the picture-word interference task. Memory & Cognition, 7(6):485-495, 1979.
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word represen- tations vector space. arXiv preprint arXiv:1301.374 2013a.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed represen- tations of words and phrases and their compositionality. In Advances in neural information pro- cessing systems, pp. 3111-3119, 2013b.
George Armitage Miller. Language and communication. 1951.
Yalda Mohsenzadeh, Caitlin Mullin, Benjamin Lahner, Radoslaw Martin Cichy, and Aude Oliva. Reliability and generalizability of similarity- based fusion of meg and fmri data in human ventral and dorsal visual streams. Vision, 3(1):8, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. CoRR, abs/2103.00020, 2021.
wwwp:xwwwwwww..//..e/eoooooese http//bb/220000 00020
Sainandan Ramakrishnan, Aishwarya Agrawal, and Stefan Lee. Overcoming language priors in vi- sual question answering with adversarial regularization. arXiv preprint arXiv:1810.03641 2018.
Ardi Roelofs. A spreading-activatio theory lemma retrieval speaking. Cognition, 42(1-3): 107-142, 1992.
Richard R Rosinski. Picture-word interference is semantically based. Child Development, pp. 643- 647, 1977.
Olivier Sigaud, Hugo Caselles-Dupré Cédric Colas, Ahmed Akakzia, Pierre- Yver Oudeyer, and Mohamed Chetouani. Towards teachable autonomous agents. arXiv preprint arXiv:2105.11977, 2021.
J Ridley Stroop. Studies of interference in serial verbal reactions. Journal of experimental psychol- ogy, 18(6):643, 1935.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Lev S Vygotsky. Thought and language. MIT press, 1934.
Sandra Waxman and Dana Markow. Words as invitations form categories: Evidence from
12-to 13-month-old infants. Cognitive psychology, 29(3):257-302, 1995.
William E Winkler. String comparator metrics and enhanced decision rules in the fellegi-sunter model of record linkage. 1990.
11

---

Published as a conference paper at ICLR 2022
A PREDICTION EXAMPLE FOR WORD-SUPERIMPOSED IMAGES.
(S/S)
(B/S)
(S/B)
(B/B)
What is the superordinate category of the image?
What is the basic category of the image?
What the superordinate category of the image?
What is the basic category of the image?
electronic
electronic
laptop
laptop
New prediction: electronic 99.81 Original prediction: outdoor 37. 6..66% 0.02%
New prediction: keyboard 17% Original prediction: tulip 89.31% 3.19%
New prediction: electronic | 45. 45.91%
New prediction: laptop | 91.56% Original prediction:
Original prediction: outdoor 37. 66.666 15.61%
tulip 89.31% 1.33%
electronic
electronic
laptop
laptop
New prediction: electronic 99.00% Original prediction: animal 45. 72% 0.19%
New prediction: dog 50..9% Original prediction: dog 60.78% 50.97%
New prediction:
New prediction: laptop | 85.38%
electronic | 40.00% Original prediction: animal 45. 72% 14.
Original prediction:
dog 60. 60.78% 4.46%
electronic
electronic
laptop
laptop p
New prediction: electronic 98. 69% Original prediction: food | 96. 88% 0.53%
New prediction: laptop | 19.38% Original prediction:
New prediction: food 72.25% Original prediction: food 96. 88% 72.25%
New prediction:
laptop 93.00% Original prediction: plate 10.00% - 2.99%
plate | 40.00% 12.31%
Figure Al: Prediction example for word-superimposed images using the CLIP model (ViT-B/32). For each stimulus condition (S/S, B/S, S/B, and B/B in Figure the "new prediction" indicates the image classification probability for the word-superimper the image classification probability for no-word images. We also show the extent to which the original prediction changed with the word attachment.
image. The original prediction means
12

---

Published as a conference paper at ICLR 2022
B REPRESENTATIONAL SIMILARITY ANALYSIS USING WORD-SUPERIMPOSER IMAGES.
original (no-word)
electronic
vehicle
outdoor
indoor
accessory
sports
kitchen
food
furniture
appliance
1.0
0.8
plant
food
Indoor
0.4
human
body
0.2
human
face
0.0
plant
Bood
indeor
outdoor
human body human face
Figure B1: Results of the RSA. In addition to the labels "animal" and "animal" in Figure show the results of other words. Based on the CLIP image encoder (ViT-B/32) outputs, we calculated the image-by-image representation dissimilarity matrices. Similar the results of the word labels "animal" and "person," presenting words on the images did not affect the no-word representation in category-selective way. These results consistent with the finding in the main text, suggesting that semantic word representations in the CLIP image encoder are different from image representations.
13

---

Published as a conference paper at ICLR 2022
C EVALUATION OF DIFFERENT IMAGE ENCODERS OF THE CLIP MODEL
The image encoder of the CLIP pre-trained model is based on ResNets (He et al. 2016) or Vision Transformers (Vaswani et al. 2017). To explore the extent which the language-biased image classification depends on the network architecture, we investigated the misclassification rate and the semantic relationship using various encoders in addition to Vision Transformer model (ViT-B/32) used in the main text. Specifically, for the ResNets, we used a ResNet-50, a ResNet- 101, and two models which follow Efficienttet--sylee model scaling ResNet 50x4 and ResNet 50x16). For the Vision Transformer, we used ViT-B/16 in addition to ViT-B/32. We used the pre-trained CLIP networks using these architectures, which are shared by the previous work (Radford et al.) 202 The experimental procedure is the same as that of the main text.
Figure CI shows the results misclassffication rates for the CLIP models using various image encoders. Even when the other architectures were used for the image encoder training, the results were consistent with the performance the ViT-B/32 version. The misclassification rate was high when the classification type was consistent with the superimposed word level (S/S and B/B). In addition, the CLIP models showed significant misclassification across different category conditions under the other conditions (S/B and B/S)..
The results of semantic similarity analysis and the RDA also did not depend on the architecture types (Figures and C3 respectively). The semantic similarity analysis showed the median similarity of the label-switched images was not different from that of the unswitched images Figures C2 for all architectures. The RDA showed that the image encoder representation for no-word images changed with the architectures (Figures C3 first and fourth rows). However, the effect of attaching words did not change with the architecture. For all architectures, the overall dissimilarity across image- by-image pairs decreased for word-superimposed images, but the representation displacements were selective the original image representation.
1.0
ViT-B/32
Misclassification rate
0.8
ViT-B/16
0.6
ResNet50x16
ResNet50x4
0.4
ResNet101
0.2
ResNet50
0.0
S/S
B/S Stimulus condition
S/B
B/B
Figure Cl: Results of misclassification rates for the CLIP models using various image encoders. The misclassification rate was plotted according to the stimulus condition (S/S, S/B, B/S, or B/B). Different color labels indicate different architectures.
14

---

Published as a conference paper at ICLR 2022
(S/S)
VIT-B/32
VIT-8/16
Resllet50x16
ResNet50x4
Restit101
ResNet50
Semantic similarity
1.0
0.8
0.6
0.4
0.2
0.0
(B/S)
VIT-8/32
VIT-B/16
ReaNet50x16
ResNet50x4
Restiet101
ResNet50
Semartic similarity
1.0
0.8
0.4
0.2
0.0
(S/B)
VIT-B/32
VIT-8/16
ResNet500x6
ResNet50x4
ResNet101
ReaNet50
Semantic similarity
1.0
0.8
1000 D
0.6
0.4
0.2
0.0
(B/B)
VIT-B/32
VIT-8/16
ResNee150x16
ResNet50x4
ResNet101
ResNee50
Semartic similarity
1.0
0.8
0.6
0.4
0.2
0.0
Unswitched
Switched
Unswitched
Switched
Unswitched
Switched
Unswitched
Switched
Unswitched
Switched
Unswitched
Switched
Figure C2: Results of semantic similarity analysis for the CLIP model using various image en- coders. The semantic similarity analysis of Figure 3 was conducted for the CLIP models with the image encoder a ViT-B/32, ViT-B/16, ResNet-50x 16, ResNet-50x4, ResNet-101, and ResNet-59. The results of the ViT-B/32 image encoder replotted from Figure Different color labels in- dicate different architectures. Each row shows the stimulus condition (S/S, B/S, S/B, or B/B).
15

---

Published as a conference paper at ICLR 2022
CLI((VI--B32)
CLIP(ViT-B/1))
CLIP(ResNet 50x16)
animal
person
CLIP(ResNet 50x4)
CLIP(ResNet 101)
CLIP(ResNet 50)
1.0
animal
0.6
0.4
person
nose
outtor
0.2
0.0
part
Figure C3: Results of the representational similarity analysis (RSA) for the CLIP model using various image encoders. The analysis method was the same as Figure except that we also used the CLIP models with other image encoders than a VIT-B/32. The results of the VIT-B/32 image encoder were replotted from Figure
16

---

Published as a conference paper at ICLR 2022
D PROMPT DEPENDENCY OF LANGUAGE-BIASED IMAGE CLASSIFICATION
Prompt "a photo ofX"
Prompt "a word label over a picture ofa
(S/S)
(B/S)
(//S)
(B/S)
SEHANIIC EDHELAITY
SCMELATTYY
SEMONTIC SUNELANSTY
08
06
0b-
06
06
SEMNNTIC
04
04
Unswitched
Lobth-switched
Unswitched
Label-switched
Unswitched
Label switched
Unswitched
Label swittchd
(S/B)
(B/B)
S/B)
(B/B)
SEHANTIC SIILLAIIY ARTTT
SEVANNTIC SIMELLARTYY LETTTT
SEMANTIIC SONILAAITY
SEMANNTIC SEMELLANNT
04
04
044
Unswitched
Labei-switched
Unswitched Label-switched
Unswitched
Label-oo sibell-witched
Unswitched
Lobel sibel--witched
Prompt: "a text that says X"
Prompt:
photo ofthe word written in red font over a picture ofa
(S/S)
(B/S)
(S/S)
(B/S)
1.0
10
10
SCHANTIC SIIELAITYY
SCMANTICC SIHELARITY
08
SIVOLARITY
08
SEMANTICC
00
0.0
0.0
0.0
Unswitched
Lobel-switched
Unswitched
Lobel-switched
Unswitched
Lobel-switched
Unswitchee
Lobel-swilcced
(B/B)
S/B)
(B/B)
SEHANTIC SIPOLARITY
0.8
SCHANNTIC SIPELAASIYY
0.
08
REMANTICC SITCOLAR
08
06
Figure DI: Results of semantic similarity analysis. The same analysis as Fig was conducted for
different prompts shown in Table GT
17

---

Published as a conference paper at ICLR 2022
E PREDICTION PROBABILITY
The prediction probability analysis indicates how label switching relies on the prediction confi dence (i.e., the probability of the classified label the label list) for no-word images and how the original prediction confidence decreases by presenting words. Figure ET shows the results of the prediction probability analysis. Different panels show different stimulus conditions, as indicated in Figure The original prediction means the probability for no-word images. In the figure, the original prediction is categorized into label-switched or unswitched conditions. The label-switched and unswitched conditions refer to if the original prediction is switched in the new prediction for soorr-super images. The new prediction shows the probability of the label before and after adding the superimposed word, for the label-switched word-superimposee images.
the original prediction, the probabilities for the label-switched images were lower than those the unswitched images. The trend was notable when the task type condition was inconsistent with the superimposed word category (the B/S and S/B conditions). These results suggest that picture- word interference is more effective when the original prediction is not confident. In addition, the original prediction probability drastically decreased in the new prediction for the word-superimposed image (two red plots in each panel of Figure p. However,
must be aware that label prediction proba- bility depends on other label predictions and that it can be lower if the other label prediction is high. We show how the original image representation is changed/unchanger by presenting words in the next section. Furthermore, the prediction probability analysis showed that the new label confidence for the label-switched images (the blue plot in Figure ET was high when the classification task type was consistent with the superimposed word category (the S/S and B/B conditions). This trend is consistent with the results of the label switching rate analysis.
(S/S)
(B/S)
Original prediction
New prediction
Original prediction
New prediction
0.8
T
0.8
MODABLLTY
0.0
0.
Unswitched
Label-switched Original lobel
New lobel
Unswitched
Lobel switched Original label
New label
(S/B)
(B/B)
Original prediction
New prediction
Original prediction
New prediction
0.8
0.6
60
Lobel-switched Original
Original
Figure E1: Results of the prediction probability analysis.
18

---

Published as a conference paper at ICLR 2022
F
EFFECT OF PSEUDOWORD
animal (control)
atipol
alipil
mzwxnmsczp
Label switching rates
57.66
26.64
28.46 %
18.25 %
Table F1: Label switching rate for nonsense pseudowords.
original (no-word)
animal
1.0
08
0.5
0.4
02
0.0
atipol
alipil
Mzwxnmsczp
plam
food
outdoor
human body
animal
food
Indoor
human
body
human
face
Figure Fl: Results of the representational analysis for nonsense pseudowords.
We conducted an additional experiment using nonsense pscudowords and random words. We cre- ated two pseudowords from the word "animal" by multilingual pscudoword generator, Wuggy Keuleers & Brysbaert 2010). We also used one random word used in previous human study Finkbeiner & Caramazza 2006). Results showed that the label switching rate decreased for these words (Table|G1). We also analyzed the RDM for the images on which these words were superim- posed. Our results showed that when nonsense pseudo and random words were superimposed on the images, the effect of words on the representation was weaker than meaningful words. The finding suggests that presenting meaningful words on the images can be factor affecting the representation of the image encoder.
19

---

Published as a conference paper at ICLR 2022
G RELATIONSHIP BETWEEN SWITCHED LABEL PREDICTION AND SUPERIMPOSED WORDS
S/S
B/S
S/B
B/B
Consistency rates
99.28 %
45.70 %
36.30 %
93.49 %
Table Gl: Consistency rates between the switched labels and the superimposed words. We calculated the ratio that the switched labels and the superimposed words were consistent in the label-switched image. This table results from the prompt "a photo of X" using the ViT/32 image encoder. When the prediction category class is same as the superimposed word class (the S/S and B/B conditions), most switched labels were the same word as superimposed on the images (more than 90%) (e.g.. "dog" prediction for superimposed word "dog") When the prediction category class different from the superimposed one (the S/S and B/B condition), parts of switched labels were the consistent category with superimposed words across different category levels (e.g., "animal" prediction for superimposed word "dog") )
H RELATIONSHIP BETWEEN NO-WORD IMAGES AND WORD-SUPERIMPOSHH IMAGES
no-word vs. no-word
animal vs. no-word
person vs. no-word
animal
plant
0.8
food
0.6
outdoor
human
body
02
human
face
0.0
anmal
plant
food
Indoor
outdoor
human
human
face
Figure HI: Relationship between no-word and word-superimposed images. We analyzed the repre- sentational dissimilarity matrices (RDMs) between no-word images and word-superimposed images for the CLIP image encoder (ViT/32). The vertical axis of each panel shows the no- word images, and the horizontal axis shows the word-superimposed images for the center and right panel. The left panel is the same as the no-word result in Figure 4 If the image encoder has a shared se- mantic representation for both the superimposed words and the images, the RDMs should show seregory-speciffe similarity when the image category is consistent with the superimposed word. For instance, when the word "animal" superimposed on the image, the similarity between the word- superimposed images and the animal images should be selectively high. However, results did not show any category-specific similarity, and the RDMs were similar even different words were super- imposed. The results consistent with that the CLIP image encoder has different representations of word forms from visual image representations.
20


Reviews:
[{"review_id": "yDeig1-Y_gn", "reviewer": "Reviewer_WKSS", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall this paper is well-written. The motivation and background are clearly stated. To evaluate the effect of word-picture interference, the authors applied multiple analyses, including assessing the model behaviors under different conditions (section 4.2, 4,5), connecting model behavior to label similarity (section 4.3, 4.4), and comparing the similarity of visual model representations (section 4.6). The methods are described in details and should be easy for readers to reproduce their analyses. The authors are trying to figure out how the visual representations of word form and object images interfere with each other and bias the classification behavior. This is an intriguing question driven by observations and theories from cognitive science.\n\nComments:\n1. In general, I agree with the authors' claims. However, I feel there exists some interesting findings in the current results remain further explanation or investigation. First, during pre-training, the CLIP model already matches the representation of \"*object in visual form*\" and \"*word in textual form*\" in the same metric space. And the authors further find that the image classification (section 4.2, 4.5) is strongly biased by the superimposed word (e.g., examples in Figure A1), even in the B/S or S/B condition (e.g., recognize tulip image labeled 'electronic' to semantic class 'keyboard', or recognize tulip image labeled 'laptop' to semantic class 'electronic'). Does it imply that the representational space of \"*words in visual form*\" should have similar structure (in the sense of pair-wise similarity) as the representational space of \"*word in textual form*\"? However, all the similarity analyses based on representations (section 4.3, 4.4, 4.6) showed non-distinguishable results across conditions, which lead to the authors' statement that \"the CLIP image encoder has different representations of word forms from visual image representations\". But it seems that at least both \"word forms\" and \"object image\" representations from CLIP image encoder captures semantic similarity in the text-based word space. What's authors interpretation on this?\n\n2. I am a bit surprised about the good performance of CLIP recognizing spelling word into the same category word. Were all the results based on pre-trained CLIP without fine-tuning? Was CLIP trained on some images with visual word forms to learn typographic features?\n\n3. Were all 'misclassified' word-embedded images classified as the superimposed label? If not, it might be better to divide the 'misclassification' condition into two cases: 'biased' (predict the superimposed label) and 'random' (predict a label not original or superimposed), just to make results in section 4.2 more informative.\n\n4. The left box in each panel of Figure 4 is a bit ambiguous to me. What is 'label-switched' for 'original images'?\n\n5. What would be the model behavior and similarity results if the image is interfered with a nonsense pseudoword?\n\n6. Typo: 'fisrt' should be 'first' on page 5.\n\n7. What does the \"semantic compositionality\" in the paper title refer to? I feel this term is weakly connected to the main text.", "summary_of_the_paper": "This paper introduced a benchmark task for evaluating how semantic processing of visual word form interfere with the recognition processing of an object image in word-embedded images. By placing category word (e.g., superordinate category like 'electronic' or basic category like 'laptop') in the center of an image, the authors evaluated the 1) the misclassification rates of CLIP; 2) the semantic/spelling similarity between original and superimposed labels under different conditions; 3) the change of model prediction probability; 4) RSA on images with different types of picture-word interference.\nIn general, this work provided interesting new insights on how language information bias the image classification in visual-language models like CLIP. The authors concluded that CLIP can recognize the visual word form as a word but fails to encode word form and object image that share the same category (e.g. word 'dog' and picture 'dog') with similar visual representations.", "summary_of_the_review": "This paper introduced a benchmark task for evaluating picture-word interference in visual-language joint learning models. The benchmark was tested with the latest model CLIP. The evaluation results brought novel insights to understand the recognition process of the CLIP image encoder, showing its distinction on visual representations of word forms and object images. This provides a new perspective for future studies to evaluate the interference of multi-modal processing in AI models. Some results and figures may need further explanation or clarification for better understanding of their findings.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "smFhWaC2zvZ", "reviewer": "Reviewer_9yiw", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "[Overall] This is a very interesting piece of work that analyzes whether the state-of-the-art artificial vision and language model; CLIP, resembles human cognition or not. \n\n[Formatting]\n- Figures and Tables should be aligned to the top of the page.\n\n[Writing]\n- I though that CLIP is trained by inputting image and words separately, but according to the authors statement in Section 5: \"how CLIP acquired the ability to read superimposed words\", it seems that it models language information from the image (superimposed text). Is this true? If so, I doubt it, and accordingly, results and discussions based on it may be results of some undiscussed reasons. Please also introduce in detail the task settings so that this kind of confusion will not occur.", "summary_of_the_paper": "The authors introduce a framework to evaluate the picture-word interference in CLIP which is known to exist in human cognition.\nIt is designed to test whether language-biased decisions occur across different category levels, and the extent to which picture-word interference in CLIP depends on the semantic similarity between superimposed words and images. Experimental results show that presenting words disturbed the CLIP image classification even across different category levels, the effect did not depend on the semantic relationship between images and words, and the superimposed word representation in the CLIP image encoder is not shared with the image representation.", "summary_of_the_review": "Work like this should is very important to understand how and why the state-of-the-art models perform well, what their limitations are, and how we can improve them.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "cyFIEPxg9A_", "reviewer": "Reviewer_P83Y", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The question of what similarity CLIP should assign to text in images\nvs.  pictured objects is potentially interesting. There doesn't seem\nto be a \"right\" answer, as both transcribing the red text and naming\nthe underlying pictured object are arguably justified. The connection\nto Rosinski's work is interesting, and understanding how CLIP deals\nwith the ambiguity of this situation is perhaps interesting from the\nperspective of adversarial examples.\n\nI had a few fundamental concerns with this work. First, I didn't\nentirely understand the motivation. Goh et al. 2021 demonstrated that\nCLIP's predictions might flip if a word was written in the image, but\nI hesitate to call this a \"misclassification\" as the authors say:\ngiven CLIP's objective (matching images/captions), it stands to reason\nthat, e.g., CLIP would assign a high score to \"white dog with the word\nelectronic written on it\", \"a white dog\", and \"electronic\" to the\nimage in the top left of Figure 1. None of these, to me, seem like\noutright misclassifications. The authors motivate their consideration\nby citing Rosinski's work, in which he showed children pictures of\nvarious objects with correct and incorrect labels, where the children\nwere specifically tasked with labeling the image (and not the\ntext). But, CLIP has no access to such asymmetric direction, so to\ncall transcribing the text incorrect, to me, is misleading.\n\nThe authors' most important argument is that CLIP doesn't behave the\nsame as a human in this setting. The evidence: label-switched\n\"misclassifications\" (where the model predicts the written word rather\nthan the object depicted) don't depend on the semantic distance to the\ndistractor word (as measured by word2vec vs. the true object class)\nnor the spelling (as measured by Jaro-Winkler), as they do with\nhumans. The alternate hypothesis would be that, like in humans, it\nwould be more difficult to name the correct pictured object (say,\n\"dog\") if the distractor were semantically related (e.g., \"cat\") or\nclose in spelling (e.g., \"bog\").\n\nBut, my concern regarding the ambiguity of the setup remains: I don't\nthink it's fair to simply call the text transcription \"incorrect\". And\nso, without accounting for that, I don't think I buy the arguments the\nauthors are making with respect to comparison to human\nexperiments. The equivalent human test, IMO, is ill-designed: handing\nchildren images with text on them and asking them to write something\ndoesn't seem fair, and, for me, neither does this setup.", "summary_of_the_paper": "The authors investigate the behavior of CLIP when its handed images\nsuperimposed with text. The authors define a label-switched image as\none for which the word being superimposed changes the classification\nprediction of the model. They construct a corpus of {12 superordinate}\nx {138 basic} x {~150 images with FMRI data} by superimposing labels\nover each image and then measuring CLIP's response. The authors\nconduct an error analysis of CLIP on these images, arguing that\nbecause CLIP's errors aren't predictable by the semantic or spelling\nsimilarity of the word/pictured object (as they are in a set of human\npsychology experiments), CLIP doesn't appear to process images in a\nsimilar fashion to humans.", "summary_of_the_review": "I think that the authors should continue to peruse this direction,\nbecause I think it's potentially quite interesting: understanding\nif/how CLIP can reason about ambiguous scenarios where more than one\nanswer is correct (in this case: transcription vs. object detection)\nis cool. But: because the \"correct\" answer (or a task description)\nisn't ever given to CLIP (as it was to the children in Rosinski's\nwork), I can't bring myself to be convinced by any of the experimental\nclaims.  My suggestion to the authors would either to i) consider\nfine-tuning CLIP with using supervised data that specifies which of\nthe tasks should be undertaken (transcription vs. detection); or ii)\nconsider designing prompts that specify what should be predicted,\ne.g., \"A photo of the word X written in red font over a picture of Y\".", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "XlDHiAWhSAc", "reviewer": "Reviewer_gSM4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The resulting dataset provides a useful tool to evaluate the bias of image recognition models that are jointly trained with language. This can be used by future researchers to evaluate whether their models acquire human-like biases.\n- The authors present different analyses to disentangle the effects of picture-word interference in CLIP.\n\nWeaknesses:\n- The writing can be improved. I found hard to follow some of the results sections. Figure 1 could also be improved to depict an example of the task itself (Fig A1 is already better), which is unclear solely by the figure and potentially confuses the reader. Similarly, Figure 2 could be improved by showing more outputs, those corresponding to the target behaviour and those corresponding to the interference behaviour.\n- The RSA analysis shows that CNNs are less affected by superimposed words than CLIP. However, as also mentioned by the authors in Section 5, CLIP was trained on a different dataset. It would have been instructive to train CLIP on ImageNet and investigate whether the language-biased modelling is still different from CNNs. It is, in fact, unclear to me whether CNNs might observe similar patterns when trained on large, noisy data.", "summary_of_the_paper": "This paper proposes a testbed for picture-word interference in image classification models. The authors specifically investigate the performance of CLIP to understand whether such language-biased model shows similar interferences to the ones observed in humans. The dataset consists of images superimposed with words, representing two different category levels (basic and superordinate). The experiments on CLIP show that the model is affected by superimposed words, but independently of their semantic relationship with the underlying image. Further analysis shows that CLIP image representations are different from the ones of superimposed words, while this does not happen in two ImageNet-based CNNs.", "summary_of_the_review": "The paper presents a useful dataset to evaluate word-superimposed image recognition models. The analysis on CLIP shows how it behaves for different categories of superimposed words. The writing can be improved, especially around the presentation of the task and some of the results. Further experiments on ImageNet-based CLIP would make the paper stronger.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a new benchmark task for models similar to CLIP for evaluating how visual word forms interfere with the visual recognition of objects in images when the former are superimposed on the latter ones. Specifically, by superimposing words belonging to different categories  (e.g., hypernyms vs basic labels) the authors study the misclassification rates of CLIP under different degrees of varying similarity between the original and superimposes labels. 

All reviewers agreed that this is a novel and interesting study which, by productively using insights from cognitive science literature on language biases, aims at shedding light on the inner-workings of a popular artificial model. The main concern raised by reviewer P83Y was regarding the claims around misclassification rates. Indeed, since CLIP was not taught (e.g., by fune-tuning or few-shot prompting) which of the two labels (i.e., the written or the visual) is the correct one, it's not fair to assess its performance on this way. While this is strictly true, the experimental protocols presented in Sections 4.3/4/5 are still a valid way to assess representational inference. Moreover, the authors have followed P83Y suggestions and incorporated a few-shot prompting experiment in Section 4.6.

All in all, I think this will make for an addition to the ICLR program and thus I'm recommending accepting this paper.

(Minor comment: WKSS rightly pointing that this paper has, at best, a loose connection to compositionality. The authors changed compositionality -> representations which is a better fit, so please make sure to change the title also in Openreview when prompted.)
Decision: Accept (Poster)