Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a testbed for picture-word interference in image classification models. The authors specifically investigate the performance of CLIP to understand whether such language-biased model shows similar interferences to the ones observed in humans. The dataset consists of images superimposed with words, representing two different category levels (basic and superordinate). The experiments on CLIP show that the model is affected by superimposed words, but independently of their semantic relationship with the underlying image. Further analysis shows that CLIP image representations are different from the ones of superimposed words, while this does not happen in two ImageNet-based CNNs.
main_review: Strengths:
- The resulting dataset provides a useful tool to evaluate the bias of image recognition models that are jointly trained with language. This can be used by future researchers to evaluate whether their models acquire human-like biases.
- The authors present different analyses to disentangle the effects of picture-word interference in CLIP.

Weaknesses:
- The writing can be improved. I found hard to follow some of the results sections. Figure 1 could also be improved to depict an example of the task itself (Fig A1 is already better), which is unclear solely by the figure and potentially confuses the reader. Similarly, Figure 2 could be improved by showing more outputs, those corresponding to the target behaviour and those corresponding to the interference behaviour.
- The RSA analysis shows that CNNs are less affected by superimposed words than CLIP. However, as also mentioned by the authors in Section 5, CLIP was trained on a different dataset. It would have been instructive to train CLIP on ImageNet and investigate whether the language-biased modelling is still different from CNNs. It is, in fact, unclear to me whether CNNs might observe similar patterns when trained on large, noisy data.
summary_of_the_review: The paper presents a useful dataset to evaluate word-superimposed image recognition models. The analysis on CLIP shows how it behaves for different categories of superimposed words. The writing can be improved, especially around the presentation of the task and some of the results. Further experiments on ImageNet-based CLIP would make the paper stronger.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper introduced a benchmark task for evaluating how semantic processing of visual word form interfere with the recognition processing of an object image in word-embedded images. By placing category word (e.g., superordinate category like 'electronic' or basic category like 'laptop') in the center of an image, the authors evaluated the 1) the misclassification rates of CLIP; 2) the semantic/spelling similarity between original and superimposed labels under different conditions; 3) the change of model prediction probability; 4) RSA on images with different types of picture-word interference.
In general, this work provided interesting new insights on how language information bias the image classification in visual-language models like CLIP. The authors concluded that CLIP can recognize the visual word form as a word but fails to encode word form and object image that share the same category (e.g. word 'dog' and picture 'dog') with similar visual representations.
main_review: Overall this paper is well-written. The motivation and background are clearly stated. To evaluate the effect of word-picture interference, the authors applied multiple analyses, including assessing the model behaviors under different conditions (section 4.2, 4,5), connecting model behavior to label similarity (section 4.3, 4.4), and comparing the similarity of visual model representations (section 4.6). The methods are described in details and should be easy for readers to reproduce their analyses. The authors are trying to figure out how the visual representations of word form and object images interfere with each other and bias the classification behavior. This is an intriguing question driven by observations and theories from cognitive science.

Comments:
1. In general, I agree with the authors' claims. However, I feel there exists some interesting findings in the current results remain further explanation or investigation. First, during pre-training, the CLIP model already matches the representation of "*object in visual form*" and "*word in textual form*" in the same metric space. And the authors further find that the image classification (section 4.2, 4.5) is strongly biased by the superimposed word (e.g., examples in Figure A1), even in the B/S or S/B condition (e.g., recognize tulip image labeled 'electronic' to semantic class 'keyboard', or recognize tulip image labeled 'laptop' to semantic class 'electronic'). Does it imply that the representational space of "*words in visual form*" should have similar structure (in the sense of pair-wise similarity) as the representational space of "*word in textual form*"? However, all the similarity analyses based on representations (section 4.3, 4.4, 4.6) showed non-distinguishable results across conditions, which lead to the authors' statement that "the CLIP image encoder has different representations of word forms from visual image representations". But it seems that at least both "word forms" and "object image" representations from CLIP image encoder captures semantic similarity in the text-based word space. What's authors interpretation on this?

2. I am a bit surprised about the good performance of CLIP recognizing spelling word into the same category word. Were all the results based on pre-trained CLIP without fine-tuning? Was CLIP trained on some images with visual word forms to learn typographic features?

3. Were all 'misclassified' word-embedded images classified as the superimposed label? If not, it might be better to divide the 'misclassification' condition into two cases: 'biased' (predict the superimposed label) and 'random' (predict a label not original or superimposed), just to make results in section 4.2 more informative.

4. The left box in each panel of Figure 4 is a bit ambiguous to me. What is 'label-switched' for 'original images'?

5. What would be the model behavior and similarity results if the image is interfered with a nonsense pseudoword?

6. Typo: 'fisrt' should be 'first' on page 5.

7. What does the "semantic compositionality" in the paper title refer to? I feel this term is weakly connected to the main text.
summary_of_the_review: This paper introduced a benchmark task for evaluating picture-word interference in visual-language joint learning models. The benchmark was tested with the latest model CLIP. The evaluation results brought novel insights to understand the recognition process of the CLIP image encoder, showing its distinction on visual representations of word forms and object images. This provides a new perspective for future studies to evaluate the interference of multi-modal processing in AI models. Some results and figures may need further explanation or clarification for better understanding of their findings.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors introduce a framework to evaluate the picture-word interference in CLIP which is known to exist in human cognition.
It is designed to test whether language-biased decisions occur across different category levels, and the extent to which picture-word interference in CLIP depends on the semantic similarity between superimposed words and images. Experimental results show that presenting words disturbed the CLIP image classification even across different category levels, the effect did not depend on the semantic relationship between images and words, and the superimposed word representation in the CLIP image encoder is not shared with the image representation.
main_review: [Overall] This is a very interesting piece of work that analyzes whether the state-of-the-art artificial vision and language model; CLIP, resembles human cognition or not. 

[Formatting]
- Figures and Tables should be aligned to the top of the page.

[Writing]
- I though that CLIP is trained by inputting image and words separately, but according to the authors statement in Section 5: "how CLIP acquired the ability to read superimposed words", it seems that it models language information from the image (superimposed text). Is this true? If so, I doubt it, and accordingly, results and discussions based on it may be results of some undiscussed reasons. Please also introduce in detail the task settings so that this kind of confusion will not occur.
summary_of_the_review: Work like this should is very important to understand how and why the state-of-the-art models perform well, what their limitations are, and how we can improve them.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors investigate the behavior of CLIP when its handed images
superimposed with text. The authors define a label-switched image as
one for which the word being superimposed changes the classification
prediction of the model. They construct a corpus of {12 superordinate}
x {138 basic} x {~150 images with FMRI data} by superimposing labels
over each image and then measuring CLIP's response. The authors
conduct an error analysis of CLIP on these images, arguing that
because CLIP's errors aren't predictable by the semantic or spelling
similarity of the word/pictured object (as they are in a set of human
psychology experiments), CLIP doesn't appear to process images in a
similar fashion to humans.
main_review: The question of what similarity CLIP should assign to text in images
vs.  pictured objects is potentially interesting. There doesn't seem
to be a "right" answer, as both transcribing the red text and naming
the underlying pictured object are arguably justified. The connection
to Rosinski's work is interesting, and understanding how CLIP deals
with the ambiguity of this situation is perhaps interesting from the
perspective of adversarial examples.

I had a few fundamental concerns with this work. First, I didn't
entirely understand the motivation. Goh et al. 2021 demonstrated that
CLIP's predictions might flip if a word was written in the image, but
I hesitate to call this a "misclassification" as the authors say:
given CLIP's objective (matching images/captions), it stands to reason
that, e.g., CLIP would assign a high score to "white dog with the word
electronic written on it", "a white dog", and "electronic" to the
image in the top left of Figure 1. None of these, to me, seem like
outright misclassifications. The authors motivate their consideration
by citing Rosinski's work, in which he showed children pictures of
various objects with correct and incorrect labels, where the children
were specifically tasked with labeling the image (and not the
text). But, CLIP has no access to such asymmetric direction, so to
call transcribing the text incorrect, to me, is misleading.

The authors' most important argument is that CLIP doesn't behave the
same as a human in this setting. The evidence: label-switched
"misclassifications" (where the model predicts the written word rather
than the object depicted) don't depend on the semantic distance to the
distractor word (as measured by word2vec vs. the true object class)
nor the spelling (as measured by Jaro-Winkler), as they do with
humans. The alternate hypothesis would be that, like in humans, it
would be more difficult to name the correct pictured object (say,
"dog") if the distractor were semantically related (e.g., "cat") or
close in spelling (e.g., "bog").

But, my concern regarding the ambiguity of the setup remains: I don't
think it's fair to simply call the text transcription "incorrect". And
so, without accounting for that, I don't think I buy the arguments the
authors are making with respect to comparison to human
experiments. The equivalent human test, IMO, is ill-designed: handing
children images with text on them and asking them to write something
doesn't seem fair, and, for me, neither does this setup.
summary_of_the_review: I think that the authors should continue to peruse this direction,
because I think it's potentially quite interesting: understanding
if/how CLIP can reason about ambiguous scenarios where more than one
answer is correct (in this case: transcription vs. object detection)
is cool. But: because the "correct" answer (or a task description)
isn't ever given to CLIP (as it was to the children in Rosinski's
work), I can't bring myself to be convinced by any of the experimental
claims.  My suggestion to the authors would either to i) consider
fine-tuning CLIP with using supervised data that specifies which of
the tasks should be undertaken (transcription vs. detection); or ii)
consider designing prompts that specify what should be predicted,
e.g., "A photo of the word X written in red font over a picture of Y".
review_title: null

Meta-review: This paper presents a new benchmark task for models similar to CLIP for evaluating how visual word forms interfere with the visual recognition of objects in images when the former are superimposed on the latter ones. Specifically, by superimposing words belonging to different categories  (e.g., hypernyms vs basic labels) the authors study the misclassification rates of CLIP under different degrees of varying similarity between the original and superimposes labels. 

All reviewers agreed that this is a novel and interesting study which, by productively using insights from cognitive science literature on language biases, aims at shedding light on the inner-workings of a popular artificial model. The main concern raised by reviewer P83Y was regarding the claims around misclassification rates. Indeed, since CLIP was not taught (e.g., by fune-tuning or few-shot prompting) which of the two labels (i.e., the written or the visual) is the correct one, it's not fair to assess its performance on this way. While this is strictly true, the experimental protocols presented in Sections 4.3/4/5 are still a valid way to assess representational inference. Moreover, the authors have followed P83Y suggestions and incorporated a few-shot prompting experiment in Section 4.6.

All in all, I think this will make for an addition to the ICLR program and thus I'm recommending accepting this paper.

(Minor comment: WKSS rightly pointing that this paper has, at best, a loose connection to compositionality. The authors changed compositionality -> representations which is a better fit, so please make sure to change the title also in Openreview when prompted.)
Decision: Accept (Poster)