How and where does CLIP process negation? (2024)

Abstract: Various benchmarks have been proposed to test linguistic understanding in pre-trained vision & language (VL) models. Here we build on the existence task from the VALSE benchmark (Parcalabescu et al., 2022) which we use to test models’ understanding of negation, a particularly interesting issue for multimodal models. However, while such VL benchmarks are useful for measuring model performance, they do not reveal anything about the internal processes through which these models arrive at their outputs in such visio-linguistic tasks. We take inspiration from the growing literature on model interpretability to explain the behaviour of VL models on the understanding of negation. Specifically, we approach these questions through an in-depth analysis of the text encoder in CLIP (Radford et al., 2021), a highly influential VL model. We localise parts of the encoder that process negation and analyse the role of attention heads in this task. Our contributions are threefold. We demonstrate how methods from the language model interpretability literature (e.g., causal tracing) can be translated to multimodal models and tasks; we provide concrete insights into how CLIP processes negation on the VALSE existence task; and we highlight inherent limitations in the VALSE dataset as a benchmark for linguistic understanding.

## introduction research in vision & language (vl) modelling has produced various pre-trained models that are capable of jointly processing image and text information by learning multimodal representations (e.g., * work carried out as m.sc. student at utrecht university @xcite @xcite @xcite . this makes them applicable to a host of downstream tasks, such as visual question answering, image caption generation or zero-shot image classification. various benchmarks have been proposed to test these models' understanding of different linguistic features, such as word order @xcite , verb meaning (hendricks and @xcite , and compositionality @xcite . the valse benchmark @xcite was introduced to test these models' ability to ground features such as existence, plurality, or spatial relations in images. an example of the existence piece is shown in figure 1 . given an image, a model must choose between a correct caption and an incorrect foil, one of which contains a negation operator. as such, this piece can be used to test a model's understanding of negation, a particularly interesting issue for multimodal models, which typically include a visual backbone pretrained on computer vision tasks such as object labelling. the models themselves are further pretrained on image-text pairs where there is likely to be a positive bias, since captions describing images will typically refer to what is depicted there. this raises the question whether vl models are capable of processing operators such as "no" in instances such as those in figure 1 . indeed, negation remains a weakness of even the most state-of-the-art large language models @xcite in line with these intuitions, initial valse results reveal that models only achieve moderate performance in this (and other) linguistic categories. however, while vl benchmarks such as valse are useful for measuring current and future model performance, they do not reveal anything about the internal processes through which these models arrive at their outputs in such visio-linguistic tasks. we aim to make use of the growing literature on model interpretability @xcite in order to explain the behaviour (and shortcomings) of vl models on the understanding of negation. to do this, we use the existence sub-task in valse, with some extensions, exploiting localisation techniques to quantify the roles that different model components play in this task. this yields the following research question: which components of vl models are responsible for the model's understanding of negation? we address two issues that arise from this general question, namely (1) the extent to which processing of negation is localised vs. distributed; and (2) whether model performance on valse-like tasks involving negation can in part be explained by high-level dataset features. specifically, we approach these questions through an in-depth analysis of clip @xcite , a highly influential vl model. clip has a relatively simple design based exclusively on transformers, which allows us to leverage interpretability techniques that target this architecture. additionally, prior work by @xcite shows that clip makes balanced use of text and image input and avoids so-called unimodal collapse @xcite @xcite , an important consideration for a study of multimodal model interpretability. finally, clip remains central to developments in both vision (e.g. the clipseg segmentation model; @xcite and vl tasks (e.g. clip is a component of several text-to-image and image-to-text models, including @xcite @xcite . in our analysis of negation, we focus on the clip text encoder. however, it is important to note that clip is pretrained with a multimodal contrastive objective, which has been shown to yield different representations compared to text-only encoders with comparable architecture but different training objectives @xcite . thus, we take the insights into the text encoder's ability to process negation as reflecting on the success or otherwise of the contrastive, multimodal pretraining in such models. the contributions of this work are threefold: firstly, we demonstrate how methods from the language model interpretability literature (e.g., causal tracing; @xcite can be translated to multimodal models and tasks; secondly, we provide concrete insights into how clip processes negation on the valse existence task; thirdly, we highlight inherent limitations in the valse dataset as a benchmark for linguistic understanding. with the introduction of clip @xcite , contrastive learning objectives have become prominent in vl models, with or without additional objectives that address multimodal fusion @xcite @xcite . models such as blip @xcite and flava @xcite ) combine contrastive objectives with unimodal pretraining of vision and language encoders. architectures such as flamingo @xcite and blip-2 @xcite reduce training cost by training relatively small networks to map between representations from pretrained image and language models. in clip, an image encoder and a text encoder process their respective inputs completely separately from each other, i.e., without any multimodal cross-attention and project them into the same latent space. the goal of contrastive learning is to maximise similarity between matching image-text pairs, minimising the similarity between non-matching pairs. during inference, clip computes the similarity of an image and a text in the form of a scaled dot product between their embeddings. contrastive objectives have been shown to yield better embedding representations (wolfe and caliskan, 2022) leading to improved performance on semantic evaluation tasks @xcite . vision-and-language benchmarks vl benchmarks focusing on specific linguistic phenomena play an important role in highlighting strengths and weaknesses in models' grounding capabilities. for example, a recent study combining several benchmarks @xcite showed that models still find certain linguistic phenomena challenging, and that grounding capabilities may be less related ## discussion the causal tracing results from section 4.1 suggest relatively strong localisation in the early (1-3) and late (8-12) layers, meaning that negation is largely represented at singular positions in these layers. in layer 4, the cte at the negator position drops sharply, and this conincides with the finding of negator-selective attention heads in layer 4 which appear to shift negation information to later positions. the locations of these attention heads also overlap with those found on the cannot dataset, which provides initial evidence that the clip text encoder uses certain attention heads for specific syntactic functions. in the middle layers localisation is generally lower with no single position restoring more than 60% of the original effect. this implies that representation of negation is distributed across positions and that the model relies on combining the representations at each position in order to make correct judgements about negations. furthermore, the first subject token position appears to play a unique role in cases with negation in the caption, which could be due to the asymmetry in the two tasks. when the negation is in the foil, the label's subject is shown in the image and, intuitively, once it is detected, a decision can be made and no further processing is necessary. conversely, when the negation is in the caption, the entire image needs to be scanned to ensure that the label's caption is in fact absent from all parts of the image. this difference could be part of the reason why the first subject token position appears to play a role up until deeper layers of the network, when the negation is in the caption. the effects of the subject position in deeper layers could imply that the subject information is in fact more deeply processed and thus more strongly represented in the final text encoder's output which, in turn, could be conducive to the model's task of "searching" for the subject in the image's representation. however, these explanations are speculative and must not be accepted without further experiments. section 4.3 highlights that the label's length and the subject's size in the image show non-negligible correlations with respect to the classification score. this suggests that clip is better at the valse existence task when labels are shorter and therefore produce less similar multimodal embeddings and when the subject in the image is sufficiently large. arguably, the more variance in classification score can be explained on the basis of such dataset variables, the less clip's benchmark score can be interpreted as an indicator of its linguistic understanding, thus calling into question the validity of the valse benchmark. however, none of the correlations found in the present study are particularly high and thus further analyses are needed to support this conclusion. ## limitations and future work the degree of localisation found in clip's text encoder is hard to interpret without reference to other results. future work could extend the present methodology to other tasks and potentially other models. our study is also limited to simple effects of individual layer/position pairs. an analysis of the interaction of certain layers or positions (e.g., by simultaneously patching activations in multiple places during causal tracing) might draw a more robust and conclusive picture of the inner processes that govern clip's understanding of negation. more generally, localisation methods may not be suited for analysing model behaviour that is shown with only moderate reliability. note that the methods used in the present study had originally been proposed and applied to language model capabilities that are shown reliably across a large corpus of data, e.g., indirect object identification @xcite , simple factual knowledge @xcite , or docstring completion @xcite . by contrast, clip does not reliably handle negation in a multimodal context (clip's accuracy is only 66.9%) and these results are based on a relatively small dataset (@xmath0. in this case, methods like causal tracing do not intuitively lend themselves to comparing situations evincing a particular model behaviour to those where the behaviour is absent. that is because they focus on the degree to which an effect that represents a particular model behaviour can be restored or ablated, but the methodology breaks down when this effect isn't present in the first place. thus, whilst illuminating the role of various components in clip's processing of negation, we cannot provide strong insights into why this processing yields correct classifications only in a fraction of cases. furthermore, since correct classification only occurs in a subset of instances of valse, which is moderately sized to begin with, the results described here require a larger and potentially more diverse dataset to obtain greater validity. with respect to the validity of the underlying valse benchmark, it might be worthwhile to conduct a larger study on dataset features (e.g., image brightness, contrast, etc.) that correlate with benchmark performance. comparisons with other vl benchmarks would further help putting these results into perspective. such features that are predictive of benchmark performance limit the validity of linguistic benchmarks and highlight variables that should be controlled for in the creation of future benchmarks.