Investigating Negation in Pre-trained Vision-and-language Models (2021)

Abstract: Pre-trained vision-and-language models have achieved impressive results on a variety of tasks, including ones that require complex reasoning beyond object recognition. However, little is known about how they achieve these results or what their limitations are. In this paper, we focus on a particular linguistic capability, namely the understanding of negation. We borrow techniques from the analysis of language models to investigate the ability of pre-trained vision-and-language models to handle negation. We find that these models severely underperform in the presence of negation.

## introduction vision-and-language models have made a lot of progress on complex tasks, going beyond recognition and towards reasoning over the two modalities @xcite . following the success of pre-trained language models such as bert @xcite on a range of language tasks, recent advances in vision-and-language have involved the introduction of pre-trained models (e.g., uniter, @xcite @xcite . these models achieve impressive results, topping task leaderboards and improving over previous approaches by a large margin. however, as with pre-trained language models, it is not clear how and why these models perform as well as they do, what information they learn and use in their predictions, or what their limitations are. while a large body of research has focused on the interpretation of pre-trained language models (e.g., @xcite @xcite , such work has been more limited for vision-and-language models. this paper focuses on a particular linguistic capability, namely the ability to understand negation. negation is universal across languages @xcite and is very important for interpreting the figure 1 : examples from the nlvr2 corpus. the input to the model consists of two images and a sentence. the model's task is to predict whether the sentence is true of the images or not. the top example shows an instance where the sentence is true and the bottom one, false. image from @xcite . for the purposes of this analysis, we focus on a particular vision-and-language task, natural language visual reasoning for real (nlvr2) @xcite . for this task, the model is presented with two images and a sentence and has to predict whether that sentence is true of the images or not (see figure 1 ). since this task involves the prediction of a truth value, it lends itself well to the exploration of negation. we consider two popular pre-trained vision-and-language models, finetuned for the task: lxmert @xcite and uniter @xcite . details on the data and models are in section 3. in order to investigate the performance of these models on examples that contain negation in a controlled manner, we annotated a portion of the original nlvr2 test set to create minimally differing instances. that is, we created pairs of instances where the images remain the same and the sentences only differ in the presence or absence of negation (see section 3 for details). we show that both models under consideration perform worse on the negated examples, compared to the corresponding non-negated examples (section 4). we also use causal mediation analysis @xcite on uniter to examine the contributions of specific neurons to the final predictions (section 5). our results show that the effects of negation are mainly seen in the upper layers of the model. we release the new test set and code for our experiments at https://github.com/radidd/ vision-and-language-negation . ## causal mediation analysis we will now take a closer look at the effect of adding negation and the contributions of specific neurons to model predictions using causal mediation analysis. this type of analysis can help us measure the effect of a change in the input (adding negation) on the output beyond looking at accuracy only. we also apply mediation analysis to find out if specific neurons in the model contribute to a change in the output more than others. we continue to work with the negation categories described previously and aim to discover if the results from this analysis correspond to the observed differences in accuracy between the categories and whether neuron effects differ by category. similarly to @xcite , we are working with a task which has correct and incorrect outputs (in their case continuations, in ours, labels). we therefore follow their definitions. we want to measure the ability of the model to predict the correct true/false label given the images and the sentence. therefore, we define our response variable as: this equation takes the following form depending on whether the gold label is true or false: the value for y is small (@xmath0, and large (@xmath1. following @xcite and @xcite , we define two do-operations: (a) negate: negate the original sentence so that the truth value changes, and (b) null: no change. we also define y x (u, l) to be the value of y when we apply the operation x to the context u. this takes the following values under the negate operation for each of the possible values of u: in order to measure the change in the response variable under the intervention (negation), we define the unit-level (per one original/negated pair) total effect as: as @xcite state, this quantity measures the margin between the probability of the correct and incorrect answers under an intervention. however, unlike @xcite , we analyse the originally correct and originally incorrect examples separately. when the model predicts the original example correctly, a larger total effect under the intervention could indicate a better handling of negation, as it suggests a higher probability of the correct label under negation. when the original example is predicted incorrectly, it is less clear what the total effect indicates. while a larger total effect suggests a move towards the correct prediction under negation, this does not necessarily mean negation is handled better. a smaller total effect suggests a move toward the incorrect label, but this means flipping the label, which is desired behaviour. we calculate the average total effect across all example pairs, for each negation type and for two different sizes of uniter: for the mediation analysis, we focus on the effects of individual neurons from the representation of the [cls] token on the response variable y. in the following definition, z refers to a single neuron. we measure the natural indirect effect (nie) of a change in the input x on the response variable y, with respect to a mediator z. as mentioned in section 2.3, this is done by fixing the input to its value without the intervention, but changing the value of the mediator z to its value under the intervention. in this case, we set the input to its value without negation (i.e. the original imagessentence pair which does not contain negation), but set the value of z (the neuron) to the value it would take if the input was the negated version of the images-sentence pair. the population level nie then is defined as: as we are concerned with the effects of specific mediators (the neurons) on the output, we do not calculate the natural direct effect (nde) which measures the effect of the input without the intervention of a mediator. we also do not consider intervening on attention heads in this work, as the length of the original and the negated sentences is different. ## conclusion our work shows that pre-trained vision-andlanguage models find it difficult to handle negation, which is a finding that is consistent with previous work on pre-trained language models. using a manually created set of minimally differing pairs we show that two vision-and-language models (lxmert and uniter) fail to reach good performance in the presence of negation. we conduct causal mediation analysis on the neurons of one model and find that the main effects of negation are found in the upper layers. however, these effects are small and do not seem to correlate well with model accuracy. while causal mediation analysis is a useful analysis tool, it is not straightforward to apply it to all models and to the analysis of attention when the input is of different length in the base case and under the intervention. in the future we would like to extend our analysis to other model variations (e.g., the uniter paired-attn model which processes the images separately from each other), and to attention heads. a negation word list not, isn't, aren't, doesn't, don't, can't, cannot, shouldn't, won't, wouldn't, no, none, nobody, nothing, nowhere, neither, nor, never, without