Attention as Grounding: Exploring Textual and Cross-Modal Attention on Entities and Relations in Language-and-Vision Transformer (2022)

Abstract: We explore how a multi-modal transformer trained for generation of longer image descriptions learns syntactic and semantic representations about entities and relations grounded in objects at the level of masked self-attention (text generation) and cross-modal attention (information fusion). We observe that cross-attention learns the visual grounding of noun phrases into objects and high-level semantic information about spatial relations, while text-to-text attention captures low-level syntactic knowledge between words. This concludes that language models in a multi-modal task learn different semantic information about objects and relations cross-modally and uni-modally (text-only). Our code is available here: https://github.com/GU-CLASP/attention-as-grounding .

## introduction in this paper, we examine what kind of knowledge is encoded in the multi-modal transformer. existing work has mostly looked at the knowledge captured in models that operate with a single modality (text). for instance, previous research has shown that the attention weights in large-scale models, e.g. bert @xcite , implicitly encode knowledge of sentence structure @xcite , partof-speech tags, syntactic dependencies @xcite , subject-verb agreement between words @xcite , and even information about textual co-reference @xcite . only a few papers have inspected what is captured by multi-modal architectures. @xcite demonstrate that the attention heads in image-and-text transformers effectively encode linguistic and cross-modal knowledge. @xcite provide the analysis of how language representations are indirectly affected by visual information in language-and-vision model. here we inspect what the model learns about two types of words in the multi-modal setting: (i) words denoting objects in the scene (e.g. "a red chair"), (ii) words depicting spatial relations between objects (e.g. "a chair next to the table"). while it is relatively simple to associate nouns with specific image regions, words describing relations are much harder to ground @xcite , possibly because visual representations are typically designed to capture objects without any explicit knowledge of relations. secondly, grounded relations depend on knowledge from both vision and language modalities which contains information about the objects and their mode of interaction (what) as well as their physical location (where) @xcite . ideally, each relation (and also other types of words) should be grounded in both modalities, but to a different degree. 1 however, studies of language-and-vision models indicate that they are frequently biased towards one modality, most often to language @xcite . therefore, the main research challenge is to develop architectures that learn to utilise an appropriate ratio of visual and language knowledge for generation (or understanding) of each word in its context. towards this goal we investigate grounding of different semantic types and answer the following questions: q1: does attention across two modalities learn visually grounded semantics of nouns? q2: what syntactic knowledge is encoded in attention on text in the multi-modal set-up? q3: what does cross-modal attention learn about grounded semantics of spatial relations? we use a two-stream multi-modal transformer @xcite , which first attends to each modality independently and then learns to attend cross-modally. this architecture uses rich relative geometry between objects, while many other twostream models @xcite simply use either coordinates of bounding boxes or their spatial location. we train the model for image paragraph generation @xcite , allowing examination of the knowledge of semantic types in extensive contexts. our experiments show how language and vision are bridged in the multi-modal transformer. in addition, our work provides insights into how multi-modal representations are learned for different word types. ## experimental set-up model we train a multi-modal transformer for image paragraph generation. the model is based on the image captioning transformer proposed by @xcite foot_1 . we use the object detector provided by @xcite foot_2 . this model comes pre-trained on object annotations from visual genome @xcite . we extract features of n objects per image, resulting in the set @xmath0, . . . , v n } with v @xmath1. we set @xmath2. the object extractor also provides us with labels ("table") and attributes ("round") for the objects, which will be used in our experiments. following @xcite , we also extract geometry information about each object @xmath3, y, w, h (centre coordinates, width, height) and use it as an additional input along with visual features. figure 1 describes the architecture of the model. in this model, each attention mechanism consists of six layers with eight attention heads in them. the image encoder (orange box) learns to combine visual and geometric features foot_3 and passes them through the standard self-attention block, consisting of multi-head self-attention, feed-forward network, residual connections and layer-normalisation. due to uni-directional nature of description generation, the text decoder (blue box) produces representation of the current token w i , based on previously generated tokens (w 1 , . . . , w i-1 ), while (w i+1 , . . . , w w ) are replaced with [m ask]. finally, the cross-attention (red box) uses information from both textual and visual streams to output a probability of the next word in the sequence. every next generated word is concatenated with the previously generated words and passed to the model to output the next word prediction. figure 2 : ground truth description of the image: it's a room with a bar on the side. there is a pink couch in the center. there's a coffee table in front of the couch. it has a light purple rug. there are three chairs at the bar. ## methods and metrics we extract the attention weights from both crossmodal attention and masked self-attention. here, we could examine attention of the model while it is generating a new description or attention of the model receiving a ground truth description using teacher-forcing. since our task is a validation task where we want to examine the behaviour of the model under fixed conditions we opted for ground truth descriptions. using generated descriptions could produce identifiable attention patterns but the descriptions are not guaranteed to contain entities and relations that are in the image and we are interested in. if the model has approximated the training data well, then the unseen ground truth descriptions will not be far off from its predictions. using ground truth descriptions that are not the model's predictions imposes more uncertainty for the model and therefore harsher conditions for evaluation of attention patterns. identifying interpretable attention patterns under these conditions therefore makes the conclusions stronger. for every generated word w i , the attention weight α per head h in each layer ℓ is extracted. in transformers the attention weights are computed as the scaled dot-product of the query matrix q with all the keys in k followed by a softmax operation. these weights are focusing on either previously generated words (masked self-attention msa, equation 1) or image objects (cross-attention ca, equation 2). we inspect how much attention is focused on specific parts of the input sequence when particular parts of the target sequence are generated. we refer to this measure as the attention focus or attention proportion. in our experiments, we calculate the proportion of total attention from a specific head that is focused on specific parts of the source sequence, e.g. previously generated words or image objects. attention proportions are generally calculated as follows: conditions on p for q1 for our experiments on visual grounding in cross-modal attention, t limits the target sequence to the text span of a noun phrase, while s defines the ground truth object that this noun phrase depicts. the attention proportion is calculated by computing the accumulated attention weight from the words in the noun phrase towards the corresponding object and then divided by the overall attention on all objects attended when this noun phrase is generated. we use spacy @xcite to extract noun phrases from image paragraphs which might introduce some errors, see appendix a for examples. we skip any phrases which contain at least one word from the list specified in appendix b. we keep determiners and adjectives in the noun phrases and any numerals if they occur. some of the paragraphs might contain noun phrases that cannot be grounded in the bounding boxes in the image; either because the bounding boxes are not identified or because the noun phrases refer to abstract concepts. these phrases typically contain words such as "room", "image" or "photo" and are generally placed at the beginning of the description (e.g., "the image is of a kitchen with . . . "). in future experiments, we plan to investigate how the model grounds general descriptions of the scene ("the nursery room"). conditions on p for q2 for the experiments on word-to-word attention, t is set to the generated word at the specific time-step t j , while s accumulates attention on words of specified part-of-speech tags when the target word t j is generated. @xcite show that masked self-attention on text is indirectly affected by vision in the multimodal set-up. nouns that often describe objects are attended to a greater extent than some other words of specific part-of-speech tag (e.g., verbs) even though this model has never seen the image directly. interestingly, the same phenomenon is not observed in text-only models such as distilgpt-2: its attention is much more local, focusing on the words that surround the target word instead of attending to more distant nouns. this finding suggests that a multi-modal transformer can learn semantic differences between words of various part-of-speech tags not just their structural arrangement which would be their syntax. therefore, we construct two sets of part-of-speech tags, which reflect semantic differences between words in terms of the possibility of their grounding. the first set contains determiners, adjectives and nouns used in descriptions of objects, while the second set includes verbs and adpositions used in descriptions of relations between objects. conditions on p for q3 to examine grounding of spatial relations, both s and t are determined based on the set of static spatial relations extracted from the texts. we extracted target -relationlandmark triplets from each description (there are likely to be multiple relations mentioned in a single image description sequence), based on the annotation schema described in @xcite and publicly available tool foot_4 . we obtained 1015 relations of region type ("clothes on hangers"), 239 relations of direction type ("a gold chandelier above the table"), and 6 relations of distance type ("a large vase in the middle of the table "). each of these relations consists of three spatial elements: a target (a cup), a landmark (a table) and a relation (on) in "a cup on the table". given that the word order describing relations is typically a targetrelation -landmark sequence, the attention proportion for masked self-attention can be extracted only in following directions: @xmath4, @xmath5, @xmath6, and @xmath7. for example, a possible t could restrict currently generated word to relation (typically expressed with adpositions), while s could limit the calculation of the attention focus to target (expressed as a noun phrase) in case of @xmath8. ## linking nouns and objects to inspect attention heads for visual grounding, we require ground truth annotations of correct linking between image objects and noun phrases. we construct such links automatically using semantic similarity between noun phrases and object labels provided by the object feature extractor. first, we use spacy @xcite and extract noun phrases on different levels of nesting. for example, a noun chunk "a window with white lace curtains" and the nested chunk "white lace curtains" are identified as two different noun phrases. potentially, this design choice allows for more accurate linking between noun phrases focusing on different objects ("window" and "curtains") and corresponding finegrained object detections. in addition, noun phrases with specific details potentially disambiguate linking when multiple objects of the same type are in the image, e.g., several windows. as for object labels, for every detected object in every image, we take the predicted label and its attribute if the extractor's confidence for this attribute is higher ## conclusion we have shown that the language model in a multimodal task captures linguistic phenomena of different kind depending on the source knowledge (text or objects) and semantic type of the output words (noun phrases or spatial relations). cross-modal attention visually grounds objects and, therefore, semantic dependencies in its deeper layers (addresses q1). text-only attention learns low-level linguistic phenomena, e.g. local syntactic dependencies (addresses q2). this is also exemplified for targetrelation-landmark descriptions which are attended in a sequential order that they appear in the text. we have also shown that there is a difference in a way objects and relations are grounded crossmodally and such grounding is particularly challenging for relations (addresses q3). the grounding of landmarks depends on relations to a greater degree than on targets in both masked and crossmodal self-attentions. this could be attributed to the auto-regressive nature of the image paragraph generation task. however, there are important differences in terms of activations across attention layers for different semantic pairs. deeper heads in the cross-modal attention tend to be activated more than the surface heads which is the opposite tendency compared to masked self-attention. overall, our work demonstrates that attention on vision and language captures considerably more diverse linguistic knowledge, both syntactic and semantic which is not linearly aligned, compared to uni-modal (language only) architectures. one possible follow-up experiment is to use attention as input to the probing classifier and identify a specific knowledge encoded by the weights. however, the performance of the probing model does not tell us whether the original model utilises acquired knowledge since it is detached from the original architecture @xcite . although attention is not necessarily an explanation @xcite , inferring linguistic properties from attention weights does not require learning a new set of parameters. other methods include finegrained analysis of features preferred by specific neurons in the model architecture by examining their maximum activation values @xcite . this method would identify the neurons that are active at each step of generation, but would not straightforwardly tell us how words and objects are linked together, which is clearly expressed in attention. our results indicate that the way relations are grounded in a transformer model is not completely transparent. future research should focus on examining the effect of different feature representations that are relevant for spatial relations (e.g., rgb-d and different models of geometry, common sense knowledge about objects' affordances) as well as the models that can be built around them. in another follow-up study we could examine grounding of relations in a different task, for example in vision-and-language navigation @xcite which is rich with descriptions of relations between objects and compare whether the same observations also hold for those models.