Attention Can Reflect Syntactic Structure (If You Let It) (2021)

Abstract: Since the popularization of the Transformer as a general-purpose feature encoder for NLP, many studies have attempted to decode linguistic structure from its novel multi-head attention mechanism. However, much of such work focused almost exclusively on English — a language with rigid word order and a lack of inflectional morphology. In this study, we present decoding experiments for multilingual BERT across 18 languages in order to test the generalizability of the claim that dependency syntax is reflected in attention patterns. We show that full trees can be decoded above baseline accuracy from single attention heads, and that individual relations are often tracked by the same heads across languages. Furthermore, in an attempt to address recent debates about the status of attention as an explanatory mechanism, we experiment with fine-tuning mBERT on a supervised parsing objective while freezing different series of parameters. Interestingly, in steering the objective to learn explicit linguistic structure, we find much of the same structure represented in the resulting attention patterns, with interesting differences with respect to which parameters are frozen.

## attention as structure transformers the focus of the present study is mbert, a multilingual variant of the exceedingly popular language model @xcite . bert is built upon the transformer architecture @xcite , which is a self-attentionbased encoder-decoder model (though only the encoder is relevant to our purposes). a transformer takes a sequence of vectors @xmath0, x 2 , ...x n ] as input and applies a positional encoding to them, in order to retain the order of words in a sentence. these inputs are then transformed into query (q), key (k), and value (v ) vectors via three separate linear transformations and passed to an attention mechanism. a single attention head computes scaled dot-product attention between k and q, outputting a weighted sum of v : searching for structure often, the line of inquiry regarding interpretability in nlp has been concerned with extracting and analyzing linguistic information from neural network models of language @xcite . recently, such investigations have targeted transformer models @xcite @xcite , at least in part because the self-attention mechanism employed by these models offers a possible window into their inner workings. with large-scale machine translation and language models being openly distributed for experimentation, several researchers have wondered if self-attention is capable of representing syntactic structure, despite not being trained with any overt parsing objective. in pursuit of this question, raganato et al. (2018) applied a maximum-spanning-tree algorithm over the attention weights of several trained mt models, comparing them with gold trees from universal dependencies @xcite . they found that, while the accuracy was not comparable to that of a supervised parser, it was nonetheless higher than several strong baselines, implying that some structure was consistently represented. @xcite corroborated the same findings for bert when decoding full trees, but observed that individual dependency relations were often tracked by specialized heads and were decodable with much higher accuracy than some fixed-offset baselines. concurrently, @xcite made a similar observation about heads specializing in specific dependency relations, proposing a coarse taxonomy of head attention functions: positional, where heads attend to adjacent tokens; syntactic, where heads attend to specific syntactic relations; and rare words, where heads point to the least frequent tokens in the sentence. htut et al. (2019) followed raganato et al. (2018) in decoding dependency trees from bert-based models, finding that finetuning on two classification tasks did not produce syntactically plausible attention patterns. lastly, @xcite modified ud annotation to better represent attention patterns and introduced a supervised head-ensembling method for consolidating shared syntactic information across heads. does attention have explanatory value? though many studies have yielded insight about how attention behaves in a variety of models, the question of whether it can be seen as a "faithful" explanation of model predictions has been subject to much recent debate. for example, @xcite present compelling arguments that attention does not offer a faithful explanation of predictions. primarily, they demonstrate that there is little correlation between standard feature importance measures and attention weights. furthermore, they contend that there exist counterfactual attention distributions, which are substantially different from learned attention weights but that do not alter a model's predictions. using a similar methodology, @xcite corroborate that attention does not provide an adequate account of an input component's importance. in response to these findings, @xcite question the assumptions underlying such claims. attention, they argue, is not a primitive, i.e., it cannot be detached from the rest of a model's components as is done in the experiments of @xcite . they propose a set of four analyses to test whether a given model's attention mechanism can provide meaningful explanation and demonstrate that the alternative attention distributions found via adversarial training methods do, in fact, perform poorly compared to standard attention mechanisms. on a theoretical level, they argue that, although attention weights do not give an exclusive "faithful" explanation, they do provide a meaningful plausible explanation. this discussion is relevant to our study because it remains unclear whether or not attending to syntactic structure serves, in practice, as plausible explanation for model behavior, or whether or not it is even capable of serving as such. indeed, the studies of @xcite and @xcite relate a convincing but incomplete picture -tree decoding accuracy just marginally exceeds baselines and various relations tend to be tracked across varying heads and layers. thus, our fine-tuning experiments (detailed in the following section) serve to enable an "easy" setting wherein we explicitly inform our models of the same structure that we are trying to extract. we posit that, if, after fine-tuning, syntactic structures were still not decodable from the attention weights, one could safely conclude that these structures are being stored via a nontransparent mechanism that may not even involve attention weights. such an insight would allow us to conclude that attention weights cannot provide even a plausible explanation for models relying on syntax. ## experimental design to examine the extent to which we can decode dependency trees from attention patterns, we run a tree decoding algorithm over mbert's attention heads -before and after fine-tuning via a parsing objective. we surmise that doing so will enable us to determine if attention can be interpreted as a reliable mechanism for capturing linguistic structure. ## decoding mbert attention the second row of ## fine-tuning experiments next, we investigate the effect fine-tuning has on uuas decoding. row 3 in given that we do manage to decode reasonable syntactic trees, we can then refine our question -what components are capable of learning these trees? one obvious candidate is the key/query component pair, given that attention weights are a scaled softmax of a composition of the two. figure 3 ( @xcite shows the difference between pretrained uuas and fine-tuned uuas per layer, across models and languages. interestingly, the best parsing accuracies do not appear to vary much depending on what component is frozen. we do see a clear trend, however, in that decoding the attention patterns of the fine-tuned model typically yields better uuas than the pretrained model, particularly in the highest layers. indeed, the lowest layer at which fine-tuning appears to improve decoding is layer 7. this implies that, regardless of which component remains frozen, the parameters facing any sort of significant and positive update tend to be those appearing towards the higher-end of the network, closer to the output. for the frozen components, the best improvements in uuas are seen at the final layer in value, which is also the only model that shows consistent improvement, as well as the highest average improvement in mean scores foot_3 for the last few layers. perhaps most interestingly, the mean uuas (figure 3 (bottom)) for our "attentive" componentskeys, queries, and their combination -does not appear to have improved by much after fine-tuning. in contrast, the maximum does show considerable improvement; this seems to imply that although all components appear to be more or less equally capable of learning decodable heads, the attentive components, when fine-tuned, appear to sharpen fewer heads. note that the only difference between keys and queries in an attention mechanism is that keys are transposed to index attention from/to appropriately. surprisingly, key and query appear to act somewhat differently, with query being almost uniformly better than key with the best heads, whilst key is slightly better with averages, implying distinctions in how both store information. furthermore, allowing both keys and queries seems to result in an interesting contradiction -the ultimate layer, which has reasonable maximums and averages for both key and query, now seems to show a uuas drop almost uniformly. this is also true for the completely unfrozen encoder. supervised parsing in addition to decoding trees from attention matrices, we also measure supervised uas/las on a held-out test set. 5 based on figure 4 , it is apparent that all settings result in generally the same uas. this is somewhat expected; lauscher et al. (2020) see better results on parsing with the entire encoder frozen, implying that the task is easy enough for a biaffine parser to learn, given frozen mbert representations. 6 the las distinction is, however, rather interesting: there is a marked difference between how important the dense layers are, as opposed to the attentive components. this is likely not reflected in our uuas probe as, strictly speaking, labelling arcs is not equivalent to searching for structure in sentences, but more akin to classifying pre-identified structures. we also note that dense appears to be better than none on average, implying that non-dense components might actually be hurting labelling capacity. in brief, consolidating the two sets of results above, we can draw three interesting conclusions about the components: our last result is especially surprising for uuas decoding. keys and queries, fundamentally, combine to form the attention weight matrix, which is precisely what we use to decode trees. one would expect that allowing these components to learn from labelled syntax would result in the best improvements to decoding, but all three have surprisingly negligible mean improvements. this indicates that we need to further improve our understanding of how attentive structure and weighting really works. ## conclusion in this study, we revisited the prospect of decoding dependency trees from the self-attention patterns of transformer-based language models. we elected to extend our experiments to 18 languages in order to gain better insight about how tree decoding accuracy might be affected in the face of (modest) typological diversity. surprisingly, across all languages, we were able to decode dependency trees from attention patterns more accurately than an adjacent-linking baseline, implying that some structure was indeed being tracked by the mechanism. in looking at specific relation types, we corroborated previous studies in showing that particular layer-head combinations tracked the same relation with regularity across languages, despite typological differences concerning word order, etc. in investigating the extent to which attention can be guided to properly capture structural relations between input words, we fine-tuned mbert as input to a dependency parser. this, we found, yielded large improvements over the pretrained attention patterns in terms of decoding accuracy, demonstrating that the attention mechanism was learning to represent the structural objective of the parser. in addition to fine-tuning the entire mbert network, we conducted a series of experiments, wherein we updated only select components of model and left the remainder frozen. most surprisingly, we observed that the transformer parameters designed for composing the attention matrix, k and q, were only modestly capable of guiding the attention to-wards resembling the dependency structure. in contrast, it was the value (v ) parameters, which are used for computing a weighted sum over the kqproduced attention, that yielded the most faithful representations of the linguistic structure via attention. though prior work @xcite seems to indicate that there is a lack of a substantial change in attention patterns after fine-tuning on syntax-and semantics-oriented classification tasks, the opposite effect has been observed with fine-tuning on negation scope resolution, where a more explanatory attention mechanism can be induced @xcite . our results are similar to the latter, and we demonstrate that given explicit syntactic annotation, attention weights do end up storing more transparently decodable structure. it is, however, still unclear which sets of transformer parameters are best suited for learning this information and storing it in the form of attention.