  Multi-headed attention heads are a mainstay in transformer-based models.
Different methods have been proposed to classify the role of each attention
head based on the relations between tokens which have high pair-wise attention.
These roles include syntactic (tokens with some syntactic relation), local
(nearby tokens), block (tokens in the same sentence) and delimiter (the special
[CLS], [SEP] tokens). There are two main challenges with existing methods for
classification: (a) there are no standard scores across studies or across
functional roles, and (b) these scores are often average quantities measured
across sentences without capturing statistical significance. In this work, we
formalize a simple yet effective score that generalizes to all the roles of
attention heads and employs hypothesis testing on this score for robust
inference. This provides us the right lens to systematically analyze attention
heads and confidently comment on many commonly posed questions on analyzing the
BERT model. In particular, we comment on the co-location of multiple functional
roles in the same attention head, the distribution of attention heads across
layers, and effect of fine-tuning for specific NLP tasks on these functional
roles.
