Using Integrated Gradients and Constituency Parse Trees to explain Linguistic Acceptability learnt by BERT (2021)

Abstract: Linguistic Acceptability is the task of determining whether a sentence is grammatical or ungrammatical. It has applications in several use cases like Question-Answering, Natural Language Generation, Neural Machine Translation, where grammatical correctness is crucial. In this paper we aim to understand the decision-making process of BERT (Devlin et al., 2019) in distinguishing between Linguistically Acceptable sentences (LA) and Linguistically Unacceptable sentences (LUA).We leverage Layer Integrated Gradients Attribution Scores (LIG) to explain the Linguistic Acceptability criteria that are learnt by BERT on the Corpus of Linguistic Acceptability (CoLA) (Warstadt et al., 2018) benchmark dataset. Our experiments on 5 categories of sentences lead to the following interesting findings: 1) LIG for LA are significantly smaller in comparison to LUA, 2) There are specific subtrees of the Constituency Parse Tree (CPT) for LA and LUA which contribute larger LIG, 3) Across the different categories of sentences we observed around 88% to 100% of the Correctly classified sentences had positive LIG, indicating a strong positive relationship to the prediction confidence of the model, and 4) Around 43% of the Misclassified sentences had negative LIG, which we believe can become correctly classified sentences if the LIG are parameterized in the loss function of the model.

## introduction linguistic acceptability is an important criteria in natural language processing and is one of the tasks in the general language understanding evaluation (glue) benchmark @xcite . with the evolution of language encoders like bert (which leverages the multi-head selfattention mechanism @xcite in its architecture) that have been a breakthrough in language understanding and achieved state-of-the-art results, the field of probing these architectures for understanding their behaviours has become important. while there have been several works on interpreting and understanding the different layers of bert with respect to lexical, syntactic and semantic behaviours @xcite @xcite @xcite , the focus on explaining the linguistic acceptability (grammaticality) learnt by bert has been sparse. some of the recent works have used probing tasks to understand the model's knowledge on particular grammatical features @xcite @xcite , relying on language model probabilities to judge grammatical acceptability on sentences that differ minimally @xcite , or probing the model's by training with boolean grammaticality judgement objectives @xcite @xcite . these methods have made significant progress in uncovering that bert has indeed learnt various aspects of grammatical knowledge, however their focus has not been on explaining the black box details of how bert arrives at a grammaticality judgement. our paper attempts to address this by explaining the model's linguistic acceptability judgement with lig and cpt (a type of grammar tree) representations. attention mechanism based methods @xcite provide interpretable understanding of the model's behaviour, however the attention scores cannot be solely relied upon since a feature could influence the output in multiple ways (for e.g. through memory cells, recurrent states etc. in lstm networks). feature attribution methods aim to understand the relationship between the model's output and the input features. box details of neural networks and provide insights that can be used to improve model performance. while previous feature attribution methods such as deeplift @xcite , layerwise relevance propagation @xcite and @xcite have provided interesting frameworks, they break at least one of the two axioms that are fundamental for attribution methods, namely sensitivity and implementation invariance @xcite . in our paper we have chosen the integrated gradients (ig) @xcite attribution method as it satisfies both the aforementioned axioms. ig is a post-hoc interpretability technique which aggregates the gradients of the input by interpolating in small steps along the straight line between a baseline (typically a vector with all zeros) and the input. a large positive or negative ig score indicates that the feature strongly increases or decreases the network output respectively, while a score close to zero indicates that the feature does not influence the network output. this can also be understood as follows: a positive score indicates that the feature tends to agree with the model's prediction, while a negative score indicates that the feature tends to disagree with the model's prediction. lig are computed as the ig between the model output and a particular layer's input or output. our work attempts to answer the following research questions: captum library approximates the above integral using the gauss-legendre approximation algorithm over 50 uniform steps of α ∈ [0, 1]. the baseline was selected as a 768 dimension zero vector. the attribution score for each word is summed across the dimensions (768 in the case of bert-base) and normalized using the euclidean norm of the scores of all the words in the sentence. we analyzed 5 different categories of sentences within the cola targeted test sets: causative-inchoative alternation (cia), reflexive-antecedent agreement (raa), subject-verb agreement (sva), subject-verb-object (svo) and wh-extraction (whe). a few sample sentences across the categories can be seen in the primary focus of our experiments relied on the lig computed between the predicted class logit and the token embedding of the words. further we also computed lig heatmaps with respect to the input (token + segment + position) embedding and across the 12 encoder layer embeddings of bert to analyze the lig characteristics. figure 1 shows the lig heatmaps of the top 10 cpt patterns for the la and lua in the whe category. the unique cpt patterns were extracted for the correctly classified sentences of each category, corresponding to which the lig of each subtree were computed. lig of a subtree is equal to the sum of the lig of the words appearing as leaf nodes in the subtree. the results in ## lig for constituency parse tree patterns cpt is a type of grammar tree which captures the relations between the constituents of a sentence. we believe that analyzing the cpt patterns will give us insights into the grammatical structure of la and lua. computing the lig for cpt patterns at different subtree levels can give us an indication into the constituents which contribute largely towards making the sentence la or lua. for each of the 5 category of sentences, we extracted all the cpt patterns for correctly classified sentences at every subtree level and picked the most frequent patterns at the root level ( ## lig and prediction confidence of the model we investigated to check if there is a relationship between the lig and the prediction confidence of the model. we found that the range of correctly classified sentences having positive lig is between 88% to 100% (cc + % in ## conclusion we have proposed a novel approach for explaining the linguistic acceptability criteria learnt by bert using lig and cpt patterns. as there is a strong relationship between lig and the prediction confidence of the model, our future work will focus on parameterizing the lig in the loss function and observing the model's performance.