CAT -probing: A Metric-based Approach to Interpret How Pre-trained Models for Programming Language Attend Code Structure (2022)

Abstract: Code pre-trained models (CodePTMs) have recently demonstrated significant success in code intelligence. To interpret these models, some probing methods have been applied. However, these methods fail to consider the inherent characteristics of codes. In this paper, to address the problem, we propose a novel probing method CAT-probing to quantitatively interpret how CodePTMs attend code structure. We first denoise the input code sequences based on the token types pre-defined by the compilers to filter those tokens whose attention scores are too small. After that, we define a new metric CAT-score to measure the commonality between the token-level attention scores generated in CodePTMs and the pair-wise distances between corresponding AST nodes. The higher the CAT-score, the stronger the ability of CodePTMs to capture code structure. We conduct extensive experiments to integrate CAT-probing with representative CodePTMs for different programming languages. Experimental results show the effectiveness of CAT-probing in CodePTM interpretation. Our codes and data are publicly available at https://github.com/nchen909/CodeAttention.

## introduction in the era of "big code" @xcite , the programming platforms, such as github and stack overflow, have generated massive opensource code data. with the assumption of "software naturalness" @xcite , pre-trained models @xcite @xcite have been applied in the domain of code intelligence. existing code pre-trained models (codeptms) can be mainly divided into two categories: structure-free methods @xcite and structure-based methods @xcite . the former only utilizes the information from raw code texts, while the latter employs code structures, such as data flow @xcite and flattened ast 1 (guo et al., 2022), to enhance the performance of pre-trained models. for more details, readers can refer to @xcite . recently, there exist works that use probing techniques @xcite @xcite to investigate what codeptms learn. for example, @xcite first probe into codeptms and construct four probing tasks to explain them. @xcite also define a series of novel diagnosing probing tasks about code syntactic structure. further, @xcite conduct qualitative structural analyses to evaluate how codeptms interpret code structure. despite the success, all these methods lack quantitative characterization on the degree of how well codeptms learn from code structure. therefore, a research question arises: can we develop a new probing way to evaluate how codeptms attend code structure quantitatively? in this paper, we propose a metric-based probing method, namely, cat-probing, to quantitatively evaluate how codeptms attention scores relate to distances between ast nodes. first, to denoise the input code sequence in the original attention scores matrix, we classify the rows/cols by token types that are pre-defined by compilers, and then retain tokens whose types have the highest proportion scores to derive a filtered attention matrix (see figure 1(b) ). meanwhile, inspired by the works @xcite , we add edges to improve the connectivity of ast and calculate the distances between nodes corresponding to the selected tokens, which generates a distance matrix as shown in figure 1(c ). after that, we define cat-score to measure the matching degree between the filtered figure 1 : visualization on the u-ast structure, the attention matrix generated in the last layer of codebert @xcite and the distance matrix. (a) a python code snippet with its corresponding u-ast. (b) heatmaps of the averaged attention weights after attention matrix filtering. (c) heatmaps of the pair-wise token distance in u-ast. in the heatmaps, the darker the color, the more salient the attention score, or the closer the nodes. in this toy example, only the token "." between "tmpbuf" and "append" is filtered. more visualization examples of filtering are given in appendix d. our main contributions can be summarized as follows: ## cat-probing 3.1 code matrices filtering as pointed out in @xcite , the attention scores in the attention matrix follow a long tail distribution, which means that the majority of attention scores are very small. to address the problem, we propose a simple but effective algorithm based on code token types to remove the small values in the attention matrix. for space limitation, we summarize the pseudocodes of the algorithm in appedix alg.1. we only keep the rows/cols corresponding to frequent token types in the original attention matrix and distance matrix to generate selected attention matrix and distance matrix. ## conclusion in this paper, we proposed a novel probing method named cat-probing to explain how codeptms attend code structure. we first denoised the input code sequences based on the token types predefined by the compilers to filter those tokens whose attention scores are too small. after that, we defined a new metric cat-score to measure the commonality between the token-level attention scores generated in codeptms and the pairwise distances between corresponding ast nodes. experiments on multiple programming languages demonstrated the effectiveness of our method.