BLOCSUM : Block Scope-based Source Code Summarization via Shared Block Representation (2023)

Abstract: Code summarization, which aims to automatically generate natural language descriptions from source code, has become an essential task in software development for better program understanding. Abstract Syntax Tree (AST), which represents the syntax structure of the source code, is helpful when utilized together with the sequence of code tokens to improve the quality of code summaries. Recent works on code summarization attempted to capture the sequential and structural information of the source code, but they considered less the property that source code consists of multiple code blocks. In this paper, we propose BLOCSUM, BLOck scope-based source Code SUMmarization via shared block representation that utilizes block-scope information by representing various structures of the code block. We propose a shared block position embedding to effectively represent the structure of code blocks and merge both code and AST.Furthermore, we develop variant ASTs to learn rich information such as block and global dependencies of the source code. To prove our approach, we perform experiments on two real-world datasets, the Java dataset and the Python dataset. We demonstrate the effectiveness of BLOCSUM through various experiments, including ablation studies and a human evaluation.

## introduction a description of source code is very important in software development because it helps developers better understand programs. advances in deep learning have enabled automatic code summarization and increased software maintenance efficiency. previous approaches on automatic source code summarization can be categorized into sequence-based, structure-based, and hybrid approaches. sequencebased approaches generated summaries by capturing the sequential information of source code @xcite @xcite @xcite @xcite . they tokenized source code into a sequence of code tokens and encoded them using seq2seq models. meanwhile, structurebased approaches used abstract syntax tree (ast) to capture the structural information of code @xcite @xcite @xcite @xcite @xcite @xcite . they parsed the source code into the ast and utilized graph models such as graph neural networks (gnns). some works flattened ast into a pre-order traversal sequence @xcite @xcite @xcite . hybrid approaches utilized both the token sequence and the asts of codes @xcite @xcite . they parallelly processed token sequences and asts with independent encoders and tried to merge them in the decoder. however, the existing approaches have some limitations. sequence-based approaches treated the source code as a single statement, so they incorporated only the sequence information of the code without any structural information such as code blocks. structural information is very important to understand code because a snippet of code can be considered as a hierarchy of blocks. structure-based approaches tried to catch such structural information of code, but they considered less sequential information of code. a snippet of code is a sequence, so sequential information is also important to understand code. another problem is that they depended only asts to capture structural information of code. however, asts are not suitable for capturing structural information because they are syntax trees for grammatical purpose. since the ast is a tree, there is only one path between every pair of nodes. any two nodes in asts are connected, but with a relatively long path, which hinders capturing the structural rela- hybrid approaches utilized both code and its ast. since both sequential and structural information of code is necessary to understand code, the approaches showed higher performances than the previous ones. however, they failed to effectively merge two different types of information. they simply adopted independent encoders for each of them and tried to merge them in the decoder. due to independent encoders, their representations are easy to be independent. it will make it hard to effectively merge both sequential information of code and structural information. since the token sequence and the ast of code are just different descriptions of the same code, they need to be encoded so that they are correlated with each other. to address these limitations mentioned above, we exploit the fact that the source code is a set of blocks consisting of multiple statements for a specific purpose. the code tokens in one code block are configured for the same purpose. as shown in figure 1a , the code block if { ... } (orange) consists of statements that are executed when a certain condition is true. so, it needs to consider the information on which block each code token belongs to. to better capture structural information, we need to give each token not only positional information but also block positional information when encoding. since asts do not have enough information to capture the structural information of code, we need to modify asts. the additional information we try to add is block dependency and global dependency between nodes. for example, as shown in figure 1b , the node "max" in the orange block of ast is only connected to its parent node, "statement", but there exists implicit block dependency that the node "max" belongs to the same orange block as the node "data" in the purple dashed line. furthermore, there exists implicit global dependency between nodes. for example, the node "max" in the orange box is the same variable as the nodes in the green dashed line. we need to add such information to asts. also, we need to match blocks in the token sequence with blocks in the ast. for example, code tokens ("if", "(", "data", ..., "}") (orange) in figure 1a can be mapped to its corresponding nodes in the ast ("if", "data", ... , "println") (orange) in figure 1b . here, the block in the code and the block in the ast are the same part with an equal role, so we will utilize information on which block of code corresponds to which block of ast. such information can make the code and ast correlated, and assist effective merging of two kinds of information. in this paper, we propose blocsum, block scope-based source code summarization via shared block representation that utilizes blockscope information of token sequences and asts. first, we propose the shared block position em-shared 𝑃 𝑐 (𝑡) 𝑃 𝑠 (𝑛) 𝑊 𝑐 (𝑡) 𝑊 𝑠 (𝑛) 𝑡 1 𝑡 2 𝑡 3 𝑡 4 … 𝑡 15 𝑡 16 𝑡 17 1 2 3 4 … 15 16 17 1 1 1 2 … 6 7 7 𝑛 1 𝑛 2 𝑛 3 𝑛 4 … 𝑛 20 𝑛 21 𝑛 22 1 2 3 4 … 20 21 22 1 2 2 3 … 6 2 1 𝐸 𝑐 (𝑡) 𝐸 𝑠 (𝑛) ast source code layer norm feed-forward layer norm multi-head self-attention masked multi-head self-attention beddings for effectively representing the structure of the code block and combining a correlation between the code and the ast encoders. furthermore, we develop simple yet effective variants of asts to learn rich information such as block and global dependencies of the source code. to validate our approach, we perform experiments on the java dataset and the python dataset. we prove the superiority of blocsum through various experiments including ablation studies and a human evaluation. ## blocsum in this section, we present the details of our model. figure 2 shows the overall architecture of bloc-sum. we first introduce the shared block position embedding and the abstract syntax tree variants and explain the architecture of blocsum in detail. ## conclusion in this paper, we proposed blocsum, block scope-based source code summarization via shared block representation that utilizes blockscope information by representing various structures of the code block. we designed two methods using the fact that a code block is a fundamental structural component of the source code. we propose the first method, the shared block position embedding, for effectively representing the structure of the code block and merging a correlation between the code and the ast encoders. furthermore, we developed to reconstruct simple yet effective ast variants to learn rich information such as block and global dependencies of the source code. experimental results demonstrated the effectiveness of blocsum and confirmed the importance of block-scope information in the code.