Code summarization (CS) is becoming a promising area in recent natural language understanding, which aims to generate sensible annotations automatically for source code and is known as programmer oriented. Previous works attempt to apply structure-based traversal (SBT) or non-sequential models like Tree-LSTM and GNN to learn structural program semantics. They both meet the following drawbacks: 1) it is shown ineffective to incorporate SBT into Transformer; 2) it is limited to capture global information through GNN ; 3) it is underestimated to capture structural semantics only using Transformer. In this paper , we propose a novel model based on structure-induced self-attention , which encodes sequential inputs with highly-effective structure modeling . Extensive experiments show that our newly-proposed model achieves new state-of-the-art results on popular benchmarks. To our best knowledge, it is the first work on code summarization that uses Transformer to model structural information with high efficiency and no extra parameters. We also provide a tutorial on how we pre-process .
Code summarization (CS) is becoming a promising area in recent language understanding, which aims to generate sensible human language automatically for programming language in the format of source code, serving in the most convenience of programmer developing. It is well known that programming languages are highly structured. Thus previous works attempt to apply structure-based traversal (SBT) or non-sequential models like Tree-LSTM and graph neural network (GNN) to learn structural program semantics. However, it is surprising that incorporating SBT into advanced encoder like Transformer instead of LSTM has been shown no performance gain, which lets GNN become the only rest means modeling such necessary structural clue in source code. To release such inconvenience , we propose structure-induced Transformer , which encodes sequential code inputs with multi-view structural clues in terms of a newly-proposed structure-induced self-attention mechanism . Extensive experiments show that our proposed structure-induced Transformer helps achieve new state-of-the-art results on benchmarks .
D
natural
R
annotations automatically for source code and is known as programmer oriented. Previous
human language automatically for programming language in the format of source code, serving in the most convenience of programmer developing. It is well known that programming languages are highly structured. Thus previous
R
GNN
graph neural network (GNN)
R
They both meet the following drawbacks: 1) it is shown ineffective to incorporate SBT into Transformer; 2) it is limited to capture global information through GNN ; 3) it is underestimated to capture structural semantics only using Transformer. In this paper
However, it is surprising that incorporating SBT into advanced encoder like Transformer instead of LSTM has been shown no performance gain, which lets GNN become the only rest means modeling such necessary structural clue in source code. To release such inconvenience
D
a novel model based on
R
self-attention
Transformer
R
inputs with highly-effective structure modeling
code inputs with multi-view structural clues in terms of a newly-proposed structure-induced self-attention mechanism
R
newly-proposed model achieves
proposed structure-induced Transformer helps achieve
R
popular benchmarks. To our best knowledge, it is the first work on code summarization that uses Transformer to model structural information with high efficiency and no extra parameters. We also provide a tutorial on how we pre-process
benchmarks