DEIM Forum 2023 1b-9-4

Utilizing Abstract Syntax Tree Embedding to Improve the Quality of
GNN-based Class Name Estimation
Hiroto MAMBA† , Yasuhiro HAYASE† , and Toshiyuki AMAGASA†† University of Tsukuba
1–1–1 Tennodai, Tsukuba, Ibaraki 305–8577, Japan
E-mail: †mamba@kde.cs.tsukuba.ac.jp, ††{hayase,amagasa}@cs.tsukuba.ac.jp
Abstract While giving comprehensible names to identiﬁers is essential in software development, it is sometimes difﬁcult
since it requires development experience and knowledge of the application domain. Among work to support the developer’s
identiﬁer naming, a GNN-based class name estimation approach learns a graph of relationships between program elements,
i.e., classes, methods, and ﬁelds, but it ignores information within the methods. This study proposes an approach that exploits
information from method bodies, which can help estimate correct class names. The proposed approach extends the existing
GNN-based approach to use embeddings of the corresponding ASTs for method nodes. An evaluation experiment measures
how correctly the proposed approach can estimate class names in large datasets of open-source Java projects. The experimental
result shows that the proposed approach improves the estimation correctness compared to the existing approach.
Key words software engineering, graph neural network, abstract syntax tree, class name estimation

1 Introduction

ing relationships and the information within method nodes. Sincemachine-learning work for software programs often utilizes AST

In software development, giving comprehensible names to iden-[7–9], it may be helpful to represent the nodes’ properties. While

tiﬁers is essential for developers’ program understanding. When

this approach is based on the existing GNN-based approach [10],developers understand programs written before, they often try to

it extends the existing approach to utilize embeddings of methods’

obtain application domain knowledge from the identiﬁer names of

ASTs.

program elements such as variables, methods, and classes. [1] Es-

An experiment investigates the contribution of the proposed ap-

pecially when a program has complex source codes or whose doc-

proach by measuring the estimation correctness using large datasets

umentation is outdated or not helpful, the names are importantof open-source Java projects. The experimental result that the pro-sources of knowledge. [2] If an identiﬁer has a meaningless nameposed approach can estimate the class names more correctly com-or one identical name is used for different concepts, those names

pared to the existing one shows that the AST helps to represent thecause difﬁculties in program understanding. [3] Since a study indi-

information within the method nodes.

cates that developers spend 60% of their working time on program

The remaining part of this paper consists as follows. Section

understanding [4], the names should be easy to understand the func-

2 describes prior work in machine learning for program code andtions and roles.

class name estimation. Section 3 describes preliminary used as

However, sometimes it is difﬁcult to give a good name to a pro-components of the proposed method. Section 4 describes the pro-gram element. Good naming requires knowledge about the ele-

posed method. Section 5 describes the experiments for the proposed

ment’s role in the project [3] or what names appear elsewhere [5].

method. Finally, Section 6 concludes this paper.

Moreover, a name may become inappropriate as the system evolves.
[3]

2 Related work

Among work to support the developer’s identiﬁer naming, an ap-

This section introduces prior work in machine learning for pro-proach [6] for class name estimation learns the relationship between

gram code. Section 2. 1 describes the general approaches for var-

program elements. They modeled these relationships into a graphious tasks; then, Section 2. 2 describes the existing approaches towith nodes representing classes, ﬁelds, and methods. Although thislearning class entities.

graph can exploit complex classes’ nature, it ignores most of the

2. 1 Machine learning approaches for software programs

information that each node itself has, which may be helpful to the

Recently, various studies have applied machine learning to source

estimation.
This study proposes an approach to estimate class names us-code for tasks such as method naming [7–9, 11–14], code summarization [15–19], method name consistency checking [20], type in-ference [21], and code translation [22, 23].

Source CodeApproaches that follow architectures for neural machine transla-

int distance(int a, int b) {return Math.abs(a - b);}tion [24, 25] learn source code as a sequence of tokens to generate
method name [14] or method summary [15].
On the other hand, various approaches exploit speciﬁc features

AST (Abstract Syntax Tree)of source code, such as abstract syntax trees (AST) or method call

MethodDeclaration

dependencies. Alon et al. [7, 11, 26] and Peng et al. [27] treat an
AST as a set of paths between two leaf nodes. Zhang et al. [28]SimpleName

Parameter

Parameter

PrimitiveType

PrimitiveType

SimpleName

PrimitiveType

SimpleName

int

a

int

b

BlockStmt

and Lin et al. [29] treat it as a set of subtrees. Hu et al. [16, 17]distance

int

ReturnStmt

treats it as one sequence by structure-based traversal. GNN-based
approaches [8, 9, 30] treat an AST as a graph with additional edges
and learn it with graph neural networks. Besides, some approaches

MethodCallExpr

NameExpr

SimpleName

BinaryExpr

SimpleName

abs

NameExpr

NameExpr

SimpleName

SimpleNameablearn dependencies between elements that appear in source code,such as methods’ call dependencies [5,31,32] or data ﬂow [9,23,33].
2. 2 Approaches for class entities

Math

As for class name estimation, Kurimoto et al. [6] proposed an
approach that learns the relationship between program elements toestimate class names by graph embedding. This relationship is rep-

AST path between token "abs" and token "b"resented as a graph with nodes of classes, methods, and ﬁelds. Thisapproach can estimate a class name even before the class is used

Figure 1 An example of an abstract syntax tree (AST) and an AST path.

from other code, in contrast to another approach [34] that learns thea sequence of nodes on the AST path, and si1 si2 · · · andcontext where the class is used.
Subsequently, a work [10] proposed an approach that learns theei1 ei2 · · · indicate words that make up tokens corresponding tosame graph using GNN. In contrast to Kurimoto et al., where mem-two leaf nodes, respectively, called subtokens.

ory usage depends on the graph size, this approach can learn frommodel calculates a feature vector zi for AST-Pathi using

The code2seq

a large dataset because it calculates embedding representations forembeddings for the subtokens Esubtoken (si1 ), Esubtoken (si2 ), · · · ,each word rather than the whole graph node.

Esubtoken (ei1 ), Esubtoken (ei2 ), · · · and embeddings for the nodes

As for utilizing ASTs, Compton et al. [35] obtained an embed-

Enode (vi1 ), Enode (vi2 ), · · · . Here, Esubtoken (·) and Enode (·) are op-

ding representation of the whole class. This approach ﬁrst calcu-

erations that look up a subtoken embedding vector and a node em-

lates embedding for each method in a Java ﬁle using code2vec [11],bedding vector, respectively. Then, the model averages the featureand then calculates entire class embedding by applying an aggrega-

vectors of all AST paths, which is used for the subsequent genera-tion function, such as summation and median. The author uses thistion of a method name. The explanation in Section 4. 1. 2 denotes

obtained class embedding for code classiﬁcation tasks.

the operation up to this point as code2seq encoder(AST), that is,
the operation from inputting an AST of a method to calculating this

3 Preliminary

averaged vector.

This section describes methods used in the proposed method de-

The code2seq has an advantage in that it can easily consider code

scribed in Section 4.

snippets with different syntactic structures but similar semantics as

3. 1 code2seq

similar ones. This is because the code2seq holds embedding vectors

code2seq [7] is an approach for method name estimation, which

for each node in an AST path and calculates the feature vector of the

exploits the abstract syntax tree (AST) of methods. This estima-

AST path from them, in contrast to having embedding vectors fortion model takes a source code of method deﬁnitions as input andonly the entire AST path [11].
3. 2 Graph neural network

outputs candidates of methods’ names.
The code2seq treats an input method as a set of paths called AST

Graph neural networks (GNN) are neural networks that takepath at the beginning of the estimation. AST paths are consideredgraph structures as input and learn them. As a one of GNN, graphbetween any two leaf nodes in the AST, and each AST path pro-convolutional network (GCN) [36] convolutes features of neighbor-vides information about a sequence of nodes on the path and tokens

ing nodes of each node and generates new features. By repeatingcorresponding to two leaf nodes (illustrated in Figure 1).

the convolution multiple times, it is possible to convolute features

The code2seq calculates feature vectors for AST paths
as follows.

Suppose that an AST path is represented as

AST-Pathi = ⟨si1 si2 · · · , vi1 vi2 · · · , ei1 ei2 · · · ⟩.

Here, vi1 vi2 · · · isof nodes that are more than one step away from each other. When()(0) (0)(0)
H (0) = h1 , h2 , · · · , hN and A ∈ RN×N represent initial nodefeatures and the adjacency matrix of a graph with N nodes, respec-the GCN (described in Section 3. 2) ﬁrst receives the initial feature

tively, this convolution is formulated as follows:

(0)

( 1
)
1
H (l+1) = σ D̃− 2 ÃD̃− 2 H (l)W (l)

(0)

(0)

h1 , h2 , · · · , h|V | of each node v1 , v2 , · · · , v|V | , and then outputs(l)where IN ∈ RN×N is an identity matrix, Ã = A + IN , D̃ii = ∑ Ãi j ,j

and W (l) is a weight matrix for l th convolution step.
3. 3 GNN-based class name estimation
A work [10] has proposed an approach to estimate class names,
which learns relationships between program elements by GNN. Figure 2 illustrates this approach. This approach takes a class’s sourcecode as input from a developer, then estimates candidates for the

(l)

(l)

the new feature h1 , h2 , · · · , h|V | after l times convolutions. Then
LSTM [37] receives the feature ht′ of the target class node vt andoutputs sequentially the probabilities of the next words of the target
class name.
d ) Training of the neural network
In the training phase, the model learns to minimize the crossentropy loss between the output probabilities and the ground-truthclass name for each word. Adam [38] optimizer is used to update
the model parameters.

name of the class. This estimation model ﬁrst constructs a graphrepresenting the program elements’ relationships. Then a GNN calculates the feature vector for each node in the graph, considering therelationships between nodes. Finally, the model generates a namefrom the calculated feature vector.
In the following explanation, a class whose name is to be estimated is called target class.
a ) Relation graphThis approach represents the relationships between program elements such as classes, methods, and ﬁelds as a graph used by Kurimoto et al. [6] (later referred to as a relation graph). This graphcontains edges for the following types of relationships between the
program elements:

4 Proposed method
This section describes an approach to estimate class names witha graph neural network using the ASTs of methods. This approachtakes the source code of a class as input from a developer and thenestimates candidates for the class name. The key idea of this approach is to learn information within methods along with relationships between program elements. While the relation graph canmodel the class’s complex nature, the graph ignores the internal
properties in each node. On the other hand, methods’ ASTs haverich structural information to represent the properties of the methods. By combining the relation graph and the method AST, this approach can estimate the class name while considering each node’s•

Class inheritance, where one class extends another class.

•

Ownership from a class to a method or a ﬁeld.

•

A relationship where one method calls another.

•

An access relationship from a method to a ﬁeld.

•

A relationship from a method to the class of its return type.

•A relationship from a ﬁeld to the class of the ﬁeld’s type.

When a developer inputs a code of a target class, the estimation
model ﬁrst constructs this relation graph. To include graph nodes
for program elements outside the code of the target class, the modeltakes not only the code of the target class but also codes referenced
by the target class.

internal properties and its neighboring nodes’ properties.
This approach is based on the existing GNN-based estimation approach [10] and extends it to include information from the AST ofeach method. This approach gives GNN an embedding containing
the AST information as the initial node feature. This embedding iscomputed using the vector generated in the middle of code2seq [7].
Given the initial feature with AST information, GNN can obtaineach node’s new feature, capturing the neighborhood method’s syntactic structure. Figure 3 illustrates this proposed method.
As another way to integrate AST information into the existing
GNN-based approach, there may be an approach to treat the AST as

In the following explanation, let G = (V, E) be the constructed
relation graph, where V = {v1 , v2 , · · · } is a set of nodes corresponding to the program elements, and E = {e1 , e2 , · · · } is a set of edges
corresponding to the relations. vt ∈ V is the node corresponding to
the target class. Besides, denote the number of nodes in G by |V |.
b ) Obtaining initial node features
Before applying GNN, the model assigns a vector to each node,called initial node feature. For each node vi , the model calculates(0)

an initial feature hi

based on the word sequence of the class nameof vi . The initial features make it possible for GNN to distinguishnodes.
c ) Generation of the target class name from initial node features
Given the initial node features of each graph node, GNN obtains
each node’s new feature, and then an RNN-based generator outputs
candidates for the target class name as word sequences. Formally,a subgraph and add an edge between the root node of the AST andthe relation graph. However, since the existing approach estimatesthe class name by considering the information of the neighboring
nodes of a target class, this alternative approach can only use theinformation of the part near the root node of the AST and ignores
most of its information.
The rest of this section describes embeddings of program element
information used as initial node features in GNN.
4. 1 Embedding of program elements’ information
This section describes embedding generations of node information, especially ASTs for methods’ nodes, to be used as initial nodefeatures.
Depending on the program element type, this estimation model
embeds each graph node in two ways: name embedding and AST

C

Name

C

"String"
returns

Source Code of Target Class
class ?
extends BufferedReader {int lineNumber;
String readLine() { ... }
...
}

F

Name

.java

M

owns

Target Class C

C

Name

Construct
Graph
.java

F

Name
"readLine"

owns

.java

M

accesses"lineNumber"

Obtain Initial
Node Features

?

.java

GNN

Generate
Names

1. LineNumberReader 80%
2. LineReader 10%
3. ...
4. ...

extends

C

C

Name

Initial NodeFeature"BufferedReader"C= Class

F = Field

M = Method

Figure 2 An illustration of the existing approach. [10]
C

Name

C

"String"000returns

Source Code of Target Class
class ?
extends BufferedReader {int lineNumber;
String readLine() { ... }
...
}

Name

F

.java

Target Class C

owns

Name
?

.java

M

0
0
0

AST

Construct
Graph
.java

F

Name
"readLine"

owns

.java

M

accesses"lineNumber"C

Embed Node
Information

000extendsC

Name"BufferedReader"

Name Embedding
AST Embedding(by code2seq encoder)C

GNN

Generate
Names

1. LineNumberReader 80%
2. LineReader 10%
3. ...
4. ...

Name Embedding000

Zero Vector

Figure 3 An illustration of the proposed method.

embedding. This approach newly employs the AST embedding in

4. 1. 1 Name embedding

addition to the name embedding, while the name embedding is the

The model computes the name embedding by feeding the word

only feature in the existing approach [10].

sequence in an element name to bi-directional LSTM. The input el-

While the name embedding is calculated based on the name ofement name is split into words by camelCase or snake case: e.g.,a program element for all graph nodes, the AST embedding is cal-a name LineNumberReader is split into three words, line, number,

culated for nodes corresponding to a method with a body. In otherand reader; a name MAX EXPONENT is split into two words max

words, a class’s or ﬁeld’s embedding is computed only from theand exponent. Then the model sequentially feeds trainable embed-

class name or ﬁeld name. And the embedding of a method node isdings for each word. After feeding, the ﬁnal hidden state of the bi-

calculated from both the method name and its AST if it is not an

directional LSTM represents the name embedding of the element.

abstract method; otherwise, it is calculated only from the method

Formally, the name embedding hnameof an element vi with a name
i

name.

wi1 , wi2 , · · · , wil is calculated as follows:

Name embedding and AST embedding are calculated as described in Section 4. 1. 1 and 4. 1. 2, respectively. In the following
explanation, let hname, hAST
denote name embedding and AST emi
ibedding of a node vi , respectively.
After obtaining the name embedding and the AST embedding,two embeddings are concatenated. That is, the embedding of a node()(0)
vi is hi = concat hname, hAST
i
i

name
name
hname
= LSTM (E(wi1 ), E(wi2 ), · · · , E(wil ))
i1 , hi2 , · · · , hil
(→
)
−
←
−
hname
= concat h name, h nameii1
ilwhere E(·) represents an operation to look up the embedding vectorof a word, and LSTM (· · · ) represents an operation to feed vectors
to the LSTM sequentially.
As for the target class, the model calculates the name embedding
similarly for other elements mentioned above but treats its name

as a dummy since the name of the target class is to be estimatedshould then output the original name of the target class. The fol-itself. Here, the dummy name is a sequence of masking tokens:lowing metrics, between the estimated name and the original class⟨SLOT ⟩ , · · · , ⟨SLOT ⟩.

name, indicate the quality of the estimation: precision, recall, f1,

4. 1. 2 AST embedding

partial(k), and exact(k). These metrics are not case-sensitive.

The code2seq [7] encoder mentioned in Section 3. 1 calculates
the AST embedding of a method if the method has a body.

Precision, recall, and f1 values are calculated by counting true
positives, false positives, and false negatives between the estimatedEach subtoken embedding used in the code2seq encoder sharesname at the ﬁrst top of the candidates and the original. For exam-

weights the same as the word embedding used in the name embed-

ple, suppose two estimated names are Reader and LinkedArrayList,

ding. This is because the subtoken embedding and the word embed-

while two original names are LineNumberReader and LinkedList,

ding are both for tokens that appear in the code.

respectively. Then the true positives are 3 (Reader, Linked, List),

Besides, zero vectors act as substitutes for the AST embedding

the false positives are 1 (Array), and the false negatives are 2 (Line,for elements without a method body. The dimension of this zero

Number). Therefore, precision is 3/4 = 0.75, recall is 3/5 = 0.6,

vector is the same as the AST embedding for the methods with aand f1 is about 0.67.

body.

partial(k) and exact(k) indicate the proportion of the estimated

Below shows the AST embedding hASTdepending on whetherithe element corresponding to a node vi has a method body or not:
{
hAST
=
i

names that are partially or exactly the same as the original name in
the top k candidates. The k values used in this experiment are 1 and10. For partial(k), when an estimated name contains any word ofcode2seq encoder(ASTi )if vi has method body0otherwise

where ASTi represents the AST of the method corresponding tonode vi .

the original name, it is counted as a partial match.
5. 1. 1 Metrics for the estimation of trivial/non-trivial words
To investigate whether there is any difference in the correlation
between the existing and proposed approaches in terms of the easea ) Utilizing weights pre-trained in code2seq encoderof estimating trivial and non-trivial words, this experiment also

Since the code2seq encoder is a part of a model designed for ameasures the metrics using only the following words:

different task, method name recommendation, utilizing pre-trained
weights in the original task may boost the learning compared
to training from scratch. If utilizing the pre-trained pre-trained•

Only the most frequent words in the class names of eachdataset, and conversely, only words other than those.
•

Only the words other than those that make up the name of theweights, the model applies the weights at the beginning of the train-

class that the target class extends, if the target class extends anothering process, and then the weights are ﬁne-tuned during the training.

class.

5 Evaluation

This experiment considers two types of words that occur mostfrequently: a word in the top 1 and words in the top 10. The rea-This section describes an experiment to evaluate the proposedson for considering these two types is that the term test occurs most

method. The main goal of this experiment is to assess whether

frequently in the two datasets used in this experiment (described

the AST embedding of the proposed method improves the cor-in Section 5. 2) and occurs signiﬁcantly more frequently than the

rectness of the class name estimation compared to the existingwords in the top 2 and below. Considering the two types of the mostapproach. This correctness is measured and compared between

frequent words, the metrics are calculated by considering only thethe existing and proposed approaches using several metrics (de-words in the top 1, only the words in the top 10, only the words in

scribed in Section 5. 1) when estimating the class names of open-

the top 2 or below (i.e., words other than test), and only the words

source Java projects. Each approach’s estimation model is trainedin the top 11 or below.

and hyperparameter-optimized (described in Section 5. 3) on the

When the metrics are calculated using only a portion of the total

datasets described in Section 5. 2, and then its metrics are computed.

words, the metrics are calculated as follows. For example, suppose

In addition, this experiment compares the metrics between thethat when the metrics are calculated using only words other thanmodel with the pre-trained weights in the code2seq encoder, which

the term test, an estimated name is PyIndexingTest, while an origi-

is a component of the proposed approach, and the model without

nal name is PyEmacsTabTest. In this case, the true positives are 1

the weights to show whether these weights affect the estimation. In(Py), the false positives are 1 (Indexing), and the false negatives are

the following descriptions, the model with the pre-trained weights

2 (Emacs and Tab). Therefore the precision is 0.5, the recall is about

is referred to as Proposed (PT), and the model without the weights

0.33, and the f1 is 0.4.

is referred to as Proposed (LFS).
5. 1 Metrics

5. 2 Datasets
As real software project codes, this experiment uses two widely-In this experiment, the estimation model takes as input a set ofused [20] existing source code corpus, both collected by Alon etsource codes where the name of the target class is hidden, andal. [7], called java-med and java-large. java-med and java-large

Table 1 Hyperparameters after optimization for each estimation model.
Exising [10] Proposed (LFS) Proposed (PT)
Embedding dim 118

505

512

Hidden dim

716

1018

1020

Batch size

96

24

84

Fanout15

11

9

Learning rate

2.8 × 10−5

1.81 × 10−5

7.24 × 10−5

As shown in Table 2, the proposed approach outperforms the existing approach in almost all metrics, except for the exact(k = 1) andexact(k = 10) of the Proposed (PT). This indicates that the AST embedding of the proposed approach is helpful for correctly estimatingclass names.
Although the Proposed (PT) can use the pre-trained weights fromthe beginning of the training, its precision falls below that of theare a set of top-starred projects on GitHub, containing 1,000 and
9,500 projects, respectively. Names in these datasets seem to havea certain level of quality since the top-starred projects have manyusers and contributors. The dataset is repartitioned into Train : Val1 :Val2 : Test = 7 : 1 : 1 : 1, independent of the partitioning by Alon etal.
5. 3 Hyperparameter
The hyperparameters are set as shown in Table 1 by optimization
with Optuna [39]. Throughout the optimization, the hyperparameters are optimized to maximize the f1 on the Val1. After the optimization, the estimation model is re-trained from scratch using theVal2 for validation. The following describes each hyperparameter,
except for the learning rate.
Embedding dim. The dimension of the two types of embeddings:
one where the name embedding and the subtoken embedding shareweights; the other to embed nodes in AST.
Hidden dim. The dimension of the components in the neural network other than the embedding above, e.g., the dimension of thehidden state in LSTM or GCN.
Batch size. The number of target classes to learn in a single iteration.
Fanout. The number of neighbors to gather information from in
GCN.
5. 4 Environments
This experiment employs uses a Tesla V100-PCIE-32GB as theGPU. The neural network implementation uses Pytorch [40] andthe Deep Graph Library [41] for GNN. An existing code2seq implementation（*1） is utilized for AST encoder and pre-training. The
extraction of element relations uses JavaParser（*2）. GCN in the proposed and existing approach has two layers.
5. 5 Experimental result
This section describes the experimental results using the above
experimental settings. Table 2 shows the results of the metrics measured on all words for each combination of model and dataset. In
addition, Table 3 and Table 4 show the metrics measured on only asubset of all words, as described in Section 5. 1. 1. Table 3 shows
the metrics measured only on the most frequent words or only onwords other than such words. Furthermore, Table 4 shows the metrics measured without the words that make up the name of the classthat the target class extends (if any).

Proposed (LFS). This decrease indicates that the Proposed (PT) is
more likely to estimate redundant words than the Proposed (LFS).
This may be due to the difference in the target task between the
proposed approach and the pre-training, i.e., the proposed approachtargets class name estimation, while the code2seq encoder, whose
weights are pre-trained, is designed for method name estimation.
This difference may cause the pre-trained weights to interfere withthe class name estimation.
The metrics measured only on trivial words and those measured
only on non-trivial words show different trends, as shown in Table
3 and Table 4. When measured only on a top 1 word or top 2-10
words, the precision scores of the Proposed (PT) on the java-med
dataset and the Proposed (LFS) on the java-large dataset are lower
than those of the existing approach. This indicates that there arecases where the proposed approach is likely to estimate trivial words
redundantly. On the other hand, the metrics measured without suchtrivial words have the same tendency as those in Table 2. In otherwords, all the values of precision, recall, and f1 of the proposed approach are better than those of the existing approach. Therefore, theproposed approach can estimate a wider variety of words, incredibly
non-trivial words, rather than trivial words compared to the existing
approach.

6 Conclusion
This paper proposes an approach that aims to exploit the information within method nodes in GNN-based class name estimation.
The key idea of this approach is to learn the information within
methods along with the relationships between program elements.
The proposed approach extends an existing GNN-based approach
to utilize the AST embedding generated in a method name estimation approach based on AST.
An experiment measures the estimation correctness using
datasets from open-source projects, which aims to investigate theeffectiveness of the AST embedding of the proposed approach. An
experimental result shows that the proposed approach can estimateclass names better than the existing one.
Future work includes the following. The ﬁrst is to investigate
other ways to embed node information well. While this work uses
AST to represent node information, machine-learning work often
uses other representations, such as dataﬂow. Depending on how theestimation model represents node information, it is valuable to in-（*1）：https://github.com/m3yrin/code2seq

vestigate how the estimation performs. The second is to evaluate（*2）：https://javaparser.org/

the contributions of the components in the approach on which thisTable 2 Metrics for each estimation model, measured on all words. Bold values represent the results
scores on the java-med and java-large datasets, respectively.
partial(k)exact(k)k=1

k = 10k = 10precision recall

java-med

0.22670.1643 0.1905 0.4429 0.5805 0.0076 0.0239

Proposed (LFS) java-med

0.27690.1968 0.2301 0.4951 0.6591 0.0092 0.0341

Proposed (PT)java-med

0.2407

0.1975 0.2170 0.5016 0.5991 0.0065 0.0184

Existing [10]java-large 0.2652

0.2197 0.2403 0.5228 0.6590 0.0085 0.0364

Proposed (LFS) java-large 0.3051

0.2336 0.2646 0.5368 0.6965 0.0149 0.0480

Existing [10]

f1

k=1

datasetTable 3 Metrics measured on only a subset of words based on their frequency.
partial(k)Metrics are measured on
top 2- words only

top 1 word only

top 11- words only

top 2-10 words only

Existing [10]datasetprecision recall

k=1

k = 10java-med

0.1843

Proposed (LFS) java-med

0.23590.1613 0.1916 0.4062 0.5895

Proposed (PT)java-med

0.21010.1608 0.1822 0.4086 0.5200

Existing [10]0.1278 0.1509 0.3393 0.4892

java-large 0.2317

0.1880 0.2076 0.4440 0.5917

Proposed (LFS) java-large 0.2900

0.1993 0.2362 0.4607 0.6451

Existing [10]0.8863 0.7630 0.8948 0.9407

java-med

0.6699Proposed (LFS) java-med

0.71630.8857 0.7920 0.8947 0.9427

Proposed (PT)java-med

0.46980.8974 0.6167 0.9045 0.9462

Existing [10]java-large 0.7388

0.8326 0.7829 0.8376 0.9154

Proposed (LFS) java-large 0.3912

0.8838 0.5424 0.8885 0.9664

Existing [10]0.1162 0.1390 0.3009 0.4451

java-med

0.1729Proposed (LFS) java-med

0.22390.1499 0.1796 0.3671 0.5465

Proposed (PT)java-med

0.20260.1481 0.1711 0.3677 0.4731

Existing [10]java-large 0.2197

0.1756 0.1952 0.4040 0.5499

Proposed (LFS) java-large 0.2885

0.1870 0.2269 0.4201 0.6036

Existing [10]java-med

0.32180.3399 0.3306 0.3608 0.4954

Proposed (LFS) java-med

0.38640.3547 0.3698 0.3746 0.5745

Proposed (PT)java-med

0.27830.3777 0.3204 0.3974 0.5064

Existing [10]java-large 0.3703

0.3877 0.3788 0.4098 0.5962

Proposed (LFS) java-large 0.2992

0.4061 0.3446 0.4312 0.6709

Table 4 Metrics measured excluding the words of the class from which
each class extends.
Existing [10]f1

JPNP20006, commissioned by the New Energy and Industrial Technology Development Organization (NEDO).

datasetprecision recall

java-med

0.1567f1

0.1193 0.1355

Proposed (LFS) java-med

0.20120.1536 0.1742

Proposed (PT)java-med

0.17340.1549 0.1636

Existing [10]java-large 0.1930

0.1616 0.1759

Proposed (LFS) java-large 0.2316

0.1872 0.2070

work is based. For example, evaluating how the estimation performs
when some graph edges are removed is interesting. The third is touse some mechanism to account for the importance of graph nodes
or edges. Although the graph seems to have many edges and nodes
that are not helpful for the estimation, the proposed approach uses
information similarly for all nodes and edges. Mechanisms such asattention may improve estimation performance by efﬁciently ignoring unimportant information.

Acknowledgments
This paper is based on results obtained from a project,

References[1] D. Lawrie, C. Morrell, H. Feild, and D. Binkley, “What’s in a name?
a study of identiﬁers,” in 14th IEEE International Conference on Program Comprehension (ICPC’06), Jun. 2006, pp. 3–12.
[2] A. De Lucia, M. Di Penta, R. Oliveto, A. Panichella, and
S. Panichella, “Using IR methods for labeling source code artifacts:
Is it worthwhile?” in 2012 20th IEEE International Conference on
Program Comprehension (ICPC), Jun. 2012, pp. 193–202.
[3] F. Deissenboeck and M. Pizka, “Concise and consistent naming,”
Software Quality Journal, vol. 14, no. 3, pp. 261–282, Sep. 2006.
[4] X. Xia, L. Bao, D. Lo, Z. Xing, A. E. Hassan, and S. Li, “Measuring
program comprehension: A Large-Scale ﬁeld study with professionals,” IEEE Trans. Software Eng., vol. 44, no. 10, pp. 951–976, Oct.
2018.
[5] 米内裕史, 早瀬康裕, and 北川博之, “ソースコード構文木とコール
グラフの統合的な埋め込みに基づくメソッド名の推定,” 研究報
告ソフトウェア工学 (SE), vol. 2020, no. 10, pp. 1–8, 2020.
[6] S. Kurimoto, Y. Hayase, H. Yonai, H. Ito, and H. Kitagawa, “Class
name recommendation based on graph embedding of program elements,” in Proceedings - Asia-Paciﬁc Software Engineering Conference, APSEC, vol. 2019-Decem, 2019, pp. 498–505.
[7] U. Alon, S. Brody, O. Levy, and E. Yahav, “code2seq: Generating
sequences from structured representations of code,” in International

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

Conference on Learning Representations, 2019. [Online]. Available:
https://openreview.net/forum?id=H1gKYo09tX
F. Ge and L. Kuang, “Keywords guided method name generation,” in2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC), vol. 0, May 2021, pp. 196–206.
M. Allamanis, M. Brockschmidt, and M. Khademi, “Learning to represent programs with graphs,” in 6th International Conference on
Learning Representations, ICLR 2018 - Conference Track Proceedings, Nov. 2018.
萬塲大登, 早瀬康裕, 天笠俊之, and 北川博之, “オブジェクト指向
プログラムの要素関係グラフを用いたクラス名の end-to-end 学
習,” 情報処理学会第 83 回全国大会, vol. 4, p. 01, 2021.
U. Alon, M. Zilberstein, O. Levy, and E. Yahav, “code2vec: learning
distributed representations of code,” Proc. ACM Program. Lang., no.
POPL, pp. 1–29, Jan. 2019.
F. Liu, G. Li, Z. Fu, S. Lu, Y. Hao, and Z. Jin, “Learning to recommend method names with global context,” in Proceedings of the44th International Conference on Software Engineering, ser. ICSE
’22. New York, NY, USA: Association for Computing Machinery,
Jul. 2022, pp. 1294–1306.
W. Ma, M. Zhao, E. Soremekun, Q. Hu, J. M. Zhang, M. Papadakis,
M. Cordy, X. Xie, and Y. L. Traon, “GraphCode2Vec: generic code
embedding via lexical and program dependence analyses,” in Proceedings of the 19th International Conference on Mining Software
Repositories, ser. MSR ’22. New York, NY, USA: Association for
Computing Machinery, Oct. 2022, pp. 524–536.
M. Allamanis, H. Peng, and C. Sutton, “A convolutional attention
network for extreme summarization of source code,” in Proceedingsof The 33rd International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, M. F. Balcan and K. Q.
Weinberger, Eds., vol. 48. New York, New York, USA: PMLR,
2016, pp. 2091–2100.
S. Iyer, I. Konstas, A. Cheung, and L. Zettlemoyer, “Summarizing
source code using a neural attention model,” in Proceedings of the54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Berlin, Germany: Association for
Computational Linguistics, Aug. 2016, pp. 2073–2083.
X. Hu, G. Li, X. Xia, D. Lo, and Z. Jin, “Deep code comment generation,” in Proceedings of the 26th Conference on Program Comprehension, ser. ICPC ’18. New York, NY, USA: Association for
Computing Machinery, May 2018, pp. 200–210.
——, “Deep code comment generation with hybrid lexical and syntactical information,” Empirical Software Engineering, vol. 25, no. 3,
pp. 2179–2217, May 2020.
W. U. Ahmad, S. Chakraborty, B. Ray, and K. W. Chang, “Atransformer-based approach for source code summarization,” arXiv,
2020.
D. Zügner, T. Kirschstein, M. Catasta, J. Leskovec, and
S. Günnemann, “Language-agnostic representation learning ofsource code from structure and context,” in International Conference on Learning Representations, 2021. [Online]. Available:
https://openreview.net/forum?id=Xh5eMZVONGF
S. Wang, M. Wen, B. Lin, and X. Mao, “Lightweight global and local contexts guided method name recommendation with prior knowledge,” in Proceedings of the 29th ACM Joint Meeting on European
Software Engineering Conference and Symposium on the Foundations of Software Engineering, ser. ESEC/FSE 2021. New York,NY, USA: Association for Computing Machinery, Aug. 2021, pp.
741–753.
M. Allamanis, E. T. Barr, S. Ducousso, and Z. Gao, “Typilus: neural
type hints,” in Proceedings of the 41st ACM SIGPLAN Conference
on Programming Language Design and Implementation, ser. PLDI
2020. New York, NY, USA: Association for Computing Machinery,Jun. 2020, pp. 91–105.
M.-A. Lachaux, B. Roziere, M. Szafraniec, and G. Lample, “DOBF:A deobfuscation Pre-Training objective for programming languages,”
https://openreview.net/pdf?id=3ez9BSHTNT, accessed: 2022-12-26.
D. Guo, S. Ren, S. Lu, Z. Feng, D. Tang, S. Liu, L. Zhou, N. Duan,

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]
[38]

[39]

A. Svyatkovskiy, S. Fu, M. Tufano, S. K. Deng, C. Clement,
D. Drain, N. Sundaresan, J. Yin, D. Jiang, and M. Zhou, “GraphCodeBERT: Pre-training code representations with data ﬂow,” in International Conference on Learning Representations, 2021.
K. Cho, B. van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares,
H. Schwenk, and Y. Bengio, “Learning phrase representations using RNN Encoder–Decoder for statistical machine translation,” inProceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar: Association for
Computational Linguistics, Oct. 2014, pp. 1724–1734.
T. Luong, H. Pham, and C. D. Manning, “Effective approaches
to attention-based neural machine translation,” in Proceedings ofthe 2015 Conference on Empirical Methods in Natural Language
Processing. Lisbon, Portugal: Association for Computational
Linguistics, Sep. 2015, pp. 1412–1421. [Online]. Available:
https://aclanthology.org/D15-1166
U. Alon, M. Zilberstein, O. Levy, and E. Yahav, “A general pathbased representation for predicting program properties,” ACM SIGPLAN Notices, vol. 53, no. 4, pp. 404–419, Jun. 2018.
H. Peng, G. Li, W. Wang, Y. Zhao, and Z. Jin, “Integrating tree path in transformer for code representation,” inAdvances in Neural Information Processing Systems, M. Ranzato,
A. Beygelzimer, Y. Dauphin, P. S. Liang, and J. W. Vaughan,Eds., vol. 34. Curran Associates, Inc., 2021, pp. 9343–9354.
[Online]. Available: https://proceedings.neurips.cc/paper/2021/ﬁle/4e0223a87610176ef0d24ef6d2dcde3a-Paper.pdf
J. Zhang, X. Wang, H. Zhang, H. Sun, K. Wang, and X. Liu, “A novel
neural source code representation based on abstract syntax tree,” in2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). ieeexplore.ieee.org, May 2019, pp. 783–794.
C. Lin, Z. Ouyang, J. Zhuang, J. Chen, H. Li, and R. Wu, “Improving code summarization with block-wise abstract syntax tree splitting,” in 2021 IEEE/ACM 29th International Conference on Program
Comprehension (ICPC). IEEE, May 2021.
K. Zhang, W. Wang, H. Zhang, G. Li, and Z. Jin, “Learning to represent programs with heterogeneous graphs,” in Proceedings of the
30th IEEE/ACM International Conference on Program Comprehension, ser. ICPC ’22. New York, NY, USA: Association for Computing Machinery, Oct. 2022, pp. 378–389.
B. Liu, T. Wang, X. Zhang, Q. Fan, G. Yin, and J. Deng, “A neuralnetwork based code summarization approach by using source codeand its call dependencies,” in ACM International Conference Proceeding Series. Association for Computing Machinery, Oct. 2019.
H. Yonai, Y. Hayase, and H. Kitagawa, “Mercem: Method name recommendation based on call graph embedding,” in 2019 26th AsiaPaciﬁc Software Engineering Conference (APSEC), Dec. 2019, pp.
134–141.
S. Gao, C. Gao, Y. He, J. Zeng, L. Y. Nie, X. Xia, and M. R. Lyu,
“Code structure guided transformer for source code summarization,”
ACM Trans. Softw. Eng. Methodol., Jul. 2022.
M. Allamanis, E. T. Barr, C. Bird, and C. Sutton, “Suggesting accurate method and class names,” in 2015 10th Joint Meeting of theEuropean Software Engineering Conference and the ACM SIGSOFT
Symposium on the Foundations of Software Engineering, ESEC/FSE
2015 - Proceedings, 2015, pp. 38–49.
R. Compton, E. Frank, P. Patros, and A. Koay, “Embedding javaclasses with code2vec: Improvements from variable obfuscation,” inMsr 2020. ACM, 2020, p. 11.
T. N. Kipf and M. Welling, “Semi-Supervised classiﬁcation
with graph convolutional networks,” in International Conference on Learning Representations, 2017. [Online]. Available:
https://openreview.net/forum?id=SJU4ayYgl
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, Nov. 1997.
D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimization,” 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, pp. 1–15, 2015.
T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna:A next-generation hyperparameter optimization framework,” in Proceedings of the 25th ACM SIGKDD International Conference on
Knowledge Discovery & Data Mining, ser. KDD ’19. New York,NY, USA: Association for Computing Machinery, Jul. 2019, pp.
2623–2631.
[40] Paszke, Gross, Massa, Lerer, and others, “Pytorch: An imperative
style, high-performance deep learning library,” Adv. Neural Inf. Process. Syst., 2019.
[41] M. Wang, D. Zheng, Z. Ye, Q. Gan, M. Li, X. Song, J. Zhou, C. Ma,
L. Yu, Y. Gai, T. Xiao, T. He, G. Karypis, J. Li, and Z. Zhang, “Deepgraph library: A graph-centric, highly-performant package for graphneural networks,” arXiv preprint arXiv:1909.01315, 2019.