Clustering Commits for
Understanding the Intents of Implementation
Kenji Yamauchi, Jiachen Yang, Keisuke Hotta, Yoshiki Higo, and Shinji Kusumoto
Graduate School of Information Science and Technology, Osaka University, Japan
1-5, Yamadaoka, Suita, Osaka, 565-0871, Japan{y-kenji, jc-yang, k-hotta, higo, kusumoto}@ist.osaka-u.ac.jp
Abstract—This paper proposes a novel technique for clusteringcommits for understanding the intents of implementation. Sucha classification of commits should be able to assist developers
to understand commits related to particular requirements, forexample, how and why has this function been implemented, or
has this function suffered from any bugs? Our technique adopts
a clustering algorithm on identifier names that are related tochanges in each commit. Such an approch allows us to take thesemantics of each commit into account without commit messages,
and so our approach is robust for the situation where somecommits lack accurate descriptions. We conducted a pilot study
to confirm that our idea answers to our objective. The pilot
study found some good examples that showed the usefulness of
our approach, and there were some undesirable results that gavesome ideas to improve it.
Keywords—Commit Classification, Version Control System,
Mining Software RepositoriesI.

I NTRODUCTION

Version control systems (VCS), such as git or Subversion,
are very helpful for managing software artifacts during software evolution. They store changes on software artifacts whendevelopers commit the changes. In other words, they record
the history of the software systems. Hence, developers canunderstand how each function was implemented by reviewing
past changes on its related artifacts, or even can revert pastchanges that seem incorrect or inappropriate.
To reap such benefits from VCSs, it is necessary tounderstand the intents of implementation in past commits. It
should be beneficial for this objective to cluster commits basedon requirements to which each commit is related. For instance,
suppose a case where a feature was implemented by multiplecommits, and the developer wants to revisit the evolution of thefunction. In this case, the developer can know which commits
she/he should review if the commits related to the requirement“implementing the feature” are classified into the same cluster.
Furthermore, when a developer wants to understand a commit
in the category, the information in the other commits including
the source code changes in them should be helpful.
There are some techniques for clustering commits to understand them. A typical approach is the one based on metadata
of commits including commit messages or committer names.
Commit messages are used because they directly represent
what the commits do. This approach classifies commits via
natural language processing or information retrieval techniqueson the data [1], [2]. These techniques work well if commits
were annotated well by its committers. Hence, it should be

best suited to the objective under the ideal situation wheredevelopers carefully annotate every commit. However, developers often commit without any messages, or with incomplete
and/or inaccurate messages. If a commit does not have acommit message, or has a commit message with incomplete
or inaccurate description, the approach will fail to classify thecommit into a proper category.
Another approach is based on syntax differences of sourcecode in each commit. This approach detects structural differences from every commit, and classifies commits based onthem [3]. It does not use commit messages, and so it works
well even though some commits lack good commit messages.
However, this approach uses only syntactic information, andso it might miss semantic information that is valuable forunderstanding commits. Hence, this approach will not besuitable for our objective, which is clustering commits tounderstand the intents of implementation in each commit.
This paper proposes a novel approach for clustering commits. Our approach detects identifier names related to changes
in every commit, and classifies every commit via the bag-ofwords model with the identifier names. Our approach does
not use commit messages, which allows it to cover the weak
point of the commit message based approach. Furthermore, it
can consider semantics of commits because identifier names
should provide us with an insight into semantics of changes
occurring in each commit [4].
We implemented our approach as a prototype tool, andconducted a pilot study with it to discuss the pros and cons of
our approach. The pilot study found some desirable examples
of clustering, which supports the usefulness of our approach.
On the other hand, the pilot study also found some undesirable
examples, but they give us insights to improve our approach.
The remainder of this paper is organized as follows. Firstof all, we introduce related work in Section II. Section III
describes our proposed technique. Section IV shows the resultof the pilot study. In addition, we discuss the validity of theresult in Section V. Finally, Section VI concludes this paper.
II.

R ELATED W ORK

Hindle et al. proposed an automated technique to classify commits into maintenance categories, namely Corrective,
Adaptive, Perfective, Feature Addition, and Non Functional[1]. Their technique trains the machine learning models withword distribution of commit messages, names of committers,
and the number of changed files per directory.

Fig. 1.

The process flow of clustering commits

Hattori and Lanza conducted a study on the nature ofcommits by classifying them with their size and commit
messages [2]. They reported that the majority of commits
change very few files, and the majority of tiny commits are
not related to development activities including adding a newfeature or testing.
Dragan et al. proposed a technique to classify commits
to understand design changes of software systems [3]. They
were interested in design changes, and so they focused on
method and class level changes. Their technique is based on
method stereotypes [5], which generalize intrinsic or atomic
behavior of the method. They classify commits into somecategories including Structure Modifier and Behavior Modifier,and they stated that these classifications assist developers inunderstanding the extent and impact of changes in the commits.
Although the objectives of the existing research differ
from each study, all of these techniques share the common
avenue, classifying commits, for their objectives. Our objective
also differs from the ones of the above studies, which is
to cluster commits based on their related requirements. To
achieve our objective, we should consider the semantics ofcommits. In addition, we should not use commit messages
for practical use in a non-ideal situation where some commits
lack commit messages. Hence, we propose a novel approach
which considers the semantics of source code changed in eachcommit by analyzing identifier names affected by the changes.
There exists a large body of work that applies information
retrieval techniques to the field of software engineering, which
includes the work of Maletic and Marcus [6]. They proposed atechnique to cluster software components with latent semantic
indexing. Although their work and ours aim at different goals,
but they are similar in that both of them use information
retrieval technique on software artifacts.
III.

P ROPOSED M ETHOD

In this section, we will discuss our proposed method
to classify the commits into clusters that correspond to therequirements.
The inputs of our method are the software repositories
organized by the VCSs. The outputs are the commits in therepositories that have been classified by the different intents
correspond to the requirements. We classify the commits basedon the modifications on the source files, i.e. the occurrence ofidentifiers in the source code between the revisions. We do nottake other information such as commit messages or committers
into account currently.
The flow of processing of our proposed method consists ofthe following three steps, as shown in the Figure 1.
S TEP 1 Obtains the syntax differences of each commits fromthe source code repository. The result has finer granularity thanthe line-based differences which can be easily obtained fromthe VCSs.
S TEP 2 Extracts the identifier names from the syntax differences which are obtained from the Step 1. Further, we
divide the identifier names into English words and count theoccurrence of each word, to generate a feature vector of thesyntax difference.
S TEP 3 Applies a clustering algorithm on the commits using
the feature vectors to form the clusters that correspond to therequirements. We consider each resulted cluster as a group ofcommits to implement a single intent of a requirement.
We treat all the operations in the commit as the modifications to the contents of the files. In other words, we consider
the addition of a new file as the modification of adding all
the contents to an empty file, and the removal of an existing
file as the modification of deleting all the contents from thefile. Further, we consider the rename operation on a file as no
modifications to the contents of the file. As a result, insteadof tracking each operation recorded in the commit, we only
process one kind of the operation, i.e. the modification of thefile contents.
In the following subsections, we will discuss each step indetail.
A. (S TEP 1) The Extraction of the Syntax Differences
Firstly, we obtain a list of modified source code files ineach commit. For each modified file, we utilize the approachof Change Distilling [7] to get the syntax difference in therevisions of the commit. The output of the Change Distillingis a list of the modifications, of each corresponding to eithera statement or an expression fragment from the source code.
We illustrate the extraction using an example with thesource code modifications, which are shown in the Figure 2.
The extracted modifications will be three syntax differences,which are private int logCounts, !IS LOGGED andlog(messages).

Fig. 3.

Fig. 2.

An example of modifications to the source codeAn example of syntax differences extracted from a commit.

B. (S TEP 2)The Extraction of the Identifier Names and theGeneration of Feature Vector
Secondly, we extract the identifier names from each syntax
difference. Previous study [8] suggested that the structural
information will aid to the information retrieval, therefore wealso extracted the method name and class name that contains
the syntax difference, in addition to the identifier names. If thesyntax difference appeared inside a method body, the names ofthe methods and the classes that contain the syntax difference
are extracted as well. For the syntax difference appeared
outside a method body, the class name is extracted, in addition
to the modified identifier names. These syntax differences
outside a method body often represent the modifications to thedeclarations of the methods or fields. The additional method
and class names will be treated as appearing once in the syntax
differences.
In the previous example of Figure 2, we extract theidentifier names from the three syntax differences. For thesyntax difference private int logCounts, we extract
the variable name logCounts together with its class name
Logger. Finally, we will get a set of identifier names:
logCounts, Logger, IS_LOGGED, Logger, checkLog,
log, messages, Logger and checkLog.
Next, we generate the feature vectors for clustering. This
step involves the following 4 sub-steps:
S TEP 2A Divide the identifier names into English words.
S TEP 2B Lemmatize the English words into their root words.
The extracted root words will be referred as feature words inour following discussion.
S TEP 2C Count the occurrences of each feature word.
S TEP 2D Convert the occurrences of the feature words into afeature vector.
1) (S TEP 2A) Divide Words: We divide the identifier
names extracted in S TEP 1 into English words, assuming they

are named in the camel case or snake case. The camel caseis a naming convention that each word begins with a capitalletter, such as getYourName. The snake case is anothernaming convention that words are separated with underscorecharacter, such as MAX_COUNTS. Furthermore, we convert thedivided words into lower case to help the further process. For
example, given the identifier names convertedName and
MAX_COUNTS, we divide them into converted name andmax counts, respectively.
2) (S TEP 2B) Lemmatization: Next, for each divided
words, we remove its prefix and suffix and lemmatize theminto their root words in English1 . For the previous example,
we will convert converted and counts to their root words
convert and count, respectively.
An example combining S TEP 2A and 2B is shown in
Figure 3. In this example, we extract from 4 identifier
names, namely convertedName, convertYourName,SERVERS_FOR_SENDING and sendToMailServer, andget a list of words convert, name, your, server,
for, send, to, mail from the lemmatization, to be
used as the feature words.
3) (S TEP 2C) Count the Occurrence of Each Feature Word:
In this step, we count the occurrence of each word in theextracted feature words. We ignore the occurrence of thestop words of English2 during this step. The stop words arethose words that frequently appear, but bring little semantic
meaning. We ignore them to avoid negative influences on
the classification. As far as the example in the Figure 3 isconcerned, we have convert twice, name once, server
twice, send twice and mail once.
4) (S TEP 2D) Convert to Feature Vector: Lastly, we convert the counted occurrences into a feature vector. For eachcommit, we define the feature vector as a Bag-Of-Words(referred as BOW) vector that takes the normalized occurrenceof feature words as its features.
As a result, the number dimensions of the total BOW
vector will be the number of unique words that appeared in
the identifier names from all the syntax differences of all
the commits. A value of 0 in the vector indicates that thecorresponding word does not appear in the syntax differenceof the commit.
For example, in commit A the feature words a, b appear
once for each, and in commit B the feature words b, c appear
1 We used the WordNetLemmatizer defined in python library
http://www.nltk.org/
2 We used the list of English stop words defined by python library
http://scikit-learn.org/

C. S TEP 3 Apply the Clustering Algorithm
Lastly, we cluster the commits using the feature vectors
obtained from S TEP 2D. Among all available clustering algorithms, we considered the following 2 requirements, and thenchoose the Repeated Bisection [9]:•No need to specify the expected number of clusters. In
our approach, it is difficult to approximate the numberof different requirements, i.e. the number of expectedclusters. Therefore, we can not use the algorithms thatneed to specify the number of expected clusters.

•

High scalability. For those long lasting projects, it isquite possible that there is a huge amount of commits
recorded in their repositories. Therefore, we need aclustering algorithm with high scalability.

Besides Repeated Bisection, there are several algorithms that
do not need to specify the number of clusters. Some of
the algorithms are costly in terms of computing complexity,
which require a long time for processing a large amount ofdata. Repeated Bisection fulfilled both requirements as it isfaster than other algorithms that do not require the number ofclusters [9]. Finding and experimenting with other clustering
algorithms will be a future work of this study.
IV.

P ILOT S TUDY

In this section, we will describe the experiments that wehave done as our pilot study. We applied our proposed method
on several software repositories and clustered the commits toconfirm whether there exists clusters that correspond to therequirements.
A. Confirm the Clustering Result
In this experiment, we applied four open source software(referred as OSS) repositories, and observed the resulted
number of clusters.
We selected four targets that are written in Java andorganized by Git, with relatively large number of commits(more than 5000) in their repositories.
In Table I, we show the clustering result for each target
repository. These repositories contain the commits that haveno modifications to the source code files, therefore we filtered these commits and count the target commits that have1 https://lucene.apache.org/solr/
3 http://www.wildfly.org/
4 http://jruby.org/

Projects
Lucene/Solr3
Jenkins CI4
WildFly5
JRuby6

Projects
Fig. 4.

The Distribution of the Number of Commits for Each Cluster

modifications in the source code. In Figure 4, we show the
distribution of the number of commits for each resulted cluster
in a boxplot. The X-axis of this figure is the four software
projects. We can see from this figure that the median numberof the commits are around 6 or 7 for every project.
1) Example of Desirable Clustering Result: Firstly we willshow an example from the properly clustered commits. We
show one cluster from the analysis of commits of Lucene/Solrin the Table II. This cluster consists of four commits. For thethree commits with ID 7 e4a64f5, 5adc910 and c5a985e, we
found the commit messages of them mentioned the same issue
ID SOLR-4275 8 that corresponds to a single implementation
requirement. We show the top 5 weighted words of thiscluster in Table III. The weight of the words are calculatedby the Repeated Bisection algorithm. By viewing the syntax
difference and the commit message of the commit a3e95d0,
we confirmed that it is related to the implementation of theoffset used in class TokenTokenizer. Further, a bug that
introduced by this commit was fixed by the commit c5a985e.
From the timestamp of the commit a3e95d0, we can confirm
that it was committed 2 years before the other 3 commits.
Therefore, we can confirm the intent of the commit a3e95d0
was implementing the requirement while the intents of the
7 The source code and other information of these commits can be accessedfrom https://github.com/apache/lucene-solr/commit/ID
8 http://issues.apache.org/jira/browse/SOLR-4275

TABLE II.
ID
e4a64f5
5adc910c5a985ea3e95d0

2 http://jenkins-ci.org/

TABLE I.

Number Of Commits for Each Cluster

twice for each. If we arrange the BOW vector in the order ofa, b, c, then the BOW feature vector for commit A and B
will be (1, 1, 0) and (0, 2, 2) respectively.

TABLE III.

C LUSTERING R ESULT# of Total Commits
11,15917,64014,28020,531# of Target Commits
7,341
8,452
10,247
13,142# of Clusters
919
1,102
1,204
1,487

D ESIRABLE C LUSTERING R ESULT (L UCENE /S OLR )Modified Files
TestTrie.java
TestTrie.java
TrieTokenizerFactory.java
TrieTokenizerFactory.java

Timestamp
6 Jan 2013
6 Jan 2013
6 Jan 201320 Jan 2011

Associated Issue ID
SOLR-4275
SOLR-4275
SOLR-4275
None

T OP 5 W EIGHTED W ORDS OF THE C LUSTER IN TABLE II
Word
trie
tokenizer
ofs
step
prec

Weight from Repeated Bisection
0.75257
0.471848
0.235517
0.228368
0.17303

commits e4a64f5, 5adc910 and c5a985e are fixing the issuesof the same requirement. To summarize, these four commits inthe same cluster have the same intent to implement the samerequirement.
2) Example of Undesirable Clustering Result: Next we willshow an example from the undesirable clustered commits. We
show one cluster from the analysis of commits of Jenkins CI
in the Table IV, which consists of three commits.
Firstly we discuss the two commits with ID b3553d6 and7d0bac1 that modifies IOUtils. The purpose of this class
was to provide the utilities about the input and output operations in the software. Therefore, it consists of a group of static
member functions that are independent in features. Therefore,
we considered that the implementations of these static methods
related to different intents. By viewing the syntax differencesin commit b3553d6 and 7d0bac1, we confirmed that they are
implementing different static methods. Therefore, we considered that they are implemented for different requirements.
Secondly we viewed the syntax difference of commit
a545a39 and found that it is modifying the access level ofthe inner classes of the class Launcher, which can also be
confirmed from the commit message. Because the implementations of these inner classes are performing IO operations,they are grouped with the other 2 commits.
As the discussion showed, the commits in Table IV were
implementing different requirements. Therefore, this cluster
was undesirable for our purpose, which is to group the commits
with the same implementation intent.
V.

VALIDITY OF THE R ESULTa single implementation intent. Therefore, it is necessary to
study the precision of the identified simultaneous modification
clusters in our further research.
VI.

This paper proposed a technique to classify commits into
requirements to understand the intents of implementation. It
used a clustering algorithm with identifier names that are
affected by changes in each commit. This approach allows
us to consider the semantics of commits without commit
messages, and so the proposed technique will work well eventhough some commits do not have enough descriptions.
We conducted a small pilot study on open source projects
with a prototype tool. The pilot study confirmed the usefulnessof our approach, and gave us some insights to improve theproposed technique.
As future work, we are going to improve the accuracy ofclassification of the proposed technique by other clustering
algorithms or by improving word extraction from identifiers.
In addition, we are going to seek other information that canbe extracted from source code and can contribute to improve
the accuracy of classification.
ACKNOWLEDGEMENT
This work was supported by MEXT/JSPS KAKENHI
25220003, 24650011, and 24680002.
R EFERENCES[1]

We discuss the validity of our result in this section.
[2]

A. Validity of the Proposed Method
Firstly, we assumed that the identifier names in the commits
can reflect the implementation intent. Therefore, our method
can generate undesirable result when the identifier names are
inadequate. Secondly, we assumed that the commits are theminimum units to divide the implementation. The existing
study suggested that a commit can contain multiple implementation contents with different intents. Previous studies [10],[11] in our research group have focused on identifying anddividing these tangled commits. They are the enabler of thisresearch.
B. Validity of the Discussion
In the discussion we considered the requirements thatare distributed among multiple source files by showing thepercentage of the simultaneous modification clusters in thesoftware repositories, in order to check the effectiveness of ourapproach. However, we did not manually check the resulted
clusters to verify whether they are really corresponding to
TABLE IV.
ID
b3553d6
7d0bac1
a545a39U NDESIRABLE C LUSTERING R ESULT (J ENKINS CI)

Modified File
IOUtils.java
IOUtils.java
Launcher.java

Commit Message
added a convenience methoddohfor serialization work these interfaces need to be public

C ONCLUSION

[3]

[4]

[5]

[6]

[7]

[8]

[9]
[10]

[11]

A. Hindle, D. M. German, M. W. Godfrey, and R. C. Holt, “Automatic Classification of Large Changes into Maintenance Categories,”in Proceedings of the 17th International Conference on Program
Comprehension, 2009, pp. 30–39.
L. P. Hattori and M. Lanza, “On the Nature of Commits,” in Proceedingsof the 4th ERCIM Workshop on Software Evolution and Evolvability,
2008, pp. 63–71.
N. Dragan, M. L. Collard, M. Hammad, and J. I. Maletic, “Using
Stereotypes to Help Characterize Commits,” in Proceedings of the 27th
International Conference on Software Maintenance, 2011, pp. 520–523.
A. Kuhn, S. Ducasse, and T. Gı̂rba, “Semantic Clustering: Identifying
Topics in Source Code,” Information and Software Technology, vol. 49,
no. 3, pp. 230–243, 2007.
N. Dragan, M. L. Collard, and J. I. Maletic, “Reverse Engineering
Method Stereotypes,” in Proceedings of the 22nd International Conference on Software Maintenance, 2006, pp. 24–34.
J. I. Maletic and A. Marcus, “Supporting Program Comprehension
Using Semantic and Structual Information,” in Proceedings of the 23rd
International Conference on Software Engineering, 2001, pp. 103–112.
B. Fluri, M. Wursch, M. PInzger, and H. C. Gall, “Change distilling:
Tree differencing for fine-grained source code change extraction,” IEEE
Transactions on Software Engineering, vol. 33, no. 11, pp. 725–743,
2007.
B. Bassett and N. A. Kraft, “Structural information based term weighting in text retrieval for feature location,” in Program Comprehension(ICPC), 2013 IEEE 21st International Conference on. IEEE, 2013,
pp. 133–141.
G. Karypis, “Cluto-a clustering toolkit,” DTIC Document, Tech. Rep.,
2002.
N. Kusunoki, K. Hotta, Y. Higo, and S. Kusumoto, “How much do code
repositories include peripheral modifications?” in Software Engineering
Conference (APSEC, 2013 20th Asia-Pacific. IEEE, 2013, pp. 19–24.
H. Kirinuki, Y. Higo, K. Hotta, and S. Kusumoto, “Hey! are youcommitting tangled changes?” in ICPC, 2014, pp. 262–265.