2013 20th Asia-Pacific Software Engineering Conference

How Much Do Code Repositories Include Peripheral Modiﬁcations?
Noa Kusunoki, Keisuke Hotta, Yoshiki Higo, Shinji Kusumoto
Graduate School of Information Science and Technology, Osaka University, Japan
1-5, Yamadaoka, Suita, Osaka, 565-0871, Japan{k-noa, k-hotta, higo, kusumoto}@ist.osaka-u.ac.jp
Excluding large commits provides us one more important
thing to worry about. Such exclusion assumes that all thesmall commits includes code changes and there are no
peripheral modiﬁcations in the commits. However, none ofresearch efforts have investigated characteristics of small
commits yet. Consequently, we investigate the following
research question.
RQ2 Do small commits consist of only modiﬁcations on
program instruction code?
In order to investigate the research questions, we propose
a new technique to distinguish the following types of modiﬁcations from each other at a ﬁne-grained level:• modiﬁcations on program instruction code,• modiﬁcations for reformatting source ﬁles,• modiﬁcations on code comments, and• modiﬁcations on other ﬁles than source ﬁles.
The remainder of this paper is organized as follows:in Section II, we deﬁne some terms used in this paperand explain the proposed technique; Section III shows anevaluation on students’ projects and an experiment on open
source software; related works are described in Section IV;
lastly, Section V concludes this paper.

Abstract—In the last decade, a variety of studies on mining
software repositories has been conducted. Mining repositories
has a potential to obtain useful knowledge for the future
development and maintenance. When software repositories aremined, large commits in them are often excluded from mining
targets because large commits include merging and we believethat large commits include peripheral modiﬁcations, which may
affect negative impacts on mining code repositories. However,
if large commits include code modiﬁcations, excluding largecommits loses such modiﬁcations unintentionally. Moreover,such data cleansing assumes that there are no peripheral
modiﬁcations in small commits. In this paper, we investigate
how much peripheral modiﬁcations are included in commits
in code repositories. As a result, we found that excluding largecommits is insufﬁcient to remove hindrances in commits formining code repositories.
Keywords-Mining software repositories, Large commit,
Source code analysis

I. I NTRODUCTION
Recently, the research areas related to mining software
repositories are very active and attract much attention [1],[2]. A software repository includes a variety of historicalinformation on the past activities related to its software.
Mining software repositories means actions and techniques
for extracting and deriving useful knowledge for the future
development and maintenance.
Historical code repository is one of the well-mined repositories. They have rich information to gain knowledge or
principles that are useful for software engineering [3], [4].
Therefore, great efforts have been spent on mining historicalcode repositories. Such research efforts have a wide variety
of interests, including defects [3]–[7], changes [8]–[13], orclones [14].
When we mine a software repository, large commits in it
are often excluded from the analysis targets. This is because
large commits often include merging [19] and we believethat large commits consist of peripheral modiﬁcations suchas reformatting source code. However, if a large commit
includes modiﬁcations on program instruction code itself,
excluding the large commit may lose important code changes
unintentionally. Consequently, in this research, we investigate the following research question.
RQ1 Do large commits consist of only peripheral modiﬁcations?
1530-1362/13 $31.00 © 2013 IEEE
DOI 10.1109/APSEC.2013.106

II. OVERVIEW OF THE I NVESTIGATION T ECHNIQUE
In order to investigate our two research questions, itis necessary to discriminate changes affecting program instructions from those not affecting them. For this objective,
the investigation technique in this research classiﬁes every
modiﬁcation in a commit into the following four categories.
• Type-A: modiﬁcations for white spaces, tabs, and newline characters.
• Type-B: modiﬁcations for comments.
• Type-C: modiﬁcations for program instruction code.
• Type-D: modiﬁcations in other ﬁles than source ﬁles,or additions/deletions of such ﬁles.
The investigation in this study desires a ﬁne-grained
analysis on modiﬁcations to achieve a high accuracy.
Hence, it analyzes every modiﬁcation at the token-level.
Herein, “a modiﬁcation at the token-level” means “addition/change/deletion of a token”. In other words, every token
modiﬁed in a commit is classiﬁed into the above categories.
Note that Type-D modiﬁcations cannot be segmentalized
into the token-level because the proposed technique focuses
19revision r
public void A()A{
method1();
method2();}

perform
lexical analysis

revision r+1

revision r
public¥s void ¥s A (

;

¥n }

¥n ¥s

// Delete Method

¥n }

;

¥n

method2 (

;

¥n ¥s

) ¥s { ¥n¥tmethod1 (

)

;

¥n

) ¥s { ¥n¥smethod1 ();) ¥s { ¥n¥tmethod1 ()) ¥s { ¥n¥smethod1 ()¥tmethod2 ()

revision r+1
public¥s void ¥s B (identify modified tokens

public void B()
B{method1();m// Delete Method}

revision r
public¥s void ¥s A (¥t); ¥n }

revision r+1
public¥s: white space

¥t

: tab

¥n

: new line character¥s void ¥s B (// Delete Method¥n }classify modifications into three categories
Type-A¥tγαType-B

Type-CA¥tmethod2: γ is deleted// Delete Method: α is changed to ββδ¥s: δ is added

Figure 1.

B

¥s

(

)

;

An Overview of the Investigation Technique

STEP-2: Detection of Modiﬁed Tokens

on modiﬁcations on source ﬁles and there is no need to split
modiﬁcations in other ﬁles.
The investigation technique takes a historical code repository as its input, and classiﬁes every modiﬁcation on every
commit into the above three categories.
The investigation technique on a commit consists of thefollowing three steps, and it repeats the steps for all thecommits included in the given repository.

This step detects added/changed/deleted tokens in thecommit. It detects mappings of tokens by comparing two
token sequences created from source ﬁles before and after
modiﬁed. The comparison is based on textual representations and types of tokens. This step identiﬁes what tokens
are added/changed/deleted in this commit with the token
mappings. It identiﬁes a pair of changed tokens, six deleted
tokens, and three added tokens from the example in Figure
1.

STEP-1: Performing lexical analysis.
STEP-2: Detecting modiﬁed tokens.
• STEP-3: Classifying modiﬁed tokens.
••

STEP-3: Classiﬁcation of Modiﬁed Tokens

Figure 1 shows how the investigation technique works on
a commit. This example assumes that only a single source
ﬁle is modiﬁed in the commit for the ease of explanations,but the investigation technique works in the same way inthe case of commits in which two or more source ﬁles are
modiﬁed. The followings brieﬂy describe each step with theexample.

The ﬁnal step classiﬁes every added/changed/deleted token detected in the previous step into the above three
categories. The classiﬁcation is performed based on typesof modiﬁed tokens. For instance, the example in Figure 1
has deletion of two tabs and addition of two white spaces,and so the classiﬁcation regards these four modiﬁcations as
Type-A. The example also has addition of a comment, which
is classiﬁed into Type-B. The other modiﬁcations, which
include the change on identiﬁes from “A” to “B”, and thedeletion of “method2”, “(”, “)”, and “;”, are classiﬁed into
Type-C in a similar fashion.

STEP-1: Lexical Analysis
The ﬁrst step performs lexical analysis on both of two
ﬁles before and after modiﬁed. Hence, this step provides
two token sequences for every ﬁle modiﬁed in the commit.
A remarkable point of the lexical analysis in this study is that
it is interested not only in usual tokens (such as identiﬁer orcomma) but also in comments, white spaces, tabs, and newline characters. This means that comments, white spaces,tabs, and new line characters are regarded as tokens as you
can see in Figure 1.

III. E XPERIMENT
We conducted an experiment on two open source systems,
Columba and SQuirreL SQL Client in order to answer the

RQs. Herein, we show the RQs again:
RQ1 Do large commits consist of only peripheral modiﬁcations?

20other hand, in SQuirreL, only 13.5% LOCs were includedin the other ﬁles modiﬁcations, which was the lowest rate.
As a result, there are different trends between the total
rate of the three type modiﬁcations of the two systems. In
Columba, all the ﬁle groups have approximately the samevalue. Meanwhile in SQuirreL, the smaller commits are, thehigher value their total rate becomes.
Next, we ignored the other ﬁles modiﬁcations. In thisexperiment, we distinguished the other ﬁles from source
ﬁles by checking ﬁle’s extension. That is because when
we conduct repository mining experiments, we often ignore
commits not related to source ﬁles. Figure 3 shows the result.
We can see that format and comment modiﬁcations often
appear in commits regardless of their size. Table II show
more quantitative data. Each cell of this table calculated withthe following formulae.

RQ2 Do small commits consist of only modiﬁcations on
program instruction code?
The numbers of revisions in the repositories were 463 and
6,737, respectively.
Figure 2 shows each type of modiﬁcations included inevery of the commits in the Columba and SQuirreL repositories. Every commit forms a vertical bar, and commits
are sorted in the descending order of their size. Each bar
consists of four colors, blue, read, gray, and water-clear,each of which corresponds to format, comment, other ﬁles,and code modiﬁcations, respectively. Their lengths reﬂect therate of their LOCs for all the lines modiﬁed in the commit.
If there is no water-clear part in a given bar, there is nocode modiﬁcation in the commit. Also, the ﬁgure includes
the LOC of each commits1 . Note that the ﬁgure has thelogarithmic axis for the commit LOC.
We divided them into ﬁve groups based on their size. The
ﬁrst group includes top 20% of the large commits, and thelast group includes top 20% of the small commits. Then, we
measured following rates F(G), C(G), and O(G) for eachgroup of the commits.

F(G) =∑ f ormat(c)c∈G∑ loc(c)× 100

F (G) =

C(G) =

c∈G∑ loc(c)× 100∑ other(c)O(G) =
loc(c) =f ormat(c) + comment(c)+other(c) + code(c)∑ loc(c)

× 100

× 100

(5)

∑ comment(c)C (G) =

c∈G

× 100

(6)

loc (c) =f ormat(c) + comment(c) + code(c)

(7)

(1)

∑ loc (c)c∈G

The table shows that around 30% modiﬁcations are eitherof the format or comment modiﬁcations in the middle
commits (20-80%). On the other hand, in the top 20%
commits, they occupy only around 10%.
Our answer to RQ1 is NO. In the experiment, the top
20% large commits included format, comment and otherﬁles modiﬁcations, however the total rate of them for all
the modiﬁcations is less than half. Thus, we conclude that
excluding large commits loses many code modiﬁcations
unintentionally.

(2)

c∈G

c∈G

∑ loc (c)

c∈G

c∈G

∑ comment(c)∑ f ormat(c)

c∈G

(3)

c∈G

(4)

where,• G is a given group of commits,• f ormat(c) is the LOC of f ormat modiﬁcations in given
commit c.
• comment(c) is the LOC of comment modiﬁcations ingiven commit c.
• other(c) is the LOC of other f iles modiﬁcations ingiven commit c.
• code(c) is the LOC of code modiﬁcations in given
commit c.
Table I shows the measurement result. The two systems
have different trends. In Columba, the other ﬁles modiﬁcations occupied more than 40% on the top 20% commits,which was the highest rate in all the ﬁve groups. On the

Table IR ATE OF LOC S INCLUDED IN MODIFICATIONS OF FORMAT, COMMENT,AND OTHER FILES

1 The number of ﬁles modiﬁed in commits can be an indicator of commit
size. However, in this experiment, we used LOC for seeing how many lines
had been modiﬁed in every commit.

21

group
top 20%21-40%
41-60%
61-80%
81-100%format
2.9%
13.0%
12.0%
9.0%
3.0%

(a) Columba
comment
other ﬁles
1.6%
40.1%
10.2%
24.5%
14.6%
22.3%
6.3%
29.0%
7.2%
34.6%

total
44.7%
47.6%
48.9%
44.4%
44.7%

group
top 20%21-40%
41-60%
61-80%
81-100%(b) SQuirreL SQL Client
format comment
other ﬁles
5.7%
3.5%
13.5%
11.0%
9.6%
19.0%
9.9%
10.9%
29.9%
6.5%
10.2%
43.7%
1.4%
3.6%
62.5%

total
22.7%
39.6%
50.6%
60.7%
67.5%

#  

 $  



 $ !" $  $     

 
 



 


 
 



 



 






 


 
 


 
 
 
 
 
 
 
 


 


 









 


 
 








 
 


 


 



 


(a) Columba

#   $  

 $ !" $  $     

 


 
 



 
 



 




 









 

 

 

 

 

 
 
 
 
 
 
 
 
 

 

 

 

 

 

 









 

 

 

 

 


 


(b) SQuirreL SQL Client
Figure 2.

Format, comment, and other ﬁles modiﬁcations

IntelliJ IDEA have the functions. Reusing the functions is a
simple and easy way.

Our answer to RQ2 is also NO. In the experiment,
middle and small commits also include such peripheral
modiﬁcations. Moreover, we found that the rate of peripheral
modiﬁcations on the middle and small commits was higher
than the one on the large commits. Thus, we conclude that
excluding large commits remains many peripheral modiﬁcations unintentionally.
Our answers to the RQs implies that excluding largecommits is insufﬁcient to exclude only peripheral modiﬁcations. We need another way to exclude them. For example,
constructing a new repository from the original one should
be a reasonable way.
1) creating a new repository,2) checking out revision r (r = 1, 2, · · · ),3) removing comments from the revision and reformatting it,4) committing the revision to the new repository,5) going back to (2) if there are more revisions in theoriginal repository.
Removing commits from source ﬁles and reformatting themis not such a complicated task. IDEs such as Eclipse or

IV. R ELATED W ORK
Fluri et al. proposed a method classifying modiﬁcations
based on their semantics [15]. In their method, ASTs are
generated from two versions of source code, each of which
is before or after the modiﬁcation. The classiﬁcation is
performed by comparing the two ASTs. On the other hand,
our method adopts token sequence comparison because of
its rapidity, however, AST comparison will work well if we
can deﬁne how modiﬁcations are separated in AST-level.
Kawrykow and Robillard proposed a method to identify
commits that are not signiﬁcant for repository mining [16].
They deﬁned “surface” modiﬁcation, which does not have
any impact on execution behavior. For example, changing
variable names is a surface modiﬁcation. They reported
that, when we mine software repositories, analyzing such
surface modiﬁcation is useless. However, the authors havea different opinion about that. The authors think that any
kind of code modiﬁcations may be useful for any purpose.

22







 

!"












&
%














&


%
%
&&




&




&




%&
&
&


&




&




&
%
%
&
&
&




&








(a) Columba






 

!"










&



%










&




&



%
&
%
%%
%
&
&%&
&

%

&
%


&
&

&


&&


&

&
&



&
%
%
%
&
&&
&



&




&



%
&





(b) SQuirreL SQL Client
Figure 3.

Format and comment modiﬁcations

Their surface modiﬁcation include some code modiﬁcations
such as renaming variables. On the other hand, our code
modiﬁcation include any kind of modiﬁcation on program
instruction code itself.
Hayashi et al. proposed a method to refactor past edit
histories [17]. Their method automatically records all the operations that developers performed, then it reorders, merges,
or deletes the operations. By using their method, we canobtain well-organized edit histories from the original one
including many modiﬁcations for different purposes.
Some researchers conducted experiments for investigatinglarge commits in software repositories. Hattori et al. reported
that 5 or less source ﬁles are modiﬁed in most commits[18]. However, they also said that, in a part of commits, 100
or more source ﬁles are modiﬁed at the same time. Hindle
et al. reported that large commits happen when we merge
branched source ﬁles into it main stream, reformat source
ﬁles, and delete unneeded source code [19].
As mentioned above, there are many research efforts
related to commits in code repositories. However, none ofthem have investigated the amount of peripheral modiﬁcations in code repositories.
V. C ONCLUSION
In this paper, we investigated how much there were
modiﬁcations of format, comment, and other ﬁles in code
repositories. We found there is a non-negligible amountof such modiﬁcations not only in large commits but also
in small commits. Our ﬁndings implies excluding largecommits is insufﬁcient as a data cleansing before mining
code repositories. We need more sophisticated ways toignore such peripheral modiﬁcations.
As a reasonable way to realize that, we suggested that constructing a new code repository for mining code repositories.
The construction will be performed as repeated operationsof (1) checking out a revision from the original repository,(2) removing comments in the source ﬁles and reformatting
the source ﬁles in the revision, and (3) commit the revision
to the new repository. Consequently, built new repositories
do not include peripheral modiﬁcations.

23

ACKNOWLEDGMENT[7] C. Boogerd and L. Moonen, “Evaluating the Relation between
Coding Standard Violations and Faults within and across Software Versions,” in Proceedings of the 6th Working Conferenceon Mining Software Repositories, 2009, pp. 41–50.

This study has been supported by Grants-in-Aid for Scientiﬁc Research (S) (25220003), Grant-in-Aid for Exploratory
Research (24650011) from Japan Society for the Promotionof Science, and Grand-in-Aid for Young Scientists (A)(24680002) from the Ministry of Education, Culture, Sports,
Science and Technology.

[8] T. Zimmermann, P. Weißgerber, S. Diehl, and A. Zeller,“Mining Version Histories to Guide Software Changes,” IEEE
Transactions on Software Engineering, vol. 31, no. 6, pp.
429–445, 2005.
[9] G. Canfora, L. Cerulo, and M. D. Penta, “Identifying Changed
Source Code Lines from Version Repositories,” in Proceedings of the 4th International Workshop on Mining Software
Repositories, 2007, pp. 14–22.

R EFERENCES[1] H. Kadgi, M. L. Collard, and J. I. Maletic, “A survey andtaxonomy of approaches for mining software repositories
in the context of software evolution,” Journal of Software
Maintenance and Evolution: Research and Practice, vol. 19,
no. 2, pp. 77–131, 2007.

[10] M. Askari and R. Holt, “Information theoretic evaluationof change prediction models for large-scale software,” inProceedings of the 2006 international workshop on Mining
software repositories, 2006, pp. 126–132.

[2] A. E. Hassan, “The Road Ahead for Mining Software Repositories,” in Proceedings of the Frontiers of Software Maintenance, 2008, pp. 48–57.

[11] H. Kagdi, J. I. Maletic, and B. Sharif, “Mining software
repositories for traceability links,” in Proceedings of the 15th
International Conference on Program Comprehension, 2007,
pp. 145–154.

[3] R. Moser, W. Pedrycz, and G. Succi, “A Comparative Analysis of the Efﬁciency of Change Metrics and Static Code
Attributes for Defect Prediction,” in Proceedings of the 30th
International Conference on Software Engineering, 2008, pp.
181–190.

[12] E. Giger, M. Pinzger, and H. C. Gall, “Can We Predict Typesof Code Changes? An Empirical Analysis,” in Proceedings of
the 9th Working Conference on Mining Software Repositories,
2012, pp. 217–226.

[4] H. Hata, O. Mizuno, and T. Kikuno, “Bug Prediction Basedon Fine-Grained Module Histories,” in Proceedings of the34th International Conference on Software Engineering,
2012, pp. 200–210.

[13] A. T. Ying, G. C. Murphy, R. Ng, and M. C. Chu-Carroll,“Predicting Source Code Changes by Mining Change History,” IEEE Transactions on Software Engineering, vol. 30,
no. 9, pp. 574–586, 2004.

[5] Y. Kamei, S. Matsumoto, A. Monden, K. Matsumoto,B. Adams, and A. E. Hassan, “Revisiting Common Bug
Prediction Findings Using Effort-Aware Models,” in Proceedings of the 26th International Conference on Software
Maintenance, 2010, pp. 1–10.

[14] F. Rahman, C. Bird, and P. Devanbu, “Clones: What is that
Smell?” in Proceedings of the 7th Working Conference on
Mining Software Repositories, 2010, pp. 72–81.
[15] B. Fluri, M. Wuersch, M. Pinzger, and H. Gall, “Change distilling: Tree differencing for ﬁne-grained source code change
extraction,” IEEE Transactions on Software Engineering,
vol. 33, no. 11, pp. 725–743, 2007.

[6] H. Zhang, “An Investigation of the Relationships between
Lines of Code and Defects,” in Proceedings of the 25th
International Conference on Software Maintenance, 2009, pp.
274–283.

[16] D. Kawrykow and M. P. Robillard, “Non-essential changes
in version histories,” in Proceedings of the 33rd International
Conference on Software Engineering, 2011, pp. 351–360.

Table IIR ATE OF LOC S INCLUDED IN MODIFICATIONS OF FORMAT AND
COMMENT(a) Columba
formatcomment
4.7%
2.6%
18.2%
13.7%
16.9%
18.7%
13.4%
14.2%
5.4%
8.9%

total
7.3%
31.9%
35.6%
27.6%
14.1%

(b) SQuirreL SQL Client
group
formatcomment
top 20%
6.4%
3.8%
21-40%
14.3%
11.8%
41-60%
12.2%
13.4%
61-80%
13.4%
16.5%
81-100%
7.2%
16.5%

total
10.3%
26.1%
25.6%
29.9%
23.7%

group
top 20%21-40%
41-60%
61-80%
81-100%[17] S. Hayashi, T. Omori, T. Zenmyo, K. Maruyama, and
M. Saeki, “Refactoring edit history of source code,” inProceedings of the 28th International Conference on Software
Maintenance, 2012, pp. 617–620.
[18] L. Hattori and M. Lanza, “On the nature of commits,” inProceedings of the 23rd International Conference on Automated Software Engineering - Workshop Proceedings (ASE
Workshops 2008), 2008, pp. 63–71.
[19] A. Hindle, D. M. German, and R. Holt, “What do largecommits tell us?: a taxonomical study of large commits,” inProceedings of the 3rd International Workshop on Mining
Software Repositories, 2008, pp. 99–108.

24