Empirical Analysis of Fault-proneness in Methods
by Focusing on their Comment Lines
Hirohisa Aman

Sousuke Amasaki

Takashi Sasaki and Minoru Kawahara

Center for Information Technology
Ehime University
Matsuyama, Japan 790–8577
Email: aman@ehime-u.ac.jp

Faculty of Computer Scienceand Systems Engineering
Okayama Prefectural University
Soja, Japan 719–1197
Email: amasaki@cse.oka-pu.ac.jp

Center for Information Technology
Ehime University
Matsuyama, Japan 790–8577
Email: sasaki@cite.ehime-u.ac.jp
Email: kawahara@cite.ehime-u.ac.jp

Abstract—This paper focuses on comments described in Java
programs, and conducts an empirical analysis about relationships between comments and fault-proneness in the programs.
The types of comments analyzed in this paper are comments
described inside a method body (inner comments), and comments
followed by a method declaration (documentation comments).
Although both of them play important roles in the program
comprehension, they seem to be described in different purposes;
The inner comments are often added to present tips about codefragments, while the documentation comments usually work as
a programmer’s manual. In the field of code refactoring, wellwritten inner comments are said to be related to “code smell”
since they may cover a lack of readability in a complicated codefragment. This paper analyzes the associations of comments withthe code quality from the aspect of fault-proneness, with using
four popular open source products. The empirical results showthat a method having inner comments tends to be 1.8 – 3.0 times
likely to be faulty. The key contribution of this work is to reveal
the usefulness of inner comments to point at faulty methods.

I.

I NTRODUCTION

Programming is one of the most basic activities in software development, and a successful quality management ofprogramming is necessary to develop a high-quality softwaresystem. Except for the cases that the programs can be automatically generated from their formal specifications, the programming activities are done by human beings (programmers).
That is to say, programmers’ minds and experiences about the
programs being developed would have much influence on thequality of their products.
The most important products produced by programmers
are sequences of executable statements to implement functions
required in the target system. Programmers often describe
some comments along with executable statements in theirsource files as well. Since those comments are eliminated orignored when the source files are compiled or interpreted,any comments are independent of the software functionality
and performance. However, in many cases, comments may be
helpful in understanding the program [1], [2]. Comments areimportant documentation artifacts [3], and have a certain levelof impact on the software quality.
In general, comments improve the readability of programs,so they are harmless to the software quality. However, comments may sometimes be added to compensate for the lack ofreadability in complicated programs. Some programmers wantto add in-depth comments to their code fragments which are
hard to be understood by other developers. In other words, anappearance of well-written comments might reflect a lack ofconfidence about the program clearness in the programmer’s
mind. In such cases, Kernighan et al. [4] recommended rewriting such confusing code rather than adding careful comments
onto those parts. In the field of code refactoring, well-writtencomments are known as artifacts related to “code smells”(signs of poor-quality programs to be refactored) [5]. While
well-written comments themselves are harmless, they can playroles as “deodorant” beside bad source code. These pros andcons of well-written comments motivated our study.
In this paper, we focus on comments described for methods
in source files, and conducts an empirical work using someopen source products written in Java to statistically analyze thefault-proneness of well-commented methods. The key contribution of this paper is to show a notable relationship between
the fault-proneness and the commenting manner in methods.
The remainder of this paper is organized as follows: Section
II describes the comments of interest in this paper. Section III
presents our empirical study using four popular open source
software projects, and discusses the results. Section IV gives
brief descriptions of related work. Finally, Section V describes
our conclusions and future work.
II.

TARGET C OMMENT T YPESThis section defines comment types to be analyzed in thefollowing section.
There are various types of comments described in a source
file. For instance, we often see the following types of comments [6]:•A property information including the file name, copyright notice, version(s), author(s), change history, etc.

•

A programmer manual explaining how to use the program, or how to use and/or call the function/method.

•

A programmer note giving an information about thedata structure and algorithm, or a tip to understand thecode fragment—how the code fragment works, what
the variable is, why the programmer wrote the codefragment, etc.

•

A comment disabling a code which is referred to as“comment out.”•A comment to represent a separator line between
functions/methods.

Since the aim of this paper is to analyze the impacts ofcomments on the fault-proneness, we focus on the part describing the sequence of executions, i.e. the functions/methods.
Then, we will focus on the following two types of comments
which are directly related to function/method implementations(see Fig. 1 for example):
1)

Documentation comments: comments followed bya function/method declaration. Documentation comments usually provide the information about how to
use the function/method. Those comments are known
as Javadoc in Java. Notice that, as shown in Fig. 1,
this paper does not limit the documentation comments
to Javadocs (/∗∗ · · · ∗/ style comments). Indeed, someJava products provide their documentation comments
in different styles using “// · · ·” type comments or “/∗· · · ∗/” type comments.

public class Example{/** [1. Documentation comments]* Returns true if this set contains no elemements.
* @return true if this set contains no elemements*/
public boolean isEmpty(){// [2. Inner comments]// check if it’s null, firstif ( ary == null ) return true;
if ( size == 0 ) return true;
return false;}// [1. Documentation comments]// Reinitialize this set.
public void reInit(){......



Fig. 1.

2)

Inner comments: comments written inside a function/method body, except for comment outed code1 .
Inner comments are often programmers’ notes that
explain the contents. Some programmers want to add
inner comments when they feel their code fragments
become complicated. This type of comment would be
related to the “code smell” and “deodorant” problem
mentioned in Sect.I.

Although the other types of comments would provide
useful information for programmers, the focus of this paper
is only on the above two types; Further analysis using othertypes of comments would be our future work.



Examples of comments in Java.
R ESEARCH OBJECTS .

TABLE I.
product

domain#filessize (KLOC)

Eclipse Checkstyle Plug-in4
Hibernate(O/R Mapping functionality)5
PMD6
SQuirreL SQL Client7

coding style checker

239

21.1

O/R mapper
code analyzer
SQL client

7501
1132
3770

522.1
68.8
387.5

We collected these four open source products from SourceForge.net3, which is one of the most popular open source
software development community sites. They were ranked in
the top 20 of Java products at SourceForge.net.
A. Research QuestionsIII.

E MPIRICAL S TUDYThis section presents our empirical work, results and discussions. Our research objects are four popular open source
products shown in Table I. We selected different sized products
from different domains for the generality of our empiricalresults. We selected them to satisfy the following all requirements: They are 1) open source products, 2) written in Java,and 3) maintained with Git.
Requirement 1) is essential to analyze all comments appearedin all source files. Requirement 2) is from our data collection
tools2 : JavaMethodExtractor, CommentCounter, and
LOCCounter. Requirement 3) is for the convenience of
our fault-data collection scheme. Since Git provides various
powerful functions for analyzing its repositories, it is easier toperform our empirical analysis than using conventional version
control systems such as CVS and Subversion. For example,“git log” command can easily and fast extract commit logs
which specific keywords (e.g. “bug”) appeared in.
1 It is hard to perfectly discriminate the comment-outed code from the normal comments (non comment-outed code). A perfect discrimination requires
thorough follow-up investigations for each comment, for each programmer.
However, it would be impossible in practice, so we use Aman’s algorithm [7]instead. While Aman’s algorithm is a simple algorithm, the most of comments(98.9%) can be successfully discriminated.
2 http://se.cite.ehime-u.ac.jp/tool/

We have a hypothesis that a more complicated, less quality
program requires more additional information for the comprehension, so such a problematic program must need somecomments to compensate for the lack of its comprehensibility.
We designed our empirical work on the following researchquestions:
RQ1:

Do the fault-proneness of methods significantly
differ by the commenting style?
If there is a significant difference in the faultproneness, our comment categorization and analysis are worthwhile.

RQ2:

Does the amount of comments have any relationships on the fault-proneness?
If they have remarkable relationships with faultproneness, it is valuable to care about the amountof comments during the development or code
reviewing.

3 http://sourceforge.net/
4 http://eclipse-cs.sourceforge.net/
5 http://hibernate.org/
6 http://pmd.sourceforge.net/
7 http://www.squirrelsql.org/

B. Metrics and Data Collection

TABLE II.

To quantitatively analyze relationships between comments
and fault-proneness in methods, we collect related data foreach method by using the following metrics:••

Lines of Inner comments (LOI)
The number of lines giving inner comments in themethod.

product
Eclipse Checkstyle
Hibernate
PMD
SQuirreLproduct
Eclipse Checkstyle
Hibernate
PMD
SQuirreL

2)

3)

Obtain the latest version of the source files, andmake the complete list of methods appeared in thesource files, except for any abstract methods; We
used our tool JavaMethodExtractor to extract
the methods. Table II shows the number of methods
in each product.
For each method, collect its change history. In theconcrete, examine whether the method is changed ornot, by checking all commits corresponding to thesource file which the target method is declared in.
Let a method be faulty if one of its changes was a
bug fixing; We decided a change to be a bug fixing
when the corresponding commit’s log includes oneof bug-fix-related words—“bug, fix, defect.” Table IIshows the numbers of faulty methods and non-faulty
methods.
For each method of its initial version, collect LOI
value and LOD value. Tables III and IV show the
distributions of LOI values and LOD values: the minimum, the 25 percentile, the median, the 75 percentile
and the maximum of metric values. These tables
reveal that there are many non-commented methods
in our target products. The ratios of non-commented
methods are shown in Table V. From Table V “thepresence or absence” of comments can be a key
category for observing the commenting styles, forexamining RQ1.

C. Analytical Procedures
We analyze our empirical data through the following manner.
1) Analysis for RQ1
Our RQ1 is “Do the fault-proneness of methods significantly differ by the commenting style?” To examine thisquestion, we conduct our empirical analysis by the following
two steps.

non-faulty
971
18, 3213, 8215, 802

faulty
80
162
149
314

min
0
0
0
0

25%
0
0
0
0

median
0
0
0
0

75%
1
0
0
0

max
26
42
69
36

TABLE IV.
D ISTRIBUTIONS OF LOD VALUES :MINIMUM , 25 PERCENTILE , MEDIAN , 75 PERCENTILE AND MAXIMUM .
product
Eclipse Checkstyle
Hibernate
PMD
SQuirreL

Then, we collected the change history of each source filefrom Git, and extracted the change history of each method in
the file. This activity consists of the following three steps:1)#methods
1, 051
18, 483
3, 9706, 116TABLE III.
D ISTRIBUTIONS OF LOI VALUES :MINIMUM , 25 PERCENTILE , MEDIAN , 75 PERCENTILE AND MAXIMUM .

Lines of Documentation comments (LOD)
The number of lines providing documentation comments for the method.

Metrics LOI and LOD are for capturing the amount of comments in a method. For example, method isEmpty in Fig.1
has LOI = 2 and LOD = 4.

N UMBERS OF METHODS , NON - FAULTY METHODS AND
FAULTY METHODS .

min
0
0
0
0

25%
3
0
0
0

median
3
0
0
0

75%
5
0
3
3

max
17
49
53
95

TABLE V.
R ATIOS OF METHODS HAVING NO INNER COMMENTS (LOI
= 0) AND ONES HAVING NO DOCUMENTATION COMMENTS (LOD = 0).
product
Eclipse Checkstyle
Hibernate
PMD
SQuirreL

LOI = 0

LOD = 0724
= 0.6891051
16248= 0.87918483
3583= 0.9033970
5467= 0.8946116

197= 0.1871051
15008= 0.81218483
2891= 0.7283970
3986= 0.6526116

1-1)

Divide the set of all methods into subsets by theircommenting styles. From the results of our datacollection (see Tables. III, IV and V), categorize
the methods into two subsets—“non-commented
methods” and “commented methods,” for innercomments (“LOI = 0” vs. “LOI > 0”) anddocumentation comments (“LOD = 0” vs. “LOD> 0”), respectively.

1-2)

For each comment type (inner comments anddocumentation comments), compare the faultproneness of methods between subsets, by performing the following statistical test (χ2 test)8 :[Null hypothesis]
The fault-proneness of non-commented methods
is at the same level as that of commented methods;[Alternative hypothesis]
The fault-proneness of non-commented methods
differs from that of commented methods.
Note that the fault-proneness of methods is quantified using the ratio of faulty methods in the subset.

8 This type of statistical test can be performed by using prop.testfunction in R (http://www.r-project.org/).

TABLE VI.

R ATIOS OF FAULTY METHODS AND STATISTICALSIGNIFICANCES OF THEIR DIFFERENCES (LOI).
LOI = 0product
Eclipse Checkstyle

Hibernate

PMD

SQuirreL38724(0.0524)115
16248(0.0071)1193583(0.0332)259
5467(0.0474)

vs
<

<

<

<

LOI > 0significance42327(0.1284)47
2235(0.0210)30387(0.0775)55649(0.0847)

TABLE VII.

R ATIOS OF FAULTY METHODS AND STATISTICALSIGNIFICANCES OF THEIR DIFFERENCES (LOD).
LOD = 0product***

Eclipse Checkstyle***

Hibernate

***

PMD

***

SQuirreL

12
197(0.0609)137
15008(0.0091)
237
2891(0.0439)198
3986(0.0497)

vs>

LOD > 0significance68854(0.0796)25
3475(0.0072)22
1079(0.0204)
116
2130(0.0545)***( significance: “***” means p-value < 0.001 )( significance: “***” means p-value < 0.001 )If the above statistical test shows a significant differencein the fault-proneness of methods, our comment categorization
and analysis are considered to be worthwhile.

Table VI and Fig. 2). These differences in the ratios of faulty
methods are statistically significant (at α = 0.001). Therefore,
there is a trend which a method with inner comments is more
faulty than a non-commented method.

2) Analysis for RQ2
Our RQ2 is “Does the amount of comments have any
relationships on the fault-proneness?” To answer this question,
we analyze the changes of fault-proneness in methods withincreasing lines of comments, through the following two steps.
2-1)

2-2)

Categorize the set of commented methods into
finer subsets, e.g. the set of methods whose LOI= 1, the set of methods whose LOI = 2, and soon.
Compare the ratios of faulty methods in the subsets, and check the changes in the ratio of faulty
methods to explore if the amount of comments
has an impact on the fault-proneness.
It is ideal to perform statistical tests like “step1-2)” in this step as well. However, by dividing
the set of commented methods into finer subsets
such as LOI = 1 and LOI = 2, the numberof methods belonging to each category becomes
smaller. Therefore, no proper conclusion could bederived from a statistical test on such a small setof methods.

If there is a clear trend of the fault-proneness over the lines
of comments, the amount of comments would be noteworthy
during the development or code reviewing.
D. Results and Discussions
We present analytical results and discussions about our
research questions in the followings.
1) RQ1: Do the fault-proneness of methods significantly differ
by the commenting style?
Tables VI and VII, and Figs. 2 and 3 show ratios of faulty
methods and statistical significances of their differences, by
LOI and LOD, respectively.
For all products, the methods having one or more innercomments (LOI > 0) are about 1.8 – 3.0 times likely to befaulty than the ones having no inner comments (LOI = 0) (seeThese results support our hypothesis that a more complicated, less quality program requires more additional information for the comprehension. Furthermore, the results are
consistent with one of the “code smells” that comments added
to programs may work as “deodorant” beside poor quality code
to be refactored [5].
The presence or absence of documentation comments did
not show any specific tendency: while PMD showed a statistically significant difference in the ratio of faulty methods,
the remaining products did not so (see Table VII and Fig.
3). Although documentation comments may provide useful
information about the method, our empirical data did not show
strong association with the fault-proneness. Further analysis of
documentation comments will be our key future work.
In general, a larger program is more faulty, so we must
consider the effect of code size (LOC) on the fault-pronenessas well. We thus perform the logistic regression analysis withusing all of LOC, LOI and LOD, to take their impacts apart:Pr( mi is faulty | xi ) =1

1 + e−y(xi ),

y(xi ) = β0 + β1 LOCi + β2 LOIi + β3 LODi ,

(1)
(2)

where mi is the i-th method and xi is the vector of metricvalues in mi , xi = (LOCi , LOIi , LODi ); LOCi signifies LOC
value of mi ; LOIi and LODi are binary variables about mias follows:
0 (LOI = 0 in mi ),LOIi =
1 (LOI > 0 in mi ),
0 (LOD = 0 in mi ),LODi =
1 (LOD > 0 in mi ).
Equations (1) and (2) express the logistic regression model
which provides the probability of that “mi is faulty” whenthree metric values (= xi ) are given. In Eq. (2), β0 , β1 , β2and β3 are constants, and β1 , β2 and β3 mean the impact levelof LOCi , LOIi and LODi on the probability (fault-proneness),
respectively. If βj is not zero, the corresponding metric hasa significant impact on the probability computation (for j =1, 2, 3).

0.00
LOI=0

LOI=0

0.06
0.04
0.02
0.00

LOI>0

LOI=0

LOI
(b) Hibernate

0.075
0.050
0.025
0.000

LOI>0

LOI=0

LOI
(c) PMD

LOI>0

LOI(d) SQuirreL

0.050
0.025
0.000

0.01000.00750.00500.00250.0000

LOD>0

LOD=0

LOD
(a) Eclipse CheckStyle

0.05

ratio of faulty methods

0.075

0.0125

ratio of faulty methods

ratio of faulty methods

ratio of faulty methods

0.00

0.08

Ratios of faulty methods: LOI= 0 vs. LOI> 0.

LOD=0

Fig. 3.

0.01

LOI>0

LOI
(a) Eclipse CheckStyle

Fig. 2.

0.02

ratio of faulty methods

0.05

ratio of faulty methods

ratio of faulty methods

ratio of faulty methods

0.10

0.03

0.04
0.03
0.02
0.01
0.00

LOD>0

LOD=0

LOD
(b) Hibernate

0.06

0.04

0.02

0.00

LOD>0

LOD=0

LOD
(c) PMD

LOD>0

LOD(d) SQuirreL

Ratios of faulty methods: LOD= 0 vs. LOD> 0.

Eclipse Checkstyle
Hibernate
PMD
SQuirreL

Intercept
(β0 )
−3.25
(***)
−4.92
(***)
−3.39
(***)
−3.16
(***)

LOC
(β1 )
0.00
()
0.01
(*)
0.03
(***)
0.01
(***)

LOI> 0
(β2 )
1.01
(***)
1.01
(***)
0.26
()
0.34
(*)

LOD> 0
(β3 )
0.41
()
−0.34
()
−0.90
(***)
0.08
()

The result of logistic regression analysis (Table VIII)
signifies that the comparisons in Tables VI and VII are notdominated by the code size (LOC). That is to say, it is worthwhile to see associations of comments with fault-proneness.
2) RQ2: Does the amount of comments have any relationships
on the fault-proneness?
The results for RQ1 showed the trends that the presenceof inner comments may be related to the fault-proneness inmethods. Then, we focus on the amount of inner comments toanswer to RQ2. Since most of LOI values in our data sets are

1

2
3
LOI

>=4

(a) Eclipse Checkstyleratio of faulty methods
0.00 0.04 0.08 0.12

( significance: “***” means p-value < 0.001; “*” means p-value < 0.05 )Table VIII shows the results of logistic regression analysis.
The asterisk(s) indicate the level of statistical significance ofthat the corresponding constant βj is not zero (for j = 1, 2, 3).
For each Eclipse Checkstyle, Hibernate and SQuirreL, thepresence of inner comments (LOI > 0) has a significant
positive impact on the fault-proneness, even though LOC isalso one of the explanatory variables. On the other hand,for PMD, the presence of inner comments has no significant
impact, but the presence of documentation comments has asignificant negative impact on the fault-proneness. This result
is corresponding to the result in Table VII and Fig. 3.

0ratio of faulty methods
0.00
0.02
0.04

product

0

1

2
3
LOI

(c) PMD
Fig. 4.

>=4

0

1

2
3
LOI

>=4

(b) Hibernate
ratio of faulty methods
0.00 0.04 0.08 0.12

R ESULTS OF LOGISTIC REGRESSION ANALYSIS :COEFFICIENTS AND THEIR SIGNIFICANCES .

ratio of faulty methods
0.00
0.10
0.20

TABLE VIII.

0

1

2
3
LOI

>=4

(d) SQuirreL

Changes in the ratio of faulty methods by LOI.

less than 4, we examined the ratios of faulty methods for LOI= 0, 1, 2, 3 and LOI ≥ 4.
Figure 4 shows the results. The ratios of faulty methods
monotonically increase from LOI = 0 to LOI = 2. The ratios
in LOI = 3 and LOI ≥ 4 are not changed monotonicallybut both of them keep higher levels than LOI = 0. Fromthese results, we can conclude that more inner comments do
not result in higher fault-proneness. Nevertheless, the presenceof inner comments is associated with the fault-proneness inmethods.

While some previous work revealed that well-commented
source files are likely to be faulty, those results were derived
from a rough-grained analysis at source file level. We performed a finer-grained analysis at method level, and found thateven one or two inner comments can point to faulty methods.
This is a novel finding about the relationships between comments and fault-proneness in methods.
E. Threats to Validity
While our empirical work used popular open source products, the other products written in languages other than Java
were not examined. Since our data collection tools can workfor only Java, our empirical work was limited to Java products.
However, our metrics and empirical analysis approach areavailable when the target product is written in C, C++ or C#
without any modifications. In many cases, writing program
elements (statements and conditions) is common among Java,C, C++ and C#, so the difference in language might notinvalidate the contribution of this paper.
Since the programming activities for commercial products are usually managed by their companies/organizations,
their commenting styles would depend on the companies/organizations, so they may bring different trends of comments. Further study on commercial products is one of ourimportant future work.
Our fault data collection was based on a keyword matchingin commit logs, so we might miss some of the true faulty
methods. However, many studies analyzing code repositories
have adopted such a keyword matching approach in the past,
and we also followed those manner to detect bug fixing commits. We would like to use a richer data-collection mechanism
collaborating with a bug tracking system in our future work.
Our analysis was based on the number of comment lines,
not on the contents of comments. The consistency between
a program and corresponding comments [6] may affect our
results. While “what the inner comments say” would be
an important factor, we empirically showed that even “thepresence or absence” of inner comments has a significant
impact on the fault-proneness in this paper.
IV.

comments as signs of parts to be refactored. Moreover, theydescribed that the existence of long inner comments may
infer a lack of external documents, so adding many innercomments were not recommended. While they studied valuable
relationships between comments and source code, their focus
was on the quality of comments, but not on the code quality.
Aman [8], [7] conducted some empirical studies on relationships between the amount of comments and faultproneness in source files. Those empirical studies reported
that well-commented programs were likely to be faulty. Those
reports are our previous work, but they focused only on the
inner comments and analyzed the fault-proneness at the source
file level. This paper studies not only the inner comments but
also the documentation ones, and performs finer analysis at
the method level rather than the source file level9 .
V.

This paper focused on two types of comments related tomethods—inner comments and documentation comments—.
We conducted an empirical analysis using four popular open
source products written in Java, to examine the relationships
between comments and fault-proneness in their methods. Our
empirical results revealed a novel finding that even one or
two inner comments can point to faulty methods. That is
to say, comments written inside a method body may tell us
complicated code fragments that we should check carefully.
Our future work includes further analysis using various
products from different domains and organizations, and written in other languages, time series analysis of relationships
between comments and fault-proneness, and further analysis
focusing on contents of comments.
ACKNOWLEDGMENTS
This work is supported by JSPS KAKENHI Grant #25330083.
R EFERENCES[1][2]

R ELATED W ORK[3]Tenny [1] and Woodfield et al. [2] conducted experiments
about program comprehension, with students and experienced
programmers as their subjects, respectively. In their experiments, the subjects evaluated the ease of understanding somevariations of a program which were different in modular design
and in commenting manner (with or without comments). Theirexperimental results showed that comments have a positive
impact on program comprehension. However, they did notevaluate the impacts of comments on fault-proneness.
Steidl et al. [6] proposed a framework to evaluate thequality of comments. They evaluated the coherence between
the documentation comments (which are called “member comments” in [6]) and the name of the corresponding method.
Their coherence evaluation presents whether the documentation comments provide useful information about the method.
They discussed the inner comments (which were called “inline
comments” in [6]) as well, and proposed to use short inner

C ONCLUSIONS

[4]
[5]
[6]

[7]

[8]

T. Tenny, “Program readability: Procedures versus comments,” IEEE
Trans. Softw. Eng., vol. 14, no. 9, pp. 1271–1279, Sep. 1988.
S. N. Woodfield, H. E. Dunsmore, and V. Y. Shen, “The effect ofmodularization and comments on program comprehension,” in Proc. 5thInt’l Conf. Softw. Eng., San Diego, California, 1981, pp. 215–223.
S. C. B. de Souza, N. Anquetil, and K. M. de Oliveira, “A study of the
documentation essential to software maintenance,” in Proc. 23rd Int’l
Conf. Design of Communication, Coventry, UK, Sept. 2005, pp. 68–75.
B. W. Kernighan and R. Pike, The practice of programming. Boston,
MA: Addison-Wesley Longman, 1999.
M. Fowler, Refactoring: Improving the Design of Existing Code. Boston,
MA: Addison-Wesley Longman, 1999.
D. Steidl, B. Hummel, and E. Juergens, “Quality analysis of sourcecode comments,” in Proc. 21st Int’l Conf. Program Comprehension, San
Francisco, California, May 2013, pp. 83–92.
H. Aman, “An empirical analysis on fault-proneness of well-commented
modules,” in Proc. 4th Int’l Workshop Empir. Softw. Eng. in Practice,
Osaka, Japan, Oct. 2012, pp. 3–9.
——, “An empirical analysis of the impact of comment statements on
fault-proneness of small-size module,” in Proc. 19th Asia-Pacific Softw.
Eng. Conf., Hong Kong, Dec. 2012, pp. 362–367.

9 The basic idea of this work was presented at the poster session of
the 8th International Symposium on Empirical Software Engineering and
Measurement (ESEM 2014), Sept.18-19, 2014 in Torino, Italy.