2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering

Do Developers Focus on Severe Code Smells?
Tsubasa Saika∗ , Eunjong Choi∗ , Norihiro Yoshida† , Shusuke Haruna∗ and Katsuro Inoue∗∗ Osaka University, Japan{t-saika@ist, ejchoi@osipp, haruna@ist, inoue@ist}.osaka-u.ac.jp† Nagoya University, Japan
yoshida@ertl.jp

Abstract—Code smells are structures in the code that suggest
the possibility of refactoring. To prioritize code smells in largescale source code, several tools for refactoring calculate theirseverity based on software metrics. Although several metrics are
known as maintainability indicators, it is still unclear whether
these severity indicators are in line with developer’s perception.
In this paper, we investigate whether developers focus on severe
code smells. The result shows that developers focus on only
particular types of severe smells and refactoring do not decrease
the severity of code smell signiﬁcantly.

opportunities. Also, if developers decreased the severity ofcode smells by refactoring in the OSS systems, the severity isin line with developers’ perception of maintainability.
We answer the following research questions :RQ1 Do developers perform refactoring more frequently
on code with more sever code smell?
RQ2 Does refactoring decrease the severity of codesmells?
Our ﬁndings from the RQs are as follows:

I. I NTRODUCTION

Developers focus on particular types of severe smells.
Therefore, the severity-based prioritization is a usefulapproach to ﬁnd the paticular type of refactoring opportunities.
• Refactoring do not decrease the severity of code smell
signiﬁcantly. Therefore, the severity is not in line with
developer’s perception of the maintainability.
•Developers tend to write source code with low maintainability even when they can easily avoid it. (e.g., because they arein a hurry or implementing urgent patches or simply makingsuboptimal choices) [1].
Code smells (also known as bad smells) are symptomsof poor design and implementation choices that may hindercode comprehension and possibly increase change and faultproneness. They are also used to ﬁnd structures in the codethat suggest the possibility of refactoring [2]. For example,
Blob Class [2] is a large and complex class that centralizes
the behavior of a portion of a system and only uses otherclasses as data holders. It can rapidly grow out of control,
making it harder to ﬁx bugs and add new features [1].
Many code quality analysis tools can detect code smell
in source code automatically (e.g., inFusion, PMD). Since
a number of code smells are detected by these tools inlarge-scale source code, developers must determine whichcode smells should be preferentially removed [3]. However,each code smell indicates only the location and the kind ofsymptoms but not the the severity.
To prioritize code smells in large-scale source code, several tools (e.g., inFusion, inCode) calculate their severity
using software metrics as quantitative indicators of software
maintainability. Although several metrics are used for not only
maintainability indicators [4], [5] but also the identiﬁcation of
refactoring opportunities [6], it is still unclear whether theseseverity indicators are in line with developer’ perception. If
the severity is not in line with developers’ perception, severitybased prioritization is inappropriate to support refactoring.
In this study, we investigate the refactorings that wereperformed by developers in three mature and large-scale OSSsystems. If developers preferentially performed refactoring oncode with high severity in the OSS systems, the severitybased prioritization is a useful approach to ﬁnding refactoring978-1-5090-1855-0/16 $31.00 © 2016 IEEE
DOI 10.1109/SANER.2016.117

II. DATASETS OF R EFACTORINGS AND C ODE S MELLSIn this study, we analyzed datasets of refactorings andcode smells that were detected in 64 releases of three JavaOSS systems: Apache Ant (Ant)1 , ArgoUML (Argo)2 , and
Xerces-J (Xerces)3 . Table I shows statistical data on eachof the systems. In this table, the last row shows the overall
number of three systems. In the following subsections, we
explain the dataset of refactorings (Section II-A) and theextracted code smells (Section II-B).
A. Dataset of Refactorings
We used the dataset of refactorings that was collected ina previous study by Bavota et al. [7]. This dataset contains
refactorings that were detected by Ref-ﬁnder [8], a templatedbased refactoring detection tool. Each refactoring was manually validated to exclude false positives. From the dataset, wecan observe distinct types, modiﬁed class names, and release
version of each refactoring. We selected this dataset because
its refactorings are reliable.
Table II depicts the total number of analyzed refactorings
and the number of distinct types of refactoring in the dataset.
1 http://ant.apache.org/
2 http://argouml.tigris.org/
3 http://xerces.apache.org/xerces-j/

1

TABLE IS TATISTICS DATA OF A NALYZED S YSTEMS
SystemXercesArgoAnt
Overall

PeriodOct. 2003 - Nov. 2006
Oct. 2002 - Apr. 2011
Aug. 2003 - Dec. 2010
-Analyzed
1.0.0-2.9.0
0.10.1-0.32.2
1.1-1.8.2-# Refactorings
7,502
3,255
1,289
12,043# Distinct Types
24211628

Blob Class
Data Class
Distorted Hierarchy
God Class
Refused Parent Bequest
Schizophrenic Class
Tradition Breaker

TABLE IIIN UMBERS OF D ETECTED C ODE S MELLSType of Code Smell
Blob Class
Data Class
Distorted Hierarchy
God Class
Refused Parent Bequest
Schizophrenic Class
Tradition Breaker
Classes w/ smells
Classes w/o smells
Overall classes

Xerces
66571
995211439991,949
17,61819,567Argo
8330160
14483311
43,37543,686Ant87280143
8140343
22,42522,768# Classes
19,567
43,686
22,76886,021TABLE IVR ESULTS OF M ANN -W HITNEY U- TEST FOR RQ1

TABLE IIOVERVIEW OF R EFACTORING DATASET
SystemXerces
Argo
Ant
Overall# Releases
34121864

All
835
102
91,255
209
91
102
2,603
83,41886,021

RQ1
n/a

RQ2

n/an/aIII. I NVESTIGATION AND R ESULTS
This section details our investigation and results, aimed
at addressing the RQ1 and RQ2. This section also identiﬁes
threats to validity of our results.
A. Do developers perform refactoring more frequently on codewith more severe code smell?
To answer RQ1, we investigated the relationship between
refactorings and the severity of code smells. We used theMann-Whitney U test [9], a nonparametric signiﬁcance test
to ﬁnd out whether refactored classes have more severe code
smells than non-refactored classes. The Mann-Whitney U
test was used to investigate the statistical signiﬁcant of thedifferences between two independent groups: refactored andnon-refactored classes that have at least one code smell. The
dependent variable is the severity of each code smell, which
is ordinal variable measured from 1 to 10.
The results of the Mann-Whitney U test can be seen in
Table IV. In this table, the check marks represent the existenceof signiﬁcant differences (p > .05) and ‘n/a’ indicates that
the Mann-Whitney U test was not applied. As we can see inthe table, there are signiﬁcant differences between refactoredclasses and non-refactored classes for Blob Class, God Class,Schizophrenic Class and Tradition Breaker. This implies thatdevelopers tend to perform refactoring more frequently oncode with more severe Blob Class, God Class, Schizophrenic
Class and Tradition Breaker.
Summary for RQ1: For Blob Class, God Class,
Schizophrenic Class and Tradition Breaker, there are statistical tendencies for developers to perform refactoring more
frequently on code with more severe code smell.

B. Dataset of Code Smells
For the detection of code smells, we used inFusion4 , a toolthat detects 24 types of code smells based on code metrics 5 .
To detect each type of code smell, inFusion uses pre-deﬁned
conditional expression consisting of logical combination ofcode metrics with different threshold values. For example,Schizophrenic Class is detected based on two metrics named
Number of Public Methods and Tight Capsule Cohesion. It
outputs detected code smells and their severity score on ascale of 1 to 10 based on how much relative metrics values
exceed the threshold value. We adopted inFusion because it
has ability to detect various types of code smells as well as tomeasure their severity.
From the detected code smells, we chose class as level ofgranularity because refactoring dataset only provides a classname with each refactoring. Table III shows the number ofclass-level code smells detected in the systems. In this table,
the last row shows the overall numbers of code smells in thesystems. As you can see in this table, only 7 class-level code
smells were detected. Because multiple code smell can bedetected in a single class, the overall number of classes might
contain more than one code smell.

B. Does refactoring decrease the severity of code smells?
To answer RQ2, we analyzed refactorings and the changesof the severity of code smell from the dataset. We conducted
the Mann-Whitney U test to reveal whether there are significant differences between the severity of refactored classes

4 http://www.intooitus.com/products/infusion
5 The details of the 24 types of code smells can be found at https://www.
intooitus.com/products/infusion/detected-ﬂaws

2

R EFERENCES

and non-refactored classes. The Mann-Whitney U test was
only applied to classes that have at least one code smell. The
dependent variable is the change of the severity of each code
smell through release versions which are ordinal variables
measured from -10 to 10. If a refactoring largely increases
or decreases the severity of code smells, the test must show
signiﬁcant differences.
The results of the Mann-Whitney U test can be seen in
Table IV. In this table, the check marks represent the existenceof signiﬁcant differences (p > .05) and ‘n/a’ indicates that
the Mann-Whitney U test was not applied because they areclasses without code smells or non-refactored classes. From
this table, we can observe signiﬁcant difference only from
Blob Class and God Class. This implies that in most case,the effect of refactoring on the changes of severity of code
smell are relatively small. Especially for Blob Class and God
Class, the severity of code smells were increased as well asdecreased, regardless of performance of refactoring. From thisresults, we can conclude that refactoring does not decrease theseverity of code smells.
Summary for RQ2: A comparison of refactored andnon-refactored classes shows that refactoring does not statistically decrease the severity of code smells.

[1] F. Palomba, G. Bavota, M. Di Penta, R. Oliveto, D. Poshyvanyk, andA. De Lucia, “Mining version histories for detecting code smells,” IEEE
Transactions on Software Engineering, vol. 41, no. 5, pp. 462–489, 2015.
[2] M. Fowler, Refactoring: Improving the Design of Existing Code. Addison
Wesley, 1999.
[3] A. Yamashita and L. Moonen., “Exploring the impact of inter-smell
relations on software maintainability: An empirical study,” in Proc. ofICSE, 2013, pp. 682–691.
[4] B. Henderson-Sellers, Object-Oriented Metrics: Measures of Complexity.
Prentice Hall, 1996.
[5] R. Marinescu and D. Ratiu, “Quantifying the quality of object-oriented
design: the factor-strategy model,” in Proc. of WCRE, 2004, pp. 192–201.
[6] F. Simon, F. Steinbruckner, and C. Lewerentz, “Metrics based refactoring,” in Proc. of CSMR, 2001, pp. 30–38.
[7] G. Bavota, B. D. Carluccio, A. D. Lucia, M. D. Penta, R. Oliveto, andO. Strollo, “When does a refactoring induce bugs? an empirical study,”
in Proc. of SCAM, 2012, pp. 104–113.
[8] M. Kim, M. Gee, A. Loh, and N. Rachatasumrit, “Ref-Finder: a refactoring reconstruction tool based on logic query templates,” in Proc. of FSE,
2010, pp. 371–372.
[9] W. J. Conover, Practical nonparametric statistics. John Wiley & Sons,
1971.

C. Threats to Validity
Some threats need to be considered when interpreting ourstudy results. The ﬁrst limitation was that the investigation
results might have been too dependent on the dataset andoutput of the inFusion. However, the dataset were validated
in the [7] and inFusion is a commercial code smell detection
tool . Therefore, we believe that the results of investigation inthis study are reliable.
Moreover, investigating different systems could have led todifferent results because our case study was conducted on three
Java OSS systems. However, we believe that our investigation
results can be generalized an applied to other open source
software systems because they spanned 64 release versions
from three separate systems.
IV. SUMMARY
To reveal whether developers focus on severe code smell,
this study investigated the effect of the severity of code smellson refactorings in 64 releases of three Java OSS systems.
Based on our investigation results, it turns out that developers perform refactoring more frequently on code with more
severe code smell for Blob Class, God Class, Schizophrenic
Class and Tradition Breaker. Therefore, it is considered thatdevelopers can use these severity of code smells as good
indicators for performing refactoring. It also revealed thatrefactorings do not decrease the severity of code smells
signiﬁcantly.
As future work, we plan to analyze additional softwaresystems to achieve the generality of our ﬁndings. In addition,
we would like to extend our investigation for more types ofcode smells such as method-level code smells.

3