Is Duplicate Code More Frequently Modified thanNon-duplicate Code in Software Evolution?:
An Empirical Study on Open Source Software
Keisuke Hotta1

Yukiko Sano1

Yoshiki Higo1

Shinji Kusumoto1

1 Graduate School of Information Science and Technology, Osaka University{k-hotta,y-sano,higo,kusumoto}@ist.osaka-u.ac.jp
ABSTRACT
Various kinds of research efforts have been performed on the basisthat the presence of duplicate code has a negative impact on software evolution. A typical example is that, if we modify a codefragment that has been duplicated to other code fragments, it isnecessary to consider whether the other code fragments have to bemodified simultaneously or not. In this research, in order to investigate how much the presence of duplicate code is related tosoftware evolution, we defined a new indicator, modification frequency. The indicator is a quantitative measure, and it allows us toobjectively compare the maintainability of duplicate code and nonduplicate code. We conducted an experiment on 15 open source
software systems, and the result showed that the presence of duplicate code does not have a negative impact on software evolution.

Categories and Subject Descriptors
D.2.8 [Software Engineering]: Metrics—Complexity measures,
Process metrics, Software science; D.2.9 [Software Engineering]:
Management—Productivity, Software quality assurance

General Terms
Measurement, Management

Keywords
Duplicate code, Software maintenance, Empirical study

1. INTRODUCTION
Recently, duplicate code is attracting a great deal of attention insoftware engineering. Duplicate code is generated by various reasons such as copy and paste programming or stereotyped code. The
presence of duplicate code makes it more difficult to maintain consistency of the code. For example, if a bug is found in a duplicate
code, we have to fix the same bug in its correspondents. In a book
dedicated to refactoring, Fowler stated that:
Number one in the stink parade is duplicate code. If you see thesame code structure in more than one place, you can be sure thatyour program will be better if you find a way to unify them [10].

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.
IWPSE-EVOL’10 September 20-21, 2010 Antwerp, Belgium
Copyright 2010 ACM 978-1-4503-0128-2/10/09 ...$10.00.

Various kinds of research efforts have been performed for resolving or improving the problems caused by the presence of duplicate
code [2]. For example, there are currently a variety of duplicate
code detection techniques available [5]. In addition, there are many
research efforts for merging duplicate code as a single function or
method, or for preventing duplications from being overlooked inmodification [8, 21]. However, only a few research efforts have investigated how much the presence of duplicate code actually has anegative impact on software evolution.
Monden et al. and Lozano et al. investigated the influence of
duplicate code on file unit or method unit, respectively [19, 20].
They found some case where the presence of duplicate code has
a negative impact on software maintenance. However, their investigations remain a matter of improvement. The units (files andmethods) are larger than duplicate code, so that it is possible thatmodifications are incorrectly counted. For example, if modifications are performed on a method where a fragment of duplicate
code exists, all the modifications are assumed as performed on the
duplicate code even if they are performed on non-duplicate code of
the method. Krinke compared stability of duplicate code and nonduplicate code [16]. In his experiment, a line-based barometer wasused. However, it was not possible that the barometer appropriately
indicates the cost of maintainability. That is, the indicator cannot
distinguish the following two cases: the first case is that consecutive 10 lines of code was modified for fixing a bug; the second caseis that 1 line modification was performed on different 10 pieces ofcode for fixing 10 different bugs. In real software maintenance, thelatter requires much more cost than the former because we haveto conduct several steps before the actual source code modificationsuch as identifying the buggy module, informing the maintainer
about the bug, identifying buggy instructions and so on.
The present paper proposes a new investigation method, which
compares duplicate code to non-duplicate code from a different
standpoint, and reports the experimental result on open source software. The features of the proposed method are as follows:• every line of code is investigated whether it is duplicate code
or not. Such a fine-grained investigation can accurately judge
whether every modification is conducted to duplicate code or
to non-duplicate code;• maintenance cost consists of not only source code modification but also several phases prior to the source code modification. In order to more appropriately estimate maintenancecost, we define an indicator that is not based on modifiedlines of code but the number of modified pieces;• we evaluate and compare modifications of duplicate code andnon-duplicate code on multiple open source software systems with multiple duplicate code detection tools. That isbecause, every detection tool detects different duplicate codefrom the same source code.
The remainder of this paper is organized as follows: Section 2
describes the previous studies on modification comparison between
duplicate code and non-duplicate code. Section 3 introduces detection tools that are used in this study, and Section 4 explains how wecompare modifications between duplicate code and non-duplicate
code. Section 5 shows the comparison result, and Section 6 discusses threat to validity of the experimental result. Finally, Section
7 concludes this paper.

2. RELATED WORKAt present, there is a huge body of work on empirical evidence
on code clones, starting with Kim et al.’s report on clone genealogies [15]. Their empirical studies on two open source software systems found 38% or 36% of groups of duplicate code were consistently changed at least one time. On the other hand, they observedthat there were groups of duplicate code that existed only for a shortperiod (5 or 10 revisions) because each instance of the groups was
modified inconsistently. Their work is the first empirical evidence
that a part of duplicate code increases the cost of source code modification.
However, Kapser and Godfrey have different opinions regarding
duplicate code, they reported that duplicate code can be a reasonable design decision based on the empirical study on two largescale open source systems [14]. They built several patterns of duplicate code in the target systems, and they discussed the pros andcons of duplicate code using the patterns. Bettenburg et al. also
reported that duplicate code does not have much a negative impact
on software quality [6]. They investigated inconsistent changes toduplicate code at release level on two open source systems. The
empirical study found that only 1.26% to 3.23% of inconsistentchanges introduced software errors into the target systems.
Monden et al. investigated the relation between software qualityand duplicate code on the file unit [20]. In their investigation, thenumber of revisions of every file was used as a barometer of quality:
if the number of revisions of a file is great, its quality is low. Theirexperiment selected a large scale legacy system, which was being
operated in a public institution, as the target. The result showed
that, modules that included duplicate code were 40% lower quality
than modules that did not include duplicate code. Moreover, theyreported that the larger duplicate code a source file included, thelower quality it was.
Lozano et al. investigated whether the presence of duplicate code
was harmful or not [18]. They developed a software tool, CloneTracker, which traces which methods include duplicate code (inshort, duplicate method) and which methods are modified in eachrevision. They conducted a pilot study, and the result showed that:
duplicate methods tend to be more frequently modified than nonduplicate methods; however, duplicate methods tend to be modified
less simultaneously than non-duplicate methods. The fact implies
that the presence of duplicate code increased cost for modification,and programmers were not aware of the duplication, so that theysometimes overlooked code fragments that had to be modified simultaneously.
Also, Lozano and Wermelinger investigated the impact of duplicate code on software maintenance [17]. Three barometers wereused in the investigation. The first one is likelihood, which indicates possibility that the method is modified in a revision. The second one is impact, which indicates the number of methods that are
simultaneously modified with the method. The third one is work,
which can be represented as a product of likelihood and impact(work = likelihood × impact). They conducted a case study on 4
open source systems for comparing the three barometers of methods including and not including duplicate code. The result was that:likelihood of methods including duplicate code was not so different from one of methods not including duplicate code; there weresome instances that impact of methods including duplicate code
were greater than one of methods not including duplicate code; ifduplicate code existed in methods for a long time, their work tended
to increase greatly.
Moreover, Lozano et al. investigated the relation between duplicate code, features of methods, and their changeability [19]. Changeability is the ease of modification. If changeability is decreased, it
will be a bottleneck of software maintenance. The result showedthat it is possible that the presence of duplicate code decreases
changeability. However changeability was more greatly affected byother properties such as length, fan-out, and complexity of methods. Consequently, they concluded that it was not necessary toconsider duplicate code as a primary option.
Krinke hypothesized that if duplicate code is less stable thannon-duplicate code, maintenance cost for duplicate code is greaterthan for non-duplicate code, and he conducted a case study in order to investigate whether the hypothesis is true or not [16]. The
targets are 200 revisions (a revision per week) of source code of 5large scale open source systems. He measured added, deleted, andchanged LOCs on duplicate code and non-duplicate code, and compared them. The result showed that non-duplicate code was more
added, deleted, and changed than duplicate code. Consequently, heconcluded that the presence of duplicate code did not necessarily
make it more difficult to maintain source code.
Eick et al. investigated whether source code decays when it isoperated and maintained for a long time [9]. They selected several
metrics such as the amount of added and deleted code, the timerequired for modification, and the number of developers as indicators of code decay. The experimental result on a 15-years-operated
large system showed that cost required for completing a single requirement tends to increase.
Göde modeled how type-1 code clones are generated and how
they evolved [11]. Type-1 code clone is a code clone that is exactly identical to its correspondents except white spaces and tabs.
He applied the model to 9 open source software systems and investigated how code clones in them are evolved. The result showed
that: the ratio of code duplication was decreasing as time passed;
the average life time of code clones was over 1 year; in the casethat code clones were modified inconsistently, there were a few
instances that additional modifications were performed to restore
their consistency.
As described above, some empirical studies reported that duplicate code should have a negative impact on software evolution
meanwhile the others reported that it should not. At present, there
is no consensus on the impact of the presence of duplicate code on
software evolution. Consequently, this research is performed as a
replication of the previous studies with solid settings.

3.

DUPLICATE CODE DETECTION TOOLSThere are currently various kinds of duplicate code detection
tools. The detection tools take the source code, and they provide
the position of the detected duplicate code in it. However, there isneither a generic nor strict definition of duplicate code. Each detection tool has its own unique definition of duplicate code, and it
detects duplicate code based on the own definition. Consequently,
different duplicate code is detected by different detection tools fromthe same source code.
The detection tools can be categorized based on their detection

1: A
2: B
3: line will be changed 1
4: line will be changed 2
5: C
6: D
7: line will be deleted 1
8: line will be deleted 2
9: E
10: F
11: G
12: H
(a) before modification

1: A
2: B
3: line changed 1
4: line changed 2
5: C
6: D
7: E
8: F
9: G
10: line added 1
11: line added 2
12: H(b) after modification

3,4c3,4< line will be changed 1< line will be changed 2--> line changed 1> line changed 2
7,8d6< line will be deleted 1< line will be deleted 2
11a10,11> line added 1> line added 2(c) diff output

Figure 1: A simple example of comparing two source files with diff (changed region is represented with identifier ‘c’ like 3,4c3,4,
deleted region is represented with identifier ‘d’ like 7,8d6, added region is represented with identifier ‘a’ like 11a10,11. The number
before and after the identifier shows the correspond lines)

techniques. Major categories should be line-based, token-based,
metrics-based, AST1 -based, and PDG2 -based. Each technique has
merits and demerits, and there is no technique that is superior toany other techniques in every way [5, 7]. The remainder of thissection describes 4 detection tools that are used in our experiment.

3.1 CCFinder
CCFinder is a token-based detection tool [13]. The major features of CCFinder are as follows:• CCFinder replaces user-defined identifiers such as variablenames or function names with special tokens before the matching process. Consequently, CCFinder can identify code fragments that use different variables as duplicate code.
• CCFinder detection speed is very fast. CCFinder can detects
duplicate code from millions lines of code within an hour.
• CCFinder can handle multiple popular programming languages
such as C/C++, Java, and COBOL.

3.2 CCFinderX

more duplicate code than traditional PDG. Also, Scorpio adopts
some heuristics for filtering out false positives. Currently, Scorpio
can handle only Java language.

4.

MEASURING MODIFICATIONFREQUENCY

This section proposes a method to estimate the influence of thepresence of duplicate code on software evolution. The influence estimation is performed by a relative comparison of cost required formaintaining duplicate code and non-duplicate code. The proposedcomparison has a different standpoint from the previous studies described in Section 2. The comparison is performed by using a new
indicator, modification frequency (in short, MF). The MF of duplicate code (in short, MFd ) and the MF of non-duplicate code (inshort, MFn ) are measured using the following steps:
STEP1: Identifies revisions where one or more source files are
modified, added, or deleted. The identified revisions are called
target revisions in the remainder of this section. Then, all
the target revisions are checked out into the local storage.

CCFinderX is a major version up from CCFinder [1]. CCFinderX is a token-based detection tool as well as CCFinder although

STEP2: Normalizes all the source files in every target revision.

the detection algorithm was changed to bucket sort from suffix tree.

STEP3: Detects duplicate code within every target revision. Then,the detection result is analyzed in order to identify the filepath, the lines of all the detected duplicate code.

CCFinderX can handle more programming languages than CCFinder.

Moreover, it can effectively use resources of multi-core CPUs forfaster duplicate code detection.

3.3 Simian
Simian is a line-based detection tool [4]. As well as CCFinder
family, Simian can handle multiple programming languages. Linebased techniques realized duplicate code detection on small memory usage and short running time. Also, Simian allows very finegrained settings. For example, we can configure that duplicate code
is not detected from import statements in the case of Java language.

3.4 Scorpio
Scorpio is a PDG-based detection tool [3, 12]. Scorpio builds
a special PDG for duplicate code detection, not traditional one, inwhich there are two types of edge representing data dependency
and control dependency. The special PDG has one more edge, execution dependency. The execution dependency allows detecting
1 Abstract Syntax Tree
2 Program Dependency Graph

STEP4: Identifies differences between two consecutive revisions.
The start lines and the end lines of all the differences are
stored.
STEP5: Counts the number of modifications on duplicate codeand non-duplicate code.
STEP6: Calculates MFd and MFn based on the duplicate code detection results and difference identification results.
The remainder of this section explains each step of the measurement in detail.

STEP1: Obtains target revisions
In order to measure MFd and MFn for a target system, it is necessary to obtain the historical data of the source code. In this research,
we used a version control system, subversion to obtain the historical data. Due to the limit of implementation, we restrict the target

Software nameEclEmma
FileZilla
FreeCol
SQuirrel SQL Client
WinMerge

Software nameThreeCAM
DatabaseToUMLAdServerBeans
NatMonitor
OpenYMSG
QMailAdmin
Tritonn
Newsstar
Hamachi-GUI
GameScanner

Table 1: Target software systems(a) Experiment 1
DomainProgramming language # of revisions
Testing
Java
788
FTP
C++
3,450
Game
Java
5,963
Database
Java
5,351
Text processing
C++
7,082

Domain3D modeling
Database
Web
Network(NAT)
Messenger
Mail
Database
Network(NNTP)
GUI,Network(VPN)
Game(b) Experiment 2
Programming language
Java
Java
Java
Java
Java
C
C/C++
C
C
C/C++

version control system to subversion. However, it is possible to use
other version control systems such as CVS.
Firstly, we identify which files are modified, added, or deleted
in each revision, and find out whether the files are source files or
not by checking their extensions. If there are 1 or more source files
in the modified files, its revision is regarded as a target revision.
After identifying all the target revisions from the historical data,
they are checked out into the local storage.

STEP2: Normalizes source code
In the STEP2, every source file in all the target revisions is normalized with the following rules:• deletes blank lines, code comments, and indents,• deletes lines including only a single open/close brace, andthe open/close brace is added to the end of the previous line.
The presence of code comments influences the measurement of
MFd and MFn . If a code comment is located within a duplicate
code, it is regarded as a part of duplicate code even if it is not a
program instruction. Thus, the LOC of duplicate code is counted
greater than it really is. Also, there is no common rule how code
comment located in the border of duplicate code and non-duplicate
code should be treated, so that a certain detection tool regards sucha code comment as duplicate code meanwhile another tool regards
it as non-duplicate code.
As mentioned above, the presence of code comments makes it
more difficult to accurately identify the start line and end line of
duplicate code. Consequently, all the code comments are removed
completely. As well as code comment, different detection tools
handle blank lines, indents, lines including only a single open or
close brace in different ways, which also influence the result of
duplicate code detection. For this reason, blank lines and indents
are removed, and lines including only a single open or close brace
are removed and the open or close brace is added to the end of theprevious line.

# of revisions
14
59
98
128
141
312
100
165
190
420

LOC of the newest revision
15,32887,282
89,661
207,376
130,283

LOC of the newest revision
3,854
19,6957,406
1,139
130,072
173,688
45,368
192,716
65,7901,214,570

STEP3: Detects duplicate code
Duplicate code is detected from every target revision, and the detection results are stored into database. Each detected duplicate
code is identified by 3-tuple, (v, f ,l). Every element of the 3-tuple
is as follows: v is the revision number that a given duplicate code
was detected; f is the absolute path to the source file where a given
duplicate code exists; l is a set of line numbers where duplicate
code exists. Note that storing only the start line and the end lineof duplicate code is not feasible because a part of duplicate code is
non-contiguous (Type-3 code clone).
This step is very time consuming. If the history of the target
software includes 1,000 revisions, duplicate code detection is performed 1,000 times. However this step is fully-automatically processing, and no manual work is required.

STEP4: Identifies differences between two consecutive revisions
In this research, we count the number of pieces of modified code,
not lines of modified code. That is, even if multiple consecutive
lines are modified, we regard it as a single modification. This isbecause that, in the process of fixing bugs or adding new functionalities, the cost required before source code modification is muchgreater than the cost of the modification cost itself. For example, if
we regard every modified line as a single modification, we cannot
distinguish 1 line modification at 10 pieces of code for fixing 10 different bugs from 10 lines modification at 1 piece of code for fixing abug. In the actual process, the former case will require much morecost than the latter case. Counting based on the number of pieces
of modified code can distinguish the two cases.
In order to identify the number of modifications in the above
manner, we use UNIX diff command. Figure 1 shows an example
of diff output. As shown in Figure 1, it is very easy to identify
multiple consecutive lines modification as a single modification,
all we have to do is just parsing the output of diff so that the startline and end line of all the modifications are identified.

MF of non-duplicate code

# of revisions

20

8000
7000
6000
5000
4000
3000
2000
1000
0

MF

15
10
5
0
EclEmma

FileZilla

FreeCol

SQuirreLRIVISIONS

MF of duplicate code

WinMerge

target software systems

Figure 2: Result of Item A on Experiment 1

STEP5: Counts the number of modifications
In this step, the number of modifications of duplicate code andnon-duplicate code is counted with the result of the previous step.
Here we assume that the variable for the number of modificationsof duplicate code is MCd and, the variable for non-duplicate codeis MCn . Firstly, MCd and MCn are initialized with 0, then they
are increased as follows: if the range of a specified modification iscompletely included in duplicate code, MCd is incremented; if it iscompletely included in non-duplicate code, MCn is incremented; if
it is located across the border of duplicate code and non-duplicate
code, both MCd and MCn are incremented. All the modifications
are processed with the above algorithm.

and (2) using the LOCs of duplicate code and non-duplicate code.
Here, we assume that:• LOCd (r) is the total lines of duplicate code in revision r.
• LOCn (r) is the total lines of non-duplicate code on r.
• LOC(r) is the total lines of code on r, so that the following
formula is satisfied:
LOC(r) = LOCd (r) + LOCn (r)Using these assumptions, the normalized MFd and MFn are defined
as follows:

STEP6: Calculates MFd and MFn∑ MCd (r)In this step, MFd and MFn are calculated with the result of duplicate
code detection and the result of modification counting. Here weassume that:

normalized MFd =∑ MCn (r)normalized MFn =• MCd (r) is the number of modifications on duplicate code between revision r and r + 1.

∑ MCd (r)

r∈R

|R|

(1)

MFd means the average number of modifications on duplicate
code per revision. As well as MFd , MFn is formalized as follows:∑ MCn (r)

r∈R

|R|

∑ LOC(r)r∈R∑ LOCd (r)

(3)

r∈R

|R|

×

∑ LOC(r)r∈R∑ LOCn (r)

(4)

r∈R

In the reminder of this paper, the normalized MFd and MFn are
called as just MFd and MFn , respectively.

Using this assumption, MFd is formalized as follows:

MFn =

|R|

×

r∈R

• R is a set of target revisions,

MFd =

r∈R

(2)

5.

MF COMPARISON BETWEEN DUPLICATEAND NON-DUPLICATE CODE

This section describes the experiment for comparing MFd and
MFn . In this experiment, we selected 15 software systems that areopen to the public in SourceForge (see Table 1). The selection was
performed based on the following criteria:

where:• the source code is managed with subversion;• MCn (r) is the number of modifications on non-duplicate code
between revision r and r + 1.

• the source code is written with C/C++ or Java;MFn means the average number of modifications on non-duplicate
code per revision. However, in these definitions, MFd and MFn arevery affected by the amount of duplicate code included the sourcecode. For example, if the amount of duplicate code is very small, it
is quite natural that MCd is much smaller than MCn . Therefore, wecannot fairly compare the MFd and MFn . In order to eliminate thebias of the amount of duplicate code, we normalize the formulae (1)• we took care not to bias the domains of the targets.
The experiment consists of the following two sub-experiments.
Experiment 1: We compare MFd and MFn on various size software systems with a scalable detection tool, CCFinder. The
purpose of this experiment is to reveal the relation between
the number of revisions and the impact of duplicate code.

Experiment 2: We compare MFd and MFn on small size softwarewith the 4 detection tools, described in Section 3. The purpose of this experiment is to investigate how MF comparison
results are different from detection tools.

MF of duplicate code

MF of non-duplicate code

25

MF

20

The following items are investigated in each sub-experiment.

15
10
5

Item A: This is for investigating whether duplicate code is modified more frequently than non-duplicate code. In this investigation, we calculate MFd and MFn on the entire period.

0
1

2

3

4

5
6
time period

7

8

9

10

(a) EclEmma
Item B: We expect that there are different MF tendencies between
the early period and the aging period of development. Consequently, we divide the entire period into 10 sub-periods andcalculate MF on every of the sub-periods.
MF

10
5
0
1

2

3

4

5
6
time period

7

8

9

10

(b) FileZilla
MF of duplicate code

MF of non-duplicate code

25

MF

20
15
10
5
0
1

2

3

4

5
6
time period

7

8

9

10

(c) FreeCol
MF of duplicate code

MF of non-duplicate code

30
25
20

MF

In experiment 1, we selected 5 software systems that are various
size and varying numbers of revisions. The ratio of duplicate codefor each target software is between 13% and 29% (see Table 2).
Figure 2 shows the result of Item A. As shown in Figure 2, MFd
is lower than MFn on all the target systems. It is generally said
that duplicate code is more frequently modified than non-duplicate
code because the same modifications have to be applied to multiple
pieces of code. However, the experimental result is different fromthe common belief.
Figure 3 shows the result of Item B. X axis is the divided periods.
Label ‘1’ is the earliest period of the development, and label ‘10’is the most recent period. In the case of EclEmma, the number of
periods that MFd is greater than MFn is the same as the number of
periods that MFn is greater than MFd .
In the case of FileZilla, FreeCol, and Winmerge, there is only aperiod that MFd is greater than MFn . In the case of Squirrel SQL
Client, MFn is greater than MFd in all the periods. This result implies that if the number of revisions becomes large, duplicate code
tends to become more stable than non-duplicate code. However, theshapes of MF transitions are different from every software system.
For WinMerge, we investigated period ‘2’, where MFn is muchgreater than MFd , and period ‘10’, where is only the period that
MFd is greater than MFn . In period ‘10’, there are many modifications on test cases. The number of revisions that test cases are
modified is 49, and the ratio of duplicate code in test cases is 88.3%.
Almost all modifications for test cases are performed on duplicate
code, so that MFd is greater than MFn . Omitting the modifications
for test cases, MFd and MFn became inverted.
The summary of experiment 1 is that, duplicate code detected by
CCFinder was modified less frequently than non-duplicate code.
Consequently, we conclude that duplicate code that can be detectedby CCFinder does not have a negative impact on software evolution
even if the target software is large and its period is long.

MF of non-duplicate code

15

15
10
5
0
1

2

3

4

5
6
time period

7

8

9

10

(d) SQuirreL
MF of duplicate code

MF of non-duplicate code

20
15

MF

5.1 Experiment1: Result and Discussion

MF of duplicate code2010

5.2 Experiment2: Result and Discussion

5

As shown in Table 1, the number of revisions of the target software in experiment 2 is smaller than the target software of experiment 1. This is because, in this experiment, 4 detection tools wereused for calculating MFs. It took much time to detect duplicate
code from every revision.
Figure 4 shows the result of Item A. In Figure 4, the detection
tools are abbreviated as follows: CCFinder → C; CCFinderX → X;
Simian → Si ; Scorpio → Sc . Scorpio does not handle C/C++, sothat there are the results of the other 3 detection tools on C/C++
systems (Figure 4(b)). MFd is less than MFn in the 22 comparison
results out of 35. In the 5 target systems out of 10, duplicate code

0
1

2

3

4

5
6
time period

7

8

9

10

(e) WinMerge

Figure 3: Result of Item B on Experiment 1

is modified less frequently than non-duplicate code in the cases of
all the detection tools. In the case of the other 2 systems, MFd isMF of duplicate code

MF of non-duplicate code

Sc

Si

80
70
60

MF

50
40
30
20
10
0
C

X

Si

Sc

C

X

ThreeCAMSiC

X

DatabaseToUML

Sc

C

X

AdServerBeans

Si

Sc

C

X

NatMonitor

Si

Sc

OpenYMSG

target software systems (Java)(a) Java software
MF of duplicate code

MF of non-duplicate code

200
180
160
140

MF

120
100
80
60
40
20
0
C

X

Si

C

QMailAdmin

X

Si

C

Tritonn

X

Si

Newsstar

C

X
Hamachi-GUI

Si

C

X

Si

GameScannertarget software systems (c/c++)(b) C/C++ software

Figure 4: Result of Item A on Experiment 2
greater than MFn in the cases of all the detection tools. And in
the remaining software, the comparison result is different for thedetection tools. Also, we compared MFs based on programming

Table 2: Ratio of duplicate code(a) Experiment 1
Software nameEclEmma
FileZilla
FreeCol
SQuirrel
WinMerge

CCFinderCCFinderX

SimianScorpio

13.1%
22.6%
23.1%
29.0%
23.6%

-

-

-

Software name
TreeCAM
DatabaseToUMLAdServerBeans
NatMonitor
OpenYMSG
QMailAdmin
Tritonn
Newsstar
Hamachi-GUI
GameScanner

CCFinderCCFinderX

SimianScorpio

29.8%
21.4%
22.7%
9.0%
17.4%
34.3%
13.8%
7.9%
36.5%
23.1%

10.5%
25.1%
18.2%
7.7%
9.9%
19.6%
7.5%
4.8%
23.1%
13.1%

4.1%
7.6%
20.3%
0.7%
5.8%
8.8%
5.5%
1.5%
18.5%
6.6%

26.2%
11.8%
15.9%
6.6%
9.9%
-

(b) Experiment 2

language and detection tool. The comparison result is shown in
Table 3. The result shows that MFd is less than MFn on all the
programming language and all the detection tools.
We investigated whether there is a statistically-significant difference between MFd and MFn by t-test. The result is that, there isno difference between them where the level of significance is 5%.
Also, there is no significant difference in the comparison based onprogramming language and detection tool. It is generally said thatthe presence of duplicate code makes it more difficult to maintain

Table 3: Comparing MFs based on programming language(a) Comparison on Programming Language
ProgrammingMF
language
duplicate code non-duplicate codeJava
20.1547
23.5330
C/C++
46.453155.0157ALL
32.176438.0545(b) Comparison on Detection Tool
DetectionMF
tool
duplicate code non-duplicate code
CCFinder36.2851
39.2862
CCFinderX37.882738.7334
Simian25.5285
39.7959
Scorpio20.5851
23.5483ALL
32.176438.0545

MF

MF of duplicate code

MF of non-duplicate code

200
180
160
140
120
100
80
60
40
20
0
C

X

Si

Sc

C

X

1

Si

Sc

C

X

Si

2

Sc

C

X

3

Si

Sc

C

X

4

Si

Sc

C

X

Si

5

Sc

C

X

6

Si

Sc

C

X

7

Si

Sc

C

X

8

Si

Sc

C

X

9

Si

Sc

Si

Sc

Si

Sc

10

time period(a) AdServerBeansMF

MF of duplicate code

MF of non-duplicate code

50
45
40
35
30
25
20
15
10
5
0
C

X

Si

Sc

C

X

1

Si

Sc

C

X

2

Si

Sc

C

X

3

Si

Sc

C

X

Si

4

Sc

C

X

Si

5

Sc

C

X

6

Si

Sc

C

X

7

Si

Sc

C

X

8

Si

Sc

C

X

9

10

time period(b) OpenYMSG
MF of duplicate code

MF of non-duplicate code

80
70
60

MF

50
40
30
20
10
0
C

X

Si
1

Sc

C

X

Si
2

Sc

C

X

Si
3

Sc

C

X

Si

Sc

C

X

4

Si

Sc

C

X

5

Si
6

Sc

C

X

Si

Sc

7

C

X

Si
8

Sc

C

X

Si
9

Sc

C

X
10

time period(c) NatMonitor

Figure 5: Result of Item B on Experiment 2
software. However, we cannot see such a tendency in the result of
this comparison.
We investigated how the software evolved in the period, and we
found that the following activities should be a part of factors that
duplicate code is modified less frequently than non-duplicate code.
Reusing stable code: When implementing new functionalities, reusing
stable code is a good way to reduce the number of introduced
bugs. If most of duplicate code is reused stable code. MFd
becomes less than MFn .
Using generated code: Automatically-generated code is rarely modified manually. Also, the generated code tends to be duplicate
code. Consequently, if most duplicate code is the generated
code, MFd becomes less than MFn .

Figure 6 is an instance of this case. The whole of the method
is duplicate code, and there are 16 correspondents in 4 source
files. Every method adds an object received by a parameter
into a collection. Their names are in a common form ‘addXXXPropertyDescriptor’ (XXX is different from every method).
All the 17 methods were added in the 10th revision, and noneof them was modified until the most recent revision. Annotation @generated was attached into all the method, whichmeans that they were generated by a code generation tool.
We measured the ratio of duplicate code for each target software.
The ratio is shown in Table 2(b). The ratio of duplicate code is very
different for each detection tool on the same software. However, as
well as in experiment 1, we cannot find out any relation between
the ratio and MF.

@generated
protected void addNamePropertyDescriptor(Object object) {itemPropertyDescriptors.add(
createItemPropertyDescriptor(((ComposeableAdapterFactory)adapterFactory).
getRootAdapterFactory(),
getResourceLocator(),getString("_UI_NamedElement_name_feature"),getString("_UI_PropertyDescriptor_description","_UI_NamedElement_name_feature","_UI_NamedElement_type"),
MetadataPackage.Literals.NAMED_ELEMENT__NAME,
true, false, false,
ItemPropertyDescriptor.GENERIC_VALUE_IMAGE,
null, null));}Figure 6: An instance of stable duplicate code

Figure 5 shows the result of Item B. Figure 5 shows the result of
only the 3 systems due to space limitation. In Figure 5(a), period‘4’ shows that MFn is greater than MFd on all the detection tool
meanwhile period ‘7’ shows exactly the opposite result. Also, in
period ‘5’, there are hardly differences between duplicate code andnon-duplicate code. We investigated the source code of period ‘4’.
In this period, many source files were created by copy-and-paste,
and a large amount of duplicate code was detected by each detection tool. The copy-and-pasted code was very stable meanwhile theother source files were modified as usual. This is the reason why
MFn is much greater than MFd in period ‘4’.
Figure 5(b) shows that duplicate code tends to be modified more
frequently than non-duplicate code in the anterior half of the period
meanwhile the opposite occurred in the posterior half. We foundthat there was a large number of duplicate code that was repeatedly
modified in the anterior half. On the other hand there was rarely
such duplicate code in the posterior half.
Figure 5(c) shows the opposite result of Figure 5(b). That is, duplicate code was modified more frequently in the posterior half ofthe period. In the anterior half, the amount of duplication was verysmall, and modifications were rarely performed on it. In the posterior half, amount of duplicate code became large, and modifications
were performed on it repeatedly. In the case of Simian detection,no duplicate code was detected except period ‘5’. This is because
Simian detects only the exact-match duplicate code meanwhile theother tools detect extract-match and renamed duplicate code in thedefault setting.
The summary of experiment 2 is as follows: we found some instances that duplicate code was modified more frequently than nonduplicate code in a short period on each detection tool; however, inthe entire period, duplicate code was modified less frequently thannon-duplicate code on every target software with all the tools. Consequently, we conclude that the presence of duplicate code does nothave a seriously-negative impact on software evolution.
The conclusion is contradictory with the previous studies [19,20]. As described in Section 2, they investigated the influence of
duplicate code on file unit or method unit. However, the units (files
and methods) are larger than duplicate code, so that modifications
can be incorrectly counted: if modifications were performed ona method where a duplicate code exists, all the modifications are
assumed as performed on duplicate code even if they were actually
performed on non-duplicate code. Thus, their studies might countmore the number of modification on duplicate code than it reallywas. On the other hand, in the proposed method, the unit is line,
which is equal to or smaller than duplicate code. Consequently,
such an incorrect count was not performed on the proposed method.
The incorrect count on duplicate code may introduce the previous

works to the different conclusion from the present paper.

6.

THREATS TO VALIDITY
This section describes threats to validity of this empirical study.

Cost required for every modification
In this empirical study, we assume that cost required for every modification is equal to one another. However, in the actual software
evolution, the cost is different between every modification. Consequently, it is possible that the comparison based on MF may not
appropriately represent the cost required for modifying duplicate
code and non-duplicate code.
Also, when we modify duplicate code, we have to consider maintaining the consistency between the modified duplicate code and its
correspondents. If the modification lacks the consistency by error,
we have to re-modify them for repairing the consistency. The effort
for consistency is not necessary for modifying non-duplicate code.
Consequently, the average cost required for duplicate code may bedifferent from the one required for non-duplicate code. In order tocompare them more appropriately, we have to consider the cost for
maintaining consistency.

Identifying the number of modifications
In this empirical study, modifying consecutive multiple lines is regarded as a single modification. However, it is possible that such an
automatically processing identifies the incorrect number of modifications. If multiple lines that were not contiguous are modified for
fixing a single bug, the proposed method presumes that multiple
modifications were performed. Also, if multiple consecutive lines
were modified for fixing two or more bugs by chance, the proposed
method presumes that only a single modification was performed.
Consequently, it is necessary to manually identify modifications if
we have to use the exactly correct number of modifications.
Beside, we investigated how many the identified modifications
occurred across the boundary of duplicate code and non-duplicate
code. If this number is high, then the analysis suspect because suchmodifications increase both the counts at same time. The investigation result is that, in the highest case, the ratio of such modifications
is 4.8%. That means that almost all modifications occurred withineither duplicate code or non-duplicate code.

Category of modifications
In this empirical study, we counted all the modifications, regardlessof their categories. As a result, the number of modifications might
be incorrectly increased by unimportant modifications such as format transformation. A part of unimportant modifications remainedeven if we had used the normalized source code described in Section 4. Consequently, manual categorization for the modifications
is required for using the exactly correct number of modifications.
Also, the code normalization that we used in this study removed
all the comments in the source files. If considerable cost was expended to make or change code comments on the development of
the target systems, we incorrectly missed the cost.

Property of target software
In this empirical study, we used only open source software systems,
so that different results may be shown with industrial software systems. It is generally said that industrial software includes more
duplicate code than open source software. Consequently, duplicate code may not be managed well in industrial software, which
may increase MFd . Also, properties of industrial software are quite
different from ones of open source software. In order to investi-

gate the impact of duplicate code on industrial software, we haveto compare MF on industrial software itself.

Release Level. In Proc. of the 16th Working Conference on
Reverse Engineering, pages 85–94, Oct. 2009.
[7] E. Burd and J. Bailey. Evaluating Clone Detection Tools for
Division of development period
Use during Preventative Maintenance. In Proc. of the 2ndIEEE International Workshop on Source Code Analysis andIn this empirical study, we divided the development period in an auManipulation, pages 36–43, Oct. 2002.
tomatic manner based on the number of revisions. However, different division may yield different results. For example, if we divide[8] M. de Wit, A. Zaidman, and A. van Deursen. Managing Codethe period based on the border of versions, we may be able to grasp
Clones Using Dynamic Change Tracking and Resolution. In
the properties of every version. Or, more fine grained division, that
Proc. of the 25th IEEE International Conference on Software
is, the period of every version is divided into multiple sub-periods,
Maintenance, pages 169–178, Sep. 2009.
which will let us how duplicate code and non-duplicate code are[9] S. G. Eick, T. L. Graves, A. F. Karr, J. S. Marron, andmodified from a start of a version to the end of the version.
A. Mockus. Does Code Decay? Assessing the Evidence from
Change Management Data. IEEE Transactions on Software
Settings of detection tools
Engineering, 27(1):1–12, Jan. 2001.
In this empirical study, we used default settings for all the detection[10] M. Fowler. Refactoring: improving the design of existing
tools. If we change the settings, different results will be shown.
code. Addison Wesley, 1999.
[11] N. Göde. Evolution of type-1 clones. In Proc. of the 9thInternational Working Conference on Source Code Analysis
7. CONCLUSION
and Manipulation, pages 77–86, Sep. 2009.
The present paper experimentally evaluated the impact of the[12] Y. Higo and S. Kusumoto. Significant and Scalable Codepresence of duplicate code in open source software systems. In orClone Detection with Program Dependency Graph. In Proc.
der to compare cost required for modifying duplicate code and nonof the 16th Working Conference on Reverse Engineering,
duplicate code, we defined a new marker, modification frequency.
pages 315–316, Oct. 2009.
In the experiment, we used 4 duplicate code detection tools for re[13]T. Kamiya, S. Kusumoto, and K. Inoue. CCFinder: A
ducing the bias of detection tool. The empirical result from 15
Multilinguistic Token-Based Code Clone Detection System
software systems showed that duplicate code tends to be modified
for Large Scale Source Code. IEEE Transactions on
less frequently than non-duplicate code. However, in the face ofSoftware Engineering, 28(7):654–670, July 2002.
statistics, there is no difference of significance between them where[14] C. J. Kapser and M. W. Godfrey. “Cloning considered
the level of significance is 5%. That is, the presence of duplicate
harmful” considered harmful: patterns of cloning incode does not have made it more difficult to develop and maintainsoftware. Empirical Software Engineering, 13(6):645–692,
software systems. At the same time, we detected some duplicate
July 2008.
code that was repeatedly modified simultaneously with its correspondents. In the future, we will conduct more experiments with
[15] M. Kim, V. Sazawal, D. Notkin, and G. C. Murphy. An
a new condition that overcomes the threats to validity described in
Empirical Study of Code Clone Genealogies. In Proc. of theSection 6. Moreover, we will try to automatically identify dupliACM SIGSOFT Symposium on the Foundations of Software
cate code that should have a seriously-negative impact on future
Engineering, pages 187–196, Sep. 2005.
development and maintenance.
[16] J. Krinke. Is cloned code more stable than non-cloned code?
In Proc. of the 8th International Working Conference on
Source Code Analysis and Manipulation, pages 57–66, Sep.
Acknowledgment
2008.
The present research is being conducted as a part of the Stage[17]
A. Lozano and M. Wermelinger. Assessing the effect ofProject, the Development of Next Generation IT Infrastructure, supclones
on changeability. In Proc. of the 24th International
ported by the Ministry of Education, Culture, Sports, Science, andConference on Software Maintenance, pages 227–236, Sep.
Technology of Japan. This study has been supported in part by2008.
Grants-in-Aid for Scientific Research (A) (21240002) and (C) (20500033)[18]
A. Lozano, M. Wermelinger, and B. Nuseibeh. Evaluatingfrom the Japan Society for the Promotion of Science, and Grandthe harmfulness of cloning: a change based experiment. In
in-Aid for Young Scientists (B) (22700031) from Ministry of EduProc. of the 4th International Workshop on Mining Software
cation, Science, Sports and Culture.
REpositories, May 2007.
[19] A. Lozano, M. Wermelinger, and B. Nuseibeh. Evaluating
8. REFERENCES
the relation between changeability decay and the[1] CCFinderX. http://www.ccfinder.net/.
characteristics of clones and methods. In Proc. of the 23rd[2] Clone Detection Literature. http:International Conference on Automated Software//www.cis.uab.edu/tairasr/clones/literature/.
Engineering, pages 100–109, Sep. 2008.
[3] Scorpio. http://www-sdl.ist.osaka-u.ac.jp/~higo/[20] A. Monden, D. Nakae, T. Kamiya, S. Sato, andcgi-bin/moin.cgi/scorpio-e/.
K. Matsumoto. Software Quality Analysis by Code Clones in[4] Simian-similarity analyser. http:Industrial Legacy Software. In Proc. of the 8th IEEE//www.redhillconsulting.com.au/products/simian/.
International Software Metrics Symposium, pages 87–94,
June 2002.
[5] S. Bellon, R. Koschke, G. Antoniol, J. Krinke, and E. Merlo.
Comparison and Evaluation of Clone Detection Tools. IEEE
[21] M. P. Robillard, W. Coelho, and G. C. Murphyy. How
Transactions on Software Engineering, 31(10):804–818, Oct.
effective developers investigate source code: An exploratory
2007.
study. IEEE Transactions on Software Engineering,30(12):889–903, Dec. 2004.
[6] N. Bettenburg, W. Shang, W. Ibrahim, B. Adams, Y. Zou,
and A. E. Hassan. Inconsistent Changes to Code Clones at