ImpactScale: Quantifying Change Impact to Predict Faults
in Large Software Systems
Kenichi Kobayashi1, Akihiko Matsuo1, Katsuro Inoue2, Yasuhiro Hayase3, Manabu Kamimura1, Toshiaki Yoshino4
1

Fujitsu Laboratories Limited
Kawasaki, Kanagawa, Japan{kenichi, a_matsuo,
kamimura.manabu}@jp.fujitsu.com

2

Osaka University
Suita, Osaka, Japaninoue@ist.osaka-u.ac.jp

Abstract—In software maintenance, both product metrics andprocess metrics are required to predict faults effectively.
However, process metrics cannot be always collected in practicalsituations. To enable accurate fault prediction without process
metrics, we define a new metric, ImpactScale. ImpactScale is thequantified value of change impact, and the change propagation
model for ImpactScale is characterized by probabilistic
propagation and relation-sensitive propagation. To evaluate
ImpactScale, we predicted faults in two large enterprise systems
using the effort-aware models and Poisson regression. The results
showed that adding ImpactScale to existing product metrics
increased the number of detected faults at 10% effort (LOC) by
over 50%. ImpactScale also improved the predicting model using
existing product metrics and dependency network measures.

I.

INTRODUCTION

Software fault prediction has achieved a great outcome as amean of supporting reviews and tests, controlling qualities, andmanaging risks [1][2]. However, in the maintenance or postrelease phase of software lifecycle, fault prediction is arelatively difficult task. Fault prediction requires metrics
correlated with faults as explanatory variables. Several studies
have reported the set of faulty modules in pre-release phasediffers from the set of ones in post-release phase [3][4]. Metrics
useful in pre-release phase may often become useless in postrelease phase. For example, Fenton et al. [3] reported the casethat McCabe’s cyclomatic complexity correlated with prerelease faults but little correlated with post-release faults.
To improve predictive performance in maintenance,
process metrics have been used in addition to product metrics.
Product metrics are metrics extracted from a snapshot of source
code, and process metrics are metrics extracted from project
activity records such as histories of changes, inspections anddefects [5]. Some studies have reported that formerly changed
code fragments tend to be fault-prone and metrics extracted
from such changes improved predictive performance [4][5][6].
However, in practical maintenance activities, documents,
records and human knowledge are often lost. In such situations,process metrics cannot be obtained. Our goal is to define a new
product metric which correlates with faults and improves
predictive performance in maintenance.
Even in long maintained software systems, one of surviving
factors of faults is software dependency [7][8]. When a module
is changed in a system, some of the modules dependent on it
doi: 10.1109/ICSM.2011.6080771 ©2011 IEEE

3

University of Tsukuba
Tsukuba, Ibaraki, Japanhayase@cs.tsukuba.ac.jp

4

Fujitsu Limited
Tokyo, Japan
yoshino.toshi@jp.fujitsu.com

should be also changed if needed. The process of change isrepeated until no change is needed. If modules under such aripple effect of changes [9] were not fully captured, faults may
happen. Logical coupling [7] is co-change information insoftware products, and it is well-known that metrics based on
logical coupling have relations with faults [5]. Hassan et al. [8]reported that logical coupling can be used to recognize change
propagation. Logical coupling can be regarded as theobservation of implicit dependency through software changes.
There are various types of software dependencies. Productmetrics regarding direct dependencies are categorized into
syntactic dependency, and process metrics regarding logicalcoupling are categorized into logical dependency [7]. Cataldo
et al. [10] compared the strengths of the correlations between
various dependencies and faults. Their examination showedthat the correlation between syntactic dependency and faults
was insignificant or weak and that the correlation between
logical dependency and faults was significant and the strongest.
The results are one of the cases that product metrics are
insufficient for fault prediction in maintenance.
Zimmermann et al. applied social network analysis (SNA)
on a software dependency graph representing relationships
between binary modules of software systems [11]. They
reported that adding network measures from SNA literature
could improve the performance of fault prediction. Although
the network measures are product metrics, they are not coveredby the syntactic dependency categorized by Cataldo.
We assumed that the network measures used byZimmermann and the logical dependency used by Cataldo
share common factors of fault-proneness. Therefore, we
assumed that change impact analysis [12] on source code
enables us to extract implicit dependency, such as relations
exposed by logical coupling. Change impact analysis is atechnique that detects affected areas of source code when somepart is changed. In Cataldo’s examination the number of logicalcouplings of a given module was most correlated with faults.
Therefore, we expected the scale of estimated areas affected byany changes correlates with fault-proneness as well as thenumber of logical couplings, and we hypothesized as follows:
Hypothesis 1: A metric that quantifies the scale of change
impact can improve the performance of fault prediction inlarge software systems.

ICSM 2011, Williamsburg, VA, USA

Accepted for publication by IEEE.  2011 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

However, it is difficult to compute the exact affected areasof change impact. Static analysis [13] has a nature that it may
derive excessive (false positive) areas. Besides, it requires
enormous computation time to improve the accuracy. Dynamic
analysis [14] can easily capture dynamically bound areas, but it
has a nature that it may fail to capture affected areas which are
seldom used. In reality, there are many cases for which
dynamic analysis cannot be performed. A practical technique tofind the affected areas has been proposed for the case where thechange to a given module is already known [15]. However,since we need to know the area before the change is given, it isdifficult to compute the areas while minimizing false positives.
Fortunately, to meet our goal it is not necessary to solve theproblem of computing the affected areas. It is enough to solve amore relaxed problem of calculating the total approximate
quantity of the affected areas. We defined a propagation modelin which change impact is propagated probabilistically andrelation-sensitively, and we hypothesized as follows and called
the quantity ImpactScale.
Hypothesis 2: The propagation model with relationsensitive propagation has enough predictive performanceto substitute for an accurate but expensive analysis.
The remainder of the paper is organized as follows: we willdefine ImpactScale in section II. The change propagation
model for ImpactScale will be also described. In section III, we
will evaluate the improvement of the predictive performanceby adding ImpactScale. In section IV, we will discuss about
ImpactScale and its definition. The threats to validity of
ImpactScale will be discussed in section V. We will talk about
related works in section VI and will conclude in section VII.
II.

IMPACTSCALE

In this section, we define a new metric, ImpactScale (abbr.
as IS), and we describe the change propagation model for it.
A. Dependency Graph and Propagation Graph
In this paper, a dependency graph is a multi-relational
directed graph, where the nodes are code entities (methods,
functions, and so on) and data entities (database tables, global
variables, and so on), and where the edges represent therelationships or dependencies between nodes. The edges arelabeled with the relation type, such as call, data, import orinheritance dependency. In a dependency graph, multiple edges
between any two nodes are allowed. Formally, the dependency
graph GD is defined as GD = <V, E>, where V is a set of nodes
and E is a set of edges. Edge e  E is defined as e = <s, t, rel>,where s  V is a source node, and t  V is a target node, andrel  Rel is a relation type. Rel is the set of all relation types.
CALL

D
D

R_
RE
A

D

L

RE
A

TE
RI

CA
L

E

T
RI

WRITE

W
R_

E

T
RI

READ

B
W

W

CALL

RE
A

Table

R_
CA
LL
CA
LL

FunctionB

R_CALL

A

CALL

A

Figure 1. Example of dependency graph GD (left part) and propagationgraph GP (right part)

The left part of Figure 1 shows an example of a dependency
graph. In the figure, there are three relation types, CALL, READ,
and WRITE. When function A calls function B, it is said that“there is a CALL relationship from function A to function B.”
Function A and B may share some status through arguments.
Since it can be hardly determined whether two functions A and
B were designed under the top-down-approach or the bottomup-approach, either change propagation from A to B or from B
to A cannot be ruled out. In order to capture remote logicaldependencies, we take a very conservative policy that change
propagation in a dependency graph has bidirectional nature.
A propagation graph consists of a dependency graph andthe reversed edges of all of its edges. A propagation graphcorresponding to GD=<V, E> is defined as GP = <V, EP>,
where EP is defined as EP = E  {reverse(e) | e  E}. reverse(e)means the reversed edge <t, s, R_rel> of an edge e = <s, t, rel>.
Relation type R_rel means the reversed relation type of rel. The
quantity of change impact is calculated on a propagation graph.
The right part of Figure 1 shows the propagation graph GP
corresponding to the dependency graph GD in Figure 1.
B. Probabilistic Propagation
Haney [9] proposed an analysis model that a change in amodule probabilistically propagates to another module, andthere are some studies which assumed that changes propagate
probabilistically [16][17]. In the propagation graph in Figure 2,
it is assumed that the change impact from v0 to v1 is propagatedwith probability r1, when there is edge e1 from v0 to v1. The
same goes for v1 and v2. It is also assumed that the change
impact from v0 to v2 is propagated with probability r1r2.

Pr=r1
v0

e1

Pr=r2
v1

e2

v2

Figure 2. Example of probabilistic propagation

C. Relation-sensitive Propagation
Since we took the aforementioned conservative propagation
policy, there is a worry that the affected areas areoverestimated. To prevent excessive estimation, we introduced
some light-weight constraints to the propagation policy. In callgraph analysis, context-sensitive analysis, which refers
previous call and status histories, is often used to improve theaccuracy of the analysis [13]. Since the analysis requires
enormous computation time, we could not use it. However, weborrowed an idea from it to solve our problem.
To prevent explosion of computation time, we used aminimal context, the relation type of the previous edge, which
is practically collectable and requires minimal computation
time. Concretely, whether propagation from one node to its
next is cut or not is determined by the projection Cut. Cut is a
projection whose domain is a pair of the relation type of theprevious edge and the relation type of the next edge, and it isdefined as Cut: Rel  Rel  {True, False}. For example, in the
propagation graph in Figure 3, at node v, the relation type of
the previous edge e1 is relprev1, and the relation type of the nextedge e3 is relnext3. The projection Cut for the graph is given forin the bottom half of the figure. The change impact from edge
e1 to edge e3 does not propagate, because Cut(relprev1, relnext3) isTrue. We call such propagation controlled by relation types
relation-sensitive propagation.

relprev1

relnext3

e1

e3

v

relprev2

relnext4e4e2

Cut(rel prev1, relnext3 )  Cut(rel prev2 , relnext4 )  True
Cut(rel prev1, relnext4 )  Cut(rel prev2 , relnext3 )  False
Figure 3. Example of relation-sensitive propagation

D. Definition of ImpactScale
In this subsection, we give the definition of ImpactScale. In
the propagation graph GP=<V, EP>, path p from node s to nodet is denoted as p = (e1, e2 ... en), where edge ei is each edge inpath p in the order of pass. The source of e1 is s, and the targetof en is t. The same node can be passed two or more times in apath. Strictly speaking, the path is a trail, in the terminology ofgraph theory. The propagation rate ri of each edge ei is given asri  RP  m(ei ) ,where RP is the base propagation rate and is hypothetically 0.5
in this paper, and m(ei) is the propagation modifier attached toedge ei, and it is normally 1.01, though it can take any value inthe range of (0, 1] or 1/RP. Qpath(p), the quantity of change
impact via path p, is given as

Q path ( p) 

n
1 n
n 1
ri  RP  m(ei ) .
RP i 1i 1

Ps,t is the set of all paths reachable from s to t. Ps,t iscalculated by path finding in the propagation graph. When asearch reaches a node in a path, projection Cut determines
whether the search goes forward to the next node through thenext edge or is terminated. The previous edge of the starting
node is not defined, thus the propagation from it must occur.
Q(s, t), the quantity of change impact from s to t, is given as

maxQ path ( p) : Ps ,t   .
Q(s, t )   pP s ,t0
: Ps ,t  Calculating Q(s, t) is essentially equivalent to solving theshortest path problem. ImpactScale, the quantity of change
impact from s, is denoted as IS(s), and it is given asIS( s)   Q(s, t )tV\ s(V\s means a set of all elements of V except s). Finally, thequadruplet ImpactScale = <IS, RP, Rel, Cut> is the definition of
ImpactScale.
E. Cut Rules
The set Rel and projection Cut can be configured arbitrarily.
In this paper, Rel is given as Rel = {CALL, READ, WRITE,
R_CALL, R_READ, R_WRITE}. CALL means a function call, READ
1

Values other than 1.0 are used to adjust propagation rates, e.g., among
different abstraction levels of entities such as database tables and variables.

means a read access, and WRITE means a write access (which
also implies a read access). R_CALL, R_READ, and R_WRITE are
the reversed relation types, respectively. Cut is given as
True
True

Cut( p, n)  TrueFalse: p  CALL  n  R_CALL: p  R_CALL  n  CALL

(Rule 1)
(Rule 2) .

: p  READ

(Rule 3): otherwise

The condition part of the above notation of projection Cut iscalled cut rules.
The above cut rules are heuristic. Their validity will be
discussed in section IV. In this paragraph we explain theirintent. The propagation from CALL to CALL is assumed to beunder the top-down-approach design. The propagation from
R_CALL to R_CALL is assumed to be under the bottom-upapproach design. Rule 1 and Rule 2 cut propagations out of theboth design. These rules are designed on the assumption thatthe propagation will rarely switch from one design approach tothe other. Rule 3 is designed based on the analogy of dataflow
analysis.
F. Computational Complexity
Most of the calculation of ImpactScale is occupied by pathfinding. Since a propagation graph for ImpactScale is a multirelational graph, ordinary graph algorithms cannot be applied.
Whaley et al. [18] presented a technique for context-sensitive
analysis. The technique duplicates nodes and reduces acomplex path finding problem with context to a simple pathfinding problem without context. By using the technique, pathfinding for ImpactScale is also reduced to path finding of asimple-relational graph, and ordinary algorithms can be applied.
When Dijkstra’s algorithm is applied to a propagation graph
GP=<V, EP>, the computational complexity of the problem isO(R|EP| + R|V|(log(R|V|)), where R is the number of relation
types. It can be calculated in practical time, even if the target
software system is fairly large.
G. Examples
The left part of Figure 4 is an example of calculating
ImpactScale of node C. Paths (CA), (CD), (CX), (CX,
XF), (CX, XF, FE), and (CX, XF, FH) arefound. Since rel(CA) is R_CALL and rel(AB) is CALL, path(CA, AB) is cut by Rule 2 at edge AB. Similarly, path(CX, XF, FH, HG) is cut by Rule 1 at edge HG.
ImpactScale of C is the sum of Q(C,A), Q(C,D), Q(C,X),
Q(C,F), Q(C,E) and Q(C,H); that is 4.0.
The right part of Figure 4 is an example of calculating
ImpactScale of node J. It has a peculiar pattern of relationsensitive propagation. Though path (JK, KL) is cut by
Rule 1 at edge KL, path (JK, KY, YK, KL) is not
cut. As a result, the change impact from node J propagates tonode L. ImpactScale of J is the sum of Q(J,K), Q(J,Y) andQ(J,L); that is 1.625. Data node Y works as the medium
between node J and node L. If node Y was missing, only path
(JK) would be found, and ImpactScale of J would equal 1.0.

K Q=0.25

In subsection A-C, we describe the experimental setup.



For RQ1, we evaluate prediction performances using
binary classification in subsection D and using faultdensity prediction and effort-aware models [22] in
subsection E.

For RQ2, we evaluate using effort-aware models andhierarchical model analysis in subsection F.

A. Target Software Systems
For evaluation, we chose large enterprise accounting
software systems from two different companies. The criteria ofselecting them were that they are maintained for the long term,
considerably large, and described in languages that can beeasily analyzed. The intent of the last criterion is to minimize
the influence of the quality of the source code analysis in thisevaluation. The collected data sets of the two systems arecalled DS1 and DS2. The profiles of them are shown in Table I.

4000

Data Set

3000
2000

Mean ISMax ISDS1

86.0

2989.6DS2

156.5

3338.2

1000
0

~950

Following subsections are organized as follows.

~900

RQ2: Does adding ImpactScale to existing product metrics
and network measures improve predictive performance?

~850

RQ1: Does adding ImpactScale to existing product metrics
improve predictive performance?

~800

To test Hypothesis 1, 2 and the significance of ImpactScale,the following research questions are our main concerns.

~750

EVALUATION

B. Measuring ImpactScale
The procedure of measuring ImpactScale is as follows.
First, we extracted call operations (CALL) and read/write
operations to databases and files (READ, WRITE) from the target
software system by using static analysis. Then, we built adependency graph. Next, for each node in the graph, we
calculated ImpactScale. In this evaluation, m(e) is always 1.0.
Total calculation time was at most several tens of seconds (theCPU used was a Core2Duo 2.5GHz). Figure 5 shows the
distributions of the measured ImpactScale and its statistics. In
the figure, most of modules have small ImpactScale and only
small part of modules have large ImpactScale. In our
experience, the distributions of ImpactScale of most systems
have similar tendencies. Figure 6 shows an example ofcalculating ImpactScale of a module in DS1.

~700

Watching code change activities to keep softwaremodularity. A prominent increase of ImpactScale may
be a symptom of modularity violations.

Calls
Fan-inFan-out
IS

~650



Sections

Description
Number of lines of code without comments and blank lines
Total sum of McCabe’s Cyclomatic Complexity of sections
Max value of McCabe’s Cyclomatic Complexity of sections
Number of sections (which corresponds to the number ofblocks in a given module)
Number of calls
Number of modules which call a given module
Number of modules which are called by a given module
ImpactScale~600

Rapid quality assessment of a whole software system

Metrics
LOC
WMC
MaxVG

~550



TABLE II. COLLECTED PRODUCT METRICS~500

Rapid effort estimation of bug-fixing tasks. It is
assumed that size of the change impact of the focusedsource code affects the man-hours of fixing tasks.

Faultymodules215
208

One module is one “program” in COBOL language, which
is a unit of the target of a call operation and corresponds to afunction in other languages. One “program” is also one sourcecode file. The collected metrics are described in Table II.

~450



269
250

~400

Quality improvement in the restricted budget andschedule. This topic is further discussed in this paper.

1.6M
3.7M

~350



5.8k
7.6k

~300

H. Use Cases
ImpactScale is used in various situations. We have beenusing it in the following cases.

DS1
DS2

~250

IS(J) = 1.0 + 0.5 + 0.125

Faults

~200

Q=0.5

Total LOC

~150

Q=0.25Modules~50

Y

Data set

~100

Rule1

Figure 4. Examples of calculating ImpactScaleIII.

TABLE I. PROFILES OF TARGET SOFTWARE SYSTEMS

CA
LL

Q=1.0

H

IS(C) = 1.0x3 + 0.5 + 0.25x2

Both systems are written in COBOL language. We collectedfault reports of both systems from over 40 months.

Number of Modules

D
RE
A

E

Q=1.0

L

R_WRITE

T
RI

Q=1.0

X

G

WRITE

W

D

Q=0.125

Rule1

CA
LL

F

LL
CA

C

R_
CA
LL

CA
LL

LL
CA
R_

B

Q=0.5

J

LL
CA

Rule2

Q=0.25

E

R_
CA
LL

Q=1.0

A

ImpactScale

Figure 5.

Measured ImpactScale

C. Cross Validation
To evaluate predictive performance using the data sets, we
performed 100 times random sub-sampling cross validations
for each evaluation. For each validation, we randomly selected
2/3 of all modules of the target data set as a training set, and theremaining 1/3 of modules were used as a test set.

The left image is the visualization of the wholesystem of DS1. Each rectangular-shaped groupof modules represents a subsystem of thesystem.
The process of calculating ImpactScale of thesource module is drawn in the figure. The pathsof change propagation are drawn as arrows (upto 3 steps), and the affected modules are
drawn in yellow. The colors of modules become
fainter in proportion to the steps from thesource module.
ImpactScale of the source module is 261.2.
This value is considerably larger than the meanvalue (86.0) of the system.
From the figure, it is found that calls, reads,
and writes between different subsystems breakthe modularity of the system. Therefore, a wide
area might be affected by a change of thesource module, and it seems that themaintenance task of the module is difficult.

ModuleDatabase Table
Call
Read
Write

Source Module
Figure 6. Example of calculating ImpactScale in the actual software

D. Binary Classification (RQ1)
We evaluate how much the predictive performance isimproved by adding ImpactScale to existing product metrics
set using a binary classifier.
1) Logistic regression
As a classifier, we used logistic regression model. An
expression of logistic regression model is represented asexp(b1 x1  b2 x2  ...  b0 ) ,y
1  exp(b1 x1  b2 x2  ...  b0 )where y is a response variable, and each xi is an explanatory
variable, and each bi is a partial regression coefficient. The
expression b1x1+b2x2+ … +b0 is called a linear predictor. To
estimate the partial regression coefficients, the maximum
likelihood estimation method is used. A response variable y(0<y<1) is interpreted as the probability of fault-proneness.
2) Model selection
To select the best prediction model, the set of the optimal
explanatory variables should be selected for the training set ofeach validation. To build predicting models, we prepared
metrics in Table II as the candidate set of explanatory variables.
For each metric X, log(X) (or log(X+1), if the metric contains
zero value) was also added into the set as the alternative
candidate. That is, for each metric X, there were three choices,X, log(X), or without X. Log-transformation for skewed data isa standard technique. To select the best model, we used AIC(Akaike’s Information Criteria). If the likelihood of the model
is higher, and if the number of parameters is fewer, AIC
becomes less. Less AIC means the model is better. To selectthe best model without ImpactScale, 37 combinations weretried, and the model with the least AIC was taken. To select thebest model with ImpactScale, 38 combinations were tried. To

avoid multicollinearity, we rejected the models with a variable
whose VIF (Variance Inflation Factor) is more than 10 [19].
3) Training and testing
For each validation, we selected and trained the best model,“MET”, using existing product metrics in Table II excluding
ImpactScale, and we selected and trained the best model,“MET+IS”, using existing product metrics and ImpactScale.
For training, if the number of faults is more than zero, it isregarded as a positive. Otherwise it is regarded as a negative.
Then, we predicted all modules in the test set. For each
module in the test set, if the response variable y is more thangiven cutoff value, the module is classified as a faulty module.
Since both DS1 and DS2 had low fault rate (3.7% and 2.7%),
we chose 0.1 for cutoff.
4) Performance measures
To evaluate predictive performance, for each validation, we
computed precision, recall and F1 values using a confusion
matrix shown in Table III. “FP” means the number of modules
predicted as faulty but observed as not faulty, and vice versa.
Recall is calculated as TP/(TP+FN). Recall close to 1.0 means
most faults are detected. Precision is calculated as TP/(TP+FP).
Precision close to 1.0 means few modules are mis-predicted as
faulty. F1 is a summarizing measure of recall and precision andcalculated as their harmonic mean, 2/(1/recall+1/precision).
TABLE III. CONFUSION MATRIX

Observed

positive
negative

Predicted
positive
negative
TP
FN
FP
TN

DS1

DS2

Perf.
measurePrecision
RecallF1
Precision
RecallF1Model MET
mean (stddev)
0.148(0.029)0.315(0.051)
0.200(0.033)0.139(0.030)0.253(0.042)0.177(0.029)Model MET+IS
mean(stddev)
0.168(0.031)
0.392(0.048)0.234(0.034)0.162(0.033)0.334(0.057)0.216(0.034)

Imprv.
by IS†+0.020+0.077+0.034+0.023+0.081+0.039†All improvements are significant (P<0.001) in Wilcoxon’s signed rank test.

E. Effort-aware evaluation (RQ1)
1) Effort-aware model and performance measures
Recent studies pointed out that the effort of testing modules
should be taken into consideration in predictive performance
evaluations [20][21][22][23]. Modules classified as fault-prone
tend to be large because of the correlation between faults andsize [3]. Arisholm et al. reported the effort of testing a moduleis roughly proportional to the size of the module [20]. In
practical maintenance tasks, budget and schedule are often very
demanding. Therefore, cost-effectiveness of fault prediction
had become important concerns for practitioners.
To involve this discussion, we use the effort-aware model
proposed by Mende et al [22]. In the effort-aware model, therelative risk Rdd(x) is used to prioritize modules to test or
inspect. Rdd(x) is defined as #errors(x) / E(x), where #errors(x)is the number of faults in module x and E(x) is the required
effort for module x. We use LOC for E(x) as previous studies[22][23]. In this case, Rdd(x) means fault density. In the effortaware model, Rdd(x) (i.e., fault density) is predicted, andmodules are tested or inspected in the descending order of fault
density. To review predictive performance, effort-based
cumulative lift chart is used as shown in Figure 7. In the figure,the solid curve represents the overall prediction outcome. The
curve is called cost-effectiveness curve [20] or effort-vs-PD
curve [21]. The X coordinate of the curve represents therelative cumulative effort, and the Y coordinate represents thefault detection rate at the spent effort. If the curve is steeper,
the prediction is more effort-effective. If the curve is under the
diagonal line, the prediction is almost random and meaningless.
The dashed curve labeled as “Optimal Model” is the outcomeof the perfect prediction and the upper limit of the performance.
Figure 7 also explains two performance measures used inthis paper. “AUC” is the Area Under the effort-vs-PD Curve[21]. AUC represents the accumulated performance in all range.
If AUC is close to 1 or its upper-limit, it means the predictive
performance is high. If AUC is around 0.5 or less, theprediction is meaningless. “ddr” is the “defect detection rate”[22]. In this paper, “ddrx” means fault detection rate at the x%

1

Faults detected
0.4
0.6
0.8

Data
set

AUC
ddr20

0.2

TABLE IV. PERFORMANCE IMPROVEMENT IN BINARY CLASSIFICATION

of effort. In maintenance, affordable effort is usually low. Thus,
this measure directly answers to practitioners’ questions suchas, “How many faults are detected in the review of the first
10% of LOC?” High ddrx means high predictive performance.
Though some previous studies used 20% as x, 20% is too big
for large systems in our industrial experience. Therefore, wemainly use ddr10 rather than ddr20 in this paper. If ddr10 is lessthan 0.1, it means the benefit of the prediction is nothing.

Optimal Model
Prediction Model

05) Results
Table IV shows the averages of performance measures of
the 100 times validations for both data sets. “Model MET” and“Model MET+IS” columns show the calculated measures forboth models. “Imprv. by IS” column shows each improvementfrom MET to MET+IS. The results show adding ImpactScale
improves the predictive performance statistically significantly
for every performance measure in both DS1 and DS2. Thisevaluation supports RQ1 is YES.

0

0.2

0.4
0.6
Effort0.8

1

Figure 7. Example of effort-based cumulative lift chart, AUC and ddr2) Fault density prediction using Poisson regression
To predict fault density, a predicting model which outputs acontinuous value or count value is required. In this paper, weused Poisson regression model [24], which is a common model
to predict the count of probabilistic events and has been used in
the software fault prediction literature [5][25]. An expressionof Poisson regression model is generally represented as
y =exp( b1x1+b2x2+ … +b0 ).
To estimate the partial regression coefficients, themaximum likelihood estimation method is used. A response
variable y (0<y) is interpreted as the expectation of the numberof faults. Since its response variable is count of events, thedensity of events can be straightforwardly derived. To predict
density, an offset term “log(LOC)” with a fixed coefficient 1.0
is added to the linear predictor as follows:y =exp( b1x1+b2x2+ … +b0+log(LOC)).
Notwithstanding the existence of the offset term, “log(LOC)”
can be still added to explanatory variables.
3) Model Selection, training and testing
We selected best models, trained and tested as subsection D
except that the number of faults is directly for training.
4) Results
Table V and Figure 8 show the results of 100 times
validations. In the figure, each red dashed curve shows theperformance of the model without ImpactScale (DS1-MET /DS2-MET). Each blue solid curve shows the performance ofthe model with ImpactScale (DS1-MET+IS / DS2-MET+IS).
White dots on the curves are measured points of ddr10 and ddr20
shown in Table V. In both DS1 and DS2, MET+IS models
outperform MET models in most range of the curves and all
three measures in the table. Especially, the improvement of
ddr10 is remarkable. The results mean that 1.5 times faults canbe detected in the 10% of effort. This evaluation supports RQ1
is YES.

TABLE V. PERFORMANCE IMPROVEMENT IN FAULT DENSITY PREDICTION AND EFFORT-AWARE MODEL
DS1-MET
mean(stddev)

DS1-MET+IS
mean(stddev)

Imprv.
by IS†

DS2-MET+IS
mean(stddev)

Imprv.
by IS†

AUC
ddr10
ddr20

0.635
(0.027)0.680(0.027)+0.045AUC
0.669(0.025)0.714(0.025)0.186(0.042)0.296(0.051)ddr10
0.225(0.047)0.343(0.046)
1.60
0.325(0.043)0.470(0.055)ddr20
0.374(0.053)0.518(0.051)1.45
The upper-limit of the AUC is 0.977.
The upper-limit of the AUC is 0.984.
†All improvements are significant (P<0.001) in Wilcoxon’s signed rank test.

+0.045Perf.
measureDS2-MET
mean(stddev)

1.53
1.39

DS1-MET

DS2-MET

DS1-MET+IS

DS2-MET+IS

0.2

0.2

Faults detected
0.4
0.6

Faults detected
0.4
0.6

0.8

0.8

1

1

Perf.
measureOptimal00Optimal
0

0.2
0.4
0.6
0.8
Effort (LOC inspected)

1

0

0.2

0.4
0.6
0.8
Effort (LOC inspected)

1(b) Effort-based cumulative lift chart of DS2(a) Effort-based cumulative lift chart of DS1

Figure 8. Effort-based comparison between models without ImpactScale (MET) and with ImpactScale (MET+IS)(100 times average)

F. Comparison with network measures (RQ2)
Zimmermann et al. introduced social network analysis(SNA) on a software dependency graph to predict faults [11].
They and several replication studies [26][27] showed theeffectiveness of network measures. Since ImpactScale is alsomeasured on a dependency graph, in order to confirm its
meaningfulness, RQ2 must be assessed.
RQ2: Does adding ImpactScale to existing product metrics
and network measures improve predictive performance?
1) Network measures in SNA
Network measures are measures of various network
topological characteristics. Zimmerman collected 58 network
measures using UCINet tool [28] and predicted faults withthem. To see the complete list of the measures and their
explanation, refer to [11] and [29]. We also used UCINet as
previous studies [11][26][27]. We collected network measureson the same dependency graph of DS1 on which ImpactScale
was measured. Since the dependency graph of DS2 is too large
for UCINet, we used only DS1 in this evaluation.
2) Principal Component Regression
We also use fault density prediction by Poisson regression
and the effort-aware model. Explanatory variables are so many
(over 60) that multicollinearity problem is inevitable. To copewith multicollinearity, we use principal component analysis(PCA) [30] as previous studies. PCA is an unsupervisedalgorithm to find principal components (PCs). Since all PCs are
orthogonal, multicollinearity can be avoided in regression
using PCs as explanatory variables. This combination is called
principal component regression.

3) Model selection, training and testing
We set up four models as follows.
MET
MET+IS
MET+SNA
MET+SNA+IS

a model consisted of existing productmetricsa model consisted of existing productmetrics and ImpactScalea model consisted of existing productmetrics and network measuresa model consisted of existing productmetrics, network measures and ImpactScale

For each validation, for each model, all of metrics andmeasures were log-transformed and then were transformed to
PCs using PCA. Next, PCs up to 99% cumulative variancewere used as explanatory variables in Poisson regression. The
average numbers of used PCs were 6.0 (MET), 7.0 (MET+IS),
27.8 (MET+SNA), and 28.8 (MET+SNA+IS). Testingprocedure is the same with subsection E.
4) Results
To compare the models, we use hierarchical modeling. The
results are shown in Figure 9. Four circles represent the four
predicting models with their performance measures, AUC andddr10. Each solid arrow means a simpler model is extended to amore complex model by adding a set of variables. For example,
arrow (a) shows that adding ImpactScale to MET improves thepredictive performance by 0.054 in AUC and by 0.074 in ddr 10.
It is consistent with the results of subsection D and E. Arrow(b) shows the effectiveness of network measures, and it
supports previous SNA studies.

To assess RQ2, we focus on arrow (d) and (e). Arrow (d)shows that adding ImpactScale to MET+SNA is significant. Itmeans that ImpactScale contains distinct explanatory factor
from network measures and that adding ImpactScale is always
meaningful. Arrow (e) shows that ImpactScale has higher
detection rate than network measures in the case effort isrestricted. However, overall predictive performance is slightly
lower. At any rate, it can be said the predictive performancedelivered by solo ImpactScale is comparable to a set of
network measures. These results support RQ2 is YES. Network
measures will be discussed further in Related works section.
(a) +IS+0.054+0.074
MET
AUC: 0.619
ddr10: 0.192(e) −SNA+IS−0.009+0.026**
MET+SNA
AUC: 0.684
ddr10: 0.240(b) +SNA+0.065+0.048

MET+SNA+IS
AUC: 0.698
ddr10: 0.279(d) +IS+0.014+0.039DISCUSSION

In this section, we discuss whether ImpactScale surely
contributes to fault prediction and whether the definition of
ImpactScale in section II is valid.
A. Contribution to Fault Prediction
First, we tested the correlations between ImpactScale andother metrics. Table VI shows the Spearman rank correlationsof all pairs of metrics in Table II in DS1. In the table, thecorrelations of ImpactScale are at most 0.38 and obviously low.
The result in DS2 is similar. For network measures, thecorrelations between ImpactScale and them are at most 0.60
and enough low to deliver the improvement shown in Figure 9.
The results mean ImpactScale is independent of other metrics.
TABLE VI. SPEARMAN RANK CORRELATIONS BETWEEN METRICS(THE FIGURES OVER 0.5 ARE SHOWN IN BOLD LETTERS.)
LOC
WMC
MaxVG
SectionCalls
Fan-inFan-out

WMC MaxVG Section
0.90
0.71
0.80
0.88
0.59
0.35
-

Calls
0.69
0.51
0.34
0.78
-

Fan-in Fan-out
-0.26
0.66
-0.13
0.48
-0.05
0.32
-0.33
0.79
-0.32
0.94
-0.32
-

0.6

LOC
WMC
MaxVG

0.4

Sections
Calls
Fan-in0.2

Fan-out
IS

0
0.2

0.4

0.6

0.8

Metrics

AUC

ddr10

LOC

0.591

0.137

WMC

0.586

0.107

MaxVG

0.578

0.073

Sections

0.596

0.156

Calls

0.579

0.173

Fan-in

0.549

0.139

Fan-out

0.580

0.162

IS

0.654

0.242

1

Figure 10. Results of fault density predictions with single metricFigure 9. Hierarchical model comparison between network measures andImpactScale. All improvements and deterioration are significant in Wilcoxon’s
signed rank test. (*: P<0.05, **: P<0.01, unmarked: P<0.001)

IV.

0.8

0

(c) +SNA+0.025+0.013*

MET+IS
AUC: 0.673
ddr10: 0.266

1

IS
0.18
0.11
0.04
0.31
0.33
0.17
0.38

Next, we predicted the fault density by Poisson regression
with single metric. The results (100 times average) of theprediction are shown in Figure 10. For each metric except
ImpactScale, its AUC is only slightly higher than 0.5, and its
ddr10 is around 0.1; therefore, it has little predictive ability. In
contrast, ImpactScale delivers obviously high predictive
performance. Therefore, ImpactScale strongly contributes tothe fault prediction in DS1. The same goes for DS2.

On the basis of the results of RQ1, RQ2 and this subsection,
we can say that Hypothesis 1 is valid in this paper.
Hypothesis 1: A metric that quantifies the scale of change
impact can improve the performance of fault prediction inlarge software systems.
B. Validity of ImpactScale Definition
In this subsection, we tested the validity of the definition of
ImpactScale. We modified the definition of ImpactScale andexamined how much the predictive performance of the model
varies by using the ddr10.
0.40
ddr
10

0.45
ddr
10

0.35

0.40
0.35

0.30

0.30
0.25
0.25
0.20

0.20

0.15
0.10

DS1
1

2

3

4

5

6

7

8

9 10

0.15
0.10

DS2
1

2

3

Limit of Distance of Path Finding

4

5

6

7

8

9 10

Figure 11. Effect of considering distant nodes

First, we tested the meaningfulness of considering distantnodes on a propagation graph. We limited the maximum lengthof path for calculating ImpactScale. Figure 11 shows the effectof considering distant nodes by changing the definition of
ImpactScale using the best model in subsection III.E including
modified ImpactScale for DS1 and DS2. For each box plot, thehorizontal axis shows each limit value of the maximum graphdistance for path finding. When the limit is 1, ImpactScale isvery similar to the sum of Fan-in and Fan-out. Therefore, if
ddr10 at limit 1 were large enough, considering distant nodes
would be meaningless. In the figure, the ddr10 curve of DS1 isincreasing until limit 5, and the ddr10 curve of DS2 isincreasing until limit 3. These results mean that theconsideration is meaningful. Besides, the minimum distances tobe considered are different depending on the software systems.
Since distant enough nodes hardly influence the ddr10 of both
DS1 and DS2, the limit distance has only to be an adequately
large number.

0.4010ddr
0.35

0.4510
ddr
0.40

DS1

0.35

0.30

0.30

0.25

0.25

0.20

0.20

0.15

0.10

DS2

0.15
MET

NoCut

MET+IS

0.10

MET

NoCut

MET+IS

Figure 12. Effect of relation-sensitive propagation

Next, we tested the meaningfulness of the relation-sensitive
propagation. In each box plot in Figure 12, the model MET+IS
is the best model with ImpactScale, which was selected in themanner in subsection III.E. The model MET is the best model
without ImpactScale. The model NoCut is the best model withmodified ImpactScale that does not cut any propagation3. On
the basis of the comparison, the ddr10 of NoCut has very little
improvement over the ddr10 of MET. Therefore, it can be said
that the definition without the cut rules is meaningless.
Conversely, it can be said that the cut rules make thecorrelation between ImpactScale and faults higher. The results
examined by using ddr20 and AUC were similar to Figure 11
and Figure 12. Consequently, we can say that Hypothesis 2 isvalid in this paper.
Hypothesis 2: The propagation model with relationsensitive propagation has enough predictive performanceto substitute for an accurate but expensive analysis.
V.

THREATS TO VALIDITY

In this section we discuss the threats to the validity of our
proposal and results. ImpactScale has no language specificfeature in its definition, but the target systems of our evaluation
are written in COBOL. Since COBOL differs from otherlanguages in paradigms and manners, some consideration
should be made to expand the discussion in this paper to otherlanguages. For example, we used a function (a program in
COBOL) as a unit for predicting faults, but in other languages
such a unit may usually be a class or a source file, which is agroup of functions. The target systems are in the domain ofaccounting software. Further evaluations for systems in various
domains of software should be performed.
We used seven traditional metrics to evaluate our results. If
target systems were written in object oriented (OO) languages,
many existing OO metrics [31] should be considered. We stillneed to study further whether ImpactScale is even meaningfulin addition to a rich set of metrics in such OO languages.
The measurement of ImpactScale requires miscellaneous
dependency information such as a call graph. If the accuracy of
the information is low, the calculated ImpactScale may notcorrectly reflect the total quantity of the affected areas. Since
ImpactScale is a summary statistic, it is expected not to besensitive to the accuracy, but we have not estimated theinfluence of the accuracy yet. In this paper, to avoid theinfluence, we chose systems written in COBOL because
dynamic bindings in such systems are usually small.
3

That is, for p,n  Rel, projection Cut(p,n) = False.

VI. RELATED WORKS
Some existing code metrics, such as Fan-in, Fan-out,
LCOM and CBO [31], are used to measure the software
structure consisting of code entities and their relations [2].
They consider only direct relations between adjacent nodes in adependency graph, but our evaluation proved that indirect anddistant nodes in the graph strongly affect faults. Therefore, weshould take those nodes into consideration to predict faults.
Process metrics regarding the amount of changes work well
as explanatory variables in fault prediction [4][5][6]. It isinteresting whether ImpactScale is independent of such process
metrics. Cataldo et al. [10] reported that the amount of changes
and logical coupling are independent of each other and bothcorrelate with the faults. Since logical coupling andImpactScale share common factors such as dependencies andchange impact, we expect that ImpactScale is also independentof the amount of changes.
Geiger et al. showed some relations between code clones
and logical couplings [32]. Since code clones cannot be fully
captured by call dependencies and data dependencies, someparts of logical couplings are not covered by ImpactScale. It iseasy to add relations of code clones into a propagation graphfor ImpactScale (e.g., add a new relation type CLONE), and the
addition may improve predictive performance further.
In recent studies, metrics traversing software dependency
graphs are emerging. Inoue et al. proposed Component Rank
[33] to find significant components by considering use dependencies between software components by using a Markov
Chain model. Hayase et al. proposed Maintenance Point [1] toestimate the maintenance effort using a dependency graphconstructed by change impact analysis. Maintenance Point iscalculated as the weighted sum of efforts of the affected
modules. Several studies [16][17] have proposed change
prediction models based on the assumption that change impact
propagates probabilistically. Tsantalis’s model [17] predicts thechange probability of a module using change history. Eachchange must be identified as propagated or originating change
by human intervention, thus the model has scalability problem.
Zimmermann et al. [11] regarded binary modules as actors
and applied SNA to the dependency graph. They andreplication studies [26][27] analyzed the target system withprincipal component regression by using over 50 network
measures in SNA and existing product metrics. They showedthat adding network measures to existing product metrics
improved the predictive performance. The principal
components composed of a lot of variables are very difficult tointerpret by human analysts. In contrast, since ImpactScale is a
designed and stand-alone metric correlated with faults, its
interpretation is easy and intuitive. Besides, ImpactScale cancapture data dependency handily by relation-sensitiveness.
VII. CONCLUSION
We defined a new product metric ImpactScale based on the
hypothesis that the scale of change impact is the significant
factor of faults in large software systems. The change
propagation model for ImpactScale is characterized byprobabilistic propagation and relation-sensitive propagation.
ImpactScale can be calculated in practical time for fairly large
systems, and its interpretation is intuitive.

We predicted faults of two large enterprise softwaresystems and evaluated the predictive performance using
precision/recall/F1 measures and the effort-aware models. In
all evaluations, the predicting models with ImpactScale moreaccurately predicted faults than did the models without
ImpactScale. By adding ImpactScale to existing productmetrics, the number of faults detected in the first 10% of thetotal LOC increased by 50% or more. We also compared
ImpactScale with network measures. The predictive performance of ImpactScale was comparable with over 50 network
measures, and adding ImpactScale enhanced the model with aset of network measures and existing product metrics. These
results support the effectiveness of a new metric, ImpactScale.
We have already applied ImpactScale to our customers’
enterprise systems to support quality assurance by using fault
prediction. In one case, we performed extra review andinspection of modules in order of predicted fault density withthe effort equivalent to 1/20 of annual man-hours in a system.
In the result, eight faults were found, and system failures wereprevented in advance (the system failed several times per year).
For future work, we would like to prove the effectiveness of
ImpactScale on other languages such as Java. When we
designed ImpactScale, we intended that it would be a predictor
for the effort and the decay of software structure. Thus, wewould like to apply ImpactScale to estimate them.
ACKNOWLEDGEMENT
We thank Prof. Kusumoto, Dr. Matsushita, and members ofSoftware Engineering Laboratory of Osaka University for theiradvice. We appreciate valuable discussions with members of
Empirical Software Engineering Group of NICTA (National
ICT Australia). We are grateful to Mr. Kamakura and membersof Cloud Application Center of Fujitsu for providing the data
sets and their support.
REFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]

Basili, V. R., Briand, L. C., and Melo, W. L., “A validation of objectoriented design metrics as quality indicators,” IEEE Trans. Softw. Eng.
22, 10, pp.751-761, 1996.
Briand, L. C., Wüst, J., Daly, J. W., and Porter, D. V., “Exploring therelationships between design measures and software quality in objectoriented systems,” J. Syst. Softw. 51, 3, pp.245-273, 2000.
Fenton, N. E., and Ohlsson, N., “Quantitative analysis of faults andfailures in a complex software system,” IEEE Trans. Softw. Eng. 26, 8,
pp.797-814, 2000.
Ostrand, T. J., and Weyuker, E. J., “The distribution of faults in a large
industrial software system,” ACM SIGSOFT Int’l Symp. on Software Testingand Analysis. ISSTA, pp.55-64, 2002.
Graves, T. L., Karr, A. F., Marron, J. S., and Siy, H., “Predicting fault
incidence using software change history,” IEEE Trans. Softw. Eng. 26, 7,
pp.653-661, 2000.
Nagappan, N., and Ball, T., “Use of relative code churn measures to predict
system defect density,” Int’l Conf. on Softw. Eng. ICSE, pp.284-292, 2005.
Gall, H., Hajek, K., and Jazayeri, M., “Detection of logical couplingbased on product release history,” IEEE Int’l Conf. on Softw. Maint.
ICSM, pp.190-198, 1998.
Hassan, A. E., and Holt, R. C., “Predicting change propagation in softwaresystems,” IEEE Int’l Conf. on Softw. Maint. ICSM, pp.284-293, 2004.
Haney, F.M., “Module connection analysis – a tools for schedulingsoftware debugging activities,” Fall Joint Computer Conference,
pp.173-180, 1972.

[10] Cataldo, M., Mockus, A., Roberts, J. A., and Herbsleb, J. D., “Software
dependencies, work dependencies, and their impact on failures,” IEEE
Trans. Softw. Eng. 36, 2, pp.864-878, 2009.
[11] Zimmermann, T., and Nagappan, N., “Predicting defects using network
analysis on dependency graphs,” Int'l Conf. on Softw. Eng. ICSE,
pp.531-540, 2008.
[12] Bohner, S. A., and Arnold, R. S. (Eds.), “Software change impact
analysis,” Bohner, S. A. and Arnold, R. S., “An introduction to softwarechange impact analysis,” IEEE Computer Society Press, pp.1-26, 1996.
[13] Grove,D., and Chambers,C., “A framework for call graph construction
algorithms,” ACM Trans. Program. Lang. Syst. 23, 6, pp.685-746, 2001.
[14] Law, J., and Rothermel, G., “Whole program path-based dynamic
impact analysis,” Int’l Conf. on Softw. Eng. ICSE, pp.308-318, 2003.
[15] Ren, X., Shah, F., Tip, F., Ryder, B. G., and Chesley, O., “Chianti: a toolfor change impact analysis of Java programs,” Conf. on Object-Oriented
Prog., Syst., Lang., and App. OOPSLA, pp.432-448, 2004.
[16] Sharafat, A. R., and Tahvildari, L., “A probabilistic approach to predictchanges in object-oriented software systems,” European Conf. on Softw.
Maint. and Reeng. CMSR, pp.27-38, 2007.
[17] Tsantalis, N., Chatzigeorgiou, A., and Stephanides, G., “Predicting theprobability of change in object-oriented systems,” IEEE Trans. Softw.
Eng. 31, 7, pp.601-614, 2005.
[18] Whaley, J., and Lam, M. S., “Cloning-based context-sensitive pointeralias analysis using binary decision diagrams,” ACM SIGPLAN Conf. on
Prog. Lang. Design and Impl. PLDI, pp.131-144, 2004.
[19] Chatterjee, S. and Hadi, A. S., "Regression analysis by example, 4th
Edition," John Wiley and Sons, 2006.
[20] Arisholm, E. and Briand, L. C., "Predicting fault-prone components in aJava legacy system," ACM/IEEE Int’l Symp. on Empirical Software
Engineering, ISESE, pp.8-17, 2006.
[21] Menzies, T., Milton, Z., Turhan, B., Cukic, B., Jiang, Y. and Bener, A.,"Defect prediction from static code features: current results, limitations,
new approaches," J. Autom. Softw. Eng., Vol.17, pp.375-407, 2010.
[22] Mende, T. and Koschke, R., "Effort-aware defect prediction models,"
European Conf. on Softw. Maint. and Reeng., CSMR, pp.107-116, 2010.
[23] Kamei, Y., Matsumoto, S., Monden, A., Matsumoto, K.., Adams, B. andHassan, A. E., "Revisiting common bug prediction findings using effortaware models," IEEE Int’l Conf. on Softw. Maint., ICSM, pp.1-10, 2010.
[24] Cameron, A. C., and Trivedi, P. K., “Regression analysis of count data,”
Cambridge University Press, 1998.
[25] Khoshgoftaar, T. M., Geleyn, E., and Gao, K., “An empirical study ofthe impact of count models predictions on module-order models,” IEEE
Int’l Softw. Metrics Symp. METRICS, pp.161-172, 2002.
[26] Tosun, A., Turhan, B. and Bener, A., "Validation of network measures
as indicators of defective modules in software systems," Int’l Conf. on
Predictor Models in Softw. Eng., PROMISE, pp.5:1-5:9, 2009.
[27] Nguyen, T., Adams, B. and Hassan, A., "Studying the impact of
dependency network measures on software quality," IEEE Int’l Conf. on
Softw. Maint. ICSM, pp.1-10, 2010.
[28] Borgatti, S.P., Everett, M.G. and Freeman, L.C., “UCINet for Windows:
software for social network analysis,” Analytic Technologies, 2002.
[29] Hanneman, R. A. and Riddle, M., "Introduction to social network
methods," University of California, Riverside, 2005.
[30] Jolliffe, I. T., “Principal component analysis, 2ed,” Springer, 2002.
[31] Chidamber, S. R., and Kemerer, C. K., “A metrics suite for object
oriented design,” IEEE Trans. Softw. Eng. 20, 6, pp.476-493, 1994.
[32] Geiger, R., Fluri, B., Gall, H., and Pinzger, M., “Relation of code clones
and change couplings,” Fundamental Approaches to Softw. Eng., LNCS
3922, Springer, pp.411-425, 2006.
[33] Inoue, K., Yokomori, R., Fujiwara, H., Yamamoto, T., Matsushita, M., andKusumoto, S., “Component Rank: relative significance rank for software
component search,” Int’l Conf. on Softw. Eng. ICSE, pp.14-24, 2003.
[34] Hayase, Y., Matsushita, M., Kusumoto, S., Inoue, K., Kobayashi, K.,
and Yoshino, T., “A metric for estimating maintenance effort based onchange impact analysis,” IEICE Transactions on Information and
Systems 90, 10, pp.2736-2745, 2007.