Prediction of Fault-proneness at Early Phase in Object-Oriented Development

Toshihiro Kamiya, Shinji Kusumoto and Katsuro Inoue
Graduate School of Engineering Science, Osaka University
1-3 Machikaneyama, Toyonaka, Osaka, 560-8531, Japan
E-mail: kamiya@ics.es.osaka-u.ac.jp
Abstract
To analyze the complexity of object-oriented software,
several metrics have been proposed. Among them, Chidamber and Kemerer’s metrics are well-known ones asobject-oriented metrics. Also, the effectiveness has been
empirically evaluated from the viewpoints of estimating
the fault-proneness of object-oriented software. In theevaluations, their metrics were applied to not design
specification but the source code because some of themmeasure an inner complexity of a class, and such information can not be obtained until the algorithm and structure of the class are determined at the end of design phase.
However, the estimation of the fault-proneness should bedone in the early phase to effectively allocate effort for
fixing the faults. This paper proposes a new method toestimate the fault-proneness of the class in the early phase,using several complexity metrics for object-oriented software. In the proposed method, we introduce four checkpoints into the analysis / design / implementation phase,and estimate the fault-prone classes using the applicable
metrics at each checkpoint.

1. Introduction
In attempt to reduce the number of delivered faults, it
was reported that most companies spend between 50-80%
of their software development effort on testing[7]. Therefore, reducing the effort of testing is a key to high productivity in software development. Software review is one of
the most effective techniques to reduce the testing effort.
In order to effectively review and test the software products, it is needed to identify the fault-prone modules sothat review and testing effort can be concentrated on themodules[1]. Several complexity metrics have already been
proposed to localize such the fault-prone modules. Then,

Chidamber and Kemerer proposed six complexity metrics
for OO software [5]. However, the metrics suite was applied to the source code because some of their metrics
measure an inner complexity of a class, and such information cannot be obtained until the algorithm and structureof the class are determined at the end of design phase. The
estimation of the fault-proneness should be done in theearly phase to effectively allocate the effort for fixing thefaults.
This paper proposes a new method to estimate thefault-proneness of the class in the early design phase,using several complexity metrics for OO software. In the
proposed method, we introduce four checkpoints into thedevelopment process based on OMT[14]. According to thedata obtained from the products (design specification andsource code) at each checkpoint, we use only the applicable metrics among the complexity metrics and estimate thefault-proneness by using the multivariate logistic regression analysis[18]. Then, we have applied the proposed
method to an experimental project held at a computercompany. The analysis result of the experiment shows thevalidity and usefulness of the proposed method in theestimation of the fault-proneness.

2. Preliminary
2.1. Object-Oriented Design Method
There exist several object-oriented design methods[2][6][14][15]. Among them, OMT[14] is a broad methodology that addresses most aspects of the OO analysis
and design technology and incorporates the lessons learned from real projects as well as from other methodologies.
OMT consists of three phases: Analysis, System Design
and Object Design. Analysis is concerned with under-standing and modeling the application and the domain
within which it operates. System Design determines theoverall architecture of the system. The system is organized
into subsystems. Concurrency is organized by grouping
objects into concurrent tasks. Object Design elaborates,refines and then optimizes the analysis models to produce
a practical design. Algorithms and data structures used in
the classes are determined.
From the viewpoint of the structure of the class, elaboration is carried out as the following six steps:
1. Identify classes in the target system.
2. Identify the reference-relationships between the classes.
3. Identify the attributes of the classes.
4. Determine the derivation-relationships between theclasses.
5. Define the operations based on the functional models.
6. Design the algorithms to implement the operations.

2.2. Complexity
Software

Metricsfor

DIT(depth of inheritance tree):
Depth of inheritance of the target class is the DIT metricfor the class.
NOC(number of children):
NOC means the number of immediate subclass subordinated to the target class in the class hierarchy.
CBO(coupling between object classes):
CBO for the target class is a count of the number of otherclasses to which it is coupled. A class is coupled toanother if it uses is methods or instance variables of another.
RFC(response for a class):
RFC = | RS | where RS is the response set for the class. RS= M ∪ (∪all i Ri ) where Ri = set of methods called bymethod i and M = set of all methods in the class.

Object-Oriented

Software metrics are quantitative measures of software
products and process[8]. Especially, the complexity metricfor a program code is the most well-known metric, and isusually used in the implementation phase and test phase,since it is a good indicator of whether the product is welldesigned, understandable, and easy to modify. For example, Software Science by Halstead[9] and Cyclomatic
Number by McCabe have been used in many softwaredevelopment organizations. Then, Chidamber and Kemerer[5] have defined a suite of metrics for OO design.
The metrics suite includes six metrics and mainly evaluate
the relationship between the classes on the design specification. Each metric evaluates the complexity of the target
class and the definitions of the metrics are as follows[5]:WMC(weighted methods per class):
Consider the target class C1, with methods M1, ..., Mn,
that are related in the class. Let c1, ..., cn be the complexity of the methods. Then WMC = Σ ci . You ought tochoose an adequate interval scale metric f that gives ci =f(Mi ).
In both [1] and [5], they made an assumption that all
method complexities are considered to be unity, thenWMC is the number of methods. We also use this assumption, so we plainly employ a term NIM(Number ofInstance Methods)[12] as WMC.

LCOM(lack of cohesion in methods):
Consider the target class C1 with $n$ methods M1, ..., Mn.
Let Ii = set of instance variables used by methods Mi .
There are n such sets I1, ..., In. Let P = { (Ii , Ij) | Ii ∩ lj =φ } and Q = { (Ii , Ij) | Ii ∩ Ij ≠ φ }. If all n sets I1, ..., In are φthen let P = φ. LCOM = max(| P | - | Q |, 0).
Besides the above metrics, NIV (Number of Instance Variables) is frequently used as a design metric of objectoriented software[12].
NIV(number of instance variables):
NIV is a number of instance variables of the target class.

2.3. Evaluation of Complexity Metrics
Chidamber and Kemerer have presented empirical dataon their metrics from actual commercial systems. They
have adopted the Weyuker’s properties to evaluate their
metrics (Weyuker has developed a list of desiderata forsoftware metrics and has evaluated a number of existing
software metrics using these properties[16]), and concluded that the metrics are generally satisfied the Weyuker’s
properties. Basili et al. empirically evaluated that Chidamber and Kemerer’s OO metrics show to be better predictors than the best set of traditional code metrics[1].

Checkpoint(CP1)
Entity and Relation(CP2)
Structure and inheritance(CP3)
Algorithm(CP4)
Implementation

Table 1. Checkpoint and available metrics.
Added Information
Available Metrics
Reference-relationship among classes and NIV, CBON
attributes of classes
Class hierarchy, methods and reused library NIV, CBON, CBOR, CBO, NIM, DIT,NOC
Algorithm of the methods
NIV, CBON, CBOR, CBO, NIM, DIT,
NOC, RFC, LCOM
Source code
NIV, CBON, CBOR, CBO, NIM, DIT,
NOC, RFC, LCOM, SLOC

In both [1] and [5], they collected the metrics values
from source codes. But the design specification is constructed in the earlier development phase. Thus, by applying the metrics to the design specification, it is possible
to estimate the fault proneness more effectively. It would
produce efficient fault detection to do the resourceallocation and coordinate the scheduling based on the
result of fault-estimation. However, it is very difficult toapply the metrics to the design specification. For example,
with respect to calculating the RFC and LCOM, we needthe detail information about the algorithms in the method
and call-relationship between the methods. These informations are usually described on the design specification
at the later design phase (just before the implementation
phase).

3. Proposed Method
3.1. Key Idea
We regard the analysis / design / implementation phaseas a series process in which the information about software
product gradually increases as the process progresses. As
described in Section 2, some of the metrics can be applied
to the design specification at the earlier design phase, andsome of them can be applied at the later design phase.
Based on the fact, as design phase progress, we estimate
the fault-proneness only using the applicable metrics tothe design specification.
At first, we introduce four checkpoints in the development process and identify which information has been
added to the design specification at each checkpoint. Then,
we define the subset of the conventional metrics applicableto the design specification developed at each checkpoint.
Finally, at each checkpoint, we estimate the fault-

proneness of the class using the multivariate logistic regression analysis with the applicable metrics.

3.2. Checkpoints and Applicable Metrics
In Section 2, we divided the development process of
OMT into the following six steps. Based on the division,
we introduce the four checkpoints in the analysis / design /implementation phase of OMT.
(CP1) Entity and relation
CP1 is the checkpoint where Steps 1, 2 and 3 have been
completed. That is, reference-relationship between classes(coupling) and attributes (instance variable) of the classes
have been determined. NIV can be calculated from theattribute information. The derivation-relationships havenot been described. CBO can be calculated from the reference-relationship between classes, but reference to thereused class is not clearly described so that the value of
CBO is not correct.
(CP2) Structure and inheritance
CP2 is the checkpoint where Steps 4 and 5 have been
completed. That is, derivation between classes and themethods in the classes have been determined and classhierarchy tree is clearly described. Thus, DIT can be calculated. NIM can be calculated from the information of
the methods. The reused classes are determined so that
CBO can be calculated correctly.
(CP3) Algorithm
CP3 is the checkpoint where Step 6 has been completed.
That is, algorithms in each method and call-relationshipbetween the methods are determined. Based on the information, LCOM and RFC can be calculated.

(CP4) Implementation
CP4 is the checkpoint where source code has been implemented. For each class, SLOC (source lines of code)
can be calculated.

wise variable selection process[18], using observed metrics and fault data.

4. Empirical Evaluation
CBO at CP1 (hereafter: CBON, CBO newly developed)
differs from original definition. However, our previous
research result showed that CBON have highly correlatedwith the fault-proneness[10]. We also introduce
CBOR(CBO reused) that is defined as CBOR = CBO –CBON. Table 1 summarizes the checkpoints and the metrics which can be calculated at each checkpoint.

3.3. Multivariate Logistic Regression Analysis
Recently, the multivariate logistic regression analysis,
whose inputs are several metrics, has been frequently used
to evaluate the fault-proneness of the program[1][3].
A multivariate logistic regression model is based onfollowing relationship equation:
P(X1, ..., Xn) = 1 / { 1 + exp(-(C0 + C1 X1 + ... + Cn Xn))}
where P is the probability that errors is found in a class,and Xis are metrics of the class. If given metrics values
make P greater than 0.5, the class is predicted to havefaults (fault prone).
To obtain particular prediction equation P(X1, ..., Xn),
we need to determine the coefficients C0, ..., Cn by stepTable 2. Descriptive statistics of the 141 C++classes.
Metrics Min. Max. MeMean
Std. Dev.
dian
NIV
0
14
3
4.00
2.67
CBO
0
5
1
1.39
1.59
CBON
0
3
0
0.53
0.99
CBOR
0
4
1
0.86
0.99
NIM
0
22
3
5.73
4.86
DIT
0
6
4
3.44
1.41
NOC
0
0
0
0.00
0.00
RFC
0
27
7
8.23
6.81
LCOM
0
190
3
22.42
36.84
SLOC
0
420
71
96.43
81.01
Ec
0
17
0
0.57
1.93
Et(min.)
0
599
0
12.68
58.94
Ec is the number of faults detected in the class.
Et is the time spent for fixing the faults.

4.1. Outline of the experiment
The experimental project was performed at a computercompany for five days in August 1997. The main characteristics of the project can be summarized as follows:(1) The Developers were new employees of the computercompany and had just graduated from college in March
1997. All the developers studied object-oriented design
and programming in C++ Language.
(2) There are sixteen developer teams of four or five developers. Each team built a mail deliver system of an
identical requirement. The predetermined system requirements, the division to subsystems and subsystem's
interface designs were handed over to each team.
(3) After a team declared that its program was finished,
the instructors executed an acceptance test.

4.2. Empirical Data
We collected complexity metrics and fault data fromeach developer. Unfortunately, we could not collect thedesign specification based on OMT in this experiment. We
assume that all the information of the design specification
Table 3. Coefficients at Each Checkpoint.
Metrics
Coefficients
CP1
CP2
CP3
CP4
C0
-3.37
-1.23
-1.31
-2.69
NIV
0.420
EL
EL
EL
CBON
EL
EL
EL
EL
CBOR
0.934
0.890
EL
CBO
EL
EL
EL
NIM
0.336
EL
EL
DIT
-1.16
-1.28
-0.663
NOC
EL
EL
RFC
0.284
EL
LCOM
EL
SLOC
0.0302The ‘EL’ means that the metric was eliminated form
the prediction equation by a backward variable elimination process. The ‘-’s are not applicable at thecheckpoint.

Table 4. Fault Prediction at CP1.
Prediction
No fault
Fault
Actual No fault
112
2
Fault
18(43)
9(37)The figures before parentheses are the number ofclasses. The figures within parentheses are the number of faults in those classes.

Table 8. Fault Predict Precision at Checkpoint.
Checkpoint
CP1 CP2 CP3 CP4Correctness(%)
82
76
85
86
Completeness(%)
33
59
63
70
Error-based Com46
75
71
83
pleteness (%)

Table 5. Fault Prediction at CP2.
Prediction
No fault
Fault
Actual No fault
109
5
Fault
11(20)
16(60)

classes are predicted to be no-faulty and actually no-faulty.
Eighteen classes are predicted to be no-faulty but actually
faulty (include 43 faults).
Here, we introduce the following three criterions toevaluate the result of the prediction:

Table 6. Fault Prediction at CP3.
Prediction
No fault
Fault
Actual No fault
111
3
Fault
9(18)
18(62)

Correctness = CPF AF / (CPF AF + CPF AN)
Completeness = CPF AF / (CPF AF + CPN AF)
Completenesserror based = EPF AF / (EPF AF + EPN AF)

Table 7. Fault Prediction at CP4.
Prediction
No fault
Fault
Actual No fault
111
3
Fault
8(14)
19(66)
were included in the source code since it was implemented
based on the design specification. So, we collected themetrics values, which can be calculated at each checkpoint,
from source code by using a metrics tool that extracts ninemetrics from C++ source code[11].
We prepared a documentation tool to collect fault data.
We have eliminated the data of developers who did notreport data or irresponsible data. As a result, seventeen
members’ data (141 classes and 80 faults) are available.
Table 2 shows common descriptive statistics of the metrics
distributions.

4.3. Analysis
Table 3 shows the prediction model’s coefficients calculated by multivariate logistic regression analysis. DIT
appeared as a negative factor to complexity. This can be
explained by the fact that there were a lot of ‘dialog’classes developed and the functionalities were very simple
but the their DIT values were relatively large. LCOM iseliminated from a equation at CP4, and this result agrees
with the results in [1].
Tables 4, 5, 6 and 7 show the obtained prediction modelby the multivariate logistic regression with the data collected at each checkpoint. For example, in Table 4, 112

where CPF AF is the number of predicted faulty and actually faulty classes, CPF AN is number of predicted faulty but
actually no-faulty classes, CPN AF is number of predictedno-faulty but actually faulty classes, Ex is number of faults
found in corresponding Cx classes.
Table 8 shows the estimation accuracy of the fault
proneness at each checkpoint. According to the progress ofthe checkpoints, the estimation accuracy is improved.
CP4 is the last phase of the development process and theestimation accuracy at CP4 is the upper limit in this experiment.
Completeness at CP1 is relatively low (33%). On theother hand, correctness is high (82%). Thus, the estimation can be used to seed the fault-prone classes. The seededclasses become the candidates that should be reviewed andtested selectively. Also, the location of the seeded classes
might be the criterion of the judgment for review. For
example, if the seeded classes are concentrated on theimportant section of the design specification and the section is difficult to test, we should redesign it.
Though the metrics for algorithms of the method cannot be used at CP2, completeness at CP2 is excellent withrespect to the upper limit at CP4. It suggests that it would
be possible to estimate the fault-proneness from the design
specification in the design phase where the algorithms are
not determined, without source code.
The result of estimation at CP3 fell short of our expectations. We consider that the accuracy would be improvedby using ‘fine-grained’ C++ design metrics[3] together at
CP3. The accuracy would be improved by cyclomatic
number as WMC.

5. Conclusion and Future WorkIn this paper, we proposed a new method to estimate thefault-proneness of the class in the early design phase,using several complexity metrics for OO software. We
have introduced four checkpoints into the analysis/design/
implementation phase, in which particular subsets of
metrics are applicable. We have applied the proposed
method to an experimental project. The analysis resultshows the validity and usefulness of the proposed method.
Future research includes the following:(1) Extension of proposed method:
We are going to extend the proposed method to deal
with other OO design specifications and other complexity
metrics[4].
(2) Metrics for dynamic complexity:
Chidamber and Kemerer’s metrics evaluate the static
complexity of the OO software. Several OO design specifications includes dynamic information. It is necessary toevaluate such dynamic complexity.
(3) Metrics tool for design specification:
Currently, UML (Unified Modeling Language) has
been proposed as the standard description language for the
OO design specification[17]. We are going to develop themeasurement tool for design specification described in
UML.

Acknowledgements
We would like to thank the support of Mr. Yukio Mohri
and Mr. Yousuke Takahashi of Nihon Unisys corporation
for the experimental projects in Section 4. We also thankMr. Shuji Takabayashi of Nara Institute of Science andTechnology for his assistance in building a metrics tool.

References[1] V. R. Basili, L. C. Briand, W. L. Mélo: A validationof object-oriented design metrics as quality indicators,
IEEE Trans. on Software Eng., Vol. 20, No. 22, pp.
751-761 (1996).
[2] G. Booch: Object Oriented Analysis and Design with
Applications, The Benjamin / Cummings (1994).
[3] L. C. Briand, P. Devanbu, and W. Mélo: An Investi-

gation into Coupling Measures for C++, Proc. of
the19th Int’l Conference on Software Eng., Boston,
USA, pp.412-421 (1997).
[4] L. C. Briand, J. Daly, V. Porter and J. W st: Predicting Fault-Prone Classes with Design Measures in
Object-Oriented Systems, Proc. of the 9th Int’l Symposium on Software Reliability Eng., Paderborn,
Germany, pp.334-343 (1998).
[5] S. R. Chidamber and C. F. Kemerer: A metrics suitefor object-oriented design, IEEE Trans. on SoftwareEng., Vol. 20, No.6, pp.476-493 (1994).
[6] P. Coad and E. Yourdon: Object Oriented Analysis,
2nd ed., Yourdon Press (1991).
[7] J. S. Collofello and S. N. Woodfield: Evaluating theeffectiveness of reliability-assurance techniques,
Journal of Systems & Software, Vol.9, No.3, pp.191195 (1989).
[8] N. E. Fenton, and S. L. Pfleeger, Software Metrics, A
Rigorous & Practical Approach. 2nd ed., Thomson,
London (1996).
[9] M. H. Halstead: Element of software science, New
York, Elsevier North-Holland (1977).
[10] T. Kamiya, S. Kusumoto, K. Inoue and Y. Mohri:
Empirical evaluation of reuse sensitiveness of complexity metrics, Information and Software Technology (to appear).
[11] T. Kamiya, S. Takayabashi, S. Kusumoto and K.
Inoue: Measurement tool for complexity of C++ program, Objecto-Oriented ’98 Symposium, Tokyo, Japan (1998) (in Japanese).
[12] M. Lorenz and J. Kidd: Object-Oriented software
metrics, New Jersey, Prentice Hall (1994).
[13] T. J. McCabe: A complexity measure, IEEE Trans.
on Software Eng., Vol. SE-2, No.4, pp.308-320(1976).
[14] J. Rumbaugh, M. Blaha, W. Premerlani, F. Eddy and
W. Lorensen: Object Oriented Modeling and Design,
Prentice Hall (1991).
[15] S. Schlaer and S. Mellor: Object Oriented System
Analysis, Prentice Hall (1988).
[16] E. J. Weyuker: Evaluating software complexity
measures, IEEE Trans. on Software Eng., Vol.14,
No.9, pp.1357-1365 (1988).
[17] UML Modeling Language, Standard Software Notation. Available at <http://www.rational.com/>.
[18] SPSS Base 8.0 Applications Guide/Professional
Statistics, SPSS (1998).