JAIST Repository
https://dspace.jaist.ac.jp/

Title

A UML Approximation of a Subset of the CK Metrics
and Their Ability to Predict Faulty Classes

Author(s)CAMARGO CRUZ, Ana Erika

CitationIssue Date

2011‑09

Type

Thesis or Dissertation

Text version

author

URL

http://hdl.handle.net/10119/9945

RightsDescription

Supervisor:Professor Ochimizu Koichiro, 情報科学研究科, 博士Japan Advanced Institute of Science and Technology

A UML Approximation of a Subset of the CKMetrics and Their Ability to Predict Faulty Classes

by

CAMARGO CRUZ Ana Erika

submitted toJapan Advanced Institute of Science and Technology
in partial fulfillment of the requirements
for the degree ofDoctor of Philosophy

Supervisor: Professor Ochimizu Koichiro

School of Information Science
Japan Advanced Institute of Science and Technology

September 2011

Abstract
Design-complexity metrics, while measured from the code, have shown to be goodpredictors of fault-prone object-oriented programs and related to several other managerial factor such as productivity, re-work effort for reusing classes and design effort, andmaintenance effort. Some of the most often used metrics are the Chidamber and Kemerer
metrics (CK). Because earlier assessments of such managerial factors are desirable, prior
to the code implementation, our research mainly concerns two topics. The first one concerns to the how can we approximate the code CK metrics using UML diagrams; and thesecond one concerns the use of such UML approximations to predict faulty object-orientedclasses.
First, we define our UML metrics, approximations of the Weighted Methods per Class(WMC), Response For Class (RFC) and Coupling Between Objects (CBO) CK code
metrics using UML communication diagrams. Second, we evaluate our UML metrics
as approximations to their corresponding code metrics. Third, in order to improve theapproximations of our UML metrics, we study the application of two different data normalization techniques, and select the best one to be used in our experiments. Finally,
because code CK metrics have shown repetitively their ability to predict faulty code inseveral previous works, we evaluate our UML CK metrics as predictors of faulty code. In
order to do so, we first construct three prediction models using logistic regression withthe source code of a package of an open source software project (Mylyn from Eclipse),
and we test them with several other of its packages. Then, we applied these models to
three different small-size software projects, using, on the one hand, their UML metrics,
and on the other hand, their corresponding code metrics for comparison.
The results of our empirical study lead us to conclude that the proposed UML RFC and
UML CBO metrics can predict fault-proneness of code almost with the same accuracy
as their respective code metrics do. The elimination of outliers and the normalization
procedure used were of great utility, not only for enabling our UML metrics to predict
fault-proneness of code using a code-based prediction model but also for improving theprediction results of our models across different software packages and projects. As for
the WMC metrics, both the proposed UML and its respective code metric showed a poor
fault-proneness prediction ability.
Our plans for future work mainly concern the exploration of other areas of researchin which our UML metrics can be applied and as for the topic of fault prediction thefollowing subjects for further study have been considered: data normalization and otherpre-processing techniques, the study of other metrics to be included in our prediction
models (such metrics should be easily obtainable before the implementation of the systemand different to design-complexity metrics), and other methodologies to predict faultproneness of code (different to logistic regression).

iAcknowledgments
Foremost, the author wishes to express her most sincere gratitude to her principal
advisor Professor OCHIMIZU Koichiro of Japan Advanced Institute of Science and Technology for his constant encouragement, advises, kind guidance and support.
The author would like to extend her gratitude to the rest of her thesis committee:
Professor OGAWA Mizuhito, Associate Professor SUZUKI Masato and Associate Professor AOKI Toshiaki of Japan Advanced Institute of Science and Technology, and Professor
MATSUMOTO Kenichi of Nara Institute of Science and Technology for their valuable
reviews, advises and suggestions.
Last but not least, the author would like to express her gratitude to her mother Mariade los Angeles Cruz Salas, her father Valentin Camargo Perez, brother Emmanuel Ivan
Camargo Cruz and sisters Alba Olivia Camargo Cruz and Daysi Iliana Camargo Cruz for
their continuous encouragement, unconditional love and moral support.

ii

Contents
Abstracti

Acknowledgmentsii

1 Introduction

1

2 Background
2.1 Measurement and Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 The Chidamber and Kemerer Metrics . . . . . . . . . . . . . . . . . . . . .
2.3 Communication Diagrams UML 2.0 . . . . . . . . . . . . . . . . . . . . . .
2.4 Prediction Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4.1 Types of Measurement Scales . . . . . . . . . . . . . . . . . . . . .
2.4.2 Statistical Prediction Techniques . . . . . . . . . . . . . . . . . . .
2.4.3 Artificial Neural Networks . . . . . . . . . . . . . . . . . . . . . . .
2.4.4 Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . .

4
4
6
7
12
12
15
18
19

3 Definition of Goals and our UML CBO, RFC and WMC Metrics
3.1 Goal Question Metric Approach . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Definition: Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4 Definition: Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5 Definition: Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5.1 Metrics for UML Communication Diagrams . . . . . . . . . . . . .
3.5.2 Metrics for Classes derived from UML Communication Diagrams . .
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

21
22
25
26
26
26
26
29
34

4 UML CK Metrics Evaluation
4.1 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2 Measures for our Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2.1 Error Approximation . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2.2 Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2.3 Precision and Accuracy . . . . . . . . . . . . . . . . . . . . . . . . .
4.3 RFC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4 CBO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5 WMC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6 The Gap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

35
35
36
36
37
39
41
42
43
44
50

iii

5 Normalization
5.1 Comparing CK Data Distributions using Boxplots . . . . . . . . . . . . . .
5.2 Data Transformation and Normalization . . . . . . . . . . . . . . . . . . .
5.2.1 Logarithm Data Transformations . . . . . . . . . . . . . . . . . . .
5.2.2 Linear Scaling to Unit Variance . . . . . . . . . . . . . . . . . . . .

52
52
62
64
66

6 Fault Prediction
6.1 Construction of Univariate Logistic Models . . . . . . . . . . . . . . . . . .
6.1.1 Independent Variables . . . . . . . . . . . . . . . . . . . . . . . . .
6.1.2 Dependent Variable . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.1.3 Procedure for Model Construction . . . . . . . . . . . . . . . . . . .
6.2 Validation of UL models . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.2.1 Importance of each independent variable . . . . . . . . . . . . . . .
6.2.2 Overall goodness of fit . . . . . . . . . . . . . . . . . . . . . . . . .
6.2.3 Discrimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.2.4 Validation using different data . . . . . . . . . . . . . . . . . . . . .
6.2.5 Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.3 Multivariate Analysis and Correlation . . . . . . . . . . . . . . . . . . . . .
6.4 Guidelines for Further Use . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

75
76
77
77
78
81
82
83
84
85
87
87
92
93

7 Related Work96
7.1 CK and UML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
7.2 Fault Prediction for Object-Oriented Software . . . . . . . . . . . . . . . . 97
7.2.1 After Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
7.2.2 Before Coding using non UML metrics . . . . . . . . . . . . . . . . 99
7.2.3 Before Coding using UML metrics . . . . . . . . . . . . . . . . . . . 102
7.3 Comparison of UML metrics and Prediction Results . . . . . . . . . . . . . 103
8 Other Applications
108
8.1 Software Implementation Progress Estimations . . . . . . . . . . . . . . . . 108
8.2 Dynamic Coupling Measures . . . . . . . . . . . . . . . . . . . . . . . . . . 109
9 Conclusions and Plans for Future Work110

Bibliography

116

Publications

121

A Theoretical Construction of our CK metricsi
A.1 The DISTANCE Framework and Number of Objects . . . . . . . . . . . . ii
A.2 For an Object: Number of Received Call Messages . . . . . . . . . . . . . . vii
A.3 For an Object: Number of Received Call Messages . . . . . . . . . . . . . . x
A.4 For a Communication Diagram: Number of Senders . . . . . . . . . . . . . xii
A.5 For an Object: The Number of Sent Call Messages . . . . . . . . . . . . . xiv
A.6 For an Object: The Number of Different Sent Call Messages . . . . . . . . xvii
A.7 For a Sender Object: The Number of Different Receiver Objects . . . . . . xix

ivChapter 1
Introduction

Nowadays, in the field of software development, the measurement of quality is performed after the software has been implemented. Once its implementation is concluded,
the software is tested and defects are detected. Such defects cause, in the majority of thecases, the software to be subjected to further analysis, correction and/or extra development, incrementing time and human resources for repairing these low-quality elements ofthe software. Therefore, there is a need to detect such elements before they are implemented.
In order to detect and evaluate problems beforehand, and to predict future trends
within the elaboration process of a certain product, whether is software, the use of efficientmetrics can serve as a baseline to perform such important tasks. In the field of software
measurement, we can find several metrics for object-oriented systems. Some of the mostused are: Weighted methods per class (WMC), Response for class (RFC), Lack of cohesionin methods (LCOM), Coupling between objects (CBO), Depth of inheritance tree (DIT),
and Number of children (NOC). These metrics are known as the Chidamber and Kemerer(CK) metrics.
The CK metrics were defined to measure complexity of the design, and they havetraditionally been measured from the code and then related to managerial factors suchas productivity, re-work effort for reusing classes and design effort, and maintenanceeffort [14, 20]. In addition, they have been widely used in prediction models to determine
how faulty classes would be after coding when their testing takes place, showing to begood predictors of fault-proneness of code [5, 9, 30, 32, 40, 43].
Although the authors of the CK metrics established that they can be gathered prior
to program execution [13], they cannot be measured accurately during the design phase,unless there exists a very well detailed documentation of the design, which is rarely foundduring the development of software projects. On the other hand, one should rememberthat a metric should be as objective as possible, preventing individual bias and avoiding inconsistencies, which means that if different people perform the measurement, they
will give similar values [7]. Therefore, written documents, although specified in detail,might be subject of different interpretations when measuring the same metric. Because
UML diagrams are part of a formal modeling language of the design of object-oriented
software, they seem to be the best source of information, in the framework of elabora1

JAISTtion of the software, for measuring the CK metrics accurately, rather than simply written
documentation.
The present paper reports the results of our research on two main topics, first how
to approximate a subset of the CK metrics using UML diagrams, because early attempts
of approximation using UML class diagrams were inefficient. And second, how suchapproximations can be used to detect those classes which would result most faulty after
their implementation.
Our hypothesis is that if the code CK metrics are per se good predictors of faultproneness, and if our UML approximations are close enough to their respective code
metrics, then we can expect to predict fault-proneness of code at design time as well as if
we were using code CK metrics.
To test the ability of our UML metrics to predict faulty classes, the following steps arecarried out. First, we build three univariate logistic prediction models using the code CK
metrics of a large-size-open-source software project. Second, we test the resulting models
using the code CK metrics of three small-size projects developed by students of JAIST.
Third, the same models were tested using the UML CK metrics of the JAIST projects,
which are approximated according to our proposal. Last, using the results of the two
previous steps, we evaluate the prediction ability of our UML CK metrics in comparison
with the ability of their corresponding code metrics.
The above approach comes with two challenges. First, we need to ensure that our
prediction models work properly across different software projects. Second, because of
the difference in obtained values between UML CK metrics and code CK metrics, ourcode-based prediction models might not work with UML CK metrics. Therefore, weshould find an appropriate manner to decrease such a difference as much as possible. To
address these two challenges, we propose to perform data normalization, more details are
given in a later Chapter.
To sum up, please see Figure 1.1, because the CK metrics, when measured from thecode, have shown repetitively to be good predictors of faulty code in previous researchworks 1.1-A, and because earlier predictions are desirable, we are proposing, first, an
approach to approximate a subset of the CK metrics using UML diagrams 1.1-B; andsecond, a methodology to enable our UML approximations to predict faulty code 1.1-C.
This document is divided into the following Chapters: In Chapter 2, we present a
brief background theory on the different areas and methodologies were needed to carryout our study. In Chapter 3, we define our UML metrics, approximations of a subset ofthe code CK metrics. In Chapter 4, we present the results of our initial evaluation of our
UML metrics as approximations of their corresponding code metrics. In Chapter 5, we
present the results of the application of two data transformations for data normalization,and final evaluation of our UML metrics. In Chapter 6, we empirically validate our UML
metrics as as predictors of faulty code. In Chapter 7, we summarize some related workto our research. In Chapter 8, we discuss other applications of our UML metrics. And
finally, in Chapter 9, we draw our conclusions and describe our plans for future work.

2

JAIST

This Study

111
000
000
111
000
111
C Prediction before coding
000
111
000
111
Design

CK metrics<UML>

Prediction
Model

Most Faulty
Classes

111000
B Approximation
000
111
000
111
000
111

Prediction

Implementation

CK metrics<Code>

Model

11
00
00
11
A Existing research studies
00
11
00
11

Figure 1.1: Research Outline

3

Most Faulty
ClassesChapter 2
Background

This Chapter presents a brief background theory on the different fields and methodologies required to carry out our research study. We first introduce the concepts of software
metrics and measurement. Second, we provide the CK metrics’ definition and guidelines according to their authors. Third, we review the concepts and elements of UML
communication diagrams, because our UML metrics are derived from this kind of diagrams. Finally, we introduce some of the techniques, which are generally used to build
prediction models, so that we can empirically validate our UML metrics as predictors offault-proneness of code.

2.1

Measurement and Metrics

Within the software engineering community, there is sometimes confusion and inconsistency over the use or the terms metric and measure. In this study, we understand metricas defined in [44]. A metric is defined to be the mathematical definition, algorithm orfunction used to obtain a quantitative assessment of a product or process. The actual
numerical value produced by a metric is a measure. Thus, for example, Cyclomatic complexity is a metric, but the value of this metric is the Cyclomatic complexity measure.
Software measurement is the continuous process of defining, collecting and analyzing
data on the software development process and its products in order to understand ancontrol the process and its products, and to supply meaningful information to improvethat process and its products. Measuring will normally comprise several metrics, again
resulting in several measurements per metric.
Metrics can be classified into various categories, the most common, according to [50],are:• Product and process metrics. A product metric is a measurement of an intermediateor final product of software development, and therefore addresses the output of asoftware development activity. Examples of such metrics are a size metric for thenumber of requirements, a complexity metric for the software code, etc. Process42.1. Measurement and Metrics

JAIST

metrics measure the characteristics of the overall development process, such as thenumber of defects found throughout the process during different kinds of reviews,
etc.
• Objective and subjective metrics. Objective metrics are absolute measures, andcount attributes or characteristics in an objective way. Subjective metrics are measurements that involve human subjective judgement.
• Direct and Indirect metrics. A direct metric is a measurement that does not dependon the measurement of any other characteristic. In contrast, a indirect metric needs
the measurement of one or more other, therefore, it always contains a calculation ofat least two other metrics.
The use of efficient metrics and measurements in any process let us have baselines tocontrol, to predict future trends and to take decisions during the process. Metrics aremeasured to learn something. Thus, it is important not just to measure but also to havethe proper guidelines of the metric used. In other words, we must be able to interpret themeaning of the metric correctly [47].
Ideally a good metric should be simple, objective, easily collected, robust and valid[7, 20]. A metric is:• simple, if its definition and usage are simple.
• objective, when different people performs the same measurement and all of themobtain similar values.
• easily collected, when the cost and effort to obtain the measure is reasonable.
• robust, when is insensitive to irrelevant changes, allowing useful comparisons. A
metric is robust, if it can cope with abnormal inputs.
• valid, if the metric is measuring what is supposed to measure, so that we can trust
its value.
Software measurement data provide visibility of the current development process andcharacteristics. Such visibility is required to reduce complexity and increase understanding of the process and products. Understanding means determining the different variables
that exist during execution of a process. Once basic understanding has been established(the variables are known), the collected and analyzed data can be used to control the process and the products, by defining corrective and preventive actions. Furthermore, basedon analysis, the collected data measurements can be used to assess the process, and therefore act as an indicator of development process problem areas, from which improvement
actions can be identified.

5

2.2. The Chidamber and Kemerer Metrics

2.2

JAIST

The Chidamber and Kemerer Metrics

The CK metrics were defined to measure unique aspects of the object-oriented approachand complexity of the design. They have been traditionally measured from the implementation of the classes. In the following paragraphs, for the sake of completeness, were-state the definition of the CK metrics as given by their authors in [13]:• Depth of inheritance tree (DIT). It is the maximum number of steps from the classnode to the root of the inheritance tree and is measured by the number of ancestor
classes.
Deeper inheritance trees constitute greater design complexity, but also promote there-usability of inherited methods. Therefore, DIT values are a trade-off between
re-usability and complexity.
• Number of children (NOC). It is the number of immediate subclasses subordinate
to a class in the hierarchy.
The greater the number of children, the greater the reuse of the class might be, but
the more complex testing and maintainability become. Therefore, NOC values arealso a trade-off between re-usability and complexity.
• Weighted methods per class (WMC). It is a count of the methods implemented ina class, or the sum of the complexities of every method of the class.
A class with high-weight methods per class is likely to be more specific, limiting its
re-usability in other applications. It also complicates its maintenance. Therefore,
low values of WMC are desirable.
• Coupling between objects (CBO). It is the number of other classes to which a classis coupled. If a method within a class uses a method or instance of a variable of adifferent class, it is said that this pair of classes is coupled.
The higher the number of couples a class has, the more complex it is to changeor to correct. Low coupling between objects improves modularity and promotes
encapsulation. Therefore, low values of CBO are desirable.
• Response for class (RFC). It is a set of all methods that can be invoked in response toa message to an object of the class. From the code, RFC is measured as the numberof methods of a given class (M), plus the number of methods of other classes directly
called by any of the methods of the given class (R). A method is counted only oncein R even, if it is executed by several methods M.
The higher RFC in a class, the more complex the class is. If a large number ofmethods can be invoked in response to a message, the testing and debugging becomes
more difficult and complex. Therefore, low values of RFC are desirable.
• Lack of cohesion of methods (LCOM). It measures the dissimilarity of methods ina class. High cohesion indicates a good subdivision class and thus promotes reusability. A method is more similar to another method if they instantiate the same
variables.
62.3. Communication Diagrams UML 2.0

JAIST

The higher LCOM in a class, the higher the dissimilarity of its methods is. Therefore, low values of LCOM are desirable.

Table 2.1: Guidelines for the CK metrics
Metrics
Weighted methods per class
Response for class
Lack of cohesion of methods
Coupling between objects
Depth of Inheritance
Number of Children

Objective
⇓
⇓
⇓
⇓
3
3

⇓: Low Values 3: Trade-off (between complexity and re-usability)

The final purpose of the definition of the CK metrics was, according to their authors, toimprove the development of the software. This can be achieved through the identificationof outliers (extreme value deviations), which might be a sign of high complexity and/or
possible design violations [13]. But how can we properly identify these outliers?
In 1994, Chidamber and Kemerer provided the definition of their metrics along with
their guidelines, which can be summarized in Table 2.1 [13]. Yet, it was not clear how
much is high or low. In other words, which the thresholds of these metrics were. In 1998,
the same authors described in [14] the managerial use of these metrics, then, it was said
that thresholds of these metrics can not be determined before their use, and they should
be derived and used locally for each dataset (each project). Finally, it was recommended
to use 80th and 20th percentiles of the distributions to determine high and low values ofthe metrics [14, 31, 46].
Furthermore, when measured from the code, the CK metrics have been related to
managerial factors such as productivity, re-work effort for reusing classes and design effort
and maintenance effort [14, 20]. In addition, the CK metrics have shown their ability to
predict fault-prone code in several empirical studies [5,9,30,32,40,43]. Further details are
given in the Chapter of Related Work.

2.3

Communication Diagrams UML 2.0

A communication diagram shows interactions organized around the parts of a composite
structure or the ’roles’ of a ’collaboration’. It models the ’objects’ and ’links’ involved in
the implementation of an ’interaction’ and ignores the others.
’Roles’ and ’connectors’ describe the configuration of ’object’s and ’links’ that may
occur when an instance of a context is executed. When the context is instantiated,’objects’ are bound to the ’roles’, and ’links’ are bound to the ’connectors’. Connectors,
in their turn, may also be bound to various kinds of temporary links, such as procedurearguments or local procedure variables.
72.3. Communication Diagrams UML 2.0

JAIST

In a communication diagram a series of ’messages’ among ’roles’ are shown which
are ’labeled arrows’ attached to ’connectors’. Each message has a ’sequence number’, an
optional ’guard condition’, a ’name’ and ’argument list’, and an optional ’return value’.
The ’sequence number’ includes the optional ’name of a thread’. All messages in the same
thread are sequentially ordered. Messages in different threads are concurrent unless there
is an explicit sequencing dependency. Various implementation details may be added, suchas a distinction between ’asynchronous’ and ’synchronous’ messages.
Only objects that are involved in the context are modeled, although there may beothers in the entire system.
UML communication diagrams are similar to sequence diagrams, both show interactions. Sequence diagrams show time sequences clearly, but do not show relationships
explicitly. Communication diagrams show object relationships clearly, but time sequencesmust be obtain from sequence numbers.
Key Elements of a Communication Diagram
In the previous Section, we have introduced communication diagrams, and we have identified many important elements to build a communication digram. In the following paragraphs, for the sake of completeness, we will re-state the definition of such elements asfound in [26].
1. Collaboration. It is a specification of a contextual relationship among objects thatinteract within a context to implement a desired functionality. The behavior of acollaboration may be specified by ’interactions’ that show ’messages’ flow in thecollaboration over time.
2. Behavior. It is a generic term for the specification of how the state of an instance of aclassier changes over time in response to external events and internal computations.
3. Object. An instance of a ’class’
4. Link. It is an individual connection among two or more objects.
5. Role. It is a description of an object in an interaction. It is possible for an object
to play multiple roles in one or many collaboration instances.
6. Message. The conveyance of information from one role to another as part of aninteraction within a context; at instance level, a communication from one object toanother.
A message may be a ’signal’ or a ’call’ of an operation. A signal is a message
transited from one object (the sender) to one or more other objects (the receivers).
The receipt of a signal may trigger a state machine transition. The sending of asignal is ’asynchronous’; the sender continues execution after sending it. A call, on
the other hand, is a call of an operation of one ’object’ (the ’receiver’) by another’object’ (the ’sender’ or caller). It can be modeled as TWO messages, a ’call message’and a later ’return message’. The receipt of an operation call may invoke a procedure82.3. Communication Diagrams UML 2.0

JAIST

or may trigger a state machine transition or both. A call may be ’asynchronous’ or’synchronous’. If it is ’synchronous’; the sender continues execution after sendingit.
(a) Structure• interaction. The interaction that owns the message• connector. The connector over which the message is transmitted• send event. The occurrence specification of sending the message. The
lifetime owing this occurrence specification is the ’sender’ of the message• receive event. The occurrence specification of receiving the message. The
lifetime owning the occurrence specification is the ’receiver’ message• signature. The signal or operation the type of message• arguments. A list of argument values compatible with the signature. A
message has arguments, which must agree in numbers and types with theparameters of the specified operation or the attributes of the specified
signal.
• return values. If any, must match the return types of the specified signal.
Any return values for an asynchronously called operation are ignored anddo not generate a reply message.
• synchronicity. Either asynchronous (signal or call) and synchronous (onlycalls).
• kind. Whether the message is complete (default, includes sender and receiver), lost (no receiver), or found (no sender). Lost and found messages
are used in advanced modeling situations for noisy systems or system withincomplete knowledge. They are unnecessary in most models.
(b) Message Format• A message is shown as a small ’labeled arrow’; attached to a ’path’ between
the ’sender’ and the ’receiver’ objects. The ’path’ is the ’connector’ used toaccess the ’target object’. The ’arrows’ point in the direction of the ’target
object’. In the case of a message from an object to itself, the message
appears on a path looping back to the same object and the keyword <<self >> appears on the target end.
• More than one message may be attached to one ’link’.
• The relative order message is shown by the ’sequence number’ portion of
the ’message label’.
• A message may be label with a ’guard condition’.
• Synchronicity. According to [26], the same ’arrow types’, used in sequence
diagrams, may be used to indicate the type of synchronicity of the message.
Figure 2.1 shows an example of synchronous and asynchronous messages
in a communication diagram, which shows the interactions that take placewhen an object sensor detects something. These interactions are:9

2.3. Communication Diagrams UML 2.0

JAIST:System:Phone Alarm:Sound Alarm

Handler:SoundbAlarm
2b. Trigger
2a. Trigger
2c.2: Trigger:Cell Handler:Supervisor
2c. Alarm

2c.1: Store(Date,Time,Cell,Sensor)

1. Alarm:Photo Cell
Sensor:Log

Asynchronous
Message

Synchronous
Message

Figure 2.1: Example: Synchronicity in Communication Diagrams– First, the sensor sends an asynchronous alarm signal to the Cell Handler. Such a signal must be asynchronous given the requirements of asensor, which does not need to wait for a return signal and is used toalert the system to an external event.
– Next the Cell Handler sends in parallel trigger signals to all alarms(in this case, a phone and a sound alarm) and an asynchronous alarm
signal to the System Handler.
– Inside the System Handler, the alarm signal is handled synchronously,
the supervisor thread first calls the internal sound alarm and thenwrites the event to the log.
This example was taken from [16].
• Message label. The general syntax for a message label is expressed in
Equation 2.1, further details are given in the next Section. Some examplesare:- A simple message: 2 : display(x, y)- A nested call with return value: 1.3.1 : p = f ind(spects) : status- Conditional within second concurrent thread 1b.4[x < 0] : invert(x, colo)- An iteration: 3.1 ∗ [i := 1..n] : update()
Message LabelThe general syntax of a message label is the following:
sequence-expression opt message
where,
10(2.1)

2.3. Communication Diagrams UML 2.0

JAIST• sequence-expression is a dot-separated list of sequence-terms followed by a colon (’:’).
Each term represents a level of procedural nesting within the overall interaction. If
all the control is asynchronous, then nesting does not occur and a simple one-level
numbering system is used. Each sequence-term has the following syntax:
label

iteration-expression opt(2.2)

where,- label is an integer and/or name. An integer serves to represent the sequential
order of the message. An example is: message 3.14 follows message 3.13 within
the activation of message 3.1. A name is used to represent a concurrent threadof control. Messages that differ in the final name are concurrent at that level ofnesting. An example is: Message 3.1a and message 3.1b are concurrent within theactivation of message 3.1. All threads of control are equal within the nesting depth.
-iteration-expression represents conditional or iterative execution. This represents
zero or more messages that are executed, depending on the conditions. The choicesare:– An iteration: ∗[iteration-clause], where iteration-clause is meant to be expressed in pseudo-code or an actual programming language; UML does notprescribe its format. an example would be : ∗[i := 1..n]– A branch: [condition-clause], where condition-clause represents a message
whose execution is contingent on the truth of the condition clause. The
condition-clause is meant to be expressed in pseudo-code or an actual programming language. An example would be: [X > y]• and message, in its turn, has the following syntax:⌊atribute =⌋opt name⌊(argumentlist )⌋opt ⌊: return-value⌋opt(2.3)

where,- name, is the name of the signal or operation. The entire name string can be
replaced by an asterisk, to indicate that any message in compatible with the model.
- attribute, is the optional name of an attribute to store the return value- argument values can be replaced by a dash, to indicate that any value is compatible
with the model. A parameter name may be include with an argument value, so anargument may have one of the following syntaxes.
argument-value(2.4)

parameter-name = argument-value

(2.5)

11

2.4. Prediction Techniques

JAIST

In order to ease the understanding of how all these elements interact with each other,
we have selected the elements we think are the most important, and we have elaborated
a meta-model of a communication diagram, such model is shown in Figure 2.2.

2.4

Prediction Techniques

As mentioned earlier, after defining our UML metrics, we evaluate them not only as
approximations of the CK metrics, but also as predictors of fault-proneness of code.
Therefore, this Section is dedicated to the study of those techniques that can help us tomodel our data and to build our fault-prediction models.
In order to build any sort of prediction model, first, the data to be modeled is collected.
Then, using a statistical technique, a prediction model can be derived. In order to selectwhich is the most suitable statistical technique to be applied, the types of measurement
scales of the variables used must be analyzed. Therefore, in the following paragraphs,
we first introduce some background theory on measurement scales, and later, we give thedetails of the candidate statistical techniques for our study. For further study on this
topic, the reader can refer to [49].
On the other hand, apart from the statistical techniques, in the field of fault-pronenessprediction, some researchers have used Artificial Neural Networks (ANN), an example is
the research work reported in [32]. ANN is a computational model used for predicting
one or more dependent variables using various independent variables. Further details,
concerning the application of this technique, are provided in a later Section.

2.4.1

Types of Measurement Scales

Measurement is a process by which numbers or symbols are attached to given characteristics or properties according to predetermining rules. It is said that measurements can beclassified into the following types: nominal, ordinal, interval and ratio. In the following
lines we described briefly these four types of scales.
• Nominal Scale
Consider we have as a variable the gender of a subject. We could use numbers to
represent subject’s genders. For example, we can arbitrarily assign number 1 forfemales and number 2 for males. The assigned numbers themselves do not haveany meaning, and therefore it would be inappropriate to compute such statistics asmean and standard deviation of the gender variable. The numbers are simply use
to categorize subjects into different groups or for counting how many are in eachcategory.
The described measurements scales are called nominal scales and the resulting data
are called nominal data. The statistics that are appropriate for nominal scales arethose which are based on counts such as mode and frequency distribution.
• Ordinal Scale

12

13

Figure 2.2: Meta-model of a UML Communication Diagram

2.4. Prediction Techniques

1

UML Desing
1

*

*

Class

Communication Diagram

1

1

1

*

1
0
0
1

Interaction
*

has−a

can−bebounds−to 1 Object

asynchronous2
1

1

Sendercan−beRoleconnects
Link

Receiver*Signal1connects
1
1
Connector*

Messageasynchronoushas−a

1 bounds−to 1Callsynchronous

1
has−a

Label Arrow

0..1
Name of a Thread

1
Sequence Number

Message Label

1

0..1
Guard Condition

Name

0..1
Argument List

0..1
Return Value

JAIST2.4. Prediction Techniques

JAIST

Suppose we want to measure person’s preferences for four brands of soda, Brands
A, B, C and D. We would ask each subject to rank the four brands from 1 to 4, byassigning a 1 to the most preferred brand and 4 to the least preferred one. Consider
a particular subject ranks the four brands as follows: Brand B - 1, Brand D - 2,
Brand A-3, Brand C - 4. Then, we could conclude that this subject prefers Brand B
to Brand D, Brand D to Brand A and Brand A to Brand C. However, even though
the differences in the successive numerical values of the ranks are equal, we cannot
state by how much the subject prefers one brand to another one. In other words,
successive categories do not represent equal differences of the measured attribute.
The described measurement scales are referred as ordinal scales and the resulting
data are called ordinal data. Valid statistics that are computed for ordinal-scaled
data are mode, media, frequency distributions and non-parametric statistics suchas rank order correlation.
• Interval Scale
Quantitative attributes are all measurable on interval scales, as any difference between the levels of an attribute can be multiplied by any real number to exceed orequal another difference. A highly familiar example of interval scale measurement istemperature with the Celsius scale. In this particular scale, the unit of measurementis 1/100 of the difference between the melting temperature and the boiling temperature of water at atmospheric pressure. The ”zero point” on an interval scale isarbitrary; and negative values can be used.
Continuing with our previous example on brand preferences, suppose that instead ofasking people to rank order the brands, we ask them to rate their brand preference
according to the following five-point scale: 1 Very high preference, 2 High preference,
3 Moderate preference, 4 Low preference and 5 Very low preference. If we assume
that successive categories represent equal degrees of preferences, then we could saythat the difference in a subject’s preference from the two brands that received ratings
1 and 2 , it is the same as the difference in the subject’s preference for two otherbrands that receive rating of 4 and 5. However, we still can not say that the subject’s
preference for a brand that received a rating of 5 is five times the preference for abrand that received a rating of 1. Moreover, suppose we multiply each rating pointby 2 and then add 10. This would result in the following scales: 12 Very highpreference, 14 High preference, 16 Moderate preference, 18 Low preference and 20
Very low preference. The differences between the new categories are equal to theones in the original categories; however, the ratio of the last to first category is not
the same for the original scale. The ratio for the original scale is 5 and 1.67 for thetransformed scale.
Measurement scales, whose successive scales categories represent equal level of thecharacteristics that is being measured, and whose base values are arbitrary, arecalled interval scales. And the resulting data are called interval data. Properties of
the interval scale are preserved under the following transformation: Yt = a + bYo ,where Yo and Yt , respectively, are the original and transformed scale, and a and b

14

2.4. Prediction Techniques

JAIST

are constants. All statistics, except those ones based on ratios such as the coefficientof variation, can be computed for interval-scaled data.
• Ratio Scale
Most measurement in the physical sciences and engineering is done on ratio scales.
Mass, length, time, plane angle, energy and electric charge are examples of physicalmeasures that are ratio scales.
Ratio scales, in addition to having all the properties of the interval scale, have anatural base value that cannot be changed. For example a subject’s age that anatural based value, which is zero. Ratio scales can be transformed by multiplying
by a constant; however they cannot be transformed by adding a constant, as thiswill change the base value. That is, only the following transformation is valid forthe ratio scale: Yt = bo .
Since the ratio scale has a natural base value, statements such as ”Subject A’s age
is twice Subject B’s age” are valid. Data resulting from ratio scales are referred asratio data. There is no restriction on the kind of statistics that can be computed
for ratio-scaled data. Variables measured using interval and ratio scales are called
metric variables.
To conclude, let us analyze which are the measurement scale of the variables used inour study. We have basically two groups of variables: the CK metrics and fault-pronenessof classes.
The CK metrics’ scale can be classified as ratio. It is no nominal because their values
are not arbitrary and do have some meaning. It is not ordinal because successive values ofthese metrics have equal differences. It is not interval because although the CK measures
are successive values representing equal level of the characteristic they are measuring, theirbase values are not arbitrary. Therefore, the scale of the CK metrics can best classified
as a ratio scale, because they have a natural base value that cannot be changed, 0.
On the other hand, to measure fault-proneness of a class, it can be done in different
ways, the two most common used by fault-prediction practitioners are using number offaults per class, or simply a categorization such as Most Faulty and Least Faulty classes.
The first case can be classified as a ratio scale, and the second case as a nominal scale.

2.4.2

Statistical Prediction Techniques

There are different statistical techniques that can be used to analyzed data. The objectiveof analyzing data is to extract the relevant information contained in the data, which canbe used to solve a given problem. This given problem, normally is formulated in one
or more null hypothesis. And the statistical techniques help us to extract that relevant
information in the data and test our null hypothesis. In the following paragraphs we will
give a brief description of these statistical methods, according to [49].
Statistical methods can be divided basically into: dependence methods and interdependence methods. Consider a data set, which consists of n observations on p variables.
The dependence methods test for the presence or absence of relationship between two
15

2.4. Prediction Techniques

JAIST

sets of variables of the data set. However, if one of these data sets is designated withdependent variables and the second one with independent variables, then the objectiveof the dependence methods is to determine if the independent variable affects the set of
dependent variables. In other words, statistical techniques only test for the presence orabsence of relationship between two sets of variables. On the other hand, interdependence
methods are used to identify how and why the variables are related among themselves.
Since we are looking for a technique that help us to find the relationship between CK
metrics acting as independent variables and fault-proneness of classes as the dependent
variable, we focus on the study of the most used dependence statistical methods.
The dependence methods for studies which consider one dependent variable and morethan one independent variables are:• Linear analysis is a technique that is used when the dependent variable and the
independent variables are measured using a metric scale, resulting in a metric datasuch as income measured in dollars. Linear analysis considers a model where thedependent variable is a linear combination of the independent variables.
• Analysis of variance (ANOVA) is an appropriated technique to estimate linear models when the independent variables are nominal or categorical.
• Discriminant analysis is appropriated when we have the dependent variable measured using a nominal scale and the independent variables are measured using aninterval or ratio scale.
• Logistic regression is an alternative procedure to Discriminant Analysis. Logistic
regression which does not make any distributional assumption for the independent
variables, as it does Discriminant Analysis, is more robust to the violation of themultivariate normality assumption (which will not hold when the independent variables are combinations of metric and nominal variables).
• Discrete discriminant analysis is appropriated when both sets of variables, the dependent variable and independent variables, are measured using categorical or nominal scale.
• Conjoint analysis is appropriated when the dependent variable is ordinal and the
independent variables are measured using a nominal scale.
Considering the subject matter of our research, in which the dependent variable isthe fault-proneness of code, which can be measured using a ratio scale, metric scale ornominal scale; and the independent variables, the CK metrics, which can be measured
using a ratio or metric scale; linear analysis, discriminant analysis or logistic regression
techniques are the techniques that can help us to build our prediction model.
Linear Regression
It is used when we have one or more independent variables (x) and one dependent variables(y). The dependent variable is a continuous metric. And the independent variables can
not be linear combination of each others.
162.4. Prediction Techniques

JAIST

The goal of this technique is to find an equation that best predict the value of thedependent variable (y), as a linear function of the independent variables (xs ). Look at
Equation 2.6, y represents the expected value of the dependent variable for a given set ofxs values, xs represents the independent variables, bi is the estimated slope of a regressionof y on xi and a is the intercept.
yexp = a + b1 x1 + b2 x2 + ...bn xn(2.6)

Two-group Discriminant Analysis
It is used when we have that our dependent variable is measured using the nominal scale,and the independent variables, on the other hand, are measured using an interval or aratio scale. Thus, we have a data set in which the dependent variable is categorical or
nominal and the independent variables are metric or continuous. Two-group discriminant
analysis is a special case of multiple linear regression.
Consider the following examples:• A medical researcher is interested in determining factors that significantly differentiate between patients who have had a heart attack and those who have no yet hada heart attack. The medical researcher then wants to use the identifies factors to
predict whether a patient is likely to have a heart attack in the future.
• The marketing manager of a customer packaged goods firm is interested in identifying salient attributes that successfully differentiate between purchasers and nonpurchasers of brands, an employing this information to predict purchase intentionsof potential customers.
Each of the above examples attempts to meet the following three objectives:
1. Identify the variables that discriminate ”best” between the two groups.
2. Used the identify variables or factors to develop an equation or function for computing a new variable or index that will represent the differences between the twogroups.
3. Used the identify variables or the computed index index to develop a rule to classify
future observations into one of the groups.
The two-group case discriminant analysis can also be thought as multiple regression.
In general, in the two-group discriminant analysis, we fit a linear equation of the type,2.7:
Group = a + b1 ∗ x1 + b2 ∗ x2 + ... + bm ∗ xm

(2.7)

Where a is a constant and b1 through bm are regression coefficients. The interpretationof the results of a two-group problem is straightforward and closely follows the logic ofmultiple regression: Those variables with the largest (standardized) regression coefficientsare the ones that contribute most to the prediction of group membership.
17

2.4. Prediction Techniques

JAIST

Logistic Regression
It is appropriated when we have one or more independent variables and our dependent
variable has just two values, in other words, when the dependent variable is dichtonomous(faulty/no faulty, male/female, healthy/unhealthy, 1/0).
Consider the following scenarios:• The marketing manager of a cable company is interested in determining the probability that a household would subscribe to a package of previous channel given
the occupant;s income, education, occupation, age, marital status, and number ofchildren.
• An auditor is interested in determining the probability that a firm will fail given a
number of financial ratios and the size of the firm (i.e., large or small).
Discriminant analysis could be used for addressing each of the above problems, however, because the independent variables are a mixture of categorical and continuous variables, the multivariate normally assumption will not hold. In these cases one could usedlogistic regression, as it does not make any assumption about the distribution of the independent variables. Logistic regression is recommended when the independent variables do
not satisfy the multivariate normality assumption.
The goal of logistic regression is to predict the probability of getting one of the twovalues of the dependent variable, given the independent variables. It uses the logit function
to link the values of the independent variables to the probability of occurrence of one ofthe values of the dependent variable.
Looking at Equation 2.8, P is the probability of occurrence of one of the values ofy, for example the probability of having a faulty class, xs variables are the independent
variables, bi variable is the estimated slope of regression of P on xi and a is the intercept.
P =

2.4.3

1
1 + e−(a+b1 x1 +b2 x2 +...bn xn )(2.8)

Artificial Neural Networks

In the field of exploratory multivariate data modeling, Artificial Neural Networks (ANN)
offer many advantages over conventional multiple linear regression methods, but also theuse of ANN have some disadvantages, for our purposes. In the following paragraphs wewill summarize some of its advantages and disadvantages cited by Kristen in [33].
ANN can identify patterns between the dependent and independent variables in data
sets. Furthermore, they can create specialized regression models or model adjustments
for all patterns discovered during certain analysis. Neural networks deal very effectively
with data discontinuities and even nonlinear transformations. Additionally, it is said thatneural networks adapt and learn in such way that they can choose models correctly forrecurring patterns.
Many researchers have used neural networks in conjunction with regression analysis.
Several researchers have also compared ANN regression to multiple linear regression. In
18

2.4. Prediction Techniques

JAIST

the majority of cases, the ANN analysis perform better than statistical regression methods.
Some advantages of using ANN are:
• ANN are nonlinear and versatile, thus they can deal very effectively with data sets
demonstrating not easily recognized or unknown nonlinearity, as well as with linear
relationships among the data.
• Prior model specification is not required. When performing ANN regression analysis,
the model under examination does not need to be specified in advance. Thus, ANN
do not require explicit relationships between inputs and outputs to be defined in
the data set.
• ANN do not make any assumptions about the population distributions of the variables used. Neither on multicollinearity of the independent variables.
• ANN can discover stable patterns in fuzzy data sets, where irrelevant data hide or
disguise the model.
On the other hand, some strong disadvantages of this technique are:
• Model parameters are unidentifiable. ANN are often referred to as black boxes,
because the parameters and coefficients they derive to approximate patterns cannot
be easily analyzed or interpreted. Unlike conventional regression techniques, ANNs
lack marginal evaluators, which obscures the network from revealing the functional
relationships among the variables.
• Although ANN often provide better results than multiple linear regression, themethod is difficult to interpret.
• Data sets are usually larger than those used in statistical regression techniques.
To summarize ANN strength is, its ability to model no-linear relationships or to finddifficult patterns. On the other hand, ANN do not make use of a function as it does
linear regression and logistic regression. Therefore, they are often described as a blackbox, because the parameters and coefficients derived are not easy to interpret and toanalyze.

2.4.4

Summary and Conclusions

In the previous sections, we have analyzed the candidate techniques to predict faultproneness of code.
We learned that in order to select the most suitable statistical technique to model ourdata, the measurement scales of the used variables must be first analyzed. Therefore,
since our CK metrics (independent variables) have a ratio scale and our fault-pronenessvariable can be expressed using either a ratio or nominal scale, the candidate statisticaltechniques to be applied in our study are: Linear Analysis, Discriminant Analysis orlogistic regression.
19

2.4. Prediction Techniques

JAIST

On the other hand, we also learned that ANN, which has been used in previous
studies of fault-proneness prediction, have the ability to model no-linear relationships or
to find difficult patterns among data, however the resulting parameters are difficulty tointerpret and analyze. Therefore, the dependent statistical techniques seemed to be more
appropriated for our purposes.
From our three candidate statistical techniques, logistic regression is the techniquethat seems to be the most suitable for our study. This mainly because the two othertechniques, linear regression and two-group discriminant analysis, make assumptions on
the normality of the distribution of the independent variables. Moreover, if we use linear
regression, we were assuming that a linear relationship exists between our independent
metrics, CK metrics, and our dependent variable, fault-proneness of code.

20Chapter 3
Definition of Goals and our UML
CBO, RFC and WMC Metrics

Metrics serve as baseline to monitor the elaboration of a product. They can also be
used to predict future trends and to detect possible flaws within the elaboration process
before the actual release of the product. A good practical example is the construction ofa house, which is based on a design detailed in some blueprints and a set of metrics cantell us how many walls the house will have, and then we could approximate the cost ofthese, among other things. However, on the field of object-oriented software, although
UML diagrams have served as the blueprints of a software solution, UML metrics have
not yet fully defined, neither explore a possible existent relationship among them, nor to
the final software product.
Different from the construction of a house, design metrics for object-oriented software,such as design complexity metrics, have been traditionally measured from the code, andnot from the design. Some of the most well known metrics are the Chidamber and Kemerer
metrics.
The few works, that have tried to approximate the CK metrics out of UML diagrams,
focus mainly on UML class diagrams. Their observations are in general that the calculation of three of the six metrics from UML class diagrams is not straightforward. These
three metrics are the CBO, RFC and LCOM. Moreover, their methods either lack accuracy (in terms of proximity to the measures obtained directly from the source code)or need very well detailed design information [4, 15, 37, 38, 53]. More details concerning
this topic can be found in the Chapter of Related Work. Because previous research work
has indicated that CBO and RFC metrics are difficult to measure using only UML class
diagrams, we present an easy approach to measure these metrics using UML communication diagrams. Moreover, we also provide a metric that approximates the number ofmethods per class, or WMC considering the method’s static complexities to be unity,
using communication diagrams.
In this Chapter, we define a set of metrics for objects measured from communication
diagrams. Then, using such metrics, we propose a UML approximation of the CBO,
21

3.1. Goal Question Metric Approach

JAIST

RFC and WMC CK metrics for classes. In order to define these metrics, we followed thebasic steps of the Goal/Question/Metric approach (GQM), which is introduced briefly inthe following section. Then, we describe our plan; after that, we define our goals andquestions; and finally, we provide the definition of our UML metrics. We used the study
cases applying the GQM approach provided in [50] as guidance.

3.1

Goal Question Metric Approach

In this Section, we briefly introduce the Goal/Question/Metric approach, according to[50].
During the development of software projects, we often find that developers work in
an unstructured and stressful way, resulting in a poor or unknown level of the software
quality. Such problems are known as the ’software crisis’. Organizations try to address
this crisis by developing software process improvement (SPI) approaches.
SPI approaches consider software process to be the main area for quality improvement,
because they contain the activities during which the product is actually created. The
earlier quality problems are detected, the easier and cheaper their solution is.
In order to improve the software development process, a definition of clear improvements goals are needed, otherwise the improvement activities will turn out to be as chaotic
as the development process itself. These improvements goals should support business objectives in the best possible way.
An organization can define improvement goals by several different ways. For example,
they could first prescribe quality objectives, from which improvement goals can be derived.
They could also conduct software process assessments in order to identify main areas
suitable for improvement, and based on the identification of such areas, improvement goals
can again be defined. Measurements also provide the means to identify improvements
goals. By applying measurement to a specific part of the process of the project, problems
within the process can be defined with respect to particular needs.
SPI practitioners usually focus in four main areas: the decrease of project cost, thedecrease of project risk, the shortening of project cycle time and the increase of quality.
Several improvement models, methods and techniques are available for such purposes.
These can be divided into two major streams: Top-down and Bottom-up approaches.
Top-down approaches are mainly based on assessments and benchmarking, examplesof these approaches are the Capability Maturity Models (CMM), the Software Process
Improvement and Capability determination model (SPICE) and BOOTSTRAP. Bottomup approaches mainly measures software development to increase understanding within a
specific context. Goal Question Metric (GQM) and the Quality Improvement Paradigm(QIP) are the typical examples of these approaches.
GQM represents a systematic approach for tailoring and integrating goals to modelsof the software process, products and quality perspectives of interest, based upon thespecific needs of the project and the organization.
The principle behind the GQM method is that measurement should be goal-oriented.
Therefore, in order to improve process, organizations have to define their measurement
goals based upon corporate goals and transform these goals into activities that can be22

3.1. Goal Question Metric Approach

JAIST

measured during the execution of the project.
GQM defines a certain goal, which then is refined into questions. Then, metrics aredefined to provide the answer to these questions. Thus, GQM defines metrics from a
top-down perspective and analyzes and interprets the measurement data bottom-up.
The GQM method different phases are:
1. The Planning phase. In this phase a project for measurement application is selected,
defined, characterized, and planned, resulting in a project plan.
2. The Definition phase. In this phase the measurement program is defined (goal,
questions, metrics, and hypothesis) and documented.
3. The Data Collection phase. During this phase the actual data collection takes place.
The Definition phase is of our main interest, since it is where the metrics and their
guidelines definition take place. To carry on with such definitions, the following steps
need to take place:• Step 1. Goal Definition
Measurement goals should be defined in an comprehensible way and should beclearly structured. For this purpose, the following template is available:
Analyze
For the purpose ofWith respect to
From the point of viewofIn the context ofthe object under measurement
understanding, controlling, or improvement theobjectthe quality focus of the object that the measurement focuses on
the people that measures the object
the environment in which measurement takes place

We should keep in mind that metrics are meant to answer the questions which,altogether, provide the information to achieve certain goal.
• Step 2. List of measurement questions and hypotheses, defined with respect to themeasurement goals.
Questions should be defined to support data interpretation towards a measurement
goal. As goals are defined in an abstract level, questions are refinements of goals toa more operational level, which is more suitable for interpretation. By answering
the question definitions, one should be able to conclude whether the goal is reached.
Therefore, during the question definition, various checks are recommended to verify
that the defined questions have the ability to support the conclusion of the goal ina satisfactory way.

23

3.1. Goal Question Metric Approach

JAIST

Subsequently, for each question, expected answers are formulated as hypotheses.
Formulating hypotheses triggers the project team to think about the current situation and therefore stimulates a better understanding of the process and/or product.
Furthermore, after measurement during data interpretation, these hypotheses ofmeasurement results will be compared with the actual measurement results. The
purpose of this comparison should not be to evaluate a possible correctness of the
hypotheses, but to encourage the project team to identify and analyze the underlying reasons that caused the actual result to deviate, or conform, from their
expectations. In other words, hypotheses are formulated to increase the learning
effect from measurement.
• Step 3. Metrics Definition.
Once goals are defined into a list of questions, metrics should be defined to provide all the quantitative information to answer the questions in a satisfactory way.
Therefore, metrics are a refinement of questions into quantitative processes and/or
product measurements.
Furthermore, factors that could possibly influence the outcome of the metrics should
also be identified. After all, factors that directly influence metrics, also influence
the answer to the questions that metrics are related to.
• Step 4. GQM planA GQM plan is a document that contains the goals, questions, metrics and hypotheses for a measurement program as defined in previous steps. The GQM plan
describes the refinement from measurement goals into questions and subsequently
from questions into metrics. As some of theses metrics may be indirect metrics,
it also describes all direct measurements that should be collected for each indirect
metric.
• Step 5. Measurement Plan
A measurement plan describes the following aspects of each direct measurementthat was identified in a GQM plan.
– It provides formal definitions of direct measurements.
– It provides textual descriptions of direct measurements.
– It defines all possible outcomes (values) of the direct measurements.
– It identifies a person that collects a particular direct measurement.
– It identifies the particular moment in time when the person should collect thedirect measurement.
– It defines by which means the direct measurement should be collected.
Furthermore, a measurement plan defines and describes both manual data collection
forms and automated data collection tools.

243.2. Planning

JAIST• Step 6. Produce Analysis Plan
An analysis plan is a document that simulates data interpretation outcomes of themetrics, graphs and tables are presented in this document. such documentation
should be related to the questions and goals as defined in the GQM plan.

3.2

Planning

The studies, that previously have provided a formal definition of the CBO and RFC
metrics using UML class diagrams, have concluded that UML class diagrams’ information
is not sufficient to capture all the information that the code normally provides. Therefore,
this study focuses on the approximation of these metrics using communication diagrams.
The main reason for which we decided to use communication diagrams can be explainedin the following paragraphs.
Recalling the definition of the CBO and RFC metrics, we have that:• CBO is the number of other ’classes’ to which a class is coupled. If a method withina class uses a method or instance of a variable of a different class, it is said that thispair of classes is coupled.
• RFC is a set of all ’methods’ that can be invoked in response to a message to an
object of the class. From the code, RFC is measured as the number of methods of agiven class (M), plus the number of methods of other classes directly called by anyof the methods of the given class (R). A method is counted only once in R even, if
it is executed by several methods M.
Both metrics require information on the number of classes or methods which are
been instantiated within a given class’s body. Class diagrams provide this information
to a limited extent. It can tells us, for example, class A uses class B, but cannot tell
us precisely that method ma of class A uses method mb of class B. However, because
communication diagrams can tell us more explicitly how the interactions between objectsof the different classes of the system take place, we can know which objects call which
methods of other objects. Therefore, communication diagrams provide a much more finegrained information in this sense. For example, suppose that we have two objects a1
and b1, a1 is an object of class A, and b1 is an object of class B. When a1 sends a callmessage mb to b1, mb can be translated into a method of class B. So that, by analyzing
all communication diagrams of a system design, we can give an approximation of thenumber of methods of class B. Moreover, not only the number of the different classes
instantiated by class A (and B ) can be known, but also the number of methods of otherclasses instantiated by class A (and B ).
In order to approximate the code CBO, RFC and WMC metrics using communication diagrams, we first provide a general set of metrics measured from communication
diagrams, then we derived from them our metrics.

25

3.3. Definition: Goal

3.3

JAIST

Definition: Goal

Based on our previous description, we define our goal in Table 3.1. For this purpose, weused the goal template definition provided in [50].
Table 3.1: Our Goal
Analyze
For the purpose ofWith respect to
From the point of view ofIn the context of

3.4

UML Communication Diagrams
obtaining an approximation of the CK RFC, CBO and
WMC metrics
their objects interactions
the researcher
software quality studies

Definition: Questions

The most important activity of the definition phase of the GQM approach is the refinementof the selected goals into questions and metrics. The questions we have defined for ourgoals are shown in Table 3.2.
Table 3.2: Questions
Q1. How can we approximate the CK-RFC metric?
Q2. How can we approximate the CK-CBO metric?
Q3. How can we approximate the CK-WMC metric?

3.5

Definition: Metrics

In the next paragraphs, we first provide the definition of a set of metrics measured directlyfrom communication diagrams. Then, we provide the definition of a set of metrics forobject-oriented classes that can answer our formulated questions in the previous Section.
Moreover, details on the how to measure these metrics manually and how they should beinterpreted are also provided.

3.5.1

Metrics for UML Communication Diagrams

First, we propose a set of seven metrics listed in Table 3.3, which can be directly measured
from UML communication diagrams.
In the following paragraphs, we explain how to measure these direct metrics througha practical example. Let us say that we would like to measure these metrics for the26

3.5. Definition: Metrics

JAIST

Table 3.3: Direct Metrics for Objects in UML Communication Diagrams
MetricAbbreviation Stands for

NSCMsgs

Number of Sent
Call Messages
NDSCMsgs Number of Different Sent Call
Messages
NDRO
Number of Different Receiver
Objects
NRCMsgs
Number of Received Call Messages
NDRCMsgs Number of Different Received
Call Messages

Definition
For an object acting as a ’sender’, the number of call messages sent to other objects (no actors).
For an object acting as a ’sender’, the number of different
call messages sent to other objects.
For an object acting as a ’sender’, the number of different
objects who ’receive’ a message from it.
For an object acting as a ’receiver’, the number of callmessages received.
For an object acting as a ’receiver’, the number of different
call messages received.

ATMControl object in the communication diagram of Figure 3.1. According to Table 3.3,
the values of such metrics are:
• NSCMsgs equals 4 because ATMControl sends call messages 1.3, 2.5, 2.7, and 2.7a
to other objects. And NDSCMsgs = NSCMsgs, because there are not repeatedmessages.
• NRCMsgs equals 2 because ATMControl receives call messages 1.2 and 2.4. And
NDRCMsgs = NRCMsgs because there are not repeated messages. Notice thatmessage 2.6 is not counted because it is a return value, is not a call message.
• NDRO equals 3 because the different objects that receive messages from ATMControl are: CustomerInterface, ATMTransaction and BankServer.

27

28

Figure 3.1: Example of a Communication Diagram<<I/O device interface>>:CarReaderIterface<<external I/O device>>:CardReader

3.5. Definition: Metrics

1: Card Reader Input<<subsystem>>:BankServer
1.2: Card
Inserted

2.5: Valid PIN(CustomerInfo)

1.1: Card
Input Data

2.6 [Valid]: Valid PIN<<state dependent
control>>:ATMControl<<<entoty>>:ATMCard
2.4: PIN Entered(Customer Info)
2.2: Card
Data

2.7a: Update Status

2.1: Card
Request1.4: Pin Prompt

1.3: Get PIN
2.7: Display Menu

2.8: Seleccion Menu<<user interface>><<entity>>:ATMTransaction:CustomerInterface:ATMCustomer

2: PIN Input

2.3: Customer Info

JAIST

3.5. Definition: Metrics

3.5.2

JAIST

Metrics for Classes derived from UML Communication
Diagrams

From our previous set of metrics, a new set is derived to be applied to object-orientedclasses, these metrics are listed in Table 3.4.
Table 3.4: Metrics for Classes from UML Communication Diagrams
Metric
NIMO

NDIMO

NIM

NDIM

NDRC

Definition
For a given class X, the Number of Instantiated Methods of Other classes, NIMO(X), is the sum of all
NSCMsgs of all objects of class X.
For a given class X, the Number of Different Instantiated Methods of Other classes, NDIMO(X), is a countof all different NDSCMsgs of all objects of class X.
For a given class X, the Number of Instantiated Methods of class X by other classes, NIM(X), is the sum of
all NRCMsgs of all objects of class X.
For a given class X, the Number of Different Instantiated Methods of class X by other classes, NDIM(X),
is a count of all different NDRCMsgs of all objects ofclass X.
For a given class X, the Number of Different ReceiverClasses, NDRC(X), is a count of all different NDRO
of all objects of class X.

To illustrate how these metrics are measured and to explain their meaning, we aregiving a simple example based on Figure 3.2 and 3.3, which illustrate a Calculator application that multiplies and sums two integer numbers. Diagram ′ A′ in Figure 3.2 shows thecommunication diagram for the multiplication use case of the application, and diagram′ ′B , in the same Figure, represents the communication diagram for the addition use caseof the application. Figure 3.2, in its turn, shows the code of the classes of the application:
Calculator, Multiplier, Adder and Display.
The metrics for the classes of our application are shown in Table 3.5, 3.6, 3.7 and 3.8.
We first measure the direct metrics from all instances of our classes in both communication
diagrams of Figure 3.2, according to Table 3.3. Then, totals per class are calculated
according to definitions provided in Table 3.4. The summary of the metrics for the classesof our application Calculator are shown in Table 3.9. From these results, the following
observations can be made:• NIMO, NDIMO and NDRC represent a degree of dependency a givenclass has to others. In other words, they represent a degree of coupling, NIMO
and NDIMO measure the number of instances to other classes’ methods by a givenclass. And NDRC is also a measure of coupling but in terms of classes, because
293.5. Definition: Metrics

JAIST

it is based on counts of all the different objects to which a given class is coupled.
Therefore, its measure can approximate the code CK CBO metric.
In our examples, class Calculator has a total of NIMO = 4, NDIMO = 3 and
NDRC = 3, which is reflected in its code, where CK CBO = 3 coupled to class
Display, Adder and Multiplier. So that, unless all of these classes work properly,class Calculator will not. Notice that message A5 of diagram ′ A′ and message B4 ofdiagram ′ B ′ are not counted because they are sent to an actor and not to an object.
The same kind of observations can be made for class Multiplier. In the case ofclasses Adder and Display, NIMO = NDIMO = NDRC = 0, indicating thatthey do not depend on any other class, which is clearly reflected in their code.
• NIM represents the total number of method instances, of a given class, madeby other classes’ objects at some point in the execution of the application.
In our examples, classes Calculator, Adder and Display have a NIM = 2, whileclass Multiplier has a NIM = 1, which indicate that class Multiplier is lessinstantiated than the other two classes. This also can can be observed in theircode.
• NDIM can approximate the number of the different (public) methods of agiven class, because it measures the different methods, of a given class, instantiatedby other classes’ objects.
In our example, we can observe from the code of the Calculator, Multiplier, Adderand Display classes that their NDIM values equal the number of their methods(2,1,1 and 1 respectively).
From our previous observations, we can draw the following hypothesis:• Hypotheses 1: For a given class X, a UML approximation of its CK WMC measure, considering the method’s static complexities to be unity, can be obtained using
Equation 3.1.
UW MC(X) = NDIM(X);(3.1)• Hypotheses 2: For a given class X, a UML approximation of its CK CBO measurecan be obtained using Equation 3.2.
UCBO(X) = NDRC(X);(3.2)• Hypotheses 3: Since NDIM can approximate the number of methods of a givenclass X, and NDIMO the number of other classes’ methods instantiated by X, we
can have an approximation of X’s code RFC measure using Equation 3.3.
URF C(X) = NDIM(X) + NDIMO(X);

30

(3.3)

3.5. Definition: Metrics

JAIST

A1: Mutiply(a,b)

A4: Display(result):Calculator

A5: result:Display

A2: result := Multiply(a,b):Multiplier:Adder
A3: result : = Sum(a,b)

Diagram A

B1: Sum(a,b):Calculator

B2: result:=Sum(a,b)
B4:result
B3: Display(result):Display:AdderDiagram B

Figure 3.2: UML Communication Diagrams for the Calculator Application
Table 3.5: Metrics for Class Calculator
Diagram NSCMsgs
A
2=
B
Total

NDSCMsgs

NRCMsgs

NDRCMsgs NRO{A2, A4}
2={B1, B3}
4={A2, A4, B1, B3}

2={A2, A4}
2={B1, B3}
3={A2, A4 = B3, B1}

1=
{A1}
1=
{B1}
2=
{A1, B1}

1=
{A1}
1=
{B1}
2=
{A1, B1}

2={: D, : M }2={: D, : A}
3={: D, : M, : A}

= NIMO= NDIMO= NIM= NDIM= NDRC:D is a Display object, :A is a Adder object, :M is a Multiplier object

31

3.5. Definition: Metrics

JAIST

class Calculator(){private int result;private Display display = new Display();
public Multiply(int a, int b){
Multiplier multiplier = new Multiplier();
result = multiplier.Multiply(a,b);
display(this.result);}
public Sum(int a, int b){
Adder adder = new Adder();this.result = adder.Sum(a,b)}

display(this.result);}class Multiplier(){class Adder(){

public int Multiply(int a, int b){
int i, result=0;

private result;public int Sum (int a, int b){

Adder adder = new Adder();

result = a + b;

for(i=1;i<=a;i++){return(result);

result = adder.Sum(result, b);}}}

return(result);

public class Display{
public void display(int result){
String resultStr = "";resultStr = Integer.toString(result) ;System.out.println ("The result is: " + resultStr);
}

}

}
}

Figure 3.3: Code of the Calculator Application

Table 3.6: Metrics for Class Adder
Diagram NSCMsgs
A
0=
B
Total

{}
0=
{}
0=
{}

= NIMONDSCMsgs

NRCMsgs

NDRCMsgs NRO

0=
{}
0=
{}
0=
{}

1=
{A3}
1=
{B2}
2=
{A3, B2}

1=
{A3}
1=
{B2}
1=
{A3 = B2}

0=
{}
0=
{}
0=
{}

= NDIMO= NIM= NDIM= NDRC:D is a Display object, :A is a Adder object, :M is a Multiplier object

32

3.5. Definition: Metrics

JAIST

Table 3.7: Metrics for Class Multiplier
Diagram NSCMsgs
A
1=

NDSCMsgs

NRCMsgs

NDRCMsgs NRO

{A3}

1=
{A3}

1=
{A2}

1=
{A2}

1=
{: A}

B

-

-

-

-

-

Total

1=
{A3}

1=
{A3}

1=
{A2}

1=
{A2}

1=
{: A}

= NIMO= NDIMO= NIM= NDIM= NDRC:D is a Display object, :A is a Adder object, :M is a Multiplier object

Table 3.8: Metrics for Class Display
Diagram NSCMsgs
A
0=
B
Total

{}
0=
{}
0=
{}

= NIMONDSCMsgs

NRCMsgs

NDRCMsgs NRO

0=
{}
0=
{}
0=
{}

1=
{A4}
1=
{B3}
2=
{A4, B3}

1=
{A4}
1=
{B3}
1=
{A4 = B3}

0=
{}
0=
{}
0=
{}

= NDIMO= NIM= NDIM= NDRC:D is a Display Object, :A is a Adder Object, :M is a Multiplier object

Table 3.9: Summary of Metrics for Calculator ApplicationClass
Calculator
Adder
Multiplier
Display

NIMO
4
0
1
0

NDIMO
3
0
1
0

NIM
2
2
1
2

NDIM
2
1
1
1

NDRC
3
0
1
0

Table 3.10: Summary of Metrics for Calculator Application: UML CK Metrics
Class
Calculator
Adder
Multiplier
Display

UCBO
3
0
1
0

URFC
5
1
2
1

33

UWMC
2
1
1
1

3.6. Summary

3.6

JAIST

Summary

To sum up the definition of our goals and metrics, we provide the answer of our questions
defined in previous Sections in terms of the our proposed metrics. This can be found in
Table 3.11.
Table 3.11: Metrics Definition based on the GQM Approach
Goal / Questions / Metrics
Analyze UML Communication Diagrams
For the purpose of obtaining an approximation of the CK RFC,
CBO and WMC metrics
With respect to their objects interactions
From the point of view of the researcher
In the context of software quality studies
Q1. How can we approximate the CK-RFC metric?
Using the metric URFC (Equation 3.3).
Q2. How can we approximate the CK-CBO metric?
Using the metric UCBO (Equation 3.2).
Q3. How can we approximate the CK-WMC metric?
Using the metric UWMC (Equation 3.1).

34

Chapter 4
UML CK Metrics Evaluation

The contents of this Chapter are related to the evaluation of our UML RFC, CBO and
WMC metrics as approximations to their respective code metrics. To perform such anevaluation, we first measure the CBO, RFC and WMC CK metrics from the code of threedifferent small-size software projects written in Java. Then, we measure the same metrics
from their UML communication diagrams (called collaboration diagrams in previous UML
versions to the 2.0 version). Such measurements are performed following the procedures
provided in the previous Chapter. Finally, we compare both sets of measures, UML andcode, and evaluate the goodness of our approximation by calculating the approximation
error of our UML metrics to their respective code metrics.

4.1

Data Collection

The four small-size software projects used for our evaluation were developed by different
teams of students of JAIST during a 3-month lecture. All of the four projects were written
in Java. Every group of students based its implementation on the analysis and basic designof the projects provided in Gomaa’s book [23], in fact, it is supposed that every groupdeveloped a more complete design before proceeding with the implementation. The four
projects are:
• Two banking systems (BNA and BNB), which were developed by different student
groups, following the same basic design (BNS).
• A cruise control and monitoring system (CRS).
• An e-commerce system (ECS).
In order to evaluate the goodness of our UML metrics as approximations to the code
WMC, CBO and RFC metrics, the following procedure was undertaken:

354.2. Measures for our Evaluation

JAIST

1. UML measures. We first calculated the UML CBO, RFC and WMC metrics perclass of every of the three different systems using the UML communication diagrams provided in [23]. This calculation was performed according to the proceduredescribed in Chapter 3, and although, the UML diagrams provided in [23] are notreferred as communication diagrams but collaboration diagrams instead, the sameprocedures to measure our UML CK metrics can be applied.
2. Code measures. The different teams of students used CVS repositories to keep recordof their implementations. CBO, RFC and WMC measures per class were collectedfrom the last version of the four systems from their respective CVS repository. Suchmeasurement was carried out using the CKJM tool, which calculates CK metrics
by processing the byte-code of compiled Java files 1 .
3. Evaluation. Finally, we compared both resulting sets of measures by using a simple
line graph, measuring the average relative errors of our approximations, and calculating the correlation coefficients between UML and code measures to determine
whether it exists a possible linear relationship between them. The numbers of classes
analyzed were 11, 10, 16 and 10 from the BNA, BNB, CRS and ECS respectively.
Notice that we are capturing design information at a very early stage of the life cycleof the software projects, since we are measuring the UML metrics from the UML diagrams
provided in Gomaa’s book, which were used by the students to elaborate a more complete
design to finally implement the systems.

4.2

Measures for our Evaluation

4.2.1

Error Approximation

The approximation error of some data refers to the discrepancy between an exact value
and some approximation to it. An approximation error can occur either because themeasurement of the data is not precise, due to the instruments used to carry out themeasurement, or because approximations are used instead of the real data, which is ourcase.
Given an approximation aprox of a correct value x, the absolute error is defined as theabsolute value of the difference between the approximation and the correct value, whichis expressed by Equation 4.1.
Errorabs = |x − approx|(4.1)

There are two problems with using the absolute error:• Significance. The absolute error can tell us the size of the error, but cannot tell us
how significant this error is. For example, if we had an absolute error of 3.52, this
could not be considered significant, if the correct value is x = 5030235.23, however
1

CKJM was written by Diomidis Spinellis and can be found in http://www.spinellis.gr/sw/ckjm/

36

4.2. Measures for our Evaluation

JAIST

if the correct value is x = 5.03023523, then our absolute error probably could bevery significant.
• Units. The absolute error will change depending on the units used, which can betroublesome if we want to use the absolute error for comparison. For example, theabsolute error of the approximation, 2.4 MV, of an actual voltage of 2.57 MV, is0.17 MV, whereas the absolute error of the approximation, 2400000 V, to an actual
voltage of 2573000 V is 173000 V. Having both results, we cannot tell which error
is more or less significant than the other.
To solve the problem of significance and units, the relative error is used. This is definedas the ratio between the absolute error and the absolute value of the correct value, see
Equation 4.2.
|x − approx|
(4.2)
|x|
In this equation, any units are canceled, so the relative errors of the approximations
2.4 MV and 2400000 V versus the actual voltages of 2.57 MV and 2573000 V, respectively,
are equal. Also, a relative error of 0.01 means that the approximation is correct to within
one part in one hundred, regardless of the size of the actual value.
For our evaluation, relative errors were calculated. In order to avoid divisions by zero,measurement observations in which the true value (x in Equation 4.2) resulted to be zero
were excluded from our evaluation.
Errorrel =4.2.2

Correlation

As it is well known in probability and statistics, correlation indicates the strength anddirection of a linear relationship between two random variables. In general statisticalusage, correlation or co-relation refers to the departure of two variables from independence.
A number of different coefficients are used for different situations. The most wellknown is the Pearson correlation coefficient, which is obtained according to the following
formula:Pn(xi − x̄)(yi − ȳ)pPnr = pPn i=1
(4.3)
2
2
(x
−
x̄)
(y
−
ȳ)
i
i
i=1
i=1
Where r is the Pearson coefficient based on a sample of paired data (xi , yi), and n is
the size of the sample.
Statistical Significance of Correlation Coefficients
Within the context of correlation, the question of statistical significance concerns therelationship between r, which is the correlation that is observed within a limited sampleof pairs of data and the existent correlation ρ in the larger reality beyond the sample, we
explain further in the following paragraphs2 .
2Onlinetextbookof“Concepts
and(http://faculty.vassar.edu/lowry/webtext.html)

37

ApplicationsofInferential

Statistics”

4.2. Measures for our Evaluation

JAIST

A sample is a relatively very small window through which the investigator glimpses
the outlines of some larger reality. In some cases, that glimpse might truly represent
the larger reality, in other cases it might misrepresent it. There is always the possibility
that the observed facts in a sample result from nothing but from mere chance. Statisticalsignificance is the logical and mathematical apparatus by which the investigator can assess
whether the yielded correlation coefficient is not just a fluke of mere chance coincidence.
The criterion for statistical significance is conventionally set at the 5% level. That is,an observed result is regarded as statistically significant (as something more than a mere
fluke) only if it had 5% or smaller likelihood of occurring by mere chance coincidence.
Table 4.1 shows the positive and negative values of r that are required for statisticalsignificance at the 5% level for various sample sizes N. For example, for any particularsample size, an observed value r is regarded as statically significant at the 5% level if andonly if its distance from zero is equal to our greater than the distance of the value of r in
Table 4.1. This, for a sample of size N=20, an observed value of r = +0.4 or r = −0.4
would be significant at the 5% level for a directional hypothesis, but non-significant for a
non-directional hypothesis.
Within the context of correlation, a directional hypothesis is one that leads the investigator to specify, in advance, one or the other of the following expectations:• Positive directional hypothesis: The relationship between two variables X and Y in
the general population is positive, the more of X the more of Y , hence a particularsample of pairs (xi , yi ) will show a positive correlation.
• Negative directional hypothesis: The relationship between two variables X and Y
in the general population is negative, the more of X the less of Y , hence a particularsample of pairs (xi , yi ) will show a negative correlation.
A non-directional hypothesis, on the other hand, leads only to the expectation thatthe correlation between X and Y within the general population might be something otherthan zero, hence a particular sample of pairs (xi , yi) will show a non-zero correlation either
positive or negative, though we have no basis for predicting which of these it will be.
Table 4.1: Correlation Coefficients Required for Statistical Significance at 5% Level
N

Directional
(±r)

5
10
16
20
27
32

0.81
0.55
0.43
0.38
0.32
0.30

38

NonDirectional
(±r)
0.88
0.63
0.50
0.44
0.38
0.35

4.2. Measures for our Evaluation

JAIST

In our study, in order to determine whether the obtained correlation coefficients between our UML approximation and code metrics can be regarded as statistically significant, and unlikely to occur by mere chance, we used as a threshold the conventional 5%
level (p-value <= 0.05) considering a non-directional hypothesis.

4.2.3

Precision and Accuracy

Precision and Accuracy are often used in the evaluation of methods or instruments ofmeasurement. Ideally a measurement device must be both accurate and precise.
Precision
Precision is defined as the degree of refinement in the method used to make a measurement, and it can be observed in the closeness to each other of the data observations of aset of measurements. The closer the observations are to each other, the more precise themethod of measurement is considered. Precision of a measurement system is also called
reproducibility or repeatability, referring to the degree to which repeated measurements
under unchanged conditions show the same results.
Precision can be usually discussed in terms of standard deviation (σ for the entire
population or s for a given sample). Assuming that the distribution of the data underobservation is a normal distribution, the observer can say that 68% of the results fall into
the range of the data mean ±1s , or 95% of them fall into the range of the data mean ±2s.
The less the different observations vary from their mean or expected value, the higher the
precision is.
Accuracy
The accuracy of a measurement is the degree of agreement or conformity with its true
value. It refers to the degree of perfection obtained in measurements.
In mathematics, the accuracy of a measure, when this is an approximation, is expressedin terms of its relative error, previously introduced. The smaller the relative error, themore accurate the measure is.
Precision, Accuracy and Errors
The theory of errors and measurement assumes that any measurement is composed of itstrue value plus some error value, which in its turn is composed of two types errors:• Random. Such type of error in measurement lead to measurable values being inconsistent when repeated measures of a constant attribute or quantity are taken.
Random errors are caused by unpredictable fluctuations in the readings of a measurement apparatus, or in the experimenter’s interpretation of the instrumental
reading; these fluctuations may be in part due to interference of the environment
with the measurement process. They are caused by factors beyond the control of
the observer. The rules of random errors are:39

4.2. Measures for our Evaluation

JAIST– Small errors occur more frequently than large ones.
– Both positive and negative errors are equally likely.
• Systematic or Bias. This type of error is considered to be predictable, and typicallyconstant or proportional to the true value.
Systematic errors are caused by imperfect calibration of measurement instruments orimperfect methods of observation, or interference of the environment with the measurement process, and always affect the results of an experiment in a predictabledirection. They are cumulative, and thus their effect can be computed and corrections can be applied.
There is a definite relationship between precision and accuracy of a measurement
system, and random and systematic errors:• The lower the number of random errors is, the higher the precision.
• The lower the number of systematic errors, the higher the accuracy.
This is illustrated in Figure 4.1, where accuracy is represented by the proximity ofmeasurement results to the true value, and precision is represented by the spread of thedata under the curve.
(−)
Systematic Errors(+)
Accuracy
Probability
density

Reference Value

Mean
Precision

Value
(+)

Random Errors(−)

Figure 4.1: RAW RFC measures40

4.3. RFC

4.3

JAIST

RFC

The graph in Figure 4.2 shows the raw RFC measures of the BNA, BNB, CRS and ECS
software projects. The line with squares represents RFC measured from UML diagrams,
and the line with circles represents RFC measured from the code. As we can observe, there
is a gap between the UML measures and the code measures, which is apparently larger inthe BNA project. Extreme deviations, such as the code of the class BNA-05 , occur mainly
because the number of calls to other classes’ methods exceeds that one estimated using
UML communication diagrams at design time, more details are given in a later section.

Raw Measures: RFC vs URFC
160
150
RFC
URFC

140
130
120
110
100
90
80
70
60
50
40
30
20
10

BNA−01
BNA−02
BNA−11
BNA−09BNA−08BNA−03
BNA−10
BNA−04
BNA−07
BNA−06
BNA−05
BNB−04
BNB−02
BNB−05
BNB−07
BNB−03
BNB−10
BNB−08
BNB−09
BNB−11
BNB−06
CRS−01
CRS−04
CRS−07
CRS−12
CRS−13
CRS−02
CRS−05
CRS−08
CRS−09
CRS−11
CRS−14
CRS−15
CRS−06
CRS−10
CRS−16
CRS−03
ECS−01
ECS−11
ECS−07
ECS−02
ECS−13
ECS−05
ECS−09ECS−10
ECS−04
ECS−060

Figure 4.2: RAW RFC measures
Accuracy and Precision
The average relative errors of our UML RFC approximations to their code RFC measures are 0.85, 0.66, 0.48 and 0.8 for the BNA, BNB, CRS and ECS respectively.
Moreover, the standard deviations of our relative errors are 0.1, 0.19, 0.25 and 0.15, for
the BNA, BNB, CRS and ECS respectively. Assuming a normal distribution of our relative
errors, we can say that 68% of them fall into the following ranges: 0.85 ± 0.1 , 0.66 ± 0.19,
0.48 ± 0.25 and 0.8 ± 0.15 for the BNA, BNB, CRS and ECS projects respectively.
Considering a threshold of 0.5 for our average relative errors and 0.25 for our standard deviations to evaluate accuracy and precision, we can say that our methodology toapproximate RFC to its code is more precise than accurate. Thereafter, the number ofrandom errors should be lower than the number of systematic errors.

41

4.4. CBO

JAIST

Linear RelationshipMoreover, the correlation coefficients between the code and UML measures tell us that
there is a strong linear relationship between them. The correlation coefficients are 0.7,
0.8, 0.57 and 0.86 for the BNA, BNB, CRS and ECS projects respectively. These correlation coefficients resulted to be statically significant to a minor p-value than the one usedconventionally; the obtained p-values are 0.016, 0.005, 0.02 and 0.0014 for the BNA, BNB,CRS and ECS projects respectively, suggesting that the three correlation coefficients areunlikely to occur by mere chance.

4.4

CBO

The graph in Figure 4.3 shows the raw CBO measures of the BNA, BNB, CRS and ECS
software projects. The line with squares represents CBO measured from UML diagrams,
and the line with circles represents CBO measured from the code. As we can observe, thegap between the UML and the code measures is not as large as the one found between
RFC measures. In fact, the average relative errors obtained for our CBO measures are
smaller than those ones obtained for our RFC approximations.
Accuracy and Precision
The obtained average relative errors are 0.52, 0.56, 0.55 and 0.49 for the BNA, BNB,CRS and ECS respectively.
Moreover, the standard deviations of our relative errors are 0.37, 0.33, 0.41 and 0.32,
for the BNA, BNB, CRS and ECS respectively. Assuming a normal distribution of our relative errors, it can be said that 68% of them fall into the range of 0.52 ± 0.37 ,0.56 ± 0.33,
0.55 ± 0.41 and 0.49 ± 0.32 for the BNA, BNB, CRS and ECS projects respectively.
Considering the same threshold values for evaluating precision and accuracy used previously, we can say that our methodology to measure CBO is more accurate than precise;and less inaccurate than our methodology to approximate RFC.
Notice that the number of classes used for this analysis changed due to CBO values
equal to zero, the number of classes used were 6, 6, 15, 6 for the BNA, BNB, CRS andECS projects respectively.
Linear RelationshipOn the other hand, the correlation coefficients between our UML and code measurestell us that there exists a stronger linear relationship between them. The correlation
coefficients are 0.85, 0.87. 0.56 and 0.92 for the BNA, BNB, CRS, and ECS projects respectively. The obtained p-values are 0.0009, 0.001, 0.024 and 0.0016 for the BNA, BNB,CRS and ECS projects respectively, which are smaller to the conventional p-value used,suggesting that the three correlation coefficients are statistically significant and unlikely
to occur by mere chance3 .
3This results are slightly different to those published in [11] due to a higher number of classes analyzed.

424.5. WMC

JAIST

Raw Measures: CBO vs UCBO

25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0

BNA−04
BNA−05
BNA−09BNA−10
BNA−07
BNA−06
BNB−02
BNB−07
BNB−05
BNB−09
BNB−10
BNB−06
CRS−01
CRS−05
CRS−07
CRS−08
CRS−09
CRS−11
CRS−13
CRS−04
CRS−10
CRS−06
CRS−02
CRS−14
CRS−15
CRS−16
CRS−03
ECS−10
ECS−13
ECS−04
ECS−05
ECS−09ECS−06CBO
UCBO

Figure 4.3: RAW CBO measures

4.5

WMC

The graph in Figure 4.4 shows the raw WMC measures of the BNA, BNS, CRS and ECS
software projects. The line with squares represents WMC measured from UML diagrams,
and the line with circles represents WMC measured from the code. As we can observe,
there is a gap between the UML and code measures, which is larger in the BNS and ECS
projects.
Accuracy and Precision
The average relative errors of our approximations are 0.69, 0.56, 0.43 and 0.94 for the
BNA, BNB, CRS and ECS respectively. Considering a threshold of 0.5 for our averagerelative errors, only our measures for the CRS data were accurate.
As for precision, the standard deviations of our relative errors are 0.15, 0.41, 0.26 and
0.79, for the BNA, BNB, CRS and ECS respectively. Assuming a normal distributionof our relative errors, we can say that 68% of them fall into the range of 0.69 ± 0.15,
0.56 ± 0.41, 0.43 ± 0.26 and 0.94 ± 0.79 for the BNA, BNB, CRS and ECS projects respectively. Considering a threshold of 0.25 for our standard deviations, only the approximations of WMC for the BNS project can be regarded as precise.
From the previous resulting measures, the only conclusion we can draw so far, is that
our methodology to approximate WMC is much more inaccurate and imprecise than our
two previous methodologies to approximate RFC and CBO.

43

4.6. The Gap

JAIST

Raw Measures: WMC vs UWMC

80
WMC
UWMC

70

60

50

40

30

20

10

BNA−11
BNA−08BNA−03
BNA−09BNA−07
BNA−10
BNA−04
BNA−01
BNA−02
BNA−06
BNA−05
BNB−02
BNB−04
BNB−03
BNB−07
BNB−08
BNB−09
BNB−10
BNB−11
BNB−05
BNB−06
CRS−04
CRS−07
CRS−01
CRS−02
CRS−06
CRS−12
CRS−13
CRS−14
CRS−15
CRS−11
CRS−09
CRS−05
CRS−08
CRS−16
CRS−10
CRS−03
ECS−06
ECS−09ECS−01
ECS−02ECS−07
ECS−11
ECS−13
ECS−05
ECS−04
ECS−100

Figure 4.4: RAW WMC measures
Linear RelationshipOn the other hand, the correlation coefficients between the UML and code measuresof the BNA, BNB, CRS and ECS projects are 0.69, 0.71, 0.51 and -0.08 respectively. The
obtained p-values are 0.01, 0.02, 0.04 and 0.82 for the BNS, CRS and ECS projects respectively, suggesting that there exists a linear relationship between code and UML WMC
measures, except for the ECS measures4 .

4.6

The Gap

In this section, we try to analyze some of the possible causes originating the difference ofvalues between our UML and code measures of our different study cases, such causes are:
1. Detail Level of the Design. Since two different designers can provide different levelsof detail in their design, even if their designs corresponds to the same system,measuring UML metrics from such designs would yield different gaps between the
resulting design measures and their respective code measures taken from their actual
implementation.
For our specific study cases, we consider that the same level of detail was used in the
different communication diagrams of the design of the BNS, CRS and ECS. Therefore, under this assumption, our UML measures of all classes can be considered to4

Our previous results published in [11] indicated no linear relationship for the BNA due to a smaller
number of classes analyzed

444.6. The Gap

JAIST

be equally proportional to their respective code metrics. This is illustrated in Figure
4.5, where the horizontal axis represents the different classes of a given system, andthe vertical axis represents CBO; the dashed line represents the different UML CBO
measures and continue line represents their respective code measures.
2. Software type. Depending on which kind of application the software is, it may require the usage of different technologies in different degrees, such as GUIs, databaseconnectivity, IPC communication, etc.
This factor may cause a proportional difference of values between code and UML
measures in different groups of classes of the system. For example, consider two
different calculator systems, A and B, which were implemented based on the same
design having exactly the same core functionalities, such as adding, multiplication,etc., with only one difference, which is that system A uses GUIs and system B uses acommand line interface. Now, let us say that we would like to measure CBO, which
is a coupling measure. In general, classes interacting with end users (eu-classes) of
system A may have a higher degree of coupling than those eu-classes of system B
because of the usage of GUI functionalities. What could be the results of measuringcode and UML CBO measures from both systems? There can be two possibilities:(a) If GUI functionalities were not decoupled from the main domain classes considered in the design of the system, we may find that although system A is using
GUIs and system B is not, their respective code measures may be proportional
to their common UML metrics. In principle, the group of main domain classesof both systems, not implementing GUIs, would have almost the same code
CBO values. The code CBO measures of the groups of classes eu-classes, on
the other hand, may be different, but still proportional to their design. Such an
imaginary situation is represented in Figure 4.6, where the horizontal axis represents the different classes ( eu-class-i represents end-user classes and class-i
other classes), and the vertical axis measures CBO. The line graph with circles
represents the UML CBO measures, the line graph with triangles represents
the code CBO of system A, and the line graph with squares the code CBO ofsystem B.
(b) If GUI functionalities were decoupled from the main domain classes consideredin the design of the system into different classes, we may find that code CBO
measures of both systems A and B are almost the same, and therefore bothsets of CBO measures would be equally proportional to their common UML
measures.
3. Developer programming style. It refers to the particular programming style of adeveloper. Software systems developed by more than one developer can affect indifferent degrees the difference of values between UML and code measures to different groups of classes of a system. The same degree of influence over all classes ofa system can be expected, if only one developer implements all of the classes of asystem, or if all the developers, involved in the implementation of the system, havemore or less the same programming style.
45

4.6. The Gap

JAIST

4. Inconsistent code. This occurs when a developer decides to implement a class orclasses changing their original design. Such factor is expected to be sporadic. However, if it occurs, then the degree in which influences the difference of values between
UML and code measures is not equal, neither proportional, for all of the classes ofa given system.
The gap or errors produced by factors 1 and 2 could be classified as systematic typeerror or bias, because both factors are assumed to be somehow proportional. On the other
hand, errors produced by factors 3 and 4 are less predictable, therefore we could classified
them as random type of errors.
In order to have a better understanding of such a gap, we compared the RFC, CBO
and WMC measures of the two different implementations BNA and BNB of the BNS
in Figures 4.7, 4.8, 4.9. In these figures the horizontal axis corresponds to the different
designed classes of the BNS and vertical axis to measures CBO. The line with circles
corresponds to the UML measures, the line with squares to the BNA implementation andthe line with diamonds to the BNB implementation. BNA tends to have higher valuesof RFC and WMC, which are metrics given in terms of methods. Our first hypothesis
was that the difference between the two implementations was due to factor 3, different
programming styles among developers can produce different CK measures. Inspecting thecode of the designed classes BNS-04 and BNS-05, of both projects BNA and BNB, wefound that:• BNA-04 and BNA-05 have much higher values of RFC and WMC than those of thedesign (BNS-04 and BNS-05) due to the usage of GUIs functionalities.
• BNB was implemented different from the original design. Designed classes BNS-04
and BNS-05 were implemented as interfaces in BNA-04 and BNA-05. The class that
actually implements these two interfaces is BNB-00 (not considered in the original
design).
• BNB-00 CK measures still much lower than the CK measures of BNA-04 and BNA05. This is due to the manner GUI functionalities are handled. BNB-00 (different
to BNA classes) decouples the GUI functionalities from those specific to the class,which is achieved by defining an inner class to handle all GUIs. BNA classes does
not make this decoupling.
• Although BNB implementation is inconsistent with the design, it handles GUIs ina smarter manner than BNA does.
Therefore, from these observation we can say that the difference between the two setsof measures of the two different implementations, BNA and BNB, is caused mainly due tofactor 4, the original design was changed in BNB. Factor 3 may also contributing to thisdifference, however to assess accurately how much is the effect of this factor is not possible
with the information we currently count with. So far, we can only say that despite the two
implementations were developed by different groups of developers, factor 3 is not causinga large difference between the measures of both implementations.
46

4.6. The Gap

JAISTSome other factors may be influencing the difference of values between UML and code
measures. To identify all of them and the degree in which each of them contributes tosuch a difference is a troublesome task. In such situations, what statisticians do is totarget systematic errors and identify some feature which correlates to this type of error.
Moreover, in a broader manner, the correction of systematic errors is also targeted byapplying data normalization methods. This kind of approach has been widely practisedin other research fields such as bio-informatics for micro-array data analysis and image
processing for feature normalization [2, 8].
For the treatment of random errors, which remain after mistakes and systematic errors
have been eliminated, application of adjustment techniques to obtain adjust observations
that fit a given math model can be applied.
Our work has been delimited and focused to decrease the systematic error of ourapproximations using data normalization methods, independently of their nature.
CBO
CODE
UML

ClassesFigure 4.5: Design Level of Detail Effect47

4.6. The Gap

JAIST
CBO
CODE SYSTEM A
CODE SYSTEM B
UML

eu−class−05

eu−class−04

eu−class−03

eu−class−02

eu−class−01

class−04

class−03

class−02

class−01

ClassesFigure 4.6: Software Type EffectFigure 4.7: RFC: Same design, two different implementations48

4.6. The Gap

JAIST

Figure 4.8: CBO: Same design, two different implementations

Figure 4.9: CBO: Same design, two different implementations49

4.7. Conclusion

4.7

JAIST

Conclusion

Table 4.2 summarizes the results of our evaluation, it shows the average relative errors of
our approximations and their corresponding standard deviations, as well as the correlation
coefficients between UML and code measures. Based on these measures, we evaluate
accuracy and precision of our UML metrics, and linear relationship to their respectivecode measures. If we suppose a normal distribution of the resulting relative errors, using
average values, the curves of their distributions would take the form of the curves shownin Figure 4.10, which can be interpreted as follows.
Considering a 0.5 threshold for our average relative errors and 0.25 for our standard
deviations, our UML CBO metric results to be the least inaccurate, and our UML RFC,
the most precise. Therefore, it can be said that the impact of random type of errors on
the gap between our UML and code measures is less harmful for RFC than for CBO, andthe impact of systematic type of errors is stronger for our RFC approximations than for
our CBO approximations. This can be explained due to the units used for measuring
CBO and RFC. Since CBO is given in terms of classes and RFC in terms of methods, it issomehow understanding that our UML approximation of RFC results to be less accurate
than our UML approximation for CBO.
Furthermore, a strong linear relationship between code and UML measures was found,for both of our UML CBO and RFC approximations in all projects.
As for our UML WMC metric, the following conclusions can be made. Because of
the resulting error approximations of our UML WMC measures to their code measures inall classes analyzed were highly inaccurate and imprecise, we conclude that the numberof designed methods per class is still far from the actual number of the implemented
methods, at least at the stage the UML communication diagrams of our projects were
designed.
Last, we mentioned that to identify the true sources of our approximation errors andtheir contribution to the overall error is a troublesome task. Therefore, our work has beendelimited and focused to reduce simply the systematic type of error of our approximation,
independently of their nature, using data normalization methods.

50

4.7. Conclusion

JAIST

Table 4.2: Summary of Evaluation
System

BNA
BNB
CRS
ECS
Average
BNA
BNB
CRS
ECS
Average
BNA
BNB
CRS
ECS
Average

Average SD
Correlation
(RE)
(RE)
(code, UML)
RF CApproximation
0.85
0.1
0.7
0.6
0.19
0.8
0.48
0.25
0.57
0.8
0.15
0.86
0.7
0.17
0.73
CBOApproximation
0.5
0.37
0.85
0.6
0.33
0.87
0.5
0.41
0.56
0.49
0.32
0.92
0.5
0.36
0.8
W MCApproximation
0.69
0.15
0.69
0.56
0.41
0.71
0.43
0.26
0.51
0.94
0.79
-0.08
0.65
0.4
0.46

Evaluation
Accuracy

Precision

Linearity

+++
++
+
++
+

++
++
+
++
+

+

+++
+++
+
+++
++

+
+
+
++
+

+
++
+

+:Fair, ++: Good, +++: Very Good

Probability Density

WMC

RFC

CBO

0

0.65 0.7

0.5

Relative Errors

Figure 4.10: Normal Distributions of Relative Errors of or UML Approximations

51

Chapter 5
Normalization

Recalling our second main goal is to test the ability our UML metrics to predict faulty
code. In order to do so, code-based prediction models are built, and tested using as their
inputs, on the one hand, UML measures, and on the other hand, their respective code measures from the JAIST projects. However, this task can be troublesome, not only because
our UML approximations are not close enough to their respective code metrics (according
to the results obtained in the previous Chapter), but also because data distributions of
the CK metrics varies from project to project, or within the same project from package
to package, as it is observed from a further data analysis presented in a later section.
In this Chapter, we first present our results and observations from further data analysis
on different datasets of code and UML CK measures using boxplots. Then, we study theapplication of two data normalization methods, which has a two-fold purpose to decrease
the difference of values (approximation error) between UML and code measures and thevariability of data spread among software projects and packages.
The data used in this Chapter are from the same four different JAIST projects usedin the previous Chapter, and from the code of an open source project, which is used in alater step to build our code-based prediction models.

5.1

Comparing CK Data Distributions using Boxplots

In order to analyze and compare the CBO, RFC and WMC data of our projects, we use
simple boxplots. A boxplot provides a visual impression of several important aspects ofthe empirical distribution of a batch data. It is especially useful for comparing several
batches of data. From a boxplot we can have at glance important characteristics of ourdata such as: location, spread, skewness, tail length and outlying data points [25].
To construct a single boxplot of a given dataset, the following information is required:• The median. To compare location of batches.
525.1. Comparing CK Data Distributions using Boxplots

JAIST• The fourth spread. To compare spread among batches. It is an indicator of how
concentrated the values are. And it is calculated as follows:
fourth-spread = dF = FU − FL

(5.1)

Where,– FU is the lower fourth of the dataset, also called first quartile, and tell us that
25% of the total of the observations are smaller to its value.
– FL , call the upper fourth or third quartile, corresponds to the value; so that
75% of the total of the observations are smaller.
Therefore, the forth spread or interquartile range (IQR) represents the range containing the middle 50% of the data.
• The identification of outliers. Commonly in statistics data values that are far enoughthe fourths are considered as potential outliers. Specifically FL − 1.5dF and FU +1.52dF are defined as the outlier cutoffs [25]. In this study, we use this convention
for the identification of outliers.
The presence of significant outliers produces biased regression estimates and reduces
the predictive power of regression models [18]. From there lies in the importance ofidentification and proper treatment of outliers.
As an example, we show in Figure 5.1 a boxplot for the CBO code measures of the
BNA project. Table 5.1 shows the values of the CBO measures for 11 classes of the BNS,
as well as the information required to build the boxplot, which was built according to thefollowing procedure:
1. A box was drawn with the ends at the lower fourth FL and upper fourth FU . For
the BNA CBO data these values are 0 and 3 respectively, but CBO = 3 does notexist, therefore the limit of the box is drawn at CBO = 4.
2. A crossbar was drawn at the median, at 1.
3. A line was drawn from each end of the box to the outlier cutoffs. The lower outlier
cutoff, FL − 1.5dF , equals -4.5, which does not exist, therefore we take the nearest
value, 0. For the upper outlier cutoff, FU − 1.5dF , we have a value of 7.5, which
neither exists, therefore we choose the closest value, 6.
4. Finally, outliers are represented by drawing circles individually, beyond the outlier
cutoffs. For the BNA the only outlier value is 13.
Having explained the functionality of a boxplot, we can easily compare the CBO, RFC
and WMC data measures of our projects and look for similarities and differences among
them using boxplots.

53

5.1. Comparing CK Data Distributions using Boxplots

Figure 5.1: Boxplot for CBO data of a Banking System

Table 5.1: CBO Code measures of the BNS
1
2
3
4
5
6
7
8
9
10
11

Class
BNA-06
BNA-07
BNA-10
BNA-05
BNA-09
BNA-04
BNA-01
BNA-02
BNA-03
BNA-08
BNA-11

CBO
13
6
4
2
2
1
0
0
0
0
0

General DataMedian = 1,
FL = 0, FU = 3 → 4,
dF = 3,FL − 1.5dF = −4.5 → 0,
FU + 1.5dF = 7.5 → 6

54

JAIST

5.1. Comparing CK Data Distributions using Boxplots

JAIST

As we mentioned earlier, apart from exploring the data from the JAIST projects,
used in the previous Chapter, we have included the CBO, RFC and WMC measurescollected from 13 packages of an open-source project named Mylyn from Eclipse (MYL).
The MYL system was written in Java by six developers in different locations. A total of
37 classes from the JAIST projects, and 602 classes from the MYL project are used forthis analysis. RFC, CBO and WMC code and UML measures were collected from theseclasses. Figures 5.2, 5.3,5.4 show the different boxplots for the code and UML CBO,
RFC and WMC measures collected respectively. In these Figures, ’COD’ stands for code
measures, ’UML’ for UML measures, and ’M-x’, where x ranges from 1 to 13, refers to
the different 13 packages of the MYL. Tables 5.2, 5.3, and 5.4 we provide the detailed
information of each of the datasets represented in these figures. From these figures andtables, the following observations can be made:• Datasets with the high number of classes tend to have more number of outliers.
• In general most of the data are positively skewed. Their right tail is longer, whichmeans that most of the data is concentrated on the left, having relatively few highvalues.
• A slight tendency for data to spread as the level increases can be observed, speciallyfor CBO and RFC data. These can be more clearly observed in Figures 5.5, 5.6, 5.7,
where level (M) versus spread (dF ) plots of our our CBO, RFC and WMC data,
respectively, are shown.
Overall, there exists a great variability among the batches, using either code or UML.
In order to promote equality of data spread and reduce such variability, data can be reexpressed by using data transformations. In the next Section, we study the application of
two data transformations and determine which of these transformations is more suitablefor our data.

55

5.1. Comparing CK Data Distributions using Boxplots

Table 5.2: Detailed Information for CBO Boxplots
CBO Boxplots
Batch

Total
MediandF
Upper OutClasses
lier Cutoff
BNA-COD
11
1
3
7.5
BNA-UML
11
1
2.5
6.25
BNB-COD
10
1
4.25
10.63
BNB-UML
10
1.5
2.75
6.88
CRS-COD
16
1.5
4
11
CRS-UML
16
1
1
3.5
ECS-COD
10
3
5.5
13.7
ECS-UML
10
0.5
6.7
16.8
MYL-P0139
3
6
15.5
MYL-P02
30
3.5
4.5
12.5
MYL-P03
56
7
10.5
30
MYL-P04
54
12
9.7
31.38
MYL-P05
23
14
19.5
57.7
MYL-P06
49
12
15
42.5
MYL-P07
24
9.5
19.5
52.5
MYL-P08
10
13
10
37.2
MYL-P09
199
10
13
37.5
MYL-P1021
7
18
48
MYL-P11*
49
5
16
41
MYL-P12
17
5
7
21.25
MYL-P13
31
29.5
29.5
80.2
(*) Dataset from which the best CBO prediction models were derived

56

JAIST

5.1. Comparing CK Data Distributions using Boxplots

Table 5.3: Detailed Information for RFC Boxplots
RFC Boxplots
Batch

Total
MediandF
Upper OutClasses
lier Cutoff
BNA-COD
11
24
42.5
122
BNA-UML
11
4
3.5
11.25
BNB-COD
10
12.5
9.75
32.38
BNB-UML
10
4.5
3.5
11.75
CRS-COD
16
9
3.7
14.6
CRS-UML
16
4
2.5
8.7
ECS-COD
10
29
10.7
52.1
ECS-UML
10
4.5
9.7
26.3
MYL-P01
39
22
24
72.5
MYL-P02
30
35
43.5
121.7
MYL-P03
56
21
27
79.2
MYL-P04
54
25
21.5
68
MYL-P05
23
21
31
92.5
MYL-P06
49
23
30
84
MYL-P07
24
22
43.7
122
MYL-P08
10
37
28.2
88
MYL-P09
199
19
28.2
82
MYL-P1021
1032
85
MYL-P11
49
18
33
88
MYL-P12*
17
13
13
42
MYL-P13
31
35
65
182
(*) Dataset from which the best RFC prediction models were derived

57

JAIST

5.1. Comparing CK Data Distributions using Boxplots

JAIST

Table 5.4: Detailed Information for WMC Boxplots
WMC Boxplots
Batch

Total
MediandF
Upper OutClasses
lier Cutoff
BNA-COD
117
7
22.5
BNA-UML
11
2
2.5
7.7
BNB-COD
10
4
1.75
7.38
BNB-UML
10
2.5
3.25
9.38
CRS-COD
1643
11.5
CRS-UML
16
3
2.2
7.4
ECS-COD
10
6
2.2
10.8
ECS-UML
104
3
9.5
MYL-P01
39
6
10.5
29
MYL-P02*
30
17.5
18
54
MYL-P03
56
6
6.7
21
MYL-P04
54
7
8.5
24
MYL-P05
23
7
5.5
18
MYL-P06
49
5
7
20.5
MYL-P07
24
6.5
8.2
26
MYL-P08
10
6.5
6.5
21.5
MYL-P09199
5
6
18
MYL-P1021
3
7
19.5
MYL-P11
49
6
8
24
MYL-P12
17
6
6
17
MYL-P1331
11
12
36(*) Dataset from which the best WMC prediction models were derived

58

59
CBO
150

100

50

0
BNA−CODBNA−UML
BNB−CODCRS−UML
ECS−CODECS−UML
MYL−P01
MYL−P02
MYL−P03
MYL−P04
MYL−P05
MYL−P06
MYL−P07

CBO (Raw Measures)

Figure 5.2: Boxplot for CBO data

BNB−UMLCRS−CODMYL−P08
MYL−P09
MYL−P10
MYL−P11
MYL−P12
MYL−P135.1. Comparing CK Data Distributions using Boxplots

JAIST

60
RFC
400

300

200

100

0
BNA−CODBNA−UML
BNB−CODCRS−UML
ECS−CODECS−UML
MYL−P01
MYL−P02
MYL−P03
MYL−P04
MYL−P05
MYL−P06
MYL−P07

RFC (Raw Measures)

Figure 5.3: Boxplot for RFC data

BNB−UMLCRS−CODMYL−P08
MYL−P09
MYL−P10
MYL−P11
MYL−P12
MYL−P135.1. Comparing CK Data Distributions using Boxplots

JAIST

61
WMC
80

60

40

20

0
BNA−CODBNA−UMLBNB−UMLCRS−CODCRS−UML
ECS−CODECS−UML
MYL−P01
MYL−P02
MYL−P03
MYL−P04
MYL−P05
MYL−P06
MYL−P07

WMC (Raw Measures)

Figure 5.4: Boxplot for WMC data

BNB−CODMYL−P08
MYL−P09
MYL−P10
MYL−P11
MYL−P12
MYL−P135.1. Comparing CK Data Distributions using Boxplots

JAIST

5.2. Data Transformation and Normalization

JAIST

Figure 5.5: CBO Level vs Spread

Figure 5.6: RFC Level vs Spread

5.2

Data Transformation and Normalization

In this Section we study the application of two data data transformations to normalize
our data, these are logarithmic data transformations and linear scaling to unit variance.

625.2. Data Transformation and Normalization

JAIST

Figure 5.7: WMC Level vs Spread
Data transformations in general enable us to re-express data in a new scale that offers
several improvements over the original scale of data collection [25]. The reasons we haveto apply the different two chosen data transformations are the following:• To promote stable spread across several datasets. It is said that when there exists
a strong relationship between spread and level (as it was observed in our previous
data explorations), power data transformations can be applied to reduce or eliminate
such dependency.
A particular case of this type of transformations is logarithmic transformations.
Transformed data will be better suited for comparison, improving normality andsymmetry of the distributions, and decreasing number of outliers, which are foundto be detrimental for statistical techniques such as logistic regression.
• To acquire a common scale among batches. Having a common scale among ourdatasets would ease data comparison and let us acquire better fault predictions
when using the same prediction model across packages and projects, using either
UML or code metrics.
A transformation that can change the scale of a dataset to a range [0, 1] is the socalled linear scaling to unit variance, which is a linear transformation. Linear transformations are often seen as more ”transparent” than logarithmic transformations
because their interpretation is easier. They only change origin and scale of theoriginal data, conserving the shape of the data distribution (different to logarithmictransformations).
Since by transforming data, we would achieve equality of data spread and/or a common scale among dataset, using either logarithmic transformations or linear scaling tounit variance, we expect that the errors of our UML approximations decrease, further
evaluation is performed in a later Section.
63

5.2. Data Transformation and Normalization

5.2.1

JAIST

Logarithm Data Transformations

Power transformations are defined as the transformation that replaces x by xp . If p = 0
then log(x) is used instead. Power transformations can help us to promote stable spreadamong our different datasets, so that data can be better suited both for visual explorationand for common analytic techniques of batch comparison.
The procedure for diagnosing the power p that can stabilize spread is explained in thefollowing lines. Based on our previous observations in Section 5.1, we can say that the
fourth-spread fd of our datasets is proportional to a power p of the median M, then we
have that:dF= cM p ;log(dF ) = log(c) + blog(M);log(dF ) = k + blog(M);(5.2)

Where, dF is the fourth-spread, M is the median and k = log(c). Thus, the logarithmsof the fourth-spreads and the logarithms of the medians are linearly related. If we use
our CBO data and plot the logarithms of the fourth-spreads versus the logarithms of themedians of the different batches, we have the resulting plot shown on Figure 5.8, whichshows a tendency for log(dF ) to increase as log(M) increases, the relationship appears tobe somehow linear, of the kind of Equation 5.2.

Figure 5.8: Spread-versus-level Plot for CBO data
Using linear regression, for our CBO spread-versus-level plot, we have that the best
linear function that models the different data points is described by Equation 5.3:
f (x) = 0.6x + 0.5(5.3)

Finally, knowing the value of the slope of the spread-versus-level plot, b, we can easily
approximate p using Equation 5.4:
64

5.2. Data Transformation and Normalization

p= 1−b

JAIST

(5.4)

Therefore, for our CBO data the more suitable value of p would be 0.4. Applying thesame methodology for our RFC and WMC data, we obtained p values almost equal tozero, −0.09 and 0.05 for RFC and WMC respectively. The spread-versus-level plots for
our RFC and WMC data are shown in Figures 5.9 and 5.10 respectively1 .

Figure 5.9: Spread-versus-level Plot for RFC data
At this time we chose to work with p = 0 to transformed all of our data. When using
power transformations with p = 0, we might face zero and infinitive values. For example
if we have a class with CBO = 0 then if we use the power transformation p = 0 thenwe will have CBO ′ = log(CBO) = log(0), which is an undefined value. A simple andcommon solution is transforming as follows: CBO ′ = log(CBO + 1) [39].
In general, CBO, RFC and WMC measures were subjected to the transformation ofequation 5.5.
x′ = log(x + 1)

(5.5)

Where:- x′ is the new normalized value.
- x is the raw value of the metric.
After applying this transformation to our code and UML data of our JAIST projects,
we compare again our UML to their corresponding code measures using simple line graphs,
as we did in our first evaluation. Figures 5.11, 5.12, 5.13 show our RFC, CBO and WMC
measures respectively; the line with squares corresponds to the UML measures, and the1

These results present a slight change to those ones published in [11] due to the inclusion of BNB data65

5.2. Data Transformation and Normalization

JAIST

Figure 5.10: Spread-versus-level Plot for WMC dataline with circles corresponds to the code measures. Although it cannot be observed
clearly, the gap between UML and code measures has decreased, in comparison with theone obtained in our first evaluation (in Chapter 4). The average relative errors of ourapproximations decreased to: 0.55, 0.35 and 0.38 for RFC, CBO and WMC respectively(considering average values of our 4 JAIST projects). For comparison with our previous
results, please refer to Table 5.5, where the subscript LOG refers to our results obtainedusing log data transformations.
Figures 5.14, 5.15 and 5.16 show the new boxplots for the RFC, CBO and WMC
measure of our different projects and packages under study. As expected, the applicationof logarithmic transformations resulted in fewer outliers, symmetry improvement anddata spread somehow more stable. Moreover, since logarithmic transformations dampens
exponential growth, it reduced our original scale across datasets, ranging from 0 to 2.5.

5.2.2

Linear Scaling to Unit VarianceLinear Scaling refers to transform a dataset to a new scale using a transformation of theform of Equation 5.6:x′ = a + bx

(5.6)

, where:- x is the original or raw measure,- x′ is the transformed measure,- b is the multiplicative component of the linear transformation, sometimes called theslope, and- a is the additive component, sometimes refers as the intercept.

66Normalized Measures (LOG): RFC vs RFC

RFC_LOG
URFC_LOG

5.2. Data Transformation and Normalization

2.2

2.0

1.8

1.6

1.4

1.2

1.0

0.8

0.6

0.4

BNA−01
BNA−02
BNA−11
BNA−09BNA−08BNA−03
BNA−10
BNA−04
BNA−07
BNA−06
BNA−05
BNB−04
BNB−02
BNB−05
BNB−07
BNB−03
BNB−10
BNB−08
BNB−09
BNB−11
BNB−06
CRS−01
CRS−04
CRS−07
CRS−12
CRS−13
CRS−02
CRS−05
CRS−08
CRS−09
CRS−11
CRS−14
CRS−15
CRS−06
CRS−10
CRS−16
CRS−03
ECS−01
ECS−11
ECS−07
ECS−02
ECS−13
ECS−05
ECS−09ECS−10
ECS−04
ECS−060.2

0.0

Normalized Measures (LOG): CBO vs CBO

CBO_LOG
UCBO_LOG

Figure 5.11: RFC measures using Logarithm Transformation

1.4

1.2

1.0

0.8

0.6

0.4

0.2

0.0

Figure 5.12: CBO measures using Logarithm Transformation67

BNA−04
BNA−05
BNA−09BNA−10
BNA−07
BNA−06
BNB−02
BNB−07
BNB−05
BNB−09
BNB−10
BNB−06
CRS−01
CRS−05
CRS−07
CRS−08
CRS−09
CRS−11
CRS−13
CRS−04
CRS−10
CRS−06
CRS−02
CRS−14
CRS−15
CRS−16
CRS−03
ECS−10
ECS−13
ECS−04
ECS−05
ECS−09ECS−06JAIST

MYL−P10
MYL−P11
MYL−P12
MYL−P13

WMC_LOG
UWMC_LOG

MYL−P095.2. Data Transformation and Normalization

MYL−P08

Normalized Measures (LOG): WMC vs WMC

MYL−P07

2.0

MYL−P06

1.8

MYL−P05

1.6

MYL−P04

1.4

MYL−P03

1.2

MYL−P02

LOG(RFC+1)

MYL−P01

1.0

ECS−UML

0.8

CRS−UML
ECS−COD

0.6

BNB−UMLCRS−COD

0.4

BNB−CODBNA−11
BNA−08BNA−03
BNA−09BNA−07
BNA−10
BNA−04
BNA−01
BNA−02
BNA−06
BNA−05
BNB−02
BNB−04
BNB−03
BNB−07
BNB−08
BNB−09
BNB−10
BNB−11
BNB−05
BNB−06
CRS−04
CRS−07
CRS−01
CRS−02
CRS−06
CRS−12
CRS−13
CRS−14
CRS−15
CRS−11
CRS−09
CRS−05
CRS−08
CRS−16
CRS−10
CRS−03
ECS−06
ECS−09ECS−01
ECS−02ECS−07
ECS−11
ECS−13
ECS−05
ECS−04
ECS−100.2

BNA−UML

0.0

BNA−COD

Figure 5.13: WMC measures using Logarithm Transformation

2.5

2.0

1.5

1.0

0.5

0.0

68

Figure 5.14: RFC Boxplots using Logarithmic Transformation

RFC

JAIST

MYL−P03
MYL−P04
MYL−P05
MYL−P06
MYL−P07
MYL−P08
MYL−P09
MYL−P10
MYL−P11
MYL−P12
MYL−P13

BNA−UML
BNB−CODBNB−UMLCRS−CODCRS−UML
ECS−CODECS−UML
MYL−P01
MYL−P02
MYL−P03
MYL−P04
MYL−P05

LOG(CBO+1)

MYL−P02

LOG(WMC+1)

MYL−P01

BNA−COD

5.2. Data Transformation and Normalization

ECS−UML

2.0

CRS−UML
ECS−COD

1.5

BNB−UMLCRS−COD

1.0

BNB−COD

0.5

BNA−UMLCBO
0.0

BNA−COD

Figure 5.15: CBO Boxplots using Logarithmic Transformation

1.5

1.0

0.5

0.0

69

Figure 5.16: WMC Boxplots using Logarithmic Transformation

WMC

MYL−P06
MYL−P07
MYL−P08
MYL−P09
MYL−P10
MYL−P11
MYL−P12
MYL−P13

JAIST

5.2. Data Transformation and Normalization

JAIST

When using linear scaling transformations the distributions of the transformed andoriginal data are similar. For example if the distribution was positively skewed before thetransformation, the distribution of the transformed data will remain the same. The median x̄ in the original scale, changes according to Equation 5.7 and the standard deviations would be affected only by the multiplicative component, according to Equation 5.8.
x̄′ = a + bx̄

(5.7)

s′x = bsx

(5.8)

Linear scaling to unit variance, as defined in [2], is a transformation based on the
widely used standard score or z-score transformation. The z-score transformation is alinear transformation with a transformed mean equals 0 and standard deviation equals 1.
The z-score transformation defined by Equation 5.9 can be easily derived, if we substitutex̄′ = 0 and s′x = 1 in Equations 5.7 and 5.8 respectively, so that the values of a and b can
be found and then used in Equation 5.6.
x′ =x − x̄
sx(5.9)

Figure 5.17: Normal distribution with media zero and standard deviation 1
Linear scaling to unit variance makes an additional shift and rescaling so that datalies into the range [0,1]. Assuming that the distribution of x is normal, the distribution ofx′ in Equation 5.9 would take the form of the curve shown in Figure 5.17. To guaranteethat 99% of the data under study lies in the range [0,1] an additional shift and rescaling
are made, taking the form of Equation 5.10. Values outside this range can be truncatedeither to 0 or 1.

70

5.2. Data Transformation and Normalization

′

x =

JAIST

x−x̄
+1
3s

(5.10)

2

Where:- x′ is the new normalized value,- x is the raw value of the metric,- x̄ is the mean of the distribution of x,- s the standard deviation of the distribution of x.
We applied linear scaling to unit variance to our raw data, and compared again our
UML metrics to their respective code metrics. Figures 5.18, 5.19 and 5.20 show the UML
RFC, CBO and WMC measures respectively; the line with squares corresponds to the
UML measures, and the line with circles corresponds to the code measures. We observethat in comparison with our previous results (applying logarithmic transformations), thegap between UML and code measures was successfully decreased for all of the threeprojects, reducing the relative errors of our approximations by 50% and standard deviations also decreased. The average relative errors for RFC, CBO and WMC are 0.19, 0.17
and 0.29 respectively (considering average values of the four different JAIST projects).
For comparison with our previous results, please refer to Table 5.5. The subscript LS
denotes the results obtained using Linear Scaling.
The data distribution of the new normalized datasets can be observed in the boxplots
in Figures 5.21, 5.22 and 5.23, as it is observed and was pointed out earlier, linear scaling
to unit variance does not change the original distribution of the data.
In conclusion by transforming our UML and code measures to a common scale [1, 0]
using linear scaling to unit variance, we greatly improved the accuracy and precision of
our UML approximations.
At this stage, we still do not know which is the effect of data normalization when
predicting faulty classes across packages and projects, but we expect better prediction
results using normalization rather than simply raw measures, this is a topic of the following
Chapter.
Table 5.5: Average Relative Errors Information
Statistic
AverageSD
Statistic
AverageSD
Statistic
Average
SD

URF C
0.7
0.17
UCBO
0.53
0.36
UW MC
0.65
0.4

URF CLOG
0.55
0.17
UCBOLOG
0.35
0.32
UW MCLOG
0.38
0.2

71

URF CLS
0.19
0.14
UCBOLS
0.17
0.11
UW MCLS
0.29
0.28

5.2. Data Transformation and Normalization

Figure 5.18: RFC measures using Linear scaling to unit variance

Figure 5.19: CBO measures using Linear scaling to unit variance
72

JAIST

5.2. Data Transformation and Normalization

Figure 5.20: WMC measures using Linear scaling to unit variance

Figure 5.21: RFC Boxplots using Linear Scaling to Unit Variance

73

JAIST

5.2. Data Transformation and Normalization

Figure 5.22: CBO Boxplots using Linear Scaling to Unit Variance

Figure 5.23: WMC Boxplots using Linear Scaling to Unit Variance

74

JAIST

Chapter 6
Fault Prediction

This Chapter has as a main purpose to evaluate our UML CBO, RFC and WMC
metrics as predictors of faulty code. The methodology used for this purpose is explained
as follows, please see Figure 6.1. We first built three code-based prediction models using
univariate logistic regression and linear scaling to unit variance to normalize data. For theconstruction of these models, the code of the MYL system, introduced in Chapter 5, wasused (Figure 6.1-A). Then, the CK UML measures collected from our JAIST projects, as
described in Chapter 4, were used as inputs of our prediction models (Figure 6.1-B). After
that, the code CK measures from three of the JAIST projects (ECS, CRS and BNS1 ) werealso used as inputs of the same prediction models (Figure 6.1-C). Finally, the prediction
results from our two previous steps were compared to evaluate the prediction ability of
our UML CK metrics (Figure 6.1-D).
A different approach could have been undertaken by building UML-based prediction
models, without the need of using code-based prediction models; however, the latter approach was adopted because our initial intentions were to include in a sole prediction
model more than one UML metric as predictors of faulty code (multivariate analysis).
For such a purpose the number of data observations required is, as a rule of thumb, 10 atleast per independent variable. This is because for every independent variable regressor
coefficients need to be estimated to build the multivariate prediction model, and unless
we provide enough data observations, such parameters would be poorly estimated. Therefore, building a multivariate prediction model, using our JAIST projects, from which UML
metrics are available, would not be proper since the number of classes per project is notlarger than 16.
Moreover, this approach also allowed us to perform a better analysis of the application of the same prediction model across different projects and packages (JAIST and MYL
projects) using raw data and normalized data.
1

In this Chapter, the code implementation used of BNS was that one referred as BNA in previous
Chapters75

6.1. Construction of Univariate Logistic Models

JAIST

111111111111111111111111111111111111111111
000000000000000000000000000000000000000000
000000000000000000000000000000000000000000
111111111111111111111111111111111111111111
PREDICTION MODELS CONSTRUCTION
using the MYL system
000000000000000000000000000000000000000000
111111111111111111111111111111111111111111
000000000000000000000000000000000000000000
111111111111111111111111111111111111111111

Implementation
Time

CODE−BASED
Prediction
Model

CK metrics<Code>

Most Faulty
Classes

11
00
00
11
A
00
11
00
11

111111111111111111111111111111111111111111111
000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000
111111111111111111111111111111111111111111111
UML METRICS VALIDATION
using JAIST projects
000000000000000000000000000000000000000000000
111111111111111111111111111111111111111111111
000000000000000000000000000000000000000000000
111111111111111111111111111111111111111111111
C

Design Time

11
00
00
11
B
00
11
00
11

CK metrics

CODE−BASED<UML>

Prediction
Model

11
00
00
11
D
00
11
00
11
Most Faulty
ClassesComparison&
Evaluation

ImplementationTime

CK metrics<Code>

1111
0000

CODE−BASED
Prediction
Model0000 Most Faulty
1111
Classes

11
00
00
11
C
00
11
00
11

Figure 6.1: Methodology to Evaluate our UML metrics as Predictors of Faulty Code

6.1

Construction of Univariate Logistic Models

To construct our prediction models, we chose to use logistic regression due to the following
reasons:
1. Logistic regression does not strictly require normally distributed variables as otherstatistical techniques such as linear regression and discriminant analysis.
2. Logistic regression parameters and the relationships among them are easier to interpret in contrast with other techniques such as Artificial Neural Networks (ANNs),
which also have been used for fault-prediction. Although, ANN are non linear anddo not make any assumption about the population distributions of the variables
used, they are often referred as black boxes because their derived parameters andcoefficients cannot be easily analyzed or interpret.
3. To construct prediction models, do not require large amount of data in comparison
with the amount of data ANNs usually require.
We constructed univariate logistic models of the kind of Equation 6.1 to predict Most
Faulty (MF) classes. In this equation x is the independent variable and P (MF ) is theprobability of having a Most Faulty class. A is the intercept value and B is the regression
coefficient of x. Figure 6.2 shows the resulting graph of this model, the horizontal axis
represents the independent variable x and the vertical axis P (MF ).

76

6.1. Construction of Univariate Logistic Models

JAIST

P(MF)

x

Figure 6.2: Univariate Logistic Model

P (MF ) =6.1.1

1
1 + e−(A+Bx)(6.1)

Independent Variables

For the construction of the models, we used 13 packages of the first-released version of
the Mylyn software (MYL), introduced in a previous Chapter. From every package of the
MYL, we obtained three univariate logistic (UL) models using CK-RFC, CK-CBO andCK-WMC as their independent variables.

6.1.2

Dependent Variable

The Mylyn web site provides well organized records of the number of faults (or bugs) foundin every released version of the system. Such records are registered in a bugzilla database2 .
This information and the information obtained from parsed CVS repository logs per classwere used to assess the number of faults per class for the first release version of the MYL.
Since logistic regression requires a dichotomous variable as dependent variable, we choseto classify our classes as Most Faulty (MF) and Least Faulty (LF). For the constructionof the UL models, classes with the number of post-release faults greater than or equal to
the second quartile of the dataset were classified as MF, and as LF otherwise.
The distribution of the number of faults per class of the different JAIST projects and
MYL packages used to construct and test our models (Section 6.2.4) are represented by
the boxplots shown in Figures 6.3 and 6.4 for the CBO models, Figures 6.5 and 6.6 forthe RFC models, and Figures 6.7 and 6.8 for the WMC models. Most of the fault data is
positively skewed, having most of the classes relatively low number of faults.
More information about the distribution of the number of faults per class of every
dataset can be found in Tables 6.1, 6.2 and 6.3, where Q1 refers to the value of thefirst quartile, Q2 to the second quartile, Q3 to the third quartile and Maximum to the
maximum number of faults found in a class of a given dataset. Notice that the distribution
2

Bugzilla is the leading open-source/free software bug tracking system77

6.1. Construction of Univariate Logistic Models

JAIST

of faults is slightly different among the data used for the construction of the different ULmodels, which is result of the elimination of outliers of CBO, RFC and WMC data (thisprocedure is further explained in the following section).
FAULT Data Distributions (CBO)15

FAULTS

105

ECS−UML

ECS−CODCRS−UMLCRS−CODBNS−UMLBNS−COD

0

Figure 6.3: Boxplots of Fault Data of JAIST Projects (for CBO Models)

6.1.3

Procedure for Model Construction

We construct several UL models to predict each of our independent variables RFC, WMC
and CBO according to the following procedure. For every package of the MYL system:
1. We created a dataset.
2. We detected and eliminated outlier data points (values equal to or greater than thethird quartile of the entire dataset) to improve normality. Outlier data points of the
CK metrics are per se focus of attention indicating high complexity and/or possible
design violations. Such data points, as recommended in [14], should be subjected
to further analysis.
3. We normalized the dataset using Linear Scaling to Unit Variance.
4. We built 3 UL models for predicting RFC, CBO and WMC independently.
Then, from the different 13 UL CBO, RFC and WMC models we chose the best
resulting models. The best CBO model was built with the data of the Mylyn package78

6.1. Construction of Univariate Logistic Models

JAIST

FAULT Data Distributions (CBO)30

25

FAULTS

20

15

10

5

MYL−P13

MYL−P12

MYL−P11

MYL−P10

MYL−P09

MYL−P08

MYL−P07

MYL−P06

MYL−P05

MYL−P04

MYL−P03

MYL−P02

MYL−P01

0

Figure 6.4: Boxplots of Fault Data of Mylyn Packages (for CBO Models)
FAULT Data Distributions (RFC)8

FAULTS

6

4

2

ECS−UML

ECS−CODCRS−UMLCRS−CODBNS−UMLBNS−COD

0

Figure 6.5: Boxplots of Fault Data of JAIST Projects (for RFC Models)79

6.1. Construction of Univariate Logistic Models

JAIST

FAULT Data Distributions (RFC)25

FAULTS

20

15

10

5

MYL−P13

MYL−P12

MYL−P11

MYL−P10

MYL−P09

MYL−P08

MYL−P07

MYL−P06

MYL−P05

MYL−P04

MYL−P03

MYL−P02

MYL−P01

0

Figure 6.6: Boxplots of Fault Data of Mylyn Packages (for RFC Models)
FAULT Data Distributions (WMC)

15

FAULTS

105

ECS−UML

ECS−CODCRS−UMLCRS−CODBNS−UMLBNS−COD

0

Figure 6.7: Boxplots of Fault Data of JAIST Projects (for WMC Models)80

6.2. Validation of UL models

JAIST

FAULT Data Distributions (WMC)
40

FAULTS

30

20

10

MYL−P13

MYL−P12

MYL−P11

MYL−P10

MYL−P09

MYL−P08

MYL−P07

MYL−P06

MYL−P05

MYL−P04

MYL−P03

MYL−P02

MYL−P01

0

Figure 6.8: Boxplots of Fault Data of Mylyn Packages (for WMC Models)labeled as MYL-P11 in Table 5.2, as for the best RFC and WMC models, packages MYLP12 and MYL-P02 in Tables 5.3 and 5.4, respectively, were used.
Table 6.4 shows the coefficients of the three best UL models found, of the kind of
Equation 6.1. The x column contains the independent variable of the model, the A
column refers to the intercept values and the B column to the regression coefficients of
the independent variable. While coefficients on the left side of columns A and B are thosefor the models using data transformations, the coefficients on the right are for the models
using the raw data.

6.2

Validation of UL models

In the next sections we give the details of the construction of every univariate logistic
model, using SAS3 . After estimating the regressor coefficients of our logistic models of
the kind of Equation 6.1, there are several steps necessary to assess the appropriateness,
adequacy and usefulness of our models [6]. First, the importance of each of the variables
is assessed by carrying out statistical tests of significance. Second, the overall goodness of
fit of the model is tested. Third, the ability of every model to discriminate between Most
Faulty and Least Faulty classes is tested. And finally, the model using different data isvalidated.
3

Statistical Analysis Software816.2. Validation of UL models

JAIST

Table 6.1: Detailed Information of Fault Data Distribution for CBO Models
Dataset

Total
Total
Q1
Q2
Q3
M aximumClassesFaults
BNS-COD
9
11
0
0
2
5
BNS-UML
10
15
0
0.5
2.75
5
CRS-COD
16
57
1
2.5
4.5
16
CRS-UML
14
38
1
2
4
8
ECS-COD
10
9
0
1
1.75
2
ECS-UML
10
9
0
1
1.75
2
MYL-P01
38
69
1
1
2
7
MYL-P02
25
118
1
2
7
17
MYL-P03
51
435
3
5
12
33
MYL-P04
50
119
1
2
3
9
MYL-P05
23
76
2
3
3
14
MYL-P06
47
153
1
3
4
15
MYL-P07
21
52
1
2
3
9
MYL-P08
10
22
1.25
2
2
5
MYL-P09
183
657
2
3
5
14
MYL-P10
21
52
1
2
3
10
MYL-P11*
48
199
1
2
6
19
MYL-P12
16
20
1
1
1.25
2
MYL-P13
27
125
1
1
3
24
(*) Dataset used for the construction of the best CBO prediction models

6.2.1

Importance of each independent variable

To determine the importance of every independent variable of our models, we use the-2LogL statistic, which has a χ2 (chi square) distribution with 1 degree of freedom (df ),which can be explained as follows.
For univariate logistic models, SAS provides us with two values of the -2LogL statistics, where L is the likelihood of a model, defined as the probability that the estimatedhypothesized model represents the input data [49]. One of these values is for the complete
model (intercept and independent variable) and the second is for a second model with only
the intercept. The difference of these two values is what determines the importance of the
independent variable in the model. The degrees of freedom used are also the difference
between the two models, 1 for our case.
For this test a significance level of α = 0.05 was chosen. A significance level is definedas the probability of making a decision to REJECT the null hypothesis of a test. If a testof significance gives a p-value lower than the α-level, the null hypothesis is rejected. The
null hypothesis of this test is: The inclusion of the independent variable does not improvethe model fit. Therefore, for a chosen α = 0.05 a −2LogL > χ21,0.05 = 3.8 is required.
The yielded -2LogL values of our models suggest that the three variables are signifi82

6.2. Validation of UL models

JAIST

Table 6.2: Detailed Information of Fault Data Distribution for RFC Models
Dataset

Total
Total
Q1
Q2
Q3
M aximumClassesFaults
BNS-COD
6
2
0
0
0.75
1
BNS-UML
6
2
0
0
0.75
1
CRS-COD
14
35
1
2
3.75
8
CRS-UML
14
35
1
2
3.75
8
ECS-COD
10
9
0
1
1.75
2
ECS-UML
10
9
0
1
1.75
2
MYL-P01
32
61
1
1
2.25
7
MYL-P02
24
138
1.75
3.5
10.25 17
MYL-P03
43
334
3
5
11
27
MYL-P04
49
117
1
2
3
9
MYL-P05
23
76
2
3
3
14
MYL-P06
46
142
1
2.5
4
15
MYL-P07
15
30
1
2
2
5
MYL-P08
10
22
1.25
2
2
5
MYL-P09
174
613
1
3
5
14
MYL-P10
21
52
1
2
3
10
MYL-P11
42
141
1
2
4
19
MYL-P12*
16
20
1
1
1.25
2
MYL-P13
25
96
1
1
2
22
(*) Dataset used for the construction of the best RFC models

cantly important for its corresponding model to fit the data, using transformed and raw
values. These values are shown in Table 6.5.

6.2.2

Overall goodness of fit

To test the overall goodness of fit, we use the Hosmer-Lemeshow (HL) test. This statistic
has a χ2 distribution. Large values of HL or small values of the probability P obtained
from it indicate that the fit of the overall goodness of fit of the model may not be good.
Moreover, it is important to take into consideration that when less than 4 df are used,then the statistic is likely to indicate that the model under testing fits the data [1].
Therefore, considering an α = 0.05, we obtain the result that all of our models passthis test. The values of this statistic of our models can be found in Table 6.5; values on
the left side of the column correspond to the models using data transformations, while
those in the right to the models using raw data.

836.2. Validation of UL models

JAIST

Table 6.3: Detailed Information of Fault Data Distribution for WMC Models
Dataset

Total
Total
Q1
Q2
Q3
M aximumClassesFaults
BNS-COD
9
11
0
1
2
4
BNS-UML
9
11
0
1
2
4
CRS-COD
16
57
1
2.5
4.5
16
CRS-UML
16
57
1
2.5
4.5
16
ECS-COD
11
10
0
1
1.5
2
ECS-UML
11
10
0
1
1.5
2
MYL-P01
37
67
1
1
2
7
MYL-P02*
29
163
1
3
10
17
MYL-P03
52
493
3
5.5
13
40
MYL-P04
48
115
1
2
3
9
MYL-P05
23
76
2
3
3
14
MYL-P06
39
92
1
1
3.5
9
MYL-P07
24
72
1.75
2
3
9
MYL-P08
10
22
1.25
2
2
5
MYL-P09
163
550
1
3
4
14
MYL-P10
20
45
1
1.5
2.25
10
MYL-P11
41
156
1
2
6
19
MYL-P12
14
16
1
1
1
2
MYL-P13
24
97
1
1
2.25
22
(*) Dataset used for the construction of the best WMC models

6.2.3

Discrimination

In this section, we measure the ability of the models to discriminate between the twogroups defined by the response. In other words how good our models classify a class as
Most Faulty or Least Faulty. For this step, the Hubert’s statistical procedure can beused [49].
The Hubert’s statistical procedure requires the following statistics are required:• O is the number of correct classification.
• e is the expected number of correct classifications due to chance.
• Z statistic follows a normal distribution, and if its value is significant at an α = 0.05,
suggests that the number of correct classifications is significantly greater than thoseobtained due to chance.
For all our models the yielded values of O are greater than those of e, which indicatesa good discrimination between the MF and the LF classes. Also, the yielded Z values
suggest that the number of correct classifications is significantly greater than that obtained84

6.2. Validation of UL models

JAIST

Table 6.4: Univariate Logistic Regression Models
x
RFC
WMC
CBO

A
-7.75 | -4.83-6.11 | -2.59-12.76 | -1.86

B
12.05 | 0.2
13.39 | 0.17
30.72 | 0.46

Table 6.5: Univariate Logistic Regression Models Validation
x
RFC
WMC
CBO

-2Logl7.06 | 7.17
14.68 | 14.93
33.39 | 33.91

HLχ26,0.12 = 10.31 | χ26,0.12 = 10.16
χ27,0.83 = 3.56 | χ27,0.82 = 3.59
χ25,0.12 = 8.581 | χ25,0.13 = 8.5

by chance. The values of these statistics can be found in Table 6.6; values on the left sideof the column correspond to the models using data transformations, while those in theright to the models using the raw data.
For the three models, the cutoff probability to discriminate between the MF and LF
classes is 0.5.

6.2.4

Validation using different data

Finally, we present the results obtained from the most important test of our empirical
study, concerning fault prediction, which is the validation of our models using different
data to those used for the construction of our models.
We test our models with different data from different packages of the same MYL
system and from the 3 other small-size software projects used in previous Chapter, usingcode and UML metrics. Such a test has two purposes: Firstly, to determine whether
Table 6.6: Discrimination Test
Group

MF

LF

Both

x
CBO
RFC
WMC
CBO
RFC
WMC
CBO
RFC
WMC

e
16.33
1
8.83
8.33
9
5.83
24.67
10
14.66

85

O
23 | 23
3|3
13 | 13
19 | 18
11 | 12
9 | 1042 | 41
14 | 15
22 | 23

Z
2.56 | 2.56
2.31 | 2.31
2.1 | 2.1
4.84 | 4.38
1.33 | 2
1.77 | 2.33
24.67 | 24.67
2.07 | 2.58
2.73 | 3.1

6.2. Validation of UL models

JAIST

the elimination of outliers and normalization procedure, described in 6.1, can improve
the results obtained using the same prediction model with other datasets (packages andprojects); secondly, to determine whether this same procedure can be used suing UML
metrics instead, so that we can predict before coding.
Based on the yielded prediction results, different threshold values were chosen to classify between MF and LF classes of the datasets for testing our models:• For testing the CBO models, in most cases the second quartile was chosen. In someother few cases, the third quartile resulted to be a better option for classificationthreshold (BNS-COD, BNS-UML, MYL-P01, MYL-12 and MYL-13). Please refer
to Table 6.1 for detailed information.
• For testing the RFC models, classes with number of faults equal or greater than the
maximum number of faults of a class within a dataset divided by two were classified
as MF and LF otherwise. So that, for example, looking at Table 6.2, for testing the
RFC model with the dataset ECS-COD, the threshold value for classification equals
1 (Maximum/2 = 2/2).
• For testing the WMC models, the same threshold value used for testing our RFC
models was chosen, except for testing with the MYL-P06 dataset, for which thesecond quartile was used. Please refer to Table 6.3 for detailed information.
Validation
After the application of our models in the different datasets, for the sake of simplicity,
we used three indicators to assess how well our models can classify or discriminate between
MF and LF classes, these indicators were:• Correctness: Number of classes (LF and MF) correctly classified.
• Specificity: Number of LF classes correctly classified.
• Sensitivity: Number of MF classes correctly classified.
Tables 6.11, 6.12 and 6.13 show the percentages of Correctness, Specificity and Sensitivity obtained from the application of our models in the different datasets using only
code measures. NRFC, NCBO and NWMC refer to the results obtained with the models
using normalized values, while RFC, CBO and WMC are the percentages obtained withthe models using the raw data.
Furthermore, in Table 6.7, we show the results obtained with our NRFC, NCBO andNWMC models in the projects: ECS, CRS and BNS, using not only their code metrics,
but also their UML metrics.

866.3. Multivariate Analysis and Correlation

JAIST

Table 6.7: Normalized code measures vs UML measures
Model

Project

ECS
NRFC CRSBNSECSNCBO CRSBNSECSNWMC CRS
BNS

6.2.5

Correctness
Code UML
80%
80%
57%
64%
33%
67%
80%
80%
88%
71%
89%
80%
55%
45%
56%
38%
78%
44%

Specificity
Code UML
100% 100%
80%
80%
50%
75%
75%
100%
88%
33%
83%
67%
25%
25%
57%
36%
67%
33%

Sensitivity
Code UML
67%
67%
0%
25%
0%
50%
83%
67%
88%
100%
100% 100%
71%
57%
50%
50%
100% 67%

Observations

From the results obtained in the previous Section, the following observation can be made:Effect of Normalization on code measures
To summarize, comparing the 16 datasets (packages and projects) using only code
measures, we can say that the normalization procedure greatly improves the results of the
RFC model, it benefits less to the CBO model, and almost no improvement is obtainedwith the WMC model, except for the fact that the NWMC model can discriminate MF
classes better than the WMC model. Note that, in most of the cases presented here, theWMC model yields 0% of Sensitivity and high percentages of Specificity, which meansthat the WMC model is classifying most or all classes as LF.
Normalized UML VS Normalized Code measures
In general, the normalized UML RFC measures yield equal, and in some cases better
results, than the normalized code RFC measures. The normalized UML CBO measures
yield nearly equal results to those obtained with the normalized code CBO measures.
Some of the percentages are better, some others equal and some others worse. The
normalized UML WMC measures did not perform better than the code WMC measures.

6.3

Multivariate Analysis and Correlation

In order to improve the prediction ability of our models, we could use more than one
metric as independent variable in a single prediction model. However, multivariate logistic
regression and linear regression require uncorrelated independent variables, and somerelationship between our dependent variable and the selected independent variables must
exist.
87

6.3. Multivariate Analysis and Correlation

JAIST

Let us say we would like to build a prediction model of the kind of Equation 6.2,
where P (MF ) is the probability of having a ”Most Faulty” class, A is the intercept value,
B, C and D are the regressor coefficients of RFC, CBO and WMC respectively. For such
construction, we would use the data of the package of the MYL system labeled as MYLP11 in Table 5.2, from which the best UL prediction model using CBO was derived. We
chose this package because from this package were also derived one of the best 5 prediction
models using RFC and WMC independently, moreover the size (49 classes) of the dataset
is enough large to calculate the coefficients A, B, C and D.
P (MF ) =1
1 + e−(A+B∗RF C+C∗CBO+D∗W M C)

(6.2)

The first step would be to find out if there exists a possible relationship between ourour independent variables and dependent variable, number of faults per class. A scatterplot can help us to determine the existence of such a relationship, when this relationshipresults to be a linear one, correlation coefficients are of great utility. For our case study,
the correlation coefficients found are shown in Table 6.8. These coefficients suggest theexistence of a strong linear relationship between number of faults and CBO, the same for
RFC and as for WMC suggest a weaker linear relationship.
The second step would be to calculate the correlation coefficients among independent
variables, RFC, CBO and WMC. These are shown in Table 6.9, which is called correlation
matrix. It lists the variable names down the first column and across the first row. The
diagonal of a correlation matrix always consists of ones, because the correlation between
a variable and itself is always perfect (1). To localize the correlation coefficient for any
pair of variables, we should find the value in the table for the row and column intersection
for those two variables. For instance, the correlation between variables RFC and CBO
is 0.9. As we can observe the three variables are highly correlated among them, which
prevents us of using these variables as predictors in the same prediction model.
Table 6.8: Correlation Between Independent and Dependent Variables

FAULTS

RFC
0.7

CBO
0.9

WMC
0.5

Table 6.9: Correlation Among Independent Variables

RFC
CBO
WMC

RFC
1
0.9
0.8

CBO
1
0.7

WMC
1

The previous results conform other studies’ findings, which suggest a high correlation
886.3. Multivariate Analysis and Correlation

JAIST

not only among the CK metrics [14, 52], but also among other sets of design-complexity
metrics, such as the QMOOD and the MOOD metrics [32, 43].
Furthermore, let us say that we wish to investigate if simple UML metrics (not anapproximation of those traditionally measured from the code) have some relationship tofault-proneness, we would face the same correlation problem. Previous studies found that
UML class metrics are also highly correlated [36, 55]. These metrics have been associatedwith the maintenance and understandability of UML class diagrams [19, 21, 36].
A common solution for the problem of correlated independent variables is to use Principal Component Analysis, which is a technique that transforms a set of correlated variables
to a new set of uncorrelated variables, named principal components [28]. Unfortunately,when using these new variables, the resulting new coefficients in the regression model are
difficult to interpret, since the dependent variables are not longer describing purely the
initial metrics, but a combination of these.
How much better a prediction model with correlated independent variables
can do?
For instance, a previous research study [43], found a good multivariate logistic model topredict faulty code across different versions of a software. Such a model was a combination
of: CBO, DIT, LCOM, NOC and WMC-McCabe metrics, which were highly correlated.
The reported results were that the only significant regressor in the model was WMC,
which made the authors wonder whether they should simply have used the univariate
WMC model.
To enhance the understanding of why correlated independent variables cannot be usedin multiple regression, a rapid analysis can be made supposing that we want to build, nota multivariate logistic regression model, but a multiple linear regression model to predict
Number of Faults, NF, using the data of our previous example. For this purpose, we use
shared variance coefficients and the multiple coefficient of correlation, which are explainedin the the following paragraphs.
The shared variance between two variables, X and Y is the amount that the variationsof the two variables tend to vary together. The percentage of shared variance or coefficientof determination is represented by the square of their correlation coefficient, r 2 . Anotherway to visualize this is with a Venn diagram, for example in Figure 6.9, the amount ofshared variance between X and Y variables is represented by their overlap of variation
r2.
The multiple coefficient of correlation, R2 , is a measure of the fit of a multiple linear
regression model. It falls somewhere between zero and one. A value of one indicates thatall data points fall exactly on a line in multidimensional space, and a value of zero indicatesthat there is not any relationship between the independent variables collectively and thedependent variable. When the independent variables are not correlated, the sum of theirshared variance (r 2 ) with the dependent variable is equal to R2 . However, when the
independent variables are correlated, this strategy overestimates the contribution of eachvariable because the variance that they share is counted more than once; and therefore
the sum of r 2 cannot be used.
For our specific case study, the coefficients of determination between NF and every of89

6.3. Multivariate Analysis and Correlation

JAIST

r = 0.00
X2

r = 0.00

Y111111000000000000111111000000111111000000111111000000111111000000111111000000111111000000111111000000111111000000Y
X 111111
000000111111000000111111000000111111000000111111000000111111000000111111

X

11
00
00
11
00
11
00
11
00
11
00
11
00
11
00
11
00
11
00
11
00
11
00
11

r = 0.20

111111111
000000000
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
Y
X
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111
000000000
111111111

r = 0.80
2

r = 0.64

2

Y

r = 0.04

r=1
2

r =1

Figure 6.9: Correlation and Shared Variance
our independent variables are shown in Table 6.10. Based on these results, we can saythat 49% of the variability of RFC can be related to NF of a class. Likewise, 81% and 35%of the variability of CBO and WMC, respectively, can be related to NF independently.
2
Now, since CBO has the highest shared variance with NF (rN
F.CBO = 0.81), we would
chose it first as predictor to build a univariate multiple linear regression model and predict NF, this is represented in Figure 6.10-a. On the other hand, because RFC’s shared
2variance with NF is the second highest (rN
F.RF C = 0.49), represented in Figure 6.10-b,
we could think that the inclusion of RFC as independent variable in our CBO univariate
prediction model could yield better results; however, because CBO and RFC are highly2
correlated to each other (r = 0.9, rCBO.RF
C = 0.81) the shared variance of RFC with NF
is very likely to be included in the shared variance with CBO, Figure 6.10-c), in otherwords, the benefits of including RFC in our regression model would not be significant,22resulting in a multiple correlation coefficient RN
F.CBO,RF C ≈ rN F.CBO = 0.81. From here
originates the need of including uncorrelated independent variables to regression models.
Table 6.10: Coefficients of Determination Among Variables
Dependent
Variable
RFC
CBO
WMC
Faults

RFC

CBO

WMC

Faults

1
0.81
0.64
0.49

1
0.49
0.81

1
0.25

1

Trying to find a prediction model that explains fault-proneness of code using only
design-complexity metrics assumes that fault-proneness of code is generated only due to90

6.3. Multivariate Analysis and Correlation

JAIST

CBO

RFC

2

r

NF.CBO2= 0.81r NF.RFC = 0.49

NF
NF

(a)

(b)

2

r CBO.RFC = 0.81

RFC

CBO

2

R

NF

2

NF. CBO,RFC~~ r CBO.RFC(c)Figure 6.10: Shared Variance and Correlated Independent Variables

916.4. Guidelines for Further Use

JAISTcomplexity in the design. Therefore, there is a need to identify other metrics, representative of other contributors of faulty code, such as the developer’s experience, teamcommunication, etc. We do agree with Fenton’s criticism [17], since the point of view thatfault-proneness of code cannot be only due to the complexity of the design, but also toother factors. All research work done for years has helped us to identify design complexity as one of the major contributors of fault-prone code. Moreover, very good predictors
have been clearly detected from their and our own results. However, they are not goodenough yet, at least, to use the same model across software projects. Therefore, we areencouraged to study other factors, which might be related to fault-proneness of code.

6.4

Guidelines for Further Use
DEFINE the scope ofyour dataset(s)package / project

Start

MEASURE the
UML metrics

Is thereany dataset foranaysis?

FORM dataset(s)YES

ELIMINATE
OutliersNO

NORMALIZE
DatasetAPPLYModels

DETECT
END

MF Classes

Figure 6.11: Flow Chart for Guidelines
In this Section, we provide a set of simple guidelines to apply the models and methodology exposed in this paper, see Figure 6.11:
1. Define the scope of your datasets. The size of a dataset can be either of a package
or of a whole project. The reader must have in mind that the models will detect
the MF classes within the chosen dataset.
2. Measure the UML metrics from the data of interest as detailed in Chapter 3, Section
3.5.
92

6.5. Conclusions

JAIST

3. Form as many datasets as needed. If you decided to work with packages, group your
UML measures in as many datasets as number of packages your project has.
4. For every dataset eliminate outlier data points. Since our UML metrics are approximation of the CK metrics and outlier data points of these metrics are, by themselves,
focus of attention indicating high complexity and/or possible design violations, suchdata points should be subjected to further analysis [14].
5. Normalize the datasets according to Chapter 5, Section 5.2.2.
6. Using the resulting normalized datasets, apply the normalized models proposed in
Section 6.1.
7. The models will yield the probability of having a MF class within the chosen dataset.

6.5

Conclusions

The results of our empirical study lead us to conclude that the proposed UML RFC andCBO metrics can predict faulty code almost with the same accuracy as their respectivecode metrics. The elimination of outliers and the normalization procedure used were significantly efficient, not just for enabling our UML metrics to predict faulty code using acode-based prediction model, but also for improving the prediction results of our models
across different packages and projects, which was found to be difficult, even across packages of the same project. On the other hand, both the UML WMC and the code WMC
metrics showed a poor fault-proneness prediction ability.
We expect that a UML-based prediction model will improve upon the results found in
this study. Our univariate logistic regression analysis yields correct classifications of about
80% when using UML metrics. Yet, this rate is not sufficient, if we consider that it canvary from project to project or from package to package. Including other metrics in themodels might improve these results, therefore we need to look for some other metrics (different to design-complexity metrics) that are easily obtainable before the implementationof the system, with low or zero correlation among them, and that are strongly related to
the fault-proneness of code. Moreover, other preprocessing techniques for datasets, suchas normalization and over/under sampling, can also be helpful. The latter technique is a
preprocessing procedure for balancing datasets with a large difference between the numberof faulty and non-faulty classes, which has been found to cause performance degradationof fault-proneness models [29]. As we can observe in Tables 6.11, 6.12,6.13 and 6.7, thepercentages of specificity and sensitivity, and in consequence correctness, generally were
improved using normalization, nonetheless specificity and sensitivity remain unbalanced
in some cases. We observed that such percentages vary according to the chosen cutoff
probability to discriminate between MF and LF classes, and that a chosen cutoff probability for a specific dataset (package or project) does not always obtain the same results inothers. Likewise, the chosen threshold of number of faults to classify classes as MF or LF
also affects such percentages. In our experiments, for the construction of our prediction
models, apart of using the number of faults greater than or equal to the second quartile of93

6.5. Conclusions

JAIST

a dataset as classification threshold, we also used the third quartile, however the former
always yields better results when predicting across datasets with the same model. Suchphenomenon is another specific topic of further investigation, and perhaps the work previously mentioned on over/under sampling techniques might be a good starting point ofresearch.
Finally, a larger number of empirical studies are needed, mainly using data from theindustry, to ensure that the resulting prediction model is robust to various real-life factors,
possible contributors of fault-prone code, such as team members’ stress and experience.
Table 6.11: RFC: Normalized vs Raw Code Measures
Project
ECS
CRS
BNS
MYL-P01
MYL-P02
MYL-P03
MYL-P04
MYL-P05
MYL-P06
MYL-P07
MYL-P08
MYL-P09
MYL-P10
MYL-P11
MYL-P12
MYL-P13

Correctness
RFC NRFC
80%
80%
71%
57%
67%
33%
63%
75%
63%
75%
72%
72%
51%
67%
48%
61%
61%
85%
62%
80%
60%
80%
75%
81%
76%
90%
74%
79%
87.5% 87.5%
56%
88%Specificity
RFC NRFC
50%
100%
100% 80%
100% 50%
59%
74%
44%
81%
78%
78%
52%
75%
52%
67%
56%
83%
59%
77%
50%
88%
74%
84%
76%
94%
72%
79%
91.7% 91.7%
50%
86%

94

Sensitivity
RFC NRFC
100% 67%
0%
0%
0%
0%
80%
80%
100% 63%
43%
43%
40%
0%
0%
0%
10%
100%
77%
92%
100% 50%
83%
63%
75%
75%
100% 100%
75%
75%
100% 100%

6.5. Conclusions

JAIST

Table 6.12: CBO: Normalized vs Raw Code Measures
Project
ECS
CRS
BNS
MYL-P01
MYL-P02
MYL-P03
MYL-P04
MYL-P05
MYL-P06
MYL-P07
MYL-P08
MYL-P09
MYL-P10
MYL-P11
MYL-P12
MYL-P13

Correctness
CBO NCBO
70%
80%
81%
80%
67%
89%
58%
61%
52%
80%
56%
56%
54%
73%
57%
43%
64%
68%
86%
81%
80%
70%
68%
73%
81%
81%
88%
88%
81%
81%
84%
67%

Specificity
CBO NCBO
100% 75%
100% 88%
100% 88%
64%
48%
89%
67%
56%
56%
31%
63%
0%
33%
35%
57%
83%
100%
33%
33%
40%
57%
80%
80%
95%
95%
78%
78%
26%
58%

Sensitivity
CBO NCBO
50%
83%
63%
88%
63%
88%
46%
85%
31%
88%
50%
50%
90%
90%
93%
50%
92%
79%
87%
73%
100% 86%
91%
86%
82%
82%
82%
82%
83%
83%
100% 88%

Table 6.13: WMC: Normalized vs Raw Code Measures
Project
ECS
CRS
BNS
MYL-P01
MYL-P02
MYL-P03
MYL-P04
MYL-P05
MYL-P06
MYL-P07
MYL-P08
MYL-P09
MYL-P10
MYL-P11
MYL-P12
MYL-P13

Correctness
WMC
NWMC
36%
55%
88%
56%
67%
78%
70%
57%
79%
76%
75%
56%
83%
38%
83%
39%
87%
67%
82%
68%
80%
50%
88%
53%
85%
60%
85%
56%
86%
64%
71%
42%

Specificity
WMC
NWMC
100%
25%
100%
57%
100%
67%
86%
57%
77%
69%
93%
50%
93%
37%
90%
43%
100%
62%
98%
63%
100%
50%
100%
49%
100%
59%
100%
49%
100%
58%
81%
73%

95

Sensitivity
WMC
NWMC
0%
71%
0%
50%
0%
100%
22%
56%
81%
81%
0%
80%
0%
90%
0%
0%
0%
10%
13%
93%
0%
50%
0%
85%
0%
67%
0%
100%
0%
100%
0%
33%

Chapter 7
Related WorkIn this Chapter, we first present some of the research work has been done on how to
obtain the CK metrics from UML diagrams. Then, we introduce some related work on
fault prediction taking place after the implementation of the code (using code metrics)
and before (using elements from the design of the software).

7.1

CK and UML

The few works, that have tried to approximate the CK metrics out of UML diagrams, focus
mainly on UML class diagrams. Their observations are in general that the calculationof three of the six metrics from UML class diagrams is not straightforward. These three
metrics are the CBO, RFC and LCOM. Moreover, their methods either lack accuracy (interms of proximity to the measures obtained directly from the source code) or need very
well detailed design information.
In [15], Twan van Enckevort tried to verify modifications to UML models using quality
metrics, among them some CK metrics measured from UML class diagrams. He indicatesthat RFC requires behavioral diagrams, LCOM cannot be measured from UML class
diagrams and that CBO is difficult to calculate, but it can be estimated by calculating
the sum of all class dependencies.
Baroni et al. have established a formal definition of the CK metrics using UML class
diagrams [4]. However, CBO, RFC and LCOM CK metrics, measured from UML class
diagrams, do not capture all the information, which normally the code provides.
Tang et al. [53] used a combination of UML class, collaboration and activity diagrams
to obtain approximations of the CBO, RFC and LCOM metrics. Strong assumptions
are made, in order to obtain accurate measures of these metrics, such as that every
non-abstract operation must be represented in one activity diagram and instance variable usages. They claim that these assumptions are not required in order to use theiralgorithms and obtain different levels of precision of the CK metrics at various stages.
Unfortunately, they do not provide any further information on which these various stages
967.2. Fault Prediction for Object-Oriented Software

JAIST

are and what the quantitative precision level of their algorithms is.
McQuillan et al. in [37] indicate that RFC can be derived by inspecting various
behavioral diagrams, but they do not say how, as for the CBO metric, they indicate thatit can be approximated from UML class diagrams, but to obtain more precise measures,
behavioral diagrams are needed. Later in [38], a formal definition of the CK metrics using
UML 2.0 is presented by the same authors. Their work is an improved version of Baroni’s
previous work using UML 1.3.
Because of the importance of coupling measures for fault-proneness detection, we
studied an alternative easy approach to measure CBO and RFC metrics using only communication diagrams. Further details are given in a later Section.

7.2

Fault Prediction for Object-Oriented Software

There is a considerable large number of works on this topic. For several years practitionersof fault prediction of object-oriented software have measured design-complexity metrics,
such as the CK metrics, from the code to build their prediction models. Another researchbranch of fault prediction for object-oriented software, in which the number of works are
not as large as in the former, focuses on how to make earlier predictions, previous to theimplementation of the code. In the following Sections, we briefly summarize some of theworks found in the literature in both branches of research work.

7.2.1

After Coding

In Tables 7.1 7.2, we have summarized some of the works on this area. Table 7.1 gives
a general description of the study and Table 7.2 gives the results obtained. In Table
7.1, the second column refers to the metrics used as predictors of faulty code, inputs of
their prediction models. The third column refers to the measure used of fault-pronenessof code, the predicted output. And finally the fourth column of this table refers to thetechnique applied for the construction of the prediction model. In Table 7.2, the second
column refers to those single metrics detected as good predictors of fault-prone code. The
third column refers to their best model. And finally, the last column refers to their general
results. More in details, these studies are:
1. A Validation of Object-Oriented Design Metrics as Quality Indicators by Basili et al.
in 1996. The goal of this study was to evaluate the CK metrics as predictors of faultprone classes. In order to validate their hypothesis, an empirical study was run overfour months. Ungraduated and graduated students of the University of Marylandwere grouped into eight teams of three students. Each team was asked to developed
a medium-sized management information system in C++. The development processof these systems were performed according to a sequential software engineering lifecycle model, derived from the Waterfall model. They collected the CK metrics fromsource code of the C++ programs delivered at the end of the implementation phase,
180 classes in total. They also collected data about faults found during the testing
phase. After the data collection, logistic regression was used to build a model thatpredicts fault-prone classes.
977.2. Fault Prediction for Object-Oriented Software

JAIST

Their prediction model was built using DIT, RFC, NOC, CBO, and the level ofreuse of a class. By using this model, they predicted correctly 138 classes out of
180, which results in a percentage of 76% of correct prediction and they detected
48 faulty classes of 58, which is 82% of faulty classes correctly predicted. This 82%
of faulty classes results is a 93% of faults of the entirely system. They found DIT,
RFC and NOC to be very significant for the prediction of faulty classes, while CBO
was found significant, and WMC was somewhat significant, but not as the previous
metrics mentioned. LCOM, on the other hand, was found not to be significant forthis study. They concluded that the studied CK metrics are useful predictors offault-proneness [5].
2. Exploring the relationships between design measures and software quality by Briand
et al.in 2000. They carried out an empirical study for exploring the relationships
between design complexity metrics (28 of coupling, 10 of cohesion, and 11 of inheritance, among them CK metrics) and the probability of fault detection in classes
during testing phase. This study used the same data collected in the study previously mentioned by [5]; and logistic regression was also used to build their prediction
models.
Coupling measures, such as RFC, and inheritance measures were found to be strongly
related to the probability of fault detection in a class. Multivariate analysis results
show that by using some of the coupling and inheritance measures, very accurate
models can be derived to detect fault-proneness of classes. Their best model obtained a percentage of correct classifications of about 82%; with this model 81% of
faulty classes were correctly detected, which can be translated in a 94% of faults
found in the entirely system [9].
3. O-O Software Quality Prediction using General Regression Neural Networks by Kanmani et al. in 2004, which discusses the application of general regression neuralnetworks for predicting software quality in terms of fault-proneness of code throughan empirical study. The software analyzed in this study was developed by studentsof the Pondicherry Engineering College. The systems were developed in a period of
about 1 month. The students were grouped in 200 teams, each team with one or 2
students. The developed software was in C++. Object design measures (10 of cohesion, 29 of coupling, 18 of inheritance measures) and size measures (7) were collectedfrom the source code, among these measures are the CK suite. Fault-proneness ofa class is measured as the fault ratio of the number of test cases not satisfied over
the total number of test cases responsible to the class.
Two groups of metrics were found to perform well at predicting fault-proneness ofcode. The first group was the size-cohesion-coupling group, and the second groupwas size-coupling-inheritance group. Both groups obtained an R-square of about
0.93. R-square measures the accuracy of the model, a perfect fit of the model would
result in an R-square value of 1 [32].
4. Early estimation of software quality using in-process testing metrics by Nachiappan
et al. in 2005. The objective of this study is to construct and to validate a set of98

7.2. Fault Prediction for Object-Oriented Software

JAIST

metrics called STREW-J (9 metrics. CBO, DIT and WMC among these) that canbe used as an early indication of an external measure of field quality. To evaluate
the predictive ability of a proposed set of metrics called STREW-J, a case study was
carried out in a junior/senior level software engineering course at North Caroline
State University in the fall 2003 semester. The system developed was an Eclipse
plug-in for collection of static code metrics, written in Java. A total of 22 projects
were analyzed, each project was developed by a group of five or six students. Fault
proneness of code was measured via black box test, number of failures per KLOC
were obtained from 45 test cases. Multiple linear regression was performed to build
the prediction model using the STEW-J metrics of fifteen projects. To construct andto test the model, the data collected of the 22 projects was split randomly 7 times.
Every time, 15 projects were taken to construct the model, and the remain 7 projects
were used to test it. To evaluate the fit of the model, in terms of predictability, it
was compute the average absolute error(AAE) and the average relative error(ARE)of the prediction of numberof f ailures/KLOC of the seven different projects not
used in the construction of the model. AAE ranks from 2.04 to 7.56 and ARE from0.82 to 2.10 of the 7 models built. They conclude that the proposed STREW-J
metrics are efficient to predict fault-proneness of code [40].
5. Empirical Validation of Three Software Metrics Suites by Olague et. al in 2007. This
paper describes an empirical validation of three suite of metrics to predict fault-proneclasses, which are are: the CK metrics, the Abreu’s Metrics for Object-Oriented Design (MOOD) and Bansiya and Davis’ Quality Metrics for Object-Oriented Design(QMOOD). Data was collected from six version of the Mozilla Rhino project, an
open source software written in Java. The CK-RFC and CK-WMC were found to
be consistent predictors of fault-proneness of code. As well as two metrics of the
QMOOD set: Class Interface Size (CIS) and Number of Methods (NOM), which issimilar to the CK-WMC. Applying multivariate logistic regression was found thatthree CK models predict more accurately fault-proneness of code, followed by one
QMOOD model. The accuracy of the CK models range from 62.6 to 90.6 percent
through the six version of Rhino. And the accuracy of the QMOOD model range
from 63.5 to 85.2 percent through the six version of Rhino [43].

7.2.2

Before Coding using non UML metrics

We found few works that use metrics from design artifacts to predict faulty code before
coding. Only one of these uses a subset of the CK metrics, which were partially obtained
from design documents [51]. Further details of these works are given in the following
paragraphs.
In [42], the relationship between design metrics and the number of function test failure
reports of software packages is investigated. From this work, a tool called ERIMET wasdeveloped to analyze design documents automatically. Then, packages are represented
by one or more graphs. Finally, some metrics, which are believed to contribute to the99

7.2. Fault Prediction for Object-Oriented Software

JAIST

Table 7.1: Studies on fault-proneness code prediction
Study

Input Metrics

Output1
2

CK
49 metrics of coupling,cohesion
andinheritancemetrics(CK among them)
CK among 64 metrics
of: inheritance, size, cohesion and coupling
STREW-J suite (3 CK
metrics and others)
CK, MOOD, QMOOD

Fault-prone classes
Fault-prone classes

345

Prediction
Technique
Logistic regression
Logistic Regression

Fault ratio

GeneralRegression
Neural Network

Fault ratio

Multiple Lineal Regression
Logistic Regression

Fault-prone classes

complexity of the code, are calculated. This calculation was possible due to the usageof a formal language named FDL, which is related to SDLs 1 process diagrams. An
example of these metrics is a modified McCabe’s Cyclomatic Complexity metric. Although
this method allows to predict before coding, it is unknown to what extent the proposed
prediction model can be used by different organizations. Therefore, it is recommend thatorganizations build their own prediction models, using their own specific data.
In [51], Ramanath et al. provide empirical evidence supporting the role of a subsetof the CK metrics in determining software defects. Complexity measures were computedfrom design documents and source code. While WMC and DIT were computed from both
sources, CBO and size were computed only from source code.
In [48], Schroter et al. used relationships between components, which are typicallydefined in the design phase, to predict fault-proneness of code. They carried out anempirical study of 52 eclipse plug-ins, where all possible import packages in ECLIPSE
were used as independent variables of their model. From their experiments, they conclude
that prediction on a package level yields better results than prediction on a file/class level.
They also found that models trained in one version can be used to predict failure-prone
components in later versions.
In the same year, Yue Jiang et al. [27] compared the performance of predictive models
which use design-level metrics with those which use code-level metrics, and with thosewhich use both. They concluded that models built from code metrics typically outperform
design metrics-based models, and that models that utilize a combination of design andcode-level metrics outperform models which use either one, or the other metrics set. Whatis of interest to us is that, by performing reverse engineering, they recovered some design
artifacts from code, and they could derive software quality metrics from these artifacts,
metrics such as Cyclomatic complexity, number of decisions, etc. (no CK metrics). For
our research, reverse engineering is not an option. We cannot reverse engineered the code
1

Specification and Description Language: A modeling language used to describe real-time systems

100

7.2. Fault Prediction for Object-Oriented Software

JAIST

Table 7.2: Studies on fault-proneness code prediction: Results
Study Good predictors
1- RFC, DIT, NOC(+++)

Best models
DIT,RFC,NOC,CBO
Level of reuse of a class

Resultsand- CBO (++)

- Correct
tion 76%- WMC (+)
2
- RFC and others
coupling (+++)- Faulty classes
tected 82%

4 coupling metrics plus 3 of inheritancede-classifica-

Faulty Classes detected
81% Correct classification
82%- NOC, DIT andothers of inheritance (++)
3

No details- 7 size metrics, 10 of cohesion and 29 of couplingR-square = 0.93 (A perfect
fit gives: R-square = 1)

- 7 size metrics, 29 of coupling and 18 of inheritance group
4

No details

STREW-J set (CBO, DIT,WMC and others)

5
- CK:
WMC

RFC,

- 3 CK Models- 1 QMOOD Model- QMOOD:
NOM, CIS

AAE: 2.04 - 7.56 , ARE:
0.82 - 2.10 through 7 different models
Through 6 versions of the
SW analyzed:- CK Models accuracy: 62.9 - 90.6%
- QMOOD Model accuracy: 63.5 - 82.5 %(+++) very significant (++) significant (+) somehow significant

101

7.2. Fault Prediction for Object-Oriented Software

JAIST

to obtain UML communication diagrams, and then measure our UML CK metrics fromthem, to later predict fault-prone code; this mainly because our findings suggest thatmeasures taken from the design (at a certain stage) are different from those taken fromthe code, as it happens with our UML WMC metric. Further details can be found in
Chapter 4 of this paper.

7.2.3

Before Coding using UML metrics

Concerning the use of UML metrics as predictors of fault-proneness of code, we found thefollowing two related works:
1. Our own previous work published in [10], where we presented the results of our
exploratory study on seven UML metrics as predictors of fault-proneness of code.
The seven metrics were derived from both UML class and collaboration diagrams,
and they were introduced as approximations of two CK metrics and three QMOOD
metrics (used in Olague’s work [43]). The proposed seven UML metrics were 2 approximations of the RFC CK metric, 2 of the CBO CK metric and 3 more UML
approximations of the CIS, NOM, DAM QMOOD metrics (some details of these
metrics are given in the following Section).
Twelve univariate logistic models were built in total using the design and implementation of one of the JAIST projects previously introduced, the ECS project; thenumber of classes analyzed were 13. Seven from the twelve models were UML based
and the rest code based. The dependent variable in the models was the probabilityof class of being Faulty during the testing phase.
Our results in this work can be summarized as follows. The prediction accuracy of
our UML models ranked from 69% to 91%. In general code metrics could predict
better than the proposed UML metrics. One of our approaches to approximate RFC
with UML diagrams performed equally to the code RFC metric. And the model using
CIS measures from UML class diagrams performed similarly to the model using code
measures, being the best UML model in this experiment at detecting fault-proneclasses with 91% of accuracy prediction, followed by the previous one mentioned,
the UML RFC model with 84.6% accuracy. In this study no normalization method
was used.
2. Recently, Nugroho et al. [41] proposed five UML metrics measured from UML classand sequence diagrams to predict faulty classes. Perhaps, this research work is themost similar to ours. Their proposed five UML metrics are classified into level ofdetail and coupling metrics. An industrial health-care system written in Java (266
UML classes and 152,017 SLOC) was used to build and test three logistic regression
models. One model was built solely with two UML metrics, the second model withcode metrics, and the third model with KSLOC and the same two UML metrics
used in their UML model.
From their five proposed UML metrics, only one, ’ImpCoupling’, resulted significantly correlated to class fault-proneness. Although their UML metric ’SDmsg’ was102

7.3. Comparison of UML metrics and Prediction Results

JAIST

not significantly correlated to fault-proneness, it was used along with the ’ImpCoupling’ in their prediction models.
They indicate that their UML metric ’SDmsg’ as a significant predictor when is
combined with other metrics, namely ’ImpCoupling’ or KSLOC. And these two
metrics were always significant in both univariate and multivariate analysis. Moreover, their hybrid model’s prediction accuracy resulted to be the best with 78%,
which is 16% above the accuracy percentage of their UML model with 62%. Last
but not least, they indicate that log transformations were applied for normalizing
their data.
For comparison purposes, the metrics used and prediction results of these two works
are further detailed in the following Section.

7.3

Comparison of UML metrics and Prediction Results

As we mentioned in the previous Section, there are only two related works concerning
UML metrics and fault prediction. These to works are our own previous work publishedin [10] (Study 1), and the research work of Nugroho et al. in [41] (Study 2). In the following lines, we give further details of the UML metrics used in both research works andsummarized their fault prediction ability, so that we can compare them with the proposed
metrics in this paper and their corresponding prediction results.
[2008] Study 1
The UML CBO and RFC metrics proposed in this paper, compare to those ones given
in our previous work [10], are more clearly defined and are given in more proper terms(related to messages of communication diagrams): ”call messages” and ”instantiated methods”. In principle our UCBO is the same as the previous defined as CBO-UML2; and as
for our URFC, the main difference is that the previous two proposals of approximation,
RFC-UML1 and RFC-UML2, still count return messages.
The other three QMOOD metrics used in our previous work were derived from class
diagrams, and their definitions are the following:• CIS, Class Interface Size, is a count of public methods in a class, which UML approximation from class diagrams would be similar to the proposed UWMC metricin this paper.
• DAM, Data Access Metric, is the ratio of the number of private or protected attributes to the total number of attributes declared in a class.
• NOM, Number of Methods.

103

7.3. Comparison of UML metrics and Prediction Results

JAIST[2010] Study 2
Although the proposed two UML coupling metrics by Nugroho et al. in [41] are notmeasured from communication diagrams and were not proposed as approximation of thecode CK metrics, at first glance, they seem to be similar to our proposed UML metrics
to approximate the code CK metrics.
Their proposed UML coupling metrics are named ’ExCoupling’ and ’ImpCoupling’,
and their respective definitions are:• ’ExCoupling’ measures the total number of incoming methods invocations MsgIN
to a particular object modeled in sequence diagrams. For a given implementation
class x, a corresponding design class x′ and j number of UML sequence diagrams inwhich x′ appears, ’ExCoupling of x is measured according to Equation 7.1:
ExCoupling(x) = MsgIN1 (x′ ) + MsgIN2 (x′ )...MsgINj (x′ )(7.1)• ’ImpCoupling’ measures the total number of outgoing methods invocations MsgOUT
FROM a particular object modeled in sequence diagrams. For a given implementation class x, a corresponding design class x′ and j number of UML sequence diagrams
in which x′ appears, ’ImpCoupling’ of x is measured according to Equation 7.2:
ImCoupling(x) = MsgOUT1 (x′ ) + MsgOUT2 (x′ )...MsgOUTj (x′ )(7.2)

Therefore, ’ExCoupling’ is similar to our metric ’NIM’ (Number of Instantiated Methods) and ’ImpCoupling to our metric ’NIMO’ (Number of Instantiated Methods of Other
classes), which are different from those metrics used to approximate the code CK metrics,
as defined in Chapter 3.
Moreover, as it was indicated earlier, only ’ImpCoupling’ resulted to be significant
predictor of fault-proneness of code. The second UML metric used in their prediction
models was the so-called ’SDmsg’ , message detailedness, which is an aggregate metricthat measures the level of detail of messages of class instances (objects) modeled in sequence diagrams, ’SDmsg’ can be calculated using Equations 7.3 and 7.4.
MsgLoDi = SDops(i) + SDret(i) + SDpar(i)(7.3)

where:- i, is a given object is a sequence diagram,- SDops measures the RATIO of messages that correspond to class methods specified
in class diagrams TO the total number of messages of the object.
- SDret measures the RATIO of return messages with label TO the total number if
messages of the object.
- SDpar measures the RATIO of messages with parameters TO the total number ofmessages of the object.
n

1X
MsgLoDi (x′ )SDMsg(x) =
n i=1
104

(7.4)

7.3. Comparison of UML metrics and Prediction Results

JAIST

where:- x is the implemented class of a designed class x′ ,- n is the number of sequence diagrams in which x′ appears.
Fault Prediction
To ease the comparison between the previous two studies and our current study, wesummarize in Tables 7.3 and 7.4 the UML metrics used and their corresponding prediction
results.
The findings of our previous study (Study 1) suggest that UML metrics can be acceptable predictors of fault-proneness of code using UML-based prediction models, whichobtained accurate predictions ranking from 69% to 90.9%. The UML metrics were suggested as approximations of the CK CBO and RFC metrics, and of the QMOOD DAM,
NOM and CIS metrics. Since only the QMOOD metrics can be approximated usingclass diagrams, our current research work (contents of this paper) was focused on how toapproximate the CK metrics that cannot be measured straightforwardly from the UML
class diagrams. The prediction models used, different to our previous study, were codebased, because our initial intentions were to perform multivariate analysis, which due tocorrelated independent metrics was not possible (as explained in Section 6.3).
If we compare the results of Study 1 with our current results, the different versionsof the UML RFC and CBO metrics obtained similar prediction results using either codebased models or UML-based models. As for the UML CIS metric, which in principle
measures the same as our UWMC metric, resulted to be better predictor than the latter.
This could be explained because CIS is measured from class diagrams, which details more
the number of public methods to be developed than communication diagrams.
The prediction accuracy of the UML-based model proposed in Study 2 is slightly lower
than those obtained in our studies. Based on our findings, we think that their prediction
accuracy could be increased by:• Using linear scaling to unit variance instead of log transformations to normalize
data.
• Using our URFC metric instead of ’ImpCoupling’ of Study 2, for the following reasons. Since, ’ImpCoupling’ is similar to our metric defined as NIMO, our URFC
could perform better, because URFC carries with it more information than NIMO.
As defined in Chapter 3, URFC is a sum of two metrics NDIM and NDIMO; being NDIMO almost similar to NIMO and in consequence to ’ImpCoupling’ (withthe difference that NDIMO counts for different messages), URFC could be better
predictor than NIMO metric.

105

Study
1.

106
2.

Model
(U):CBO1(U): CBO2(U): RFC1(U): RFC2(U): CIS(U): DAM(U): NOM(U): CBO(U): RFC(U): CIS(U): DAM(U): NOM(U):CBO1(U): CBO2(U): RFC1(U): RFC2(M): SDmsg + ImpCoupling(M): SDmsg + ImpCoupling+ KSLOC

Normalization Construction Data

Test Data

None

13 ECS classes [UML]

[S]

None

13 ECS classes [Code][S]

13 ECS classes [UML]

11 BNS classes [UML]Log

266 IHS classes [UML]266 IHS [UML + Code]

[S]

LS

17 MYL-P12 classes[Code]

None

This(U): RFC

10 ECS classes [UML]
10 ECS classes [Code]
16 CRS classes [UML]
16 CRS classes [Code]11 BNS classes [UML]
11 BNS classes [Code]

IHS: Integrated Health-care System, ECS: E-commerce System (JAIST), CRS: Cruise control System (JAIST), BNS: Banking System (JAIST)

80%
80%
64%
57%
67%
33%

JAIST

(U): Univariate Model, (M): Multivariate Model, LS: Linear Scaling to Unit Variance[Code]: Code Measures, [UML]: UML Measures, [UML + Code]: UML and Code Measures, [S]: Same Measures used in the Construction

Accuracy
69.20%
69.20%
76.92%
84.61%
90.9%
72.7%
72.7%
92.30%
84.61%
90.9%
36.3%
90.9%
72.7%
63.6%
72.7%
72.7%
62%
78%

7.3. Comparison of UML metrics and Prediction Results

Table 7.3: Fault Prediction using UML Metrics

StudyThis

Normalization Construction Data

(U): CBOLS

49 MYL-P11 classes[Code](U): WMC

LS

30 MYL-P02 classes[Code]

107

Model

Test Data
10 ECS classes [UML]
10 ECS classes [Code]
16 CRS classes [UML]
16 CRS classes [Code]11 BNS classes [UML]
11 BNS classes [Code]
10 ECS classes [UML]
10 ECS classes [Code]
16 CRS classes [UML]
16 CRS classes [Code]11 BNS classes [UML]
11 BNS classes [Code]

Accuracy
80%
80%
71%
88%
80%
89%
45%
55%
38%
56%
44%
78%

7.3. Comparison of UML metrics and Prediction Results

Table 7.4: Fault Prediction using UML Metrics(U): Univariate Model, (M): Multivariate Model, LS: Linear Scaling to Unit Variance
IHS: Integrated Health-care System, ECS: E-commerce System (JAIST), CRS: Cruise control System (JAIST), BNS: Banking System (JAIST)

JAISTChapter 8
Other Applications

In this Chapter, we discuss the potential use of our UML metrics in other two areasof research. The first one is on software implementation progress estimations and thesecond one is on dynamic coupling measures and their relationship with change pronenessof code. In the first area, we have started to do some research work, which details can befound in [12].

8.1

Software Implementation Progress Estimations

Scheduling and monitoring progress of software development are crucial activities to meetthe deliverable deadlines and, in general, to control a project. One of the reasons software
products are delivered late is the inability to recognize on time that the project is fallingbehind schedule. Specifically talking of software implementation, progress reports areoften estimated subjectively by software developers. Such reports are often open to misconception leading to inaccurate schedule estimations [34, 54], which is one of the major
causes of software schedule slippage [35]. Therefore, objective and accurate measurements
for assessing software implementation progress timely are clearly needed.
The number of Source Lines of Code (SLoC) and used memory in bytes are frequently
used to measure the size of a software product [31]. However, assessing the percentage ofwhat is being developed in terms of the same metrics is not possible, because their final
values remain unknown till the project is completed.
Considering that the overall functionality of a software is achieved by relating every
single part or module composing the software, code metrics measuring such relationshipmight be able to tell us how much of the totality of the software is being developed, only
if, the final values of the same metrics are known beforehand. One of the available objectoriented metrics that can measure a kind of relationship between the different modules ofthe software is the CK CBO metric. Now, let us say that we would like to use the CBO
metric to track software implementation progress, we still would need a target value ofwhat must be reached at the end of the implementation in terms of the same metric.
1088.2. Dynamic Coupling Measures

JAIST

Based on our evaluation results of our UML CBO metric in Chapter 5, Section 5.2.2,
which values were close to the final code CBO values in different small-size software
projects, we hypothesize that the same UML CBO metric can serve as the sought target
value to estimate implementation progress readily.
In [12], we discuss our first attempts of estimating software implementation progress
readily. Our approach is based on CBO measures using as a baseline UML approximations of the same measures. The assessment of the implementation progress consists,
basically, in the normalization of both UML and code CBO measures, so that, as thecode approximates to its design, decreasing the error approximation, some progress is”earned”.
Our initial results suggest that our methodology can be helpful in the estimation ofsoftware implementation progress, which encourages us to perform further improvementsof our approach.

8.2

Dynamic Coupling Measures

Coupling has been traditionally measured through structural properties and static codeanalysis. However, because of polymorphism, dynamic binding and the common presenceof unused code in commercial software, the resulting coupling measures are imprecise as
they do not perfectly reflect the actual coupling taking place among classes at runtime.
The type of coupling that can define and precisely measured based on dynamic analysis ofsystems is referred as dynamic coupling. A number of code dynamic coupling measures
have been defined and the related to change proneness of classes, their results indicatethat such measures are good indicators of change proneness and can be complement ofexisting static measures for coupling, more information can be found in [3].
Using the same reasoning, it is possible that coupling measures derived from communication diagrams, which reflect the dynamic behavior of a system, could provide better
information than those metrics derived from UML static views.
On this investigation line, a very interesting work was recently published by Ah-rim
Han et al. [24], who have proposed a compound measure of dynamic coupling that isderived from UML structural and behavioral information. They used such a measure to
predict change proneness of classes. From their experiment results, they conclude that
although the goodness-of-the-fit of their prediction model is apparently lower than a codebased model, their model can be of great aid in early stages of the development of thesoftware. It might be worth of investigation the application of our UML CBO and RFC
metrics, an in general all metrics derived from UML communication diagrams (Chapter
3, Section 3.5), to factors such as change proneness of classes.

109

Chapter 9
Conclusions and Plans for FutureWork

The CK metrics were defined to measure complexity of the design, and they havetraditionally been measured from the code and then related to various managerial factors
such as productivity, re-work effort for reusing classes and design effort, maintenanceeffort [14, 20], and fault-proneness of code [5, 9, 30, 32, 40, 43].
In this paper, we proposed a simple approach to approximate CBO, RFC and WMC
CK metrics using UML communication diagrams. Special focus on the CBO and RFC wasgiven, because previous research studies reported such approximations difficult to achieveusing only class diagrams. Our UML metrics were defined using the Goal/Question/Metricapproach. And, an empirical study was carried out to evaluate our metrics, first, as approximations of their corresponding code CK metrics, and second as predictors of faultprone code.
The results of our empirical study lead us to conclude that the proposed UML RFC
and CBO metrics can predict faulty code almost with the same accuracy as their corresponding code metrics. The elimination of outliers and the normalization procedure usedwere significantly efficient, not just for enabling our UML metrics to predict faulty codeusing a code-based prediction model, but also for improving the prediction results of ourmodels across different packages and projects, which was found to be difficult, even across
packages of the same project. On the other hand, both the UML WMC and the code
WMC metrics showed a poor fault-proneness prediction ability.
We expect our prediction results can be improved by using UML-based prediction
models, or by the inclusion of other metrics (different to design-complexity metrics) inthe predictions models. Such metrics should be easily obtainable before the implementation of the system, with low or zero correlation among them, and strongly related tofault-proneness of code. Moreover, other preprocessing techniques for datasets, such as
normalization and over/under sampling, can also be helpful.
There is also the need to explore different methodologies to predict fault-proneness.
Recalling the main reason we chose logistic regression as prediction technique is because
110JAIST
this technique, different to other statistical techniques, does not require normally distributed variables, and the resulting models are easy to interpret and to understand.
Therefore, we are looking for prediction techniques with similar characteristics; and, if
possible (considering our previous discussion in Chapter 6) that can address the problem of correlation among independent variables efficiently. Some statistical data mining
techniques have been recently adopted by fault-proneness prediction researchers, such as
bayesian networks and decision trees, however we still do not know the extent of usefulnessof these techniques for our specific purposes.
Our main intentions are to direct others’ researchers attention towards our UML metrics, not only as early predictors of faulty code, but also as early measures of designcomplexity metrics and/or coupling, which could be used in other research areas suchas change proneness of software and software implementation progress, as previously discussed in Chapter 8. Moreover, we hope that the results presented in this paper encourage
those who have access to real-life software projects to carry out more experiments, so thatexternal validity of our study can be extended.
In the field of empirical software engineering studies, apart of the internal and external validity of their designed models (in our case our fault-prediction models), there
exists another important aspect of validity for the metrics used, which is called constructvalidity. A metric exhibits construct validity when it measures what it is purported tomeasure. When software metrics are defined using plain-language terms, problems canarise because they can be easily misinterpreted, resulting in other different measures tothe ones originally intended [45]. On this topic, as complementary work of our study, we
present the validation of the construction of our UML metrics in Appendix A.
To sum up, our plans for future work, mainly concern the exploration of other areasof research in which our UML metrics can be applied, and concerning the topic of fault
prediction, the following subjects for further study have been considered: data normalization and other pre-processing techniques, the inclusion of other metrics in our prediction
models (different to design-complexity metrics), as well as the study of other prediction
techniques (different to logistic regression).

111

List of Tables
2.1

Guidelines for the CK metrics . . . . . . . . . . . . . . . . . . . . . . . . .

7

3.1 Our Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Direct Metrics for Objects in UML Communication Diagrams . . . . . . . .
3.4 Metrics for Classes from UML Communication Diagrams . . . . . . . . . .
3.5 Metrics for Class Calculator . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6 Metrics for Class Adder . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.7 Metrics for Class Multiplier . . . . . . . . . . . . . . . . . . . . . . . . . .
3.8 Metrics for Class Display . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.9 Summary of Metrics for Calculator Application . . . . . . . . . . . . . . .
3.10 Summary of Metrics for Calculator Application: UML CK Metrics . . . . .
3.11 Metrics Definition based on the GQM Approach . . . . . . . . . . . . . . .

26
26
27
29
31
32
33
33
33
33
34

4.1
4.2

Correlation Coefficients Required for Statistical Significance at 5% Level . 38
Summary of Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.1
5.2
5.3
5.4
5.5

CBO Code measures of the BNS . . . . . . . . . . . . . . . . . . . . . . . .
Detailed Information for CBO Boxplots . . . . . . . . . . . . . . . . . . . .
Detailed Information for RFC Boxplots . . . . . . . . . . . . . . . . . . . .
Detailed Information for WMC Boxplots . . . . . . . . . . . . . . . . . . .
Average Relative Errors Information . . . . . . . . . . . . . . . . . . . . .

54
56
57
58
71

6.1 Detailed Information of Fault Data Distribution for CBO Models . . . . .
6.2 Detailed Information of Fault Data Distribution for RFC Models . . . . . .
6.3 Detailed Information of Fault Data Distribution for WMC Models . . . . .
6.4 Univariate Logistic Regression Models . . . . . . . . . . . . . . . . . . . .
6.5 Univariate Logistic Regression Models Validation . . . . . . . . . . . . . .
6.6 Discrimination Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.7 Normalized code measures vs UML measures . . . . . . . . . . . . . . . . .
6.8 Correlation Between Independent and Dependent Variables . . . . . . . . .
6.9 Correlation Among Independent Variables . . . . . . . . . . . . . . . . . .
6.10 Coefficients of Determination Among Variables . . . . . . . . . . . . . . . .
6.11 RFC: Normalized vs Raw Code Measures . . . . . . . . . . . . . . . . . . .
6.12 CBO: Normalized vs Raw Code Measures . . . . . . . . . . . . . . . . . . .
6.13 WMC: Normalized vs Raw Code Measures . . . . . . . . . . . . . . . . . .

82
83
84
85
85
85
87
88
88
90
94
95
95

112

List of Tables

JAIST

7.1
7.2
7.3
7.4

Studies on fault-proneness code prediction . . . . . . . . . . . . . . . . . . 100
Studies on fault-proneness code prediction: Results . . . . . . . . . . . . . 101
Fault Prediction using UML Metrics . . . . . . . . . . . . . . . . . . . . . 106
Fault Prediction using UML Metrics . . . . . . . . . . . . . . . . . . . . . 107

A.1
A.2
A.3
A.4

Abstract Functions to Model Direct Metrics for Objects . . . . . . . . . . . xxii
Abstract Functions to Model Direct Metrics for Objects . . . . . . . . . . . xxiii
Abstract Functions to Model Metrics for Classes . . . . . . . . . . . . . . . xxiv
Abstract Functions to Model Metrics for Classes . . . . . . . . . . . . . . . xxv

113

List of Figures
1.1

Research Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

2.1
2.2

Example: Synchronicity in Communication Diagrams . . . . . . . . . . . . 10
Meta-model of a UML Communication Diagram . . . . . . . . . . . . . . . 13

3.1
3.2
3.3

Example of a Communication Diagram . . . . . . . . . . . . . . . . . . . . 28
UML Communication Diagrams for the Calculator Application . . . . . . . 31
Code of the Calculator Application . . . . . . . . . . . . . . . . . . . . . . 32

4.1 RAW RFC measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2 RAW RFC measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3 RAW CBO measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4 RAW WMC measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5 Design Level of Detail Effect . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6 Software Type Effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.7 RFC: Same design, two different implementations . . . . . . . . . . . . . .
4.8 CBO: Same design, two different implementations . . . . . . . . . . . . . .
4.9 CBO: Same design, two different implementations . . . . . . . . . . . . . .
4.10 Normal Distributions of Relative Errors of or UML Approximations . . . .

40
41
43
44
47
48
48
49
49
51

5.1 Boxplot for CBO data of a Banking System . . . . . . . . . . . . . . . . .
5.2 Boxplot for CBO data . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3 Boxplot for RFC data . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.4 Boxplot for WMC data . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.5 CBO Level vs Spread . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.6 RFC Level vs Spread . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.7 WMC Level vs Spread . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.8 Spread-versus-level Plot for CBO data . . . . . . . . . . . . . . . . . . . .
5.9 Spread-versus-level Plot for RFC data . . . . . . . . . . . . . . . . . . . . .
5.10 Spread-versus-level Plot for WMC data . . . . . . . . . . . . . . . . . . . .
5.11 RFC measures using Logarithm Transformation . . . . . . . . . . . . . . .
5.12 CBO measures using Logarithm Transformation . . . . . . . . . . . . . . .
5.13 WMC measures using Logarithm Transformation . . . . . . . . . . . . . .
5.14 RFC Boxplots using Logarithmic Transformation . . . . . . . . . . . . . .
5.15 CBO Boxplots using Logarithmic Transformation . . . . . . . . . . . . . .
5.16 WMC Boxplots using Logarithmic Transformation . . . . . . . . . . . . . .

54
59
60
61
62
62
63
64
65
66
67
67
68
68
69
69

114

List of Figures

JAIST

5.17 Normal distribution with media zero and standard deviation 1 . . . . . . .
5.18 RFC measures using Linear scaling to unit variance . . . . . . . . . . . . .
5.19 CBO measures using Linear scaling to unit variance . . . . . . . . . . . . .
5.20 WMC measures using Linear scaling to unit variance . . . . . . . . . . . .
5.21 RFC Boxplots using Linear Scaling to Unit Variance . . . . . . . . . . . .
5.22 CBO Boxplots using Linear Scaling to Unit Variance . . . . . . . . . . . .
5.23 WMC Boxplots using Linear Scaling to Unit Variance . . . . . . . . . . . .

70
72
72
73
73
74
74

6.1 Methodology to Evaluate our UML metrics as Predictors of Faulty Code .
6.2 Univariate Logistic Model . . . . . . . . . . . . . . . . . . . . . . . . . . .
6.3 Boxplots of Fault Data of JAIST Projects (for CBO Models) . . . . . . . .
6.4 Boxplots of Fault Data of Mylyn Packages (for CBO Models) . . . . . . . .
6.5 Boxplots of Fault Data of JAIST Projects (for RFC Models) . . . . . . . .
6.6 Boxplots of Fault Data of Mylyn Packages (for RFC Models) . . . . . . . .
6.7 Boxplots of Fault Data of JAIST Projects (for WMC Models) . . . . . . .
6.8 Boxplots of Fault Data of Mylyn Packages (for WMC Models) . . . . . . .
6.9 Correlation and Shared Variance . . . . . . . . . . . . . . . . . . . . . . . .
6.10 Shared Variance and Correlated Independent Variables . . . . . . . . . . .
6.11 Flow Chart for Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . .

76
77
78
79
79
80
80
81
90
91
92

A.1
A.2
A.3
A.4

Distanced-based Measure Construction incorporated in our Empirical Study ii
Example of Different Two Communication Diagrams . . . . . . . . . . . . iv
Transformation of SObjectes(CmDA) → SObjects(CmDA′ ) . . . . . . . . vi
Example of Different Two Communication Diagrams . . . . . . . . . . . . viii

115

Bibliography[1] Abdelmonem Afifi, Virginia Clark, and Susanne May. Computer-Aided Multivariate
Analysis. Chapman & Hall/CRC, USA, 2004.
[2] Selim Aksoy and Robert M. Haralick. Feature normalization and likelihood-basedsimilarity measures for image retrieval. Pattern Recogn. Lett., 22(5):563–582, 2001.
[3] E. Arisholm, L.C. Briand, and A. Foyen. Dynamic coupling measurement for objectoriented software. Software Engineering, IEEE Transactions on, 30(8):491 – 506,
2004.
[4] A. Baroni and F. B. Abreu. An OCL based formalization of the MOOSE metrics
suite. In 7th ECOOP workshop on quantitative approaches in object oriented softw.
eng., 2003.
[5] Victor R. Basili, Lionel C. Briand, and Walcélio L. Melo. A validation of objectoriented design metrics as quality indicators. IEEE Trans. Softw. Eng., 22(10):751–
761, 1996.
[6] Viv Bewick, Liz Cheek, and Jonathan Ball. Statistics review 14: Logistic regression.
Critical Care, 9(1), 2005.
[7] Thirumalesh Bhat and Nachiappan Nagappan. Building scalable failure-pronenessmodels using complexity metrics for large scale software systems. In APSEC ’06:
Proc. of the XIII Asia Pacific Softw. Eng. Conference, pages 361–366, Washington,
DC, USA, 2006. IEEE Comp. Soc.
[8] B. Bolstad, R. Irizarray, M. Astrand, and T. Speed. A comparison of normalization
methods for high density oligonucleotide array data based on bias and variance.
Bioinformatics, 19(2):185–193, 2003.
[9] Lionel C. Briand, Jürgen Wüst, John W. Daly, and D. Victor Porter. Exploring therelationship between design measures and software quality in object-oriented systems.
J. Syst. Softw., 51(3):245–273, 2000.
[10] Ana E. Camargo C. and Koichiro Ochimizu. Quality prediction model for object
oriented software using UML metrics. In Proc. of the 4th World Congress for Softw.
Quality, Bethesda, Maryland, USA, 2008. ASQ.

116

Bibliography

JAIST[11] Ana Erika Camargo Cruz and Koichiro Ochimizu. A UML approximation of threechidamber-kemerer metrics and their ability to predict faulty code across software
projects. IEICE TRANSACTIONS on Information and Systems, E93-D(11):3038–
3050, 2010.
[12] Ana Erika Camargo Cruz and Koichiro Ochimizu. Towards objective estimations ofsoftware implementation progress. Technical report, IEICE Tech. Rep., March 2011.
[13] S. R. Chidamber and C. F. Kemerer. A metrics suite for object oriented design.
IEEE Trans. Softw. Eng., 20(6):476–493, 1994.
[14] Shyam R. Chidamber, David P. Darcy, and Chris F. Kemerer. Managerial use of
metrics for object-oriented software: An exploratory analysis. IEEE Trans. Softw.
Eng., 24(8):629–639, 1998.
[15] Twan van Enckevort. Refactoring UML models: using openarchitectureware to measure uml model quality and perform pattern matching on UML models with ocl
queries. In OOPSLA ’09: Proceeding of the 24th ACM SIGPLAN Conf. companion
on Object oriented programming systems languages and applications, pages 635–646,
New York, NY, USA, 2009. ACM.
[16] Hans-Erik Eriksson, Magnus Penker, and David Fado. UML 2 Toolkit. John Wiley& Sons, Inc., New York, NY, USA, 2003.
[17] Norman E. Fenton and Martin Neil. A critique of software defect prediction models.
IEEE Trans. Softw. Eng., 25(5):675–689, 1999.
[18] George Fernández. Data mining using SAS applications. CRC Press, Inc., Boca
Raton, FL, USA, 2002.
[19] Marcela Genero, Mario Piatini, and Esperanza Manso. Finding ”early” indicators of
UML class diagrams understandability and modifiability. In ISESE ’04: Proc. of the2004 Intl. Symposium on Empirical Softw. Eng., pages 207–216, Washington, DC,
USA, 2004. IEEE Comp. Soc.
[20] Marcela Genero, Mario Piattini, and Coral Caleron. A survey of metrics for UML
class diagrams. Journal of Object Technology, 4:59–92, 2005.
[21] Marcela Genero, Mario Piattini, Esperanza Manso, and Giovanni Cantone. Building UML class diagram maintainability prediction models based on early metrics.
In METRICS ’03: Proc. of the 9th Intl. Symposium on Softw. Metrics, page 263,
Washington, DC, USA, 2003. IEEE Comp. Soc.
[22] Marcela Genero, Geert Poels, and Mario Piattini. Defining and validating metrics forassessing the understandability of entity-relationship diagrams. Data Knowl. Eng.,
64:534–557, March 2008.
[23] Hassan Gomaa. Designing Concurrent, Distributed, and Real-Time Applications with
UML. Addison Wesley-Object Technology Series Editors, Boston, MA, USA, 2000.
117

Bibliography

JAIST[24] Ah-Rim Han, Sang-Uk Jeon, Doo-Hwan Bae, and Jang-Eui Hong. Measuring behavioral dependency for improving change-proneness prediction in UML-based designmodels. J. Syst. Softw., 83:222–234, February 2010.
[25] David C. Hoaglin, Frederick Mosteller, and John W. Tukey. Understanading Robustand Exploratory Data Analysis. John Wiley & Sons, Inc., USA, 2000.
[26] Rumbaugh James, Jacobson Ivar, and Grady Booch. The Unified Modeling Language
Reference Manual, Second Edition. Addison-Wesley, Massachusetts, USA, 2006.
[27] Yue Jiang, Bojan Cuki, Tim Menzies, and Nick Bartlow. Comparing design andcode metrics for software quality prediction. In PROMISE ’08: Proc. of the 4th Intl.
workshop on Predictor Models in Softw. Eng., pages 11–18, New York, NY, USA,2008. ACM.
[28] I.T. Jolliffe. Principal Component Analysis. Springer-Verlag, New York, NY, USA,1986.
[29] Yasutaka Kamei, Akito Monden, Shinsuke Matsumoto, Takeshi Kakimoto, and Kenichi Matsumoto. The effects of over and under sampling on fault-prone module
detection. In ESEM ’07: Proc. of the First Intl. Symposium on Empirical Softw.
Eng. and Measurement, pages 196–204, Washington, DC, USA, 2007. IEEE Comp.
Soc.
[30] Yasutaka Kamei, Akito Monden, Shuji Morisaki, and Ken-ichi Matsumoto. A hybrid
faulty module prediction using association rule mining and logistic regression analysis. In ESEM ’08: Proc. of the Second ACM-IEEE Intl. symposium on Empirical
Softw. Eng. and Measurement, pages 279–281, New York, NY, USA, 2008. ACM.
[31] Stephen H. Kan. Metrics and Models in Software Quality Engineering. AddisonWesley Longman Publishing Co., Inc., Boston, MA, USA, 2002.
[32] S. Kanmani, V. Rhymend Uthariaraj, V. Sankaranarayanan, and P. Thambidurai.
Object oriented software quality prediction using general regression neural networks.
SIGSOFT Softw. Eng. Notes, 29(5):1–6, 2004.
[33] Bell DeTienne Kristen and Lewis Lee W. Artificial neural networks for the management researcher: The state of the art. The Forum: A Publication of the Research
Methods Division of the Academy of Management.
[34] Jinhua Li, Zhibing Ma, and Huanzhen Dong. Monitoring software projects withearned value analysis and use case point. ACIS International Conference on Computer and Information Science, pages 475–480, 2008.
[35] Z. Ma, J.S. Collofello, and D.E. Smith-Daniels. Causes and solutions for scheduleslippage: a survey of software projects. In Performance, Computing, and Communications Conference, 2000. IPCCC ’00. Conference Proceeding of the IEEE International, pages 373 –379, February 2000.
118

Bibliography

JAIST[36] M Esperanza Manso, Marcela Genero, and Mario Piattini. No-redundant metrics for
UML class diagram structural complexity. In Lecture Notes on Comp. Sci., pages
127–142. Springer, 2003.
[37] Jacqueline A. Mc Quillan and James F. Power. A definition of the chidamber and kemerer metrics suite for the unified modeling language. Technical report, Departmentof Comp. Sci., Co. Kildare, Ireland, 2006.
[38] Jacqueline A. Mc Quillan and James F. Power. On the application of software metrics
to UML models. Models in Softw. Eng., pages 217–226, 2007.
[39] Frederick Mosteller and John W. Tukey. Data Analysis and Regression. AddisonWesley Publishing Company, USA, 1977.
[40] Nachiappan Nagappan, Laurie Williams, Mladen Vouk, and Jason Osborne. Early
estimation of software quality using in-process testing metrics: a controlled casestudy. In 3-WoSQ: Proc. of the third workshop on Softw. quality, pages 1–7, New
York, NY, USA, 2005. ACM.
[41] Ariadi Nugroho, Michel R. V. Chaudron, and Erik Arisholm. Assessing UML design
metrics for predicting fault-prone classes in a java system. In MSR, pages 21–30,
2010.
[42] Niclas Ohlsson and Hans Alberg. Predicting fault-prone software modules in telephone switches. IEEE Trans. Softw. Eng., 22(12):886–894, 1996.
[43] Hector M. Olague, Sampson Gholston, and Stephen Quattlebaum. Empirical validation of three software metrics suites to predict fault-proneness of object-orientedclasses developed using highly iterative or agile software development processes. IEEE
Trans. Softw. Eng., 33(6):402–419, 2007. Senior Member-Letha H. Etzkorn.
[44] Wendy W. Peng and Dolores R. Wallace. Software Error Analysis. U.S. Dept.
of Commerce, National Institute of Standards and Technology, Gaithersburg, MD,
USA, 1999.
[45] Geert Poels and Guido Dedene. Distance: a framework for software measure construction. Open access publications from katholieke universiteit leuven, Katholieke
Universiteit Leuven, 1999.
[46] Linda Rosenberg. Applying and interpreting object oriented metrics. In Software
Assurance Technology Conference.
[47] Linda Rosenberg. Applying and interpreting object oriented metrics. In Software
Technology Conference, Utah, USA, 1998.
[48] Adrian Schröter, Thomas Zimmermann, and Andreas Zeller. Predicting component
failures at design time. In ISESE ’06: Proc. of the 2006 ACM/IEEE Intl. symposiumon Empirical Softw. Eng., pages 18–27, New York, NY, USA, 2006. ACM.
119

Bibliography

JAIST[49] Subhash Sharma. Applied Multivariate Techniques. Addison-Wiley & Sons, Inc.,
USA, 1996.
[50] Rini van Solingen and Egon Berghout. The Goal/Question/Metric Method: A Practical Guide for Quality Improvement of Software Development. McGraw-Hill Publishing Company, England, UK, 1999.
[51] Ramanath Subramanyam and M.S. Krishnan. Empirical analysis of ck metrics forobject-oriented design complexity: Implications for software defects. IEEE Trans.
Softw. Eng., 29(4):297–310, 2003.
[52] Giancarlo Succi, Witold Pedrycz, Snezana Djokic, Paolo Zuliani, and Barbara Russo.
An empirical exploration of the distributions of the chidamber and kemerer objectoriented metrics suite. Empirical Softw. Eng., 10(1):81–104, 2005.
[53] Mei-Huei Tang and Mei-Hwa Chen. Measuring oo design metrics from UML. In UML’02: Proc. of the 5th Intl. Conference on The Unified Modeling Language, pages 368–
382, London, UK, 2002. Springer-Verlag.
[54] M. P. Ware, F. G. Wilkie, M. Shapcott, and N. G. Lester. The use of productmeasures in tracking code development to completion within small to medium sizedenterprises. In Proceedings of the 2008 Sixth International Conference on Software
Engineering Research, Management and Applications, pages 263–270, Washington,
DC, USA, 2008. IEEE Computer Society.
[55] Tong Yi and Fangjun Wu. Empirical analysis of entropy distance metric for UML
class diagrams. SIGSOFT Softw. Eng. Notes, 29(5):1–6, 2004.

120

Publications[1] Camargo Cruz Ana E., Ochimizu Koichiro, ”A UML Approximation of Three ChidamberKemerer Metrics and Their Ability to Predict Faulty Code across Software Projects”,
IEICE TRANSACTIONS on Information and Systems, Vol.E93-D,No.11,Nov. 2010,
pp. 3038-3050, Japan.
[2] Camargo Cruz Ana E., ”Exploratory Study of a UML Metric for Fault Prediction”,
Proceedings of the 32nd. International Conference on Software Engineering (ICSE2010), Vol. 2 pp. 361-364, Doctoral Symposium, May 2-8 2010, Cape Town, South
Africa.
[3] Camargo Cruz Ana E., Ochimizu Koichiro, ”Towards Logistic Regression Models for Predicting Fault-prone Code across Software Projects”, Proceedings of theThird International Symposium on Empirical Software Engineering and Measurement (ESEM 2009), IEEE Catalog Number: CFP09ENM, pp. 460-463, Oct 15th16th 2009, Lake Buena Vista, USA.
[4] Camargo Cruz Ana E., Ochimizu Koichiro, ”Quality prediction model for object
oriented software using UML metrics”, Proceedings of the 4th World Congress for
Software Quality, Septemeber 2008, Bethesda, Maryland, USA.

121

Appendix A
Theoretical Construction of our CK
metrics

In the field of empirical software engineering studies, most of the critiques concern
their internal and external validity. In our empirical study, these two types of validity
were correctly addressed for our prediction models according to [6]. On the other hand,
there exists another important aspect of validity for the metrics used, which is called construct validity. A metric exhibits construct validity when it measures what it is purported
to measure. When software metrics are defined using plain-language terms, problems canarise because they can be easily misinterpreted, resulting in other different measures tothe ones originally intended [45].
As complementary work of our empirical study, the contents of this Appendix haveas a main purpose to validate the construction of our CK UML metrics, for which the
DISTANCE framework is applied. The DISTANCE framework has been used to validate
the construction of metrics for UML class diagrams [22], and according to its authors, itis specially suited to satisfy the measurement needs of empirical research, offering boththe modeling of the software attributes of interest and the definition of the corresponding
software metrics, which altogether ensures the construct validity of the resultant metrics[45].
The framework is called DISTANCE as it builds upon the concepts of distance anddissimilarity. It models software attributes as conceptual distances between the software
entities they characterize and other software entities that serve as reference points [45].
Suppose that from a ’UML communication diagram’ we are interested to know its ’numberof different objects’. Being the attribute of interest the ’number of different objects’, thesoftware entities that can characterize such an attribute are precisely the ’objects’ of thegiven ’communication diagram’. And the obvious reference point is a ’communication diagram’ without ’objects’. Then, once identified the software entities that characterize theattribute of interest and the reference point, distances are measured using some functions
that are called ’metrics’ in mathematics.
Following the basic example provided by the authors of the DISTANCE frameworkiA.1. The DISTANCE Framework and Number of Objects

JAIST

1
0
0
1
0
1
11
00
00
11
00
11

Definition of study
goals

Evaluation of
our UML metrcis
as predictors of

Identification of
empirical hyptheses

faulty code&
Study deisgn

Evaluation of
our UML metrcis
as approximation of
the code CK metrics

Distance−based
measureconstruction

Data Collection andPreparation of
Data Colection

Analysis

Figure A.1: Distanced-based Measure Construction incorporated in our Empirical Study
approach in [45], we illustrate in Figure A.1 the how this distance-based measure construction can be incorporated into our empirical study. The procedures provided by the
DISTANCE framework for attribute modeling and measure construction, are embedded
into a goal-oriented approach. The resulting metrics then are further validated throughan empirical study.
In this Appendix, we validate theoretically the construction of our metrics, defined in
Chapter 3, using the DISTANCE Framework approach. First, we validate the constructionof the five direct metrics measured from communication diagrams in Table 3.3. Moreover,in order to ease the understanding of such construct validations and definitions, we modeland validate the construction of two more metrics (Number of Objects and Number of
Senders). Finally, we provide the means for modeling and validate the construction of
our second set of UML metrics for classes in Table 3.4.

A.1

The DISTANCE Framework and Number of Objects

This Section has as a main purpose the understanding of the application of the DISTANCE
framework approach to validate the construction of software metrics or measures. For
such purpose, we follow and apply its five basics steps to validate the construction of the’Number of Objects’ metric, defined in Chapter 3.
The DISTANCE Framework is called that way because it builds upon the conceptsii

A.1. The DISTANCE Framework and Number of Objects

JAIST

of distance and dissimilarity entities. Think of ’UML communication diagrams’ as software entities, and as the attribute of interest their ’numerousness of objects’. Software
attributes are modeled as conceptual distances between the software entities they characterize and other software entities that serve as reference points. The distances are thenmeasured by functions that are called ’metrics’ in mathematics. To avoid inconsistency
with the DISTANCE framework, in this section the term ’metric’ refers to such functions,and to refer to any of our UML metrics, the term ’measure’ is used instead.
The distance-based measure construction process consists of five basic steps. This
process is triggered by a request to find or build a measurement instrument for a software
attribute attr, such as the ’numerousness of objects’. The five basic steps to validate theconstruction of its measure ’Number of Objects’ NObjects are:
• Step 1: Find a measurement abstraction
Measurement abstractions are used in software measurement to emphasize an
attribute of interest, while simultaneously de-emphasizing other attributes.
Formally in this step, for the set of software entities P that are characterized
by an attribute attr, a set of software entities M must be found that can beused as measurement abstractions to emphasize attr and to define the mappingabs : P → M.
In our case, in order to find M let us say that:– P is the ’Universe of communication diagrams’ UCmD relevant to the Universeof Discourse UoD.
– attr, the attribute of interest, is the ’numerousness of objects’.
Moreover, we should consider that the ’numerousness of objects’ in a communication
diagram CmD within UCmD is determined by its ’set of objects’ SObjects(CmD).
And, if UObjects is the ’Universe of Objects’ relevant to UoD, then the ’set ofobjects’ of the ’communication diagram’ CmD is a subset of UObjects. The set
that contains all the ’sets of objects’ of the UCmD is the ’power set of UObjects’,denoted by ℘(UObjects).
Therefore, the set of software entities M that emphasize the ’numerousness of objects’ of set of software entities P is the ℘(UObjects). And the abstraction
function abs results in a projection function that maps a ’communication diagram’CmD onto its ’set of objects’ SObjects(CmD).
absN Objects : UCmD → ℘(UObjects) : CmD → SObjects(CmD)
Example: For the communication diagram A in Figure A.2, we have that :absN Objects (CmDA) = SObjects(CmDA)= {c, m, s, d}(A.1)

And as for the communication diagram A’ in the same Figure A.2, we have that :iii

A.1. The DISTANCE Framework and Number of Objects

JAIST

A5:result

A1: Mutiply(a,b)
A4:Display(a)
d

c
User
A2:result:= Multiply(a,b)
msA3:result:= Sum(a,b)

Diagram A

A1: Mutiply(a,b)
c
User
A2:result:= Multiply(a,b)
msA3:result:= Sum(a,b)

Diagram A’Figure A.2: Example of Different Two Communication Diagramsiv

A.1. The DISTANCE Framework and Number of Objects

absN Objects (CmDA′ ) = SObjects(CmDA′ )= {c, m, s}

JAIST

(A.2)

• Step 2: Model distances between measurement abstractions
Distances between the elements of M are modeled as sequences of elementary transformations. Such sequences represent a series of atomic changes applied to an element of M to arrive at another element of M. The number of atomic changes
that are required to transform one element into the other determines the distance
between these elements.
The formal outcome of this step is a set Te of elementary transformations on M.
Continuing with our example, because the elements of M are elements of ℘(UObjects), ’sets of objects’, Te must contain two elementary transformations: one for adding
an ’object’ to a set and one for removing an ’object’ from a set.
Given two ’sets of objects’ s1 and s2 , where s1 , s2 ∈ ℘(UObjects), s1 can always betransformed into s2 by removing first all objects from s1 that are not in s2 , and thenadding all objects to s1 that are not in s2 , but not in the original s1 . In the worst
imaginary case, s1 can be transformed into s2 , via an empty set of interactions.
Formally, Te = {t0N Objects , t1N Objects }, where:– t0N Objects : ℘(UObjects) → ℘(UObjects) : s → s ∪ {obj}, with obj ∈ UObjects– t1N Objects : ℘(UObjects) → ℘(UObjects) : s → s − {obj}, with obj ∈ UObjects
Example: The distance between absN Objects (CmDA) and absN Objects (CmDA′ ) canbe modeled as a sequence of one elementary transformation:
SObjects(CmDA) = {c, m, s, d}SObjects(CmDA′ ) = {c, m, s}= t1N Objects (SObjects(CmDA), d)(A.3)

This is example is represented in Figure A.3.
• Step 3: Quantify distances between measurement abstractions
A metric δ : M × M → ℜ is defined to quantify the distances between the elementsof M. Formally, this step results in the definition of a metric space (M, δ).
In our case, to quantify the distances between elements of M, or ℘(UObjects),
we can define the metric δN Objects by using the symmetric difference model. The
symmetric difference model can always be used to define a metric when the set ofmeasurement abstractions is a power set. Therefore, δN Objects is:
δN Objects : ℘(UObjects) × ℘(UObjects) → ℜ : (s, s′ ) → (|s − s′ | + |s′ − s|)This definition is equivalent to state that the distance between two ’sets of objects’is measured by the count of elementary transformations in the shortest sequence ofelementary transformations between these two sets.
v

A.1. The DISTANCE Framework and Number of Objects
Objects of CmDA’

Objects of CmDA

c

m

s

d

c

s

m
s

{d}

Objects of CmDA
c

JAIST

m
d

t 1NObjects (SObjects(CmDA),{d})= SObjects(CmDA’)

Figure A.3: Transformation of SObjectes(CmDA) → SObjects(CmDA′ )Notice that for any element in s but not in s′ and for any element in s′ but not ins, an elementary transformation is needed.
Example: Applying the symmetric difference model, the distance between the setof objects of CmDA and CmDA′ equals 1:
δN Objects (absN Objects (CmDA), absN Objects (CmDA′ )) =|{c, m, s, d} − {c, m, s}| + |{c, m, s} − {c, m, s, d}|=|{d}| + |∅|
= 1

(A.4)

• Step 4: Find a reference abstraction
We need to determine what the model of a software in P must look like in case that
entity is characterized by the theoretical lowest value of attr. This hypothetical’null’ model or reference model can then be used as reference point or norm formeasurement.
The result of this step is thus the definition of a function ref that returns for eachsoftware entity in P a reference abstraction for attr in M; ref : P → M.
For our example, the obvious reference point for measurement is the ’empty setof objects’, which means that a communication diagram without ’objects’ is thelowest possible value for the NObjects measure. Therefore, we define the following
function:refN Objects : UCmD → ℘(UObjects) : CmD → ∅• Step 5. Define the software measure
The results of the previous steps are combined to produce a formal definition of thesoftware attribute attr and its measure.
vi

A.2. For an Object: Number of Received Call Messages

JAIST

The extent to which attr characterizes a software entity p ∈ P is defined by the
distance between the actual model of p for attr (i.e. abs(p)) and the reference model
for attr (i.e. ref (p)). The larger this distance is, the more the actual measurementof abstraction differs from the norm that has been set, and thus the greater theextent to which attr characterizes p. Hence, the value of attr for p is the value
returned by the metric δ for the pair (abs(p), ref (p)).
The formal outcome of this step is the measure µ : P → ℜ defined such that∀p ∈ P : µ(p) = δ(abs(p), ref (p)).
For our example, the ’numerousness of objects’ of a ’communication diagram’ CmD
can be defined as the distance between its ’set of objects’, SObjects(CmD), andthe ’empty set of objects’, ∅. Therefore, its measure ’Number of Objects’ NObjects
can be defined as a function that returns the value of the metric δN Objects for thepair of sets SObjects(CmD) and ∅ for any communication diagram.
∀CmD ∈ UCmD : NObjects(CmD)=
δN Objects (SObjects(CmD), ∅)=|SObjects(CmD) − ∅| + |∅ − SObjects(CmD)| =|SObjects(CmD)|(A.5)

Consequently, a measure that returns the count of ’objects’ of a communication
diagram qualifies as a measure of the ’numerousness of objects’.
Example: The ’Number of Objects’ of the ’communication diagram A’ CmDA in
Figure A.2 equals 4:
δN Objects (SObjects(CmDA), ∅) =|{c, m, s, d} − ∅| +|∅ − {c, m, s, d}|=|{c, m, s, d}| + |∅|
=
4

(A.6)

The details underlying the theoretical principles of the DISTANCE framework canfound in [45].

A.2

For an Object: Number of Received Call Messages• Step 1: Find a measurement abstraction
The set of software entities modeled is the ’Universe of Objects’ within a communication diagram, UObjects, that are relevant to the Universe of Discourse UoD.
The attribute of interest are: The numerousness of Received Call Messages.
A call message can be distinguished from another, if its sequence expression of its
label are different, e.g. the call message A1 : print() is different to the call message
A2 : print().
vii

A.2. For an Object: Number of Received Call Messages

JAIST

Let be UObjects as the ’Universe of Objects’ relevant to UoD. All the ’sets ofReceived Call Messages’ of an object of UObjects are elements of the ’power set of
URCMsgs’, denoted by ℘(URCMsgs).
Therefore, the set of software entities that can be used as measurement abstractionsis ℘(URCMsgs), and the abstraction function abs results in a projection functionthat maps ’objects’ onto their ’sets of Received Call Messages’:
absN RCM sgs : UObjects → ℘(URCMsgs) : Obj → SRCMsgs(Obj)A9:result
A5:b
A1: Mutiply(a,b)
A2:Display(a)
d

c

A3:a

A4:Display(b)User

A8:Display(result)A6:result:= Multiply(a,b)
msA7:result:= Sum(a,b)

Diagram C

Figure A.4: Example of Different Two Communication Diagrams
Example: For the communication diagram C in Figure A.4, we have that :absN RCM sgs (c) = SRCMsgs(c)= {A1}(A.7)

absN RCM sgs (m) = SRCMsgs(m)= {A6}(A.8)

absN RCM sgs (s) = SRCMsgs(s)= {A7}(A.9)

absN RCM sgs (d) = SRCMsgs(d)= {A2, A4, A8}(A.10)• Step 2: Model distances between measurement abstraction
As explained previously, in our first validation, since the elements of ℘(URCMsgs)
are ’sets of Received Call Messages’, Te must contain two elementary transformations: one for adding a ’Received Call Messages’ to a set and one for removing a’Received Call Message’ from a set. Formally, Te = {t0N RCM sgs , t1N RCM sgs }, where:– t0N RCM sgs : ℘(URCMsgs) → ℘(URCMsgs) : s → s ∪ {rcmsg}, with rcmsg ∈URCMsgsviii

A.2. For an Object: Number of Received Call Messages

JAIST– t1N RCM sgs : ℘(URCMsgs) → ℘(URCMsgs) : s → s − {rcmsg}, with rcmsg ∈URCMsgs
Example: The distance between absN RCM sgs (c) and absN RCM sgs (d) can be modeled
as follows. Given that:
SRCMsgs(c) = {A1}
SRCMsgs(d) = {A2, A4, A8}(A.11)

Recalling that for any element in SRCMsgs(c) and not in SRCMsgs(d) and viceversa, an elementary transformation is needed, the sequence of elementary transformations needed to model the distance between absN RCM sgs (c) and absN RCM sgs (d)are equal to 4:

SRCMsgs(d)=t0N RCM sgs (t0N RCM sgs (t0N RCM sgs (t1N RCM sgs (SRCMsgs(c), A1), A2), A4), A8)(A.12)• Step 3: Quantify distances between measurement abstractions
The function δN RCM sgs that models the distances in ℘(URCMsgs) is defined as:
δN RCM sgs : ℘(URCMsgs) × ℘(URCMsgs) → ℜ : (s, s′ ) → (|s − s′ | + |s′ − s|)
Example: Continuing with our previous example, applying the symmetric difference model, the distance between the ’sets of Received Call Messages’ of c and dequals 4:
δN RCM sgs (absN RCM sgs (c), absN RCM sgs (d))=|{A1} − {A2, A4, A8}| + |{A2, A4, A8} − {A1}| =|{A1}| + |{A2, A4, A8}|
= 4

(A.13)• Step 4: Find a reference abstraction
The obvious reference point for measurement is the ’empty set of received call messages’, which means that an ’object’ without ’received call messages’ is the lowest
possible value for the NRCMsgs measure. Therefore, the reference abstraction
function can be defined as:
refN RCM sgs : UObjects → ℘(URCMsgs) : Obj → ∅• Step 5. Define the software measure.
The ’numerousness of Received Call Messages’ of a given ’object’ Obj, can be definedas the distance between its ’set of received call messages’ SRCMsgs(Obj), and the’empty set of received call messages’ ∅. Therefore, its measure ’Number of Received
Call Messages’ NRCMsgs can be defined as a function that returns the value ofthe metric δN RCM sgs for the pair of sets SRCMsgs(object) and ∅ for any object.

ix

A.3. For an Object: Number of Received Call Messages

JAIST∀Obj ∈ UObjects : NRCMsgs(Obj)=δN RCM sgs (SRCMsgs(Obj), ∅)=|SRCMsgs(Obj) − ∅| + |∅ − SRCMsgs(Obj)| =|SRCMsgs(Obj)|(A.14)

Consequently, a measure that returns the count of ’received call messages’ of an’object’ qualifies as a measure of the ’numerousness of Received Call Messages’.
Example: The ’Number of Received Call Messages’ of object d in diagram C of
Figure A.4 equals 3:
δN RCM sgs (SRCMsgs(d), ∅) =|{A2, A4, A8} − ∅| +|∅ − {A2, A4, A8}|=|{A2, A4, A8}| + |∅|
=
3

A.3

(A.15)For an Object: Number of Received Call Messages• Step 1: Find a measurement abstraction
The set of software entities modeled is the ’Universe of Objects’ in a communication
diagram, UObjects, that are relevant to the Universe of Discourse UoD.
The attribute of interest are: the numerousness of Different Received Call Messages.
A call message can be distinguished from another, if their names are different, e.g.
the call message A1 : print() is exactly the same as the call message A2 : print().
Let be UObjects as the ’Universe of Objects’ relevant to UoD. All the ’sets ofDifferent Received Call Messages’ of an object of UObjects are elements of thepower set of UDRCMsgs, denoted by ℘(UDRCMsgs).
Therefore, the set of software entities that can be used as measurement abstractionsis ℘(UDRCMsgs), and the abstraction function abs results in a projection functionthat maps ’objects’ onto their ’sets of Different Received Call Messages’:
absN DRCM sgs : UObjects → ℘(UDRCMsgs) : Obj → SDRCMsgs(Obj)
Example: For the communication diagram C in Figure A.4, we have that :absN DRCM sgs (c) = SDRCMsgs(c)= {A1}(A.16)

absN DRCM sgs (m) = SDRCMsgs(m)= {A6}(A.17)xA.3. For an Object: Number of Received Call Messages

JAIST

absN DRCM sgs (s) = SDRCMsgs(s)= {A7}(A.18)

absN DRCM sgs (d) = SDRCMsgs(d)= {A2 = A4 = A8}(A.19)

Notice the difference between the absN DRCM sgs (d) and absN RCM sgs (d) defined in theprevious Section.
• Step 2: Model distances between measurement abstraction
Just as in the validation of the construction of our previous measures, since theelements of ℘(UDRCMsgs) are ’sets of Different Received Call Messages’, Te must
contain two elementary transformations: one for adding a ’Different Received Call
Messages’ to a set and one for removing a ’Different Received Call Message’ from aset. Formally, Te = {t0N DRCM sgs , t1N DRCM sgs }, where:– t0N DRCM sgs : ℘(UDRCMsgs) → ℘(UDRCMsgs) : s → s ∪ {drcmsg}, withdrcmsg ∈ UDRCMsgs– t1N DRCM sgs : ℘(UDRCMsgs) → ℘(UDRCMsgs) : s → s − {drcmsg}, withdrcmsg ∈ UDRCMsgs
Example: The distance between absN DRCM sgs (c) and absN DRCM sgs (m) can be modeled as follows. Given that:
SDRCMsgs(c) = {A1}
SDRCMsgs(m) = {A6}

(A.20)

Recalling that for any element in SDRCMsgs(c) and not in SDRCMsgs(m) andvice-versa, an elementary transformation is needed, the sequence of elementarytransformations needed to model the distance between absN DRCM sgs (c) and absN DRCM sgs (m)equals 2.
SDRCMsgs(m)=t0N DRCM sgs (t1N DRCM sgs (SDRCMsgs(c), A1), A6)(A.21)• Step 3: Quantify distances between measurement abstractions
The function δN DRCM sgs that models the distances in ℘(UDRCMsgs) is:δN DRCM sgs : ℘(UDRCMsgs) × ℘(UDRCMsgs) → ℜ : (s, s′ ) → (|s − s′ | + |s′ − s|)
Example: Continuing with our previous example, applying the symmetric difference model, the distance between the ’sets of Different Received Call Messages’ ofc and m equals 2:
δN DRCM sgs (absN DRCM sgs (c), absN DRCM sgs (m)) =|{A1} − {A6}| + |{A6} − {A1}|=|{A1}| + |{A6}|= 2
xi(A.22)

A.4. For a Communication Diagram: Number of Senders

JAIST• Step 4: Find a reference abstraction
The obvious reference point for measurement is the empty set of ’different received
call messages’, which means that an ’object’ without ’different received call messages’ is the lowest possible value for the NDRCMsgs measure. Therefore, thereference abstraction function can be defined as:
refN DRCM sgs : UObjects → ℘(UDRCMsgs) : Obj → ∅• Step 5. Define the software measure.
The ’numerousness of Different Received Call Messages’ of a given ’object’, Obj,
can be defined as the distance between its ’set of different received call messages’,
SDRCMsgs(Obj), and the ’empty set of different received call messages’, ∅. Therefore, its measure ’Number of Different Received Call Messages’ NDRCMsgs canbe defined as a function that returns the value of the metric δN DRCM sgs for the pairof sets SDRCMsgs(Obj) and ∅ for any object.
∀Obj ∈ UObjects : NDRCMsgs(Obj)δN DRCM sgs (SDRCMsgs(Obj), ∅)=|SDRCMsgs(Obj) − ∅| + |∅ − SDRCMsgs(Obj)| =|SDRCMsgs(Obj)|(A.23)

Consequently, a measure that returns the count of ’different received call messages’of an ’object’ qualifies as a measure of the ’numerousness of Different Received Call
Messages’.
Example: The ’Number of Different Received Call Messages’ d object equals 1:
δN DRCM sgs (SRCMsgs(d), ∅) =|{A2 = A4 = A8} − ∅| +|∅ − {A2 = A4 = A8}|=|{A2 = A4 = A8}| + |∅|=1(A.24)

Notice the difference between this metric and the δN RCM sgs for the same object in theprevious Section.

A.4

For a Communication Diagram: Number of Senders• Step 1: Find a measurement abstraction
The set of software entities modeled is the ’Universe of Communication Diagrams’,
UCmD, that are relevant to the Universe of Discourse UoD.
The attribute of interest are: The numerousness of objects or actors that act assenders.

xii

A.4. For a Communication Diagram: Number of Senders

JAIST

Let be USenders as the ’Universe of Sender Objects’ relevant to UoD. All the ’sets
of senders’ within the UCmD are elements of the ’power set of USenderes’, denotedby ℘(USenders).
Therefore, the set of software entities that can be used as measurement abstractionsis ℘(USenders), and the abstraction function abs results in a projection functionthat maps ’communication diagrams’ onto their ’sets of senders’:
absN Senders : UCmD → ℘(USenders) : CmD → SSenders(CmD)
Example: For the communication diagram A in Figure A.2, we have that :absN Senders (CmDA) = SSenders(CmDA)= {User, c, m, d}(A.25)

And as for the communication diagram A’ in the same Figure A.2, we have that :absN Senders (CmDA′ ) = SSenders(CmDA′ )= {User, c, m}(A.26)• Step 2: Model distances between measurement abstraction
Just as in the construction validation of our previous measures, since the elements of℘(USenders) are ’sets of senders’, Te must contain two elementary transformations:
one for adding a ’sender’ to a set and one for removing a ’sender’ from a set.
Formally, Te = {t0N Senders , t1N Senders }, where:– t0N Senders : ℘(USenders) → ℘(USenders) : s → s ∪ {sender}, with sender ∈USenders– t1N Senders : ℘(USenders) → ℘(USenders) : s → s − {sender}, with sender ∈USenders
Example: The distance between absN Senders (CmDA) and absN Senders (CmDA′ ) in
Figure A.2 can be modeled as follows:
SSenders(CmDA) = {User, c, m, d}
SSenders(CmDA′ ) = {User, c, m}= t1N Senders (SSenders(CmDA), d)(A.27)

Therefore, the number of elementary transformations needed to model the distance
between absN Senders (CmDA) and absN Senders (CmDA′ ) equals 1.
• Step 3: Quantify distances between measurement abstractions
The function δN Senders that models the distances in ℘(USenders) is:δN Senders : ℘(USenders) × ℘(USenders) → ℜ : (s, s′ ) → (|s − s′ | + |s′ − s|)
Example: Continuing with our previous example, applying the symmetric difference model, the distance between the ’sets of senders’ of CmDA and CmDA′ equals
1:xiii

A.5. For an Object: The Number of Sent Call Messages

δN Senders (absN Senders (CmDA), absN Senders (CmDA′ ))=|{User, c, m, d} − {User, c, m}| + |{User, c, m} − {User, c, m, d}| =|{d}| + |∅|
= 1

JAIST

(A.28)• Step 4: Find a reference abstraction
The obvious reference point for measurement is the ’empty set of senders’, whichmeans that a communication diagram without ’senders’ is the lowest possible value
for the NSenders measure. Therefore, the reference abstraction function can be
defined as:
refN Senders : UCmD → ℘(USenders) : CmD → ∅• Step 5. Define the software measure.
The ’numerousness of Senders’ in a communication diagram, CmD, can be definedas the distance between its ’set of senders’, SSenders(CmD), and the ’empty set of
senders’, ∅. Therefore, its measure ’Number of Senders’ NSenders can be defined
as a function that returns the value of the metric δN Senders for the pair of sets
SSenders(CmD) and ∅ for any communication diagram.
∀CmD ∈ UCmD : NSenders(CmD)=
δN Senders (SSenders(CmD), ∅)=|SSenders(CmD) − ∅| + |∅ − SSenders(CmD)| =|SSenders(CmD)|(A.29)

Consequently, a measure that returns the count of ’senders’ in a communication
diagram qualifies as a measure of the ’numerousness of Senders’.
Example: The ’Number of Senders’ in communication diagram A, CmDA, in Figure
A.2 equals 4:
δN Senders (SSenders(mDA), ∅) =|{User, c, m, d} − ∅| +|∅ − {User, c, m, d}|=|{d}| + |∅|
=
4

A.5

(A.30)For an Object: The Number of Sent Call Messages• Step 1: Find a measurement abstraction
The set of entities modeled is the ’Universe of Senders’ (objects or actors), USenders,in a communication diagram that are relevant to the Universe of Discourse UoD.
The attribute of interest is: The numerousness of Sent Call Messages.
xivA.5. For an Object: The Number of Sent Call Messages

JAIST

A call message can be distinguished from another, if its sequence expression of its
label are different, e.g. the call message A1 : print() is different to the call message
A2 : print().
Let be USCMsgs as the ’Universe of Sent Call Messages’ relevant to UoD. All the’sets of Sent Call Messages’ within the USenders are elements of the ’power set of
USCMsgs’, denoted by ℘(USCMsgs).
Therefore, the set of software entities that can be used as measurement abstractionsis ℘(USCMsgs), and the abstraction function abs results in a projection functionthat maps ’senders’, objects or actors, onto their ’sets of Sent Call Messages’:
absN SCM sgs : USenders → ℘(USCMsgs) : sender → SSCMsgs(sender)
Example: For the communication diagram C in Figure A.4, we have that :absN SCM sgs (User) = SSCMsgs(User)= {A1}(A.31)

absN SCM sgs (c) = SSCMsgs(c)= {A2, A4, A6, A8}(A.32)

absN SCM sgs (m) = SSCMsgs(m)= {A7}(A.33)

absN SCM sgs (s) = SSCMsgs(s)= ∅(A.34)

absN SCM sgs (d) = SSCMsgs(d)= ∅(A.35)

Notice that messages A3, A5 and A9 are not call messages.
• Step 2: Model distances between measurement abstraction
Since the elements of ℘(USCMsgs) are ’sets of senders’, Te must contain two elementary transformations: one for adding a ’Sent Call Message’ to a set and one forremoving a ’Sent Call Message’ from a set. Formally, Te = {t0N SCM sgs , t1N SCM sgs },where:– t0N SCM sgs : ℘(USCMsgs) → ℘(USCMsgs) : s → s ∪ {scmsg}, with scmsg ∈USCMSgs– t1N SCM sgs : ℘(USCMsgs) → ℘(USCMsgs) : s → s − {scmsg}, with scmsg ∈USCMsgs
Example: The distance between absN SCM sgs (c) and absN SCM sgs (s) can be modeled
as follows:xv

A.5. For an Object: The Number of Sent Call Messages

JAIST

SSCMsgs(s) = ∅SSCMsgs(c) = {A2, A4, A6, A8}= t0N SCM sgs (t0N SCM sgs (t0N SCM sgs (t0N SCM sgs (SSCMsgs(s), A2), A4), A6), A8)(A.36)
Therefore, a total of 4 elementary transformations are sufficient to model the distance between absN SCM sgs (c) and absN SCM sgs (s).
• Step 3: Quantify distances between measurement abstractions
The function δN SCM sgs that models the distances in ℘(USCMsgs) is:δN SCM sgs : ℘(USCMsgs) × ℘(USCMsgs) → ℜ : (s, s′ ) → (|s − s′ | + |s′ − s|)
Example: Continuing with our previous example, applying the symmetric difference model, the distance between the sets of Sent Call Messages of s and d equals
4:
δN SCM sgs (absN SCM sgs (s), absN SCM sgs (d))=|∅ − {A2, A4, A6, A8}| + |{A2, A4, A6, A8} − ∅| =|∅| + |{A2, A4, A6, A8}|
= 4

(A.37)• Step 4: Find a reference abstraction
The obvious reference point for measurement is the empty set of ’Sent Call Messages’, which means that a sender without ’Sent Call Messages’ is the lowest possible
value for the NSCMsgs measure. Therefore, the reference abstraction function canbe defined as:
refN SCM sgs : USenders → ℘(USCMsgs) : sender → ∅• Step 5. Define the software measure.
The ’numerousness of Sent Call Messages’ of a ’sender’, sender, can be defined as
the distance between its ’set of Sent Call Messages’, SSenders(sender), and the’empty set of Sent Call Messages’, ∅. Therefore, its measure ’Number of Sent Call
Messages’ NSCMsgs can be defined as a function that returns the value of themetric δN SCM sgs for the pair of sets SSCMsgs(sender) and ∅:∀sender ∈ USenders : NSCMsgs(sender)
δN SCM sgs (SSCMsgs(sender), ∅)=|SSCMsgs(sender) − ∅| + |∅ − SSCMsgs(sender)| =|SSCMsgs(sender)|(A.38)

Consequently, a measure that returns the count of ’Sent Call Messages’ of a ’sender’in a communication diagram qualifies as a measure of the ’numerousness of Sent
Call Messages’.
Example: The ’Number of Sent Call Messages’ of c in diagram C of Figure A.4
equals 4:xvi

A.6. For an Object: The Number of Different Sent Call Messages

JAISTδN SCM sgs (SSCMsgs(c), ∅) =|{A2, A4, A6, A8} − ∅| +|∅ − {A2, A4, A6, A8}|=|{A2, A4, A6, A8}| + |∅|
=
4

A.6

(A.39)For an Object: The Number of Different Sent
Call Messages• Step 1: Find a measurement abstraction
The set of entities modeled is the ’Universe of Senders’, actors or objects, USenders,that are relevant to the Universe of Discourse UoD.
The attribute of interest is: The numerousness of Different Sent Call Messages.
A call message can be distinguished from another, if their names are different, e.g.
the call message A1 : print() is exactly the same as the call message A2 : print().
Let be UDSCMsgs as the ’Universe of Different Sent Call Messages’ relevant to
UoD. All the ’sets of Different Sent Call Messages’ within the USenders are elements of the power set of UDSCMsgs, denoted by ℘(UDSCMsgs).
Therefore, the set of software entities that can be used as measurement abstractionsis ℘(UDSCMsgs), and the abstraction function abs results in a projection function that maps ’senders’, objects or actors, onto their ’sets of Different Sent Call
Messages’:
absN DSCM sgs : USenders → ℘(UDSCMsgs) : sender → SDSCMsgs(sender)
Example: For the communication diagram C in Figure A.4, we have that :absN DSCM sgs (User) = SDSCMsgs(User)= {A1}(A.40)

absN DSCM sgs (c) = SDSCMsgs(c)= {A2 = A4 = A8, A6}(A.41)

absN DSCM sgs (m) = SDSCMsgs(m)= {A6}(A.42)

absN DSCM sgs (s) = SDSCMsgs(s)= ∅(A.43)

absN DSCM sgs (d) = SDSCMsgs(d)= ∅(A.44)

Notice the difference between absN DSCM sgs (c) and absN DSCM sgs (c) defined in theprevious Section.
xvii

A.6. For an Object: The Number of Different Sent Call Messages

JAIST• Step 2: Model distances between measurement abstraction
Since the elements of ℘(UDSCMsgs) are ’sets of senders’, Te must contain twoelementary transformations: one for adding a ’Different Sent Call Message’ to a
set and one for removing a ’Different Sent Call Message’ from a set. Formally,
Te = {t0N DSCM sgs , t1N DSCM sgs }, where:– t0N DSCM sgs : ℘(UDSCMsgs) → ℘(UDSCMsgs) : s → s ∪ {dscmsg}, withdscmsg ∈ UDSCMSgs– t1N DSCM sgs : ℘(UDSCMsgs) → ℘(UDSCMsgs) : s → s − {dscmsg}, withdscmsg ∈ UDSCMsgs
Example: The distance between absN DSCM sgs (c) and absN DSCM sgs (s) can be modeled by the following sequence of elementary transformations:SDSCMsgs(s) = ∅SDSCMsgs(c) = {A2 = A4 = A8, A6}= t0N DSCM sgs (t0N DSCM sgs (SDSCMsgs(s), A2), A6)(A.45)

Therefore, a sequence of 2 elementary transformations is sufficient to model the
distance between absN DSCM sgs (c) and absN DSCM sgs (s).
• Step 3: Quantify distances between measurement abstractions
The function δN DSCM sgs that models the distances in ℘(UDSCMsgs) is:δN DSCM sgs : ℘(UDSCMsgs) × ℘(UDSCMsgs) → ℜ : (s, s′) → (|s − s′ | + |s′ − s|)
Example: Continuing with our previous example, applying the symmetric difference model, the distance between the sets of Different Sent Call Messages of s andc equals 2:
δN DSCM sgs (absN DSCM sgs (s), absN DSCM sgs (c)) =|∅ − {A2, A6}| + |{A2, A6} − ∅|=|∅| + |{A2, A6}|
= 2

(A.46)• Step 4: Find a reference abstraction
The obvious reference point for measurement is the ’empty set of Different Sent
Call Messages’, which means that a sender without ’Different Sent Call Messages’is the lowest possible value for the NDSCMsgs measure. Therefore, the reference
abstraction function can be defined as follows:
refN DSCM sgs : USenders → ℘(UDSCMsgs) : sender → ∅• Step 5. Define the software measure.
The ’numerousness of Different Sent Call Messages’ of a ’sender’, sender, can be defined as the distance between its ’set of Different Sent Call Messages’, SDSCMsgs(sender),xviii

A.7. For a Sender Object: The Number of Different Receiver Objects

JAIST

and the ’empty set of Different Sent Call Messages’, ∅. Therefore, its measure ’Number of Different Sent Call Messages’ NDSCMsgs can be defined as a function that
returns the value of the metric δN DSCM sgs for the pair of sets SDSCMsgs(sender)and ∅:∀sender ∈ USenders : NDSCMsgs(sender)
δN DSCM sgs (SDSCMsgs(sender), ∅)=|SDSCMsgs(sender) − ∅| + |∅ − SDSCMsgs(sender)| =|SDSCMsgs(sender)|(A.47)

Consequently, a measure that returns the count of ’Different Sent Call Messages’ of
a ’sender’ in a communication diagram qualifies as a measure of the ’numerousness
of Different Sent Call Messages’.
Example: The ’Number of Different Sent Call Messages’ of c in diagram C of
Figure A.4 equals 2:
δN DSCM sgs (SDSCMsgs(c), ∅) =|{A2, A6} − ∅| +|∅ − {A2, A6}|=|{A2, A6}| + |∅|
=
2

A.7

(A.48)For a Sender Object: The Number of Different
Receiver Objects• Step 1: Find a measurement abstraction
The set of entities modeled is the ’Universe of senders’, actors or objects, USenders,that are relevant to the Universe of Discourse UoD.
The attribute of interest is: The numerousness of Receiver Objects.
Let be UReceivers as the ’Universe of receiver’ objects relevant to UoD. All the ’setsof receivers’ within the USenders are elements of the ’power set of UReceivers’,
denoted by ℘(UReceivers).
Therefore, the set of software entities that can be used as measurement abstractionsis ℘(UReceivers), and the abstraction function abs results in a projection functionthat maps ’senders’, objects or actors, onto their ’sets of receivers’:
absN Receivers : USenders → ℘(UReceivers) : sender → SReceivers(sender)
Example: For the communication diagram C in Figure A.4, we have that :absN Receivers (User) = SReceivers(User)
= {c}

xix

(A.49)

A.7. For a Sender Object: The Number of Different Receiver Objects

JAIST

absN Receivers (c) = SReceivers(c)= {m, d}(A.50)

absN Receivers (m) = SReceivers(m)= {s}(A.51)

absN Receivers (s) = SReceivers(s)= ∅(A.52)

absN Receivers (d) = SReceivers(d)= ∅(A.53)• Step 2: Model distances between measurement abstraction
Since the elements of ℘(UReceivers) are ’sets of receiver’ objects, Te must containtwo elementary transformations: one for adding a ’receiver’ to a set and one forremoving a ’receiver’ from a set. Formally, Te = {t0N Receivers , t1N Receivers }, where:– t0N Receivers : ℘(UReceivers) → ℘(UReceivers) : s → s ∪ {receiver}, withreceiver ∈ UReceivers– t1N Receivers : ℘(UReceivers) → ℘(UReceivers) : s → s − {receiver}, withreceiver ∈ UReceivers
Example: The distance between absN Receivers(c) and absN Receivers(s) in diagram C of
Figure A.4 can be modeled by the following sequence of elementary transformations:
SReceivers(s) = ∅SReceivers(c) = {m, d}= t0N Receivers (t0N Receivers (SReceivers(s), m), d)(A.54)

Therefore, a sequence of 2 elementary transformations are needed to model the
distance between absN Receivers(c) and absN Receivers(s) .
• Step 3: Quantify distances between measurement abstractions
The function δN Receivers that models the distances in ℘(UReceivers) is:δN Receivers : ℘(UReceivers) × ℘(UReceivers) → ℜ : (s, s′ ) → (|s − s′ | + |s′ − s|)
Example: Continuing with our previous example, applying the symmetric difference model, the distance between the ’sets of receivers’ of s and c equals 2:
δN Receivers (absN Receivers (s), absN Receivers (c)) =|∅ − {m, d}| + |{m, d} − ∅|=|∅| + |{m, d}|= 2

xx(A.55)

A.7. For a Sender Object: The Number of Different Receiver Objects

JAIST• Step 4: Find a reference abstraction
The obvious reference point for measurement is the ’empty set of receivers’, whichmeans that a ’sender’ without ’receivers’ is the lowest possible value for the NReceivers
measure. Therefore, the reference abstraction function can be defined as follows:
refN Receivers : USenders → ℘(UReceivers) : sender → ∅• Step 5. Define the software measure.
The ’numerousness of Receivers’ of a ’sender’ sender can be defined as the distance between its ’set of receivers’, SReceivers(sender) and the ’empty set of receivers’, ∅. Therefore, its measure ’Number of Receivers’ NReceivers can be defined as a function that returns the value of the metric δN Receivers for the pair of sets
SReceivers(sender) and ∅:∀sender ∈ USenders : NReceivers(sender)=
δN Receivers (SReceivers(sender), ∅)=|SReceivers(sender) − ∅| + |∅ − SReceivers(sender)| =|SReceivers(sender)|(A.56)

Consequently, a measure that returns the count of ’receiver objects’ of a ’sender’ ina ’communication diagram’ qualifies as a measure of its ’numerousness of Receivers’.
Example: The ’Number of Receivers’ of c in diagram C of Figure A.4 equals 2:
δN Receivers (SReceivers(c), ∅) =|{m, d} − ∅| +|∅ − {m, d}|=|{m, d}| + |∅| + |{m, d}|=2(A.57)

As we could observe, through all the Chapter, all of the validated measures can bemodeled by means of its abstraction function and. Tables A.1 and A.2 summarize theabstract functions of the proposed measures for UML communication diagrams. Furthermore, we also provide the abstract functions for the UML class metrics as defined in
Chapter 3. Such functions are listed in Tables A.3 and A.4, which in their turn serve us
to approximate the RFC, CBO and WMC CK metrics, as indicated in Table 3.4.

xxi

A.7. For a Sender Object: The Number of Different Receiver Objects

JAIST

Table A.1: Abstract Functions to Model Direct Metrics for Objects
MeasureAbstract Function

NObjects (Number of Objects)

absN Objects : U CmD → ℘(U Objects) : CmD → SObjects(CmD)
where:• N Objects is the Universe of objects in a communication diagram
relevant to the UoD U CmD,
• U CmD is the Universe of Communication Diagrams,• SObjects(CmD) ( U Objects is the set of objects of a communication diagram.

NRCMsgs (Number of Received
Call Messages)

absN RCM sgs : U Objects → ℘(U RCM sgs) : obj → SRCM sgs(obj)
where :
• U RCM sgs is the Universe of Received Call Messages relevant to the
UoD U Objects,• U Objects is the Universe of of Objects within a Communication Diagram,• SRCM sgs(obj) ( U RCM sgs is the set of received call messages ofan object, obj.

NDRCMsgs(Number of Different Received
Call Messages)

absN DRCM sgs : U Objects → ℘(U DRCM sgs) : obj → SDRCM sgs(obj)
where:• U DRCM sgs is the Universe of Different Received Call Messages relevant to the UoD U Objects,• U Objects is the Universe of Objects within a Communication Diagram• SDRCM sgs(obj) ( U DRCM sgs is the set of different received callmessages of an object, obj.

xxii

A.7. For a Sender Object: The Number of Different Receiver Objects

JAIST

Table A.2: Abstract Functions to Model Direct Metrics for Objects
MeasureAbstract Function

NSenders (Number of Senders)

absN Senders : U CmD → ℘(U Senders) : CmD → SSenders(CmD)
where:• U Senders is the Universe of objects or actors acting as senders in acommunication diagram relevant to the UoD U cmD,
• U CmD is the Universe of Communication Diagrams.
• SSenders(CmD) ( U Senders is the set of senders of a communication diagram.

NSCMsgs (Number of Sent Call
Messages)

absN SCM sgs : U Senders → ℘(U SCM sgs) : sndr → SSCM sgs(sndr)
where:NDSCMsgs(Number of Different Sent Call
Messages)

absN DSCM sgs: U Senders
SDSCM sgs(sndr)
where:

NDRO (Number
of Different Receiver Objects)

absN Receivers : U Senders → ℘(U Receivers) : sndr → SReceivers(sndr)
where:• U SCM sgs is the Universe of Sent Call Messages relevant to the UoD
U Senders,• U Senders is the Universe of objects or actors acting as senders within
a communication diagram,• SSCM sgs(sndr) ( U SCM sgs is the set of sent call messages of asender sndr.
→℘(U DSCM sgs)

:

sndr

→

• U DSCM sgs is the Universe of Different Sent Call Messages relevant
to an UoD U Senders,• U Senders is the Universe of objects or actors acting as senders within
a communication diagram,• SDSCM sgs(sndr) ( U DSCM sgs is the set of different sent callmessages of a sender sndr.

• U Receivers is the Universe of the different objects acting as receivers
for a sender to an UoD U Senders,• U Senders is the Universe of objects or actors acting as senders within
a communication diagram,• SReceiverss(sndr) ( U Receivers is the set of receiver objects of asender sndr.

xxiii

A.7. For a Sender Object: The Number of Different Receiver Objects

JAIST

Table A.3: Abstract Functions to Model Metrics for Classes
MeasureAbstract Function

NClasses (Number of Classes)

absN Classes : U SysDsgs → ℘(U Objects) : SysDsg → SObjects(SysDsg)
where:• N Classes is the Universe of objects of a system design relevant to
the UoD U SysDsgs,• U SysDsgs is the Universe of UML Designs for different objectoriented software systems, in which for each of theme there exists
a set of UML communication diagrams,• SObjects(SysDsg) ( U Objects is the set of classes in a UML Design,derived from its UML communication diagrams.

NIM(Numberof
InstantiatedMethods)

absN IM : U Classes → ℘(U RCM sgs) : cls → SRCM sgs(cls)where :
• U RCM sgs is the Universe of Received Call Messages relevant to the
UoD U Classes,• U Classes is the Universe of classes within a UML Design of an
object-oriented software system,• SRCM sgs(cls) ( U RCM sgs is the set of received call messages by
all objects of the class cls. A call message can be distinguished fromanother, if its sequence expression of its label are different, e.g. thecall message A1 : print() is different to the call message A2 : print().

NDIM(Number of Different
InstantiatedMethods)

absN DIM : U Classes → ℘(U DRCM sgs) : cls → SDRCM sgs(cls)where:• U DRCM sgs is the Universe of Different Received Call Messages relevant to the UoD U Classes,• U Classes is the Universe of classes within a UML Design of an
object-oriented software system,• SDRCM sgs(cls) ( U DRCM sgs is the set of different received callmessages of all objects of the class class. A call message can be
distinguished from another, if their names are different, e.g. thecall message A1 : print() is exactly the same as the call message
A2 : print().

xxiv

A.7. For a Sender Object: The Number of Different Receiver Objects

JAIST

Table A.4: Abstract Functions to Model Metrics for Classes
MeasureAbstract Function

NIMO (NumberofInstantiatedMethods of OtherClasses)

absN IM O : U Classes → ℘(U SCM sgs) : cls → SSCM sgs(cls)where:NDIMO (Number of Different
InstantiatedMethods of OtherClasses)

absN DIM O : U Classes → ℘(U DSCM sgs) : cls → SDSCM sgs(cls)where:NDRC (Number
of Different Receiver Classes)

absN DRC : U Classes → ℘(U Receivers) : cls → SReceivers(cls)where:• U SCM sgs is the Universe of Sent Call Messages relevant to the UoD
U Classes,• U Classes is the Universe of classes within a UML Design of an
object-oriented software system,• SSCM sgs(cls) ( U SCM sgs is the set of sent call messages of all
objects of a class cls. A call message can be distinguished fromanother, if its sequence expression of its label are different, e.g. thecall message A1 : print() is different to the call message A2 : print().

• U DSCM sgs is the Universe of Different Sent Call Messages relevant
to an UoD U Classes,• U Classes is the Universe of classes within a UML Design of an
object-oriented software system,• SDSCM sgs(cls) ( U DSCM sgs is the set of different call messages
sent by the objects of the class cls. A call message can be distinguished from another, if their names are different, e.g. the call message A1 : print() is exactly the same as the call message A2 : print().

• U Receivers is the Universe of the different objects acting as receiversof a message sent by any of the objects of the class cls to an UoD
U Class,• U Classes is the Universe of classes within a UML Design of an
object-oriented software system,• SReceiverss(cls) ( U Receivers is the set of objects that receive acall message from any of the objects of the class cls.

xxv