Evaluation of a Business Application Framework
Using Complexity and Functionality Metrics
Hikaru Fujiwara1, Shinji Kusumoto1, Katsuro Inoue1, Toshifusa Ootsubo2 andKatsuhiko Yuura2
1
Graduate School of Engineering Science, Osaka University 1-3 Machikaneyama,
Toyonaka, Osaka, 560-8531, Japan Tel: +81-6-6850-6571, Fax: +81-6-6850-6574
2
Business Solution Systems Division, Hitachi Ltd.
h-fujiwr@ics.es.osaka-u.ac.jp

Abstract. This paper experimentally evaluates the usefulness of a businessapplication framework from a viewpoint of saving cost and quality of thesoftware in a company. Here, we conducted two case studies. In the casestudies, four kinds of applications are developed. Each of them is developed in
two ways: using framework-based reuse and conventional module-based reuse.
Then, we evaluate the difference among them using the several functionality
and complexity metrics. As the results, the framework-based reuse would be
more efficient than the module-based reuse in the company.

1. Introduction
As the size and the complexity of software increase, it becomes important to develop
high-quality software cost-effectively within a specified period. In order to attain it,
several software engineering techniques have been developed. Reuse is one of themost famous techniques among them.
The definition of reuse is the use of an existing software component in a newcontext, either elsewhere in the same system or in another system [4]. It is generally
said that software reuse is improved productivity and quality, resulting in cost saving.
Several research studies have demonstrated the actual benefits[2][7][9].
On the other hand, object-oriented development intends to construct the softwareby reusing several software components which have high cohesion and are easy to
combine. By reusing the components which have high quality, the improvement of
the quality of the software will be attained and development period will be also
decreased [8]. As in development with object-oriented languages, developers reuse aparticular library called a framework [3]. A framework is collection of classes thatprovide a set of services for a particular domain; a framework thus exports a numberof individual classes and mechanisms which clients can use or adapt. In a typicaldevelopment with the framework, at first, developers take out the primitive parts of aprogram. Then, they develop the other parts needed for the developing application,and combine them into a program. The Microsoft Foundation Classes (called MFC) isa framework for building applications that conform to the Microsoft Windows userinterface standards.

In a department of Hitachi Ltd., a business application framework for a specificdomain has been developed and introduced. However, it is difficult to transfer the
new framework to the development because developers in the department use theoriginal reuse technique that is a conventional module-based reuse. From a viewpointof saving cost and improving the productivity, it is clear that framework-based reuseis more appropriate than the module-based reuse. For effective technical transfer, it isnecessary to motivate the developers who will use the new technique by showing thebenefit of using it quantitatively.
This paper experimentally evaluates the usefulness of the framework from aviewpoint of saving cost and quality of the software. Here, we conducted two casestudies. In the case studies, four kinds of applications are developed. Each of them isdeveloped in two ways: using framework-based reuse and conventional module-based
reuse. Then, we evaluate the difference among them using the several functionality
and complexity metrics. As the results, the framework-based reuse would be more
efficient than the module-based reuse in the department.
Section 2 introduces the characteristics of the target application software and the
proposed framework. Then, Section 3 describes the case studies and Section 4analyzes the empirical results. Finally, Section 5 concludes and mentions about future
research topics.

2. Preliminary

2.1 Target application software
In a department of Hitachi Ltd., a series of on-line application software for a local
government has been developed. Here, “series” implies that there exist multiple
projects where similar applications are successively developed for several local
governments. The functions of the application software includes such as a family
registration, payment of various taxes, treatment of health insurance. Since the data
handled by each local government are different, the details of each applications aredifferent, but the fundamental processing of each function is the same. To put it
simply, it controls the processing of conventional database operations.
So far, to develop the series of the applications, conventional module-based reusehas been used. Figure 1 shows the typical example. Figure 1 (a) is the module
structure of an application that makes a new record for a health insurance for a local
government A. Module A1 is a main module and controls the transition of the screens.
Users manipulate the application through the screens that are changed based on theusers' input. Each of the modules A11, A12 and A13 corresponds to the process of datainquiry, showing the inquiry result and updating the record. In the conventional
module-based reuse, each of the modules is reused in the next development. Figure 1(b) shows the module structure of the same application for a local government B.
Here, modules B11, B12 and B13 are reused by modifying the corresponding modules
A11, A12 and A13. The reused modules are modified in a skillful way in thedepartment.

Fig. 1. Conventional module-based reuse
2.2Proposed framework
Recently, many companies have started to introduce object-oriented technology into
their software development environments. The application software described in
Section 2.1 will be also developed by object-oriented approach and implemented
using Java. So, the new framework is going to be introduced. The framework is
intended to reuse the processing of the transition of the screens in addition to themodule-based reuse. In the framework, typical transitions of screens are prepared, thatare data inquiry, data renewal, data addition, data removal and so on. For example,
data renewal consists of the three kinds of screen manipulations: data inquiry,
showing the inquiry result and updating the record.
For example, Figure 2 explains the framework-based reuse. Figure 2 (a) shows theapplication software that makes a new record for a health insurance for a local
government A. F1 is a framework where a screen transition pattern for data renewal is
got together. By using F1, the processing of data inquiry, showing the inquiry result
and updating the record are simply implemented together. The specific information to
the local government A is given by parameters. So, in case of developing the similar
application for a local government B, it is easily done using the framework F1 and thespecific information to the local government B shown in Figure 2(b).

Fig. 2. Framework-based reuse

3. Case Studies
Here, we explain the case studies to evaluate the usefulness of the proposed
framework.
3.1 Design of case study
We conducted two case studies: Case studies 1 and 2. In case study 1, four kinds ofprograms are developed in two ways: conventional and framework-based reuse. Here,
Ca, Cb, Cc and Cd represent the four kinds of programs developed using theframework-based reuse. On the other hand, Pa, Pb, Pc and Pd represent the ones
developed using the conventional reuse. The functions implemented in Ci and Pi(i=a,b,c,d) are the same. We compare Ci and Pi from the viewpoint of the productivity
and quality.
In case study 2, a program is also developed in two ways. In this case, each of fourfunctions (called fa, fb, fc and fd) are continuously added to the program. That is, using
the framework reuse, at first, a program (called Ca), includes only fa, is developed andthen a program (called Ca+b) is developed by adding the function fb to Ca. Similarly,

Ca+b+c is developed by adding fc to Ca+b and finally Ca+b+c+d is completed by adding fd
to Ca+b+c. On the other hand, using the conventional reuse, programs Pa, Pa+b, Pa+b+c
and Pa+b+c+d are developed. The functions between Ci and Pi (i=a, a+b, a+b+c,
a+b+c+d) are the same. We compare the differences between the two successive
programs from the viewpoint of the productivity and quality.
3.2 Metrics
Unfortunately, we could not collect the actual development effort and the number offaults injected in the development. So, we used the following metrics to indirectly
evaluate the productivity and quality: OOFP (Object-Oriented Function Point)[5] andC&K metrics (Chidamber and Kemerer's metrics)[6].
Function point (FP) was proposed to help measure the functionality of softwaresystems and to estimate the effort required for the software development. However,
the original function point is not proposed to object-oriented program. Recently,
OOFP has been proposed in [5] and relatively easy to calculate from the programs.
So, we used the OOFP to indirectly measure the productivity.
On the other hand, C&K metrics is one of the most famous metrics to evaluate thecomplexity of object-oriented software. Several studies reported the relationshipbetween the values of C&K metrics and the number of faults injected. For example,
Basili et al. empirically evaluated that C&K metrics show to be better predictors offault-proneness of the class than the traditional code metrics [1]. Thus, we used C&K
metrics to indirectly measure the quality of the applications.
3.3 OOFP
Caldiera et al. have defined an adaptation of traditional FP, called OOFP, to enable
the measurement of object oriented analysis and design specifications[5].
In traditional developments, the central concepts used in counting function points
are logical files and transactions that operate on those files. In OO systems, the core
concept is no longer related to file or data bases; instead the central concept is the“object”. A class is the candidate for mapping logical files into the OO paradigm. In
the FP method, logical files (LFs) are divided into internal logical files (ILFs) andexternal interface files (EIFs). Classes within the application boundary correspond to
ILFs. Classes outside the application boundary correspond to EIFs. For each ILF/EIF,
it is necessary to compute the number of DETs (Data Element Types) and RETs
(Record Element Types). Simple attributes, such as integers and strings, areconsidered as DETs. A complex attribute, such as an attribute whose type is a class ora reference to another class, is considered as RET.
Transactions in FP method are classified as inputs, outputs and inquiries. In
OOFP, they are simply treated as generic Service Requests (SRs). Each service
request (method) in each class in the system is examined. Abstract methods are notcounted. Concrete methods are only counted once (in the class they are declared),even if they are inherited by several subclasses, because they are only coded once. If
a method is to be counted, the data types referenced in it are classified: (1) simple

items are simple data items (such as integers and strings) referenced as arguments ofthe method, and simple global variables referenced in the method (2) complex items
are complex (class or reference to class) arguments, objects or complex global
variables referenced by the method. Simple items are considered as DETs, andComplex items are considered as File Types Referenced (FTRs).
Finally, OOFP is calculated by summing up the number of ILF, EIF and SR
weighted by each complexity based on DET, RET and FTR.
3.4 C&K metrics
Chidamber and Kemerer's metrics are based on measurement theory, and reflect the
viewpoints of experienced object-oriented software developers. Chidamber and
Kemerer defined the following six metrics[6]:1. WMC(Weighted Method per Class) : Assume that a class C includes methods
M1, ..., Mn. Let ci(i = 1..n) be the static complexity of method Mi. Then,
WMC(C)= ∑in=1 c i . If it can be supposed that all methods of the given class C are
equally complex, then WMC(C) is simply the number of methods defined in theclass C [1].
2. DIT(Depth of Inheritance Tree of a class) : DIT(C) is the depth of class C in theinheritance tree of a given program. If it can be supposed that the wholeinheritance graph is a tree, then DIT(C) is the length of path from the root to theclass C [1].
3. NOC(Number Of Children) : NOC(C) is the number of immediate sub-classes
subordinated to a class C in the class hierarchy of a given program.
4. CBO(Coupling Between Object class) : CBO(C) is the number of couplings
between a class C and any other class. The coupling means that the class C uses
member functions and/or instance variables in any other class.
5. RFC(Response For a Class) : Let Ms(C) be a set of methods in a class C, anddefine Mr(C) = {Mj | Mj is a method called by any Mi ∉ Ms(C) and Mj ∉ Ms(C)}.
Then RFC(C)= | Mr(C) U Ms(C) |.
6. LCOM(Lack of Cohesion in Methods) : Assume that a class C includes methods
M1, M2,...,Mn. For each i (i = 1..n), let Ii be a set of instance variables used bymethod Mi. LCOM(C) is defined as the number of such pairs of method (Mi, Mj)
that Ii I Ij = φ , minus such pairs of method (Mk, Ml) that Ik I Il ≠ φ , when theformer is greater than the latter. Otherwise, LCOM(C) is defined to be zero.
3.5 Application of the metrics
Here, we explain how to apply the metrics to the programs. In order to evaluate theeffect of the framework, we pay attention to the newly developed part of the
programs. Figure 3 shows rough structure of the Ca and Pa in case study 1. In Ca, FW
represents the framework. On the other hand, Figure 4 shows rough structure of theCa, Pa, Ca+b and Pa+b in case study 2. In both cases, the values of metrics are calculated
from the newly developed part of the programs (shaded portion).

Tables 1 and 2 show the measurement results of case studies 1 and 2. The values of
C&K metrics are the average values per a class. Here, the values of NOC and DIT areomitted since these applications were developed without class inheritance and thevalues became 0.

Fig. 3. Measurement in case study 1

Fig. 4. Measurement in case study 2

4. Analysis

4.1 Case study 1
With respect to OOFP, the values of programs developed using framework aresmaller than ones of programs developed using conventional reuse1. Especially, thevalues of Ca, Cb and Cd are much smaller than ones of corresponding programs.
Next, we analyze the values of C&K metrics:
CBO, RFC: The values of Ci are higher complexity than ones of Pi. It means thatthe newly developed classes of Ci include a lot of method calls. We examined theclasses of Ci that have high CBO and RFC values and found that the called method
are mostly included in the classes in the framework. So, if the classes of theframework have high quality, the complexity does not affect the quality of theoverall application programs.
WMC, LCOM: The values of Ci are higher than ones of Pi. We also examined theclasses of Ci that have high WMC and LCOM values and found that there aremany simple methods, that set/get the values of attributes in the class of
1 Since P s have been developed scratch, they actually had no reused part.
iframework. The size of these methods is about one or two lines of codes.
Therefore, the complexity does not affect the quality of the overall application
programs, too.
Table 1. Result in case study 1
Ca
Cb
Cc
Cd
Pa
Pb
Pc
Pd

OOFP
176
180
418
252
526
526
671
672

CBO
3.8
5.8
5.4
4.1
2.1
2.3
3.0
2.4

RFC
14.4
18.2
33.1
17.8
5.8
5.9
7.4
8.6

WMC
7.4
7.6
8.1
6.4
3.3
3.3
3.4
4.3

LCOM
21.4
23.2
28.7
13.8
2.1
2.1
2.0
15.7

4.2 Case study 2
Using Table 2, we evaluate the amount of increased values of the metrics. Here, ∆ b,
∆ c, ∆ d represent the increased values of OOFP by adding the function fb, fc and fd,
respectively. For Ci (using the framework), the values of ∆ b, ∆ c and ∆ d are 75, 327
and 165, respectively. On the other hand, for Pi (not using the framework), the valuesof ∆ b, ∆ c and ∆ d are 89, 234 and 235. For ∆ c, the values of Ci has higher value thanones of Pi. This is because in order to implement the function fc, a lot of data items
must be handled compared to the function fb and fd. Also, it is found that theadjustment between the function fc and the framework is not good. One way to copewith it is to add the new components to the framework to fit the function like fc.
Next, we analyze the values of C&K metrics. Here, ( δ b, δ c, δ d) represent theincreased values of C&K metrics by adding the function fb, fc and fd, respectively.
CBO, WMC: There is few difference between the increased values in Ci and Pi.
RFC: The increased value of δ c in Ci is 13.4 (= 30.1 - 16.7) and much larger thanone (1.7 = 8.6 - 6.9) in Pi. Using the framework, the number of method call isincreased. The called method are mostly included in the classes in the framework.
So, if the classes of the framework have high quality, the complexity does notaffect the quality of the overall application programs.
LCOM: There are not so much difference among Ci. On the other hand, theincreased value in δ d (8.2 = 10.0 - 1.8) in Pi is larger that in δ b and δ c.
For Ca+b+c+d and Pa+b+c+d, the values of OOFP are 743 and 1084. So, totally, byusing the framework, the effort of the development would be reduced. On the other
hand, with respect to C&K metrics, the complexity of Ca+b+c+d is higher than Pa+b+c+d.
However, the complexity is caused by the interaction among the newly developedclass and the framework classes. So, we consider that if the framework has highquality, the complexity does not affect the quality of the overall application programs.

Table 2. Result in case study 2
Ca
Ca+b
Ca+b+cCa+b+c+d
Pa
Pa+b
Pa+b+c
Pa+b++c+d

OOFP
176
251
578
743
526
615
849
1084

CBO
3.8
5.0
5.9
5.8
2.1
2.8
3.7
4.0

RFC
14.4
16.7
30.1
28.3
5.8
6.9
8.6
10.5

WMC
7.4
7.6
8.3
7.9
3.3
3.3
3.3
3.9

LCOM
21.4
20.1
28.6
25.6
2.1
2.0
1.8
10.0

5. Conclusion
We have experimentally evaluated the usefulness of the framework from a view pointof the effectiveness of the saving cost and the quality of the software. As the results of
two case studies, the framework-based reuse is more efficient than the module-based
reuse in the department.
Actually, the value of FW's OOFP is 1298. However, as the result of case study 1,
the value of OOFP in framework-based reuse about 2.5 times as effective as inconventional reuse. So, it is expected that the department developing the series ofproject will save the effort after three or four applications have developed, whereas
the investment for FW was spent.
In order to show the usefulness of the framework, we are going to apply theframework to many software development projects. In addition, it is necessary to add
the new components to the framework to increase the applicability of it.

References
1. V. R. Basili, L. C. Briand and W. L. Melo: “A validation of object-oriented design metrics asquality indicators”, IEEE Trans. on Software Eng. Vol. 20, No. 22, pp. 751-761 (1996).
2. V. R. Basili, G. Caldiera, F. McGarry, R. Pajerski, G. Page and S. Waligora: “The software
engineering laboratory - an operational software experience”, Proc. of ICSE14, pp. 370381 (1992).
3. G. Booch:
Object-Oriented Analysis and Design with Applications, The
Benjamin/Cummings Publishing (1994).
4. C. Braun: Reuse, in John J. Marciniak, editor, Encyclopedia of Software Engineering, vol. 2,
John Wiley & Sons, pp. 1055-1069 (1994).
5. G. Caldiera, G. Antoniol, R. Fiutem and C. Lokan: “Definition and experimental evaluationof function points for object-oriented systems”, IEEE, Proc. of METRICS98, pp.167-178(1998).
6. S. R. Chidamber and C. F. Kemerer: “A metrics suite for object-oriented design”, IEEE
Trans. on Software Eng., Vol. 20, No. 6, pp. 476-493 (1994).
7. S. Isoda: “Experience report on a software reuse project: Its structure, activities, andstatistical results”, Proc. of ICSE14, pp.320-326 (1992).

8. I. Jacobson, M. Griss and P. Jacobson: Software Reuse ---Architecture Process andOrganization for Business Success---, Addison-Wesley (1997).
9. B. Keepence and M. Mannion: “Using patterns to model variability in product families”,
IEEE Software, Vol. 16, No. 4, pp. 102-108 (1999).