IEICE TRANS. FUNDAMENTALS, VOL. E00{A, NO. 1 JANUARY 1998

1

PAPER Special Section on Concurrent Systems Technology
Experimental Evaluation of Two-Phase Project Control
for Software Development ProcessyOsamu MIZUNO ,Student Memberyy

Yasunari TAKAGISUMMARY

In this paper, we consider a simple development process
consisting of design and debug phases, which is derived from actual concurrent development process for embedded software ata certain company. Then we propose two-phase project control
that examines the initial development plan at the end of designphase, updates it to the current status of the development process
and executes the debug phase under the new plan.
In order to show the usefulness, we de ne three imaginary
projects based on actually executed projects in a certain company: the project that executes debug phase under initial plan,
the project that applies the proposed approach, and the projectthat follows a uniform plan.
Moreover, to execute these projects, we use the project simulator, which has already been developed based on GSPN model.
Judging from the number of residual faults in all products, wefound that project B is the best among them.
key words:

software development, project management, software

fault, Petri-net model, software test and debug

1.

and

y

Members
Nonmembersy, Shinji KUSUMOTO , Tohru KIKUNO ,Introduction

Generally, the actual software development process is
a concurrent process in the sense that many activities
are executed in parallel by team members[1][2]. For example, the module design activities for subsystems areconcurrently executed, and the coding and the coding
review are concurrently executed. The former is for theeciency, but the latter may be for the delay in theschedule.
Therefore, it is not easy to manage the progressof development for such a concurrent process[7]. Especially, using the development plan initially constructedbased on insucient data on the target project, it isvery dicult to control the process of development.
Furthermore, it is also dicult to estimate the development cost. Many failed examples are shown in so-called
death march projects[14].
Especially, for the test phase, the following problems or diculties are pointed out with respect to controlling the progress of test phase[6][11].
{ to determine the amount of e orts for the test
phase
{ to estimate the quality of delivered codey
The authors are with the Graduate School of Engineering Science, Osaka University.
yy
The authors are with the OMRON Corporation.

,

Keishi SAKAMOTO

yy

,

,

These problems are tightly coupled with each other. If
the e orts are too small, the quality will become very
poor. Even if the e orts are too large, the quality may
not be improved so much.
Based on these observations, we propose a new
method for controlling the progress of software development. The key idea is to update or modify the initial plan at an intermediate stage of the development,
and to apply the updated plan to the succeeding stage.
Then we propose to control the progress of design andcoding phases using the initial plan, and then control
the progress of test and debug phases using the updated
plan. We call this method two-phase project control.
The updating of the plan should re ect the resultsof development using the initial plan. For instance, thenumber of faults introduced into the product and thenumber of residual faults in design and coding phases
are the data to be considered.
In this paper, we con rm the usefulness of thetwo-phase project control using the project simulator
developed based on the Generalized Stochastic Petrinet(GSPN) model[10]. Additionally, in the evaluation
we apply the real data, collected from the actual software development projects in a certain company, to thesimulator.
This paper is organized as follows: Section 2 shows
the actual software development process in the certain company, and then proposes a simple processmodel. Section 3 proposes the two-phase project control method, and Section 4 explains only the outline ofthe project simulator. Section 5 shows the case study
and the simulation results. Finally, Section 6 summarizes the main results and the future research work.
2.

Software Process Model

2.1 Actual Software Process
An example of the actual software process is shownin Fig. 1. This process is currently used for a certain company, which develops the computer control
systems with embedded software. The main products
include ATMs(Automated Teller Machine), POS(Point
Of Sales) terminals and ticket vending machines.
The following concurrent executions are observed
for this actual process in Fig. 1.

IEICE TRANS. FUNDAMENTALS, VOL. E00{A, NO. 1 JANUARY 1998

2
Concept Design(CD)

Design Phase

A1

Concept Design Review
(CDR)

eCD

A2

Function Design
(FD)

A3

A4

Function Design Review
(FDR)

A5

A6

Concept Design Review
(CDR)

eFD

Structure Design
(SD)

A7

A8

A9

Structure Design Review
(SDR)

A10

A11

A12

Module Design
(MD)

A13

A14

A15

A16

Module Design Review
(MDR)

A17

A18

A19

A20

Coding
(PG)

A21

A22

A23

A24

Coding Review
(PGR)

A25

A26

A27

A28

Unit Test
(UT)

A29

A30

A31

A32

Unit Debug
(UDB)

A33

A34

Integration Test
(IT)

A37

A38

A39

Integration Debug
(IDB)

A40

A41

A42

A35

A43

A44

Function Debug
(FDB)

A45

A46

Verification Test
(VT)

A47

Verification Debug
(VDB)

A48

Fig. 1

Function Design
(FD)

eSD

eMD

ePG

Integration Debug(IDB)

ModuleDesign(MD)

Function Debug
(FDB)

dMDR

Coding
(PG)

A13

A16

A14

A15

dFDB

Verification Test
(VT)
Verification Debug
(VDB)

dPGR

ddesign=dCDR+dFDR+dSDR+dMDR+dPGR
rdesign=eCD+eFD+eSD+eMD+ePG - ddesign

A16

dIDB

Function Test
(FT)

Coding Review(PGR)

tain functions.
For example, module designs with , ,andare included in Fig. 1. This implies thatthe target system consists of four subsystems andthat module designs are executed by four groups
in the team.
 Certain activity is unwillingly executed with previous activity concurrently.
This concurrency is not explicitly shown in Fig. 1,
but it is recognized by the interviews with project
manager in the certain company. Additionally, theanalysis of e ects of this concurrency on the quality and productivity has already been reported[8].
However, this actual development process is toocomplicated for further discussions in this paper. Then,
we propose a simple process model, shown in Fig. 2, in
which the fundamental ow structure in Fig. 1 is remained. By this simpli cation, for example four module design activities , ,andin Fig. 1 aremerged into one module design(MD) in Fig. 2. (But,the simple process in Fig. 2 is used to de ne the proposed methods clearly. The actual simulation in Sect. 5
is performed using the actual process in Fig. 1.)
Additionally, we divide logically the whole process
into two phases from the viewpoint of faults: design

dUDB

Integration Test
(IT)

Module Design Review(MDR) Several activities are executed concurrently for cer-

A15

Unit Debug(UDB)

Structure Design Review
(SDR)
dSDR

Actual development process

A14

Unit Test(UT)

Structure Design(SD)

Fig. 2

A13

Debug PhasedCDR

FunctionDesign Review
(FDR)
dFDR

A36

Function Test
(FT)

Concept Design
(CD)

dVDB

ddebug=dUDB+dIDB+dFDB+dVDB
rdebug=rdesign - ddebug

Simple development process

phase and debug phase. The reason is that the activities in the design phase are essentially for increasing
the size of the product and thus introducing faults, but
the activities in the debug phase are for detecting andremoving faults from the product(thus decreasing thesize of the product).
2.2 Simple Software Process Model
Then, we assume that software development process
consists of two successive phases: design and debug
phases. Design phase includes ten activities (Concept
Design (CD), Concept Design Review (CDR), Function
Design (FD), Function Design Review (FDR) Structure
Design (SD), Structure Design Review (SDR), Module
Design (MD), Module Design Review (MDR), Coding(PG) and Code Review (PGR)). Debug phase includes
eight activities (Unit Test (UT), Unit Debug (UDB),
Integration Test (IT), Integration Debug (IDB), Function Test (FT), Function Debug (FDB), Veri cation
Test (VT) and Veri cation Debug (VDB)).
As shown in Fig. 2, we introduce several parameters to make the discussions clear. Let i be the numberof faults introduced into the product developed in design/coding activity . Let j be the number of detected
and removed faults in reviews/debug activity . Let
design be the total of CDR , FDR , SDR , MDR andPGR . Let design be the number of residual faults indesign phase and is calculated by 6 i 0 design . Also, letdebug be the total of UDB , IDB , FDB , and V DB .
Let debug be the number of residual faults in debug
e

i

d

j

d

d

d

d

d

d

d

d

r

e

d

r

d

d

d

MIZUNO et al: EXPERIMENTAL EVALUATION OF TWO-PHASE PROJECT CONTROL FOR SOFTWARE DEVELOPMENT PROCESS

3phase and is calculated by design 0 debug .
Strictly speaking, acceptance test and debug are
executed after VDB. Since these are not activities ofdevelopment team, we omit them from Fig. 1 and Fig. 2.
Let last be the number of residual faults in the acceptance test and debug. Thus we can consider that last is
the number of residual faults in the last product(usuallyprogram codes).
However, as a matter of fact, we cannot collect allvalues of the above parameters. Thus, in this paper,
we aim to estimate the value of design , design , debug ,debug and last . In Sect. 5, we evaluate the usefulnessof the proposed project control method by comparing
the estimated values of design , design , debug , debug
and last with the actual ones.
r

Debug Phase

Design Phased

Debug under
init
the initial plan p debug

Design under
init
the initial plan p design

r

r

d

r

r

Fig. 3dSingle-phase control

Design Phase

Debug Phase

Design under
init
the initial plan p design

Debug under
upd
the updated plan p debug

r

d

r

d

r

Fig. 4Two-phase controlr

2.3 Project Planp

The initial project plan consists of the assignments ofboth developers and the time to each activity. The assignments are determined, at the beginning of project,based on the documents such as WBS(Work Breakdown Structure) charts, Organization charts of project,
PERT charts and a list of software products to be developed. Simultaneously, the initial plan also re ects
experience and knowledge of the project manager.
First, we perform an estimation using the initialplan. However, some di erence usually occurs between
the initial plan and the actual progress of the project.
If the developers follow the initial plan regardless of theexistence of the di erence, then the fatal confusion may
be caused. On the contrary, if we can take the progress
into consideration and reconstruct a new plan, then wemight get more accurate estimation. In Section 3, we
will propose such a new project control method.
3.

In this Section, we explain about proposed project control methods and project plan. The detail of projectplan will be explained in subsection 4.3.
3.1 Single-Phase Control
In single-phase control of software development process,we construct the initial project plan at the beginningof project, and execute whole project under the initialproject plan (See Fig. 3).
In the following, design and debug denote theproject plan of design and debug phases, respectively.
Then we often use = ( design debug ) to denote awhole plan of project.
init
Let the init
design and debug denote the initial plansof design and debug phases, respectively. We assume
initthat both init
design and debug are constructed at the beginning of the project. Then, the project plan of thesingle-phase controlled project is described as follows:
p

p

p

p

p

p

p

p

p

p

;p

p

;p

3.2 Two-Phase Control
Next, in two-phase control of software development
process, we construct two versions of project
plans
upd
init
= ( init= ( init
design debug ) anddesign debug ). At
the beginning of project, we construct the project planinit
= ( init
design debug ) and execute the design phase underinit . Then we stop the execution of the project at
designthe end of design phase, and update
the plan of debug
updphase init
debug into a new plan debug taking the development situation into account. After that we continue
theproject under the updated new project plan upd
debug (SeeFig. 4).
For simplicity, we represent the project plan of thetwo-phase controlled project as follows:upd= ( init
design debug )As already mentioned, in the project plan for
two-phase control, init
design is the initial project plan for
upd
design phase and debug is the updated plan of debug
phase, which is constructed at the end of design phase.
p

p

p

;p

p

p

0

p

;p

;p

p

p

p

p

p

Project Control Methodsinit= ( init
design debug )

p

;p

p

p

p

4.

[8]

Simulator based on GSPN Model

4.1 Outline of the GSPN Model
The proposed model for project simulator consists ofProject model and Process model. Figure 5 shows theoutline of the proposed model.
Project model includes three key components: activities, products and developers. Some attributes areattached to each of them, to be shown in Table 1.
Process model includes a set of Activity models
which include speci cations of design, coding, review,
test, debug activities, and so on. Activity model is described by an extended GSPN, to be shown in Fig. 6.

IEICE TRANS. FUNDAMENTALS, VOL. E00{A, NO. 1 JANUARY 1998

4
Project Model
activities{ Ai }
products{ Pi }

entry condition
exit condition
input products
output products

attribute valuesof Ai, Pi, Mi

review

Outline of the proposed model

4.2 Workload
In [10], we de ne the term \workload" of an activity as
the total time needed for a developer who has the standard capability to complete the activity. Additionally,an eciency of the activity under such a condition isquanti ed as 1. The value of eciency depends on theenvironment, such as the number of the developers, thenecessity of communication and performance of CASE
tools. Then, the development time is calculated as theresult of dividing the workload by the eciency of theactivity.
If we get the workload of an activity, then we canestimate the development time appropriate for speci cseveral activity conditions dependent on a given environment.
In the proposed model, the workload is assigned
to each activity depending on the input products forthe activity. That is, for example, workload of design
activity ( design ) is de ned as the following formula:W

design = sdesign 2 Kdesign :W

Here, design denotes the size of input product of design activity and design denotes the workload parameter for design activity. Before simulation, workload
parameter must be given to each activity of the target
project.
Consuming of the workload assigned to an activity corresponds to the progress of the activity in thedevelopment. Growth of product can be modeled bychanging values of the size or the number of faults inthe output product.
sK

deadline

size
number of faults
completion rate

Attribute of developer Mi
experience level

workload

4.3 Project Model and Project Plan

debug

Process Model
Fig. 5

workforce

Attributes of product Pi

test

codingdesignProject template

Attributes of activity Ai
typedevelopers{ Mi }

firing rates oftransitions in GSPN

Table 1

Project model focuses on three key components: activities, products and developers, and attaches several
attributes to each of them (See Table 1). The project
template with values assigned for each attribute is generally called the project plan.
An activity has eight kinds of attributes, which
are type, entry/exit conditions, input/output products,
workforce, deadline and workload.
(1) Type shows which the activity corresponds toand describes currently one of design, coding, review,
test and debug.
(2) Entry condition and (3) exit condition specify
conditions for beginning and ending the activity, respectively.
(4) Input products describes the products given to
the activity as the input products and the workload parameter, that is the degree of contribution of each input
products to determine workload of the activity.
(5) Output products describes the output productsthat are developed in the activity and the weight assigned to each product. The variation of the productsize and that of the number of faults are distributed tothe products according to weights. Thus, the sum ofeach weight must be one.
(6) Workforce speci es tuples of the developers
who engage in the activity and the ratio of time in which
each developer can engage in the activity to his or her
business hours.
(7) Deadline represents the appointed date for thecompletion of the activity, which is xed on the development plan.
(8) Workload represents a tuple of the workload
assigned to the activity and consumed amount of it.
Next, a product has three kinds of attributes(SeeTable 1), which are size, the number of faults and completion rate. (1) Size represents the product size in document pages or the lines of source code. (2) Number offaults counts faults in the product. (3) Completion rate
represents the ratio of the consumed workload to theassigned workload.
Then, a developer has an attribute experiencelevel(See Table 1) which is determined according tohis/her length of service. We classify developers' experience levels into the following three levels: novice,MIZUNO et al: EXPERIMENTAL EVALUATION OF TWO-PHASE PROJECT CONTROL FOR SOFTWARE DEVELOPMENT PROCESS

5

t1 rcmmoves to the place . When the token exists in theplace , only the transition which represents writing behavior is enabled. If the transition res, thenproduct size is increased by one, and the number offaults could be increased according to the fault injection rate in (to be explained later). After the ring of, the token moves back to the place .
The ring rates of the transitions are formulatedby the following ten functions cm, th , wr , pr , rd ,dt , md, ps , lc and in . These functions should beconcretely speci ed based on the property of the target
project.
In the following, is the number of the developers engage in the activity, is developer's experiencelevel, 6 is the sum of each developer's experience,is the total size of the input products, is the completion rate of the input products, is the number offaults of the input products, is the number of thedays from the current date to the deadline of the activity. Activity model parameters (e.g., cm th wr
and in) are given to each activity and concerned withthe developers' behavior. For example, cm th wr
and in correspond to communicating, thinking, writing and fault injection, respectively.
P2

P2

t2 rtht3 rwr

t3

t3

s

P1

P2

f

p

(a)

attributes of token

transition

t1

t2

t3

s
f
w

executionfunction-w = w+1s = s+1
f = f+1(pin)product size
number of faults
consumed workload

t3

f

f

(c)

(b)
Fig. 6

P1

f

f

f

f

f

f

f

f

M

Activity modelL

L

standard and expert levels. They are quanti ed as discrete values 1, 2 and 3, respectively.
4.4 Activity Model for Process
Activity model is prepared for each type of activities
such as design, coding, review, test, debug and so on.
The descriptions of Activity models are given using
an extended GSPN. Figure 6(a) shows an example ofthe description of the design activity. In the extended
GSPN, a token has three attributes: product size ,number of faults and consumed workload , as shownin Fig. 6(b). These attributes are used to represent thecurrent status of development that varies over the execution of each Activity model.
Transitions used here are timed transitions. The
ring delay of each transition is exponentially distributed and the average ring delay of a transition is
speci ed by a ring rate assigned to it. In Fig. 6(a), thering rate cm of transition means that the averagering delay of transition is 1 cm.
In addition, each transition has a function (called
execution function) to be evaluated on its ring, asshown in Fig. 6(c). The execution of the function updates the attribute values of the token. Intuitively
speaking, each transition corresponds to the developers'
behavior such as thinking, writing and communicatingor an event which occurs during execution of activity.
Places correspond to waiting states for occurrences of
behaviors or events.
[Example] Consider transitions
and in the design
Activity model depicted in Fig. 6. These two transitions represent communicating and thinking behavior,respectively, and are enabled to re when a token exists in the place . If the communicating transition
res, it has no e ect on the attributes values, and thetoken returns to the place and only time elapses by
the ring delay. On the other hand, if the transition
res by evaluating its execution function, then consumed workload is increased by one, and the token
s

f

w

r

t1

t1

=r

t1

P1

t1

P1

t2

w

t2

S

R

F

D

K

;K

;K

K

;K

;K

K

K

(1) Communicating rate: cm = cm( cm 6 )(2) Thinking rate: th = th ( th 6 )(3) Writing rate: wr = wr ( wr 6 )(4) Preparing rate: pr = pr ( pr 6 )(5) Reading rate: rd = rd ( rd 6 )(6) Fault detecting rate: dt = dt ( dt 6)(7) Fault modifying rate: md = md ( md 6 )(8) Test-case passing rate: ps = ps ( ps )(9) Fault localizing rate: lc = lc ( lc 6
)
r

r

r

f

f

r

r

f

f

; M;

L

K

; M;

L

K

r

; M;

; M;

f

r

r

r

; M;

K

K

f

K

f

K

L; S

L

; M;

f

K

f

K

K

L; R

L; S; F

; M;

L

;M

; M;

L; S; F

These make it possible to dynamically determine
the frequency of communications or the diculty inthinking and writing according to the number of developers, experience levels of developers and/or completion rates of input products.
Moreover, the increase of product size at every
ring of writing transition and the consumption ofworkload at every ring of thinking transition are described by the corresponding execution functions. At
each ring of the transition, the values of token's attributes can be changed by evaluating its executionfunction.
Activity model handles fault injections in the design activity as the stochastic events whose occurrences
depend on the fault injection rate in. In general, in
is formulated by the following function:
s

t3

t2

p

p

IEICE TRANS. FUNDAMENTALS, VOL. E00{A, NO. 1 JANUARY 1998

6(10) Fault injection rate: in = in( in
p

f

K

; M;

6

L; D; R)

By using this function, it is possible to take account of dynamic in uence on the fault injection ratecaused by the stress from deadline of the activity ordevelopers' experience levels.
Though the functions (1){(3) are used in Fig. 6,
the rests (4){(9) are not included in Fig. 6. Figure 6
shows an example of design and coding activities andthus the improvement of model should be conducted
through case studies. Besides we also modeled otheractivities(review, test and debug), in which the functions (4){(9) are used. The detail of activity models isshown in [9].
4.5 Simulator
We have designed and implemented a simulator which
supports description of the target process, executes theprocess described by Activity model and analyses thesimulation results statistically.
The main simulation results include design ,design , debug and debug . As already explained, acceptance test and debug are executed just after veri cation debug in Fig. 2. From the simulation for these
activities, we also get last.
The system consists of ve functional units:project control unit, activity simulator, user interface
unit, display unit and editor.
Simulations proceed at the intervals of unit time .
At rst, project control unit determines activities to be
executed, based on the current status of the progressand entry/exit conditions of each activities. Next, foreach executable activity, project control unit delivers
the parameters to activity simulator and directs it toexecute activities for a day. Then, activity simulator
executes all of the activities, which are directed to execute by project control unit, using given parametersand extended GSPN. The execution of an activity isexpressed by the consumption of its workload. Whenan activity consumes all of assigned workload, the activity is regarded to be completed.
The execution of simulation is able to be suspended
or restarted at any time. Moreover, at every unit timeof simulation, intermediate simulation results can be
stored in the simulation database. The intermediate
simulation results are stored in the same format of theoriginal project description. Thus, it is possible torestart the simulation using the intermediate simulation results as the input. Also it is possible to change
the values of parameters(attributes of project) at any
time of the simulation. For example, we can modify the
number of developers at any time of the simulation andsimulate it immediately.
d

r

d

r

r

y

y

Currently, one unit time is a day (8 hours).

Table 2

Project 0
Project 1
Project 2
Project 3
Project 4

5.

Target projects

Type of Project
ActualImaginaryImaginaryImaginaryImaginary

Project Plan
actual ; P actual )= (Pdesign
debug
init
init
p0 = (Pdesign ; Pdebug )actual ; P init )p2 = (Pdesign debug
actual actual
p1 = (Pdesign ; Pdebug )actual uniform )p3 = (Pdesign ; Pdebug
p1

Case Study

In order to evaluate the usefulness of the proposed
project control method, we apply the simulator tosimilar software development projects, which are constructed based on the actual project data in an organization. In the following, we call all of software development projects thus constructed as imaginary softwaredevelopment projects.
5.1 Characteristics of Target Project
The main characteristics of the project are summarized
as follows[13]:(1)Development e ort of the project was 62 (mandays).
(2)The size of the system of the project was about 6.9(Ksteps).
(3)The project uses a standard waterfall model.
5.2 Experimental Projects
Now, we consider ve development projects: Project 0,
Project 1, Project 2, Project 3 and Project 4, based on
the development data of actual project. Table 2 summarizes the characteristic attributes of these projects.
Project 1 and Project 2 correspond to the project that
executes debug phase under initial plan. Clearly,
Project 3 is the project that applies the proposed approach and Project 4 is the project that follows a uniform plan. The details will be explained in subsections
5.2.1, 5.2.2 and 5.2.3.
Project 0 is the only actual project and the rests
Project 1 through Project 4 are imaginary projects. All
imaginary projects are executed by the project simulator mentioned in Sect. 4. Since we evaluate all imaginary projects by comparing with Project 0(for which
we have data collected from actual project in the company), we assume that the design phase of any imaginary projects(except for Project 1) is executed under
actual. From this assumption, we canthe actual plan design
get the simulation results with high accuracy for the
debug phase, and can compare in detail the resultant
data among ve development projects.
PMIZUNO et al: EXPERIMENTAL EVALUATION OF TWO-PHASE PROJECT CONTROL FOR SOFTWARE DEVELOPMENT PROCESS

7

5.2.1 Actual development
This is the actual development executed in a certain
company.
init and init denote initial project plans
Let design
debug
for the design and debug phases, respectively, whichwere actually constructed at the beginning of theactualproject in the company. On the other hand, let design
actualand debug denote speci c project plans for the designand debug phases, which we constructed based on therecords and the data collected at the completed target
project.
In summary, the project was originally planned to
be executed under the following initial plan :
init
init
= ( design
debug )
From the actual resultant data of the project, we caninterpret that the project was executed under the following project plan :actual actual)= ( design
debug
Therefore, we call this actual project (which was executed by the developers under the project plan ) as
Project 0.
P

P

P

P

p0

p0

P

;P

p1

p1

P

;P

p1

5.2.2 Imaginary projects(single-phase control)
Here, we de ne two imaginary development projects
which are executed under single-phase controlled plan.
At rst, we de ne an imaginary project, Project 1.
In Project 1, design phase and debug phase are executed
init and init , respectively.
under the initial plan design
debug
That is, the whole project is executed by project simulator under the initial plan .
Next, we consider an imaginary project, Project 2,
in which design phase is executed under the actual
actual and debug phase is executed under theplan design
init . By this, we want to make a caseinitial plan debugthat the plan for design phase in Project 2 was not updated(which is the di erence from Project 0).
Thus, Project 2 is executed under the following
plan of single-phase control:
actual init )= ( design
debug
Note that two projects Project 1 and Project 2 haveinit for debug phase, and thatthe same project plan debug
they have the di erent plans for design phase.
P

P

p0

P

P

At rst, we consider an imaginary project Project 3
in which design phase is executed under the actual plan
actualdesign , and its debug phase is also executed under theactual . That is, Project 3 is executed unactual plan debug
der the project plan .
Note that Project 3 is very close to Project 0, since
Project 3 is entirely simulated by using actual data ofProject 0. The di erence is that Project 0 is executedby developers, while Project 3 is executed by project
simulator.
Finally, we consider another imaginary project
Project 4, in which design phase is executed under theactual , and its debug phase is executed unactual plan design
uniform . We construct
der the virtual uniform plan debug
uniform as follows: First, we obtainthe uniform plan debug
the total time to debug from the debug plan data. Next,
each debug activity in this project is assigned uniformly
the same amount of workload to be consumed. With
init is also
respect to other attributes, the value in debug
uniform
used in debug . Thus, Project 4 is executed under
the following uniform project plan .
actual uniform )= ( design
debug
P

P

p1

P

P

P

P

P

p3

p3

P

;P

P

5.2.3 Imaginary projects(two-phase control)
Now, we de ne two imaginary development projects
Project 3 and Project 4 under two-phase project control method. As mentioned before, we assume that thedesign phase of Project 3 and Project 4 is executed unactual .
der the actual plan designP;P

5.3 Assumptions on Simulation
Here, we formulate each ring rate and fault injection
rate in the Activity models. For example, we show theformulas used in the design activity. In the following
formulas, is the number of the developers engaged
in the activity, 6 is the sum of each developer's experience level, is the completion rate of the input
products, is the number of the days from the current date to the deadline of the activity. cm th wr
and in are parameters given to each activity and concerned with communicating, thinking, writing and fault
injection rate, respectively.
M

L

R

D

K

;K

;K

K

2

r

cm = Kcm 2 6L 2 R

rth = Kth 2 M 2 M = Kth 2 6Lr

wr = Kwr 2 M 2 M = Kwr 2 6L

p2

p2

P

M

6

p

L

6

L

in = Kin 2 6L 2 R 2 D 2 M
MHere, we explain the sources of these formulas. The
relationship between the number of developers and thefrequency of communication is discussed in [3]. Similarly, it is said that incomplete input products induce
frequent inquiries about the omission of the description[4]. On the other hand, from the reference [12],the individual capability of a developer has an e ect on
both productivity and quality of software. The in uence of mental stress caused by the deadline is discussed

IEICE TRANS. FUNDAMENTALS, VOL. E00{A, NO. 1 JANUARY 1998

8
Table 3

init
debug
actualdebug
uniform
P
debug
P

P

Assignment of workload
UT & IT & FT & VT &
UDB
IDB
FDB
VDB
44
44
80
56
45
27
96
64
52
52
52
52

r

P

P

P

P

P

P

P

P

5.4 Simulation Result

r

r

p0

P

P

r

r

d

:

r

r

r

P

By using the project simulator, we iterate simulations for 1000 times per each project and calculate theaverage value of quality metrics.
Project 0 (actual development) has no information about residual faults at the end of the design/debug phase. As for the residual faults in thelast product( last ), we can nd the value from detectedfaults at the acceptance test, but as for the residual
faults in other products, we cannot obtain such datafrom the company.
As for Project 1, debug became 7.42. It gives theworst value among all projects. One of the reason isthat Project 1 was executed using the initial projectplan with no check points. (In some sense, Project 1
is essentially the same as Project 0. However Project 0
can be interpreted that a check point was prepared at
the end of design phase.)
In Project 2, the design phase is executed under theactual , and the debug phase is executed
actual plan design
init . The result debug (= 6 04)
under the initial plan debug
is better than that of Projects 1 and 4, but worse thanthat of Project 3. From the de nition of Project 2, theinit never re ects the situation at the
project plan debug
end of design phase, and thus it could not remove faults
suciently.
P:r

in [5]. Based on the knowledge from these sources andthe interviews with developers in a certain company, we
determined these formulas for convenience. Although,these formulas are not theoretically proved, we can simulated the projects appropriately using these formulasand values of parameters(to be explained in subsection
5.4).
Next, we explain an assignment of workload to eachactivity. As we mentioned before, we use veuniform
project
init , actual , init , actual andplans, design
design debug debug
debug .
Among them, Table 3 shows a part of project plans andsummarizes the assignment of workload to each activity(UT and UDB, IT and IDB, FT and FDB, and VT
init , this assignment isand VDB)(see Fig. 2). For debug
actual ,obtained from the plan of the project. For debug
the assignmentuniform
is obtained from the resultant data ofproject. For debug , the same amount of workload is
assigned uniformly to each activity.
As spaces are limited, we omit the description of
Project model, formulas of ring rates, and parameters
used by other activity models.
P

The value of debug (= 4 69) in Project 3 becomes
the best in this experiment. Regarding to debug , wecan say that the debug phase works very e ectively unactual . Since the plan of debug
der the project plan debug
phase was updated from the initial one by considering
the resultant situation of the design phase, the updated
plan might give adequate control for the succeeding debug phase(as shown in Table 3), and improve the valueof debug compared with Project 2.
To tell the truth, we predicted that Project 4 would
not show good performance because we constructed the
debug plan without considering the situation of designphase. The result, debug = 6 97, is better than that of
Project 1, but is worse than that of Projects 2 and 3.
As explained in subsection 5.2, the design phasesof Projects 2, 3 and 4 were executed under the sameactual project plan. That is, on the Table 4, the values of design and design are almost the same among
Projects 2, 3 and 4.
From the viewpoint of residual faults in all products(that is, the value of debug ) after debug phase,
Project 3 has the least faults, and is the most superior to other Projects 1, 2 and 4. Besides, judging fromresidual faults in the last product(the value of last ) after acceptance test and debug, there are no essential
di erences among four projects.
Here, we investigate the reason why di erences
appear in ve experimental projects. From the further investigations of the actual development data ofProject 0, the time assignment in the project plan
actualdebug for the debug phase was rather reasonable. Especially, the data of Project 0 implies that if concept
design takes longer time compared with the estimatedtime in the initial plan, then developers need to spend
longer times on veri cation test and debug.
Of course, there are many other factors, such as developers and fault density, to a ect the fault injection
and removal in the project. So we cannot conclude thatthe time assignment (that is, workload assignment) toeach activity in debug phase is the most e ective factor
to the number of detected/residual faults. However,as far as we concern the simulation results, updating
the time assignment in the project plan of debug phasewill e ect in uentially the number of detected/residual
faults and quality of the products at the end of debug
phase.

r:

6.

Conclusion

In this paper, we have proposed the two-phase projectcontrol method to realize the high quality of the products. In order to evaluate the proposed method, we
have de ned four imaginary projects using the actual
project data from a certain company. Finally, from theresults of project simulation, we have con rmed the effectiveness of the proposed method.
In the experimental evaluations, we have assumed

MIZUNO et al: EXPERIMENTAL EVALUATION OF TWO-PHASE PROJECT CONTROL FOR SOFTWARE DEVELOPMENT PROCESS

9

Table 4

Values of quality metrics

Executor

Plan

Design Phasedevelopmers
simulator
simulator
simulator
simulator

actual actual
p1 = (Pdesign ; Pdebug )
init
init
p0 = (Pdesign ; Pdebug )actual ; P init )p2 = (Pdesign debug
actual actual
p1 = (Pdesign ; Pdebug )actual uniform )p3 = (Pdesign ; Pdebug

designd

Project 0
Project 1
Project 2
Project 3
Project 4

actual was constructed by the exthat project plan debug
perts with project managements. This construction of
actualinit
actualdebug or the updating debug into debug is one of themost important and challenging future research works.
We consider that the functions such as cm , th ,etc. in subsection 4.4 are basically generalized ones.
But the formulas in subsection 5.3 are considered to betarget speci c. Now, we are applying this model andsimulator to much more projects. If we can get successful simulation results from the applications also, then it
will become a good evidence for generalizing our resultof the case study.
Additionally, we have simpli ed the actual development process shown in Fig. 1 to make our discussions
clear. In order to show the practical e ectiveness of thetwo-phase project control method, we are now trying toextend the process model shown in Fig. 2 to distinctly
and explicitly include such concurrent activities as described in subsection 2.1.
P

P

P

P

f

f

19
22.80
19.14
19.23
19.58

r

design

|
25.39
19.83
19.71
19.87

Debug Phase

debug

d

r

16
17.97
13.79
15.02
12.90

debug

Acceptance
Test

|
7.42
6.04
4.69
6.97

r

last

1
0.82
0.94
1.16
1.13

ulator based on generalized stochastic petri-net," Proc.
19th International Conference on Software Engineering,pp.293{302, 1997.
[11] J. D. Musa, A. Iannino and K. Okumoto: \Software reliability: measurement, prediction, application," McGrawHill, 1987.
[12] H. Sackman, W. J. Erickson and E. E. Grant: \Exploratory experimental studies comparing online and ofine programming performance," Commun. ACM, vol.11,
no.1, pp.3{11, 1968.
[13] Y. Takagi, T. Tanaka, N. Niihara, K. Sakamoto, S.
Kusumoto and T. Kikuno: \Analysis of review's e ectiveness based on software metrics," Proc. of the 6th International Symposium on Software Reliability Engineering,pp.34{39, 1995.
[14] E. Yourdon: \Death March : The Complete Software Developer's Guide to Surviving `Mission Impossible'
Projects," Prentice Hall Computer Books, 1997.

References[1] T. K. Abdel-Hamid: \The dynamics of software projectstang: A system dynamics based simulation approach,"
IEEE Trans. Software Eng., vol.15, no.2, pp.109{110,1989.
[2] M. Aoyama: \Agile software process model," Proc. 21th
Annual International Computer Software and Applications Conference, pp.454{459, 1997.
[3] F. P. Brooks Jr.: \The Mythical Man-Month," Addison
Wesley, 1975.
[4] B. Curtis, H. Krasner and N. Iscoe: \A eld study ofthe software design process for large systems," Commun.
ACM, vol.31, no.11, pp.1268{1287, 1988.
[5] T. Furuyama, Y. Arai and K. Iio: \Fault generation
model and mental stress e ect analysis," The Journal of
Systems and Software, vol.26, pp.31{42, 1994.
[6] F. J. Heemstra: \Software cost estimation," Informationand Software Technology, vol.34, no.10, pp.627{639, 1992.
[7] W. S. Humphrey: \Managing the Software Process," Addison Wesley, MA, 1989.
[8] Y. Hirayama, O. Mizuno, S. Kusumoto, T. Kikuno:\Hierarchical Project Management Model for Quantitative Evaluation of Software Process," Proc. International
Symposium on Software Engineering for the Next Generation, pp.40{49, 1996.
[9] Y. Hirayama: \Quantitative evaluation of software process based on hierarchical project management model,"Master dissertation, Osaka University, 1996.
[10] S. Kusumoto, O. Mizuno, Y.Hirayama, T. Kikuno, Y.
Takagi and K.Sakamoto: \A new software project sim-

Mizunoreceived the B.E.
degree in information and computer sciences from Osaka University, in 1996. He
is currently Master course student in the
Department of Informatics and Mathematical Science, Graduate School of Engineering Science, Osaka University. His research interests are software process, software quality assurance technique. He is a
student member of the IEEE.

Osamu

received the B.E.,
M.E. and Dr. Eng. degrees in informationand computer sciences from Osaka University, in 1988, 1990 and 1993, respectively. He is currently Assistant Professor in the Department of Informatics andMathematical Science, Graduate Schoolof Engineering Science, Osaka University.
His research interests are software metrics, software quality assurance technique.
He is a member of the IEEE and IPSJ.
Shinji Kusumoto

Kikunoreceived the B.E.,
M.E. and Ph.D. degrees in electrical engineering from Osaka University, in 1970,
Tohru

IEICE TRANS. FUNDAMENTALS, VOL. E00{A, NO. 1 JANUARY 1998

10
1972, and 1975, respectively. He joined
Hiroshima University from 1975 to 1987,
from 1988 he has been working for Osaka
University. He is currently Professor in
the Department of Informatics and Mathematical Science, Graduate School of Engineering Science, Osaka University. His
research interests include analysis and design of fault-tolerant systems, quantitative evaluation of softwaredevelopment process and design of testing procedure of computer
protocols. He was a general co-chairman of the Second International Workshop on Responsive Computer Systems in 1992. He
is a member of the IEEE, ACM and IPSJ.

received the B.E.
degree in information and computer science, from Nagoya Institute of Technology, Nagoya, Japan, in 1985. He has beenworking for OMRON Corporation.

Yasunari Takagi

Keishi Sakamoto
received the B.E.
degree in electrical engineering, from
Kobe University, Kobe, Hyogo, Japan, in
1969. He has been working for OMRONCorporation.