Vol. 42

No. 3

Transactions of Information Processing Society of Japan

Mar. 2001

Regular Paper

Software Project Simulator for Effective Process Improvement
Shinji Kusumoto†, Osamu Mizuno†, Tohru Kikuno†, Yuji
Hirayama†† , Yasunari Takagi†† and Keishi Sakamoto†††In this paper, we propose a new model for describing software processes and an estimation
method for the quality, cost and delivery date of a software project. The model was developed
based on the experience and the data from the software development process at OMRON
corporation. The model consists of a Project model and a Process model. The Project model
focuses on three key components: activity, product and developer of the project. Processmodel includes a set of Activity models, each of which specifies design, coding, review, test,
and debug activities respectively using GSPN(Generalized Stochastic Petri-Net). The newmodel can take the influence of human factors into account by introducing the concept of“workload”. Next, we develop a simulator which supports description of the process andexecutes the process described by the Activity model. As the result of its execution, we getthe estimated values for the quality, cost and delivery date of the target process. Then, weapply the simulator to real software processes at OMRON corporation and compare theseestimated values with actual data. The experimental results show the applicability of the
proposed simulator to improve real software process.

a process improvement procedure which describes the current process using Petri-net andestimates the benefits gained by the improvement. In the procedure, the current software
process is described accurately and in detail,and then a feasible action plan is presented
to developers. Also, benefit estimation is performed to evaluate the impacts of an action
plan before the action plan is actually implemented. In22) , we applied the proposed procedures to a practical project. We estimatedthat by applying the action plan to the next
project, 10% of the total effort/KLOC would
be reduced at test phases. Then, actually, theestimated effort reduction had been attained.
Then, our research objective shifted to executethe process improvement, especially benefit estimation, systematically from the viewpoint ofquality, cost and delivery date.
This paper proposes a new model based on
Generalized Stochastic Petri-net(GSPN)7) forsoftware projects. The model consists of a
Project model and a Process model. The
Project model focuses on three key components:
activity, product and developer of the project.
The Process model includes a set of Activity
models, each of which specifies design, coding,review, test, and debug activities respectively
using GSPN. The model can take the influenceof human factors into account by introducing
the concept of “workload”. The workload of anactivity is defined as total time needed for adeveloper with the average capability to com-

1. Introduction
Software quality is a major problem facing
software engineering. It is very important toestablish effective procedures, methods, notations, tools, and practices for promoting software quality. Moreover, quality affects productivity, since products with higher quality require less rework and maintenance. It is generally said that in order to improve productivity and quality of software products, the software development process has to be improved.
That is, to consistently improve products, theprocess used for developing them should be understood, defined, measured, and progressively
improved18) .
There are numerous studies and reports4) 9)12)
regarding the improvement of software development processes. The importance of the following two key activities is commonly stressed forthe improvement of software development processes: (1) to understand and analyze the current status of the software development process,and (2) to construct and execute the improvement plan of the process based on the analysis
results.
We have described an actual experience ofsoftware process improvement22) . In order toimprove the software process, we have proposed† Graduate School of Engineering Science, Osaka
University†† OMRON Corporation††† SPI consultant
396

Vol. 42

No. 3

Software Project Simulator

plete the activity. The workload can reflect thenecessity of communication and performance of
CASE tools, and thus the simulator can evaluate the dynamic aspect of the project. Additionally, so-called parallel executions of several
activities (for example, design and coding) canbe easily specified in the Process model to reflect the reality of the software project.
Next, in order to support description of theprocess and execute the process described by an
Activity model, we develop a software project
simulator. The kernel part of the simulator isdeveloped using C language and the display andthe editor are developed using Tcl/Tk. As theresults of simulation, we get the estimated values for the quality, cost and delivery date of
the target process. Then, we conduct empirical evaluation. In the experiment, we apply
the simulator to real software projects at OMRON corporation and compare the estimatedvalues with actual data. The experimental results show that the estimated values are quite
close to the actual value. As the result, we canshow the applicability of the proposed simulator
to improve real software project in the future.
The rest of this paper is organized as follows:
Section 2 reviews the related works and our pastexperience. Next, Section 3 proposes the new
project model. Section 4 describes the project
simulator based on the proposed model. Then,
Section 5 applies the simulator to the several
software projects in OMRON corporation andevaluates the usefulness of the proposed model.
Section 6 discusses the applicability of the proposed method and future research works. Finally, Section 7 concludes this paper.
2. Preliminaries
2.1 Past experience at OMRON corporation
OMRON Corporation consists of several
headquarters. One of them is the Electronic
Fund Transfer Systems H.Q. (shortly called
EF T S). The EFTS consists of several divisions including the Development Division andseveral business divisions. The SEPG (Software Engineering Process Group) is one of thegroups within the Development Division. The
SEPG was established in 1992 to improve thesoftware development process of the development departments, each of which belongs toeach business division. Moreover, the SEPG
has been cooperating with several Universities
for process improvement.

397In22) , we reported an actual experience ofsoftware process improvement at OM RON corporation. For effective technology transfer,a software engineering process group (shortly
called SEP G) had set three principal goals as
follows: (1) motivate developers to improve on
their process, (2) describe and define current
software process correctly and in detail, (3)present a feasible action plan for developers tofollow. To attain these goals, the SEPG proposed a process improvement procedure by describing the current process and estimating thebenefits gained by the improvement.
The project to be improved was one of a series of embedded software developments. “Series” implies that there exist multiple projects
where similar products are successively developed. Conventionally, these projects continue
for at least three years. When the process improvement started, two projects had alreadybeen finished. Next project would start after
four months.
Firstly, the SEPG held a series of interviews
with developers and depicted a flow map in theform of a Petri net19) which describes the current software process. Secondly, the group constructed an action plan from the in-depth analysis of the current flow map, and estimated thebenefits obtained if this plan were to be rigorously followed. As a result, both the action planand the benefit estimation were agreed by thedevelopers as a feasible action plan. Furthermore, by applying the action plan to a practicalproject, it was confirmed that, compared to asimilar project, approximately 10% of the total
effort/KLOC was reduced at test phases. Then,
we could suggest that the principal goals andthe proposed procedure are effective to reducethe development effort at OMRON corporation.
Then, our research objective shifted to develop an adequate process description language
for a real project and then to execute the process improvement, especially benefit estimation, systematically from the viewpoint of quality, cost and delivery date.
2.2 Related works
Several methods have already been proposed
to model and evaluate the software development process. Kellner has proposed the evaluation method of software processes describedby the modeling tool STATEMATE13) . Thismethod has demonstrated how process models
could be applied for software project management. Next, Lee and Murata have proposed

398Transactions of Information Processing Society of Japan

a β-distributed stochastic Petri-net model forsoftware project management14). This modelis an integrated model of program evaluation
and review technique (PERT) and Petri-net,
and suitable to deal with the uncertainty andconcurrency problems of large software project
management. FUNSOFT Nets and SPADE arealso the model based on Petri-net2)3) and are
more oriented to process enactment than toprocess analysis and simulation. They mainly
focus on the evaluation of the time constraintsof the process.
Furthermore, Tvedt and Collofello have evaluated the effectiveness of process improvement
on software inspections by using the system dynamics model24) . This method makes it possible to predict the impact of process improvements through the cause-effect relationships on
software development.
These methods are based on evaluation fromthe viewpoints of only the cost and delivery
date. Thus they cannot totally evaluate theimportant factors of software project: quality,
cost and delivery date.
Raffo has extended Kellner’s method in order to evaluate the quality of software, andapplied it to Kellner’s Software Process example20) . This method, however, has not been applied to real software development processes.
3. Proposed Model
3.1 Overview
It is necessary to evaluate the software processes from the viewpoints of quality, cost anddelivery date. At first, following the policyin22) , we decide to develop the model basedon Petri-net. Among many kinds of Petrinet model, we select the Generalized Stochastic
Petri-net(GSPN)7) .
Fundamental activities in the software development processes in22) can be described by introducing the probability of the injection andremoval of a fault as the firing rate of the transitions in order to evaluate the number of residual faults in the products.
Then, the concept of “workload” is considered in order to describe the fluctuations of thedevelopment period and product size. The interpretations of workload will be given in thenext subsection.
There also exist several dynamic factors,
which affect the behaviors of the developers, inthe software development as follows:( 1 ) Communication overheads5) .

Mar. 2001( 2 ) Difference of experience15) 21) .
( 3 ) Confusion by incompleteness6) .
( 4 ) Stress by delivery date8) .
In the practical development, since these factors change the development period dynamically, it is very difficult to estimate the development period precisely. So, three attributes:
developers’ experience level, completion rate ofproducts and deadline for activities, are incorporated to take the dynamic influence of humanfactors into account. Especially, “completionrate” of products makes it possible for developers to start concurrently the successive(andthus the next) activity based on incomplete
documents developed by current activity. The
degree of incompleteness is controlled by completion rate. All these attributes will be taken
into a Project model later.
3.2 Key concept “Workload”
Generally speaking, effort is used to measure
the amount of the activity. But, the effort
doesn’t become clear until the activity is over,and thus the effort is not suitable to determine
the amount of uncompleted activity. Additionally, the effort includes not only the amountof work needed for purely execution of the activity, but also the amount of communication
among the developers. For example, let us consider an activity of 10 person-days. Even if thisactivity is performed by a developer in 10 days,
it could not be performed by 10 developers ina day. One of the main reason is that time tocommunicate among the developers increases as
the number of the developers increases.
Here, we define the term “workload” of anactivity as total time needed for a developer
who has the average capability to complete theactivity. The similar concept to workload has
been presented in1) . Our “workload” could beconsidered as an actual instance of the conceptin1) . Additionally, an efficiency of the activity under such a condition is quantified as 1.
The value of efficiency depends on the environment, such as the number of the developers, thenecessity of communication and performance of
CASE tools. Then, the development time is calculated as the result of dividing the workloadby the efficiency of the activity.
[Example 1] Consider the following two casesof an activity whose workload is 20 hours.
Case 1: Two developers, with the standard
capability execute the activity and ten percent of the total development time is spentVol. 42

No. 3

Software Project Simulator

for communication.
For this case, if each developer takes partin the activity for 10 hours, then the attained workload becomes 18 (=10×2×0.9).
Case 2: Four developers, with the standard
capability, execute the activity and twenty
percent of the total development time isspent for communication.
For this case, if each developer takes
part in the activity for 5 hours, then the attained workload becomes 16 (=5×4×0.8).
If we get the workload of an activity, then we
can estimate the development time appropriate for specific several activity conditions dependent on a given environment.
In the proposed model, the workload is assigned to each activity depending on the input
products for the activity. That is, for example,
workload of design activity (Wdesign ) is definedas the following formula:Wdesign = sdesign × Kdesign .
Here, sdesign denotes the size of input product of design activity and Kdesign denotes theworkload parameter for design activity. Before
simulation, workload parameter must be given
to each activity of the target project.
Consuming of the workload assigned to an
activity corresponds to the progress of the activity in the development. Growth of productcan be modeled by changing values of the sizeor the number of faults in the output product.









Project Model

activities {Ai}products {Pi}developers {Mi}deliverfiring rates oftransitions

update attributevalues of Ai, Pi, Mi





test

designcodingreview

debug

Process Model

Fig. 1 Structure of the proposed model

3.3 Structure of new model
The proposed model consists of a Projectmodel and a Process model. Figure 1 shows

399

Attributes of activity Ai
type
entry condition
exit condition
input products
output products
workforce
deadline
workload
Attributes of product Pi
size
number of faults
completion rate
Attribute of developer Mi
experience level
Fig. 2 Project template

the structure of the proposed model.
The Project model includes three key components: activities, products and developers.
Some attributes are attached to each of them,as shown in Figures 2.
The Process model includes a set of Activity
models which include specifications of design,coding, review, test, debug activities, and soon.
3.4 Project model
The Project model focuses on three key components: activities, products and developers,
and attaches several attributes to each of them(See Figure 2).
An activity has eight kinds of attributes,
which are type, entry/exit conditions, input/output products, workforce, deadline andworkload. (1) Type shows which the activity
corresponds to and describes currently one ofdesign, coding, review, test and debug. (2)Entry condition and (3) exit condition specify
conditions for beginning and ending the activity, respectively. (4) Input products describe theproducts given to the activity as the input products and the degree of contribution of each input products to determine workload of the activity. (5) Output products describe the output
products that are developed in the activity andthe weight assigned to each product. The variation of the product size and that of the numberof faults are distributed to the products according to weights. Thus, the sum of each weightmust be one. (6) Workforce specifies tuples ofthe developers who engage in the activity andthe ratio of time in which each developer can en-

400Transactions of Information Processing Society of Japan

A1
type
entry condition
exit condition
input products
output products
workforce
deadlineFD(A1, non-executed)(A1, consumed)(P0, 7.0)(P1, 0.3), (P2, 0.5), (P3, 0.2)
(M1, 1.0)
20

A2
type
entry condition
exit condition
input products
output products
workforce
deadline

PG(A1, done)(A2, consumed)(P1, 1.2)(P4, 1.0)(M2, 1.0), (M3, 1.0)
35

A3
type
entry condition
exit condition
input products
output products
workforce
deadline

PG(A1, done)(A3, consumed)(P2, 1.1)(P5, 1.0)(M1, 1.0)
35

P0
size
number of faults
completion rate

8
0
1.0

M1
experience level3

M2
experience level2

M3
experience level

1

Fig. 3 Example of project description

rcm
t1
r
th
t2
P1

t3

rw r

P2

transition

t1

executionfunction

-

t2

t3

s=s+1
w =w +1 f=f+1 (p )inFig. 4 Activity model

gage in the activity to his or her business hours.
(7) Deadline represents the appointed date forthe completion of the activity, which is fixed on
the development plan. (8) Workload represents
a tuple of the workload assigned to the activity
and consumed amount of it.
Next, a product has three kinds of attributes,

Mar. 2001

which are size, the number of faults and completion rate. (1) Size represents the product size in
document pages or the lines of source code. (2)Number of faults counts faults in the product.
(3) Completion rate represents the ratio of theconsumed workload to the assigned workload.
Then, a developer has an attributes experience level which is determined according tohis/her length of service. We classify developers’ experience levels into the following threelevels: novice, standard and expert levels. They
are quantified as discrete values 1, 2 and 3, respectively.
[Example 2] Figure 3 shows an example of
the Project model description. This project iscomposed of three activities (A1 , A2 , A3 ), six
products and three developers. Now, let us
explain the description of activity A1 . Typeshows that the activity A1 is a design activity. Entry c.(the condition for starting the activity) represents that only if A1 has not been
started, it can be started at any time. Exitc.(the condition for ending the activity) represents that if all of the workload assigned to
A1 are consumed, its execution ends. Input
p. and output p. show input and output
products of the activity, respectively. Input
p.(= (P0 , 7.0)) represents that the product P0
is given to A1 as an input product, and theworkload equivalent to seven times as much as
the size of the product P0 is assigned to A1 .
Output p.(= (P1 , 0.3), (P2, 0.5), (P3, 0.2)) represents that A1 develops three products P1 , P2
and P3 , and the increase or decrease of the sizeand faults is distributed to P1 , P2 and P3 ina three-five-two ratio, respectively. Workforce
represents that the developer M1 engages in A1
at the full rate of his/her business hours. Deadline(the appointed date for the completion of
the activity) represents that the deadline of A1
is specified to be 20 days after the beginning of
the project.
The description of product P0 shows that thesize of P0 is 8 pages, no fault exists in it anddevelopment of P0 is fully completed in termsof excluding omissions from the description.
The descriptions of developers M1 , M2 andM3 show that their experience levels are 3(expert), 2(standard) and 1(novice), respectively.
Note that workloads of activities A1 , · · · , A3
and attributes of all products except for the initial input product P0 are determined during theexecution of the model. Thus, they are not yet

Vol. 42

No. 3

Software Project Simulator

specified in Figure 3.
Parallel execution of several activities can beeasily defined by utilizing both attributes entry condition of activity and completion rate ofproduct. With respect to example of the parallel execution, please refer to10) .
3.5 Activity model
An activity model is prepared for each typeof activities such as design, coding, review, test,
debug and so on. The descriptions of the Activity models are given using an extended GSPN.
Figure 4 shows an example of the descriptionof the design activity. In the extended GSPN,
a token has three attributes: product size s,
number of faults f and consumed workload w.
These attributes are used to represent the current status of development that varies over the
execution of each Activity model.
Transitions used here are timed transitions.
The firing delay of each transition is exponentially distributed and the average firing delay of
a transition is specified by a firing rate assigned
to it. In Figure 4, the firing rate rcm of transition t1 means that the average firing delay oftransition t1 is 1/rcm .
In addition, each transition has a function(called execution function) to be evaluated on
its firing. The execution of the function updates
the attribute values of the token. Intuitively
speaking, each transition corresponds to the developers’ behavior such as thinking, writing andcommunicating or an event which occurs during execution of activity. Places correspond to
waiting states for occurrences of behaviors or
events.
[Example 3] Figure 4 shows a descriptionof the design activity.
Here, we consider
three kinds of developers’ behaviors in the design activity: communicating among developers, thinking for problem solution and writingfor putting down the solution in documents.
Transitions t1 , t2 and t3 in Figure 4 correspondto communicating, thinking and writing and are
given the firing rates rcm , rth and rwr , respectively.
The firing rates of the transitions are formulated by the following ten functions fcm , fth ,fwr , fpr , frd , fdt , fmd , fps , flc and fin . These
functions should be concretely specified basedon the property of the target project.
In the following, M is the number of the de-401velopers who engage in the activity, L is developer’s experience level, ΣL is the sum of eachdeveloper’s experience, S is the total size of the
input products, R is the completion rate of the
input products, F is the number of faults of the
input products, D is the number of the days
from the current date to the deadline of theactivity. Kcm , Kth, Kwr and Kin are parameters given to each activity and concerned with
communicating, thinking, writing and fault injection rate, respectively☆ .
(1) Communicating rate rcm
rcm = fcm (M, ΣL, R)(2) Thinking rate rth
rth = fth (M, ΣL)(3) Writing rate rwr
rwr = fwr (M, ΣL)(4) Preparing rate rpr
rpr = fpr (M, ΣL, S)(5) Reading rate rrd
rrd = frd (M, ΣL)(6) Fault detecting rate rdt
rdt = fdt (M, ΣL, S, F )(7) Fault modifying rate rmd
rmd = fmd (M, ΣL)(8) Testcase passing rate rps
rps = fps (M )(9) Fault localizing rate rlc
rlc = flc (M, ΣL, S, F )These make it possible to dynamically determine the frequency of communications or
the difficulty in thinking and writing according
to the number of developers, experience levelsof developers and/or completion rates of input
products.
Moreover, the increase of product size s at
every firing of writing transition t3 and the consumption of workload at every firing of thinking
transition t2 are described by the corresponding
execution functions. At each firing of the transition, the values of token’s attributes can bechanged by evaluating its execution function.
The activity model handles fault injectionsin the design activity as the stochastic events
whose occurrences depend on the fault injection
rate pin . In general, pin is formulated by thefollowing function:(10) Fault injection rate pin
pin = fin (M, ΣL, D, R)
By using this function, it is possible to take☆

The functions fcm , fth , fwr and fin should be concretely given based on the characteristics of the
project to be applied.

402

Transactions of Information Processing Society of Japanaccount of dynamic influence on the fault injection rate caused by the stress from deadline of
the activity or developers’ experience levels.
Though the functions (1)–(3) are used in Figure 4, the rests (4)–(9) are not included in Figure 4. Figure 4 shows an example of designand coding activities and thus the improvement of model should be conducted throughcase studies. Besides we also modeled other activities(review, test and debug), in which thefunctions (4)–(9) are used. The detail of theactivity models is shown in11) .
[Example 4] In the design Activity model
depicted in Figure 4, for example, transitions
t1 and t2 which represent communicating andthinking behavior, respectively, are enabled tofire when a token exists in the place P1 . If thecommunicating transition t1 fires, it has no effect on the attributes values, and the token returns to the place P1 and only time elapses by
the firing delay. On the other hand, if the transition t2 fires by evaluating its execution function, then consumed workload w is increased
by one, and the token moves to the place P2 .
When the token exists in the place P2 , only thetransition t3 which represents writing behavior
is enabled. If the transition t3 fires, then product size s is increased by one, and the numberof faults f could be increased according to thefault injection rate pin . After the firing of t3 ,the token moves back to the place P1 .
3.6 Simulation by model
The development process specified by themodel is carried out by repeating the interaction between the Project model and the Activity models at specified intervals. Each interaction cycle consists of the following three steps:
Step 1. Based on values of attributes of activities, products and developers, the Projectmodel computes the firing rates of transitions of the Activity models.
Step 2. The Project model delivers the firing
rates to the corresponding activity model,
and then the Process model executes theprocess described by GSPN.
Step 3. The Process model returns the execution results to the Project model. Then,the Project model updates relevant attributes of activities and products based on
the returned values.
4. Simulation Environment
In order to quantitatively evaluate software

Mar. 2001

processes described by the proposed model,
a simulation environment which executes theprocess automatically is indispensable. We
have designed and implemented a simulator
which supports description of the target process, executes the process described by the activity model and analyses the simulation results
statistically. In this Section, we explain theoverview of the simulator. The experimental
evaluation of the simulator will be described in
Section 5.
Project manager

Data flow
Control flow

User Interface Unit
EditorProject
Description

Project
Description

Project Control Unit

Project
DB

Activity Simulator
Description of
Activity model

Simulation
results
Simulation
results

Display Unit

Simulation
DB

Petri-net
DB

Fig. 5 System architecture

4.1 System architecture
Figure 5 shows the system architecture of thesimulator. The system consists of five functional units: project control unit, activity simulator, user interface unit, display unit and editor. In Figure 5, solid lines represent data flows
and dotted lines represent control flows, respectively.
The followings are the function of each unit.
(1) Project control unit: Project control unit
decides which Activity model is to be simulated by activity simulator according to theproject description. It defines the relationship among activities, products and developers of the Project model, and sets thevalues of attributes of each activity, product and developer. Next, it delivers thename of the Activity model and the values of parameters to the activity simulator.
When it receives the results of simulation,
it updates the values of attributes.
(2) Activity simulator: Activity simulator
simulates activities specified by the Activ-

Vol. 42

No. 3

Software Project Simulator

ity model using data such as name of the
Activity model and parameters given fromproject control unit. At first, it gets an
Activity model with the same name from
Petri-net database. Next, it simulates activities specified by the model using given
parameter values. The results of simulation are returned to project control unit atregular intervals and are stored in the simulation database.
(3) User interface unit: User interface unit
manages exchanges of data or commands
between user and system(editor, projectcontrol part and display part).
(4) Display unit: Display unit displays thedata received from the activity simulator.
It can also provide the data about the previous simulation results stored in the simulation database. The data include graphical information and statistical analysis of
the simulation results.
(5) Editor: Editor supports to create theproject description which is an input ofproject control unit. By using this editor,
we can describe a project and set up allparameters needed in the proposed model.
The output of editor is stored in the project
database.
Since high speed computation is necessary forthe project control unit and activity simulator,
we implemented them using C language. On
the other hand, the display unit and the editor, for which user-friendliness is strongly desirable, are implemented using Tcl/Tk. The
program size becomes about 3500 lines (C language: 1000 lines, Tcl/Tk: 2500 lines).
4.2 Behavior of simulator
Simulations proceeds at the intervals of unit
time☆ . At first, project control unit determines
activities to be executed, based on the current
status of the progress and entry/exit conditionsof each activities. Next, for each executable
activity, project control unit delivers the parameters to activity simulator and directs it toexecute activities for a day. Then, activity simulator executes all of the activities, which are
directed to execute by project control unit, using given parameters and extended GSPN. The
execution of an activity is expressed by the consumption of its workload. When an activity
consumes all of assigned workload, the activity
is regarded to be completed.
☆

Currently, one unit time is a day (8 hours).

403

The execution of simulation is able to be suspended or restarted at any time. Moreover,at every unit time of simulation, intermediate
simulation results can be stored in the simulation database. The intermediate simulation
results are stored in the same format of theoriginal project description. Thus, it is possible to restart the simulation using the intermediate simulation results as the input. Also
it is possible to change the values of parameters(attributes of project) at any time of thesimulation. For example, we can modify the
number of developers at any time of the simulation and simulate it immediately.

Fig. 6 Example of simulation

Figure 6 shows the execution example of simulation which is in progress. In the main window, we can see the progress of the simulation.
We can also get various information from thewindow such as control flow between activities,
consumed/assigned workload of each activity,
assigned developers on a activity, size of product, the number of residual faults in productand so on. These information supports to investigate the progress of simulation in various
ways. In Figure 6, another window graphicallyshows the change of residual faults.
5. Empirical Evaluation
In order to evaluate the usefulness of the proposed method, we conduct an empirical evaluation. In the experiment, we apply the simulator
to three similar software development projects
P R1 , P R2 and P R3 in OMRON corporation.
5.1 Assumptions
Here, we formulate each firing rate and fault
injection rate in the design and coding Activity models on the following assumptions (H1)(H3). In the following formulas, M is the number of the developers engaged in the activity,

404Transactions of Information Processing Society of Japan

ΣL is the sum of each developer’s experiencelevel, R is the completion rate of the input
products, D is the number of the days fromthe current date to the deadline of the activity.
Kcm , Kth , Kwr and Kin are parameters given toeach activity and concerned with communicating, thinking, writing and fault injection rate,
respectively.
(H1) Communicating rate rcm is proportional
to the squared number of developers and inversely proportional to the developers’ experience levels and completion rates of the
input products.
M2
rcm = Kcm ×
ΣL × R
The validity of (H1) comes from the followings: (1) The number of communication
paths among M developers is M (M −1)/2,
and a novice developer needs more communications because of his/her immatureknowledge, and (2) Incomplete input products induce frequent inquiries about theomission of the description6) .
(H2) Thinking rate rth and writing rate rwr
are proportional to the average developer’s
experience and the number of developers.
This assumption is based on the assertion
that the individual capability of the developer is strongly related to the productivityof software15)21) .
ΣL
rth = Kth ×× M = Kth × ΣLM
ΣL
× M = Kwr × ΣL
rwr = Kwr ×
M(H3) Fault injection rate pin is proportional
to number of developers and inversely proportional to the average experience level ofthe developers, completion rate of the input products and the number of the days
from the current date to the deadline of theactivity.
M
pin = Kin ××M
ΣL × R × D
The validity of (H3) comes from the followings: (1) The individual capabilities ofdevelopers are also related to the quality ofsoftware15)21) , (2) Incompleteness of products prevents developers from performing
their work correctly, and (3) The analysisin8) shows that mental stress by deadlines
is the largest cause of generated faults.
As spaces are limited, we omit the descriptionof the Project model, formulas of firing rates,and parameters used by other activity models.

Mar. 2001

5.2 Characteristics of target projects
The main characteristics of the projects are
summarized as follows:(1)Development effort is 170-330 person-days
per project.
(2)The size of the system is about 15K steps.
(3)Project members are almost unchanged
through all projects.
(4)Each project uses a standard waterfall
model.
Table 1 Analysis result of simulation
Development
Development
duration
effortSimulation
242
312(St. dev.)(6.54)(12.08)Actual value
248
329

Residual
faults15(4.21)
26

5.2.1 Outline of the experiment
In order to execute the process specified by
the process description, it is necessary to determine the values of parameters for every activity based on the collected data of projects P R1
and P R2 . Now, however, we cannot obtain allof the necessary data, and thus we must initiate
the simulation of the development process using
virtual data for some parameters. For example,
with respect to the parameters Kcm , Kth , Kwr
and Kin of the design activity, we assign thesuitable values at first and, then, changed thevalues so that the simulated results of the design activity at P R1 and P R2 become the sameas the actual data of it. Also, we calculated
the value of some parameters deterministicallyusing actual input data. For example, with respect to the parameter Input product rate ofeach activity, we can calculate it using the actual data of effort and product size of P R1 andP R2 . Then, we get a common project description for those similar projects P R1 and P R2.
Then, we describe the project P R3 by adding
some attributes, that is peculiar to P R3(e.g.
the number of developers), to the common
project description. Successively, we simulate
it on the simulator and get the estimated valueof P R3 with respect to the development duration, development effort and residual faults. In
the case study, we repeated the simulation one
hundred times and calculated the average values of the development duration, development
effort and residual faults of P R3 .
Finally, we compare these estimated values of
P R3 with the actual values of P R3 .

Vol. 42

No. 3

Software Project Simulator

5.2.2 Simulation results
Table 1 shows both the estimated and theactual values of project P R3 . In Table 1, estimated values of development duration, development effort and the number of residual faults
are 242(days), 312(person-days) and 15, respectively. On the other hand, actual values of themare 248(days), 329(person-days) and 26, respectively.
For the development duration and development effort, the estimated values are quite close
to the actual values. On the other hand, for thenumber of residual faults, the difference (=11)
is about three times as large as the standard
deviation (=4.21).
Here, we investigate the reason why the error occurs in estimating the residual faults. As
the results of examining the data of P R1, P R2
and P R3, it is found that for projects P R1 andP R2 , the average number of developers allocated to the test and debug activities was 10.5.
On the other hand, for project P R3 , the average number of developers in the activities was
20. We consider that this developer allocation
plan induces the error in estimating the residual faults. Though we discussed the bad influence of communication overhead in the earlier
sections, the good influence of increasing thenumber of developers appeared in the simulation results remarkably. We expect that theaccuracy of the estimation can be improved byrevising the equation of the fault injection rate.
It is one of the important future research works.
6. Discussions/Future works
6.1 Parameter setting of the proposed
model
Before simulating the target project, we mustcustomize the simulator by tuning up the values of parameters so that each activity in the
project can simulate actual situation. But, it
is generally very hard to determine parameters, since these are tightly related each other.
Therefore, in this paper we used heuristic values in Section 5. It is necessary to develop thesystematic method or algorithms to determine
the parameters. We consider that the valuesof parameters can be determined by stepwise
method. In16) , we empirically found certain relationships between the parameters. We thenchose several projects for the parameter determination, and determined the values of parameters for each project so that the results of simulation became the same as the actual results.

405

We are going to generalize the stepwise method
to efficiently determine the parameters of the
proposed model.
6.2 Application to other projects
In the actual process improvement activity,
usually several action plans are appeared as
candidates. In such case, it is necessary to identify the most appropriate plan for the current
software development. In such situation, byusing our simulator, we can choose the mostappropriate one. In17) , we used the proposed
simulator to compare two strategies of projectplanning in software development process. One
is to construct the initial project plan at the
beginning of project, and execute whole project
under the initial project plan. The other is toconstruct the initial project plan at the beginning of project and execute the project under
it until the end of design phase, then, construct
the revised plan based on the data from designphase and execute the rest of the project baseon the revised plan. Clearly, the latter strategy
is more appropriate than the former. However,it needs a great deal of effort to collect the datafrom the design phase and revise the plan. So,
the managers generally don’t want to adopt thelatter strategy. If it is necessary for the managers to adopt the latter strategy, we must show
the effect of it. From the results of project simulation, we have confirmed the effectiveness of
the latter strategy. We are going to use our simulator in actual software process improvement
activities in OMRON corporation.
6.3 Tailoring the proposed model toother software organizations
Currently, the proposed model has been built
for the software development process in OMRON corporation. Based on the data and experience from OMRON corporation, we determined the details of the model (For example,
the several attributes of the project templatein Figure 2 and the description of the activity
model in Figure 4). In order to apply the modelto the software development processes in otherorganizations, we have to tailor the model. For
example, we prepared five kinds of the activity
models (design, coding, review, testing, debug).
If necessary, we should reconstruct them andadd other kinds of the activity models. Also,in the activity models, the firing delay of eachtransition is exponentially distributed. It may
be appropriate to modify the distribution tonormal distribution and so on.

406Transactions of Information Processing Society of Japan

7. Conclusion
We have proposed a new model for software
project which can evaluate the software processfrom the viewpoints of the quality, cost and development period. In the new model, by introducing the concept of workload and the attribute of completion rate, it is possible to evaluate the dynamic aspect of software project.
Next, we have developed an integrated simulator to support the estimation of software
project based on the proposed model. Finally,
we have applied the simulator to real software
projects and compare the estimated values withactual data. The experimental results show theapplicability of the proposed simulator to manage real software project.
Up to the present, numerous studies in software engineering develop new methods, tools,or techniques to improve some aspect of software development or maintenance. However, it
has been very difficult to introduce them to theactual software development. One of the reason
is that relatively little evidence has been gathered on which of these new developments are
effective23) . We consider that the collaboration
research between industry and academia in software engineering may be one solution to above
problem and our results show a good suggestion
to efficiently introduce the software engineering
technique to the actual software organization.
We would like to continue the collaboration research to develop a framework for effective technology transfer.
References
1) T. K. Abdel-Hamid: “The dynamics of software project staffing: A system dynamics based
simulation approach”, IEEE Trans. of Software
Engineering, Vol.15, No.2, pp.109-119 (1989).
2) P. Armenise, S. Bandinelli, C. Ghezzi and A.
Morzenti: “Software processes representation
languages: Survey and assessment”, Proc. of
4th Conf. Software Engineering and Knowledge
Eng., pp.455-462(1992).
3) S. C. Bandinelli, A. Fuggetta and C. Ghezzi:“Software process model evolution in the
SPADE Environment”, IEEE Trans. Software Engineering, Vol. 19, No.12, pp.11281144(1993).
4) V. R. Basili and H. D. Rombach: “The TAME
project: Towards improvement-oriented software environment”, IEEE Trans. Software Engineering, Vol.14, No.6, pp.758-773 (1988).

Mar. 2001

5) F. P. Brooks, Jr.: The Mythical Man-Month,
Addison-Wesley (1975).
6) B. Curtis, H. Krasner and N. Iscoe: “Afield study of the software design process forlarge systems”, Communications of the ACM,
Vol.31, No.11, pp.1268-1287 (1988).
7) K. Furusawa, Y. Hirayama, S. Kusumoto
and T. Kikuno: “Modeling and quantitativeevaluation of software process based on a
Generalized Stochastic Petri-net”, Proc. of
15th Software Reliability Symposium, pp.99104 (1994)(in Japanese).
8) T. Furuyama, Y. Arai and K. Iio: “Fault generation model and mental stress effect analysis”, The Journal of Systems and Software,
Vol.26, pp.31-42 (1994).
9) M. V. Genuchten:“Why is software late? An
empirical study of reason for delay in softwaredevelopment”, Trans. on IEEE Softw. Eng., 17,8, pp.582-590(1991).
10) Y. Hirayama, O. Mizuno, S. Kusumoto andT. Kikuno: “Hierarchical project management
model for quantitative evaluation of software
process”, Proc. of the International Symposium
on Software Engineering for the Next Generation, pp.40-49 (1996).
11) Y. Hirayama: “Quantitative evaluation ofsoftware process based on hierarchical project
management model,” Master dissertation, Osaka University, 1996.
12) A. Johnson:“Software process improvement
experience in the DP/MIS function”, Proc. of
ICSE16, pp.323-329(1993).
13) M. I. Kellner: “Software process modeling support for management planning and control”,
Proc. of the 1st International Conference on
Software Process, pp.8-28 (1993).
14) G. Lee and T. Murata: “A β-distributed
stochastic Petri net model for software projecttime/cost management”, The Journal of Systems and Software,Vol.26, No.2, pp.149-165(1994).
15) K. Matsumoto, S. Kusumoto, T. Kikuno andK. Torii: “An experimental evaluation of teamperformance in program development basedon model – Extension of programmer performance model”, Journal of Information Processing, Vol.15, No.3, pp.466-473 (1992).
16) O. Mizuno, S. Kusumoto and T. Kikuno:“Customization of software project simulator
for improving estimation accuracy, Proc. ofProc. of 9th International Symposium on Software Reliability Engineering, Vol.2, pp.47-48(Paderborn, Nov. 1998).
17) O. Mizuno, S. Kusumoto, T. Kikuno, Y. Tak-

Vol. 42

No. 3

Software Project Simulator

agi and K.Sakamoto: “Experimental evaluationof two-phase project control for software development process”, IEICE Transactions on Fundamentals of Electronics, Communications andComputer Sciences, Vol.E81-A, No.4, pp.605614(April, 1998).
18) M. C. Paulk, W. S. Humphrey and G. J.
Pandelios:“Software process assessments: Issues and lessons learned”, Proc. of ISQE92,
pp.4B41-4B58(1992).
19) J. L. Peterson: Petri Net Theory and the Modeling of Systems, Prentice-Hall(1981).
20) D. M. Raffo: “Evaluating the impact of process improvements quantitatively using processmodeling”, Proc. of CASCON93, Vol.1, pp.290313 (1993).
21) H. Sackman, W. J. Erickson and E. E. Grant:“Exploratory experimental studies comparing
online and offline programming performance”,
Communications of the ACM, Vol.11, No.1,
pp.3-11 (1968).
22) T. Tanaka, K. Sakamoto, S. Kusumoto, K.
Matsumoto and T. Kikuno: “Improvement ofsoftware process by process description andbenefit estimation”, Proc. of the 17th International Conference on Software Engineering,pp.123-132 (1995).
23) W. F. Tichy, N. Harbermann and L.
Prechelt:“Future directions in software engineering,” ACM SIGSOFT, Software Engineering Notes, Vol.18, No.1, pp.35-48(1993).
24) J. D. Tvedt and J. S. Collofello: “Evaluating the effectiveness of process improvements
on software development cycle time via system
dynamics modeling”, Proc. of COMPSAC95,
pp.318-325 (1995).

407

Osamu Mizuno was born in
1973. He received B.E. and M.E.
degrees in information and computer sciences from Osaka University in 1996 and 1998, respectively. He has been working for
Osaka University since 1999. He
is currently a Research associate in the Department of Informatics and Mathematical Science at Osaka University. His research interests
include the software process and the software
quality assurance technique. He is a member of
the IEEE.
Tohru Kikuno was born in1947. He received M.S. and
Ph.D. degrees from Osaka University in 1972 and 1975, respectively. He joined Hiroshima University from 1975 to 1987. Since
1990, he has been a Professor
in the Department of Informatics and Mathematical Science at Osaka University. His research interests include the quantitative evaluation of software development processes andthe analysis and design of fault-tolerant systems. He served as a program co-chair of the 1st
International Symposium on Object-Oriented
Real-Time Distributed Computing (ISORC’98)
and of the 5th International Conference on
Real-Time Computing Systems and Applications (RTCSA’98). He also served as a generalco-chair of the 2nd International Symposium on
Object-Oriented Real-Time Distributed Computing (ISORC ’99).

(Received ??? ??, ????)(Accepted ??? ??, ????)
Shinji Kusumoto was bornin 1965. He received the B.E.,
M.E. and D.E. degrees in information and computer sciencesfrom Osaka University in 1988,
1990 and 1993, respectively. He
is currently Associate Professor
in the Department of Informatics and Mathematical Sciences, Graduate School of Engineering Science, Osaka University. His research interests are software metrics, software quality assurance technique. He is a member of the IEEE,
IEICE and JFPUG.

Yuji Hirayama was born in
1971. He received B.E. and M.E.
degrees in information and computer sciences from Osaka University in 1994 and 1996, respectively. He has been working for
OMRON Corporation.
Yasunari Takagi received
B.E. degree in information andcomputer science, from Nagoya
Institute of Technology in 1985.
He has been working for OMRON Corporation.

408

Transactions of Information Processing Society of Japan

Keishi Sakamoto received
B.E. degree in electrical engineering, from Kobe University in1969 and D. E. degree in information science, from Nara Institute of Science and Technology
in 2000. He had been working
for OMRON Corporation from 1969 to 2000.
Currently, he is Software Process Improvement(SPI) consultant.

Mar. 2001