Customization of Software Project Simulator
for Improving Estimation Accuracy
Osamu Mizuno, Shinji Kusumoto and Tohru Kikuno
Department of Informatics and Mathematical Science,
Graduate School of Engineering Science, Osaka University, Japano-mizuno@ics.es.osaka-u.ac.jp
t1 rcmAbstract
We have developed a Petri-net based project simulator forpredicting development effort and residual fault content. The
simulator requires customization based on four parameters
to reflect project characteristics. In this paper, we propose anew method for determining these parameters and validate
the effectiveness of the method in two industrial experiments.

1. Project Simulator
The software project simulator consists of Project modeland Process model. Project model specifies three key components: activities, products and developers. Process model
defines a set of Activity models that are described by anextended GSPN(Generalized Stochastic Petri-Net).

1.1. Activity Model
Figure 1(a) shows an example of the description of thedesign activity. In the extended GSPN, a token has three attributes: product size s, the number of faults f and consumed
workload1 w, as shown in Fig. 1(b).
Transitions used here are timed transitions. The firing
delay of each transition is exponentially distributed and theaverage firing delay of a transition is specified by a firing rate
assigned to it. In Fig. 1(a), the firing rate rcm of transition t1means that the average firing delay of transition t1 is 1=rcm .
Each transition has a function (called an execution function) to be evaluated on its firing, as shown in Fig. 1(c). The
execution of the function updates the attribute values of thetoken.

1.2. Customization using parameters
The firing rates rcm , rth and rwr of the transitions areformulated by functions fcm , fth and fwr , respectively[2].
Additionally, fault injections are interpreted as the stochastic
events whose occurrences depend on the fault injection rate
pin . In general, pin is formulated by function fin .
1 We define the term “workload” of an activity as the total time needed

for a developer who has the standard capability to complete the activity[2].

t2 rtht3 rwrP2

P1(a)attributes of token

transition

t1

t2

t3

sfw

executionfunction-w = w+1

s = s+1
f = f+1(pin)product size
number of faults
consumed workload

(c)

(b)

Figure 1. Activity model
In the previous study[2], we define concrete functions forall of these formulas. Since, we consider only design andcoding activities in this paper, we present concrete functions
for rcm , rth , rwr and pin as follows:(1) Communicating rate: rcm = Kcm 2 LM
2R2L 2 M = Kth 2 L(2) Thinking rate: rth = Kth 2 M

L 2 M = Kwr 2 L(3) Writing rate: rwr = Kwr 2 M(4) Fault injection rate: pin = Kin 2 L2M
R2D 2 M

Here, M is the number of the developers engaged in theactivity, L is the sum of each developer’s experience level,
R is the completion rate of the input products, D is thenumber of the days from the current date to the deadline ofthe activity.
Before simulating a certain project, we must customize thesimulator by tuning up the values of parameters so that eachactivity in the project can simulate actual situation. But, it isgenerally very hard to determine parameters, since these aretightly related each other. Therefore, in the previous studies,
we used heuristic values shown in Table 1.
Table 1. Old values of parameters[2]DesignCoding

Kcm Kth Kwr Kin
0.10
0.10

0.20
0.20

0.20
0.20

15.5
17.0

3.1. Experiment 1/* Phase 1 */for i := 0:02 to 0:30 step 0:02 do begin
for j := 0:05 to 0:30 step 0:05 do begin
for k := 0:05 to 0:30 step 0:05 do begin#Effort := (the amount of effort obtainedby simulation under (Kcm ; Kth ; Kwr ) = (i; j; k ));if (#Effort is within 5% of Effortactual ) thenStore (i; j; k) as a candidate;
end;
end;
end
Choose the best candidate (Kcm ; Kth ; Kwr )best .
/* Phase 2 */lo := 1:0 ; hi := 100
while lo hi do begin
Kin := (lo + hi)=2 ;#Fault := (the number of faults obtainedby simulation under Kin and (Kcm ; Kth ; Kwr )best );if #Fault is within 5% of Faultactual ) then break;if Faultactual #Fault then hi := Kin else lo := K in ;end6

6Figure 2. Algorithm for determining parameters
Table 2. New values of parametersDesignCoding

Kcm Kth Kwr Kin
0.05
0.04

0.26
0.20

0.22
0.15

18.6
43.8

2. Proposed Algorithm
For customization of the simulator, we assume that anactual project data, for which the data on the amount ofefforts and the number of faults are collected, is given.
Based on the lessons learned from the previous studies, weconsider the following algorithm with two phases: Phase 1 isto determine Kcm , Kth and Kwr , and Phase 2 is to determine
Kin with the values of Kcm , Kth and Kwr obtained atPhase 1.
The outline of the algorithm is shown in Fig. 2 In Fig. 2,variables Effortactual and Faultactual are the target values
for the simulation and are obtained from the actual projectdata.

By applying the data of the project P R1 to the algorithm
shown in Fig. 2, we determine the values of parameters forthe design and coding activities. The values of parameters
obtained are shown in Table 2.
Comparing the values in Table 1 and Table 2, the values ofKcm , Kth , Kwr are almost the same. But the values of Kin
are quite different each other.

3.2. Experiment 2
In order to evaluate the validity of the proposed algorithm,
we estimate the number of faults in another project P R2
by the project simulator using the values of parameters in
Table 2.
Table 3. Simulated result of P R2

Actual project data
New estimation
Estimation in [2]Effort(person-days)
41.7
38.6
37.1

Residual faults(# of faults)
17.0
17.9
13.2

Table 3 shows the simulated result of the project P R2 (Forcomparison, we also show the simulation result in [2]). From
Table 3, the simulated effort is 38.6 (person-days) and theactual effort is 41.7 (person-days). The simulated value ofresidual faults is 17.9 and the actual value is 17. Thus, thenew estimations of both effort and the residual faults for P R2
are very close to the actual data. Furthermore, with respect
to estimation accuracy, the new estimation by the proposed
method is superior to the one in [2].
As a result, we can say that the values of parameters obtained from the project P R1 are applicable to the estimationof the software quality in another project P R2 . Unfortunately, we don’t have any more generalized data yet.

4. Future WorkIn this study, we have discussed only the design and coding
activities. Thus, determining the values of parameters in thetest and debug activities is an important future research work.
Additionally, we should contrast our approach for simulationand prediction with other approach[1].

3. Experimental Evaluation
References
In order to confirm the effectiveness of proposed method,
we have performed two experimental evaluations. In Experiments 1 and 2, we use two actual project data P R1 andP R2 offered from a certain company. Both projects are thedevelopment of the embedded software, and for almost thesame application. The sizes of P R1 and P R2 are 12.3 Kstep
and 9.6 Kstep, respectively.
In Experiment 1, we determine the value of firing parameters from the data of P R1 . Successively, in Experiment 2,
we execute the simulation of P R2 and evaluate the efforts
and residual faults in P R2 .

[1] L. C. Briand, K. E. Emam, B. Freimut and O. Laitenberger: “Quantitative evaluation of capture-recapture
models to control software inspections,” Proc. 8th International Symposium on Software Reliability Engineering, pp.234–244, 1997.
[2] S. Kusumoto, O. Mizuno, Y. Hirayama, T. Kikuno,
Y. Takagi and K. Sakamoto: “A new software project
simulator based on generalized stochastic petri-net,”
Proc. 19th International Conference on Software Engineering, pp.293–302, 1997.