Productivity Analysis of Japanese Enterprise Software
Development Projects
Masateru Tsunoda

Akito Monden

Hiroshi Yadohisa

Nara Institute of Science andNara Institute of Science andTechnologyTechnology
Kansai Science City, 630-0192 Japan Kansai Science City, 630-0192 Japanmasate-t@is.naist.jp

akito-m@is.naist.jp

Doshisha University
1-3 Miyakodani, Tatara, Kyotanabe,
Kyoto, 610-0394 Japanhyadohis@mail.doshisha.ac.jp

Nahomi Kikuchi

Ken-ichi Matsumoto

Software Engineering Center
Information-technology Promotion Agency
2-28-8, Hon-Komagome,
Bunkyo-ku, Tokyo, 113-6591 Japan

Nara Institute of Science andTechnology
Kansai Science City, 630-0192 Japan

matumoto@is.naist.jp

n-kiku@ipa.go.jp
ware productivity is one of the key factors in drawing up the early
effort estimation in a project, although the productivity greatly
varies according to project attributes such as business area, application type, programming language, team size and experience, etc[8]. Past researches have shown that some attributes do affect theproductivity [2][7][8][9]. Unfortunately, many of these attributes,
e.g. business area and programming language, are usually notcontrollable by a software development company.

ABSTRACTTo clarify the relation between controllable attributes of a software development and its productivity, this paper experimentally
analyzed a software project repository (SEC repository), consisting of 253 enterprise software development projects in Japanesecompanies, established by Software Engineering Center (SEC),
Information-technology Promotion Agency, Japan. In the experiment, as controllable attributes, we focused on the outsourcing
ratio of a software project, defined as an effort outsourced to subcontract companies divided by a whole development effort, andon the effort allocation balance among development phases. Ourmajor findings include both larger outsourcing ratio and smaller
upstream process effort leads to worse productivity.

This paper focuses on the project attributes that are controllable
by a software development company. Through the analysis of theyear 2005 version of the SEC repository consisting of 253 samples of enterprise software development in Japanese companies,
this paper seeks to clarify the relation between project attributes
and software development productivity. The repository has beendeveloped and maintained by Software Engineering Center (SEC),
Information-technology Promotion Agency, Japan [10].

Categories and Subject Descriptors
D.2.9 [Software Engineering]: Management – Cost estimation,productivity; K.6.1 [Management of Computing and Information Systems]: Project and People Management – Strategic information systems planning;

In the analysis, as controllable project attributes, this paper focuses on the outsourcing ratio of a software project, and on theeffort allocation balance among development phases. While highoutsourcing ratio is one of the major distinctive properties ofJapanese enterprise software development, there is no past researches focused on the relation between outsourcing ratio andproductivity of software development in Japan.

General Terms: Management, measurement, economics
Keywords
Software productivity, subcontract, upstream process, customsoftware, software project repository

The reminder of this paper first describes details of the SEC repository we used (Section 2). Next, describes analyses we conducted to clarify the relation between project attributes and productivity (Section 3). Afterward, related works are described (Section 4); and in the end, conclusions and future topics will beshown (Section 5).

1. INTRODUCTION
Estimation of software development effort is required throughouta software development lifecycle to set up, evaluate and revise aproject plan including resource allocation and scheduling. Soft-2. THE SEC REPOSITORY
The year 2005 version of the SEC repository consists of 1009
software projects held in 15 Japanese companies. These projects
are custom enterprise software, which is the majority in Japan[4][5]. Each project is characterized by about 400 attributes (metrics); however, 87.7% of them were unrecorded on average. Since
the project size is a necessary attribute to calculate the productivity, we excluded projects having missing value in the functionPermission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,requires prior specific permission and/or a fee.
MSR’06, May 22–23, 2006, Shanghai, China.
Copyright 2006 ACM 1-59593-085-X/06/0005...$5.00.

14666

Producti vity

0.60

6$
$
$
$0.40In the analysis, we mainly focused to two factors to analyze productivity. One is the outsourcing ratio, defined as an effort outsourced to subcontract companies divided by a whole development effort. The other is the upstream process ratio, defined as thesum of requirement analysis process effort and design processeffort divided by a whole development effort. In many cases, thesetwo factors are controllable by a software development company.

666$

$
$
$
$
$
$
$
$
$

0.20

3. PRODUCTIVITY ANALYSIS
0.00

n=64

n=125

n=64

n=253

LowMiddleHighOv erall

There are several factors that may influence software productivity.
One of the most considerable factors is the project size [1][3]since development processes (e.g. human resource allocation)vary according to the project size. Therefore, we also focus to theFP, which is one of project size metrics, in addition to the outsourcing ratio and the upstream process ratio.

Function Point

Figure 1. Boxplots of productivity for project groups classified by FP$$$

Before focusing on the outsourcing ratio and the upstream processratio, we analyzed the influence of FP (project size), which is themost basic factor in productivity analyses.

$$
$

0.60

Producti vity

3.1 Project Size

In this analysis projects were classified into three groups by their
FP. Projects whose FPs are equal to or less than lower quartile
were classified as the “low” group. Projects whose FPs are equal
to or greater than upper quartile were classified into the “high”
group. The rest projects were classified into the “middle” group.

$
$

0.40
0.20
0.00

n=58

n=80

n=21

n=159

LowMiddleHighOv erall

To visually explore the difference in productivity among threegroups, boxplots were used. Figure 1 shows boxplots of productivity for the three project groups (FP=low, FP=middle, andFP=middle) plus “overall” group for whole projects. Circles indicate outliers, and stars indicate extreme outliers. Upper quartile ofFP is about 4.1 times larger than lower quartile. The figure shows
there are several high productivity projects in the low group. Median of productivity of the low group is about 1.4 times larger
than the high group. Median of productivity of the middle groupis only about 1.1 times larger than the high group. Using MannWhitney U test, we confirmed that the difference of productivity
is statistically significant at the 0.05 level between the low groupand the high group. P-values are shown in the column “All” ofTable 1. The column “Group 1” and the column “Group 2” indicate paired group when testing. Italic indicates p-value < 0.05.

Function PointFigure 2. Boxplots of productivity for project groups classified by FP (Outsourcing ratio is zero)
Table 1. P-values for project groups classified by FPGroup 1Group 2All

Low
Low
MiddleMiddleHighHigh

0.072
0.017
0.296Outsourcing
ratio is zero
0.789
0.610
0.744

Next we excluded the influence of outsourcing ratio since there
was a high correlation between FP and outsourcing ratio (Spearman's rank correlation was 0.51). Figure 2 shows boxplots for
projects whose outsourcing ratios were zero. Median of productivity is almost same among the three groups. The differences in
productivity among the three groups are not statistically significant at all. P-values are shown in the column “Outsourcing ratio iszero” of Table 1.

point (FP) attribute from our analysis. Also, we excluded maintenance projects and enhancement projects, and focused to new
development projects and re-development projects because maintenance/enhancement processes are often very different fromnew/re-development processes. As a result, 253 projects wereselected for the analysis. All these projects were waterfall processdevelopment.

Above all, the SEC repository showed that the project size alone
do not directly influence the productivity. However, it can be said
that large size projects have high outsourcing ratio; and thus, it
indirectly leads to worse productivity.

Productivity is defined as FP divided by (total) development effort(person-hour). FP methods consist of IFPUG (34%), SPR (26%),
NESMA (3%), others (31%), and missing (5%). Productivity ofthe upper quartile project is about 3.3 times larger than the lower
quartile project. Because of biased distribution of productivity andother metrics, we used the median of metrics instead of the average in our analysis. We also used non-parametric tests in theanalysis.

3.2 Outsourcing Ratio
Figure 3 shows boxplots of productivity for three project groups
classified by the outsourcing ratio. (Note that the number of projects of the low group is different from the high group because
there are many projects whose outsourcing ratios are zero and all

15

$$
$

Producti vity

0.60$$
$
$
$
$
$
$
$

6$

0.40
$
$

0.20
0.00

ever, these results do not mean that the outsourcing ratio must be
suppressed because the development cost would be higher if acompany stopped outsourcing. (Unfortunately, because the SEC
repository does not record cost factors, we were not able to analyze relations between cost and outsourcing ratio.) In addition, ifthe software size is too large to be developed in-house, a company
needs to outsource a part of software development. Anyway, acompany must be aware of the trade-offs between increase ofeffort and decrease of cost.

666$
$

n=159

n=30

n=63

n=253

LowMiddle

HighOv erall

3.3 Upstream Process Ratio
Figure 5 shows boxplots of productivity for three project groups
classified by upstream process ratio. The figure shows there arehigh productivity projects in the high group. Median of productivity of the high group is about 1.8 times larger than the low group.
The differences in productivity are statistically significant at the0.05 level between the high group and other groups. P-vales areshown in the column “All” of Table 3.

Outsourcing Ratio

Figure 3. Boxplots of productivity for project groups classified by outsourcing ratio$$$

666Productivity

0.60$$$
$

0.40
0.20
0.00

Next we excluded the influences of FP and outsourcing ratio.
Figure 6 shows boxplots for projects whose FPs are in the “middle” group. The differences of productivity are statistically significant at the 0.05 level between the high group and other groups.
P-values are shown in the column “Function point is middle” ofTable 3. Similarly, Figure 7 is boxplots of projects whose outsourcing ratio is zero. The differences in productivity are not statistically significant at the 0.05 level among the three groups. Pvalues are shown in the column “Outsourcing ratio is zero” ofTable 3. Although there was no strong significance among thesegroups, significance p=0.083 (< 0.10) was observed between
middle and high group.

$$n=80

n=17

n=27

n=125

LowMiddleHighOv erall

Outsourcing RatioFigure 4. Boxplots of productivity for project groups classified by outsourcing ratio (FP is middle)

From these results, high upstream process ratio alone has someinfluence to the productivity. This suggests that high upstream
process ratio may avoid additional effort (reworks) on downstream processes.

Table 2. P-values for project groups classified by outsourcing ratio

4. RELATED WORKGroup 1
Low
Low
MiddleGroup 2Middle
HighHighAll
3.2E-04
8.8E-140.002

Maxwell et al. [8] and Premraj et al. [9] analyzed the influence ofthe business sector type on productivity, using Finnish softwaredevelopment project dataset collected by Software Technology
Transfer Finland (STTF). Lokan et al. [6] also showed productivity analysis focused on the business sector using a dataset of International Software Benchmarking Standards Group (ISBSG). In
these researches, projects in the manufacturing sector have thehighest productivity, and projects in the banking/Insurance sector
have the lowest productivity. Same tendency was observed in theSEC repository.

FP is middle
3.9E-04
1.8E-070.104of them are classified as the lower quartile group.) The figure
shows there are high productivity projects in the low group. Median of productivity of the low group is about 2.6 times larger
than the high group. The differences of productivity are statistically significant at the 0.05 level among the three groups. Pvalues are shown in the column “All” of Table 2.

Blackburn et al. [2] analyzed the influence of requirement analysis
process ratio on the productivity. They found that requirement
analysis process ratio leads to higher productivity of coding process. Our finding about the influence of upstream ratio follows
their result.

Next we excluded the influence of FP. As described in Section 3.1,
high FP projects tend to have high outsourcing ratio. Figure 4
shows boxplots for projects whose FPs are in the “middle” group.
Figure 4 shows similar tendency to Figure 3. The differences in
productivity are statistically significant at the 0.05 level between
the low group and other groups. P-values are shown in the column“FP is middle” of Table 2.

5. CONCLUSIONSWe analyzed the influences of outsourcing ratio and upstream
process ratio on the development productivity of Japanese enterprise software development projects. We found that both lower
outsourcing ratio projects and higher upstream process ratio pro-

These results suggest that lower outsourcing ratio projects havehigher productivity. We consider that higher outsourcing ratio
introduces communication overhead between companies. How-16

Producti vity

0.60
0.40

66

6
66

6
66$
$
$$
$$$$$
$
$
$
$
$

$
$

jects have significantly higher productivity. Our future work is toanalyze the influences of other factors thoroughly.

6. ACKNOWLEDGMENTS
The authors would like to thank Software Engineering Center,Information-technology Promotion Agency, Japan for offering theSEC Repository. This work is supported by the EASE (EmpiricalApproach to Software Engineering) project of the ComprehensiveDevelopment of e-Society Foundation Software program of theMinistry of Education, Culture, Sports, Science and Technologyof Japan.

0.20
0.00

n=50

n=99

n=50

n=253

LowMiddleHigh

Ov erall

7. REFERENCES

Ups tream Proc ess Ratio[1] Banker, R., Chang, H., and Kemerer, C. Evidence on economies of scale in software development. Information andSoftware Technology, 36, 5 (1994), 275-282.

Figure 5. Boxplots of productivity for project groups classified by upstream process ratio
666Producti vity0.60

$

60.40

[2] Blackburn, J., Scudder, G., and Wassenhove. L. Improving
Speed and Productivity of Software Development: A Global
Survey of Software Developers. IEEE Trans. on SoftwareEng., 22, 12 (1996), 875-885.

666$

$
$
$
$

[3] Boehm, B. Software Engineering Economics. Prentice Hall,
1981.

$

[4] Cusumano, M., MacCormack, A., Kemerer, C., and Crandall,
B. Software Development Worldwide: The State of the Practice. IEEE Software, 20, 6 (2003), 28-34.

0.20
0.00

n=23

n=45

n=25

n=125LowMiddleHighOv erall[5] Duvall, L. A study of software management: The state ofpractice in the United States and Japan. Journal of Systems
and Software, 31, 2 (1995), 109-124.

Ups tream Proc ess Ratio[6] Lokan, C., Wright, T., Hill, P., and Stringer, M. Organizational Benchmarking Using the ISBSG Data Repository.
IEEE Software, 18, 5 (2001), 26-32.

Figure 6. Boxplots of productivity for project groups classified by upstream process ratio (FP is middle)
$$
$

0.60

Producti vity[7]$$$$$$[8] Maxwell, K., and Forselius, P. Benchmarking SoftwareDevelopment Productivity. IEEE Software, 17, 1 (2000), 8088.

0.40
0.20
0.00

Maxwell, K., Wassenhove, L., and Dutta, S. Software Development Productivity of European Space, Military, and Industrial Applications. IEEE Trans. on Software Eng., 22, 10(1996), 706-718.

n=23

n=67

n=38

n=159

LowMiddle

HighOv erall[9] Premraj, R., Shepperd, M., Kitchenham, B., and Forselius, P.
An Empirical Analysis of Software Productivity over Time.
In Proceedings of 11th IEEE International Software Metrics
Symposium (METRICS'05) (Como, Italy, September), 2005,37.
[10] Software Engineering Center, Information-technology Promotion Agency, Japan The 2005 White Paper on Software
Development Projects (in Japanese). Nikkei Business Publications, 2005.

Ups tream Proc ess Ratio

Figure 7. Boxplots of productivity for project groups classified by upstream process ratio (Outsourcing ratio is zero)
Table 3. P-values for project groups classified by upstream process ratio
Group 1

Group 2

All

Low
Low
MiddleMiddleHighHigh

0.819
0.009
0.012

FP is Outsourcing
middle ratio is zero
0.974
0.595
0.016
0.222
0.011
0.083

17