A Health Index of Open Source Projects Focusing
on Pareto Distribution of Developer’s Contribution
Hirohisa Aman∗ , Aji Ery Burhandenny† , Sousuke Amasaki‡ , Tomoyuki Yokogawa‡ and Minoru Kawahara∗∗ Center for Information Technology, Ehime University, Matsuyama, Ehime 790-8577, Japan† Graduate School of Science and Engineering, Ehime University, Matsuyama, Ehime 790-8577, Japan‡ Faculty of Computer Science and Systems Engineering, Okayama Prefectural University, Soja, Okayama 719–1197, Japan

Abstract—Open source software (OSS) products have beenbroadly utilized for the IT business as well as the personal use
in recent years. Software companies can receive much benefit
from OSS products in terms of cost to develop and maintaintheir products. However, there are also risks that products ofinterest might become no longer being successfully maintainedby the OSS developers because a successful maintenance is not
obligation of developers. In order to evaluate a project’s healthfrom a perspective of a macroscopic trend analysis, this paper
focuses on the distribution of the developer’s contribution toan OSS project, and analyzes the relationships of distribution
features with the quality of products. The empirical analysis
with 32 popular OSS projects shows that the length of tail in the
Pareto chart can be a health index of an OSS project in termsof the balance between bug fix and feature upgrade.

I. I NTRODUCTION
Recently, open source software (OSS) products have become more and more popular, and have been widely used invarious areas of information and communication technology.
Leading examples include Linux, Apache HTTP, Firefox andPostgreSQL. These days, many OSS products are utilized for
not only personal purposes but also for business. According toa survey report on the utilization of OSS in software-relatedcompanies [1], 78 percent of the survey respondents said thatthey run part or all of its operations on OSS products. In
addition, 66 percent of respondents also said their software
products are built on OSS ones. Thus, many companies areusing and focusing on OSS products in their business, andthis trend is highly expected to continue.
However, there are also risks associated with the quality
control and maintenance of their software products. For example, if a software company had utilized an OSS product for
running their commercial service, and encountered a failurecaused by a fault in the OSS product, it would be unsure
whether the faulty part of the program would be quickly
detected and fixed in the OSS project or not; The company’sengineers might have to fight the suspicious programs to fix
the fault by themselves, even if they have no experienceor knowledge in regard to maintaining the faulty part ofthe programs. In order to minimize such a risk, software
companies should adopt the OSS products which are expected
to be actively developed and maintained both now and in
the future. Thus, there is a need for evaluating the abilityof the OSS project to steadily run—the “health level” of the
OSS project. While there are some related criteria such as

the popularity and the user assessment, they are based on the
products rather than the project. Recently, there have beensome studies focusing on the project operation, including thecontributions of OSS developers [2]. It would be promising tofocus on the developers in order to discuss the project’s healthbecause their contributions drive the OSS project. While ananalysis of individual’s contribution must be beneficial, sucha detailed analysis tend to be unfit for capturing the entire
spectrum of the project. Therefore, in this paper, we willconduct a macroscopic analysis of the developer’s contribution
to the OSS project, and propose a health index of project.
The key contributions of this paper are the followings.
1) A quantitative analysis of developer’s contribution isconducted for 32 popular OSS projects, and reports theirfeatures from the perspective of the Pareto principle.
2) The analysis result shows that the length of tail in the
Pareto chart can be a useful index for evaluating whether
the OSS project has been developed in a healthy balance
between bug fix and feature upgrade.
The remainder of this paper is organized as follows: Section
II describes available data from OSS projects, and proposes
metrics of the developer’s contribution. Then, Section III
reports on our empirical analysis and discussions. Section IV
briefly discusses our related work. Finally, Section V gives theconclusion of this paper and our future work.
II. OSS P ROJECT AND D EVELOPER ’ S C ONTRIBUTION
A repository of an OSS project is usually available on theInternet, so anyone can get the products and see any changes
that have been made in the past. While an OSS project may
use not only a repository but also a bug tracking system anda mailing list system to manage their bug reports and e-mails
exchanged among the developers, the access to those material
is sometimes restricted to the developers participated in the
project. Thus, in this paper, we will focus on data managed ina repository which anyone can freely access.
We can investigate various data in regard to each commit toan OSS project through the project’s repository, e.g., when thefile change was made, what the difference from the previous
revision was, and who made it. Recently, many studies analyzing repositories have been reported in the Mining Software
Repositories (MSR) community [3]: for example, commit log
is analyzed to predict fault-prone modules for more effective
testings and code review [4], [5], [6].

In this paper, we will analyze the developer’s contribution
in a macroscopic manner. We begin with quantifying eachdeveloper’s contribution by using the following two metrics.
1) Number of Commits made by the developer (NoC), and
2) Ratio of Files in which the developer has been involved(RoF):n(d),RoF(d) =N
where d signifies a developer, N is the number of files
included in the latest version of the product, and n(d)is the the number of files which have been added or
modified by d and are included in the latest version.
Table I presents an example, where 2 developers A andB added or modified 8 files through 10 commits; each row
corresponds to each commit made by the developer shownin the rightmost column, and a check mark (✓) signifies thecorresponding file was added or modified at the time. For
instance, developer A added 4 files whose file numbers are
1–4 at the commit no.1, and developer B did 4 files whose
file numbers are 5–8 at the commit no.2. At the commit
no.5, developer B modified 2 files whose numbers are 2
and 6. In this example, we have NoC(A) = 4 and NoC(B)= 6, because developer A made the commits whose numbers
are 1, 3, 7 and 8, and developer B did the remaining 6
commits. For metric RoF, we obtain the following values:
RoF(A) = 4/8 = 0.5 and RoF(B) = 7/8 = 0.875, since
N = 8, n(A) = 4 and n(B) = 7 as shown in Table I.
The NoC is a simple metric observing the degree of thedeveloper’s contribution—the more commits, the higher the
NoC value. A highly contributing developer would actively
commit files to the repository, and then the developer would
have a high value of NoC. The RoF is a metric to show
how widely the developer has been involved in the product.
If a developer has a high value of RoF, the developer musthave contributed to many parts of the product, which means
there is a high contribution to the project. In this paper, we
try to capture an OSS project’s status of development andmaintenance through the above metrics, and examine whether
they can be useful in evaluating the organization of project.
III. E MPIRICAL A NALYSISThis section conducts an empirical analysis on the above
two metrics as health indexes of OSS projects.
A. Research Questions
Our aim is to analyze the usefulness of our two metrics forassessing the project’s health. While we proposed two different
metrics observing each developer’s contribution, we do not aim
to evaluate the individual developer by using these metrics.
Instead, we will focus on the “distribution” of metric values
rather than the individual’s metric value. Now we tackle thefollowing two research questions.
RQ1: Can the distribution of metric values show a difference among OSS projects?
RQ2: Can the distribution be noteworthy for making a
useful health index of OSS project?

TABLE IE XAMPLE OF 10 COMMITS INVOLVING 8 FILES BY 2 DEVELOPERS .
commit
no.
1
2
3
4
5
6
7
8
9
10
involved
developer

1
✓

2
✓

3
✓

✓
✓
✓
✓
A

✓
A
B

file no.
4
5
✓
✓
✓

✓

✓
✓

✓
A
B

✓
A
B

B

6

7

8

✓

✓

✓

✓

✓

✓

✓

✓

B

B

B

✓

committing
developer
A
B
A
B
B
B
A
A
B
B

RQ1 is a basic question in regard to the meaningfulness of
our two metrics. If there is no difference in the metric values
among projects, the proposed metrics are not sensible and are
not beneficial to our discussion of OSS project’s health. RQ2
is a key question concerning the contribution of this paper. We
will try to find a feature of the distribution of metric values
which would be useful in assessing the project’s health.
In this empirical analysis, we use the following bug fix rate
(BFR) as a criterion for evaluating a project:number of bug fixing commits.
number of all commits
Any OSS project may have not only bug fixing commits but
also other commits for enhancements. The balance between
the bug fix and the enhancement is a useful measure to assess
the project’s evolution. It is generally undesirable to have many
bug fixing commits during the development. A low value of
BFR would signify a successful evolution of the software
product, and it means the project has been running healthily.
While a project with an extremely low BFR value may also
be unhealthy, we assume that projects of interest have many
commits for both bug fixes and enhancements, and we mainly
focus on their balance through BFR in this paper.
BFR =B. Empirical Objects and Data Collection
We collected 60 popular OSS projects shown in Table II.
The key reasons why we used them are 1) they are popular
projects, and 2) their products have been managed with Git.
Reason 1) is for increasing the value and the generalityof our results. Since any finding derived from minor projects
must be worthless, we focused only on popular OSS projects.
They are ranked in top 100 of popularity at SourceForge.net1
or widely known projects in the world. Moreover, we will
excluded the projects which have been driven by a few
developers, from our data set because they may have projectspecific trends, i.e., a lack of generality in their data. As wewill describe later, 28 out of 60 projects end up being excluded
since the number of unique developers is less than 10. Reason
2) is for the ease of data access: we can easily make a cloneof repository, so our data collection is quickly performed.
1 https://sourceforge.net/

TABLE IIOSS PROJECTS ANALYZED IN THIS PAPER .
no.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

project
android-x86
Angry IP Scanner
AutoClicker
Catacombae
Cool Reader
Dev-C++
Dia Diagram Editor
Ditto
Eclipse Checkstyle Plug-in
Eclipse Tomcat Plugin
eXo Platform
Expat XML Parser
Ext2 Filesystems
Utilities (Ext2 Util)
Firebird
FlightGear
FreeMindGit
GNU ARM Eclipse
GPartedHibernate ORM
HPCC Platform
KDiff3
libjpeg-turbo
Linux
LinuxonAndroid
MediaPortal
MediathekView
milter manager
Money Manager Ex
MoodleNagios Core
NAPS2

no.
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

project
net-snmp
Notepad++ Compare plugin
Notepad++ Plugin Manager
Notepad++ Python Script
Pandora FMS
Password SafePMD
PNP4Nagios
ProjectLibre
PSeIntrEFIndRuby
Ruby on Rails
Scribus
Smoothwall
SQuirreL SQL Client(SQuirreL)
SystemRescueCd
TEncoder Video Converter
The OpenGL Extension
Wrangler Library(OpenGL Lib)
tuxboot
Ubuntuzilla: Mozilla
Software Installer
UNetbootin
Universal Media Server(Universal Media)
Win32 Disk Imager
Wine
Wineskin
WordPress
xCAT

For each project, we collected data in the following scheme.
1) Get commit data of each developer.
Collect the commit logs, and classify them according to
the developer. If there are two or more developers who
seem to be the same one—they have the same name ore-mail address—, merge their data into a single set.
2) Collect metric values of NoC and RoF.
For each developer, calculate the NoC value and the RoF
value.
3) Calculate the BFR value.
For each project, count all commits and all bug fixing
commits, and calculate the BFR value as the ratio of thelatter to the former. These numbers can be obtained bycounting the unique hash codes which appear in all of thecommit logs and in the bug fix commit logs, respectively.
The decision as to whether a commit was a bug fix or not
is made by checking the appearance of a bug fix-related
keyword in the commit message, according to generally
accepted practice in the MSR community, proposed byŚliwerski et al. [7].
C. Results and Discussions
Table III shows the total numbers of files, developers andcommits in each project, except for the ones whose number ofdevelopers is less than 10; 28 out of 60 projects are excluded
from our data set due to the small number of developers, so
32 projects are shown in Table III.

TABLE IIIN UMBERS OF FILES , DEVELOPERS AND COMMITS IN EACH PROJECT.
no.
1
2
5
11
12
13
15
17
20
21
23
24
26
29
30
31
33
37
38
39
40
44
45
47
48
51
54
55
56
57
59
60

project
android-x86
Angry IP Scanner
Cool Reader
eXo Platform
Expat XML Parser
Ext2 Util
FlightGear
Git
Hibernate ORM
HPCC Platform
libjpeg-turbo
LinuxMediaPortal
Money Manager Ex
MoodleNagios Core
net-snmp
Pandora FMS
Password SafePMD
PNP4Nagios
Ruby
Ruby on Rails
Smoothwall
SQuirreL
OpenGL Lib
UNetbootin
Universal Media
Win32 Disk Imager
Wine
WordPress
xCAT#files49, 177769
3, 3781, 983
194
1, 918913
2, 941
10, 360
9, 406433
54, 376
6, 918640
14, 968866
2, 1984, 7052, 621
2, 123701
4, 3643, 1491, 506
7, 444270
270
1, 29747
141, 484
1, 5962, 150#developers
12, 121
11
21
179
20
151
57
1, 421267
52
15
13, 888
42
16
597
19
61
33
56
99
29
97
3, 551
14
31
29
10
41
10
3, 172
71
79#commits
594, 738
1, 0344, 077
11, 474
1, 394
5, 4944, 65055, 4317, 14916, 505
1, 629700, 908
7, 8393, 65281, 772
2, 262
26, 14412, 328
10, 2318, 6981, 14343, 977
60, 180
1, 5487, 956
1, 118628
6, 753
159
22, 707, 259
34, 263
15, 760

We first checked the correlations among three numbers:
we calculated Spearman’s rank correlation coefficient ρ [8]rather than Pearson’s coefficient because of their skewed distributions. They were ρ(#files, #developers)= 0.718, ρ(#files,#commits)= 0.803 and ρ(#developers, #commits)= 0.872.
These three numbers seem to be strongly related each other,i.e., we can simply say that the more files, the more developers
and the more commits. It is a natural trend, so our data set
may provide a general data of OSS projects.
1) Can the distribution of metric values show a difference
among OSS projects? (for RQ1): Tables IV and V show the
distributions of NoC values and of RoF values, respectively—
the quartiles in the set of developers. The minimum values of
RoF result in 0% because some files were modified in the past
but are not included in the latest version of the product.
In a majority of projects, the maximum values of NoC areover 1, 000, i.e., there are some highly-contributing developers
in those projects. However, such developers are a minority inthe projects other than WordPress since these medians are lessthan 100. On the other hand, most of WordPress’s developers
have committed a few dozen times or more, and the projectshows a different distribution than the others.
While there are variations of RoF values among projects,
the 75 percentiles of RoF values are less than 10% except
for 5 projects (Expat XML Parser, Money Manager Ex,
Smoothwall, Win32 Disk Imager and WordPress). That is tosay, in many projects, there are a few developers who added or
modified files extensively in the directory trees. For example,

0

2000

4000

6000

8000

10000

12000

75%
0
8.8
0.3
1.3
12.9
0.4
4.7
0.2
0.1
2.8
5.3
0
0.7
30.9
0.2
2.9
1.4
6.8
0.6
0.1
1
3.8
0.1
14.1
0.5
4.4
9.1
0.5
28.7
0
15.4
3.4

max
16.9
71.5
98
43.6
60.3
69.3
40.5
50.4
85
70.1
99.3
15.3
71.1
92.3
34.6
72.9
96
38.5
93.3
95.5
99.1
60.9
25.8
48.9
67.9
89.6
100
81.9
70.2
40.8
52.1
28.2

0.8
0.6

0.15

0.4

0.10

RoF

0.2

0.05

0.0

Cumulative Percentage

1.0

0.20

50%
0
0.5
0.1
0.1
1
0.1
0.5
0.1
0
0.5
0.9
0
0.1
8.1
0
0.4
0.1
1.7
0
0
0.4
1.2
0.1
2.7
0
0.7
0.4
0.2
12.8
0
4.8
0.9

0.00

0.8
0.6
0.4

0

0.0

0.2

5000

Cumulative Percentage

15000
10000

NoC

25%
0
0.1
0
0
0.5
0.1
0.1
0
0
0
0.2
0
0
0.6
0
0.2
0
0.4
0
0
0.1
0.3
0
0.4
0
0.4
0.4
0.1
5.3
0
1.1
0.3

1.0

20000

min
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

14000

0

2000

4000

6000

8000

10000

Developers

Developers(a) NoC(b) RoF

12000

14000

5

10

15

20

25

30

1.0
0.8
0.6
0.4
0.2
0.0

Cumulative Percentage

1.0
0.8
0.6

RoF
0.4
0.2

0.4
0.2
0.0
0

0.0

0.8
0.6

400
300

NoC

200

Cumulative Percentage

500

1.0

600

Fig. 1. Pareto charts of NoC values and RoF values in Linux.

100

0

5

10

15

20

Developers

Developers(a) NoC(b) RoF

25

30

20

30

40

50

60

70

1.0
0.8
0.6

0.4

0.4
0.2
0.0

RoF

0.3
0.2
0.1
0.0

Cumulative Percentage

0.5

1.0
0.8
0.6
0.4
10

Cumulative Percentage

0.2
0

0.6

Fig. 2. Pareto charts of NoC values and RoF values in OpenGL Lib.

0.0

the maximum RoF value in Linux is 15.3%; Since Linux is
an operating system and provides a wide range of functions,
it is harder to comprehensively contribute the development of
all files than the other products. Notice that we discuss thecontribution “as the author” in this paper, so we do not denythat there are “reviewers” checking a wide range of the files
comprehensively. As is the case in NoC, WordPress projectshows a different distribution of RoF values from the others.
To quantitatively compare these distributions, we introduce
the notion of the Pareto chart [9]. Figures 1, 2 and 3 show
the Pareto charts of NoC values and RoF values in Linux,
OpenGL Lib and WordPress, respectively; we show only threeprojects’ Pareto charts for lack of space. The vertical axes
signify the degree of the developer’s contribution (the left handside) and the cumulative percentage (the right hand side), andthe horizontal axis corresponds to developers in decreasingorder of contribution.
Since a minority of developers look making most of thetotal contributions, the Pareto principle seems to roughly holdfor most projects. The Pareto principle says that 80% of thetotal contributions are made by 20% of the developers where
the percentages may vary, and the Pareto chart has a “longtail” according to the remaining 80% of the developers. We
checked the rate of the dominant developers who made 80%of the total contributions, and calculated the relative length of
the tail (see Table VI): WordPress required more developers
to make 80% of the total contributions than the other projects.
That is to say, WordPress had a shorter tail than the others.

project
android-x86
Angry IP Scanner
Cool Reader
eXo Platform
Expat XML Parser
Ext2 Util
FlightGear
Git
Hibernate ORM
HPCC Platform
libjpeg-turbo
LinuxMediaPortal
Money Manager Ex
MoodleNagios Core
net-snmp
Pandora FMS
Password SafePMD
PNP4Nagios
Ruby
Ruby on Rails
Smoothwall
SQuirreL
OpenGL Lib
UNetbootin
Universal Media
Win32 Disk Imager
Wine
WordPress
xCAT

0

max
18, 199466
3, 8591, 242549
3, 923969
16, 671
2, 357
7, 6581, 55220, 876
3, 834
1, 315
7, 948974
7, 8182, 6515, 3343, 352922
13, 090
4, 423534
3, 553425
606
4, 35576108, 116
7, 1532, 754

8000

75%
11
77
7
76.5
28.5
5
100
7
5
157
9
12
29
241
37
10.5
23
406
33.5
27
8
294
3
176
24
6
2
20
25
34
432.5
141

6000

50%
2
3
2
18
2
2
4
2
1
45
2
3
8
51
5
5
2
79
6
3
3
83
1
46.5
4
2
1.5
5
5.5
4
156
25

4000

25%
1
1
1
3
1
1
2
1
1
1.5
2
1
2
6
1
1.5
1
34
1.5
1
1
21
1
6
1
1
1
1
2
1
31.5
11.5

NoC

min
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1

2000

project
android-x86
Angry IP Scanner
Cool Reader
eXo Platform
Expat XML Parser
Ext2 Util
FlightGear
Git
Hibernate ORM
HPCC Platform
libjpeg-turbo
LinuxMediaPortal
Money Manager Ex
MoodleNagios Core
net-snmp
Pandora FMS
Password SafePMD
PNP4Nagios
Ruby
Ruby on Rails
Smoothwall
SQuirreL
OpenGL Lib
UNetbootin
Universal Media
Win32 Disk Imager
Wine
WordPress
xCAT

TABLE VD ISTRIBUTIONS OF DEVELOPERS ’ CONTRIBUTIONS : RO F (%).

0

TABLE IV
D ISTRIBUTIONS OF DEVELOPERS ’ CONTRIBUTIONS : N O C.

0

10

20

30

40

50

Developers

Developers(a) NoC(b) RoF

60

70

Fig. 3. Pareto charts of NoC values and RoF values in WordPress.

project
net-snmp
Pandora FMS
Password SafePMD
PNP4Nagios
Ruby
Ruby on Rails
Smoothwall
SQuirreL
OpenGL Lib
UNetbootin
Universal Media
Win32 Disk Imager
Wine
WordPress
xCAT

BFR
0.192
0.379
0.064
0.021
0.144
0.238
0.136
0.181
0.052
0.132
0.234
0.157
0.289
0.009
0.536
0.271

clearly see the projects with longer tails in RoF values tend
to have lower BFR values. Since the RoF of a developer isthe ratio of files which the developer has been involved in, ahealthier project may be a project such that various functions
have been developed by various developers cooperatively.
While many OSS projects have been successfully run thanks
to talented developers who have made a lot of contributions,
the workload balance must be a significant factor to assess
the project’s health. Since it is not sure whether a highlycontributing developer will continuously provide their help
to the project in the future, a high workload on a certain
developer can be a threat to the project’s health. Focusing
on the structure of developers in the OSS project would be
useful in understanding and evaluating the project.
The answer to RQ2 results in “Yes”: the relative length of
the tail in the RoF Pareto chart has a negative correlation withthe project’s BFR, so it can be one of useful health indexes.
D. Threats to Validity

0.5

Since our BFR data is based on a simple keyword matching,it might overlook some bug fix commits. However, such amethod has been widely used in the MSR community and hasa certain level of validity, therefore the data collection scheme
would not be a serious threat to validity of our empirical study.
Although we used only 32 projects, they are popular ones
from different domains and have been developed by 10 ormore developers. Moreover, their programming languages are
not the same. Thus, while our data selection was not a random
sampling, it would not become a significant threat to validity.

0.4r(length of tail in RoF, BFR)= −0.455
where t = −2.7975, df = 30 and p = 0.0089.
While the length of tail in NoC values seems to have nocorrelation with BFR values, the length of tail in RoF values
would have a negative correlations with BFR values. That is
to say, the longer tail in RoF, the lower BFR.
According to the notion of the Pareto principle, we consider
80% to be a baseline of the length of tail, then we compare
BFR values between two groups of projects—the projects
whose relative length of tail in RoF is shorter than 80% versus
the ones whose relative length of tail in RoF is longer than orequal to 80%. Figure 4 shows the results. From the figure, we
•

BFR
0.300
0.082
0.391
0.025
0.220
0.413
0.208
0.190
0.082
0.229
0.175
0.299
0.111
0.206
0.164
0.409

0.3

The answer to RQ1 results in “Yes”: while the distributions
seems to roughly obey the Pareto principle, there are differences in the relative length of tail among projects. The lengthof tail in the NoC value ranges from 75.4% to 98.7%, and thatin the RoF value ranges from 63.6% to 98%, respectively.
2) Can the distribution be noteworthy for making a usefulhealth index of OSS project? (for RQ2): Table VII shows BFRvalues in each project. WordPress project had the maximum
BFR, therefore we can expect that there is a correlation between the length of tail and the BFR. We calculated Pearson’s
correlation coefficient for each pair and did a t-test whether
the correlation is statistically significant (̸= 0) or not. Theywere
• r(length of tail in NoC, BFR)= −0.244
where t = −1.3766, df = 30 and p = 0.1788;

project
android-x86
Angry IP Scanner
Cool Reader
eXo Platform
Expat XML Parser
Ext2 Util
FlightGear
Git
Hibernate ORM
HPCC Platform
libjpeg-turbo
LinuxMediaPortal
Money Manager Ex
MoodleNagios Core

BFR

RoF
dominant other(tail)
7.8
92.2
8.3
91.7
4.5
95.5
12.2
87.8
19.0
81.0
94.7
5.3
19.0
81.0
12.1
87.9
2.6
97.4
11.3
88.7
12.5
87.5
8.3
91.7
4.7
95.3
29.4
70.6
7.2
92.8
10.0
90.0
9.7
90.3
23.5
76.5
5.3
94.7
2.0
98.0
3.3
96.7
21.4
78.6
11.7
88.3
20.0
80.0
6.3
93.8
13.3
86.7
9.1
90.9
7.1
92.9
36.4
63.6
6.2
93.8
33.3
66.7
26.3
73.8

0.2

NoC
dominant other(tail)
5.6
94.4
9.1
90.9
4.8
95.2
24.6
75.4
10.0
90.0
98.7
1.3
17.5
82.5
2.6
97.4
4.1
95.9
9.6
90.4
6.7
93.3
5.6
94.4
4.8
95.2
18.8
81.3
6.9
93.1
5.3
94.7
8.2
91.8
21.2
78.8
3.6
96.4
6.1
93.9
3.4
96.6
14.4
85.6
2.6
97.4
21.4
78.6
6.5
93.5
6.9
93.1
10.0
90.0
4.9
95.1
20.0
80.0
5.9
94.1
23.9
76.1
17.7
82.3

0.1

project
android-x86
Angry IP Scanner
Cool Reader
eXo Platform
Expat XML Parser
Ext2 Util
FlightGear
Git
Hibernate ORM
HPCC Platform
libjpeg-turbo
LinuxMediaPortal
Money Manager Ex
MoodleNagios Core
net-snmp
Pandora FMS
Password SafePMD
PNP4Nagios
Ruby
Ruby on Rails
Smoothwall
SQuirreL
OpenGL Lib
UNetbootin
Universal Media
Win32 Disk Imager
Wine
WordPress
xCAT

TABLE VII
BFR VALUES OF SURVEYED OSS PROJECTS .

0.0

TABLE VI
P ERCENTAGES OF DOMINANT DEVELOPERS AND THE OTHERS

tail < 80%

tail >= 80%

Fig. 4. Comparison of BFR values: OSS projects whose tail length of RoF
is shorter than 80% vs. longer than or equal to 80%.

We focused only on the number of commits and not on
what kind of changes were made. While impacts of changes
should also be considered, we missed those analyses in thispaper. Moreover, although we checked only commit “authors,”
there are other ways of contributing OSS projects: for example,
posting a bug report to the bug tracking system or to the
support mailing list. Such another kind of contribution would
also be important factor for evaluating the OSS project’s healthlevel. Addressing these issues are our significant future work.
IV. R ELATED W ORK
Raja and Tretter [10] focused on the survivability of OSS
projects, and they proposed “viability index” which is a linear
combination of three sub indexes on 1) vigor, 2) resilienceand 3) organization: 1) the vigor index is a measure of project
growth; 2) the resilience index means an ability to react to thechanges in the project’s environment; 3) the organization index
is a measure of the diversity of interactions among project
members. While the viability index can produce remarkableevaluations of OSS projects, it requires to find appropriate
coefficients to combine the above three factors in their model.
Although our approach is simpler and less informative than
Raja and Tretter model, our one does not need any parametertuning but it captures yet another feature of projects, so that
it would be a useful supplement in evaluating OSS projects.
Yamashita et al. [11] investigated many OSS projects fromthe perspective of the Pareto principle, i.e., whether the coredevelopers’s work dominates the project or not. Their work
is an important replication study of the prior work [12],[13], [14], [15], [16], [17], by conducting a large-scale datacollection from the GitHub. They showed that a majority of
projects are not compliant with the Pareto principle in termsof the core developers’s contribution to projects. That is tosay, there seems to be a wide variance in the proportion ofcore developers to non-core ones among projects. While theirresults look difference from our results, their work included
the projects driven by a few developers as well. In our analysis,
we excluded such projects and then focused on the length oftail in the Pareto chart. If we include the projects driven bysmall number of developers (≤ 10), we cannot say that mostprojects seem to obey the Pareto principle. Notice that our
main focus is to capture a feature of OSS project by using thelength of tail in the Pareto chart. Moreover, we analyzed therelationships with the bug fix rates in OSS projects as well.
V. C ONCLUSION AND F UTURE W ORK
In this paper, we focused on developers’ contributions to
OSS projects, and quantified them with using two metrics,
the number of commits (NoC) and the ratio of files which thedeveloper has added or modified (RoF). Then, we examined 32
popular OSS projects which have been driven by 10 or moredevelopers, and showed the distribution of the developer’s
contribution roughly obeyed the Pareto principle. That is tosay, the developers could be categorized into two types: thedominant developers (making 80% of the total contributions)and the others. The latter developers form a long tail in the

Pareto chart, and the length of the tail in the chart showed aninteresting trend: the longer tail in the Pareto chart of RoF,
the lower rate of bug fixes in the project. Thus, it would be auseful index to assess the project’s health.
To examine the usefulness of proposed index in predicting
project’s healthiness, we have to perform further investigations
with other OSS projects which are not only well maintained
but also no longer maintained in the future: we plan to collecta lot of projects from GitHub and analyze them. Moreover, afurther analysis focusing on not only the commits but also thepull requests is also our important future work.
ACKNOWLEDGMENT
The authors would like to thank the anonymous reviewers
for their helpful comments on the first version of this paper.
This work was supported by JSPS KAKENHI #16K00099.
R EFERENCES[1] Black Duck Software, “The ninth annual future of open source survey| black duck,” https://www.blackducksoftware.jp/future-of-open-source,
Apr. 2015.
[2] D. Posnett, R. D’Souza, P. Devanbu, and V. Filkov, “Dual ecologicalmeasures of focus in software development,” in Proc. Int’l Conf. Softw.
Eng., May 2013, pp. 452–461.
[3] A. E. Hassan, “The road ahead for mining software repositories,” inProc. Frontiers of Softw. Maintenance, 2008, pp. 48–57.
[4] A. E. Hassan and R. C. Holt, “The top ten list: Dynamic fault prediction,”in Proc. 21st IEEE Int’l Conf. Softw. Maintenance, 2005, pp. 263–272.
[5] Y. Kamei, S. Matsumoto, A. Monden, K. ichi Matsumoto, B. Adams, andA. E. Hassan, “Revisiting common bug prediction findings using effortaware models,” in Proc. 2010 IEEE Int’l Conf. Softw. Maintenance,
2010, pp. 1–10.
[6] F. Rahman, D. Posnett, A. Hindle, E. Barr, and P. Devanbu, “Bugcachefor inspections : Hit or miss?” in Proc. 19th ACM SIGSOFT Symp. &13th European Conf. Foundations of Softw. Eng., 2011, pp. 322–331.
[7] J. Śliwerski, T. Zimmermann, and A. Zeller, “When do changes induce
fixes?” in Proc. Int’l Workshop on Mining Softw. Repositories, May
2005, pp. 1–5.
[8] S. Siegel, Nonparametric statistics for the behavioral sciences.
McGraw-Hill, 1956.
[9] J. M. Juran and F. M. Gryna, Eds., Juran’s Quality Control Handbook.
New York: McGraw-Hill, 1988.
[10] U. Raja and M. J. Tretter, “Defining and evaluating a measure of open
source project survivability,” IEEE Trans. Softw. Eng., vol. 38, no. 1,
pp. 163–174, Jan 2012.
[11] K. Yamashita, S. McIntosh, Y. Kamei, A. E. Hassan, and N. Ubayashi,“Revisiting the applicability of the pareto principle to core development
teams in open source software projects,” in Proc. 14th Int’l Workshop
on the Principles of Softw. Evolution, Aug. 2015, pp. 46–55.
[12] A. Mockus, R. T. Fielding, and J. D. Herbsleb, “Two case studies ofopen source software development: Apache and mozilla,” ACM Trans.
Softw. Eng. Methodol., vol. 11, no. 3, pp. 309–346, July 2002.
[13] T. T. Dinh-Trong and J. M. Bieman, “The FreeBSD project: a replicationcase study of open source development,” IEEE Trans. Softw. Eng.,
vol. 31, no. 6, pp. 481–494, June 2005.
[14] S. Koch and G. Schneider, “Effort, cooperation and coordination in an
open source software project: GNOME,” Inf. Syst. J., vol. 12, no. 1, pp.
27–42, Jan. 2002.
[15] M. Goeminne and T. Mens, “Evidence for the pareto principle inopen source software activity,” in Joint Proc. 1st Int’l Workshop Model
Driven Softw. Maintenance & 5th Int’l Workshop Soft. Quality &Maintainability, Mar. 2011, pp. 74–82.
[16] G. Robles, S. Koch, J. M. González-Barahona, and J. Carlos, “Remoteanalysis and measurement of libre software systems by means of the
CVSAnalY tool,” in Proc. 2nd ICSE Workshop Remote Analysis &Measurement of Softw. Syst., May 2004, pp. 51–55.
[17] J. Geldenhuys, “Finding the core developers,” in Proc. 36th EuromicroConf. Softw. Eng. & Advanced Applications, Sept. 2010, pp. 447–450.