Team Activities Measurement Method
for Open Source Software Development
Using the Gini Coefficient

Ayako Masuda
FeliCa Networks, Inc.

Agenda1. Introduction
2. Summary
3. How to Measure the Team Activities in the OSSD
4. Result of the Measurement
5. Discussion
6. Conclusion11-1. Background (1/3)•••••

Open source software (OSS) has made remarkable contributions as social
infrastructure.
The model of the OSS development (OSSD) team is becoming the de facto
standard.
However, there are many areas where OSSD team activities are not clear.
We are conducting empirical research on OSSD team activities.
Our previous research revealed the characteristic of the OSSD team, which
was that the team consists of a small number of enthusiastic contributors(core members) and a large number of small contributors.

Ranking of additional lines in atom.

2

1-1. Background (2/3)•We tried to analyze the test activities of OSSD, but the 48 OSSD teams
analyzed in this study did not confirm the existence of the members
specializing in testing.

•In OSSD, in order to use a lot of other software, it is necessary to test
including dependencies in the development stage.
Moreover, developers fix bugs and respond to new requirements throughfrequent releases.
One of the factors is that products are changing from software products
themselves to services.
In OSSD, the speed of response time supports product quality.

••••

In other words, it is considered that the difference in quality requirements isthe difference in testing activities.

31-1. Background (3/3)••••

Key factors for evaluating GitHub projects are its community structure andtesting activity.
In particular, evidence of testing in a GitHub project implies that thedevelopers have spent considerable time and effort to ensure that theproduct adheres to its intended behavior.
For example, 47.8% of the openlayers project, which is one of our target
project codes, are test codes.
Thus, half of the OSSD team activity is related to testing.

•The OSSD team develops source codes for realizing requirements and,further, test codes for testing the source codes.

•This study measures the team activity in OSSD, and it includes aspects ofsoftware testing activity.

4

1-2. The Purpose of This Study• This study focused on the load distribution of software development
activities and measures the variance of contributors’ activity usingthe Gini coefficient.
•The Gini coefficient and Lorenz curve are common indexes used to analyze thedistribution of household income*.

* Gastwirth, 1972; Nakamura, 2005

5

Agenda1. Introduction
2. Summary
3. How to Measure the Team Activities in the OSSD
4. Result of the Measurement
5. Discussion
6. Conclusion

6

Summary (1/3)•Measurement of load distribution using Gini coefficient– This study focused on the load distribution of software development
activities and measures the variance of contributors' activity using theGini coefficient.
– Effort Person-Months in the months with commit record in themeasurement period was used as the unit of measurement to measureamount of activity, and the Gini coefficient, commonly used in field ofeconomics, was used as an index corresponding to the occupancy rate.
– It was confirmed that this method can be used to calculate the Gini
coefficient of Effort Person-Months in real OSSD projects.

7Summary (2/3)•The influence of activity variance– The influence of activity variance was investigated by confirming
whether the Gini coefficient of Effort Person-Months correlated with thedevelopment period, contributor number, total Effort Person-Months
number, total commits numbered, repository size, star number, fork
number, and issue number.
– However, it was found that there was almost no correlation with items
other than the development period.
– Given that there is almost no impact of large variance in activity amounton commit, which is directly linked to the deliverable, it is assumed that
there is almost no lowering of occupancy rate of personnel due to freeparticipation.

8Summary (3/3)•The characteristics of OSSD from the viewpoint of Gini coefficient– The OSSD projects were classified with the Gini coefficient in order toinvestigate the characteristics of the OSSD project from the Gini
coefficient, and the distribution of each item among the projects werecompared, as well.
– However, OSSD projects with a long development period did not have any
characteristics other than the trend that the Gini coefficient was large.

9Agenda

1. Introduction
2. Summary
3. How to Measure the Team Activities in the OSSD
4. Result of the Measurement
5. Discussion
6. Conclusion

10

Analysis Target
The require conditions of the project :・Continued development for a certain period of time・Participation by a certain number of contributors
・Continuous entry of new contributors

Previous Study: ProjectExtracted

Aug. 11, 2017

50 Projects
UpdatedThis Study

1. Using GitHub Advanced Search function,
extract projects that satisfy the following conditions:・Registered with GitHub in 2012 and continued until 2017
・Repository size is 15 MB or larger
・More than 200 forks
・More than 1,000 stars showing evaluation by OSS users
2. In the project extracted by 1,
both values of the number of contributors and commits
belong to the second quartile or more

Nov. 30, 201848 Projects

In updated process, because there were two projects in which
the repository was moved, 48 OSSD project data were used.
11Analysis Data
Project A
Project A
authormonth
AggregatedProjectB
Extractedaaa
2012-12
Project C: Project

48 PJ’s Commit Records

Effort Person-Months (EM)・Collects commit records for each contributor
for the entire period of the project from Git log
・EM for each contributor was calculated
on a monthly basis
・EM was counted in contributor units

Commitfiles

adddelete

EM

668

866

3465

5519

1

aaa

2013-01

450

720

2370

13197

1

aaa

2013-02

366

683

3128

3569

1:
bbb

2013-11

1

2

38

3

1

bbb

2013-12

1

1

1

1

1

AggregatedProject A
authorStart

commitfiles

adddeleteEMaaa

2012-12702

3112

217356

88372

49

bbb

2013-11

1332

5442

127000

91494

137

ccc

2014-12

5487

20775

573063478714286:
yyy

2015-12

6914

33529

553354444034541

zzz

2017-12

1648

6552

279058134294422

12

Agenda1. Introduction
2. Summary
3. How to Measure the Team Activities in the OSSD
4. Result of the Measurement
5. Discussion
6. Conclusion

13

Selected 3 OSSD Projects Data

Product

Started inopenlayers

meteorllvm

A JavaScript library that
displays dynamic maps
on web pages.

A real-time web
application frameworkthat enables reactive
web programming.

A compiler base that canbe used for any
programming language.

May 2006

May 2006

June 2001

Period (year)

127

17

Contributors (number)

267476919

Repository size (KB)

77,614

77,585

940,703

Forks (number)

1.778

5,028

1,720

Stars (number)

4,453

40,706

3,540

Issues (number)

620

267

0

Effort-Months (EM)

152

144

2,181

0.7217878

0.5706287

0.6091536

Gini Coefficient (value)

14

Selected 3 OSSD Projects Data
openlayers

meteorllvm

Distribution of Effort Person-Months

Distribution of Effort Person-Months

Distribution of Effort Person-Months

Effort Person-Months for each contributor was found to be a power-law distribution

Lorenz curve

Lorenz curve

Lorenz curve

Gini : 0.7217878

Gini : 0.5706287

Gini : 0.6091536

The variance of contributor's activity amount is large.
The Gini coefficient of Effort Person-Months can be calculated in three extracted OSSD projects.15

Result of the Measurement
The Gini Coefficient of Effort Person-Months in All PJs.

Distribution of each item in all PJs

16

Influence of variance of activity amount• To investigate the impact of activity variance, it was found if the Gini
coefficient of Effort Person-Months correlated with the development
period, contributor number, total Effort Person-Months number, totalcommits numbered, repository size, star number, fork number, andissue number.

• According to Table , although the development period had acorrelation, the results showed weak or almost nil correlation withother items.
This implies that the influence of the variance of the activity amount on
the development activity is weak.
17

Classification
by Gini coefficient difference•••

We classified the Gini coefficients and tried to found the characteristics ofthe project on the basis of the differences among the groups.
In the group A with a large Gini coefficient, the development period wasobserved to be long.
However, the variance among the individual OSSD projects was large, andthe characteristics of the project could not be investigated by comparing thegroups.

Group AGroup BGroup C18

Agenda1. Introduction
2. Summary
3. How to Measure the Team Activities in the OSSD
4. Result of the Measurement
5. Discussion
6. Conclusion

19Discussion•The distribution of contributors' activity amount in an OSSD project obeys apower-law distribution with a small number of enthusiastic core members
corresponding to the head and a large number of members engaged in asmall amount of activity, such as beginners corresponding to the tail.

•Given that there is almost no impact of large variance in activity amount oncommits, which is directly linked to the deliverable, it is considered that
there is almost no lowering of occupancy rate of personnel due to freeparticipation.

•OSSD projects are diverse in terms of the target and difficulty level ofdevelopment, project operation, and so on.

•Among various OSSD projects, the Gini coefficient tended to be large forthose with a long development period.

20

Discussion•The Gini coefficient of Effort Person-Months calculated over the entire
measurement period was compared with the Gini coefficient calculated overthe last year in the measurement period.

Comparison of the Gini coefficientfor the whole period and the last year••••

The Gini coefficient for the last year of the measurement period was lower
than the coefficient of the whole period.
This result is because the cumulative number of small activity contributors
who have committed only once is small in relation to the whole period.
This difference in the Gini coefficient indicates the occupancy rate ofcontributors whose period of participation in the OSSD project was short.
The time series analysis of the OSSD project will help clarifying the activityof the contributors.

21

Agenda1. Introduction
2. Summary
3. How to Measure the Team Activities in the OSSD
4. Result of the Measurement
5. Discussion
6. Conclusion

22

Conclusion
Findings•••••

In OSSD, development activities and testing activities are not separated, and testing
activities are conducted in the process of development.
Development activities are promoted mainly by a few core members.
Some projects have been promoted by the same core members, while others havenew core members appearing one after another.
The measurement of load distribution of the activity of OSSD can use Gini coefficientas an index.
Given that there is almost no impact of large variance in activity amount on commit,
which is directly linked to the deliverable, it is assumed that there is almost nolowering of occupancy rate of personnel due to free participation.

Future Task
•Future research should focus on a time series analysis in the OSSD projects andclarify the activity situation of the contributors.

23Thank you for your attention.