KnowledgeBasedSoftwareEngineering
A.aplinskas,H.Praneviius,T.Nakatani(Eds.)
Technologija,2010©2010H.Aman, T.Ohkochi.Allrightsreserved

46

ANAPPLICATIONOFGROWTHCURVEMODELFOR
PREDICTINGCODECHURNINOPENSOURCE
DEVELOPMENT
HirohisaAMAN1andTakahiroOHKOCHI
GraduateSchoolofScienceandEngineering,EhimeUniversity,Japan

Abstract.Quantitativequalitymanagementandpredictionaresignificantchallengesin
opensourcesoftwaredevelopment.Whileanyonecouldaccesstosourcecodethroughthecoderepository,itisnoteasytoperformaquantitativeassessmentand/orprediction
ofthesoftwarequality.Toprovideameanofquantitativemanagementutilizingacoderepository,thispaperproposesamathematicalmodelingofsourcecodechurn—agrowthcurvemodelbasedonanonhomogeneousPoissonprocess.Anempiricalworkusing12
packagesinEclipseisperformed,andtheresultsshow:(1)aninflectionSshapedcurve
modelcouldbewellfittedtoactualdata,and(2)theproposedmodelcouldpredictthecodechurninthenext5monthswithlessthan10%error.
Keywords. Open source development, quantitative management, source code churn,
prediction,growthcurvemodel.

1. Introduction
Inrecentyears,opensourcesoftware(OSS)hasbeenactiveinvariousfieldssuchasoperatingsystem, Web server, embedded system, and so on. Some vendors use OSS in their products
and/or services as key components[7]. However there is a difficulty in a quality assurance of
OSS;AlthoughanyonecouldaccesstoasourcecodeofOSSthroughthepubliccoderepository,
itisnoteasytoperformaquantitativeassessmentand/orpredictionoftheOSSquality.Whileasourcecodeisadetailedrawdataofthedevelopmentactivities,theevolutionarychangeofthewhole software could not be understood by simply observing each code. Such difficulty may
leadtoabarriertoindustrialpromotionofOSSutilization.
ForanOSSproject,asourcecodeisamajorproductreflectingthedevelopmentactivities.To
perform a quantitative management of OSS development, it is important to observe not onlythe source code but also its changehistory showing the evolutionary process. Especially on
change history, many repository mining techniques have been studied in recent days[8]. Arepository mining is a powerful method to find some knowledge from a code repository.
However it is a method for getting information through statistical analysis and/or machinelearning,notformodelingtheobservedphenomenamathematically.
Asanotherrelatedwork,softwareevolutioninopensourcedevelopmenthasbeenstudiedas
well.GodfreyandTu[5]investigatedagrowthpatternofsourcecodeinLinux,andXieetal.[15]
1

Corresponding Author: Hirohisa AMAN, 3, Bunkyocho, Matsuyamashi, Ehime, 7908577 Japan,
aman@cs.ehimeu.ac.jp.H.Amanetal./AnApplicationofGrowthCurveModelforPredictingCodeChurn…

47

also discussed the evolution patterns through their empirical study using many open source
releases. Paulson et al.[11] investigated some growths in some major open source software
includingLinux,ApacheandGcc.Whilesomesoftwareevolutionpatternshavebeendiscussed,
there are few studies focusing on a mathematical model of tendency in software evolution
(codechange).
Towardamathematicalmodelingofthechangehistory,weintroduceanotherperspectiveontheanalysisofcoderepositoryinthispaper—astochasticmodeling.Weconsideracodechange
tobeastochasticeventbecausemanyprogrammerswouldcontinueconcurrentlycommittingtotherepository;inotherwords,itwouldbeuncertaintywhenthenextcodechangewouldbe
done.
Inthispaper,weproposeastochasticmodelofcodechangeevents(codechurn)inorderto
formulate a novel method for explaining code churn mathematically. The contribution of thispaperistopresentsomeexperimentaltrialsshowingthatourmathematicalmodelhasahighlevelofabilitytopredictcodechurninalargescaleOSS.
The rest of the paper is organized as follows: Section 2 describes the code churn in open
sourcedevelopment,andintroducesbasicgrowthcurvemodels.Thenastochasticmodelingofcodechurnrelatedtothegrowthcurvemodelisproposed.Section3performssomeempirical
work using Eclipse. In the empirical work, some packages of source code are collected from
Eclipserepository,andsometrialsareperformedtofitgrowthcurvestotheactualcodechurn.
Then the model’s ability to predict code churn is evaluated. Finally, Section 4 presents ourconclusionandfuturework.

2. CodeChurnandGrowthCurveModel
2.1. CodeChurninOpenSourceDevelopment
In an open source development, source code changes are basic events reflecting thedevelopment activities. It is important to observe the code churn for a quantitative
management of the open source development; term “code churn” includes code addition,
deletionandmodification.WemeasuretheamountofcodechurnwiththetotalchangedLOC
thatisthetotalofaddedLOC(Ga),deletedLOC(Gd)andmodifiedLOC(Gm),exceptingblanklinesandcomments:
codechurn=Ga+Gd+Gm;
where Gm is the greater LOC in corresponding fragments modified through versionupgrade.
Generallyacodemodificationcorrespondstotwodifferentfragments—theolderxlinesweremodifiedtothenewerylines.WedefinetheamountofmodifiedLOCasmax(x;y).Forexample,
Fig.1showsacodechurninJavacodefragments.Thedifferencesbetweenthefragmentscould
becapturedbyusingGNUdiff[4]2,andFig.2showstheexecutionresult.Inthiscase,
x Line1wasdeleted:Gd =1;
x Line3wasmodifiedtonewlines3and4:Gm =max(1; 2)=2;
x Newline5wasadded:Ga =1.
Thuswegetcodechurn=1+1+2=4:
MostOSSprojectshavepubliccoderepositoriestoautomaticallymanagetheirsourcecodeandcodechurn.Whileanyonecouldaccesstosourcecodeandcodechurnthroughthepublic
repository,it is noteasyto understand the developmentalstatus.Forinstance,a major open
sourcedevelopmentsite,SourceForge.net[12],givessomestatisticalinformationincludingthe
downloadhistory,trackertrafficstatistics,andsoon.Howeverlittlestatisticsaboutcodechurn2diff2.8.1;theexecutionoptionwas“Bw.”

48

H.Amanetal./AnApplicationofGrowthCurveModelforPredictingCodeChurn…

couldbeeasily obtainedatsuch developmentsite.Whilethenumberofversionupgradeand
some simple diff results such as “Changes since 1.3: +8 -6 lines” could be
obtained,itisnoteasytoorganizeallofsuchdataaboutcodechurnscatteredintherepository,
anditisnoteasytogetusefulstatisticsforastatisticalanalysisofcodechurntendency.Figure1.Exampleofcodechurn.


Figure2.ExecutionresultofdiffforfragmentsshowninFig.1.
Togetstatisticsaboutcodechurn,wecollectmonthlysnapshots(sourcecode)fromthecoderepository,andexaminemonthlydifferencesinthesourcecode;wehavedevelopedatoolfor
collecting changed LOC, DiffLineCounter.jar[1]. Then we propose a mathematical
model for explaining and predicting code churn from a stochastic viewpoint. We consider acodechurntobeastochasticeventsimilartoa“customerarrival”inthequeueingtheory[10],
andtrytomodelcodechurnwithaPoissonprocessmodel,especiallyagrowthcurvemodel.

2.2. GrowthCurveModel
There has been considerable research on the software reliability growth model[3, 14] in
which the cumulative numbers of faults or failures are explained with some mathematical
models. Most of major models are based on the nonhomogeneous Poisson process (NHPP)
considering a fault (or failure) detection to be a stochastic event: Let N(t) be the random
variablerepresentingthecumulativenumberoffaults(orfailures)detectedbytimet,andthe
probabilityofgettingN(t)=nisgivenasthePoissondistributionfunction,
where

H.Amanetal./AnApplicationofGrowthCurveModelforPredictingCodeChurn…

49

Table1.BasicNHPPmodels.

H(t) is the mean value function of N(t), i.e., the expected number of faults (or failures)
detectedbytimet,andh(t)isthefault(failure)detectionrateatt.NHPPisaPoissonprocess
with a time dependent fault (failure) detection rate, i.e., h(t) is variable over time; if h(t) is aconstant, O, then we get H(t) = Ot, and the stochastic process is homogeneous Poisson
process(HPP), not NHPP. Table 1 shows the mean value functions of basic NHPP models. In
Table1,aistheexpectednumberofallfaults(failures),bistheparameterrelatedtothefault
3(failure) detection rate, and c is the odds ratio of nondetectable faults to detectable faults .
Parameters a, b and c are estimated from actual data by using statistical method such as
maximumlikelihoodmethodorleastsquaremethod.Wenowproposemodelingcodechurnin
opensourcedevelopment,justastheabovefaultdetection:achangeofcodeisconsideredasa
stochasticevent,andtheamountofcodechurnisexplainedbyagrowthcurvemodelbasedon
NHPP.Althoughastrictlymodelingshouldtakeconsiderationsforthetypeandamountofeachcodechurn,firstwetrytoexploretheavailabilityofabovemacromodels. 

3. EmpiricalWorkInthissection,weexplorethetrendofcodechurnfromapracticalopensourcedevelopment
project,Eclipse[2].ThenwetrytofittheaboveNHPPmodelstotheactualdata,anddiscussthe
abilitytopredictthecodechurn.

3.1. ExperimentalObjectsOur experimental objects are 12 packages included in Eclipse, shown in Table 2, that have
beenactivelydevelopedwithmorecodechurn.Sinceapackageisasetofsourcefilesthatare
related to each other structurally and/or semantically, we considered a package to be an
appropriate unit of analysis for a code churn. We collected monthly code churn in those
packages until Oct.1st, 2009. Although there are several releases of Eclipse in our datacollection period, we consider the crossrelease issue would be of little concern, since all thetargetpackagesprovidecorefunctionalitiesofEclipse,andsuchcorefunctionalitieswouldbe
continuouslymaintainedthroughthereleases.

3.2. PreliminaryWork:FittingTo find an appropriate mathematical model of code churn, we perform preliminary
experimentsforfittingtheabovethreeNHPPmodelstopracticaldataofEclipse.Figure3showsaninstanceofinflectionSshapedcurvefittedtoactualdatainorg.eclipse.core.runtime,whosemeanvaluefunctionisestimatedas

3This is from the notion that some faults would be nondetectable in early testing stage since they
were hidden by another faults. Such nondetectable faults would become detectable after the otherdetectablefaultswerefixed.

50

H.Amanetal./AnApplicationofGrowthCurveModelforPredictingCodeChurn…


Table2.Packagesusedintheempiricalstudy.
No.
1
2
3
4
5
6
7
8
9
10
11
12

packageorg.eclipse.core.internal.content
org.eclipse.core.internal.jobs
org.eclipse.core.internal.localstoreorg.eclipse.core.internal.plugins
org.eclipse.core.internal.preferences
org.eclipse.core.internal.registryorg.eclipse.core.internal.runtime
org.eclipse.core.resources
org.eclipse.core.runtime
org.eclipse.debug.core.modelorg.eclipse.jdt.core.dom
org.eclipse.jdt.internal.compiler.parser

Figure3.FittingofaninflectionSshapedcurvetoactualdatain“org.eclipse.core.runtime.”
In Fig.3, the actual data is the monthly cumulative code churn (changed LOC) during thedevelopment, and the fitted curve is obtained through the leastsquare method. R[13] could
automatically calculate the curve parameters so that the square error with actual data is
minimal. In this example (Fig.3), while the early stage of development is roughly traced, thelaterstageiswellfittedandaconvergenceofthecodechurnmaybemodeledsuccessfully.For
all models shown in Table 1 and all packages shown in Table 2, we perform to estimate thecurveparametersusingR.SeeTable4inAppendixAforthelistofestimatedparameters.
Table 3 shows the mean magnitude of relative error (MMRE) as the results of fitting threemodels showninTable1;thelessMMREvalue indicatesthe betterfitting. Inour preliminaryH.Amanetal./AnApplicationofGrowthCurveModelforPredictingCodeChurn…

51

work,theinflectionSshapedmodelcouldshowbetterfittingstotherealcodechurninEclipse
than the other models. We now consider the inflection Sshaped model to be a preferable
modelforexplainingcodechurn,andwenexttrytousethemodelforpredictingcodechurnin
thefollowingempiricalwork.

3.3. ProcedureandResults:PredictingWeperformedexperimentsforpredictingcodechurnwiththeinflectionSshapedmodel.In
our experiments, we predicted code churn in the next n month(s), for n = 1, 2, …, 24. Noticethat data before the inflection point of Sshaped curve could not build any stable prediction
model,soweperformedtheexperimentsusingdatainlaterstageofthepackagedevelopment.
Table3.MMREinfittings.
No.
1
2
3
4
5
6
7
8
9
10
11
12
Total

exponential
0.235
0.04870.149
0.04120.04575.44
0.324
0.348
0.191
0.272
0.981
0.189
0.689

delayedSshaped
0.117
0.02940.182
0.0381
0.0633
2.12
0.146
0.12
0.163
0.11
0.435
0.17
0.308

inflectionSshaped
0.164
0.02840.16
0.03180.04260.615
0.0962
0.22
0.142
0.176
0.225
0.135
0.17



Experimentalprocedureforpredictingcodechurninthenextnmonths:

Giventhecumulativecodechurnc(t)inthefirsttmonths(t=1,…,N;Nisthefinalmonthinthedatacollection.Nvariesfrompackagetopackagesinceeachpackagehasadifferenttimeofstartingthedevelopment.).
(1) Iteratethefollowing(2)and(3)forx=T0;T0+1,…,N–x;T0isthebeginningmonthofthelaterdevelopmentalstage.WeputT0=34fromtheactualdatadistribution.
(2) BuildtheinflectionSshapedmodelHx(t)usingc(t)(t=1,…,x),wheret=xisregardedas
thecurrentmonthonthepackagedevelopment.
(3) Predictcodechurninthenextnmonths,c(t=x+n),andcalculatethemagnitudeofrelative
error(MRE):MRE=|c(x+n)–Hx(x+n)|/c(x+n).
(4) CalculateMMRE,themeanofMREsobtainedin(3).

WhileMMRE=0isidealforanypredictionmodel,itwouldbenotexpectedintherealworld,
andweshouldintroducearealisticthresholdvalueofMMREtobeacceptableasaprediction
model. Myrtveit et al.[9] have stated that MMRE = 0:25 is an acceptable level of an effort
predictionmodel.ThispaperwillconsiderthatMMRE=0:25isabasiccriterionofaprediction
modelaswell.MoreoverwewillempiricallyregardMMRE< 0:10asahighpracticalutilityofa
predictionmodel.Figure 4 shows the prediction results in the above 12 packages. The Xaxis represents thepredictiontimen asin“thenextn monthsprediction,”andtheYaxisrepresentsMMREofthe
prediction.From Fig.4, we can see that the proposedmodels could predictcode churnin thenext 13 months with MMRE < 0:25 for all packages. Moreover, the proposed models could
predictcodechurninthenext5monthswithMMRE < 0:10, andthe resultsshowapractical
utilityoftheproposedones.

52

H.Amanetal./AnApplicationofGrowthCurveModelforPredictingCodeChurn…


Figure4.MMREofpredictedcodechurninthenextnmonth(s).

4. ConclusionandFutureWork
Thispaperhasconsideredacodechurninopensourcedevelopmenttobeastochasticevent,
andproposedapredictionmodelbasedonthenonhomogeneousPoissonprocess(NHPP).The
empirical work using Eclipse source code has showed that: (1) an inflection Sshaped curve
wouldbebettermodelforfittingandpredictingpracticalcodechurn;(2)theproposedmodel
could predict code churn in the next 5 months with less than 10% error, and in the next 13
monthswithlessthan25%error,repectively.
In the above small trial using Eclipse, the proposed approach have been supported: an
inflectionSshapedcurvemodelcouldpredictcodechurnproperly,andasimilartendencyhas
beenindicatedforallexperimentalpackages.WeconcludethataNHPPbasedmodelmightbe
apromisingmodelforpredictingcodechurninopensourcedevelopment.
Toprovethegeneralityofourproposal,wewillperformfurtherexperimentsformanyother
OSSpackagesas ourfuturework, since theabove empiricalworkwaslimitedinonly 12core
packagesofEclipse.Anotherfutureworkincludesananalysisofcodechurnitself.Intheabove
empiricalwork,weomittedthecontentsandsemanticsofcodechanges.Wewillhavetofocus
oncodechurnindetailwithsomerelatedwork,e.g.[6],formoreaccuratepredictionmodel.

H.Amanetal./AnApplicationofGrowthCurveModelforPredictingCodeChurn…

53

Acknowledgment
Theauthorswouldliketothanktheanonymousreviewersfortheirthoughtfulcommentsand
helpfulsuggestionsthefirstversionofthispaper.Thisresearchwaspartiallysupportedbythe
MinistryofEducation,Science,SportsandCulture,GrantinAidforYoungScientists(B),2010,
22700035.

References[1]

H. Aman. DiffLineCounter, [Online] Available from: http://www.hpc.cs.ehimeu.ac.jp/
˜aman/project/tool/DiffLineCounter.html[Accessed31March2010].
[2] Eclipse.org,[Online]Availablefrom:http://www.eclipse.org/[Accessed31March2010].
[3] A.L.Goel.SoftwareReliabilityModels:Assumptions,Limitations,andApplicability.IEEETrans.Softw.
Eng.1985;SE11(12):1411–1423.
[4] Free Software Foundation, Diffutils – GNU Project, [Online] Available from:
http://www.gnu.org/software/diffutils/diffutils.html[Accessed31March2010].
[5] M.W.GodfreyandQ.Tu.EvolutioninOpenSourceSoftware:ACaseStudy.Proc.InternationalConf.
onSoftwareMaintenance.2000;131.
[6] S. Kim, E.J. Whitehead Jr. and Y. Zhang. Classifying Software Changes: Clean or Buggy?. IEEE Trans.
Softw.Eng.2008;34(2):181–196.
[7] G.Lawton.TheChangingFaceofOpenSource.IEEEComputer2009;42(5):14–17.
[8] MSRHomePage,[Online]Availablefrom:http://www.msrconf.org[Accessed31March2010].
[9] I.Myrtveit,etal.ReliabilityandValidityinComparativeStudies ofSoftwarePredictionModels.IEEE
Trans.Softw.Eng.2005;31(5):380–391.
[10] R. Nelson. Probability, stochastic processes, and queueing theory: the mathematics of com puter
performancemodeling.NewYork:SpringerVerlag;1995.
[11] J.W. Paulson, G. Succi and A. Eberlein, An Empirical Study of OpenSource and Closed SourceSoftwareProducts.IEEETrans.Softw.Eng.2004;30(4):246–256.
[12] SourceForge.net,[Online]Availablefrom:http://sourceforge.net/[Accessed31March2010].
[13] TheRProjectforStatisticalComputing,[Online]Availablefrom:http://www.rproject.org/[Accessed
31March2010].
[14] A.Wood.PredictingSoftwareReliability.IEEEComputer1996;29(11):61–68.
[15] G.Xie, J. Chen andI. Neamtiu.Towards aBetterUnderstandingofSoftware Evolution:An Empirical
StudyonOpenSourceSoftware.Proc.InternationalConf.onSoftwareMaintenance.2009;51–60.


A.Parametersofgrowthcurvesfittedtoactualdata
Table 4 shows the estimated parameters of growth curves fitted to actual data in our
preliminarywork.

54

H.Amanetal./AnApplicationofGrowthCurveModelforPredictingCodeChurn…

Table4.Parametersofgrowthcurvesfittedtoactualdata.
No.
1


2


3


4


5


6


7


8


9


10


11


12






a
b
c
a
b
c
a
b
c
a
b
c
a
b
c
a
b
c
a
b
c
a
b
c
a
b
c
a
b
c
a
b
c
a
b
c

exponential
6037.380.115—
9131.020.122—
19600.46
0.011—
9545.110.098—
6942.220.292—
10645.46
0.068—
23268.11
0.013—14897.73
0.026—
16377.10
0.021—
5009.160.023—
163389.47
0.020—108777.07
0.016—

delayedSshaped
5957.08
0.237—
9015.270.259—
13537.54
0.044—9446.240.204—
6853.450.759—
10330.53
0.152—
17309.13
0.049—
13179.17
0.071—14043.96
0.059—
4357.770.063—
134594.52
0.061—
86055.38
0.053—

inflectionSshaped
5972.240.195
1.92
9051.06
0.190
1.31
12646.03
0.060
7.31
9486.790.142
1.02
7051.960.090
0.81110101.12
0.356
81.2
15770.28
0.091
19.413240.86
0.062
2.4513382.07
0.092
12.7
4389.720.054
2.41
120721.60
0.257
1035.1179979.82
0.087
13.0