A Survival Analysis of Source Files Modified
by New Developers
Hirohisa Aman1 , Sousuke Amasaki2 ,Tomoyuki Yokogawa2, and Minoru Kawahara1
1

2

Ehime University, Matsuyama, Ehime 790–8577, Japan
Okayama Prefectural University, Soja, Okayama 719–1197, Japan

Abstract. This paper proposes an application of the survival analysis
to bug-ﬁx events occurred in source ﬁles. When a source ﬁle is modiﬁed,
it has a risk of creating a bug (fault). In this paper, such a risk is analyzed
from a viewpoint of the survival time—the time that the source ﬁle cansurvive without any bug ﬁx. Through an empirical study with 100 open
source software (OSS) projects, the following ﬁndings are reported: (1)Source ﬁles modiﬁed by new developers have about 26% shorter survival
time than the others. (2) The above tendency may be inverted if the OSS
project has more developers relative to the total number of source ﬁles.
Keywords: open source development, survival analysis, time to bug ﬁx

1

Introduction

Open source software (OSS) products have been more and more popular in the
IT world. Many users and companies have utilized in their social life or business.
As an OSS product has more users or stakeholders, post-release failures occurredin the product have larger impacts [4]. Hence, the quality management of OSS
products has gotten a lot of attention recently. A software product usually evolves
through its functional enhancements and bug (fault) ﬁxes. Although it is idealthat developers never create any bugs, the bug-free evolution would be hard inreality; some code modiﬁcations are also creations of new bugs [6, 8]. Nonetheless,frequent bug ﬁxes are always undesirable in software development.
There have been many studies in regard to bug-ﬁx prediction in the past.
Rahman et al. [11] reported the trend that recently-bug-ﬁxed source ﬁles arelikely to be ﬁxed again. Google utilized their results and released the prediction
tool working on Git repositories [5]. Bird et al. [3] and Posnett et al. [10] focusedon the ownership of source ﬁles and reported that a source ﬁle having lower
ownership is likely to be more fault-prone; low ownership of a source ﬁle meansthat the ﬁle has not been developed and maintained by speciﬁc core developer(s),i.e., the ﬁle has been modiﬁed by various developers.
According to the previous work, the change history and ownership of source
ﬁles are promising data for analyzing the occurrence of bug ﬁxes. However, mostprevious studies focused on the number of bug ﬁxes or the bug-ﬁx rate; weconsider that the time to bug ﬁx would be yet another noteworthy feature to
be analyzed. For example, suppose bug ﬁxes were made in two ﬁles fA and fB .
If fA was modiﬁed “one day” ago and fB was done “one year” ago, we should
preferentially examine the precedent modiﬁcation of fA than fB . In order toanalyze such a diﬀerence, we will focus on the survival analysis [12]. The survival
analysis is popular in the medical ﬁeld, which analyzes the survival rate and thesurvival time of patients who were received speciﬁc treatments. In this paper, we
propose an application of the survival analysis to the bug-ﬁx survival time—thetime to bug ﬁx. The key contributions of this paper are as follows:
1. An application of the survival analysis to the bug-ﬁx events in source ﬁles:
To the best of our knowledge, although there have been studies utilizing thesurvival analysis to the substantivity of OSS projects [2, 14], there have notbeen a study applying it to the time to bug ﬁx.
2. An empirical report with many OSS projects: We collected data from 100
OSS projects, and report the results of the bug-ﬁx survival analysis.

2Survival Analysis and Its Application to Bug Fix

2.1

Survival Analysis

The survival analysis is a statistical method for analyzing the time to the occurrence of an event (e.g., a patient death in a clinical site).
Let t be the elapsed time from the start of our observation, and S(t) be theprobability that the event of interest had not been occurred until t, i.e., a subject
survives at t. S(t) is a monotonically decreasing function and called the survival
rate function. The expected survival time μt is computed as:(1)μt = − t dS(t) .
That is to say, μt is the area under S(t) (see Fig. 1).

Fig. 1. Example of S(t) and μt (area of the hatched part).

Needless to say, there may be a subject which has survived after the end of
our event observation. Such a subject is called a “censor sample.” If we have acensor sample, we cannot get true S(t). However, when our event occurrence time
is discrete type (t = t1 , t2 , . . . , ti , . . .), we can estimate it by using the KaplanMeier (KM) method (see [7] for the details). The KM method is a popular
non-parametric method for estimating S(t) using the cumulative hazard as:

S(ti ) =i

{ 1 − λ(ti ) } ,

(2)

k=1

where λ(ti ) is the hazard at ti and λ(ti ) = ei /ni ; ni is the number of subjects
which have survived at least just before ti and ei is the number of subjects whichdied (encountered the event) at ti , respectively.
2.2

Application of Survival Analysis to Bug Fix

Now we consider another survival analysis in which a subject and an event are asource ﬁle and a bug ﬁx, respectively. Then, we can develop a model of the time
to bug ﬁx in a source ﬁle. While many studies have been done for predicting
bug ﬁxes in the past, most of them focused on the number of bug ﬁxes or
the occurrence rate. We will focus on yet another point of view in this paper.
According to the previous work regarding the ﬁle ownership [3, 10], modiﬁcations
made by new developers may have higher risks of causing future bug ﬁxes. By
analyzing the survival time to bug ﬁx, we will evaluate such risks. We describe
the survival analysis of the time to bug ﬁx in the remainder of this section.
Let f be a source ﬁle, and tE be the time when the observation was ﬁnished,respectively. Then, let tM (< tE ) be the time when f was ﬁnally modiﬁed (orcreated) but the modiﬁcation was not a bug ﬁx. If a bug is ﬁxed in f at tB , thelife time of f to the bug ﬁx, L(f ), is computed as L(f ) = tB − tM . If bug ﬁxes
occurred in f twice or more, use the oldest bug-ﬁx time as tB ; if no bug ﬁx was
observed until tE , f is a censor sample. Then, we deﬁne L(f ) = tE − tM .
Next, we check the developer who made the modiﬁcation at tM , and examine
whether he/she is a new developer who has never been involved in f before tM .
Then, we categorize f into two types, NEW and CONV—if f is modiﬁed bya new developer at tM , we consider f to be Type NEW; otherwise, f is TypeCONV which means that the modiﬁcation is made by a conventional developer.
Table 1 presents a simple example where eight ﬁles f1 , f2 , · · ·, f8 have beendeveloped and maintained by three developers (ID = 1, 2, 3). Symbols “A,” “M”
and “B” in the table signify a creation of new ﬁle, a modiﬁcation of a ﬁle, and a
bug ﬁx of a ﬁle, respectively. Their subscripts denote the developer ID who madethose work. For each ﬁle, the modiﬁcation (or the creation) according to its tM
is marked with an asterisk. For example, f1 , f4 and f5 are modiﬁed by developer
1 at t = 21. f1 is Type NEW because its ﬁnal modiﬁcation at t = 26 is madeby developer 2 and it is the ﬁrst time for developer 2 to modify f1 at that time.
Similarly, f5 is Type CONV since its ﬁnal modiﬁcation before the bug ﬁx (att = 21) is made by developer 1 and he/she had already been involved in f5 . Since
f1 , f4 , f6 and f8 have no bug ﬁx within the observation duration (0 ≤ t ≤ tE ),they are censor samples and their life times are computed as L(fi ) = tE − tM .
The remaining ﬁles’ life times are obtained as L(fi ) = tB − tM .
The KM method can estimates the survival rate function S(t) with Equation(2). Figure 2 shows the estimated S(t) of each type. We can see that NEW has
a shorter life time to bug ﬁx than CONV. The expected survival time of NEW

Table 1. Example of source ﬁle development history.
ttE

15 18 20

21 25 26 30 31 35

ﬁle
0
f1
f2
f3
f4
f5
f6
f7
f8

5

12

A1 M1
A1
A1
A1
A1

M∗2 B1A2
M2 M∗2
A∗2

M∗2

M∗2

M1
B2

M∗3

M1
M∗1 B2
B2

A∗3

type

tB

tM L(fi )

NEW
NEW
NEW
NEW
CONV
CONV
CONV
CONV

—
25
18
—
25
—
25
—

26
20
15
31
21
15
12
30

9
5
3
4
4
20
13
5

and CONV are 6 and 13.375, respectively3 . While both type have the same bugﬁx rate (50%), they have remarkable diﬀerences in terms of expected survival
time—6 vs. 13.375: the expected survival time of Type NEW is shorter than thehalf of Type CONV’s survival time.

Fig. 2. Estimated S(t)’s for NEW and CONV types in Table 1.

3

Empirical Study

3.1

DatasetWe collected 100 local copies of OSS projects’ repositories which are available on
the GitHub, and obtained data to be analyzed from those repositories. Our subject projects consist of 50 Java projects and 50 C++ ones. The main reason why
we selected these projects is their popularities; we believe that a ﬁnding derived
from more popular projects is more attractive for more researches and practitioners. These projects have high “stars” scores at GitHub: We performed project
searches sorted by “most stars” option, where the search keywords were “Java”
and “C++,” respectively. Table 2 presents the names of collected projects.
3.2

Procedure

For each project, we conducted our empirical study in the following four steps.
3

NEW: 1 × 3 + 0.750 × (5 − 3) + 0.375 × (9 − 5) = 6; CONV: 1 × 4 + 0.750 × (13 −4) + 0375 × (20 − 13) = 13.375.

Table 2. Analyzed OSS projects (in decreasing order of “stars”).
Java projects

C++ projects

RxJava, elasticsearch, retrofit, okhttp,java-design-patterns,
guava,
leakcanary,
zxing, libgdx, interviews, fastjson, dubbo,
Android-CleanArchitecture,
realm-java,
MaterialDrawer, ExoPlayer, deeplearning4j,
BottomBar, spark, vert.x, dagger, presto,
junit4, Android-Bootstrap, dropwizard, UltimateRecyclerView, uCrop, jedis, auto, guice,mybatis-3, jadx, metrics, mockito, HikariCP,
webmagic, buck, j2objc, jsoup, lombok,
rebound, swagger-core, pinpoint, scribejava,okio, android-classyshark, async-http-client,mosby, CoreNLP, dex2jar

folly, imgui, json, libphonenumber, openFrameworks, Catch, proxygen, capnproto,
rapidjson, libzmq, libsass, muduo, crow,tiny-dnn, ppsspp, dlib, spdlog, Cpp-Primer,
openpose, pybind11, GamePlay, re2, concurrentqueue, envoy, oclint, zopfli, nghttp2,
cpprestsdk, websocketpp, osrm-backend,
BansheeEngine,swig,
i2cdevlib,
algorithms, AtomicGameEngine, mlpack, thrust,
iaito, glog, cpr, cpp-ethereum, magnum,
cppcheck, gosu, phxpaxos, deepdetect,actor-framework, cereal, oryol, cling

1. Made a copy of the repository, and obtained the set of source ﬁles includedin the latest version (F ). Source ﬁles for testings, demos and documents were
excluded from F . Let tE be the time when the repository was copied.
2. For each f ∈ F , extracted its change history from the commit logs, anddecided tB : if the commit message of f contained a bug ﬁx-related keyword,
we considered that a bug ﬁx was performed in f at the commit [13].
3. For each f ∈ F , determined tM and and f ’s type—NEW or CONV. The
identiﬁcation of developer was performed in accordance with the following
rules [1]: if two developers had the same name or the same e-mail address,
we considered that they are the same developer.
4. Estimated the survival rate function S(t) for each types, NEW and CONV,
using the KM method. Then, computed the expected survival time with
Equation (1): let μN and μC be the expected survival time in NEW and
CONV, respectively. To compare their diﬀerences across projects, deﬁne thefollowing criterion, Δμ:μC − μN
Δμ =
.
(3)
μC
If Δμ has a larger positive value, Type NEW ﬁles have shorter expectedsurvival times than Type CONV ones. While “μC − μN ” directly shows thediﬀerence of two survival times, we considered it is better to normalize thediﬀerence with using one of those times because there would be dispersionsof survival times among projects; such a raw diﬀerence would not be suitablefor comparing diﬀerent projects.
3.3

Results

Table 3 and Fig. 3 show distributions of Δμ values. The median and the average(mean) of Δμ in the 100 OSS products are 0.394 and 0.258, respectively (seeTable 3). Moreover, a majority of projects show Δμ > 0 (see Fig. 3). In otherwords, the expected survival times of Type NEW source ﬁles tend to be about
26% shorter than that of Type CONV on average.

Table 3. Distribution of Δμ values in analyzed OSS projects.
min

25%

median mean

75%

max

−1.675

0.048

0.394

0.628

0.997

0.258

Δμ

1

0

−1

Fig. 3. Boxplot of Δμ values in analyzed OSS projects.

3.4

Discussions

Through the survival analysis, we have understood a major trend of time to bug
ﬁx in many OSS projects—how long time a source ﬁle tends to take until a bug
ﬁx, rather than whether a bug ﬁx would occur or not. A source ﬁle modiﬁedby a new developer (Type NEW) would have a higher risk of a latent bug andthe time to bug ﬁx would be about 26% shorter than another type of source ﬁle(Type CONV). This trend seems to support the previous work regarding the ﬁleownership [3, 10].
While the above results show an overall trend, some projects had small Δμvalues or the opposite trends. To examine if there is a statistically signiﬁcantdiﬀerence between NEW and CONV in terms of S(t), we performed the logrank test [9] (at a 5% signiﬁcance level). The test results were as follows: there
seem to be signiﬁcant diﬀerences in 65 products (does not in 35 products); in
56 out of 65 products, Type NEW tends to have a shorter survival time (denote
it by “NEW < CONV”); the remaining 9 products show the opposite tendency(“CONV < NEW”).
To explore a diﬀerence between two cases “(a) NEW < CONV” and “(b)CONV < NEW,” we examined the numbers of developers (= nd ) and source
ﬁles (= nf ) in those projects, and calculated the ratio between them: r = nd /nf .
While the average of r values in case (a) is about 0.079, that in case (b) is 0.246.
That is to say, r in case (b) is about 3 times larger than case (a); furthermore,
the average r of the outliers shown in Fig. 3 is 3.93 which is about 50 times
larger than case (a). Hence, case (b) tends to have more variety in terms ofdevelopers relative to the number of source ﬁles to be maintained, and theywould be projects which more new developers can actively contribute to. We
would like to do a further analysis from such a viewpoint as our future work.
Now, we have to notice that we have checked bug “ﬁxing” events but notbug (fault) “inducing” ones. In other words, even if a bug ﬁx occurred aftera new developer’s commit, it is not always true that the corresponding fault
was induced by the new developer. We need to perform a further analysis offault-inducing commits in the future for an enhanced discussion.
3.5

Threats to Validity

We collected only Java and C++ data from only Git repositories. Since ouranalysis method is applicable to any other programming languages and version
control systems (VCSs) without any change, the limitations of language andVCS are not serious threats to validity. Nonetheless, there is a risk of getting
diﬀerent results in OSS projects other than the ones we examined. To mitigate
such a threat, we collected empirical data from a set of many popular projects.
While we focused on whether a source ﬁle modiﬁcation was made by a new
developer or not, we did not examine his/her experience of maintaining othersource ﬁles and expertise. Since our ﬁndings in this paper are derived from a lotof samples, impacts of individuals on our results might not be serious threats. A
further analysis on individuals is our important future work.
We decided whether a commit is a bug ﬁx or not, by using a keyword matching method [13], and it is a popular way of detecting bug-ﬁx commits in themining software repository community. However, our analysis was based on theassumption that all commit messages provided proper information in regard totheir code changes. The assumption can be a threat to validity. For example,
some bug-ﬁx commits might be missed in our dataset because some developers
might not appropriately describe their commit messages even if they performed
bug ﬁxes. The development of a more accurate detection method is one of our
future challenges.

4

Conclusion

For a bug ﬁx of a source ﬁle in an OSS development, we focused on the developer
who made the last modiﬁcation of the ﬁle before the bug ﬁx—whether the developer had an experience of modifying the ﬁle at the time or not. In accordancewith the above developer type, we categorized source ﬁles into the following two
types, NEW and CONV: if a source ﬁle’s last modiﬁcation before its bug ﬁx(or the end of our observation) was made by a new developer who had never
modiﬁed that ﬁle, the source ﬁle is Type NEW; otherwise, it is Type CONV.
Then, we considered to compare these two types in terms of the time to bug
ﬁx through a survival analysis, and conducted an empirical study with 100 OSS
projects which are available on the GitHub. The empirical results showed thatthe expected survival time of Type NEW source ﬁles is about 26% shorter thanthat of Type CONV ones. That is to say, when a source ﬁle is modiﬁed by a new
developer who had not been involved in the maintenance of the ﬁle, that ﬁle islikely to require a bug ﬁx sooner. In such a case, a more careful review would be
useful to prevent a quality degradation.

On the other hand, if a project has more developers relative to the total
number of source ﬁles, the project tends to produce the opposite tendency: TypeNEW has a longer expected survival time. Such a project may be easier for more
new developers to contribute. A further analysis on those points of view is our
future work. Moreover, we plan to take into account the degree of a developer’s
familiarity with a ﬁle and to perform a further analysis toward a just-in-time
defect prediction.
Acknowledgment
This work was supported by JSPS KAKENHI #16K00099. The authors would
like to thank the anonymous reviewers for their helpful comments.

References
1. Bird, C., Gourley, A., Devanbu, P., Gertz, M., Swaminathan, A.: Mining emailsocial networks. In: Proc. Int’l Workshop Mining Softw. Repositories. pp. 137–143.
(2006)
2. Bird, C., Gourley, A., Devanbu, P., Swaminathan, A., Hsu, G.: Open borders?
immigration in open source projects. In: Proc. 4th Int’l Workshop Mining Softw.
Repositories (2007)
3. Bird, C., Nagappan, N., Murphy, B., Gall, H., Devanbu, P.: Don’t touch my code!:
Examining the eﬀects of ownership on software quality. In: Proc. 19th ACM SIGSOFT Symp. & 13th European Conf. Foundations of Softw. Eng. pp. 4–14. (2011)
4. Black Duck: the 2017 Open Source 360◦ Survey.
https://www.blackducksoftware.com/about/news-events/releases/open-source360-organizations-increase-reliance-open-source (2017)
5. Google: Bugspots. https://github.com/igrigorik/bugspots (2011)
6. Jones, C.: Applied Software Measurement: Global Analysis of Productivity andQuality. McGraw-Hill, New York, 3rd ed. (2008)
7. Kaplan, E.L., Meier, P.: Nonparametric estimation from incomplete observations.
J. American Statistical Association 53(282), 457–481 (1958)
8. Li, Y., Li, D., Huang, F., Lee, S.Y., Ai, J.: An exploratory analysis on software developers’ bug-introducing tendency over time. In: Proc. Int’l Conf. Softw. Analysis,
Testing and Evolution. pp.12–17. (2016)
9. Peto, R., Peto, J.: Asymptotically eﬃcient rank invariant test procedures. J. Royal
Statistical Society. Series A (General) 135(2), 185–207 (1972)
10. Posnett, D., D’Souza, R., Devanbu, P., Filkov, V.: Dual ecological measures of focus
in software development. In: Proc. Int’l Conf. Softw. Eng. pp. 452–461. (2013)
11. Rahman, F., Posnett, D., Hindle, A., Barr, E., Devanbu, P.: Bugcache for inspections: Hit or miss? In: Proc. 19th ACM SIGSOFT Symp. & 13th European Conf.
Foundations of Softw. Eng. pp. 322–331. (2011)
12. Rupert G. Miller, J.: Survival Analysis. John Wiley & Sons, Hoboken, NJ (2011)13. Śliwerski, J., Zimmermann, T., Zeller, A.: When do changes induce ﬁxes? In: Proc.
Int’l Workshop Mining Softw. Repositories. pp. 1–5 (2005)
14. Samoladas, I., Angelis, L., Stamelos, I.: Survival analysis on the duration of open
source projects. Inf. & Softw. Tech. 52, 902–922 (2010)