Exploring the Adoption of Fuzz Testing in
Open-Source Software: A Case Study of the Go
Community
Olivier Nourry∗ , Masanari Kondo∗ , Mahmoud Alfadel† , Shane McIntosh† , Yasutaka Kamei∗∗ Kyushu University, Japan — oliviern@posl.ait.kyushu-u.ac.jp, {kondo,kamei}@ait.kyushu-u.ac.jp† Software REBELs, University of Waterloo, Canada — {malfadel,shane.mcintosh}@uwaterloo.ca

Abstract—Fuzz testing (or fuzzing) is a software testing technique aimed at identifying software vulnerabilities. Recently, theGo community added native support for fuzz testing into theirstandard library. Using that feature, developers can write unit
tests to perform deterministic and fuzz testing of their softwaresystems against unexpected inputs. Although the availability of
support makes fuzz testing more accessible for the Go community
at large, little is known about the degree to which Go developers
adopt fuzz testing during software development. Therefore, inthis paper, we set out to study the evolution of fuzz testing
practices in open-source Go projects. More specifically, we strive
to understand whether the introduction of support for fuzz testingin the Go standard library has led to the adoption of fuzz testingas part of the standard testing processes of Go projects. To
achieve our goal, we study 1) to what extent fuzz tests are usedin open-source Go projects, 2) who writes and maintains fuzz
tests in Go projects, and finally, 3) how tightly coupled are fuzz
tests with source code (as compared to non-fuzz tests). We findthat fuzz testing only represents 3.15% of testing functions in
open-source projects. Our results also suggest that fuzz testingdevelopment is not being conducted as part of standard testing
activities. For developers contributing to fuzzing, we find that a
median of only 12.50% of their testing-related commits containfuzz tests. Finally, we perform a qualitative analysis and findthat fuzz testing is mostly used by critical software systems, suchas blockchain technologies or network infrastructure projects,
to test the most critical features of their systems (e.g., dataprocessing functions, database endpoints). Our results lead us
to conclude that fuzz testing is best used in combination withdeterministic testing (e.g., unit testing) where fuzzing is used tothoroughly test important features, and deterministic testing isused to test other features.

I. I NTRODUCTION
Anaconda’s 2022 yearly survey on the state of data science [1] showed that the fear of vulnerabilities (such as
the Log4j vulnerability1 ) in open-source software has caused
private companies to start scaling back their use of open-source
software. To address the issue of vulnerabilities in opensource software systems, developers are increasingly turning
to automated testing strategies, such as fuzzing [2, 3]. Fuzzing
is a software testing technique where a large number of inputs
are sent to a software system with the goal of triggering unexpected behaviors. While unit tests allow developers to test asoftware system against expected inputs, fuzzing complements
unit tests by testing against unexpected or abnormal inputs.

Using fuzzing, developers have been able to detect multiple
software vulnerabilities and fix them before an attacker can
exploit them (e.g., CVE-2016-6978,2 CVE-2017-3732,3 CVE2019-164114 ).
As of 2024, fuzzing is mostly conducted using specialized
external tools called “fuzzers” that generate inputs and send
them to a target software system. Since the launch of OSSFuzz in 2016 [4, 5], a substantial portion of fuzzing activities
in the open-source software ecosystem has been conducted via
external services. These external services allow open-source
projects to benefit from continuous fuzzing without carrying
the heavy computational costs of fuzzing.
Due to growing interest in fuzzing [2, 3] within the opensource community, other initiatives aimed at making fuzzing
more accessible have also been launched. For instance, in
2021, the Go community decided to integrate fuzzing (issue #44551)5 within the Go programming language itself toempower developers to write fuzz tests just as easily as unit
tests. Additionally, the efforts of the open-source community
to make the adoption of fuzzing easier have motivated the
addition of Go features to facilitate the adoption of OSS-Fuzz(issue #50192)6 and have also led OSS-Fuzz developers to add
native Go fuzzing support [6].
Since unit tests do not scale well to test against large
amounts of abnormal inputs, the increased accessibility of
fuzzing now provides developers with the opportunity toadopt fuzz testing as part of their standard software testing
practices. With the considerable security benefits provided by
combining deterministic testing, such as unit testing, withscalable approaches, such as fuzzing, we believe that the
research community has an important role to play in making
the adoption of fuzzing as easy as possible.
As of 2024 (two years after the introduction of fuzz tests
into the Go standard library), we still do not know if opensource communities have started using fuzzing to maketheir open-source projects more secure and avoid the
spread of vulnerabilities throughout the ecosystem. We
therefore set out to study the co-evolution of fuzz testing
2 https://nvd.nist.gov/vuln/detail/CVE-2019-16411
3 https://nvd.nist.gov/vuln/detail/CVE-2017-3732
4 https://nvd.nist.gov/vuln/detail/CVE-2016-6978
5 https://github.com/golang/go/issues/44551

1 https://www.cisa.gov/news-events/news/apache-log4j-vulnerability-guidance

6 https://github.com/golang/go/issues/50192

alongside standard testing activities in open-source projects
hosted on GitHub. To provide an overview of the state of fuzz
testing in OSS projects, we conduct a conceptual replicationof McIntosh et al.’s analysis of build maintenance effort [7]and aim to answer the following preliminary questions:(PQ1) Do developers use fuzz testing?
(PQ2) Is fuzzing test code more prone to change than nonfuzzing test code?
Building upon the results of these preliminary results, we
further investigate to answer the following research questions:(RQ1) Is fuzz testing development conducted as part of thestandard testing process?
(RQ2) Who conducts fuzz testing in open-source projects?
Based on the preliminary and research questions mentioned
above, we summarize the findings of this paper as follows:• (PQ1) We find that Go fuzz tests are rarely added bydevelopers two years after their introduction into thestandard Go library. Our analysis reveals that only 3.58%of the studied projects contain any fuzz tests and thosefuzz tests only account for 1.35% (on median) of all test
functions.
• (PQ2) We observe significantly more commits contributing to non-fuzz test code than fuzz test code. Changes tonon-fuzzing test code also tend to be larger than changes
to fuzz tests.
• (RQ1) We find that neither non-fuzzing test code changes
nor source code changes are tightly coupled with fuzz testchanges. Our results suggest that fuzz test development
is not being conducted alongside other types of testdevelopment, such as unit tests.
• (RQ2) We find that most projects rely on few contributors
to contribute most of the testing code (both fuzzing testcode and non-fuzzing test code). Conversely, we find thatsignificantly more developers have contributed to nonfuzzing test code, whereas most developers have not madeany contribution towards fuzzing activities.
Our results lead us to conclude that non-fuzz testing is still
the backbone of testing activities in OSS projects. From ourinvestigation, we also believe that fuzzing might be best usedon parts of a codebase that require rigorous testing. From thenumber of developers that have contributed to the developmentof fuzz tests, we also conclude that more work needs to bedone to lower the barrier of entry to fuzz testing.
II. BACKGROUND AND R ELATED W ORK
In this section, we define key concepts and then situate ourwork with respect to the literature.
A. Background
Challenges in the adoption of fuzzing. Every activity
related to software development and maintenance brings its
own set of challenges. Kim et al. conducted a field study anda series of interviews with refactoring experts at Microsoft
to study the challenges of refactoring [8]. They find thatrefactoring activities are both risky and costly with developers

spending on average 10% of their work hours refactoring code.
Similarly, Greiler et al. interviewed 25 senior practitioners tolearn more about the challenges of testing plug-in-based systems [9]. Their study reveals that even experienced developers
face difficulties writing tests for their software systems.
As the adoption of fuzzing into the software testing process increases over time [10, 11, 12], software testers anddevelopers will need to manage the challenges incurred byfuzz testing. In our prior work, we conducted a survey withfuzzing experts that revealed over 22 distinct challenges ofconducting fuzzing activities [13]. We found that the usabilityof fuzzers (i.e., setting up, building, or using a fuzzer) is
the key challenge that experts are facing when conducting
fuzz testing. Thus, making fuzzing more accessible could
lower barriers to the adoption of fuzzing into standard testing
activities.
Native Go fuzzing. Fuzz tests were integrated into the Go
language itself in release 1.18 on March 15th, 2022 [14].
To ease the adoption of this native fuzzing feature, theGo language developers purposely designed fuzz tests tobe similar to unit tests to facilitate the conversion between
normal tests and fuzz tests.7 We surmise that this nativeand straightforward feature could help address the usability
challenges of conducting fuzzing activities in OSS projects.
With this new feature simplifying the development of fuzz
tests, we set out to understand if open-source communities
are starting to use fuzz testing to make their projects moresecure. Thus, we analyze how frequently developers employ
fuzz tests and how they evolve alongside non-fuzzing tests.
Listing 1 shows an example of a fuzz test retrieved from thebabylonchain/babylon project. This target of the test is
the GenKeyPair function, which is designed to generate apair of encryption keys. As we can see, the structure to write
Go fuzz tests closely resembles Go unit tests.
Developers first define the seed values that will be sent to
the target function using f.Add. Then, developers must write
the code that will test the target function and wrap it withthe f.Fuzz function to make the test into a fuzz test. Unlike
unit testing, the initial values provided to fuzz tests will also
be used to generate new inputs for further testing.
B. Related work
The co-evolution of source code and non-source code.
McIntosh et al. investigated build maintenance efforts andthe logical coupling between source code, test files andbuild files [7]. They find that build maintenance introducesconsiderable overhead to software development (up to 27%
on source code development and 44% on test development).
Their results reveal that most developers are impacted by build
maintenance. Finally, they find that hiring build experts cangreatly reduce the number of developers impacted by build
maintenance.
7 https://github.com/golang/go/issues/44551#issuecomment-785176317
8 Retrieved
from babylonchain/babylon: https://github.com/
babylonchain/babylon/blob/682c2d238ca305243d361a512bd0c401eedef1c6/
btctxformatter/formatter test.go#L96

// This fuzzer checks if decoder won't panic with whatever,→bytes we point it atfunc FuzzDecodingWontPanic(f *testing.F) {f.Add(randNBytes(firstPartLength),,→
uint8(rand.Intn(99)))
f.Add(randNBytes(secondPartLength),,→
uint8(rand.Intn(99)))
f.Fuzz(func(t *testing.T, bytes []byte, tagIdx,→
uint8) {tag := []byte{0, 1, 2, 3}
decoded, err :=,→IsBabylonCheckpointData(tag,,→CurrentVersion, bytes)
if err == nil {if decoded.Index != 0 &&,→decoded.Index != 1 {t.Errorf("With correct,→
decoding index should,→be either 0 or 1")
}
}
})
}

Listing 1: Fuzz test example8
Zaidman et al. studied the co-evolution of production andtest code in both industrial and open source projects [15]. Theyfind that projects do not increase their testing activity as they
get closer to a new release. They also find that an increase in
test code is often linked with an increase in coverage. Jiang
and Adams studied the co-evolution of infrastructure codeand source code in 265 OpenStack projects [16]. They findthat infrastructure files are frequently modified and are tightly
coupled with test files.
In this study, we examine the co-evolution of fuzzing codeand non-fuzzing testing code. Thus, our work aligns withprevious research that investigates the co-evolution of sourceand non-source code. Our work differs from past studies byinvestigating developer usage of standard test code and fuzzingcode when testing open-source projects.
The usage of new language features. The adoption and usageof new language features has been a recurring topic of researchin software engineering studies. Dyer et al. investigated theuse of new Java features in over 31,000 open-source Javaprojects [17]. They find several areas where developers could
use new Java features but do not. Their investigation also
shows that the adoption of a new language feature is likely
to be driven by compiler support. Moreover, they find that
contributors increasingly start using a feature once the featureis used for the first time in the codebase.
The reluctance of developers to adopt new language features
seems to be a common pattern in studies investigating theevolution of a programming language. Mazinanian et al. [18]studied the use of lambda expressions in Java and surveyed 97
developers that adopted the lambda expression feature. Theirinvestigation shows that the use of a new feature can take sometime before it is widely adopted. They also find that developers
tend to use built-in features inefficiently by using generalfeatures over specialized ones. Similarly, Scarsbrook et al.
studied the adoption of TypeScript features over time [19].
Their observations align with the results of Mazinanian et al.
where they find that the Typescript community tends not touse new language features.
Similar to these studies, our study provides better insights
into how developers are adopting fuzz testing as a new feature.
Our preliminary analysis (described in Section IV-A) showsthat fuzz testing is still not widely adopted with only 3.58%
of projects having fuzz tests. Furthermore, fuzz tests make up
just 1.35% of testing functions. Our observations align withthe conclusions of the previous studies, i.e., we also find thatnew features are not always adopted by developers.
Empirical studies on fuzzing. With the increased availabilityof open-source empirical data brought by the release of OSSFuzz, researchers have been able to conduct more empiricalstudies on fuzzing in recent years. Ding et al. conducted alarge-scale empirical study of over 23,000 OSS-Fuzz bugs
spanning over 316 projects [20]. Their results show thatfuzzing is an effective technique to find bugs and that developers typically fix detected bugs quickly. Yet they also find thatsome flaky bugs detected by fuzzing are problematic to address
or reproduce. Moreover, they find that fuzzing campaigns tend
to find large numbers of bugs in short timeframes requiring
developers to deal with high numbers of bug reports at once.
Böhme et al. conducted a four cpu year-long fuzz campaign
to study the increasing cost of fuzzing over time [21]. Theirstudy shows that the computational cost of fuzzing gets exponentially more expensive over time in order to find bugs. Theirresults prove that improving fuzzers yields more potential
improvements to the efficiency of a fuzzing campaign thanadding more computational power.
The challenges of adopting fuzzing were also studied inboth an industrial and an academic setting. Our prior work
mined fuzzing-related GitHub issues of open-source software
repositories [13]. After manually labeling these issues, wecreated a taxonomy of 22 challenges related to conducting
fuzzing activities and validated their results by surveying
fuzzing practitioners. Some of the challenges described in ourprior work had previously been highlighted in an industrial
setting by Liang et al., where a team of developers at Huawei
tried to start fuzzing one of their libraries [22].
This paper makes the following contributions: (1) we conduct a large-scale empirical study on 11,341 projects to study
the usage and adoption of native Go fuzzing, (2) we shed lighton how open-source communities conduct testing activities,and (3) we propose a list of candidate functions for developers
to fuzz based on our investigation of how fuzz testing is being
applied in practice.
III. S TUDIED DATA
Since the goal of our study is to investigate the usage andevolution of fuzz testing in OSS projects, we first need to findprojects that contain fuzz tests. In this section, we describe ourB. Locating Code to be Analyzed
GitHub62,702
projects
1) Selection of
Target projects

60,982
projects

Extracting projects
via GitHub Search
Removing fork
projects
Extracting the top
20% projects

12,120projects
11,341projects

Removing duplicatedprojects
Finding test files

636,532
test files
Parsing the code
2) Locating
Code andCommits tobe Analyzed

AST files

2,581
fuzz functions

Locating test
Functions andcommits

3,000,877
non-fuzz
functionsFig. 1. Overview of the data collection process

approach for selecting projects and locating code and commits
to be analyzed within the studied projects. Figure 1 provides
an overview of the data collection process.
A. Selection of Target Projects
We begin with the GitHub search tool [23] to find projects.
To collect all projects, we apply the language filter, where
we specify that the projects must be written in Go (i.e., theprimary language of the project is Go). This produces a list of
62,702 projects (as of March 2024). We then filter out projects
that are forks, which reduced the list to 60,982 projects.
Next, since we are interested in how open-source projects
use fuzzing, we need to mitigate the likelihood of toy projects
being included in our dataset. Therefore, we select only
those projects within the top 20% (12,120 projects) in termsof number of commits (x > 246 commits). Because larger
projects tend to have larger communities, we also assume thatlarge projects are more likely to have open-source contributors
conducting fuzz testing within their development team. By
selecting only the large projects, we therefore increase ourodds of finding fuzz tests within the studied projects.
While inspecting our data for inconsistencies, we observe
duplicated projects due to project renaming. For these cases,
although two projects have different URLs, clicking the
GitHub URL would redirect to the most up-to-date GitHub
repository with the correct URL. To find these cases, we therefore used the python requests package to automatically verify
the redirecting URL. To deduplicate our data, we therefore
remove any project in our dataset whose URL redirects to thelatest up-to-date repository. This process further reduces thenumber of projects to 11,341. We then clone these repositories
and proceed with the code parsing process.

Finding test files. When programming using the Go language, developers must follow specific rules in order to write
test code. As described in the official Go documentation [24],all test code must be located in a file with the “ test.go” suffix.
To find test code in our studied projects, we therefore search
for all files ending in “ test.go”. In total, we detect 636,532
test files in our dataset.
Parsing the code. After locating the test files, we use theparser package of the Go standard library9 to parse all Go test
files in our dataset. The parser produces Abstract Syntax Trees(ASTs) in JSON format, which show all test function names
along with their return and parameter types. Additionally, theparser logs the first and last line numbers of each function in
the test files. The parsing process is applied to all test files forevery revision in the history of our studied projects.
Locating test functions. Next, we set out to detect test
functions (both non-fuzzing and fuzzing test functions) within
the test files. To write a fuzz test, developers must follow
the naming conventions that is specified in the official Go
documentation [25], i.e., the function name of a fuzz test must
start with the word “Fuzz (e.g., FuzzTestOne)”. Moreover, thefuzz target must be a function call to the (*testing.F).Fuzz
function with *testing.T being the type of the first parameter.
To locate fuzz tests, we traverse the ASTs that we obtainfrom parsing the test files in search of functions that satisfied
the naming constraints. Since the parser logs the name of all
functions in the test files, we also search for functions thatsatisfy the naming constraints for non-fuzzing tests, such asunit tests (the function name starts with the word “Test”).
Labeling commits. After locating fuzzing tests/code, we
extract the starting and end lines of each test function forevery revision from the ASTs. For each commit, we then use
git to extract the names of the changed files along with theline numbers where the changes are recorded. From there, wematch the line numbers from the AST output and the “diff”
output to label all commits as “non-fuzzing”, “fuzzing”, or“source” according to whether the changes modify fuzzing
code, non-fuzzing test code, or code outside of test files.
Note that these categories are not mutually exclusive, and one
commit may have multiple labels.
IV. P RELIMINARY A NALYSIS
A. Do Developers Use Fuzz Testing? (PQ1)
Motivation. Fuzz tests were only added officially to the
Go language in 2022. Because this feature is so recent, we
still do not know to what extent native Go fuzzing is used bydevelopers to uncover software vulnerabilities. We therefore
aim to know what ratio of test files contain fuzz tests andhow many fuzz tests projects tend to have.
Approach. From the process described in Section III-B,
we were able to find the number of test files, non-fuzz test
functions, and fuzz test functions in our studied projects. In
9 https://pkg.go.dev/go/parser

this first preliminary analysis, we therefore aim to report how
much fuzz testing was being conducted in open-source projects
at the time of data collection (March 2024).
To get an overview of how much fuzz testing is used inpractice, we first calculate the ratio of projects that containat least one fuzz test. From this subset of projects, we thencalculate what percentage of test files contain fuzz tests. Using
the percentage of test files that contain fuzz tests in eachproject, we also calculate the median percentage of test files
that contain fuzz tests across all projects.
From the file level, we then dig deeper to the function levelto calculate the proportion of test functions that are fuzz tests.
To do so, we divide the number of fuzzing test functions over
the overall number of test functions in a given project. We thenrepeat the same aggregation process we did for the file-level
analysis to calculate the median percentage of fuzz functions
over all test functions across all projects.
Results. We find that most projects still do not makeuse of native Go fuzzing features with only 406 projects(3.58%) out of 11,341 projects containing fuzz tests. Whenlooking at the 406 projects that contain fuzzing code, we findthat the median ratio of test files that contain fuzz tests isonly 3.15%. Figure 2 shows the distribution for the number of
non-fuzz test functions and the number of fuzz test functionsin projects that contain at least one fuzz test. As shown in
the figure, non-fuzzing tests are still far more common thanfuzz tests with a median number of 207 non-fuzzing tests per
project versus only 2 fuzz tests per project.
Continuing with our subset projects that contain fuzzing
code, we then calculate what percentage of test functions are
fuzz functions in each project. Using the ratio of fuzz functions
in each project, we then calculate the median ratio of fuzz tests
across all projects and find that fuzz tests only account for
1.35% of testing functions. Overall, our results indicate that
two years after being introduced, fuzz tests only account fora small percentage of testing activities in our studied projects.

105

102

104
103
102

101

101
100

100

Fig. 2. Number of non-fuzz tests (left) and fuzz tests (right) per projectcontaining at least one fuzz test (log scaled)

We find that most projects do not conduct fuzz testing
to uncover software vulnerabilities. For projects that
do fuzz their software system, fuzz tests only accountfor a small percentage of total testing functions. These
results indicate that more work needs to be done tospread the adoption of fuzzing among open-source
communities.
B. Is Fuzzing Test Code More Prone to Change Than NonFuzzing Test Code? (PQ2)
Motivation. While the results of PQ1 reveal the extent towhich developers use fuzz testing, these results do not provide
insights into how frequently developers develop fuzz tests or
the amount of effort required to maintain fuzzing activities.
To get a basic idea of the amount of effort that developers
put into fuzzing their software systems, we investigate themonthly commit activity of fuzz tests and the rate of change
of fuzzing test code per commit (churn). We then calculate thesame metrics for standard test code (i.e., non-fuzzing test code)
to compare the rate of development of standard tests and fuzz
tests. We hypothesize that non-fuzzing test code requires more
maintenance efforts than fuzzing test code because it requires
developers to manually write code to test against new inputs.
Fuzz tests, however, use the fuzzing engine to automaticallygenerate new inputs and may therefore not require as many
interventions from developers.
Approach. Starting from the filtered dataset described in
Section III-B, we further filter out commits dating before theintegration of fuzz tests into the official Go standard library.
To determine if a commit changed fuzzing test code or
standard test code, we first used the starting and ending
line numbers of each test function obtained from the parsing
process described in Section III-B. Next, we used git’s difffeature to find out what lines were changed in each commit
in our dataset. By matching the line numbers of test functions
with the output of the diff command, we were able to find allchanges that happened in either fuzz test functions or non-fuzz
test functions. From there, we further filtered our dataset toremove all commits that did not modify a test function.
For the remaining commits, we calculated the testing code
churn in each commit by summing up the number of lines
added, deleted, or modified in test functions. Using the numberof lines changed (i.e., added, deleted, or modified) in fuzzing
test code and non-fuzzing test code, we then calculated the median number of lines changed per commit across all projects.
To get an overview of testing activities since the introductionof fuzz tests into the standard library, we then plotted themonthly number of commits involving fuzzing test code andnon-fuzzing test code across all projects.
Results. We find that non-fuzzing test functions tend
to have larger changes than fuzz test code changes. From
the filtering process described in Section III-B, the removalof commits predating the introduction of fuzz tests into thestandard library, and the removal of commits that do not target
test functions, we end up with a dataset of 125,927 commits.

V. I S F UZZ T ESTING D EVELOPMENT C ONDUCTED A SPART OF THE S TANDARD T ESTING P ROCESS ? (RQ1)

5000

Commits

4000
3000
2000

Test commits
Fuzz commits

1000
2022-03
04
05
06
07
08
09
10
11
12
2023-01
02
03
04
05
06
07
08
09
10
11
12
2024-01
02
03

0

Fig. 3. Number of commits modifying non-fuzzing test functions and numberof commits modifying fuzzing test functions per month since fuzz tests wereadded to the standard library. Note: The drop at the start of 2024 is because
the data had not been made available yet.

Using the number of fuzz test code lines impacted in eachcommit in our dataset, we first calculate the median number oflines impacted inside fuzz test functions per “fuzzing commit”.
We then repeat the same process for non-fuzzing test functions
and find a median of 17 lines of code changed per fuzzing
commits and 33 lines of code changed per non-fuzzing test
functions. Our results indicate that fuzz tests might require lessdevelopment efforts to maintain than non-fuzzing tests. This
could be due in part to fuzz tests not requiring developers tomanually write each input used during testing.
We find that open-source communities commit significantly more non-fuzzing test code than fuzzing test code
on a monthly basis. We continue our preliminary analysis byinvestigating the monthly number of commits involved withtest code modifications. Figure 3 shows the monthly number ofcommits modifying non-fuzzing test functions and the monthly
number of commits modifying fuzzing test functions between
the introduction of fuzz tests into the standard library in March2022 and the time of data collection in March 2024. During
this period, we calculate 102 median number of commits
per month that change fuzz test code across our studied
projects. This is a stark contrast to non-fuzzing test code where
the median monthly number of commits that modify nonfuzzing test functions is 4,535. Overall, our results indicatethat developers still see non-fuzzing tests such as unit tests as
the main avenue to test a software system. Consequently, it islikely that most of the testing efforts in open-source projects
are dedicated to deterministic testing over fuzz testing.
We find that open-source software projects both commit significantly more often and commit larger changes
to non-fuzzing test code over fuzzing test code. Our
results suggest that open-source communities tend to
spend more development efforts on non-fuzz testing
than fuzz testing.

Motivation. In Section IV-B, we found that non-fuzzing
test code is committed at a much higher rate than fuzzing test
code. In this section, we aim to get a better understanding of
how open-source communities conduct their testing activities.
Specifically, we investigate if changes to fuzzing test code andstandard test code are typically done at the same time. From
this analysis, we aim to find out if fuzzing development isconducted as part of standard test development or not.
Approach. After removing commits older than the introduction of native Go fuzzing, we use association rules [26, 27,
28, 29] to measure the coupling between source code changes,
non-fuzzing test code changes, and fuzz testing code changes.
In formal terms, association rules are defined as statisticaldescriptions of the co-occurrence of elements in a dataset [26].
In other words, they are rules used to find associations andrelationships between different types of data points. Table I
shows how we calculated the association rules used in thispaper.
TABLE IA SSOCIATION RULE CALCULATIONS
RuleCalculation

Support(X)# type X revisions# total revisions

Conf(X –>Y)Support(X,Y)
Support(X)

Lift(X–>Y)

Conf(X–>Y)Support(Y)For our use case, we first use the support rule to know the
proportion of commits that contain a specific type of change(source, non-fuzzing test, fuzz test). To measure the coupling
relationship between our studied changes (source, non-fuzzing
test code, fuzzing test code), we then use the confidence rule
which measures the strength of the implication that a change to
X will be accompanied by a change to Y. For example, let’s
assume our dataset contains 9 commits in total (5 commitsthat modify the source code, and 4 commits that change nonfuzzing test code). If one of these commits changes both thesource code and non-fuzzing test code, then the Conf (Source→ Test) would be measured as (1/5) = 0.2 and the Conf (Test→ Source) would be measured as (1/4) = 0.25. Note that theconfidence value is not symmetrical as shown in the example
above (Conf (Source → Test) ̸= Conf (Test → Source)).
The last rule we use is the Lift association rule. The lift
value represents the ratio between the observed rate of cooccurrence and the rate of co-occurrence that would havebeen expected due to random chance. A lift value of 1
indicates that two variables are independent of each other. A
positive lift value greater than 1 means that we are observing
more instances of an event than we would expect from arandom distribution, and a value lower than 1 indicates that
we are observing fewer instances than we would expect from

TABLE II
M EDIAN ASSOCIATION RULE VALUES CALCULATED ACROSS OUR DATASET

Support.
Confidence

Lift

Association
SourceTest
Fuzz
Source/Test
Source/Fuzz
Fuzz/Test
Source → Test
Source → Fuzz
Test → Source
Test → Fuzz
Fuzz → Source
Fuzz → Test
Source → Test
Source → Fuzz
Test → Fuzz

Median
0.667
0.309
0.009
0.249
0.005
0.004
0.369
0.007
0.861
0.015
0.909
0.750
1.239
1.204
2.393

a random distribution. For our use case, we use the lift value
to know if our distributions are likely obtained due to chanceor not. Note, the lift value is symmetrical (Lift(X →Y) =Lift(Y→X))
Results. Table II shows the median association rule values
across all projects. Looking at the support metrics, we findthat source code changes occur at a 2:1 rate when comparedwith test changes (0.667 and 0.309 respectively). The median
support value for fuzz tests (0.009) further indicates that mostcommits do not contain changes to fuzz test code. This is
in line with our preliminary results where we found thatdevelopers do not use fuzz tests as much as non-fuzzing tests.
Source code changes are much more likely to be accompanied by non-fuzzing test code changes than fuzzing testcode changes. The median Conf(Source → Test) value reveals
that almost 37% of source code changes are accompanied by
non-fuzzing test code changes. This result shows that testing
activities are well integrated within the software development process in our studied projects. However, the medianConf(Source → Fuzz) value shows that fuzz testing is not aconsistent part of software development two years after fuzz
tests were added to the standard library. From the lift values,
we observe a small positive relationship between changes forfiles involving source code changes. For standard test and fuzz
test changes, however, the higher lift value indicates that the
coupling values we calculated are unlikely to be due to chance.
Fuzz tests do not seem to be conducted as part ofregular testing activities. With the Conf(Test → Fuzz) value
being twice as large as the Conf(Source → Fuzz) value,
we find that fuzz tests are more likely to be added during
testing activities than source code development. However, with
only 1.5% of testing commits containing fuzzing test codechanges, our results indicate that fuzz testing is not a common
testing activity in our studied projects. This pattern is further
highlighted by the Conf(Fuzz → Source) and Conf(Fuzz →Test) values (0.909 and 0.750 respectively) and the distributionof confidence values in Figure 4 which show that fuzz testchanges seem to be accompanied by both source code changes
and standard test code changes indiscriminately.

1.0
0.8
0.6
0.4
0.2
0.0
0.4
0.3
0.2
0.1
0.0

Source->Test

Source->Fuzz

1.0
0.8
0.6
0.4
0.2
0.0
1.0
0.8
0.6
0.4
0.2
0.0

Test->Source

Test->Fuzz

1.0
0.8
0.6
0.4
0.2
0.0
1.0
0.8
0.6
0.4
0.2
0.0

Fuzz->Source

Fuzz->Test

Fig. 4. Distribution of confidence values across studied projects.

We find that source code changes are much more
likely to be accompanied by non-fuzzing test codechanges than fuzzing test code changes. We also findthat fuzzing test code changes are committed with both
source code changes and standard test code changes.
Our results indicate that fuzzing development is not
being done as part of regular testing activities.
VI. W HO C ONDUCTS F UZZ T ESTING IN O PEN -S OURCEP ROJECTS ? (RQ2)
Motivation In PQ1, we found that fuzz testing is still
an underutilized feature by open-source contributors whichsuggests that there is room to improve the security of opensource software systems. While we cannot know the exact
reasons behind the low usage of fuzz testing, the testimonies
of fuzzing experts in Nourry et al.’s 2023 survey revealed thatfuzzing has a high barrier of entry [13].
In this section, we aim to find out if fuzzing is mostly
conducted by projects that have fuzzing or security experts
within their contributors. Moreover, we want to know if theaverage open-source contributor contributes to fuzz testing
activities. Finding out empirically if projects distribute thefuzzing workload among contributors will also help us understand if the barrier of entry to fuzzing is still too high after
introducing fuzz tests into the standard library.
Approach As we did in Section V, we first removed
commits older than the introduction of native Go fuzzing.
From the mining process described in Section III-B, we also
extracted the author of every commit. To find out if an average
open-source developer is likely to work on non-fuzzing testcode but not fuzzing test code, we first find out the number ofdevelopers conducting testing activities in each project. Thus,
we sum up the number of unique developers that deleted,modified, or added fuzzing test code during the lifetime of a
project. We then perform the same calculation for non-fuzzing
test code in order to compare the involvement of open-source
contributors with standard testing activities (e.g., writing unit
tests) versus fuzzing activities.

10 https://github.com/golang/go/issues/44551#issuecomment-785176317

TABLE IIIM EDIAN NUMBER OF DEVELOPERS PARTICIPATING TO STANDARDTESTING ACTIVITIES , FUZZING ACTIVITIES , AND BOTH TYPES OF TESTING
ACTIVITIES ACROSS ALL PROJECTS .

Test
Fuzz
Test + Fuzz

# of developers% of developers

32
2
1

23.96%
1.31%
0.50%# of developers responsiblefor 80% of changes
2
1
X

We find that both standard testing and fuzz testing
rely on a few contributors to take on most of the testing
workload. Table III shows the number of developers required
to cover 80% of normal test changes and fuzz test changes.
Since only a handful of developers contribute to writing fuzz
test code, it is not surprising that a few developers can accountfor a large portion of fuzz test code changes. For standard
testing, however, we previously found a 32 median numberof developers that have written non-fuzz test code acrossprojects which initially indicated that the testing workload
might be shared among the contributors. However, as we
can see in Table III, we find that the median number ofdevelopers required to take on at least 80% of the non-fuzz
testing workload is only two. From our results, we therefore
find that open-source projects mostly rely on a few dedicatedcontributors to conduct software testing activities (both fuzz
testing and standard testing).
100

Percentage of fuzzing commits

To get a better understanding of the workload distribution ofsoftware testing activities in open-source projects, we analyzed
the distribution of testing commits in our studied projects.
Thus, we first grouped together all commits belonging to eachproject. Then, for each developer, we calculated the numberof commits that modified non-fuzzing test functions and thenumber of commits that modified fuzz test functions.
Once we knew how many non-fuzz test commits and how
many fuzz commits each developer contributed to a project,
we sorted the developers based on their number of commits ineach project. Starting from the contributors with the most commits, we then calculated the cumulative distribution of nonfuzz test commits and fuzz commits in each project. Finally,
we calculated how many developers are required to achieve
80% of the fuzz testing workload in each project and thencalculated the median number of developers required acrossall projects. We repeated the same process for non-fuzzing testcode in order to compare how developers distribute the testing
workload for fuzz testing versus standard testing.
Lastly, we aimed to determine if developers contributing
to fuzz testing are “fuzzing experts” that focus most of theiropen-source contributions toward fuzzing activities. To findout, we sampled developers that had at least one fuzz commit(i.e., a commit with fuzz test code changes). We then counted
how many testing commits each developer contributed ineach project. In this context, we define “testing commits”
as a commit where the developer adds, deletes, or modifies
a non-fuzzing test function or a fuzz test function. Anychange in test a file that does not modify a test function is
not considered as a testing commit. For each developer, wetherefore calculated what percentage of testing commits arechanging fuzz functions to find out if fuzzing developers arefocusing specifically on fuzzing activities or not.
Results. We find that only a small number of contributors have contributed to fuzz testing activities whereas
a significant portion of contributors have written nonfuzz tests. Table III shows the median number of developers
across projects that participate to standard testing activities,
fuzz testing activities and the median number of developersthat participate to both. Our results show that almost 24% of
open-source contributors (median of 32 developers per project)contributed to standard test code for our selected projects. This
indicates that open-source communities view software testing(specifically non-fuzz testing) as a shared effort to achieve
better code quality and security. Conversely, we find that only
1.31% (median of 2 developers per project) of developers havecontributed fuzz testing code development.
These results show that large open-source projects mostly
rely on a few contributors knowledgeable about fuzzing among
their community to conduct fuzzing activities. Additionally,
these results also indicate that most developers do not feel
comfortable writing fuzz test code even though Go developers
purposefully designed fuzz tests to be similar to unit tests.10

80
60
40
20
0

Fig. 5. Percentage of testing commits that modify fuzzing code for developers
conducting fuzzing activities.

We find that fuzzing developers primarily contributeto standard testing activities over fuzzing activities. To
know if contributors contributing to fuzz testing in opensource projects are mostly dedicated to fuzzing activities, we
calculated what percentage of testing commits (i.e., commitsthat change either a non-fuzz test or a fuzz test.) containchanges to fuzzing test code. Figure 5 shows what percentageof testing commits modify a fuzz test function. With a medianpercentage of 12.50% fuzz commits, we find that fuzzing
developers dedicate most of their contributions towards nonfuzz testing rather than fuzz testing. These results indicate that
open-source contributors conducting fuzz testing are mostly“testing experts” that use fuzzing alongside non-fuzz testingrather than “fuzzing experts” that focus solely on fuzzing.
We find that fewer contributors have contributed tofuzz testing activities than non-fuzz testing activities.
We also find that most test code is developed by afew contributors who focus on testing. Our results
suggest that fuzzing development is not conducted byfuzzing specialists but rather by contributors dedicatedto testing activities in general.

VII. D ISCUSSION
A. What Projects Are Making The Most Use of Fuzz Testing?
Our results in this study have shown that fuzz testing only
represents a small fraction of ongoing testing activities in
OSS projects. While it is surprising how little fuzz tests areused in practice, there is a clear complexity and resource
cost gap between fuzz testing and standard testing that cannot
be ignored. The case study led by Liang et al. [22] on the
adoption of fuzzing at Huawei empirically showed that evenexperienced developers can experience difficulties learning
fuzz testing. Computationally, Böhme et al.’s study on thecost of fuzzing [21] has also shown that long fuzz sessions
are extremely inefficient to run. Due to these limitations, it
is possible that projects might not want to have most of their
testing infrastructure revolve around fuzzing.
While fuzz testing is more complex and more computationally expensive than standard testing, it still has its place within
the realm of software testing. As of 2024, fuzz testing is one of
the best automated approach available to test at scale and findvulnerabilities. However, due to their high cost, it is relevant
to find out in what context fuzz tests are suitable and whichfunctions warrant the high computational cost. For example,
unit tests might be better suited to test simple functions whilefuzz tests might be more suitable for critical features related
to security or data management.
Projects revolving around high-risk fields such as avionics,
finance, and networking might therefore be more comfortablewith the high cost of fuzzing in exchange for better security.
We therefore raise the question: What kind of projects use
fuzz testing the most as of 2024?
Table V shows the ten projects in our dataset that had thehighest number of fuzz test functions. From these top 10
projects, we find that seven of them are directly related todifferent types of infrastructure systems such as blockchain
networks, cloud computing technologies, and networking infrastructure. In fact, we find that half of the top 10 projects
using fuzz testing the most are projects developing blockchain
networks where financial transactions must happen securely
over the network. The other three projects not related to
networking are comprised of custom libraries for softwaredevelopment and a custom database driver.

TABLE VT OP TEN PROJECTS WITH THE HIGHEST NUMBER OF FUZZ FUNCTIONS INOUR DATASET.
Project URL

Fuzz functions

Project description

babylonchain/babylon
lightningnetwork/lnd
spacemeshos/go-spacemesh

160
104
94

cncf/cncf-fuzzing73

ava-labs/avalanchego
luxdefi/node
thepudds/fzgenmtoohey31/iter
gravitational/teleport

65
59
59
50
50

scylladb/scylla-go-driver47

Crypto/Blockchain network
Crypto/Blockchain network
Crypto/Blockchain network
Repository used to fuzz
CNCF projects
Crypto/Blockchain network
Quantum crypto/blockchain network
Fuzz wrapper generator in Go
Go package for iterator functions
Network infrastructure platform
Experimental, high performance
ScyllaDB driver

For most of the projects on the top 10 list, we observea clear pattern where projects using fuzz testing a lot are
dealing with some critical feature involving data integrity or
network security. A vulnerability in these kinds of systems canresult in catastrophic consequences costing billions of dollars,
widespread network security breaches, or large data breaches.
Interestingly, we also observe that the list of top 10 projects
is quite representative of the rest of our dataset of projects
that use fuzzing. After performing a quick manual check on
the rest of our dataset, we find that projects that fuzz are
overwhelmingly comprised of blockchain-related projects andnetwork/cloud infrastructure projects. The remaining projects
consist mostly of custom-made libraries and database-related
projects.
B. What Are the Main Uses of Fuzzing?
From our manual investigation, we were able to notice apattern in what types of projects use fuzzing the most. Since
most of these projects deal with high-risk or critical features,
we are interested to know what features these communities
deem worth fuzzing. To find out, two authors manually investigated the GitHub repositories of the projects listed in Table V
and inspected their fuzz tests.
Since large open-source communities usually have contribution guidelines, test function names are typically descriptiveof the purpose of the test (e.g., FuzzParamsQuery, FuzzEncodingDecoding, etc.).
By inspecting the name of the fuzz test functions and thefuzzing test code, the authors were therefore able to determine
the purpose of each fuzz test and to make a list of reasons why
open-source communities fuzz their software systems.
Table IV shows the types of functions fuzzed by opensource communities and gives examples found during themanual review process. During the manual investigation, we
found that most projects tend to use fuzz testing to fuzz critical
functions. That is, functions that handle core features of asystem and where any bug or unusual behavior can bring down
the entire system or cause serious consequences. For example,
a function that rewards users with cryptocurrency coins while
mining cryptocurrency would be considered a critical feature.

TABLE IV
F UNCTIONS AND FUNCTIONALITIES THAT ARE BEING FUZZED IN OPEN - SOURCE GO PROJECTS .

Data
FeaturesInternal Codebase

Label
Data manipulation
Queries
Database
Project specific features
Networking features
Generative functions
Parsers
Storage
Setup functions
Utility code

ExamplesData encoding/decoding, hashing, zipping/unzipping
Request queries, message queue queries, db queries
Database endpoints, reads, storage
Authentication functions, play/pause/stop functions
Packet fuzzing, request headers fuzzing
Public key generations, token creation, uuid generations, etc.
Request parsers, query parser, packet parsers, data parsers
Storing to a server, into memory or auth token storage
Loading configs, database setup scripts, env setup
Compression, Remote Procedure Calls (RPC),
custom address iterators, file deletion functions, timestamp/dates.

From Table IV, we can see that several of these functionalities such as data encoding/decoding or database queries areused in projects that would not be considered as “high-risk”.
While it is logical to fuzz critical functions for projects insensitive fields (i.e., financial, networking, etc.), we are alsointerested to know if projects not involved in such fields also
fuzz their critical features.
We therefore randomly sampled 5 projects from our datasetthat were not involved with blockchain technologies or did
not develop critical open-source infrastructure. We performed
the same process as before and investigated their usage offuzz testing. Interestingly, we find that these projects do
not fuzz their most important features as much as projects
developing critical software systems. For example, the sourcegraph/zoekt11 project is a large open-source project providing
source code search functionalities. In this project, we findmany functions used to parse, query, and manipulate data
which are key features of the system but find very few fuzz
tests within their testing codebase.
C. Summary and Takeaway
We find that fuzz testing is mostly used by software systems
that require a higher level of security than most projects. Our
manual investigation reveals that projects use fuzz tests to test
functions that are critical for the system to run properly.
Our qualitative analysis’ results are consistent with ourquantitative results where we found that fuzz testing is not
used by most projects and only represents a small fraction oftest code for projects that do use them. From our qualitative
analysis, it seems more likely that fuzz tests are not ignored
by developers but rather only used for specific use cases wherefuzz testing excels over other types of testing.
For developers, we highly encourage every project to use
fuzz tests to detect vulnerabilities since the risk of spreadingvulnerabilities is always present due to the interdependent
nature of the open-source ecosystem [30, 31]. For those notfamiliar with fuzz testing, we recommend fuzzing functions
such as the ones listed in Table IV.
11 https://github.com/sourcegraph/zoekt

For researchers, our manual investigation can serve asa starting point to explore the possibility of automaticallydetecting important functions that should be fuzzed given a
codebase. If such a detection approach could be designed andpaired with automatic test generation tools, we could see asignificant increase in the number of developers that conduct
fuzz testing.
VIII. T HREATS T O VALIDITY
Internal threats to validity concern factors internal to ourstudy that could affect or bias our results. In this study, weused the top 20% largest open-source projects on GitHub to
conduct our analysis. This could bias our results by not being
representative of smaller projects. Using this process, however,
we reduce the risk of analyzing toy projects and increase thechance that our studied projects are conducting real opensource software development.
External threats to validity concern the ability to generalize
the result of this study. In this study, we only analyze theevolution of fuzz testing in OSS projects. Our results may not
be generalizable to other languages that do not offer native
support for fuzzing testing. Because Go is one of the first
language to support native fuzz tests, we, however, believe that
our study is representative of the current use of fuzz testingby the average developer (i.e., developers that are not fuzzing
experts like the ones working on OSS-Fuzz).
Construct threats to validity concern the relationship between theory and observation. To find and mine Go projects,
we used the GitHub search tool published by Dabic et al.
and specified “Go” as the main language in the search field.
While the projects returned by the tool all have Go as theirmain language, the proportion of Go language over the entire
codebase may differ (e.g., some projects might be 100% in
Go while others only have 45% of their code in Go). It
is possible that the use of fuzz testing may differ between
projects developed purely in Go versus projects using Go for
specific parts of their codebase. Additionally, we identified
fuzz test functions based on their arguments as specified in
the official Go documentation. It is however possible that

developers might have used aliased imports which would causesome of our identified functions
IX. C ONCLUSION
In this study, we investigated the usage of native Go
fuzzing in open-source projects. Our results show that very
few contributors develop fuzz tests in open-source projects.
We also find that developers conducting fuzzing tend totarget specific functions rather than use fuzzing as a general
testing strategy. Our results also revealed that less than 4%of our studied projects conduct fuzzing activities. Due to therisk of spreading vulnerabilities throughout the open-source
ecosystem, we highly encourage projects to start adding somefuzz tests to their testing codebase.
For developers, we recommend using fuzz tests in combination with unit tests to get the best balance between computation
cost and code robustness. For developers looking to startfuzzing their software, we propose a list of functionalities thatare strong candidates for fuzz testing.
For researchers, we provide new empirical evidence of how
open-source communities conduct software testing activities.
Our results provide researchers and software engineers witha better understanding of the factors limiting the adoption offuzz testing. From our results, we hope that better tools andmethodologies that help developers adopt fuzz testing can be
developed by the software engineering community.
X. R EPLICATION PACKAGE
We provide all required scripts to replicate the data
mining process, the data filtering process, and the data
handling to obtain our results. The scripts can be foundat the following link: https://anonymous.4open.science/r/
ICSME2024-GoFuzzEvolutionReplication-64E8/README.
md
XI. ACKNOWLEDGEMENTS
We gratefully acknowledge the financial support of the
Natural Sciences and Engineering Research Council (NSERC)
of Canada. We also gratefully acknowledge the financial
support of: (1) JSPS for the KAKENHI grants (JP21H04877,
JP22K18630, JP22K17874, JP24K02921), and Bilateral Program grant JPJSBP120239929; and (2) the Inamori ResearchInstitute for Science for supporting Yasutaka Kamei via the
InaRIS Fellowship.
R EFERENCES[1] Author,“Stateofdata
sciencereport,”2022. [Online]. Available: https://www.anaconda.com/
state-of-data-science-report-2022[2] K. Serebryany, “OSS-Fuzz - Google’s continuous
fuzzing service for open source software,” 2017,https://www.usenix.org/conference/usenixsecurity17/
technical-sessions/presentation/serebryany, Last accessed
6 April 2024.
[3] “ClusterFuzz,” https://google.github.io/clusterfuzz/, Last
accessed 6 April 2024.

[4] Google, “Announcing oss-fuzz: Continuous fuzzing
for open source software,” 2016. [Online].
Available: https://opensource.googleblog.com/2016/12/
announcing-oss-fuzz-continuous-fuzzing.html[5] ——, “Google, oss-fuzz,” https://google.github.io/ossfuzz/.
[6] G. OSS-Fuzz, “Integrating a go project into ossfuzz,” https://google.github.io/oss-fuzz/getting-started/
new-project-guide/go-lang/.
[7] S. McIntosh, B. Adams, T. H. Nguyen, Y. Kamei, andA. E. Hassan, “An empirical study of build maintenance effort,” in Proceedings of the 33rd International
Conference on Software Engineering. Association for
Computing Machinery, 2011, p. 141–150.
[8] M. Kim, T. Zimmermann, and N. Nagappan, “An empirical study of refactoringchallenges and benefits at
microsoft,” IEEE Transactions on Software Engineering,pp. 633–649, 2014.
[9] M. Greiler, A. van Deursen, and M.-A. Storey, “Testconfessions: A study of testing practices for plug-in
systems,” in Proceedings of the 34th International Conference on Software Engineering, 2012, pp. 244–254.
[10] M. Moroz and K. Serebryany, “Guided inprocess
fuzzingofChromecomponents,”
2016,
https://security.googleblog.com/2016/08/
guided-in-process-fuzzing-of-chrome.html,
Last
accessed 6 April 2024.
[11] P. Godefroid, “Fuzzing: Hack, art, and science,” Communications of the ACM, pp. 70–76, 2020.
[12] S. Mallissery and Y.-S. Wu, “Demystify the fuzzing
methods: A comprehensive survey,” ACM Computing
Survey, pp. 1–38, 2023.
[13] O. Nourry, Y. Kashiwa, B. Lin, G. Bavota, M. Lanza, and
Y. Kamei, “The human side of fuzzing: Challenges faced
by developers during fuzzing activities.” Association for
Computing Machinery, 2023, pp. 1–26.
[14] “Go version 1.18 release notes,” https://go.dev/blog/go1.
18.
[15] A. Zaidman, B. Van Rompaey, A. van Deursen, and
S. Demeyer, “Studying the co-evolution of production
and test code in open source and industrial developer
test processes through repository mining,” Empirical
Software Engineering, pp. 325–364, 2011.
[16] Y. Jiang and B. Adams, “Co-evolution of infrastructureand source code: an empirical study,” in Proceedingsof the 12th Working Conference on Mining Software
Repositories. IEEE Press, 2015, p. 45–55.
[17] R. Dyer, H. Rajan, H. A. Nguyen, and T. N. Nguyen,“Mining billions of ast nodes to study actual and potential
usage of java language features,” in Proceedings of the36th International Conference on Software Engineering. Association for Computing Machinery, 2014, p.
779–790.
[18] D. Mazinanian, A. Ketkar, N. Tsantalis, and D. Dig,
“Understanding the use of lambda expressions in java,”
Proc. ACM Program. Lang., pp. 1–31, 2017.

[19] J. Scarsbrook, M. Utting, and R. Ko, “Typescript’s
evolution: An analysis of feature adoption over time,”
in 2023 IEEE/ACM 20th International Conference on
Mining Software Repositories, 2023, pp. 109–114.
[20] Z. Y. Ding and C. Le Goues, “An empirical studyof oss-fuzz bugs,” 2021 IEEE/ACM 18th International
Conference on Mining Software Repositories (MSR), pp.
131–142, 2021.
[21] M. Böhme and B. Falk, “Fuzzing: on the exponential
cost of vulnerability discovery,” in Proceedings of the
28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundationsof Software Engineering. Association for Computing
Machinery, 2020, p. 713–724.
[22] J. Liang, M. Wang, Y. Chen, Y. Jiang, and R. Zhang,“Fuzz testing in practice: Obstacles and solutions,” in2018 IEEE 25th International Conference on Software
Analysis, Evolution and Reengineering (SANER). IEEE,
2018, pp. 562–566.
[23] O. Dabic, E. Aghajani, and G. Bavota, “Samplingprojects in github for MSR studies,” in 18th IEEE/ACM
International Conference on Mining Software Repositories, MSR 2021. IEEE, 2021, pp. 560–564.
[24] “Go documentation, adding a test,” https://go.dev/doc/
tutorial/add-a-test.
[25] “Go fuzz tests,” https://go.dev/doc/security/fuzz/.
[26] R. Agrawal, T. Imieliński, and A. Swami, “Mining association rules between sets of items in large databases,”in Proceedings of the 1993 ACM SIGMOD International
Conference on Management of Data. Association for

Computing Machinery, 1993, p. 207–216.
[27] T. Zimmermann, P. Weibgerber, S. Diehl, and A. Zeller,“Mining version histories to guide software changes,”in Proceedings of the 26th International Conference on
Software Engineering, 2004, pp. 563–572.
[28] M. Mondal, B. Roy, C. K. Roy, and K. A. Schneider, “Associating code clones with association rules for change
impact analysis,” in Proceedings of the 27th International
Conference on Software Analysis, Evolution and Reengineering (SANER), 2020, pp. 93–103.
[29] Z. Lubsen, A. Zaidman, and M. Pinzger, “Using association rules to study the co-evolution of production & test
code,” in Proceedings of the 6th International Working
Conference on Mining Software Repositories, 2009, pp.
151–154.
[30] M. Valiev, B. Vasilescu, and J. Herbsleb, “Ecosystemlevel determinants of sustained activity in open-source
projects: a case study of the pypi ecosystem,” in Proceedings of the 2018 26th ACM Joint Meeting on European
Software Engineering Conference and Symposium on theFoundations of Software Engineering. Association for
Computing Machinery, 2018, p. 644–655.
[31] C. Liu, S. Chen, L. Fan, B. Chen, Y. Liu, and X. Peng,“Demystifying the vulnerability propagation and its evolution via dependency trees in the npm ecosystem,” inProceedings of the 44th International Conference on
Software Engineering. IEEE/ACM, 2022, p. 672–684.