Does Shortening the Release Cycle Affect Refactoring
Activities: A Case Study of the JDT Core, Platform
SWT, and UI projects
Olivier Nourry, Yutaro Kashiwa, Yasutaka Kamei, Naoyasu Ubayashi
Principles Of Software engineering and programming Languages Group (POSL),
Kyushu University, Fukuoka, Japan

Abstract[Context] Several large-scale companies such as Google and Netflix choseto adopt short release cycles (e.g., rapid releases) in recent years. Although
this allows these companies to provide updates and features faster for theirusers, it also causes developers to have less time to dedicate to development
activities other than feature development. [Objective] In this paper, we investigate how refactoring activities were impacted by the adoption of shorterreleases. [Method] We extract all refactorings applied over a period of two
years during traditional yearly releases and almost two years during shorter
quarterly releases in three Eclipse projects. We then analyze both time periods’ refactoring activities to understand how refactoring activities can be
impacted by shortening the release cycles. [Results] We observe reduced
refactoring activities in one project and a decrease in more complex refactoring operations after shortening the release cycles. We also find that weekly
efforts dedicated to refactoring activities was lower across all projects after
shortening the release cycles. [Conclusion] Shorter releases may impact
software development tasks such as refactoring in unintended ways. Not applying specific types of refactoring may also affect the software’s quality inthe long term. Using this case study and past work on shorter releases, potential short release adopters can now better plan their transition to shorterreleases knowing which areas of development may be affected.
Email address: oliviern@posl.ait.kyushu-u.ac.jp, {kashiwa, kamei,ubayashi}@ait.kyushu-u.ac.jp (Olivier Nourry, Yutaro Kashiwa, Yasutaka Kamei,
Naoyasu Ubayashi)

Preprint submitted to Information and Software Technology

July 26, 2021

Keywords: Release Policy Change, Refactoring, Release Engineering
1. Introduction
Refactoring in software systems has been investigated extensively throughout the years. Multiple aspects such as the impacts on code quality [1] andtesting activities [2] have been investigated in order to determine how important it is to refactor the code in a software development project. Past
studies [3, 4, 5] have concluded that refactoring is beneficial to a softwaredevelopment project and that it is worth dedicating some development time
to refactor the code.
Modern-day software companies, however, have started changing the way
they develop software systems. Over the past few years, large companies such
as Google, Facebook, Firefox, and Netflix have all decided to adopt a shortrelease cycle model for various reasons such as keeping up with customer
expectations or keeping up with competitors [6].
This short release cycle model has been referred to as rapid releases (RR),or short releases (SR), and the old yearly release model as traditional releases(TR). Several studies have been conducted in order to better understand theimpacts of shorter releases by comparing various metrics and activities before and after shortening the release cycles. For example, a previous study [2]showed that shorter releases’ builds contain more code commits than traditional releases’ builds. Another study [1] also found that short release cycles
had a significant impact on the number of files being affected per commit.
The code quality, number of bugs, integration, and testing in shorter releases
have also all been subject to investigation in past studies [1, 2, 7] but to thebest of our knowledge, the impacts of shortening the release cycles on refactoring activities have not yet been investigated. In this study, we therefore
aim to further contribute to the empirical knowledge related to the impactsof shortening the release cycles on software development activities.
In this case study, we investigate if shortening the release cycles had
any impacts on refactoring activities in three software development projects.
Due to the possible negative impacts of not refactoring, it is important forpotential short release adopters to be aware of the possible downsides related
to refactoring activities that shorter releases can bring to a project. We
examine four aspects of refactoring activities: the amount of refactoring done,
the efforts devoted to refactoring, the type of refactoring applied in the code,and the human resources dedicated to refactoring. These aspects allow us to2draw an overall picture of refactoring during short releases and traditional
releases in order to determine if the refactoring activities are impacted after
adopting a shorter release cycle.
Paper Organization: Section 2 introduces previous studies involving
refactoring and short releases. Section 3 describes our dataset selection, its
extraction and the filtering process, and Section 4 presents our results. In
Section 5, we summarize our findings, discuss their implications and addresssome of the threats to the validity of this study. We conclude in Section 6
where we briefly summarize our findings and make our final comments on
refactoring and short releases.
2. Background2.1. Refactoring
Refactoring has been widely defined as changes that improve code quality but preserve the software system’s external behavior [8]. Thanks to theimprovement of refactoring detection algorithms [9, 10, 11, 12, 13] that automatically identify refactoring activities from source code histories, many
researchers have conducted empirical studies [3, 14, 15, 16, 17, 18, 19] on
software evolution and software refactoring.
Refactoring and Software Quality. In a previous study [20], Lacerda etal. performed a systematic literature review and concluded that refactoring
has a strong relationship with code quality attributes such as understandability, reusability, and maintainability. During their literature review, theyalso identified that code quality is one of the relationship linking code smells
and refactoring. Code quality is therefore a good representation of how wellrefactoring can remove code smells.
While leading a case study [21] on the impacts of refactoring on software
quality and productivity in an agile environment, Moser et al. also foundempirical evidence that refactoring improves code quality factors, reducescode complexity and coupling, and improves cohesion. Their results also
showed empirical evidence that refactoring increases development productivity. In another study [22], Moser et al. also proposed a new methodology toassess if refactoring improves quality. Using their new proposed approach,they conducted a case study in a near industrial environment and found thatrefactoring has a positive effect on reusability and promotes ad-hoc reuse ofobject-oriented classes.
Kim et al. [4] found that there is an increase in the number of bug fixes
3

after API-level refactorings while investigating the role of refactorings in three
evolving open source projects. They also found that less time is required to
fix bugs after API-level refactorings are applied to the code.
While refactoring has many benefits, refactoring code changes carry thesame risks associated with any other codebase change such as the introduction of new defects as shown in a previous study [23] that found thatrefactoring can account for a significant part of API breaking changes.
Refactoring automation. To mitigate the risks of introducing new defects, it has been suggested that refactoring tools should be used wheneverpossible. Multiple automated tools have been developed over time to ease theprocess of refactoring for developers. These tools were designed to help developers in several ways ranging from automated refactoring suggestion [24],to automated refactoring validation that checks if a manual refactoring wasapplied correctly [25] and all the way to fully automated refactoring applications [26]. Popular IDEs such as IntelliJ IDEA and Eclipse have also added
refactoring support by adding their own built-in refactoring tools to theirsoftware. These tools generally cover a wide range of refactoring patterns
and can help developers save time when refactoring their code.
Xing and Stroulia [27] studied the proportion of code modifications forrefactoring changes and which types are the most frequent types of refactorings being applied in the Eclipse JDT project. They found that about
70% of structural changes may be due to refactorings code changes and that
60% of refactorings can be automatically applied by a refactoring tool if the
refactoring tool can gather the information relevant to the refactoring being
applied.
Although refactoring tools are becoming more commonly used, Silva et
al. [16] found that many developers still do not trust automated refactoring for complex cases or believe that automation does not need to be usedin trivial cases. A previous study led by Brett et al. [28] also found thatrefactoring tools themselves can contain multiple bugs. It is therefore very
important that refactoring is done properly in order to avoid introducing
more defects. Because each release is usually longer when using a traditional
release cycle model, the evolution of the software system is well controlled
due to developers planning ahead of time how much time is to be allocatedto each refactoring task. New release methodologies however reduce thatplanning time a lot by shortening release cycles [29]. This paper differs fromprevious refactoring and software evolution studies by taking into account
this new shorter releases approach to software development.
42.2. Short Releases
Many software development projects have been switching from a traditional release cycle to a shorter release cycle to deliver new features faster(a few weeks to a few months to ship a major release) rather than providingthem yearly. This satisfies users and also allows developers to receive quick
feedback from users [30]. For instance, the Firefox and Eclipse projects shortened the release intervals for major/main releases, respectively. Specifically,
the Firefox project started releasing a product in 6 weeks rather than yearly.
From Eclipse 4.9, the Eclipse project abandoned service releases (i.e., patch
releases) and replaced all the service releases with main releases.
Many studies have examined the impacts of shortening the release cycles on development activities (e.g., testing, bug-fixing, and integrating) tofind that shortening release intervals affects development activities [2, 7, 30,
31, 32, 33, 34, 35]. For bug-fixing activities, the average bug-fixing time is
reduced by adopting shorter releases [31]. As for testing activities, shorterreleases narrow the scope of the tests and increase the number of test executions [2].
Research motivation. In the field of the consumer market, Tonietto etal. [36] reported about code behaviour changes when developers are undertime pressure. When time is limited, people tend to perform fewer tasks andare less likely to complete time-consuming tasks because of the perception of
having less time than they actually have when compared to an unboundedtime [36]. In this way, while having limited time, developers change thescope of their tasks and the amount of effort put on specific tasks in orderto optimize their activities.
Refactoring, for instance, is not trivial work when it must be completed
in a short amount of time. In fact, past studies have found that certain
types of refactoring can be quite challenging even without time limitations.
For example, Garrido et al. [37] have previously studied the challenges ofrefactoring a C program file that uses preprocessor directives. Vassallo et
al. [38] also investigated the advantages and barriers of continuous refactoringin CI and found out that the main barrier to continuous refactoring was thelack of time. The same study also states that this is even more true in agile
projects because there is more emphasis on new changes. We use this findingby Vassallo et al. to motivate our study of the impacts of shorter releases on
refactoring practices.
To the best of our knowledge, our work is the first to examine the impact
5

II.Data Extraction

I. Project
Selection

Analyze refactoringfor each revisionCoderepositories
Extract informationof each revision+- instance ID- commit ID- refactoring type- refactored lines

III. Filtering- instance ID- commit ID- refactoring type- refactored lines- commit date- committer name- changed lines- parent commit IDs

Removemerge commits

Refactoring data

- commit ID- commit date- committer name- changed lines- parent commit IDs

Refactoring andchange data

Change data

Filter and
Separate- instance ID- commit ID- refactoring type- refactored lines- commit date- committer name- changed lines

TR
RR

Release note

Datasets

Figure 1: An overview of the data extraction procedureof switching from traditional yearly releases to shorter releases on refactoring
activities. Since refactoring plays a crucial role in software development, thiscase study provides further insight for projects planning to transition toshorter releases. In this study, we focus on the process change and compare
the amounts, efforts, contents, and human resources of refactorings between
traditional releases and shorter releases in order to investigate if shortening
the release cycles had any impacts on refactoring activities.
3. Study Design
This section explains the design of our case study in order to address
our research questions introduced in Section 3.1. To measure the impactsof switching to a shorter release cycle on refactoring activities, we compare
two datasets derived from traditional yearly releases and shorter quarterly
releases. Figure 1 depicts an overview of the data extraction process. We first
apply a refactoring analysis tool to each revision in code repositories. Next,
we measure the relevant metrics and merge them with the outputs of the
refactoring tool. Finally, the merged outputs are divided into two datasets
based on whether the commit date is before or after the cycle change date.
3.1. Research Questions
In this paper, we structure our study around the following four researchquestions.
RQ1 : Do developers still refactor as much? (Amount)
RQ2 : Are refactoring efforts affected by shorter releases? (Effort)
RQ3 : Do shorter releases impact what type of refactoring is applied to thecode? (Contents)

6

RQ4 : Does shortening the release cycles affect the human resources dedicated to refactoring activities and their refactoring workload? (Humanresource)
In previous studies [1, 2, 7] that investigate the impacts of shorter releases
on various development tasks, three main aspects have been used to measure
the impacts: the frequency of the task (RQ1), the characteristics of the task
(RQ3) and the human resources dedicated to the task (RQ4). In addition
to investigating these three main aspects, in RQ2, we diverge from the main
aspects used by previous studies in order to investigate the impacts of shorterreleases on refactoring efforts by measuring the amplitude of the refactoringchanges. In this paper, we design our research questions in a way that will
help paint a broad picture of what impacts, if any, switching to shorterreleases has on refactoring activities. We also try to align our analysis withprevious studies so that we can compare our results afterwards. We compare
the results of our empirical study with previous studies in Table 11 in thediscussion section to get an overall view of the possible impacts of shortening
the releases on development activities.
3.2. Project Selection
To address our research questions, we perform a case study analyzing
refactoring in traditional yearly releases and shorter quarterly releases. The
projects used in this study were picked because they satisfied the following
three criteria:
Criterion1 : Experience in shortening release cycles. We need tocompare the amount, effort, contents and human-resource of refactoring
between traditional and shorter releases. Therefore, our studied systems
must have development experience following both a traditional release
cycle and a shorter release cycle.
Criterion2 : Java System. Since we need to analyze refactoring activities,
we use a state of the art refactoring analysis tool that can detect various
refactoring types (See Section 3.3). However, this tool has the limitationof only being able to analyze Java code. Thus, our studied systems must
be written in Java.
Criterion3 : Large-size repositories In order to acquire reliable results,
we need many instances of refactoring. Our studied systems must have at
7

least 1,000 commits during the traditional releases period and a significant
number of commits during the shorter releases period in their masterbranch.
Because short release cycles are not a widely used approach yet in opensource software development, the number of open source projects that havedeveloped for a significant period of time following both yearly traditional
releases and shorter releases is limited. Considering this limiting factor andour three selection criteria, we decided to pick three Java projects from the
Eclipse Foundation which are known to be participating in the simultaneous release initiative. These projects are the jdt.core, platform.ui, andplatform.swt projects, which are mature (over 10 years of development)
and still active.
The Eclipse Foundation, which owns these projects, follows a simultaneous release scheme to deliver products in sync with all the projects in Eclipse.
Figure 2 depicts the transition of Eclipse’s release cycle. So far, the release
cycle has undergone changes twice before the adoption of their current releasemodel. Until Eclipse version 4.5, Eclipse delivered yearly new features andbug-fix in the main release in June and then provided patch releases twice in
September and February in order to fix bugs found in the main release. In
Eclipse 4.6, they added a new patch release (named service release) in December and moved the February patch release to March making all releases
released quarterly. Furthermore, from Eclipse 4.8, a shorter release cycle wasadopted which abolished patch releases and made every quarterly release into
a main release.
3.3. Data Extraction
In order to mine the refactoring patterns, we used the state-of-the-art
refactoring analysis tool developed by Tsantalis et al. called RefactoringMiner [10]. RefactoringMiner can detect up to 40 different types of refactoring patterns (as of February 1, 2020)1 . We used this tool due to its stateof the art performance (high precision and high recall [10]) as well as thelarge diversity of patterns that the tool can detect. The tool can detect simple patterns such as renaming and moving pieces of code or pushing down
and pulling up methods. It is also able to detect more time consuming or
1

We used build “8ccec9050ad6dcc4d575cce3713f5283d5988c5d” since RefactoringMineris still being developed.

8main releaseservice release

added one
service release

adopted quarterly
main releases
our dataset

4.4

4.5

4.6

4.7

4.8

4.10 4.12
4.9 4.11 4.13

Figure 2: Range of our dataset

complex patterns such as extracting a superclass or merge and split typesof refactorings such as split parameter or merge attribute. After cloning therepositories of the studied projects, we ran the RefactoringMiner tool on every commit. The output of each commit contains the information of multiple
refactoring instances, including the refactoring types and the refactored lines
in the source code.
Additionally, we extracted the basic information of each commit such ascommit date, parent commit ids, changed lines, and committer name. We
used the commit date and the parent commit ids for our filtering process
and to create our datasets. Then, we utilized the changed lines and thecommitter name for RQ2 and RQ4, respectively. Finally, we merged thebasic information with the output of the RefactoringMiner, based on thecommit id.
3.4. Filtering
Merge commits. Before making datasets, we need to remove commits
with duplicated refactoring instances. Code repositories contain merge commits, which will cause refactoring instances to be redundantly detected eventhough the same snippet has previously been marked as refactored in the parent commits. Therefore, we eliminate commits with multiple parent commits
from our dataset.
Commits during early development periods. Since the goal of thisstudy is to find the impacts of shortening the release cycle, in order to havea fair comparison between the traditional releases data and short releases
data, we selected a subset of the traditional releases data that was as close
as possible to the switch to shorter releases. Using this subset, we can ensurethat we are not comparing early development commits and contributions thatare aimed at developing the core of the application with recent contributions

9

Table 1: Summary of the dataset
Refactoring Refactoring % Refactoring
Project
Release Commits
Instances
Commits
Commits
TR
1,306
3,435
354
27.10%
jdt.core
SR
618
816
146
23.60%
TR
1,877
3,466
168
9.00%
platform.swt
SR
1,243
1,409
115
9.30%
TR
1,475
1,914
270
18.30%
platform.ui
SR
1,340
3,182
224
16.70%

aimed at software maintenance and update [1]. Thus, we do not use the dataprior to Eclipse 4.6 (June 22nd, 2016). We would like to note that the yearly
release cycle (main release) was broken on Eclipse 4.8 (June 27th, 2018) wherequarterly releases were adopted (See Figure 2). All commits authored prior toJune 27th, 2018 (Eclipse 4.8) were categorized as traditional release commits
and all commits done on June 27th, 2018, and after were categorized as shortrelease commits. Although the release periods have a different length, ourdata is normalized so that our results are not affected by it. In RQ1 and
RQ2, we compare weekly median data to normalize between the two release
periods, in RQ3 we normalize using a ranking-based approach, and finally in
RQ4 we use percentages and medians to normalize our data.
The main idea behind this choice is that the project would have thesame maturity and frequency of code contribution from the contributors if
no release cycle change happened. Taking a subset that was too early in the
project’s life would result in comparing traditional releases commits aimed
at developing the core of the application against shorter releases commits
which are more oriented towards maintenance and feature additions since
the project is much older at that point.
Dataset Summary: The final dataset, as shown in Table 1 is madeof traditional releases data and short releases data extending over a similar
length of time. Henceforth, traditional releases refer to the time period between July 22nd, 2016, and June 27th, 2018, and short releases refer to thetime period after June 27th, 2018.
4. Case Study Results
4.1. (RQ1) Do Developers Still Refactor as Much?
Motivation. Due to the shorter releases, developers do not have as muchtime to dedicate to tasks not related to feature development and maintenance10

such as testing [39, 40, 41, 42] or refactoring. For example, Vassallo et al. [38]found that lack of time is the main barrier to refactoring while investigating
continuous refactoring in CI. We therefore investigate if adopting shorterrelease cycles affects the overall project’s refactoring activity by comparing
how much refactoring was being done during both release periods.
Approach. Refactoring is not usually done as an everyday task but rather
when it is needed to implement new features or when the code becomes
unreadable [18]. For these reasons, we decided to aggregate our data to
study refactoring activity on a weekly basis.
For RQ1, we focus our results around the median weekly values to compare the refactoring done during the traditional releases period and theshorter releases period. To extract the overall refactoring activity in ourchosen projects, we use the following two metrics:
Refactoring Instances: We use refactoring instances as our main metric to
see exactly how much refactoring was applied to the code during eachperiod. The number of instances directly reflects how many times thecode was changed for refactoring purposes.
Refactoring Commits: We use refactoring commits as a secondary metricto study refactoring activities. The number of refactoring commits gives
us an insight as to how often the code is refactored.
We start our investigation by counting the number of refactoring instances
per week in each project and extracting the weekly median number of refactoring instances. We then compare the median values between traditional
releases and short releases to see if shorter releases impacted how much refactoring is applied to the code. We then proceed to look at refactoring commits to see if there is more weekly refactoring activities after shortening thereleases. We extract the weekly median number of commits during the traditional releases period and the shorter releases period and then compare theweekly median percentage of refactoring commits during both periods.
We finally use a Mann-Whitney test for statistical tests and Cliff’s delta
for effect size, which are non-parametric, to verify if there is a statisticallysignificant difference in the number of refactoring instances and refactoring
commits between the traditional releases and the shorter releases. We selectthese non-parametric tests because refactoring activities sometimes show outliers and are not assured to follow a normal distribution.
11

Findings 1. We observe a decreased amount of refactoring with alower number of refactoring instances applied in the JDT.CORE
project after shortening the release cycle
As shown in Table 2, we find that the weekly median number of refactoring instances is very similar during both the traditional releases and theshortened releases for the two platform projects. The jdt.core project’s
median refactoring instances per week, however, is twice as much during the
traditional releases period than the short releases period. Although both theplatform projects show a small increase in the median number of weekly refactoring instances, a Mann-Whitney test on the weekly number of instances foreach project confirms that out of the three projects, the jdt.core projectis the only one showing a statistically significant difference in the number of
refactoring instances per week (p-value < 0.01). It is also the only projectwith a non-negligible effect size for the number of refactoring instances per
week. The two platform projects were found to have no statistically significant difference in the weekly number of instances.
Findings 2. We find a decrease in refactoring activities with a lower
number of weekly refactoring commits in the JDT.CORE project
after adopting shorter releases.
The refactoring commits data shows less of a difference than the refactoring instances data when looking at the weekly median number of refactoring
commits. Out of the three projects, we find that only the jdt.core project
has a difference in the weekly median number of refactoring commits. We
however find that this measure does not tell the whole story; when looking
at the percentage of weekly commits that contain refactoring changes in Table 2, we find that there was an overall decrease in median percentage of
refactoring commits of almost 6% in the jdt.core project after adopting
shorter releases. Although the decrease is much smaller than the one foundin the jdt.core project, both platform projects also show a decrease in the
median percentage of refactoring commits.
Using a Mann-Whitney test and Cliff’s delta, we find that our results
remain consistent with our previous observation; only the jdt.core project
has a statistically significant difference in weekly refactoring commits and a
non-negligible effect size. The platform.swt and platform.ui projects do
not show any statistically significant difference in the number of refactoring
commits between the traditional releases and the shorter releases.

12

Table 2: Weekly refactoring activities of each project during TR and SR
Project
jdt.core
platform.swt
platform.ui

Release
TR
SR
TR
SR
TR
SR

Refactoring instances
Refactoring commits
Median #SS
ESMedian # Median %
SS
10.5
0.002
3.0
25.00%
0.002
d=0.285.0
p<0.01
2.0
19.09%
p<0.01
2.0
0.523
1.0
7.14%
0.867
d=-0.06
2.5
p>0.01
1.0
7.02%
p>0.01
5.0
0.48822.0
16.67%
0.469
d=-0.06
6.0
p>0.01
2.0
14.29%
p>0.01*SS: statistical significance ES: effect sizeES
d=0.28d=-0.01d=-0.06

From our investigation, we conclude that adopting shorter releases may
have affected the amount of refactoring done and the refactoring activities of the jdt.core project. We found no clear positive or negative
impacts of shortening the release cycle in the platform projects.
4.2. (RQ2) Are Refactoring Efforts Affected by Shorter Releases?
Motivation. In some cases, we find commits with multiple refactoring instances affecting only a few lines of code. In other cases, we find single
commits with only one instance that required changing hundreds of lines ofcode. Because of these different cases, we cannot determine how much effort
is put into refactoring using only the number of refactoring commits andinstances.
Since past studies have found that refactoring is more often done manuallythan using refactoring tools [18, 19, 16], we need to take into account thatsome refactorings require significantly more time and code changes whileothers can be done very quickly and easily. In this RQ, we aim to find out
how much refactoring efforts developers dedicate to refactoring by comparing
the refactoring code churn of refactoring commits during both the traditional
releases and the shorter releases.
Approach. Because code churn has been used several times as a proxymeasure of effort or as part of a subset of multiple metrics used to measureeffort in past studies [43, 44, 45], we decided to use the refactoring code
churn as our metric to estimate how much effort is put by developers into
refactoring during the yearly releases and the quarterly releases.
The first thing we did was to aggregate all files that had been refactored
at least once. We then looked at the Git history to extract every code change

13and code churn from this subset of files. For every refactoring instance inour dataset, we used the output of the RefactoringMiner to find the specificsource code lines involved with refactoring.
In some cases such as refactorings that move source code folders, wefound that the RefactoringMiner associated some code changes with a specific refactoring instance when the changes were actually caused by a different
refactoring instance. This meant that some lines of code involved with refactoring were counted twice when summing up code churn for all the refactoring
instances. To avoid counting the lines of code multiple times for the samecode changes, we compared the code change history with the lines flagged by
the RefactoringMiner to make sure each line was only counted once.
To calculate the refactoring code churn for a specific file, we therefore
compared the code change history of the file with the source code lines thatwere flagged by the RefactoringMiner as being involved with refactoring. We
discarded all code changes that were not directly caused by refactoring inorder to only keep the history of refactoring code changes for that file. We
repeated this process for every file in our subset to get our final dataset for
RQ2.
After extracting every file’s individual refactoring code churn, we aggregated our data by project in order to measure the change in refactoring effort
on a project basis. We then calculated the total refactoring churn involved
during the traditional releases and the shortened releases for each project.
Similarly to our approach in RQ1, we then split our data on a weekly basis.
We use the median of weekly refactoring code churn to get an overview ofthe difference in refactoring efforts over time for each project. We then lookat the weekly refactoring code churn distribution to see exactly how refactoring efforts changed between the yearly releases and the quarterly releases
for each project.
Although multiple studies [16, 18, 19] find that most refactoring is being
done manually, some of these studies also find that a small subset of patterns
associated with “Renaming” types of refactoring are more often applied using
tools than manually. Due to the high likelihood of tool usage, we excluded“Renaming” types of refactoring from RQ2’s analysis.
Findings 3. The median weekly refactoring churn of the JDT.
CORE project shows a statistically significant difference between
the traditional releases and the shorter releases. We proceed to look at
the weekly data starting with the weekly median refactoring churn as shownin Table 3. Surprisingly, we find that the highest gap in median weekly refac14

Table 3: Refactoring Churn in LoC during TR and SR
Total Refactoring Weekly Refactoring
Project
Release
Churn
Churn MedianTR
18,49846.5
jdt.coreSR
10,97320.5
TR35,131
11
platform.swt
SR
4,993
5.5
TR
6,207
26
platform.ui
SR45,657
20.5

toring, with more than double the weekly refactoring churn during traditional
releases, is from the jdt.core project which had the lowest difference in total refactoring churn. The platform.swt shows a small decrease in weekly
refactoring churn median and the platform.ui project shows no impact at
all.
A Mann-Whitney test on the weekly refactoring code churn confirms our
findings by revealing no statistically significant difference in the two platformprojects. The jdt.core project, unlike the platform projects, was found tohave a statistically significant difference in weekly refactoring churn.
From our investigation, we conclude that adopting shorter releases may
have affected the weekly amount of efforts dedicated to refactoring in
the JDT.CORE project. While we also notice an overall downward
trend of refactoring efforts during the shorter releases in the platformprojects, the observed decrease in refactoring efforts was not as significant in the platform projects as it was in the jdt.core project.

4.3. (RQ3) Do Shorter Releases Impact What Type of Refactoring is Applied
to the Code?
Motivation. Past studies have suggested that testing activities are affectedby time constraints [39, 40, 41, 42]. Because developers must follow stricter
time constraints during the quarterly releases, we investigate if the type of
refactorings applied is also affected by the new time constraints. For example, it is possible that only small and simple refactorings such as renaming
would be applied; leaving large refactorings that change the structure of thecode on the side due to the lack of time. In this section, we look at the15

type of refactorings being done during the traditional yearly releases and thequarterly releases to see if the shorter releases allow developers to still takecare of all types of refactorings after shortening the release cycle.
In RQ3, we conduct two analyses (Section 4.3.1 and 4.3.2) to evaluate
which refactoring types are the most common. We first start with a finegrain analysis between yearly releases and quarterly releases based on the
refactoring patterns individually. We then conduct a coarse-grain analysis bycategorizing the refactoring patterns into distinct categories and analyzing
the variation of these categories between the traditional releases and theshorter releases.
4.3.1. Refactoring pattern analysis (Fine-granularity)
Approach. Previous work [46] found that the number of commits is an
accurate measure to estimate workload. Therefore, to get a general idea of
which patterns were the most often applied in each project, we first examined
how many commits for each refactoring pattern there are in all three projects
during both time periods. To better visualize the variation in popularity foreach refactoring pattern, we ranked every pattern during both time periods
based on the number of commits that implemented them. This allows us tonormalize the number of commits between the traditional releases and theshorter releases. Because we are not interested in the amount of refactoring
done but rather the type of refactoring being done in RQ3, using the ranks
to compare between both time periods allows us to abstract how much refactoring is being done and focus exclusively on which refactoring types are themost common during each release period. Lower ranks represent patterns
that are frequently applied to the code and high ranks represent patterns
that are not often applied to the code.
To see if the same patterns tend to have a high frequency of application
during both time periods or if the new release model affected which refactoring patterns are being applied by the developers, we then selected the top 10
refactoring patterns with the most commits in traditional releases and the
top 10 refactoring patterns with the most commits in the shorter releases
respectively. To see which patterns were the most affected by the adoptionof shorter releases, we also calculated the variation in ranking (ranking ∆)of each refactoring pattern from our top 10 list of patterns with the highest
number of commits.
Pattern Ranking: Each refactoring pattern is assigned a rank for the yearly16

platform.swt project

0

25

50

75

Extract Method
Rename Method
Rename Variable
Rename Parameter
Extract Variable
Change Variable Type
Change Parameter Type
Change Return Type
Rename Attribute
Change Attribute Type
Extract And Move Method
Inline Variable
Inline Method
Parameterize Variable
Move Attribute
Replace Variable With Attribute
Rename Class
Push Down Method
Pull Up AttributeMove Source Folder
Move Method
Push Down AttributeMove Class
Extract Subclass
Extract AttributeMove And Rename Class
Merge Variable
Extract Superclass
Extract Interface
Extract Class

100

SR
TR

0

Number of commits

platform.ui project

Refactoring pattern

SR
TR
Refactoring pattern

Refactoring patternjdt.core project
Extract Method
Rename Method
Change Variable Type
Extract Variable
Rename Variable
Change Attribute Type
Change Parameter Type
Change Return Type
Rename Attribute
Rename Parameter
Extract And Move Method
Inline Method
Move Method
Parameterize Variable
Move Attribute
Pull Up Method
Inline Variable
Extract Class
Pull Up Attribute
Rename Class
Move Class
Extract SuperclassPush Down Method
Push Down Attribute
Replace Variable With Attribute
Extract Subclass
Extract Attribute
Extract Interface
Move And Rename Class
Move Source Folder
Move And Rename Attribute
Merge Parameter

10

20

30

Number of commits

Change Variable Type
Change Attribute Type
Rename Variable
Extract Variable
Extract Method
Change Parameter Type
Change Return Type
Rename Method
Rename Attribute
Rename Parameter
Move Source Folder
Move Class
Move Attribute
Inline Method
Rename Class
Move Method
Move And Rename Class
Replace Variable With Attribute
Inline Variable
Extract Attribute
Parameterize Variable
Extract And Move Method
Extract Class
Pull Up Method
Extract Superclass
Split Attribute
Push Down Method
Move And Rename Attribute
Extract Interface

SR
TR

0

20

40

60

80

Number of commits

Figure 3: Number of refactoring commits per pattern during both release periods

releases period and the quarterly releases period for each project based on
how many commits implement them. Common refactoring patterns are
assigned low ranking values and uncommon patterns are assigned highranking values.
Ranking ∆: We calculate the ranking ∆ for each refactoring pattern bycalculating the variation in ranking between yearly releases and quarterly releases. A positive ∆ indicates that a pattern’s rank increased andtherefore became less common during the shorter releases and a negativedelta indicates that a refactoring pattern’s rank decreased and therefore
became more common during the shorter releases.
Findings 4. We find that most refactoring patterns that were common during traditional releases were also common after shortening
the release cycles. To get a general view of which patterns were popularduring both time periods, we first extracted how many commits implement
each refactoring pattern for each project during the traditional releases andthe shorter releases. From Figure 3, we notice that the distribution of refactoring patterns that were popular in each project is similar during both time
periods. Looking at the refactoring patterns with the most commits for eachproject, we also find that the three projects share a lot of similarities withpatterns such as Extract Method being applied frequently during both thetraditional releases and the shortened releases in all three projects.
From the ranking assignment process previously described, we calculate
17the rank of each refactoring pattern and find that the most common refactoring patterns remain very similar even after shortening the releases. As shownin Table 4, 9 of the top 10 refactoring patterns are common in both the traditional releases and the shorter releases in the jdt.core project. From
Table 5, we find that 8 out of 10 patterns are common in both time periodsin the platform.swt project and from Table 6, we find that all 10 patterns
with the highest ranks during the traditional releases remain the same during
the shorter releases in the platform.ui project. From these results, we findthat most patterns that were popular during traditional releases remained
popular after adopting shorter releases.
Looking at the ranking ∆ of each pattern in Table 4, we notice that
the Change refactoring patterns have significantly gone down in popularity
during the shorter releases in the jdt.core project. We also observe that
the Rename refactoring patterns have been oppositely affected and becamemuch more common during the shorter release cycle. From Table 5, we findsimilar results as in the jdt.core project with Change refactoring patterns
decreasing in popularity and Rename refactoring patterns increasing in popularity in the platform.swt project. These results align with a previous
study’s findings [47] where they observed that Eclipse puts a heavy emphasis on backward compatibility. It is therefore not surprising that developers
would be reluctant to perform refactorings that can affect the API such aschanging method signatures.
While these results seem to confirm our initial assumption that simpler
refactorings such as renaming are increasing in popularity due to the shorterreleases, from Table 6 we find that the patterns in the platform.ui project
were affected differently than in the jdt.core project: the Extract refactorings had a small decrease in popularity, the Rename refactorings saw nochange at all and the Change refactorings saw a small increase in popularity. The platform.ui results therefore suggest that developers have started
modifying their refactoring behaviors after adopting shorter releases by taking advantage of the more frequent major releases and applying refactorings
that modify the API. Since we cannot draw a clear conclusion from our finegrain analysis, we proceed with our coarse-grain analysis.

18

Table 4: Top 10 most common refactoring patterns in the JDT.CORE project during eachrelease period
Pattern
Ranking in TR Ranking in SR Ranking ∆Change Return Type (M)615+9Change Variable Type (L)26+4Change Attribute Type (M) 57+2Extract Method (M)11+0Change Parameter Type (M) 88+0Rename Parameter (H)99+0Extract Variable (L)33+0Rename Method (H)42-2
Rename Variable (L)74-3
Rename Attribute (H)105-5Parameterize Variable (H)1510-5(L): Low level refactoring, (M): Medium level refactoring, (H): High level refactoring

4.3.2. Categorization analysis (Coarse granularity)
Approach. To investigate the impacts of shorter releases on the different
types of refactorings being applied rather than the impacts on each refactoring pattern’s individual frequency, we proceeded to aggregate the refactoring
patterns by categorizing them. We were then able to abstract our analysis and use our refactoring categories to continue our comparison between
traditional releases and shorter releases.
We decided to use Murphy et al.’s categorization model [19] because it isless likely to be affected by subjectivity than estimating the complexity of a
specific code change.
Following the categorization criteria given in the original paper [19], thepatterns detected by the RefactoringMiner were categorized into three different categories: high-level refactorings which only affect the signatures ofclasses methods and field, medium-level refactorings which change the signature of classes, methods, and field while also significantly changing blocks
of code and finally low-level refactorings which only affect blocks of code. In
short, medium-level refactorings require to change the code in several places,
unlike high-level and low-level refactorings that only target a specific section of the code and therefore are less likely to affect the structure of thecodebase. Based on these characteristics, we hypothesize that the frequencyof medium-level refactorings should be reduced during the shorter releases
due to the combination of the higher complexity involved with changing thestructure of the code and shorter release time.
19

Table 5: Top 10 most common refactoring patterns in the PLATFORM.SWT project
during each release period
Pattern
Ranking in TR Ranking in SR Ranking ∆Inline Variable (L)917+8Change Variable Type (L)48+4Change Return Type (M)711+4Extract Method (M)12+1
Rename Variable (L)23+1Extract Variable (L)55+0Change Attribute Type (M)1010+0Rename Method (H)3
1-2
Rename Parameter (H)64-2Change Parameter Type (M)86-2Extract And Move Method (M) 11
9-2
Rename Attribute (H)127-5(L): Low level refactoring, (M): Medium level refactoring, (H): High level refactoring
Table 6: Top 10 most common refactoring patterns in the PLATFORM.UI project during
each release period
Pattern
Ranking in TR Ranking in SR Ranking ∆Extract Variable (L)26+4Extract Method (M)57+2Rename Variable (L)3
4+1
Rename Method (H)88+0Rename Attribute (H)99+0Rename Parameter (H)1010+0Change Variable Type (L)11+0Change Attribute Type (M) 42-2Change Return Type (M)75-2Change Parameter Type (M) 63-3(L): Low level refactoring, (M): Medium level refactoring, (H): High level refactoring

To categorize the refactoring patterns that were not previously categorized in Murphy et al.’s previous study [19], two authors separately classified
each pattern detected by the RefactoringMiner. The two authors then combined the result of their individual classification to get the final classificationfor each pattern. For the cases where there were conflicts, the two authors
discussed and agreed on the best categorization based on the criteria definedin Murphy et al.’s previous study [19]. After categorizing every refactoring
pattern, we started to investigate if the type of refactorings applied changed
between the traditional releases and the shorter releases.
We started by investigating which refactoring category (high-level, mediumlevel, low-level) was the most impacted by the change in release cycle. We
20

first looked at how many patterns in each refactoring category became morecommon after shortening the release cycles (ranking ∆ < 0), how many patterns became less common (ranking ∆ > 0) and how many patterns were notaffected by the change (ranking ∆ = 0). We continued our investigation bylooking at the overall variation in ranking of each refactoring category. To
do so, we aggregated every refactoring pattern according to the refactoring
category it belonged to. We then summed up the ranking ∆ to see which
type of refactoring (high-level, medium-level, low-level) showed the largest
increase or decrease in popularity during both time periods.
Refactoring category ∆: To determine if a refactoring category became moreor less common, we sum the ranking ∆ for each pattern belonging to thatcategory.
Findings 5. We find that medium-level refactoring patterns becameless popular in the JDT.CORE project after shortening the release
cycles.
From traditional yearly releases to quarterly releases, the jdt.core project
has one less medium-level refactoring and one more high-level refactoring in
its top 10 as shown in Table 4. From Table 5, we find that the platform.swt project has one more low-level refactoring and one less high-levelrefactoring in its top 10 and the platform.ui project shows no difference
between both periods as shown in Table 6. If we look at the top end of
the rankings, however, we notice that in both the jdt.core project and theplatform.swt project, medium-level refactorings are less common thanhigh-level and low-level refactorings during the shorter releases. This decrease in medium-level refactorings indicate that developers may not be as
inclined to apply more complex refactorings during shorter releases because
medium-level refactorings require to change the code in several places [19].
From the ranking ∆ shown in Table 7, we find that in both the jdt.core
project and the platform.swt project, high-level refactorings got significantly more common after adopting shorter releases. The main difference
between the two project, however, is that in the jdt.core project, mediumlevel refactorings became much less common whereas in the platform.swt
project, low-level refactorings became less common. Surprisingly, the platform.ui shows a different trend from the jdt.core project and the platform.swt by having an increased amount of medium-level refactoring during the shorter releases. In fact, the platform.ui project is also the projectwith the smallest degree of variation in terms of rankings after adopting
21

Table 7: Ranking and workload variation between TR and SR
Refactoring
jdt.core platform.swt platform.ui
type
Low-levelSum of ∆+1+13+5
Medium-level Sum of ∆
+11
+1
-5
High-level
Sum of ∆-12-9+0

shorter releases on top of being the only project that increased medium-levelrefactorings. From these results, it appears that only the jdt.core projectsatisfies our initial hypothesis that medium-level refactorings should see asignificant decrease during the shorter releases. Our results also suggest thatthe platform.ui is the only one out of the three project that was able tokeep up with its refactoring workload in all three categories of refactorings
even after shortening the release cycle.
We therefore confirm our hypothesis and find that developers do not
apply complex refactorings as much after shortening the releases in thejdt.core project. Although the platform.swt project also shows a decrease in medium-level refactoring, the decrease is not significant enoughto confirm our initial hypothesis. The platform.ui shows opposite resultswith medium-level refactoring becoming more common during the shorterreleases.
Findings 6. We find that patterns that are commonly applied using
refactoring tools have greatly increased in popularity during theshorter releases. If we take a look at ‘Renaming’ refactoring patterns which
are commonly implemented using refactoring tools [16, 18], we also find thatthese types of pattern became way more common during the shorter releases
in two of our three projects. Using the data from Tables 4, 5, and 6, we sum
the ∆ values of ‘Renaming’ refactoring patterns in each project and find aclear increase in popularity in the jdt.core project and the platform.swt
with a sum of ∆ of -10 and -8 respectively. The platform.ui is the only
project where we find a small decrease in popularity with a sum of ∆ of +1.
These results indicate that shorter releases may increase the use of refactoring
tools by developers.

22

We find that developers do not apply complex refactorings as muchduring the shorter releases in the jdt.core project. We also findthat patterns that are commonly implemented using refactoring tools
became more common in two of the three analyzed projects after shortening the releases.

4.4. (RQ4) Does Shortening the Releases Affect the Human Resources Dedicated to Refactoring Activities and their Refactoring Workload?
Motivation. In this RQ, we aim to determine how human resources wereaffected after switching to shorter release cycles. A previous study about the
impacts of shorter releases on testing found that switching to shorter releases
led to a higher workload for testers due to fewer testers being involved withtesting activities [2]. Since releases are shorter and development teams haveto keep up with the development of new features, we want to see if the numberof developers that refactor is affected when adopting shorter releases.
Approach. For the first part, we investigated if the number of developersthat refactored was impacted by adopting shorter releases. We first extract
the total number of developers, the number of developers that refactored
during the traditional releases and the number of developers that refactored
during the shorter releases. We then calculated the percentage of developersthat refactored during each time period and compared the results between
both periods. To see if shorter releases promote developer involvement withrefactoring activities, we then extracted how many developers refactored during both periods and how many developers started refactoring only after
shortening the release cycles.
For the second part, we investigated how the refactoring contribution of
the developers was affected by adopting shorter releases. Using refactoring
instances as our metric, we measured every developer’s overall contribution
to refactoring activities. We first extracted the median number of refactoring
instances per developer. Then, to understand how the refactoring workload isshared among developers that refactor, we looked at the median percentage of
refactoring instances contribution per developer during both release periods.
Findings 7. We find no clear impacts of adopting shorter releaseson the number of developers that refactor.

23

Table 8: Developer resources during TR and SR
Release# Dev # Refactoring Dev % Refactoring Dev
TR
43
20
46.51%
SR
35
14
40.00%
jdt.core
T R ∧ SR8
T R ∧ SR
6
TR
55
2952.73%
SR
45
18
40.00%
platform.swt
T R ∧ SR13
T R ∧ SR
5
TR
100
46
46.00%
SR
105
55
52.38%
platform.ui
T R ∧ SR16
T R ∧ SR
39
Project

From our initial data extraction, as shown in Table 8, we find that two outof the three projects have a higher number of refactoring developers as well
as a higher percentage of developers that refactor during traditional releases.
We also notice that for the same two projects, namely the jdt.core andplatform.swt projects, most of the developers that are refactoring during
the shorter releases were already refactoring during traditional releases. The
platform.ui project is the only project of the three that differs. In fact, notonly is there a higher percentage of developers that refactor during the shorterreleases but most of the developers refactoring during that time period did
not refactor during traditional releases.
Because the platform.ui project shows the exact opposite trend whencompared with the jdt.core project and the platform.swt project, wecannot conclude what impacts shortening the release cycles had on the number of developers that refactor. From our observations, a project can be positively affected by making more developers involved with refactoring as shownwith the platform.ui project, or negatively impacted by having less developers refactor the code as shown with the jdt.core and platform.swt
projects.
Findings 8. The refactoring workload is more evenly distributed
among developers that refactor during the shorter releases.
We investigate how the contribution of refactoring developers is affectedby the change in release cycle. Using the median number of refactoring
instances as our metric, we find that all three projects have less than half thenumber of refactoring instances per developer during the shorter releases as24

Table 9: Refactoring workload contribution of developers during TR and SR
Project
jdt.core
platform.swt
platform.ui

Release
TR
SR
TR
SR
TR
SR

Median #refactoring
instances
28.0
5.0
7.0
3.5
5.5
2.0

Median %refactoring
instances
0.82%
3.42%
0.20%
3.04%
0.29%
0.89%

shown in Table 9. In the case of the jdt.core project, we observe almost
six times less refactoring instances per developer. These results indicatethat there is a clear downward trend in the number of refactoring instances
applied to the code per developer and that most of the refactoring workload
may therefore be taken care of by ‘refactoring experts’.
We therefore analyze how the total refactoring workload is split amongdevelopers. To measure the refactoring contribution of each developer thatrefactors, we measure what percentage of the total number of refactoring instances each developer has contributed. We then use the median to determine
if the workload is more evenly shared among refactoring contributors or if
most of them only take on a minor portion of the total refactoring workload.
We find that in both the jdt.core and platform.swt projects, eachdeveloper take on a higher percentage of the total workload during the shorterreleases than during the traditional releases. While the platform.swt
project also has a higher percentage of refactoring instances contribution perdeveloper during the shorter releases, the increment is much smaller than theother two projects.
We find no negative impacts on the amount of human resources dedicated to refactoring after shortening the release cycle. We also findthat the refactoring workload is more evenly distributed among thedevelopers during shorter releases.

25

Figure 4: Evolution of refactoring activities over time

5. Discussion
5.1. Evolution of Refactoring Activities
Intuitively, it would make sense if refactoring activities diminish overtime as a project ages and becomes more mature. If that was the case, thenit would be less likely that the impacts observed in RQ1 and subsequent
RQs would be consequences of shortening the release cycle. To verify thishypothesis, we plotted the evolution of refactoring activities since the first
named version (3.2) in 2006. From Figure 4, we notice that the jdt.core
seems to already have been on a downward trend of refactoring when therelease cycle was shortened in 2018. The two platform projects, however,show a very stable amount of refactoring commits regardless of how many
non-refactoring commits are being contributed to the projects.
Another concern we face is that the results and the statistical significancewe observed in the jdt.core project in RQ1 may not be due to the event
we are studying: the adoption of shorter release cycles. Since our results
might be due to the way we separate our data, we decided to evaluate two
separate chunks of data from the same time period (i.e., early TR and lateTR, early SR and late SR). By comparing two subsets of data from the sametime period, we can determine if the statistically significant difference in thenumber of commits that we found in RQ1 could be related to the change
in release cycles or if this statistical significance is due to the seemingly
decreasing refactoring activity previously observed.
We therefore used a Mann-Whitney test on the number of refactoring
commits in the first half of the traditional releases period versus the second
half of the traditional releases period and did the same for the shorter releases
period for all 3 projects. Our results, as shown in Table 10, show that there
is no statistically significant difference in the number of refactoring commits

26

Table 10: p−values for the number of refactoring commits during early versus late TR
and early versus late SR

Project
jdt.core
platform.swt
platform.ui

early TR and late TR
statistical significance
0.853
0.236
0.0215early SR and late SR
statistical significance
0.565
0.407
0.0193during the traditional releases period and shorter releases period for boththe jdt.core project and the platform.swt project. This indicates to us
that the the statistically significant difference in the number of refactoring
commits found between the yearly releases period and the quarterly releases
period for the jdt.core project may have been caused by the change inrelease cycles.
5.2. Refactoring and Software Quality
In RQ3, we observed that certain complex refactoring operations becameless common after shortening the releases. Using a previous survey based
study led by Silva et al. [16], we investigate the reasons why developers apply
these patterns that became less popular. More specifically, we are interested
to know how they think the quality of the code is improved by applying thistype of refactoring patterns. In their survey, Silva et al. investigated therefactoring behavior of developers based on eight refactoring patterns. Twoof these patterns; ‘Extract Method’ and ‘Move Method’, are classified as
medium-level refactoring patterns per our classification in RQ3. We therefore
use these two refactoring patterns as our basis to investigate how developers
think they improve the quality of the code when they apply medium-levelrefactorings.
During the survey led by Silva et al., developers confessed that theyusually extract methods to improve the reusability and the readability of thecode. In the case of the ‘Move Method’ refactorings, the developers’ answers
point to a desire to increase cohesion and reusability. Because readability andreusability are not metrics that can easily be measured, we cannot empiricallyverify that the developers’ refactorings achieved the intended purpose.
A past study led by Hindle et al. [48] however defined and studied theconcept of software “naturalness”. This code “naturalness” studies how “natural” a software’s code feels similar to how a natural language can feel natural27

or unnatural. A previous study led by Baishakhi and Hellendoorn [49] foundthat buggy code tends to be more unnatural and becomes more natural as
bugs are fixed. We therefore conduct a small literature survey and find two
studies led by Arima et al. [50] and Bin et al. [51] that studied the relationshipbetween refactoring patterns and the naturalness of the code. Interestingly,
both of these studies have found that the ‘Extract Method’ refactoring improve the naturalness of the code. The ‘Move Method’ refactoring, however,
was found to improve the code naturalness in one study and decrease it in
the other. We therefore hypothesize that the readability and reusability may
indeed increase when these two refactoring patterns are applied.
To conclude, the code’s quality may decrease in the long term if somerefactoring patterns are ignored due to their time cost or complexity. Developers should therefore try not to modify their refactoring behaviors even if
the release cycles become shorter. On the contrary, they should make sure
all types of refactorings are applied to maintain the code’s quality in the longterm.
5.3. Summary and Implications
To get an overall view of the impacts of shorter releases, including our new
findings on refactoring activities, we aggregated the results of related work
studying the impacts of shortening release cycles on different developmenttasks as shown in Table 11. Although other studies use the Firefox project for
their datasets, we notice that the impacts observed by previous studies are
similar to our findings: although reducing the release cycle length has some
impacts, these impacts are manageable. For example, shorter releases had
an impact on how many platforms are tested in Firefox but each supported
platform is tested more thoroughly [2]. From a previous study investigating
short releases and bugs, one solution to keep up with the workload was tohave more developers work on each release [1]. Another study investigating
the impacts of short releases on the integration delay of fixed issues in theFirefox project [7] also found that developers adjusted to shorter releases byassigning an issue to a release rather than queuing issues in a backlog.
From this case study’s results we were able to see how shortening the releases may affect a project’s refactoring activities by looking at the jdt.core
project’s evolution. From RQ1 and RQ2, the decreased weekly number of
refactoring instances, the decreased weekly percentage of commits that refactor the code and the lower weekly median number of lines refactored indicatethat refactoring activities in the jdt.core project may have been negatively
28

Table 11: Impact of shorter releases on software development tasks
Refactoring
Our Paper
Short releases
may reduce howmuch refactoring is applied tothe code

Integration[7]
Issuesarequeued up as
backlog in traditional releases
versus a perrelease basis in
shorter releases.

RQ2:
Does
adopting shorterreleases affect theeffort dedicated tothe task
RQ3:
Does
adopting shorterreleases affect thecharacteristics ofthe task

Shorter releases
may
impactrefactoring
efforts

Not studied

In some cases,
refactorings
thataffectmany parts ofthe code arenotappliedasfrequently
during shorterreleases

RQ4: Does thechange to shorterreleases affect thehuman resources
dedicated to the
task

The
refactoring
workloadis split more
evenly amongdevelopers

Issues addressed
more
quickly
but take moretime to be integrated. The
total time fromthe issue report
date
to
its
integrationis
not significantly
different.
Not studied

RQ1:
Does
adopting shorterreleases affect theamount or thefrequency of the
task

Bugs
[1]
Harder bugs arepropagated tolater releases.
Fewer changes
per developers
but the changes
touch
more
files.
Not studied

Testing[2]
Fewer platforms
tested in total
but each supported
morethoroughly.

Negligible difference in thenumber of postrelease
bugs.
Bugs are fixed
more
quickly
but fewer bugs
are being fixed.

More tests executions per day
but the tests focus on a smaller
subset of the
corpus. Overall
fewer tests butlarger builds.

More developers
working on each
short release.

Fewertestersbuthigherworkload.

Not studied

affected by the shorter releases. From RQ3, we also found that specific typesof refactorings are becoming less common after changing the release cycle in
the jdt.core project. Past studies have shown that refactoring improves
software quality and maintainability [3, 18] which leads to fewer bugs in thelong term. In Section 5.2, we also found that the refactoring patterns thatwere less common during the shorter releases were associated with readability
and reusability; two aspects of software quality that can deteriorate in thelong term if not addressed. Short releases adopters should therefore take into
account the risks of shortening their release cycles on refactoring activities
29and assign their resources accordingly.
By looking at the platform.ui project, however, our results also show
how a project could benefit from the shorter releases while also keeping up
with its refactoring workload. From RQ1 and RQ2, we found that the shorterreleases did not impact the amount of refactoring being done in the platform.ui project. From RQ3, we even find that medium-level refactorings
increased after shortening the releases. These results indicate that the developers are adapting their refactoring behaviors with the new release cycles.
Since every release is a major release with the new release model, the developers now have the flexibility of refactoring their API during any releaseand our results show that they are actively taking advantage of that. From
RQ4, we also find that a higher percentage of developers refactor during theshorter releases in the platform.ui project. From this case study, shortreleases adopters can therefore also see how their project could benefit fromshortening their release cycles.
While it is not yet clear as to why some projects adapt better to shorterreleases than others, we hypothesize that one of the possible reason as to why
the jdt.core is having a harder time keeping up with its refactoring workload might be due to the nature of the code itself. Since the platform.ui
project is a UI project, the chances of having to refactor the API is most
likely lower than in the jdt.core project which is made of core application
code. As we have seen previously, the Eclipse community also tends to be
more reluctant to refactor their API in order to maintain a strong backward compatibility [47]. The same study also finds that the developers areaware of how many downstream users can get affected by their changes andare therefore sensitive about making breaking changes. Since the jdt.core
project makes up the core of the Eclipse application and can affect all otheraspects of the platform, we therefore assume that the developers might beless inclined to refactor their API as much as in other projects.
5.4. Threats to Validity
Internal threats: In RQ1 and RQ2, we aggregate our data on a weekly
basis and use the weekly median values for our analysis. Aggregating the dataon a weekly basis allows us to have enough data points to follow the evolutionof refactoring activities while also keeping the time frame short enough to
avoid having outliers significantly affect our results.
In RQ2, we removed all ‘Rename’ refactoring types because more than
50% of developers admit using refactoring tools for renaming in previous
30

studies investigating the usage of refactoring tools [16]. Still, several refactoring types (e.g., ‘Extract Local’) can be performed by tools, which might
affect the results of RQ2. As of today, there is still no tool that can allow us
to detect if a refactoring was performed manually or automatically. Filteringout patterns that are commonly implemented using refactoring tools basedon previous studies’ findings is therefore the best approach we currently haveto minimize refactoring tools’ impacts on our results.
In RQ3, we categorize each refactoring pattern based on a previous study
authored by Murphy et al. [19]. We know that this categorization is not
optimal because it does not allow us to accurately categorize refactoring
patterns based on the complexity of the task or the efforts required to apply
each refactoring pattern. However, it does allow us to get a hint of bothcomplexity and efforts by taking into account which sections of the codeneed to be changed to apply each refactoring pattern.
External threats: Because we only used open-source software projects
from the Eclipse foundation, the results shown in this paper may not be
representative of other software systems. We also found that the Eclipse
foundation gradually adopted shorter releases over a long period of time
unlike other known projects using short releases such as Firefox. The impactsof shorter releases on the projects investigated in this study may therefore
differ from other projects that chose to adopt short releases more abruptly.
Because we observed different impacts of shorter release cycles on all threeof our projects, it is difficult to generalize our results.
Constructive threats: Code churn has been used by multiple paststudies as a measure of effort [52, 53, 54, 55]. However, it has also beensuggested that code churn is not a direct measure of effort but rather one ofmany metrics used to measure development effort [56]. We therefore only geta general idea of the actual efforts put into refactoring by using refactoring
churn. Although we used a state of the art refactoring detection tool that wasused several times in previous studies [57, 58, 59, 60], we have no guaranteethat all refactorings applied to the code were detected.
6. Conclusion
In this study, we investigated the impacts of shortening the release cycles on refactoring activities in three Eclipse projects which all developed
using both traditional yearly releases and shorter quarterly cycles. Throughthis case study, we found that shortening the release cycles may impact the31refactoring activities of a software project in various ways. We also foundthat some of the refactoring patterns that were less common after shortening
the releases are usually applied to improve the readability and reusability of
the code. Not applying them could therefore affect the software’s quality inthe long term. This case study further contributes to the empirical knowledge related to shortening the release cycles. With the results we present
in this paper, potential short releases adopters now have a better insight ofthe potential impacts on refactoring activities that can occur from shortening their release cycle. When aggregated with previous studies that study
the impacts of shortening the release cycles, our results allow short releases
adopters to have an overall picture of which area of development can be impacted when shortening the release cycles. It also allows them to dedicate
resources accordingly to make the transition as smooth as possible.
Intuitively, it would make sense to observe less refactoring, more bugs, orother issues when shortening a project’s development cycle due to the shortamount of time available to developers. One interesting factor that however
seems common across studies investigating the impacts of shorter release cycles is that developers can adapt to the shorter releases and manage softwaredevelopment tasks even with less time between each release. Although theimpacts of shorter seem manageable, it would be interesting to study in the
future what problems early adopters such as Google and Firefox have encountered when adopting shorter releases and how these issues were solved.
In the case of our study, for example, while we cannot prove with certainty
that developers are using more refactoring tools to keep up the with refactoring workload, our results do seem to indicate that more tools may havebeen used to mitigate the impacts of the shorter releases.
Acknowledgment
We gratefully acknowledge the financial support of JSPS and SNSF forthe project “SENSOR” (JPJSJRP20191502), and JSPS for the KAKENHI
grants (JP21H04877, JP21K17725).

32

References[1] F. Khomh, B. Adams, T. Dhaliwal, Y. Zou, Understanding the impact
of rapid releases on software quality, Empirical Software Engineering
20 (2] [2015] 336–373.
[2] M. V. Mäntylä, B. Adams, F. Khomh, E. Engström, K. Petersen,
On rapid releases and software testing: A case study and a semisystematic literature review, Empirical Software Engineering 20 (5][2015] 1384–1425.
[3] F. Palomba, A. Zaidman, R. Oliveto, A. De Lucia, An exploratory study
on the relationship between changes and refactoring, in: Proceedings ofthe 25th International Conference on Program Comprehension, 2017, p.
176–185.
[4] M. Kim, D. Cai, S. Kim, An empirical investigation into the role of apilevel refactorings during software evolution, in: Proceedings of the 33rd
International Conference on Software Engineering, 2011, pp. 151–160.
[5] K. Stroggylos, D. Spinellis, Refactoring–does it improve software quality?, in: Proceedings of the 5th International Workshop on Software
Quality, 2007, p. 10.
[6] A. Silva, G. Carneiro, F. Brito e Abreu, M. Monteiro, Frequent releases
in open source software: A systematic review, Information 8 (3] [2017]109.
[7] D. A. da Costa, S. McIntosh, C. Treude, U. Kulesza, A. E. Hassan,
The Impact of Rapid Release Cycles on the Integration Delay of Fixed
Issues, Empirical Software Engineering 23 (2] [2018] 835–904.
[8] M. Fowler, K. Beck, J. Brant, W. Opdyke, D. Roberts, Refactoring:
improving the design of existing code, Addison-Wesley, 1999.
[9] M. Kim, M. Gee, A. Loh, N. Rachatasumrit, Ref-finder: A refactoring
reconstruction tool based on logic query templates, in: Proceedings ofthe Eighteenth ACM SIGSOFT International Symposium on Foundations of Software Engineering, 2010, p. 371–372.

33

[10] N. Tsantalis, M. Mansouri, L. M. Eshkevari, D. Mazinanian, D. Dig,
Accurate and efficient refactoring detection in commit history, in: Proceedings of the 40th International Conference on Software Engineering,
2018, pp. 483–494.
[11] D. Silva, M. T. Valente, Refdiff: Detecting refactorings in version histories, in: Proceedings of the 2017 IEEE/ACM 14th International Conference on Mining Software Repositories, 2017, pp. 269–279.
[12] D. Dig, C. Comertoglu, D. Marinov, R. Johnson, Automated detectionof refactorings in evolving components, in: Proceedings of the 20th European Conference on Object-Oriented Programming, 2006, p. 404–428.
[13] Z. Xing, E. Stroulia, Umldiff: An algorithm for object-oriented design
differencing, in: Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering, 2005, pp. 54–65.
[14] C. Vassallo, G. Grano, F. Palomba, H. Gall, A. Bacchelli, A large-scale
empirical exploration on refactoring activities in open source software
projects, Science of Computer Programming 180 [2019] 1–15.
[15] A. Peruma, M. W. Mkaouer, M. J. Decker, C. D. Newman, An empirical investigation of how and why developers rename identifiers, in:Proceedings of the 2nd International Workshop on Refactoring, 2018, p.
26–33.
[16] D. Silva, N. Tsantalis, M. T. Valente, Why we refactor? confessions ofgithub contributors, in: Proceedings of the 2016 24th ACM SIGSOFTInternational Symposium on Foundations of Software Engineering, 2016,p. 858–870.
[17] G. Bavota, B. De Carluccio, A. De Lucia, M. Di Penta, R. Oliveto,O. Strollo, When does a refactoring induce bugs? an empirical study, in:Proceedings of the 2012 IEEE 12th International Working Conference
on Source Code Analysis and Manipulation, 2012, p. 104–113.
[18] M. Kim, T. Zimmermann, N. Nagappan, A field study of refactoring
challenges and benefits, in: Proceedings of the ACM SIGSOFT 20th
International Symposium on the Foundations of Software Engineering,
2012, pp. 1–11.
34

[19] E. Murphy-Hill, C. Parnin, A. P. Black, How we refactor, and how
we know it, in: Proceedings of the 31st International Conference on
Software Engineering, 2009, p. 287–297.
[20] G. Lacerda, F. Petrillo, M. Pimenta, Y. Guéhéneuc, Code smells andrefactoring: A tertiary systematic review of challenges and observations,
Journal of Systems and Software [2020] 110610.
[21] R. Moser, P. Abrahamsson, W. Pedrycz, A. Sillitti, G. Succi, A casestudy on the impact of refactoring on quality and productivity in anagile team, 2007.
[22] R. Moser, A. Sillitti, P. Abrahamsson, G. Succi, Does refactoring improve reusability?, 2006, pp. 287–297.
[23] D. Dig, R. Johnson, The role of refactorings in api evolution, 2005, pp.
389–398.
[24] S. Herbold, J. Grabowski, H. Neukirchen, Automated refactoring suggestions using the results of code analysis tools, 2009, pp. 104 – 109.
[25] X. Ge, E. Murphy-Hill, Manual refactoring changes with automatedrefactoring validation, in: Proceedings of the 36th International Conference on Software Engineering, 2014, p. 1095–1105.
[26] H. K. Wright, D. Jasper, M. Klimek, C. Carruth, Z. Wan, Large-scale
automated refactoring using clangmr, in: 2013 IEEE International Conference on Software Maintenance, 2013, pp. 548–551.
[27] Z. Xing, E. Stroulia, Refactoring practice: How it is and how it should
be supported - an eclipse case study, in: Proceedings of the 22nd IEEE
International Conference on Software Maintenance, 2006, pp. 458–468.
[28] B. Daniel, D. Dig, K. Garcia, D. Marinov, Automated testing of refactoring engines, 2007, pp. 185–194.
[29] M. Michlmayr, B. Fitzgerald, K.-J. Stol, Why and how should open
source projects adopt time-based releases?, IEEE Software 32 (2] [2015]
55–63.

35

[30] F. Khomh, T. Dhaliwal, Y. Zou, B. Adams, Do faster releases improve software quality? An empirical case study of Mozilla Firefox, in:Proceedings of the 9th IEEE Working Conference on Mining Software
Repositories, 2012, pp. 179–188.
[31] O. Baysal, I. Davis, M. W. Godfrey, A Tale of Two Browsers, in: Proceedings of the 8th Working Conference on Mining Software Repositories, 2011, p. 238–241.
[32] V. M. Mika, F. Khomh, B. Adams, E. Engstr, K. Petersen, On Rapid
Releases and Software Testing, in: Proceedings of the 2013 IEEE International Conference on Software Maintenance, 2013, pp. 20–29.
[33] E. Kula, A. Rastogi, H. Huijgens, A. Van Deursen, G. Gousios, Releasing fast and slow: An exploratory case study at ING, in: Proceedingsof the 2019 27th ACM Joint Meeting European Software Engineering
Conference and Symposium on the Foundations of Software Engineering, 2019, pp. 785–795.
[34] M. Claes, M. Mantyla, M. Kuutila, B. Adams, Abnormal Working
Hours: Effect of Rapid Releases and Implications to Work Content, in:Proceedings of the 14th International Conference on Mining Software
Repositories, 2017, pp. 243–247.
[35] N. Kerzazi, F. Khomh, Factors impacting rapid releases: An industrial
case study, in: Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 2014, pp.
98–107.
[36] G. N. Tonietto, S. A. Malkoc, S. M. Nowlis, When an Hour Feels Shorter:
Future Boundary Tasks Alter Consumption by Contracting Time, Journal of Consumer Research 45 (5] [2018] 1085–1102.
[37] A. Garrido, R. Johnson, Challenges of refactoring c programs, 2002, p.
6–14.
[38] C. Vassallo, F. Palomba, H. C. Gall, Continuous refactoring in ci: A
preliminary study on the perceived advantages and barriers, in: International Conference on Software Maintenance and Evolution, 2018, pp.
564–568.
36

[39] H. Do, S. Mirarab, L. Tahvildari, G. Rothermel, The effects of time
constraints on test case prioritization: A series of controlled experiments,
IEEE Transactions on Software Engineering 36 (5] [2010] 593–617.
[40] H. Do, G. Rothermel, An empirical study of regression testing techniques incorporating context and lifetime factors and improved costbenefit models, in: Proceedings of the 14th ACM SIGSOFT International Symposium on Foundations of Software Engineering, 2006, p.
141–151.
[41] K. R. Walcott, M. L. Soffa, G. M. Kapfhammer, R. S. Roos, Timeaware
test suite prioritization, in: Proceedings of the 2006 International Symposium on Software Testing and Analysis, 2006, p. 1–12.
[42] Jung-Min Kim, A. Porter, A history-based test prioritization techniquefor regression testing in resource constrained environments, in: Proceedings of the 24th International Conference on Software Engineering, 2002,
pp. 119–129.
[43] S. Karus, M. Dumas, Predicting coding effort in projects containing
xml, in: Proceedings of the 2012 16th European Conference on Software
Maintenance and Reengineering, 2012, p. 203–212.
[44] T. Mende, R. Koschke, Revisiting the evaluation of defect prediction
models, in: Proceedings of the 5th International Conference on Predictor
Models in Software Engineering, 2009, pp. 1–10.
[45] Y. Kamei, S. Matsumoto, A. Monden, K.-i. Matsumoto, B. Adams, A. E.
Hassan, Revisiting common bug prediction findings using effort-awaremodels, in: Proceedings of the 2010 IEEE International Conference on
Software Maintenance, 2010, p. 1–10.
[46] M. Tufano, F. Palomba, G. Bavota, R. Oliveto, M. D. Penta, A. De Lucia, D. Poshyvanyk, When and why your code starts to smell bad (andwhether the smells go away), IEEE Transactions on Software Engineering 43 (11] [2017] 1063–1088.
[47] C. Bogart, C. Kästner, J. Herbsleb, F. Thung, How to break an api:
Cost negotiation and community values in three software ecosystems, in:Proceedings of the 2016 24th ACM SIGSOFT International Symposium
on Foundations of Software Engineering, 2016, p. 109–120.
37

[48] A. Hindle, E. T. Barr, Z. Su, M. Gabel, P. Devanbu, On the naturalnessof software, in: Proceedings of the 34th International Conference on
Software Engineering, 2012, p. 837–847.
[49] B. Ray, V. Hellendoorn, S. Godhane, Z. Tu, A. Bacchelli, P. Devanbu,
On the ”naturalness” of buggy code, in: Proceedings of the 38th International Conference on Software Engineering, 2016, p. 428–439.
[50] R. Arima, Y. Higo, S. Kusumoto, Toward refactoring evaluation withcode naturalness, in: Proceedings of the 26th Conference on Program
Comprehension, 2018, p. 316–319.
[51] B. Lin, C. Nagy, G. Bavota, M. Lanza, On the impact of refactoring
operations on code naturalness, in: 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering, 2019, pp.
594–598.
[52] E. Arisholm, L. Briand, E. Johannessen, A systematic and comprehensive investigation of methods to build and evaluate fault prediction models, Journal of Systems and Software 83 (1] [2010] 2–17.
[53] S. Karus, M. Dumas, Predicting coding effort in projects containing
xml, in: Proceedings of the 2012 16th European Conference on Software
Maintenance and Reengineering, 2012, p. 203–212.
[54] T. Mende, R. Koschke, Revisiting the evaluation of defect prediction
models, in: Proceedings of the 5th International Conference on Predictor
Models in Software Engineering, 2009, pp. 1–10.
[55] T. Menzies, Z. Milton, B. Turhan, B. Cukic, Y. Jiang, A. Bener, Defectprediction from static code features: Current results, limitations, newapproaches, Automated Software Engineering [2010] 375–407.
[56] E. Shihab, Y. Kamei, B. Adams, A. E. Hassan, Is lines of code a goodmeasure of effort in effort-aware models?, Information and Software
Technology [2013] 1981–1993.
[57] P. Thongtanunam, W. Shang, A. E. Hassan, Will this clone be shortlived? towards a better understanding of the characteristics of shortlived clones, Empirical Software Engineering 24 (2] [2019] 937–972.

38

[58] L. Sousa, D. Cedrim, W. Oizumi, A. Bibiano, A. Oliveira, A. Garcia,
D. Oliveira, M. Kim, Characterizing and identifying composite refactorings: Concepts, heuristics and patterns, 2020.
[59] M. Iammarino, F. Zampetti, L. Aversano, M. Di Penta, Self-admitted
technical debt removal and refactoring actions: Co-occurrence or more?,in: 2019 IEEE International Conference on Software Maintenance andEvolution, 2019, pp. 186–190.
[60] E. C. Neto, D. A. d. Costa, U. Kulesza, Revisiting and improving szz implementations, in: 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, 2019, pp. 1–12.

39