2021 IEEE 15th International Workshop on Software Clones (IWSC)

Type-2 Code Clone Detection for DockerﬁlesTomoaki Tsuru, Tasuku Nakagawa, Shinsuke Matsumoto, Yoshiki Higo, and Shinji Kusumoto
Graduate School of Information Science and Technology, Osaka University, Japan
E-mail: {t-tsuru, t-nakagw, shinsuke, higo, kusumoto}@ist.osaka-u.ac.jp

Abstract—This paper proposes a Type-2 code clone
detection technique for Dockerﬁles. Docker is a platform for realizing containerized virtual environmentsthat has been attracting signiﬁcant attention as a technology support for server infrastructures. In Docker,
the procedure to realize a virtual environment is described in the form of a source code called a Dockerﬁle. Therefore, code clones such as repetitions andduplications with similar structures are often included.
In this study, we propose a Type-2 code clone detection technique for deriving patterns within Dockerﬁles.
More speciﬁcally, our technique detects code clones bynormalizing tokens properly, separating Docker syntax
and shell syntax, and then adopting suﬃx array algorithm for each syntax. We conducted application experiments on approximately 5,000 Dockerﬁles available
on GitHub and found that our technique could detect
Type-2 code clones with a high precision. Our technique
derived some patterns in Dockerﬁles, such as those used
for building a new distribution on a container.

I. Introduction
Docker has been the gold standard for providing
container-level virtualization. Docker is one of the key
technologies that realizes infrastructure as code (IaC)
where the procedure of container (i.e., virtual environment) construction is explicitly described as source code
called Dockerﬁle. IaC enables rapid reproduction of exactly the same infrastructure environment no matter who
executes it or when it is executed [1]. However, it has been
pointed out that there are still some immature domains ofresearch because Docker and IaC-relating technologies arein their infancy [2].
This paper focuses on code clones (clones) included inDockerﬁles. A clone is a code fragment that is identical or similar to another fragment. In the ﬁeld of software engineering, numerous studies has been conducted
on code clone detection for various programming languages [3] [4] [5]. Clone detection can be applied forvarious applications; recommendation for usage of application programming interfaces (APIs) [6], refactoring recommendations for redundant code fragments [7], suggestions
to change fragments requiring the same change [8], andlicense violation detection [9].
Clone detection techniques have been proposed for not
only procedural languages such as Java and C but also
various other languages. On the other hand, there havebeen a few proposals for Dockerﬁle clone detection In one
such study, Oumaziz et al. have proposed a duplicate code
detection technique for Dockerﬁles [10]. This technique

isolates tokens in Dockerﬁles and detects duplicates using
inverted index algorithm. However, this technique cannot
detect clones except for variations in variables (Type-2
clones) because it only targets duplicate code fragmentsthat match perfectly (Type-1).
In this study, we propose a Type-2 code clone detection
technique for Dockerﬁles. The contributions of this study
are as follows:
Consideration and deﬁnition of Type-2 clones inDockerﬁles: Type-2 clones allow for diﬀerences intokens that do not aﬀect the behavior of the program. Therefore, it is essential that variable names,
function names, and constants be normalized properly
to facilitate for their detection. Dockerﬁle contains
not only variables but also various constant tokens
such as ﬁle paths and usernames. In addition, there
are many notations that do not aﬀect the behaviorof Dockerﬁles, such as parameter order and option
aliases. We consider Type-2 clones in Dockerﬁles byorganizing the tokens to be normalized.
• Proposal of a clone detection technique in Dockerﬁles that considers multiple syntaxes: Dockerﬁle is anested language that allows the syntax of multiple
languages to be written in a single ﬁle [11] [12]. Morespeciﬁcally, it uses shell syntax to describe the internal processing of the container and Docker syntax todescribe processing with the outside of the containersuch as copying ﬁles from the host operating system(OS) or specifying the container network ports. Our
proposed clone detection technique ﬁrst generates abstract syntax trees (ASTs) from the given Dockerﬁlesand then normalizes the tokens. Our technique also
separates Dockerﬁles into two syntaxes and then uses
suﬃx array algorithm to achieve multi-syntactic clone
detection.
• Type-2 clone detection for publicly accessible Dockerﬁles and pattern derivation in Dockerﬁle: Type-2
clone detection using our proposed technique is performed on 4,817 Dockerﬁles in 725 Dockerﬁle repositories available on GitHub. The results of these experiments showed that Type-2 clones are detected witha precision of 95 %. We also visually derive patterns
that appear in multiple Dockerﬁles, such as those used
for building a new distribution on a container andsoftware installations using make commands.
•

Work licensed under Creative Commons Attribution 4.0 License. https://creativecommons.org/licenses/by/4.0/
1
DOI 10.1109/IWSC53727.2021.00007

1.
Construct
ASTs

2.
Normalize
tokens3.
Hashing per
instructions
0
1
2
5

4.
Integrate bysyntax05.
Construct & compare
suffix array (SA)
3

0

4

0
0
3
4

Docker syntax

3
4
0

0

6.
Outputclone sets
4

4

0

0

1

SA of Docker syntax
3
1

1

2

1

1

2

1
1
1
2
2

Shell syntax
0
1
4
5

2

Dockerfile Original ASTs

1
2
2

2
1

1

1

1

2

1

2

SA of shell syntax
0 1 2 3 1 0 1 4 2Docker & shell syntax
Normalized ASTs Arrays ofhash values

0
0
3
4
1
1
1
2
2

1
1
1
2
0
4
2

4 2
2 3 1 0 1 4 2
0 1 4 21 4 2
2
3 1 0 1 4 2

3 1 0 1 4 2

SA of Docker & shell syntax

1

2

2

Blue Docker syntax
Orange Shell syntax
Different token
Normalized token
Delimiter of files

Fig. 1: Overview of Type-2 clone detection for Dockerﬁlescontainers as source code. Dockerﬁle is the source code describing the procedure for building containers. Dockerﬁlecomposes an image which is the template of the container.
Dockerﬁles is a nested language within which shell syntax
in the RUN instruction can be embedded [11] [12].
Docker and other IaC-related technologies are in theirinfancy, and there are various IaC-related studies such as
characterizing Dockerﬁles smells [17] and code completionfor Dockerﬁles [18]. On the other hand, it has been pointed
out that there are some IaC domains that are still underresearched [2].

II. Preliminaries
A. Code Clone
A code clone (clone) is a code fragment that is identicalor similar to another fragment. A set of code fragment that
is identical or similar to another fragment is called a clone
set. Clones are classiﬁed as follows based on the degree ofsimilarity to their correspondences [3].
• Type-1: an identical copy except for variations inwhite spaces and comments.
• Type-2: a syntactically identical copy except for variations in identiﬁers such as variables and functions.
• Type-3: a copy with further statement-level modiﬁcations such as added or removed statements.
So far, numerous clone detection techniques andtools have been proposed for general programming languages [3] [4] [5]. General applications of clone detection
include recommendation for usage of application programming interfaces (APIs) [6], refactoring recommendations
for redundant code fragments [7], suggestions for changes
to fragments requiring the same change [8], and license
violation detection [9]. Clone detection research has also
been tackled in various languages other than programming
languages, such as build tools [13] and requirement speciﬁcations [14].

C. Duplicates in DockerﬁlesThere are a few studies on code clone detection in Dockerﬁles. Oumaziz et al. propose a duplicate code detection
technique for Dockerﬁles [10]. They use inverted index
algorithm [19] with some Dockerﬁle instructions as a chunkto detect duplicate code after parsing the Dockerﬁles.
The size of chunks goes from one Dockerﬁle instruction
to the maximal number of instructions contained in theDockerﬁle. In Dockerﬁle, there is the practice of writing
multiple shell commands inside a single RUN instruction.
In their technique, RUN instructions are separated into
internal shell commands during parsing in order to detect
duplicate codes inherent in shell commands.
Oumaziz et al. only detect exact duplicates (Type-1
clones) because they do not apply token normalization.
This means their technique cannot detect Type-2 clones,
which are caused by variations in the tokens such as
variables. In addition, even if similar Docker instruction
or shell command sequences appear in multiple Dockerﬁleprojects, their technique cannot be applied to patternderivation in Dockerﬁles due to the presence variations intoken strings.

B. Docker
Docker is a platform that implements operation-systemlevel virtualization called container. Docker has attracted
signiﬁcant amounts of attention in recent years; 87 % of
information technology companies and a variety of opensource software use Docker [15] [16].
Docker is one of the key technologies to realize infrastructure conﬁguration automation called infrastructure
as code (IaC) by describing the procedure for building

2III. Proposed TechniqueA B A C D A B
Sequence of tokens

We propose a Type-2 clone detection technique forDockerﬁles. This technique can derive patterns that appear in multiple Dockerﬁles. Figure 1 shows an overview
of Type-2 clone detection for Dockerﬁles. The input of
our technique is a set of Dockerﬁles and the output isthe detected clone sets. The ﬂow of our technique isdescribed in Figure 1. This technique ﬁrst constructs ASTs
corresponding to each of the multiple Dockerﬁles given as
input (step 1) and then normalizes each token (step 2).
This normalization process allows us to extend the Type1 clone detection to Type-2 clone detection. Next, perinstruction hashing (step 3) and per-syntax integration(step 4) are applied. Finally, Docker syntax and shell
syntax are separated in advance, followed by suﬃx arrayalgorithm (step 5) and clone set detection (step 6) toachieve per-syntax clone detection.
Suﬃx array, which is a kind of data structure used instring search algorithms, consists of an array of suﬃxes
in a search string that have been sorted in lexicographic
order [20]. Suﬃx array has also been applied in the ﬁeldof clone detection [21] [22]. Inverted index algorithm candetect clones scalably by setting the appropriate size ofchunks [19]. The smaller the size of chunks in inverted
index algorithm, the longer it takes to detect clones usingthat algorithm. On the other hand, suﬃx array algorithm
can detect clones in linear time without predetermining
the chunk size [20]. Therefore, our proposed techniqueuses suﬃx array for code clone detection. Figure 2 shows
the procedure for constructing a suﬃx array based on thetoken sequence “A B A C D A B” and the clone detection
technique. First, each suﬃx of the input token sequenceis stored in an array via suﬃx enumeration (step 1).
Next, the suﬃx array is sorted in lexicographic order(step 2). Finally, this algorithm outputs clone sets forward
matching subtokens for suﬃxes in the suﬃx array (step 3).
In the token sequence example of “A B A C D A B”, there
are three clones: “A”, “A B”, and “B”.
Our proposed technique adopted the technique of constructing ASTs in Dockerﬁles by Henkel et al. [11]. Thisis because their ASTs have tokens parsed not only atDocker syntax level but also at the shell command level,
which makes it easier to normalize the tokens and separate Docker syntax and shell syntax with our proposed
technique. Henkel et al. presents ASTs in Dockerﬁles as
JavaScript Object Notation (JSON) format. They take
Dockerﬁles as input and output ASTs according to thefollowing procedure. First, they convert Dockerﬁles into
concrete syntax trees at the Docker syntax level. Next,
they construct syntax trees at the shell script grammar
level for the RUN instruction whose argument is shell
syntax. In their technique, Bash Shell, which is generally
used in Dockerﬁles, is adopted as shell script for parsing.
Finally, shell command is parsed to construct AST, which
is output in JSON. Note that Henkel et al. limit the

1. Enumerate suffixes
3. Output clone sets

2. Sort in lexicographic order
Index

SuffixIndex

Suffix

0

A B A C D A B

5 A B

1

B A C D A B

2

A C D A B

3

C D A B

4

D A B

5

A B

3 C D A B

6B

4 D A B

0 A B A C D A B
2 A C D A B
6 B
1 B A C D A B

0 2 5
Clone set “A”

0 2
Clone set “A B”

1 3
Clone set “B”

Fig. 2: Procedure for constructing a suﬃx array and clone
detection techniqueparsing to the top 50 shell commands that are mostfrequently used in Dockerﬁle.
In the following sections, we discuss the tokens to benormalized in step 2 (Section III-A) and the reason for
pre-separating Docker syntax and shell syntax in step 4(Section III-B).
A. Considering Normalized Tokens
To detect Type-2 clones, it is necessary to properly
normalize the tokens in source codes. In general clone
studies targeting Java, C and other similar programming
languages, identiﬁer names such as variable names andfunction names are often selected as the normalization
targets [23]. This is because variable and function names
are a kind of label information, and the same behavior
can be realized even if the labels are diﬀerent. This is also
true in Dockerﬁles, where there are temporary variables(variables declared with ARG instructions), etc., that are
subject to normalization. Furthermore, in terms of theidentity of the behavior in source codes, the order ofparameters and options in shell script (ls -l . andls . -l) and the aliases of option names (ls -list andls -l) in shell script, there are many other notations that
can achieve the same behavior. Hence, Type-2 clones inDockerﬁle can be detected by properly normalizing thesetokens.
This study deﬁnes the tokens to be normalized as shownin Table I and the tokens that are not to be normalized
as shown in Table II. Figure 3 shows an example of token
normalization using our proposed technique. One of thetokens to be normalized is the order parameters andoptions in shell commands. Shell commands deﬁne theorder of the parameters and options for each commands.
Some commands allow the order of parameters andoptions to be diﬀerent. As a result, commands thathave the same behavior despite having variations ofparameters and options can exist in multiple Dockerﬁles.
RUN apt install --yes --no-install-recommends
and RUN apt install --no-install-recommends --yes33 FROM ubuntu:18.04

3 FROM ubuntu:$$TAG

...

...

16 ARG PGP_KEYSERVER=ha.pool.sks-keyservers.net

16 ARG $$VAR=ha.pool.sks-keyservers.netBrown: Normalization of tokens...
Bule-green: Alias of options
59 RUN set –e –u -x; ¥Underline: Separate or reordering options¥
60
Gray: Remove options
61$$VAR="$(apt-mark showmanual)”: ¥apt-get update; ¥6263
apt-get install --no-install-recommends –-yes ¥...

59 RUN set -eux; ¥¥60
61
savedAptMark="$(apt-mark showmanual)"; ¥apt-get update; ¥6263
apt-get install --yes --no-install-recommends ¥
...

72
73

...

72
73

; ¥
rm -rf /var/lib/apt/lists/*; ¥
...

83
84
85

...

83
84
85

for key in $OPENSSL_PGP_KEY_IDS; do ¥gpg --batch --keyserver "$PGP_KEYSERVER" --recv-keys "$key"; ¥done; ¥
...

255

; ¥
rm –-force --recursive $$PATH; ¥$OPENSSL_PGP_KEY_IDS
is not normalized
because it isthe environment variable

for $$VAR in $OPENSSL_PGP_KEY_IDS; do ¥gpg --batch --keyserver "$$OP_ARG” --recv-keys ”$$OP_ARG"; ¥done; ¥...

255

chown -R rabbitmq:rabbitmq "$RABBITMQ_HOME"; ¥...
...

chown -R $$USER:$$GROUP "$RABBITMQ_HOME"; ¥...
...

(a) Before normalization(b) After normalization

Fig. 3: Example of normalization for an actual Dockerﬁle 1

are examples of clones with variations of the order of
options.
The option controlling logs and the option controllinginteractions are also contained in the tokens to be normalized in our proposed technique. The options for controlling
logs, such as the log output option --verbose in wget
command and the error output option --show-error incurl command, only control the log output. The presence or absence of these options does not change thebehavior of Dockerﬁles. Additionally, assuming that theDockerﬁles have been built without errors, the options

for controlling interactions such as --force, which is aforce execution option for rm command, do not change
the behavior of Dockerﬁles with or without those options.
Therefore, our proposed technique also normalizes theoptions for controlling logs and the options for controlling interactions. These options are removed during thenormalization process because their presence or absence
do not change the behavior of Dockerﬁles. For example,
the command RUN curl -fsSL $URL has four options: the--fail option (-f) to prohibit log output in case of connection failure, the --silent option (-s) to prohibit log
output, the --show-error option (-S) to allow log output

1 https://github.com/docker-library/rabbitmq/blob/master/
Dockerﬁle-ubuntu.template

TABLE I: Normalized tokens
Token
Temporary variables
File paths
URLs
Username and group names
Tags included FROM instruction
Arguments of options
Order of parameters and options
Aliases of options
Options for controlling logs
Options for controlling interactions
Options for controlling output ﬁlesReason and example of normalization
Temporary variables do not aﬀect the external processes in Docker container.
ARG tag=bionic → ARG $$VAR=bionic
Variations of ﬁle paths do not aﬀect behavior.
cd webapps/convertigo → cd $$PATH
URLs do not aﬀect resources outside Docker container.
wget http://example.com/app/app.tar.gz → wget $URL
Variations of usernames and group names do not aﬀect behavior.
USER spark:spark → USER $$USER:$$GROUP
Diﬀerences in container functionality among variations of tags are insigniﬁcant.
FROM ubuntu:16.04 → FROM ubuntu:$$TAG
The roles of tokens set as arguments are assigned for each option.
gpg --keyserver $KEY_SERVER → gpg --keyserver $OP_ARG
Variations of the order of options allow for equivalent commands to be executed.
set -xe → set -e -x
Aliases of options allow for equivalent commands to be executed.
rm -fr → rm --force --recursive
Variations with or without options for controlling logs do not aﬀect behavior.
wget -q → wget
Variations with or without options for controlling logs do not aﬀect behavior if Dockerﬁles have no errors.
apt-get install wget -y → apt-get install wget
Variations of ﬁle paths do not aﬀect behaviors.
wget -o $file → wget

TABLE II: Denormalized tokens
TokensEnvironment Variables
Images of the Docker Container
Network Ports of the Docker Container

ReasonEnvironment variables aﬀect external processes in the Docker container.
Each image contains a diﬀerent package manager.
Variations in port numbers aﬀect processes outside the Docker container.

4

only in case of error, and the --location redirection
option (-L). The --fail, --silent, and --show-error
options are classiﬁed as options for controlling logs, andthe --location option is an option controlling interaction. Therefore, the command RUN curl -fsSL $URL isnormalized to RUN curl $URL.

Technically, these automatically generated Dockerﬁles are
Type-2 clones, but since they are obvious clones thathave been generated automatically, they should not be
detected. Therefore, for repositories that contain templateﬁles, we exclude the Dockerﬁles in the repository and use
the template ﬁles themselves as clone detection targets.

B. Docker Syntax and Shell Syntax Segregation

B. Results

Dockerﬁle is a nested language that contains the shell
syntax [11] [12]. We expect to derive patterns that do notdepend on a speciﬁc Dockerﬁle by detecting clones in eachsyntax. Therefore, we separate Docker syntax and shell
syntax to detect Type-2 clones. Clones that contain bothDocker syntax and shell syntax are also detected becausesuch clones that contain both syntaxes are useful.

1) Number of clones and precisions: Table III shows theexperimental results with the number of clone segments,
the number of clone sets, the number of clone segments per
clone set, the length of clone segments per clone set, andthe precision detected in this experiment. Comparing our
proposed and no-normalization detection techniques, perdetected clone set, our proposed technique has a larger
number of clone segments and a longer length of clone
segment than those for the no-normalization detection
technique. The increased number and lengths of clone
segments per detected clone set identiﬁed by our proposed
technique met expectations. On the other hand, our proposed technique detected fewer clone sets than the nonormalization detection technique. The reason why our
proposed technique detects fewer clone sets than the nonormalization detection technique is that segments thatare not related to Type-1 clones are detected as Type-2
clones by the normalization process. As a result, independent clone sets in Type-1 are considered as equivalent clone
sets in Type-2.
Comparing the precision of our proposed techniquewith that of the no-normalization detection technique, thepresicion of our proposed technique is lower than thatof the no-normalization detection technique. The morenormalization is applied, the lower the precision tends to
be in general [4]. Generally speaking, Type-2 clones are
more diﬃcult to detect than Type-1 clones, and Type-3
clones are even more diﬃcult to detect than Type-2 clones.
However, our proposed technique has a high precision of
95 % and is useful as a Type-2 clone detection technique.
However, the high precision of 95 % achieved by our
proposed technique demonstrates its usefulness as a Type2 clone detection technique, particularly since, as reported
by Sheneamer et al., the precisions of most clone detection
techniques for general programming languages are lessthan 50 % [4]. It should also be noted that the reason
why the precision of our proposed technique is higher thanthat of clone detection techniques for general programming
languages is that, unlike general programming languages,

IV. Experiments
This study conﬁrms that our proposed technique candetect Type-2 clones of Dockerﬁles and patterns. As acomparison, we use the Type-1 clone detection technique
without normalization in our proposed technique, which isreferred to hereafter as the “no-normalization” detection
technique. The metrics are the number of detected clone
segments, the number of detected clone sets, the number ofclone segments per detected clone set, the length of clone
segments per detected clone set, and the precision. The
number of clone segments per clone set and the lengthof clone segments per clone set by our proposed techniqueare expected to increase since Type-2 clones include Type1 clones. Precision is the rate of candidates that are actual
clones among the detected clones. The higher the precision
is, the fewer false clone are detected by the clone detection technique. Therefore, a higher precision of a clone
detection indicates a superior clone detection technique.
We sampled the detected clone sets to a conﬁdence level of
95 % and a conﬁdence interval of 5 % for each techniques.
The precision is based on visual conﬁrmation because theset of true clones in the target Dockerﬁles is unknown.
A. Subjects
This experiment targets 4,817 Dockerﬁles contained in
725 repositories that are popular on GitHub. In Docker,
there is a practice that the entire ﬂow of container building is often described in a template ﬁle, within which
the deﬁning minor diﬀerences between distributions andversions in temporary variables that can be replaced withtext are deﬁned [10]. The target Dockerﬁles are automatically generated by executing or compiling this template.

TABLE III: Number of clones and precisions# of clone segments# of clone sets
# of clone segments per clone set
Length of clone segments per clone set
Precision

Proposed technique (no-normalization)
Docker syntax Shell syntax Both syntaxes
217,766
433,161
930,134
50,60887,126
204,840
4.30
4.97
4.54
18.36
11.96
20.78
98.70 %
99.48 %
100.00 %5

Proposed technique (normalization)
Docker syntax Shell syntax Both syntaxes
240,639458,737
987,597
50,756
84,011
203,103
4.74
5.46
4.86
18.64
12.63
21.30
98.44 %
96.09 %
95.31 %

1 FROM scratch

1 FROM scratch

2 MAINTAINER James Mills, prologic at shortcircuit dot net dot au

2 MAINTAINER Vlad Glagolev <stealth@sourcemage.org>

3 ADD rootfs.tar.xz /4 CMD ["/bin/bash"]

3 ADD smgl-stable-0.62-docker-x86_64.tar.xz /4 CMD ["/bin/bash"](a) Clone segments in a Dockerﬁle of CRUX image 4(b) Clone segments in a Dockerﬁle of Source Mage image 5

Fig. 4: Distribution building pattern for Type-2 clone detected by our proposed technique
3 RUN SWIPL_VER=7.5.11 && ¥...
12&& cd swipl-$SWIPL_VER && ./configure && make&& make install ¥49 RUN wget https://github.com/protocolbuffers/protobuf/releases/
download/v3.7.1/protobuf-cpp-3.7.1.tar.gz ¥...
51&& cd protobuf-3.7.1 ¥52&& ./configure --disable-shared --with-pic && make&& make install && ldconfig ¥(a) Clone segments in a Dockerﬁle of SWI-Prolog image 6(b) Clone segments in one Dockerﬁle on GitHub 7

Fig. 5: Software installation pattern using make commands for Type-2 clone detected by our proposed technique

Docker syntax and shell syntax assign runtime roles totokens.
2) Examples of detected Type-2 clone: We visually derived some Dockerﬁles patterns from among the Type-2
clones detected by our proposed technique in this experiment. Figure 4 and Figure 5 show some of the derived
patterns.
Figure 4 shows the pattern used for building a new distribution on a container. Dockerﬁles copy the tar archivefrom the host OS to an empty container, called scratch,and then extract the tar archive on the container to build
a new distribution. Note that the MAINTAINER instruction
sets the author as metadata in the container. We consider
the MAINTAINER instruction to be a comment when it isused in general languages to detect clones because this
instruction does not change the behavior of the container.
This pattern was derived as 13 clone segments from 5
repositories.
Figure 5 shows the software installation pattern using
make commands. Both Dockerﬁles conﬁgure a Makeﬁleunder the directory where the target software is locatedafter being changed to that directory. Next, the binaryﬁle generated by make command is installed into thedirectory speciﬁed by install label in the Makeﬁle. The--disable-shared option in configure command (line
52 of Figure 5(b)) controls the output of dynamic andstatic libraries while the --with-pic option in the samecommand controls the output of position-independentcodes. These option are classiﬁed as options for controlling
output ﬁles. This experiment detects line 12 of Figure 5(a)and lines 51-52 of Figure 5(b) as Type-2 clones because
these options are classiﬁed as options for controlling out-

put ﬁles. This pattern was derived as 12 clone segmentsfrom 5 repositories.
V. Related Works
A. Clone Detection Techniques for General Programming
Languages
Numerous clone detection techniques have been proposed for general programming languages [3] [4] [5].
Typical lexical clone detection tools include
CCFinder [24] and Clone Miner [21]. CCFinder [24]detects token-based clones using suﬃx tree algorithm [25].
Clone Miner [21] detects token-based clones using suﬃx
array algorithm [20]. The goal of this study is to derive
patterns in Dockerﬁles. We focus on syntactical clone
detection rather than lexical detection because it isanticipated that clone detection in Dockerﬁle instructions
enable pattern derivation in Dockerﬁles.
Falke et al. detect AST-based clones using suﬃx tree
algorithm [26]. This algorithm can detect clones in linear time and space. Suﬃx array algorithm [20] is morememory eﬃcient than suﬃx tree algorithm while the twoalgorithms take the same amount of time. Therefore, ourproposed technique uses suﬃx array algorithm to detectclones.
B. Clone Detection for Various Other Languages
Clone detection research has also been tackled in various
languages other than programming languages, such asbuild tools [13] and requirements speciﬁcations [14]. McIntosh et al. detects Type-1 clones for build systems such as
CMake/Autotools (C/C++) and Ant/Maven (Java) [13].
Juergens et al. detects clones with exact word matches(Type-1) in the quality assessments of software requirement speciﬁcations [14]. Bellon et al. classiﬁes clones
into three types (Type-1, Type-2, and Type-3) based on
the degree of similarity to their correspondences whileassuming a general programming language [3]. On theother hand, clones detected in those languages is mostly
an exact duplicate (Type-1) because the classiﬁcation ofclones in source code written in languages other than

4 https://github.com/cruxlinux/docker-crux/blob/master/
Dockerﬁle5 https://github.com/vaygr/docker-sourcemage/blob/master/
stable/Dockerﬁle6 https://github.com/swi-prolog/docker-swipl/blob/master/7.5.
11/alpine/Dockerﬁle7 https://github.com/eclipse-openj9/openj9/blob/master/buildenv/docker/jdk8/x86 64/ubuntu16/jitserver/buildenv/
Dockerﬁle6

general programming languages are undeﬁned. Problems
caused by clones in those languages may be missed because
they do not detect clones that are not perfect matches.
Therefore, this study deﬁnes and detects Type-2 clones inDockerﬁles, which is one of the source codes that describe
the procedure for building a virtual environment.

[10] M. A. Oumaziz, J.-R. Falleri, X. Blanc, T. F. Bissyandé, andJ. Klein, “Handling Duplicates in Dockerﬁles Families: Learningfrom Experts,” in Proc. International Conference on Software
Maintenance and Evolution, 2019, pp. 524–535.
[11] J. Henkel, C. Bird, S. K. Lahiri, and T. Reps, “A Datasetof Dockerﬁles,” in Proc. International Conference on Mining
Software Repositories, 2020, pp. 528–532.
[12] ——, “Learning from, Understanding, and Supporting DevOps
Artifacts for Docker,” in Proc. International Conference on
Software Engineering, 2020, pp. 38–49.
[13] S. McIntosh, M. Poehlmann, E. Juergens, A. Mockus, B. Adams,
A. E. Hassan, B. Haupt, and C. Wagner, “Collecting and Leveraging a Benchmark of Build System Clones to Aid in Quality
Assessments,” in Proc. International Conference on Software
Engineering, 2014, pp. 145–154.
[14] E. Juergens, F. Deissenboeck, M. Feilkas, B. Hummel,
B. Schaetz, S. Wagner, C. Domann, and J. Streit, “Can Clone
Detection Support Quality Assessments of Requirements Speciﬁcations?” in Proc. International Conference on Software Engineering, 2010, pp. 79–88.
[15] Portworx,“AnnualContainer
Adoption
Report,”https://portworx.com/wp-content/uploads/2019/05/2019container-adoption-survey.pdf, 2019, [Online; accessed 18. Jan.
2021].
[16] J. Cito, G. Schermann, J. E. Wittern, P. Leitner, S. Zumberi,and H. C. Gall, “An Empirical Analysis of the Docker Container
Ecosystem on GitHub,” in Proc. International Conference on
Mining Software Repositories, 2017, pp. 323–333.
[17] Y. Wu, Y. Zhang, T. Wang, and H. Wang, “Characterizing the
Occurrence of Dockerﬁle Smells in Open-Source Software: An
Empirical Study,” IEEE Access, vol. 8, pp. 34 127–34 139, 2020.
[18] K. Hanayama, S. Matsumoto, and S. Kusumoto, “Humpback:Code Completion System for Dockerﬁles Based on Language
Models,” in Proc. Workshop on Natural Language Processing
Advancements for Software Engineering, 2020, pp. 1–4.
[19] B. Hummel, E. Juergens, L. Heinemann, and M. Conradt,“Index-Based Code Clone Detection: Incremental, Distributed,
Scalable,” in Proc. International Conference on Software Maintenance, 2010, pp. 1–9.
[20] U. Manber and G. Myers, “Suﬃx Arrays: A New Method for
On-Line String Searches,” Siam Journal on Computing, vol. 22,
no. 5, pp. 935–948, 1993.
[21] H. A. Basit and S. Jarzabek, “Eﬃcient Token Based Clone Detection with Flexible Tokenization,” in Proc. European Software
Engineering Conference and the ACM SIGSOFT Symposium on
the Foundations of Software Engineering, 2007, pp. 513–516.
[22] G. Li, Y. Wu, C. K. Roy, J. Sun, X. Peng, N. Zhan, B. Hu, andJ. Ma, “SAGA: Eﬃcient and Large-Scale Detection of NearMiss Clones with GPU Acceleration,” in Proc. International
Conference on Software Analysis, Evolution and Reengineering,
2020, pp. 272–283.
[23] C. K. Roy, J. R. Cordy, and R. Koschke, “Comparison and
Evaluation of Code Clone Detection Techniques and Tools:
A Qualitative Approach,” Science of Computer Programming,
vol. 74, no. 7, pp. 470–495, 2009.
[24] T. Kamiya, S. Kusumoto, and K. Inoue, “CCFinder: A Multilinguistic Token-Based Code Clone Detection System for Large
Scale Source Code,” IEEE Transactions on Software Engineering, vol. 28, no. 7, pp. 654–670, 2002.
[25] D. Gusﬁeld, Algorithms on Strings, Trees, and Sequences. Cambridge University Press, 1997.
[26] R. Falke, P. Frenzel, and R. Koschke, “Empirical evaluation ofclone detection using syntax suﬃx trees,” Empirical Software
Engineering, vol. 13, no. 6, pp. 601–643, 2008.
[27] M. I. Abouelhoda, S. Kurtz, and E. Ohlebusch, “Replacing
suﬃx trees with enhanced suﬃx arrays,” Journal of discretealgorithms, vol. 2, no. 1, pp. 53–86, 2004.

VI. Conclusion
In this study, we proposed deﬁnition and a detection
technique for Type-2 clones in Dockerﬁles. We conducted
experiments on 4,817 ﬁles in 725 GitHub repositories. The
experiment results showed that our proposed technique
was eﬀective because it achieved the precisions higher than
95 % for Docker syntax, shell syntax, and both syntaxes.
We were also able to visually derive Type-2 clones that
could be patterns, such as those used for building a new
distribution on a container and software installation using
make commands.
As a future work, we plan to propose a technique fordetecting clones with statement-level modiﬁcations (Type3) in Dockerﬁles because some such clones that are caused
by variations in shell commands that have the same
behavior, such as package managers or variations in theorder of Docker instructions or shell commands.
Acknowledgment
This research was supported in part by JSPS KAKENHI
Japan (Grant Numbers: JP21H04877, 20H04166).
References[1] Y. Jiang and B. Adams, “Co-evolution of Infrastructure andSource Code - An Empirical Study,” in Proc. Working Conference on Mining Software Repositories, 2015, pp. 45–55.
[2] R. Jabbari, N. bin Ali, K. Petersen, and B. Tanveer, “Whatis DevOps? A Systematic Mapping Study on Deﬁnitions andPractices,” in Proc. the Scientiﬁc Workshop Proceedings ofXP2016, 2016, pp. 1–11.
[3] S. Bellon, R. Koschke, G. Antoniol, J. Krinke, and E. Merlo,“Comparison and Evaluation of Clone Detection Tools,” IEEE
Transactions on Software Engineering, vol. 33, no. 9, pp. 577–
591, 2007.
[4] A. Sheneamer and J. Kalita, “A Survey of Software Clone
Detection Techniques,” International Journal of Computer Applications, vol. 137, no. 10, pp. 1–21, 2016.
[5] H. Min and Z. Li Ping, “Survey on Software Clone Detection
Research,” in Proc. International Conference on Management
Engineering, Software Engineering and Service Sciences, 2019,
pp. 9–16.
[6] I. Keivanloo, J. Rilling, and Y. Zou, “Spotting Working Code
Examples,” in Proc. International Conference on Software Engineering, 2014, pp. 664–675.
[7] N. Yoshida, S. Numata, E. Choiz, and K. Inoue, “Proactive
Clone Recommendation System for Extract Method Refactoring,” in Proc. International Workshop on Refactoring, 2019, pp.
67–70.
[8] M. Mondal, C. K. Roy, and K. A. Schneider, “An Exploratory
Study on Change Suggestions for Methods Using Clone Detection,” in Proc. International Conference on Computer Scienceand Software Engineering, 2016, pp. 85–95.
[9] A. Monden, S. Okahara, Y. Manabe, and K. Matsumoto,“Guilty or Not Guilty: Using Clone Metrics to Determine Open
Source Licensing Violations,” IEEE software, vol. 28, no. 2, pp.
42–47, 2010.

7