Humpback: Code Completion System for Dockerfiles
Based on Language Models
Kaisei Hanayamaa , Shinsuke Matsumotoa and Shinji Kusumotoa
aGraduate School of Information Science and Technology, Osaka University, Suita, Osaka, 565–0871, Japan

Abstract

The object of this study is Docker, the de facto standard containerization platform. Containers in Docker are built by creating files called Dockerfiles. Managing the infrastructure as code makes it possible to incorporate knowledge gained fromconventional software development. However, infrastructure as code is a relatively new technology, some domains of whichhave not been fully researched. In this study, we focus on code completion and aim to construct a system that supports thedevelopment of Dockerfiles. The proposed code completion system, Humpback, applies machine learning to a pre-collected
dataset with long short-term memory to create language models and uses model switching to overcome a Docker-specificcode completion problem. Evaluation experiments show that Humpback has a high average accuracy of 96.9%.

Keywords

Docker, code completion, machine learning, language model, long short-term memory

1. Introduction

on which a container is created. Dockerfiles can have anested language; embedded scripting languages (mainlybash) are described in a nested state in the top-level syntax [1]. The contents of Dockerfiles differ considerably
depending on the base image. For example, for an baseimage includes Ubuntu, the apt-get command is usedin the RUN instruction, whereas for a CentOS base image,the dnf command is used. For accurate code completion,base image differences must thus be taken into account.
The contributions of this paper are as follows:
1. Model switching is used to overcome a Dockerspecific problem. We introduce model switching toovercome the problem caused by base image differences.
With model switching, language models for predictions
are switched depending on the base image. Long shortterm memory (LSTM) [6] is employed to generate language models (section 3.2).
2. A Docker-specific code completion system, Humpback, is implemented. Figure 1 shows a screenshot of
Humpback. Humpback is available online and can beused in a web browser.2 Evaluation experiments showthat Humpback has a high accuracy of 96.9% and is usefulfor developing Dockerfiles (section 4.4).

Server virtualization is broadly used for cost reductionand efficient resource utilization. Among various methods of virtualization, containerization has become mainstream [1]. Containerization creates logical compartments (i.e., containers) on the host operating system.
Each container provides an independent environment.
Docker1 is the de facto standard containerization platform [2]. Containers in Docker are configured by writing imperative instructions in files called Dockerfiles.
The process of managing infrastructure configuration
through machine-readable definition files is called infrastructure as code (IaC). IaC enables developers to manage
infrastructure configuration in the same way as application code, allowing automated scaling and the preventionof human error [3]. However, IaC is a relatively new technology and thus some areas are still in progress [4], suchas development support and static analysis.
In this study, we focus on code completion, a widelyused feature in software development [5]. We believethat providing a code completion system for an emerging
technology such as Docker can considerably improve
productivity, by reusing existing knowledge, and reduce
common errors.
One concern when building a Docker-specific code 2. Backgroundcompletion system is base image differences. A base image, which includes a Linux distribution, is an image file 2.1. Code completion
NLPaSE 2020: Natural Language Processing Advancements for
Software Engineering, December 01, 2020, Singapore
email: k-hanaym@ist.osaka-u.ac.jp (K. Hanayama);
shinsuke@ist.osaka-u.ac.jp (S. Matsumoto);
kusumoto@ist.osaka-u.ac.jp (S. Kusumoto)© 2020 Copyright for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).

CEUR

Workshop
Proceedings

CEUR Workshop Proceedings (CEUR-WS.org)
https://www.docker.com/

Code completion is extensively used in software development [5]. A pop-up dialog is used to display candidate
words after the user has typed some characters. Developers select the desired word from the list, reducing typos
and other common errors. Another benefit is the faciliation of the use of descriptive (i.e., long) names for

http://ceur-ws.org
ISSN 1613-0073

12

https://sdl.ist.osaka-u.ac.jp/~k-hanaym/humpback/support and static analysis, can be applied to IaC.

3. Humpback: code completion
system for Dockerfiles
3.1. System overview
We propose Humpback, a code completion system forDockerfiles. Humpback helps developers to reduce errors and enhance efficiency when writing Dockerfiles.
Various methods have been used to implement code comFigure 1: Screenshot of Humpback
pletion systems. Here, we employ language models. Statistically processing pre-collected Dockerfiles and performing contextual predictions makes it possible to reusevariables. Manually entering long variable names is cum- existing knowledge. We also introduce model switching
to overcome the problem, caused by base image differbersome and error-prone.
Traditional code completion systems display all candi- ences.
dates, which can be extremely long list. A large numberof intelligent code completion systems have been pro- 3.2. Methodology
posed to overcome this problem [7]. Systems that use
statistical language models such as N-gram and recurrent The methodology of Humpback is divided into the learnneural network (RNN)-based approaches have achieved ing phase and the prediction phase.

high performance. Given token sequence 𝑤 of length 𝑚,
the language model gives the probability 𝑃 (𝑤1 , ..., 𝑤𝑚 ).
This probability indicates the relative likelihood of words,
which allows the construction of code completion systems. Intelligent code completion considers the context
and calculates probabilities based on language models
to narrow the list of candidate words. Compared to a
traditional code completion system, an intelligent one
more effectively enhances developer productivity.

2.2. Docker, infrastructure as code, andchallenges

3.2.1. Learning phase
The learning phase includes file collection, data processing, and language model generation. Figure 2 shows anoverview of the learning phase.
File collection: We search for GitHub repositories
with Dockerfiles using the GitHub API3 , pull these repositories in order of their star count (i.e., popularity), andextract the Dockerfiles.
Data processing: The contents of the collected Dockerfiles are divided into token sequences. The inputs
are paired with the expected outputs. For example, if
there is a statement FROM centos RUN dnf, then FROM
expects centos and FROM centos expects RUN. Next,
these data are encoded using integer values for the learnerto interpret efficiently. The number of elements in thetraining data varies. Therefore, 0-padding is performed
to obtain fixed-length data.
Language model generation: Humpback uses language models for prediction. We assume that the contents of Dockerfiles are time-series data. LSTM [6], animproved RNN architecture used in the field of deeplearning, is employed to generate language models. The
middle layer of the RNN is replaced with LSTM blocks,
which allow for learning with long-term dependency.

Docker, an open containerization platform, isolates applications from the development environment with containers, allowing efficient resource utilization. Docker has
become the de facto standard container technology; over87% of information technology companies use Docker [2].
Containers in Docker can be built by interactively executing commands or by creating configuration files called
Dockerfiles. Dockerfiles set up containers through imperative instructions, enabling reproducible builds. A
process for specifing the environment in which softwaresystems will be tested and/or deployed by configuration
scripts is called IaC. Developers can manage infrastructure configuration in the same way as application code,
allowing automated scaling and the prevention of human
3.2.2. Prediction phaseerror [3]. Interest in IaC has thus grown [8].
Research on IaC is still in its infancy [4]. There are rela- Humpback uses model switching to overcome the probtively few studies on IaC, and most of them propose tools lem caused by base image differences. Pre-trained lanfor implementing the practices of IaC itself. Knowledge
3in software engineering, such as that on development
https://docs.github.com/en/graphql

…

GitHub API
Training data

Dockerfiles

File collection

Data processing

FROM: 1
RUN : 3…

FROM centos RUN

1 25

3

FROM centos RUN dnf

1 253 52

1. Divide the contentsof Dockerfiles
and convert them into
Input Expected Output

Language model

Language model generation

1 25

FROM centos

LSTM

2. Encode training datawith integer values

00

1 250

1 25

3

1 25 3 52

3. Padding by 0 to maketraining data of
fixed length

Figure 2: Overview of learning phase# of versions# of epochs

Durationframework. Candidate words are presented immediately,
and thus developers can use Humpback without slowing
down their development process.

6199-

815594-3d15h12m
2h30m
3h00m
-4. Evaluation Experiment

Table 1
Details of dataset and learning
Distribution
DebianUbuntu
AlpineOthers# of Dockerfiles
17,011 (80.2%)
1,497 (7.0%)
1,105 (5.2%)
1,577 (7.4%)

guage models for each base image are prepared in advance. Humpback switches models for prediction depending on the input data. For instance, if the base imageof input data is Ubuntu, a model trained with Dockerfiles
whose base images are Ubuntu is used.
However, it is impossible to identify the Linux distribution from the base image name in some cases. For
example, we can guess that “openjdk:11-jdk” will include the Java development environment, but cannot
guess its Linux distribution. We created a base image
detector to determine the Linux distribution for a thegiven Dockerfile. First, the base image detector builds
a container from the Dockerfile. Then, it identifies thedistribution based on the /etc/os-release file. We
analyzed the base images of the entire dataset (section
4.2). With these results, Humpback can switch models forprediction even if the Linux distribution is not explicitlyspecified. For example, the base image detecor identified
the distribution of openjdk:11-jdk as Debian.

4.1. Evaluation metrics
We conducted evaluation experiments to verify that model
switching improves the accuracy of code completion.
Top-k accuracy (𝐴𝑐𝑐(𝑘)) and the mean reciprocal rank(𝑀 𝑅𝑅) [9] are used as metrics for evaluating accuracy:∑︀|𝑄| 1𝑁1𝐴𝑐𝑐(𝑘) = 𝑡𝑜𝑝−𝑘, 𝑀 𝑅𝑅 = |𝑄|
𝑖=1 𝑟𝑎𝑛𝑘𝑖 where|𝑄|
𝑁𝑡𝑜𝑝−𝑘 refers to the number of relevant recommendations in top 𝑘 suggestions, |𝑄| represents the total number of queries, and 𝑟𝑎𝑛𝑘𝑖 denotes the rank position of thefirst relevant word for the 𝑖-th query. For both 𝐴𝑐𝑐(𝑘)and 𝑀 𝑅𝑅, a value closer to 1 indicates better modelperformance.

4.2. Dataset

We collected 21,190 Dockerfiles using the GitHub API.
The numbers of Dockerfiles and their versions for various
Linux distributions are shown on the left side of Table 1.
The major distributions in the dataset are Alpine Linux,Debian GNU/Linux, and Ubuntu. The dataset for Ubuntu
has the most variety, with 19 versions in 1,497 files. In the3.3. Implementationtable, “Others” includes Amazon Linux, CentOS, Fedora,
Three libraries/frameworks are used to implement Hump- Oracle Linux Server, and VMware Photon OS/Linux.
The number of epochs and the learning duration areback, namely TensorFlow4 , a software library for machine learning, Keras5 , a high-level neural network li- shown on the right side of Table 1. Hyperparameters suchbrary, and Optuna6 , a hyperparameter auto-optimization as the activation/optimization function, and number ofunits in each layer were optimized using Optuna.
456

https://www.tensorflow.org/
https://keras.io/
https://preferred.jp/en/projects/optuna/Table 2
Average scores in experiment (Gen.: generic model, Hump.: Humpback)Distribution
AlpineDebianUbuntuTop-1 accuracy
Gen.
Hump.

Docker syntax
Top-5 accuracy
Gen.
Hump.

94.2%
97.2%
91.9%

98.3%
98.0%
98.9%

96.2%
98.8%
96.9%

98.0%
99.5%
99.4%

MRR
Gen.
Hump.

Top-1 accuracy
Gen.
Hump.

Shell syntax
Top-5 accuracy
Gen.
Hump.

Gen.

0.96110.9762
0.9542

92.4%
95.5%
96.6%

96.8%
98.3%
98.9%

0.94330.96890.97664.3. Experiment design
We compared the recommendation accuracy for the threemajor distributions in the dataset, both with and withoutmodel switching. For the case without model switching,we created a generic model that was trained with all
Dockerfiles. Two syntaxes were defined; descriptionsin the RUN instruction were defined as Shell syntax andother descriptions were defined as Docker syntax. There
were three axes of comparison: the presence or absence ofmodel switching, the Linux distribution, and the syntax.
We first extracted 100 Dockerfiles from the dataset
and set the correct answer to a random position in eachDockerfile. Next, the contents from the beginning of thefile to just before the correct answer were given to thelanguage models and predictions were generated. Then𝐴𝑐𝑐(𝑘) and 𝑀 𝑅𝑅 were computed by comparing thepredictions against the correct answer. Ten rounds of theabove process were performed for each comparison axis.

4.4. Experiment results
Table 2 shows the average scores of 𝐴𝑐𝑐(1), 𝐴𝑐𝑐(5), and𝑀 𝑅𝑅. “Gen.” refers to the generic model (i.e., withoutmodel switching). “Hump.” refers to Humpback (i.e., withmodel switching). The numbers in bold indicate the bestscores in a given category.
Prediction with Humpback is more accurate for almost
all evaluation axis. Model switching is thus beneficial
for building Docker-specific code completion systems.
Humpback achieved an outstanding average Top-1 accuracy of 96.9% (up to 98.8% for Debian, Docker syntax).
Moreover, the accuracy improved by up to 5.0% (Ubuntu,Docker syntax) compared to that for the generic model.
As described in section 3.3, the candidate words are presented instantly. With its quickness and high accuracy,
Humpback can significantly improve productivity.

5. Conclusion
In this study, we proposed Humpback, a code completion
system for Dockerfiles. Humpback is available online
and can be used in a web browser. We introduced model
switching to overcome a Docker-specific problem. Evaluation experiments showed that Humpback has a highaverage accuracy of 96.9%, and that model switching
improves the accuracy of Humpback. In future work,

0.9706
0.99130.9810

94.4%
97.5%
97.7%

96.9%
99.5%
99.5%

MRR
Hump.
0.95650.98430.9855we will further improve the accuracy of Humpback andcompare Humpback with other code completion systems.

Acknowledgments
This work was supported in part by MEXT/JSPS KAKENHI Grant No. 18H03222.

References[1] J. Henkel, C. Bird, S. K. Lahiri, T. Reps, A dataset ofdockerfiles, in: International Working Conferenceon Mining Software Repositories, 2020, pp. 1–5.
[2] Portworx, Container adoption survey, 2019. URL:
https://portworx.com/wp-content/uploads/2019/05/2019-container-adoption-survey.pdf.
[3] M. Artac, T. Borovssak, E. Di Nitto, M. Guerriero,
D. A. Tamburri, Devops: Introducing infrastructureas-code, in: International Conference on Software
Engineering Companion, 2017, pp. 497–498.
[4] A. Rahman, R. Mahdavi-Hezaveh, L. Williams, A
systematic mapping study of infrastructure as coderesearch, Information and Software Technology 108(2019) 65–77.
[5] M. Bruch, M. Monperrus, M. Mezini, Learningfrom examples to improve code completion systems,
in: European Software Engineering Conference andSymposium on the Foundations of Software Engineering, 2009, pp. 213–222.
[6] F. A. Gers, J. Schmidhuber, F. Cummins, Learning
to forget: continual prediction with lstm, in: International Conference on Artificial Neural Networks,
volume 2, 1999, pp. 850–855.
[7] A. Svyatkovskiy, S. Fu, Y. Zhao, N. Sundaresan,
Pythia: Ai-assisted code completion system, in: International Conference on Knowledge Discovery andData Mining, 2019, pp. 2727–2735.
[8] C. Parnin, E. Helms, C. Atlee, H. Boughton, M. Ghattas, A. Glover, J. Holman, J. Micco, B. Murphy, T. Savor, M. Stumm, S. Whitaker, L. Williams, The top 10
adages in continuous deployment, IEEE Software 34(2017) 86–95.
[9] D. R. Radev, H. Qi, H. Wu, W. Fan, Evaluating webbased question answering systems, in: International
Conference on Language Resources and Evaluation,
2002, pp. 1153–1156.