Fixing slow down speeds w/ curl in alpine docker images
Tags: docker, productivity, devops, linux
Published: 2019-11-21 20:21:47

# The problem

After playing around with using the jenkins:alpine base image and attempting to download a .war (60 MB) file using curl, I discovered that it was EXTREMELY slow. I am talking about the difference between it taking a minute and a half on my host machine and 15 mins during the docker image build.

```dockerfile
FROM jenkins:alpine

ARG JENKINS_VERSION
ENV JENKINS_WAR="/usr/share/jenkins/jenkins.war"

USER root
RUN curl -o ${JENKINS_WAR} -L \
	https://updates.jenkins-ci.org/download/war/${JENKINS_VERSION}/jenkins.war
RUN chown jenkins:jenkins ${JENKINS_WAR}
USER jenkins
```

Obviously, this is ridiculous and there had to be something wrong. After searching online, I found there was very little documentation on how to resolve this issue, so I started doing my own experimenting.

####Overview
downloading .war of 60 MB took
- minute and a half on host machine
![host machine](https://thepracticaldev.s3.amazonaws.com/i/bjkpo00e3jrz6ddzrclj.PNG)
- 15 min during docker alpine image build
![alpine slow](https://thepracticaldev.s3.amazonaws.com/i/b5f7m5hceuhk59o4xijm.PNG)

####NOTICE
- made some updates were made on 12.4.2019

# What did NOT work :thumbsdown:

###Adding `docker build ...` options
I was unable to get any increase from adding in options to `docker build ...` such as ...
- `--network host` : performs the commands using the host network
- `-m 1GB` : allocates more memory to the image build

###Fixing alpine DNS issues
I also read online about some DNS issues that alpine has and followed some of their instructions for fixing them. None of that helped increase download speeds.

###Using the docker "ADD" command (12.4.2019)
Using `ADD` is easily the cleanest and most straightforward option. However, I found it was just as slow as using curl. See the secondary solution at the end of this article to see how I was able to still use the simplicity of `ADD` and while also getting an increase in speed.
```dockerfile
FROM jenkins:alpine
...
ADD https://updates.jenkins-ci.org/download/war/${JENKINS_VERSION} ${JENKINS_WAR}
...

```

###Changing image base _(some success)_
I noticed that when switched out from an alpine based image, I was able to significantly increase my download speeds. However, it still wasn't as fast as my host machine.

Notice the change in "TAG" in first line the `FROM jenkins:TAG` from "alpine" to "latest". The "latest" tag does not use alpine.

```dockerfile
FROM jenkins:latest

ARG JENKINS_VERSION
ENV JENKINS_WAR="/usr/share/jenkins/jenkins.war"

USER root
RUN curl -o ${JENKINS_WAR} -L \
	https://updates.jenkins-ci.org/download/war/${JENKINS_VERSION}/jenkins.war
RUN chown jenkins:jenkins ${JENKINS_WAR}
USER jenkins
```

![non alpine faster](https://thepracticaldev.s3.amazonaws.com/i/3vqmrpxc7jxhksyqbcvg.PNG)

###Increasing curl limit rate _(some success)_
Using the alpine base image, I found that setting `--limit-rate 1G` as a `curl ...` option increased the download speed. I could have stopped at this point, but the download was still taking about 1-5 mins. It seemed rather unstable to me, so I kept experimenting.

```dockerfile
FROM jenkins:alpine

ARG JENKINS_VERSION
ENV JENKINS_WAR="/usr/share/jenkins/jenkins.war"

USER root
RUN curl --limit-rate 1G -o ${JENKINS_WAR} -L \
	https://updates.jenkins-ci.org/download/war/${JENKINS_VERSION}/jenkins.war
RUN chown jenkins:jenkins ${JENKINS_WAR}
USER jenkins
```

![Alt Text](https://thepracticaldev.s3.amazonaws.com/i/s58w9n6g6skve9gjcr1s.PNG)


# The solution :smile:
I found that by combining switching the image base (from alpine to debian) and setting `--limit-rate 1G` curl option I was able to make the download happen in **2 seconds**.

I still wanted to use alpine as my base image, so I changed the dockerfile to use multi-stage.

```dockerfile
FROM debian:buster-slim as downloader

RUN apt update && apt install curl -y

USER root
ARG JENKINS_VERSION
WORKDIR /home/root

# running curl inside the debian image instead of alpine
RUN curl --limit-rate 1G -o jenkins.war -L \
	https://updates.jenkins-ci.org/download/war/${JENKINS_VERSION}/jenkins.war


FROM jenkins:alpine

ENV JENKINS_WAR="/usr/share/jenkins/jenkins.war"
# getting the downloaded .war file from the debian image
COPY --from=downloader /home/root/jenkins.war ${JENKINS_WAR}

USER root
RUN chown jenkins:jenkins ${JENKINS_WAR}
USER jenkins

```

![debian image fastest](https://thepracticaldev.s3.amazonaws.com/i/dps7mhqd1qi1slfqliqe.PNG)

# Another _better_ solution :rocket: (12.4.2019)
It suddenly hit me that the `ADD` or `COPY` docker commands _must_ have the capability to use URLs. Once I confirmed that `ADD` can do this, I felt so dumb. When I first tried using `ADD` to download the jenkins.war file it was still very slow :cry: 

BUT, then I thought, "what about switching the base image and then using ADD?". And what do you know, it was FAST :smile: It took about 3 seconds to download. This is more simple because the code is easier to follow and it removes the need to run `apt update` and `apt install curl`. This results in a smaller image overall.


```dockerfile
FROM debian:buster-slim as downloader

ARG JENKINS_VERSION
WORKDIR /home/root
ADD https://updates.jenkins-ci.org/download/war/${JENKINS_VERSION}/jenkins.war \
jenkins.war

FROM jenkins:alpine

ENV JENKINS_WAR="/usr/share/jenkins/jenkins.war"
COPY --from=downloader /home/root/jenkins.war ${JENKINS_WAR}

USER root
RUN chown jenkins:jenkins ${JENKINS_WAR}
```