Tips on Building a Docker image (quickly) in a CI/CD Pipeline
Tags: docker, python, dotnet, java
Published: 2021-08-05 17:50:03

Building docker images has been evolving over the years and there are new strategies to do this.  One of the pain points that comes up frequently is the (lack of) speed when building an image on a CI/CD pipeline.  Many pipelines are now docker containers themselves and are a total blank slate.  We lose out on many optimizations if we don't realize it.

Below are a series of tips and tricks to make your docker image speed up in your build pipeline.  I have seen build times drop from 10's of minutes to less than 60 seconds, though your miles may vary!

- **Use `.dockerfileignore`**: The first step of every docker build is that docker will tarball the entire directory and
  send the data to docker.  This is output as the line `Sending build context to Docker daemon  XX.XXMB`  This should only be
  the source code and configuration files that are used in your build.  It really should not be more than 10's of MB.  Make
  sure to exclude everything that is not needed.
- Things that don't often change should be pushed up in the file.  The initial package installs, static environment variables,
  should be moved up.  Docker has a concept of caching, but the first line in the dockerfile that has a change, all lines
  after it will no longer be cached.  The last line of the dockerfile should be adding in your code, or compiling it!
  
  A typical file should have the following flow:
  ```dockerfile
  FROM base-cotnainer
  
  # Add all static environment variables, folders, users...
  ENV IS_APPLICATION=1
  RUN mkdir app
  WORKDIR /app  
  RUN useradd -ms /bin/sh newuser
  COMMAND ["python3", "main.py"]
  
  # Install all base image packages
  RUN apt install ...
  
  # Copy over your library depency files (Python example below)
  COPY Pipfile /app/Pipfile
  COPY Pipfile.lock /app/Pipfile.lock
  
  # Install your libraries (Python example below)
  RUN pipenv install --deploy
  
  # Lastly, copy the code over
  COPY . /app/
  ```
- Don't use a wildcard with the `COPY` command.  Docker will never cache that line. Be explicit!

- Build your containers once.  There should be zero reason to build your application multiple times.

- Build your containers at the same time.  If you have an application, test, integration, performance, et.al container,
  build them all as parallel tasks/steps/jobs in your pipeline.  Much of the docker builds is around I/O tasks and can and should be parallelized.

- Copy your library dependency file definitions manually and then run the library install before you copy your source code!  Doing
  this step first will allow you cache the pulling of libraries (which feels like a full download of the internet).  Below are
  language specific examples of doing this:
  
    - **Javascript**
      ```dockerfile
      FROM [...]
      mkdir /app
      WORKDIR /app
      # ...
      COPY package.json /app
      COPY package-lock.json /app
      RUN npm install
    
      COPY [all your code and configuration]    
      ```
  - **Python (requirements.txt)**
    ```dockerfile
    FROM [...]
    mkdir /app
    WORKDIR /app
    # ...
    COPY requirements.txt /app
    RUN pip3 install
      
    COPY [all your code and configuration]    
    ```
  - **Python (pipenv)**
    ```dockerfile
    FROM [...]
    mkdir /app
    WORKDIR /app
    # ...
    COPY Pipfile /app
    COPY Pipfile.lock /app
    RUN pipenv install --deploy
      
    COPY [all your code and configuration]    
    ```
  - **Java (Maven)**
    
    ```dockerfile
    FROM [...]
    mkdir /app
    WORKDIR /app
    # ...
    COPY pom.xml /app/pom.xml
    RUN mvn install
      
    COPY [all your code and configuration]    
    RUN mvn package
    ```
  - **Java (Gradle)**
    
    Add this to your `build.gradle` file: 
    ```
    task downloadDependencies(type: Exec) {
    configurations.testRuntime.files
    commandLine 'echo', 'Downloaded all dependencies'
    }
    ```
    Example Dockerfile to pull dependencies as a cached layer.
    ```dockerfile
    FROM [...]
    mkdir /app
    WORKDIR /app
    # ...
    COPY build.gradle /app/build.gradle
    RUN ./gradlew downloadDependencies
      
    COPY [all your code and configuration]    
    RUN ./gradlew build
    ```
  - **DotNet (Nuget)**
    ```dockerfile
    FROM [...]
    mkdir /app
    WORKDIR /app
    # ...
    # Copy all solution files .sln (NO WILDCARDS!)
    COPY WebApp.sln /app/WebApp.sln 
    # COPY all .csproj files (NO WILDCARDS!)
    COPY WebApp.Application/WebApp.Application.csproj /app/WebApp.Application/WebApp.Application.csproj
    # Yep.. gotta manually add them all (... Pokemon!)
    
    # This command will pull all library 
    RUN dotnet restore
      
    COPY [all your code and configuration]    
    ```
  - **Don't** pre-pull images.  It's slow and it doesn't work.  Using the new `BUILDKIT`
    defined below, Docker will be smarter and only pull the image layers that are cached.  This is faster,more
    efficient and it works.
- Use docker's new library `BUILDKIT` and inline cache information into your builds.  (This is a simple and big one)
There are three things we need to set to get proper image caching in a pipeline:
  1. Set the `DOCKER_BUILDKIT` environment variable to 1
  2. Add a build argument to tell the new BUILDKIT library to line caching information in each image layer: `--build-arg BUILDKIT_INLINE_CACHE=1`
  3. Tell the docker build command where you can get your image from to cache `--cache-from [remote-repo-url]/[CONTAINER_NAME]:latest`
The full command would look something like this: `DOCKER_BUILDKIT=1 docker build --build-arg BUILDKIT_INLINE_CACHE=1 --cache-from [remote-repo-url]/[CONTAINER_NAME]:latest` --tag [CONTAINER_NAME] .`

  Lastly, don't forget to push an updated image to your remote docker repo!