Working with Elixir Releases and performing CI/CD in containers
Tags: elixir, phoenix, docker, ci
Published: 2021-01-02 01:35:11


*Originally published at [https://geeks.wego.com](https://geeks.wego.com/elixir-releases-ci-cd-in-containers/) and  https://darnahsan.medium.com/working-with-elixir-releases-and-performing-ci-cd-in-containers-383ba03b6768 on December 24, 2020.*

Elixir has seen fast adoption in [Wego](https://www.wego.com) with multiple services already in production after the weekend project [UltronEx](https://dev.to/darnahsan/ruby-vs-elixir-performance-ultron-is-dead-long-live-ultronx-42ao) became a daily driver for [monitoring real time data](https://dev.to/darnahsan/real-time-dashboard-with-websocket-in-elixir-5ald) it has spurred a movement to explore new options and build robust resilient systems.

As We run almost all our work loads in containers we take advantage of the same build pipelines for building container images to run CI/CD as well. Elixir runs on [BEAM](https://blog.erlang.org/a-brief-BEAM-primer/) and has some nice features such as hot redeploys that is something anti to the concept of containers where they are immutable images none the less it does not mean Elixir is not container friendly. Mix, the elixir build tool similar to what Rake is for Ruby and has great support to generate a bundle with [ERTS](https://erlang.org/doc/apps/erts/index.html) called a [Release](https://elixir-lang.org/getting-started/mix-otp/config-and-releases.html) that you can take and run anywhere as long as it is the same platform it was built on. This is what works really well with containers keeping the image size significantly small without any external dependencies. Even though this makes the Elixir Release easy to work with it also strips it of its tool chain that you require to perform actions such as running migration or executing tasks as there is no Mix available in a Release. The below chart shows the different approaches and their features

![](https://cdn-images-1.medium.com/max/2000/1*obH6IHAdpZ7f2gIwyPxgjw.png)

One might ask why use a Release then for that there is a great thread on Elixir forum on why to [Always use Releases](https://elixirforum.com/t/always-use-releases/4573/21). Let’s say If you were to package your app with Mix and run it using mix run --no-halt to overcome issues with missing Mix what difference does it make on the container size ? The elixir image for 1.11.2-erlang-23.1.3 is about 214MB which is used as the base to either build a release in a multi stage Dockerfile or use it to run the app using Mix. Compared to that Debian image is only 69.2MB. When we use an Elixir Release for a Phoenix app and copy that to a stage based on Debian the final image size comes to 101MB whereas if we were to use Mix and not generate a release the final image size is 627MB a whopping 6X the release image size.

![](https://cdn-images-1.medium.com/max/2400/0*bhEOsbFr_KDeTtwe.png)

![](https://cdn-images-1.medium.com/max/2400/0*QLZhL17fymeoYMaw.png)

![](https://cdn-images-1.medium.com/max/2400/0*dQEXLDfZSieJEx7h.png)

![](https://cdn-images-1.medium.com/max/2400/0*EjYkDkwOMRVJCW7N.png)

That is some significant advantage releases have over other approaches and also goes on to show the impact multi stage build files can have over image sizes.

So that sorts that we should always use Releases but how to then get the lost functionality of Mix ? For this Elixir and Docker both do their part to seamlessly make things similar to what you could do with a Ruby CI/CD containerised process with rake. For running [database migration with releases ](https://hexdocs.pm/phoenix/releases.html#ecto-migrations-and-custom-commands)Elixir has a work around with [Ecto Migrator](https://hexdocs.pm/ecto_sql/Ecto.Migrator.html) where in you create a separate file that executes the migrations, it is a standardised code and should actually IMO be part of Ecto

    defmodule MyApp.Release do
      [@app](http://twitter.com/app) :my_app

    def migrate do
        load_app()

    for repo <- repos() do
          {:ok, _, _} = Ecto.Migrator.with_repo(repo, &Ecto.Migrator.run(&1, :up, all: true))
        end
      end

    def rollback(repo, version) do
        load_app()
        {:ok, _, _} = Ecto.Migrator.with_repo(repo, &Ecto.Migrator.run(&1, :down, to: version))
      end

    defp repos do
        Application.fetch_env!([@app](http://twitter.com/app), :ecto_repos)
      end

    defp load_app do
        Application.load([@app](http://twitter.com/app))
      end
    end

After creating your release you can run you migrations in production with eval for e.g

    $ _build/prod/rel/my_app/bin/my_app eval "MyApp.Release.migrate"

Migration is a special case and Ecto has some nice features to access the Database to make this all possible but for running some tasks in production environment requires a separate approach and some understanding of an Elixir application or for a web application built in Phoenix. Every Elixir application is built of multiple applications where a Phoenix app is a list of applications where Web and Database are separate applications which also gives Elixir applications its robustness where any single application can not bring down your entire Service. For this you can take lead from the [elixir release docs](https://hexdocs.pm/phoenix/releases.html) and base understanding of elixir application to formulate your application container image to be used to execute tasks and run the application service as well. Let us take a Phoenix app GhostRider as an example. Here we split the application children into two separate lists base and web . Based on what is required we can handle what children load from an environment variable which can conveniently be passed to a container at runtime.

    defmodule GhostRider.Application do
      # See [https://hexdocs.pm/elixir/Application.html](https://hexdocs.pm/elixir/Application.html)
      # for more information on OTP Applications
      [@moduledoc](http://twitter.com/moduledoc) false

    use Application

    def start(_type, _args) do
        base = [
          # Start the Ecto repository
          GhostRider.Repo,
          {Finch, name: GhostRiderFinch}
        ]

    web = [
          # Start the Telemetry supervisor
          GhostRiderWeb.Telemetry,
          # Start the PubSub system
          {Phoenix.PubSub, name: GhostRider.PubSub},
          # Start the Endpoint (http/https)
          GhostRiderWeb.Endpoint
          # Start a worker by calling: GhostRider.Worker.start_link(arg)
          # {GhostRider.Worker, arg},
        ]

    children =
          case System.get_env("APP_LEVEL") do
            "TASK" -> base
            _ -> base ++ web
          end

    # See [https://hexdocs.pm/elixir/Supervisor.html](https://hexdocs.pm/elixir/Supervisor.html)
        # for other strategies and supported options
        opts = [strategy: :one_for_one, name: GhostRider.Supervisor]
        Supervisor.start_link(children, opts)
      end

    # Tell Phoenix to update the endpoint configuration
      # whenever the application is updated.
      def config_change(changed, _new, removed) do
        GhostRiderWeb.Endpoint.config_change(changed, removed)
        :ok
      end
    end

Though this approach would work but applications only start when a release is started. To overcome this we need to use the eval to execute a wrapper function bound to perform operation but in order to execute that operation we need to start our application using Application.ensure_all_started(@app) along with the environment variable controlling which children should be loaded.

![With all children loaded for the application](https://cdn-images-1.medium.com/max/2000/1*W7c6WpwnyFVqAts9X-xz2A.png)*With all children loaded for the application*

![With limited children loaded for the application](https://cdn-images-1.medium.com/max/2000/1*wMnK2kMqznTlTtRfJQqYmA.png)*With limited children loaded for the application*

With Migrations and Task sorted and capable of working with containers the last piece of the CI/CD puzzle is how do you run your test when there is no Mix in your release and the release is byte code and no test folder is available ? This is where docker shines with its multi stage builds and we leverage docker-compose to target a stage to be able to execute tests in the build to ensure they are consistent across different technology stacks we have.

Now some docker and docker-compose recipes to bring this all together to work in a CI/CD pipeline. One thing I have to admit is lack of resources around containerising Elixir/Phoenix applications, something the community should work towards improving. Below is a file for building a Phoenix application and you can see there is a stage called tester this is what we will target using docker-compose

    # [https://elixirforum.com/t/could-use-some-feedback-on-this-multistage-dockerfile-1st-elixir-phoenix-deployment/30862/10](https://elixirforum.com/t/could-use-some-feedback-on-this-multistage-dockerfile-1st-elixir-phoenix-deployment/30862/10)?

    ########################
    ### Dependency stage ###
    ########################
    FROM hexpm/elixir:1.11.2-erlang-23.1.3-debian-buster-20201012 AS base

    # install build dependencies
    RUN apt-get -qq update && \
        apt-get -qq -y install build-essential npm git python --fix-missing --no-install-recommends

    # prepare build dir
    WORKDIR /app

    ARG MIX_ENV
    ARG RELEASE_ENV

    ENV LANG=C.UTF-8 LC_ALL=C.UTF-8

    # Update timezone
    ENV TZ=Asia/Singapore

    # install hex + rebar
    RUN mix local.hex --force && \
        mix local.rebar --force

    # set build ENV
    ENV MIX_ENV=${MIX_ENV}
    ENV RELEASE_ENV=${RELEASE_ENV}

    COPY mix.exs mix.lock ./
    COPY config config

    # install mix dependencies
    RUN mix deps.get --only ${MIX_ENV}
    RUN mix deps.compile

    COPY lib ./lib
    COPY priv ./priv
    #########################
    ####### Test Stage ######
    #########################

    FROM base as tester
    WORKDIR /app
    COPY test test

    ########################
    # Build Phoenix assets #
    ########################
    # Using stretch for now because it includes Python
    # Otherwise you get errors, could use a smaller image though
    FROM node:14.15.3-stretch AS assets
    WORKDIR /app

    COPY --from=base /app/deps /app/deps/
    COPY assets/package.json assets/package-lock.json ./assets/
    RUN npm --prefix ./assets ci --progress=false --no-audit --loglevel=error

    COPY lib ./lib
    COPY priv ./priv
    COPY assets/ ./assets/

    RUN npm run --prefix ./assets deploy

    #########################
    # Create Phoenix digest #
    #########################
    FROM base AS digest
    WORKDIR /app

    # set build ENV
    ENV MIX_ENV=${MIX_ENV}
    ENV RELEASE_ENV=${RELEASE_ENV}

    COPY --from=assets /app/priv ./priv
    RUN mix phx.digest

    #######################
    #### Create release ###
    #######################
    FROM digest AS release
    WORKDIR /app

    ARG MIX_ENV
    ARG RELEASE_ENV
    ENV MIX_ENV=${MIX_ENV}
    ENV RELEASE_ENV=${RELEASE_ENV}

    COPY --from=digest /app/priv/static ./priv/static

    RUN mix do compile, release

    #################################################
    # Create the actual image that will be deployed #
    #################################################
    FROM debian:buster-slim AS deploy

    # Install stable dependencies that don't change often
    RUN apt-get update && \
      apt-get install -y --no-install-recommends \
      apt-utils \
      openssl \
      curl \
      wget && \
      rm -rf /var/lib/apt/lists/*

    # Set WORKDIR after setting user to nobody so it automatically gets the right permissions
    # When the app starts it will need to be able to create a tmp directory in /app
    WORKDIR /app

    ARG MIX_ENV
    ARG RELEASE_ENV

    ENV MIX_ENV=${MIX_ENV}
    ENV RELEASE_ENV=${RELEASE_ENV}

    ENV LANG=C.UTF-8 LC_ALL=C.UTF-8

    # Update timezone
    ENV TZ=Asia/Singapore

    HEALTHCHECK --start-period=10s \
      --interval=15s \
      --timeout=5s \
      --retries=3 \
      CMD curl -sSf [http://localhost:8080/heartbeat](http://localhost:8080/heartbeat) || exit 1

    COPY --from=release /app/_build/${MIX_ENV}/rel/ghost_rider ./

    ENV HOME=/app
    # Exposes port to the host machine
    EXPOSE 8080

    CMD ["bin/ghost_rider", "start"]

A standard docker-compose file to run the application would be similar to this which would spin up a Postgres database and start the application for you.

    version: "3.7"
    services:
      ghost-rider:
        build:
          context: .
          args:
            MIX_ENV: "${MIX_ENV}"
            RELEASE_ENV: "${RELEASE_ENV}"
        image: wego/ghost-rider
        depends_on:
          - ghost-rider-db
        restart: on-failure
        ports:
          - "8080:8080"
        environment:
          MIX_ENV: "${MIX_ENV}"
          RELEASE_ENV: "${RELEASE_ENV}"
          PORT: "8080"
          SECRET_KEY_BASE: "${SECRET_KEY_BASE}"
          DB_NAME: ghost-rider
          DB_PASSWORD: ghost-rider
          DB_HOST: ghost-rider-db
          APP_LEVEL: WEB

    ghost-rider-db:
        image: postgres
        restart: always
        ports:
          - 5432:5432
        environment:
          POSTGRES_PASSWORD: ghost-rider
          POSTGRES_DB: "${POSTGRES_DB}"
          PGDATA: /var/lib/postgresql/data/pgdata
        volumes:
          - "postgresql-data:/var/lib/postgresql/data"

    volumes:
      postgresql-data:

To build and run your container image you would execute

    $ docker-compose -f docker-compose.yml up -d

To run your migration

    $ docker-compose run --rm ghost-rider bin/ghost_rider eval "Release.migrate"

To run the test you would need to create another docker-compose file to override the default values and define the target it should execute up to. If you want you could create a completely new file as well

    version: "3.7"
    services:
      ghost-rider:
        build:
          target: tester
          args:
            - MIX_ENV=test
            - RELEASE_ENV=test
        image: wego/ghost-rider-test
        ports:
          - "8080:8080"
        environment:
          - MIX_ENV=test
          - RELEASE_ENV=test
          - DB_NAME=ghost-rider-test
          - DB_PASSWORD=ghost-rider
          - DB_HOST=ghost-rider-db

With this you can setup your test database and run test simply by doing

    $ docker-compose -f docker-compose.yml -f docker-compose.test.yml run --rm ghost-rider mix ecto.setup

    $ docker-compose -f docker-compose.yml -f docker-compose.test.yml run --rm ghost-rider mix test

All it takes is some understanding of Elixir and it can be used with Docker tools as any other language to build images and perform CI/CD without any friction.

I hope this helps with containerising Elixir and Phoenix applications as they are amazing technologies along with Docker and containers that make things really simple build pipelines for CI/CD.