AWS SSM Parameters for Elixir secrets
Tags: elixir, aws, devops
Published: 2022-04-29 11:17:33

This is a very short article about using [AWS Systems Manager Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html) to load secret configuration in a simple way. One of the motivations for the approach I'm about to describe is the desire to avoid what I consider to be an anti-pattern. If you are not interested in that discussion, then feel free to skip to [the code](#orgabc0d87).


# The Anti-Pattern

Elixir 1.9 introduced ["releases"](https://github.com/elixir-lang/elixir/blob/v1.9/CHANGELOG.md#releases), and with it an [overhaul](https://github.com/elixir-lang/elixir/blob/v1.9/CHANGELOG.md#configuration-overhaul) of how configuration is done. Before this, all configuration was resolved at build time, something that was not a problem for those who ran their apps "from source", as "build time" and "run time" were effectively the same. But for those who wanted to ship a pre-built app, this caused problems.

```elixir
config :my_app,
  # when is this resolved?
  database_url: System.get_env("DATABASE_URL")
```

People solved this problem by adopting a pattern: set the variable to a *placeholder value* and defer the lookup till later.

```elixir
config :my_app,
  normal_var: "I am totally normal",
  deferred_var: {:system, "SEE_YOU_LATER"}
```

In order for this to work, the consuming code would have to recognise this pattern.

```elixir
def consume_config() do
  case Application.get_env(:my_app, :deferred_var) do
    {:system, var} -> System.get_env(var)
    value -> value
  end
end
```

The problem with this "anti-pattern" is it confused newcomers who wrongly assumed it was a feature of the configuration system itself, rather than what it actually was: an ad-hoc workaround, that had to be implemented for every library and variable one chose to support.

Happily, this is not a problem anymore, and the story around runtime configuration is much better: just put it all in `config/runtime.exs`.

Recently, I wanted to use AWS SSM to store secret configuration, and it looked like the way to go was to use [Config.Provider](https://hexdocs.pm/elixir/1.13.4/Config.Provider.html). Looking around to see how others had solved the same problem, a familiar pattern cropped up.

```elixir
config :my_app,
  some_var: {:ssm, "THIS_LOOKS_FAMILIAR"}
```

Because `Config.Provider` s run after `config/runtime.exs` is loaded, you needed some *placeholder value* that could later be detected and replaced.

This, I thought, was an instance of the same anti-pattern.

Deciding I didn't want to re-introduce the anti-pattern, I came up with a different approach.


<a id="orgabc0d87"></a>

# A different approach

First, add the required dependencies and declare a `Config.Provider`.

```elixir
defmodule Example.MixProject do
  def project do
    [
      ...
      releases: releases()
    ]
  end

  defp releases() do
    [
      example: [
        config_providers: [
          {Example.ConfigProvider, []}
        ]
      ]
    ]
  end

  defp deps do
    [
      ...
      {:ex_aws, "2.0"},
      {:ex_aws_ssm, "2.0"}
    ]
  end
end
```

And then, implement it as follows.

```elixir
defmodule Example.ConfigProvider do
  @behaviour Config.Provider

  @impl true
  def init([]) do
    []
  end

  @impl true
  def load(config, _state) do
    {:ok, _} = Application.ensure_all_started(:hackney)
    {:ok, _} = Application.ensure_all_started(:ex_aws)

    params = get_parameters(config[:ex_aws]) |> Enum.into(%{})

    Config.Reader.merge(config,
      example: [
        {ExampleWeb.Endpoint, secret_key_base: params["secret_key_base"]},
        {Example.Repo, url: params["database_url"]}
      ]
    )
  end

  defp get_parameters(config) do
    ExAws.SSM.get_parameters_by_path(prefix(), with_decryption: true)
    |> ExAws.request(config)
    |> case do
         {:ok, %{"Parameters" => params}} ->
           params
           |> Enum.map(fn param ->
             {String.trim_leading(param["Name"], prefix()), param["Value"]}
           end)

         error ->
           raise "Failed to fetch parameters: #{inspect(error)}"
       end
  end

  defp prefix() do
    "/example/prod/"
  end
end
```

Essentially, instead of using a generic `Config.Provider` that relies on *placeholder values* to do its job, I wrote a one-off `Config.Provider` that added the configuration I wanted.


# Regaining what was lost

This worked fine for me in the project I was working on, but looking back at it, I think it suffers from two problems.

There was something nice about the "wrong" approach: it was declarative and the configuration was all in one place.

I thought about an alternative that would help the code regain the clarity that was lost and this is what I came up with.

First, I started with what I thought was the simplest possible API.

```elixir
config :my_app,
  some_var: System.get_env("SOME_VAR"),
  database_url: Secret.get_env("DATABASE_URL")
```

Just replace `System.get_env(...)` with `Secret.get_env(...)`.

And got the following (admittedly rough) code working.

```elixir
defmodule Secret do
  def get_env(key, default \\ nil) do
    Map.get(get_all_env(), key, default)
  end

  defp get_all_env() do
    case :ets.whereis(__MODULE__) do
      :undefined ->
        :ets.new(__MODULE__, [:set, :public, :named_table])

        {:ok, _} = Application.ensure_all_started(:hackney)
        {:ok, _} = Application.ensure_all_started(:ex_aws)

        env = fetch_env()
        :ets.insert(__MODULE__, {:env, env})
        env

      ref ->
        [{:env, env}] = :ets.lookup(ref, :env)
        env
    end
  end

  defp fetch_env() do
    config = [
      region: System.fetch_env!("AWS_REGION"),
      access_key_id: System.fetch_env!("AWS_ACCESS_KEY_ID"),
      secret_access_key: System.fetch_env!("AWS_SECRET_ACCESS_KEY")
    ]

    get_parameters(config) |> Enum.into(%{})
  end

  defp get_parameters(config) do
    # same as above
  end
end
```

```shell
$ aws ssm put-parameter \
  --name "/example/prod/DATABASE_URL" \
  --value "ecto://postgres:postgres@localhost/example_prod" \
  --type SecureString \
  --overwrite
```

```elixir
> Secret.get_env("DATABASE_URL")

"ecto://postgres:postgres@localhost/example_prod"
```

I have yet to switch to this approach, but it is something I will consider next time I need to accomplish the same task.