Using Elixir's "with" statement.
Tags: elixir, functional, programming, tutorial
Published: 2021-12-07 12:22:04

Elixir has many features to make code readable, maintainable, and correct. One of them is the `with` statement.   Unfortunately, the `with` statement can be confusing to work with, because there are some hidden assumptions on how it's used.

In this post, I will introduce you to why, where, and when you would want to use a `with` statement, and how you can deal with some of the challenges and underlying assumptions. Additionally, you’ll read about a readability and differentiation trick using annotation tuples.

## 1. The worst "case".

The use-case for `with` statements is apparent when we look at the worst `case` statement implementation. Nesting `case` statements quickly leads to a pyramid of unreadable code (literally! tilt your head 90°):

```elixir
case is_email_address?(email) do
  true ->
    case String.length(code) === 6 do
      true ->
        case EmailConfirmations.get_email_confirmation(email) do
          %EmailConfirmation{} ->
            case EmailAddresses.get_email_address(email) do
              nil ->
                case Users.create_user(data) do
                  {:ok, user} ->
                    case EmailAddresses.create_email_address(user, email) do
                      {:ok, email_address} ->
                        ...

                      {:error, error} ->
                        :error
                    end

                  {:error, error} ->
                    :error
                end

              _email_address ->
                :error
            end

          nil ->
            :error
        end

      false ->
        :error
    end

  false ->
    :error
end
```

- What this code does well, is matching all possible returns from every function that is called; therefore the code is correct.

- What this code does badly, is it's unreadable for you, me, and any other developer looking at it.

We need something better to maintain readability, while keeping the case statement's functionality and correctness.

## 2. Simple "with" statements.

`with` statements work like `case` statements, but with a focus on successful function call results. The following statements are equivalent:

```elixir
case Users.create_user(attrs) do
  {:ok, user} -> ...
  {:error, changeset} -> ...
end

with {:ok, user} <- Users.create_user(attrs) do
  ...
else
  {:error, changeset} -> ...
end
```

Let's see some examples.

### Implicit  handling of non-matching results.

`with` statements do not need to handle non-matching clauses. Instead, the `with` statement will return the non-matching clause directly to the parent scope.

The Phoenix framework comes with context generators that will output controller functions with `with` statements like this:

```elixir
def create(conn, params) do
  with {:ok, user} <- Users.create_user(params) do
    conn      
    |> put_status(:accepted)      
    |> render("show.json", user: user)
  end
end
```

Where the function `Users.create_user/1` can return either `{:ok, user}` or `{:error, changeset}`.

As you can see, the clause `{:error, changeset}` is not caught in the `with` statement. Let's see what this `with` statement does:

- When the expected clause matches, because it is `{:ok, user}`, we continue with the function inside the `with` statement and `user` is available within the scope.

- When the expected clause does not match, for example we receive `{:error, changeset}`, then the result is immediately returned to the parent scope (i.e. `create/2`).

Therefore, the function `create/2` returns either the result of the successful clause (an updated `conn`) or `{:error, changeset}`.

### Explicit handling of non-matching results.

We can explicitly handle the non-matching clauses in a `with` statement by using the `else` clause. The `else` clause requires us to pattern match on **all non-matching results**.

```elixir
def create(conn, params) do
  with {:ok, user} <- Users.create_user(params) do
    conn      
    |> put_status(:accepted)      
    |> render("show.json", user: user)
  else
    {:error, changeset} ->
      {:error, changeset}
  end
end
```

This does the same as the previous example. It will return either the result of the successful clause (an updated `conn`) or `{:error, changeset}`.

Because we are explicitly handling non-matching clauses, we must add pattern matches for **all non-matching clauses** in the `else` block. If we miss a returned clause, we will receive a `** (WithClauseError) no with clause matching` error.

> This is the reason why Dialyzer will complain `"The pattern can never match the type"`, because if a returned clause is not explicitly handled, it may crash the process at runtime.

## 3. Chained functions in "with" statements.

Chaining function calls in a `with` statement is fairly straightforward. We can rewrite the "case pyramid" from the introduction by chaining the consecutive expressions in a `with` statement:

```elixir
with true <- is_email_address?(email),
     true <- String.length(code) === 6,
     %EmailConfirmation{} <- EmailConfirmations.get_email_confirmation(email),
     nil <- EmailAddresses.get_email_address(email),
     {:ok, user} <- Users.create_user(data),
     {:ok, email_address} <- EmailAddresses.create_email_address(user, email) do
  ...
end
```

Much cleaner and shorter than the original. And with syntax-highlighting in an IDE, this code will be much more readable.

### Challenges.

In the previous examples, we always return `:error` when a function call does not return the desired result. But this is rarely what we want in practice.

- We may want to differentiate between similar clauses in the `else` block.

- We need to handle **all non-matching clauses** in the `else` block, of **every chained function call**, or we will introduce app-crashing bugs.

- Dialyzer will definitely complain if any of the functions does not return an expected clause (i.e. `:error` in this example).

## 4. Advanced "with" statements.

Let's add custom error-handling, differentiation of function calls and results, and readability improvements.

### Custom error-handling.

Let's introduce some custom error-handling to the rewritten "case pyramid" in a `with` statement:

```elixir
with true <- is_email_address?(email),
     true <- String.length(code) === 6,
     %EmailConfirmation{} <- EmailConfirmations.get_email_confirmation(email),
     nil <- EmailAddresses.get_email_address(email),
     {:ok, user} <- Users.create_user(data),
     {:ok, email_address} <- EmailAddresses.create_email_address(user, email) do
  ...
else
  false -> 
    {:error, :bad_request}

  nil -> 
    {:error, :not_found}

  %EmailAddress{} -> 
    {:error, :conflict}

  {:error, changeset} -> 
    {:error, changeset}
end
```

Great! Now we can return different error messages depending on  which non-matching clause is returned.

But what if the `with` statement has multiple functions that return the same non-matching results, and we want to handle each return differently?

For example, when two functions can return `false`, and one version must return `{:error, :bad_request}`, and the other must return `{:error, :conflict}`, like so:

```elixir
with true <- is_email_address?(email),
     true <- EmailAddresses.is_available(email) do
  ...
else
  false ->
    # Return either '{:error, :bad_request} or '{:error, :conflict}'
end
```

Let's see how we can differentiate the returned `false` clause.

### Differentiating non-matching clauses.

We can use a simple mathematical trick to differentiate the returned results. If `x * y = z`, then `x * y + c = z + c` must be true as well. Therefore we can add some annotation tuples to the function calls and clauses, and write the previous example as:

```elixir
with {:is_email, true} <- {:is_email, is_email_address?(email)},
     {:is_available, true} <- {:is_available, EmailAddresses.is_available(email)} do
  ...
else
  {:is_email, false} ->
    {:error, :bad_request}

  {:is_available, false} ->
    {:error, :conflict}
end
```

By using tuples on both sides of the function calls, we can differentiate the returning values.

### Improving readability for large "with" statements.

We can improve readability of large `with` statements by using the trick from the previous section. That is, by annotating function calls and pattern matches in the `with` and `else` block with tuples.

Let's rewrite the "case pyramid" `with` statement with annotation tuples:

```elixir
with {:is_email, true} <- {:is_email, is_email_address?(email)},
     {:is_code, true} <- {:is_code, String.length(code) === 6},
     {:confirmation_fetch, %EmailConfirmation{}} <- {:confirmation_fetch, EmailConfirmations.get_email_confirmation(email)},
     {:email_fetch, nil} <- {:email_fetch, EmailAddresses.get_email_address(email)},
     {:user_create, {:ok, user}} <- {:user_create, Users.create_user(data)},
     {:email_create, {:ok, email_address}} <- {:email_create, EmailAddresses.create_email_address(user, email)} do
  ...
else
  {_, false} -> 
    {:error, :bad_request}

  {:confirmation_fetch, nil} -> 
    {:error, :not_found}

  {:email_fetch, %EmailAddress{}} -> 
    {:error, :conflict}

  {_, {:error, changeset}} -> 
    {:error, changeset}
end
```

In this `with` statement we chain six functions, and add the following tuples to make the code more readable: `{:is_email, _}`, `{:is_code, _}`, `{:confirmation_fetch, _}`, `{:email_fetch, _}`, `{:user_create, _}`, and `{:email_create, _}`.

Again, in an IDE with syntax-highlighting this will look much better.

## 5. Common issues and how to solve them.

Be mindful of all the possible returned values from the functions you call in a `with` statement. Properly handling matching and non-matching clauses will solve most of your issues.

1. Use formatting to make `with` blocks more readable.

2. Use the `else` block in a with statement to handle non-matching results manually.

3. Always pattern match **all possible returned non-matching values** in `else` blocks to prevent app-crashing bugs.

4. Use pattern matches on structs. For example `with %User{is_banned: false} <- Users.get_user(id) do`.

5. To differentiate function calls and results, annotate the function call and their results with tuples (e.g. `{:is_email, true}` and `{:is_available, true}`).

6. To make large `with` statements more readable, consider annotating function calls and their results with tuples (e.g. `{:user_fetch, user}` and `{:user_update, updated_user}`).

7. Dialyzer may show the warning `"The pattern can never match the type"`. This means you've missed a result from one (or multiple) of the called functions in the `else` block. Add the missing return clause, and the warning will resolve.

---

I hope you learned something today. I'm a big fan of Elixir's `with` clause, and of all the other readability, maintainability, and code correctness features that Elixir offers. The next post will be about Dialyzer, and why  you should (always) use it for development. Follow me to get a notification in January!