Using Ecto changesets for API request parameter validation
Tags: elixir, ecto, phoenix, tutorial
Published: 2022-02-24 20:23:26

As Elixir developers, we typically use `Ecto.Changeset` to validate proposed database records changes in Elixir. It has comprehensive field validation capabilities and standardized error reporting, both of which we can customize.

In this post, I will share how you can use `Ecto.Changeset` beyond the database context, and use it as an API validation mechanism. Combine it with `Gettext`, and you get easy error translation for your APIs.

## Regular Ecto

If you’ve worked with Elixir, then you’ve probably seen Ecto schemas and changesets like this:

```elixir
@primary_key {:user_id, :binary_id, autogenerate: true}

schema "users" do
  field :username, :string, unique: true
  field :password, :string, virtual: true, redact: true
  field :password_hash, :string
  field :first_name, :string
  field :last_name, :string
  field :is_banned, :boolean, default: false
  field :is_deleted, :boolean, default: false

  embeds_one :photo, Photo

  timestamps()
end

def changeset(user, attrs) do
  user
  |> cast(attrs, [:username, :first_name, :last_name, :password])
  |> validate_required([:username, :first_name, :password])
  |> shared_validations()
end
```

This is a regular `Ecto.Schema` for a `users` table, with a `changeset/2` function that receives the `User` struct and the proposed database record changes as `attrs`.

### Passing valid data to changeset/2

When passing valid data to the changeset function `changeset/2`, it will return the tuple `{:ok, %Ecto.Changeset{valid?: true, changes: changes, ...}}`. Where `changes` is a `map` of validated database record changes, like:

```elixir
%{first_name: "Jane", last_name: "Doe", username: "janedoe"}
```

### Passing invalid data to changeset/2

When passing invalid data to the `changeset/2` function, it will return the tuple `{:error, %Ecto.Changeset{valid?: false, errors: errors, ...}}`. Where `errors` is a `keyword list`, like:

```elixir
errors: [
  first_name: {"should be at least %{count} character(s)",
    [count: 3, validation: :length, kind: :min, type: :string]},
  last_name: {"should be at most %{count} character(s)",
    [count: 20, validation: :length, kind: :max, type: :string]},
  username: {"can't be blank", [validation: :required]}
]
```

Which is a bit unreadable, but with the help of `Ecto.Changeset.traverse_errors(changeset, &translate_error/1)` we can translate these errors in a human-readable `map`, like:

```elixir
%{
  "first_name" => ["should be at least 3 character(s)"],
  "last_name" => ["should be at most 20 character(s)"],
  "username" => ["can't be blank"]
}
```

We’ll be using the `changes` and `errors` fields later in this post, so let’s keep them in mind. 😉

> When creating a Phoenix project, `translate_error/1` is provided by default. If you’ve chosen to install `Gettext` (installed by default for a Phoenix app), you get easy language translation of these errors for free.

## API validation

Using `Ecto.Changeset` for database record changes is nice, but let’s take a look at API validation. I’ll use a JSON-based API in the following examples, because that’s what I’m most familiar with.

### Basic controller

Below is an example of a basic Phoenix controller, which creates a `User` and an `EmailAddress`, and verifies and updates an existing `EmailConfirmation`. Each action has a function call. These three function calls happen in succession, and each must succeed before another can be called.

That means we will want to validate the request parameters **before** we start any of the function calls; because if one succeeds, a later one may fail when it receives invalid data.

```elixir
# API.AccountController

action_fallback(API.FallbackController)

def create(conn, %{"account" => params}) do
  with {:ok, email_confirmation} <- verify_email_confirmation(params),
       {:ok, email_address} <- create_email_address(params),
       {:ok, user} <- create_user(params) do
    conn
    |> put_status(:created)
    |> render("create.json", user: user, email_address: email_address)
  end
end

def create(_conn, _params), do: {:error, :bad_request}
```

Let’s see how we can validate the input parameters, before we call any of the functions.

### Basic validation

We could add a pattern match in the controller arguments, and add some manual validation in private functions in the same controller file.

With the pattern match `def create(conn, %{"email" => _, "code" => _, "username” => _, "first_name" => _ "last_name" => _} = params) do` we can guarantee all the fields that we need are present. There’s no validation of the values yet.

Because we lack an API validation library, we need to manually confirm the validity of each field in the controller:

- `email` must be a valid email address, e.g. “john@example.com”
- `code` must be a valid confirmation code, e.g. “123456”
- `username` must be a valid string without spaces and special characters, e.g. “johndoe”
- `first_name` and `last_name` must be strings with a minimum length of 3 characters and a maximum length of 20 characters.

We could do this with a private function:

```elixir
# API.AccountController

defp validate_params(%{"email" => email, "first_name" => first_name, ...}) do
  with true <- is_email(email),
       true <- String.length(first_name) >= 3,
       true <- String.length(last_name) <= 20,
       ... do
    :ok
  else
    false -> :error
  end
end
```

But that gets dirty, fast. ❌

### Using Ecto

Instead of doing validations manually, let’s create an `Ecto.Schema` and an accompanying changeset called `AccountController.CreateAction` that contains our validations, in the controller namespace:

```elixir
# API.AccountController.CreateAction

embedded_schema do
  field(:email, :string)
  field(:code, :string)
  field(:username, :string)
  field(:first_name, :string)
  field(:last_name, :string)
end

def changeset(attrs) do
  %CreateAction{}
  |> cast(attrs, [:email, :code, :first_name, :last_name])
  |> validate_required([:email, :code, :first_name, :last_name])
  |> validate_length(:code, is: 6)
  |> validate_length(:username, min: 3, max: 10)
  |> validate_length(:first_name, min: 3, max: 20)
  |> validate_length(:last_name, min: 1, max: 20)
  |> validate_format(:email_address, @email_regex)
  |> validate_format(:username, @username_regex)
  |> validate_format(:first_name, @name_regex)
  |> validate_format(:last_name, @name_regex)
  |> update_change(:email, &String.downcase/1)
  |> update_change(:username, &String.downcase/1)
end

def validate_params(params) do
  case changeset(params) do
    %Ecto.Changeset{valid?: false} = changeset -> 
      {:error, changeset}

    %Ecto.Changeset{valid?: true, changes: changes} -> 
      {:ok, changes}
  end
end
```

I’m going a bit overboard with the validations, to show the extent to which you can validate API fields and values. It can be very fine-grained. 👌

In my own projects, I tend to abstract these validations into shared functions. For example, the `first_name` and `last_name` validations happen in both the `CreateAction` and in `User` schemas, so they share a separate validation function in `User`.

For example:

```elixir
def validate_name(changeset, field) do
  changeset
  |> validate_format(field, @name_regex)
  |> validate_length(field, min: 3, max: 20)
end
```

Very nice. ✅

### Implementation

OK. Let’s implement the `validate_params/1` function of `API.AccountController.CreateAction` in the controller:

```elixir
# API.AccountController

def create(conn, params) do
  with {:ok, attrs} <- CreateAction.validate_params(params),
       {:ok, email_confirmation} <- verify_email_confirmation(attrs),
       {:ok, email_address} <- create_email_address(attrs),
       {:ok, user} <- create_user(attrs) do
    conn
    |> put_status(:created)
    |> render("create.json", user: user, email_address: email_address)
  end
end
```

Much cleaner. So what happens here?

We call `CreateAction.validate_params/1` before any other function gets involved. `validate_params/1` receives the request parameters as a `map`, and validates them using `changeset/1`, returning either `{:ok, attrs}` or `{:error, changeset}`.

If the request parameters are valid, then the `Ecto.Changeset` `struct` contains the `valid?: true` and `changes: changes` fields. `changes` is the `map` of validated request parameters that we want to pass to our subsequent function calls as `{:ok, attrs}`.

If the request parameters are invalid, then the `Ecto.Changeset` `struct` contains the `valid?: false` field, and we pass the `Ecto.Changeset` back to our controller function as `{:error, changeset}`, where it gets picked up by the `FallbackController`.

## Error messages

So when the API request body contains invalid parameters, we receive `{:error, %Ecto.Changeset{}}`. To process this error, we need a `FallbackController`. Luckily, this is provided by default in a Phoenix project.

> If you’re missing `FallbackController`, then you can run one of the `mix phx.gen` tasks from https://hexdocs.pm/phoenix/Mix.Tasks.Phx.Gen.html and it will be generated for you.

### Catching errors with FallbackController

The default `FallbackController` contains a fallback function like this:

```elixir
# API.FallbackController

def call(conn, {:error, %Ecto.Changeset{} = changeset}) do
  conn
  |> put_status(:unprocessable_entity)
  |> put_view(ChangesetView)
  |> render("error.json", changeset: changeset)
end
```

Whenever a function inside a controller returns `{:error, %Ecto.Changeset{}}`, it is caught by this fallback function inside `FallbackController`. The function then renders the `changeset` errors as a message, and returns the connection with a `422` HTTP status code (`:unprocessable_entity`).

For example, it returns error messages like this:

```elixir
%{
  "errors" => %{
    "code" => ["should be 6 character(s)"],
    "email_address" => ["has invalid format"]
  }
}
```

You can customize the error parsing with `Ecto.Changeset.traverse_errors/2`, but the default provided by Phoenix is a nice format for a frontend system to handle.

### Translating error messages

If you have `Gettext` installed (which is installed by default in a Phoenix project), then you can add custom error translations for any language you need.

Since the error messages returned from `Ecto.Changeset` are always in a simple and specific format, like `"is in valid"` `"can't be blank"` and `"should be at least 8 character(s)"`, we can easily add error translations for our API.

I won’t dive into the details of `Gettext`, but the previous example of a rendered error could easily be translated into this, in Spanish 🇪🇸:

```elixir
%{
  "errors" => %{
    "code" => ["debe tener 6 caracter(es)"],
    "email_address" => ["tiene un formato inválido"]
  }
}
```

Not sure if this translates correctly, because I don't speak much Spanish. But it's a nice feature to have, right? 🤷

---

I hope you learned something today, and that this post will help you build better APIs in Elixir. The next post will be about Dialyzer, and why you should (always) use it for development. Godspeed, alchemists! ⚗️

### More information

- Read about `Ecto.Schema` and `embedded_schema` here: https://hexdocs.pm/ecto/Ecto.Schema.html
- Read about `Ecto.Changeset` and error messages from changeset validations here: https://hexdocs.pm/ecto/Ecto.Changeset.html
- Read about `FallbackController` here: https://hexdocs.pm/phoenix/Phoenix.Controller.html#action_fallback/1
- Read about `Plug` HTTP status codes here: https://hexdocs.pm/plug/Plug.Conn.Status.html
- Read about Phoenix generators here: https://hexdocs.pm/phoenix/Mix.Tasks.Phx.Gen.html
- Read about `Gettext` here: https://hexdocs.pm/gettext/Gettext.html