Absinthe for Large Elixir Applications
Tags: elixir
Published: 2023-06-13 10:32:52

In our introduction to Absinthe, we covered the basics of Absinthe and GraphQL for an Elixir application.

Now we'll dig deeper and see how we can customize Absinthe for larger-scale Elixir applications.

Let's get going!

## Defining Resolvers for a Large Elixir App

We've seen that creating a query and returning a result in Absinthe is really easy. But if you have big resolution logic, the schema can soon get very heavy. It's common practice to define resolver functions in a separate module for large apps using Absinthe.

First, create a module that defines functions acting as field resolvers:

```elixir
# lib/my_app_web/schema/resolvers/blog.ex
defmodule MyAppWeb.Schema.Resolvers.Blog do
  def post(%{id: post_id}, _resolution) do
    # complex resolution logic here
    # post = ...
    {:ok, post}
  end
end
```

Then, update your schema to use this resolver:

```elixir
defmodule MyAppWeb.Schema do
  use Absinthe.Schema

  query do
    field :post, :post do
      arg :id, non_null(:id)
      resolve &MyAppWeb.Schema.Resolvers.Blog.post/2
    end
  end
end
```

This also allows you to unit-test that resolution logic separately from your GraphQL integration tests. We will discuss more about testing schemas later on in this series.

The resolvers additionally help you to reduce code duplication at several places in your schema. For example, let's say you need to expose a field under a different name from what is defined on the struct.

We can build the object like this:

```elixir
object :post do
  # ...
  field :published_at, :datetime do
    resolve fn %Post{} = post, _args, _resolution ->
      {:ok, post.inserted_at}
    end
  end
end
```

If you find yourself doing that at several places in the schema, it might be better to create a resolver:

```elixir
defmodule MyAppWeb.Schema.Resolvers.Base do
  @doc """
  Returns a resolver function that returns the value at `field` from `parent`
  """
  def alias(field) do
    fn parent, _args, _resolution -> {:ok, Map.get(parent, field)} end
  end
end
```

And then use it in your object like this:

```elixir
object :post do
  # ...
  field :published_at, :datetime, resolve: Resolvers.Base.alias(:inserted_at)
end
```

## Avoiding N+1 Queries in Elixir

Let’s get back to our schema, but with a query that returns a list of posts:

```elixir
defmodule MyAppWeb.Schema do
  use Absinthe.Schema

  object :post do
    # ...
    field :author, non_null(:author),
      resolve: &Resolvers.Blog.post_author/3
  end

  query do
    field :posts, non_null(list_of(:post)),
      resolve: &Resolvers.Blog.list_posts/2
  end
end
```

Here is the full [`Resolvers.Blog`](https://gist.github.com/sapandiwakar/2585851aa844b9cbf4f7fb812f049ca5#file-blog-ex) if you are interested.

Now, perform a query that includes the author of each post:

```graphql
query {
  posts {
    id
    author {
      id
      firstName
    }
  }
}
```

When this query is executed, we first fetch all posts from the database. Then we fetch the author by their id for each post — that's not very efficient.

But that’s an easy problem to solve. We can just preload the authors in `Resolvers.Blog.list_posts/2`.

However, what if someone makes a query that doesn’t need an author? We will still be unnecessarily fetching all authors, defeating the whole purpose of having a composable query like this.

One possible solution is to be smart when resolving the initial list of posts and load authors only when the user has selected the `author` field in the query. If you remember, we get an `Absinthe.Resolution` struct as the last argument to the resolver function.
`resolution.definition.selections` contains all the selected fields (`Absinthe.Blueprint.Document.Field` structs).

We can check if the `author` was selected here, and preload it right there:

```elixir
defmodule Resolvers.Blog do
  def list_posts(%{}, %Absinthe.Resolution{} = res) do
    posts = Blog.list_posts()

    res.definition.selections
    |> Enum.any?(&(&1.name == "author"))
    |> if do
      {:ok, Repo.preload(posts, :author)}
    else
      {:ok, posts}
    end
  end
end
```

That works, but if you have many fields (or several nested levels), it can soon become too cumbersome. This is where [`dataloader`](https://hexdocs.pm/absinthe/dataloader.html) can help.

## Dataloader to the Rescue!

Dataloader provides an efficient way to load linked data by batching queries. It does this by first performing a full pass through the query resolution phase, collecting all the ids of objects to be loaded. Dataloader then loads them all in one go instead of making a request to the database for each of them separately.

It’s a separate dependency, so first add it to your `mix.exs` inside `deps`.

You then need a one-time setup in your schema:

```elixir
defmodule MyAppWeb.Schema do
  use Absinthe.Schema

  # ... schema definition here ...

  # Add dataloader to the context
  def context(ctx) do
    loader =
      Dataloader.new
      |> Dataloader.add_source(Blog, Blog.data())
      # ... add other sources here ...

    Map.put(ctx, :loader, loader)
  end

  # Add dataloader to the plugins
  def plugins do
    [Absinthe.Middleware.Dataloader] ++ Absinthe.Plugin.defaults()
  end
end
```

You can add custom fields to the context through the `context` method in the schema. The context's value is available to all steps in the GraphQL request cycle (e.g., inside resolvers or middleware) inside the `%Absinthe.Resolution{}` struct. Context is also where we usually store user details if we authenticate users.

Check out the [Absinthe Context and Authentication](https://hexdocs.pm/absinthe/context-and-authentication.html) guide to explore this further.

In addition, we add `dataloader` to `plugins` using the `plugins/0` callback on the schema. This allows `dataloader` to hook into the resolution pipeline. If you want to learn more about plugins, read the [Writing Middleware and Plugins guide for Absinthe](https://hexdocs.pm/absinthe/middleware-and-plugins.html).

`DataLoader.add_source/3` expects the name of the data source as its second argument and a module implementing the `Dataloader.Source` protocol. Dataloader supports an [Ecto-based data source](https://hexdocs.pm/dataloader/Dataloader.Ecto.html) out of the box, which is what we will need for our example.

Let's update our Phoenix context (`Blog` in our example) to return the Ecto data source from `data/0`:

```elixir
# my_app/blog.ex
defmodule MyApp.Blog do
  def data(), do: Dataloader.Ecto.new(MyApp.Repo, query: &query/2)

  def query(queryable, _params), do: queryable
end
```

It isn't really documented very well, but if you are feeling adventurous, most of the data-loading magic lies inside the [Ecto data source](https://github.com/absinthe-graphql/dataloader/blob/master/lib/dataloader/ecto.ex).
It uses the `query/2` function we passed to generate a base query when it needs to load some data.

For example, if we try to load `Author` with ids 1 through 5, it will make a single query like `from a in Author, where a.id in [1, 2, 3, 4, 5]`, instead of making 5 different queries.

This function is our opportunity to filter results and return a query that will finally be used to fetch the items.
For now, we just return the queryable as it is, which means that we don’t need any special filtering.

## Use Dataloader Inside the Absinthe Schema

To use Dataloader inside our schema, we must now modify our object `post` to use the [`dataloader` helper](https://hexdocs.pm/absinthe/Absinthe.Resolution.Helpers.html#dataloader/3).

```elixir
import Absinthe.Resolution.Helpers, only: [dataloader: 2]

object :post do
  # ... other fields
  field :author, non_null(:author),
    resolve: dataloader(MyApp.Blog, :author)
  end
end
```

- The first argument to `dataloader/2` is the source name (as registered in the schema).
- The next argument is the name of the field in the parent object (`author` field in parent object `post`).

Note that the data source should be the one that will resolve the field. So if the post belongs to an author with type `MyApp.Accounts.User`, you must use `dataloader(MyApp.Accounts, :author)` as the resolver and support `data/0` and `query/2` inside the `Accounts` context.

[Here is the full code](https://gist.github.com/sapandiwakar/067b283d14905f0c9dcafb378c783cd7) if you are interested. I know it's a lot to take in, so let's go through the execution of the query above.

Absinthe first invokes our posts resolver (`Resolvers.Blog.list_posts/2`), and returns the list of posts. Absinthe then checks for the fields it needs inside `post` and encounters a selection for `author`.
This is where dataloader takes over:

1. It collects the `author_id` for all posts that will be returned in our result. Let's say we need to load authors `[1, 2, 3, 4, 5]`.
2. It then calls `MyApp.Blog.query(Author, %{})` to get the initial query. In our example, we are simply returning `Author` (but in a real application, this could be filtered by business case — for example, if we need only authors that have an active account, we could return `where(queryable, [a], a.active)`, instead of just returning `queryable`).
3. Finally, it loads the required ids from the above query — `from a in Author, where a.id in [1, 2, 3, 4, 5]`.

As you can see, we only performed a single query instead of 5 different ones.

Nesting also works out of the box, so if each author has an `organization` field and we select that in the query, Dataloader will load all organizations in one batch.

## Organizing Your Absinthe Schema with Imports

As your schema starts growing, you will soon notice that putting all type definitions and query fields in the same file is not sensible. This is where [`import_types`](https://hexdocs.pm/absinthe/Absinthe.Schema.Notation.html#import_types/2) and [`import_fields`](https://hexdocs.pm/absinthe/Absinthe.Schema.Notation.html#import_fields/2) come into play.

The level to split at depends on the size of your API and your application, but it is a common practice to split by business context (the same as your Phoenix contexts).

Here is a structure that works well.

1. Create a module that contains queries (and another for mutations) related to each model:

   ```elixir
   # lib/my_app_web/schema/types/blog/post/queries.ex
   defmodule MyAppWeb.Schema.Types.Blog.Post.Queries do
     use Absinthe.Schema.Notation

     object :post_queries do
       field :posts, list_of(:post), resolve: Resolvers.Blog.posts/2
       # ... all queries related to post here
     end
   end

   ```

2. Create a module for types related to each model. Also import the query and mutation types here.

   ```elixir
   # lib/my_app_web/schema/types/blog/post.ex
   defmodule MyAppWeb.Schema.Types.Blog.Post do
     use Absinthe.Schema.Notation

     import_types(MyAppWeb.Schema.Types.Blog.Post.Queries)
     import_types(MyAppWeb.Schema.Types.Blog.Post.Mutations)

     object :post do
       field :title, not_null(:string)
       # ...
     end

     # all types related to blog here
   end
   ```

3. Create a module for types related to each context. This should only import the types from model-specific modules.

   ```elixir
   # lib/my_app_web/schema/types/blog.ex
   defmodule MyAppWeb.Schema.Types.Blog do
     use Absinthe.Schema.Notation

     alias MyAppWeb.Schema.Types

     import_types(Types.Blog.Post)
     # ... import all types related to blog here

     object :blog_queries do
       import_fields(:post_queries)
       # ... import all queries related to blog here
     end

     object :blog_mutations do
       import_fields(:post_mutations)
       # ... import all queries related to blog here
     end
   end
   ```

4. Finally, import the context-specific types to your schema.

   ```elixir
   # lib/my_app_web/schema.ex
   defmodule MyAppWeb.Schema do
     use Absinthe.Schema

     import_types(MyAppWeb.Schema.Types.Blog)

     query do
       import_fields :blog_queries
     end

     mutation do
       import_fields :blog_mutations
     end
   end
   ```

This way, your schema remains very clear, declaring only what it imports. All specific queries are further down the pipeline.

This may seem like overkill for our small API example.
But we have been using it in production for a large app with several contexts, and it’s been a boon to keep our schema manageable.

## Wrap Up

In this post, the second part of our series on Absinthe, we customized Absinthe for an Elixir application pushing a lot of data. We started by defining resolvers for a big Elixir application and covered how to avoid N+1 queries.

Finally, we dived into Dataloader (which helps to load linked data) in some detail and explored how to organize our Absinthe schema.

Next up, we'll look at creating mutations and subscriptions with Absinthe.

Until then, happy coding!

**P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, [subscribe to our Elixir Alchemy newsletter and never miss a single post](/elixir-alchemy)!**