Deconstructing Elixir's GenServers
Tags: elixir, learning, genserver
Published: 2018-06-12 14:10:02

---
title: Deconstructing Elixir's GenServers
description: Elixir uses GenServers to keep state and run code asynchronously. But how does this work? In this episode of Elixir Alchemy, we'll deconstruct Elixir's GenServer module to see how it works under the hood.
cover_image: https://blog.appsignal.com/images/blog/2018-06/genservers.jpg
tags: elixir, learning, genserver
published: true
canonical_url: https://blog.appsignal.com/2018/06/12/elixir-alchemy-deconstructing-genservers.html
---

Instead of using object instances like in object oriented languages, Elixir uses [*GenServers*](https://elixir-lang.org/getting-started/mix-otp/genserver.html) (generic servers) to store state in separate processes. GenServers can keep state and run code asynchronously, which is useful for keeping data available without passing it from function to function.

But how does this work? In this episode of Elixir Alchemy, we'll deconstruct Elixir's `GenServer` module to see how it functions. We'll learn how a GenServer communicates with other processes, and how it keeps track of its state by implementing part of its functionality ourselves.

## A key-value store

Before we can dive under the hood of the `GenServer` module, we'll take it for a spin to build a *key-value store* that can store and retrieve values by key.

``` elixir
defmodule KeyValue do
  # 1
  def init(state \\ %{}) do
    {:ok, state}
  end

  # 2
  def handle_cast({:put, key, value}, state) do
    {:noreply, Map.put(state, key, value)}
  end

  # 3
  def handle_call({:get, key}, _from, state) do
    {:reply, Map.fetch!(state, key), state}
  end
end
```

This example has three *callback functions* to initialize the store, put new values in the store, and query values by key.

1. The `init/1` function takes a state or defaults to an empty map. The `GenServer` API requires the state to be returned in an ok-tuple.

2. The `handle_cast/2` callback is used to *cast* an asynchronous message to our server, without waiting for a response. This server responds to a cast with a message matching `{:put, key, value}` and puts a new key in map that holds the state. It returns a `:noreply`-tuple to indicate the function doesn't return a response, with the updated state.

3. To make synchronous calls, the `handle_call/3` callback is used. This example responds to *call* with a message matching `{:get, key}`, which finds the key in the state and returns it in a `:reply`-tuple along with the state.

To try our key-value store, we use the `start/2`, `cast/3` and `call/2` functions on `GenServer`. Internally, these use the internal callbacks defined in the key-value store.

    iex(1)> {:ok, pid} = GenServer.start(KeyValue, %{})
    {:ok, #PID<0.113.0>}
    iex(2)> GenServer.cast(pid, {:put, :foo, "bar"})
    :ok
    iex(3)> GenServer.call(pid, {:get, :foo})
    {:ok, "bar"}

We initialize a new store, which returns the *pid* (Process ID) in an ok-tuple. The pid is used to refer to our store process later when calling `cast` and `call` to set and retrieve a value.

### Convenience functions

Since having to call to our server through `GenServer`'s functions gets tedious, they're usually abstracted away by adding functions to the server.

``` elixir
defmodule KeyValue do
  use GenServer

  def start do
    GenServer.start(KeyValue, %{})
  end

  def put(server, key, value) do
    GenServer.cast(server, {:put, key, value})
  end

  def get(server, key) do
    GenServer.call(server, {:get, key})
  end

  # Callbacks

  def init(state) do
    {:ok, state}
  end

  def handle_cast({:put, key, value}, state) do
    {:noreply, Map.put(state, key, value)}
  end

  def handle_call({:get, key}, _from, state) do
    {:reply, Map.fetch!(state, key), state}
  end
end
```

By adding the `start/0`, `get/2` and `put/3` functions, we don't have to worry about the server's internal implementation when calling its API.

    iex(2)> {:ok, pid} = KeyValue.start
    {:ok, #PID<0.113.0>}
    iex(3)> KeyValue.put(pid, :foo, "bar")
    :ok
    iex(4)> KeyValue.get(pid, :foo)
    "bar"

## GenServer internals

To understand how our key-value server works under the hood, we'll dive into the `GenServer` module. It relies on *message passing* and *recursion* for communication and keeping state.

### Message passing between processes

As we discussed when we talked about [processes in Elixir](https://blog.appsignal.com/2017/05/18/elixir-alchemy-demystifying-processes-in-elixir.html), processes communicate by sending messages to each other. Most message passing is abstracted away while working in Elixir, but you can send a message to another process by using the `send/2` function.

``` elixir
send(pid, :hello)
```

The function takes the recipient's `pid` and a message to send. Usually, the message is an atom or a tuple when multiple arguments need to be passed.

In the other process, the recipient receives the message in its mailbox. It can react to incoming messages using the `receive` function.

``` elixir
receive do
  :hello ->
    IO.puts "Hello there!"
end
```

By using pattern matching, different actions can be taken for different messages. In `GenServer`, tuples with the `:"$gen_call"` and `:"$gen_cast"` keys are used to query and update the internal state, for example.

Message passing is the first ingredient needed to build a server. In a GenServer, messages are passed to a separate spawned process to update and query its state.

### State through recursion

Besides accepting messages, the process spawned by a GenServer uses looping to keep track of its initial state.

``` elixir
def loop(state) do
  receive do
    message -> loop(message)
  end
end
```

In this example, a default state is passed to the `loop/1` function, which calls the `receive` function to check for new messages in the mailbox. If there are none, it will block the process and wait for one to come in. When a message is received, the function calls itself with the received message as its new state. The state is updated and the process is back to waiting for messages to come in.

GenServers have a loop method internally, which reads incoming messages and calls the corresponding callback functions. After that, it calls itself again with the updated state.

## Implementing our own GenServer

Now that we understand how a GenServer functions, we can partially implement our own version of the `GenServer` module to understand how message passing and recursive state works together to make our key-value store work.

In Elixir itself, the `GenServer` module is mostly a wrapper around Erlang's tried and tested `gen_server`, but we'll implement ours in Elixir.

``` elixir
defmodule MyGenServer do
  # 1
  def start(module, state) do
    {:ok, state} = module.init(state)
    {:ok, spawn(__MODULE__, :loop, [state, module])}
  end

  # 2
  def loop(state, module) do
    state =
      receive do
        {:"$gen_cast", from, message} ->
          {:noreply, state} = module.handle_cast(message, state)
          state

        {:"$gen_call", {pid, reference} = from, message} ->
          {:reply, reply, state} = module.handle_call(message, from, state)
          send(pid, {reference, reply})
          state
      end

    loop(state, module)
  end

  # ...

end
```

1. The `start/2` function starts the server process. It calls the `init/1` function on the passed module to allow it to set initial state and then [spawns](https://hexdocs.pm/elixir/Kernel.html#spawn/3) a process that runs the `loop/2` function with the `state` and `module` variables as its arguments.

2. The `loop/2` function accepts incoming messages.
  - When a `:"$gen_cast"`-message comes in, the `handle_cast/2` function is called on the module passed to the `start/2` function (which is `KeyValue` in our example). It expects a `:noreply`-tuple to be returned with the new state. The new state is then used to run the `loop/2` function again to wait for more messages.
  - `:"$gen_call"`-messages call the `handle_call/3` function. Here, a `:reply`-tuple is expected, with the reply and the new state. The reply is then sent back to the process that sent the message. Then, like when handing casts, the updated state is used to call the `loop/2` function again.

The server part of our custom GenServer is done. We can now start our key-value store with our custom GenServer. Because our custom GenServer's API matches Elixir's, we can use the built-in version to communicate with a key value store started with our custom GenServer.

    iex> {:ok, pid} = MyGenServer.start(KeyValue, %{})
    {:ok, #PID<0.119.0>}
    iex> GenServer.cast(pid, {:put, :foo, "bar"})
    :ok
    iex> GenServer.call(pid, {:get, :foo})
    "bar"

To understand the `cast/2` and `call/2` functions, we'll implement those on our GenServer as well.

``` elixir
defmodule MyGenServer do
  # ...

  # 1
  def cast(pid, message) do
    send(pid, {:"$gen_cast", {self(), nil}, message})
    :ok
  end

  # 2
  def call(pid, message) do
    send(pid, {:"$gen_call", {self(), nil}, message})

    receive do
      {_, response} -> response
    end
  end
end
```

1. To cast a message to our server, we'll send a message in the cast-format to our server and return an ok-atom.
2. Since a call requires a response, we'll wait for a message to be sent back using the `receive` function and return that.

```
iex> {:ok, pid} = MyGenServer.start(KeyValue, %{})
{:ok, #PID<0.119.0>}
iex> MyGenServer.cast(pid, {:put, :foo, "bar"})
:ok
iex> MyGenServer.call(pid, {:get, :foo})
"bar"
```

Now, we can use our implementation to start our key-value store and communicate with it without needing Elixir's `GenServer` module.


``` elixir
defmodule MyGenServer do
  def start(module, state) do
    {:ok, state} = module.init(state)
    {:ok, spawn(__MODULE__, :loop, [state, module])}
  end

  def loop(state, module) do
    state =
      receive do
        {:"$gen_cast", from, message} ->
          {:noreply, state} = module.handle_cast(message, state)
          state

        {:"$gen_call", {pid, reference} = from, message} ->
          {:reply, reply, state} = module.handle_call(message, from, state)
          send(pid, {reference, reply})
          state
      end

    loop(state, module)
  end

  def cast(pid, message) do
    send(pid, {:"$gen_cast", {self(), nil}, message})
    :ok
  end

  def call(pid, message) do
    send(pid, {:"$gen_call", {self(), nil}, message})

    receive do
      {_, response} -> response
    end
  end
end
```

## Using GenServers in Elixir

We can now switch the convenience funtions in our key-value store over to our own GenServer implementation to verify everything still works.

``` elixir
defmodule KeyValue do
  def start do
    MyGenServer.start(KeyValue, %{})
  end

  def put(server, key, value) do
    MyGenServer.cast(server, {:put, key, value})
  end

  def get(server, key) do
    MyGenServer.call(server, {:get, key})
  end

  # Callbacks

  def init(state) do
    {:ok, state}
  end

  def handle_cast({:put, key, value}, state) do
    {:noreply, Map.put(state, key, value)}
  end

  def handle_call({:get, key}, _from, state) do
    {:reply, Map.fetch!(state, key), state}
  end
end
```

    iex> {:ok, pid} = KeyValue.start
    {:ok, #PID<0.113.0>}
    iex> KeyValue.put(pid, :foo, "bar")
    :ok
    iex> KeyValue.get(pid, :foo)
    "bar"

This concludes our dive into Elixir's GenServer internals. Our example implements parts of what Elixir's built-in `GenServer` can do. Although our implementation only covers part of what `GenServer` and `gen_server` do in Elixir and Erlang, it gives a peek into how a GenServer works internally.

*Note*: Our own GenServer implementation functions, but shouldn't be used for anything but educational purposes. Instead, use Elixir's built-in GenServer. It's based on Erlang's `gen_server`, which has been used in production apps for decades.

Did we clear up some confusion about Elixir's GenServers? We'd love to know what you think of this article, so please don't hesitate to [let us know](https://twitter.com/appsignal). We'd also love to know if you have any Elixir subjects you'd like to know more about.