A deeper dive in Elixir's Plug
Tags: elixir, plug, cowboy, router
Published: 2019-01-06 14:36:08

---
title: "A deeper dive in Elixir's Plug"
published: true
description: "Trip down Elixir's Plug, what it means, how it works and a bit more"
tags: [elixir, plug, cowboy, router]
---

*This post was originally published on [my blog](https://ieftimov.com) on [December 31st, 2018](https://ieftimov.com/a-deeper-dive-in-elixir-plug).*

Being new to Elixir and Phoenix, I spend quite some time in the projects' documentation. One thing that stood out for me recently is the first sentence of [Phoenix's Plug documentation](https://hexdocs.pm/phoenix/plug.html#content):

> Plug lives at the heart of Phoenix’s HTTP layer and Phoenix puts Plug front and center.

So naturally, I felt compelled to take a deeper dive into Plug and understand it better. I hope the following article will help you out in understanding Plug.

## What's Plug?

![](https://thepracticaldev.s3.amazonaws.com/i/wc1em7dxm4kmw7zq7efb.jpg)

As [the readme](https://github.com/elixir-plug/plug#plug) puts it, Plug is:
1. A specification for composable modules between web applications
2. Connection adapters for different web servers in the Erlang VM

But, what does this mean? Well, it basically states that Plug 1) *defines the way* you build web apps in Elixir and 2) it provides you with tools to write apps that are understood by web servers.

Let's take a dive and see what that means.

## Web servers, yeehaw!

One of the most popular HTTP servers for Erlang is [Cowboy](https://github.com/ninenines/cowboy). It is a small, fast and modern HTTP server for Erlang/OTP. If you were to write any web application in Elixir it will run on Cowboy, because the Elixir core team has built a Plug adapter for Cowboy, conveniently named [plug_cowboy](https://github.com/elixir-plug/plug_cowboy).

This means that if you include this package in your package, you will get the Elixir interface to talk to the Cowboy web server (and vice-versa). It means that you can send and receive requests and other stuff that web servers can do.

So why is this important?

Well, to understand Plug we need to understand how it works. Basically, using the adapter (`plug_cowboy`), Plug can accept the connection request that comes in Cowboy and turn it into a meaningful struct, also known as `Plug.Conn`.

This means that Plug uses `plug_cowboy` to understand Cowboy's nitty-gritty details. By doing this Plug allows us to easily build handler functions and modules that can receive, handle and respond to requests.

Of course, the idea behind Plug is not to work only with Cowboy. If you look at [this SO answer](https://stackoverflow.com/a/26323883/6015550) from José Valim (Elixir's BDFL) he clearly states "Plug is meant to be a generic adapter for different web servers. Currently we support just Cowboy but there is work to support others."

## Enter Plug

Okay, now that we've scratched the surface of Cowboy and it's Plug adapter, let's look at Plug itself.

If you look at [Plug's README](https://github.com/elixir-plug/plug/blob/master/README.md), you will notice that there are two flavours of plugs, a function or a module.

The most minimal plug can be a function, it just takes a `Plug.Conn` struct (that we will explore more later) and some options. The function will manipulate the struct and return it at the end. Here's the example from the `README`:

```elixir
def hello_world_plug(conn, _opts) do
  conn
  |> put_resp_content_type("text/plain")
  |> send_resp(200, "Hello world")
end
```
<small>Code blatantly copied from [Plug's docs](https://github.com/elixir-plug/plug/blob/master/README.md).</small>

If you look at the function, it's quite simple. It receives the connection struct, puts its content type to `text/plain` and returns a response with an HTTP 200 status and `"Hello world"` as the body.

The second flavour is the module Plug. This means that instead of just having a function that will be invoked as part of the request lifecycle, you can define a module that takes a connection and initialized options and returns the connection:

```elixir
defmodule MyPlug do
  def init([]), do: false
  def call(conn, _opts), do: conn
end
```

<small>Code blatantly copied from [Plug's docs](https://github.com/elixir-plug/plug/blob/master/README.md).</small>

Having this in mind, let's take a step further and see how we can use Plug in a tiny application.

## Plugging a plug as an endpoint

So far, the most important things we covered was what's Plug and what is it used for on a high level. We also took a look at two different types of plugs.

Now, let's see how we can mount a Plug on a Cowboy server and essentially use it as an endpoint:

```elixir
defmodule PlugTest do
  import Plug.Conn

  def init(options) do
    # initialize options

    options
  end

  def call(conn, _opts) do
    conn
    |> put_resp_content_type("text/plain")
    |> send_resp(200, "Hello world")
  end
end
```

What this module will do is, when mounted on a Cowboy server, will set the `Content-Type` header to `text/plain` and will return an HTTP 200 with a body of `Hello world`.

Let's fire up IEx and test this ourselves:

```text
› iex -S mix
Erlang/OTP 21 [erts-10.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] [dtrace]

Interactive Elixir (1.7.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> {:ok, _ } = Plug.Cowboy.http PlugTest, [], port: 3000
{:ok, #PID<0.202.0>}
```

This starts the Cowboy server as a BEAM process, listening on port 3000. If we `cURL` it we'll see the response body and it's headers:

```
› curl -v 127.0.0.1:3000
> GET / HTTP/1.1
> Host: 127.0.0.1:3000
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< cache-control: max-age=0, private, must-revalidate
< content-length: 11
< content-type: text/plain; charset=utf-8
< date: Tue, 25 Dec 2018 22:54:54 GMT
< server: Cowboy
<
* Connection #0 to host 127.0.0.1 left intact
Hello world
```

You see, the `Content-Type` of the response is set to `text/plain` and the body is `Hello world`. In this example, the plug is essentially an endpoint by itself, serving plain text to our `cURL` command (or to a browser). As you might be able to imagine at this point, you can plug in much more elaborate Plugs to a Cowboy server and it will serve them just fine.

To shut down the endpoint all you need to do is:

```text
iex(2)> Plug.Cowboy.shutdown PlugTest.HTTP
:ok
```

What we are witnessing here is probably the tiniest web application one can write in Elixir. It's an app that takes a request and returns a valid response over HTTP with a status and a body.

So, how does this actually work? How do we accept the request and build a response here?

## Diving into the `Plug.Conn`

To understand this, we need to zoom in the `call/2` function of our module `PlugTest`. I will also throw in an `IO.inspect` right at the end of the function so we can inspect what this struct is:

```elixir
def call(conn, _opts) do
  conn
  |> put_resp_content_type("text/plain")
  |> send_resp(200, "Hello world")
  |> IO.inspect
end
```

If you start the Cowboy instance again via your IEx session and you hit `127.0.0.1:3000` via `cURL` (or a browser), you should see something like this in your IEx session:

```
%Plug.Conn{
  adapter: {Plug.Cowboy.Conn, :...},
  assigns: %{},
  before_send: [],
  body_params: %Plug.Conn.Unfetched{aspect: :body_params},
  cookies: %Plug.Conn.Unfetched{aspect: :cookies},
  halted: false,
  host: "127.0.0.1",
  method: "GET",
  owner: #PID<0.316.0>,
  params: %Plug.Conn.Unfetched{aspect: :params},
  path_info: [],
  path_params: %{},
  port: 3000,
  private: %{},
  query_params: %Plug.Conn.Unfetched{aspect: :query_params},
  query_string: "",
  remote_ip: {127, 0, 0, 1},
  req_cookies: %Plug.Conn.Unfetched{aspect: :cookies},
  req_headers: [
    {"accept",
     "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8"},
    {"accept-encoding", "gzip, deflate, br"},
    {"accept-language", "en-US,en;q=0.9"},
    {"connection", "keep-alive"},
    {"host", "127.0.0.1:3000"},
    {"upgrade-insecure-requests", "1"},
    {"user-agent",
     "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36"}
  ],
  request_path: "/",
  resp_body: nil,
  resp_cookies: %{},
  resp_headers: [
    {"cache-control", "max-age=0, private, must-revalidate"},
    {"content-type", "text/plain; charset=utf-8"}
  ],
  scheme: :http,
  script_name: [],
  secret_key_base: nil,
  state: :sent,
  status: 200
}
```

What are we actually looking at? Well, it's actually the Plug representation of a connection. This is a direct interface to the underlying web server and the request that the Cowboy server has received.

Some of the attributes of the struct are pretty self-explanatory, like `scheme`, `method`, `host`, `request_path`, etc. If you would like to go into detail what each of these fields is, I suggest taking a look at `Plug.Conn`'s [documentation](https://hexdocs.pm/plug/Plug.Conn.html#module-request-fields).

But, to understand better the `Plug.Conn` struct, we need to understand the connection lifecycle of each connection struct.

### Connection lifecycle

Just like any map in Elixir `Plug.Conn` allows us to pattern match on it. Let's modify the little endpoint we created before and try to add some extra `IO.inspect` function calls:

```elixir
defmodule PlugTest do
  import Plug.Conn

  def init(options) do
    # initialize options

    options
  end

  def call(conn, _opts) do
    conn
    |> inspect_state
    |> put_resp_content_type("text/plain")
    |> inspect_state
    |> put_private(:foo, :bar)
    |> inspect_state
    |> resp(200, "Hello world")
    |> inspect_state
    |> send_resp()
    |> inspect_state
  end

  defp inspect_state(conn = %{state: state}) do
    IO.inspect state
    conn
  end
end
```

Because `Plug.Conn` allows pattern matching, we can get the `state` of the connection, print it out and return the connection itself so the pipeline in the `call/2` function would continue working as expected.

Let's mount this plug on a Cowboy instance and hit it with a simple `cURL` request:

```bash
iex(6)> Plug.Cowboy.http PlugTest, [], port: 3000
{:ok, #PID<0.453.0>}

# curl 127.0.0.1:3000

iex(21)> :unset
:unset
:unset
:set
:sent
```

You see, when the connection enters the plug it's state changes from `:unset` to `:set` to finally `:sent`. This means that once the plug is invoked the state of the connection is `:unset`. Then we do multiple actions, or in other words, we invoke multiple functions on the `Plug.Conn` which add more information to the connection. Obviously, since all variables in Elixir are immutable, each of these function returns a new `Plug.Conn` instance, instead of mutating the existing one.

Once the body and the status of the connection are set, then the state changes to `:set`. Up until that moment, the state is fixed as `:unset`. Once we send the response back to the client the state is changed to `:sent`.

What we need to understand here is that whether we have one or more plugs in a pipeline, they will all receive a `Plug.Conn`, call functions on it, whether to extract or add data to it and then the connection will be passed on to the next plug. Eventually, in the pipeline, there will be a plug (in the form of an endpoint or a Phoenix controller) that will set the body and the response status and send the response back to the client.

There are a bit more details to this, but this is just enough to wrap our minds around `Plug` and `Plug.Conn` in general.

## Next-level `Plug`ging using `Plug.Router`

Now that we understand how `Plug.Conn` works and how plugs can change the connection by invoking functions defined in the `Plug.Conn` module, let's look at a more advanced feature of plugs - turning a plug into a router.

In our first example, we saw the simplest of the Elixir web apps - a simple plug that takes the request and returns a simple response with a text body and an HTTP 200. But, what if we want to handle different routes or HTTP methods? What if we want to gracefully handle any request to an unknown route with an HTTP 404?

One nicety that `Plug` comes with is a module called `Plug.Router`, you can see its documentation [here](https://hexdocs.pm/plug/Plug.Router.html#content). The router module contains a DSL that allows us to define a routing algorithm for incoming requests and writing handlers (powered by Plug) for the routes. If you are coming from Ruby land, while `Plug` is basically Rack, this DSL is Sinatra.rb.

Let's create a tiny router using `Plug.Router`, add some plugs to its pipeline and some endpoints.

**Quick aside: What is a pipeline?**

Although it has the same name as the pipeline operator (`|>`), a pipeline in Plug's context is a list of plugs executed one after another. That's really it. The last plug in that pipeline is usually an endpoint that will set the body and the status of the response and return the response to the client.

Now, back to our router:

```elixir
defmodule MyRouter do
  use Plug.Router

  plug :match
  plug :dispatch

  get "/hello" do
    send_resp(conn, 200, "world")
  end

  match _ do
    send_resp(conn, 404, "oops")
  end
end
```

<small>Code blatantly copied from [`Plug.Router`'s docs](https://hexdocs.pm/plug/Plug.Router.html).</small>

The first thing that you will notice here is that all routers are modules as well. By `use`ing the `Plug.Router` module, we include some functions that make our lives easier, like `get` or `match`.

If you notice at the top of the module we have two lines:

```elixir
plug :match
plug :dispatch
```

This is the router's *pipeline*. All of the requests coming to the router will pass through these two plugs: `match` and `dispatch`. The first one does the matching of the route that we define (e.g. `/hello`), while the other one will invoke the function defined for a particular route. This means that if we would like to add other plugs, most of the time they will be invoked between the two mandatory ones (`match` and `dispatch`).

Let's mount our router on a Cowboy server and see it's behaviour:

```bash
iex(29)> Plug.Cowboy.http MyRouter, [], port: 3000
{:ok, #PID<0.1500.0>}
```

When we hit `127.0.0.1:3000/hello`, we will get the following:

```bash
› curl -v 127.0.0.1:3000/hello
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 3000 (#0)
> GET /hello HTTP/1.1
> Host: 127.0.0.1:3000
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< cache-control: max-age=0, private, must-revalidate
< content-length: 5
< date: Thu, 27 Dec 2018 22:50:47 GMT
< server: Cowboy
<
* Connection #0 to host 127.0.0.1 left intact
world
```

As you can see, we received `world` as the response body and an HTTP 200. But if we hit any other URL, the router will match the other route:

```bash
› curl -v 127.0.0.1:3000/foo
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 3000 (#0)
> GET /foo HTTP/1.1
> Host: 127.0.0.1:3000
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< cache-control: max-age=0, private, must-revalidate
< content-length: 4
< date: Thu, 27 Dec 2018 22:51:56 GMT
< server: Cowboy
<
* Connection #0 to host 127.0.0.1 left intact
oops
```

As you can see, because the `/hello` route didn't match we defaulted to the other route, also known as "catch all" route, which returned `oops` as the response body and an HTTP 404 status.

If you would like to learn more about `Plug.Router` and its route matching macros you can read more in [its documentation](https://hexdocs.pm/plug/Plug.Router.html#module-routes). We still need to cover some more distance with Plug.

## Built-in Plugs

In the previous section, we mentioned the plugs `match` and `dispatch`, and plug pipelines. We also mentioned that we can plug in other plugs in the pipeline so we can inspect or change the `Plug.Conn` of each request.

What is very exciting here is that `Plug` also comes with already built-in plugs. That means that there's a list of plugs that you can plug-in in any Plug-based application:

- `Plug.CSRFProtection`
- `Plug.Head`
- `Plug.Logger`
- `Plug.MethodOverride`
- `Plug.Parsers`
- `Plug.RequestId`
- `Plug.SSL`
- `Plug.Session`
- `Plug.Static`

Let's try to understand how a couple of them work and how we can plug them in our `MyRouter` router module.

### `Plug.Head`

This is a rather simple plug. It's so simple, I will add [all of its code](https://github.com/elixir-plug/plug/blob/v1.7.1/lib/plug/head.ex#L1) here:

```elixir
defmodule Plug.Head do
  @behaviour Plug

  alias Plug.Conn

  def init([]), do: []

  def call(%Conn{method: "HEAD"} = conn, []), do: %{conn | method: "GET"}
  def call(conn, []), do: conn
end
```

What this plug does is it turns any HTTP `HEAD` request into a `GET` request. That's all. Its `call` function receives a `Plug.Conn`, matches only the ones that have a `method: "HEAD"` and returns a new `Plug.Conn` with the `method` changed to `"GET"`.

If you've been wondering what the `HEAD` method is for, this is from [RFC 2616](https://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html):

> The HEAD method is identical to GET except that the server MUST NOT return a
> message-body in the response. The metainformation contained in the HTTP headers
> in response to a HEAD request SHOULD be identical to the information sent in
> response to a GET request. This method can be used for obtaining
> metainformation about the entity implied by the request without transferring
> the entity-body itself. This method is often used for testing hypertext links
> for validity, accessibility, and recent modification.

Let's plug this plug in our `Plug.Router` (pun totally intended):

```elixir
defmodule MyRouter do
  use Plug.Router

  plug Plug.Head
  plug :match
  plug :dispatch

  get "/hello" do
    send_resp(conn, 200, "world")
  end

  match _ do
    send_resp(conn, 404, "oops")
  end
end
```

Once we `cURL` the routes we would get the following behaviour:

```bash
› curl -I 127.0.0.1:3000/hello
HTTP/1.1 200 OK
cache-control: max-age=0, private, must-revalidate
content-length: 5
date: Thu, 27 Dec 2018 23:25:13 GMT
server: Cowboy

› curl -I 127.0.0.1:3000/foo
HTTP/1.1 404 Not Found
cache-control: max-age=0, private, must-revalidate
content-length: 4
date: Thu, 27 Dec 2018 23:25:17 GMT
server: Cowboy
```

As you can see, although we didn't explicitly match the `HEAD` routes using the `head` macro, the `Plug.Head` plug remapped the `HEAD` requests to `GET` and our handlers still kept on working as expected (the first one returned an HTTP 200, and the second one an HTTP 404).

### `Plug.Logger`

This one is a bit more complicated so we cannot inline [all of its code](https://github.com/elixir-plug/plug/blob/v1.7.1/lib/plug/logger.ex#L1) in this article. Basically, if we would plug this plug in our router, it will log all of the incoming requests and response statuses, like so:

```
  GET /index.html
  Sent 200 in 572ms
```

This plug uses Elixir's `Logger` ([docs](https://hexdocs.pm/logger/Logger.html) under the hood, which supports four different logging levels:

- `:debug` - for debug-related messages
- `:info` - for information of any kind (default level)
- `:warn` - for warnings
- `:error` - for errors

If we would look at the source of its `call/2` function, we would notice two logical units. The first one is:

```elixir
def call(conn, level) do
  Logger.log(level, fn ->
    [conn.method, ?\s, conn.request_path]
  end)

  # Snipped...
end
```

This one will take Elixir's `Logger` and using the logging `level` will log the information to the backend (by default it's `console`). The information that is logged is the method of the request (e.g. `GET`, `POST`, etc) and the request path (e.g. `/foo/bar`). This results in the first line of the log:

```
GET /index.html
```

The second logical unit is a bit more elaborate:

```elixir
def call(conn, level) do
  # Snipped...

  start = System.monotonic_time()

  Conn.register_before_send(conn, fn conn ->
    Logger.log(level, fn ->
      stop = System.monotonic_time()
      diff = System.convert_time_unit(stop - start, :native, :microsecond)
      status = Integer.to_string(conn.status)

      [connection_type(conn), ?\s, status, " in ", formatted_diff(diff)]
    end)

    conn
  end)
end
```

In short: this section records the time between the `start` and the `stop` (end) of the request and prints out the `diff`erence between the two (or in other words - the amount of time the response took). Also, it prints out the HTTP status of the response.

To do this it uses `Plug.Conn.register_before_send/2` ([docs](https://hexdocs.pm/plug/Plug.Conn.html#register_before_send/2)) which is a utility function that registers callbacks to be invoked before the response is sent. This means that the function which will calculate the `diff` and log it to the `Logger` with the response status will be invoked by `Plug.Conn` right before the response is sent to the client.

## Wrapping up with Plug

You actually made it this far - I applaud you. I hope that this was a nice journey for you in Plug and it's related modules/functions and that you learned something new.

We looked at quite a bit of details in and around `Plug`. For some of the modules that we spoke about we barely scratched the surface. For example, `Plug.Conn` has quite a bit of more useful functions. Or `Plug.Router` has more functions in its DSL where you can write more elaborate and thoughtful APIs or web apps. In line with this, `Plug` also offers more built-in plugs. It even has a plug which can serve static files with ease, and plugging it in your Plug-based apps is a breeze.

But, aside from all the things that we skipped in this article, I hope that you understood how powerful the Plug model is and how much power it provides us with such simplicity and unobtrusiveness.

In future posts, we will look at even more details about other plugs in `Plug`, but until then please shoot me a comment or a message if you've found this article helpful (or not).