Livebook for Elixir: Just What the Docs Ordered
Tags: elixir
Published: 2022-05-31 10:48:03

While initially conceived as a tool for data exploration (much like [Jupyter](https://jupyter.org/) for Python), Livebook has deservedly become a sensation in the Elixir community.

It has been fantastic to see all the wonderful ways teams are leveraging Livebook for a range of different use cases. We have seen Livebooks being used to:

- Create interactive documentation for libraries.
- Build onboarding material and guides.
- Audit and explore potential dependencies in your app.

Livebooks have also been used as the default REPL interface for project development.

In this post, we'll show how you can easily create interactive documentation with Livebook and outline some top tips for using Livebook. We will assume you have [installed Livebook](https://Github.com/livebook-dev/livebook), following the guidance in their README.

## But First: What is Livebook for Elixir?

Livebooks are supercharged markdown files where you can add sections of arbitrary executable Elixir code. They are inspired by similar notebooks for other languages (like Python's Jupyter), but Livebooks leverage [LiveView](https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.html) and other BEAM goodies, so they are even better.

Livebook files get their own `.livemd` extension, and (somewhat confusingly) we create and run those Livebook markdown files using a Phoenix application also called [Livebook](https://Github.com/livebook-dev/livebook).

That phoenix app runs in the browser and enables a whole host of interactive features like collaboration, as we will see.

The expectation is that you will install the Livebook repo locally and start a Livebook server from somewhere on your machine where there are Livebooks (the files).

The Livebook app will show you the working directory of where you started the Livebook server, so you can select any given Livebook to run from there.

Let's now look at a library to document.

## Livebook Docs: A Single Source of Truth in Elixir

Our library is going to accept various inputs and rate them according to the following chonk chart:

![alt chart showing cats of various sizes](https://blog.appsignal.com/images/blog/2022-05/chonk.jpg)

It will output the chonk rating accordingly. First, let's create the library.

```sh
mix new chonk_o_meter && cd chonk_o_meter
```

Let's add `ex_doc` to our deps in our `mix.exs`:

```elixir
defp deps do
  [
    {:ex_doc, ">= 0.0.0", runtime: false, only: [:docs, :dev]},
  ]
end
```

Now, download the chonk chart image above and put it into the root of the library in a directory called `images` so we can refer to it in our README. In the `README.md`, let's put a title and a short explanation of what the library aims to do:

```md
# Chonk 'O' Meter

Chonk O Meter is a state-of-the-art size estimator. It will rate the size of anything according to the following chart:

![alt chart showing cats of various sizes](./images/chonk.jpg)
```

So far, so good. Now we will open the module and write our moduledoc. But here's the thing: what we really want to do is just copy what we already wrote for the README.

Having one source of truth for that information is super valuable as the library develops because having to update the documentation in multiple places is a recipe for errors!

It's good to repeat the same information in different formats because different people will come to the library via different paths.

Some may see the repo first (and therefore the README), whereas others may see the library on Hex first — and so only see the moduledoc.

You may think that it's easy enough to copy and paste, but once we add our Livebook into the mix, there will be three places we need to update docs when something changes!

Instead, let's be a bit smarter. We will section off a part of the README and read that section into the moduledoc at compile time. Sandwich the introduction to the library between two markdown comments in the README:

```md
<!-- README START -->

.... Library introduction here.

<!-- README END -->
```

Now, we can write the introduction as if it were a moduledoc in between those two comments, like so:

```md
<!-- README START -->

Chonk O Meter is a state-of-the-art size estimator. It will rate the size of anything according to the following chart:

![alt chart showing cats of various sizes](./images/chonk.jpg)

<!-- README END -->
```

In the main module `ChonkOMeter`, we can do this:

```elixir
defmodule ChonkOMeter do
  @moduledoc File.read!(Path.expand("./README.md"))
             |> String.split("<!-- README START -->")
             |> Enum.at(1)
             |> String.split("<!-- README END -->")
             |> List.first()
end
```

This will extract the guide sandwiched between the markdown comments and set it as the moduledoc. Now we only need to change the README to update both!

We can generate the documentation and view it locally to verify that this works as expected. If you run `mix docs`, a `doc` folder will appear with an `index.html` file. We can `open doc/index.html` to view our documentation in the browser.

## Adding an Image to the Moduledoc

If you navigate to the module's documentation, you will notice that the image is missing. Hex allows you to point to assets in your docs as long as they are included inside the `doc` directory (generated when you create docs with the `mix docs` command).

Usually, that command overwrites the whole `doc` folder. So, to ensure that our pictures are always copied there, we can use an alias in our `mix.exs` file. We will turn the `mix docs` command into one that runs `mix docs` and then copies all images inside the `/images` directory into a `doc/images` directory.

```elixir
defmodule ChonkOMeter.MixProject do
  use Mix.Project

  def project do
    [
      ...
      aliases: aliases(),
      ...
    ]
  end

  defp aliases() do
    [docs: ["docs", &copy_pictures/1]]
  end

  defp copy_pictures(_) do
    File.cp_r(Path.expand("./images/"), Path.expand("./doc/images/"))
  end
end
```

If you run `mix docs` now, you will see that the `images/` directory gets copied over to the `doc` folder. Open the `doc/index.html` file and you should now see the chonk chart appear!

## Doctests in Elixir's Livebook

It's important to note that in writing our moduledoc in this way, we don't lose any of the usual capabilities ex_docs give us.

Anything you can normally do in a doctest, you can still do here. To demonstrate that, let's add a doctest to our README. First, we'll need a function to test:

```elixir
defmodule ChonkOMeter do
  @moduledoc File.read!(Path.expand("./README.md"))
             |> String.split("<!-- README START -->")
             |> Enum.at(1)
             |> String.split("<!-- README END -->")

  @doc """
  Returns the Chonk rating for a given number of story points.
  """
  def story_points(points) when is_integer(points) and points >= 0 and points < 3 do
    "A Fine Boi"
  end

  def story_points(points) when is_integer(points) and points >= 3 and points < 5 do
    "He Chomnk"
  end

  def story_points(points) when is_integer(points) and points >= 5 and points < 8 do
    "A Heckin' Chonker"
  end

  def story_points(points) when is_integer(points) and points >= 8 and points < 10 do
    "H E F T Y C H O N K"
  end

  def story_points(points) when is_integer(points) and points >= 10 and points < 15 do
    "Mega Chonk"
  end

  def story_points(points) when is_integer(points) and points >= 15 do
    "Oh Lawd He Comin'"
  end
end
```

Now remove the boilerplate in our test file, so it looks like this:

```elixir
defmodule ChonkOMeterTest do
  use ExUnit.Case
  doctest ChonkOMeter
end
```

Run the tests to ensure there are none for now:

```sh
mix test
```

Finally, in our README, we can add the usual doctest syntax:

```md
<!-- README START -->

Chonk O Meter is a state-of-the-art size estimator. It will rate the size of anything according to the following chart:

![alt chart showing cats of various sizes](./images/chonk.jpg)

For example:

    iex> ChonkOMeter.story_points(10)
    "Mega Chonk"

<!-- README END -->
```

If you run the tests, you will notice that the change in the README has not triggered a re-compilation, meaning the app still thinks there are no doctests. To fix this, we just need to add an [`@external_resource`](https://hexdocs.pm/elixir/Module.html#module-external_resource) module attribute into the main module. This tells mix to recompile when the README changes:

```elixir
defmodule ChonkOMeter do
  @external_resource Path.expand("./README.md")
  # ^^ Add this line ^^
  @moduledoc File.read!(Path.expand("./README.md"))
             |> String.split("<!-- README START -->")
             |> Enum.at(1)
             |> String.split("<!-- README END -->")

  @doc """
  Returns the Chonk rating for a given number of story points.
  """
  def story_points(points) when is_integer(points) and points >= 0 and points < 3 do
    "A Fine Boi"
  end

  def story_points(points) when is_integer(points) and points >= 3 and points < 5 do
    "He Chomnk"
  end

  def story_points(points) when is_integer(points) and points >= 5 and points < 8 do
    "A Heckin' Chonker"
  end

  def story_points(points) when is_integer(points) and points >= 8 and points < 10 do
    "H E F T Y C H O N K"
  end

  def story_points(points) when is_integer(points) and points >= 10 and points < 15 do
    "Mega Chonk"
  end

  def story_points(points) when is_integer(points) and points >= 15 do
    "Oh Lawd He Comin'"
  end
end
```

When we run our tests, this now results in one passing doctest! We can also add a doctest to our function doc like so:

```elixir
...
  @doc """
  Returns the Chonk rating for a given number of story points.

      iex> ChonkOMeter.story_points(5)
      "A Heckin' Chonker"
  """
  def story_points(points) when is_integer(points) and points >= 0 and points < 3 do
    "A Fine Boi"
  end

  def story_points(points) when is_integer(points) and points >= 3 and points < 5 do
    "He Chomnk"
  end

  def story_points(points) when is_integer(points) and points >= 5 and points < 8 do
    "A Heckin' Chonker"
  end

  def story_points(points) when is_integer(points) and points >= 8 and points < 10 do
    "H E F T Y C H O N K"
  end

  def story_points(points) when is_integer(points) and points >= 10 and points < 15 do
    "Mega Chonk"
  end

  def story_points(points) when is_integer(points) and points >= 15 do
    "Oh Lawd He Comin'"
  end
...
```

## Adding Livebook to Elixir

So let's recap. Right now, we have a README as the source of truth for our moduledoc. We can have doctests and images and all the usual goodies that a moduledoc is allowed, but we don't have to repeat ourselves and risk copy/paste errors.

We want to keep that same energy going for our Livebook, to avoid repeating ourselves manually, but still have an interactive playground for our library on top of the usual moduledocs.

To do that, we can generate our Livebook from our module. To help with this, I've written a library we can include called [livebook_helpers](https://github.com/Adzz/livebook_helpers):

```elixir
defp deps do
  [
    {:ex_doc, ">= 0.0.0", runtime: false, only: [:docs, :dev]},
    {:livebook_helpers, ">= 0.0.0", only: [:docs, :dev]},
  ]
end
```

Once we have fetched the deps with `mix deps.get`, running `mix help` shows one extra mix task:

```sh
...
mix create_livebook_from_module # Creates a livebook from the docs in the given module.
...
```

We can see from [the docs](https://hexdocs.pm/livebook_helpers/Mix.Tasks.CreateLivebookFromModule.html) that we run the mix task by providing a module and a path to a Livebook. Let's try that:

```sh
mix create_livebook_from_module ChonkOMeter "chonk_o_meter_introduction"
```

You should see a successful output that links to the generated Livebook! 🎉 There is one last thing we can do to make our workflow seamless. Let's add `create_livebook_from_module` to the end of the `mix docs` command.

```elixir
defmodule ChonkOMeter.MixProject do
  use Mix.Project

  def project do
    [
      ...
      aliases: aliases(),
      ...
    ]
  end

  defp aliases() do
    [docs: ["docs", &copy_pictures/1, &create_livebook/1]]
  end

  defp copy_pictures(_) do
    File.cp_r(Path.expand("./images/"), Path.expand("./doc/images/"))
  end

  defp create_livebook(_) do
    Mix.Task.run("create_livebook_from_module", ["ChonkOMeter", "chonk_o_meter_introduction"])
  end
end
```

Whenever we run `mix docs`, we will copy over any static images used in the README and generate a Livebook from our main module!

![alt picture of the generated livebook showing the same moduledocs](https://blog.appsignal.com/images/blog/2022-05/livebook.png)

## Running Livebook in Elixir

So far, so good! We have a nice pipeline to create a useful Livebook, but now we need to think about running the Livebook. Start the Livebook app like so:

```sh
livebook server
```

By default, the Elixir sections only have access to the Elixir and Erlang standard library. If we run our generated library and then attempt to run an Elixir cell that calls the library, it will fail because the library code is not there. To solve this, we have two options — `Mix.install` or Livebook runtime.

### Add `Mix.install` to Livebook

We could add a section to the beginning of the Livebook that does this:

```elixir
Mix.install([:chonk_o_meter])
```

![alt text](https://blog.appsignal.com/images/blog/2022-05/mix_install.png)

When called, we will get the latest version of the library from Hex. It will be made available to all subsequent Elixir cells, just like when you run `Mix.install` inside an IEx REPL.

You can also easily specify a version and provide live documentation for any version of a given library:

```elixir
Mix.install([{chonk_o_meter: ">=0.0.1"}])
```

`LivebookHelpers` can even generate a Livebook with a `Mix.install` at the beginning if we supply deps to the mix task:

```sh
mix create_livebook_from_module ChonkOMeter "chonk_o_meter_introduction" "[:chonk_o_meter"]"
```

This works great for any library that is deployed to Hex. However, you'll run into problems if, for example, you want a Livebook for a `main` branch. In that case, you can do this:

```elixir
Mix.install [{:chonk_o_meter, path: "./"}]
```

This tells mix to look in the provided path for a local version of the library. This, of course, makes some assumptions about _where_ the Livebook will run, so it's good to make that clear. If you put the Livebook at the root of the repo and a user starts the Livebook server from there, then the path `"./"` will work.

If you don't want to rely on this, though, Livebook has your back with a powerful feature: runtime!

### Livebook Runtime

There are three kinds of runtime, but they all let you point to code and call it in any Elixir cell within your Livebook.

The three runtime options are:

- Embedded
- Attached node
- Mix standalone

You select the runtime by clicking the cog symbol here:

![alt text](https://blog.appsignal.com/images/blog/2022-05/runtime.png)

Let's look at each runtime option below.

### Embedded Mode

Embedded Mode lets us run the notebook code within the Livebook node itself! This is really for specific cases where there is no option to start a separate Elixir runtime — for example, on embedded devices.

Code defined in one notebook may interfere with code from another notebook. So this mode should only be used if you have no alternative and is not relevant here.

### Attached Node

The attached node runtime connects a Livebook to a running Elixir app via the usual Erlang magic that we use to connect two running nodes. It looks like this:

![alt attached node option](https://blog.appsignal.com/images/blog/2022-05/attached.png)

As long as the app starts with a cookie that you know and a `sname`, you can give them to Livebook and connect. This is like getting a remote shell in a running app but with a more full-featured text-editing environment.

Using an attached node gives you complete control over how your app starts. You get much more control than the mix standalone and can do all sorts of things, like set env vars and start other services (like a database).

An attached node could be especially relevant for creating internal (live!) documentation for closed-source repos at work, but is not relevant for us and our library.

### Mix Standalone

Finally, the Mix standalone runtime lets you point to a mix project which Livebook will compile and start (analogous to running `iex -S mix` in your terminal).

![alt mix standalone option](https://blog.appsignal.com/images/blog/2022-05/standalone.png)

Mix standalone confers a great advantage over `Mix.install`, as you can recompile it! It's useful if you write a Livebook from scratch; in that case, you'll likely add something to the library that you'll then want to use in the Livebook.

Instead of having to kill the Livebook server and restart to access the new functions, you can add an Elixir cell with the following:

```elixir
IEx.Helpers.recompile()
```

This will recompile the connected mix app (i.e., our library) when you call it, meaning you get access to any new functionality therein.

## Livebook for Docs: Try It Yourself in Your Elixir App

That concludes our tour of Livebook for docs. You can [see the `chonk_o_meter` library example here](https://github.com/Adzz/chonk_o_meter). For an example of these ideas in action in a real library, [check out my data_schema library](https://Github.com/Adzz/data_schema/blob/main/lib/data_schema.ex#L2) too.

Currently, GitHub doesn't recognize the `.livemd` extension, so if you play around with Livebook, I would encourage you to push to public repos. The more we do this, the more chance we have of getting GitHub to parse the files with nice syntax highlighting and markdown rendering.

Until that happens, though, we can put a magic line at the top of a Livebook to force GitHub to render it as markdown:

```md
<!-- vim: syntax=markdown -->
```

The Livebook that [Livebook helpers](https://Github.com/Adzz/livebook_helpers) generates will include this line for you. Now you have all the knowledge you need go forth and create live docs!

Happy coding!

**P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, [subscribe to our Elixir Alchemy newsletter and never miss a single post](/elixir-alchemy)!**