Shell scripting with Elixir
Tags: elixir
Published: 2024-02-12 05:17:42

---
title: Shell scripting with Elixir
published: true
date: 2024-02-11 20:00:00 UTC
tags: Elixir
canonical_url: https://arathunku.com/b/2024/shell-scripting-with-elixir/
---

When simple bash scripts start to become unwieldy, you may consider reaching out for something “higher” level, like Perl, Ruby or Python. I’m reaching out for Elixir. Maybe the startup times are not perfect for every use case, but Elixir is extremely versatile. It’s easy to add dependencies, debug, iterate and even write tests, all in a single file! I believe, due to [LiveBook](livebook.dev), it really hits its stride in recent years; the ecosystem leans heavily into ergonomic developer experience and great out-of-the-box defaults ([Req](https://github.com/wojtekmach/req) is amazing!). In few lines of code, you can connect to Postgres, send HTTP requests or start HTTP server.

For my own use cases I’ve written scripts for displaying weather(focus only on rain, v. important for dog walks when your dog hates rain) in [`i3status-rust`](https://github.com/greshake/i3status-rust), transforming various CSV files from bank exports into [`beancount`](https://beancount.github.io/) format, or creating [Telegram bots](https://github.com/visciang/telegram).

A simple script may be just:

```elixir
#!/usr/bin/env elixir
IO.puts("Hello world")
```

Well, if it were that simple, I’d just do `echo "Hello world"` and skip Elixir, but then there’d be no point in writing this blog post, right? So, I have another, a bit more involved and what may seem like over-complicated template. It’s a starting point for more complex scripts, and I can either remove some of the parts I don’t need or start extending it.

## The template

```elixir
#!/usr/bin/env -S ERL_FLAGS=+B elixir
Mix.install([])

if System.get_env("DEPS_ONLY") == "true" do
 System.halt(0)
 Process.sleep(:infinity)
end

defmodule Hello do
 @moduledoc """
 <!-- TODO -->

 ## Usage

 $ bin/hello --help

 """

 @args [help: :boolean]
 def main(args) do
 {parsed, args} = OptionParser.parse!(args, strict: @args)
 cmd(parsed, args)
 end

 defp cmd([help: true], _), do: IO.puts(@moduledoc)
 defp cmd(_parsed, _args) do
 IO.puts(@moduledoc)
 System.stop(1)
 end
end

Hello.main(System.argv())
```

First of all, the shebang is not the usual thing you’d expect. It configures an additional flag for [`erl`](https://www.erlang.org/doc/man/erl.html). This flag, per [docs](https://erlang.org/documentation/doc-4.9.1/erts-4.9.1/doc/html/erl.html), “De-activates the break handler for (SIGINT) ^C and ^\ “. I’ll expand on it in [Signals](#signals-some-are-not-like-the-others) section.

Now, we’re executing, and there’s a ready-to-go [`Mix.install/2`](https://hexdocs.pm/mix/Mix.html#install/2) statement where we can add additional dependencies. To make HTTP requests, we can add battries-included HTTP client like `{:req, "~> 0.4"}`. Processing JSON? `{:jason, "~> 1.4"}`. We can search for packages with `mix hex.search [package-name]` or lookup the latest version `mix hex.info [package-name]`

```sh
$ mix hex.info req
Req is a batteries-included HTTP client for Elixir.

Config: {:req, "~> 0.4.8"}
Releases: 0.4.8, 0.4.7, 0.4.6, 0.4.5, 0.4.4, 0.4.3, 0.4.2, 0.4.1, ...

Licenses: Apache-2.0
Links:
 Changelog: https://hexdocs.pm/req/changelog.html
 GitHub: https://github.com/wojtekmach/req
```

Another unusual block is `if System.get_env("DEPS_ONLY") do ... end`. If the script doesn’t have any dependencies, I’d just delete it or let it be. It’s useful in cases when the script has dependencies. We can use this block to cache dependencies and compilation of the script, skipping the execution of the rest of the script. This is handy for CI setups or when building container images. For CI, I’d also define a directory where the dependencies should be cached. for GitLab CI, a job may looks like this:

```yaml
print-hello:
 # https://hub.docker.com/r/hexpm/elixir/tags?page=1
 image: hexpm/elixir:1.16.1-erlang-26.2.2-debian-bookworm-20240130-slim
 variables:
   MIX_INSTALL_DIR: "$CI_PROJECT_DIR/.cache/mix"
 cache:
   - key: elixir-cache
 paths:
   - .cache
 script:
   - DEPS_ONLY=true bin/hello
   - bin/hello
```

After that, there’s a module where we’ll be documenting what the script is about, defining options for parsing arguments and failing gracefully when invalid options are passed, ensuring proper error exit code. That’s also where we’d extend the script, before last `cmd/2`. CLI argument parsing is done with built-in [OptionParser](https://hexdocs.pm/elixir/OptionParser.html).

This structure may seem a bit verbose and may look like a lot of boilerplate, but again, if it were simple, we’d have just stayed with the bash in the first place. Here, this structure can easily grow with the script.

I used to define inline functions like `print_help = fn -> ... end` or `process_args = fn (args) -> .. end` but in the end, working within a module is cleaner, and no need to look if given function is anonymous function (`.()` call) or module’s function.

With the template in place, we’re ready to add some logic to it. Elixir can do quite a bit just with the standard library, but there are also some gotchas. Let’s go through some common needs.

### **Output**

[`IO`](https://hexdocs.pm/elixir/IO.html) will probably be the most often used module. It can be used to write stdout with functions like `IO.write/1`, `IO.puts/1`, or to `stderr` with their equivalent 2-argument calls like `IO.puts(:stderr, "Error")`. We can also read inputs with `IO.read/2`. Any writing or reading can be also handled as a stream with [`IO.stream/2`](https://hexdocs.pm/elixir/IO.html#stream/2).

```elixir
IO.puts("Hello")
IO.puts(:stderr, "Invalid argument")

IO.stream(:stdin, :line) # that's the default
|> Enum.map(&String.trim_trailing(&1, "\n"))
|> Enum.map(&String.reverse/1)
|> Enum.map(&IO.puts/1)
```

### **Colors**

There’s no need to define color codes manually. With [IO.ANSI](https://hexdocs.pm/elixir/IO.ANSI.html), we can add text and background colors easily.

```elixir
iex(1)> IO.ANSI.blue_background()
"\e[44m"
iex(2)> v <> "Example" <> IO.ANSI.reset()
"\e[44mExample\e[0m"
iex(3)> v |> IO.puts()
Example
:ok
```

![Pasted_image_20240211121425.png](https://arathunku.com/b/2024/shell-scripting-with-elixir/Pasted_image_20240211121425.png)

### **Exit code**

Another quick one, there’s [`System.stop(exit_code)`](https://hexdocs.pm/elixir/1.12/System.html#stop/1) to gently shutdown VM, what may not be obvious is that it’s async process. Make sure to call `Process.sleep(:infinity)` after it to block the execution. This ensures that all the applications are taken down gently. There’s alternative of [`System.halt/1`](https://hexdocs.pm/elixir/1.12/System.html#halt/1) and it forces immediate shutdown of Erlang runtime system.

### **Subprocesses**

For one-off commands, [`System.cmd/3`](https://hexdocs.pm/elixir/main/System.html#cmd/3) is enough.

```elixir
iex(1)> {output, 0} = System.cmd("git", ["rev-parse", "--show-toplevel"])
{"/home/arathunku/code/github.com/arathunku/elixir-cli-template-example\n", 0}
```

With pattern matching we ensure immediate exit if there’s any other exit code than the successful one, and we can process the output.

It gets more tricky if you want to create another BEAM process while continuing with the rest of the execution. If something goes wrong or Erlang system crashes, OS process might get left behind. In these cases, instead of reinveting a wheel of managing OS processes, it’s good occasion to make a use of this `Mix.install/2` at the beginning and add [MuonTrap](https://github.com/fhunleth/muontrap). It will ensure the processes are, as described in README, well-behaved.

```elixir
Mix.install([{:muontrap, "~> 1.0"}])

defmodule Hello do
  ...
  defp cmd([], []) do
    _pid = spawn_link(fn ->
      MuonTrap.cmd("ping", ["-i", "5", "localhost"], into: IO.stream(:stdio, :line))
      System.stop()
    end)
    Process.sleep(:infinity)
  end
end
```

### Signals, some are not like the others

It’s tricky and it will probably not behave as you’d expect based on your experience in other languages. Some signals are handled by default by Erlang system, for more details check [nice documentation in PR](https://github.com/erlang/otp/pull/1315). For scripting… usually signals don’t matter that much, at least in my case. If you spawn a GenServer, it’ll receive [`terminate/2`](https://hexdocs.pm/elixir/GenServer.html#c:terminate/2) for cleanup, assuming gentle shutdown.

We can still skip `BREAK` menu(^C) and exit immediately if we start with `ERL_FLAGS=+B elixir`. This is why it’s in the template at the beginning. Some other signals can be cough by swapping default `erl_signal_server`, but not all of them\*. In the example above we’ll do just that, handle what we’re interested it and defer rest to the default handler.

> \*At this moment, `INT` cannot be trapped, [see this issue](https://github.com/erlang/otp/issues/4583)

```elixir
defmodule Signals do
 @behaviour :gen_event
 def init(_), do: {:ok, nil}

 def handle_event(:sigusr1, state) do
 IO.puts("USR1, continue...")
 {:ok, state}
 end

 def handle_event(:sigterm, _state) do
   IO.puts("Ok, ok, let me take a moment and exit...")
   Process.sleep(3)
   System.stop()
 end

 def handle_event(signal, state) do
   :erl_signal_handler.handle_event(signal, state)
 end

 def handle_call(_, state), do: {:ok, :ok, state}

 def terminate(reason, _state) do
   IO.puts("Goodbye! #{inspect(reason)}")
 end
end

with :ok <- :gen_event.swap_handler(
 :erl_signal_server, {:erl_signal_handler, []}, { __MODULE__ , []}
) do
  IO.puts("I'll wait for signals!")
  Process.sleep(:infinity)
else
 err ->
   IO.warn("Something went wrong. err=#{inspect(err)}")
end
```

### **Testing**

In Rust, we can write tests next to the code with `#[cfg(test)]`, and these will run when `cargo test` is executed. Did you know you can do kind of a similar thing in Elixir scripts? There’s no magic here, we need to create a test module and trigger [`ExUnit`](https://hexdocs.pm/ex_unit/ExUnit.html)

```elixir
if System.get_env("MIX_ENV") == "test" do
 ExUnit.start()

 defmodule HelloTest do
 use ExUnit.Case, async: true
 import ExUnit.CaptureIO

 test "prints a message when no arguments are passed" do
 assert capture_io(fn -> Tests.main([]) end) == "Hello World\n"
 end

 test "prints help for unknown arguments" do
 assert capture_io(fn -> Tests.main(["--help"]) end) =~ "Example of adding ExUnit"
 end
 end
else
 Hello.main(System.argv())
end
```

## Going beyond scripts

[`OptionParser`](https://hexdocs.pm/elixir/OptionParser.html) is good. Maybe it doesn’t do all the stuff that something like [Rust clap](https://docs.rs/clap/latest/clap/) does but it’s absolutely getting the job done.

You can go beyond simple CLI scripts and build full TUI apps. Progress bars? Charts? Text editor? Here, `Ratatouille` comes to the rescue. [Simple TUI counter example](https://github.com/arathunku/elixir-cli-template-example/blob/main/bin/tui-counter) or more advanced ones - [more advanced examples](https://github.com/ndreynolds/ratatouille/tree/cc7b6a37e0b1757cd89cc5084343814a79dd86dc?tab=readme-ov-file#example-applications).

If you’d like to see even more examples with [`Mix.install/2`](https://hexdocs.pm/mix/Mix.html#install/2), make sure to check [mix\_install\_examples](https://github.com/wojtekmach/mix_install_examples)! There’re examples of HTTP servers, CSV parsing, web scraping, machine learning or [full Phoenix LiveView file uploader](https://github.com/wojtekmach/mix_install_examples/blob/main/phoenix_live_view_upload_image.exs)(!!!), but at this point you may consider just using `mix new ...` and setting up a proper Mix project. After that, you can use [burrito](https://github.com/burrito-elixir/burrito) to ship a single Elixir CLI binary for end-users.

Is Elixir more complicated than Ruby od Python as a bash replacement? It depends, of course it depends. If you already know Python or Ruby well, you’ll probably prefer them but Elixir can absolutely be used too! It’s a bit on the slow side to start up, may not be a best choice to implement PS1, but if the startup speed doesn’t matter that much and you want go have very ergonomic language for scripting at your hand - it’s great.

## Closing note

Writing this article was kind of a strange experience. You may think it’s a bit light on examples and details, and it is done on purpose. There’s strong focus on documentation within the community and I didn’t want to repeat what’s already out there in official docs and libraries. Just look at this beautiful [`System.cmd/3`](https://hexdocs.pm/elixir/main/System.html#cmd/3) documentation or [`IO.ANSI`](https://hexdocs.pm/elixir/main/IO.ANSI.html) docs, and it’s all available in [`iex`](https://hexdocs.pm/iex/IEx.html) with [`h/1`](https://hexdocs.pm/iex/IEx.Helpers.html#h/1).

When writing the article, I’ve dumped all my scripts and tests into this repository. Thanks for reading!