TOML: the annoying parts
Tags: tooling
Published: 2020-11-18 01:57:37

If your project needs a configuration file, [TOML](https://toml.io/en/) is usually the best format.
It's definitely easier to read and write than JSON, and much less fragile and ambiguous than YAML. There are comments, real booleans, and multi-line string literals.
It also offers better flexibility than classic INI-like formats, and there's a [spec](https://toml.io/en/latest) where users can learn the syntax.

I went for TOML when I wrote the [Soupault](https://soupault.neocities.org) static site generator/HTML processor, and in these 1.5 years that I've been using it and working on it, I never thought "I wish I'd chosen something else". None of the users told me they wish it used something else either.
However, there are still annoying points that I hope will be improved some day.

## Paste-ability

Don't we all like to paste configuration snippets from tutorials and other people's configs?

When nested tables are involved, TOML doesn't make it especially easy to paste, or even discuss configuration options.

Consider this config:

```toml
[settings]
  verbose  = true
  debug    = false
```

How are we going to talk about those options in the docs?
Let's try this: "When `settings.verbose = true`, build progress output will be printed to the console".
Users cannot paste `settings.verbose = true` into that config because TOML doesn't allow re-defining options.

This is an invalid config because the `verbose` option is re-defined:

```toml
settings.verbose = true

[settings]
  verbose  = false
  debug = false
```

It gets funnier. This is technically valid TOML:

```toml
[settings]
  verbose  = false
  debug = false

settings.verbose = true
```

It just doesn't mean what one may think it means: the last option is interpeted as `settings.settings.verbose`.

So, best you can do when talking about options is to use lengthy descriptions like "Option `verbose` in the `[settings]` table".
Well, or avoid using tables, but it defeats the purpose.

If I were designing a format from scratch, I would likely avoid having both `[table]\n key` and `table.key` notations because they clearly don't compose.

## Unordered tables

Sometimes I wish the standard mandated that tables were ordered dicts. If you need order, you can only use lists of tables (and their syntax and behaviour are rather clumsy and not obvious), or provide a different way to specify the order.

When I wanted to make it possible to use output of one "widget" as an input for another, I chose to add an explicit `after` option.

```toml
# Highlights the link to the current section in the navigation menu
[widgets.section-link-highlight]
  after = "insert-nav-menu"
  widget = "section-link-highlight"

  active_link_class = "nav-active"
  selector = "div#nav"
```
Maybe the fact that users can specify multiple dependencies with `after = ["widget-1", "widget-2"]` is a good thing, but it may also make complex configs hard to follow.

I suppose one reason TOML designers made tables unordered is that it's how hash tables/dicts work in most languages. In Python you have `OrderedDict`, and in OCaml or Haskell you can use a _(string × α) list_. There's no equally common and straightforward way to do that in JS or many other languages.

## Automatic manipulation

Unordered tables and "more than one way to do it" mean parsing and formatting TOML objects isn't a lossless operation. None of existing libraries can parse a TOML file and print it back as it came.

It's hard to blame them. Preserving comments, line numbers, and whether a table was normal or inline for example—these are all pretty hard tasks. They are also unnecessary if you only want to de/serialize TOML.

If you are writing a validator or convertor, then things are different.

When I had to make some [breaking changes](https://soupault.neocities.org/blog/soupault-2.0.0-beta1-release/) to the config syntax, I made a convertor from the old format to new to simplify migration for users.

However, what it outputs is quite far from the config that comes in. You'll lose all your comments at the very least.
I tried to alleviate it by making the convertor output a detailed log of the changes it's making so that users can replicate them by hand.

## Conclusion

I don't regret going with TOML for the configuration format. There's still room for improvement in its tooling and usage conventions though, and I hope that I or someone else will be able to make those improvements.