Doctave CLI 0.2.0: A Benchmarking Story
Tags: markdown, showdev, writing, rust
Published: 2021-09-15 12:34:15

The [Doctave CLI](https://www.github.com/Doctave/doctave) is a free to use open source documentation generator. It takes your Markdown files and converts them into a beautiful [documentation site](https://cli.doctave.com). Today we have released version 0.2.0, which brings some cosmetic improvements and a fully _in-memory development server_ for local development. We'll also talk about how this surprisingly _did not_ make the CLI faster like we expected.

## Serving your docs from memory

Before Doctave would rewrite the whole documentation site to disk every time you made a change to your Markdown files. It would pretty much `rm -r` the `site` directory, used to house the generated HTML, and regenerate the docs site from scratch. Because Doctave (which is written in Rust) is able to generate the whole site in ~tens of milliseconds, we hadn't looked into optimizations here yet.

But there were two non-performance related issues that bothered us about this approach:

1. You would end up with a `site` directory that didn't really serve any purpose while developing
   your docs
2. While very minor in scale in our case, constantly writing and deleting small files from disk
   isn't something that modern SSDs like to do

This is why Doctave 0.2.0 never writes the HTML documentation site to disk until you run
`doctave build --release`. In `serve` mode when you write your documentation, the embedded web server serves them directly from memory.

```
$ doctave serve Doctave | Serve Starting development server...

Server running on http://0.0.0.0:4001/

    File docs/README.md updated.
    Site rebuilt in 31.698781ms
```

Wait a minute - shouldn't this be faster now that we are not writing files to disk?

## Why is this not faster?

After completing this feature, I was surprised to see that Doctave wasn't generating sites noticeably faster than before. I was expecting a speedup due to us not spending as much time on IO. This was not happening.

After double checking to make sure the code was doing what I expected, it was time to benchmark things to see where the time was being spent. After some digging I found the answer: _generating the search index_.

Doctave comes with offline search built in. We use the [elasticlunr-rs](https://github.com/mattico/elasticlunr-rs) crate to generate a search index that is compatible with the [elasticlunr.js](https://github.com/weixsong/elasticlunr.js) library. You can see it in action by going to [our docs](https://cli.doctave.com) (built with the CLI, naturally) and hitting the letter `s` on your keyboard to focus on the search bar. The searching happens entirely client-side.

It turned out that we are spending _~70% of the site generation time creating the search index_. This is completely reasonable, as this is a somewhat CPU intensive task that parses the input files and generates the index. The time spent doing IO writing files to disk is completely negligible in comparison, and thus did not move the needle much at all. On top of that, the CLI was already writing files to disk in parallel. Moving things in-memory did not save us much time.

So while Doctave 0.2.0 is technically faster, this was not a big performance win. There is some more work we can do to parallelize the build process further - maybe in the next release. In the meantime, Doctave will be more friendly on your SSD and not pollute your workspace as much.

Still, this was a change for the better. It just wasn't the big performance win I suspected.

## Conclusions

I think this was an instructive little story about performance and benchmarking that was worth sharing. It's often the case that when we make assumptions about the performance of a system that turn out to be wrong once you measure things. This is just another reminder: **always measure**.

Finally, do try out [Doctave 0.2.0](https://github.com/Doctave/doctave)! If you're looking for a batteries-included documentation generator that doesn't require plugins, doesn't pollute your repository with loads of files, and doesn't need a specialized environment to run, Doctave may be a good choice for you. It also comes with [Mermaid JS diagram](https://mermaid-js.github.io/mermaid/#/) support and dark mode!

You can host sites generated by Doctave on GitHub pages, or your favorite static site hosting provider. We are also building a [specialized host](https://www.doctave.com) for teams using Doctave on multiple projects - check it out if you're using Doctave at your organization.

Let me know if you're using it to document your project. I can be reached at `nik@doctave.com`. I'd be excited to hear what you think.

### Want to try out Doctave?

If you're on Mac using Homebrew, you can install Doctave with the following command: 

```
brew install doctave/doctave/doctave
```

If you're a Rust fan and want to use Cargo, you can do so too: 

```
cargo install --git https://github.com/Doctave/doctave --tag 0.2.0
```

We also have [prebuilt static binaries](https://github.com/Doctave/doctave/releases/tag/0.2.0) for Mac, Linux, and Windows.