Rendering markdown with Markdoc in Rails
Tags: markdoc, stripe, rails, markdown
Published: 2022-07-13 18:42:00

Markdoc is Stripe’s newly open-sourced framework for authoring content. It’s the tools used to build the Stripe documentation and the framework on top of which many of the interesting and powerful features of the Stripe docs are built.

Every couple years, I can't resist yak shaving and rebuilding my personal site, [cjav.dev](https://cjav.dev). It’s built with [11ty](https://www.11ty.dev/). Currently, I'm just rendering long lists of links to blog posts, videos, and podcast episodes. It’s all haphazardly organized and I typically use CMD+F to find an old post that I need to reference. One benefit to the current system is how easy it is to author content and re-deploy. If I have an idea for a new blog post, I create a new markdown file and get to work. Once it’s got some words, no need to polish, I’ll commit and push which deploys the site to [Netlify](https://www.netlify.com/).

Over the past year, I’ve built some fun integrations with the [Transistor.fm](https://www.youtube.com/watch?v=ktEjaABp4aw) and YouTube APIs, and even built a little tool to visualize my progress towards my goals on [Strava](https://www.youtube.com/watch?v=KXUxGUKtcbM). 

Some goals for the 2022 rebuild: 

1. Easy to author content
2. Easy to deploy
3. Easy to add API integrations
4. Better surfacing of content
5. Playground for automations (background jobs, small database)

I [polled Twitter](https://twitter.com/cjav_dev/status/1542560813734567944) with a few options for a potential stack. My statistically insignificant poll shows [bridgetown.rb](https://www.bridgetownrb.com/) followed closely by [Markdoc](https://markdoc.dev/) + Next.js as the winners.

Some initial experimentation with Bridgetown.rb was very smooth. All of my existing content ported over nicely, the authoring experience was solid, and I could use Tailwind out of the box ([h/t](https://twitter.com/andrewmcodes/status/1542568534336688129) @andrewmcodes). It had support for API calls with Roda and I could see that working great to clone exactly what I have today. Bridgetown.rb is a great choice for any Rubyist building a personal site today.

As part of this exploration, I wanted to see if I could get the new Markdoc tools from Stripe to render my existing content and enable me to build custom Markdoc components that play nicely inside of a Rails app.

Markdoc ships with two built in renderers, one for React and one for HTML. The HTML example includes some Lit components 🔥.  Since Rails already has lots of nice HTML-based tooling, I wanted to stick with the HTML renderer. 

At a high level, I knew I needed to:


1. Add Markdoc to the app
2. Create an entry point for the Markdoc framework
3. Add a build step that would recompile the Markdoc rendering logic if I added or removed components
4. Create a way to call into the built JavaScript from Ruby
5. Build some front end Lit components that work with the web components rendered by Markdoc

Let’s walk through the steps with a brand-new Rails application:


## Create a new Rails application

```bash
rails new markdoc-rails -j esbuild
cd markdoc-rails
```

We’ll use `esbuild` to build our client side JavaScript, but also our markdoc renderer.


## Install Markdoc

This’ll add `@markdoc/markdoc` to the root package.json. Which works, but I’m not sure this is the best approach. Maybe it should be in an isolated package.json in the `Rails.root/markdoc` directory where we’ll wire up the rendering logic. Leave a comment if you have a strong opinion! 

```bash
npm install @markdoc/markdoc
```

We’ll create a new directory in the root of the rails app called `markdoc` this will contain the Markdoc rendering scripts and the schemas for our custom Markdoc components that we’ll use to juice up our content and make it more interactive, but still easy to author.


```bash
mkdir -p markdoc/{src,dist}
```

A new file named `markdoc/src/index.js` will contain our source for our rendering JavaScript and might look something like this (to start):

```javascript
const Markdoc = require('@markdoc/markdoc')

const tags = {};

function markdocRender(doc) {
  const ast = Markdoc.parse(doc);
  const content = Markdoc.transform(ast, { tags });
  return Markdoc.renderers.html(content);
}

globalThis.markdocRender = markdocRender;

// is this thing on?
console.log(markdocRender(`
# Testing 1.2.3.

- bang 💥
`))
```

Notice that I’m attaching the `markdocRender` function to the `globalThis`. That’s so we expose that method from the built JavaScript and can access it from Ruby later. If you know a better way to do this, please share 🤗.

Depending on your node version, you should be able to run this now with:

```bash
node markdoc/src/index.js
```

which prints the following to the console:

```bash
<article><h1>Testing 1.2.3.</h1><ul><li>bang 💥</li></ul></article>
```


## Add a build step

Eventually, we’ll want to call this JavaScript from a Rails controller using [ExecJS](https://github.com/rails/execjs) or [MiniRacer](https://github.com/rubyjs/mini_racer) or some similar tool. None of the Ruby-to-JavaScript gems I found were sophisticated enough to know how to load npm modules with common.js or ES module syntax, so my solution is to just build the JavaScript with a watcher and have that run as part of `bin/dev`. 

Let’s update package.json with two new build scripts, one for our client side JS (standard), and this new one that will build our Markdoc renderer:

```json
  // ...
  "scripts": {
    "build": "esbuild app/javascript/*.* --bundle --sourcemap --outdir=app/assets/builds",
    "build:markdoc": "esbuild markdoc/src/index.js --bundle --sourcemap --outdir=markdoc/dist"
  }
}
```

Then, we can update our `Procfile.dev` so that foreman can spin up this blog builder for us when we start the application:

```yaml
web: bin/rails server -p 3000
js: yarn build --watch
markdoc: yarn build:markdoc --watch
```

We’re ready to move in to Ruby 💎.


## Create a controller and route

For my use-case, I wanted a route like `/posts` to render the list of markdown files and `/posts/filename` to render the content of the post. Ultimately, I might want to build a frame around the content of a post (think nav, header, footer etc.) so I want to have a show template where I’ll embed the content of a post written in markdown.

Generate a Posts controller

```bash
rails g controller Posts index show
```

Create routes


```ruby
# config/routes.rb
Rails.application.routes.draw do
  resources :posts, only: [:index, :show]
end
```

You can look at the [source on GitHub](https://github.com/cjavilla-stripe/markdoc-rails/blob/main/app/controllers/posts_controller.rb) if you’re interested in how I build the index route, as I’m most excited to show you the show route where the Markdoc magic happens.


## Rendering markdown with Markdoc from Ruby

First, we’ll need some markdown to play with. I created a few files in a new directory in the root of the app at `Rails.root/posts`:


`/posts/2022-06-04-test.md` looks like this:

```markdown
---
title: Test with links
date: 2022-06-04
---

# Another test

\[back\](/posts)

\[out\](https://twitter.com/cjav_dev)
```

`/posts/2022-07-11-test-with-code.md` looks like this (well sorta, dev.to also has a hard time with escaping backticks!):

```markdown
---
title: Test with code
date: 2022-07-11
---

# This is a test

## This is a subtitle

- bullet

\`\`\`ruby
  class Cat
    def meow
      puts "hi"
    end
  end
\`\`\`

{% callout type="warning" %}
This is a warning callout
{% /callout %}
```

When we visit `http://localhost:3000/posts/2022-06-04-test` we should see the rendered content of the first post.

In the show action of our PostsController, the name of the post is available in `params[:id]`. I thought about splitting up the path later so that it was based on date, maybe something like `/posts/2022/06/04/test` (which is what bridgetown does). For now, we’ll keep it simple.

```ruby
def show
  post_name = params[:id]
```

Next, we’ll need the raw markdown content which we’ll read in from the file on disk. Long term this directory could be configurable, just slapping on `.md` is probably a bit rigid, but it works fine.

```ruby
def show
  post_name = params[:id]

  content = File.read(File.join(Rails.root, "posts", "#{post_name}.md"))
```

Many of the posts on my personal site use code fences with the three backticks. Since this content is passed from Ruby to JavaScript as a string, we’ll escape those backticks. It took me several tries to figure out how to actually do this. In the second argument to `gsub` I was passing a double quoted string at first. A pattern that I’ve solidified in my head is: double quotes when doing string interpolation, single quotes for non string interpolation. But the core difference between double and single quotes comes down to the escape sequences for each! Checkout Tom de Bruijn’s [article here](https://blog.appsignal.com/2016/12/21/ruby-magic-escaping-in-ruby.html) for much more detail.

```ruby
def show
  post_name = params[:id]

  content = File.read(File.join(Rails.root, "posts", "#{post_name}.md"))

  # Escape fences so that MiniRacer can evaluate:
  content = content.gsub(/`/, '\\\`')
```

Now that we have the content of the post, we need to pass that down to JavaScript. I also tried a few things for this part starting with ExecJS and ultimately landing on MiniRacer, but ExecJS probably would’ve worked fine.

First add the gem:

```bash
bundle add mini_racer
```

Then we’ll read in the built JavaScript and evaluate with MiniRacer.

```ruby
# Read in the built Markdoc rendering script
entry_point = File.read(File.join(Rails.root, "markdoc", "dist", "index.js"))

# Evaluate the JavaScript
context = MiniRacer::Context.new
context.eval(entry_point) 
```

Finally, we call our `markdocRender` JavaScript function passing in the raw markdown content we read in earlier and store it in an instance variable so that we can render that in our view template:

```ruby
@rendered_content = context.eval("markdocRender(`#{content}`)")
```

All together now:

```ruby
class PostsController < ApplicationController
  def show
    post_name = params[:id]

    content = File.read(File.join(Rails.root, "posts", "#{post_name}.md"))

    # Escape fences so that MiniRacer can evaluate:
    content = content.gsub(/`/, '\\\`')

    # Run from the dist entry point
    # New markdoc components aren't added for my blog
    # too frequently, but when they are we need to rebiuld
    # with `npm run build:markdoc`
    entry_point = File.read(File.join(Rails.root, "markdoc", "dist", "index.js"))

    # Render as markdoc
    context = MiniRacer::Context.new
    context.eval(entry_point)
    @rendered_content = context.eval("markdocRender(`#{content}`)")
  end
end
```

Our show template will work with rendering the html safe `@rendered_content` 🎉:

```html
<%= @rendered_content.html_safe %>
```

![Screenshot showing a rendered post](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h21rkkcnitn4pfubzoma.png)
 

## Adding Markdoc components with [Lit](https://lit.dev/)

One of the major benefits of Markdoc is the authoring system is very clean. You write markdown and maybe use a few components but for the most part, the authoring experience is focused on the content instead of the code around it. This is powered by Markdoc components which are implemented outside of the markdown files and used as higher level snippets while authoring.

In the Markdoc docs, you’ll find this Callout example:

```markdown
{% callout title="Yellow Snow" type="warning" %}
Don't eat the yellow snow.
{% /callout %}
```

Which could render with an icon and a yellow background or however we want all callouts across our blog to render.

This works by creating new [tags](https://markdoc.dev/docs/tags) that extend the markdown syntax. 

We’ll define a new `callout` tag by creating a Markdoc schema:

```javascript
// markdoc/src/schema/Callout.markdoc.js
module.exports = {
  render: 'Callout',
  description: 'Display the enclosed content in a callout box',
  children: ['paragraph', 'tag', 'list'],
  attributes: {
    type: {
      type: String,
      default: 'note',
      matches: ['caution', 'check', 'note', 'warning'],
      errorLevel: 'critical',
      description:
        'Controls the color and icon of the callout. Can be: "caution", "check", "note", "warning"'
    },
    title: {
      type: String,
      description: 'The title displayed at the top of the callout'
    }
  }
};
```

To unlock tag extensions, we’ll need to update our rendering logic to pass in a configuration object during the transform step:

```javascript
const Markdoc = require('@markdoc/markdoc')
const callout = require('./schema/Callout.markdoc')

const tags = {
  callout
};

function markdocRender(doc) {
  const ast = Markdoc.parse(doc);
  const content = Markdoc.transform(ast, { tags });
  return Markdoc.renderers.html(content);
}

globalThis.markdocRender = markdocRender;
```

Now markdoc will render `\{% callout %}` tags as `<markdoc-callout>` web components. 


![Screenshot of the HTML for the markdoc-callout component](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/z47kimsfyhbwpw28hqu9.png)
 


We’re done extending markdown by adding our new markdoc tag, now we’ll add support for this new MarkdocCallout webcomponent using Lit.

Since this is a client side feature, I’ll create a new directory in `app/javascript/components` and add a new file called `callout.ts`. We’ll also create an `index.js` file that simply imports all of our components one by one, e.g.: `import './callout'`. 

Here we can build whatever logic we’d like to see for our custom callout:

```typescript
import { html, css, LitElement } from 'lit';
import {customElement} from 'lit/decorators.js';

@customElement('markdoc-callout')
export class MarkdocCallout extends LitElement {
  static styles = css`
    .note {
      background-color: #8792a2;
    }
    .caution {
      background-color: #d97917;
    }
    .check {
      background-color: #000000;
    }
    .warning {
      background-color: #ffd848;
    }
  `;

  static properties = {
    type: { type: String },
    title: { type: String },
  };

  constructor() {
    super();
    this.type = 'note';
  }

  render() {
    return html`<p class="${this.type}"><strong>${this.title}</strong><slot></slot></p>`;
  }
}
```

After updating our blog post to have this callout:

```markdown
{% callout title="Yellow Snow" type="warning" %}
Don't eat the yellow snow.
{% /callout %}
```

We now see this new component rendered to the screen (checkout the #shadow-root in the dev tools, too!):

![Screenshot of callout component rendered](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tshm2bgfztn0ts1jhl0g.png)


## Conclusion

This was a fun exercise. Checkout [the code](https://github.com/cjavilla-stripe/markdoc-rails) and if you go down the Rails + Markdoc path, I'd love to hear about it. Next todo is to experiment with deployment. Since we’re calling down into JavaScript from Ruby, I’m not 100% sure how many deployment environments that’ll play nicely in. Drop a comment and share your favorite personal sites for inspiration, too!