Using LibUV in Neovim
Tags: vim, neovim, lua, libuv
Published: 2020-01-08 13:07:53

---
title: Using LibUV in Neovim
published: true
date: 2020-01-07 00:00:00 UTC
tags: [vim, neovim, lua, libuv]
canonical_url: https://teukka.tech/vimloop.html
---

## Off The Main Loop: Async Actions within Neovim

Neovim embeds the libuv library in the editor and exposes lua (and to some extent vimscript through the `jobstart` function) bindings for interacting with the library’s API. This allows actions in the editor to happen asynchronously, keeping the main editor loop clear for user input. What this means practically for users is that actions which take a long time such as grepping through large projects, generating ctags, or linting can now be done in the background without blocking the user’s ability to keep editing text.

Let’s explore how to use the libuv lua bindings in Neovim to create useful asynchronous actions. I will cover two examples: using the [pandoc](https://pandoc.org) program to convert a markdown file to html, and using [ripgrep](https://github.com/BurntSushi/ripgrep) to search within a project. Both of these examples use the libuv bindings differently, but still illustrate well the capabilities of libuv inside Neovim.

## First Example: Convert Markdown Files with Pandoc

Pandoc is a powerful tool for converting files into and different formats. I write many Markdown documents and it is useful to convert them into HTML for my blog. In order to do this in an asynchronous way, we will use `vim.loop` from Neovim’s lua library. I would recommend going through the documentation on `vim.loop` as you code along with this post. Let’s start by creating a basic lua function outline in our `packagepath`:

```lua
-- in ~/.config/nvim/lua/markdown.lua
local M = {}
local loop = vim.loop
local api = vim.api

function M.convertFile()
  local shortname = vim.fn.expand('%:t:r')
  local fullname = api.nvim_buf_get_name(0)
 -- loop logic goes here
end

return M

```

In this snippet, we first get the shortened name of the file (i.e. `vimlooppost` in `vimlooppost.md`), and the full name of the file (i.e. `~/blog/posts/vimlooppost.md`). We will use both of these variables later when we spawn the pandoc process. In order to get a better understanding of what `vim.loop.spawn` does, let’s head over to the [libuv documentation](https://github.com/luvit/luv/blob/master/docs.md#uvspawnfile-options-onexit). Here we can see that it takes an options table, an `onexit` callback, and returns the processId and the handle of the spawned process. Using this information, let’s build out the rest of the `convertFile` function:

```lua

-- in ~/.config/nvim/lua/markdown.lua
local M = {}
local loop = vim.loop
local api = vim.api

function M.convertFile()
  local shortname = vim.fn.expand('%:t:r')
  local fullname = api.nvim_buf_get_name(0)
  handle = vim.loop.spawn('pandoc', {
    args = {fullname, '--to=html5', '-o', string.format('%s.html', shortname), '-s', '--highlight-style', 'tango', '-c', '--css=pandoc.css'}
  },
  function()
    print('DOCUMENT CONVERSION COMPLETE')
    handle:close()
  end
  )
end

return M

```

Pandoc receives as arguments the strings listed in the `args` table, and when the process finishes, we echo a success message and close the handle. Using this in our vimrc is as simple as:

```vim
" in ftplug/markdown.vim

nnoremap <leader>c :lua require'markdown'.convertFile()<CR>

```

## Second Example: Async Grep

Much like in the first example, we will be relying on an external program to do the heavy lifting, ripgrep. Unlike in the first example, we don’t only want to kick off a background process, but also use the values generated in this process inside of Neovim. For that, we will need to pass a set of file descriptors to `vim.loop.spawn`:

```lua
-- in ~/.config/nvim/lua/tools.lua
local M = {}
local loop = vim.loop
local api = vim.api

function M.asyncGrep(term)
  local stdout = loop.new_pipe(false) -- create file descriptor for stdout
  local stderr = loop.new_pipe(false) -- create file descriptor for stdout
  handle = loop.spawn('rg', {
    args = {term, '--vimgrep', '--smart-case'},
    stdio = {stdout,stderr}
  },
  function()
    stdout:read_stop()
    stderr:read_stop()
    stdout:close()
    stderr:close()
    handle:close()
  end
  )
  loop.read_start(stdout, onread) -- TODO implement onread handler
  loop.read_start(stderr, onread)
end
return M

```

Our function, `asyncGrep` will take a search term as an argument which it then passes to `ripgrep` in the `loop.spawn` call. After the process is spawned, we need to start reading the output into our file descriptors, which will then call the `onread` callback which we have yet to implement. Let’s implement the `onread` callback now:

```lua
-- in tools.lua
local results = {}
local function onread(err, data)
  if err then
    -- print('ERROR: ', err)
    -- TODO handle err
  end
  if data then
    table.insert(results, data)
  end
end

```

The `onread` callback takes data written to our file descriptor by the `ripgrep` process and appends a table called `results`. We now want to add functionality to our `asyncGrep` function that will allow us to use these results to set the quickfix list:

```lua
function M.asyncGrep(term)
  local stdout = loop.new_pipe(false) -- create file descriptor for stdout
  local stderr = loop.new_pipe(false) -- create file descriptor for stdout
  local function setQF()
    vim.fn.setqflist({}, 'r', {title = 'Search Results', lines = results})
    api.nvim_command('cwindow')
    local count = #results
    for i=0, count do results[i]=nil end -- clear the table for next search
  end
  handle = loop.spawn('rg', {
    args = {term, '--vimgrep', '--smart-case'},
    stdio = {stdout,stderr}
  },
  function()
    stdout:read_stop()
    stderr:read_stop()
    stdout:close()
    stderr:close()
    handle:close()
    setQF()
  end
  )
  loop.read_start(stdout, onread) -- TODO implement onread handler
  loop.read_start(stderr, onread)
end

```

If you run this function, you will encounter this error message: `lua/tools.lua:122: E5560: vimL function must not be called in a lua loop callback`. In order for any vim functions to be called within a lua loop callback, they need to be wrapped in `vim.schedule_wrap`. Wrapping vim functions in `vim.schedule_wrap` is necessary since it schedules the callbacks to be invoked when it is safe, bridging the gap between the libuv event loop and the internal Neovim main loop. To learn more about `vim.schedule_wrap`, check out `:h schedule_wrap`. Let’s fix this error and see what our function looks like when it’s all put together:

```lua
-- in ~/.config/nvim/lua/tools.lua
local M = {}
local loop = vim.loop
local api = vim.api
local results = {}

local function onread(err, data)
  if err then
    -- print('ERROR: ', err)
    -- TODO handle err
  end
  if data then
    table.insert(results, data)
  end
end

function M.asyncGrep(term)
  local stdout = vim.loop.new_pipe(false)
  local stderr = vim.loop.new_pipe(false)
  local function setQF()
    vim.fn.setqflist({}, 'r', {title = 'Search Results', lines = results})
    api.nvim_command('cwindow')
    local count = #results
    for i=0, count do results[i]=nil end -- clear the table for the next search
  end
  handle = vim.loop.spawn('rg', {
    args = {term, '--vimgrep', '--smart-case'},
    stdio = {stdout,stderr}
  },
  vim.schedule_wrap(function()
    stdout:read_stop()
    stderr:read_stop()
    stdout:close()
    stderr:close()
    handle:close()
    setQF()
  end
  )
  )
  vim.loop.read_start(stdout, onread)
  vim.loop.read_start(stderr, onread)
end

return M

```

Let’s use this newly created async function in our vimrc:

```vim
" in init.vim
command! -nargs=+ -complete=dir -bar Grep lua require'tools'.asyncGrep(<q-args>)

```

Now we can call `:Grep searchTerm` and get results without blocking the main editor loop!

## What’s Next?

Having libuv bindings in Neovim unlocks a lot of potential for extending the functionality of your editor. The ability to asynchronously spawn other process can be used for linting, file watching, formatting, and much more. Free yourself from main loop blockage and try experimenting with `vim.loop`.