What is io_uring?
Tags: programming, linux, computerscience, opensource
Published: 2023-04-07 15:44:28

`io_uring` is a new asynchronous I/O API for Linux created by Jens Axboe from Facebook.

It aims to provide an API without the limitations of similar interfaces

- [`read(2)`](https://man7.org/linux/man-pages/man2/read.2.html)/[`write(3)`](https://man7.org/linux/man-pages/man2/write.2.html) are synchronous
- [`aio_read(3)`](https://man7.org/linux/man-pages/man3/aio_read.3.html)/[`aio_write(3)`](https://man7.org/linux/man-pages/man3/aio_write.3.html) provide asynchronous functionality, but only supports with files opened with `O_DIRECT` or in unbuffered mode
- [`select(2)`](http://man7.org/linux/man-pages/man2/select.2.html)/[`poll(2)`](http://man7.org/linux/man-pages/man2/poll.2.html)/[`epoll(7)`](http://man7.org/linux/man-pages/man7/epoll.7.html) work well with socks but do not behave as expected with regular files (always “ready”)

To have a more consistency API between file descriptors (sockets and regular files) we can use [`libuv`](https://libuv.org/) (will probably explore it in the future) or [`liburing/io_uring`](https://github.com/axboe/liburing) (the star of the show).

### How does it work?

As the name suggests, it uses ring buffers as the main interface for kernel-user space communication.

There are two ring buffers, one for submission of requests (submission queue or SQ) and the other that informs you about completion of those requests (completion queue or CQ).

These ring buffers are shared between kernel and user space.

1. Set ring buffers up with [`io_uring_setup()`](https://unixism.net/loti/ref-iouring/io_uring_setup.html#c.io_uring_setup) and then map them into user space with two [`mmap(2)`](http://man7.org/linux/man-pages/man2/mmap.2.html) calls
2. Create a submission queue entry (SQE) describing what operation you want to perform (read or write a file, accept client connections, etc.) and add it to SQ
3. Call [`io_uring_enter()`](https://unixism.net/loti/ref-iouring/io_uring_enter.html#c.io_uring_enter) syscall to signal SQEs are ready to be processed
    1. Multiple SQEs can be added before making the syscall
    2. [`io_uring_enter()`](https://unixism.net/loti/ref-iouring/io_uring_enter.html#c.io_uring_enter) can also wait for requests to be processed by the kernel before it returns, so you know you’re ready to read off the completion queue for results
4. Requests are processed by the kernel and completion queue events (CQEs) are added to the CQ
5. Read CQEs off the head of the completion queue ring buffer. There is one CQE corresponding to each SQE and it contains the status of that particular request

Ordering in the CQ may not correspond to the request order in the SQ. This may happen because all requests are performed in parallel, and their results will be added to the CQ as they become available. This is done for performance reasons. If a file is on an HDD and another on an SSD, we don’t want the HDD request to block the faster SSD request.

There is a [polling mode available](https://unixism.net/loti/tutorial/sq_poll.html#sq-poll), in which the kernel polls for new entries in the submission queue. This avoids the syscall overhead of calling [`io_uring_enter()`](https://unixism.net/loti/ref-iouring/io_uring_enter.html#c.io_uring_enter) every time you submit entries for processing.

Because of the shared ring buffers between the kernel and user space, io_uring can be a zero-copy system.

### How to use it?

Most sources indicate that the kernel interface was adopted in Linux kernel version 5.1. But from what I saw in the linux git, the `linux/io_ring` is only present in [linux 6.0](https://github.com/torvalds/linux/tree/v6.0) (does anyone know where it might be declared in previous versions?).

There is also a `liburing` library that provides an API to interact with the kernel interface easily from userspace.

I will eventually try to interact with `io_uring` using Go, so keep an eye on future articles if that interests you.

### References

[https://en.wikipedia.org/wiki/Io_uring](https://en.wikipedia.org/wiki/Io_uring)

[https://unixism.net/loti/index.html](https://unixism.net/loti/index.html)