The Backend Revolution or Why io_uring Is So Important.
Tags: backend, architecture, tutorial, beginners
Published: 2020-06-03 04:37:08

---
title: The Backend Revolution or Why io_uring Is So Important.
published: true
description: New era in backend development
tags: #backend #architecture #tutorial #beginners
---
Not so many backend (and application in general) developers watching development of Linux kernel. But those who do, definitely noticed appearance and rapid development of __io_uring__. For those who is not aware what it is I'll provide brief introduction below. But main topic of this post not the io_uring itself. Instead I want to show you that we are watching the beginning of the real revolution in backend development.

## I/O Anatomy of The Backend
As I've explained in the [article dedicated to Data Dependency Graph](https://dev.to/siy/data-dependency-analyses-in-backend-applications-27pp), each backend application at very high level has only three parts - set of inputs, set of outputs and set of transformations from inputs to outputs:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/u6l6gmpmdqcqppylgnfj.jpg)

This is rather logical view, since it presents logical meaning of inputs and outputs from the point of view of the architecture of the application. In other words, it preserves inputs and outputs as we see them while designing and implementing the application. Nevertheless, if we look at the same application from the low level I/O perspective, we quickly realize that _every_ input, output or internal communication (calls to other services located outside our application) in fact consists of _both_, input and output. How this happens? Lets take a look at, for example, REST entry point: it receives request from client (_input_ part), processes it and sends response (_output_ part). Same is true for every other point where we interacting with world outside the application. 
Lets try to draw a diagram with I/O which happens during processing of request of very simple REST entry point. This entry point prepares arguments from incoming request, calls external service, formats response and sends it back. All I/O inputs (i.e. where data received) I'll keep pointing to left and all I/O outputs (i.e. where data sent) will point to right:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/kxxsox92vsb29d6efxbh.png)

Vertical lines depict interaction between application and external world, i.e. I/O operations. 
Now lets ignore for the moment that all steps in this diagram are logically belong to same request and focus only on the I/O:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/2066ucgqdmf4a1wqp4no.png)

Some important observations: 
- From the point of view of I/O application has much more inputs and outputs that we usually taking into account at logical level.
- The whole application is an __asynchronous system driven by I/O events__. 
- At such a low level _structure_ of the application remains exactly the same - set of inputs, set of outputs and set of transformations between them. But this time we have completely different sets of inputs, outputs and transformations.

While in this very simple example we have only one input for every transformation, in more complex cases outputs may depend on more than one input. Also, usually there is some kind of context which is passed across all transformations related to same logical processing pipeline. For example, in the REST application there are request and response contexts (often combined into one) which form the context passed through all processing steps of same entry point.

I want to emphasize: __all__ backend applications have identical structure shown above. The internal architecture and software stack used for implementation are irrelevant. Old-fashioned Java monolith and fancy serverless functions have same structure from point of view of I/O.

### Few Words About Transformations
At first look transformations at the diagrams above should be functions. After all they take some parameters (input data + context) and produce some output (which is then sent). But such a function is only part of the transformation. There is an very important stage - synchronization. It often overlooked (especially if we're dealing with synchronous code), but it's very important since it directly affects on application performance characteristics (see below). The purpose of the synchronization stage is to collect all necessary parameters and then pass them to transformation function. So, whole transformation contains two parts - synchronization stage followed by transformation function:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/tn1gxim6kzki2zeqi36j.png)

The synchronization part be one of two types - __All__ and __Any__. The [article dedicated to Data Dependency Graph](https://dev.to/siy/data-dependency-analyses-in-backend-applications-27pp) explains these in more details.

## I/O and Application Performance
The look at the application as a set of Input-Transformation-Output elements is crucial step for understanding application performance bottlenecks. There are only three factors which affect application performance:
- Input throughput
- Output throughput
- Transformation throughput

Lets start from the last item: transformation throughput. This part depends on several factors - overhead introduced by underlying software stack, for example. It also depends on the complexity of performed transformations. Some applications are computationally intensive (for example, blockchain and AI apps). Nevertheless, most backend applications have no significant computational load and transformation throughput is limited by other factors:
- Overhead introduced by software stack
- Overhead introduced by synchronous processing

Last item appears in traditional synchronous applications when they create and maintain large number of threads, vast majority of which just waits for the I/O.

OK, how about Input and Output? Well, they have their own (big) set of issues. First of all, wast majority of traditional API's provided by operating systems are synchronous. There are some exceptions but they cover only subset of necessary functionality (usually network). Next issue is that _every_ I/O operation means context switch. We prepare parameters and perform call to operating system (OS). At this point OS needs to save whole context, switch to internal OS stack, copy (if necessary) data between user space and kernel space, perform requested operation and upon completion perform same steps in reverse order. Beside that every context switch is a huge stress to CPU pipelines which heavily impacts performance.

Lets summarize: how application looks and behaves from the point of view of I/O is a completely different from how we ought to implement our applications. 

## New Era of I/O With io_uring
The [__io_uring__](https://unixism.net/loti/what_is_io_uring.html) is a completely new approach to I/O API. 

First of all, the API is consistent across all types of I/O. Network and file I/O handled consistently. 

Next, the API is asynchronous. Application submits requests and then is notified when request is completed. This API naturally implements so called [__Proactor__ pattern](https://en.wikipedia.org/wiki/Proactor_pattern). While far less famous than [__Reactor__ pattern](https://en.wikipedia.org/wiki/Reactor_pattern), __Proactor__ is better fit for pure asynchronous API.

The __io_uring__ API uses two ring buffers for communication between application and kernel (hence the API name) and designed in way which enables _natural batching_ of requests and responses. Beside other consequences (see below), this means that number of context switches is significantly reduced. There is no correspondence _One I/O = One syscall = Two context switches_ anymore.

Finally __io_uring__ can be configured (although this requires root privileges) to perform I/O without calling OS at all (to be precise - don't call OS as long as there is stable stream of I/O operations).

There are a number of other useful features, for example it's possible to pre-configure memory buffers for I/O so they will be directly accessible for application and for kernel, so it is possible to perform I/O without copying any data (nor even remapping memory).

## Ideal Backend Application
Well, there are no ideal applications. Nevertheless, I propose to call _Ideal Backend Application_ any backend application which is able to saturate Input at given hardware (i.e. reach hardware+OS limits). This is not so easy to achieve. In fact before __io_uring__ was introduced, it was almost impossible for traditional user-level application to reach I/O limits. 
Now, with __io_uring__ this is possible, but why this is important and why such an application is ideal? Main property of such application: it's impossible to overload it!
The whole range of loads, from zero to saturated Input is a _normal operating range_ for such application. Just imagine: no performance degradation, no excessive memory (or other resources) consumption and no relevant failures. Such an application is always  stable and reliable, its _ideal_ from maintenance and support point of views.

## Are we ready for this revolution?
Well, we definitely have close to zero support for __io_uring__ in popular stacks used for backend. Unfortunately many of them are not designed to accommodate such I/O model easily. Worse is that many frameworks support or promote synchronous processing model. Adding __io_uring__ support in such frameworks will provide no sensible benefits.
Good news is that virtually any stack which uses Promise-based asynchronous processing model will get significant performance boost since this model is very close to __io_uring__ API and how application looks from the point of view of I/O.

## Conclusion
We're facing revolutionary change in the backend development. And to get most benefits from this change we need to adopt asynchronous processing and at least some functional programming approaches. This combination provides minimal "abstraction overhead" and allows to get most out of hardware. Some software stacks are already adopted this model (for example, Node and Deno) and show quite good performance despite far from the best possible performance of JS and single threaded design.