Writing Asynchronous Non-Blocking Rest API using JAVA.
By: Md Sajedul Karim
Tags: Spring, Java, API, Servers, Asynchronous Programming

Writing Asynchronous Non-Blocking Rest API using JAVA.

Image Credit — www.pexels.com

During an API call from a client, the server creates and assigns a thread of that request. This thread waits and is appointed until requests are served to that client.

Each server can open a limited number of threads, and if that number exceeds, then new clients request will store in the request queue of that server.

For the Tomcat server, its default number of threads maxThreads is 200, representing the maximum number of concurrent threads allowed to run at any given time.

There are other metrics, maxConnections , representing the total number of concurrent connections that the server will accept and process. Any additional incoming connections will be placed in a queue until a thread becomes available. The default value for NIO/NIO2 mode is 10,000 and APR/Native is 8,192.

Another metrics is acceptCount , which represents the maximum number of TCP requests that can wait in a queue at the OS level when there are no worker threads available. The default value is 100.

For a high traffic application, If a request takes much time or takes a long time to serve, it will create a huge problem. To overcome this thread management problem, we can use asynchronous programming.

According to visualstudiomagazine.com, the definition of asynchronous programming is

Asynchronous programming is a means of parallel programming in which a unit of work runs separately from the main application thread and notifies the calling thread of its completion, failure or progress.

How synchronous applications create problems?

For synchronous applications, what problem create during the request process? Let's figure out the problem. Here is the request flow:

The request comes to the tomcat server with a listening port. Tomcat server has its internal server configuration like maxThreads , maxConnections Etc. If the server currently exceeds the max thread value, then the request waits into the queue.

, Etc. If the server currently exceeds the max thread value, then the request waits into the queue. Now server passes that request into the application, which means the server assigns a new thread into that request and passes this request into the application. This thread waits until the request is served.

If API takes a long time, then that thread waits for its response. Here, tomcat has bt default 200 thread so that it could serve a max of 200 requests at a time. In this case, clients will face heavy delays during API calls.

Today we will discuss using CompletableFuture how can we write the asynchronous REST API to make the tomcat free to overcome the thread overflow problem?

To know details about the terms CompletableFuture , you may check this article

CompletableFuture for Asynchronous Programming

To overcome this issue, you may check the below code using. CompletableFuture .

Here what are we doing here? We are doing the below things:

It is simple endpoints that is a simple GET method and its return type is CompletableFuture<String>

method and its return type is During processing the request, it was just calling the method processRequest where only have a Thread.sleep for 10s.

where only have a for 10s. Both methods have an entry and exit log.

After calling the endpoint from browser/postman, it sends a response after 10 seconds. The log of this API call is given below.

Image Credit — Author

Here,

The first and second line of log is showing under nio-8080-exec-8 thread. It was also showing requests received and servlet thread released. This thread is from non-blocking IO, and it is from the embedded server.

thread. It was also showing requests received and servlet thread released. This thread is from non-blocking IO, and it is from the embedded server. From the first 2 lines of the log, it realized that it is releasing before execution of processRequest method part that is under CompletableFuture part.

method part that is under part. On last 2 lines of the log is for CompletableFuture part. It is handling under onPool-worker-2 the thread. That mean’s it is separate from the tomcat thread, and this thread is from Spring. ForkJoinPool .

Summery of thought

Tomcat server passing requests into applications after getting calls from clients.

Application assigning this task into another thread by CompletableFuture . This thread is responsible for executing this assigned task and respond to clients by tomcat connections.

. This thread is responsible for executing this assigned task and respond to clients by tomcat connections. The application immediately releasing the tomcat assigned thread. This released thread will back to the tomcat thread pool, and it will ready to serve the new API request.

With this mechanism, tomcat now can handle large traffic.

We can accept more requests by modifying the tomcat server.xml file.

We can be finetuning server.xml file so that it can handle a large number of incoming traffic. This fine-tuning be done based on our server’s resources like processor capability, the number of core, RAM, etc.

Code credit — Author

Final Thoughts

To reduce server thread waiting time, you may apply the asynchronous API. Also, you may try Deferred Result or Spring WebFlux.

For writing a secure REST API, you may check this article.