Let’s build a simple distributed computing system, for modern cloud | by Tharindu Bandara | Towards Data Science

The way of writing software is much different now. Being cloud native is one of the key goals for almost every software system architecture out there.
In this article, we will explore how can we build a simple and extensible distributed computing framework, tailored for the modern cloud. We are going to design, build and test a fresh framework from scratch while discussing the key design decisions.
This article provides a good guide to understand how distributed systems work, and implementing extensible systems targetting modern cloud infrastructures.
At a high level, there are three major components in this system.
Client — Actual users or user agents submitting jobsMaster — Accepts jobs from clients, orchestrate the system and get the jobs executed, and finally return expected outcomes to clientsWorker — Accepts a job, execute it, and provide the expected outcomes.
Client — Actual users or user agents submitting jobs
Master — Accepts jobs from clients, orchestrate the system and get the jobs executed, and finally return expected outcomes to clients
Worker — Accepts a job, execute it, and provide the expected outcomes.
Altogether clients submit jobs to the master, master schedules and get them executed with the available workers, and finally return job outcome back to the client.
We are developing this system for the modern cloud and in a community first approach. Therefore throughout the design, we will keep following three things in our mind.
Simple — Simple systems are easy to implement and easy to diagnoseModern — Systems run on the latest infrastructures utilizing the latest technologies are usually much faster, efficient, and obviously up-to date.Extensible — Extensible systems are vital for community as requirements differ from one use-case to other, thus allow developers to avoid implementing the common components, re-use and achieve use-cases with minimum implementation.
Simple — Simple systems are easy to implement and easy to diagnose
Modern — Systems run on the latest infrastructures utilizing the latest technologies are usually much faster, efficient, and obviously up-to date.
Extensible — Extensible systems are vital for community as requirements differ from one use-case to other, thus allow developers to avoid implementing the common components, re-use and achieve use-cases with minimum implementation.
This is the system architecture of the distributed computing framework.
The above image is pretty self-explanatory. To explain some of the key elements of it,
Worker microservice
A worker has a self-isolated workspace which allows it to be containarized and act independantly. Also, the system expects that all workers are homogenious in terms of the resource capacities. i.e. Fixed CPU power, RAM, Storage, etc. In other words, workers do not scale.
This makes workers simple thus easier to manage. Workers are actually gRPC servers in-terms of the default implementation. But it is also extensible thus can be implemented with any framework, or even from scratch.
The default gRPC implementation ensure that master-worker communication happens via RPC(remote procedure calls). This takes away all the complexity when compared to the other popular method, RESTful APIs. Besides, gRPC is also scalable, code-friendly, and fast(about 7 times faster that the RESTful APIs).
Master microservice
Master talks with clients and negtiates with workers. It acts as a gRPC client to the workers, and a server serving RESTful APIs to the clients. RESTful APIs are built on top of Spring Boot framework + Open API 3 code generator in an API first development approach.
RESTful APIs are much human friendly thus suits better for Master and Client communications. Also, these API calls do not require to be blazing fast as they are not frequent.
Master is also expected to be operating on its own self-isolated environment thus making it ideal for containerisation.
The master consists of several loosely coupled components. Each of these components have their own responsibilities and are independant in-terms of the work they do. This makes the Master itself much simpler to understand. Also, each of the components are extensible which makes the behaviour of the master can be modified and extended at a more fine-grained level.
Task Manager — Orchestrate data flow among the component. It includes a periodically running task which checks and copies completed jobs from worker services, and dispatch scheduled jobs afterwards.Task Scheduler — Persist jobs in a priority queue until they get dispatched to workers.Task Prioratizer — Assign a priority value based on diffrent attributes of the job such as deadline, etc. The default implementation implements the “Earliest deadline first” approach.Task Distributor — Communicates with the worker pool and dispatch a task to a free worker. If there are no free workers available, then it puts the task back to scheduling.Worker Pool Manager — Manages the communication with worker services.Central Store — Store job data until they get completed, and to be obtained by clients. The default implementation uses local storage based store, in which tasks are stored in folders created in a given isolated workspace.
Task Manager — Orchestrate data flow among the component. It includes a periodically running task which checks and copies completed jobs from worker services, and dispatch scheduled jobs afterwards.
Task Scheduler — Persist jobs in a priority queue until they get dispatched to workers.
Task Prioratizer — Assign a priority value based on diffrent attributes of the job such as deadline, etc. The default implementation implements the “Earliest deadline first” approach.
Task Distributor — Communicates with the worker pool and dispatch a task to a free worker. If there are no free workers available, then it puts the task back to scheduling.
Worker Pool Manager — Manages the communication with worker services.
Central Store — Store job data until they get completed, and to be obtained by clients. The default implementation uses local storage based store, in which tasks are stored in folders created in a given isolated workspace.
As mentioned earlier, anyone can plug-in their own implementation of any of the above components, thus making the system extensible by design.
Before we begin, the complete implementation of the system that we discussed so far can be found in the link below. It is an open source project distributed under the Apache 2.0 licence.
github.com
The system we just designed, is named as ORION. This project has four main components.
Master — Contains the Spring boot + Open API 3 + gRPC client code based implementation of the Master microservice. This is a pure JAVA module.
Worker — Cotains the gRPC server based implementation of the Worker microservice. This also is a pure JAVA module.
Monitor App — Contains the React + Material UI based implementation of the ORION monitoring SPA(single page application). This is a pure JavaScript module.
Integration — Contains test scripts, and test resources to configure the test environment. The scripts are Postman test collections and test resources includes the data requires to execute the collection.
Configuring and running the test suit
We will be using the v0.1 release of the ORION to conduct our test scenarios.
Download the v0.1 release artifacts and extract both zipped files.
Create an empty folder for the testing workspace. Let’s name it as theworkspace .
Open the following file,
<downloaded-artifacts>/integration/environment/init-setup.sh
Edit the following variables with the correct file paths.
WORKER_JAR_PATH=<path to the orion-v0.1-worker.jar file>MASTER_JAR_PATH=<path to the orion-v0.1-master.jar file>WORKSPACE=<path to the workspace directory>
Save the changes.
Open the following file.
<downloaded-artifacts>/integration/environment/stop-setup.sh
Edit the following variable with the correct file path.
WORKSPACE=<path to the workspace directory>
Save the changes.
Now we are ready to start the ORION system. Open a terminal window and execute the following to run the init-setup.sh script.
cd <downloaded-artifacts>/integration/environmentsh init-setup.sh
This copies relavant files to the workspace directory and starts three microservices for the master and two workers.
If you observe the workspace directory, following folders are created.
active-processes — PIDs of the three services.
logs — Log files from the three services.
master — Artifacts related to the master service.
worker-1 — Artifacts related to the first worker service.
worker-2 — Artifacts related to the second worker service.
Execute the following command to observe master logs.
tail -f <workspace>/logs/master.log
Following tailing log should be there if the service started successfully.
...2021-06-25 02:14:22.022  INFO 10125 --- [           main] o.c.orion.master.RESTfulEndpoint         : Started RESTfulEndpoint in 1.194 seconds (JVM running for 1.658)
Likewise, logs of the worker services can be tailed in the same manner using the correct file name in the same folder.
Now start Postman and import the following test script collection.
<downloaded-artifacts>/integration/test-scripts/ORION Test Suit - Executing heavy tasks.postman_collection
There are four task submission API calls in this collection. You need to set the executable shell script and input text file for each of them(these files are submitted as multi-part form data). Relevant artifacts can be found in the corresponding folder.
<downloaded-artifacts>/integration/test-scripts/task-files/task-<number>
Tasks we are using in this test are nothing more than a shell script and an input file. Following is one of those scripts.
...sleep 50cat in.txt > out.txt 2>&1
What it does it wait for some time and then create a new file named out.txt and copy everything from in.txt to the out.txt. This is to mimic a task that takes some time to process, and involves input and output files.
We are now almost ready to execute the test suit. But before that, lets start the monitoring app.
Execute the following in a terminal(you need to have npm installed at this point).
cd <downloaded-artifacts>/sdnpm inpm start
This should start the monitoring app.
Once tasks are submitted, this will show how they first inserted into the queue, sent to workers for execution, and finally stored in the central store upon completion.
Let’s start the test. In the postman collection, submit all four tasks. You should recieve the following response saying each task is executing in the ORION.
Now open the monitoring app window and observe the life cycle of the submitted jobs!
For each task, progress and it’s unique id is shown.
Finally, stop the ORION system by executing the following.
cd <downloaded-artifacts>/integration/environmentsh stop-setup.sh
In this article, we gradually developed a distributed computing system tailored for the modern cloud. We started by designing the architecture focusing on being simple, modern and extensible. Then we moved to test the implemented system and successfully ran a test suite and observed how submitted tasks are handled.
Thank you for staying until the end of this article. Your ideas, suggestions, and questions regarding this work, are highly appreciated.