Distributed Systems with Rails
Tags: ruby, rails
Published: 2022-03-10 20:57:15

---
title: Distributed Systems with Rails
published: true
description: How to build distributed systems with Ruby on Rails
tags: ruby, rails
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2dnb0k3ygf541k5t8noy.jpg
---

When most people think of __distributed systems__, domains like [microservices](https://en.wikipedia.org/wiki/Microservices) and [blockchains](https://en.wikipedia.org/wiki/Blockchain) or some variant of orchestrated complexity come to mind. 

![Distributed system complexity](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xordwelxtqsit65folk5.png) 

But let's take a moment to consider [Wikipedia's definition](https://en.wikipedia.org/wiki/Distributed_computing).

> A distributed system is a system whose components are located on different networked computers, which communicate and coordinate their actions by passing messages to one another from any system. The components interact with one another in order to achieve a common goal.
> 
> Three significant characteristics of distributed systems are: 
>
>       1. concurrency of components
>       2. lack of a global clock
>       3. independent failure of components

If we stretch this definition a bit, we'll discover that Ruby on Rails supports creating distributed systems. In fact, most full-stack web frameworks do. 

Consider that you're already running multiple services, often on separate hardware, things like databases and caches. Now examine purpose-driven deployments. A server dedicated to standard web requests and another for API requests. Or think of the third-party services that web applications rely on for things like logging, email, search, analytics, image processing, etc... Such services are endless. Just look at what's offered in [Heroku's marketplace](https://elements.heroku.com/addons).

![Typical full stack deployment](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cxm0tyd3296fqak5uh7z.png) 
 
While it may not be self-evident, thinking of Rails as a distributed system platform will help frame the mental model for the ensuing discussion. 

---

I want to focus on a particular feature in Rails that helps accomplish some distributed system characteristics, [ActiveJob](https://edgeguides.rubyonrails.org/active_job_basics.html). ActiveJob provides background job processing for Rails. 

The principal component in the Rails toolbelt for managing background jobs is the backplane or [data-bus](https://en.wikipedia.org/wiki/Software_bus) which allows us to store metadata about expensive operations that should move off the [critical path](https://seebrock3r.medium.com/a-hot-path-is-a-code-path-that-is-perf-critical-either-because-its-something-that-is-really-85ee62fe68f6) and be processed in the background. These days, [Redis](https://redis.io/) is generally used as this backplane for serious Rails applications. It's a multipurpose in-memory data store that serves several functions: a cache, message broker, and database. 

ActiveJob uses Redis as a [message broker](https://en.wikipedia.org/wiki/Message_broker) by saving or queueing metadata about operations to run in the background. It also dequeues that metadata and runs those operations. [Sidekiq](https://sidekiq.org/) is another popular library that serves the same purpose. Sidekiq implements the ActiveJob interface and includes several advanced features not available in ActiveJob.

It's common for Rails applications to run ActiveJob background workers on separate hardware, but sophisticated deployments might even run isolated worker pools on different machines for different types of background work. _It's also viable to run multiple instances of Redis to further separate different types of background work._

![Rails based distributed system deployment](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m1tjmcuzwxn6185cou4d.png) 

The possibilities are endless, and Rails supports all of them. I hope it's becoming clear that purpose-driven deployments can comprise a distributed system even when all the code is contained inside a monolith. For our purposes, it might help to think of distributed systems in terms of __units of deployment__ instead of isolated __units of execution__.

---

Now, let's consider some scenarios where segregating different types of background work makes sense.

- Limiting resource contention on a centralized database
- Constraining requests to third party API endpoints
- Partitioning customers to ensure a single user can't exhaust system resources and cause problems for everyone
- Isolating ETL processes to prevent disrupting production
 
All of these scenarios can be managed by a single monolithic Rails codebase. __Why might this be a good idea?__ Team productivity is exceptionally high in a well-designed Rails monolith. Even if the code is more tightly coupled, it's a tradeoff worth considering as you'll get several benefits of 
__"distributed systems"__ without all the downsides.

---

Stay tuned for the [next post](https://dev.to/hopsoft/sophisticated-simple-and-affordable-background-workers-14l4) covering how to structure a queueing system to manage all that background work.