Techniques to manage software growth
Published: 2020-04-26 09:02:10

Almost all software services will undergo some kind of growth spurt during their lives. For example, if you are hosting a small travel website and expect a high number of requests during certain seasons or months - we might know that we'll need to increase our system's capability. This means we give additional processing power (more memory or CPU) so that we can handle increased traffic.

Scaling up - The idea of adding on capabilities to a pre-existing system is known as scaling up. When we "scale up", we're improving upon the system that already exists and making them more capable of handling more traffic. However, there are physical limits to vertical scaling. E.g. you can't keep adding more memory or CPU to an existing system. This is where another form of scaling comes into play: scaling out.

Scaling out - Instead of choosing to increase the capacity by improving on the system we already have, we can choose to add more systems to help expand the capabilities.

Scaling out techniques - 

1) Partitioning - This technique is to take a portion of software service that is handled/located in one centralized place, and split it up and out throughout the system. When we partition and distribute a single software service, it means that the entire service no longer lives in one place, reducing the likelihood of the entire service failing. A common element considered for partitioning is data records stored in databases. Database partitioning is a design principle whereby rows of a database table are held separately, rather than being split into the column. Each partition forms part of a shard, which may, in turn, be located on a separate database server or physical location. The advantage is it improves search performance since the tables are divided and distributed into multiple servers, the total number of rows in each table in each database is reduced.

2) Caching - This technique involves creating replicas or copies of a resource to live close to the client that is accessing them. When a resource is cached, it means the original resource is copied and then put in a place that is easy-to-access and is physically close to the requesting client. This significantly improves performance. However, the challenge of caching is to ensure those copies will not get out of sync which could lead to consistency issues. Some popular caching services include Memcached, Redis.

3) Asynchronous communication - When we add more systems to scale out, we are inadvertently introducing network latency issues. The technique to solve the network latency issue is to use asynchronous communication. The idea behind asynchronous communication is to do something useful instead of waiting for another service. This way we can elegantly handle time between request and response, mask delay at our end. Some popular asynchronous communication frameworks include Akka, RabbitMQ

Closing thoughts 

"Valleys of Death" - those points in the software's growth where you're bigger, but not quite big enough to have the next level to scale. Scaling techniques give us a way to think, get bigger to scale software to the next level.