Concept of Load Balancing
By: Manikant Pathak
Tags: Distributed Systems, Scalability

Hi all, this is my first topic on medium.com and am planning to publish more as and when I learn something. My idea is to categorise each topic into three major questions: What, Why and How which will followed by few helpful links. This approach has come-in handy to me when I seek crisp and specific answers to questions and, hope the same helps you all too. Please do leave your feedback (positive/negative) as both will help me.

What is load balancing?

In very simple language, load balancing in tech is nothing but balancing the requests / network traffic across multiple application servers based on a pre-defined algorithm.We will talk more on the algorithm for load balancing in the how section of this topic.

Why do we need load balancing?

Consider a scenario where you build an application and deploy it on multiple servers but no load balancer (hardware or software) is defined. Now when a new request comes, it can be routed to any of the server and there are chances that the subsequent requests goes to the same server or the server itself may be down. In this case user will not have the best experience as the response time will be more or there may not be any response at all. A load balancer will be helpful here, as it will ensure that the requests are getting directed only to servers which are healthy and then among the healthy servers, the re-direction will happen based on the algorithm configured.

Another scenario where a load balancer will be helpful is re-directing the requests based on the client IP, to to the nearest server to reduce network latency and provide good user-experience. Example: If servers are located in Asia, Europe and North America and if a request from North America is served from Asia, response time is more compared to request from Asia going to server in Asia. This is where re-direction based on client location will ensure a better user experience.

With a load balancer in place and application deployed across multiple servers one of the biggest advantage is that there is no single point of failure for the application, apart from the case where the load balancer itself is down. This risk can also be mitigated and more on this can be found in next section.

How do we achieve efficient load balancing?

Load balancers can be either Hardware Load Balancer or Software Load Balancer. Hardware load balancers are expensive and very few organization actually use it. So we will focus on Software load balancers in this topic.

How does a load balancer know which servers are healthy?

This is done by doing a health check-up for all members within a load balancer group regularly. A health check essentially sends a request to each member of the load balancer group to establish the availability of each member server to accept client requests and then analysing their response. More details on health checks can be found here.

Which server to select from the list of healthy servers?

This is decided based on the algorithm defined and some of the commonly used algorithms are:

Least Connection Method: This will direct the incoming requests to server which is having least active connections. This approach is most helpful when we have persistent connection which are unevenly distributed to the servers.

This will direct the incoming requests to server which is having least active connections. This approach is most helpful when we have persistent connection which are unevenly distributed to the servers. Least Response Time Method: This will direct the incoming requests to the server which is having fewest active connection and least average response time.

This will direct the incoming requests to the server which is having fewest active connection and least average response time. Round Robin Method — This method chooses the first available server for the incoming requests and after sending the request to server pushes it down the queue, so that turn for this server comes only when the request is sent to all other servers. This is most commonly used technique and is specially helpful when all the servers are of same specification with not many persistent connections.

— This method chooses the first available server for the incoming requests and after sending the request to server pushes it down the queue, so that turn for this server comes only when the request is sent to all other servers. This is most commonly used technique and is specially helpful when all the servers are of same specification with not many persistent connections. Weighted Round Robin Method — This method is an extension to the round robin method where a wight(an integer value that indicates the processing capacity) is assigned to each server based on their processing capacities. With this the servers with higher weights receive new connections before those with less weights and servers with higher weights get more connections than those with less weights.

— This method is an extension to the round robin method where a wight(an integer value that indicates the processing capacity) is assigned to each server based on their processing capacities. With this the servers with higher weights receive new connections before those with less weights and servers with higher weights get more connections than those with less weights. IP Hash — Under this method, a hash of the IP address of the client is calculated to redirect the request to a server.

How to handle cases where load balancer itself is down and can become a single point of failure?

To overcome this, a second load balancer can be connected to the first to form a cluster. Each Load Balancer monitors the health of the other and, since both of them are equally capable of serving traffic and failure detection, in the event the main load balancer fails, the second load balancer takes over and start serving the requests.