# Towards a Highly Available Model for Processing Service Requests Based on Distributed Hash Tables
## 2 Materials and Methods
### System Scalability Based on the Intrinsic Properties of the Chord DHT
Note that, in order to obtain the best performance, the current novel solution for high availability was tested by running it on nodes that were fully loaded by the service. Each load balancing node has an equal probability to be contacted. We have gradually deployed a Chord network, having a maximum of 128 peers, and on top of each peer there is also the application service that is running. The network that interconnects the peers, and the network that connects each of the peers with the load balancers representing the front end of the service, is a high speed network. The observed ping delay when contacting one node from another node in the datacenter does not exceed 2 ms, and is usually around 1 ms if the nodes are not in the same rack, and around 0.3 ms if the nodes are in the same rack. The decision to stop at 128 nodes is based on the observation that, in practice, this number of nodes is eloquent for showing system scalability, since, to the author's knowledge, most of the common highly available applications are deployed on a few nodes, certainly no more than 100. From the performed experiments, notice that the extra delay for fully processing the client's requests by the service, introduced by deploying it on a DHT of about 128 peers in our datacenter, has a mean value no larger than 15 ms, which makes it negligible and thus validated the fact that this improved novel solution for high availability is transparent to the service's clients. ### Practical Evaluating of the System's Scalability

Indeed, from the evaluation experiments performed in our datacenter, which involved a maximum of 128 nodes, the results show that the number of messages exchanged by the peers in order to find the destination of a message routed via Chord, i.e., to find the Chord node that is responsible for handling and processing a particular client's request, was no larger than 7 hops. It is obvious that \(7=log_{2}(128)\), meaning that our experimental findings are in accordance with [9]. If extrapolating the measurements performed for 128 peers to about 1024 peers, we obtain no more than 10 hops, or inter-peer messages, to reach a Chord ID destination, which means that for a number of peers that is 10\(x\) greater than the number of peers that we have launched, the number of exchanged Chord messages remains almost the same, since the integer part of the logarithm function grows very slowly. Moreover, if identifying a very loaded peer by the requests of the clients it must serve, there exists the means to decide deploying a new peer, in order to re-balance the load. Thus, the extended network, integrating this new peer, will have almost the same system inter-peer routing communication delay, since the maximum number of routing hops for messages among peers is represented by the graphic representation of the function \([log_{2}(K)]\), where \([X]\) takes the integer part of the real value \(X\). The performed experiment to show the novel solution's improved _scalability_ was the following: the load balancing node was deployed and then, gradually, 1, 2, \(\ldots\), 127 and 128 improved Chord peers were deployed to help handle the already loaded service that needs to process its clients' requests, as described in Section 2. This was actually the way to gradually instantiate an infrastructure that is able to provide improved high availability via scalability for the request processing service. Each time, we waited for the infrastructure to converge to a fully connected DHT. On this logical and physical infrastructure, messages were sent via the DHT communication layer towards various nodes in order to see an increasing number of DHT routing hops and the time needed to traverse those hops. As can be seen from Figure 5, the number of messages was no larger than 7, i.e., as expected, the base 2 logarithm of the maximum number of deployed peers: \(log_{2}(No_{pers})\).