# Computational and Communication Infrastructure Challenges for Resilient Cloud Services
## 5 Platform Resilience Mechanisms
### Traffic Management and Fault Tolerance
#### 5.1.1 Network Infrastructure Adaptation and Fault Tolerance
We configured Jmeter with 1000 threads (users) and a specific ramp-up period for each case. Each thread pulls an image from a 100-image repository and sends the request to the API gateway service, which is routed to the facemask service. We used JMeter to simulate users concurrently sending requests to the service. The purpose of each test case and the expected results are described below. * Case 1: The ability of the system to keep instances alive in the face of malfunctions. One of the characteristics of a fault-tolerant system is to launch new instances or restart existing ones in case of failure. In this experiment, we sought to validate the capacity of our platform to maintain deployed resources in the presence of development-level and architecture-level failures. For this case, we used a Jmeter ramp-up period of 5 min and set the number of replicas for the facemask deployment to four. As shown in Figure 3, Kubernetes Replicasets help to keep a specified number of pods alive. In the first test, we deleted the pods of the facemask deployment. In this case, Kubernetes took care of making the service available again and restarted the pods in approximately 4 s. We ran the test with and without using the retry pattern. We got 0.5% of requests with errors when not using the retry pattern and no errors when using it. In the second test, we ran the experiments in the presence or absence of failures and using or not the retry pattern. In the case of failures, we injected a 20 s service interruption and got 10% of requests with errors when not using the retry pattern. We should note that the retry pattern allows us to keep the error percentage at 0 percent. However, as expected, the maximum response time increases during the failure. Table 1 shows the request response times and the percentage of error in each of the evaluated cases. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Case** & **Average(s)** & **Min(s)** & **Max(s)** & **\%Error** \\ \hline No retry pattern and no induced failures & 1.6 & 1.1 & 2.1 & 0 \\ Retry pattern and no induced failures & 1.7 & 1.2 & 2.3 & 0 \\ No retry pattern and induced failures & 1.1 & 0.5 & 2 & 10 \\ Retry Pattern and induced failures & 10.2 & 1.5 & 17.4 & 0 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Performance of the facemask service with 1000 users and induced failures. Figure 3: Microservice ReplicaSet control. Case 2: The ability of the system to create new instances in the presence of many different requests. We assessed the ability to balance the resource load to ensure that the system responded successfully to demands from multiple users. In this case, we configured load balancing policies to allow the number of active instances to be increased or decreased by considering the number of user requests. To achieve this, we used the HPA Kubernetes services, which allowed the platform to respond to a high volume of requests and improved the response times and the number of simultaneous requests it could process. HPA will enable requests to be evenly distributed across replicas, defining the total instances and the metric to be considered for tuning on the platform. Figure 4 shows how the HPA works. For this case, we used a Jmeter ramp-up period of 2 min and set the number of replicas for the facemask deployment to two. We used the CPU utilization as the metric to scale the service to four instances. We set the maximum CPU utilization to 60% and verified that Kubernetes created the additional instances when the CPU utilization threshold is exceeded in less than 2 min while processing the 1000 requests. This traffic represents a bigger demand than expected during the rush time (1000 users in 5 min) and the platform quickly responds by autoscaling the pods. * Case 3: The ability of the system to route traffic in a controlled manner. Communication channels were established using Calico's network policies to define routes for the most important services, such as the API gateway and facemask detection, to improve the security of data delivery between microservices.