# Computational and Communication Infrastructure Challenges for Resilient Cloud Services
## 5 Platform Resilience Mechanisms
### Traffic Management and Fault Tolerance
#### 5.1.1 Network Infrastructure Adaptation and Fault Tolerance
One node was configured as the master node and the other as a worker. Each node has 64 GB of RAM, an Intel Xeon CPU E5-2650 v4 with 24 cores, 2 TB of HDD, and Ubuntu 18.04 as the operating system. For our deployment, we used Kubernetes 1.17, Docker 1.19.3, and Python 3.7. We defined a number of requirements for each implemented Python script that the Docker container needs. Listing 2 shows a list of requirements for each microservice, and Listing 3 shows the basic configuration in a Dockerfile to create the container for the facemask service. The Dockerfile for the Apigateway is similar, only changing the exposed port to 5000. ```
opencv-contrib-python=4.5.4.58 numpy=1.21.5 requests=2.18.4 click=8.0.4 Flask=2.0.3 itsdangerous=2.1.2 Jinja2=3.1.1 MarkupSafe=2.1.1 Werkzeug=2.0.3
```
Listing 2: Requirements for Python applications. ```
FROMpython:3.7 LABELmaintainer=\"HeberthMartinez\" RUNapt-getupdate RUNaptinstalllibgl1-mesa-glx-y RUNmdiff/app WORKDIR/app COPY./app RUNpipinstall-rrequirements.txt EXPOSE4000
```
Listing 3: Dockerfile for the facemask service. ### Deployment

Our microservices-based architecture uses Kubernetes for container orchestration. Kubernetes allows the separation of developed applications into small groups of containers and provides tools to enable management and making decisions during application failures. Figure 2: Test scenario. A resource group is defined during service deployment in a Kubernetes cluster, allowing deployed containers to be monitored. As shown in Figure 2, we deployed our application using two Kubernetes services. One service runs an ApiGateway that receives client requests and routes them to a service running the facemask detection algorithm. Each service routes the traffic to a set of pods, which are \"logical\" host grouping Docker containers. Pods can be replicated inside a single multicore server or across multiple servers. We implemented a Retry pattern in the ApiGateway service configured to retry a request to the facemask service a maximum of 10 times with a period of 10 s between retries in case of request timeout the internal logic described in Listing 4. Our platform leverages the implemented resilience mechanisms to scale the number of pods according to the demand. We use CPU utilization as the metric to decide the number of active pod replicas. For our application, an acceptable maximum response time for a request is 20 s.
```
start whileappisruning initFlaskapp defineroute\"facemask\" receivedatafromtheclient sendrequesttofacemaskserviceonport4000 responsedatatoclientwithstatuscode200 ifnotexist sendmessageerror\"error\"andstatuscode500 defineroute\"facemask-retry implementretrypattern receivedatafromtheclient sendrequesttofacemaskserviceonport4000 checkresponseandretryrequest10 timeseach10 secondsifnotisvalid responsedatatoclientwithstatuscode200 ifnotexist sendmessageerror\"error\"andstatuscode500 exposeport500 end
```
**Listing 4**. : Apigateway pseudocode. For the facemask detection service, we implemented a Python Flask application that receives the request from the client with a JSON body that contains the image URL to process. The service downloads the image and processes it with the help of a neural network, and returns a JSON with objects detected information. Listing 5 shows the internal logic of the service. ```
start whileappisruning initFlaskapp defineroute\"detect\" receivedatafromtheclient downloadimagefromurlsendbyclient loadimagewiththehelpofopencvmodule loadnewlandnetworkwiththehelpofopencvmodule loadconfigurationfileandlabelsfilewiththehelpofopencvmodule processimagewithneuralnetwork createJSONresponsewithallobjectdetected,confidencevalueandlabeldetected sendJSONresponsetoclientwithstatuscode200 ifnotexist sendmessageerror\"error\"andstatuscode500 exposeport4000 end
```

**Listing 5**. : Facemask pseudocode. ### Test Cases

We designed three test cases to validate the fault tolerance capability of our platform. We used JMeter tool [63] to simulate users concurrently sending requests to the service.