# Vendor-Agnostic Reconfiguration of Kubernetes Clusters in Cloud Federations
## 7 Evaluation
### Experimental Results and Findings
#### 7.3.2 Experiment 2: Disruption Impact on Running Applications When Reconfiguring  
The goal of this experiment is to determine the disruption impact of the controller and reconfiguration tactics on running applications. Experimental Setup  
Likewise, we deploy our test applications and the traffic generator on different nodes to ensure that the traffic generator does not become a performance bottleneck. We use the traffic generator to generate consecutive loads of 100 requests per second. We only run each load for three seconds to evaluate the performance of the test applications at a certain point in time. The controller reconfigures features during these loads. This experiment is only performed on default GKE clusters but compared with proprietary customization interface of GKE. Our GKE clusters had two e2-highcpu-4 worker nodes. As stated in Section 5, the reconfigurators for the Calico network plugin and static CPUManager policy requires restarting of the Kubelet agent on all worker nodes. During a restart of the Kubelet, if a pod has readiness and liveness probes set, these probes will not be executed by the stopped Kubelet, so the pod might be observed by the control plane as  
Figure 9: Performance overhead of the three Reconfigurator modules of the autonomic controller in comparison with native configuration using proprietary customization interfaces of two K8s vendors (GKE and kubeadm). An explanation for each subfigure (**a–f**) is given above it. not ready anymore to serve traffic, and thus the Kube-proxy will not forward requests to it. To explore the actual impact of a Kubelet restart on such pods, we set a readiness probe for the test SaaS application that the Kubelet should execute every second. Experimental Results and Findings  
Figures 10 and 11 present the disruption impact on the SaaS application and the Cassandra database when the controller reconfigures each feature. We found that when the controller reconfigures each feature, for a short period of time, there is increased network and CPU usage of nodes. This can be explained by the container image pulling and container creation process for Daemonset pods and dependent plug-in components. This increase is especially noticeable when pulling and installing the Calico network plugin: the node's network usage goes from around 90 Kb/s during normal operation to 2.53 Mb/s (Figure 12) and the CPU usage goes from about 600 to 800 cpu milliseconds (Figure 13). For the static CPU manager policy, the controller only deploys Daemonset pods to reconfigure the Kubelet settings (Section 5.2). Adding a replica of the PodPreset API has minimal disruption. This is because the container image of the PodPresets wehook is not deployed to every node as in the case of the Calico network plugin and the CPUManagement policy. Figure 11: Disruption impact of the three Reconfigurator modules on SaaS application. Figure 10: Disruption impact of three Reconfigurator modules on Cassandra database. More importantly, we found a sharp increase in latency for those reconfigurators that require restarting the Kubelet. Although the Kubelet restarts quickly, the traffic generator seems not able to communicate with the test applications during a short period of time (Figure 14). As such, we believe that restarting the Kubelet for all nodes using a privileged Daemonset is a risky strategy that may result in short-term unavailability of cluster services. When installing the Calico CNI plugin using the proprietary customization interface of GKE, the disruption to the applications is much smaller. This is because GKE creates new worker nodes running in CNI mode and gradually migrates workloads from the existing nodes to the new ones. As a result, all Pods are restarted and therefore all use the new CNI plugin. In our solutions, existing Pods are not restarted, but they also do not use the new CNI plugin. Figure 14: Disruption Impact on SaaS application (During Kubelet restart, no traffic routed to the Application). Figure 12: Cassandra node network usage when reconfiguring.