# Q8S: Emulation of Heterogeneous Kubernetes Clusters Using QEMU
## 3 Results
### Implementation
The slowest operation by far was the setup of the emulated nodes, including running cloud-init and installing Kubernetes, which took about 23 min for the fastest node and 39 min for the slowest node. Moreover, all x86_64 nodes completed their deployment after 25 min, while this stage took the ARM64 nodes at least 35 min.Another longer stage was the initial setup of each of the hosts, i.e., the installation of QEMU and other dependencies, which took 24 min. Requirements

In this part we discuss how the presented prototype for QSS aligns with the functional requirements, which we defined in Section 2. As we were able to perform the benchmarking on a Kubernetes cluster deployed through QSS, we can consider FR5 to be completed. The benchmarking was performed on an OpenStack cluster for which we did not request any additional privileges, fulfilling \begin{table}
\begin{tabular}{l c c c c c} \hline \hline \multirow{2}{*}{**Node Type**} & \multirow{2}{*}{**Idle**} & \multicolumn{2}{c}{**Pod-to-Pod**} & \multicolumn{2}{c}{**Pod-to-Service**} \\  & & **TCP** & **UDP** & **TCP** & **UDP** \\ \hline OpenStack, client node & 350 & 387 & 379 & 388 & 365 \\ Emulated, client node & 429 & 442 & 438 & 441 & 433 \\ OpenStack, server node & 431 & 431 & 436 & 436 & 435 \\ Emulated, server node & 400 & 407 & 407 & 408 & 412 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of RAM usage in MB with nodes idling and during pod-to-pod and pod-to-service communication. Measured by k8s-bench-suite, 5 min per measurement \begin{table}
\begin{tabular}{l c c} \hline \hline
**Stage** & **Time in min** & **Time Since Start in min** \\ \hline OS instances ready & 2 & 2 \\ Distributed configuration files & 8 & 10 \\ Host setup completed & 24 & 34 \\ All workers joined & 39 & 73 \\ All workers ready & 3 & 76 \\ Flannel Network set up & 1 & 77 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Time taken to complete various stages of deployment for a cluster with 50 emulated worker nodes, including 30 emulated x86_64 nodes and 20 emulated ARM64 nodes, using QSS. FR1 and FR6. Specifying the node settings including system architecture, CPU speed, core count, memory and storage size also worked, which completes FR2. Q8S was also able to deploy the desired amount of node types and set up a working emulation for them, which matches FR3 and FR4. Table 6 provides an overview of the status of the requirements. However, while Q8S completes the goals we set out for it, we still see further features and improvements that should be made. These include support for limiting bandwidth and latency along with simulating network failure rates to emulate edge nodes. Moreover, Q8S works slow and a full deployment can take over an hour, highlighting the need to parallelize and optimize the process. Nevertheless, once the deployment is complete, the created cluster can be used for multiple rounds of experiments without having to rerun Q8S, making Q8S a valuable tool for emulating instead of simulating heterogeneous Kubernetes clusters. ## 4 Discussion

While Q8S fulfills its initial requirements, several limitations and considerations remain that should be mentioned. ### Limitations

The current implementation of Q8S only supports up to 252 nodes in an emulated cluster as the /24 subnet in OpenStack is used and some IPs are already reserved for the starting instance and the OpenStack router. This could be expanded to a /16 subnet, however, this would require changing the implementation on how the matching of IPs between the hosts and QEMU VMs works as having an identical last section of the IP address does not work with subnets larger than /24. It should also be noted that when running an emulation, the compute capabilities of the emulated node cannot exceed those of the host node. Moreover, as the hosts also consume some CPU time and memory in order to run their operating systems and QEMU, when setting the emulated nodes resources to close to those of the host, some of these are not guaranteed to be available to the emulated node.