# Q8S: Emulation of Heterogeneous Kubernetes Clusters Using QEMU
## 3 Results
### Implementation
The emulation introduces performance overhead resulting in higher resource consumption compared to a native performance as was shown in Table 3. If Q8S is to be used for gathering training data for scheduling algorithms or CPU metrics are used for evaluating a given algorithm, this overhead has to be taken into account. Another limitation is that due to the usage of QEMU and libvirt, only hardware that is supported by these applications can be emulated via Q8S. The supported hardware is further restricted due to the reliance on Ubuntu Cloud-Images, which typically builds \begin{table}
\begin{tabular}{l l l} \hline \hline
**Requ.** & **Description** & **Fulfilled** \\ \hline FR1 & Use existing OpenStack cloud for resources & yes \\ FR2 & Specify different node settings & yes \\ FR2.1 & Emulated system architecture & yes, x86\_64 and ARM64 \\ FR2.2 & CPU speed and core count & yes, speed via CPU model \\ FR2.3 & Memory size & yes \\ FR2.4 & Storage size & yes \\ FR3 & Specify amount of different nodes in the cluster & yes \\ FR4 & Emulation of worker node hardware & yes \\ FR5 & Working Kubernetes cluster as defined & yes \\ FR6 & No additional privileges in OpenStack & yes \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Fulfillment of Requirements and Constraints. for 64-bit x86 and ARM systems. Supporting other architectures, such as 32-bit systems, would require building Cloud-Images with 32-bit support or significantly adjusting the installation process of Q8S to not rely on Ubuntu Cloud-Images. However, in order to deploy a Kubernetes cluster on such infrastructure, the respective container images must also be available. According to the image manifests available for the official Kubernetes repositories, only four architectures are supported: amd64 (i.e., x86_64), arm64, ppc64 le and s390x. As ppc64 le and s390x have either a negligible market share or are effectively discontinued, we conclude that the majority of system that will run Kubernetes utilize either x86_64 or ARM64. ### Implications

With Q8S, we provide a tool for emulating heterogeneous Kubernetes clusters. This enables researchers and developers of software for such clusters, for example, alternative scheduling systems, to perform tests and gather more detailed data compared to using simulated clusters if access to real heterogeneous hardware is not available. Furthermore, an emulated cluster can potentially be used to gather data to train machine learning models or have scheduling algorithm actively learn via reinforcement learning. Q8S, therefore, extends the selection of available tools to researchers beyond existing simulation tools such as CloudSim, K8 sSim and EMUSIM. ## 5 Conclusions

In this work, we have identified the need for emulated heterogeneous Kubernetes clusters in order to provide more accurate environments compared to utilizing simulation tools. To meet this need, we have designed and implemented Q8S as a tool for automatically deploying such clusters on top of OpenStack clouds by using QEMU to emulate heterogeneous nodes, e.g., ARM nodes on top of x86 machines. Moreover, we have validated the functionality of Q8S and benchmarked the performance overhead imposed by the emulation layer and found it to be acceptable. We provide our prototype implementation of Q8S under an open-source license to enable its adaptation and further development. Future WorkThe prototype of Q8S is functional but could be further improved. Many other simulations support setting bandwidth and latency for connections, which is to be implemented in Q8S. Similarly, support for network and node failures, which may occur more frequently for heterogeneous clusters with components outside of a data center, are also not yet supported. Due to the multiple layers involved when deploying a cluster via Q8S, if any of the deployment scripts get stuck and fail to complete, it can be difficult to identify the issue or even to retrieve the logs of the failed script.