# Q8S: Emulation of Heterogeneous Kubernetes Clusters Using QEMU
## 1 Introduction
### Related Work  
CloudSim [10] is an open-source engine for simulating cloud computing environments. In CloudSim, a data center may operate a number of hosts, each of which runs one or more VMs, and tasks are executed in the VMs. The compute capabilities of hosts can be specified in core numbers and Million Instructions Per Second (MIPS) per core as well as memory, bandwidth, and storage. The simulation can then calculate for a set of tasks, each of which requires a number of compute instructions as well as memory and access to storage, what tasks would end up on which nodes and how long the execution of all tasks would take. CloudSim is an event-based simulation and does not emulate any hardware. Moreover, the heterogeneity of hosts is limited to CPU counts, speed, memory, and storage size and bandwidth; it does not include different system architectures. DynamicCloudSim [11] and Cloud2 Sim [12] provide further developments on top of CloudSim. DynamicCloudSim introduces performance variance by sampling CPU, IO, and bandwidth speed from a normal distribution instead of using a static value. Cloud2 Sim adds the option to run distributed simulations on CloudSim, as CloudSim itself can only be executed on a single host, which results in long compute times for large simulations. EMUSIM [16] is another extension of CloudSim; it utilizes emulated hardware to run applications to improve the accuracy of a simulation. By running the target application in a small-scale emulated environment with varying input sizes, EMUSIM is able to collect metrics regarding the scalability of the application. These metrics are then used in CloudSim to perform a more detailed simulation. The same limitations apply for the support of heterogeneous clusters as for CloudSim. K8 sSim [17], a tool similar to CloudSim, is capable of simulating Kubernetes clusters, including specifying CPU capacity, memory, and GPU count per node. Workloads can be configured with specific requirements and be scheduled with customizable scheduling algorithms. The simulation collects data on total execution time and wait times for workloads, which serve to evaluate a given scheduling algorithm. This can serve as a starting point for validating a new scheduling algorithm; however, it does not properly support heterogeneous clusters, as it does not run any workloads or performs hardware emulation. CloudNativeSim [13] also takes inspiration from CloudSim and provides a toolkit for simulating cloud applications that utilize microservice architectures. It implements many additional features such as various scaling techniques, request processing, and Quality of Service (QoS) metrics. While it employs containerization and pods as an abstraction for instances, it does not fully simulate Kubernetes deployments. Moreover, its support for heterogeneous environments is limited to simulating instances with different code environments and hardware capacities. PerficientCloudSim [14] provides an extension of CloudSim for it to support large-scale simulations using heterogeneous resources such as different CPU types. Different CPU types are simulated by considering their performance distribution in terms of the instructions they can process per second. While this provides a tool for testing scheduling systems on across different CPU types, it does not cover potential issues with varying CPU architectures. ServiceSim [15] is another toolkit built on top of CloudSim and has a focus on simulations of microservices in cloud-edge environments. One of its main goals is to enable the validation of policies against QoS requirements in heterogeneous networks. However, it does not specifically consider that edge nodes may employ heterogeneous hardware. faas-sim [31] is a simulation framework, which also supports edge environments but specifically focuses on serverless edge computing. Function life cycle management is based on Kubernetes and supports heterogeneous nodes via varying capabilities and capacities.