## 5 Performance Evaluation
### Experimental Results
#### 5.2.2 Kafka Broker Virtual Network Function Experiment
Thus, this can be regarded as a lower bound of the message rate. Figure 9: Message delivery ratio of a Kafka broker VNF running in a Kata container on a (blue) QEMU hypervisor, and a (red) Firecracker hypervisor. The second KPI we analyzed was the CPU consumption, shown in Figure 10. Again, it emerged that the two frameworks performed very similarly to each other up to five topics, and then the Kata + QEMU solution consumed more CPU time. This is easily explainable, since it was also able to handle a slightly larger amount of Kafka traffic. The general comment was that such a CPU consumption is very low (less than 4.5% of a CPU core for handling a traffic rate of about 25 Mbps). Thus, again, the resource footprint of the proposed solution was more affordable than in edge/MEC nodes. Regarding the concerns of the final KPI--that is, the latency in handling Kafka messages-- the two solutions looked, again, very similar. The latency values were approximately 3 ms for handling a traffic rate of 25 Mbps. This value was low enough to support practically all the use cases surveyed in previous sections. Since the deployed VNF could be modeled in the user plane as a single server deterministic model with a constant offered traffic rate (D/D/1 model in [PERSON]'s notation), it was possible to evaluate the input rate at which the system was no longer stable and started losing messages. This rate value could be identified as the one corresponding to five topics. In fact, when five topics were used, the message loss was nearly negligible; thus, the service arrival rate \(\lambda_{p}\) could be considered equal to the Kafka broker service rate \(\mu_{p}\) in saturation. The estimated rate was 4 Kpps, being the message size equal to \(L_{p}=800\) B. Thus, the service capacity \(C\) was equal to \(C=L_{p}\mu_{p}\). Once this value was known, it was possible to estimate the average service time in more general conditions, using the general formulation provided by the G/G/1 model. The use of a single server model was motivated by the fact that the service consisted of the execution of work by all components of the IoT service (Kafka broker) to satisfy each user request. For this reason, the service process statistics were indicated as general in [PERSON]'s notation. Regarding the average service rate, given the average packet length \(\mathbb{E}[L]\), then \(\mu=\frac{C}{\mathbb{E}[L]}\). If the statistics on service and arrival processes are known, it is possible to use [PERSON]'s formula to approximate the mean waiting time \(T_{w}\)[49], as follows:  
\[T_{w}\approx\left(\frac{\rho}{1-\rho}\right)\left(\frac{c_{a}^{2}+c_{s}^{2}}{ 2}\right)\frac{1}{\mu}, \tag{1}\]  
Figure 10: Percentage CPU consumption of a Kafka broker VNF running in a Kata container on a (blue) QEMU hypervisor, and a (red) Firecracker hypervisor. where \(\rho=\frac{\lambda}{\mu}\) is referred to as average utilization, whereas \(c_{a}\) and \(c_{s}\) are the coefficients of variation of arrival times and service times, respectively. This means that \(c_{a}=\sigma_{a}\lambda\), with \(\sigma_{a}\) being the standard deviation of arrival times, and \(c_{s}=\sigma_{s}\mu\), where \(\sigma_{s}\) is the standard deviation of arrival times. From the equation above, it follows that the overall mean service time \(T_{s}\) can be estimated as follows:  
\[T_{s}\approx T_{w}+\frac{1}{\mu}. \tag{2}\]  
When we move away from deterministic arrival or services processes, the condition of \(\rho\approx 1\) cannot be clearly approached, as the system becomes unstable. The Kingman's approximation is known to work well in situations with heavy load that are, thus, close to saturation. This is indeed a possible situation in edge computing scenarios, where the available resources are limited. For lighter load conditions, a good approximation is given by the M/G/1 model, from which the Pollaczek-Khinchine formula [50] is derived.