## 5 Performance Evaluation
### Experimental Results
#### 5.2.1 NetBricks Virtual Network Function Experiment  
The experimental comparison was organized as follows. An image of Ubuntu 17.10 hosting NetBricks was deployed with a Kata container (v.1.11.2) using both QEMU, which was the baseline solution, and Firecracker v.0.21.1. These two hypervisors are currently supported by Kata. The widespread QEMU was expected to guarantee an extensive hardware support, while Firecracker brought a technology that minimized resource footprint and workloads. The first experimental observation was relevant to the disk footprint of the two hypervisors. As expected, they required a significantly different storage. In particular, QEMU required 16 MB of disk space, whereas Firecracker required just 2.6 MB. This was significant since, in situations of limited availability of resources, as it should be in MEC nodes or resource-limited devices, a saving of disk space of more than 80% is a huge benefit. If a large number of VNFs (from hundreds to thousands) has to run concurrently on the same node, the storage requirement of Firecracker would be almost negligible, whereas that of QEMU could become significant.  
More insights can be obtained by executing VNFs in the proposed architecture. Figure 8 shows RAM usage related to the execution of NetBricks on a Kata container, with QEMU and Firecracker as a function of time. Both of them were executed in the following two different scenarios: deployment (labeled as \"Hypervisor\" in Figure 8) and deployment with subsequent VNF invocation (labeled as \"VNF\"). In this example, the VNF realized with NetBricks is a packet generator included in the framework, _packet-test_. We present the performance achieved by this VNF to show the impact of a single VNF on the resource footprint.  
We can observe that, within a second, from a starting point of 559 MB of RAM already allocated on the server by other processes, deploying a NetBricks container (dotted line) Firecracker resulted in an overhead of 3 MB (red curve) while the QEMU was 20 MB (blue curves). Furthermore, in the container deployment with the execution of the packet generator VNF, Firecracker was lighter than QEMU, with a difference of about 10 MB over a total of about 41 MB; that is, a saving in memory footprint of about 25%. The vertical lines in Figure 8 show the average boot system + user time; Firecracker deployed NetBricks in \(T_{boot,F}=\)71 ms and QEMU in \(T_{boot,Q}=\)83 ms, respectively. This implies a 14% saving in boot time, which was significant. From our experiments, it emerged that the resource footprint and deployment latency values were similar to those characterizing most of the VNFs made available in NetBricks. Thus, the presented use case is representative of real scenarios, such as those illustrated in Section 3. From the experimental results we can identify the most suitable deployment strategies for VNFs in future IoT applications. Essentially, the selected strategies depend on the latency requirements of applications. For example, uRLLC applications characterizing the IoV and some smart manufacturing applications are highly sensitive to latency values. Even in some smart health applications, such as remote surgery, the VNF deployment time should be larger than the latency tolerance. For all other applications, including entertainment and traffic management services in IoV, remote patient monitoring in smart health applications, or smart grid surveillance, the dynamic behavior of the proposed solution can significantly reduce the resource footprint and maximize the usage of computing and storage resources, both in MEC-based applications and in the core of a cloud network. In fact, the boot time of a VNF is by far inferior to the tolerated latency; thus, it is possible to start up a new VNF instance only when strictly necessary, and possible prediction errors, due to the AI-based scheduling system, can be tolerated.