## 4 New Virtualization Solutions for Network Functions
* CCM (CIS cluster management) focuses on cluster management and provides lifecycle management, configuration management, performance and fault management [30]. * CIR (container image registry) manages OS container images. These new management entities, introduced to manage CNF clusters, can be mapped onto the functions offered by Kubernetes, which is the natural implementation solution for container orchestration. The ETSI standard [30] does not specify interfaces and reference points for these new entities, thus leaving a considerable freedom in their implementation. Some documents from ETSI [31; 32] or cloud providers [33] show some possibilities, together with a mapping to Kubernetes functions, but a fully agreed solution has not yet been reached. The main difficulty is due to the fact that the way ETSI NFV operates is significantly different from the Kubernetes approach. In fact, the NFV model uses the declarative approach to manage imperative operations, whereas Kubernetes uses it in an intent-based style. In more detail, NFV specifies procedures and interfaces between NFVO, VNFM, and VIM, whereas Kubernetes manages internal and external communications using artifacts, APIs, and manifest files, specifying only the intent, which is the desired final operating state. Figure 2: IoT mMTC services belonging to different scenarios can be deployed in a flexible and dynamic way by leveraging NFV features. The proposed lightweight architecture is shown as the NFV infrastructure block, compliant with the ETSI NFV standard. In terms of mapping, Kubernetes has an active role in both cluster provisioning (NFVI) and control (NFVO, VNFM, and VIM). In fact, the CIS function can be associated with a cluster of Kubernetes Worker nodes, capable of providing CNFs. Instead, the control counterpart (CISM) can be associated with the functions offered by the Kubernetes Master nodes. Kubernetes API implements container management service interfaces exposed by the CISM. A important role is also played by Helm, which is a tool for managing OS containers that are deployed on Kubernetes CIS clusters. The Helm CLI allows for the implementation of the OS container workload management service interface exposed by the CISM [31; 32]. In terms of what concerns the CCM, it has the ability to scale the number of CIS clusters (i.e., Kubernetes clusters) based on the resources that are necessary to serve the workload offered, and is not present in the Kubernetes ecosystem; while there are dedicated proprietary tools in cloud providers such as Google, AWS or Microsoft that offer this function, porting it to the edge is more complicated. The main point is whether there are enough resources on the edge node(s) to serve the offered workload. If the answer is positive, it is necessary to introduce further tools able to provide and configure resources as additional Kubernetes clusters, either virtualized or bare metal. However, if these resources are not available locally, it is necessary to perform a bursting operation in the edge-to-cloud continuum (see, e.g., Ref. [34]), with possible performance issues. The adoption of the serverless approach should help to limit the bursting events towards remote clouds. Regarding CIR, Kubernetes does not offer any specific service to host container images, but relies on external services. Instead, it is available as a Kubernetes Registry service, which is an image pull secret used to authenticate a private image registry, in order to pull a container image hosted on it. In this regard, it is also useful to point out the fully serverless approach allows a developer to write the code and upload it without additional operations. It will be executed in a pre-determined environment with a collection of libraries and OS installed. Thus, CIR is mainly related to the classic approach, based on containers, or for those users that, while using serverless, prefer to prepare a custom image to pull and deploy. In general, the presence of Kubernetes also significantly changes the way in which other management entities work.