# Vendor-Agnostic Reconfiguration of Kubernetes Clusters in Cloud Federations
###### Abstract
This problem is called vendor lock-in [2]. At present, there are many methods and architectures that tackle such challenges like the Broker Model [2] or Volunteer Federations of providers [3]. The recent trend of using the standardized container orchestration platform Kubernetes (K8s) introduces less complexity to avoid vendor lock-in because Kubernetes hides most heterogeneity of the underlying infrastructure and provide a unifying computing platform [4,5]. More specifically, a K8s cluster can run on different IaaS providers, and with the help of the K8s control plane, the cluster itself can achieve automatic scaling, container management and other functionality across multiple cloud providers. Although a single K8s cluster has good support for deployment across different availability zones within a single cloud provider, a single cluster has bottlenecks in scalability [6] and cannot be deployed across multiple georegions or different cloud providers. Therefore, cluster federation solutions, which interconnect multiple Kubernetes clusters on different cloud regions or cloud providers, has the natural advantage of improving cluster performance and scalability and can further facilitate geodistribution as well as management of ultralarge systems. There already exist many K8s vendors that target edge and fog computing environments such as K3s and KubeEdge [7]. As 5G networks and edge computing converge, we hypothesize that federations of K8s clusters can run across any random topology of edge, fog and cloud sites, yet all must be configured in the same manner in order to achieve consistent application behavior and consistent application and cluster management. It is unlikely that such sets of distributed K8s clusters can all be offered by the same K8s vendor or edge, fog and cloud provider. This is especially unlikely when envisioning next-generation ultralow-latency and ultrareliable 5G applications that require that workloads run closely to the end users [8]. As such, cluster federations across edge, fog and cloud computing environments will inherently involve multiple K8s vendors. However, member clusters in a cluster federation need to be configured in the same manner in order to preserve (i) _application behavior consistency_, (ii) _application management consistency_ and (iii) _cluster management consistency_. (1) Applications deployed in a cluster federation have to perform consistently on each member cluster to meet commercial and technical requirements and to comply with security policies defined by governance solutions. For example, in order to achieve the same user experience in different regions, an application needs to have the same performance level and use the same security standards for data plane security on each member cluster; (2) Management and deployment of applications must also be performed consistently in order to simplify the process of application management. For instance, only if each member cluster supports exactly the same APIs, application administrators can manage their applications uniformly in a federation setting; this is especially a problem when APIs have been deprecated but they are still highly demanded by using community, e.g., the PodPreset API in the OpenShift community [9]; (3) Management consistency is also needed at the level of the control plane of each cluster. For example, from a security perspective, each cluster needs to have the same security guarantees; e.g., the etch databases running in the control plane of each cluster may be required to encrypt data at rest. In order to meet these three consistency requirements, it is important that the K8s control plane and K8s agents are configured in the same way across different member clusters. Moreover, if member clusters belong to different K8s vendors, these three consistency requirements cannot always be met due to differences in default cluster configurations and, more importantly, the _inability to modify the default configurations_.