# Performance Evaluation of Open-Source Serverless Platforms for Kubernetes
###### Abstract  
Serverless computing has grown massively in popularity over the last few years, and has provided developers with a way to deploy function-sized code units without having to take care of the actual servers or deal with logging, monitoring, and scaling of their code. High-performance computing (HPC) clusters can profit from improved serverless resource sharing capabilities compared to reservation-based systems such as Slurm. However, before running self-hosted serverless platforms in HPC becomes a viable option, serverless platforms must be able to deliver a decent level of performance. Other researchers have already pointed out that there is a distinct lack of studies in the area of comparative benchmarks on serverless platforms, especially for open-source self-hosted platforms. This study takes a step towards filling this gap by systematically benchmarking two promising self-hosted Kubernetes-based serverless platforms in comparison. While the resulting benchmarks signal potential, they demonstrate that many opportunities for performance improvements in serverless computing are being left on the table. serverless; open source; Kubernetes; benchmark; performance; self-hosted; HPC +
Footnote †: journal:  
CPU (or GPU) due to isolation technologies such as container isolation. Thus, computing resources can be shared dynamically by any number of users, instead of being reserved by one user at a time. Nevertheless, in practice it is not clear whether open-source serverless platforms can live up to the performance requirements of HPC workloads such that they are a valid alternative. Furthermore, serverless employs a different execution model compared to common distributed computing clusters, which usually depend on the Message Passing Interface (MPI) ([[https://www.mpi-forum.org/](https://www.mpi-forum.org/)]([https://www.mpi-forum.org/](https://www.mpi-forum.org/)) accessed on 27 May 2022) to coordinate computing efforts, necessitating either the adoption of applications and programs that are designed to operate on serverless platforms or the adaptation of existing applications and programs to the serverless model. Thus, switching to serverless could incur additional costs and work requirements beyond setting up a serverless platform. This makes it necessary to ensure that the chosen open-source serverless platform is able to deliver the desired improvements in resource utilization by benchmarking these platforms against each other. Despite this, there is currently a distinct lack of comparative studies on open-source serverless platforms. Most studies of serverless platforms focus on the most popular instance, AWS, which is present in 88% of serverless studies, followed by Microsoft (26%) and Google (23%), with open-source platforms combined amounting to only 14% according to [PERSON] et al. [4]. Even fewer studies have made an effort to compare the performance of multiple open-source platforms. Moreover, it is difficult to produce comparable benchmarks for serverless platforms, as there are no standardized benchmarks for them; due to the many variations in the architectures of serverless platforms, it is difficult to design universal benchmarks [5]. This paper focuses on two open-source platforms running on top of Kubernetes ([[https://kubernetes.io/](https://kubernetes.io/)]([https://kubernetes.io/](https://kubernetes.io/)) accessed on 27 May 2022), namely, OpenFaa5 ([[https://openfaas.com/](https://openfaas.com/)]([https://openfaas.com/](https://openfaas.com/)) accessed on 27 May 2022) and Nuclio ([[https://nuclio.io/](https://nuclio.io/)]([https://nuclio.io/](https://nuclio.io/)) accessed on 27 May 2022). In this paper, Kubernetes is used as a basis for running the serverless platforms under study for two reasons. First, Kubernetes is either supported by or even depended on by almost all open-source serverless platforms. Second, [PERSON] et al.