# Performance Evaluation of Open-Source Serverless Platforms for Kubernetes
## 2 Materials and Methods
### Workload Design  
Two application-motivated workloads that could be easily ported to other FaaS systems were designed for the experiments. First, an image processing workload that represents CPU-heavy tasks, which takes in images, performs a set of transformations on them, and outputs the resulting images. This workload is a placeholder for other workloads that do some form of data processing by taking in data, executing code on the data while relying on a C/C++ library, and saving the processed data. The image transformations were performed through ImageMagick ([[https://imagemagick.org](https://imagemagick.org)]([https://imagemagick.org](https://imagemagick.org)) accessed on 27 May 2022) using its C++ API. Furthermore, this workload comes in two variants, the regular processing variant and a null-processing (NP) variant that skips the transformations and sends back the images after loading them and sleeping for 1 s. The purpose of the NP variant is to provide an estimate of how the system would behave without having to perform the CPU-heavy task such that more throughput-focused tasks can be represented with this workload. The C++ implementations of the workload variants were the same for both OpenFaaS and Nuclio. The implementation of the regular variant can be seen in Appendix A.1. The C++ code was compiled with the following command: <<+ 'Magick++-config --cnxflags --cppflags' -O2 -o imageproc imageproc.cpp 'Magick++-config --ldflags --libs'. The second sample workload focused on reversing of bytes as a simple task that requires fully loading the input before it can be sent back. The purpose of this workload is to additionally vary the number of bytes sent with each function invocation to determine the optimal input size for each platform. Moreover, this workload comes in two variants and two implementation languages. In the first variant, referred to as single, only a single request is sent and the time to respond to that request is measured. In the second variant, referred to as parallel, a number of requests are sent, similar to the image processing workload, and the time to process all requests is measured. Implementation was carried out in C++ and in Go, and can be found in Appendices A.2 and A.3, respectively. As discussed earlier, the function implementations must be embeddedinto function environments. Neither OpenFaaS nor Nuclio offers a dedicated C++ function environment, only general-purpose templates that can be used to deploy C++ functions. Internally, these templates work by running a web server written in Go, starting the user function by forking a sub process, and sending the input data to it via the standard input stream and receiving outputs via the standard output stream. When considering performance, this is not optimal, as it effectively means that the web server receives the input data and then has to copy it into the memory of the sub process. This extra work could be avoided by allowing the user function to directly operate on the input data. The Nuclio template for Go does this by having its users implement their Go functions as Go plugins ([[https://pkg.go.dev/plugin](https://pkg.go.dev/plugin)]([https://pkg.go.dev/plugin](https://pkg.go.dev/plugin)) accessed on 27 May 2022), which produces binaries that can be loaded at runtime by a Go program. This avoids the extra work of copying the input data, as these plugins become native extensions of the web server that can operate on the original input data. Therefore, it is reasonable to include a Go implementation of the workload for Nuclio. OpenFaaS uses the same streaming technique for Go templates as it does for the general purpose templates, thus, a Go implementation of the workload for OpenFaaS was left out.