# Performance Evaluation of Open-Source Serverless Platforms for Kubernetes
## 3 Results
### Byte Reversal  
For the byte reversal workload, we measured the time needed to complete 1 or 100 requests for the single and parallel variants of the workload, respectively. The requests carried from 1 B up to 16 MiB per request, with the Go variant only going up to 2 MiB as all requests using a larger request body failed. Furthermore, the parallel variants were tested using 1, 2, 4, 8, and 16 function instances, while the single variant was only tested for 1 and 4 function instances. The analysis depends on the throughput in bytes per second calculated from the size of the requests and the time needed to successfully complete all requests of a test run. \(y\) bytes per second is calculated as \(y=n\cdot\frac{b}{t}\), where \(n\) is the number of requests, \(b\) is bytes per request, and \(t\) is the time needed to process \(n\) requests of size \(b\). Figure 3 shows the results of the benchmarks as box plots, with OpenFaaS (Figure 3a,b), Nuclio (Figure 3c,d), and Nuclio Go (Figure 3e,f), respectively, in each row, single requests on the left side (Figure 3a,c,e), and parallel requests on the right side (Figure 3b,d,f). Each graph shows the throughput rate in bytes per second on the y-axis and the request size on the x-axis. As the request size was doubled after each test run, a logarithmic scale with base 2 is used on the x-axis. Each graph shows the results for the number of function instances that reached the highest peak throughput rate. This is notable for the single variant with four function instances for OpenFaaS (Figure 3a) and Nuclio (Figure 3c) and only one function instance for Nuclio Go (Figure 3e), as well as for the parallel variants with four function instances for OpenFaaS (Figure 3b) and with eight function instances for Nuclio (Figure 3d) and Nuclio Go (Figure 3f). Furthermore, the concrete peak throughput rates in MiB/s are listed in Table 2 along with the number of function instances used. The graphs for the single request variant (Figure 3a,c,e) show a significant number of outliers, especially the Nuclio Go implementation, which is caused by the fact that every single box in each plot relies on 1000 data points. However, the outliers may offset the average throughput rate used to discuss the results. To ensure that a mean average-based discussion is valid, Table 3 shows the arithmetic mean (as Mean), geometric mean (as Gmean), and harmonic mean (as Hmean) for the single variant. The table shows that \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Variant**} & \multicolumn{2}{c}{**OpenFaaS**} & \multicolumn{2}{c}{**Nuclio**} & \multicolumn{2}{c}{**Nuclio Go**} \\ \cline{2-7}  & **Throughput** & **Instances** & **Throughput** & **Instances** & **Throughput** & **Instances** \\ \hline Single & 7.47 & 4 & 9.64 & 4 & 12.71 & 1 \\ \hline Parallel & 20.00 & 4 & 33.27 & 8 & 33.25 & 8 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Peak average input processing rates in MiB/s for byte reversal workload, single and parallel variants. the geometric mean and harmonic mean are all within 90% of the arithmetic mean, even for Nuclio Go. Therefore, the outliers do not significantly distort the results; thus, the remaining discussion uses the arithmetic mean. Figure 3: Throughput-focused workload: reversing input with OpenFaaS, Nuclio, and Nuclio Go. (**a**) OpenFaaS, single request, four function instances; (**b**) OpenFaaS, parallel, four function instances; (**c**) Nuclio, single request, four function instances; (**d**) Nuclio, parallel, eight function instances; (**e**) Nuclio Go, single request, one function instance; (**f**) Nuclio Go, parallel, eight function instances. In Table 2, it can be seen that, for single requests, Nuclio processes about 29% more bytes per second than OpenFaaS, while Nuclio Go processes 32% more bytes per second than Nuclio and 59% more than OpenFaaS. Each of them reached their highest throughput rate for the largest number of bytes per request that they could handle, that is, 16 MiB for OpenFaaS and Nuclio and 2 MiB for Nuclio Go.