## 5 Evaluation Results
### Results of Performance Analysis
#### 5.3.2 Latency with Cold Starts  
While the evaluation of performance metrics with container request latency, API response time, and CPU utilization provides valuable insights, these results can be further detailed by discussing additional factors that can influence overall performance. One such consideration with serverless cloud computing is cold starts, a system-level challenge faced when a serverless instance is initializing itself and its run-able code, typically exacerbated by serverless platforms that enact scale-to-zero policies [52]. However, one study notes [53] that the cold start problem also arises \"when the auto scaler provisions additional instances to handle the traffic\". This therefore means that cold starts are a near-unavoidable issue with serverless cloud computing and that optimizations must be considered to lessen the impact of cold starts that, when occurring, introduce latency to requests. To better facilitate these optimizations, we might first consider understanding each of the platforms' cold start policies regarding when a container is set for recycling such that cold starts will now occur as new containers are spun up. This study proposes such an experiment to find these values [54], wherein a log file from a container's /tmp directory is read to determine whether a machine is cold or warm. Table 5 lists the results of this testing, showing how AWS appears to support less container reuse compared to GCP and Azure since AWS yielded a significantly higher percentage of cold starts. However, within [54]'s testing, there was a noticeable difference in cold start times for AWS, wherein the average function invocation time was lower for AWS, with an average 246.146 ms duration for invocation in contrast to GCP's 6284.838 ms duration. This therefore shows that AWS has likely applied its optimizations to favor cold starts, whereas GCP likely favors longer-living containers and avoiding cold starts if possible. Of note is that these findings contrast with those who found in [55] that, while Azure had a relatively consistent 20-min container recycling time and AWS had a steady preference for recycling containers given last-function invocation durations greater than 28 min, GCP had no discernible pattern and cold starts were observed to occur randomly from 3 min of this duration to 1 h. Given the prevalence of the shorter durations for which containers were observed to be enduring a cold start, this then challenges the claim made in the previous paragraph, which concludes that GCP favors keeping containers warm. Since a cold start will occur within the lifetime of an application and thus introduce latency which an end user might potentially find undesirable, we must still consider the means by which we might optimize for cold start conditions. [PERSON] suggested the methods of performance and log monitoring for indicators of startup bottlenecks [56]: increasing the memory allocated to a container, choosing faster runtimes, keeping shared data in memory, shrinking the package size to contain only the minified and optimized code that is to be run, and keeping a pool of pre-warmed functions to ensure that, for some number of requests, there is that number of containers that are in a warm state and able to immediately handle those requests. The suggestion of decreasing the package size when possible is further substantiated by [PERSON] [57], who found that, when deploying containers of sizes 1 KB, 14 MB, and 35 MB, there was a nearly 5 s average difference in the cold start times of AWS and GCP and, for Azure, there was roughly 10 s of average startup duration difference. These differences show the importance of reducing the code bundle such that request latency is reduced, ushering in a better user experience.