## 1 Introduction
Therefore, no infrastructure management issues, such as file system operations, horizontal autoscaling of functions, load balancing, and system monitoring, need be addressed by developers. They can, therefore, focus solely on the application code. To optimize the performance obtainable through the serverless model, it is necessary to configure some parameters that determine its operation. In particular, the horizontal autoscaling of functions, typical of serverless platforms, must adapt to the edge system and its current state. In this paper, we use the concept of state to describe the essential configuration of the edge system and the value of the operational parameters, such as the current occupancy of resources. The main objective of this paper was to compare reinforcement learning algorithms capable of adapting to the state of the system, in order to determine the optimal operating parameters that determine the horizontal autoscaling events of the functions. In particular, these algorithms are able to learn how to configure the threshold value on CPU occupancy to trigger autoscaling events, in order to maximize the number of functions that can be instantiated in the edge, guaranteeing adequate latency and success rate values in accessing services. Some papers have already addressed related problems. For example, in [10], a comparison of OpenFaaS, AWS Greengrass, and Apache OpenWhisk with different workloads is shown. The edge computing cluster is based on Raspberry Pis. In [8], a performance comparison of FaaS platforms is shown, namely, OpenFaaS, Kubeless, Fission, and Knative. By using different runtimes written in Python, NodeJS, and Golang the authors evaluate the latency for accessing compute-intensive and I/O-intensive functions. A serverless model is considered in [9], where function invocation on the edge is based on a multi-armed bandit problem. In a recent paper [7], we propose the use of a reinforcement learning algorithm for minimizing the average observed latency for accessing a serverless function by using a Q-learning agent to learn the optimal HPA threshold. Nevertheless, the need to carry out the analysis shown in this paper depends on the fact that the results of the previous research activities, illustrated also in what follows, are often difficult to compare. The different characteristics of the systems, their different configuration, the service request arrival processes, and the deployed functions are so different as to make it difficult to understand what is the best choice to make in a real edge computing system. Moreover, different objective functions and differentiability properties, due to the use of different reward functions, can significantly influence the achievable performance. For this reason, we focused on three different algorithms inspired by different principles: value-based, policy-based, and a combination of both. These principles bring together almost all the proposals present in the literature. In particular, we compared the performance of the deep Q-network (DQN), advantage actor-critic (A2C), and proximal policy optimization (PPO) algorithms in an edge computing cluster orchestrated using Kubernetes. This choice was based on the fact that Kubernetes is currently the most popular open-source container orchestration and management technology. On this platform, we installed OpenFaa, which is one of the most popular serverless technologies. We used this integrated platform to deploy functions that emulate the behavior of an IoT-oriented edge system. In summary, the main contributions of this paper are as follows:  
* We provide insights into the typical problems of edge systems when they are managed with the considered technologies. * We compare the learning capabilities of different, popular reinforcement learning algorithms, based on system configuration and desired results. * We critically review the results, based on the type of algorithm--value-based, policy-based or mixed--and on the basis of the actual function invocation process. The paper is organized as follows.