## 6 Experimental Setup and Results  
### System Architecture and Technologies  
This section includes a detailed overview of the experimental setup, to allow the interested reader to reproduce the experiments. The edge computing system was implemented by using an HP Omen server 45L GT22-1013 nI with an Intel(r) i9 14900k processor at 5.8 GHz (eight hyperthreaded performance cores + 16 efficient cores, for a total of 32 threads) and 64 GB RAM. By using the Hyper-V hypervisor, we instantiated two Ubuntu 22.04.2 virtual machines for implementing a two-nodes computing cluster managed by Kubernetes (k8s). The two nodes implemented the master node and a worker node of the Kubernetes cluster. The master node was configured with 4 virtual CPU cores (vcpus) and 12 GB of RAM. The worker node was configured with 24 vcpus and 32 GB of RAM. The decision to use a single computer node was made in order to emulate a single-node edge system, a fairly common situation. However, this did not compromise the generality of the results. In fact, regardless of the number of nodes used, the resources, in terms of CPU cores and RAM, were abstracted by Kubernetes and managed as a whole. Thus, what really impacted on performance was the volume of available resources. The FaaS capabilities were introduced by using OpenFaas. This is a serverless platform that enables creation and deployment of applications through serverless computing. Its architecture is container-based and makes use of Docker containers to execute functions. Every function is packaged within a self-contained image, allowing developers to use the desired programming language. The OpenFaas integration with Kubernetes allows deploying functions quickly on Kubernetes clusters, leveraging its scalability capabilities. The interaction with this tool is mediated by a built-in API Gateway that exposes the cluster functionality. The API Gateway manages the routing of requests to the corresponding functions and manages security policies. In the experimental system, we installed the Community Edition of OpenFaaS, which is the only open-source distribution of OpenFaaS. Although the Community Edition offers a wide range of features and tools, it has some limitations compared to the commercial version. In particular, the HPA allows replicating only five instances of the same function. For this reason, in our experiments we used the Kubernetes HPA for scaling functions. The reinforcement learning algorithms were implemented by using the Stable-Baselines3 library ([[https://stable-baselines3.readthedocs.io/en/master/](https://stable-baselines3.readthedocs.io/en/master/)]([https://stable-baselines3.readthedocs.io/en/master/](https://stable-baselines3.readthedocs.io/en/master/)), accessed on 1 July 2024), which is a Python library that provides implementations of different reinforcement learning algorithms. It is compatible with PyTorch ([[https://pytorch.org/](https://pytorch.org/)]([https://pytorch.org/](https://pytorch.org/)), accessed on 1 July 2024) and Gymnasium ([[https://gymnasium.farama.org/index.html](https://gymnasium.farama.org/index.html)]([https://gymnasium.farama.org/index.html](https://gymnasium.farama.org/index.html)), accessed on 1 July 2024). The latter is a tool that allows implementing a personalized environment through a set of open APIs. The environment that we implemented included \begin{table}
\begin{tabular}{c c} \hline \hline
**Hyperparameter** & **Value** \\ \hline Learning rate & 0.0001 \\ \hline Entropy coefficient, c & 0.01 \\ \hline Discount factor, \(\gamma\) & 0.8 \\ \hline Learning coefficients, \(c_{1}\),\(c_{2}\) & 0.5, 0 \\ \hline Batch size & 10 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Hyperparameters used in experiments. the action space, the state definition, the reward function, and any details needed to allow interaction with the reinforcement learning agent.