## 6 Experimental Setup and Results
### System Architecture and Technologies
The requests sent to the edge computing system were generated via the Hey traffic generator ([[https://github.com/rakyll/hey](https://github.com/rakyll/hey)]([https://github.com/rakyll/hey](https://github.com/rakyll/hey)), accessed on 1 July 2024). This is a tool used to generate and simulate load towards an application, facilitating the testing and development process. Hey allows sending customized HTTP requests by generating output information on the status of the requests and response times. It also allows some simple forms of concurrent generation of parallel requests as well as the control of the generation rate. Data from the Azure Public Dataset was used for both training and tests, in particular the Azure Functions Trace 2019 ([[https://github.com/Azure/AzurePublicDataset](https://github.com/Azure/AzurePublicDataset)]([https://github.com/Azure/AzurePublicDataset](https://github.com/Azure/AzurePublicDataset)), accessed on 1 July 2024), by selecting one of the functions characterized by a greater frequency of invocation. In the invocation dataset used, each row, which represents a day of requests, presents an almost periodic trend. In other words, there is a pattern of requests that highlight a sort of daily periodicity. Furthermore, during a day, sometimes rapid changes in the frequency of requests occur, while in other intervals there are none at all. The peaks of the request frequency are around the value of 1800 requests/minute. The experimental tests were carried out with a compute-intensive function. Specifically, the function factored a given input number in the HTTP request. The reason we used a compute-intensive function was that the horizontal autoscaling policy is based on CPU occupancy. If we had used a different autoscaling policy--for example, based on memory occupancy--to carry out a similar analysis, we would have had to use a memory-bound function. For training, we set a maximum acceptable latency threshold \(\tau=0.5\) s for this service, i.e., the SLA. If the observed service latency exceeded this threshold, the quality of service was assumed to be compromised. Therefore, the reward function \(R_{TOT}\) used took into account both the observed latency and the losses. We assumed a loss when the HTTP response code was different from \"200 OK\". Thus, we formalized this performance metric as a combination of the latency reward \(L_{t}\) and the loss reward \(T_{s}\), as follows:  
\[L_{t}= \begin{cases}min(0,-w_{1}\frac{|l_{m}-\tau|}{\tau}),&l_{m}>\tau\\ w_{1}(1-\frac{|l_{m}-\tau|}{\tau}),&0<l_{m}\leq\tau\end{cases}  \tag{15}\]  
\[T_{s}= \begin{cases}0,&\text{success rate}=1\\ -w_{2},&\text{otherwise}\end{cases}  \tag{16}\]  
\[R_{TOT}=\frac{L_{t}+T_{s}}{w_{TOT}} \tag{17}\]  
where \(l_{m}\) was the average response latency in one minute and \(w_{i}\) was the weight of different contributions, set to \(w_{1}=w_{2}=w_{TOT}=10\). The reward represented the numerical value that returned to the agent based on its actions. When the agent took optimal actions, it was rewarded with the best achievable reward in the state; otherwise, it was assigned a negative or zero reward. In this way, the reward served as a performance measure for the agent during the training process. The reward formulation was developed gradually, to avoid making the system unstable. This function was piecewise differentiable, with a discontinuity on the latency threshold. Therefore, gradient ascent algorithms could converge the policy parameters towards the threshold from both directions. Before defining the final formulation, we conducted an in-depth base case analysis, to identify the weaknesses of each case. In more detail, we should not keep the service latency as low as possible, but just within the SLA value. In fact, the other fundamental key performance indicator (KPI) for an edge system is resource utilization efficiency. Since the considered function was CPU-intensive, we were interested to keep high the utilization efficient of the processor. This led to the current version of the reward.