## 3 Related Work  
### Reinforcement Learning in Edge Computing  
Many different contributions are available in the literature on the use of reinforcement learning in the edge context, starting from general surveys that focus on some research aspects of interest. For example, paper [4] provides an overview of research problems in cloud computing and edge computing and recent progress in addressing them with the help of computational intelligence techniques.The authors discuss future research trends and provide readers with some insights into research directions. Also, paper [15] shows a survey of federated reinforcement learning (FRL). The authors propose to classify algorithms in two categories, namely horizontal federated reinforcement learning and vertical federated reinforcement learning. Furthermore, the contributions on FRL are organized according to application fields, including those involving edge computing. Several papers show proposals for the management of computing and storage resources, which are typically limited in edge systems, through intelligent orchestration mechanisms. For example, in [16] the authors propose a deep reinforcement learning-based task orchestrator called DeepEdge, which is able to meet some of the requirements of heavily-loaded mobile networks. The problem is modeled as a Markov decision process,and a double deep Q-network (DDQN) algorithm is used to optimize robustness. Four different applications are considered, namely, image rendering, infotainment, pervasive health, and augmented reality. Paper [17] presents a federated reinforcement learning algorithm called FEMRL for edge computing systems. This algorithms inspires model-based reinforcement learning. In this proposal, clients train a local model based on locally sampled data. At the edge server, a global model is then created, using local models. In turn, the updated policy is then sent back to clients for subsequent sampling. Both theoretical and experimental results are shown. Paper [18] addresses the problem of handling a large number of devices in edge networks used by mobile users. In particular, the problem of unbalanced resource allocation is considered. For this purpose, a deep-Q-network algorithm is proposed for workload scheduling. The experimental results highlight good performance, in terms of service time and resource efficiency. The application scenario in [19] is vehicle-to-pedestrian communication scenarios in edge computing. The paper shows a proposal consisting of a communication task allocation algorithm based on deep reinforcement learning. The final objective is optimal spectrum and power allocation. The experimental results show that the learning agent can effectively improve vehicle-to-infrastructure communication rates and meet delay constraints. The analysis in [3] deals with the resource constraints of IoT devices and wireless networks. The authors propose a joint computation-offloading and multi-user-scheduling algorithm for IoT edge computing. The research objective consists of minimizing the long-term average delay. The proposed solution is tailored to an infinite-horizon average-reward continuous-time Markov decision process model. Deep reinforcement learning and a neural network are used to approximate the value functions. Paper [20] focuses on the deployment of large language models in edge computing systems. The research objective is enhancing privacy and computational efficiency. The analysis analyzes the impact of different splitting points, and the results are used to define a framework that will inspire model-based reinforcement learning algorithms. A reward surrogate model is defined, to balance inference performance and computational load. Paper [21] deals with a task assignment problem in three-layer edge/fog computing systems and proposes a reinforcement learning approach for solving the problem.