## 1 Introduction
### Related Work  
Recently, researchers have focused on reducing delay and energy consumption through computational offloading and resource allocation techniques. In [11], the authors specifically examined how fog and cloud computing can work together to minimize service delay for IoT devices. To accomplish this, they optimized computational offloading, computing resource allocation, radio bandwidth allocation, and transmit power assignment, in a cloud/fog computing system. The objective was to reduce the weighted total of delay and energy consumption, and the task was described as a mixed-integer non-linear programming problem. The researchers used a sub-optimal approach with minimal complexity to address this problem, as described in [11; 15]. To accelerate the performance of offloading, the learning-based offloading strategy goes one step ahead. A reinforcement learning-basedcomputing offloading strategy was investigated in [1; 2], and [16] achieved a significant performance gain in IoT applications. In [17], the author analyzed a multi-user, multi-edge node computational offloading issue constrained by the availability of computing resources. Subsequently, a model-free reinforcement learning offloading method was applied to the time-varying channel information and CPU cycles to learn the long-term offloading approaches and enhance their utility. Due to the dimensionality curse and the limitations of Q-learning, the DRL method has been increasingly applied to huge state-space-based complex control issues. Deep reinforcement learning (DRL) is investigated in [18] as a potential solution for tackling network dynamics, resource diversity, and the integration of managing resources with mode selection in mobile edge computing networks (MECNs). In [19], the authors presented a joint optimization challenge, where they addressed mode selection, channel assignment, power allocation, and discrete phase shift selection to optimize the average sum data rate of device-to-device pairs. In [12], the authors explore using a DRL approach to managing computation offloading and multi-user scheduling in internet of things (IoT) edge computing systems. The aim of this research was to reduce the overall delay and power consumption, taking into account the unpredictable arrival of data traffic, by optimizing the weighted sum of these two factors over an extended period of time. Similarly, in [20] employs an actor-critic DRL strategy to handle content caching, compute offloading, and radio resource allocation difficulties in edge-enabled IoT systems. They offer a combined optimization method for the edge-enabled IoT network to decrease energy usage and end-to-end delay. Moreover, in [21], the authors proposed a deep Q-network-based power allocation method for NOMA-enabled network devices to improve energy efficiency and reduce power consumption. They examined wireless network delays and dynamic energy-efficient resource allocation, optimizing the system's energy efficiency while meeting delay constraints. The authors proposed a strategy in [11; 22] for offloading computation tasks in mobile edge computing (MEC) networks by utilizing a double deep Q-network-based approach that considers the time-varying network dynamics. Their approach describes the computational offloading policy as a Markov chain of decisions with the objective of maximizing long-term utility. Another approach, i.e., deep reinforcement learning-based online offloading (DROO), is studied in [23] for precise offloading decisions and wireless resource allocations under the time-varying conditions of a wireless channel. DROO uses a deep neural network as a scalable method for advanced learning. Several studies have explored deep reinforcement learning for intelligent resource allocation and management, such as in [23], where a DRL-based communication link scheduling algorithm is explored for a cache-enabled opportunistic interference alignment wireless network.