## 4 DQN-Based Priority/Demanding Resource Management
### DQN-Based Solutions
The target Q-network is periodically synchronized with the online network to stabilize learning. This process is repeated for multiple episodes, allowing the MEC server to learn and refine its resource allocation strategies, ultimately returning the optimal action for each time slot based on the learned policy. ``` Input Output & Discount factor, learning rate, exploration factor greedy, replay memory size & Output & Optimal action of each slot \(a^{*}(t)\) & Online and target-\(Q\) network parameters with random weight \(\vartheta\) and \(\vartheta^{-}\), respectively. (\(\vartheta^{-}=\vartheta\)), replay memory \(B\) & & episode = 1, 2,..., k \(\to K\)do & &



consumption. More servers could handle more tasks; however, the infrastructure would consume more energy. IIoT devices are in industrial setups that generate data and require computation. The numbers [50; 100; 150] indicate different scenarios with varying numbers of devices. The size of the tasks generated can range from 5 MB to 30 MB. Bandwidth is the maximum rate of data transfer across the network. A bandwidth of 20 MHz ensures that data can move quickly between the IIoT devices and MEC servers, reducing the time and energy needed for communication. CPU frequency determines how fast the MEC servers can process data. Depending on the workload, a range of 5 GHz to 20 GHz means the servers can adjust their processing speed. A maximum link latency of 1.5 milliseconds ensures that the system responds quickly, which is critical in industrial settings. The operation is divided into 1000 time slots, which are small intervals of time used to schedule tasks and allocate resources. The replay memory buffer is where data is temporarily stored while being processed. A size of 3000 units means the system can hold a significant amount of data at once, which helps manage tasks efficiently and conserve energy. ReLU is a mathematical function used in neural networks within the system to make decisions, such as task scheduling or resource allocation. A discount factor of 0.95 means the system values long-term efficiency slightly more than short-term gains, encouraging energy-efficient decisions over time. The learning rate is 0.001, ensuring the system learns gradually and avoids making large, energy-inefficient changes based on new data. The batch size of 32 is a balance between computational efficiency and energy use, allowing the system to learn effectively without overloading the servers or consuming excessive power. ### Performance Evaluation

With the above setup simulation infrastructure, we deploy our management scheme and reference approaches in the controller to evaluate the performance. DQG-PD leverages the virtualization and the proposed DQN agent to guide the allocation process. Priority on industrial services and demand are observed from the infrastructure plane. The target networks discover the action batches on offloading decisions resource (bandwidth and computing resources) virtualization properties by exploration. We compared this with (1) meta-heuristic balancing the resources and demand and (2) single-agent DRL with a reward emphasized for service with higher priority (low PDB and PELR). To evaluate the performance of our contribution domains, we set 3 slices of different priority levels with demanding conditions. The traffic rates are configured to be high congestion following four different settings to capture the bottleneck and constrained resource evaluation. For the performance metrics, we focus on (1) the reward (resources,