# Intelligent End-Edge Computation Offloading Based on Lyapunov-Guided Deep Reinforcement Learning
## 2 Related Work  
For industrial tasks, different algorithms were proposed to balance the computation offloading and resource allocation in different scenarios. Most existing works take delay or energy consumption minimization as the optimization objective. For example, [PERSON]. [7] employed the double and dueling architectures on the basis of deep Q-network, and proposed the D3 QN-based multi-priority computation offloading scheme to minimize overall task delay. [PERSON]. [8] used Lyapunov optimization to decompose the multi-layer multi-timescale resource allocation problem into three sub-problems, and employed a deep actor-critic algorithm to minimize the total queuing delay of all devices. [PERSON]. [9] proposed a multi-agent deep reinforcement learning (MADRL)-based scheduling algorithm, which actor-critic (AC) framework with estimation and target networks is designed for policy and value iterations to minimize delay. Similarly, [PERSON]. [10] proposed multi-agent double actor-critic algorithm to reduce the task processing delay while improving the blockchain transaction throughput. [PERSON]. [11] designed a joint communication and computation resource allocation mechanism based on Q-learning to minimize the total task delay cost. On the other hand, [PERSON]. [12] incorporated a data fusion system in the architecture and designed a joint computation offloading and resource allocation scheme to minimize the overall queuing delay of the system. Ref. [13] designed an algorithm that decomposes the problem into task offloading and channel allocation sub-problems, and proposed a low-complexity heuristic algorithm to solve the sub-problem efficiently for total weighted task processing latency minimization. Ref. [14] developed a novel online SBS peer offloading framework by leveraging the Lyapunov technique, in order to maximize the long-term system performance while keeping the energy consumption of SBSs below individual long-term constraints. Ref. [15] aimed to optimize the average energy consumption by using non-orthogonal multiple access (NOMA) technique to improve the spectral efficiency and access, which is solved using successive convex approximation. Ref. [16] combined NOMA technique and frequency division multiple access and used long short-term memory network to optimize the objective. Ref. [17] used the air-ground integrated computing networks multi-agent deep deterministic policy gradient algorithm to minimize the average energy consumption by jointly optimizing the computation task allocation and wireless resource allocation. Furthermore, there are also many works that combined time delay and energy consumption as optimization objectives. For example, Ref. [18] added a greedy algorithm to multi-intelligence depth determination of policy gradients to minimize the weighted sum of delay and energy consumption. Ref. [19] established a dual queue model containing data and computation queues, used the proximal policy optimization algorithm to minimize the weighted sum of average time delay and system energy consumption. Ref. [20] proposed an online joint offloading and resource allocation framework under the long-term MEC energy constraint, aimed at guaranteeing the users' QoE. Ref. [21] aimed to minimize the task expected cost, which includes the task execution time delay and energy consumption, and applied the time division multiple access technique to optimize the objective function. In addition, there are also some works that used other variables as the optimization objectives. For example, Ref. [22] considered the computation rate and energy consumption in computation offloading, proposed a deep deterministic policy gradient-based multiple continuous variable decision model to make the optimal offloading decision in edge computing. Ref. [23] proposed a computation offloading algorithm based on deep Q-Learning and used asynchronous federated deep Q-Learning to offload task. Ref.