# Intelligent End-Edge Computation Offloading Based on Lyapunov-Guided Deep Reinforcement Learning
## 1 Introduction  
In the context of the fourth industrial revolution represented by \"Industry 4.0\", communication technology and manufacturing technology are deeply integrated, enhancing networking and intelligence industrial of production process [1]. Thus, it is necessary to establish an interconnected, intelligent and stable industrial wireless network [2]. Industrial wireless technology, an innovation of this century, has become a hot spot after the field bus. Industrial wireless changes the information transfer of the existing control system. It boasts robust anti-interference capabilities, low power consumption, high reliability, and numerous other technical virtues [3]. In the future, industrial control technology will necessitate stringent requirements for reliability, low latency, and high-speed performance, further underscoring the significance of these advancements. The industrial production process involves a large number of industrial tasks and the rapid increase of CPU-bound applications and delay-sensitive applications has added new challenges to the development of industrial wireless networks. Confronted with this deluge of industrial tasks, the conventional cloud computing model encounters limitations in addressing data processing and computational demands. Mobile edge computing (MEC) has emerged to enhance data processing efficiency. By strategically deploying edge servers at the network's periphery, the MEC system effectively brings the computing prowess of the cloud center closer to the edge, satisfying the escalating demand for computational augmentation at industrial terminals while fostering proximity between industrial terminals and edge servers. This arrangementcurtails transmission time for industrial tasks, mitigating the latency in cloud computing. Nonetheless, when large-scale industrial tasks migrate to these edge servers concurrently, network congestion occurs, escalating task processing time delays and reducing computational throughput. Consequently, computation offloading in industrial wireless networks confronts the following pivotal challenges. Firstly, devising offloading strategies that are prudently calibrated to performance metrics. Secondly, dynamically allocating industrial tasks according to actual demand, ensuring optimal resource utilization. Enhancing resource utilization and optimizing performance metrics like energy consumption, time delay, and computation rate are essential to address these challenges. When tackling dynamic challenges in the computation offloading strategies, ensuring system stability is very important. The Lyapunov optimization theory, originating from control theory, emerges as a robust approach to validate system stability and has been ingeniously adapted to construct queuing models for dynamic systems. Incorporating queue size into the objective function resolution, it fosters a relatively stable system state, thereby achieving the desired optimization objectives. Solving dynamic problems such as mixed-integer nonlinear programming (MINLP) requires high computation complexity, especially in large networks. The Lyapunov optimization theory excels in dynamically adapting to the ever-changing network conditions at every instant of system evolution, negating the need for manual intervention in adjusting control variables [4]. It is adaptable to real-time control of dynamically changing systems. Also, it ensures relatively low computation complexity, which is more straightforward than the direct solution of other algorithms [5]. Therefore, the application of Lyapunov optimization can transform the MINLP problem into continuous deterministic sub-problems while providing theoretical guarantee for the long-term stability of the system. Traditional model-driven computation offloading and resource allocation strategies rely on exhaustive system information to construct accurate system models.