###### Abstract
In recent years, real-time scheduling of concurrent tasks and workflow scheduling on cloud computing or edge computing have received special attention from researchers [3,6,7]. The optimization objectives in such studies mostly focus on resources, QoS, and time. The methods employed for scheduling are predominantly heuristic, including rule-based and search-based methods. A small portion of the research has utilized deep learning methods, primarily for assessing the effectiveness of solutions using neural networks, rather than having the neural networks make decisions. These heuristic methods often lack analysis of computation time in experiments. However, it can also be observed that researchers are gradually integrating the latest deep learning research into traditional methods for such problems. The traditional methods for solving scheduling problems are primarily heuristic methods, which can be further classified into two categories. The first class are rule-based heuristic scheduling methods, which typically rely on predefined rules for scheduling without optimizing the scheduling results. The second class belongs to search-based heuristic algorithms. This method often utilizes various rules to search the solution space and find a feasible solution to the optimization problem within acceptable costs (time or space), which is close to the optimal solution. Up to now, there have been numerous metaheuristic works on cloud/edge computing scheduling [8]. Examples of such methods include Particle Swarm Optimization (PSO) [9,10], Ant Colony Optimization (ACO) [11], and Differential Evolution (DE) [12], which have been adopted to optimize the makespan or cost (e.g., resource cost) while satisfying other QoS constraints. It is worth noting that the makespan and cost of executing a workflow on the cloud are two conflicting optimization objectives. For instance, reducing the makespan of the workflow requires more resources, which often leads to higher execution costs. As a result, many algorithms struggle to strike a balance between these two objectives. Furthermore, such methods often face challenges in meeting the real-time requirements of certain scenarios. In recent years, with the development of artificial intelligence, Reinforcement Learning (RL) has become an active area of research in machine learning [13,14]. Its ability to address decision-making problems has inspired researchers to use RL to tackle optimization problems. Most research leverages Deep Reinforcement Learning (DRL) to learn solutions for problem-solving by designing evaluation metrics for the problem and enabling neural networks to continuously attempt and learn through trial. With the trained network, excellent solutions can be obtained in a short amount of time. Many studies have utilized Deep Reinforcement Learning to solve resource management problems [15], workshop job scheduling problems [16], and cloud-edge task allocation problems [17]. While using Deep Reinforcement Learning methods can avoid designing highly complex heuristic rules and offer faster computation times, there are still challenges in designing the environment states and network models, as well as the need to balance different optimization objectives. Many heuristic methods utilize search and neural 94 network techniques, but their computational efficiency falls short in real-time scheduling 95 environments. Previous research has encountered several challenges: (1) Heuristic methods often require the design of overly complex rules, and scheduling rules need to be designed while satisfying problem constraints. When the constraints change, the scheduling rules may need to be modified accordingly. The methods of deep reinforcement learning become increasingly challenging to train as the problem constraints increase, leading to an increase in the required input features. (2) Many scheduling methods, while capable of producing excellent results, have computation times that are unacceptable for real-time environments.