# Task Scheduling Mechanism Based on Reinforcement Learning in Cloud Computing
## 2 Related Works
[18] proposed an improved GA combined with a greedy strategy to optimize the task scheduling process, reducing the number of algorithm iterations and significantly improving performance in terms of minimizing the total completion time, average response time, and QoS. [PERSON] et al. [19] introduced a hybrid heuristic algorithm based on genetic algorithms and lists for scheduling heterogeneous computing tasks. However, these optimization methods provide an optimal solution and are primarily designed for static optimization problems, making them less suitable for dynamic optimization problems. In response to the aforementioned issues, Reinforcement Learning (RL) [20; 21; 22] has been actively applied in cloud computing task scheduling. RL is a direct learning strategy that enables an agent to learn appropriate policies through trial and error, acquiring rewards from the environment without any prior knowledge. The agent observes its own state and selects actions to obtain rewards associated with the chosen actions from the environment. Based on past experiences, the agent learns the optimal actions to maximize rewards. Q-Learning is a classic RL algorithm that has been applied in task scheduling. [PERSON] et al. [23] and [PERSON] et al. [24] proposed a sensor node cooperative RL approach for task scheduling to optimize the trade-off between energy and application performance in sensor networks. Each node considers the local state observations of its neighbors and shares knowledge with other agents to enhance the learning process and achieve better results than individual agents. [PERSON] et al. [25] combined the Q-Learning RL method with an improved supervised learning model, Support Vector Machine (ISVM-Q), for task scheduling in Wireless Sensor Networks (WSNs). The ISVM model takes state-action pairs as inputs and calculates Q-value estimates. Based on this estimation, the agent selects the optimal tasks to be performed. Experimental results have demonstrated that the ISVM method improves application performance by putting sensors and communication modules in sleep mode when necessary while preserving network energy. [PERSON] et al. [26] proposed Q-Grid, an RL-based hierarchical protocol, to improve information dissemination rates by minimizing latency and hop count. The protocol divides the geographic region into small grids and searches for the optimal grid next to the destination based on the traffic of neighboring grids, establishing a Q-Value table. [PERSON] et al. [27] proposed QEEC, an efficient energy-saving cloud computing task scheduling framework based on Q-Learning. This framework utilizes the M/M/S queue model for task reallocation and employs a Q-Learning algorithm based on an \(\hat{\mathrm{I}}\)-Greedy policy [28] to achieve energy-efficient utilization of cloud resources during task scheduling. [PERSON] et al. [29] introduced a global-perspective flexible task scheduling scheme based on Q-Learning, aiming to improve task scheduling success rate, reduce latency, and prolong the lifespan of the Internet of Things (IoT). [PERSON] et al. [30] proposed a Q-Learning approach based on multi-objective optimization and the FANET routing protocol. They were able to achieve a higher packet delivery rate and lower latency and energy consumption, leading to better performance. For RL, the process can be divided into exploration and exploitation. Exploration refers to the selection of actions to gain new knowledge, while exploitation refers to selecting the best action based on stored knowledge. The agent should strike a balance between these two. Resolving this trade-off problem allows the agent to achieve more efficient learning, as an inefficient exploration strategy would result in more suboptimal actions and poorer performance. In the exploration phase of Q-Learning, the \(\varepsilon\)-Greedy algorithm is commonly used, which employs a completely random policy for exploration. However, this algorithm converges slowly and is prone to becoming trapped in local optima.