# Task Scheduling Mechanism Based on Reinforcement Learning in Cloud Computing
## 1 Introduction
(3) Load Balancing Scheduling Methods: the aim of these methods is to achieve load balancing among devices in order to improve resource utilization and system performance. Examples include round-robin (RR) scheduling, random scheduling, and queue-length-based scheduling. These methods can balance task loads, but fail to consider task characteristics and device performance differences. Existing cloud computing task scheduling faces several challenges. (1) Large-Scale Task Processing: cloud computing environments may have a massive number of tasks that need to be scheduled and processed. Handling such a large-scale task processing introduces complexity in task scheduling and increases computational complexity. (2) Dynamic Environment: task arrivals and departures in the cloud computing environment are dynamically changing. The number of tasks and their resource requirements can change at any time. Therefore, task scheduling needs to exhibit real-time adaptability and flexibility to promptly respond to changing demands. (3) Resource Allocation and Load Balancing: task scheduling requires the rational allocation and utilization of resources in the cloud computing environment in order to achieve load balancing and maximize resource utilization. Inadequate resource allocation or load imbalance can lead to system performance degradation and resource wastage. Based on the need for better task scheduling performance, this study applies reinforcement learning to cloud computing task scheduling and proposes a two-stage dynamic cloud task scheduling framework called Q-Learning-based Multi-Task Scheduling Framework (QMTSF). The first stage occurs in the cloud data center,; when user tasks arrive, they and are initially stored in a global queue (GQ). Then, based on the characteristics of the tasks, they are dynamically assigned to appropriate servers. The second stage takes place within each server, specifically, the server queue (SQ). A time window is used to determine when the tasks in the queue are assigned to each virtual machine (VM). For each time window, the server's task scheduler first prioritizes all tasks in the queue based on their deadline constraints, then employs a UCB-based Q-Reinforcement Learning (UQRL) strategy to assign tasks to suitable VMs. An incentive mechanism is applied to reward assignments that minimize the makespan of tasks. The main contributions of this paper are as follows:  
1. We propose Q-learning-based Multi-Task Scheduling Framework (QMTSF) to address task scheduling issues in cloud computing. Task assignment is performed in the cloud data center, while task scheduling for different VMs is conducted within each server. 2. For the task assignment phase in the cloud data center, we introduce a method based on server task type priorities to allocate tasks to suitable servers. 3. For task scheduling within each server, we first employ a dynamic sorting strategy to prioritize tasks based on their deadlines. This enables the Q-Learning-based task scheduler to consider the more urgent tasks earlier, contributing to improved service quality. Additionally, we apply an improved Q-Learning algorithm based on the UCB strategy (UQRL) to the task scheduling within each server, resulting in reduced average task processing times and overall makespan. 4. We conducted experiments to validate the effectiveness and superiority of QMTSF, comparing its performance against the PSO algorithm, random scheduling, and RR scheduling. The experimental results confirm the effectiveness of QMTSF. The structure of this paper is as follows: Section 2 provides an overview of the current research progress and achievements in cloud computing task scheduling; Section 3 introduces the QMTSF system model proposed in this paper; Section 4 presents the dynamic task allocation algorithm and UQRL algorithm used in QMTSF; and Section 5 presents the results of the simulation experiments. Finally, Section 6 concludes the paper.