# Task Scheduling Mechanism Based on Reinforcement Learning in Cloud Computing
## 2 Related Works
In the exploration-exploitation trade-off, only the immediate benefits are considered, without taking into account the cost of obtaining those benefits (i.e., exploration attempts). Excessive consideration of immediate benefits may overlook the value of other potential strategies, while excessive focus on exploration attempts may neglect the maximum benefit from exploitation, leading to a lower overall success rate. Thus, a good choice is to balance current benefits and exploration attempts with the aim of achieve the highest possible success rate within a limited number of rounds. The Upper Confidence Bound (UCB) algorithm is an algorithm that uses the current benefits and exploration attempts as the criteria for selection. The UCB strategy was proposed as a method of balancing exploration and exploitation in the multi-armed bandit problem [31; 32]. This method balances the relationship between exploration and exploitation by considering the number of times an action is chosen, and has better performance [33] than methods such as \(\varepsilon\)-Greedy. Several improved methods based on UCB have been proposed, such as UCB1-tuned [34] and Discounted UCB [35]. The UCB approach adopts an optimistic attitude towards uncertainty, and uses the upper confidence bound of option returns as the estimated value of returns. The basic idea is that the more times a certain option is tried, the narrower the confidence interval for the estimated returns becomes, and the lower the uncertainty. Options with higher mean values tend to be selected multiple times, representing the conservative part of the algorithm (exploitation). On the other hand, options with fewer attempts have wider confidence intervals and higher uncertainty. Options with wider confidence intervals are more likely to be chosen multiple times, representing the aggressive part of the algorithm (exploration). [PERSON] et al. [36] studied the performance of applying UCB to the Q-Learning algorithm and conducted numerical analysis, demonstrating its balance between exploitation and exploration and its superior performance compared to other methods. [PERSON] et al. [37] applied UCB-2 to the Q-Learning algorithm, allowing multiple agents to act in parallel and achieving a near-linear speedup in execution time. Their approach outperformed existing parallel algorithms in terms of sample complexity. [PERSON] et al. [38] formulated network optimization as a [PERSON] decision process problem and developed a UCB-based exploration reinforcement learning algorithm called Q\({}^{+}\)-Learning. This approach improves convergence speed, minimizes performance loss during the learning process, and achieves optimal throughput. [PERSON] et al. [39] utilized an energy-efficient Q-Learning algorithm based on the UCB strategy to dynamically activate and deactivate SCCs (sub-carrier components) in order to maximize user throughput with the minimum number of active SCCs, thereby maximizing energy efficiency. [PERSON] et al. [40] proposed a user scheduling algorithm based on the UCB strategy in reinforcement learning that continuously updates the estimated behavioral value of each user. This approach avoids the phenomenon of \"extreme unfairness\" during the exploration phase, reduces algorithm complexity, and improves system throughput. [PERSON] et al. [41] introduced a variant of Q-Learning called Q-greedyUCB, which combines the average reward algorithm of Q-Learning and the UCB algorithm to achieve an optimal delay-power trade-off scheduling strategy in communication systems. Simulation results demonstrate that this algorithm is more effective than \(\varepsilon\)-greedy and standard Q-Learning in terms of cumulative reward and convergence speed.