# SLA-DQTS: SLA Constrained Adaptive Online Task Scheduling Based on DDQN in Cloud Computing
## 4 Algorithm Design
### Reward  
We prioritize task overdue time to ensure it when the load changes. To this end, we consider the two goals of makespan and task cost. \begin{table}
\begin{tabular}{l l} \hline \hline
**Notation** & **Description** \\ \hline \(t_{i}\) & Task instance \(i\) \\ \(t_{i\_{i}}^{mips}\) & Computing resource requirements for task \(i\) \\ \(t_{i\_{i}}^{mips\_l}\) & Bandwidth resource requirements for task \(i\) \\ \(t_{i\_{i}}^{l}\) & Number of tasks corresponding to computing resources for task \(i\) \\ \(t_{i}^{low\_l}\) & Number of tasks corresponding to bandwidth resources for task \(i\) \\ \(t_{i}^{dead}\) & Task processing time expected by user for task \(i\) \\ \(t_{i\_{start}}^{load}\) & Time when task \(i\) is submitted to task center \\ \(t_{i\_{firstish}}^{start}\) & Time when task \(i\) is executed by node \\ \(t_{i}^{t}\) & Time when task \(i\) is completed \\ \(t_{i}^{cost}\) & Execution cost for task \(i\) \\ \(vm_{j}\) & VM instance \(j\) \\ \(vm_{j}^{mips}\) & Number of instructions executed per second by VM \\ \(vm_{j}^{bw}\) & Bandwidth of VM \\ \(vm_{j}^{price}\) & Price per second of VM \\ \(ET_{ij}^{l}\) & Time to execute \(t_{i}\) by \(vm_{j}\) \\ \(vm_{j}^{bustime}\) & Time from completion of all tasks assigned to \(vm_{j}\) calculated by clock at current moment until VM is idle \\ \(t_{i\_}AVM\) & Machine set of executable \(t_{i}\) requirements \\ \(CLU\_VM\) & VM set in cluster \\ \(OT_{ij}\) & Overdue time calculation equation for task \(t_{i}\) allocated to \(vm_{j}\) at current clock \\ \(r\) & Reward function \\ \(r^{1}\) & Average task processing speed of this batch \\ \(r^{2}\) & Task cost performance of this batch \\ \(r^{3}\) & Average task overdue time of batch \\ \(r^{1}_{rate}\) & Weight value of \(r^{1}\) in reward \\ \(r^{2}_{rate}\) & Weight value of \(r^{2}\) in reward \\ \(time_{start}^{finish}\) & Time from submission until all batch tasks completed \\ \(t_{k}^{et}\) & Value by which task exceeds expected value; if not exceeded, it is zero, so \\  & it is always greater than or equal to 0 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Main notation used in this paper. However, these goals conflict. For example, a low-cost strategy will increase the utilization rate of low processing power VM, which will increase the time spent. If makespan is the main goal, the utilization rate of high processing power VM will increase. Therefore, it is necessary to balance the importance of goals, and the dimensions between them must be as close as possible. Our tasks arrive in batches, so we calculate the reward of a batch and take the average value as the reward of each task in it. In the reward function, we set the overdue time as the main factor to optimize the task overdue time under high load, while under low load, we balance the cost and throughput according to the set weight. Thus, we define the reward function as  
\[r=\left(r^{1}*r_{rate}^{1}+r_{i}^{2}*r_{rate}^{2}\right)*max\left(1-r^{3},0.1 \right)-r^{3} \tag{11}\] \[\text{Subject to }(r_{rate}^{1}+r_{rate}^{2}=1)^{\prime},\]  
where \(r^{1}\), \(r^{2}\), and \(r^{3}\) are related to makespan, task cost, and task overdue time, respectively. When \(r^{3}\) is greater than 1, task overdue time becomes the main goal, and the proportions of \(r^{1}\) and \(r^{2}\) are reduced to a very low level. When \(r^{3}\) returns to the low-load state, \(r^{3}\) is 0, and the reward is only related to \(r^{1}\) and \(r^{2}\) to achieve the dynamic switching of scheduling strategy between high and low loads.