# A Novel Fault-Tolerant Aware Task Scheduler Using Deep Reinforcement Learning in Cloud Computing
## 3 Problem Formulation and Proposed System Architecture  
This section represents the formulation of the problem and proposed system architecture. Initially, the task scheduling problem is formulated in such a way that \(k\) tasks are indicated as \(T^{k}=\left\{T^{1},T^{2},\ T^{3},......T^{k}\right\}\), \(n\) virtual machines are indicated as \(V^{n}=\left\{V^{1},V^{2},V^{3}......V^{n}\right\}\), \(s\) hosts are indicated as \(PH^{s}=\left\{PH^{1},\ PH^{2},......PH^{s}\right\}\), and \(z\) datacenters are indicated as \(D^{z}=\left\{D^{1},\ D^{2},....D^{z}\right\}\). After consideration of the above entities, \(T^{k}\) tasks are mapped to \(V^{n}\) VMs which are placed in \(PH^{s}\) hosts and placed in \(D^{t}\) datacenters by considering priorities of the task and VMs while minimizing makespan, rate of failure, and energy consumption. The Figure 1 represents the system architecture of the proposed deep reinforcement learning fault-tolerant task scheduling algorithm. In this architecture, initially a variety of users submit heterogeneous tasks to the cloud platform. At the cloud provider, these submitted tasks are captured by a broker which is an agent-based entity that takes all tasks and submits these tasks to the task manager. In our research, at the task manager level, we used logic to calculate task priorities and VM priorities. All tasks submitted by the broker to the task manager are to be prioritized based on task length and run-time capacities. VM priorities are calculated using the unit cost of electricity at the corresponding datacenter. These priorities are fed to the reinforcement-based scheduler. The scheduler is connected to the resource manager as it needs to track resources in the cloud. With consideration of these priorities, the scheduler will add these task sequences to the execution queue from which tasks are assigned to VMs based on the priorities. In this scheduler, we used a deep Q-learning model which works based on reinforcement learning, in which it takes input and analyzes tasks fed to it and schedules tasks to appropriate VMs. This is an agent-based \begin{table}
\begin{tabular}{c c c} \hline
**Author** & **Methodology** & **Parameters** \\ \hline
[17] & RL to make a scheduling model for cloud computing &
\begin{tabular}{c} Makespan, total power cost in datacenters, energy consumption, migration time \\ \end{tabular} \\
[18] & RL task scheduling with Q-learning &
\begin{tabular}{c} Makespan, total power cost, energy consumption \\ Makespan, energy consumption, packet delivery \\ ratio, trust value \\ \end{tabular} \\
[20] & FELARE &
\begin{tabular}{c} Ontime task completion rate, energy saving \\ Makespan, resource usage \\ Computation time of tasks \\ Makespan \\ \end{tabular} \\
[21] & RLFTWS &
\begin{tabular}{c} Makespan \\ Communication delay, reliability \\ Turnaround time, makespan \\ Processing cost, load balancing tasks \\ Makespan, resource utilization \\ Total execution time and resource utilization ratio \\ Task completion time, makespan \\ \end{tabular} \\
[22] & Dynamic task scheduling algorithm based on an improved GA &
\begin{tabular}{c} Total execution time and resource utilization ratio \\ Task completion time, makespan \\ Makespan, computation time \\ \end{tabular} \\
[30] & DRL-based task scheduling &
\begin{tabular}{c} Energy consumption, total cost, makespan \\ Cost and throughput, makespan \\ Execution time, degree of imbalance \\ Response time, makespan, CPU utilization \\ SLA violation, makespan \\ Estimated completion time, resource utilization \\ Degree of imbalance, cost, makespan \\ Energy consumption, resource utilization \\ Response time, degree of imbalance \\ Makespan, total cost \\ Makespan \\ Task response time \\ Makespan \\ \end{tabular} \\ \hline \end{tabular}
\end{table} 
Table 1: Cont. learning model; initially, it randomly schedules tasks based on priorities and looks for outcomes and if tasks are precisely distributed to VMs.