# A Novel Fault-Tolerant Aware Task Scheduler Using Deep Reinforcement Learning in Cloud Computing
## 3 Problem Formulation and Proposed System Architecture
Then, a positive reward will be raised by the DQN model and it learns, for a certain number of tasks generated previously, the best schedule and if there is a random distribution of task-generated schedules or if the outcome is not good then it considers it as a negative reward and it will learn from previous generated actions. In this research, we used a reward-based agent model which learns on its own based on the situation, i.e., type and number of tasks generated in this scenario. In our research, our main objective using this DRFTSA scheduler is to minimize the rate of failure which is a QoS parameter and it is very important from the point of view of the cloud provider. Failure in the cloud environment is often because many users execute their requests from all around the world and they are running their tasks from remote locations. Therefore, there is a chance to have a single point of failure in the cloud environment due to the unavailability of the VMs, network issues, etc. Our proposed DRFTSA scheduler keeps track of all the tasks submitted to it and, based on considered priorities, the scheduler needs to make a decision on its own and generate schedules effectively. Mathematical modeling of the proposed DRFTSA is shown below in Table 2. \begin{table}
\begin{tabular}{c c} \hline \hline
**Name** & **Notation** \\ \hline \(Crlo^{V}\) & Current execution of tasks \\ \(Crlo^{PH_{s}}\) & Current execution of tasks on all physical machines/nodes \\ \(PR^{cap}_{V}\) & Processing capacity of VMs \\ \(T^{k}_{le}\) & Task length \\ \(T^{pi}\) & Task priorities \\ \(V^{pi}_{n}\) & Virtual machine priorities based on unit electricity cost \\ \(ms_{k}\) & Makespan of \(k\) tasks \\ \(enc_{cons}(V^{n})\) & Energy consumption of \(n\) tasks \\ \(Rate_{failure}\) & Rate of failure \\ \(act^{T}\) & Action space for \(T\) tasks \\ \(sta^{T}\) & State space for \(T\) tasks \\ \(Rew_{fun}\) & Reward function \\ \(M\dot{\omega}\) & Replay memory \\ \(\mu\) & Learning rate of agent \\ \(\dot{\theta}\) & Time for agent learning \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Mathematical notations used in proposed DRFTSA. Figure 1: System architecture of DRFTSA. In the first step, after submission of various heterogeneous tasks by the users to the cloud platform, the broker captures those tasks and in turn submits tasks to the task manager. At this level, to calculate priorities of tasks, initially the task manager needs to know the currently running tasks on VMs and then it needs to know the current execution of tasks load on all physical machines/nodes. The current execution of tasks running on VMs is calculated using below Equation (1). \[Crlo^{V}=\sum Crlo_{n}^{T} \tag{1}\]  
where \(Crlo_{n}^{T}\) indicates current execution of tasks on \(n\) VMs. Now, we calculate the current execution of tasks on all physical machines/nodes using Equation (2). \[Crlo^{PH_{s}}=\frac{Crlo^{V}}{PH^{s}} \tag{2}\]  
where \(Crlo^{PH_{s}}\) indicates the current execution of tasks running on \(s\) physical machines/nodes. In this research, an important aspect is to identify the appropriate VM which can accommodate tasks submitted to the cloud console. Therefore, the scheduler needs to make a decision whether to assign tasks to a specific VM or not. This decision is mainly based on the processing capacity of a VM. It is calculated in Equation (3) below. \[PR_{V}^{cap}=PR_{no}\times PR_{MIPS} \tag{3}\]  
where \(PR_{V}^{cap}\) indicates the processing capacity of a VM, the number of processing elements is indicated by \(PR_{no}\), and the number of instructions processed per second is indicated by \(PR_{MIPS}\). After submitting the requests to the task manager, it need to identify two priorities, i.e., the task and VM priorities which prioritize the input task sequence which can be fed to the scheduler. Basically, task priority depends on its length and the run-time capacity of the VM. Therefore, task length is calculated in Equation (4) below.