# [PERSON]: An HPC Job Scheduler Using Deep Reinforcement Learning and Remaining Time Prediction
## 3 RLSchert
### Job Scheduler Based on Deep Reinforcement Learning
\begin{split} job_{j}=&(ID,res,enter\_time,start\_time, finish\_time,finish\_time\_hat,\\ & len\_real,len\_param,[run_{1},run_{2},\cdots,run_{n}])\end{split}  \tag{6}\]  
**Queue:** After a new job is generated, the simulator puts it into the backlog, and then, the job enters the waiting queue and waits for the scheduler to schedule it. The size of the waiting queue is unlimited, but the number of visible jobs to the scheduler is \(num\_nw\). Controlling the number of visible jobs reduces the complexity of the scheduler and is beneficial for the training scheduling policy by DRL. Some slots in the visible jobs may be empty. For example, the number of jobs is less than \(num\_nw\). The running queue is a fixed-length queue that contains jobs running in the cluster and provides information for the scheduler to decide to kill a job. The ready queue is a temporary queue for each job in the cluster. We used the remaining time prediction, and the finish time of the job may change during job runs. For example, the allocated time of a job is too short, and additional time needs to be added, so the following jobs need to be reallocated. To simplify the simulator's design, we put jobs with \(start\_time\) greater than the current time into the ready queue and wait for reallocating. **Cluster:** The cluster is the computing resources. We did not consider the modeling of computing nodes and network communication. All computing resources in the cluster were regarded as a resource pool. The formal definition of the cluster is shown in Equation (7), where \(num\_res\) represents the number of resource types. In this work, we only considered the CPU, which means \(num\_res=1\). \(time\_horizon\) is the maximum time step available to the scheduler, which is greater than the maximum runtime of the jobs. \(res\_slot\) is the number of CPU cores in the cluster. \(avbl\_slot\) represents the number of available resources at each time step, which is defined as a vector \(<v_{1},v_{2},\cdots,v_{n}>\). The same as DeepRM, \(canvas\) is the resource occupation of each job in the cluster, which is a matrix \(M\). The \(M_{ij}\) indicates which job occupies the \(j\)-th CPU at time step \(i\). The job is also assigned a unique color. \[cluster= (num\_res,time\_horizon,res\_slot,avbl\_slot,canvas) \tag{7}\]  
**Scheduler:** The scheduler selects jobs from the waiting queue to the cluster according to the information such as the waiting queue, the running queue, the computing cluster, and the job features, or kills a job from the running jobs for rescheduling. The kill policy includes a rule-based policy and a PPO-based policy. A job is killed by the rule-based policy means that: the time of the job has been running is less than the predicted time before running (\(len\_param\)), and \(len\_param<10\), the predicted time during running (\(run_{i}\)) is greater than 10, while the related error \(|run_{i}-len\_param|>1\). The PPO-based kill policy is described in detail below. The scheduler is a state-action function, which is defined in Equation (8), where \(cluster\_status\), \(Waiting\_queue\), and \(Running\_queue\) represent the status of the cluster, the waiting queue, and the running queue, respectively. \(A\) is the action, which can be the slot index in the waiting queue, or the slot index where the job is terminated, or the empty action that controls the progress of the simulator. \[f:(cluster\_status,Waiting\_queue,Running\_queue)\to A \tag{8}\]