# MASA: Multi-Application Scheduling Algorithm for Heterogeneous Resource Platform
## 4 Multi-Application Scheduling Algorithm (MASA) Based on DRL
### Framework of MASA  
#### 4.1.1 NN-Scheduler  
The state encoding network design references graph neural networks and self-attention mechanisms [10] to achieve scalable network input dimensions. The input states contain the jobs DAG's attributes (e.g., number of remaining tasks, examined task computational execution time, the PE types to which the tasks can be mapped, and DAG dependency structure), and the outputs are feature vectors at the task-level, the job-level, and the global-level, respectively. The task-level feature vector is performed by the \(\mathrm{g}_{1}\)-network, which transforms the attribute information of each job DAG through a nonlinear transformation into a task-level feature vector \(\mathrm{e}_{\mathrm{v}}^{\mathrm{i}}\). \(\mathrm{e}_{\mathrm{v}}^{\mathrm{i}}\) captures the dependencies of the front and back nodes, as shown in Equation (6):  
\[\mathrm{e}_{\mathrm{v}}^{\mathrm{i}}=\mathrm{g}\left[\sum_{\mathrm{u}\in \mathrm{sub}(\mathrm{n}_{\mathrm{v}}^{\mathrm{i}})}\mathrm{f}(\mathrm{u}) \right]+\mathrm{x}_{\mathrm{v}}^{\mathrm{i}} \tag{6}\]  
where \(\mathrm{x}_{\mathrm{v}}^{\mathrm{i}}\) represents the attribute information of each node \(\mathrm{n}_{\mathrm{v}}^{\mathrm{i}}\) and \(\mathrm{sub}\left(\mathrm{n}_{\mathrm{v}}^{\mathrm{i}}\right)\) is the set of child nodes of \(\mathrm{n}_{\mathrm{v}}^{\mathrm{i}}\). \(\mathrm{g}_{1}\) (\(\ast\)) contains the nonlinear transformation, where \(\mathrm{f}\sim\log(\cdot/\mathrm{n})\), \(\mathrm{g}\sim\exp(\mathrm{n}\times\cdot)\), \(\mathrm{n}\rightarrow\infty\). The \(\mathrm{g}_{2}\)-network's input vectors, \(\mathrm{e}_{\mathrm{v}}^{\mathrm{i}}\), derived from \(\left|\mathrm{J}\right|\times\left|\mathrm{j}\right|\) space. We adopt the attention mechanism to represent the complex relationship between \(\mathrm{e}_{\mathrm{v}}^{\mathrm{i}}\), transforming the raw data into scalable states to capture the feature vectors \(\mathrm{y}^{\mathrm{i}}\) of the entire job DAG. The job-level feature vector \(\mathrm{y}^{\mathrm{i}}\) is obtained by summarizing all the feature vectors \(\mathrm{e}_{\mathrm{v}}^{\mathrm{i}}\) in the DAG, as indicated in the blue box in Figure 3. Equation (7) calculates the weighted vector \(\mathrm{\varphi}_{\mathrm{score}}^{\mathrm{i}}\), while Equation (8) represents the attention function that computes the weighted sum. This sum indicates the correlation between the weighted vector \(\mathrm{\varphi}_{\mathrm{score}}^{\mathrm{i}}\) and the vector sequence \(\mathrm{e}^{\mathrm{i}}=\left\{\mathrm{e}_{1}^{\mathrm{i}},\mathrm{e}_{2}^{ \mathrm{i}}\ \ldots\mathrm{e}_{\left|\mathrm{j}\right|}^{\mathrm{i}}\right\}\), where \(\left|\mathrm{j}\right|\) is the number of tasks in job \(\mathrm{j}_{\mathrm{i}}\). \[\mathrm{\varphi}_{\mathrm{score}}^{\mathrm{i}}\ =\ \mathrm{softmax}\left(\frac{\mathrm{ \varphi}_{\mathrm{q}}\cdot\mathrm{\varphi}_{\mathrm{k}}}{\sqrt{\mathrm{d}_{ \mathrm{k}}}}\right)\cdot\mathrm{\varphi}_{\mathrm{v}}=\mathrm{softmax}\left( \frac{\mathrm{(w}_{\mathrm{q}}\cdot\mathrm{e}_{\mathrm{v}}^{\mathrm{i}})^{ \mathrm{T}}\mathrm{(w}_{\mathrm{k}}\cdot\mathrm{e}_{\mathrm{v}}^{\mathrm{i}})}{ \sqrt{\mathrm{d}_{\mathrm{k}}}}\right)\mathrm{(w}_{\mathrm{v}}\cdot\mathrm{e}_{ \mathrm{v}}^{\mathrm{i}}\right) \tag{7}\]  
Figure 3: NN-Scheduler Implementation. \[y^{i}=att\Big{(}\varphi^{i}_{score}\,,e^{i}\Big{)}=\sum_{v=1}^{||i|}\varphi^{i}_{ score}\ \cdot e^{i}_{v} \tag{8}\]  
In order to be able to better represent the connection between all job DAGs, a global feature vector \(z\), \(z=\psi\left(y^{1},y^{2}\dots y^{n}\right)\) represents a nonlinear function. 2. Policy network for task prioritization  
The prioritization of tasks is achieved using the policy network after obtaining the feature vectors that correspond to the tasks in the job DAG.