## 7 Conclusions and Future Works  
In conclusion, this work presents a scheduling policy for DL training tasks in a heterogeneous GPU cluster, addressing the variability of training performance. It develops upon the model-similarity-based scheduling policy by implementing a round-based mechanism and job packing. The round-based mechanism empowers the scheduler to periodically adjust scheduling decisions for a given job to optimise training performance. Furthermore, job packing enables the concurrent training of multiple models to improve GPU utilisation, particularly when a GPU is allocated for training a small model. As a result, it mitigates queuing delay during high cluster load.  
This work conducts various experiments considering three influential factors on training performance: GPU architecture, the number of GPUs used in training, and batch size. While batch size is typically a user choice, and its adjustment is optional for a scheduler to improve cluster efficiency, this work demonstrates improvements when adjusting the batch size along with a selection of GPU architecture and the number of GPUs used in training. When comparing our work to the state of the art, our work outperforms across all evaluation metrics, particularly under high cluster loads. We conclude that the model-similarity-based scheduling policy is more effective when implemented with the round-based mechanism and job packing than when implementing the model-similarity-based scheduling policy alone.  
Future works can be extended by improving the round-based mechanism to address its current drawbacks, where resources may idle if a job is completed before the round ends. One potential solution to address this drawback is to implement a predictive approach at the beginning of each round. This approach involves assessing whether the jobs scheduled for the current round are likely to finish before the round ends. If such a scenario is anticipated, the scheduler can pre-determine the scheduling of a job in the queue that is compatible with the allocated resources of the potentially finished job.  
Conceptualization, P.T. and J.S.; Methodology, P.T. and J.S.; Validation, P.T.; Investigation, P.T.; Data curation, P.T.; Writing--original draft, P.T.; Writing--review & editing, K.L., F.L., A.G., J.S. and P.B.; Visualization, P.T.; Supervision, K.L., F.L., A.G., J.S. and P.B. All authors have read and agreed to the published version of the manuscript.  
This research received no external funding.  
Not applicable.  
* **Data Availability Statement:** Publicly available datasets were analysed in this study. This data can be found here: [[https://gitlab.uni.lu/phhanapol/model-similarity-based-scheduling-policy](https://gitlab.uni.lu/phhanapol/model-similarity-based-scheduling-policy)]([https://gitlab.uni.lu/phhanapol/model-similarity-based-scheduling-policy](https://gitlab.uni.lu/phhanapol/model-similarity-based-scheduling-policy)).  
The authors are grateful for the HPC of the University of Luxembourg (ULHPC) and Grid'5000, which provided computing resources throughout this research.  
The authors declare no conflicts of interest.  
\begin{table}
\begin{tabular}{l l} \hline \hline
**Policy** & **Description** \\ \hline fifo & First in, first out (in YARN [39]) \\ isolation & Dominant Resource Fairness [40] \\ min\_total\_duration & Gavel to minimise makespan [8] \\ max\_min\_fairness & Tiresias [41] \\ finish\_time\_fairness & Themis [42] \\ similarity & Our scheduling policy \\ similarity\_gpu & Decision on GPU architecture \\ similarity\_gpu\_nppu & Decision on GPU architecture and the number of GPUs \\ similarity\_gpu\_nppu\_bs & Decision on GPU architecture, the number of GPUs, and \\  & the adjustment of batch size \\ \hline \hline \end{tabular}
\end{table}
Table 3: Scheduling policies for comparison.