## 1 Introduction  
Deep learning (DL) represents a paradigm within machine learning (ML) wherein iterative learning occurs through neural networks processing input data to acquire the problem-solving ability [1]. Over the past decade, DL has garnered significant success across diverse domains, including but not limited to image processing and natural language processing [2, 3]. Despite these achievements, it is noteworthy that training DL models is a time-consuming and resource-intensive task [4]. In order to reduce the consumption of time, Graphics Processing Units (GPUs) for parallel processing have emerged as a viable alternative to accelerate the training process [5]. Furthermore, the adoption of parallel training across multiple GPUs proves to be a practical strategy for reducing the overall training time. Intensive demand of resources is a much harder issue to mitigate, as it depends largely on the complexity of DL models (i.e., architectures) and amount of data used. Nowadays, a multitude of GPU architectures present themselves as viable choices for training DL models. Nevertheless, determining GPU performance, especially in the context of DL, cannot solely rely on published performance data, as prevailing DL models are often trained for benchmarking purposes. It is essential to recognise that the most recently released GPU architecture may not necessarily constitute the optimal choice for certain models [6]. The training performance is contingent upon factors such as GPU architecture,the number of GPUs used, hyperparameters (such as the batch size, which must be tuned according to the available GPU memory), and the specific characteristics inherent to the DL model (such as the number of layers or the activation function) under consideration [7]. It then becomes clear that selecting the most suitable GPU architecture for a specific model significantly expedites the training process [8]. Training DL models can be conducted within a GPU cluster, potentially featuring heterogeneous GPU architectures. This diversity in GPU architectures provides a range of options for DL users. It adds complexity to the training, as these options mainly rely on the users' experience. Even for experienced users, choosing GPU architectures is often a matter of trial and error. Consequently, in a heterogeneous GPU cluster, an effective scheduler must both manage resources within a multi-tenant environment and consider the variable training performance inherent to DL models. Therefore, this paper focuses on designing scheduling policies that take these aspects into account to minimise the time required to complete a set of jobs. With insights into the variables affecting training performance, cutting-edge scheduling techniques can enhance scheduling policies for DL training tasks, especially within a heterogeneous GPU cluster. Firstly, a round-based mechanism enables a scheduler to periodically re-schedule, addressing potential sub-optimal allocations [8]. Secondly, job packing emerges as a valuable strategy for augmenting cluster utilisation. It is particularly effective when dealing with small model training in isolation, as this scenario typically results in the underutilisation of a GPU. Simultaneously, job packing reduces job waiting time [9]. Those approaches provide good performances and can be combined when establishing a scheduling policy. This paper presents a new approach combining the works of [PERSON] et al. [8] and [PERSON] [9] as well as building upon our earlier research [7] to extend the contributions made in the previous study. The key contributions of the paper are as follows:  
* **Implementation of a round-based mechanism:** We implement a round-based mechanism to take advantage of re-scheduling. * **Integration of job packing:** We incorporate the concept of job packing to enhance cluster utilisation and reduce waiting time.