## 5 Experimentation Setup
### Experiments  
**Scheduling decision.** The decision variables influencing the training performance, which are GPU architecture, the number of GPUs used in training, and batch size, are defined in Section 3. The adjustment of the batch size is an optional service within a GPU cluster to improve cluster efficiency. We will conduct the experiments by varying these three parameters. We divide the experiments as follows.  
1. **Making a decision for GPU architecture.** The scheduler selects the GPU architecture for a given job to maximise its throughput. We assume that users provide information about the number of GPUs used in training and the batch size.
2. **Making a decision for GPU architecture and number of GPUs used in training.** The scheduler selects GPU architecture and its number in training for a given job to maximise its throughput. In this case, we assume that users configure the batch size.
3. **Making a decision for GPU architecture, number of GPUs used in training, and adjustment of batch size.** In practice, batch size is a user choice, and the scheduler does not modify the user-identified batch size. However, suggesting a batch size along with resource allocation can be an optional service, potentially achieving higher throughput based on the allocated resources. In this experiment, the scheduler selects GPU architecture and its number in training and batch size for a given job to maximise its throughput. The results can be compared with the experiments without changing batch size to evaluate the improvement.  
\begin{table}
\begin{tabular}{l l l} \hline \hline \multicolumn{1}{c}{**Components**} & \multicolumn{1}{c}{**Elements**} \\ \hline
**DL models** & \{1...21\} CNN models & \\  & Their number of layers and model parameters are described as follows: \\ VGG [32] & VGG16: & 16 layers with 138.4 M parameters \\  & VGG19: & 19 layers with 143.7 M parameters \\ ResNet [33] & ResNet50: & 107 layers with 25.6 M parameters \\  & ResNet50V2: & 103 layers with 25.6 M parameters \\  & ResNet101: & 209 layers with 44.7 M parameters \\  & ResNet101V2: & 205 layers with 44.7 M parameters \\  & ResNet152: & 311 layers with 60.4 M parameters \\  & ResNet152V2: & 307 layers with 60.4 M parameters \\ MobileNet [34; 35] & MobileNet: & 55 layers with 4.3 M parameters \\  & MobileNetV2: & 105 layers with 3.5 M parameters \\ DenseNet [36] & DenseNet121: & 242 layers with 8.1 M parameters \\  & DenseNet169: & 338 layers with 14.3 M parameters \\  & DenseNet201: & 402 layers with 20.2 M parameters \\ EfficientNet [37] & EfficientNetB0: & 132 layers with 5.3 M parameters \\  & EfficientNetB1: & 186 layers with 7.9 M parameters \\  & EfficientNetB2: & 186 layers with 9.2 M parameters \\  & EfficientNetB3: & 210 layers with 12.3 M parameters \\  & EfficientNetB4: & 258 layers with 19.5 M parameters \\  & EfficientNetB5: & 312 layers with 30.6 M parameters \\  & EfficientNetB6: & 360 layers with 43.3 M parameters \\  & EfficientNetB7: & 438 layers with 66.7 M parameters \\ \hline
**Batch size** & \{32,64,128,256,512\} & \\ \hline
**GPU architecture** & Nvidia A100 \{1,2\} & \\
**and GPU count** & Nvidia A40 \{1,2\} & \\  & Nvidia RTX1080 Ti \{1,2\} & \\  & Nvidia K40M \{1,2\} & \\  & Nvidia RTX2080 Ti \{1,2,3,4\} & \\  & Nvidia T4 \{1,2,3,4\} & \\ \hline
**Total** & 1680 jobs & \\ \hline \hline \end{tabular}
\end{table}
Table 2: Jobs and GPU architectures used in evaluation.  
**Time per iteration.** The scheduling decisions are re-evaluated periodically with a round duration of 3, 6, 12, and 30 min. The experiments are conducted under two conditions: a scheduling decision that can be changed in the next round and, as a control case, a scheduling decision without any changes. We repeat the experiment ten times for each possible parameter combination and average the results.