# Log Analysis-Based Resource and Execution Time Improvement in HPC: A Case Study
## 4 Simulation and Results  
In our experiment, we have simulated certain large-scale jobs that were run on Tachyon2. This system has a scheduling formula based on the job size weighting, but the backfilling algorithm is not applied. Figure 6 shows the scenario of the conservative backfilling algorithm applied to Tachyon2. As discussed, the queued job has two properties, which are the resource and time requirements [30]. Here,  
Figure 5: Conservative backfilling. it is assumed that the priority is higher as the number is smaller. If large-scale job #0 is inserted into the queue list, it should wait until it has a suitable resource. The conservative backfilling can execute low priority jobs (box #1-#6) first during the waiting time of job #0. Job #6 has a lower priority than #5, but it runs first because the available resources are acquired first and do not delay predecessor #5. As shown in the graphs of Figures 3 and 4, the simulation is performed by selecting the job that were the main cause of the increase in the number of available nodes due to large-scale operation. This job was a large-scale job that required 2048 nodes (16,384 cores, 8 cores/node), and the job ran after waiting about 40 hours and 16 minutes. During the waiting time, the computing nodes are emptied to match the requested resources of the job, as shown in Figure 7.  
As a result, the latency of the entire subordinated job increases (\(Job_{lat}\)). If a small job with a low priority leapfrogs first for the resources that are emptied during the waiting time, it can use the  
Figure 6: Applying backfilling to available resources before the large-scale job. Figure 7: Increase in available resources for the resource allocation of large jobs. resources efficiently. When a job is submitted to the scheduler, it has a unique job ID, submission time, resource requirements, execution, and queue status, as shown in Figure 8a. The queue status indicates whether the job is currently waiting (_qw_), running (_r_), pending (_h_), or terminating (_e_). This information can be checked with a command such as _qstat_ provided by the scheduler. The simulation procedure is performed as follows:  
1. Extract jobs that do not exceed 2048 nodes between submission time and start time (Waiting time, \(T_{randy}\)) during subordinated jobs, as shown in Figure 8a;
2. Backfilling is performed on the extracted jobs, as shown in Figure 8b. Each timestamp specifies the available idle processors (total number of cores);
3. Resource profile update - Scans the profile (request resource) and finds the first point in the available processor. This operation is performed for the job to be backfilled in 1. Starting from this point, it continues to scan the processor to see if it is available as a profile (request resource) until the job is terminated as expected. Figure 8 shows the updated resources profile. Conservative backfill does not delay the start time of predecessors. Therefore, job 3029155 is excluded because it ends after the start time of the target job. Moreover, job 2019182 is out of resources due to the backfill operation in front. The calculation process is shown in Appendix A.  
Figure 8: Applying backfilling for a large-scale job. Resource Efficiency can be obtained as Equations (3)-(5). The backfilling job has two properties (Resource usage (\(P\)) and Runtime (\(T\))), as shown in Figure 5. The resource efficiency by backfilling scheduling is measured as follows:  
\[\text{Backfilling jobs: }B_{1}(P_{1},T_{1})\text{, }\cdots\text{, }B_{n}(P_{n},T_{n}) \tag{3}\]  
\[\text{Resource Efficiency}=\sum_{i=1}^{n}P_{i}T_{i} \tag{4}\]  
\[ \begin{array}{c}\text{R}(128\text{cores}\times 12\text{hours})\times 7\text{ jobs}+(4,096\text{cores}\times 12\text{hours})\times 2\text{jobs}\\ +(512\text{cores}\ast 12\text{hours})\times 2\text{jobs}\\ =72,192\text{seconds}\ (\text{Reduced overall job execution time})\end{array}  \tag{5}\]  
Backfill scheduling recognizes the status of jobs and resources every scheduling cycle event covered in Section 2.4 and attempts scheduling policy.