# Log Analysis-Based Resource and Execution Time Improvement in HPC: A Case Study
## 2 System Configuration
### Job Execution Logs  
Tachyon2 stores information about job execution at the start and endpoint. When a job is submitted, it is queued while available resources are ready. When available resources are obtained, the job is assigned to the compute resources. At this time, the user's job description script is backed up, and the library, compiler, and node information required to run the program are stored. At this time, the user's job scripts, libraries and compilers required to run the program, and the allocated resource information are stored [19].  
When the job is completed on the Tachyon2 system, the scheduler (SGE) leaves the job execution information, then regenerates this log and saves it again daily in the format shown in Table 2. This data is used in accounting, statistical extraction, user job tracking, and so on. This log contains the log recording time, unique job ID, job name and date information. In addition, such a log format contains several important pieces of date information that are related to the job. For example, Submit-Date, Start-Date, and End-Date are representing the submission, starting, and ending date information in the format of \(yyyymmddhhmmss\). While such date information is provided in the perspective of jobs, there are pieces of resource utilization perspective information, such as Wait-Time (\(T_{\mathit{ready}}\)) and Execution-Time (\(T_{\mathit{running}}\)). Such time information provides how long the jobs were queued for waiting resources and utilized assigned computing resources. Finally, the log format also provides pieces status information, such as the number of used CPU, Exit-Code, Failed-Code, and the number of threads [6].  
As mentioned earlier, Tachyon2 uses an exclusive policy that executes only one job per node. Even if the number of cores used by a job is 4 (CPUS), it is subtracted after multiplying by 8 (E-CPU), which is the total number of cores installed on the node. Resource usage is calculated for all nodes on which the job has been run. Each of the CPU USAGE (s), MEM USAGE (KB), and MAX VMEM (KB) is expressed as the sum of CPU time, memory usage, and virtual memory usage of all nodes where the job is executed. Normally, jobs in HPC are executed using multiple resources, and the scheduler recognizes the events of each processor at the end of the job and logs the status. Most of the job is normally completed and then terminated, but it can be abnormally terminated for reasons such as forced termination by the user, error in the job submission script, exceeding the wall time limit of the job scheduler, software and hardware problems, etc. At the end of the job, the scheduler leaves each signal code according to the above causes [20]. The code for this cause is logged in the EXIT-CODE(#19) and FAILED (#20) fields. In the case of the OpenMP (Open Multi-Processing) code, the number of threads is stored in the OMP_NUM_THREADS (# 21) field.  
This study confirms that the utilization of idle resources is effectively used when the large-scale job is submitted in Tachyon2. It extracts statistical data and analyzes patterns of large-scale jobs using thousands of cores or more from the job log of Tachyon2 stored in the form of Table 2.