Characterizing I/O Optimization Effect Through Holistic Log Data Analysis
Channel: The Virtual Institute for I/O
Date: 26/06/2020

Presented by Yuichi Tsujita (RIKEN)

Recent HPC systems utilize parallel file systems such as GPFS and Lustre to ope with the huge demand of data-intensive applications. Although most of the HPC systems provide performance tuning tools on compute nodes, there is not enough chance to tune I/O activities on parallel file systems including high speed interconnects among compute nodes and file systems. We propose an I/O performance optimization framework using log data of parallel file systems and interconnects in a holistic way for effective use of HPC systems including I/O nodes and parallel file systems. We demonstrate our framework at the K computer with two I/O benchmarks for the original and the enhanced MPI-IO implementations. Its I/O analysis has revealed that I/O performance improvements achieved by the enhanced MPI-IO implementation are due to effective utilization of parallel file systems and interconnects among I/O nodes compared with the original MPI-IO implementation.