A cluster of data servers and a parallel file system are often used to provide high-throughput I/O service to parallel programs running on a compute cluster. To exploit I/O parallelism parallel file systems stripe file data across the data servers. While this practice is effective in serving asynchronous requests, it may break individual program's spatial locality, which can seriously degrade I/O performance when the data servers concurrently serve synchronous requests from multiple I/O-intensive programs. In this paper we propose a scheme, IOrchestrator, to improve I/O performance of multi-node storage systems by orchestrating I/O services among programs when such inter-data-server coordination is dynamically determined to be cost effective. We have implemented IOrchestrator in the PVFS2 parallel file system. Our experiments with representative parallel benchmarks show that IOrchestrator can significantly improve I/O performanceby up to a factor of 2.5-delivered by a cluster of data servers servicing concurrently-running parallel programs. Notably, we have not observed any scenarios in which the use of IOrchestrator causes substantial performance degradation.
Zhang et al REF study the access to disks by multiple applications running in the system by focusing on cases when I/O requests from multiple applications might break the spatial locality of individual programs -a situation that can seriously degrade I/O performance when the data servers concurrently serve synchronous requests from multiple I/Ointensive programs.
IOrchestrator: Improving the Performance of Multi-node I/O Systems via Inter-Server Coordination
2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis
2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis
Computer Science