The scientific community is exploring the suitability of cloud infrastructure to handle High Performance Computing (HPC) applications. The goal of Magellan, a project funded through DOE ASCR, is to investigate the potential role of cloud computing to address the computing needs of the Department of Energy's Office of Science, especially for mid-range computing and data-intensive applications which are not served through existing DOE centers today. Prior work has shown that applications with significant communication or I/O tend to perform poorly in virtualized cloud environments. However, there is a limited understanding of the I/O characteristics in virtualized cloud environments. This paper will present our results in benchmarking the I/O performance over different cloud and HPC platforms to identify the major bottlenecks in existing infrastructure. We compare the I/O performance using IOR benchmark on two cloud platforms -Amazon and Magellan. We analyze the performance of different storage options available, different instance types in multiple availability zones. Finally, we perform large-scale tests in order to analyze the variability in the I/O patterns over time and region. Our results highlight the overhead and variability in I/O performance on both public and private cloud solutions. Our results will help applications decide between the different storage options enabling applications to make effective choices.
Ghoshal et al. REF compared I/O performance on two cloud platforms, Amazon EC2 and Magellan, using the IOR benchmark on the CC1 instance type.
I/O performance of virtualized cloud environments
DataCloud-SC '11
Engineering