Design and Implementation of the Tianhe2 Data Storage and Management System With the convergence of highperformance computing (HPC), big data and artificial intelligence (AI), the HPC community is pushing for xe2x80x9ctriple usexe2x80x9d systems to expedite scientific discoveries. However, supporting these converged applications on HPC systems presents formidable challenges in terms of storage and data management due to the explosive growth of scientific data and the fundamental differences in IO characteristics among HPC, big data and AI workloads. In this paper, we discuss the driving force behind the converging trend, highlight three data management challenges, and summarize our efforts in addressing these data management challenges on a typical HPC system at the parallel file system, data management middleware, and user application levels. As HPC systems are approaching the border of exascale computing, this paper sheds light on how to enable applicationdriven data management as a preliminary step toward the deep convergence of exascale computing ecosystems, big data, and AI.

High Performance IO For Large Scale Deep Learning. Training deep learning (DL) models on petascale datasets is essential for achieving competitive and stateoftheart performance in applications such as speech, video analytics, and object recognition. However, existing distributed filesystems were not developed for the access patterns and usability requirements of DL jobs. In this paper, we describe AIStore, a highly scalable, easytodeploy storage system, and WebDataset, a standardsbased storage format and library that permits efficient access to very large datasets. We compare system performance experimentally using image classification workloads and storing training data on a variety of backends, including local SSDs, singlenode NFS, and two identical baremetal clusters: HDFS and AIStore.