Pattern analysis based data management method and memorydisk integrated system for high performance computing Abstract   This research proposes a memorydisk integrated system (MDIS) with pattern analysis based data management method. The proposed system consists of a pattern adaptive prefetcher with DRAMbased dual buffers and a PCMbased persistent memory structure. The PCMbased persistent memory module is exploited as conventional main memory and storage layers simultaneously. In this system, to compensate the PCMs unfavorable characteristics, a pattern analysis based prefetching method by using dual DRAM buffers with a small amount of space are designed as a sort of caching layer. The proposed pattern analysis based data management method monitors the miss rate to determine the prefetch policy, e.g., when and what to prefetch. These schemes enable flexible adaptation to the application execution characteristics showing unpredictable and inconsistent workload patterns. In order to evaluate our system, we implemented a tracedriven simulator system and launched the YCSB on Redis, Apache Storm, Apache Spark, and OpenStack Swift. The experimental result shows that the MDIS with a pattern adaptive prefetcher can reduce the total access time by 13.9% compared to the conventional scheme and thus the overall performance of the memorydisk integrated system improves.

IO Acceleration via MultiTiered Data Buffering and Prefetching Modern HighPerformance Computing (HPC) systems are adding extra layers to the memory and storage hierarchy, named deep memory and storage hierarchy (DMSH), to increase IO performance. New hardware technologies, such as NVMe and SSD, have been introduced in burst buffer installations to reduce the pressure for external storage and boost the burstiness of modern IO systems. The DMSH has demonstrated its strength and potential in practice. However, each layer of DMSH is an independent heterogeneous system and data movement among more layers is significantly more complex even without considering heterogeneity. How to efficiently utilize the DMSH is a subject of research facing the HPC community. Further, accessing data with a highthroughput and lowlatency is more imperative than ever. Data prefetching is a wellknown technique for hiding read latency by requesting data before it is needed to move it from a highlatency medium (e.g., disk) to a lowlatency one (e.g., main memory). However, existing solutions do not consider the new deep memory and storage hierarchy and also suffer from underutilization of prefetching resources and unnecessary evictions. Additionally, existing approaches implement a clientpull model where understanding the applicationxe2x80x99s IO behavior drives prefetching decisions. Moving towards exascale, where machines run multiple applications concurrently by accessing files in a workflow, a more datacentric approach resolves challenges such as cache pollution and redundancy. In this paper, we present the design and implementation of Hermes: a new, heterogeneousaware, multitiered, dynamic, and distributed IO buffering system. Hermes enables, manages, supervises, and, in some sense, extends IO buffering to fully integrate into the DMSH. We introduce three novel data placement policies to efficiently utilize all layers and we present three novel techniques to perform memory, metadata, and communication management in hierarchical buffering systems. Additionally, we demonstrate the benefits of a truly hierarchical data prefetcher that adopts a serverpush approach to data prefetching. Our evaluation shows that, in addition to automatic data movement through the hierarchy, Hermes can significantly accelerate IO and outperforms by more than 2x stateoftheart buffering platforms. Lastly, results show 10%xe2x80x9335% performance gains over existing prefetchers and over 50% when compared to systems with no prefetching.