Research on Improving Reliability, Energy Efficiency and Scalability in Distributed and Parallel File Systems

With the increasing popularity of cloud computing and “Big d ata” applications, current data centers are often required to manage petabytes or exabytes of da ta. To store this huge amount of data, thousands or tens of thousands storage nodes are required at a single site. This imposes three major challenges for storage system designers: (1) Reliability— node failure in these datacenters is a normal occurrence rather than a rare situation. This makes data reliability a great concern [68, 70]. (2) Energy efficiency—a data center can consume up to 100 times more energy than a standard office building. More than 10 % of this energy consumption can be attributed to storage syst ems. Thus, reducing the energy consumption of the storage system is key to reducing the overall consumption of the data center. (3) Scalability—with the continuou sly increasing size of data, maintaining the scalability of the storage systems is essential. That is , the expansion of the storage system should be completed efficiently and without limitations on t he total number of storage nodes or performance. This thesis proposes three ways to improve the above three ke y features for current large-scale storage systems. Firstly, we define the problem of “reverse l ookup”, namely finding the list of objects (blocks) for a failed node. As the first step of failur e recovery, this process is directly related to the recovery/reconstruction time. While existing solutions use metadata traversal or data distribution reversing methods for reverse lookup, which a re either time consuming or expensive, a deterministic block placement can achieve fast and efficie nt reverse lookup. However, the deterministic placement solutions are designed for centralized , small-scale storage architectures such as RAID etc. Due to their lacking of scalability, they canno t be directly applied in large-scale storage systems. In this paper, we propose Group-Shifted De clust ring (G-SD), a deterministic data layout for multi-way replication. G-SD addresses the s calability issue of our previous Shifted Declustering layout and supports fast and efficient reverse lookup. iii Secondly, we define a problem: “how to balance the performanc e, energy, and recovery in degradation mode for an energy efficient storage system? ”. While ext nsive researches have been proposed to tradeoff performance for energy efficiency under normal m ode [23, 46, 15, 56], the system entersdegradation modewhen node failure occurs, in which node reconstruction is in itiated. This very process requires a number of disks to be spun up and requi res a substantial amount of I/O bandwidth, which will not only compromise energy efficiency but also performance. Without considering the I/O bandwidth contention between recovery and performance, we find that the current energy proportional solutions cannot answer this question accurately. This thesis present PERP, a mathematical model to minimize the energy consumption for a storage systems with respect to performance and recovery. PERP answers this problem by prov iding the accurate number of nodes and the assigned recovery bandwidth at each time frame. Thirdly, current distributed file systems such as Google Fil System(GFS) [49] and Hadoop Distributed File System (HDFS) [18], employ a pseudo-random me thod for replica distribution and a centralized lookup table (block map) to record all replica l ocations. This lookup table requires a large amount of memory and consumes a considerable amount of CPU/network resources on the metadata server. With the booming size of “Big Data”, the met ada a server becomes a scalability and performance bottleneck. While current approaches such as HDFS Federation attempt to “horizontally” extend scalability by allowing multiple metada ta servers, we believe a more promising optimization option is to “vertically” scale up each metada ta server. We propose Deister, a novel block management scheme that builds on top of a deterministi c declustering distribution method Intersected Shifted Declustering (ISD). Thus both replica distribution and location lookup can be achieved without a centralized lookup table.