Abstract-Worldwide data centers consume about 300 billion kWh of energy per year, which accounts for 2 % of total electricity use. As MapReduce becomes the mainstream paradigm for data intensive computing in data centers, optimizing MapReduce energy efficiency can greatly mitigate energy requirements and reduce energy bills. Numerous studies have attempted to improve MapReduce energy efficiency, but few have approached this problem from understanding and reducing the energy impact of data movements. As data movements are often performance and energy bottlenecks, we propose a data movement centric approach and present an analysis framework with methods and metrics for evaluating costly built-in MapReduce data move ments. Our experimental investigation leverages the fine-grained performance and power profiling framework eTune and reveals unique system-level and component-level energy characteristics of data movements. It also shows the scalability of energy efficiency with Map Reduce workload and system parameters. These energy characteristics can be exploited in system design and resource allocation to improve data-intensive computing energy efficiency. Index Terms-MapReduce, Data Movement, Energy-Efficient Computing Power and energy consumption are among the top concerns in data center design and operation. Today's data centers usually consume multiple megawatts of power. Worldwide, the annual total energy consumption of data centers is about 300 billion kWh, accounting for 2% of the total electricity use [10]. This massive power/energy demand not only limits the affordability and sustainability of data centers, but also causes considerable environmental concerns and policy re sponses. Improving energy efficiency of data center applica tions can effectively address this problem, but requires us first understanding the power and energy characteristics of data center workloads for guided design. In this paper, we focus on energy efficiency of MapReduce based data intensive workloads. Particularly, we study the power, energy, and energy efficiency of data movements in MapReduce [4] applications. This focus is due to three rea sons. First, data movements consume a significant portion of system energy in data centers by keeping computer servers waiting for data or moving data around [1] . Second, MapRe-978-1-4799-0623-9/13/$31.00 © 2013 IEEE duce has become a major computing paradigm in data centers for processing large scale data sets, thanks to its automatic parallel processing and ultra high scalability. Third, due to technology revolution in data acquisition and storage, data centers have experienced the doubling of data volume every two years [5] . Efficient storing and processing of massive data is not only a basic requirement but also a practical challenge to most data centers. Researchers have attempted several methods to improve energy efficiency of MapReduce data movements. These meth ods include: reducing the volume of data in motion with data compression [3] , increasing data movement speed using high speed interconnects [16] , [17] , applying dynamic voltage and frequency scaling (DVFS) to reduce CPU power consumption during data movements [18], and partitioning data storage into multiple zones based on data access patterns to facilitate different power managements While most of these methods are capable of improving overall energy efficiency of MapReduce workloads, they don't provide details about the power and energy requirements of built-in MapReduce data movements. As shown by the power profile of a MapReduce application in Figure 1 , system power significantly changes with MapReduce execution phases and data movements; each MapReduce data movement incurs distinct power consumption. System hardware and software designs must recognize such differences and use suitable strategies to achieve optimal energy efficiency for each data movement. In this work, we study power and energy characteristics of data movements using first-hand data collected on a physical Hadoop MapReduce compute cluster. We describe an analysis framework to evaluate the energy efficiency of MapReduce data movements, and characterize the power and energy of three major types of MapReduce data movements: HDFS (Hadoop File System) read, HDFS write, and data shuffle. In addition, we discuss various impacting factors of data movement energy efficiency. The remainder of this paper is organized as follows. We present the analysis framework in Section II. We analyze the overall energy profiles of typical data movements in Section III
Wirtz REF proposes a centric data movement approach and present an analytical framework with methods and metrics for evaluating costly built-in data movements in MapReduce.
Power and energy characteristics of MapReduce data movements
2013 International Green Computing Conference Proceedings
2013 International Green Computing Conference Proceedings
Computer Science