to compress or not to compress - compute vs. io tradeoffs for mapreduce energy efficiency
Compression enables us to shift resource bottlenecks between IO and CPU. In modern datacenters, where energy efficiency is a growing concern, the benefits of using compression have not been completely exploited. As MapReduce represents a common computation framework for Internet datacenters, we develop a decision algorithm that helps MapReduce users identify when and where to use compression. For some jobs, using compression gives energy savings of up to 60\%. We believe our findings will provide signficant impact on improving datacenter energy efficiency.
compression mapreduce hadoop