SWIMProjectUCB/SWIM — pull request #6 by adferguson (2012-06-18 04:45:59+00:00)
Title: Account for overhead from SequenceFile serialization

First, take HDFS block size directly as command line argument to GenerateReplayScript. Then, determine the amount of usable data we can store within a single block when serialized as a SequenceFile. Use this (smaller) number as the size of each input partition. Using this, we report to the user the values to set in randomwriter_conf.xsl

This eliminates the problem of two maps being run for each input partition (the second map was due to the overhead of serializing the input data as a SequenceFile).