bixolabs/cascading.avro — pull request #22 by edwardnewett (2013-08-21 20:11:36+00:00)
Title: Only include data from the valid range of a BytesWritable

This issue came up when I noticed that serializing the resulting avro included additional bytes at the end that increase every time the object is deserialized/updated/re-serialized.

I found that BytesWritable automatically resizes its internal buffer capacity to be 50% larger when set() is called, and the method getBytes() returns the entire buffer. From the Javadoc of BytesWritable.getBytes():
/**
* Get the data from the BytesWritable.
* @return The data is only valid between 0 and getLength() - 1.
*/

An example of where a BytesWritable gets set with additional capacity is in
org.apache.hadoop.madred.JobClient$RawSplit (at least in versions 0.20 and 1.03)

This pull request includes two tests that demonstrate the issue and the fix to only include the valid range.