Cascading/cascading — pull request #3 by ipostelnik (2012-06-07 17:16:35+00:00)
Title: Reduce size of serialized MultiInputSplit object

This patch significantly reduces the size of the serialized MultiInputSplit object in cases where you have lots of input files. In my test with a job that had 10K input files, the JobTracker memory usage barely went up relative to baseline, compared to 12 GB jump with the original code.

I've tested it against 0.20-based hadoop cluster on jobs with both single and multiple sources. I've only tested with file based taps, since that's all have available to me.