kevinweil/elephant-bird — pull request #398 by jcoveney (2014-05-19 21:33:44+00:00)
Title: Add CombineFileInputFormat support (esp. for lzo)

This PR makes it so that users of our InputFormats can use them as CombineFileInputFormats if they so desire. I did not go for full generality, especially given the nastiness of Hadoop configuration. Since for interop with cascading we need things to be mapreduce and mapred, things are fairly EB specific though it would be easy to make them more general, and that's something I'd like to do in a future PR.

This isn't 100% done, but I'd like some eyeballs to validate approach and mainly I need to add things. I'd like to...
* documentation
* get rid of some TODO's (progress, file filtering, that sort of thing)
* clean up the split consolidation logic (might leave this for a separate PR)