davies/spark — pull request #2 by JoshRosen (2014-12-16 07:40:08+00:00)
Title: More genericization in ConfigurableCombineFileRecordReader

@davies Opening this as a PR against your PR, since this seemed easier than trying to describe this in a comment.

The main idea here is to remove the last traces of WholeTextFile-specific stuff from the ConfigurableCombineFileRecordReader class so that it just acts as plumbing for piping the `Configurable` configuration to the right place.

This seems clearer to me, since `ConfigurableCombineFileRecordReader` sounds like the name of a generic thing that's not WholeTextFiles-specific, so the hardcoding of some of those values was confusing to me.

Let me know what you think. If this looks good, feel free to merge this into your PR (I've opened this against your PR branch).