the point is well taken (which, roughly, is "file i/o in parallel is hard"), but I'd like to push forward with this HDFS stuff with read-only for awhile. Appending to existing files (i.e. write operations) is not supported on all HDFS systems, and I think we cover a lot of uses cases by just being good a reading many large files. postscript: this PR is not good at reading many large files.