bigdatagenomics/adam — pull request #472 by tdanford (2014-11-12 21:05:41+00:00)
Title: Multi loader

[ADAM-463] Added AvroMultiParquetRDD

First, updated the FileLocator to include a childLocators method, and added its implementation in all the child FileLocator implementations.

Next, added an HDFSFileLocator implementation.

Finally, added AvroMultiParquetRDD, which takes a FileLocator which indicates the _parent_ directory (in HDFS, or S3) of a set of parquet files, and which loads _all_ row groups from all the parquet files that are immediate children of that FileLocator.

(Joint work with Carl Yeksigian.)