kite-sdk/kite — pull request #120 by gabrielreid (2014-06-25 08:36:21+00:00)
Title: Don't disable CombineFileInputFormat on avro files

The current implementation of DataSourceTarget disables Crunch's use of CombineFileInputFormat for all input types, even though it is still technically compatible with Avro files. In our case, with day-based partitions, we can sometimes run into multiple partitions that contain a small amount of data, and in this case the use of CombineFileInputFormat is very advantageous.

This change re-enables the use of CombineFileInputFormat only for Avro files, while still disabling it for any other type of input dataset.