jghoman/haivvreo — Issue #10: is data locality working?
Author: koertkuipers
Created: 2011-12-30 21:09:16+00:00
State: open

i have a folder with 24 avro files, yet when i run a hive query over it i get only 4 mappers. any idea why? also the mappers seem to be unaware of data-locality i think, not entirely sure.

the 24 avro files are ach 24Mb in size. is it being smart and grouping them? the 4 resulting lzo compressed text files are 138Mb in size.

when i run a cascading job over the same data i get 24 mappers with data-locality as expected.