vinted/camus — pull request #15 by saulius (2014-10-13 08:48:55+00:00)
Title: Sweeper with named outputs

I got excited when I found http://avro.apache.org/docs/1.7.5/api/java/org/apache/avro/mapred/AvroMultipleOutputs.html because this is an answer to our excessive sweeper runs for every portal that sometimes hang the cluster. This change redirects entry to a portal-specific named output which gets written to a `country=${portal}` output subdirectory later. So just a single run is necessary through ETLed dataset to sweep for all portals
t.

The only *caveat* is that I need to write a script to change partitioning schema from `country=${portal}/year=${year}/month=${month}/day=${day}` to `year=${year}/month=${month}/day=${day}/country=${portal}`. But that's manageable.

@ljank @vidma