Merge pull request #117 from ehrlichja/develop

Cleanup and clarity
fixing recursion issue when source subdirectory doesn't contain the source subdir
Merge pull request #118 from ehrlichja/develop

reverting switch on string to an if/else for jdk 1.6 compatibility.
fixing topic recursion and hadoop2 dependency
fix for sweeper code
improving sweeper support for large number of input files
Changes to support new partitioning that produces fewer output files
Changes to support new partitioning that produces fewer output files
merged
adding priority queuing for sweeper
adding support for multiple priority levels in the sweeper code
removing unused imports
fixing current topic not being set
Merge pull request #119 from kengoodhope/master

Adding pluggablility to work allocation, along with a new work allocation strategy to group small topics partitions within the same task.
adding option to control output file size
Merge branch 'PartitioningAndSweeperTest'
Add gzip compression support
Looks like the avro-1.7 SchemaParser was also called during this path, and it cannot under the '@' chars in the field names (allowed in avro 1.4).

Use the RelaxedSchemaParser to get around this problem.

The stack trace when the error happened:

2014-11-01 09:14:52,842 WARN [Thread-5] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.avro.SchemaParseException: Illegal initial character: @fields
        at org.apache.avro.Schema.validateName(Schema.java:1079)
        at org.apache.avro.Schema.access$200(Schema.java:79)
        at org.apache.avro.Schema$Field.<init>(Schema.java:372)
        at org.apache.avro.Schema.parse(Schema.java:1215)
        at org.apache.avro.Schema$Parser.parse(Schema.java:965)
        at org.apache.avro.Schema$Parser.parse(Schema.java:953)
        at org.apache.avro.Schema.parse(Schema.java:1005)
        at org.apache.avro.hadoop.io.AvroSerialization.getKeyWriterSchema(AvroSerialization.java:197)
        at org.apache.avro.mapreduce.AvroJob.getMapOutputKeySchema(AvroJob.java:174)
        at com.linkedin.camus.sweeper.mapreduce.AvroKeyMapper.setup(AvroKeyMapper.java:38)
Merge pull request #124 from HenryCaiHaiying/master

Looks like the avro-1.7 SchemaParser was also called during this path, a...
Merge pull request #122 from yagnik/support-gzip

Add gzip compression support