linkedin/camus — pull request #19 by mykidong (2013-07-25 05:40:07+00:00)
Title: CamusTrevniJob: Produce Trevni Data instead of Avro From Kafka.

I think, it is nice to produce trevni data from kafka, because trevni data format is column-major which can be read very fast in m/r environments, in particular, some specified column can be read instead of reading all the columns in case of row based data like avro.

com.linkedin.camus.etl.kafka.CamusTrevniJob added: it is based CamusJob to produce the trevni data instead of avro data.

com.linkedin.camus.etl.kafka.mapred.EtlTrevniMultiOutputFormat: it is based EtlMultiOutputFormat. The most part of modification is method, getDataRecordWriter() in which AvroColumnWriter as a trevni data writer is used.

I have tried extending CamusJob and EtlMultiOutputFormat because just small modification was needed, but some codes are private which cannot be accessed.