Added Pig processing and some additional refactoring of the Avro process