introduce cruch and hadoop to build deps
Index pipeline
enable snappy compression for pipeline intermediant output
add tests and fix bug for cross project reference
add a tool to dump intermediat MR output
move to hadoop 2.0
Merge branch 'parser-pipeline-0129'
fix log4j conf