Merge pull request #16 from pwendell/master

Support writing input data to HDFS.
Merge pull request #17 from pwendell/master

Add ability to hash records.
Update spark repo to point to the new main repo (no longer in incubating)
Merge pull request #19 from holdenk/update-spark-perf-to-master

Update spark repo to point to the new main repo (no longer in incubating)
Better support for running with an existing Spark installation.
Temporariliy removing Shark
Switch to using spark-submit.
Changes to streaming tests
More clean-up
More clean-up
Ignore external log4j when running tests