Actually one other thing.  I am just thinking about how we can make it easy for people that are folking Shark to use this script. They might have their own git repositories for Spark and Hive they have to define. And we want to make it easy to merge stuff so the configuration can stay in a separate file.

Can I suggest this small change?

First, source the run-tests-from-scratch-config.sh file for private configurations if the file exists. 

Then, make the Spark git repo + spark branch and Hive git repo + branch into their own variables, and set default values if they are not set already. 

This would make this script very portable, and configurations can be done entirely in the private run-tests-from-scratch-config.sh file.