I've been thinking about this and I'm not sure I believe adding a config file will make life easier for orgs who want to set up Jenkins for their forks of Shark. Maybe some other users can chime in here.

My thinking is that they would need to modify and commit this file to their fork of the repo so that Jenkins would check it out every time it does a build, and this is different than the pattern we have for the [spark|shark]-env.sh config files and I think that would make using a config file a bit more confusing.

I do definitely agree with you that it should be more obvious that orgs can configure things to work in their private environments via env variables. We can recommend either (a) just export the appropriate env vars in the Jenkins project before calling bin/run-tests-from-scratch and or (b) create and commit (to their private repo or to somewhere they can fetch via wget) a text file containing these and source that file before calling bin/run-tests-from-scratch.

So, based on those thoughts, I think we should:

1) Make sure all variables that orgs might want to use custom values for respect having their default values being overridden when env vars are set. E.g. the following would have to be changed: HIVE_WAREHOUSE, HIVE_DEV_HOME, HIVE_HOME, SPARK_MEM, SPARK_MASTER_MEM, ...

2) Clearly document these variables. This includes (1) in the comment at the top of the script, (2) in the usage text that prints with -h, and (3) by moving them all to the top of the file (and having parsing and checking of them come in a section below where they are all set instead of inline with them as it is now).

What do you think?