azkaban/azkaban — pull request #330 by markgrover (2014-10-12 19:54:04+00:00)
Title: Updating the azkaban-web-start.sh to be more robust and fixing some bugs...

....
Old pull request (https://github.com/azkaban/azkaban/pull/318) was mistakenly closed by me. So here it is again.

1. HADOOP_HOME was not being checked correctly. The literal was being checked instead of the variable. There was no check for the directory existing either.
2. It was being assumed that the Hadoop MR1 native bits were under HADOOP_HOME. With Hadoop 2 coming out, MR1 bits are usually picked from a separate location. So, I added a new variable called HADOOP_MR1_HOME which if exists, and has native bits, those native bits would be added to the Java Library Path.
3. Hadoop HDFS home doesn't have to be the same location as Hadoop Home. Many distributions including Apache Bigtop, CDH, etc. put HDFS in a separate location other than HADOOP_HOME. Added support for a new variable called HADOOP_HDFS_HOME, which if it's set and exists, will be used to pick HDFS jars in the classpth.
4. It was also previously assumed that Hadoop configuration always exists under $HADOOP_HOME/conf. That's not always the case. Added support for a new variable called HADOOP_CONF_DIR which can be used to specify the location of hadoop configuration.
5. Added HIVE_CONF_DIR in the same vein, as well.

All of this was done with backwards compatibility in mind, so any existing setup and variables should take precendence and this wouldn't break anything. If existing variables are defined, the script will leverage them.

Example invocation:

HADOOP_HOME=/opt/cloudera/parcels/CDH/lib/hadoop HADOOP_HDFS_HOME=/opt/cloudera/parcels/CDH/lib/hadoop-hdfs HADOOP_CONF_DIR=/etc/hadoop/conf HIVE_HOME=/opt/cloudera/parcels/CDH/lib/hive HADOOP_CONF_DIR=/etc/hadoop/conf bin/start-web.sh