apache/incubator-spark — pull request #282 by liyinan926 (2013-12-20 22:28:11+00:00)
Title: Added support for Kerberos-secured HDFS access

This PR added support for Spark to access Kerberos-secured HDFS, which may be useful in certain cluster setup. The basic idea is to schedule a timer task in the Executor constructor that periodically refreshes (kinit) Kerberos credential so the executors can read/write Kerberos-secured HDFS. Kerberos login is through a principal and a keytab file available on each slave node. Three new configuration parameters are introduced: spark.hadoop.security.kerberos to tell whether Kerberos-secure HDFS is used and whether this support should be enabled or not, spark.hadoop.kerberos.keytab to point to the path for the keytab file, and finally spark.hadoop.kerberos.principal to give the principal name.