mesos/spark — pull request #750 by jerryshao (2013-07-30 09:03:19+00:00)
Title: Add Spark multi-user support for standalone mode

This patch add multi-user support for standalone mode.

Currently Spark ExecutorBackend do not distinguish the user who submits app, instead it uses the user who start Spark cluster to communicate with hdfs, this will introduce file permission issue. This patch solves this issue in two aspects:

1. ExecutorBackend use the app's user to access hdfs, this will keep the same file permission with the app's user.
2. For security hdfs, client driver get delegation token and distribute to cluster, ExecutorBackend use delegation token to get service authentication and access hdfs.

This patch does not solve delegation token renew issue when communicate with security hadoop, for long-run apps like Spark Streaming, Shark Server, this will make access failure when delegation token expires.

Any advice about renew mechanism is really appreciated, I will add it in this patch.

Thanks
Jerry