mesos/spark — pull request #362 by stephenh (2013-01-11 07:17:11+00:00)
Title: Add SparkContext.hadoopConfiguration

The hadoopConfiguration also pulls in any spark.hadoop.* system properties and explicitly checks for the AWS creds for Hadoop's s3/s3n file systems (based on Shark's code).

I've been using this in our cluster to avoid using "s3n://user:pass@..." style URLs, with the AWS creds being appended to the spark-env.sh file, and it's working well.