apache/incubator-spark — pull request #136 by squito (2013-11-04 17:42:38+00:00)
Title: Whitelist+spreadout

This is really two completely independent features, but they touch a lot of the same files so it would be a pain to merge them separately.

1) change the spread-out option "spark.deploy.spreadOut" from per-cluster to per-app. Its a pain to have to take down a cluster, especially if its shared, just to try out different uses of this option. This was discussed a while back on the dev mailing list but I just got around to it. https://groups.google.com/forum/#!searchin/spark-developers/spread$20out/spark-developers/kZjKapuzjT4/Hn-SbXRxzjoJ

2) add a way to whitelist a subset of the workers in the standalone cluster for a specific app. in general, a user would never want to mess with this, they should just be able to let the app get assigned to wherever. But its useful for cluster administration, eg. if you want to benchmark specific nodes. Or a user might even want this if there are some performance differences between their nodes that aren't captured by just cores & memory.