I won't push further on this, as it's not a blocker for this patch. However, most of the set()s inside Spark are part of SparkContext and SparkEnv initialization code, and thus are not fully at odds with an immutable SparkConf, as long as it was finalized after initialization. In other places, it's used as a hack to pass around information in a global, type-unsafe namespace.

I was not contradicting your initial point about documentation, though a builder pattern implies immutability through code rather than comments.