I agree this is less than ideal, but:

1) This just failed a job for me, so, I'm all for deferring cleaning up an approach that is going away, but personally we'll run with this patch (or something equivalent) just to avoid "crap, that 3am report run failed, now I have to re-run it", even if the race condition only happens once/month/whatever.

2) The race happens anytime someone does System.getProperties.iterator() and someone else does System.setProperty mid-iteration. We could wrap all of Spark's calls to "System.get/setProperties" with a global lock, but that is a) doing even more work on a system that is going away, and b) only guaranteed to fix callers of System.setProperty in the Spark codebase (technically upstream jars could do a System.setProperty and cause the same issue; granted this is unlikely).

So, totally agree this is less than ideal, but I think it's the least amount of effort to avoid (very rarely, but it happens) blowing up jobs before moving to the better config system.