(Just a side note, I've proposed some significant refactoring to this file here: https://github.com/pwendell/incubator-spark/pull/1. This comment is still relevant there, just don't spend too much time reviewing this file until that's merged.)