Ah, ok, I understand now.  In that case, to make things simpler, would it possibly make sense to not load the jars to the current thread and only load them for the SparkContext/executors?  Classloader stuff can be confusing to deal with and keeping it as isolated as possible could make things easier for users.  This would also line up a little more with how the MR distributed cache works - jars that get added to it don't become accessible for to driver code.