mesos/spark — pull request #611 by squito (2013-05-15 05:57:19+00:00)
Title: Use default classloaders for akka & deserializing task results

This changes spark to use the default class loader, instead of the classloader of the current class, when starting akka & when deserializing task results, as discussed here: https://groups.google.com/forum/?fromgroups=#!topic/spark-developers/SUmLIHpw1x0

The change is very small, but the effects probably aren't obvious. Its needed for when you want to add a jar of user code after the spark context is started with `sc.addJar`. The `Executor`s already updated to use the code of the jar, but before this change, they could send a message back with a class the driver didn't know how to load, which lead to a `ClassNotFoundException`.

The thing is, before the change, it *might* work. You might get the jar in all of the currently active akka threads. And for a lot of tasks, that was good enough. But, on some workloads, akka decides to launch a whole bunch of new threads. Then those threads wont' have a classloader that knows about the new jar.

This change alone isn't a complete solution -- if you call `sc.addJar`, it still doesn't add the jar to the class loader for the driver. You still need to reset your classloader to let you add jars, just like is done in `Executor`s. With this change, at least its possible to do that. Or I could add something similar to the driver itself.

I ran tests with standalone jobs & the shell, in local mode & the cluster, as Matei suggested. But I'd appreciate other ideas on how to test it, particularly given my experience that only some workloads can cause akka to spawn more threads.