apache/spark — pull request #1890 by GrahamDennis (2014-08-11 11:06:43+00:00)
Title: [SPARK-2878]: Fix custom spark.kryo.registrator

This is a work-in-progress, and I'm looking for feedback on my current approach. My aim here is to add the user jars specified in SparkConf into the Executor processes before they start any tasks, and to add these jars to the class path of all threads.

So far I have just implemented sending the user jars to the Executor processes before registering the executor with the master. Next steps are to add these jars to all threads, and/or share a pool of kryo instances created with a known class loader. This will also need implementing for Mesos & YARN.