SPARK-1877: ClassNotFoundException when loading RDD with serialized obje...

When I load a RDD that has custom serialized objects, Spark throws ClassNotFoundException. This happens only when Spark is deployed as a standalone cluster, it works fine when Spark is local.

I debugged the issue and I noticed that ObjectInputStream.resolveClass does not use ExecutorURLClassLoader set by SparkSubmit. You have to explicitly set the classloader in SparkContext.objectFile for ObjectInputStream when deserializing objects.