apache/spark — pull request #3518 by ilganeli (2014-11-30 18:36:20+00:00)
Title: [SPARK-3694] RDD and Task serialization debugging output

Hi all - in addition to what was explicitly requested in the original JIRA, I also added the ability to have a trace of the serialization for RDDs so that you can see which specific dependency is unserializable. For debugging task serialization, I added a debug log output that shows the file and jar dependencies. However, I am unsure whether I can add more functionality there. For the RDD, it is possible to attempt to serialize each dependency in turn, which is why I can identify which component fails. For task debugging, I did not see a straightforward way to do the same thing. If anyone can suggest an approach here, I would be happily to implement it.