apache/spark — pull request #1568 by davies (2014-07-24 06:59:55+00:00)
Title: [SPARK-2652] [PySpark] Turning some default configs for PySpark

Add several default configs for PySpark, related to serialization in JVM.

spark.serializer = org.apache.spark.serializer.KryoSerializer
spark.kryo.referenceTracking = False
spark.serializer.objectStreamReset = 1

This will help to reduce the memory usage during RDD.partitionBy()