apache/spark — pull request #766 by CrazyJvm (2014-05-14 07:47:42+00:00)
Title: default task number misleading in several places

private[streaming] def defaultPartitioner(numPartitions: Int = self.ssc.sc.defaultParallelism){
new HashPartitioner(numPartitions)
}

it represents that the default task number in Spark Streaming relies on the variable defaultParallelism in SparkContext, which is decided by the config property spark.default.parallelism

the property "spark.default.parallelism" refers to https://github.com/apache/spark/pull/389