apache/spark — pull request #3694 by scwf (2014-12-15 01:46:39+00:00)
Title: [Core] Adding a parallelismRatio to control the partitions num of shuffledRDD

Adding parallelismRatio to control the partitions num of shuffledRDD, the rule is:

Math.max(1, parallelismRatio * number of partitions of the largest upstream RDD)

The ratio is 1.0 by default to make it compatible with the old version. When we have a good experience on it, we can change this.