mesos/spark — pull request #420 by stephenh (2013-01-28 06:13:23+00:00)
Title: Fix defaultMinSplits to use max.

I am probably just thinking about this wrong, but this:

math.min(defaultParallelism, 2)

Seems like it would always be 2. When is defaultParallelism going to be 0 or 1? (Perhaps if it's unset.) But AFAIK in standalone mode it's number of cores, e.g. much larger than 2.

Given this hint is passed into Hadoop's InputFormat, wouldn't we want to get back ~number of core splits instead of just 2 splits?