apache/spark — pull request #2857 by YanTangZhai (2014-10-20 13:07:25+00:00)
Title: [SPARK-4009][SQL]HiveTableScan should use makeRDDForTable instead of makeRDDForPartitionedTable for partitioned table when partitionPruningPred is None

HiveTableScan should use makeRDDForTable instead of makeRDDForPartitionedTable for partitioned table when partitionPruningPred is None.
If a table has many partitions for example more than 20 thousands while it has a few data for example less than 512MB, some sql querying the table will produce more than 20000 RDDs. The job would submit failed with exception: java stack overflow.