apache/spark — pull request #213 by marmbrus (2014-03-24 06:02:09+00:00)
Title: Unify the logic for column pruning, projection, and filtering of table scans.

This removes a bunch of duplicated logic, dead code and casting when planning parquet table scans and hive table scans.

Other changes:
- Fix tests now that we are doing a better job of column pruning (i.e. since pruning predicates are applied before we even start scanning tuples, columns required by these predicates do not need to be included unless they are also included in the final output of this logical plan fragment.
- Add rule to simplify trivial filters. This was required to avoid `WHERE false` from getting pushed into table scans, since `HiveTableScan` (reasonably) refuses to apply partition pruning predicates to non-partitioned tables.