[SPARK-2534] Avoid pulling in the entire RDD in various operators (branch-1.0 backport)

This backports #1450 into branch-1.0.

Author: Reynold Xin <rxin@apache.org>

Closes #1469 from rxin/closure-1.0 and squashes the following commits:

b474a92 [Reynold Xin] [SPARK-2534] Avoid pulling in the entire RDD in various operators