[SPARK-2441][SQL] Add more efficient distinct operator.

Author: Michael Armbrust <michael@databricks.com>

Closes #1366 from marmbrus/partialDistinct and squashes the following commits:

12a31ab [Michael Armbrust] Add more efficient distinct operator.

(cherry picked from commit 7e26b57615f6c1d3f9058f9c19c05ec91f017f4c)
Signed-off-by: Reynold Xin <rxin@apache.org>