[SPARK-1170] Cleaned up the API; other improvements