Its a NPE. Matei suggested replacing it with a less obtuse error message in https://issues.apache.org/jira/browse/SPARK-1857 . I think this is a better place to put the check since:
1) Other people might try and access the partitioner and then they get sensible error messages
2) Checking for a NPE inside of the lookup() function isn't any more intuitive
3) We already need to add a similar check to context/SparkContext here