> Not a blocker for this PR, but thinking ahead for ranking problems, is there a
> more robust way to distinguish classification v. regression?

In scikit-learn, it is in general the type of classifier that you use,
and you get catastrophic behavior if you do it wrong.