OK, @jnothman and @GaelVaroquaux , reworked as requested.

Somewhat subjective point, but if we want to fail hard on invalid arguments, shouldn't we do it at constructor time and not at fit/transform (i.e., fail hard __and fast__)?

I think I've noticed this behavior on typos in hyperparameters for other estimators as well.