Wouldn't it better to fit `base_estimator` on the complete dataset while issuing an `ConvergenceWarning` with an explicit error message rather that failing?

Any opinion @GaelVaroquaux  and others?