Actually the way of computing the residuals makes me think that this current implementation of RANSAC only works for regression and not classification. Hence we should drop the classifier / perceptron case for now and make it explicit in the docstring that `base_estimator` should be a regression model or `None`.