scikit-learn/scikit-learn — pull request #3850 by amueller (2014-11-13 16:16:31+00:00)
Title: MRG Make ParameterSampler sample without replacement

Fixes #3792.
I think this issue came up before so maybe it is worth addressing.
This makes the ParametersSampler sample without replacement if all parameters are lists.
This has a couple of gotchas:

* It changes current behavior, in particular I raise an error if there are less possible settings than ``n_iter``.
* I don't know if there is a way to detect finite support in a scipy distribution, so we can not do it there. That means that ``'bla' : bernoulli(0.5)`` is not equivalent any more to ``'bla': [0, 1]``.

Possibilities to resolve these would be:
* give a warning instead of a value error when ``n_iter`` > ``grid_size`` and produce a smaller grid (or sample with replacement in this case?). Might invalidate the ``__len__`` implementation.
* add a parameter ``with_replacement``, set it to ``None``, and go to a deprecation cycle to set it to ``False``. The parameter would only do something in the case all parameters are lists, though.
* Hack into the scipy dists, check if something is discrete and use lower and upper to get the support. If a user tries to implement his own distribution with finite support, we would still be screwed.