Randomized search on hyper parameters.
The search strategy starts evaluating all the candidates with a small amount of resources and iteratively selects the best candidates, using more and more resources.
The candidates are sampled at random from the parameter space and the number of sampled candidates is determined by n_candidates.
Read more in the User guide.
Note
This estimator is still experimental for now: the predictions and the API might change without any deprecation cycle. To use it, you need to explicitly import enable_halving_search_cv:
This is assumed to implement the scikit-learn estimator interface. Either estimator needs to provide a score function, or scoring must be passed.
Dictionary with parameters names (string) as keys and distributions or lists of parameters to try. Distributions must provide a rvs method for sampling (such as those from scipy.stats.distributions). If a list is given, it is sampled uniformly.
The number of candidate parameters to sample, at the first iteration. Using ‘exhaust’ will sample enough candidates so that the last iteration uses as many resources as possible, based on min_resources, max_resources and factor. In this case, min_resources cannot be ‘exhaust’.
The ‘halving’ parameter, which determines the proportion of candidates that are selected for each subsequent iteration. For example, factor=3 means that only one third of the candidates are selected.
Defines the resource that increases with each iteration. By default, the resource is the number of samples. It can also be set to any parameter of the base estimator that accepts positive integer values, e.g. ‘n_iterations’ or ‘n_estimators’ for a gradient boosting estimator. In this case max_resources cannot be ‘auto’ and must be set explicitly.
The maximum number of resources that any candidate is allowed to use for a given iteration. By default, this is set n_samples when resource='n_samples' (default), else an error is raised.
The minimum amount of resource that any candidate is allowed to use for a given iteration. Equivalently, this defines the amount of resources r0 that are allocated for each candidate at the first iteration.
problem
classification problem
Note that the amount of resources used at each iteration is always a multiple of min_resources.
This is only relevant in cases where there isn’t enough resources to reduce the remaining candidates to at most factor after the last iteration. If True, then the search process will ‘replay’ the first iteration for as long as needed until the number of candidates is small enough. This is False by default, which means that the last iteration may evaluate more than factor candidates. See Aggressive elimination of candidates for more details.
Determines the cross-validation splitting strategy. Possible inputs for cv are:
For integer/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
Refer User Guide for the various cross-validation strategies that can be used here.
Note
Due to implementation details, the folds produced by cv must be the same across multiple calls to cv.split(). For built-in scikit-learn iterators, this can be achieved by deactivating shuffling (shuffle=False), or by setting the cv’s random_state parameter to an integer.
A single string (see The scoring parameter: defining model evaluation rules) or a callable (see Defining your scoring strategy from metric functions) to evaluate the predictions on the test set. If None, the estimator’s score method is used.
If True, refit an estimator using the best found parameters on the whole dataset.
The refitted estimator is made available at the best_estimator_ attribute and permits using predict directly on this HalvingRandomSearchCV instance.
Value to assign to the score if an error occurs in estimator fitting. If set to ‘raise’, the error is raised. If a numeric value is given, FitFailedWarning is raised. This parameter does not affect the refit step, which will always raise the error. Default is np.nan
If False, the cv_results_ attribute will not include training scores. Computing training scores is used to get insights on how different parameter settings impact the overfitting/underfitting trade-off. However computing the scores on the training set can be computationally expensive and is not strictly required to select the parameters that yield the best generalization performance.
Pseudo random number generator state used for subsampling the dataset when resources != 'n_samples'. Also used for random uniform sampling from lists of possible values instead of scipy.stats distributions. Pass an int for reproducible output across multiple function calls. See Glossary.
Number of jobs to run in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Controls the verbosity: the higher, the more messages.
The amount of resources used at each iteration.
The number of candidate parameters that were evaluated at each iteration.
The number of candidate parameters that are left after the last iteration. It corresponds to ceil(n_candidates[-1] / factor)
The maximum number of resources that any candidate is allowed to use for a given iteration. Note that since the number of resources used at each iteration must be a multiple of min_resources_, the actual number of resources used at the last iteration may be smaller than max_resources_.
The amount of resources that are allocated for each candidate at the first iteration.
The actual number of iterations that were run. This is equal to n_required_iterations_ if aggressive_elimination is True. Else, this is equal to min(n_possible_iterations_,
n_required_iterations_).
The number of iterations that are possible starting with min_resources_ resources and without exceeding max_resources_.
The number of iterations that are required to end up with less than factor candidates at the last iteration, starting with min_resources_ resources. This will be smaller than n_possible_iterations_ when there isn’t enough resources.
A dict with keys as column headers and values as columns, that can be imported into a pandas DataFrame. It contains many informations for analysing the results of a search. Please refer to the User guide for details.
Estimator that was chosen by the search, i.e. estimator which gave highest score (or smallest loss if specified) on the left out data. Not available if refit=False.
Mean cross-validated score of the best_estimator.
Parameter setting that gave the best results on the hold out data.
The index (of the cv_results_ arrays) which corresponds to the best candidate parameter setting.
The dict at search.cv_results_['params'][search.best_index_] gives the parameter setting for the best model, that gives the highest mean score (search.best_score_).
Scorer function used on the held out data to choose the best parameters for the model.
The number of cross-validation splits (folds/iterations).
Seconds used for refitting the best model on the whole dataset.
This is present only if refit is not False.
See also
Search over a grid of parameters using successive halving.
The parameters selected are those that maximize the score of the held-out data, according to the scoring parameter.
decision_function(X)
Call decision_function on the estimator with the best found parameters.
fit(X[, y, groups])
Run fit with all sets of parameters.
get_params([deep])
Get parameters for this estimator.
inverse_transform(Xt)
Call inverse_transform on the estimator with the best found params.
predict(X)
Call predict on the estimator with the best found parameters.
predict_log_proba(X)
Call predict_log_proba on the estimator with the best found parameters.
predict_proba(X)
Call predict_proba on the estimator with the best found parameters.
score(X[, y])
Returns the score on the given data, if the estimator has been refit.
score_samples(X)
Call score_samples on the estimator with the best found parameters.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Call transform on the estimator with the best found parameters.
Call decision_function on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports decision_function.
Must fulfill the input assumptions of the underlying estimator.
Run fit with all sets of parameters.
Training vector, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Parameters passed to the fit method of the estimator
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Call inverse_transform on the estimator with the best found params.
Only available if the underlying estimator implements inverse_transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.
Call predict on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict.
Must fulfill the input assumptions of the underlying estimator.
Call predict_log_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_log_proba.
Must fulfill the input assumptions of the underlying estimator.
Call predict_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_proba.
Must fulfill the input assumptions of the underlying estimator.
Returns the score on the given data, if the estimator has been refit.
This uses the score defined by scoring where provided, and the best_estimator_.score method otherwise.
Input data, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Call score_samples on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports score_samples.
New in version 0.24.
Data to predict on. Must fulfill input requirements of the underlying estimator.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Call transform on the estimator with the best found parameters.
Only available if the underlying estimator supports transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.
Release Highlights for scikit-learn 0.24
Successive Halving Iterations


Call decision_function on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports decision_function.
Must fulfill the input assumptions of the underlying estimator.


Run fit with all sets of parameters.
Training vector, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Parameters passed to the fit method of the estimator


Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.


Call inverse_transform on the estimator with the best found params.
Only available if the underlying estimator implements inverse_transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.


Call predict on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict.
Must fulfill the input assumptions of the underlying estimator.


Call predict_log_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_log_proba.
Must fulfill the input assumptions of the underlying estimator.


Call predict_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_proba.
Must fulfill the input assumptions of the underlying estimator.


Returns the score on the given data, if the estimator has been refit.
This uses the score defined by scoring where provided, and the best_estimator_.score method otherwise.
Input data, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.


Call score_samples on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports score_samples.
New in version 0.24.
Data to predict on. Must fulfill input requirements of the underlying estimator.


Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.


Call transform on the estimator with the best found parameters.
Only available if the underlying estimator supports transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.


K-Folds cross-validator
Provides train/test indices to split data in train/test sets. Split dataset into k consecutive folds (without shuffling by default).
Each fold is then used once as a validation while the k - 1 remaining folds form the training set.
Read more in the User Guide.
Number of folds. Must be at least 2.
Changed in version 0.22: n_splits default value changed from 3 to 5.
Whether to shuffle the data before splitting into batches. Note that the samples within each split will not be shuffled.
When shuffle is True, random_state affects the ordering of the indices, which controls the randomness of each fold. Otherwise, this parameter has no effect. Pass an int for reproducible output across multiple function calls. See Glossary.
See also
Takes group information into account to avoid building folds with imbalanced class distributions (for binary or multiclass classification tasks).
K-fold iterator variant with non-overlapping groups.
Repeats K-Fold n times.
The first n_samples % n_splits folds have size n_samples // n_splits + 1, other folds have size n_samples // n_splits, where n_samples is the number of samples.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.


K-Folds cross-validator
Provides train/test indices to split data in train/test sets. Split dataset into k consecutive folds (without shuffling by default).
Each fold is then used once as a validation while the k - 1 remaining folds form the training set.
Read more in the User Guide.
Number of folds. Must be at least 2.
Changed in version 0.22: n_splits default value changed from 3 to 5.
Whether to shuffle the data before splitting into batches. Note that the samples within each split will not be shuffled.
When shuffle is True, random_state affects the ordering of the indices, which controls the randomness of each fold. Otherwise, this parameter has no effect. Pass an int for reproducible output across multiple function calls. See Glossary.
See also
Takes group information into account to avoid building folds with imbalanced class distributions (for binary or multiclass classification tasks).
K-fold iterator variant with non-overlapping groups.
Repeats K-Fold n times.
The first n_samples % n_splits folds have size n_samples // n_splits + 1, other folds have size n_samples // n_splits, where n_samples is the number of samples.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.
Feature agglomeration vs. univariate selection
Gradient Boosting Out-of-Bag estimates
Nested versus non-nested cross-validation
Visualizing cross-validation behavior in scikit-learn
Cross-validation on diabetes Dataset Exercise


Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.


Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.


Learning curve.
Determines cross-validated training and test scores for different training set sizes.
A cross-validation generator splits the whole dataset k times in training and test data. Subsets of the training set with varying sizes will be used to train the estimator and a score for each training subset size and the test set will be computed. Afterwards, the scores will be averaged over all k runs for each training subset size.
Read more in the User Guide.
An object of that type which is cloned for each validation.
Training vector, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Relative or absolute numbers of training examples that will be used to generate the learning curve. If the dtype is float, it is regarded as a fraction of the maximum size of the training set (that is determined by the selected validation method), i.e. it has to be within (0, 1]. Otherwise it is interpreted as absolute sizes of the training sets. Note that for classification the number of samples usually have to be big enough to contain at least one sample from each class.
Determines the cross-validation splitting strategy. Possible inputs for cv are:
For int/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
Refer User Guide for the various cross-validation strategies that can be used here.
Changed in version 0.22: cv default value if None changed from 3-fold to 5-fold.
A str (see model evaluation documentation) or a scorer callable object / function with signature scorer(estimator, X, y).
If the estimator supports incremental learning, this will be used to speed up fitting for different training set sizes.
Number of jobs to run in parallel. Training the estimator and computing the score are parallelized over the different training and test sets. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Number of predispatched jobs for parallel execution (default is all). The option can reduce the allocated memory. The str can be an expression like ‘2*n_jobs’.
Controls the verbosity: the higher, the more messages.
Whether to shuffle training data before taking prefixes of it based on``train_sizes``.
Used when shuffle is True. Pass an int for reproducible output across multiple function calls. See Glossary.
Value to assign to the score if an error occurs in estimator fitting. If set to ‘raise’, the error is raised. If a numeric value is given, FitFailedWarning is raised.
New in version 0.20.
Whether to return the fit and score times.
Parameters to pass to the fit method of the estimator.
New in version 0.24.
Numbers of training examples that has been used to generate the learning curve. Note that the number of ticks might be less than n_ticks because duplicate entries will be removed.
Scores on training sets.
Scores on test set.
Times spent for fitting in seconds. Only present if return_times is True.
Times spent for scoring in seconds. Only present if return_times is True.
See examples/model_selection/plot_learning_curve.py


Leave One Group Out cross-validator
Provides train/test indices to split data according to a third-party provided group. This group information can be used to encode arbitrary domain specific stratifications of the samples as integers.
For instance the groups could be the year of collection of the samples and thus allow for cross-validation against time-based splits.
Read more in the User Guide.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Group labels for the samples used while splitting the dataset into train/test set. This ‘groups’ parameter must always be specified to calculate the number of splits, though the other parameters can be omitted.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.


Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Group labels for the samples used while splitting the dataset into train/test set. This ‘groups’ parameter must always be specified to calculate the number of splits, though the other parameters can be omitted.
Returns the number of splitting iterations in the cross-validator.


Leave-One-Out cross-validator
Provides train/test indices to split data in train/test sets. Each sample is used once as a test set (singleton) while the remaining samples form the training set.
Note: LeaveOneOut() is equivalent to KFold(n_splits=n) and LeavePOut(p=1) where n is the number of samples.
Due to the high number of test sets (which is the same as the number of samples) this cross-validation method can be very costly. For large datasets one should favor KFold, ShuffleSplit or StratifiedKFold.
Read more in the User Guide.
See also
For splitting the data according to explicit, domain-specific stratification of the dataset.
K-fold iterator variant with non-overlapping groups.
get_n_splits(X[, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Training data, where n_samples is the number of samples and n_features is the number of features.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.


Returns the number of splitting iterations in the cross-validator
Training data, where n_samples is the number of samples and n_features is the number of features.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.


Leave P Group(s) Out cross-validator
Provides train/test indices to split data according to a third-party provided group. This group information can be used to encode arbitrary domain specific stratifications of the samples as integers.
For instance the groups could be the year of collection of the samples and thus allow for cross-validation against time-based splits.
The difference between LeavePGroupsOut and LeaveOneGroupOut is that the former builds the test sets with all the samples assigned to p different values of the groups while the latter uses samples all assigned the same groups.
Read more in the User Guide.
Number of groups (p) to leave out in the test split.
See also
K-fold iterator variant with non-overlapping groups.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Group labels for the samples used while splitting the dataset into train/test set. This ‘groups’ parameter must always be specified to calculate the number of splits, though the other parameters can be omitted.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.


Leave-P-Out cross-validator
Provides train/test indices to split data in train/test sets. This results in testing on all distinct samples of size p, while the remaining n - p samples form the training set in each iteration.
Note: LeavePOut(p) is NOT equivalent to KFold(n_splits=n_samples // p) which creates non-overlapping test sets.
Due to the high number of iterations which grows combinatorically with the number of samples this cross-validation method can be very costly. For large datasets one should favor KFold, StratifiedKFold or ShuffleSplit.
Read more in the User Guide.
Size of the test sets. Must be strictly less than the number of samples.
get_n_splits(X[, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Training data, where n_samples is the number of samples and n_features is the number of features.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.


Returns the number of splitting iterations in the cross-validator
Training data, where n_samples is the number of samples and n_features is the number of features.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.


Grid of parameters with a discrete number of values for each.
Can be used to iterate over parameter value combinations with the Python built-in function iter. The order of the generated parameter combinations is deterministic.
Read more in the User Guide.
The parameter grid to explore, as a dictionary mapping estimator parameters to sequences of allowed values.
An empty dict signifies default parameters.
A sequence of dicts signifies a sequence of grids to search, and is useful to avoid exploring parameter combinations that make no sense or have no effect. See the examples below.
See also
Uses ParameterGrid to perform a full parallelized parameter search.


Generator on parameters sampled from given distributions.
Non-deterministic iterable over random candidate combinations for hyper- parameter search. If all parameters are presented as a list, sampling without replacement is performed. If at least one parameter is given as a distribution, sampling with replacement is used. It is highly recommended to use continuous distributions for continuous parameters.
Read more in the User Guide.
Dictionary with parameters names (str) as keys and distributions or lists of parameters to try. Distributions must provide a rvs method for sampling (such as those from scipy.stats.distributions). If a list is given, it is sampled uniformly. If a list of dicts is given, first a dict is sampled uniformly, and then a parameter is sampled using that dict as above.
Number of parameter settings that are produced.
Pseudo random number generator state used for random uniform sampling from lists of possible values instead of scipy.stats distributions. Pass an int for reproducible output across multiple function calls. See Glossary.
Yields dictionaries mapping each estimator parameter to as sampled value.


Evaluate the significance of a cross-validated score with permutations
Permutes targets to generate ‘randomized data’ and compute the empirical p-value against the null hypothesis that features and targets are independent.
The p-value represents the fraction of randomized data sets where the estimator performed as well or better than in the original data. A small p-value suggests that there is a real dependency between features and targets which has been used by the estimator to give good predictions. A large p-value may be due to lack of real dependency between features and targets or the estimator was not able to use the dependency to give good predictions.
Read more in the User Guide.
The object to use to fit the data.
The data to fit.
The target variable to try to predict in the case of supervised learning.
Labels to constrain permutation within groups, i.e. y values are permuted among samples with the same group identifier. When not specified, y values are permuted among all samples.
When a grouped cross-validator is used, the group labels are also passed on to the split method of the cross-validator. The cross-validator uses them for grouping the samples while splitting the dataset into train/test set.
A single str (see The scoring parameter: defining model evaluation rules) or a callable (see Defining your scoring strategy from metric functions) to evaluate the predictions on the test set.
If None the estimator’s score method is used.
Determines the cross-validation splitting strategy. Possible inputs for cv are:
For int/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
Refer User Guide for the various cross-validation strategies that can be used here.
Changed in version 0.22: cv default value if None changed from 3-fold to 5-fold.
Number of times to permute y.
Number of jobs to run in parallel. Training the estimator and computing the cross-validated score are parallelized over the permutations. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Pass an int for reproducible output for permutation of y values among samples. See Glossary.
The verbosity level.
Parameters to pass to the fit method of the estimator.
New in version 0.24.
The true score without permuting targets.
The scores obtained for each permutations.
The p-value, which approximates the probability that the score would be obtained by chance. This is calculated as:
(C + 1) / (n_permutations + 1)
Where C is the number of permutations whose score >= the true score.
The best possible p-value is 1/(n_permutations + 1), the worst is 1.0.
This function implements Test 1 in:
Ojala and Garriga. Permutation Tests for Studying Classifier Performance. The Journal of Machine Learning Research (2010) vol. 11


Predefined split cross-validator
Provides train/test indices to split data into train/test sets using a predefined scheme specified by the user with the test_fold parameter.
Read more in the User Guide.
New in version 0.16.
The entry test_fold[i] represents the index of the test set that sample i belongs to. It is possible to exclude sample i from any test set (i.e. include sample i in every training set) by setting test_fold[i] equal to -1.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split([X, y, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
The training set indices for that split.
The testing set indices for that split.


Generate indices to split data into training and test set.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
The training set indices for that split.
The testing set indices for that split.


Randomized search on hyper parameters.
RandomizedSearchCV implements a “fit” and a “score” method. It also implements “score_samples”, “predict”, “predict_proba”, “decision_function”, “transform” and “inverse_transform” if they are implemented in the estimator used.
The parameters of the estimator used to apply these methods are optimized by cross-validated search over parameter settings.
In contrast to GridSearchCV, not all parameter values are tried out, but rather a fixed number of parameter settings is sampled from the specified distributions. The number of parameter settings that are tried is given by n_iter.
If all parameters are presented as a list, sampling without replacement is performed. If at least one parameter is given as a distribution, sampling with replacement is used. It is highly recommended to use continuous distributions for continuous parameters.
Read more in the User Guide.
New in version 0.14.
A object of that type is instantiated for each grid point. This is assumed to implement the scikit-learn estimator interface. Either estimator needs to provide a score function, or scoring must be passed.
Dictionary with parameters names (str) as keys and distributions or lists of parameters to try. Distributions must provide a rvs method for sampling (such as those from scipy.stats.distributions). If a list is given, it is sampled uniformly. If a list of dicts is given, first a dict is sampled uniformly, and then a parameter is sampled using that dict as above.
Number of parameter settings that are sampled. n_iter trades off runtime vs quality of the solution.
Strategy to evaluate the performance of the cross-validated model on the test set.
If scoring represents a single score, one can use:
If scoring reprents multiple scores, one can use:
See Specifying multiple metrics for evaluation for an example.
If None, the estimator’s score method is used.
Number of jobs to run in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Changed in version v0.20: n_jobs default changed from 1 to None
Controls the number of jobs that get dispatched during parallel execution. Reducing this number can be useful to avoid an explosion of memory consumption when more jobs get dispatched than CPUs can process. This parameter can be:
Determines the cross-validation splitting strategy. Possible inputs for cv are:
For integer/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
Refer User Guide for the various cross-validation strategies that can be used here.
Changed in version 0.22: cv default value if None changed from 3-fold to 5-fold.
Refit an estimator using the best found parameters on the whole dataset.
For multiple metric evaluation, this needs to be a str denoting the scorer that would be used to find the best parameters for refitting the estimator at the end.
Where there are considerations other than maximum score in choosing a best estimator, refit can be set to a function which returns the selected best_index_ given the cv_results. In that case, the best_estimator_ and best_params_ will be set according to the returned best_index_ while the best_score_ attribute will not be available.
The refitted estimator is made available at the best_estimator_ attribute and permits using predict directly on this RandomizedSearchCV instance.
Also for multiple metric evaluation, the attributes best_index_, best_score_ and best_params_ will only be available if refit is set and all of them will be determined w.r.t this specific scorer.
See scoring parameter to know more about multiple metric evaluation.
Changed in version 0.20: Support for callable added.
Controls the verbosity: the higher, the more messages.
Pseudo random number generator state used for random uniform sampling from lists of possible values instead of scipy.stats distributions. Pass an int for reproducible output across multiple function calls. See Glossary.
Value to assign to the score if an error occurs in estimator fitting. If set to ‘raise’, the error is raised. If a numeric value is given, FitFailedWarning is raised. This parameter does not affect the refit step, which will always raise the error.
If False, the cv_results_ attribute will not include training scores. Computing training scores is used to get insights on how different parameter settings impact the overfitting/underfitting trade-off. However computing the scores on the training set can be computationally expensive and is not strictly required to select the parameters that yield the best generalization performance.
New in version 0.19.
Changed in version 0.21: Default value was changed from True to False
A dict with keys as column headers and values as columns, that can be imported into a pandas DataFrame.
For instance the below given table
param_kernel
param_gamma
split0_test_score
…
rank_test_score
‘rbf’
0.1
0.80
…
1
‘rbf’
0.2
0.84
…
3
‘rbf’
0.3
0.70
…
2
will be represented by a cv_results_ dict of:
NOTE
The key 'params' is used to store a list of parameter settings dicts for all the parameter candidates.
The mean_fit_time, std_fit_time, mean_score_time and std_score_time are all in seconds.
For multi-metric evaluation, the scores for all the scorers are available in the cv_results_ dict at the keys ending with that scorer’s name ('_<scorer_name>') instead of '_score' shown above. (‘split0_test_precision’, ‘mean_train_precision’ etc.)
Estimator that was chosen by the search, i.e. estimator which gave highest score (or smallest loss if specified) on the left out data. Not available if refit=False.
For multi-metric evaluation, this attribute is present only if refit is specified.
See refit parameter for more information on allowed values.
Mean cross-validated score of the best_estimator.
For multi-metric evaluation, this is not available if refit is False. See refit parameter for more information.
This attribute is not available if refit is a function.
Parameter setting that gave the best results on the hold out data.
For multi-metric evaluation, this is not available if refit is False. See refit parameter for more information.
The index (of the cv_results_ arrays) which corresponds to the best candidate parameter setting.
The dict at search.cv_results_['params'][search.best_index_] gives the parameter setting for the best model, that gives the highest mean score (search.best_score_).
For multi-metric evaluation, this is not available if refit is False. See refit parameter for more information.
Scorer function used on the held out data to choose the best parameters for the model.
For multi-metric evaluation, this attribute holds the validated scoring dict which maps the scorer key to the scorer callable.
The number of cross-validation splits (folds/iterations).
Seconds used for refitting the best model on the whole dataset.
This is present only if refit is not False.
New in version 0.20.
Whether or not the scorers compute several metrics.
See also
Does exhaustive search over a grid of parameters.
A generator over parameter settings, constructed from param_distributions.
The parameters selected are those that maximize the score of the held-out data, according to the scoring parameter.
If n_jobs was set to a value higher than one, the data is copied for each parameter setting(and not n_jobs times). This is done for efficiency reasons if individual jobs take very little time, but may raise errors if the dataset is large and not enough memory is available. A workaround in this case is to set pre_dispatch. Then, the memory is copied only pre_dispatch many times. A reasonable value for pre_dispatch is 2 *
n_jobs.
decision_function(X)
Call decision_function on the estimator with the best found parameters.
fit(X[, y, groups])
Run fit with all sets of parameters.
get_params([deep])
Get parameters for this estimator.
inverse_transform(Xt)
Call inverse_transform on the estimator with the best found params.
predict(X)
Call predict on the estimator with the best found parameters.
predict_log_proba(X)
Call predict_log_proba on the estimator with the best found parameters.
predict_proba(X)
Call predict_proba on the estimator with the best found parameters.
score(X[, y])
Returns the score on the given data, if the estimator has been refit.
score_samples(X)
Call score_samples on the estimator with the best found parameters.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Call transform on the estimator with the best found parameters.
Call decision_function on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports decision_function.
Must fulfill the input assumptions of the underlying estimator.
Run fit with all sets of parameters.
Training vector, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Parameters passed to the fit method of the estimator
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Call inverse_transform on the estimator with the best found params.
Only available if the underlying estimator implements inverse_transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.
Call predict on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict.
Must fulfill the input assumptions of the underlying estimator.
Call predict_log_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_log_proba.
Must fulfill the input assumptions of the underlying estimator.
Call predict_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_proba.
Must fulfill the input assumptions of the underlying estimator.
Returns the score on the given data, if the estimator has been refit.
This uses the score defined by scoring where provided, and the best_estimator_.score method otherwise.
Input data, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Call score_samples on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports score_samples.
New in version 0.24.
Data to predict on. Must fulfill input requirements of the underlying estimator.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Call transform on the estimator with the best found parameters.
Only available if the underlying estimator supports transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.


Randomized search on hyper parameters.
RandomizedSearchCV implements a “fit” and a “score” method. It also implements “score_samples”, “predict”, “predict_proba”, “decision_function”, “transform” and “inverse_transform” if they are implemented in the estimator used.
The parameters of the estimator used to apply these methods are optimized by cross-validated search over parameter settings.
In contrast to GridSearchCV, not all parameter values are tried out, but rather a fixed number of parameter settings is sampled from the specified distributions. The number of parameter settings that are tried is given by n_iter.
If all parameters are presented as a list, sampling without replacement is performed. If at least one parameter is given as a distribution, sampling with replacement is used. It is highly recommended to use continuous distributions for continuous parameters.
Read more in the User Guide.
New in version 0.14.
A object of that type is instantiated for each grid point. This is assumed to implement the scikit-learn estimator interface. Either estimator needs to provide a score function, or scoring must be passed.
Dictionary with parameters names (str) as keys and distributions or lists of parameters to try. Distributions must provide a rvs method for sampling (such as those from scipy.stats.distributions). If a list is given, it is sampled uniformly. If a list of dicts is given, first a dict is sampled uniformly, and then a parameter is sampled using that dict as above.
Number of parameter settings that are sampled. n_iter trades off runtime vs quality of the solution.
Strategy to evaluate the performance of the cross-validated model on the test set.
If scoring represents a single score, one can use:
If scoring reprents multiple scores, one can use:
See Specifying multiple metrics for evaluation for an example.
If None, the estimator’s score method is used.
Number of jobs to run in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Changed in version v0.20: n_jobs default changed from 1 to None
Controls the number of jobs that get dispatched during parallel execution. Reducing this number can be useful to avoid an explosion of memory consumption when more jobs get dispatched than CPUs can process. This parameter can be:
Determines the cross-validation splitting strategy. Possible inputs for cv are:
For integer/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
Refer User Guide for the various cross-validation strategies that can be used here.
Changed in version 0.22: cv default value if None changed from 3-fold to 5-fold.
Refit an estimator using the best found parameters on the whole dataset.
For multiple metric evaluation, this needs to be a str denoting the scorer that would be used to find the best parameters for refitting the estimator at the end.
Where there are considerations other than maximum score in choosing a best estimator, refit can be set to a function which returns the selected best_index_ given the cv_results. In that case, the best_estimator_ and best_params_ will be set according to the returned best_index_ while the best_score_ attribute will not be available.
The refitted estimator is made available at the best_estimator_ attribute and permits using predict directly on this RandomizedSearchCV instance.
Also for multiple metric evaluation, the attributes best_index_, best_score_ and best_params_ will only be available if refit is set and all of them will be determined w.r.t this specific scorer.
See scoring parameter to know more about multiple metric evaluation.
Changed in version 0.20: Support for callable added.
Controls the verbosity: the higher, the more messages.
Pseudo random number generator state used for random uniform sampling from lists of possible values instead of scipy.stats distributions. Pass an int for reproducible output across multiple function calls. See Glossary.
Value to assign to the score if an error occurs in estimator fitting. If set to ‘raise’, the error is raised. If a numeric value is given, FitFailedWarning is raised. This parameter does not affect the refit step, which will always raise the error.
If False, the cv_results_ attribute will not include training scores. Computing training scores is used to get insights on how different parameter settings impact the overfitting/underfitting trade-off. However computing the scores on the training set can be computationally expensive and is not strictly required to select the parameters that yield the best generalization performance.
New in version 0.19.
Changed in version 0.21: Default value was changed from True to False
A dict with keys as column headers and values as columns, that can be imported into a pandas DataFrame.
For instance the below given table
param_kernel
param_gamma
split0_test_score
…
rank_test_score
‘rbf’
0.1
0.80
…
1
‘rbf’
0.2
0.84
…
3
‘rbf’
0.3
0.70
…
2
will be represented by a cv_results_ dict of:
NOTE
The key 'params' is used to store a list of parameter settings dicts for all the parameter candidates.
The mean_fit_time, std_fit_time, mean_score_time and std_score_time are all in seconds.
For multi-metric evaluation, the scores for all the scorers are available in the cv_results_ dict at the keys ending with that scorer’s name ('_<scorer_name>') instead of '_score' shown above. (‘split0_test_precision’, ‘mean_train_precision’ etc.)
Estimator that was chosen by the search, i.e. estimator which gave highest score (or smallest loss if specified) on the left out data. Not available if refit=False.
For multi-metric evaluation, this attribute is present only if refit is specified.
See refit parameter for more information on allowed values.
Mean cross-validated score of the best_estimator.
For multi-metric evaluation, this is not available if refit is False. See refit parameter for more information.
This attribute is not available if refit is a function.
Parameter setting that gave the best results on the hold out data.
For multi-metric evaluation, this is not available if refit is False. See refit parameter for more information.
The index (of the cv_results_ arrays) which corresponds to the best candidate parameter setting.
The dict at search.cv_results_['params'][search.best_index_] gives the parameter setting for the best model, that gives the highest mean score (search.best_score_).
For multi-metric evaluation, this is not available if refit is False. See refit parameter for more information.
Scorer function used on the held out data to choose the best parameters for the model.
For multi-metric evaluation, this attribute holds the validated scoring dict which maps the scorer key to the scorer callable.
The number of cross-validation splits (folds/iterations).
Seconds used for refitting the best model on the whole dataset.
This is present only if refit is not False.
New in version 0.20.
Whether or not the scorers compute several metrics.
See also
Does exhaustive search over a grid of parameters.
A generator over parameter settings, constructed from param_distributions.
The parameters selected are those that maximize the score of the held-out data, according to the scoring parameter.
If n_jobs was set to a value higher than one, the data is copied for each parameter setting(and not n_jobs times). This is done for efficiency reasons if individual jobs take very little time, but may raise errors if the dataset is large and not enough memory is available. A workaround in this case is to set pre_dispatch. Then, the memory is copied only pre_dispatch many times. A reasonable value for pre_dispatch is 2 *
n_jobs.
decision_function(X)
Call decision_function on the estimator with the best found parameters.
fit(X[, y, groups])
Run fit with all sets of parameters.
get_params([deep])
Get parameters for this estimator.
inverse_transform(Xt)
Call inverse_transform on the estimator with the best found params.
predict(X)
Call predict on the estimator with the best found parameters.
predict_log_proba(X)
Call predict_log_proba on the estimator with the best found parameters.
predict_proba(X)
Call predict_proba on the estimator with the best found parameters.
score(X[, y])
Returns the score on the given data, if the estimator has been refit.
score_samples(X)
Call score_samples on the estimator with the best found parameters.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Call transform on the estimator with the best found parameters.
Call decision_function on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports decision_function.
Must fulfill the input assumptions of the underlying estimator.
Run fit with all sets of parameters.
Training vector, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Parameters passed to the fit method of the estimator
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Call inverse_transform on the estimator with the best found params.
Only available if the underlying estimator implements inverse_transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.
Call predict on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict.
Must fulfill the input assumptions of the underlying estimator.
Call predict_log_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_log_proba.
Must fulfill the input assumptions of the underlying estimator.
Call predict_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_proba.
Must fulfill the input assumptions of the underlying estimator.
Returns the score on the given data, if the estimator has been refit.
This uses the score defined by scoring where provided, and the best_estimator_.score method otherwise.
Input data, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Call score_samples on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports score_samples.
New in version 0.24.
Data to predict on. Must fulfill input requirements of the underlying estimator.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Call transform on the estimator with the best found parameters.
Only available if the underlying estimator supports transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.
Release Highlights for scikit-learn 0.24
Comparing randomized search and grid search for hyperparameter estimation


Call decision_function on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports decision_function.
Must fulfill the input assumptions of the underlying estimator.


Run fit with all sets of parameters.
Training vector, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Parameters passed to the fit method of the estimator


Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.


Call inverse_transform on the estimator with the best found params.
Only available if the underlying estimator implements inverse_transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.


Call predict on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict.
Must fulfill the input assumptions of the underlying estimator.


Call predict_log_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_log_proba.
Must fulfill the input assumptions of the underlying estimator.


Call predict_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_proba.
Must fulfill the input assumptions of the underlying estimator.


Returns the score on the given data, if the estimator has been refit.
This uses the score defined by scoring where provided, and the best_estimator_.score method otherwise.
Input data, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.


Call score_samples on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports score_samples.
New in version 0.24.
Data to predict on. Must fulfill input requirements of the underlying estimator.


Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.


Call transform on the estimator with the best found parameters.
Only available if the underlying estimator supports transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.


Repeated K-Fold cross validator.
Repeats K-Fold n times with different randomization in each repetition.
Read more in the User Guide.
Number of folds. Must be at least 2.
Number of times cross-validator needs to be repeated.
Controls the randomness of each repeated cross-validation instance. Pass an int for reproducible output across multiple function calls. See Glossary.
See also
Repeats Stratified K-Fold n times.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generates indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility. np.zeros(n_samples) may be used as a placeholder.
Always ignored, exists for compatibility. np.zeros(n_samples) may be used as a placeholder.
Group labels for the samples used while splitting the dataset into train/test set.
Returns the number of splitting iterations in the cross-validator.
Generates indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.


Repeated K-Fold cross validator.
Repeats K-Fold n times with different randomization in each repetition.
Read more in the User Guide.
Number of folds. Must be at least 2.
Number of times cross-validator needs to be repeated.
Controls the randomness of each repeated cross-validation instance. Pass an int for reproducible output across multiple function calls. See Glossary.
See also
Repeats Stratified K-Fold n times.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generates indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility. np.zeros(n_samples) may be used as a placeholder.
Always ignored, exists for compatibility. np.zeros(n_samples) may be used as a placeholder.
Group labels for the samples used while splitting the dataset into train/test set.
Returns the number of splitting iterations in the cross-validator.
Generates indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.
Common pitfalls in interpretation of coefficients of linear models


Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility. np.zeros(n_samples) may be used as a placeholder.
Always ignored, exists for compatibility. np.zeros(n_samples) may be used as a placeholder.
Group labels for the samples used while splitting the dataset into train/test set.
Returns the number of splitting iterations in the cross-validator.


Generates indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.


Repeated Stratified K-Fold cross validator.
Repeats Stratified K-Fold n times with different randomization in each repetition.
Read more in the User Guide.
Number of folds. Must be at least 2.
Number of times cross-validator needs to be repeated.
Controls the generation of the random states for each repetition. Pass an int for reproducible output across multiple function calls. See Glossary.
See also
Repeats K-Fold n times.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generates indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility. np.zeros(n_samples) may be used as a placeholder.
Always ignored, exists for compatibility. np.zeros(n_samples) may be used as a placeholder.
Group labels for the samples used while splitting the dataset into train/test set.
Returns the number of splitting iterations in the cross-validator.
Generates indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.


Repeated Stratified K-Fold cross validator.
Repeats Stratified K-Fold n times with different randomization in each repetition.
Read more in the User Guide.
Number of folds. Must be at least 2.
Number of times cross-validator needs to be repeated.
Controls the generation of the random states for each repetition. Pass an int for reproducible output across multiple function calls. See Glossary.
See also
Repeats K-Fold n times.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generates indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility. np.zeros(n_samples) may be used as a placeholder.
Always ignored, exists for compatibility. np.zeros(n_samples) may be used as a placeholder.
Group labels for the samples used while splitting the dataset into train/test set.
Returns the number of splitting iterations in the cross-validator.
Generates indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.
Statistical comparison of models using grid search


Random permutation cross-validator
Yields indices to split data into training and test sets.
Note: contrary to other cross-validation strategies, random splits do not guarantee that all folds will be different, although this is still very likely for sizeable datasets.
Read more in the User Guide.
Number of re-shuffling & splitting iterations.
If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the test split. If int, represents the absolute number of test samples. If None, the value is set to the complement of the train size. If train_size is also None, it will be set to 0.1.
If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the train split. If int, represents the absolute number of train samples. If None, the value is automatically set to the complement of the test size.
Controls the randomness of the training and testing indices produced. Pass an int for reproducible output across multiple function calls. See Glossary.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.


Random permutation cross-validator
Yields indices to split data into training and test sets.
Note: contrary to other cross-validation strategies, random splits do not guarantee that all folds will be different, although this is still very likely for sizeable datasets.
Read more in the User Guide.
Number of re-shuffling & splitting iterations.
If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the test split. If int, represents the absolute number of test samples. If None, the value is set to the complement of the train size. If train_size is also None, it will be set to 0.1.
If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the train split. If int, represents the absolute number of train samples. If None, the value is automatically set to the complement of the test size.
Controls the randomness of the training and testing indices produced. Pass an int for reproducible output across multiple function calls. See Glossary.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.
Visualizing cross-validation behavior in scikit-learn
Plotting Learning Curves
Scaling the regularization parameter for SVCs


Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.


Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.


Stratified K-Folds cross-validator.
Provides train/test indices to split data in train/test sets.
This cross-validation object is a variation of KFold that returns stratified folds. The folds are made by preserving the percentage of samples for each class.
Read more in the User Guide.
Number of folds. Must be at least 2.
Changed in version 0.22: n_splits default value changed from 3 to 5.
Whether to shuffle each class’s samples before splitting into batches. Note that the samples within each split will not be shuffled.
When shuffle is True, random_state affects the ordering of the indices, which controls the randomness of each fold for each class. Otherwise, leave random_state as None. Pass an int for reproducible output across multiple function calls. See Glossary.
See also
Repeats Stratified K-Fold n times.
The implementation is designed to:
Changed in version 0.22: The previous implementation did not follow the last constraint.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X, y[, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
Note that providing y is sufficient to generate the splits and hence np.zeros(n_samples) may be used as a placeholder for X instead of actual training data.
The target variable for supervised learning problems. Stratification is done based on the y labels.
Always ignored, exists for compatibility.
The training set indices for that split.
The testing set indices for that split.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.


Stratified K-Folds cross-validator.
Provides train/test indices to split data in train/test sets.
This cross-validation object is a variation of KFold that returns stratified folds. The folds are made by preserving the percentage of samples for each class.
Read more in the User Guide.
Number of folds. Must be at least 2.
Changed in version 0.22: n_splits default value changed from 3 to 5.
Whether to shuffle each class’s samples before splitting into batches. Note that the samples within each split will not be shuffled.
When shuffle is True, random_state affects the ordering of the indices, which controls the randomness of each fold for each class. Otherwise, leave random_state as None. Pass an int for reproducible output across multiple function calls. See Glossary.
See also
Repeats Stratified K-Fold n times.
The implementation is designed to:
Changed in version 0.22: The previous implementation did not follow the last constraint.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X, y[, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
Note that providing y is sufficient to generate the splits and hence np.zeros(n_samples) may be used as a placeholder for X instead of actual training data.
The target variable for supervised learning problems. Stratification is done based on the y labels.
Always ignored, exists for compatibility.
The training set indices for that split.
The testing set indices for that split.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.
Recursive feature elimination with cross-validation
Test with permutations the significance of a classification score
GMM covariances
Receiver Operating Characteristic (ROC) with cross validation
Visualizing cross-validation behavior in scikit-learn
Effect of varying threshold for self-training


Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
Note that providing y is sufficient to generate the splits and hence np.zeros(n_samples) may be used as a placeholder for X instead of actual training data.
The target variable for supervised learning problems. Stratification is done based on the y labels.
Always ignored, exists for compatibility.
The training set indices for that split.
The testing set indices for that split.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.


Stratified ShuffleSplit cross-validator
Provides train/test indices to split data in train/test sets.
This cross-validation object is a merge of StratifiedKFold and ShuffleSplit, which returns stratified randomized folds. The folds are made by preserving the percentage of samples for each class.
Note: like the ShuffleSplit strategy, stratified random splits do not guarantee that all folds will be different, although this is still very likely for sizeable datasets.
Read more in the User Guide.
Number of re-shuffling & splitting iterations.
If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the test split. If int, represents the absolute number of test samples. If None, the value is set to the complement of the train size. If train_size is also None, it will be set to 0.1.
If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the train split. If int, represents the absolute number of train samples. If None, the value is automatically set to the complement of the test size.
Controls the randomness of the training and testing indices produced. Pass an int for reproducible output across multiple function calls. See Glossary.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X, y[, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
Note that providing y is sufficient to generate the splits and hence np.zeros(n_samples) may be used as a placeholder for X instead of actual training data.
The target variable for supervised learning problems. Stratification is done based on the y labels.
Always ignored, exists for compatibility.
The training set indices for that split.
The testing set indices for that split.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.


Stratified ShuffleSplit cross-validator
Provides train/test indices to split data in train/test sets.
This cross-validation object is a merge of StratifiedKFold and ShuffleSplit, which returns stratified randomized folds. The folds are made by preserving the percentage of samples for each class.
Note: like the ShuffleSplit strategy, stratified random splits do not guarantee that all folds will be different, although this is still very likely for sizeable datasets.
Read more in the User Guide.
Number of re-shuffling & splitting iterations.
If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the test split. If int, represents the absolute number of test samples. If None, the value is set to the complement of the train size. If train_size is also None, it will be set to 0.1.
If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the train split. If int, represents the absolute number of train samples. If None, the value is automatically set to the complement of the test size.
Controls the randomness of the training and testing indices produced. Pass an int for reproducible output across multiple function calls. See Glossary.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X, y[, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
Note that providing y is sufficient to generate the splits and hence np.zeros(n_samples) may be used as a placeholder for X instead of actual training data.
The target variable for supervised learning problems. Stratification is done based on the y labels.
Always ignored, exists for compatibility.
The training set indices for that split.
The testing set indices for that split.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.
Visualizing cross-validation behavior in scikit-learn
RBF SVM parameters


Time Series cross-validator
Provides train/test indices to split time series data samples that are observed at fixed time intervals, in train/test sets. In each split, test indices must be higher than before, and thus shuffling in cross validator is inappropriate.
This cross-validation object is a variation of KFold. In the kth split, it returns first k folds as train set and the (k+1)th fold as test set.
Note that unlike standard cross-validation methods, successive training sets are supersets of those that come before them.
Read more in the User Guide.
New in version 0.18.
Number of splits. Must be at least 2.
Changed in version 0.22: n_splits default value changed from 3 to 5.
Maximum size for a single training set.
Used to limit the size of the test set. Defaults to n_samples // (n_splits + 1), which is the maximum allowed value with gap=0.
New in version 0.24.
Number of samples to exclude from the end of each train set before the test set.
New in version 0.24.
The training set has size i * n_samples // (n_splits + 1)
+ n_samples % (n_splits + 1) in the i th split, with a test set of size n_samples//(n_splits + 1) by default, where n_samples is the number of samples.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
The training set indices for that split.
The testing set indices for that split.


Time Series cross-validator
Provides train/test indices to split time series data samples that are observed at fixed time intervals, in train/test sets. In each split, test indices must be higher than before, and thus shuffling in cross validator is inappropriate.
This cross-validation object is a variation of KFold. In the kth split, it returns first k folds as train set and the (k+1)th fold as test set.
Note that unlike standard cross-validation methods, successive training sets are supersets of those that come before them.
Read more in the User Guide.
New in version 0.18.
Number of splits. Must be at least 2.
Changed in version 0.22: n_splits default value changed from 3 to 5.
Maximum size for a single training set.
Used to limit the size of the test set. Defaults to n_samples // (n_splits + 1), which is the maximum allowed value with gap=0.
New in version 0.24.
Number of samples to exclude from the end of each train set before the test set.
New in version 0.24.
The training set has size i * n_samples // (n_splits + 1)
+ n_samples % (n_splits + 1) in the i th split, with a test set of size n_samples//(n_splits + 1) by default, where n_samples is the number of samples.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
The training set indices for that split.
The testing set indices for that split.
Visualizing cross-validation behavior in scikit-learn


Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
The training set indices for that split.
The testing set indices for that split.


Split arrays or matrices into random train and test subsets
Quick utility that wraps input validation and next(ShuffleSplit().split(X, y)) and application to input data into a single call for splitting (and optionally subsampling) data in a oneliner.
Read more in the User Guide.
Allowed inputs are lists, numpy arrays, scipy-sparse matrices or pandas dataframes.
If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the test split. If int, represents the absolute number of test samples. If None, the value is set to the complement of the train size. If train_size is also None, it will be set to 0.25.
If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to include in the train split. If int, represents the absolute number of train samples. If None, the value is automatically set to the complement of the test size.
Controls the shuffling applied to the data before applying the split. Pass an int for reproducible output across multiple function calls. See Glossary.
Whether or not to shuffle the data before splitting. If shuffle=False then stratify must be None.
If not None, data is split in a stratified fashion, using this as the class labels. Read more in the User Guide.
List containing train-test split of inputs.
New in version 0.16: If the input is sparse, the output will be a scipy.sparse.csr_matrix. Else, output type is the same as the input type.


Validation curve.
Determine training and test scores for varying parameter values.
Compute scores for an estimator with different values of a specified parameter. This is similar to grid search with one parameter. However, this will also compute training scores and is merely a utility for plotting the results.
Read more in the User Guide.
An object of that type which is cloned for each validation.
Training vector, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Name of the parameter that will be varied.
The values of the parameter that will be evaluated.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Determines the cross-validation splitting strategy. Possible inputs for cv are:
For int/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
Refer User Guide for the various cross-validation strategies that can be used here.
Changed in version 0.22: cv default value if None changed from 3-fold to 5-fold.
A str (see model evaluation documentation) or a scorer callable object / function with signature scorer(estimator, X, y).
Number of jobs to run in parallel. Training the estimator and computing the score are parallelized over the combinations of each parameter value and each cross-validation split. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Number of predispatched jobs for parallel execution (default is all). The option can reduce the allocated memory. The str can be an expression like ‘2*n_jobs’.
Controls the verbosity: the higher, the more messages.
Parameters to pass to the fit method of the estimator.
New in version 0.24.
Value to assign to the score if an error occurs in estimator fitting. If set to ‘raise’, the error is raised. If a numeric value is given, FitFailedWarning is raised.
New in version 0.20.
Scores on training sets.
Scores on test set.
See Plotting Validation Curves


One-vs-one multiclass strategy
This strategy consists in fitting one classifier per class pair. At prediction time, the class which received the most votes is selected. Since it requires to fit n_classes * (n_classes - 1) / 2 classifiers, this method is usually slower than one-vs-the-rest, due to its O(n_classes^2) complexity. However, this method may be advantageous for algorithms such as kernel algorithms which don’t scale well with n_samples. This is because each individual learning problem only involves a small subset of the data whereas, with one-vs-the-rest, the complete dataset is used n_classes times.
Read more in the User Guide.
An estimator object implementing fit and one of decision_function or predict_proba.
The number of jobs to use for the computation: the n_classes * (
n_classes - 1) / 2 OVO problems are computed in parallel.
None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Estimators used for predictions.
Array containing labels.
Number of classes
Indices of samples used when training the estimators. None when estimator’s pairwise tag is False.
Deprecated since version 0.24: The _pairwise attribute is deprecated in 0.24. From 1.1 (renaming of 0.25) and onward, pairwise_indices_ will use the pairwise estimator tag instead.
decision_function(X)
Decision function for the OneVsOneClassifier.
fit(X, y)
Fit underlying estimators.
get_params([deep])
Get parameters for this estimator.
partial_fit(X, y[, classes])
Partially fit underlying estimators
predict(X)
Estimate the best class label for each sample in X.
score(X, y[, sample_weight])
Return the mean accuracy on the given test data and labels.
set_params(**params)
Set the parameters of this estimator.
Decision function for the OneVsOneClassifier.
The decision values for the samples are computed by adding the normalized sum of pair-wise classification confidence levels to the votes in order to disambiguate between the decision values when the votes for all the classes are equal leading to a tie.
Changed in version 0.19: output shape changed to (n_samples,) to conform to scikit-learn conventions for binary classification.
Fit underlying estimators.
Data.
Multi-class targets.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Partially fit underlying estimators
Should be used when memory is inefficient to train all data. Chunks of data can be passed in several iteration, where the first call should have an array of all target variables.
Data.
Multi-class targets.
Classes across all calls to partial_fit. Can be obtained via np.unique(y_all), where y_all is the target vector of the entire dataset. This argument is only required in the first call of partial_fit and can be omitted in the subsequent calls.
Estimate the best class label for each sample in X.
This is implemented as argmax(decision_function(X), axis=1) which will return the label of the class with most votes by estimators predicting the outcome of a decision for each possible class pair.
Data.
Predicted multi-class targets.
Return the mean accuracy on the given test data and labels.
In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.
Test samples.
True labels for X.
Sample weights.
Mean accuracy of self.predict(X) wrt. y.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.


Decision function for the OneVsOneClassifier.
The decision values for the samples are computed by adding the normalized sum of pair-wise classification confidence levels to the votes in order to disambiguate between the decision values when the votes for all the classes are equal leading to a tie.
Changed in version 0.19: output shape changed to (n_samples,) to conform to scikit-learn conventions for binary classification.


Fit underlying estimators.
Data.
Multi-class targets.


Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.


Partially fit underlying estimators
Should be used when memory is inefficient to train all data. Chunks of data can be passed in several iteration, where the first call should have an array of all target variables.
Data.
Multi-class targets.
Classes across all calls to partial_fit. Can be obtained via np.unique(y_all), where y_all is the target vector of the entire dataset. This argument is only required in the first call of partial_fit and can be omitted in the subsequent calls.


Estimate the best class label for each sample in X.
This is implemented as argmax(decision_function(X), axis=1) which will return the label of the class with most votes by estimators predicting the outcome of a decision for each possible class pair.
Data.
Predicted multi-class targets.


Return the mean accuracy on the given test data and labels.
In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.
Test samples.
True labels for X.
Sample weights.
Mean accuracy of self.predict(X) wrt. y.


Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.