Generate random samples from the fitted Gaussian distribution.
Number of samples to generate.
Randomly generated sample
Component labels


Compute the per-sample average log-likelihood of the given data X.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Log likelihood of the Gaussian mixture given X.


Compute the weighted log probabilities for each sample.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Log probabilities of each data point in X.


Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.


Gaussian Mixture.
Representation of a Gaussian mixture model probability distribution. This class allows to estimate the parameters of a Gaussian mixture distribution.
Read more in the User Guide.
New in version 0.18.
The number of mixture components.
String describing the type of covariance parameters to use. Must be one of:
each component has its own general covariance matrix
all components share the same general covariance matrix
each component has its own diagonal covariance matrix
each component has its own single variance
The convergence threshold. EM iterations will stop when the lower bound average gain is below this threshold.
Non-negative regularization added to the diagonal of covariance. Allows to assure that the covariance matrices are all positive.
The number of EM iterations to perform.
The number of initializations to perform. The best results are kept.
The method used to initialize the weights, the means and the precisions. Must be one of:
The user-provided initial weights. If it is None, weights are initialized using the init_params method.
The user-provided initial means, If it is None, means are initialized using the init_params method.
The user-provided initial precisions (inverse of the covariance matrices). If it is None, precisions are initialized using the ‘init_params’ method. The shape depends on ‘covariance_type’:
Controls the random seed given to the method chosen to initialize the parameters (see init_params). In addition, it controls the generation of random samples from the fitted distribution (see the method sample). Pass an int for reproducible output across multiple function calls. See Glossary.
If ‘warm_start’ is True, the solution of the last fitting is used as initialization for the next call of fit(). This can speed up convergence when fit is called several times on similar problems. In that case, ‘n_init’ is ignored and only a single initialization occurs upon the first call. See the Glossary.
Enable verbose output. If 1 then it prints the current initialization and each iteration step. If greater than 1 then it prints also the log probability and the time needed for each step.
Number of iteration done before the next print.
The weights of each mixture components.
The mean of each mixture component.
The covariance of each mixture component. The shape depends on covariance_type:
The precision matrices for each component in the mixture. A precision matrix is the inverse of a covariance matrix. A covariance matrix is symmetric positive definite so the mixture of Gaussian can be equivalently parameterized by the precision matrices. Storing the precision matrices instead of the covariance matrices makes it more efficient to compute the log-likelihood of new samples at test time. The shape depends on covariance_type:
The cholesky decomposition of the precision matrices of each mixture component. A precision matrix is the inverse of a covariance matrix. A covariance matrix is symmetric positive definite so the mixture of Gaussian can be equivalently parameterized by the precision matrices. Storing the precision matrices instead of the covariance matrices makes it more efficient to compute the log-likelihood of new samples at test time. The shape depends on covariance_type:
True when convergence was reached in fit(), False otherwise.
Number of step used by the best fit of EM to reach the convergence.
Lower bound value on the log-likelihood (of the training data with respect to the model) of the best fit of EM.
See also
Gaussian mixture model fit with a variational inference.
aic(X)
Akaike information criterion for the current model on the input X.
bic(X)
Bayesian information criterion for the current model on the input X.
fit(X[, y])
Estimate model parameters with the EM algorithm.
fit_predict(X[, y])
Estimate model parameters using X and predict the labels for X.
get_params([deep])
Get parameters for this estimator.
predict(X)
Predict the labels for the data samples in X using trained model.
predict_proba(X)
Predict posterior probability of each component given the data.
sample([n_samples])
Generate random samples from the fitted Gaussian distribution.
score(X[, y])
Compute the per-sample average log-likelihood of the given data X.
score_samples(X)
Compute the weighted log probabilities for each sample.
set_params(**params)
Set the parameters of this estimator.
Akaike information criterion for the current model on the input X.
The lower the better.
Bayesian information criterion for the current model on the input X.
The lower the better.
Estimate model parameters with the EM algorithm.
The method fits the model n_init times and sets the parameters with which the model has the largest likelihood or lower bound. Within each trial, the method iterates between E-step and M-step for max_iter times until the change of likelihood or lower bound is less than tol, otherwise, a ConvergenceWarning is raised. If warm_start is True, then n_init is ignored and a single initialization is performed upon the first call. Upon consecutive calls, training starts where it left off.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Estimate model parameters using X and predict the labels for X.
The method fits the model n_init times and sets the parameters with which the model has the largest likelihood or lower bound. Within each trial, the method iterates between E-step and M-step for max_iter times until the change of likelihood or lower bound is less than tol, otherwise, a ConvergenceWarning is raised. After fitting, it predicts the most probable label for the input data points.
New in version 0.20.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Component labels.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Predict the labels for the data samples in X using trained model.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Component labels.
Predict posterior probability of each component given the data.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Returns the probability each Gaussian (state) in the model given each sample.
Generate random samples from the fitted Gaussian distribution.
Number of samples to generate.
Randomly generated sample
Component labels
Compute the per-sample average log-likelihood of the given data X.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Log likelihood of the Gaussian mixture given X.
Compute the weighted log probabilities for each sample.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Log probabilities of each data point in X.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.


Gaussian Mixture.
Representation of a Gaussian mixture model probability distribution. This class allows to estimate the parameters of a Gaussian mixture distribution.
Read more in the User Guide.
New in version 0.18.
The number of mixture components.
String describing the type of covariance parameters to use. Must be one of:
each component has its own general covariance matrix
all components share the same general covariance matrix
each component has its own diagonal covariance matrix
each component has its own single variance
The convergence threshold. EM iterations will stop when the lower bound average gain is below this threshold.
Non-negative regularization added to the diagonal of covariance. Allows to assure that the covariance matrices are all positive.
The number of EM iterations to perform.
The number of initializations to perform. The best results are kept.
The method used to initialize the weights, the means and the precisions. Must be one of:
The user-provided initial weights. If it is None, weights are initialized using the init_params method.
The user-provided initial means, If it is None, means are initialized using the init_params method.
The user-provided initial precisions (inverse of the covariance matrices). If it is None, precisions are initialized using the ‘init_params’ method. The shape depends on ‘covariance_type’:
Controls the random seed given to the method chosen to initialize the parameters (see init_params). In addition, it controls the generation of random samples from the fitted distribution (see the method sample). Pass an int for reproducible output across multiple function calls. See Glossary.
If ‘warm_start’ is True, the solution of the last fitting is used as initialization for the next call of fit(). This can speed up convergence when fit is called several times on similar problems. In that case, ‘n_init’ is ignored and only a single initialization occurs upon the first call. See the Glossary.
Enable verbose output. If 1 then it prints the current initialization and each iteration step. If greater than 1 then it prints also the log probability and the time needed for each step.
Number of iteration done before the next print.
The weights of each mixture components.
The mean of each mixture component.
The covariance of each mixture component. The shape depends on covariance_type:
The precision matrices for each component in the mixture. A precision matrix is the inverse of a covariance matrix. A covariance matrix is symmetric positive definite so the mixture of Gaussian can be equivalently parameterized by the precision matrices. Storing the precision matrices instead of the covariance matrices makes it more efficient to compute the log-likelihood of new samples at test time. The shape depends on covariance_type:
The cholesky decomposition of the precision matrices of each mixture component. A precision matrix is the inverse of a covariance matrix. A covariance matrix is symmetric positive definite so the mixture of Gaussian can be equivalently parameterized by the precision matrices. Storing the precision matrices instead of the covariance matrices makes it more efficient to compute the log-likelihood of new samples at test time. The shape depends on covariance_type:
True when convergence was reached in fit(), False otherwise.
Number of step used by the best fit of EM to reach the convergence.
Lower bound value on the log-likelihood (of the training data with respect to the model) of the best fit of EM.
See also
Gaussian mixture model fit with a variational inference.
aic(X)
Akaike information criterion for the current model on the input X.
bic(X)
Bayesian information criterion for the current model on the input X.
fit(X[, y])
Estimate model parameters with the EM algorithm.
fit_predict(X[, y])
Estimate model parameters using X and predict the labels for X.
get_params([deep])
Get parameters for this estimator.
predict(X)
Predict the labels for the data samples in X using trained model.
predict_proba(X)
Predict posterior probability of each component given the data.
sample([n_samples])
Generate random samples from the fitted Gaussian distribution.
score(X[, y])
Compute the per-sample average log-likelihood of the given data X.
score_samples(X)
Compute the weighted log probabilities for each sample.
set_params(**params)
Set the parameters of this estimator.
Akaike information criterion for the current model on the input X.
The lower the better.
Bayesian information criterion for the current model on the input X.
The lower the better.
Estimate model parameters with the EM algorithm.
The method fits the model n_init times and sets the parameters with which the model has the largest likelihood or lower bound. Within each trial, the method iterates between E-step and M-step for max_iter times until the change of likelihood or lower bound is less than tol, otherwise, a ConvergenceWarning is raised. If warm_start is True, then n_init is ignored and a single initialization is performed upon the first call. Upon consecutive calls, training starts where it left off.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Estimate model parameters using X and predict the labels for X.
The method fits the model n_init times and sets the parameters with which the model has the largest likelihood or lower bound. Within each trial, the method iterates between E-step and M-step for max_iter times until the change of likelihood or lower bound is less than tol, otherwise, a ConvergenceWarning is raised. After fitting, it predicts the most probable label for the input data points.
New in version 0.20.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Component labels.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Predict the labels for the data samples in X using trained model.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Component labels.
Predict posterior probability of each component given the data.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Returns the probability each Gaussian (state) in the model given each sample.
Generate random samples from the fitted Gaussian distribution.
Number of samples to generate.
Randomly generated sample
Component labels
Compute the per-sample average log-likelihood of the given data X.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Log likelihood of the Gaussian mixture given X.
Compute the weighted log probabilities for each sample.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Log probabilities of each data point in X.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Comparing different clustering algorithms on toy datasets
Density Estimation for a Gaussian mixture
Gaussian Mixture Model Ellipsoids
Gaussian Mixture Model Selection
GMM covariances
Gaussian Mixture Model Sine Curve


Akaike information criterion for the current model on the input X.
The lower the better.


Bayesian information criterion for the current model on the input X.
The lower the better.


Estimate model parameters with the EM algorithm.
The method fits the model n_init times and sets the parameters with which the model has the largest likelihood or lower bound. Within each trial, the method iterates between E-step and M-step for max_iter times until the change of likelihood or lower bound is less than tol, otherwise, a ConvergenceWarning is raised. If warm_start is True, then n_init is ignored and a single initialization is performed upon the first call. Upon consecutive calls, training starts where it left off.
List of n_features-dimensional data points. Each row corresponds to a single data point.


Estimate model parameters using X and predict the labels for X.
The method fits the model n_init times and sets the parameters with which the model has the largest likelihood or lower bound. Within each trial, the method iterates between E-step and M-step for max_iter times until the change of likelihood or lower bound is less than tol, otherwise, a ConvergenceWarning is raised. After fitting, it predicts the most probable label for the input data points.
New in version 0.20.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Component labels.


Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.


Predict the labels for the data samples in X using trained model.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Component labels.


Predict posterior probability of each component given the data.
List of n_features-dimensional data points. Each row corresponds to a single data point.
Returns the probability each Gaussian (state) in the model given each sample.


Input checker utility for building a cross-validator
Determines the cross-validation splitting strategy. Possible inputs for cv are: - None, to use the default 5-fold cross validation, - integer, to specify the number of folds. - CV splitter, - An iterable yielding (train, test) splits as arrays of indices.
For integer/None inputs, if classifier is True and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
Refer User Guide for the various cross-validation strategies that can be used here.
Changed in version 0.22: cv default value changed from 3-fold to 5-fold.
The target variable for supervised learning problems.
Whether the task is a classification task, in which case stratified KFold will be used.
The return value is a cross-validator which generates the train/test splits via the split method.


Evaluate metric(s) by cross-validation and also record fit/score times.
Read more in the User Guide.
The object to use to fit the data.
The data to fit. Can be for example a list, or an array.
The target variable to try to predict in the case of supervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Strategy to evaluate the performance of the cross-validated model on the test set.
If scoring represents a single score, one can use:
If scoring reprents multiple scores, one can use:
See Specifying multiple metrics for evaluation for an example.
Determines the cross-validation splitting strategy. Possible inputs for cv are:
For int/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
Refer User Guide for the various cross-validation strategies that can be used here.
Changed in version 0.22: cv default value if None changed from 3-fold to 5-fold.
Number of jobs to run in parallel. Training the estimator and computing the score are parallelized over the cross-validation splits. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
The verbosity level.
Parameters to pass to the fit method of the estimator.
Controls the number of jobs that get dispatched during parallel execution. Reducing this number can be useful to avoid an explosion of memory consumption when more jobs get dispatched than CPUs can process. This parameter can be:
Whether to include train scores. Computing training scores is used to get insights on how different parameter settings impact the overfitting/underfitting trade-off. However computing the scores on the training set can be computationally expensive and is not strictly required to select the parameters that yield the best generalization performance.
New in version 0.19.
Changed in version 0.21: Default value was changed from True to False
Whether to return the estimators fitted on each split.
New in version 0.20.
Value to assign to the score if an error occurs in estimator fitting. If set to ‘raise’, the error is raised. If a numeric value is given, FitFailedWarning is raised.
New in version 0.20.
Array of scores of the estimator for each run of the cross validation.
A dict of arrays containing the score/time arrays for each scorer is returned. The possible keys for this dict are:
The score array for test scores on each cv split. Suffix _score in test_score changes to a specific metric like test_r2 or test_auc if there are multiple scoring metrics in the scoring parameter.
The score array for train scores on each cv split. Suffix _score in train_score changes to a specific metric like train_r2 or train_auc if there are multiple scoring metrics in the scoring parameter. This is available only if return_train_score parameter is True.
The time for fitting the estimator on the train set for each cv split.
The time for scoring the estimator on the test set for each cv split. (Note time for scoring on the train set is not included even if return_train_score is set to True
The estimator objects for each cv split. This is available only if return_estimator parameter is set to True.
See also
Run cross-validation for single metric evaluation.
Get predictions from each split of cross-validation for diagnostic purposes.
Make a scorer from a performance metric or loss function.
Single metric evaluation using cross_validate
Multiple metric evaluation using cross_validate (please refer the scoring parameter doc for more information)


Generate cross-validated estimates for each input data point
The data is split according to the cv parameter. Each sample belongs to exactly one test set, and its prediction is computed with an estimator fitted on the corresponding training set.
Passing these predictions into an evaluation metric may not be a valid way to measure generalization performance. Results can differ from cross_validate and cross_val_score unless all tests sets have equal size and the metric decomposes over samples.
Read more in the User Guide.
The object to use to fit the data.
The data to fit. Can be, for example a list, or an array at least 2d.
The target variable to try to predict in the case of supervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Determines the cross-validation splitting strategy. Possible inputs for cv are:
For int/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
Refer User Guide for the various cross-validation strategies that can be used here.
Changed in version 0.22: cv default value if None changed from 3-fold to 5-fold.
Number of jobs to run in parallel. Training the estimator and predicting are parallelized over the cross-validation splits. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
The verbosity level.
Parameters to pass to the fit method of the estimator.
Controls the number of jobs that get dispatched during parallel execution. Reducing this number can be useful to avoid an explosion of memory consumption when more jobs get dispatched than CPUs can process. This parameter can be:
The method to be invoked by estimator.
This is the result of calling method. Shape:
See also
Calculate score for each CV split.
Calculate one or more scores and timings for each CV split.
In the case that one or more classes are absent in a training portion, a default score needs to be assigned to all instances for that class if method produces columns per class, as in {‘decision_function’, ‘predict_proba’, ‘predict_log_proba’}. For predict_proba this value is 0. In order to ensure finite output, we approximate negative infinity by the minimum finite float value for the dtype in other cases.


Evaluate a score by cross-validation
Read more in the User Guide.
The object to use to fit the data.
The data to fit. Can be for example a list, or an array.
The target variable to try to predict in the case of supervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
A str (see model evaluation documentation) or a scorer callable object / function with signature scorer(estimator, X, y) which should return only a single value.
Similar to cross_validate but only a single metric is permitted.
If None, the estimator’s default scorer (if available) is used.
Determines the cross-validation splitting strategy. Possible inputs for cv are:
For int/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
Refer User Guide for the various cross-validation strategies that can be used here.
Changed in version 0.22: cv default value if None changed from 3-fold to 5-fold.
Number of jobs to run in parallel. Training the estimator and computing the score are parallelized over the cross-validation splits. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
The verbosity level.
Parameters to pass to the fit method of the estimator.
Controls the number of jobs that get dispatched during parallel execution. Reducing this number can be useful to avoid an explosion of memory consumption when more jobs get dispatched than CPUs can process. This parameter can be:
Value to assign to the score if an error occurs in estimator fitting. If set to ‘raise’, the error is raised. If a numeric value is given, FitFailedWarning is raised.
New in version 0.20.
Array of scores of the estimator for each run of the cross validation.
See also
To run cross-validation on multiple metrics and also to return train scores, fit times and score times.
Get predictions from each split of cross-validation for diagnostic purposes.
Make a scorer from a performance metric or loss function.


Exhaustive search over specified parameter values for an estimator.
Important members are fit, predict.
GridSearchCV implements a “fit” and a “score” method. It also implements “score_samples”, “predict”, “predict_proba”, “decision_function”, “transform” and “inverse_transform” if they are implemented in the estimator used.
The parameters of the estimator used to apply these methods are optimized by cross-validated grid-search over a parameter grid.
Read more in the User Guide.
This is assumed to implement the scikit-learn estimator interface. Either estimator needs to provide a score function, or scoring must be passed.
Dictionary with parameters names (str) as keys and lists of parameter settings to try as values, or a list of such dictionaries, in which case the grids spanned by each dictionary in the list are explored. This enables searching over any sequence of parameter settings.
Strategy to evaluate the performance of the cross-validated model on the test set.
If scoring represents a single score, one can use:
If scoring reprents multiple scores, one can use:
See Specifying multiple metrics for evaluation for an example.
Number of jobs to run in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Changed in version v0.20: n_jobs default changed from 1 to None
Controls the number of jobs that get dispatched during parallel execution. Reducing this number can be useful to avoid an explosion of memory consumption when more jobs get dispatched than CPUs can process. This parameter can be:
Determines the cross-validation splitting strategy. Possible inputs for cv are:
For integer/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
Refer User Guide for the various cross-validation strategies that can be used here.
Changed in version 0.22: cv default value if None changed from 3-fold to 5-fold.
Refit an estimator using the best found parameters on the whole dataset.
For multiple metric evaluation, this needs to be a str denoting the scorer that would be used to find the best parameters for refitting the estimator at the end.
Where there are considerations other than maximum score in choosing a best estimator, refit can be set to a function which returns the selected best_index_ given cv_results_. In that case, the best_estimator_ and best_params_ will be set according to the returned best_index_ while the best_score_ attribute will not be available.
The refitted estimator is made available at the best_estimator_ attribute and permits using predict directly on this GridSearchCV instance.
Also for multiple metric evaluation, the attributes best_index_, best_score_ and best_params_ will only be available if refit is set and all of them will be determined w.r.t this specific scorer.
See scoring parameter to know more about multiple metric evaluation.
Changed in version 0.20: Support for callable added.
Controls the verbosity: the higher, the more messages.
Value to assign to the score if an error occurs in estimator fitting. If set to ‘raise’, the error is raised. If a numeric value is given, FitFailedWarning is raised. This parameter does not affect the refit step, which will always raise the error.
If False, the cv_results_ attribute will not include training scores. Computing training scores is used to get insights on how different parameter settings impact the overfitting/underfitting trade-off. However computing the scores on the training set can be computationally expensive and is not strictly required to select the parameters that yield the best generalization performance.
New in version 0.19.
Changed in version 0.21: Default value was changed from True to False
A dict with keys as column headers and values as columns, that can be imported into a pandas DataFrame.
For instance the below given table
param_kernel
param_gamma
param_degree
split0_test_score
…
rank_t…
‘poly’
–
2
0.80
…
2
‘poly’
–
3
0.70
…
4
‘rbf’
0.1
–
0.80
…
3
‘rbf’
0.2
–
0.93
…
1
will be represented by a cv_results_ dict of:
NOTE
The key 'params' is used to store a list of parameter settings dicts for all the parameter candidates.
The mean_fit_time, std_fit_time, mean_score_time and std_score_time are all in seconds.
For multi-metric evaluation, the scores for all the scorers are available in the cv_results_ dict at the keys ending with that scorer’s name ('_<scorer_name>') instead of '_score' shown above. (‘split0_test_precision’, ‘mean_train_precision’ etc.)
Estimator that was chosen by the search, i.e. estimator which gave highest score (or smallest loss if specified) on the left out data. Not available if refit=False.
See refit parameter for more information on allowed values.
Mean cross-validated score of the best_estimator
For multi-metric evaluation, this is present only if refit is specified.
This attribute is not available if refit is a function.
Parameter setting that gave the best results on the hold out data.
For multi-metric evaluation, this is present only if refit is specified.
The index (of the cv_results_ arrays) which corresponds to the best candidate parameter setting.
The dict at search.cv_results_['params'][search.best_index_] gives the parameter setting for the best model, that gives the highest mean score (search.best_score_).
For multi-metric evaluation, this is present only if refit is specified.
Scorer function used on the held out data to choose the best parameters for the model.
For multi-metric evaluation, this attribute holds the validated scoring dict which maps the scorer key to the scorer callable.
The number of cross-validation splits (folds/iterations).
Seconds used for refitting the best model on the whole dataset.
This is present only if refit is not False.
New in version 0.20.
Whether or not the scorers compute several metrics.
See also
Generates all the combinations of a hyperparameter grid.
Utility function to split the data into a development set usable for fitting a GridSearchCV instance and an evaluation set for its final evaluation.
Make a scorer from a performance metric or loss function.
The parameters selected are those that maximize the score of the left out data, unless an explicit score is passed in which case it is used instead.
If n_jobs was set to a value higher than one, the data is copied for each point in the grid (and not n_jobs times). This is done for efficiency reasons if individual jobs take very little time, but may raise errors if the dataset is large and not enough memory is available. A workaround in this case is to set pre_dispatch. Then, the memory is copied only pre_dispatch many times. A reasonable value for pre_dispatch is 2 *
n_jobs.
decision_function(X)
Call decision_function on the estimator with the best found parameters.
fit(X[, y, groups])
Run fit with all sets of parameters.
get_params([deep])
Get parameters for this estimator.
inverse_transform(Xt)
Call inverse_transform on the estimator with the best found params.
predict(X)
Call predict on the estimator with the best found parameters.
predict_log_proba(X)
Call predict_log_proba on the estimator with the best found parameters.
predict_proba(X)
Call predict_proba on the estimator with the best found parameters.
score(X[, y])
Returns the score on the given data, if the estimator has been refit.
score_samples(X)
Call score_samples on the estimator with the best found parameters.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Call transform on the estimator with the best found parameters.
Call decision_function on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports decision_function.
Must fulfill the input assumptions of the underlying estimator.
Run fit with all sets of parameters.
Training vector, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Parameters passed to the fit method of the estimator
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Call inverse_transform on the estimator with the best found params.
Only available if the underlying estimator implements inverse_transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.
Call predict on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict.
Must fulfill the input assumptions of the underlying estimator.
Call predict_log_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_log_proba.
Must fulfill the input assumptions of the underlying estimator.
Call predict_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_proba.
Must fulfill the input assumptions of the underlying estimator.
Returns the score on the given data, if the estimator has been refit.
This uses the score defined by scoring where provided, and the best_estimator_.score method otherwise.
Input data, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Call score_samples on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports score_samples.
New in version 0.24.
Data to predict on. Must fulfill input requirements of the underlying estimator.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Call transform on the estimator with the best found parameters.
Only available if the underlying estimator supports transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.


Exhaustive search over specified parameter values for an estimator.
Important members are fit, predict.
GridSearchCV implements a “fit” and a “score” method. It also implements “score_samples”, “predict”, “predict_proba”, “decision_function”, “transform” and “inverse_transform” if they are implemented in the estimator used.
The parameters of the estimator used to apply these methods are optimized by cross-validated grid-search over a parameter grid.
Read more in the User Guide.
This is assumed to implement the scikit-learn estimator interface. Either estimator needs to provide a score function, or scoring must be passed.
Dictionary with parameters names (str) as keys and lists of parameter settings to try as values, or a list of such dictionaries, in which case the grids spanned by each dictionary in the list are explored. This enables searching over any sequence of parameter settings.
Strategy to evaluate the performance of the cross-validated model on the test set.
If scoring represents a single score, one can use:
If scoring reprents multiple scores, one can use:
See Specifying multiple metrics for evaluation for an example.
Number of jobs to run in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Changed in version v0.20: n_jobs default changed from 1 to None
Controls the number of jobs that get dispatched during parallel execution. Reducing this number can be useful to avoid an explosion of memory consumption when more jobs get dispatched than CPUs can process. This parameter can be:
Determines the cross-validation splitting strategy. Possible inputs for cv are:
For integer/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
Refer User Guide for the various cross-validation strategies that can be used here.
Changed in version 0.22: cv default value if None changed from 3-fold to 5-fold.
Refit an estimator using the best found parameters on the whole dataset.
For multiple metric evaluation, this needs to be a str denoting the scorer that would be used to find the best parameters for refitting the estimator at the end.
Where there are considerations other than maximum score in choosing a best estimator, refit can be set to a function which returns the selected best_index_ given cv_results_. In that case, the best_estimator_ and best_params_ will be set according to the returned best_index_ while the best_score_ attribute will not be available.
The refitted estimator is made available at the best_estimator_ attribute and permits using predict directly on this GridSearchCV instance.
Also for multiple metric evaluation, the attributes best_index_, best_score_ and best_params_ will only be available if refit is set and all of them will be determined w.r.t this specific scorer.
See scoring parameter to know more about multiple metric evaluation.
Changed in version 0.20: Support for callable added.
Controls the verbosity: the higher, the more messages.
Value to assign to the score if an error occurs in estimator fitting. If set to ‘raise’, the error is raised. If a numeric value is given, FitFailedWarning is raised. This parameter does not affect the refit step, which will always raise the error.
If False, the cv_results_ attribute will not include training scores. Computing training scores is used to get insights on how different parameter settings impact the overfitting/underfitting trade-off. However computing the scores on the training set can be computationally expensive and is not strictly required to select the parameters that yield the best generalization performance.
New in version 0.19.
Changed in version 0.21: Default value was changed from True to False
A dict with keys as column headers and values as columns, that can be imported into a pandas DataFrame.
For instance the below given table
param_kernel
param_gamma
param_degree
split0_test_score
…
rank_t…
‘poly’
–
2
0.80
…
2
‘poly’
–
3
0.70
…
4
‘rbf’
0.1
–
0.80
…
3
‘rbf’
0.2
–
0.93
…
1
will be represented by a cv_results_ dict of:
NOTE
The key 'params' is used to store a list of parameter settings dicts for all the parameter candidates.
The mean_fit_time, std_fit_time, mean_score_time and std_score_time are all in seconds.
For multi-metric evaluation, the scores for all the scorers are available in the cv_results_ dict at the keys ending with that scorer’s name ('_<scorer_name>') instead of '_score' shown above. (‘split0_test_precision’, ‘mean_train_precision’ etc.)
Estimator that was chosen by the search, i.e. estimator which gave highest score (or smallest loss if specified) on the left out data. Not available if refit=False.
See refit parameter for more information on allowed values.
Mean cross-validated score of the best_estimator
For multi-metric evaluation, this is present only if refit is specified.
This attribute is not available if refit is a function.
Parameter setting that gave the best results on the hold out data.
For multi-metric evaluation, this is present only if refit is specified.
The index (of the cv_results_ arrays) which corresponds to the best candidate parameter setting.
The dict at search.cv_results_['params'][search.best_index_] gives the parameter setting for the best model, that gives the highest mean score (search.best_score_).
For multi-metric evaluation, this is present only if refit is specified.
Scorer function used on the held out data to choose the best parameters for the model.
For multi-metric evaluation, this attribute holds the validated scoring dict which maps the scorer key to the scorer callable.
The number of cross-validation splits (folds/iterations).
Seconds used for refitting the best model on the whole dataset.
This is present only if refit is not False.
New in version 0.20.
Whether or not the scorers compute several metrics.
See also
Generates all the combinations of a hyperparameter grid.
Utility function to split the data into a development set usable for fitting a GridSearchCV instance and an evaluation set for its final evaluation.
Make a scorer from a performance metric or loss function.
The parameters selected are those that maximize the score of the left out data, unless an explicit score is passed in which case it is used instead.
If n_jobs was set to a value higher than one, the data is copied for each point in the grid (and not n_jobs times). This is done for efficiency reasons if individual jobs take very little time, but may raise errors if the dataset is large and not enough memory is available. A workaround in this case is to set pre_dispatch. Then, the memory is copied only pre_dispatch many times. A reasonable value for pre_dispatch is 2 *
n_jobs.
decision_function(X)
Call decision_function on the estimator with the best found parameters.
fit(X[, y, groups])
Run fit with all sets of parameters.
get_params([deep])
Get parameters for this estimator.
inverse_transform(Xt)
Call inverse_transform on the estimator with the best found params.
predict(X)
Call predict on the estimator with the best found parameters.
predict_log_proba(X)
Call predict_log_proba on the estimator with the best found parameters.
predict_proba(X)
Call predict_proba on the estimator with the best found parameters.
score(X[, y])
Returns the score on the given data, if the estimator has been refit.
score_samples(X)
Call score_samples on the estimator with the best found parameters.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Call transform on the estimator with the best found parameters.
Call decision_function on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports decision_function.
Must fulfill the input assumptions of the underlying estimator.
Run fit with all sets of parameters.
Training vector, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Parameters passed to the fit method of the estimator
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Call inverse_transform on the estimator with the best found params.
Only available if the underlying estimator implements inverse_transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.
Call predict on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict.
Must fulfill the input assumptions of the underlying estimator.
Call predict_log_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_log_proba.
Must fulfill the input assumptions of the underlying estimator.
Call predict_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_proba.
Must fulfill the input assumptions of the underlying estimator.
Returns the score on the given data, if the estimator has been refit.
This uses the score defined by scoring where provided, and the best_estimator_.score method otherwise.
Input data, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Call score_samples on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports score_samples.
New in version 0.24.
Data to predict on. Must fulfill input requirements of the underlying estimator.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Call transform on the estimator with the best found parameters.
Only available if the underlying estimator supports transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.
Release Highlights for scikit-learn 0.24
Feature agglomeration vs. univariate selection
Shrinkage covariance estimation: LedoitWolf vs OAS and max-likelihood
Model selection with Probabilistic PCA and Factor Analysis (FA)
Faces recognition example using eigenfaces and SVMs
Comparison of kernel ridge and Gaussian process regression
Comparison of kernel ridge regression and SVR
Parameter estimation using grid search with cross-validation
Comparing randomized search and grid search for hyperparameter estimation
Nested versus non-nested cross-validation
Demonstration of multi-metric evaluation on cross_val_score and GridSearchCV
Sample pipeline for text feature extraction and evaluation
Balance model complexity and cross-validated score
Comparison between grid search and successive halving
Statistical comparison of models using grid search
Kernel Density Estimation
Caching nearest neighbors
Concatenating multiple feature extraction methods
Pipelining: chaining a PCA and a logistic regression
Selecting dimensionality reduction with Pipeline and GridSearchCV
Column Transformer with Mixed Types
Feature discretization
Scaling the regularization parameter for SVCs
RBF SVM parameters
Cross-validation on diabetes Dataset Exercise


Call decision_function on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports decision_function.
Must fulfill the input assumptions of the underlying estimator.


Run fit with all sets of parameters.
Training vector, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Parameters passed to the fit method of the estimator


Call inverse_transform on the estimator with the best found params.
Only available if the underlying estimator implements inverse_transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.


Call predict on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict.
Must fulfill the input assumptions of the underlying estimator.


Call predict_log_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_log_proba.
Must fulfill the input assumptions of the underlying estimator.


Call predict_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_proba.
Must fulfill the input assumptions of the underlying estimator.


Returns the score on the given data, if the estimator has been refit.
This uses the score defined by scoring where provided, and the best_estimator_.score method otherwise.
Input data, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.


Call score_samples on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports score_samples.
New in version 0.24.
Data to predict on. Must fulfill input requirements of the underlying estimator.


Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.


Call transform on the estimator with the best found parameters.
Only available if the underlying estimator supports transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.


K-fold iterator variant with non-overlapping groups.
The same group will not appear in two different folds (the number of distinct groups has to be at least equal to the number of folds).
The folds are approximately balanced in the sense that the number of distinct groups is approximately the same in each fold.
Read more in the User Guide.
Number of folds. Must be at least 2.
Changed in version 0.22: n_splits default value changed from 3 to 5.
See also
For splitting the data according to explicit domain-specific stratification of the dataset.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.


K-fold iterator variant with non-overlapping groups.
The same group will not appear in two different folds (the number of distinct groups has to be at least equal to the number of folds).
The folds are approximately balanced in the sense that the number of distinct groups is approximately the same in each fold.
Read more in the User Guide.
Number of folds. Must be at least 2.
Changed in version 0.22: n_splits default value changed from 3 to 5.
See also
For splitting the data according to explicit domain-specific stratification of the dataset.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.
Visualizing cross-validation behavior in scikit-learn


Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.


Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.


Shuffle-Group(s)-Out cross-validation iterator
Provides randomized train/test indices to split data according to a third-party provided group. This group information can be used to encode arbitrary domain specific stratifications of the samples as integers.
For instance the groups could be the year of collection of the samples and thus allow for cross-validation against time-based splits.
The difference between LeavePGroupsOut and GroupShuffleSplit is that the former generates splits using all subsets of size p unique groups, whereas GroupShuffleSplit generates a user-determined number of random test splits, each with a user-determined fraction of unique groups.
For example, a less computationally intensive alternative to LeavePGroupsOut(p=10) would be GroupShuffleSplit(test_size=10, n_splits=100).
Note: The parameters test_size and train_size refer to groups, and not to samples, as in ShuffleSplit.
Read more in the User Guide.
Number of re-shuffling & splitting iterations.
If float, should be between 0.0 and 1.0 and represent the proportion of groups to include in the test split (rounded up). If int, represents the absolute number of test groups. If None, the value is set to the complement of the train size. The default will change in version 0.21. It will remain 0.2 only if train_size is unspecified, otherwise it will complement the specified train_size.
If float, should be between 0.0 and 1.0 and represent the proportion of the groups to include in the train split. If int, represents the absolute number of train groups. If None, the value is automatically set to the complement of the test size.
Controls the randomness of the training and testing indices produced. Pass an int for reproducible output across multiple function calls. See Glossary.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.


Shuffle-Group(s)-Out cross-validation iterator
Provides randomized train/test indices to split data according to a third-party provided group. This group information can be used to encode arbitrary domain specific stratifications of the samples as integers.
For instance the groups could be the year of collection of the samples and thus allow for cross-validation against time-based splits.
The difference between LeavePGroupsOut and GroupShuffleSplit is that the former generates splits using all subsets of size p unique groups, whereas GroupShuffleSplit generates a user-determined number of random test splits, each with a user-determined fraction of unique groups.
For example, a less computationally intensive alternative to LeavePGroupsOut(p=10) would be GroupShuffleSplit(test_size=10, n_splits=100).
Note: The parameters test_size and train_size refer to groups, and not to samples, as in ShuffleSplit.
Read more in the User Guide.
Number of re-shuffling & splitting iterations.
If float, should be between 0.0 and 1.0 and represent the proportion of groups to include in the test split (rounded up). If int, represents the absolute number of test groups. If None, the value is set to the complement of the train size. The default will change in version 0.21. It will remain 0.2 only if train_size is unspecified, otherwise it will complement the specified train_size.
If float, should be between 0.0 and 1.0 and represent the proportion of the groups to include in the train split. If int, represents the absolute number of train groups. If None, the value is automatically set to the complement of the test size.
Controls the randomness of the training and testing indices produced. Pass an int for reproducible output across multiple function calls. See Glossary.
get_n_splits([X, y, groups])
Returns the number of splitting iterations in the cross-validator
split(X[, y, groups])
Generate indices to split data into training and test set.
Returns the number of splitting iterations in the cross-validator
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Always ignored, exists for compatibility.
Returns the number of splitting iterations in the cross-validator.
Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.
Visualizing cross-validation behavior in scikit-learn


Generate indices to split data into training and test set.
Training data, where n_samples is the number of samples and n_features is the number of features.
The target variable for supervised learning problems.
Group labels for the samples used while splitting the dataset into train/test set.
The training set indices for that split.
The testing set indices for that split.
Randomized CV splitters may return different results for each call of split. You can make the results identical by setting random_state to an integer.


Search over specified parameter values with successive halving.
The search strategy starts evaluating all the candidates with a small amount of resources and iteratively selects the best candidates, using more and more resources.
Read more in the User guide.
Note
This estimator is still experimental for now: the predictions and the API might change without any deprecation cycle. To use it, you need to explicitly import enable_halving_search_cv:
This is assumed to implement the scikit-learn estimator interface. Either estimator needs to provide a score function, or scoring must be passed.
Dictionary with parameters names (string) as keys and lists of parameter settings to try as values, or a list of such dictionaries, in which case the grids spanned by each dictionary in the list are explored. This enables searching over any sequence of parameter settings.
The ‘halving’ parameter, which determines the proportion of candidates that are selected for each subsequent iteration. For example, factor=3 means that only one third of the candidates are selected.
Defines the resource that increases with each iteration. By default, the resource is the number of samples. It can also be set to any parameter of the base estimator that accepts positive integer values, e.g. ‘n_iterations’ or ‘n_estimators’ for a gradient boosting estimator. In this case max_resources cannot be ‘auto’ and must be set explicitly.
The maximum amount of resource that any candidate is allowed to use for a given iteration. By default, this is set to n_samples when resource='n_samples' (default), else an error is raised.
The minimum amount of resource that any candidate is allowed to use for a given iteration. Equivalently, this defines the amount of resources r0 that are allocated for each candidate at the first iteration.
problem
classification problem
Note that the amount of resources used at each iteration is always a multiple of min_resources.
This is only relevant in cases where there isn’t enough resources to reduce the remaining candidates to at most factor after the last iteration. If True, then the search process will ‘replay’ the first iteration for as long as needed until the number of candidates is small enough. This is False by default, which means that the last iteration may evaluate more than factor candidates. See Aggressive elimination of candidates for more details.
Determines the cross-validation splitting strategy. Possible inputs for cv are:
For integer/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
Refer User Guide for the various cross-validation strategies that can be used here.
Note
Due to implementation details, the folds produced by cv must be the same across multiple calls to cv.split(). For built-in scikit-learn iterators, this can be achieved by deactivating shuffling (shuffle=False), or by setting the cv’s random_state parameter to an integer.
A single string (see The scoring parameter: defining model evaluation rules) or a callable (see Defining your scoring strategy from metric functions) to evaluate the predictions on the test set. If None, the estimator’s score method is used.
If True, refit an estimator using the best found parameters on the whole dataset.
The refitted estimator is made available at the best_estimator_ attribute and permits using predict directly on this HalvingGridSearchCV instance.
Value to assign to the score if an error occurs in estimator fitting. If set to ‘raise’, the error is raised. If a numeric value is given, FitFailedWarning is raised. This parameter does not affect the refit step, which will always raise the error. Default is np.nan
If False, the cv_results_ attribute will not include training scores. Computing training scores is used to get insights on how different parameter settings impact the overfitting/underfitting trade-off. However computing the scores on the training set can be computationally expensive and is not strictly required to select the parameters that yield the best generalization performance.
Pseudo random number generator state used for subsampling the dataset when resources != 'n_samples'. Ignored otherwise. Pass an int for reproducible output across multiple function calls. See Glossary.
Number of jobs to run in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Controls the verbosity: the higher, the more messages.
The amount of resources used at each iteration.
The number of candidate parameters that were evaluated at each iteration.
The number of candidate parameters that are left after the last iteration. It corresponds to ceil(n_candidates[-1] / factor)
The maximum number of resources that any candidate is allowed to use for a given iteration. Note that since the number of resources used at each iteration must be a multiple of min_resources_, the actual number of resources used at the last iteration may be smaller than max_resources_.
The amount of resources that are allocated for each candidate at the first iteration.
The actual number of iterations that were run. This is equal to n_required_iterations_ if aggressive_elimination is True. Else, this is equal to min(n_possible_iterations_,
n_required_iterations_).
The number of iterations that are possible starting with min_resources_ resources and without exceeding max_resources_.
The number of iterations that are required to end up with less than factor candidates at the last iteration, starting with min_resources_ resources. This will be smaller than n_possible_iterations_ when there isn’t enough resources.
A dict with keys as column headers and values as columns, that can be imported into a pandas DataFrame. It contains many informations for analysing the results of a search. Please refer to the User guide for details.
Estimator that was chosen by the search, i.e. estimator which gave highest score (or smallest loss if specified) on the left out data. Not available if refit=False.
Mean cross-validated score of the best_estimator.
Parameter setting that gave the best results on the hold out data.
The index (of the cv_results_ arrays) which corresponds to the best candidate parameter setting.
The dict at search.cv_results_['params'][search.best_index_] gives the parameter setting for the best model, that gives the highest mean score (search.best_score_).
Scorer function used on the held out data to choose the best parameters for the model.
The number of cross-validation splits (folds/iterations).
Seconds used for refitting the best model on the whole dataset.
This is present only if refit is not False.
See also
Random search over a set of parameters using successive halving.
The parameters selected are those that maximize the score of the held-out data, according to the scoring parameter.
decision_function(X)
Call decision_function on the estimator with the best found parameters.
fit(X[, y, groups])
Run fit with all sets of parameters.
get_params([deep])
Get parameters for this estimator.
inverse_transform(Xt)
Call inverse_transform on the estimator with the best found params.
predict(X)
Call predict on the estimator with the best found parameters.
predict_log_proba(X)
Call predict_log_proba on the estimator with the best found parameters.
predict_proba(X)
Call predict_proba on the estimator with the best found parameters.
score(X[, y])
Returns the score on the given data, if the estimator has been refit.
score_samples(X)
Call score_samples on the estimator with the best found parameters.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Call transform on the estimator with the best found parameters.
Call decision_function on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports decision_function.
Must fulfill the input assumptions of the underlying estimator.
Run fit with all sets of parameters.
Training vector, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Parameters passed to the fit method of the estimator
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Call inverse_transform on the estimator with the best found params.
Only available if the underlying estimator implements inverse_transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.
Call predict on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict.
Must fulfill the input assumptions of the underlying estimator.
Call predict_log_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_log_proba.
Must fulfill the input assumptions of the underlying estimator.
Call predict_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_proba.
Must fulfill the input assumptions of the underlying estimator.
Returns the score on the given data, if the estimator has been refit.
This uses the score defined by scoring where provided, and the best_estimator_.score method otherwise.
Input data, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Call score_samples on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports score_samples.
New in version 0.24.
Data to predict on. Must fulfill input requirements of the underlying estimator.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Call transform on the estimator with the best found parameters.
Only available if the underlying estimator supports transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.


Search over specified parameter values with successive halving.
The search strategy starts evaluating all the candidates with a small amount of resources and iteratively selects the best candidates, using more and more resources.
Read more in the User guide.
Note
This estimator is still experimental for now: the predictions and the API might change without any deprecation cycle. To use it, you need to explicitly import enable_halving_search_cv:
This is assumed to implement the scikit-learn estimator interface. Either estimator needs to provide a score function, or scoring must be passed.
Dictionary with parameters names (string) as keys and lists of parameter settings to try as values, or a list of such dictionaries, in which case the grids spanned by each dictionary in the list are explored. This enables searching over any sequence of parameter settings.
The ‘halving’ parameter, which determines the proportion of candidates that are selected for each subsequent iteration. For example, factor=3 means that only one third of the candidates are selected.
Defines the resource that increases with each iteration. By default, the resource is the number of samples. It can also be set to any parameter of the base estimator that accepts positive integer values, e.g. ‘n_iterations’ or ‘n_estimators’ for a gradient boosting estimator. In this case max_resources cannot be ‘auto’ and must be set explicitly.
The maximum amount of resource that any candidate is allowed to use for a given iteration. By default, this is set to n_samples when resource='n_samples' (default), else an error is raised.
The minimum amount of resource that any candidate is allowed to use for a given iteration. Equivalently, this defines the amount of resources r0 that are allocated for each candidate at the first iteration.
problem
classification problem
Note that the amount of resources used at each iteration is always a multiple of min_resources.
This is only relevant in cases where there isn’t enough resources to reduce the remaining candidates to at most factor after the last iteration. If True, then the search process will ‘replay’ the first iteration for as long as needed until the number of candidates is small enough. This is False by default, which means that the last iteration may evaluate more than factor candidates. See Aggressive elimination of candidates for more details.
Determines the cross-validation splitting strategy. Possible inputs for cv are:
For integer/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
Refer User Guide for the various cross-validation strategies that can be used here.
Note
Due to implementation details, the folds produced by cv must be the same across multiple calls to cv.split(). For built-in scikit-learn iterators, this can be achieved by deactivating shuffling (shuffle=False), or by setting the cv’s random_state parameter to an integer.
A single string (see The scoring parameter: defining model evaluation rules) or a callable (see Defining your scoring strategy from metric functions) to evaluate the predictions on the test set. If None, the estimator’s score method is used.
If True, refit an estimator using the best found parameters on the whole dataset.
The refitted estimator is made available at the best_estimator_ attribute and permits using predict directly on this HalvingGridSearchCV instance.
Value to assign to the score if an error occurs in estimator fitting. If set to ‘raise’, the error is raised. If a numeric value is given, FitFailedWarning is raised. This parameter does not affect the refit step, which will always raise the error. Default is np.nan
If False, the cv_results_ attribute will not include training scores. Computing training scores is used to get insights on how different parameter settings impact the overfitting/underfitting trade-off. However computing the scores on the training set can be computationally expensive and is not strictly required to select the parameters that yield the best generalization performance.
Pseudo random number generator state used for subsampling the dataset when resources != 'n_samples'. Ignored otherwise. Pass an int for reproducible output across multiple function calls. See Glossary.
Number of jobs to run in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Controls the verbosity: the higher, the more messages.
The amount of resources used at each iteration.
The number of candidate parameters that were evaluated at each iteration.
The number of candidate parameters that are left after the last iteration. It corresponds to ceil(n_candidates[-1] / factor)
The maximum number of resources that any candidate is allowed to use for a given iteration. Note that since the number of resources used at each iteration must be a multiple of min_resources_, the actual number of resources used at the last iteration may be smaller than max_resources_.
The amount of resources that are allocated for each candidate at the first iteration.
The actual number of iterations that were run. This is equal to n_required_iterations_ if aggressive_elimination is True. Else, this is equal to min(n_possible_iterations_,
n_required_iterations_).
The number of iterations that are possible starting with min_resources_ resources and without exceeding max_resources_.
The number of iterations that are required to end up with less than factor candidates at the last iteration, starting with min_resources_ resources. This will be smaller than n_possible_iterations_ when there isn’t enough resources.
A dict with keys as column headers and values as columns, that can be imported into a pandas DataFrame. It contains many informations for analysing the results of a search. Please refer to the User guide for details.
Estimator that was chosen by the search, i.e. estimator which gave highest score (or smallest loss if specified) on the left out data. Not available if refit=False.
Mean cross-validated score of the best_estimator.
Parameter setting that gave the best results on the hold out data.
The index (of the cv_results_ arrays) which corresponds to the best candidate parameter setting.
The dict at search.cv_results_['params'][search.best_index_] gives the parameter setting for the best model, that gives the highest mean score (search.best_score_).
Scorer function used on the held out data to choose the best parameters for the model.
The number of cross-validation splits (folds/iterations).
Seconds used for refitting the best model on the whole dataset.
This is present only if refit is not False.
See also
Random search over a set of parameters using successive halving.
The parameters selected are those that maximize the score of the held-out data, according to the scoring parameter.
decision_function(X)
Call decision_function on the estimator with the best found parameters.
fit(X[, y, groups])
Run fit with all sets of parameters.
get_params([deep])
Get parameters for this estimator.
inverse_transform(Xt)
Call inverse_transform on the estimator with the best found params.
predict(X)
Call predict on the estimator with the best found parameters.
predict_log_proba(X)
Call predict_log_proba on the estimator with the best found parameters.
predict_proba(X)
Call predict_proba on the estimator with the best found parameters.
score(X[, y])
Returns the score on the given data, if the estimator has been refit.
score_samples(X)
Call score_samples on the estimator with the best found parameters.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Call transform on the estimator with the best found parameters.
Call decision_function on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports decision_function.
Must fulfill the input assumptions of the underlying estimator.
Run fit with all sets of parameters.
Training vector, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Parameters passed to the fit method of the estimator
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Call inverse_transform on the estimator with the best found params.
Only available if the underlying estimator implements inverse_transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.
Call predict on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict.
Must fulfill the input assumptions of the underlying estimator.
Call predict_log_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_log_proba.
Must fulfill the input assumptions of the underlying estimator.
Call predict_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_proba.
Must fulfill the input assumptions of the underlying estimator.
Returns the score on the given data, if the estimator has been refit.
This uses the score defined by scoring where provided, and the best_estimator_.score method otherwise.
Input data, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Call score_samples on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports score_samples.
New in version 0.24.
Data to predict on. Must fulfill input requirements of the underlying estimator.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Call transform on the estimator with the best found parameters.
Only available if the underlying estimator supports transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.
Release Highlights for scikit-learn 0.24
Successive Halving Iterations
Comparison between grid search and successive halving


Call decision_function on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports decision_function.
Must fulfill the input assumptions of the underlying estimator.


Run fit with all sets of parameters.
Training vector, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Parameters passed to the fit method of the estimator


Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.


Call inverse_transform on the estimator with the best found params.
Only available if the underlying estimator implements inverse_transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.


Call predict on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict.
Must fulfill the input assumptions of the underlying estimator.


Call predict_log_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_log_proba.
Must fulfill the input assumptions of the underlying estimator.


Call predict_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_proba.
Must fulfill the input assumptions of the underlying estimator.


Returns the score on the given data, if the estimator has been refit.
This uses the score defined by scoring where provided, and the best_estimator_.score method otherwise.
Input data, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.


Call score_samples on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports score_samples.
New in version 0.24.
Data to predict on. Must fulfill input requirements of the underlying estimator.


Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.


Call transform on the estimator with the best found parameters.
Only available if the underlying estimator supports transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.


Randomized search on hyper parameters.
The search strategy starts evaluating all the candidates with a small amount of resources and iteratively selects the best candidates, using more and more resources.
The candidates are sampled at random from the parameter space and the number of sampled candidates is determined by n_candidates.
Read more in the User guide.
Note
This estimator is still experimental for now: the predictions and the API might change without any deprecation cycle. To use it, you need to explicitly import enable_halving_search_cv:
This is assumed to implement the scikit-learn estimator interface. Either estimator needs to provide a score function, or scoring must be passed.
Dictionary with parameters names (string) as keys and distributions or lists of parameters to try. Distributions must provide a rvs method for sampling (such as those from scipy.stats.distributions). If a list is given, it is sampled uniformly.
The number of candidate parameters to sample, at the first iteration. Using ‘exhaust’ will sample enough candidates so that the last iteration uses as many resources as possible, based on min_resources, max_resources and factor. In this case, min_resources cannot be ‘exhaust’.
The ‘halving’ parameter, which determines the proportion of candidates that are selected for each subsequent iteration. For example, factor=3 means that only one third of the candidates are selected.
Defines the resource that increases with each iteration. By default, the resource is the number of samples. It can also be set to any parameter of the base estimator that accepts positive integer values, e.g. ‘n_iterations’ or ‘n_estimators’ for a gradient boosting estimator. In this case max_resources cannot be ‘auto’ and must be set explicitly.
The maximum number of resources that any candidate is allowed to use for a given iteration. By default, this is set n_samples when resource='n_samples' (default), else an error is raised.
The minimum amount of resource that any candidate is allowed to use for a given iteration. Equivalently, this defines the amount of resources r0 that are allocated for each candidate at the first iteration.
problem
classification problem
Note that the amount of resources used at each iteration is always a multiple of min_resources.
This is only relevant in cases where there isn’t enough resources to reduce the remaining candidates to at most factor after the last iteration. If True, then the search process will ‘replay’ the first iteration for as long as needed until the number of candidates is small enough. This is False by default, which means that the last iteration may evaluate more than factor candidates. See Aggressive elimination of candidates for more details.
Determines the cross-validation splitting strategy. Possible inputs for cv are:
For integer/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
Refer User Guide for the various cross-validation strategies that can be used here.
Note
Due to implementation details, the folds produced by cv must be the same across multiple calls to cv.split(). For built-in scikit-learn iterators, this can be achieved by deactivating shuffling (shuffle=False), or by setting the cv’s random_state parameter to an integer.
A single string (see The scoring parameter: defining model evaluation rules) or a callable (see Defining your scoring strategy from metric functions) to evaluate the predictions on the test set. If None, the estimator’s score method is used.
If True, refit an estimator using the best found parameters on the whole dataset.
The refitted estimator is made available at the best_estimator_ attribute and permits using predict directly on this HalvingRandomSearchCV instance.
Value to assign to the score if an error occurs in estimator fitting. If set to ‘raise’, the error is raised. If a numeric value is given, FitFailedWarning is raised. This parameter does not affect the refit step, which will always raise the error. Default is np.nan
If False, the cv_results_ attribute will not include training scores. Computing training scores is used to get insights on how different parameter settings impact the overfitting/underfitting trade-off. However computing the scores on the training set can be computationally expensive and is not strictly required to select the parameters that yield the best generalization performance.
Pseudo random number generator state used for subsampling the dataset when resources != 'n_samples'. Also used for random uniform sampling from lists of possible values instead of scipy.stats distributions. Pass an int for reproducible output across multiple function calls. See Glossary.
Number of jobs to run in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Controls the verbosity: the higher, the more messages.
The amount of resources used at each iteration.
The number of candidate parameters that were evaluated at each iteration.
The number of candidate parameters that are left after the last iteration. It corresponds to ceil(n_candidates[-1] / factor)
The maximum number of resources that any candidate is allowed to use for a given iteration. Note that since the number of resources used at each iteration must be a multiple of min_resources_, the actual number of resources used at the last iteration may be smaller than max_resources_.
The amount of resources that are allocated for each candidate at the first iteration.
The actual number of iterations that were run. This is equal to n_required_iterations_ if aggressive_elimination is True. Else, this is equal to min(n_possible_iterations_,
n_required_iterations_).
The number of iterations that are possible starting with min_resources_ resources and without exceeding max_resources_.
The number of iterations that are required to end up with less than factor candidates at the last iteration, starting with min_resources_ resources. This will be smaller than n_possible_iterations_ when there isn’t enough resources.
A dict with keys as column headers and values as columns, that can be imported into a pandas DataFrame. It contains many informations for analysing the results of a search. Please refer to the User guide for details.
Estimator that was chosen by the search, i.e. estimator which gave highest score (or smallest loss if specified) on the left out data. Not available if refit=False.
Mean cross-validated score of the best_estimator.
Parameter setting that gave the best results on the hold out data.
The index (of the cv_results_ arrays) which corresponds to the best candidate parameter setting.
The dict at search.cv_results_['params'][search.best_index_] gives the parameter setting for the best model, that gives the highest mean score (search.best_score_).
Scorer function used on the held out data to choose the best parameters for the model.
The number of cross-validation splits (folds/iterations).
Seconds used for refitting the best model on the whole dataset.
This is present only if refit is not False.
See also
Search over a grid of parameters using successive halving.
The parameters selected are those that maximize the score of the held-out data, according to the scoring parameter.
decision_function(X)
Call decision_function on the estimator with the best found parameters.
fit(X[, y, groups])
Run fit with all sets of parameters.
get_params([deep])
Get parameters for this estimator.
inverse_transform(Xt)
Call inverse_transform on the estimator with the best found params.
predict(X)
Call predict on the estimator with the best found parameters.
predict_log_proba(X)
Call predict_log_proba on the estimator with the best found parameters.
predict_proba(X)
Call predict_proba on the estimator with the best found parameters.
score(X[, y])
Returns the score on the given data, if the estimator has been refit.
score_samples(X)
Call score_samples on the estimator with the best found parameters.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Call transform on the estimator with the best found parameters.
Call decision_function on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports decision_function.
Must fulfill the input assumptions of the underlying estimator.
Run fit with all sets of parameters.
Training vector, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Group labels for the samples used while splitting the dataset into train/test set. Only used in conjunction with a “Group” cv instance (e.g., GroupKFold).
Parameters passed to the fit method of the estimator
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Call inverse_transform on the estimator with the best found params.
Only available if the underlying estimator implements inverse_transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.
Call predict on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict.
Must fulfill the input assumptions of the underlying estimator.
Call predict_log_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_log_proba.
Must fulfill the input assumptions of the underlying estimator.
Call predict_proba on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports predict_proba.
Must fulfill the input assumptions of the underlying estimator.
Returns the score on the given data, if the estimator has been refit.
This uses the score defined by scoring where provided, and the best_estimator_.score method otherwise.
Input data, where n_samples is the number of samples and n_features is the number of features.
Target relative to X for classification or regression; None for unsupervised learning.
Call score_samples on the estimator with the best found parameters.
Only available if refit=True and the underlying estimator supports score_samples.
New in version 0.24.
Data to predict on. Must fulfill input requirements of the underlying estimator.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Call transform on the estimator with the best found parameters.
Only available if the underlying estimator supports transform and refit=True.
Must fulfill the input assumptions of the underlying estimator.