Compute Non-negative Matrix Factorization (NMF).
Find two non-negative matrices (W, H) whose product approximates the non- negative matrix X. This factorization can be used for example for dimensionality reduction, source separation or topic extraction.
The objective function is:
Where:
\(||A||_{Fro}^2 = \sum_{i,j} A_{ij}^2\) (Frobenius norm)
\(||vec(A)||_1 = \sum_{i,j} abs(A_{ij})\) (Elementwise L1 norm)
For multiplicative-update (‘mu’) solver, the Frobenius norm \((0.5 * ||X - WH||_{Fro}^2)\) can be changed into another beta-divergence loss, by changing the beta_loss parameter.
The objective function is minimized with an alternating minimization of W and H. If H is given and update_H=False, it solves for W only.
Constant matrix.
If init=’custom’, it is used as initial guess for the solution.
If init=’custom’, it is used as initial guess for the solution. If update_H=False, it is used as a constant, to solve for W only.
Number of components, if n_components is not set all features are kept.
Method used to initialize the procedure.
Valid options:
sqrt(X.mean() / n_components)
initialization (better for sparseness)
(better when sparsity is not desired)
(generally faster, less accurate alternative to NNDSVDa for when sparsity is not desired)
Changed in version 0.23: The default value of init changed from ‘random’ to None in 0.23.
Set to True, both W and H will be estimated from initial guesses. Set to False, only W will be estimated.
Numerical solver to use:
Alternating Least Squares (Fast HALS).
New in version 0.17: Coordinate Descent solver.
New in version 0.19: Multiplicative Update solver.
Beta divergence to be minimized, measuring the distance between X and the dot product WH. Note that values different from ‘frobenius’ (or 2) and ‘kullback-leibler’ (or 1) lead to significantly slower fits. Note that for beta_loss <= 0 (or ‘itakura-saito’), the input matrix X cannot contain zeros. Used only in ‘mu’ solver.
New in version 0.19.
Tolerance of the stopping condition.
Maximum number of iterations before timing out.
Constant that multiplies the regularization terms.
The regularization mixing parameter, with 0 <= l1_ratio <= 1. For l1_ratio = 0 the penalty is an elementwise L2 penalty (aka Frobenius Norm). For l1_ratio = 1 it is an elementwise L1 penalty. For 0 < l1_ratio < 1, the penalty is a combination of L1 and L2.
Select whether the regularization affects the components (H), the transformation (W), both or none of them.
Used for NMF initialisation (when init == ‘nndsvdar’ or ‘random’), and in Coordinate Descent. Pass an int for reproducible results across multiple function calls. See Glossary.
The verbosity level.
If true, randomize the order of coordinates in the CD solver.
Solution to the non-negative least squares problem.
Solution to the non-negative least squares problem.
Actual number of iterations.
Cichocki, Andrzej, and P. H. A. N. Anh-Huy. “Fast local algorithms for large scale nonnegative matrix and tensor factorizations.” IEICE transactions on fundamentals of electronics, communications and computer sciences 92.3: 708-721, 2009.
Fevotte, C., & Idier, J. (2011). Algorithms for nonnegative matrix factorization with the beta-divergence. Neural Computation, 23(9).


Sparse coding
Each row of the result is the solution to a sparse coding problem. The goal is to find a sparse array code such that:
Read more in the User Guide.
Data matrix.
The dictionary matrix against which to solve the sparse coding of the data. Some of the algorithms assume normalized rows for meaningful output.
Precomputed Gram matrix, dictionary * dictionary'.
Precomputed covariance, dictionary' * X.
The algorithm used:
Number of nonzero coefficients to target in each column of the solution. This is only used by algorithm='lars' and algorithm='omp' and is overridden by alpha in the omp case. If None, then n_nonzero_coefs=int(n_features / 10).
If algorithm='lasso_lars' or algorithm='lasso_cd', alpha is the penalty applied to the L1 norm. If algorithm='threshold', alpha is the absolute value of the threshold below which coefficients will be squashed to zero. If algorithm='omp', alpha is the tolerance parameter: the value of the reconstruction error targeted. In this case, it overrides n_nonzero_coefs. If None, default to 1.
Whether to copy the precomputed covariance matrix; if False, it may be overwritten.
Initialization value of the sparse codes. Only used if algorithm='lasso_cd'.
Maximum number of iterations to perform if algorithm='lasso_cd' or 'lasso_lars'.
Number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
If False, the input arrays X and dictionary will not be checked.
Controls the verbosity; the higher, the more messages.
Whether to enforce positivity when finding the encoding.
New in version 0.20.
The sparse codes
See also


Reshape a 2D image into a collection of patches
The resulting patches are allocated in a dedicated array.
Read more in the User Guide.
The original image data. For color images, the last dimension specifies the channel: a RGB image would have n_channels=3.
The dimensions of one patch.
The maximum number of patches to extract. If max_patches is a float between 0 and 1, it is taken to be a proportion of the total number of patches.
Determines the random number generator used for random sampling when max_patches is not None. Use an int to make the randomness deterministic. See Glossary.
The collection of patches extracted from the image, where n_patches is either max_patches or the total number of patches that can be extracted.
Online learning of a dictionary of parts of faces
Image denoising using dictionary learning


Graph of the pixel-to-pixel connections
Edges exist if 2 voxels are connected.
Dimension in x axis
Dimension in y axis
Dimension in z axis
An optional mask of the image, to consider only part of the pixels.
The class to use to build the returned adjacency matrix.
The data of the returned sparse matrix. By default it is int
For scikit-learn versions 0.14.1 and prior, return_as=np.ndarray was handled by returning a dense np.matrix instance. Going forward, np.ndarray returns an np.ndarray, as expected.
For compatibility, user code relying on this method should wrap its calls in np.asarray to avoid type issues.


Graph of the pixel-to-pixel gradient connections
Edges are weighted with the gradient values.
Read more in the User Guide.
2D or 3D image.
An optional mask of the image, to consider only part of the pixels.
The class to use to build the returned adjacency matrix.
The data of the returned sparse matrix. By default it is the dtype of img
For scikit-learn versions 0.14.1 and prior, return_as=np.ndarray was handled by returning a dense np.matrix instance. Going forward, np.ndarray returns an np.ndarray, as expected.
For compatibility, user code relying on this method should wrap its calls in np.asarray to avoid type issues.


Reconstruct the image from all of its patches.
Patches are assumed to overlap and the image is constructed by filling in the patches from left to right, top to bottom, averaging the overlapping regions.
Read more in the User Guide.
The complete set of patches. If the patches contain colour information, channels are indexed along the last dimension: RGB patches would have n_channels=3.
The size of the image that will be reconstructed.
The reconstructed image.
Image denoising using dictionary learning


Compute chi-squared stats between each non-negative feature and class.
This score can be used to select the n_features features with the highest values for the test chi-squared statistic from X, which must contain only non-negative features such as booleans or frequencies (e.g., term counts in document classification), relative to the classes.
Recall that the chi-square test measures dependence between stochastic variables, so using this function “weeds out” the features that are the most likely to be independent of class and therefore irrelevant for classification.
Read more in the User Guide.
Sample vectors.
Target vector (class labels).
chi2 statistics of each feature.
p-values of each feature.
See also
ANOVA F-value between label/feature for classification tasks.
F-value between label/feature for regression tasks.
Complexity of this algorithm is O(n_classes * n_features).
Selecting dimensionality reduction with Pipeline and GridSearchCV
SVM-Anova: SVM with univariate feature selection
Classification of text documents using sparse features


Compute the ANOVA F-value for the provided sample.
Read more in the User Guide.
The set of regressors that will be tested sequentially.
The data matrix.
The set of F values.
The set of p-values.
See also
Chi-squared stats of non-negative features for classification tasks.
F-value between label/feature for regression tasks.
Pipeline Anova SVM
Univariate Feature Selection


Univariate linear regression tests.
Linear model for testing the individual effect of each of many regressors. This is a scoring function to be used in a feature selection procedure, not a free standing feature selection procedure.
This is done in 2 steps:
For more on usage see the User Guide.
The set of regressors that will be tested sequentially.
The data matrix
If true, X and y will be centered.
F values of features.
p-values of F-scores.
See also
Mutual information for a continuous target.
ANOVA F-value between label/feature for classification tasks.
Chi-squared stats of non-negative features for classification tasks.
Select features based on the k highest scores.
Select features based on a false positive rate test.
Select features based on an estimated false discovery rate.
Select features based on family-wise error rate.
Select features based on percentile of the highest scores.
Feature agglomeration vs. univariate selection
Comparison of F-test and mutual information


Estimate mutual information for a discrete target variable.
Mutual information (MI) [1] between two random variables is a non-negative value, which measures the dependency between the variables. It is equal to zero if and only if two random variables are independent, and higher values mean higher dependency.
The function relies on nonparametric methods based on entropy estimation from k-nearest neighbors distances as described in [2] and [3]. Both methods are based on the idea originally proposed in [4].
It can be used for univariate features selection, read more in the User Guide.
Feature matrix.
Target vector.
If bool, then determines whether to consider all features discrete or continuous. If array, then it should be either a boolean mask with shape (n_features,) or array with indices of discrete features. If ‘auto’, it is assigned to False for dense X and to True for sparse X.
Number of neighbors to use for MI estimation for continuous variables, see [2] and [3]. Higher values reduce variance of the estimation, but could introduce a bias.
Whether to make a copy of the given data. If set to False, the initial data will be overwritten.
Determines random number generation for adding small noise to continuous variables in order to remove repeated values. Pass an int for reproducible results across multiple function calls. See Glossary.
Estimated mutual information between each feature and the target.
Mutual Information on Wikipedia.
A. Kraskov, H. Stogbauer and P. Grassberger, “Estimating mutual information”. Phys. Rev. E 69, 2004.
B. C. Ross “Mutual Information between Discrete and Continuous Data Sets”. PLoS ONE 9(2), 2014.
L. F. Kozachenko, N. N. Leonenko, “Sample Estimate of the Entropy of a Random Vector:, Probl. Peredachi Inf., 23:2 (1987), 9-16


Estimate mutual information for a continuous target variable.
Mutual information (MI) [1] between two random variables is a non-negative value, which measures the dependency between the variables. It is equal to zero if and only if two random variables are independent, and higher values mean higher dependency.
The function relies on nonparametric methods based on entropy estimation from k-nearest neighbors distances as described in [2] and [3]. Both methods are based on the idea originally proposed in [4].
It can be used for univariate features selection, read more in the User Guide.
Feature matrix.
Target vector.
If bool, then determines whether to consider all features discrete or continuous. If array, then it should be either a boolean mask with shape (n_features,) or array with indices of discrete features. If ‘auto’, it is assigned to False for dense X and to True for sparse X.
Number of neighbors to use for MI estimation for continuous variables, see [2] and [3]. Higher values reduce variance of the estimation, but could introduce a bias.
Whether to make a copy of the given data. If set to False, the initial data will be overwritten.
Determines random number generation for adding small noise to continuous variables in order to remove repeated values. Pass an int for reproducible results across multiple function calls. See Glossary.
Estimated mutual information between each feature and the target.
Mutual Information on Wikipedia.
A. Kraskov, H. Stogbauer and P. Grassberger, “Estimating mutual information”. Phys. Rev. E 69, 2004.
B. C. Ross “Mutual Information between Discrete and Continuous Data Sets”. PLoS ONE 9(2), 2014.
L. F. Kozachenko, N. N. Leonenko, “Sample Estimate of the Entropy of a Random Vector”, Probl. Peredachi Inf., 23:2 (1987), 9-16
Comparison of F-test and mutual information


Retrieve current values for configuration set by set_config
Keys are parameter names that can be passed to set_config.
See also
Context manager for global scikit-learn configuration.
Set global scikit-learn configuration.


Partial dependence of features.
Partial dependence of a feature (or a set of features) corresponds to the average response of an estimator for each possible value of the feature.
Read more in the User Guide.
Warning
For GradientBoostingClassifier and GradientBoostingRegressor, the 'recursion' method (used by default) will not account for the init predictor of the boosting process. In practice, this will produce the same values as 'brute' up to a constant offset in the target response, provided that init is a constant estimator (which is the default). However, if init is not a constant estimator, the partial dependence values are incorrect for 'recursion' because the offset will be sample-dependent. It is preferable to use the 'brute' method. Note that this only applies to GradientBoostingClassifier and GradientBoostingRegressor, not to HistGradientBoostingClassifier and HistGradientBoostingRegressor.
A fitted estimator object implementing predict, predict_proba, or decision_function. Multioutput-multiclass classifiers are not supported.
X is used to generate a grid of values for the target features (where the partial dependence will be evaluated), and also to generate values for the complement features when the method is ‘brute’.
The feature (e.g. [0]) or pair of interacting features (e.g. [(0, 1)]) for which the partial dependency should be computed.
Specifies whether to use predict_proba or decision_function as the target response. For regressors this parameter is ignored and the response is always the output of predict. By default, predict_proba is tried first and we revert to decision_function if it doesn’t exist. If method is ‘recursion’, the response is always the output of decision_function.
The lower and upper percentile used to create the extreme values for the grid. Must be in [0, 1].
The number of equally spaced points on the grid, for each target feature.
The method used to calculate the averaged predictions:
Please see this note for differences between the 'brute' and 'recursion' method.
Whether to return the partial dependence averaged across all the samples in the dataset or one line per sample or both. See Returns below.
Note that the fast method='recursion' option is only available for kind='average'. Plotting individual dependencies requires using the slower method='brute' option.
New in version 0.24.
Deprecated since version 0.24: kind='legacy' is deprecated and will be removed in version 1.1. kind='average' will be the new default. It is intended to migrate from the ndarray output to Bunch output.
The predictions for all the points in the grid, averaged over all samples in X (or over the training data if method is ‘recursion’).
Dictionary-like object, with the following attributes.
The predictions for all the points in the grid for all samples in X. This is also known as Individual Conditional Expectation (ICE)
The predictions for all the points in the grid, averaged over all samples in X (or over the training data if method is ‘recursion’). Only available when kind=’both’.
The values with which the grid has been created. The generated grid is a cartesian product of the arrays in values. len(values) == len(features). The size of each array values[j] is either grid_resolution, or the number of unique values in X[:, j], whichever is smaller.
n_outputs corresponds to the number of classes in a multi-class setting, or to the number of tasks for multi-output regression. For classical regression and binary classification n_outputs==1. n_values_feature_j corresponds to the size values[j].
The values with which the grid has been created. The generated grid is a cartesian product of the arrays in values. len(values) ==
len(features). The size of each array values[j] is either grid_resolution, or the number of unique values in X[:, j], whichever is smaller. Only available when kind="legacy".
See also
Plot Partial Dependence.
Partial Dependence visualization.
Partial Dependence and Individual Conditional Expectation Plots


Permutation importance for feature evaluation [BRE].
The estimator is required to be a fitted estimator. X can be the data set used to train the estimator or a hold-out set. The permutation importance of a feature is calculated as follows. First, a baseline metric, defined by scoring, is evaluated on a (potentially different) dataset defined by the X. Next, a feature column from the validation set is permuted and the metric is evaluated again. The permutation importance is defined to be the difference between the baseline metric and metric from permutating the feature column.
Read more in the User Guide.
An estimator that has already been fitted and is compatible with scorer.
Data on which permutation importance will be computed.
Targets for supervised or None for unsupervised.
Scorer to use. It can be a single string (see The scoring parameter: defining model evaluation rules) or a callable (see Defining your scoring strategy from metric functions). If None, the estimator’s default scorer is used.
Number of times to permute a feature.
Number of jobs to run in parallel. The computation is done by computing permutation score for each columns and parallelized over the columns. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Pseudo-random number generator to control the permutations of each feature. Pass an int to get reproducible results across function calls. See :term: Glossary <random_state>.
Sample weights used in scoring.
New in version 0.24.
Dictionary-like object, with the following attributes.
Mean of feature importance over n_repeats.
Standard deviation over n_repeats.
Raw permutation importance scores.
L. Breiman, “Random Forests”, Machine Learning, 45(1), 5-32, 2001. https://doi.org/10.1023/A:1010933404324
Release Highlights for scikit-learn 0.22
Feature importances with forests of trees
Gradient Boosting regression
Permutation Importance with Multicollinear or Correlated Features
Permutation Importance vs Random Forest Feature Importance (MDI)


Partial dependence (PD) and individual conditional expectation (ICE) plots.
Partial dependence plots, individual conditional expectation plots or an overlay of both of them can be plotted by setting the kind parameter. The len(features) plots are arranged in a grid with n_cols columns. Two-way partial dependence plots are plotted as contour plots. The deciles of the feature values will be shown with tick marks on the x-axes for one-way plots, and on both axes for two-way plots.
Read more in the User Guide.
Note
plot_partial_dependence does not support using the same axes with multiple calls. To plot the the partial dependence for multiple estimators, please pass the axes created by the first call to the second call:
Warning
For GradientBoostingClassifier and GradientBoostingRegressor, the 'recursion' method (used by default) will not account for the init predictor of the boosting process. In practice, this will produce the same values as 'brute' up to a constant offset in the target response, provided that init is a constant estimator (which is the default). However, if init is not a constant estimator, the partial dependence values are incorrect for 'recursion' because the offset will be sample-dependent. It is preferable to use the 'brute' method. Note that this only applies to GradientBoostingClassifier and GradientBoostingRegressor, not to HistGradientBoostingClassifier and HistGradientBoostingRegressor.
A fitted estimator object implementing predict, predict_proba, or decision_function. Multioutput-multiclass classifiers are not supported.
X is used to generate a grid of values for the target features (where the partial dependence will be evaluated), and also to generate values for the complement features when the method is 'brute'.
The target features for which to create the PDPs. If features[i] is an integer or a string, a one-way PDP is created; if features[i] is a tuple, a two-way PDP is created (only supported with kind='average'). Each tuple must be of size 2. if any entry is a string, then it must be in feature_names.
Name of each feature; feature_names[i] holds the name of the feature with index i. By default, the name of the feature corresponds to their numerical index for NumPy array and their column name for pandas dataframe.
Ignored in binary classification or classical regression settings.
Specifies whether to use predict_proba or decision_function as the target response. For regressors this parameter is ignored and the response is always the output of predict. By default, predict_proba is tried first and we revert to decision_function if it doesn’t exist. If method is 'recursion', the response is always the output of decision_function.
The maximum number of columns in the grid plot. Only active when ax is a single axis or None.
The number of equally spaced points on the axes of the plots, for each target feature.
The lower and upper percentile used to create the extreme values for the PDP axes. Must be in [0, 1].
The method used to calculate the averaged predictions:
Please see this note for differences between the 'brute' and 'recursion' method.
The number of CPUs to use to compute the partial dependences. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Verbose output during PD computations.
Dict with keywords passed to the matplotlib.pyplot.plot call. For one-way partial dependence plots.
Dict with keywords passed to the matplotlib.pyplot.contourf call. For two-way partial dependence plots.
New in version 0.22.
Whether to plot the partial dependence averaged across all the samples in the dataset or one line per sample or both.
Note that the fast method='recursion' option is only available for kind='average'. Plotting individual dependencies requires using the slower method='brute' option.
New in version 0.24.
Sampling for ICE curves when kind is ‘individual’ or ‘both’. If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to be used to plot ICE curves. If int, represents the absolute number samples to use.
Note that the full dataset is still used to calculate averaged partial dependence when kind='both'.
New in version 0.24.
Controls the randomness of the selected samples when subsamples is not None and kind is either 'both' or 'individual'. See Glossary for details.
New in version 0.24.
See also
Compute Partial Dependence values.
Partial Dependence visualization.
Release Highlights for scikit-learn 0.23
Release Highlights for scikit-learn 0.24
Monotonic Constraints
Partial Dependence and Individual Conditional Expectation Plots
Advanced Plotting With Partial Dependence


Determine whether y is monotonically correlated with x.
y is found increasing or decreasing with respect to x based on a Spearman correlation test.
Training data.
Training target.
Whether the relationship is increasing or decreasing.
The Spearman correlation coefficient is estimated from the data, and the sign of the resulting estimate is used as the result.
In the event that the 95% confidence interval based on Fisher transform spans zero, a warning is raised.
Fisher transformation. Wikipedia. https://en.wikipedia.org/wiki/Fisher_transformation


Solve the isotonic regression model.
Read more in the User Guide.
The data.
Weights on each point of the regression. If None, weight is set to 1 (equal weights).
Lower bound on the lowest predicted value (the minimum value may still be higher). If not set, defaults to -inf.
Upper bound on the highest predicted value (the maximum may still be lower). If not set, defaults to +inf.
Whether to compute y_ is increasing (if set to True) or decreasing (if set to False)
Isotonic fit of y.
“Active set algorithms for isotonic regression; A unifying framework” by Michael J. Best and Nilotpal Chakravarti, section 3.


Compute elastic net path with coordinate descent.
The elastic net optimization function varies for mono and multi-outputs.
For mono-output tasks it is:
For multi-output tasks it is:
Where:
i.e. the sum of norm of each row.
Read more in the User Guide.
Training data. Pass directly as Fortran-contiguous data to avoid unnecessary memory duplication. If y is mono-output then X can be sparse.
Target values.
Number between 0 and 1 passed to elastic net (scaling between l1 and l2 penalties). l1_ratio=1 corresponds to the Lasso.
Length of the path. eps=1e-3 means that alpha_min / alpha_max = 1e-3.
Number of alphas along the regularization path.
List of alphas where to compute the models. If None alphas are set automatically.
Whether to use a precomputed Gram matrix to speed up calculations. If set to 'auto' let us decide. The Gram matrix can also be passed as argument.
Xy = np.dot(X.T, y) that can be precomputed. It is useful only when the Gram matrix is precomputed.
If True, X will be copied; else, it may be overwritten.
The initial values of the coefficients.
Amount of verbosity.
Whether to return the number of iterations or not.
If set to True, forces coefficients to be positive. (Only allowed when y.ndim == 1).
If set to False, the input validation checks are skipped (including the Gram matrix when provided). It is assumed that they are handled by the caller.
Keyword arguments passed to the coordinate descent solver.
The alphas along the path where models are computed.
Coefficients along the path.
The dual gaps at the end of the optimization for each alpha.
The number of iterations taken by the coordinate descent optimizer to reach the specified tolerance for each alpha. (Is returned when return_n_iter is set to True).
See also
For an example, see examples/linear_model/plot_lasso_coordinate_descent_path.py.
Lasso and Elastic Net


Compute Least Angle Regression or Lasso path using LARS algorithm [1]
The optimization objective for the case method=’lasso’ is:
in the case of method=’lars’, the objective function is only known in the form of an implicit equation (see discussion in [1])
Read more in the User Guide.
Input data. Note that if X is None then the Gram matrix must be specified, i.e., cannot be None or False.
Input targets.
Xy = np.dot(X.T, y) that can be precomputed. It is useful only when the Gram matrix is precomputed.
Precomputed Gram matrix (X’ * X), if 'auto', the Gram matrix is precomputed from the given X, if there are more samples than features.
Maximum number of iterations to perform, set to infinity for no limit.
Minimum correlation along the path. It corresponds to the regularization parameter alpha parameter in the Lasso.
Specifies the returned model. Select 'lar' for Least Angle Regression, 'lasso' for the Lasso.
If False, X is overwritten.
The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. Unlike the tol parameter in some iterative optimization-based algorithms, this parameter does not control the tolerance of the optimization.
If False, Gram is overwritten.
Controls output verbosity.
If return_path==True returns the entire path, else returns only the last point of the path.
Whether to return the number of iterations.
Restrict coefficients to be >= 0. This option is only allowed with method ‘lasso’. Note that the model coefficients will not converge to the ordinary-least-squares solution for small values of alpha. Only coefficients up to the smallest alpha value (alphas_[alphas_ > 0.].min() when fit_path=True) reached by the stepwise Lars-Lasso algorithm are typically in congruence with the solution of the coordinate descent lasso_path function.
Maximum of covariances (in absolute value) at each iteration. n_alphas is either max_iter, n_features or the number of nodes in the path with alpha >= alpha_min, whichever is smaller.
Indices of active variables at the end of the path.
Coefficients along the path
Number of iterations run. Returned only if return_n_iter is set to True.
See also
“Least Angle Regression”, Efron et al. http://statweb.stanford.edu/~tibs/ftp/lars.pdf
Wikipedia entry on the Least-angle regression
Wikipedia entry on the Lasso
Lasso path using LARS


lars_path in the sufficient stats mode [1]
The optimization objective for the case method=’lasso’ is:
in the case of method=’lars’, the objective function is only known in the form of an implicit equation (see discussion in [1])
Read more in the User Guide.
Xy = np.dot(X.T, y).
Gram = np.dot(X.T * X).
Equivalent size of sample.
Maximum number of iterations to perform, set to infinity for no limit.
Minimum correlation along the path. It corresponds to the regularization parameter alpha parameter in the Lasso.
Specifies the returned model. Select 'lar' for Least Angle Regression, 'lasso' for the Lasso.
If False, X is overwritten.
The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. Unlike the tol parameter in some iterative optimization-based algorithms, this parameter does not control the tolerance of the optimization.
If False, Gram is overwritten.
Controls output verbosity.
If return_path==True returns the entire path, else returns only the last point of the path.
Whether to return the number of iterations.
Restrict coefficients to be >= 0. This option is only allowed with method ‘lasso’. Note that the model coefficients will not converge to the ordinary-least-squares solution for small values of alpha. Only coefficients up to the smallest alpha value (alphas_[alphas_ > 0.].min() when fit_path=True) reached by the stepwise Lars-Lasso algorithm are typically in congruence with the solution of the coordinate descent lasso_path function.
Maximum of covariances (in absolute value) at each iteration. n_alphas is either max_iter, n_features or the number of nodes in the path with alpha >= alpha_min, whichever is smaller.
Indices of active variables at the end of the path.
Coefficients along the path
Number of iterations run. Returned only if return_n_iter is set to True.
See also
“Least Angle Regression”, Efron et al. http://statweb.stanford.edu/~tibs/ftp/lars.pdf
Wikipedia entry on the Least-angle regression
Wikipedia entry on the Lasso


Compute Lasso path with coordinate descent
The Lasso optimization function varies for mono and multi-outputs.
For mono-output tasks it is:
For multi-output tasks it is:
Where:
i.e. the sum of norm of each row.
Read more in the User Guide.
Training data. Pass directly as Fortran-contiguous data to avoid unnecessary memory duplication. If y is mono-output then X can be sparse.
Target values
Length of the path. eps=1e-3 means that alpha_min / alpha_max = 1e-3
Number of alphas along the regularization path
List of alphas where to compute the models. If None alphas are set automatically
Whether to use a precomputed Gram matrix to speed up calculations. If set to 'auto' let us decide. The Gram matrix can also be passed as argument.
Xy = np.dot(X.T, y) that can be precomputed. It is useful only when the Gram matrix is precomputed.
If True, X will be copied; else, it may be overwritten.
The initial values of the coefficients.
Amount of verbosity.
whether to return the number of iterations or not.
If set to True, forces coefficients to be positive. (Only allowed when y.ndim == 1).
keyword arguments passed to the coordinate descent solver.
The alphas along the path where models are computed.
Coefficients along the path.
The dual gaps at the end of the optimization for each alpha.
The number of iterations taken by the coordinate descent optimizer to reach the specified tolerance for each alpha.
See also
For an example, see examples/linear_model/plot_lasso_coordinate_descent_path.py.
To avoid unnecessary memory duplication the X argument of the fit method should be directly passed as a Fortran-contiguous numpy array.
Note that in certain cases, the Lars solver may be significantly faster to implement this functionality. In particular, linear interpolation can be used to retrieve model coefficients between the values output by lars_path
Comparing lasso_path and lars_path with interpolation:
Lasso and Elastic Net


Orthogonal Matching Pursuit (OMP).
Solves n_targets Orthogonal Matching Pursuit problems. An instance of the problem has the form:
When parametrized by the number of non-zero coefficients using n_nonzero_coefs: argmin ||y - Xgamma||^2 subject to ||gamma||_0 <= n_{nonzero coefs}
When parametrized by error using the parameter tol: argmin ||gamma||_0 subject to ||y - Xgamma||^2 <= tol
Read more in the User Guide.
Input data. Columns are assumed to have unit norm.
Input targets.
Desired number of non-zero entries in the solution. If None (by default) this value is set to 10% of n_features.
Maximum norm of the residual. If not None, overrides n_nonzero_coefs.
Whether to perform precomputations. Improves performance when n_targets or n_samples is very large.
Whether the design matrix X must be copied by the algorithm. A false value is only helpful if X is already Fortran-ordered, otherwise a copy is made anyway.
Whether to return every value of the nonzero coefficients along the forward path. Useful for cross-validation.
Whether or not to return the number of iterations.
Coefficients of the OMP solution. If return_path=True, this contains the whole coefficient path. In this case its shape is (n_features, n_features) or (n_features, n_targets, n_features) and iterating over the last axis yields coefficients in increasing order of active features.
Number of active features across every target. Returned only if return_n_iter is set to True.
See also
Orthogonal matching pursuit was introduced in S. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Transactions on Signal Processing, Vol. 41, No. 12. (December 1993), pp. 3397-3415. (http://blanche.polytechnique.fr/~mallat/papiers/MallatPursuit93.pdf)
This implementation is based on Rubinstein, R., Zibulevsky, M. and Elad, M., Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit Technical Report - CS Technion, April 2008. https://www.cs.technion.ac.il/~ronrubin/Publications/KSVD-OMP-v2.pdf


Gram Orthogonal Matching Pursuit (OMP).
Solves n_targets Orthogonal Matching Pursuit problems using only the Gram matrix X.T * X and the product X.T * y.
Read more in the User Guide.
Gram matrix of the input data: X.T * X.
Input targets multiplied by X: X.T * y.
Desired number of non-zero entries in the solution. If None (by default) this value is set to 10% of n_features.
Maximum norm of the residual. If not None, overrides n_nonzero_coefs.
Squared L2 norms of the lines of y. Required if tol is not None.
Whether the gram matrix must be copied by the algorithm. A false value is only helpful if it is already Fortran-ordered, otherwise a copy is made anyway.
Whether the covariance vector Xy must be copied by the algorithm. If False, it may be overwritten.
Whether to return every value of the nonzero coefficients along the forward path. Useful for cross-validation.
Whether or not to return the number of iterations.
Coefficients of the OMP solution. If return_path=True, this contains the whole coefficient path. In this case its shape is (n_features, n_features) or (n_features, n_targets, n_features) and iterating over the last axis yields coefficients in increasing order of active features.
Number of active features across every target. Returned only if return_n_iter is set to True.
See also
Orthogonal matching pursuit was introduced in G. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries, IEEE Transactions on Signal Processing, Vol. 41, No. 12. (December 1993), pp. 3397-3415. (http://blanche.polytechnique.fr/~mallat/papiers/MallatPursuit93.pdf)
This implementation is based on Rubinstein, R., Zibulevsky, M. and Elad, M., Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit Technical Report - CS Technion, April 2008. https://www.cs.technion.ac.il/~ronrubin/Publications/KSVD-OMP-v2.pdf


Passive Aggressive Regressor
Read more in the User Guide.
Maximum step size (regularization). Defaults to 1.0.
Whether the intercept should be estimated or not. If False, the data is assumed to be already centered. Defaults to True.
The maximum number of passes over the training data (aka epochs). It only impacts the behavior in the fit method, and not the partial_fit method.
New in version 0.19.
The stopping criterion. If it is not None, the iterations will stop when (loss > previous_loss - tol).
New in version 0.19.
Whether to use early stopping to terminate training when validation. score is not improving. If set to True, it will automatically set aside a fraction of training data as validation and terminate training when validation score is not improving by at least tol for n_iter_no_change consecutive epochs.
New in version 0.20.
The proportion of training data to set aside as validation set for early stopping. Must be between 0 and 1. Only used if early_stopping is True.
New in version 0.20.
Number of iterations with no improvement to wait before early stopping.
New in version 0.20.
Whether or not the training data should be shuffled after each epoch.
The verbosity level
The loss function to be used: epsilon_insensitive: equivalent to PA-I in the reference paper. squared_epsilon_insensitive: equivalent to PA-II in the reference paper.
If the difference between the current prediction and the correct label is below this threshold, the model is not updated.
Used to shuffle the training data, when shuffle is set to True. Pass an int for reproducible output across multiple function calls. See Glossary.
When set to True, reuse the solution of the previous call to fit as initialization, otherwise, just erase the previous solution. See the Glossary.
Repeatedly calling fit or partial_fit when warm_start is True can result in a different solution than when calling fit a single time because of the way the data is shuffled.
When set to True, computes the averaged SGD weights and stores the result in the coef_ attribute. If set to an int greater than 1, averaging will begin once the total number of samples seen reaches average. So average=10 will begin averaging after seeing 10 samples.
New in version 0.19: parameter average to use weights averaging in SGD
Weights assigned to the features.
Constants in decision function.
The actual number of iterations to reach the stopping criterion.
Number of weight updates performed during training. Same as (n_iter_ * n_samples).
See also
Online Passive-Aggressive Algorithms <http://jmlr.csail.mit.edu/papers/volume7/crammer06a/crammer06a.pdf> K. Crammer, O. Dekel, J. Keshat, S. Shalev-Shwartz, Y. Singer - JMLR (2006)


Solve the ridge equation by the method of normal equations.
Read more in the User Guide.
Training data
Target values
Regularization strength; must be a positive float. Regularization improves the conditioning of the problem and reduces the variance of the estimates. Larger values specify stronger regularization. Alpha corresponds to 1 / (2C) in other linear models such as LogisticRegression or LinearSVC. If an array is passed, penalties are assumed to be specific to the targets. Hence they must correspond in number.
Individual weights for each sample. If given a float, every sample will have the same weight. If sample_weight is not None and solver=’auto’, the solver will be set to ‘cholesky’.
New in version 0.17.
Solver to use in the computational routines:
All last five solvers support both dense and sparse data. However, only ‘sag’ and ‘sparse_cg’ supports sparse input when fit_intercept is True.
New in version 0.17: Stochastic Average Gradient descent solver.
New in version 0.19: SAGA solver.
Maximum number of iterations for conjugate gradient solver. For the ‘sparse_cg’ and ‘lsqr’ solvers, the default value is determined by scipy.sparse.linalg. For ‘sag’ and saga solver, the default value is 1000.
Precision of the solution.
Verbosity level. Setting verbose > 0 will display additional information depending on the solver used.
Used when solver == ‘sag’ or ‘saga’ to shuffle the data. See Glossary for details.
If True, the method also returns n_iter, the actual number of iteration performed by the solver.
New in version 0.17.
If True and if X is sparse, the method also returns the intercept, and the solver is automatically changed to ‘sag’. This is only a temporary fix for fitting the intercept with sparse data. For dense data, use sklearn.linear_model._preprocess_data before your regression.
New in version 0.17.
If False, the input arrays X and y will not be checked.
New in version 0.21.
Weight vector(s).
The actual number of iteration performed by the solver. Only returned if return_n_iter is True.
The intercept of the model. Only returned if return_intercept is True and if X is a scipy sparse array.
This function won’t compute the intercept.


Perform a Locally Linear Embedding analysis on the data.
Read more in the User Guide.
Sample data, shape = (n_samples, n_features), in the form of a numpy array or a NearestNeighbors object.
number of neighbors to consider for each point.
number of coordinates for the manifold.
regularization constant, multiplies the trace of the local covariance matrix of the distances.
auto : algorithm will attempt to choose the best method for input data
For this method, M may be a dense matrix, sparse matrix, or general linear operator. Warning: ARPACK can be unstable for some problems. It is best to try several random seeds in order to check results.
decomposition. For this method, M must be an array or matrix type. This method should be avoided for large problems.
Tolerance for ‘arpack’ method Not used if eigen_solver==’dense’.
maximum number of iterations for the arpack solver.
see reference [1]
n_neighbors > n_components * (1 + (n_components + 1) / 2. see reference [2]
see reference [3]
see reference [4]
Tolerance for Hessian eigenmapping method. Only used if method == ‘hessian’
Tolerance for modified LLE method. Only used if method == ‘modified’
Determines the random number generator when solver == ‘arpack’. Pass an int for reproducible results across multiple function calls. See :term: Glossary <random_state>.
The number of parallel jobs to run for neighbors search. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Embedding vectors.
Reconstruction error for the embedding vectors. Equivalent to norm(Y - W Y, 'fro')**2, where W are the reconstruction weights.
Roweis, S. & Saul, L. Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323 (2000).
Donoho, D. & Grimes, C. Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. Proc Natl Acad Sci U S A. 100:5591 (2003).
Zhang, Z. & Wang, J. MLLE: Modified Locally Linear Embedding Using Multiple Weights. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.70.382
Zhang, Z. & Zha, H. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. Journal of Shanghai Univ. 8:406 (2004)
Swiss Roll reduction with LLE


Computes multidimensional scaling using the SMACOF algorithm.
The SMACOF (Scaling by MAjorizing a COmplicated Function) algorithm is a multidimensional scaling algorithm which minimizes an objective function (the stress) using a majorization technique. Stress majorization, also known as the Guttman Transform, guarantees a monotone convergence of stress, and is more powerful than traditional techniques such as gradient descent.
The SMACOF algorithm for metric MDS can summarized by the following steps:
The nonmetric algorithm adds a monotonic regression step before computing the stress.
Pairwise dissimilarities between the points. Must be symmetric.
Compute metric or nonmetric SMACOF algorithm.
Number of dimensions in which to immerse the dissimilarities. If an init array is provided, this option is overridden and the shape of init is used to determine the dimensionality of the embedding space.
Starting configuration of the embedding to initialize the algorithm. By default, the algorithm is initialized with a randomly chosen array.
Number of times the SMACOF algorithm will be run with different initializations. The final results will be the best output of the runs, determined by the run with the smallest final stress. If init is provided, this option is overridden and a single run is performed.
The number of jobs to use for the computation. If multiple initializations are used (n_init), each run of the algorithm is computed in parallel.
None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Maximum number of iterations of the SMACOF algorithm for a single run.
Level of verbosity.
Relative tolerance with respect to stress at which to declare convergence.
Determines the random number generator used to initialize the centers. Pass an int for reproducible results across multiple function calls. See :term: Glossary <random_state>.
Whether or not to return the number of iterations.
Coordinates of the points in a n_components-space.
The final value of the stress (sum of squared distance of the disparities and the distances for all constrained points).
The number of iterations corresponding to the best stress. Returned only if return_n_iter is set to True.
“Modern Multidimensional Scaling - Theory and Applications” Borg, I.; Groenen P. Springer Series in Statistics (1997)
“Nonmetric multidimensional scaling: a numerical method” Kruskal, J. Psychometrika, 29 (1964)
“Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis” Kruskal, J. Psychometrika, 29, (1964)


Project the sample on the first eigenvectors of the graph Laplacian.
The adjacency matrix is used to compute a normalized graph Laplacian whose spectrum (especially the eigenvectors associated to the smallest eigenvalues) has an interpretation in terms of minimal number of cuts necessary to split the graph into comparably sized components.
This embedding can also ‘work’ even if the adjacency variable is not strictly the adjacency matrix of a graph but more generally an affinity or similarity matrix between samples (for instance the heat kernel of a euclidean distance matrix or a k-NN matrix).
However care must taken to always make the affinity matrix symmetric so that the eigenvector decomposition works as expected.
Note : Laplacian Eigenmaps is the actual algorithm implemented here.
Read more in the User Guide.
The adjacency matrix of the graph to embed.
The dimension of the projection subspace.
The eigenvalue decomposition strategy to use. AMG requires pyamg to be installed. It can be faster on very large, sparse problems, but may also lead to instabilities. If None, then 'arpack' is used.
Determines the random number generator used for the initialization of the lobpcg eigenvectors decomposition when solver == ‘amg’. Pass an int for reproducible results across multiple function calls. See :term: Glossary <random_state>.
Stopping criterion for eigendecomposition of the Laplacian matrix when using arpack eigen_solver.
If True, then compute normalized Laplacian.
Whether to drop the first eigenvector. For spectral embedding, this should be True as the first eigenvector should be constant vector for connected graph, but for spectral clustering, this should be kept as False to retain the first eigenvector.
The reduced samples.
Spectral Embedding (Laplacian Eigenmaps) is most useful when the graph has one connected component. If there graph has many components, the first few eigenvectors will simply uncover the connected components of the graph.


Expresses to what extent the local structure is retained.
The trustworthiness is within [0, 1]. It is defined as
where for each sample i, \(\mathcal{N}_{i}^{k}\) are its k nearest neighbors in the output space, and every sample j is its \(r(i, j)\)-th nearest neighbor in the input space. In other words, any unexpected nearest neighbors in the output space are penalised in proportion to their rank in the input space.
If the metric is ‘precomputed’ X must be a square distance matrix. Otherwise it contains a sample per row.
Embedding of the training data in low-dimensional space.
Number of neighbors k that will be considered.
Which metric to use for computing pairwise distances between samples from the original input space. If metric is ‘precomputed’, X must be a matrix of pairwise distances or squared distances. Otherwise, see the documentation of argument metric in sklearn.pairwise.pairwise_distances for a list of available metrics.
New in version 0.20.
Trustworthiness of the low-dimensional embedding.


Accuracy classification score.
In multilabel classification, this function computes subset accuracy: the set of labels predicted for a sample must exactly match the corresponding set of labels in y_true.
Read more in the User Guide.
Ground truth (correct) labels.
Predicted labels, as returned by a classifier.
If False, return the number of correctly classified samples. Otherwise, return the fraction of correctly classified samples.
Sample weights.
If normalize == True, return the fraction of correctly classified samples (float), else returns the number of correctly classified samples (int).
The best performance is 1 with normalize == True and the number of samples with normalize == False.
See also
In binary and multiclass classification, this function is equal to the jaccard_score function.
In the multilabel case with binary label indicators:
Plot classification probability
Multi-class AdaBoosted Decision Trees
Probabilistic predictions with Gaussian process classification (GPC)
Demonstration of multi-metric evaluation on cross_val_score and GridSearchCV
Importance of Feature Scaling
Effect of varying threshold for self-training
Classification of text documents using sparse features


Adjusted Mutual Information between two clusterings.
Adjusted Mutual Information (AMI) is an adjustment of the Mutual Information (MI) score to account for chance. It accounts for the fact that the MI is generally higher for two clusterings with a larger number of clusters, regardless of whether there is actually more information shared. For two clusterings \(U\) and \(V\), the AMI is given as:
This metric is independent of the absolute values of the labels: a permutation of the class or cluster label values won’t change the score value in any way.
This metric is furthermore symmetric: switching label_true with label_pred will return the same score value. This can be useful to measure the agreement of two independent label assignments strategies on the same dataset when the real ground truth is not known.
Be mindful that this function is an order of magnitude slower than other metrics, such as the Adjusted Rand Index.
Read more in the User Guide.
A clustering of the data into disjoint subsets.
A clustering of the data into disjoint subsets.
How to compute the normalizer in the denominator. Possible options are ‘min’, ‘geometric’, ‘arithmetic’, and ‘max’.
New in version 0.20.
Changed in version 0.22: The default value of average_method changed from ‘max’ to ‘arithmetic’.
The AMI returns a value of 1 when the two partitions are identical (ie perfectly matched). Random partitions (independent labellings) have an expected AMI around 0 on average hence can be negative.
See also
Adjusted Rand Index.
Mutual Information (not adjusted for chance).
Vinh, Epps, and Bailey, (2010). Information Theoretic Measures for Clusterings Comparison: Variants, Properties, Normalization and Correction for Chance, JMLR
Wikipedia entry for the Adjusted Mutual Information
Perfect labelings are both homogeneous and complete, hence have score 1.0:
If classes members are completely split across different clusters, the assignment is totally in-complete, hence the AMI is null:
Demo of affinity propagation clustering algorithm
Demo of DBSCAN clustering algorithm
Adjustment for chance in clustering performance evaluation
A demo of K-Means clustering on the handwritten digits data


Rand index adjusted for chance.
The Rand Index computes a similarity measure between two clusterings by considering all pairs of samples and counting pairs that are assigned in the same or different clusters in the predicted and true clusterings.
The raw RI score is then “adjusted for chance” into the ARI score using the following scheme:
The adjusted Rand index is thus ensured to have a value close to 0.0 for random labeling independently of the number of clusters and samples and exactly 1.0 when the clusterings are identical (up to a permutation).
ARI is a symmetric measure:
Read more in the User Guide.
Ground truth class labels to be used as a reference
Cluster labels to evaluate
Similarity score between -1.0 and 1.0. Random labelings have an ARI close to 0.0. 1.0 stands for perfect match.
See also
Adjusted Mutual Information.
L. Hubert and P. Arabie, Comparing Partitions, Journal of Classification 1985 https://link.springer.com/article/10.1007%2FBF01908075
D. Steinley, Properties of the Hubert-Arabie adjusted Rand index, Psychological Methods 2004
https://en.wikipedia.org/wiki/Rand_index#Adjusted_Rand_index
Perfectly matching labelings have a score of 1 even
Labelings that assign all classes members to the same clusters are complete but may not always be pure, hence penalized:
ARI is symmetric, so labelings that have pure clusters with members coming from the same classes but unnecessary splits are penalized:
If classes members are completely split across different clusters, the assignment is totally incomplete, hence the ARI is very low:
Demo of affinity propagation clustering algorithm
Demo of DBSCAN clustering algorithm
Adjustment for chance in clustering performance evaluation
A demo of K-Means clustering on the handwritten digits data
Clustering text documents using k-means


Compute Area Under the Curve (AUC) using the trapezoidal rule.
This is a general function, given points on a curve. For computing the area under the ROC-curve, see roc_auc_score. For an alternative way to summarize a precision-recall curve, see average_precision_score.
x coordinates. These must be either monotonic increasing or monotonic decreasing.
y coordinates.
See also
Compute the area under the ROC curve.
Compute average precision from prediction scores.
Compute precision-recall pairs for different probability thresholds.
Species distribution modeling
Poisson regression and non-normal loss
Tweedie regression on insurance claims
Receiver Operating Characteristic (ROC) with cross validation
Receiver Operating Characteristic (ROC)
Precision-Recall


Compute average precision (AP) from prediction scores.
AP summarizes a precision-recall curve as the weighted mean of precisions achieved at each threshold, with the increase in recall from the previous threshold used as the weight:
where \(P_n\) and \(R_n\) are the precision and recall at the nth threshold [1]. This implementation is not interpolated and is different from computing the area under the precision-recall curve with the trapezoidal rule, which uses linear interpolation and can be too optimistic.
Note: this implementation is restricted to the binary classification task or multilabel classification task.
Read more in the User Guide.
True binary labels or binary label indicators.
Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by decision_function on some classifiers).
If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data:
Calculate metrics globally by considering each element of the label indicator matrix as a label.
Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.
Calculate metrics for each label, and find their average, weighted by support (the number of true instances for each label).
Calculate metrics for each instance, and find their average.
Will be ignored when y_true is binary.
The label of the positive class. Only applied to binary y_true. For multilabel-indicator y_true, pos_label is fixed to 1.
Sample weights.
See also
Compute the area under the ROC curve.
Compute precision-recall pairs for different probability thresholds.
Changed in version 0.19: Instead of linearly interpolating between operating points, precisions are weighted by the change in recall since the last operating point.
Wikipedia entry for the Average precision
Precision-Recall


Compute the balanced accuracy.
The balanced accuracy in binary and multiclass classification problems to deal with imbalanced datasets. It is defined as the average of recall obtained on each class.
The best value is 1 and the worst value is 0 when adjusted=False.
Read more in the User Guide.
New in version 0.20.
Ground truth (correct) target values.
Estimated targets as returned by a classifier.
Sample weights.
When true, the result is adjusted for chance, so that random performance would score 0, and perfect performance scores 1.
See also
Some literature promotes alternative definitions of balanced accuracy. Our definition is equivalent to accuracy_score with class-balanced sample weights, and shares desirable properties with the binary case. See the User Guide.
Brodersen, K.H.; Ong, C.S.; Stephan, K.E.; Buhmann, J.M. (2010). The balanced accuracy and its posterior distribution. Proceedings of the 20th International Conference on Pattern Recognition, 3121-24.
John. D. Kelleher, Brian Mac Namee, Aoife D’Arcy, (2015). Fundamentals of Machine Learning for Predictive Data Analytics: Algorithms, Worked Examples, and Case Studies.


Compute the Brier score loss.
The smaller the Brier score loss, the better, hence the naming with “loss”. The Brier score measures the mean squared difference between the predicted probability and the actual outcome. The Brier score always takes on a value between zero and one, since this is the largest possible difference between a predicted probability (which must be between zero and one) and the actual outcome (which can take on values of only 0 and 1). It can be decomposed is the sum of refinement loss and calibration loss.
The Brier score is appropriate for binary and categorical outcomes that can be structured as true or false, but is inappropriate for ordinal variables which can take on three or more values (this is because the Brier score assumes that all possible outcomes are equivalently “distant” from one another). Which label is considered to be the positive label is controlled via the parameter pos_label, which defaults to the greater label unless y_true is all 0 or all -1, in which case pos_label defaults to 1.
Read more in the User Guide.
True targets.
Probabilities of the positive class.
Sample weights.
Label of the positive class. pos_label will be infered in the following manner:
Brier score loss.
Wikipedia entry for the Brier score.
Probability Calibration curves
Probability calibration of classifiers


Compute the Calinski and Harabasz score.
It is also known as the Variance Ratio Criterion.
The score is defined as ratio between the within-cluster dispersion and the between-cluster dispersion.
Read more in the User Guide.
A list of n_features-dimensional data points. Each row corresponds to a single data point.
Predicted labels for each sample.
The resulting Calinski-Harabasz score.
T. Calinski and J. Harabasz, 1974. “A dendrite method for cluster analysis”. Communications in Statistics


Determine scorer from user options.
A TypeError will be thrown if the estimator cannot be scored.
The object to use to fit the data.
A string (see model evaluation documentation) or a scorer callable object / function with signature scorer(estimator, X, y).
If no scoring is specified and the estimator has no score function, we can either return None or raise an exception.
A scorer callable object / function with signature scorer(estimator, X, y).


Build a text report showing the main classification metrics.
Read more in the User Guide.
Ground truth (correct) target values.
Estimated targets as returned by a classifier.
Optional list of label indices to include in the report.
Optional display names matching the labels (same order).
Sample weights.
Number of digits for formatting output floating point values. When output_dict is True, this will be ignored and the returned values will not be rounded.
If True, return output as dict.
New in version 0.20.
Sets the value to return when there is a zero division. If set to “warn”, this acts as 0, but warnings are also raised.
Text summary of the precision, recall, F1 score for each class. Dictionary returned if output_dict is True. Dictionary has the following structure:
The reported averages include macro average (averaging the unweighted mean per label), weighted average (averaging the support-weighted mean per label), and sample average (only for multilabel classification). Micro average (averaging the total true positives, false negatives and false positives) is only shown for multi-label or multi-class with a subset of classes, because it corresponds to accuracy otherwise and would be the same for all metrics. See also precision_recall_fscore_support for more details on averages.
Note that in binary classification, recall of the positive class is also known as “sensitivity”; recall of the negative class is “specificity”.
See also
Recognizing hand-written digits
Faces recognition example using eigenfaces and SVMs
Pipeline Anova SVM
Parameter estimation using grid search with cross-validation
Restricted Boltzmann Machine features for digit classification
Column Transformer with Heterogeneous Data Sources
Label Propagation digits: Demonstrating performance
Label Propagation digits active learning
Classification of text documents using sparse features


Build a contingency matrix describing the relationship between labels.
Ground truth class labels to be used as a reference.
Cluster labels to evaluate.
If a float, that value is added to all values in the contingency matrix. This helps to stop NaN propagation. If None, nothing is adjusted.
If True, return a sparse CSR continency matrix. If eps is not None and sparse is True will raise ValueError.
New in version 0.18.
Output dtype. Ignored if eps is not None.
New in version 0.24.
Matrix \(C\) such that \(C_{i, j}\) is the number of samples in true class \(i\) and in predicted class \(j\). If eps is None, the dtype of this array will be integer unless set otherwise with the dtype argument. If eps is given, the dtype will be float. Will be a sklearn.sparse.csr_matrix if sparse=True.


Pair confusion matrix arising from two clusterings.
The pair confusion matrix \(C\) computes a 2 by 2 similarity matrix between two clusterings by considering all pairs of samples and counting pairs that are assigned into the same or into different clusters under the true and predicted clusterings.
Considering a pair of samples that is clustered together a positive pair, then as in binary classification the count of true negatives is \(C_{00}\), false negatives is \(C_{10}\), true positives is \(C_{11}\) and false positives is \(C_{01}\).
Read more in the User Guide.
Ground truth class labels to be used as a reference.
Cluster labels to evaluate.
The contingency matrix.
See also
Rand Score
Adjusted Rand Score
Adjusted Mutual Information
Perfectly matching labelings have all non-zero entries on the diagonal regardless of actual label values:
Labelings that assign all classes members to the same clusters are complete but may be not always pure, hence penalized, and have some off-diagonal non-zero entries:
Note that the matrix is not symmetric.


Cohen’s kappa: a statistic that measures inter-annotator agreement.
This function computes Cohen’s kappa [1], a score that expresses the level of agreement between two annotators on a classification problem. It is defined as
where \(p_o\) is the empirical probability of agreement on the label assigned to any sample (the observed agreement ratio), and \(p_e\) is the expected agreement when both annotators assign labels randomly. \(p_e\) is estimated using a per-annotator empirical prior over the class labels [2].
Read more in the User Guide.
Labels assigned by the first annotator.
Labels assigned by the second annotator. The kappa statistic is symmetric, so swapping y1 and y2 doesn’t change the value.
List of labels to index the matrix. This may be used to select a subset of labels. If None, all labels that appear at least once in y1 or y2 are used.
Weighting type to calculate the score. None means no weighted; “linear” means linear weighted; “quadratic” means quadratic weighted.
Sample weights.
The kappa statistic, which is a number between -1 and 1. The maximum value means complete agreement; zero or lower means chance agreement.
J. Cohen (1960). “A coefficient of agreement for nominal scales”. Educational and Psychological Measurement 20(1):37-46. doi:10.1177/001316446002000104.
R. Artstein and M. Poesio (2008). “Inter-coder agreement for computational linguistics”. Computational Linguistics 34(4):555-596.
Wikipedia entry for the Cohen’s kappa.


Completeness metric of a cluster labeling given a ground truth.
A clustering result satisfies completeness if all the data points that are members of a given class are elements of the same cluster.
This metric is independent of the absolute values of the labels: a permutation of the class or cluster label values won’t change the score value in any way.
This metric is not symmetric: switching label_true with label_pred will return the homogeneity_score which will be different in general.
Read more in the User Guide.
ground truth class labels to be used as a reference
cluster labels to evaluate
score between 0.0 and 1.0. 1.0 stands for perfectly complete labeling
See also
Andrew Rosenberg and Julia Hirschberg, 2007. V-Measure: A conditional entropy-based external cluster evaluation measure
Perfect labelings are complete:
Non-perfect labelings that assign all classes members to the same clusters are still complete:
If classes members are split across different clusters, the assignment cannot be complete:
Release Highlights for scikit-learn 0.23
Demo of affinity propagation clustering algorithm
Demo of DBSCAN clustering algorithm
A demo of K-Means clustering on the handwritten digits data
Clustering text documents using k-means


Compute confusion matrix to evaluate the accuracy of a classification.
By definition a confusion matrix \(C\) is such that \(C_{i, j}\) is equal to the number of observations known to be in group \(i\) and predicted to be in group \(j\).
Thus in binary classification, the count of true negatives is \(C_{0,0}\), false negatives is \(C_{1,0}\), true positives is \(C_{1,1}\) and false positives is \(C_{0,1}\).
Read more in the User Guide.
Ground truth (correct) target values.
Estimated targets as returned by a classifier.
List of labels to index the matrix. This may be used to reorder or select a subset of labels. If None is given, those that appear at least once in y_true or y_pred are used in sorted order.
Sample weights.
New in version 0.18.
Normalizes confusion matrix over the true (rows), predicted (columns) conditions or all the population. If None, confusion matrix will not be normalized.
Confusion matrix whose i-th row and j-th column entry indicates the number of samples with true label being i-th class and predicted label being j-th class.
See also
Plot Confusion Matrix.
Confusion Matrix visualization.
Wikipedia entry for the Confusion matrix (Wikipedia and other references may use a different convention for axes).
In the binary case, we can extract true positives, etc as follows:
Faces recognition example using eigenfaces and SVMs
Visualizations with Display Objects
Label Propagation digits: Demonstrating performance
Label Propagation digits active learning
Classification of text documents using sparse features


The similarity of two sets of biclusters.
Similarity between individual biclusters is computed. Then the best matching between sets is found using the Hungarian algorithm. The final score is the sum of similarities divided by the size of the larger set.
Read more in the User Guide.
Tuple of row and column indicators for a set of biclusters.
Another set of biclusters like a.
May be the string “jaccard” to use the Jaccard coefficient, or any function that takes four arguments, each of which is a 1d indicator vector: (a_rows, a_columns, b_rows, b_columns).
A demo of the Spectral Co-Clustering algorithm
A demo of the Spectral Biclustering algorithm


Coverage error measure.
Compute how far we need to go through the ranked scores to cover all true labels. The best value is equal to the average number of labels in y_true per sample.
Ties in y_scores are broken by giving maximal rank that would have been assigned to all tied values.
Note: Our implementation’s score is 1 greater than the one given in Tsoumakas et al., 2010. This extends it to handle the degenerate case in which an instance has 0 true labels.
Read more in the User Guide.
True binary labels in binary indicator format.
Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by “decision_function” on some classifiers).
Sample weights.
Tsoumakas, G., Katakis, I., & Vlahavas, I. (2010). Mining multi-label data. In Data mining and knowledge discovery handbook (pp. 667-685). Springer US.


Computes the Davies-Bouldin score.
The score is defined as the average similarity measure of each cluster with its most similar cluster, where similarity is the ratio of within-cluster distances to between-cluster distances. Thus, clusters which are farther apart and less dispersed will result in a better score.
The minimum score is zero, with lower values indicating better clustering.
Read more in the User Guide.
New in version 0.20.
A list of n_features-dimensional data points. Each row corresponds to a single data point.
Predicted labels for each sample.
The resulting Davies-Bouldin score.
Davies, David L.; Bouldin, Donald W. (1979). “A Cluster Separation Measure”. IEEE Transactions on Pattern Analysis and Machine Intelligence. PAMI-1 (2): 224-227


Compute Discounted Cumulative Gain.
Sum the true scores ranked in the order induced by the predicted scores, after applying a logarithmic discount.
This ranking metric yields a high value if true labels are ranked high by y_score.
Usually the Normalized Discounted Cumulative Gain (NDCG, computed by ndcg_score) is preferred.
True targets of multilabel classification, or true scores of entities to be ranked.
Target scores, can either be probability estimates, confidence values, or non-thresholded measure of decisions (as returned by “decision_function” on some classifiers).
Only consider the highest k scores in the ranking. If None, use all outputs.
Base of the logarithm used for the discount. A low value means a sharper discount (top results are more important).
Sample weights. If None, all samples are given the same weight.
Assume that there are no ties in y_score (which is likely to be the case if y_score is continuous) for efficiency gains.
The averaged sample DCG scores.
See also
The Discounted Cumulative Gain divided by the Ideal Discounted Cumulative Gain (the DCG obtained for a perfect ranking), in order to have a score between 0 and 1.
Wikipedia entry for Discounted Cumulative Gain.
Jarvelin, K., & Kekalainen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS), 20(4), 422-446.
Wang, Y., Wang, L., Li, Y., He, D., Chen, W., & Liu, T. Y. (2013, May). A theoretical analysis of NDCG ranking measures. In Proceedings of the 26th Annual Conference on Learning Theory (COLT 2013).
McSherry, F., & Najork, M. (2008, March). Computing information retrieval performance measures efficiently in the presence of tied scores. In European conference on information retrieval (pp. 414-421). Springer, Berlin, Heidelberg.


Compute error rates for different probability thresholds.
Note
This metric is used for evaluation of ranking and error tradeoffs of a binary classification task.
Read more in the User Guide.
New in version 0.24.
True binary labels. If labels are not either {-1, 1} or {0, 1}, then pos_label should be explicitly given.
Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by “decision_function” on some classifiers).
The label of the positive class. When pos_label=None, if y_true is in {-1, 1} or {0, 1}, pos_label is set to 1, otherwise an error will be raised.
Sample weights.
False positive rate (FPR) such that element i is the false positive rate of predictions with score >= thresholds[i]. This is occasionally referred to as false acceptance propability or fall-out.
False negative rate (FNR) such that element i is the false negative rate of predictions with score >= thresholds[i]. This is occasionally referred to as false rejection or miss rate.
Decreasing score values.
See also
Plot detection error tradeoff (DET) curve.
DET curve visualization.
Compute Receiver operating characteristic (ROC) curve.
Compute precision-recall curve.
Detection error tradeoff (DET) curve


Explained variance regression score function.
Best possible score is 1.0, lower values are worse.
Read more in the User Guide.
Ground truth (correct) target values.
Estimated target values.
Sample weights.
Defines aggregating of multiple output scores. Array-like value defines weights used to average scores.
Returns a full set of scores in case of multioutput input.
Scores of all outputs are averaged with uniform weight.
Scores of all outputs are averaged, weighted by the variances of each individual output.
The explained variance or ndarray if ‘multioutput’ is ‘raw_values’.
This is not a symmetric function.


Compute the F1 score, also known as balanced F-score or F-measure.
The F1 score can be interpreted as a weighted average of the precision and recall, where an F1 score reaches its best value at 1 and worst score at 0. The relative contribution of precision and recall to the F1 score are equal. The formula for the F1 score is:
In the multi-class and multi-label case, this is the average of the F1 score of each class with weighting depending on the average parameter.
Read more in the User Guide.
Ground truth (correct) target values.
Estimated targets as returned by a classifier.
The set of labels to include when average != 'binary', and their order if average is None. Labels present in the data can be excluded, for example to calculate a multiclass average ignoring a majority negative class, while labels not present in the data will result in 0 components in a macro average. For multilabel targets, labels are column indices. By default, all labels in y_true and y_pred are used in sorted order.
Changed in version 0.17: Parameter labels improved for multiclass problem.
The class to report if average='binary' and the data is binary. If the data are multiclass or multilabel, this will be ignored; setting labels=[pos_label] and average != 'binary' will report scores for that label only.
This parameter is required for multiclass/multilabel targets. If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data:
Only report results for the class specified by pos_label. This is applicable only if targets (y_{true,pred}) are binary.
Calculate metrics globally by counting the total true positives, false negatives and false positives.
Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.
Calculate metrics for each label, and find their average weighted by support (the number of true instances for each label). This alters ‘macro’ to account for label imbalance; it can result in an F-score that is not between precision and recall.
Calculate metrics for each instance, and find their average (only meaningful for multilabel classification where this differs from accuracy_score).
Sample weights.
Sets the value to return when there is a zero division, i.e. when all predictions and labels are negative. If set to “warn”, this acts as 0, but warnings are also raised.
F1 score of the positive class in binary classification or weighted average of the F1 scores of each class for the multiclass task.
See also
When true positive + false positive == 0, precision is undefined. When true positive + false negative == 0, recall is undefined. In such cases, by default the metric will be set to 0, as will f-score, and UndefinedMetricWarning will be raised. This behavior can be modified with zero_division.
Wikipedia entry for the F1-score.
Probability Calibration curves
Precision-Recall
Semi-supervised Classification on a Text Dataset


Compute the F-beta score.
The F-beta score is the weighted harmonic mean of precision and recall, reaching its optimal value at 1 and its worst value at 0.
The beta parameter determines the weight of recall in the combined score. beta < 1 lends more weight to precision, while beta > 1 favors recall (beta -> 0 considers only precision, beta -> +inf only recall).
Read more in the User Guide.
Ground truth (correct) target values.
Estimated targets as returned by a classifier.
Determines the weight of recall in the combined score.
The set of labels to include when average != 'binary', and their order if average is None. Labels present in the data can be excluded, for example to calculate a multiclass average ignoring a majority negative class, while labels not present in the data will result in 0 components in a macro average. For multilabel targets, labels are column indices. By default, all labels in y_true and y_pred are used in sorted order.
Changed in version 0.17: Parameter labels improved for multiclass problem.
The class to report if average='binary' and the data is binary. If the data are multiclass or multilabel, this will be ignored; setting labels=[pos_label] and average != 'binary' will report scores for that label only.
This parameter is required for multiclass/multilabel targets. If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data:
Only report results for the class specified by pos_label. This is applicable only if targets (y_{true,pred}) are binary.
Calculate metrics globally by counting the total true positives, false negatives and false positives.
Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.
Calculate metrics for each label, and find their average weighted by support (the number of true instances for each label). This alters ‘macro’ to account for label imbalance; it can result in an F-score that is not between precision and recall.
Calculate metrics for each instance, and find their average (only meaningful for multilabel classification where this differs from accuracy_score).
Sample weights.
Sets the value to return when there is a zero division, i.e. when all predictions and labels are negative. If set to “warn”, this acts as 0, but warnings are also raised.
F-beta score of the positive class in binary classification or weighted average of the F-beta score of each class for the multiclass task.
See also
When true positive + false positive == 0 or true positive + false negative == 0, f-score returns 0 and raises UndefinedMetricWarning. This behavior can be modified with zero_division.
R. Baeza-Yates and B. Ribeiro-Neto (2011). Modern Information Retrieval. Addison Wesley, pp. 327-328.
Wikipedia entry for the F1-score.


Measure the similarity of two clusterings of a set of points.
New in version 0.18.
The Fowlkes-Mallows index (FMI) is defined as the geometric mean between of the precision and recall:
Where TP is the number of True Positive (i.e. the number of pair of points that belongs in the same clusters in both labels_true and labels_pred), FP is the number of False Positive (i.e. the number of pair of points that belongs in the same clusters in labels_true and not in labels_pred) and FN is the number of False Negative (i.e the number of pair of points that belongs in the same clusters in labels_pred and not in labels_True).
The score ranges from 0 to 1. A high value indicates a good similarity between two clusters.
Read more in the User Guide.
A clustering of the data into disjoint subsets.
A clustering of the data into disjoint subsets.
Compute contingency matrix internally with sparse matrix.
The resulting Fowlkes-Mallows score.
E. B. Fowkles and C. L. Mallows, 1983. “A method for comparing two hierarchical clusterings”. Journal of the American Statistical Association
Wikipedia entry for the Fowlkes-Mallows Index
Perfect labelings are both homogeneous and complete, hence have score 1.0:
If classes members are completely split across different clusters, the assignment is totally random, hence the FMI is null:


Get a scorer from string.
Read more in the User Guide.
Scoring method as string. If callable it is returned as is.
The scorer.


Compute the average Hamming loss.
The Hamming loss is the fraction of labels that are incorrectly predicted.
Read more in the User Guide.
Ground truth (correct) labels.
Predicted labels, as returned by a classifier.
Sample weights.
New in version 0.18.
Return the average Hamming loss between element of y_true and y_pred.
See also
In multiclass classification, the Hamming loss corresponds to the Hamming distance between y_true and y_pred which is equivalent to the subset zero_one_loss function, when normalize parameter is set to True.
In multilabel classification, the Hamming loss is different from the subset zero-one loss. The zero-one loss considers the entire set of labels for a given sample incorrect if it does not entirely match the true set of labels. Hamming loss is more forgiving in that it penalizes only the individual labels.
The Hamming loss is upperbounded by the subset zero-one loss, when normalize parameter is set to True. It is always between 0 and 1, lower being better.
Grigorios Tsoumakas, Ioannis Katakis. Multi-Label Classification: An Overview. International Journal of Data Warehousing & Mining, 3(3), 1-13, July-September 2007.
Wikipedia entry on the Hamming distance.
In the multilabel case with binary label indicators:
Model Complexity Influence


Average hinge loss (non-regularized).
In binary class case, assuming labels in y_true are encoded with +1 and -1, when a prediction mistake is made, margin = y_true * pred_decision is always negative (since the signs disagree), implying 1 - margin is always greater than 1. The cumulated hinge loss is therefore an upper bound of the number of mistakes made by the classifier.
In multiclass case, the function expects that either all the labels are included in y_true or an optional labels argument is provided which contains all the labels. The multilabel margin is calculated according to Crammer-Singer’s method. As in the binary case, the cumulated hinge loss is an upper bound of the number of mistakes made by the classifier.
Read more in the User Guide.
True target, consisting of integers of two values. The positive label must be greater than the negative label.
Predicted decisions, as output by decision_function (floats).
Contains all the labels for the problem. Used in multiclass hinge loss.
Sample weights.
Wikipedia entry on the Hinge loss.
Koby Crammer, Yoram Singer. On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines. Journal of Machine Learning Research 2, (2001), 265-292.
L1 AND L2 Regularization for Multiclass Hinge Loss Models by Robert C. Moore, John DeNero.
In the multiclass case:


Compute the homogeneity and completeness and V-Measure scores at once.
Those metrics are based on normalized conditional entropy measures of the clustering labeling to evaluate given the knowledge of a Ground Truth class labels of the same samples.
A clustering result satisfies homogeneity if all of its clusters contain only data points which are members of a single class.
A clustering result satisfies completeness if all the data points that are members of a given class are elements of the same cluster.
Both scores have positive values between 0.0 and 1.0, larger values being desirable.
Those 3 metrics are independent of the absolute values of the labels: a permutation of the class or cluster label values won’t change the score values in any way.
V-Measure is furthermore symmetric: swapping labels_true and label_pred will give the same score. This does not hold for homogeneity and completeness. V-Measure is identical to normalized_mutual_info_score with the arithmetic averaging method.
Read more in the User Guide.
ground truth class labels to be used as a reference
cluster labels to evaluate
Ratio of weight attributed to homogeneity vs completeness. If beta is greater than 1, completeness is weighted more strongly in the calculation. If beta is less than 1, homogeneity is weighted more strongly.
score between 0.0 and 1.0. 1.0 stands for perfectly homogeneous labeling
score between 0.0 and 1.0. 1.0 stands for perfectly complete labeling
harmonic mean of the first two
See also


Homogeneity metric of a cluster labeling given a ground truth.
A clustering result satisfies homogeneity if all of its clusters contain only data points which are members of a single class.
This metric is independent of the absolute values of the labels: a permutation of the class or cluster label values won’t change the score value in any way.
This metric is not symmetric: switching label_true with label_pred will return the completeness_score which will be different in general.
Read more in the User Guide.
ground truth class labels to be used as a reference
cluster labels to evaluate
score between 0.0 and 1.0. 1.0 stands for perfectly homogeneous labeling
See also
Andrew Rosenberg and Julia Hirschberg, 2007. V-Measure: A conditional entropy-based external cluster evaluation measure
Perfect labelings are homogeneous:
Non-perfect labelings that further split classes into more clusters can be perfectly homogeneous:
Clusters that include samples from different classes do not make for an homogeneous labeling:
Demo of affinity propagation clustering algorithm
Demo of DBSCAN clustering algorithm
A demo of K-Means clustering on the handwritten digits data
Clustering text documents using k-means


Jaccard similarity coefficient score.
The Jaccard index [1], or Jaccard similarity coefficient, defined as the size of the intersection divided by the size of the union of two label sets, is used to compare set of predicted labels for a sample to the corresponding set of labels in y_true.
Read more in the User Guide.
Ground truth (correct) labels.
Predicted labels, as returned by a classifier.
The set of labels to include when average != 'binary', and their order if average is None. Labels present in the data can be excluded, for example to calculate a multiclass average ignoring a majority negative class, while labels not present in the data will result in 0 components in a macro average. For multilabel targets, labels are column indices. By default, all labels in y_true and y_pred are used in sorted order.
The class to report if average='binary' and the data is binary. If the data are multiclass or multilabel, this will be ignored; setting labels=[pos_label] and average != 'binary' will report scores for that label only.
If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data:
Only report results for the class specified by pos_label. This is applicable only if targets (y_{true,pred}) are binary.
Calculate metrics globally by counting the total true positives, false negatives and false positives.
Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.
Calculate metrics for each label, and find their average, weighted by support (the number of true instances for each label). This alters ‘macro’ to account for label imbalance.
Calculate metrics for each instance, and find their average (only meaningful for multilabel classification).
Sample weights.
Sets the value to return when there is a zero division, i.e. when there there are no negative values in predictions and labels. If set to “warn”, this acts like 0, but a warning is also raised.
See also
jaccard_score may be a poor metric if there are no positives for some samples or classes. Jaccard is undefined if there are no true or predicted labels, and our implementation will return a score of 0 with a warning.
Wikipedia entry for the Jaccard index.
In the binary case:
In the multilabel case:
In the multiclass case:
Classifier Chain


Compute ranking-based average precision.
Label ranking average precision (LRAP) is the average over each ground truth label assigned to each sample, of the ratio of true vs. total labels with lower score.
This metric is used in multilabel ranking problem, where the goal is to give better rank to the labels associated to each sample.
The obtained score is always strictly greater than 0 and the best value is 1.
Read more in the User Guide.
True binary labels in binary indicator format.
Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by “decision_function” on some classifiers).
Sample weights.
New in version 0.20.


Compute Ranking loss measure.
Compute the average number of label pairs that are incorrectly ordered given y_score weighted by the size of the label set and the number of labels not in the label set.
This is similar to the error set size, but weighted by the number of relevant and irrelevant labels. The best performance is achieved with a ranking loss of zero.
Read more in the User Guide.
New in version 0.17: A function label_ranking_loss
True binary labels in binary indicator format.
Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by “decision_function” on some classifiers).
Sample weights.
Tsoumakas, G., Katakis, I., & Vlahavas, I. (2010). Mining multi-label data. In Data mining and knowledge discovery handbook (pp. 667-685). Springer US.


Log loss, aka logistic loss or cross-entropy loss.
This is the loss function used in (multinomial) logistic regression and extensions of it such as neural networks, defined as the negative log-likelihood of a logistic model that returns y_pred probabilities for its training data y_true. The log loss is only defined for two or more labels. For a single sample with true label \(y \in \{0,1\}\) and and a probability estimate \(p = \operatorname{Pr}(y = 1)\), the log loss is:
Read more in the User Guide.
Ground truth (correct) labels for n_samples samples.
Predicted probabilities, as returned by a classifier’s predict_proba method. If y_pred.shape = (n_samples,) the probabilities provided are assumed to be that of the positive class. The labels in y_pred are assumed to be ordered alphabetically, as done by preprocessing.LabelBinarizer.
Log loss is undefined for p=0 or p=1, so probabilities are clipped to max(eps, min(1 - eps, p)).
If true, return the mean loss per sample. Otherwise, return the sum of the per-sample losses.
Sample weights.
If not provided, labels will be inferred from y_true. If labels is None and y_pred has shape (n_samples,) the labels are assumed to be binary and are inferred from y_true.
New in version 0.18.
The logarithm used is the natural logarithm (base-e).
C.M. Bishop (2006). Pattern Recognition and Machine Learning. Springer, p. 209.
Probability Calibration for 3-class classification
Probabilistic predictions with Gaussian process classification (GPC)


Make a scorer from a performance metric or loss function.
This factory function wraps scoring functions for use in GridSearchCV and cross_val_score. It takes a score function, such as accuracy_score, mean_squared_error, adjusted_rand_index or average_precision and returns a callable that scores an estimator’s output. The signature of the call is (estimator, X, y) where estimator is the model to be evaluated, X is the data and y is the ground truth labeling (or None in the case of unsupervised models).
Read more in the User Guide.
Score function (or loss function) with signature score_func(y, y_pred, **kwargs).
Whether score_func is a score function (default), meaning high is good, or a loss function, meaning low is good. In the latter case, the scorer object will sign-flip the outcome of the score_func.
Whether score_func requires predict_proba to get probability estimates out of a classifier.
If True, for binary y_true, the score function is supposed to accept a 1D y_pred (i.e., probability of the positive class, shape (n_samples,)).
Whether score_func takes a continuous decision certainty. This only works for binary classification using estimators that have either a decision_function or predict_proba method.
If True, for binary y_true, the score function is supposed to accept a 1D y_pred (i.e., probability of the positive class or the decision function, shape (n_samples,)).
For example average_precision or the area under the roc curve can not be computed using discrete predictions alone.
Additional parameters to be passed to score_func.
Callable object that returns a scalar score; greater is better.
If needs_proba=False and needs_threshold=False, the score function is supposed to accept the output of predict. If needs_proba=True, the score function is supposed to accept the output of predict_proba (For binary y_true, the score function is supposed to accept probability of the positive class). If needs_threshold=True, the score function is supposed to accept the output of decision_function.
Demonstration of multi-metric evaluation on cross_val_score and GridSearchCV


Compute the Matthews correlation coefficient (MCC).
The Matthews correlation coefficient is used in machine learning as a measure of the quality of binary and multiclass classifications. It takes into account true and false positives and negatives and is generally regarded as a balanced measure which can be used even if the classes are of very different sizes. The MCC is in essence a correlation coefficient value between -1 and +1. A coefficient of +1 represents a perfect prediction, 0 an average random prediction and -1 an inverse prediction. The statistic is also known as the phi coefficient. [source: Wikipedia]
Binary and multiclass labels are supported. Only in the binary case does this relate to information about true and false positives and negatives. See references below.
Read more in the User Guide.
Ground truth (correct) target values.
Estimated targets as returned by a classifier.
Sample weights.
New in version 0.18.
The Matthews correlation coefficient (+1 represents a perfect prediction, 0 an average random prediction and -1 and inverse prediction).
Baldi, Brunak, Chauvin, Andersen and Nielsen, (2000). Assessing the accuracy of prediction algorithms for classification: an overview.
Wikipedia entry for the Matthews Correlation Coefficient.
Gorodkin, (2004). Comparing two K-category assignments by a K-category correlation coefficient.
Jurman, Riccadonna, Furlanello, (2012). A Comparison of MCC and CEN Error Measures in MultiClass Prediction.


max_error metric calculates the maximum residual error.
Read more in the User Guide.
Ground truth (correct) target values.
Estimated target values.
A positive floating point value (the best value is 0.0).


Mean absolute error regression loss.
Read more in the User Guide.
Ground truth (correct) target values.
Estimated target values.
Sample weights.
Defines aggregating of multiple output values. Array-like value defines weights used to average errors.
Returns a full set of errors in case of multioutput input.
Errors of all outputs are averaged with uniform weight.
If multioutput is ‘raw_values’, then mean absolute error is returned for each output separately. If multioutput is ‘uniform_average’ or an ndarray of weights, then the weighted average of all output errors is returned.
MAE output is non-negative floating point. The best value is 0.0.
Poisson regression and non-normal loss
Tweedie regression on insurance claims


Mean absolute percentage error regression loss.
Note here that we do not represent the output as a percentage in range [0, 100]. Instead, we represent it in range [0, 1/eps]. Read more in the User Guide.
New in version 0.24.
Ground truth (correct) target values.
Estimated target values.
Sample weights.
Defines aggregating of multiple output values. Array-like value defines weights used to average errors. If input is list then the shape must be (n_outputs,).
Returns a full set of errors in case of multioutput input.
Errors of all outputs are averaged with uniform weight.
If multioutput is ‘raw_values’, then mean absolute percentage error is returned for each output separately. If multioutput is ‘uniform_average’ or an ndarray of weights, then the weighted average of all output errors is returned.
MAPE output is non-negative floating point. The best value is 0.0. But note the fact that bad predictions can lead to arbitarily large MAPE values, especially if some y_true values are very close to zero. Note that we return a large value instead of inf when y_true is zero.


Mean Gamma deviance regression loss.
Gamma deviance is equivalent to the Tweedie deviance with the power parameter power=2. It is invariant to scaling of the target variable, and measures relative errors.
Read more in the User Guide.
Ground truth (correct) target values. Requires y_true > 0.
Estimated target values. Requires y_pred > 0.
Sample weights.
A non-negative floating point value (the best value is 0.0).