sklearn.metrics.normalized_mutual_info_score(labels_true, labels_pred, *, average_method='arithmetic') [source]

Normalized Mutual Information between two clusterings. Normalized Mutual Information (NMI) is a normalization of the Mutual Information (MI) score to scale the results between 0 (no mutual information) and 1 (perfect correlation). In this function, mutual information is normalized by some generalized mean of H(labels_true) and H(labels_pred)), defined by the average_method. This measure is not adjusted for chance. Therefore adjusted_mutual_info_score might be preferred. This metric is independent of the absolute values of the labels: a permutation of the class or cluster label values won’t change the score value in any way. This metric is furthermore symmetric: switching label_true with label_pred will return the same score value. This can be useful to measure the agreement of two independent label assignments strategies on the same dataset when the real ground truth is not known. Read more in the User Guide.  Parameters

labels_trueint array, shape = [n_samples]

A clustering of the data into disjoint subsets.
labels_predint array-like of shape (n_samples,)

A clustering of the data into disjoint subsets.
average_methodstr, default=’arithmetic’

How to compute the normalizer in the denominator. Possible options are ‘min’, ‘geometric’, ‘arithmetic’, and ‘max’.  New in version 0.20.   Changed in version 0.22: The default value of average_method changed from ‘geometric’ to ‘arithmetic’.     Returns

nmifloat

score between 0.0 and 1.0. 1.0 stands for perfectly complete labeling      See also
v_measure_score

V-Measure (NMI with arithmetic mean option).
adjusted_rand_score

Adjusted Rand Index.
adjusted_mutual_info_score

Adjusted Mutual Information (adjusted against chance).    Examples Perfect labelings are both homogeneous and complete, hence have score 1.0: >>> from sklearn.metrics.cluster import normalized_mutual_info_score
>>> normalized_mutual_info_score([0, 0, 1, 1], [0, 0, 1, 1])
...
1.0
>>> normalized_mutual_info_score([0, 0, 1, 1], [1, 1, 0, 0])
...
1.0
 If classes members are completely split across different clusters, the assignment is totally in-complete, hence the NMI is null: >>> normalized_mutual_info_score([0, 0, 0, 0], [0, 1, 2, 3])
...
0.0


sklearn.metrics.pairwise.additive_chi2_kernel(X, Y=None) [source]

Computes the additive chi-squared kernel between observations in X and Y. The chi-squared kernel is computed between each pair of rows in X and Y. X and Y have to be non-negative. This kernel is most commonly applied to histograms. The chi-squared kernel is given by: k(x, y) = -Sum [(x - y)^2 / (x + y)]
 It can be interpreted as a weighted difference per entry. Read more in the User Guide.  Parameters

Xarray-like of shape (n_samples_X, n_features)

Yndarray of shape (n_samples_Y, n_features), default=None
  Returns

kernel_matrixndarray of shape (n_samples_X, n_samples_Y)
    See also
chi2_kernel

The exponentiated version of the kernel, which is usually preferable.
sklearn.kernel_approximation.AdditiveChi2Sampler

A Fourier approximation to this kernel.    Notes As the negative of a distance, this kernel is only conditionally positive definite. References  Zhang, J. and Marszalek, M. and Lazebnik, S. and Schmid, C. Local features and kernels for classification of texture and object categories: A comprehensive study International Journal of Computer Vision 2007 https://research.microsoft.com/en-us/um/people/manik/projects/trade-off/papers/ZhangIJCV06.pdf


sklearn.metrics.pairwise.chi2_kernel(X, Y=None, gamma=1.0) [source]

Computes the exponential chi-squared kernel X and Y. The chi-squared kernel is computed between each pair of rows in X and Y. X and Y have to be non-negative. This kernel is most commonly applied to histograms. The chi-squared kernel is given by: k(x, y) = exp(-gamma Sum [(x - y)^2 / (x + y)])
 It can be interpreted as a weighted difference per entry. Read more in the User Guide.  Parameters

Xarray-like of shape (n_samples_X, n_features)

Yndarray of shape (n_samples_Y, n_features), default=None

gammafloat, default=1.

Scaling parameter of the chi2 kernel.    Returns

kernel_matrixndarray of shape (n_samples_X, n_samples_Y)
    See also
additive_chi2_kernel

The additive version of this kernel.
sklearn.kernel_approximation.AdditiveChi2Sampler

A Fourier approximation to the additive version of this kernel.    References  Zhang, J. and Marszalek, M. and Lazebnik, S. and Schmid, C. Local features and kernels for classification of texture and object categories: A comprehensive study International Journal of Computer Vision 2007 https://research.microsoft.com/en-us/um/people/manik/projects/trade-off/papers/ZhangIJCV06.pdf


sklearn.metrics.pairwise.cosine_distances(X, Y=None) [source]

Compute cosine distance between samples in X and Y. Cosine distance is defined as 1.0 minus the cosine similarity. Read more in the User Guide.  Parameters

X{array-like, sparse matrix} of shape (n_samples_X, n_features)

Matrix X.
Y{array-like, sparse matrix} of shape (n_samples_Y, n_features), default=None

Matrix Y.    Returns

distance matrixndarray of shape (n_samples_X, n_samples_Y)
    See also
cosine_similarity

scipy.spatial.distance.cosine

Dense matrices only.


sklearn.metrics.pairwise.cosine_similarity(X, Y=None, dense_output=True) [source]

Compute cosine similarity between samples in X and Y. Cosine similarity, or the cosine kernel, computes similarity as the normalized dot product of X and Y: K(X, Y) = <X, Y> / (||X||*||Y||) On L2-normalized data, this function is equivalent to linear_kernel. Read more in the User Guide.  Parameters

X{ndarray, sparse matrix} of shape (n_samples_X, n_features)

Input data.
Y{ndarray, sparse matrix} of shape (n_samples_Y, n_features), default=None

Input data. If None, the output will be the pairwise similarities between all samples in X.
dense_outputbool, default=True

Whether to return dense output even when the input is sparse. If False, the output is sparse if both input arrays are sparse.  New in version 0.17: parameter dense_output for dense output.     Returns

kernel matrixndarray of shape (n_samples_X, n_samples_Y)


sklearn.metrics.pairwise.distance_metrics() [source]

Valid metrics for pairwise_distances. This function simply returns the valid pairwise distance metrics. It exists to allow for a description of the mapping for each of the valid strings. The valid distance metrics, and the function they map to, are:
metric Function
‘cityblock’ metrics.pairwise.manhattan_distances
‘cosine’ metrics.pairwise.cosine_distances
‘euclidean’ metrics.pairwise.euclidean_distances
‘haversine’ metrics.pairwise.haversine_distances
‘l1’ metrics.pairwise.manhattan_distances
‘l2’ metrics.pairwise.euclidean_distances
‘manhattan’ metrics.pairwise.manhattan_distances
‘nan_euclidean’ metrics.pairwise.nan_euclidean_distances   Read more in the User Guide.


sklearn.metrics.pairwise.euclidean_distances(X, Y=None, *, Y_norm_squared=None, squared=False, X_norm_squared=None) [source]

Considering the rows of X (and Y=X) as vectors, compute the distance matrix between each pair of vectors. For efficiency reasons, the euclidean distance between a pair of row vector x and y is computed as: dist(x, y) = sqrt(dot(x, x) - 2 * dot(x, y) + dot(y, y))
 This formulation has two advantages over other ways of computing distances. First, it is computationally efficient when dealing with sparse data. Second, if one argument varies but the other remains unchanged, then dot(x, x) and/or dot(y, y) can be pre-computed. However, this is not the most precise way of doing this computation, because this equation potentially suffers from “catastrophic cancellation”. Also, the distance matrix returned by this function may not be exactly symmetric as required by, e.g., scipy.spatial.distance functions. Read more in the User Guide.  Parameters

X{array-like, sparse matrix} of shape (n_samples_X, n_features)

Y{array-like, sparse matrix} of shape (n_samples_Y, n_features), default=None

Y_norm_squaredarray-like of shape (n_samples_Y,), default=None

Pre-computed dot-products of vectors in Y (e.g., (Y**2).sum(axis=1)) May be ignored in some cases, see the note below.
squaredbool, default=False

Return squared Euclidean distances.
X_norm_squaredarray-like of shape (n_samples,), default=None

Pre-computed dot-products of vectors in X (e.g., (X**2).sum(axis=1)) May be ignored in some cases, see the note below.    Returns

distancesndarray of shape (n_samples_X, n_samples_Y)
    See also
paired_distances

Distances betweens pairs of elements of X and Y.    Notes To achieve better accuracy, X_norm_squared and Y_norm_squared may be unused if they are passed as float32. Examples >>> from sklearn.metrics.pairwise import euclidean_distances
>>> X = [[0, 1], [1, 1]]
>>> # distance between rows of X
>>> euclidean_distances(X, X)
array([[0., 1.],
       [1., 0.]])
>>> # get distance to origin
>>> euclidean_distances(X, [[0, 0]])
array([[1.        ],
       [1.41421356]])


sklearn.metrics.pairwise.haversine_distances(X, Y=None) [source]

Compute the Haversine distance between samples in X and Y. The Haversine (or great circle) distance is the angular distance between two points on the surface of a sphere. The first coordinate of each point is assumed to be the latitude, the second is the longitude, given in radians. The dimension of the data must be 2.  \[D(x, y) = 2\arcsin[\sqrt{\sin^2((x1 - y1) / 2) + \cos(x1)\cos(y1)\sin^2((x2 - y2) / 2)}]\]  Parameters

Xarray-like of shape (n_samples_X, 2)

Yarray-like of shape (n_samples_Y, 2), default=None
  Returns

distancendarray of shape (n_samples_X, n_samples_Y)
   Notes As the Earth is nearly spherical, the haversine formula provides a good approximation of the distance between two points of the Earth surface, with a less than 1% error on average. Examples We want to calculate the distance between the Ezeiza Airport (Buenos Aires, Argentina) and the Charles de Gaulle Airport (Paris, France). >>> from sklearn.metrics.pairwise import haversine_distances
>>> from math import radians
>>> bsas = [-34.83333, -58.5166646]
>>> paris = [49.0083899664, 2.53844117956]
>>> bsas_in_radians = [radians(_) for _ in bsas]
>>> paris_in_radians = [radians(_) for _ in paris]
>>> result = haversine_distances([bsas_in_radians, paris_in_radians])
>>> result * 6371000/1000  # multiply by Earth radius to get kilometers
array([[    0.        , 11099.54035582],
       [11099.54035582,     0.        ]])


sklearn.metrics.pairwise.kernel_metrics() [source]

Valid metrics for pairwise_kernels. This function simply returns the valid pairwise distance metrics. It exists, however, to allow for a verbose description of the mapping for each of the valid strings.  The valid distance metrics, and the function they map to, are:

metric Function
‘additive_chi2’ sklearn.pairwise.additive_chi2_kernel
‘chi2’ sklearn.pairwise.chi2_kernel
‘linear’ sklearn.pairwise.linear_kernel
‘poly’ sklearn.pairwise.polynomial_kernel
‘polynomial’ sklearn.pairwise.polynomial_kernel
‘rbf’ sklearn.pairwise.rbf_kernel
‘laplacian’ sklearn.pairwise.laplacian_kernel
‘sigmoid’ sklearn.pairwise.sigmoid_kernel
‘cosine’ sklearn.pairwise.cosine_similarity     Read more in the User Guide.


sklearn.metrics.pairwise.laplacian_kernel(X, Y=None, gamma=None) [source]

Compute the laplacian kernel between X and Y. The laplacian kernel is defined as: K(x, y) = exp(-gamma ||x-y||_1)
 for each pair of rows x in X and y in Y. Read more in the User Guide.  New in version 0.17.   Parameters

Xndarray of shape (n_samples_X, n_features)

Yndarray of shape (n_samples_Y, n_features), default=None

gammafloat, default=None

If None, defaults to 1.0 / n_features.    Returns

kernel_matrixndarray of shape (n_samples_X, n_samples_Y)


sklearn.metrics.pairwise.linear_kernel(X, Y=None, dense_output=True) [source]

Compute the linear kernel between X and Y. Read more in the User Guide.  Parameters

Xndarray of shape (n_samples_X, n_features)

Yndarray of shape (n_samples_Y, n_features), default=None

dense_outputbool, default=True

Whether to return dense output even when the input is sparse. If False, the output is sparse if both input arrays are sparse.  New in version 0.20.     Returns

Gram matrixndarray of shape (n_samples_X, n_samples_Y)


sklearn.metrics.pairwise.manhattan_distances(X, Y=None, *, sum_over_features=True) [source]

Compute the L1 distances between the vectors in X and Y. With sum_over_features equal to False it returns the componentwise distances. Read more in the User Guide.  Parameters

Xarray-like of shape (n_samples_X, n_features)

Yarray-like of shape (n_samples_Y, n_features), default=None

sum_over_featuresbool, default=True

If True the function returns the pairwise distance matrix else it returns the componentwise L1 pairwise-distances. Not supported for sparse matrix inputs.    Returns

Dndarray of shape (n_samples_X * n_samples_Y, n_features) or (n_samples_X, n_samples_Y)

If sum_over_features is False shape is (n_samples_X * n_samples_Y, n_features) and D contains the componentwise L1 pairwise-distances (ie. absolute difference), else shape is (n_samples_X, n_samples_Y) and D contains the pairwise L1 distances.     Notes When X and/or Y are CSR sparse matrices and they are not already in canonical format, this function modifies them in-place to make them canonical. Examples >>> from sklearn.metrics.pairwise import manhattan_distances
>>> manhattan_distances([[3]], [[3]])
array([[0.]])
>>> manhattan_distances([[3]], [[2]])
array([[1.]])
>>> manhattan_distances([[2]], [[3]])
array([[1.]])
>>> manhattan_distances([[1, 2], [3, 4]],         [[1, 2], [0, 3]])
array([[0., 2.],
       [4., 4.]])
>>> import numpy as np
>>> X = np.ones((1, 2))
>>> y = np.full((2, 2), 2.)
>>> manhattan_distances(X, y, sum_over_features=False)
array([[1., 1.],
       [1., 1.]])


sklearn.metrics.pairwise.nan_euclidean_distances(X, Y=None, *, squared=False, missing_values=nan, copy=True) [source]

Calculate the euclidean distances in the presence of missing values. Compute the euclidean distance between each pair of samples in X and Y, where Y=X is assumed if Y=None. When calculating the distance between a pair of samples, this formulation ignores feature coordinates with a missing value in either sample and scales up the weight of the remaining coordinates: dist(x,y) = sqrt(weight * sq. distance from present coordinates) where, weight = Total # of coordinates / # of present coordinates For example, the distance between [3, na, na, 6] and [1, na, 4, 5] is:  \[\sqrt{\frac{4}{2}((3-1)^2 + (6-5)^2)}\] If all the coordinates are missing or if there are no common present coordinates then NaN is returned for that pair. Read more in the User Guide.  New in version 0.22.   Parameters

Xarray-like of shape=(n_samples_X, n_features)

Yarray-like of shape=(n_samples_Y, n_features), default=None

squaredbool, default=False

Return squared Euclidean distances.
missing_valuesnp.nan or int, default=np.nan

Representation of missing value.
copybool, default=True

Make and use a deep copy of X and Y (if Y exists).    Returns

distancesndarray of shape (n_samples_X, n_samples_Y)
    See also
paired_distances

Distances between pairs of elements of X and Y.    References  John K. Dixon, “Pattern Recognition with Partly Missing Data”, IEEE Transactions on Systems, Man, and Cybernetics, Volume: 9, Issue: 10, pp. 617 - 621, Oct. 1979. http://ieeexplore.ieee.org/abstract/document/4310090/
  Examples >>> from sklearn.metrics.pairwise import nan_euclidean_distances
>>> nan = float("NaN")
>>> X = [[0, 1], [1, nan]]
>>> nan_euclidean_distances(X, X) # distance between rows of X
array([[0.        , 1.41421356],
       [1.41421356, 0.        ]])
 >>> # get distance to origin
>>> nan_euclidean_distances(X, [[0, 0]])
array([[1.        ],
       [1.41421356]])


sklearn.metrics.pairwise.paired_cosine_distances(X, Y) [source]

Computes the paired cosine distances between X and Y. Read more in the User Guide.  Parameters

Xarray-like of shape (n_samples, n_features)

Yarray-like of shape (n_samples, n_features)
  Returns

distancesndarray of shape (n_samples,)
   Notes The cosine distance is equivalent to the half the squared euclidean distance if each sample is normalized to unit norm.


sklearn.metrics.pairwise.paired_distances(X, Y, *, metric='euclidean', **kwds) [source]

Computes the paired distances between X and Y. Computes the distances between (X[0], Y[0]), (X[1], Y[1]), etc… Read more in the User Guide.  Parameters

Xndarray of shape (n_samples, n_features)

Array 1 for distance computation.
Yndarray of shape (n_samples, n_features)

Array 2 for distance computation.
metricstr or callable, default=”euclidean”

The metric to use when calculating distance between instances in a feature array. If metric is a string, it must be one of the options specified in PAIRED_DISTANCES, including “euclidean”, “manhattan”, or “cosine”. Alternatively, if metric is a callable function, it is called on each pair of instances (rows) and the resulting value recorded. The callable should take two arrays from X as input and return a value indicating the distance between them.    Returns

distancesndarray of shape (n_samples,)
    See also
pairwise_distances

Computes the distance between every pair of samples.    Examples >>> from sklearn.metrics.pairwise import paired_distances
>>> X = [[0, 1], [1, 1]]
>>> Y = [[0, 1], [2, 1]]
>>> paired_distances(X, Y)
array([0., 1.])


sklearn.metrics.pairwise.paired_euclidean_distances(X, Y) [source]

Computes the paired euclidean distances between X and Y. Read more in the User Guide.  Parameters

Xarray-like of shape (n_samples, n_features)

Yarray-like of shape (n_samples, n_features)
  Returns

distancesndarray of shape (n_samples,)


sklearn.metrics.pairwise.paired_manhattan_distances(X, Y) [source]

Compute the L1 distances between the vectors in X and Y. Read more in the User Guide.  Parameters

Xarray-like of shape (n_samples, n_features)

Yarray-like of shape (n_samples, n_features)
  Returns

distancesndarray of shape (n_samples,)


sklearn.metrics.pairwise.pairwise_kernels(X, Y=None, metric='linear', *, filter_params=False, n_jobs=None, **kwds) [source]

Compute the kernel between arrays X and optional array Y. This method takes either a vector array or a kernel matrix, and returns a kernel matrix. If the input is a vector array, the kernels are computed. If the input is a kernel matrix, it is returned instead. This method provides a safe way to take a kernel matrix as input, while preserving compatibility with many other algorithms that take a vector array. If Y is given (default is None), then the returned matrix is the pairwise kernel between the arrays from both X and Y.  Valid values for metric are:

[‘additive_chi2’, ‘chi2’, ‘linear’, ‘poly’, ‘polynomial’, ‘rbf’, ‘laplacian’, ‘sigmoid’, ‘cosine’]   Read more in the User Guide.  Parameters

Xndarray of shape (n_samples_X, n_samples_X) or (n_samples_X, n_features)

Array of pairwise kernels between samples, or a feature array. The shape of the array should be (n_samples_X, n_samples_X) if metric == “precomputed” and (n_samples_X, n_features) otherwise.
Yndarray of shape (n_samples_Y, n_features), default=None

A second feature array only if X has shape (n_samples_X, n_features).
metricstr or callable, default=”linear”

The metric to use when calculating kernel between instances in a feature array. If metric is a string, it must be one of the metrics in pairwise.PAIRWISE_KERNEL_FUNCTIONS. If metric is “precomputed”, X is assumed to be a kernel matrix. Alternatively, if metric is a callable function, it is called on each pair of instances (rows) and the resulting value recorded. The callable should take two rows from X as input and return the corresponding kernel value as a single number. This means that callables from sklearn.metrics.pairwise are not allowed, as they operate on matrices, not single samples. Use the string identifying the kernel instead.
filter_paramsbool, default=False

Whether to filter invalid parameters or not.
n_jobsint, default=None

The number of jobs to use for the computation. This works by breaking down the pairwise matrix into n_jobs even slices and computing them in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
**kwdsoptional keyword parameters

Any further parameters are passed directly to the kernel function.    Returns

Kndarray of shape (n_samples_X, n_samples_X) or (n_samples_X, n_samples_Y)

A kernel matrix K such that K_{i, j} is the kernel between the ith and jth vectors of the given matrix X, if Y is None. If Y is not None, then K_{i, j} is the kernel between the ith array from X and the jth array from Y.     Notes If metric is ‘precomputed’, Y is ignored and X is returned.


sklearn.metrics.pairwise.polynomial_kernel(X, Y=None, degree=3, gamma=None, coef0=1) [source]

Compute the polynomial kernel between X and Y: K(X, Y) = (gamma <X, Y> + coef0)^degree
 Read more in the User Guide.  Parameters

Xndarray of shape (n_samples_X, n_features)

Yndarray of shape (n_samples_Y, n_features), default=None

degreeint, default=3

gammafloat, default=None

If None, defaults to 1.0 / n_features.
coef0float, default=1
  Returns

Gram matrixndarray of shape (n_samples_X, n_samples_Y)


sklearn.metrics.pairwise.rbf_kernel(X, Y=None, gamma=None) [source]

Compute the rbf (gaussian) kernel between X and Y: K(x, y) = exp(-gamma ||x-y||^2)
 for each pair of rows x in X and y in Y. Read more in the User Guide.  Parameters

Xndarray of shape (n_samples_X, n_features)

Yndarray of shape (n_samples_Y, n_features), default=None

gammafloat, default=None

If None, defaults to 1.0 / n_features.    Returns

kernel_matrixndarray of shape (n_samples_X, n_samples_Y)


sklearn.metrics.pairwise.sigmoid_kernel(X, Y=None, gamma=None, coef0=1) [source]

Compute the sigmoid kernel between X and Y: K(X, Y) = tanh(gamma <X, Y> + coef0)
 Read more in the User Guide.  Parameters

Xndarray of shape (n_samples_X, n_features)

Yndarray of shape (n_samples_Y, n_features), default=None

gammafloat, default=None

If None, defaults to 1.0 / n_features.
coef0float, default=1
  Returns

Gram matrixndarray of shape (n_samples_X, n_samples_Y)


sklearn.metrics.pairwise_distances(X, Y=None, metric='euclidean', *, n_jobs=None, force_all_finite=True, **kwds) [source]

Compute the distance matrix from a vector array X and optional Y. This method takes either a vector array or a distance matrix, and returns a distance matrix. If the input is a vector array, the distances are computed. If the input is a distances matrix, it is returned instead. This method provides a safe way to take a distance matrix as input, while preserving compatibility with many other algorithms that take a vector array. If Y is given (default is None), then the returned matrix is the pairwise distance between the arrays from both X and Y. Valid values for metric are:  From scikit-learn: [‘cityblock’, ‘cosine’, ‘euclidean’, ‘l1’, ‘l2’, ‘manhattan’]. These metrics support sparse matrix inputs. [‘nan_euclidean’] but it does not yet support sparse matrices. From scipy.spatial.distance: [‘braycurtis’, ‘canberra’, ‘chebyshev’, ‘correlation’, ‘dice’, ‘hamming’, ‘jaccard’, ‘kulsinski’, ‘mahalanobis’, ‘minkowski’, ‘rogerstanimoto’, ‘russellrao’, ‘seuclidean’, ‘sokalmichener’, ‘sokalsneath’, ‘sqeuclidean’, ‘yule’] See the documentation for scipy.spatial.distance for details on these metrics. These metrics do not support sparse matrix inputs.  Note that in the case of ‘cityblock’, ‘cosine’ and ‘euclidean’ (which are valid scipy.spatial.distance metrics), the scikit-learn implementation will be used, which is faster and has support for sparse matrices (except for ‘cityblock’). For a verbose description of the metrics from scikit-learn, see the __doc__ of the sklearn.pairwise.distance_metrics function. Read more in the User Guide.  Parameters

Xndarray of shape (n_samples_X, n_samples_X) or (n_samples_X, n_features)

Array of pairwise distances between samples, or a feature array. The shape of the array should be (n_samples_X, n_samples_X) if metric == “precomputed” and (n_samples_X, n_features) otherwise.
Yndarray of shape (n_samples_Y, n_features), default=None

An optional second feature array. Only allowed if metric != “precomputed”.
metricstr or callable, default=’euclidean’

The metric to use when calculating distance between instances in a feature array. If metric is a string, it must be one of the options allowed by scipy.spatial.distance.pdist for its metric parameter, or a metric listed in pairwise.PAIRWISE_DISTANCE_FUNCTIONS. If metric is “precomputed”, X is assumed to be a distance matrix. Alternatively, if metric is a callable function, it is called on each pair of instances (rows) and the resulting value recorded. The callable should take two arrays from X as input and return a value indicating the distance between them.
n_jobsint, default=None

The number of jobs to use for the computation. This works by breaking down the pairwise matrix into n_jobs even slices and computing them in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
force_all_finitebool or ‘allow-nan’, default=True

Whether to raise an error on np.inf, np.nan, pd.NA in array. Ignored for a metric listed in pairwise.PAIRWISE_DISTANCE_FUNCTIONS. The possibilities are:  True: Force all values of array to be finite. False: accepts np.inf, np.nan, pd.NA in array. ‘allow-nan’: accepts only np.nan and pd.NA values in array. Values cannot be infinite.   New in version 0.22: force_all_finite accepts the string 'allow-nan'.   Changed in version 0.23: Accepts pd.NA and converts it into np.nan.
**kwdsoptional keyword parameters

Any further parameters are passed directly to the distance function. If using a scipy.spatial.distance metric, the parameters are still metric dependent. See the scipy docs for usage examples.    Returns

Dndarray of shape (n_samples_X, n_samples_X) or (n_samples_X, n_samples_Y)

A distance matrix D such that D_{i, j} is the distance between the ith and jth vectors of the given matrix X, if Y is None. If Y is not None, then D_{i, j} is the distance between the ith array from X and the jth array from Y.      See also
pairwise_distances_chunked

Performs the same calculation as this function, but returns a generator of chunks of the distance matrix, in order to limit memory usage.
paired_distances

Computes the distances between corresponding elements of two arrays.


sklearn.metrics.pairwise_distances_argmin(X, Y, *, axis=1, metric='euclidean', metric_kwargs=None) [source]

Compute minimum distances between one point and a set of points. This function computes for each row in X, the index of the row of Y which is closest (according to the specified distance). This is mostly equivalent to calling: pairwise_distances(X, Y=Y, metric=metric).argmin(axis=axis) but uses much less memory, and is faster for large arrays. This function works with dense 2D arrays only.  Parameters

Xarray-like of shape (n_samples_X, n_features)

Array containing points.
Yarray-like of shape (n_samples_Y, n_features)

Arrays containing points.
axisint, default=1

Axis along which the argmin and distances are to be computed.
metricstr or callable, default=”euclidean”

Metric to use for distance computation. Any metric from scikit-learn or scipy.spatial.distance can be used. If metric is a callable function, it is called on each pair of instances (rows) and the resulting value recorded. The callable should take two arrays as input and return one value indicating the distance between them. This works for Scipy’s metrics, but is less efficient than passing the metric name as a string. Distance matrices are not supported. Valid values for metric are:  from scikit-learn: [‘cityblock’, ‘cosine’, ‘euclidean’, ‘l1’, ‘l2’, ‘manhattan’] from scipy.spatial.distance: [‘braycurtis’, ‘canberra’, ‘chebyshev’, ‘correlation’, ‘dice’, ‘hamming’, ‘jaccard’, ‘kulsinski’, ‘mahalanobis’, ‘minkowski’, ‘rogerstanimoto’, ‘russellrao’, ‘seuclidean’, ‘sokalmichener’, ‘sokalsneath’, ‘sqeuclidean’, ‘yule’]  See the documentation for scipy.spatial.distance for details on these metrics.
metric_kwargsdict, default=None

Keyword arguments to pass to specified metric function.    Returns

argminnumpy.ndarray

Y[argmin[i], :] is the row in Y that is closest to X[i, :].      See also
sklearn.metrics.pairwise_distances

sklearn.metrics.pairwise_distances_argmin_min


sklearn.metrics.pairwise_distances_argmin_min(X, Y, *, axis=1, metric='euclidean', metric_kwargs=None) [source]

Compute minimum distances between one point and a set of points. This function computes for each row in X, the index of the row of Y which is closest (according to the specified distance). The minimal distances are also returned. This is mostly equivalent to calling:  (pairwise_distances(X, Y=Y, metric=metric).argmin(axis=axis),

pairwise_distances(X, Y=Y, metric=metric).min(axis=axis))   but uses much less memory, and is faster for large arrays.  Parameters

X{array-like, sparse matrix} of shape (n_samples_X, n_features)

Array containing points.
Y{array-like, sparse matrix} of shape (n_samples_Y, n_features)

Array containing points.
axisint, default=1

Axis along which the argmin and distances are to be computed.
metricstr or callable, default=’euclidean’

Metric to use for distance computation. Any metric from scikit-learn or scipy.spatial.distance can be used. If metric is a callable function, it is called on each pair of instances (rows) and the resulting value recorded. The callable should take two arrays as input and return one value indicating the distance between them. This works for Scipy’s metrics, but is less efficient than passing the metric name as a string. Distance matrices are not supported. Valid values for metric are:  from scikit-learn: [‘cityblock’, ‘cosine’, ‘euclidean’, ‘l1’, ‘l2’, ‘manhattan’] from scipy.spatial.distance: [‘braycurtis’, ‘canberra’, ‘chebyshev’, ‘correlation’, ‘dice’, ‘hamming’, ‘jaccard’, ‘kulsinski’, ‘mahalanobis’, ‘minkowski’, ‘rogerstanimoto’, ‘russellrao’, ‘seuclidean’, ‘sokalmichener’, ‘sokalsneath’, ‘sqeuclidean’, ‘yule’]  See the documentation for scipy.spatial.distance for details on these metrics.
metric_kwargsdict, default=None

Keyword arguments to pass to specified metric function.    Returns

argminndarray

Y[argmin[i], :] is the row in Y that is closest to X[i, :].
distancesndarray

distances[i] is the distance between the i-th row in X and the argmin[i]-th row in Y.      See also
sklearn.metrics.pairwise_distances

sklearn.metrics.pairwise_distances_argmin


sklearn.metrics.pairwise_distances_chunked(X, Y=None, *, reduce_func=None, metric='euclidean', n_jobs=None, working_memory=None, **kwds) [source]

Generate a distance matrix chunk by chunk with optional reduction. In cases where not all of a pairwise distance matrix needs to be stored at once, this is used to calculate pairwise distances in working_memory-sized chunks. If reduce_func is given, it is run on each chunk and its return values are concatenated into lists, arrays or sparse matrices.  Parameters

Xndarray of shape (n_samples_X, n_samples_X) or (n_samples_X, n_features)

Array of pairwise distances between samples, or a feature array. The shape the array should be (n_samples_X, n_samples_X) if metric=’precomputed’ and (n_samples_X, n_features) otherwise.
Yndarray of shape (n_samples_Y, n_features), default=None

An optional second feature array. Only allowed if metric != “precomputed”.
reduce_funccallable, default=None

The function which is applied on each chunk of the distance matrix, reducing it to needed values. reduce_func(D_chunk, start) is called repeatedly, where D_chunk is a contiguous vertical slice of the pairwise distance matrix, starting at row start. It should return one of: None; an array, a list, or a sparse matrix of length D_chunk.shape[0]; or a tuple of such objects. Returning None is useful for in-place operations, rather than reductions. If None, pairwise_distances_chunked returns a generator of vertical chunks of the distance matrix.
metricstr or callable, default=’euclidean’

The metric to use when calculating distance between instances in a feature array. If metric is a string, it must be one of the options allowed by scipy.spatial.distance.pdist for its metric parameter, or a metric listed in pairwise.PAIRWISE_DISTANCE_FUNCTIONS. If metric is “precomputed”, X is assumed to be a distance matrix. Alternatively, if metric is a callable function, it is called on each pair of instances (rows) and the resulting value recorded. The callable should take two arrays from X as input and return a value indicating the distance between them.
n_jobsint, default=None

The number of jobs to use for the computation. This works by breaking down the pairwise matrix into n_jobs even slices and computing them in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
working_memoryint, default=None

The sought maximum memory for temporary distance matrix chunks. When None (default), the value of sklearn.get_config()['working_memory'] is used.
`**kwds`optional keyword parameters

Any further parameters are passed directly to the distance function. If using a scipy.spatial.distance metric, the parameters are still metric dependent. See the scipy docs for usage examples.    Yields

D_chunk{ndarray, sparse matrix}

A contiguous slice of distance matrix, optionally processed by reduce_func.     Examples Without reduce_func: >>> import numpy as np
>>> from sklearn.metrics import pairwise_distances_chunked
>>> X = np.random.RandomState(0).rand(5, 3)
>>> D_chunk = next(pairwise_distances_chunked(X))
>>> D_chunk
array([[0.  ..., 0.29..., 0.41..., 0.19..., 0.57...],
       [0.29..., 0.  ..., 0.57..., 0.41..., 0.76...],
       [0.41..., 0.57..., 0.  ..., 0.44..., 0.90...],
       [0.19..., 0.41..., 0.44..., 0.  ..., 0.51...],
       [0.57..., 0.76..., 0.90..., 0.51..., 0.  ...]])
 Retrieve all neighbors and average distance within radius r: >>> r = .2
>>> def reduce_func(D_chunk, start):
...     neigh = [np.flatnonzero(d < r) for d in D_chunk]
...     avg_dist = (D_chunk * (D_chunk < r)).mean(axis=1)
...     return neigh, avg_dist
>>> gen = pairwise_distances_chunked(X, reduce_func=reduce_func)
>>> neigh, avg_dist = next(gen)
>>> neigh
[array([0, 3]), array([1]), array([2]), array([0, 3]), array([4])]
>>> avg_dist
array([0.039..., 0.        , 0.        , 0.039..., 0.        ])
 Where r is defined per sample, we need to make use of start: >>> r = [.2, .4, .4, .3, .1]
>>> def reduce_func(D_chunk, start):
...     neigh = [np.flatnonzero(d < r[i])
...              for i, d in enumerate(D_chunk, start)]
...     return neigh
>>> neigh = next(pairwise_distances_chunked(X, reduce_func=reduce_func))
>>> neigh
[array([0, 3]), array([0, 1]), array([2]), array([0, 3]), array([4])]
 Force row-by-row generation by reducing working_memory: >>> gen = pairwise_distances_chunked(X, reduce_func=reduce_func,
...                                  working_memory=0)
>>> next(gen)
[array([0, 3])]
>>> next(gen)
[array([0, 1])]


sklearn.metrics.plot_confusion_matrix(estimator, X, y_true, *, labels=None, sample_weight=None, normalize=None, display_labels=None, include_values=True, xticks_rotation='horizontal', values_format=None, cmap='viridis', ax=None, colorbar=True) [source]

Plot Confusion Matrix. Read more in the User Guide.  Parameters

estimatorestimator instance

Fitted classifier or a fitted Pipeline in which the last estimator is a classifier.
X{array-like, sparse matrix} of shape (n_samples, n_features)

Input values.
y_truearray-like of shape (n_samples,)

Target values.
labelsarray-like of shape (n_classes,), default=None

List of labels to index the matrix. This may be used to reorder or select a subset of labels. If None is given, those that appear at least once in y_true or y_pred are used in sorted order.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
normalize{‘true’, ‘pred’, ‘all’}, default=None

Normalizes confusion matrix over the true (rows), predicted (columns) conditions or all the population. If None, confusion matrix will not be normalized.
display_labelsarray-like of shape (n_classes,), default=None

Target names used for plotting. By default, labels will be used if it is defined, otherwise the unique labels of y_true and y_pred will be used.
include_valuesbool, default=True

Includes values in confusion matrix.
xticks_rotation{‘vertical’, ‘horizontal’} or float, default=’horizontal’

Rotation of xtick labels.
values_formatstr, default=None

Format specification for values in confusion matrix. If None, the format specification is ‘d’ or ‘.2g’ whichever is shorter.
cmapstr or matplotlib Colormap, default=’viridis’

Colormap recognized by matplotlib.
axmatplotlib Axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
colorbarbool, default=True

Whether or not to add a colorbar to the plot.  New in version 0.24.     Returns

displayConfusionMatrixDisplay
    See also
confusion_matrix

Compute Confusion Matrix to evaluate the accuracy of a classification.
ConfusionMatrixDisplay

Confusion Matrix visualization.    Examples >>> import matplotlib.pyplot as plt
>>> from sklearn.datasets import make_classification
>>> from sklearn.metrics import plot_confusion_matrix
>>> from sklearn.model_selection import train_test_split
>>> from sklearn.svm import SVC
>>> X, y = make_classification(random_state=0)
>>> X_train, X_test, y_train, y_test = train_test_split(
...         X, y, random_state=0)
>>> clf = SVC(random_state=0)
>>> clf.fit(X_train, y_train)
SVC(random_state=0)
>>> plot_confusion_matrix(clf, X_test, y_test)
>>> plt.show()


sklearn.metrics.plot_det_curve(estimator, X, y, *, sample_weight=None, response_method='auto', name=None, ax=None, pos_label=None, **kwargs) [source]

Plot detection error tradeoff (DET) curve. Extra keyword arguments will be passed to matplotlib’s plot. Read more in the User Guide.  New in version 0.24.   Parameters

estimatorestimator instance

Fitted classifier or a fitted Pipeline in which the last estimator is a classifier.
X{array-like, sparse matrix} of shape (n_samples, n_features)

Input values.
yarray-like of shape (n_samples,)

Target values.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
response_method{‘predict_proba’, ‘decision_function’, ‘auto’} default=’auto’

Specifies whether to use predict_proba or decision_function as the predicted target response. If set to ‘auto’, predict_proba is tried first and if it does not exist decision_function is tried next.
namestr, default=None

Name of DET curve for labeling. If None, use the name of the estimator.
axmatplotlib axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
pos_labelstr or int, default=None

The label of the positive class. When pos_label=None, if y_true is in {-1, 1} or {0, 1}, pos_label is set to 1, otherwise an error will be raised.    Returns

displayDetCurveDisplay

Object that stores computed values.      See also
det_curve

Compute error rates for different probability thresholds.
DetCurveDisplay

DET curve visualization.
plot_roc_curve

Plot Receiver operating characteristic (ROC) curve.    Examples >>> import matplotlib.pyplot as plt
>>> from sklearn import datasets, metrics, model_selection, svm
>>> X, y = datasets.make_classification(random_state=0)
>>> X_train, X_test, y_train, y_test = model_selection.train_test_split(
...     X, y, random_state=0)
>>> clf = svm.SVC(random_state=0)
>>> clf.fit(X_train, y_train)
SVC(random_state=0)
>>> metrics.plot_det_curve(clf, X_test, y_test)
>>> plt.show()


sklearn.metrics.plot_precision_recall_curve(estimator, X, y, *, sample_weight=None, response_method='auto', name=None, ax=None, pos_label=None, **kwargs) [source]

Plot Precision Recall Curve for binary classifiers. Extra keyword arguments will be passed to matplotlib’s plot. Read more in the User Guide.  Parameters

estimatorestimator instance

Fitted classifier or a fitted Pipeline in which the last estimator is a classifier.
X{array-like, sparse matrix} of shape (n_samples, n_features)

Input values.
yarray-like of shape (n_samples,)

Binary target values.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
response_method{‘predict_proba’, ‘decision_function’, ‘auto’}, default=’auto’

Specifies whether to use predict_proba or decision_function as the target response. If set to ‘auto’, predict_proba is tried first and if it does not exist decision_function is tried next.
namestr, default=None

Name for labeling curve. If None, the name of the estimator is used.
axmatplotlib axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
pos_labelstr or int, default=None

The class considered as the positive class when computing the precision and recall metrics. By default, estimators.classes_[1] is considered as the positive class.  New in version 0.24.
**kwargsdict

Keyword arguments to be passed to matplotlib’s plot.    Returns

displayPrecisionRecallDisplay

Object that stores computed values.      See also
precision_recall_curve

Compute precision-recall pairs for different probability thresholds.
PrecisionRecallDisplay

Precision Recall visualization.


sklearn.metrics.plot_roc_curve(estimator, X, y, *, sample_weight=None, drop_intermediate=True, response_method='auto', name=None, ax=None, pos_label=None, **kwargs) [source]

Plot Receiver operating characteristic (ROC) curve. Extra keyword arguments will be passed to matplotlib’s plot. Read more in the User Guide.  Parameters

estimatorestimator instance

Fitted classifier or a fitted Pipeline in which the last estimator is a classifier.
X{array-like, sparse matrix} of shape (n_samples, n_features)

Input values.
yarray-like of shape (n_samples,)

Target values.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
drop_intermediateboolean, default=True

Whether to drop some suboptimal thresholds which would not appear on a plotted ROC curve. This is useful in order to create lighter ROC curves.
response_method{‘predict_proba’, ‘decision_function’, ‘auto’} default=’auto’

Specifies whether to use predict_proba or decision_function as the target response. If set to ‘auto’, predict_proba is tried first and if it does not exist decision_function is tried next.
namestr, default=None

Name of ROC Curve for labeling. If None, use the name of the estimator.
axmatplotlib axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
pos_labelstr or int, default=None

The class considered as the positive class when computing the roc auc metrics. By default, estimators.classes_[1] is considered as the positive class.  New in version 0.24.     Returns

displayRocCurveDisplay

Object that stores computed values.      See also
roc_curve

Compute Receiver operating characteristic (ROC) curve.
RocCurveDisplay

ROC Curve visualization.
roc_auc_score

Compute the area under the ROC curve.    Examples >>> import matplotlib.pyplot as plt
>>> from sklearn import datasets, metrics, model_selection, svm
>>> X, y = datasets.make_classification(random_state=0)
>>> X_train, X_test, y_train, y_test = model_selection.train_test_split(
...     X, y, random_state=0)
>>> clf = svm.SVC(random_state=0)
>>> clf.fit(X_train, y_train)
SVC(random_state=0)
>>> metrics.plot_roc_curve(clf, X_test, y_test)
>>> plt.show()


class sklearn.metrics.PrecisionRecallDisplay(precision, recall, *, average_precision=None, estimator_name=None, pos_label=None) [source]

Precision Recall visualization. It is recommend to use plot_precision_recall_curve to create a visualizer. All parameters are stored as attributes. Read more in the User Guide.  Parameters

precisionndarray

Precision values.
recallndarray

Recall values.
average_precisionfloat, default=None

Average precision. If None, the average precision is not shown.
estimator_namestr, default=None

Name of estimator. If None, then the estimator name is not shown.
pos_labelstr or int, default=None

The class considered as the positive class. If None, the class will not be shown in the legend.  New in version 0.24.     Attributes

line_matplotlib Artist

Precision recall curve.
ax_matplotlib Axes

Axes with precision recall curve.
figure_matplotlib Figure

Figure containing the curve.      See also
precision_recall_curve

Compute precision-recall pairs for different probability thresholds.
plot_precision_recall_curve

Plot Precision Recall Curve for binary classifiers.    Examples >>> from sklearn.datasets import make_classification
>>> from sklearn.metrics import (precision_recall_curve,
...                              PrecisionRecallDisplay)
>>> from sklearn.model_selection import train_test_split
>>> from sklearn.svm import SVC
>>> X, y = make_classification(random_state=0)
>>> X_train, X_test, y_train, y_test = train_test_split(X, y,
...                                                     random_state=0)
>>> clf = SVC(random_state=0)
>>> clf.fit(X_train, y_train)
SVC(random_state=0)
>>> predictions = clf.predict(X_test)
>>> precision, recall, _ = precision_recall_curve(y_test, predictions)
>>> disp = PrecisionRecallDisplay(precision=precision, recall=recall)
>>> disp.plot()
 Methods
plot([ax, name]) Plot visualization.
plot(ax=None, *, name=None, **kwargs) [source]

Plot visualization. Extra keyword arguments will be passed to matplotlib’s plot.  Parameters

axMatplotlib Axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
namestr, default=None

Name of precision recall curve for labeling. If None, use the name of the estimator.
**kwargsdict

Keyword arguments to be passed to matplotlib’s plot.    Returns

displayPrecisionRecallDisplay

Object that stores computed values.


sklearn.metrics.PrecisionRecallDisplay
class sklearn.metrics.PrecisionRecallDisplay(precision, recall, *, average_precision=None, estimator_name=None, pos_label=None) [source]

Precision Recall visualization. It is recommend to use plot_precision_recall_curve to create a visualizer. All parameters are stored as attributes. Read more in the User Guide.  Parameters

precisionndarray

Precision values.
recallndarray

Recall values.
average_precisionfloat, default=None

Average precision. If None, the average precision is not shown.
estimator_namestr, default=None

Name of estimator. If None, then the estimator name is not shown.
pos_labelstr or int, default=None

The class considered as the positive class. If None, the class will not be shown in the legend.  New in version 0.24.     Attributes

line_matplotlib Artist

Precision recall curve.
ax_matplotlib Axes

Axes with precision recall curve.
figure_matplotlib Figure

Figure containing the curve.      See also
precision_recall_curve

Compute precision-recall pairs for different probability thresholds.
plot_precision_recall_curve

Plot Precision Recall Curve for binary classifiers.    Examples >>> from sklearn.datasets import make_classification
>>> from sklearn.metrics import (precision_recall_curve,
...                              PrecisionRecallDisplay)
>>> from sklearn.model_selection import train_test_split
>>> from sklearn.svm import SVC
>>> X, y = make_classification(random_state=0)
>>> X_train, X_test, y_train, y_test = train_test_split(X, y,
...                                                     random_state=0)
>>> clf = SVC(random_state=0)
>>> clf.fit(X_train, y_train)
SVC(random_state=0)
>>> predictions = clf.predict(X_test)
>>> precision, recall, _ = precision_recall_curve(y_test, predictions)
>>> disp = PrecisionRecallDisplay(precision=precision, recall=recall)
>>> disp.plot()
 Methods
plot([ax, name]) Plot visualization.
plot(ax=None, *, name=None, **kwargs) [source]

Plot visualization. Extra keyword arguments will be passed to matplotlib’s plot.  Parameters

axMatplotlib Axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
namestr, default=None

Name of precision recall curve for labeling. If None, use the name of the estimator.
**kwargsdict

Keyword arguments to be passed to matplotlib’s plot.    Returns

displayPrecisionRecallDisplay

Object that stores computed values.

 Examples using sklearn.metrics.PrecisionRecallDisplay

  Visualizations with Display Objects


plot(ax=None, *, name=None, **kwargs) [source]

Plot visualization. Extra keyword arguments will be passed to matplotlib’s plot.  Parameters

axMatplotlib Axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
namestr, default=None

Name of precision recall curve for labeling. If None, use the name of the estimator.
**kwargsdict

Keyword arguments to be passed to matplotlib’s plot.    Returns

displayPrecisionRecallDisplay

Object that stores computed values.


sklearn.metrics.precision_recall_curve(y_true, probas_pred, *, pos_label=None, sample_weight=None) [source]

Compute precision-recall pairs for different probability thresholds. Note: this implementation is restricted to the binary classification task. The precision is the ratio tp / (tp + fp) where tp is the number of true positives and fp the number of false positives. The precision is intuitively the ability of the classifier not to label as positive a sample that is negative. The recall is the ratio tp / (tp + fn) where tp is the number of true positives and fn the number of false negatives. The recall is intuitively the ability of the classifier to find all the positive samples. The last precision and recall values are 1. and 0. respectively and do not have a corresponding threshold. This ensures that the graph starts on the y axis. Read more in the User Guide.  Parameters

y_truendarray of shape (n_samples,)

True binary labels. If labels are not either {-1, 1} or {0, 1}, then pos_label should be explicitly given.
probas_predndarray of shape (n_samples,)

Estimated probabilities or output of a decision function.
pos_labelint or str, default=None

The label of the positive class. When pos_label=None, if y_true is in {-1, 1} or {0, 1}, pos_label is set to 1, otherwise an error will be raised.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

precisionndarray of shape (n_thresholds + 1,)

Precision values such that element i is the precision of predictions with score >= thresholds[i] and the last element is 1.
recallndarray of shape (n_thresholds + 1,)

Decreasing recall values such that element i is the recall of predictions with score >= thresholds[i] and the last element is 0.
thresholdsndarray of shape (n_thresholds,)

Increasing thresholds on the decision function used to compute precision and recall. n_thresholds <= len(np.unique(probas_pred)).      See also
plot_precision_recall_curve

Plot Precision Recall Curve for binary classifiers.
PrecisionRecallDisplay

Precision Recall visualization.
average_precision_score

Compute average precision from prediction scores.
det_curve

Compute error rates for different probability thresholds.
roc_curve

Compute Receiver operating characteristic (ROC) curve.    Examples >>> import numpy as np
>>> from sklearn.metrics import precision_recall_curve
>>> y_true = np.array([0, 0, 1, 1])
>>> y_scores = np.array([0.1, 0.4, 0.35, 0.8])
>>> precision, recall, thresholds = precision_recall_curve(
...     y_true, y_scores)
>>> precision
array([0.66666667, 0.5       , 1.        , 1.        ])
>>> recall
array([1. , 0.5, 0.5, 0. ])
>>> thresholds
array([0.35, 0.4 , 0.8 ])


sklearn.metrics.precision_recall_fscore_support(y_true, y_pred, *, beta=1.0, labels=None, pos_label=1, average=None, warn_for='precision', 'recall', 'f-score', sample_weight=None, zero_division='warn') [source]

Compute precision, recall, F-measure and support for each class. The precision is the ratio tp / (tp + fp) where tp is the number of true positives and fp the number of false positives. The precision is intuitively the ability of the classifier not to label as positive a sample that is negative. The recall is the ratio tp / (tp + fn) where tp is the number of true positives and fn the number of false negatives. The recall is intuitively the ability of the classifier to find all the positive samples. The F-beta score can be interpreted as a weighted harmonic mean of the precision and recall, where an F-beta score reaches its best value at 1 and worst score at 0. The F-beta score weights recall more than precision by a factor of beta. beta == 1.0 means recall and precision are equally important. The support is the number of occurrences of each class in y_true. If pos_label is None and in binary classification, this function returns the average precision, recall and F-measure if average is one of 'micro', 'macro', 'weighted' or 'samples'. Read more in the User Guide.  Parameters

y_true1d array-like, or label indicator array / sparse matrix

Ground truth (correct) target values.
y_pred1d array-like, or label indicator array / sparse matrix

Estimated targets as returned by a classifier.
betafloat, default=1.0

The strength of recall versus precision in the F-score.
labelsarray-like, default=None

The set of labels to include when average != 'binary', and their order if average is None. Labels present in the data can be excluded, for example to calculate a multiclass average ignoring a majority negative class, while labels not present in the data will result in 0 components in a macro average. For multilabel targets, labels are column indices. By default, all labels in y_true and y_pred are used in sorted order.
pos_labelstr or int, default=1

The class to report if average='binary' and the data is binary. If the data are multiclass or multilabel, this will be ignored; setting labels=[pos_label] and average != 'binary' will report scores for that label only.
average{‘binary’, ‘micro’, ‘macro’, ‘samples’,’weighted’}, default=None

If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data:
'binary':

Only report results for the class specified by pos_label. This is applicable only if targets (y_{true,pred}) are binary.
'micro':

Calculate metrics globally by counting the total true positives, false negatives and false positives.
'macro':

Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.
'weighted':

Calculate metrics for each label, and find their average weighted by support (the number of true instances for each label). This alters ‘macro’ to account for label imbalance; it can result in an F-score that is not between precision and recall.
'samples':

Calculate metrics for each instance, and find their average (only meaningful for multilabel classification where this differs from accuracy_score).
warn_fortuple or set, for internal use

This determines which warnings will be made in the case that this function is being used to return only one of its metrics.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
zero_division“warn”, 0 or 1, default=”warn”

 Sets the value to return when there is a zero division:

 recall: when there are no positive labels precision: when there are no positive predictions f-score: both    If set to “warn”, this acts as 0, but warnings are also raised.    Returns

precisionfloat (if average is not None) or array of float, shape = [n_unique_labels]

recallfloat (if average is not None) or array of float, , shape = [n_unique_labels]

fbeta_scorefloat (if average is not None) or array of float, shape = [n_unique_labels]

supportNone (if average is not None) or array of int, shape = [n_unique_labels]

The number of occurrences of each label in y_true.     Notes When true positive + false positive == 0, precision is undefined. When true positive + false negative == 0, recall is undefined. In such cases, by default the metric will be set to 0, as will f-score, and UndefinedMetricWarning will be raised. This behavior can be modified with zero_division. References
1
Wikipedia entry for the Precision and recall.
2
Wikipedia entry for the F1-score.
3
Discriminative Methods for Multi-labeled Classification Advances in Knowledge Discovery and Data Mining (2004), pp. 22-30 by Shantanu Godbole, Sunita Sarawagi.   Examples >>> import numpy as np
>>> from sklearn.metrics import precision_recall_fscore_support
>>> y_true = np.array(['cat', 'dog', 'pig', 'cat', 'dog', 'pig'])
>>> y_pred = np.array(['cat', 'pig', 'dog', 'cat', 'cat', 'dog'])
>>> precision_recall_fscore_support(y_true, y_pred, average='macro')
(0.22..., 0.33..., 0.26..., None)
>>> precision_recall_fscore_support(y_true, y_pred, average='micro')
(0.33..., 0.33..., 0.33..., None)
>>> precision_recall_fscore_support(y_true, y_pred, average='weighted')
(0.22..., 0.33..., 0.26..., None)
 It is possible to compute per-label precisions, recalls, F1-scores and supports instead of averaging: >>> precision_recall_fscore_support(y_true, y_pred, average=None,
... labels=['pig', 'dog', 'cat'])
(array([0.        , 0.        , 0.66...]),
 array([0., 0., 1.]), array([0. , 0. , 0.8]),
 array([2, 2, 2]))


sklearn.metrics.precision_score(y_true, y_pred, *, labels=None, pos_label=1, average='binary', sample_weight=None, zero_division='warn') [source]

Compute the precision. The precision is the ratio tp / (tp + fp) where tp is the number of true positives and fp the number of false positives. The precision is intuitively the ability of the classifier not to label as positive a sample that is negative. The best value is 1 and the worst value is 0. Read more in the User Guide.  Parameters

y_true1d array-like, or label indicator array / sparse matrix

Ground truth (correct) target values.
y_pred1d array-like, or label indicator array / sparse matrix

Estimated targets as returned by a classifier.
labelsarray-like, default=None

The set of labels to include when average != 'binary', and their order if average is None. Labels present in the data can be excluded, for example to calculate a multiclass average ignoring a majority negative class, while labels not present in the data will result in 0 components in a macro average. For multilabel targets, labels are column indices. By default, all labels in y_true and y_pred are used in sorted order.  Changed in version 0.17: Parameter labels improved for multiclass problem.
pos_labelstr or int, default=1

The class to report if average='binary' and the data is binary. If the data are multiclass or multilabel, this will be ignored; setting labels=[pos_label] and average != 'binary' will report scores for that label only.
average{‘micro’, ‘macro’, ‘samples’, ‘weighted’, ‘binary’} default=’binary’

This parameter is required for multiclass/multilabel targets. If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data:
'binary':

Only report results for the class specified by pos_label. This is applicable only if targets (y_{true,pred}) are binary.
'micro':

Calculate metrics globally by counting the total true positives, false negatives and false positives.
'macro':

Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.
'weighted':

Calculate metrics for each label, and find their average weighted by support (the number of true instances for each label). This alters ‘macro’ to account for label imbalance; it can result in an F-score that is not between precision and recall.
'samples':

Calculate metrics for each instance, and find their average (only meaningful for multilabel classification where this differs from accuracy_score).
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
zero_division“warn”, 0 or 1, default=”warn”

Sets the value to return when there is a zero division. If set to “warn”, this acts as 0, but warnings are also raised.    Returns

precisionfloat (if average is not None) or array of float of shape

(n_unique_labels,) Precision of the positive class in binary classification or weighted average of the precision of each class for the multiclass task.      See also
precision_recall_fscore_support, multilabel_confusion_matrix

  Notes When true positive + false positive == 0, precision returns 0 and raises UndefinedMetricWarning. This behavior can be modified with zero_division. Examples >>> from sklearn.metrics import precision_score
>>> y_true = [0, 1, 2, 0, 1, 2]
>>> y_pred = [0, 2, 1, 0, 0, 1]
>>> precision_score(y_true, y_pred, average='macro')
0.22...
>>> precision_score(y_true, y_pred, average='micro')
0.33...
>>> precision_score(y_true, y_pred, average='weighted')
0.22...
>>> precision_score(y_true, y_pred, average=None)
array([0.66..., 0.        , 0.        ])
>>> y_pred = [0, 0, 0, 0, 0, 0]
>>> precision_score(y_true, y_pred, average=None)
array([0.33..., 0.        , 0.        ])
>>> precision_score(y_true, y_pred, average=None, zero_division=1)
array([0.33..., 1.        , 1.        ])


sklearn.metrics.r2_score(y_true, y_pred, *, sample_weight=None, multioutput='uniform_average') [source]

R^2 (coefficient of determination) regression score function. Best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a R^2 score of 0.0. Read more in the User Guide.  Parameters

y_truearray-like of shape (n_samples,) or (n_samples, n_outputs)

Ground truth (correct) target values.
y_predarray-like of shape (n_samples,) or (n_samples, n_outputs)

Estimated target values.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
multioutput{‘raw_values’, ‘uniform_average’, ‘variance_weighted’}, array-like of shape (n_outputs,) or None, default=’uniform_average’

Defines aggregating of multiple output scores. Array-like value defines weights used to average scores. Default is “uniform_average”.  ‘raw_values’ :

Returns a full set of scores in case of multioutput input.  ‘uniform_average’ :

Scores of all outputs are averaged with uniform weight.  ‘variance_weighted’ :

Scores of all outputs are averaged, weighted by the variances of each individual output.    Changed in version 0.19: Default value of multioutput is ‘uniform_average’.     Returns

zfloat or ndarray of floats

The R^2 score or ndarray of scores if ‘multioutput’ is ‘raw_values’.     Notes This is not a symmetric function. Unlike most other scores, R^2 score may be negative (it need not actually be the square of a quantity R). This metric is not well-defined for single samples and will return a NaN value if n_samples is less than two. References
1
Wikipedia entry on the Coefficient of determination   Examples >>> from sklearn.metrics import r2_score
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> r2_score(y_true, y_pred)
0.948...
>>> y_true = [[0.5, 1], [-1, 1], [7, -6]]
>>> y_pred = [[0, 2], [-1, 2], [8, -5]]
>>> r2_score(y_true, y_pred,
...          multioutput='variance_weighted')
0.938...
>>> y_true = [1, 2, 3]
>>> y_pred = [1, 2, 3]
>>> r2_score(y_true, y_pred)
1.0
>>> y_true = [1, 2, 3]
>>> y_pred = [2, 2, 2]
>>> r2_score(y_true, y_pred)
0.0
>>> y_true = [1, 2, 3]
>>> y_pred = [3, 2, 1]
>>> r2_score(y_true, y_pred)
-3.0


sklearn.metrics.rand_score(labels_true, labels_pred) [source]

Rand index. The Rand Index computes a similarity measure between two clusterings by considering all pairs of samples and counting pairs that are assigned in the same or different clusters in the predicted and true clusterings. The raw RI score is: RI = (number of agreeing pairs) / (number of pairs) Read more in the User Guide.  Parameters

labels_truearray-like of shape (n_samples,), dtype=integral

Ground truth class labels to be used as a reference.
labels_predarray-like of shape (n_samples,), dtype=integral

Cluster labels to evaluate.    Returns

RIfloat

Similarity score between 0.0 and 1.0, inclusive, 1.0 stands for perfect match.      See also
adjusted_rand_score

Adjusted Rand Score
adjusted_mutual_info_score

Adjusted Mutual Information    References Examples Perfectly matching labelings have a score of 1 even >>> from sklearn.metrics.cluster import rand_score
>>> rand_score([0, 0, 1, 1], [1, 1, 0, 0])
1.0
 Labelings that assign all classes members to the same clusters are complete but may not always be pure, hence penalized: >>> rand_score([0, 0, 1, 2], [0, 0, 1, 1])
0.83...


sklearn.metrics.recall_score(y_true, y_pred, *, labels=None, pos_label=1, average='binary', sample_weight=None, zero_division='warn') [source]

Compute the recall. The recall is the ratio tp / (tp + fn) where tp is the number of true positives and fn the number of false negatives. The recall is intuitively the ability of the classifier to find all the positive samples. The best value is 1 and the worst value is 0. Read more in the User Guide.  Parameters

y_true1d array-like, or label indicator array / sparse matrix

Ground truth (correct) target values.
y_pred1d array-like, or label indicator array / sparse matrix

Estimated targets as returned by a classifier.
labelsarray-like, default=None

The set of labels to include when average != 'binary', and their order if average is None. Labels present in the data can be excluded, for example to calculate a multiclass average ignoring a majority negative class, while labels not present in the data will result in 0 components in a macro average. For multilabel targets, labels are column indices. By default, all labels in y_true and y_pred are used in sorted order.  Changed in version 0.17: Parameter labels improved for multiclass problem.
pos_labelstr or int, default=1

The class to report if average='binary' and the data is binary. If the data are multiclass or multilabel, this will be ignored; setting labels=[pos_label] and average != 'binary' will report scores for that label only.
average{‘micro’, ‘macro’, ‘samples’, ‘weighted’, ‘binary’} default=’binary’

This parameter is required for multiclass/multilabel targets. If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data:
'binary':

Only report results for the class specified by pos_label. This is applicable only if targets (y_{true,pred}) are binary.
'micro':

Calculate metrics globally by counting the total true positives, false negatives and false positives.
'macro':

Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.
'weighted':

Calculate metrics for each label, and find their average weighted by support (the number of true instances for each label). This alters ‘macro’ to account for label imbalance; it can result in an F-score that is not between precision and recall.
'samples':

Calculate metrics for each instance, and find their average (only meaningful for multilabel classification where this differs from accuracy_score).
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
zero_division“warn”, 0 or 1, default=”warn”

Sets the value to return when there is a zero division. If set to “warn”, this acts as 0, but warnings are also raised.    Returns

recallfloat (if average is not None) or array of float of shape

(n_unique_labels,) Recall of the positive class in binary classification or weighted average of the recall of each class for the multiclass task.      See also
precision_recall_fscore_support, balanced_accuracy_score

multilabel_confusion_matrix

  Notes When true positive + false negative == 0, recall returns 0 and raises UndefinedMetricWarning. This behavior can be modified with zero_division. Examples >>> from sklearn.metrics import recall_score
>>> y_true = [0, 1, 2, 0, 1, 2]
>>> y_pred = [0, 2, 1, 0, 0, 1]
>>> recall_score(y_true, y_pred, average='macro')
0.33...
>>> recall_score(y_true, y_pred, average='micro')
0.33...
>>> recall_score(y_true, y_pred, average='weighted')
0.33...
>>> recall_score(y_true, y_pred, average=None)
array([1., 0., 0.])
>>> y_true = [0, 0, 0, 0, 0, 0]
>>> recall_score(y_true, y_pred, average=None)
array([0.5, 0. , 0. ])
>>> recall_score(y_true, y_pred, average=None, zero_division=1)
array([0.5, 1. , 1. ])


class sklearn.metrics.RocCurveDisplay(*, fpr, tpr, roc_auc=None, estimator_name=None, pos_label=None) [source]

ROC Curve visualization. It is recommend to use plot_roc_curve to create a visualizer. All parameters are stored as attributes. Read more in the User Guide.  Parameters

fprndarray

False positive rate.
tprndarray

True positive rate.
roc_aucfloat, default=None

Area under ROC curve. If None, the roc_auc score is not shown.
estimator_namestr, default=None

Name of estimator. If None, the estimator name is not shown.
pos_labelstr or int, default=None

The class considered as the positive class when computing the roc auc metrics. By default, estimators.classes_[1] is considered as the positive class.  New in version 0.24.     Attributes

line_matplotlib Artist

ROC Curve.
ax_matplotlib Axes

Axes with ROC Curve.
figure_matplotlib Figure

Figure containing the curve.      See also
roc_curve

Compute Receiver operating characteristic (ROC) curve.
plot_roc_curve

Plot Receiver operating characteristic (ROC) curve.
roc_auc_score

Compute the area under the ROC curve.    Examples >>> import matplotlib.pyplot as plt
>>> import numpy as np
>>> from sklearn import metrics
>>> y = np.array([0, 0, 1, 1])
>>> pred = np.array([0.1, 0.4, 0.35, 0.8])
>>> fpr, tpr, thresholds = metrics.roc_curve(y, pred)
>>> roc_auc = metrics.auc(fpr, tpr)
>>> display = metrics.RocCurveDisplay(fpr=fpr, tpr=tpr, roc_auc=roc_auc,                                          estimator_name='example estimator')
>>> display.plot()
>>> plt.show()
 Methods
plot([ax, name]) Plot visualization
plot(ax=None, *, name=None, **kwargs) [source]

Plot visualization Extra keyword arguments will be passed to matplotlib’s plot.  Parameters

axmatplotlib axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
namestr, default=None

Name of ROC Curve for labeling. If None, use the name of the estimator.    Returns

displayRocCurveDisplay

Object that stores computed values.


sklearn.metrics.RocCurveDisplay
class sklearn.metrics.RocCurveDisplay(*, fpr, tpr, roc_auc=None, estimator_name=None, pos_label=None) [source]

ROC Curve visualization. It is recommend to use plot_roc_curve to create a visualizer. All parameters are stored as attributes. Read more in the User Guide.  Parameters

fprndarray

False positive rate.
tprndarray

True positive rate.
roc_aucfloat, default=None

Area under ROC curve. If None, the roc_auc score is not shown.
estimator_namestr, default=None

Name of estimator. If None, the estimator name is not shown.
pos_labelstr or int, default=None

The class considered as the positive class when computing the roc auc metrics. By default, estimators.classes_[1] is considered as the positive class.  New in version 0.24.     Attributes

line_matplotlib Artist

ROC Curve.
ax_matplotlib Axes

Axes with ROC Curve.
figure_matplotlib Figure

Figure containing the curve.      See also
roc_curve

Compute Receiver operating characteristic (ROC) curve.
plot_roc_curve

Plot Receiver operating characteristic (ROC) curve.
roc_auc_score

Compute the area under the ROC curve.    Examples >>> import matplotlib.pyplot as plt
>>> import numpy as np
>>> from sklearn import metrics
>>> y = np.array([0, 0, 1, 1])
>>> pred = np.array([0.1, 0.4, 0.35, 0.8])
>>> fpr, tpr, thresholds = metrics.roc_curve(y, pred)
>>> roc_auc = metrics.auc(fpr, tpr)
>>> display = metrics.RocCurveDisplay(fpr=fpr, tpr=tpr, roc_auc=roc_auc,                                          estimator_name='example estimator')
>>> display.plot()
>>> plt.show()
 Methods
plot([ax, name]) Plot visualization
plot(ax=None, *, name=None, **kwargs) [source]

Plot visualization Extra keyword arguments will be passed to matplotlib’s plot.  Parameters

axmatplotlib axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
namestr, default=None

Name of ROC Curve for labeling. If None, use the name of the estimator.    Returns

displayRocCurveDisplay

Object that stores computed values.

 Examples using sklearn.metrics.RocCurveDisplay

  Visualizations with Display Objects


plot(ax=None, *, name=None, **kwargs) [source]

Plot visualization Extra keyword arguments will be passed to matplotlib’s plot.  Parameters

axmatplotlib axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
namestr, default=None

Name of ROC Curve for labeling. If None, use the name of the estimator.    Returns

displayRocCurveDisplay

Object that stores computed values.


sklearn.metrics.roc_auc_score(y_true, y_score, *, average='macro', sample_weight=None, max_fpr=None, multi_class='raise', labels=None) [source]

Compute Area Under the Receiver Operating Characteristic Curve (ROC AUC) from prediction scores. Note: this implementation can be used with binary, multiclass and multilabel classification, but some restrictions apply (see Parameters). Read more in the User Guide.  Parameters

y_truearray-like of shape (n_samples,) or (n_samples, n_classes)

True labels or binary label indicators. The binary and multiclass cases expect labels with shape (n_samples,) while the multilabel case expects binary label indicators with shape (n_samples, n_classes).
y_scorearray-like of shape (n_samples,) or (n_samples, n_classes)

Target scores.  In the binary case, it corresponds to an array of shape (n_samples,). Both probability estimates and non-thresholded decision values can be provided. The probability estimates correspond to the probability of the class with the greater label, i.e. estimator.classes_[1] and thus estimator.predict_proba(X, y)[:, 1]. The decision values corresponds to the output of estimator.decision_function(X, y). See more information in the User guide; In the multiclass case, it corresponds to an array of shape (n_samples, n_classes) of probability estimates provided by the predict_proba method. The probability estimates must sum to 1 across the possible classes. In addition, the order of the class scores must correspond to the order of labels, if provided, or else to the numerical or lexicographical order of the labels in y_true. See more information in the User guide; In the multilabel case, it corresponds to an array of shape (n_samples, n_classes). Probability estimates are provided by the predict_proba method and the non-thresholded decision values by the decision_function method. The probability estimates correspond to the probability of the class with the greater label for each output of the classifier. See more information in the User guide.
average{‘micro’, ‘macro’, ‘samples’, ‘weighted’} or None, default=’macro’

If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data: Note: multiclass ROC AUC currently only handles the ‘macro’ and ‘weighted’ averages.
'micro':

Calculate metrics globally by considering each element of the label indicator matrix as a label.
'macro':

Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.
'weighted':

Calculate metrics for each label, and find their average, weighted by support (the number of true instances for each label).
'samples':

Calculate metrics for each instance, and find their average.   Will be ignored when y_true is binary.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
max_fprfloat > 0 and <= 1, default=None

If not None, the standardized partial AUC [2] over the range [0, max_fpr] is returned. For the multiclass case, max_fpr, should be either equal to None or 1.0 as AUC ROC partial computation currently is not supported for multiclass.
multi_class{‘raise’, ‘ovr’, ‘ovo’}, default=’raise’

Only used for multiclass targets. Determines the type of configuration to use. The default value raises an error, so either 'ovr' or 'ovo' must be passed explicitly.
'ovr':

Stands for One-vs-rest. Computes the AUC of each class against the rest [3] [4]. This treats the multiclass case in the same way as the multilabel case. Sensitive to class imbalance even when average == 'macro', because class imbalance affects the composition of each of the ‘rest’ groupings.
'ovo':

Stands for One-vs-one. Computes the average AUC of all possible pairwise combinations of classes [5]. Insensitive to class imbalance when average == 'macro'.
labelsarray-like of shape (n_classes,), default=None

Only used for multiclass targets. List of labels that index the classes in y_score. If None, the numerical or lexicographical order of the labels in y_true is used.    Returns

aucfloat
    See also
average_precision_score

Area under the precision-recall curve.
roc_curve

Compute Receiver operating characteristic (ROC) curve.
plot_roc_curve

Plot Receiver operating characteristic (ROC) curve.    References
1
Wikipedia entry for the Receiver operating characteristic
2
Analyzing a portion of the ROC curve. McClish, 1989
3
Provost, F., Domingos, P. (2000). Well-trained PETs: Improving probability estimation trees (Section 6.2), CeDER Working Paper #IS-00-04, Stern School of Business, New York University.
4
Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861-874.
5
Hand, D.J., Till, R.J. (2001). A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems. Machine Learning, 45(2), 171-186.   Examples Binary case: >>> from sklearn.datasets import load_breast_cancer
>>> from sklearn.linear_model import LogisticRegression
>>> from sklearn.metrics import roc_auc_score
>>> X, y = load_breast_cancer(return_X_y=True)
>>> clf = LogisticRegression(solver="liblinear", random_state=0).fit(X, y)
>>> roc_auc_score(y, clf.predict_proba(X)[:, 1])
0.99...
>>> roc_auc_score(y, clf.decision_function(X))
0.99...
 Multiclass case: >>> from sklearn.datasets import load_iris
>>> X, y = load_iris(return_X_y=True)
>>> clf = LogisticRegression(solver="liblinear").fit(X, y)
>>> roc_auc_score(y, clf.predict_proba(X), multi_class='ovr')
0.99...
 Multilabel case: >>> from sklearn.datasets import make_multilabel_classification
>>> from sklearn.multioutput import MultiOutputClassifier
>>> X, y = make_multilabel_classification(random_state=0)
>>> clf = MultiOutputClassifier(clf).fit(X, y)
>>> # get a list of n_output containing probability arrays of shape
>>> # (n_samples, n_classes)
>>> y_pred = clf.predict_proba(X)
>>> # extract the positive columns for each output
>>> y_pred = np.transpose([pred[:, 1] for pred in y_pred])
>>> roc_auc_score(y, y_pred, average=None)
array([0.82..., 0.86..., 0.94..., 0.85... , 0.94...])
>>> from sklearn.linear_model import RidgeClassifierCV
>>> clf = RidgeClassifierCV().fit(X, y)
>>> roc_auc_score(y, clf.decision_function(X), average=None)
array([0.81..., 0.84... , 0.93..., 0.87..., 0.94...])


sklearn.metrics.roc_curve(y_true, y_score, *, pos_label=None, sample_weight=None, drop_intermediate=True) [source]

Compute Receiver operating characteristic (ROC). Note: this implementation is restricted to the binary classification task. Read more in the User Guide.  Parameters

y_truendarray of shape (n_samples,)

True binary labels. If labels are not either {-1, 1} or {0, 1}, then pos_label should be explicitly given.
y_scorendarray of shape (n_samples,)

Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by “decision_function” on some classifiers).
pos_labelint or str, default=None

The label of the positive class. When pos_label=None, if y_true is in {-1, 1} or {0, 1}, pos_label is set to 1, otherwise an error will be raised.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
drop_intermediatebool, default=True

Whether to drop some suboptimal thresholds which would not appear on a plotted ROC curve. This is useful in order to create lighter ROC curves.  New in version 0.17: parameter drop_intermediate.     Returns

fprndarray of shape (>2,)

Increasing false positive rates such that element i is the false positive rate of predictions with score >= thresholds[i].
tprndarray of shape (>2,)

Increasing true positive rates such that element i is the true positive rate of predictions with score >= thresholds[i].
thresholdsndarray of shape = (n_thresholds,)

Decreasing thresholds on the decision function used to compute fpr and tpr. thresholds[0] represents no instances being predicted and is arbitrarily set to max(y_score) + 1.      See also
plot_roc_curve

Plot Receiver operating characteristic (ROC) curve.
RocCurveDisplay

ROC Curve visualization.
det_curve

Compute error rates for different probability thresholds.
roc_auc_score

Compute the area under the ROC curve.    Notes Since the thresholds are sorted from low to high values, they are reversed upon returning them to ensure they correspond to both fpr and tpr, which are sorted in reversed order during their calculation. References
1
Wikipedia entry for the Receiver operating characteristic
2
Fawcett T. An introduction to ROC analysis[J]. Pattern Recognition Letters, 2006, 27(8):861-874.   Examples >>> import numpy as np
>>> from sklearn import metrics
>>> y = np.array([1, 1, 2, 2])
>>> scores = np.array([0.1, 0.4, 0.35, 0.8])
>>> fpr, tpr, thresholds = metrics.roc_curve(y, scores, pos_label=2)
>>> fpr
array([0. , 0. , 0.5, 0.5, 1. ])
>>> tpr
array([0. , 0.5, 0.5, 1. , 1. ])
>>> thresholds
array([1.8 , 0.8 , 0.4 , 0.35, 0.1 ])


sklearn.metrics.silhouette_samples(X, labels, *, metric='euclidean', **kwds) [source]

Compute the Silhouette Coefficient for each sample. The Silhouette Coefficient is a measure of how well samples are clustered with samples that are similar to themselves. Clustering models with a high Silhouette Coefficient are said to be dense, where samples in the same cluster are similar to each other, and well separated, where samples in different clusters are not very similar to each other. The Silhouette Coefficient is calculated using the mean intra-cluster distance (a) and the mean nearest-cluster distance (b) for each sample. The Silhouette Coefficient for a sample is (b - a) / max(a,
b). Note that Silhouette Coefficient is only defined if number of labels is 2 <= n_labels <= n_samples - 1. This function returns the Silhouette Coefficient for each sample. The best value is 1 and the worst value is -1. Values near 0 indicate overlapping clusters. Read more in the User Guide.  Parameters

Xarray-like of shape (n_samples_a, n_samples_a) if metric == “precomputed” or (n_samples_a, n_features) otherwise

An array of pairwise distances between samples, or a feature array.
labelsarray-like of shape (n_samples,)

Label values for each sample.
metricstr or callable, default=’euclidean’

The metric to use when calculating distance between instances in a feature array. If metric is a string, it must be one of the options allowed by sklearn.metrics.pairwise.pairwise_distances. If X is the distance array itself, use “precomputed” as the metric. Precomputed distance matrices must have 0 along the diagonal.
`**kwds`optional keyword parameters

Any further parameters are passed directly to the distance function. If using a scipy.spatial.distance metric, the parameters are still metric dependent. See the scipy docs for usage examples.    Returns

silhouettearray-like of shape (n_samples,)

Silhouette Coefficients for each sample.     References
1
Peter J. Rousseeuw (1987). “Silhouettes: a Graphical Aid to the Interpretation and Validation of Cluster Analysis”. Computational and Applied Mathematics 20: 53-65.
2
Wikipedia entry on the Silhouette Coefficient


sklearn.metrics.silhouette_score(X, labels, *, metric='euclidean', sample_size=None, random_state=None, **kwds) [source]

Compute the mean Silhouette Coefficient of all samples. The Silhouette Coefficient is calculated using the mean intra-cluster distance (a) and the mean nearest-cluster distance (b) for each sample. The Silhouette Coefficient for a sample is (b - a) / max(a,
b). To clarify, b is the distance between a sample and the nearest cluster that the sample is not a part of. Note that Silhouette Coefficient is only defined if number of labels is 2 <= n_labels <= n_samples - 1. This function returns the mean Silhouette Coefficient over all samples. To obtain the values for each sample, use silhouette_samples. The best value is 1 and the worst value is -1. Values near 0 indicate overlapping clusters. Negative values generally indicate that a sample has been assigned to the wrong cluster, as a different cluster is more similar. Read more in the User Guide.  Parameters

Xarray-like of shape (n_samples_a, n_samples_a) if metric == “precomputed” or (n_samples_a, n_features) otherwise

An array of pairwise distances between samples, or a feature array.
labelsarray-like of shape (n_samples,)

Predicted labels for each sample.
metricstr or callable, default=’euclidean’

The metric to use when calculating distance between instances in a feature array. If metric is a string, it must be one of the options allowed by metrics.pairwise.pairwise_distances. If X is the distance array itself, use metric="precomputed".
sample_sizeint, default=None

The size of the sample to use when computing the Silhouette Coefficient on a random subset of the data. If sample_size is None, no sampling is used.
random_stateint, RandomState instance or None, default=None

Determines random number generation for selecting a subset of samples. Used when sample_size is not None. Pass an int for reproducible results across multiple function calls. See Glossary.
**kwdsoptional keyword parameters

Any further parameters are passed directly to the distance function. If using a scipy.spatial.distance metric, the parameters are still metric dependent. See the scipy docs for usage examples.    Returns

silhouettefloat

Mean Silhouette Coefficient for all samples.     References
1
Peter J. Rousseeuw (1987). “Silhouettes: a Graphical Aid to the Interpretation and Validation of Cluster Analysis”. Computational and Applied Mathematics 20: 53-65.
2
Wikipedia entry on the Silhouette Coefficient


sklearn.metrics.top_k_accuracy_score(y_true, y_score, *, k=2, normalize=True, sample_weight=None, labels=None) [source]

Top-k Accuracy classification score. This metric computes the number of times where the correct label is among the top k labels predicted (ranked by predicted scores). Note that the multilabel case isn’t covered here. Read more in the User Guide  Parameters

y_truearray-like of shape (n_samples,)

True labels.
y_scorearray-like of shape (n_samples,) or (n_samples, n_classes)

Target scores. These can be either probability estimates or non-thresholded decision values (as returned by decision_function on some classifiers). The binary case expects scores with shape (n_samples,) while the multiclass case expects scores with shape (n_samples, n_classes). In the nulticlass case, the order of the class scores must correspond to the order of labels, if provided, or else to the numerical or lexicographical order of the labels in y_true.
kint, default=2

Number of most likely outcomes considered to find the correct label.
normalizebool, default=True

If True, return the fraction of correctly classified samples. Otherwise, return the number of correctly classified samples.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights. If None, all samples are given the same weight.
labelsarray-like of shape (n_classes,), default=None

Multiclass only. List of labels that index the classes in y_score. If None, the numerical or lexicographical order of the labels in y_true is used.    Returns

scorefloat

The top-k accuracy score. The best performance is 1 with normalize == True and the number of samples with normalize == False.      See also
accuracy_score

  Notes In cases where two or more labels are assigned equal predicted scores, the labels with the highest indices will be chosen first. This might impact the result if the correct label falls after the threshold because of that. Examples >>> import numpy as np
>>> from sklearn.metrics import top_k_accuracy_score
>>> y_true = np.array([0, 1, 2, 2])
>>> y_score = np.array([[0.5, 0.2, 0.2],  # 0 is in top 2
...                     [0.3, 0.4, 0.2],  # 1 is in top 2
...                     [0.2, 0.4, 0.3],  # 2 is in top 2
...                     [0.7, 0.2, 0.1]]) # 2 isn't in top 2
>>> top_k_accuracy_score(y_true, y_score, k=2)
0.75
>>> # Not normalizing gives the number of "correctly" classified samples
>>> top_k_accuracy_score(y_true, y_score, k=2, normalize=False)
3


sklearn.metrics.v_measure_score(labels_true, labels_pred, *, beta=1.0) [source]

V-measure cluster labeling given a ground truth. This score is identical to normalized_mutual_info_score with the 'arithmetic' option for averaging. The V-measure is the harmonic mean between homogeneity and completeness: v = (1 + beta) * homogeneity * completeness
     / (beta * homogeneity + completeness)
 This metric is independent of the absolute values of the labels: a permutation of the class or cluster label values won’t change the score value in any way. This metric is furthermore symmetric: switching label_true with label_pred will return the same score value. This can be useful to measure the agreement of two independent label assignments strategies on the same dataset when the real ground truth is not known. Read more in the User Guide.  Parameters

labels_trueint array, shape = [n_samples]

ground truth class labels to be used as a reference
labels_predarray-like of shape (n_samples,)

cluster labels to evaluate
betafloat, default=1.0

Ratio of weight attributed to homogeneity vs completeness. If beta is greater than 1, completeness is weighted more strongly in the calculation. If beta is less than 1, homogeneity is weighted more strongly.    Returns

v_measurefloat

score between 0.0 and 1.0. 1.0 stands for perfectly complete labeling      See also
homogeneity_score

completeness_score

normalized_mutual_info_score

  References
1
Andrew Rosenberg and Julia Hirschberg, 2007. V-Measure: A conditional entropy-based external cluster evaluation measure   Examples Perfect labelings are both homogeneous and complete, hence have score 1.0: >>> from sklearn.metrics.cluster import v_measure_score
>>> v_measure_score([0, 0, 1, 1], [0, 0, 1, 1])
1.0
>>> v_measure_score([0, 0, 1, 1], [1, 1, 0, 0])
1.0
 Labelings that assign all classes members to the same clusters are complete be not homogeneous, hence penalized: >>> print("%.6f" % v_measure_score([0, 0, 1, 2], [0, 0, 1, 1]))
0.8...
>>> print("%.6f" % v_measure_score([0, 1, 2, 3], [0, 0, 1, 1]))
0.66...
 Labelings that have pure clusters with members coming from the same classes are homogeneous but un-necessary splits harms completeness and thus penalize V-measure as well: >>> print("%.6f" % v_measure_score([0, 0, 1, 1], [0, 0, 1, 2]))
0.8...
>>> print("%.6f" % v_measure_score([0, 0, 1, 1], [0, 1, 2, 3]))
0.66...
 If classes members are completely split across different clusters, the assignment is totally incomplete, hence the V-Measure is null: >>> print("%.6f" % v_measure_score([0, 0, 0, 0], [0, 1, 2, 3]))
0.0...
 Clusters that include samples from totally different classes totally destroy the homogeneity of the labeling, hence: >>> print("%.6f" % v_measure_score([0, 0, 1, 1], [0, 0, 0, 0]))
0.0...


sklearn.metrics.zero_one_loss(y_true, y_pred, *, normalize=True, sample_weight=None) [source]

Zero-one classification loss. If normalize is True, return the fraction of misclassifications (float), else it returns the number of misclassifications (int). The best performance is 0. Read more in the User Guide.  Parameters

y_true1d array-like, or label indicator array / sparse matrix

Ground truth (correct) labels.
y_pred1d array-like, or label indicator array / sparse matrix

Predicted labels, as returned by a classifier.
normalizebool, default=True

If False, return the number of misclassifications. Otherwise, return the fraction of misclassifications.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

lossfloat or int,

If normalize == True, return the fraction of misclassifications (float), else it returns the number of misclassifications (int).      See also
accuracy_score, hamming_loss, jaccard_score

  Notes In multilabel classification, the zero_one_loss function corresponds to the subset zero-one loss: for each sample, the entire set of labels must be correctly predicted, otherwise the loss for that sample is equal to one. Examples >>> from sklearn.metrics import zero_one_loss
>>> y_pred = [1, 2, 3, 4]
>>> y_true = [2, 2, 3, 4]
>>> zero_one_loss(y_true, y_pred)
0.25
>>> zero_one_loss(y_true, y_pred, normalize=False)
1
 In the multilabel case with binary label indicators: >>> import numpy as np
>>> zero_one_loss(np.array([[0, 1], [1, 1]]), np.ones((2, 2)))
0.5