fit_transform(X, y=None) [source]

Fit X into an embedded space and return that transformed output.  Parameters

Xndarray of shape (n_samples, n_features) or (n_samples, n_samples)

If the metric is ‘precomputed’ X must be a square distance matrix. Otherwise it contains a sample per row. If the method is ‘exact’, X may be a sparse matrix of type ‘csr’, ‘csc’ or ‘coo’. If the method is ‘barnes_hut’ and the metric is ‘precomputed’, X may be a precomputed sparse graph.
yIgnored
  Returns

X_newndarray of shape (n_samples, n_components)

Embedding of the training data in low-dimensional space.


get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.


set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.


sklearn.metrics.accuracy_score(y_true, y_pred, *, normalize=True, sample_weight=None) [source]

Accuracy classification score. In multilabel classification, this function computes subset accuracy: the set of labels predicted for a sample must exactly match the corresponding set of labels in y_true. Read more in the User Guide.  Parameters

y_true1d array-like, or label indicator array / sparse matrix

Ground truth (correct) labels.
y_pred1d array-like, or label indicator array / sparse matrix

Predicted labels, as returned by a classifier.
normalizebool, default=True

If False, return the number of correctly classified samples. Otherwise, return the fraction of correctly classified samples.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

scorefloat

If normalize == True, return the fraction of correctly classified samples (float), else returns the number of correctly classified samples (int). The best performance is 1 with normalize == True and the number of samples with normalize == False.      See also
jaccard_score, hamming_loss, zero_one_loss

  Notes In binary and multiclass classification, this function is equal to the jaccard_score function. Examples >>> from sklearn.metrics import accuracy_score
>>> y_pred = [0, 2, 1, 3]
>>> y_true = [0, 1, 2, 3]
>>> accuracy_score(y_true, y_pred)
0.5
>>> accuracy_score(y_true, y_pred, normalize=False)
2
 In the multilabel case with binary label indicators: >>> import numpy as np
>>> accuracy_score(np.array([[0, 1], [1, 1]]), np.ones((2, 2)))
0.5


sklearn.metrics.adjusted_mutual_info_score(labels_true, labels_pred, *, average_method='arithmetic') [source]

Adjusted Mutual Information between two clusterings. Adjusted Mutual Information (AMI) is an adjustment of the Mutual Information (MI) score to account for chance. It accounts for the fact that the MI is generally higher for two clusterings with a larger number of clusters, regardless of whether there is actually more information shared. For two clusterings \(U\) and \(V\), the AMI is given as: AMI(U, V) = [MI(U, V) - E(MI(U, V))] / [avg(H(U), H(V)) - E(MI(U, V))]
 This metric is independent of the absolute values of the labels: a permutation of the class or cluster label values won’t change the score value in any way. This metric is furthermore symmetric: switching label_true with label_pred will return the same score value. This can be useful to measure the agreement of two independent label assignments strategies on the same dataset when the real ground truth is not known. Be mindful that this function is an order of magnitude slower than other metrics, such as the Adjusted Rand Index. Read more in the User Guide.  Parameters

labels_trueint array, shape = [n_samples]

A clustering of the data into disjoint subsets.
labels_predint array-like of shape (n_samples,)

A clustering of the data into disjoint subsets.
average_methodstr, default=’arithmetic’

How to compute the normalizer in the denominator. Possible options are ‘min’, ‘geometric’, ‘arithmetic’, and ‘max’.  New in version 0.20.   Changed in version 0.22: The default value of average_method changed from ‘max’ to ‘arithmetic’.     Returns
 ami: float (upperlimited by 1.0)

The AMI returns a value of 1 when the two partitions are identical (ie perfectly matched). Random partitions (independent labellings) have an expected AMI around 0 on average hence can be negative.      See also
adjusted_rand_score

Adjusted Rand Index.
mutual_info_score

Mutual Information (not adjusted for chance).    References
1
Vinh, Epps, and Bailey, (2010). Information Theoretic Measures for Clusterings Comparison: Variants, Properties, Normalization and Correction for Chance, JMLR
2
Wikipedia entry for the Adjusted Mutual Information   Examples Perfect labelings are both homogeneous and complete, hence have score 1.0: >>> from sklearn.metrics.cluster import adjusted_mutual_info_score
>>> adjusted_mutual_info_score([0, 0, 1, 1], [0, 0, 1, 1])
...
1.0
>>> adjusted_mutual_info_score([0, 0, 1, 1], [1, 1, 0, 0])
...
1.0
 If classes members are completely split across different clusters, the assignment is totally in-complete, hence the AMI is null: >>> adjusted_mutual_info_score([0, 0, 0, 0], [0, 1, 2, 3])
...
0.0


sklearn.metrics.adjusted_rand_score(labels_true, labels_pred) [source]

Rand index adjusted for chance. The Rand Index computes a similarity measure between two clusterings by considering all pairs of samples and counting pairs that are assigned in the same or different clusters in the predicted and true clusterings. The raw RI score is then “adjusted for chance” into the ARI score using the following scheme: ARI = (RI - Expected_RI) / (max(RI) - Expected_RI)
 The adjusted Rand index is thus ensured to have a value close to 0.0 for random labeling independently of the number of clusters and samples and exactly 1.0 when the clusterings are identical (up to a permutation). ARI is a symmetric measure: adjusted_rand_score(a, b) == adjusted_rand_score(b, a)
 Read more in the User Guide.  Parameters

labels_trueint array, shape = [n_samples]

Ground truth class labels to be used as a reference
labels_predarray-like of shape (n_samples,)

Cluster labels to evaluate    Returns

ARIfloat

Similarity score between -1.0 and 1.0. Random labelings have an ARI close to 0.0. 1.0 stands for perfect match.      See also
adjusted_mutual_info_score

Adjusted Mutual Information.    References
Hubert1985
L. Hubert and P. Arabie, Comparing Partitions, Journal of Classification 1985 https://link.springer.com/article/10.1007%2FBF01908075
Steinley2004
D. Steinley, Properties of the Hubert-Arabie adjusted Rand index, Psychological Methods 2004
wk
https://en.wikipedia.org/wiki/Rand_index#Adjusted_Rand_index   Examples Perfectly matching labelings have a score of 1 even >>> from sklearn.metrics.cluster import adjusted_rand_score
>>> adjusted_rand_score([0, 0, 1, 1], [0, 0, 1, 1])
1.0
>>> adjusted_rand_score([0, 0, 1, 1], [1, 1, 0, 0])
1.0
 Labelings that assign all classes members to the same clusters are complete but may not always be pure, hence penalized: >>> adjusted_rand_score([0, 0, 1, 2], [0, 0, 1, 1])
0.57...
 ARI is symmetric, so labelings that have pure clusters with members coming from the same classes but unnecessary splits are penalized: >>> adjusted_rand_score([0, 0, 1, 1], [0, 0, 1, 2])
0.57...
 If classes members are completely split across different clusters, the assignment is totally incomplete, hence the ARI is very low: >>> adjusted_rand_score([0, 0, 0, 0], [0, 1, 2, 3])
0.0


sklearn.metrics.auc(x, y) [source]

Compute Area Under the Curve (AUC) using the trapezoidal rule. This is a general function, given points on a curve. For computing the area under the ROC-curve, see roc_auc_score. For an alternative way to summarize a precision-recall curve, see average_precision_score.  Parameters

xndarray of shape (n,)

x coordinates. These must be either monotonic increasing or monotonic decreasing.
yndarray of shape, (n,)

y coordinates.    Returns

aucfloat
    See also
roc_auc_score

Compute the area under the ROC curve.
average_precision_score

Compute average precision from prediction scores.
precision_recall_curve

Compute precision-recall pairs for different probability thresholds.    Examples >>> import numpy as np
>>> from sklearn import metrics
>>> y = np.array([1, 1, 2, 2])
>>> pred = np.array([0.1, 0.4, 0.35, 0.8])
>>> fpr, tpr, thresholds = metrics.roc_curve(y, pred, pos_label=2)
>>> metrics.auc(fpr, tpr)
0.75


sklearn.metrics.average_precision_score(y_true, y_score, *, average='macro', pos_label=1, sample_weight=None) [source]

Compute average precision (AP) from prediction scores. AP summarizes a precision-recall curve as the weighted mean of precisions achieved at each threshold, with the increase in recall from the previous threshold used as the weight:  \[\text{AP} = \sum_n (R_n - R_{n-1}) P_n\] where \(P_n\) and \(R_n\) are the precision and recall at the nth threshold [1]. This implementation is not interpolated and is different from computing the area under the precision-recall curve with the trapezoidal rule, which uses linear interpolation and can be too optimistic. Note: this implementation is restricted to the binary classification task or multilabel classification task. Read more in the User Guide.  Parameters

y_truendarray of shape (n_samples,) or (n_samples, n_classes)

True binary labels or binary label indicators.
y_scorendarray of shape (n_samples,) or (n_samples, n_classes)

Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by decision_function on some classifiers).
average{‘micro’, ‘samples’, ‘weighted’, ‘macro’} or None, default=’macro’

If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data:
'micro':

Calculate metrics globally by considering each element of the label indicator matrix as a label.
'macro':

Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.
'weighted':

Calculate metrics for each label, and find their average, weighted by support (the number of true instances for each label).
'samples':

Calculate metrics for each instance, and find their average.   Will be ignored when y_true is binary.
pos_labelint or str, default=1

The label of the positive class. Only applied to binary y_true. For multilabel-indicator y_true, pos_label is fixed to 1.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

average_precisionfloat
    See also
roc_auc_score

Compute the area under the ROC curve.
precision_recall_curve

Compute precision-recall pairs for different probability thresholds.    Notes  Changed in version 0.19: Instead of linearly interpolating between operating points, precisions are weighted by the change in recall since the last operating point.  References
1
Wikipedia entry for the Average precision   Examples >>> import numpy as np
>>> from sklearn.metrics import average_precision_score
>>> y_true = np.array([0, 0, 1, 1])
>>> y_scores = np.array([0.1, 0.4, 0.35, 0.8])
>>> average_precision_score(y_true, y_scores)
0.83...


sklearn.metrics.balanced_accuracy_score(y_true, y_pred, *, sample_weight=None, adjusted=False) [source]

Compute the balanced accuracy. The balanced accuracy in binary and multiclass classification problems to deal with imbalanced datasets. It is defined as the average of recall obtained on each class. The best value is 1 and the worst value is 0 when adjusted=False. Read more in the User Guide.  New in version 0.20.   Parameters

y_true1d array-like

Ground truth (correct) target values.
y_pred1d array-like

Estimated targets as returned by a classifier.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
adjustedbool, default=False

When true, the result is adjusted for chance, so that random performance would score 0, and perfect performance scores 1.    Returns

balanced_accuracyfloat
    See also
recall_score, roc_auc_score

  Notes Some literature promotes alternative definitions of balanced accuracy. Our definition is equivalent to accuracy_score with class-balanced sample weights, and shares desirable properties with the binary case. See the User Guide. References
1
Brodersen, K.H.; Ong, C.S.; Stephan, K.E.; Buhmann, J.M. (2010). The balanced accuracy and its posterior distribution. Proceedings of the 20th International Conference on Pattern Recognition, 3121-24.
2
John. D. Kelleher, Brian Mac Namee, Aoife D’Arcy, (2015). Fundamentals of Machine Learning for Predictive Data Analytics: Algorithms, Worked Examples, and Case Studies.   Examples >>> from sklearn.metrics import balanced_accuracy_score
>>> y_true = [0, 1, 0, 0, 1, 0]
>>> y_pred = [0, 1, 0, 0, 0, 1]
>>> balanced_accuracy_score(y_true, y_pred)
0.625


sklearn.metrics.brier_score_loss(y_true, y_prob, *, sample_weight=None, pos_label=None) [source]

Compute the Brier score loss. The smaller the Brier score loss, the better, hence the naming with “loss”. The Brier score measures the mean squared difference between the predicted probability and the actual outcome. The Brier score always takes on a value between zero and one, since this is the largest possible difference between a predicted probability (which must be between zero and one) and the actual outcome (which can take on values of only 0 and 1). It can be decomposed is the sum of refinement loss and calibration loss. The Brier score is appropriate for binary and categorical outcomes that can be structured as true or false, but is inappropriate for ordinal variables which can take on three or more values (this is because the Brier score assumes that all possible outcomes are equivalently “distant” from one another). Which label is considered to be the positive label is controlled via the parameter pos_label, which defaults to the greater label unless y_true is all 0 or all -1, in which case pos_label defaults to 1. Read more in the User Guide.  Parameters

y_truearray of shape (n_samples,)

True targets.
y_probarray of shape (n_samples,)

Probabilities of the positive class.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
pos_labelint or str, default=None

Label of the positive class. pos_label will be infered in the following manner:  if y_true in {-1, 1} or {0, 1}, pos_label defaults to 1; else if y_true contains string, an error will be raised and pos_label should be explicitely specified; otherwise, pos_label defaults to the greater label, i.e. np.unique(y_true)[-1].     Returns

scorefloat

Brier score loss.     References
1
Wikipedia entry for the Brier score.   Examples >>> import numpy as np
>>> from sklearn.metrics import brier_score_loss
>>> y_true = np.array([0, 1, 1, 0])
>>> y_true_categorical = np.array(["spam", "ham", "ham", "spam"])
>>> y_prob = np.array([0.1, 0.9, 0.8, 0.3])
>>> brier_score_loss(y_true, y_prob)
0.037...
>>> brier_score_loss(y_true, 1-y_prob, pos_label=0)
0.037...
>>> brier_score_loss(y_true_categorical, y_prob, pos_label="ham")
0.037...
>>> brier_score_loss(y_true, np.array(y_prob) > 0.5)
0.0


sklearn.metrics.calinski_harabasz_score(X, labels) [source]

Compute the Calinski and Harabasz score. It is also known as the Variance Ratio Criterion. The score is defined as ratio between the within-cluster dispersion and the between-cluster dispersion. Read more in the User Guide.  Parameters

Xarray-like of shape (n_samples, n_features)

A list of n_features-dimensional data points. Each row corresponds to a single data point.
labelsarray-like of shape (n_samples,)

Predicted labels for each sample.    Returns

scorefloat

The resulting Calinski-Harabasz score.     References
1
T. Calinski and J. Harabasz, 1974. “A dendrite method for cluster analysis”. Communications in Statistics


sklearn.metrics.check_scoring(estimator, scoring=None, *, allow_none=False) [source]

Determine scorer from user options. A TypeError will be thrown if the estimator cannot be scored.  Parameters

estimatorestimator object implementing ‘fit’

The object to use to fit the data.
scoringstr or callable, default=None

A string (see model evaluation documentation) or a scorer callable object / function with signature scorer(estimator, X, y).
allow_nonebool, default=False

If no scoring is specified and the estimator has no score function, we can either return None or raise an exception.    Returns

scoringcallable

A scorer callable object / function with signature scorer(estimator, X, y).


sklearn.metrics.classification_report(y_true, y_pred, *, labels=None, target_names=None, sample_weight=None, digits=2, output_dict=False, zero_division='warn') [source]

Build a text report showing the main classification metrics. Read more in the User Guide.  Parameters

y_true1d array-like, or label indicator array / sparse matrix

Ground truth (correct) target values.
y_pred1d array-like, or label indicator array / sparse matrix

Estimated targets as returned by a classifier.
labelsarray-like of shape (n_labels,), default=None

Optional list of label indices to include in the report.
target_nameslist of str of shape (n_labels,), default=None

Optional display names matching the labels (same order).
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
digitsint, default=2

Number of digits for formatting output floating point values. When output_dict is True, this will be ignored and the returned values will not be rounded.
output_dictbool, default=False

If True, return output as dict.  New in version 0.20.
zero_division“warn”, 0 or 1, default=”warn”

Sets the value to return when there is a zero division. If set to “warn”, this acts as 0, but warnings are also raised.    Returns

reportstring / dict

Text summary of the precision, recall, F1 score for each class. Dictionary returned if output_dict is True. Dictionary has the following structure: {'label 1': {'precision':0.5,
             'recall':1.0,
             'f1-score':0.67,
             'support':1},
 'label 2': { ... },
  ...
}
 The reported averages include macro average (averaging the unweighted mean per label), weighted average (averaging the support-weighted mean per label), and sample average (only for multilabel classification). Micro average (averaging the total true positives, false negatives and false positives) is only shown for multi-label or multi-class with a subset of classes, because it corresponds to accuracy otherwise and would be the same for all metrics. See also precision_recall_fscore_support for more details on averages. Note that in binary classification, recall of the positive class is also known as “sensitivity”; recall of the negative class is “specificity”.      See also
precision_recall_fscore_support, confusion_matrix

multilabel_confusion_matrix

  Examples >>> from sklearn.metrics import classification_report
>>> y_true = [0, 1, 2, 2, 2]
>>> y_pred = [0, 0, 2, 2, 1]
>>> target_names = ['class 0', 'class 1', 'class 2']
>>> print(classification_report(y_true, y_pred, target_names=target_names))
              precision    recall  f1-score   support

     class 0       0.50      1.00      0.67         1
     class 1       0.00      0.00      0.00         1
     class 2       1.00      0.67      0.80         3

    accuracy                           0.60         5
   macro avg       0.50      0.56      0.49         5
weighted avg       0.70      0.60      0.61         5

>>> y_pred = [1, 1, 0]
>>> y_true = [1, 1, 1]
>>> print(classification_report(y_true, y_pred, labels=[1, 2, 3]))
              precision    recall  f1-score   support

           1       1.00      0.67      0.80         3
           2       0.00      0.00      0.00         0
           3       0.00      0.00      0.00         0

   micro avg       1.00      0.67      0.80         3
   macro avg       0.33      0.22      0.27         3
weighted avg       1.00      0.67      0.80         3


sklearn.metrics.cluster.contingency_matrix(labels_true, labels_pred, *, eps=None, sparse=False, dtype=<class 'numpy.int64'>) [source]

Build a contingency matrix describing the relationship between labels.  Parameters

labels_trueint array, shape = [n_samples]

Ground truth class labels to be used as a reference.
labels_predarray-like of shape (n_samples,)

Cluster labels to evaluate.
epsfloat, default=None

If a float, that value is added to all values in the contingency matrix. This helps to stop NaN propagation. If None, nothing is adjusted.
sparsebool, default=False

If True, return a sparse CSR continency matrix. If eps is not None and sparse is True will raise ValueError.  New in version 0.18.
dtypenumeric type, default=np.int64

Output dtype. Ignored if eps is not None.  New in version 0.24.     Returns

contingency{array-like, sparse}, shape=[n_classes_true, n_classes_pred]

Matrix \(C\) such that \(C_{i, j}\) is the number of samples in true class \(i\) and in predicted class \(j\). If eps is None, the dtype of this array will be integer unless set otherwise with the dtype argument. If eps is given, the dtype will be float. Will be a sklearn.sparse.csr_matrix if sparse=True.


sklearn.metrics.cluster.pair_confusion_matrix(labels_true, labels_pred) [source]

Pair confusion matrix arising from two clusterings. The pair confusion matrix \(C\) computes a 2 by 2 similarity matrix between two clusterings by considering all pairs of samples and counting pairs that are assigned into the same or into different clusters under the true and predicted clusterings. Considering a pair of samples that is clustered together a positive pair, then as in binary classification the count of true negatives is \(C_{00}\), false negatives is \(C_{10}\), true positives is \(C_{11}\) and false positives is \(C_{01}\). Read more in the User Guide.  Parameters

labels_truearray-like of shape (n_samples,), dtype=integral

Ground truth class labels to be used as a reference.
labels_predarray-like of shape (n_samples,), dtype=integral

Cluster labels to evaluate.    Returns

Cndarray of shape (2, 2), dtype=np.int64

The contingency matrix.      See also
rand_score

Rand Score
adjusted_rand_score

Adjusted Rand Score
adjusted_mutual_info_score

Adjusted Mutual Information    References Examples Perfectly matching labelings have all non-zero entries on the diagonal regardless of actual label values: >>> from sklearn.metrics.cluster import pair_confusion_matrix
>>> pair_confusion_matrix([0, 0, 1, 1], [1, 1, 0, 0])
array([[8, 0],
       [0, 4]]...
 Labelings that assign all classes members to the same clusters are complete but may be not always pure, hence penalized, and have some off-diagonal non-zero entries: >>> pair_confusion_matrix([0, 0, 1, 2], [0, 0, 1, 1])
array([[8, 2],
       [0, 2]]...
 Note that the matrix is not symmetric.


sklearn.metrics.cohen_kappa_score(y1, y2, *, labels=None, weights=None, sample_weight=None) [source]

Cohen’s kappa: a statistic that measures inter-annotator agreement. This function computes Cohen’s kappa [1], a score that expresses the level of agreement between two annotators on a classification problem. It is defined as  \[\kappa = (p_o - p_e) / (1 - p_e)\] where \(p_o\) is the empirical probability of agreement on the label assigned to any sample (the observed agreement ratio), and \(p_e\) is the expected agreement when both annotators assign labels randomly. \(p_e\) is estimated using a per-annotator empirical prior over the class labels [2]. Read more in the User Guide.  Parameters

y1array of shape (n_samples,)

Labels assigned by the first annotator.
y2array of shape (n_samples,)

Labels assigned by the second annotator. The kappa statistic is symmetric, so swapping y1 and y2 doesn’t change the value.
labelsarray-like of shape (n_classes,), default=None

List of labels to index the matrix. This may be used to select a subset of labels. If None, all labels that appear at least once in y1 or y2 are used.
weights{‘linear’, ‘quadratic’}, default=None

Weighting type to calculate the score. None means no weighted; “linear” means linear weighted; “quadratic” means quadratic weighted.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

kappafloat

The kappa statistic, which is a number between -1 and 1. The maximum value means complete agreement; zero or lower means chance agreement.     References
1
J. Cohen (1960). “A coefficient of agreement for nominal scales”. Educational and Psychological Measurement 20(1):37-46. doi:10.1177/001316446002000104.
2
R. Artstein and M. Poesio (2008). “Inter-coder agreement for computational linguistics”. Computational Linguistics 34(4):555-596.
3
Wikipedia entry for the Cohen’s kappa.


sklearn.metrics.completeness_score(labels_true, labels_pred) [source]

Completeness metric of a cluster labeling given a ground truth. A clustering result satisfies completeness if all the data points that are members of a given class are elements of the same cluster. This metric is independent of the absolute values of the labels: a permutation of the class or cluster label values won’t change the score value in any way. This metric is not symmetric: switching label_true with label_pred will return the homogeneity_score which will be different in general. Read more in the User Guide.  Parameters

labels_trueint array, shape = [n_samples]

ground truth class labels to be used as a reference
labels_predarray-like of shape (n_samples,)

cluster labels to evaluate    Returns

completenessfloat

score between 0.0 and 1.0. 1.0 stands for perfectly complete labeling      See also
homogeneity_score

v_measure_score

  References
1
Andrew Rosenberg and Julia Hirschberg, 2007. V-Measure: A conditional entropy-based external cluster evaluation measure   Examples Perfect labelings are complete: >>> from sklearn.metrics.cluster import completeness_score
>>> completeness_score([0, 0, 1, 1], [1, 1, 0, 0])
1.0
 Non-perfect labelings that assign all classes members to the same clusters are still complete: >>> print(completeness_score([0, 0, 1, 1], [0, 0, 0, 0]))
1.0
>>> print(completeness_score([0, 1, 2, 3], [0, 0, 1, 1]))
0.999...
 If classes members are split across different clusters, the assignment cannot be complete: >>> print(completeness_score([0, 0, 1, 1], [0, 1, 0, 1]))
0.0
>>> print(completeness_score([0, 0, 0, 0], [0, 1, 2, 3]))
0.0


class sklearn.metrics.ConfusionMatrixDisplay(confusion_matrix, *, display_labels=None) [source]

Confusion Matrix visualization. It is recommend to use plot_confusion_matrix to create a ConfusionMatrixDisplay. All parameters are stored as attributes. Read more in the User Guide.  Parameters

confusion_matrixndarray of shape (n_classes, n_classes)

Confusion matrix.
display_labelsndarray of shape (n_classes,), default=None

Display labels for plot. If None, display labels are set from 0 to n_classes - 1.    Attributes

im_matplotlib AxesImage

Image representing the confusion matrix.
text_ndarray of shape (n_classes, n_classes), dtype=matplotlib Text, or None

Array of matplotlib axes. None if include_values is false.
ax_matplotlib Axes

Axes with confusion matrix.
figure_matplotlib Figure

Figure containing the confusion matrix.      See also
confusion_matrix

Compute Confusion Matrix to evaluate the accuracy of a classification.
plot_confusion_matrix

Plot Confusion Matrix.    Examples >>> from sklearn.datasets import make_classification
>>> from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay
>>> from sklearn.model_selection import train_test_split
>>> from sklearn.svm import SVC
>>> X, y = make_classification(random_state=0)
>>> X_train, X_test, y_train, y_test = train_test_split(X, y,
...                                                     random_state=0)
>>> clf = SVC(random_state=0)
>>> clf.fit(X_train, y_train)
SVC(random_state=0)
>>> predictions = clf.predict(X_test)
>>> cm = confusion_matrix(y_test, predictions, labels=clf.classes_)
>>> disp = ConfusionMatrixDisplay(confusion_matrix=cm,
...                               display_labels=clf.classes_)
>>> disp.plot()
 Methods
plot(*[, include_values, cmap, …]) Plot visualization.
plot(*, include_values=True, cmap='viridis', xticks_rotation='horizontal', values_format=None, ax=None, colorbar=True) [source]

Plot visualization.  Parameters

include_valuesbool, default=True

Includes values in confusion matrix.
cmapstr or matplotlib Colormap, default=’viridis’

Colormap recognized by matplotlib.
xticks_rotation{‘vertical’, ‘horizontal’} or float, default=’horizontal’

Rotation of xtick labels.
values_formatstr, default=None

Format specification for values in confusion matrix. If None, the format specification is ‘d’ or ‘.2g’ whichever is shorter.
axmatplotlib axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
colorbarbool, default=True

Whether or not to add a colorbar to the plot.    Returns

displayConfusionMatrixDisplay


sklearn.metrics.ConfusionMatrixDisplay
class sklearn.metrics.ConfusionMatrixDisplay(confusion_matrix, *, display_labels=None) [source]

Confusion Matrix visualization. It is recommend to use plot_confusion_matrix to create a ConfusionMatrixDisplay. All parameters are stored as attributes. Read more in the User Guide.  Parameters

confusion_matrixndarray of shape (n_classes, n_classes)

Confusion matrix.
display_labelsndarray of shape (n_classes,), default=None

Display labels for plot. If None, display labels are set from 0 to n_classes - 1.    Attributes

im_matplotlib AxesImage

Image representing the confusion matrix.
text_ndarray of shape (n_classes, n_classes), dtype=matplotlib Text, or None

Array of matplotlib axes. None if include_values is false.
ax_matplotlib Axes

Axes with confusion matrix.
figure_matplotlib Figure

Figure containing the confusion matrix.      See also
confusion_matrix

Compute Confusion Matrix to evaluate the accuracy of a classification.
plot_confusion_matrix

Plot Confusion Matrix.    Examples >>> from sklearn.datasets import make_classification
>>> from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay
>>> from sklearn.model_selection import train_test_split
>>> from sklearn.svm import SVC
>>> X, y = make_classification(random_state=0)
>>> X_train, X_test, y_train, y_test = train_test_split(X, y,
...                                                     random_state=0)
>>> clf = SVC(random_state=0)
>>> clf.fit(X_train, y_train)
SVC(random_state=0)
>>> predictions = clf.predict(X_test)
>>> cm = confusion_matrix(y_test, predictions, labels=clf.classes_)
>>> disp = ConfusionMatrixDisplay(confusion_matrix=cm,
...                               display_labels=clf.classes_)
>>> disp.plot()
 Methods
plot(*[, include_values, cmap, …]) Plot visualization.
plot(*, include_values=True, cmap='viridis', xticks_rotation='horizontal', values_format=None, ax=None, colorbar=True) [source]

Plot visualization.  Parameters

include_valuesbool, default=True

Includes values in confusion matrix.
cmapstr or matplotlib Colormap, default=’viridis’

Colormap recognized by matplotlib.
xticks_rotation{‘vertical’, ‘horizontal’} or float, default=’horizontal’

Rotation of xtick labels.
values_formatstr, default=None

Format specification for values in confusion matrix. If None, the format specification is ‘d’ or ‘.2g’ whichever is shorter.
axmatplotlib axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
colorbarbool, default=True

Whether or not to add a colorbar to the plot.    Returns

displayConfusionMatrixDisplay

 Examples using sklearn.metrics.ConfusionMatrixDisplay

  Visualizations with Display Objects


plot(*, include_values=True, cmap='viridis', xticks_rotation='horizontal', values_format=None, ax=None, colorbar=True) [source]

Plot visualization.  Parameters

include_valuesbool, default=True

Includes values in confusion matrix.
cmapstr or matplotlib Colormap, default=’viridis’

Colormap recognized by matplotlib.
xticks_rotation{‘vertical’, ‘horizontal’} or float, default=’horizontal’

Rotation of xtick labels.
values_formatstr, default=None

Format specification for values in confusion matrix. If None, the format specification is ‘d’ or ‘.2g’ whichever is shorter.
axmatplotlib axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
colorbarbool, default=True

Whether or not to add a colorbar to the plot.    Returns

displayConfusionMatrixDisplay


sklearn.metrics.confusion_matrix(y_true, y_pred, *, labels=None, sample_weight=None, normalize=None) [source]

Compute confusion matrix to evaluate the accuracy of a classification. By definition a confusion matrix \(C\) is such that \(C_{i, j}\) is equal to the number of observations known to be in group \(i\) and predicted to be in group \(j\). Thus in binary classification, the count of true negatives is \(C_{0,0}\), false negatives is \(C_{1,0}\), true positives is \(C_{1,1}\) and false positives is \(C_{0,1}\). Read more in the User Guide.  Parameters

y_truearray-like of shape (n_samples,)

Ground truth (correct) target values.
y_predarray-like of shape (n_samples,)

Estimated targets as returned by a classifier.
labelsarray-like of shape (n_classes), default=None

List of labels to index the matrix. This may be used to reorder or select a subset of labels. If None is given, those that appear at least once in y_true or y_pred are used in sorted order.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.  New in version 0.18.
normalize{‘true’, ‘pred’, ‘all’}, default=None

Normalizes confusion matrix over the true (rows), predicted (columns) conditions or all the population. If None, confusion matrix will not be normalized.    Returns

Cndarray of shape (n_classes, n_classes)

Confusion matrix whose i-th row and j-th column entry indicates the number of samples with true label being i-th class and predicted label being j-th class.      See also
plot_confusion_matrix

Plot Confusion Matrix.
ConfusionMatrixDisplay

Confusion Matrix visualization.    References
1
Wikipedia entry for the Confusion matrix (Wikipedia and other references may use a different convention for axes).   Examples >>> from sklearn.metrics import confusion_matrix
>>> y_true = [2, 0, 2, 2, 0, 1]
>>> y_pred = [0, 0, 2, 2, 0, 2]
>>> confusion_matrix(y_true, y_pred)
array([[2, 0, 0],
       [0, 0, 1],
       [1, 0, 2]])
 >>> y_true = ["cat", "ant", "cat", "cat", "ant", "bird"]
>>> y_pred = ["ant", "ant", "cat", "cat", "ant", "cat"]
>>> confusion_matrix(y_true, y_pred, labels=["ant", "bird", "cat"])
array([[2, 0, 0],
       [0, 0, 1],
       [1, 0, 2]])
 In the binary case, we can extract true positives, etc as follows: >>> tn, fp, fn, tp = confusion_matrix([0, 1, 0, 1], [1, 1, 1, 0]).ravel()
>>> (tn, fp, fn, tp)
(0, 2, 1, 1)


sklearn.metrics.consensus_score(a, b, *, similarity='jaccard') [source]

The similarity of two sets of biclusters. Similarity between individual biclusters is computed. Then the best matching between sets is found using the Hungarian algorithm. The final score is the sum of similarities divided by the size of the larger set. Read more in the User Guide.  Parameters

a(rows, columns)

Tuple of row and column indicators for a set of biclusters.
b(rows, columns)

Another set of biclusters like a.
similarity‘jaccard’ or callable, default=’jaccard’

May be the string “jaccard” to use the Jaccard coefficient, or any function that takes four arguments, each of which is a 1d indicator vector: (a_rows, a_columns, b_rows, b_columns).     References  Hochreiter, Bodenhofer, et. al., 2010. FABIA: factor analysis for bicluster acquisition.


sklearn.metrics.coverage_error(y_true, y_score, *, sample_weight=None) [source]

Coverage error measure. Compute how far we need to go through the ranked scores to cover all true labels. The best value is equal to the average number of labels in y_true per sample. Ties in y_scores are broken by giving maximal rank that would have been assigned to all tied values. Note: Our implementation’s score is 1 greater than the one given in Tsoumakas et al., 2010. This extends it to handle the degenerate case in which an instance has 0 true labels. Read more in the User Guide.  Parameters

y_truendarray of shape (n_samples, n_labels)

True binary labels in binary indicator format.
y_scorendarray of shape (n_samples, n_labels)

Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by “decision_function” on some classifiers).
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

coverage_errorfloat
   References
1
Tsoumakas, G., Katakis, I., & Vlahavas, I. (2010). Mining multi-label data. In Data mining and knowledge discovery handbook (pp. 667-685). Springer US.


sklearn.metrics.davies_bouldin_score(X, labels) [source]

Computes the Davies-Bouldin score. The score is defined as the average similarity measure of each cluster with its most similar cluster, where similarity is the ratio of within-cluster distances to between-cluster distances. Thus, clusters which are farther apart and less dispersed will result in a better score. The minimum score is zero, with lower values indicating better clustering. Read more in the User Guide.  New in version 0.20.   Parameters

Xarray-like of shape (n_samples, n_features)

A list of n_features-dimensional data points. Each row corresponds to a single data point.
labelsarray-like of shape (n_samples,)

Predicted labels for each sample.    Returns
 score: float

The resulting Davies-Bouldin score.     References
1
Davies, David L.; Bouldin, Donald W. (1979). “A Cluster Separation Measure”. IEEE Transactions on Pattern Analysis and Machine Intelligence. PAMI-1 (2): 224-227


sklearn.metrics.dcg_score(y_true, y_score, *, k=None, log_base=2, sample_weight=None, ignore_ties=False) [source]

Compute Discounted Cumulative Gain. Sum the true scores ranked in the order induced by the predicted scores, after applying a logarithmic discount. This ranking metric yields a high value if true labels are ranked high by y_score. Usually the Normalized Discounted Cumulative Gain (NDCG, computed by ndcg_score) is preferred.  Parameters

y_truendarray of shape (n_samples, n_labels)

True targets of multilabel classification, or true scores of entities to be ranked.
y_scorendarray of shape (n_samples, n_labels)

Target scores, can either be probability estimates, confidence values, or non-thresholded measure of decisions (as returned by “decision_function” on some classifiers).
kint, default=None

Only consider the highest k scores in the ranking. If None, use all outputs.
log_basefloat, default=2

Base of the logarithm used for the discount. A low value means a sharper discount (top results are more important).
sample_weightndarray of shape (n_samples,), default=None

Sample weights. If None, all samples are given the same weight.
ignore_tiesbool, default=False

Assume that there are no ties in y_score (which is likely to be the case if y_score is continuous) for efficiency gains.    Returns

discounted_cumulative_gainfloat

The averaged sample DCG scores.      See also
ndcg_score

The Discounted Cumulative Gain divided by the Ideal Discounted Cumulative Gain (the DCG obtained for a perfect ranking), in order to have a score between 0 and 1.    References Wikipedia entry for Discounted Cumulative Gain. Jarvelin, K., & Kekalainen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS), 20(4), 422-446. Wang, Y., Wang, L., Li, Y., He, D., Chen, W., & Liu, T. Y. (2013, May). A theoretical analysis of NDCG ranking measures. In Proceedings of the 26th Annual Conference on Learning Theory (COLT 2013). McSherry, F., & Najork, M. (2008, March). Computing information retrieval performance measures efficiently in the presence of tied scores. In European conference on information retrieval (pp. 414-421). Springer, Berlin, Heidelberg. Examples >>> from sklearn.metrics import dcg_score
>>> # we have groud-truth relevance of some answers to a query:
>>> true_relevance = np.asarray([[10, 0, 0, 1, 5]])
>>> # we predict scores for the answers
>>> scores = np.asarray([[.1, .2, .3, 4, 70]])
>>> dcg_score(true_relevance, scores)
9.49...
>>> # we can set k to truncate the sum; only top k answers contribute
>>> dcg_score(true_relevance, scores, k=2)
5.63...
>>> # now we have some ties in our prediction
>>> scores = np.asarray([[1, 0, 0, 0, 1]])
>>> # by default ties are averaged, so here we get the average true
>>> # relevance of our top predictions: (10 + 5) / 2 = 7.5
>>> dcg_score(true_relevance, scores, k=1)
7.5
>>> # we can choose to ignore ties for faster results, but only
>>> # if we know there aren't ties in our scores, otherwise we get
>>> # wrong results:
>>> dcg_score(true_relevance,
...           scores, k=1, ignore_ties=True)
5.0


class sklearn.metrics.DetCurveDisplay(*, fpr, fnr, estimator_name=None, pos_label=None) [source]

DET curve visualization. It is recommend to use plot_det_curve to create a visualizer. All parameters are stored as attributes. Read more in the User Guide.  New in version 0.24.   Parameters

fprndarray

False positive rate.
tprndarray

True positive rate.
estimator_namestr, default=None

Name of estimator. If None, the estimator name is not shown.
pos_labelstr or int, default=None

The label of the positive class.    Attributes

line_matplotlib Artist

DET Curve.
ax_matplotlib Axes

Axes with DET Curve.
figure_matplotlib Figure

Figure containing the curve.      See also
det_curve

Compute error rates for different probability thresholds.
plot_det_curve

Plot detection error tradeoff (DET) curve.    Examples >>> import matplotlib.pyplot as plt
>>> import numpy as np
>>> from sklearn import metrics
>>> y = np.array([0, 0, 1, 1])
>>> pred = np.array([0.1, 0.4, 0.35, 0.8])
>>> fpr, fnr, thresholds = metrics.det_curve(y, pred)
>>> display = metrics.DetCurveDisplay(
...     fpr=fpr, fnr=fnr, estimator_name='example estimator'
... )
>>> display.plot()
>>> plt.show()
 Methods
plot([ax, name]) Plot visualization.
plot(ax=None, *, name=None, **kwargs) [source]

Plot visualization.  Parameters

axmatplotlib axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
namestr, default=None

Name of DET curve for labeling. If None, use the name of the estimator.    Returns

displayDetCurveDisplay

Object that stores computed values.


sklearn.metrics.DetCurveDisplay
class sklearn.metrics.DetCurveDisplay(*, fpr, fnr, estimator_name=None, pos_label=None) [source]

DET curve visualization. It is recommend to use plot_det_curve to create a visualizer. All parameters are stored as attributes. Read more in the User Guide.  New in version 0.24.   Parameters

fprndarray

False positive rate.
tprndarray

True positive rate.
estimator_namestr, default=None

Name of estimator. If None, the estimator name is not shown.
pos_labelstr or int, default=None

The label of the positive class.    Attributes

line_matplotlib Artist

DET Curve.
ax_matplotlib Axes

Axes with DET Curve.
figure_matplotlib Figure

Figure containing the curve.      See also
det_curve

Compute error rates for different probability thresholds.
plot_det_curve

Plot detection error tradeoff (DET) curve.    Examples >>> import matplotlib.pyplot as plt
>>> import numpy as np
>>> from sklearn import metrics
>>> y = np.array([0, 0, 1, 1])
>>> pred = np.array([0.1, 0.4, 0.35, 0.8])
>>> fpr, fnr, thresholds = metrics.det_curve(y, pred)
>>> display = metrics.DetCurveDisplay(
...     fpr=fpr, fnr=fnr, estimator_name='example estimator'
... )
>>> display.plot()
>>> plt.show()
 Methods
plot([ax, name]) Plot visualization.
plot(ax=None, *, name=None, **kwargs) [source]

Plot visualization.  Parameters

axmatplotlib axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
namestr, default=None

Name of DET curve for labeling. If None, use the name of the estimator.    Returns

displayDetCurveDisplay

Object that stores computed values.


plot(ax=None, *, name=None, **kwargs) [source]

Plot visualization.  Parameters

axmatplotlib axes, default=None

Axes object to plot on. If None, a new figure and axes is created.
namestr, default=None

Name of DET curve for labeling. If None, use the name of the estimator.    Returns

displayDetCurveDisplay

Object that stores computed values.


sklearn.metrics.det_curve(y_true, y_score, pos_label=None, sample_weight=None) [source]

Compute error rates for different probability thresholds.  Note This metric is used for evaluation of ranking and error tradeoffs of a binary classification task.  Read more in the User Guide.  New in version 0.24.   Parameters

y_truendarray of shape (n_samples,)

True binary labels. If labels are not either {-1, 1} or {0, 1}, then pos_label should be explicitly given.
y_scorendarray of shape of (n_samples,)

Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by “decision_function” on some classifiers).
pos_labelint or str, default=None

The label of the positive class. When pos_label=None, if y_true is in {-1, 1} or {0, 1}, pos_label is set to 1, otherwise an error will be raised.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

fprndarray of shape (n_thresholds,)

False positive rate (FPR) such that element i is the false positive rate of predictions with score >= thresholds[i]. This is occasionally referred to as false acceptance propability or fall-out.
fnrndarray of shape (n_thresholds,)

False negative rate (FNR) such that element i is the false negative rate of predictions with score >= thresholds[i]. This is occasionally referred to as false rejection or miss rate.
thresholdsndarray of shape (n_thresholds,)

Decreasing score values.      See also
plot_det_curve

Plot detection error tradeoff (DET) curve.
DetCurveDisplay

DET curve visualization.
roc_curve

Compute Receiver operating characteristic (ROC) curve.
precision_recall_curve

Compute precision-recall curve.    Examples >>> import numpy as np
>>> from sklearn.metrics import det_curve
>>> y_true = np.array([0, 0, 1, 1])
>>> y_scores = np.array([0.1, 0.4, 0.35, 0.8])
>>> fpr, fnr, thresholds = det_curve(y_true, y_scores)
>>> fpr
array([0.5, 0.5, 0. ])
>>> fnr
array([0. , 0.5, 0.5])
>>> thresholds
array([0.35, 0.4 , 0.8 ])


sklearn.metrics.explained_variance_score(y_true, y_pred, *, sample_weight=None, multioutput='uniform_average') [source]

Explained variance regression score function. Best possible score is 1.0, lower values are worse. Read more in the User Guide.  Parameters

y_truearray-like of shape (n_samples,) or (n_samples, n_outputs)

Ground truth (correct) target values.
y_predarray-like of shape (n_samples,) or (n_samples, n_outputs)

Estimated target values.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
multioutput{‘raw_values’, ‘uniform_average’, ‘variance_weighted’} or array-like of shape (n_outputs,), default=’uniform_average’

Defines aggregating of multiple output scores. Array-like value defines weights used to average scores.  ‘raw_values’ :

Returns a full set of scores in case of multioutput input.  ‘uniform_average’ :

Scores of all outputs are averaged with uniform weight.  ‘variance_weighted’ :

Scores of all outputs are averaged, weighted by the variances of each individual output.      Returns

scorefloat or ndarray of floats

The explained variance or ndarray if ‘multioutput’ is ‘raw_values’.     Notes This is not a symmetric function. Examples >>> from sklearn.metrics import explained_variance_score
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> explained_variance_score(y_true, y_pred)
0.957...
>>> y_true = [[0.5, 1], [-1, 1], [7, -6]]
>>> y_pred = [[0, 2], [-1, 2], [8, -5]]
>>> explained_variance_score(y_true, y_pred, multioutput='uniform_average')
0.983...


sklearn.metrics.f1_score(y_true, y_pred, *, labels=None, pos_label=1, average='binary', sample_weight=None, zero_division='warn') [source]

Compute the F1 score, also known as balanced F-score or F-measure. The F1 score can be interpreted as a weighted average of the precision and recall, where an F1 score reaches its best value at 1 and worst score at 0. The relative contribution of precision and recall to the F1 score are equal. The formula for the F1 score is: F1 = 2 * (precision * recall) / (precision + recall)
 In the multi-class and multi-label case, this is the average of the F1 score of each class with weighting depending on the average parameter. Read more in the User Guide.  Parameters

y_true1d array-like, or label indicator array / sparse matrix

Ground truth (correct) target values.
y_pred1d array-like, or label indicator array / sparse matrix

Estimated targets as returned by a classifier.
labelsarray-like, default=None

The set of labels to include when average != 'binary', and their order if average is None. Labels present in the data can be excluded, for example to calculate a multiclass average ignoring a majority negative class, while labels not present in the data will result in 0 components in a macro average. For multilabel targets, labels are column indices. By default, all labels in y_true and y_pred are used in sorted order.  Changed in version 0.17: Parameter labels improved for multiclass problem.
pos_labelstr or int, default=1

The class to report if average='binary' and the data is binary. If the data are multiclass or multilabel, this will be ignored; setting labels=[pos_label] and average != 'binary' will report scores for that label only.
average{‘micro’, ‘macro’, ‘samples’,’weighted’, ‘binary’} or None, default=’binary’

This parameter is required for multiclass/multilabel targets. If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data:
'binary':

Only report results for the class specified by pos_label. This is applicable only if targets (y_{true,pred}) are binary.
'micro':

Calculate metrics globally by counting the total true positives, false negatives and false positives.
'macro':

Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.
'weighted':

Calculate metrics for each label, and find their average weighted by support (the number of true instances for each label). This alters ‘macro’ to account for label imbalance; it can result in an F-score that is not between precision and recall.
'samples':

Calculate metrics for each instance, and find their average (only meaningful for multilabel classification where this differs from accuracy_score).
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
zero_division“warn”, 0 or 1, default=”warn”

Sets the value to return when there is a zero division, i.e. when all predictions and labels are negative. If set to “warn”, this acts as 0, but warnings are also raised.    Returns

f1_scorefloat or array of float, shape = [n_unique_labels]

F1 score of the positive class in binary classification or weighted average of the F1 scores of each class for the multiclass task.      See also
fbeta_score, precision_recall_fscore_support, jaccard_score

multilabel_confusion_matrix

  Notes When true positive + false positive == 0, precision is undefined. When true positive + false negative == 0, recall is undefined. In such cases, by default the metric will be set to 0, as will f-score, and UndefinedMetricWarning will be raised. This behavior can be modified with zero_division. References
1
Wikipedia entry for the F1-score.   Examples >>> from sklearn.metrics import f1_score
>>> y_true = [0, 1, 2, 0, 1, 2]
>>> y_pred = [0, 2, 1, 0, 0, 1]
>>> f1_score(y_true, y_pred, average='macro')
0.26...
>>> f1_score(y_true, y_pred, average='micro')
0.33...
>>> f1_score(y_true, y_pred, average='weighted')
0.26...
>>> f1_score(y_true, y_pred, average=None)
array([0.8, 0. , 0. ])
>>> y_true = [0, 0, 0, 0, 0, 0]
>>> y_pred = [0, 0, 0, 0, 0, 0]
>>> f1_score(y_true, y_pred, zero_division=1)
1.0...


sklearn.metrics.fbeta_score(y_true, y_pred, *, beta, labels=None, pos_label=1, average='binary', sample_weight=None, zero_division='warn') [source]

Compute the F-beta score. The F-beta score is the weighted harmonic mean of precision and recall, reaching its optimal value at 1 and its worst value at 0. The beta parameter determines the weight of recall in the combined score. beta < 1 lends more weight to precision, while beta > 1 favors recall (beta -> 0 considers only precision, beta -> +inf only recall). Read more in the User Guide.  Parameters

y_true1d array-like, or label indicator array / sparse matrix

Ground truth (correct) target values.
y_pred1d array-like, or label indicator array / sparse matrix

Estimated targets as returned by a classifier.
betafloat

Determines the weight of recall in the combined score.
labelsarray-like, default=None

The set of labels to include when average != 'binary', and their order if average is None. Labels present in the data can be excluded, for example to calculate a multiclass average ignoring a majority negative class, while labels not present in the data will result in 0 components in a macro average. For multilabel targets, labels are column indices. By default, all labels in y_true and y_pred are used in sorted order.  Changed in version 0.17: Parameter labels improved for multiclass problem.
pos_labelstr or int, default=1

The class to report if average='binary' and the data is binary. If the data are multiclass or multilabel, this will be ignored; setting labels=[pos_label] and average != 'binary' will report scores for that label only.
average{‘micro’, ‘macro’, ‘samples’, ‘weighted’, ‘binary’} or None default=’binary’

This parameter is required for multiclass/multilabel targets. If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data:
'binary':

Only report results for the class specified by pos_label. This is applicable only if targets (y_{true,pred}) are binary.
'micro':

Calculate metrics globally by counting the total true positives, false negatives and false positives.
'macro':

Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.
'weighted':

Calculate metrics for each label, and find their average weighted by support (the number of true instances for each label). This alters ‘macro’ to account for label imbalance; it can result in an F-score that is not between precision and recall.
'samples':

Calculate metrics for each instance, and find their average (only meaningful for multilabel classification where this differs from accuracy_score).
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
zero_division“warn”, 0 or 1, default=”warn”

Sets the value to return when there is a zero division, i.e. when all predictions and labels are negative. If set to “warn”, this acts as 0, but warnings are also raised.    Returns

fbeta_scorefloat (if average is not None) or array of float, shape = [n_unique_labels]

F-beta score of the positive class in binary classification or weighted average of the F-beta score of each class for the multiclass task.      See also
precision_recall_fscore_support, multilabel_confusion_matrix

  Notes When true positive + false positive == 0 or true positive + false negative == 0, f-score returns 0 and raises UndefinedMetricWarning. This behavior can be modified with zero_division. References
1
R. Baeza-Yates and B. Ribeiro-Neto (2011). Modern Information Retrieval. Addison Wesley, pp. 327-328.
2
Wikipedia entry for the F1-score.   Examples >>> from sklearn.metrics import fbeta_score
>>> y_true = [0, 1, 2, 0, 1, 2]
>>> y_pred = [0, 2, 1, 0, 0, 1]
>>> fbeta_score(y_true, y_pred, average='macro', beta=0.5)
0.23...
>>> fbeta_score(y_true, y_pred, average='micro', beta=0.5)
0.33...
>>> fbeta_score(y_true, y_pred, average='weighted', beta=0.5)
0.23...
>>> fbeta_score(y_true, y_pred, average=None, beta=0.5)
array([0.71..., 0.        , 0.        ])


sklearn.metrics.fowlkes_mallows_score(labels_true, labels_pred, *, sparse=False) [source]

Measure the similarity of two clusterings of a set of points.  New in version 0.18.  The Fowlkes-Mallows index (FMI) is defined as the geometric mean between of the precision and recall: FMI = TP / sqrt((TP + FP) * (TP + FN))
 Where TP is the number of True Positive (i.e. the number of pair of points that belongs in the same clusters in both labels_true and labels_pred), FP is the number of False Positive (i.e. the number of pair of points that belongs in the same clusters in labels_true and not in labels_pred) and FN is the number of False Negative (i.e the number of pair of points that belongs in the same clusters in labels_pred and not in labels_True). The score ranges from 0 to 1. A high value indicates a good similarity between two clusters. Read more in the User Guide.  Parameters

labels_trueint array, shape = (n_samples,)

A clustering of the data into disjoint subsets.
labels_predarray, shape = (n_samples, )

A clustering of the data into disjoint subsets.
sparsebool, default=False

Compute contingency matrix internally with sparse matrix.    Returns

scorefloat

The resulting Fowlkes-Mallows score.     References
1
E. B. Fowkles and C. L. Mallows, 1983. “A method for comparing two hierarchical clusterings”. Journal of the American Statistical Association
2
Wikipedia entry for the Fowlkes-Mallows Index   Examples Perfect labelings are both homogeneous and complete, hence have score 1.0: >>> from sklearn.metrics.cluster import fowlkes_mallows_score
>>> fowlkes_mallows_score([0, 0, 1, 1], [0, 0, 1, 1])
1.0
>>> fowlkes_mallows_score([0, 0, 1, 1], [1, 1, 0, 0])
1.0
 If classes members are completely split across different clusters, the assignment is totally random, hence the FMI is null: >>> fowlkes_mallows_score([0, 0, 0, 0], [0, 1, 2, 3])
0.0


sklearn.metrics.get_scorer(scoring) [source]

Get a scorer from string. Read more in the User Guide.  Parameters

scoringstr or callable

Scoring method as string. If callable it is returned as is.    Returns

scorercallable

The scorer.


sklearn.metrics.hamming_loss(y_true, y_pred, *, sample_weight=None) [source]

Compute the average Hamming loss. The Hamming loss is the fraction of labels that are incorrectly predicted. Read more in the User Guide.  Parameters

y_true1d array-like, or label indicator array / sparse matrix

Ground truth (correct) labels.
y_pred1d array-like, or label indicator array / sparse matrix

Predicted labels, as returned by a classifier.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.  New in version 0.18.     Returns

lossfloat or int

Return the average Hamming loss between element of y_true and y_pred.      See also
accuracy_score, jaccard_score, zero_one_loss

  Notes In multiclass classification, the Hamming loss corresponds to the Hamming distance between y_true and y_pred which is equivalent to the subset zero_one_loss function, when normalize parameter is set to True. In multilabel classification, the Hamming loss is different from the subset zero-one loss. The zero-one loss considers the entire set of labels for a given sample incorrect if it does not entirely match the true set of labels. Hamming loss is more forgiving in that it penalizes only the individual labels. The Hamming loss is upperbounded by the subset zero-one loss, when normalize parameter is set to True. It is always between 0 and 1, lower being better. References
1
Grigorios Tsoumakas, Ioannis Katakis. Multi-Label Classification: An Overview. International Journal of Data Warehousing & Mining, 3(3), 1-13, July-September 2007.
2
Wikipedia entry on the Hamming distance.   Examples >>> from sklearn.metrics import hamming_loss
>>> y_pred = [1, 2, 3, 4]
>>> y_true = [2, 2, 3, 4]
>>> hamming_loss(y_true, y_pred)
0.25
 In the multilabel case with binary label indicators: >>> import numpy as np
>>> hamming_loss(np.array([[0, 1], [1, 1]]), np.zeros((2, 2)))
0.75


sklearn.metrics.hinge_loss(y_true, pred_decision, *, labels=None, sample_weight=None) [source]

Average hinge loss (non-regularized). In binary class case, assuming labels in y_true are encoded with +1 and -1, when a prediction mistake is made, margin = y_true * pred_decision is always negative (since the signs disagree), implying 1 - margin is always greater than 1. The cumulated hinge loss is therefore an upper bound of the number of mistakes made by the classifier. In multiclass case, the function expects that either all the labels are included in y_true or an optional labels argument is provided which contains all the labels. The multilabel margin is calculated according to Crammer-Singer’s method. As in the binary case, the cumulated hinge loss is an upper bound of the number of mistakes made by the classifier. Read more in the User Guide.  Parameters

y_truearray of shape (n_samples,)

True target, consisting of integers of two values. The positive label must be greater than the negative label.
pred_decisionarray of shape (n_samples,) or (n_samples, n_classes)

Predicted decisions, as output by decision_function (floats).
labelsarray-like, default=None

Contains all the labels for the problem. Used in multiclass hinge loss.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

lossfloat
   References
1
Wikipedia entry on the Hinge loss.
2
Koby Crammer, Yoram Singer. On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines. Journal of Machine Learning Research 2, (2001), 265-292.
3
L1 AND L2 Regularization for Multiclass Hinge Loss Models by Robert C. Moore, John DeNero.   Examples >>> from sklearn import svm
>>> from sklearn.metrics import hinge_loss
>>> X = [[0], [1]]
>>> y = [-1, 1]
>>> est = svm.LinearSVC(random_state=0)
>>> est.fit(X, y)
LinearSVC(random_state=0)
>>> pred_decision = est.decision_function([[-2], [3], [0.5]])
>>> pred_decision
array([-2.18...,  2.36...,  0.09...])
>>> hinge_loss([-1, 1, 1], pred_decision)
0.30...
 In the multiclass case: >>> import numpy as np
>>> X = np.array([[0], [1], [2], [3]])
>>> Y = np.array([0, 1, 2, 3])
>>> labels = np.array([0, 1, 2, 3])
>>> est = svm.LinearSVC()
>>> est.fit(X, Y)
LinearSVC()
>>> pred_decision = est.decision_function([[-1], [2], [3]])
>>> y_true = [0, 2, 3]
>>> hinge_loss(y_true, pred_decision, labels=labels)
0.56...


sklearn.metrics.homogeneity_completeness_v_measure(labels_true, labels_pred, *, beta=1.0) [source]

Compute the homogeneity and completeness and V-Measure scores at once. Those metrics are based on normalized conditional entropy measures of the clustering labeling to evaluate given the knowledge of a Ground Truth class labels of the same samples. A clustering result satisfies homogeneity if all of its clusters contain only data points which are members of a single class. A clustering result satisfies completeness if all the data points that are members of a given class are elements of the same cluster. Both scores have positive values between 0.0 and 1.0, larger values being desirable. Those 3 metrics are independent of the absolute values of the labels: a permutation of the class or cluster label values won’t change the score values in any way. V-Measure is furthermore symmetric: swapping labels_true and label_pred will give the same score. This does not hold for homogeneity and completeness. V-Measure is identical to normalized_mutual_info_score with the arithmetic averaging method. Read more in the User Guide.  Parameters

labels_trueint array, shape = [n_samples]

ground truth class labels to be used as a reference
labels_predarray-like of shape (n_samples,)

cluster labels to evaluate
betafloat, default=1.0

Ratio of weight attributed to homogeneity vs completeness. If beta is greater than 1, completeness is weighted more strongly in the calculation. If beta is less than 1, homogeneity is weighted more strongly.    Returns

homogeneityfloat

score between 0.0 and 1.0. 1.0 stands for perfectly homogeneous labeling
completenessfloat

score between 0.0 and 1.0. 1.0 stands for perfectly complete labeling
v_measurefloat

harmonic mean of the first two      See also
homogeneity_score

completeness_score

v_measure_score


sklearn.metrics.homogeneity_score(labels_true, labels_pred) [source]

Homogeneity metric of a cluster labeling given a ground truth. A clustering result satisfies homogeneity if all of its clusters contain only data points which are members of a single class. This metric is independent of the absolute values of the labels: a permutation of the class or cluster label values won’t change the score value in any way. This metric is not symmetric: switching label_true with label_pred will return the completeness_score which will be different in general. Read more in the User Guide.  Parameters

labels_trueint array, shape = [n_samples]

ground truth class labels to be used as a reference
labels_predarray-like of shape (n_samples,)

cluster labels to evaluate    Returns

homogeneityfloat

score between 0.0 and 1.0. 1.0 stands for perfectly homogeneous labeling      See also
completeness_score

v_measure_score

  References
1
Andrew Rosenberg and Julia Hirschberg, 2007. V-Measure: A conditional entropy-based external cluster evaluation measure   Examples Perfect labelings are homogeneous: >>> from sklearn.metrics.cluster import homogeneity_score
>>> homogeneity_score([0, 0, 1, 1], [1, 1, 0, 0])
1.0
 Non-perfect labelings that further split classes into more clusters can be perfectly homogeneous: >>> print("%.6f" % homogeneity_score([0, 0, 1, 1], [0, 0, 1, 2]))
1.000000
>>> print("%.6f" % homogeneity_score([0, 0, 1, 1], [0, 1, 2, 3]))
1.000000
 Clusters that include samples from different classes do not make for an homogeneous labeling: >>> print("%.6f" % homogeneity_score([0, 0, 1, 1], [0, 1, 0, 1]))
0.0...
>>> print("%.6f" % homogeneity_score([0, 0, 1, 1], [0, 0, 0, 0]))
0.0...


sklearn.metrics.jaccard_score(y_true, y_pred, *, labels=None, pos_label=1, average='binary', sample_weight=None, zero_division='warn') [source]

Jaccard similarity coefficient score. The Jaccard index [1], or Jaccard similarity coefficient, defined as the size of the intersection divided by the size of the union of two label sets, is used to compare set of predicted labels for a sample to the corresponding set of labels in y_true. Read more in the User Guide.  Parameters

y_true1d array-like, or label indicator array / sparse matrix

Ground truth (correct) labels.
y_pred1d array-like, or label indicator array / sparse matrix

Predicted labels, as returned by a classifier.
labelsarray-like of shape (n_classes,), default=None

The set of labels to include when average != 'binary', and their order if average is None. Labels present in the data can be excluded, for example to calculate a multiclass average ignoring a majority negative class, while labels not present in the data will result in 0 components in a macro average. For multilabel targets, labels are column indices. By default, all labels in y_true and y_pred are used in sorted order.
pos_labelstr or int, default=1

The class to report if average='binary' and the data is binary. If the data are multiclass or multilabel, this will be ignored; setting labels=[pos_label] and average != 'binary' will report scores for that label only.
average{None, ‘micro’, ‘macro’, ‘samples’, ‘weighted’, ‘binary’}, default=’binary’

If None, the scores for each class are returned. Otherwise, this determines the type of averaging performed on the data:
'binary':

Only report results for the class specified by pos_label. This is applicable only if targets (y_{true,pred}) are binary.
'micro':

Calculate metrics globally by counting the total true positives, false negatives and false positives.
'macro':

Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.
'weighted':

Calculate metrics for each label, and find their average, weighted by support (the number of true instances for each label). This alters ‘macro’ to account for label imbalance.
'samples':

Calculate metrics for each instance, and find their average (only meaningful for multilabel classification).
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
zero_division“warn”, {0.0, 1.0}, default=”warn”

Sets the value to return when there is a zero division, i.e. when there there are no negative values in predictions and labels. If set to “warn”, this acts like 0, but a warning is also raised.    Returns

scorefloat (if average is not None) or array of floats, shape = [n_unique_labels]
    See also
accuracy_score, f_score, multilabel_confusion_matrix

  Notes jaccard_score may be a poor metric if there are no positives for some samples or classes. Jaccard is undefined if there are no true or predicted labels, and our implementation will return a score of 0 with a warning. References
1
Wikipedia entry for the Jaccard index.   Examples >>> import numpy as np
>>> from sklearn.metrics import jaccard_score
>>> y_true = np.array([[0, 1, 1],
...                    [1, 1, 0]])
>>> y_pred = np.array([[1, 1, 1],
...                    [1, 0, 0]])
 In the binary case: >>> jaccard_score(y_true[0], y_pred[0])
0.6666...
 In the multilabel case: >>> jaccard_score(y_true, y_pred, average='samples')
0.5833...
>>> jaccard_score(y_true, y_pred, average='macro')
0.6666...
>>> jaccard_score(y_true, y_pred, average=None)
array([0.5, 0.5, 1. ])
 In the multiclass case: >>> y_pred = [0, 2, 1, 2]
>>> y_true = [0, 1, 2, 2]
>>> jaccard_score(y_true, y_pred, average=None)
array([1. , 0. , 0.33...])


sklearn.metrics.label_ranking_average_precision_score(y_true, y_score, *, sample_weight=None) [source]

Compute ranking-based average precision. Label ranking average precision (LRAP) is the average over each ground truth label assigned to each sample, of the ratio of true vs. total labels with lower score. This metric is used in multilabel ranking problem, where the goal is to give better rank to the labels associated to each sample. The obtained score is always strictly greater than 0 and the best value is 1. Read more in the User Guide.  Parameters

y_true{ndarray, sparse matrix} of shape (n_samples, n_labels)

True binary labels in binary indicator format.
y_scorendarray of shape (n_samples, n_labels)

Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by “decision_function” on some classifiers).
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.  New in version 0.20.     Returns

scorefloat
   Examples >>> import numpy as np
>>> from sklearn.metrics import label_ranking_average_precision_score
>>> y_true = np.array([[1, 0, 0], [0, 0, 1]])
>>> y_score = np.array([[0.75, 0.5, 1], [1, 0.2, 0.1]])
>>> label_ranking_average_precision_score(y_true, y_score)
0.416...


sklearn.metrics.label_ranking_loss(y_true, y_score, *, sample_weight=None) [source]

Compute Ranking loss measure. Compute the average number of label pairs that are incorrectly ordered given y_score weighted by the size of the label set and the number of labels not in the label set. This is similar to the error set size, but weighted by the number of relevant and irrelevant labels. The best performance is achieved with a ranking loss of zero. Read more in the User Guide.  New in version 0.17: A function label_ranking_loss   Parameters

y_true{ndarray, sparse matrix} of shape (n_samples, n_labels)

True binary labels in binary indicator format.
y_scorendarray of shape (n_samples, n_labels)

Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by “decision_function” on some classifiers).
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

lossfloat
   References
1
Tsoumakas, G., Katakis, I., & Vlahavas, I. (2010). Mining multi-label data. In Data mining and knowledge discovery handbook (pp. 667-685). Springer US.


sklearn.metrics.log_loss(y_true, y_pred, *, eps=1e-15, normalize=True, sample_weight=None, labels=None) [source]

Log loss, aka logistic loss or cross-entropy loss. This is the loss function used in (multinomial) logistic regression and extensions of it such as neural networks, defined as the negative log-likelihood of a logistic model that returns y_pred probabilities for its training data y_true. The log loss is only defined for two or more labels. For a single sample with true label \(y \in \{0,1\}\) and and a probability estimate \(p = \operatorname{Pr}(y = 1)\), the log loss is:  \[L_{\log}(y, p) = -(y \log (p) + (1 - y) \log (1 - p))\] Read more in the User Guide.  Parameters

y_truearray-like or label indicator matrix

Ground truth (correct) labels for n_samples samples.
y_predarray-like of float, shape = (n_samples, n_classes) or (n_samples,)

Predicted probabilities, as returned by a classifier’s predict_proba method. If y_pred.shape = (n_samples,) the probabilities provided are assumed to be that of the positive class. The labels in y_pred are assumed to be ordered alphabetically, as done by preprocessing.LabelBinarizer.
epsfloat, default=1e-15

Log loss is undefined for p=0 or p=1, so probabilities are clipped to max(eps, min(1 - eps, p)).
normalizebool, default=True

If true, return the mean loss per sample. Otherwise, return the sum of the per-sample losses.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
labelsarray-like, default=None

If not provided, labels will be inferred from y_true. If labels is None and y_pred has shape (n_samples,) the labels are assumed to be binary and are inferred from y_true.  New in version 0.18.     Returns

lossfloat
   Notes The logarithm used is the natural logarithm (base-e). References C.M. Bishop (2006). Pattern Recognition and Machine Learning. Springer, p. 209. Examples >>> from sklearn.metrics import log_loss
>>> log_loss(["spam", "ham", "ham", "spam"],
...          [[.1, .9], [.9, .1], [.8, .2], [.35, .65]])
0.21616...


sklearn.metrics.make_scorer(score_func, *, greater_is_better=True, needs_proba=False, needs_threshold=False, **kwargs) [source]

Make a scorer from a performance metric or loss function. This factory function wraps scoring functions for use in GridSearchCV and cross_val_score. It takes a score function, such as accuracy_score, mean_squared_error, adjusted_rand_index or average_precision and returns a callable that scores an estimator’s output. The signature of the call is (estimator, X, y) where estimator is the model to be evaluated, X is the data and y is the ground truth labeling (or None in the case of unsupervised models). Read more in the User Guide.  Parameters

score_funccallable

Score function (or loss function) with signature score_func(y, y_pred, **kwargs).
greater_is_betterbool, default=True

Whether score_func is a score function (default), meaning high is good, or a loss function, meaning low is good. In the latter case, the scorer object will sign-flip the outcome of the score_func.
needs_probabool, default=False

Whether score_func requires predict_proba to get probability estimates out of a classifier. If True, for binary y_true, the score function is supposed to accept a 1D y_pred (i.e., probability of the positive class, shape (n_samples,)).
needs_thresholdbool, default=False

Whether score_func takes a continuous decision certainty. This only works for binary classification using estimators that have either a decision_function or predict_proba method. If True, for binary y_true, the score function is supposed to accept a 1D y_pred (i.e., probability of the positive class or the decision function, shape (n_samples,)). For example average_precision or the area under the roc curve can not be computed using discrete predictions alone.
**kwargsadditional arguments

Additional parameters to be passed to score_func.    Returns

scorercallable

Callable object that returns a scalar score; greater is better.     Notes If needs_proba=False and needs_threshold=False, the score function is supposed to accept the output of predict. If needs_proba=True, the score function is supposed to accept the output of predict_proba (For binary y_true, the score function is supposed to accept probability of the positive class). If needs_threshold=True, the score function is supposed to accept the output of decision_function. Examples >>> from sklearn.metrics import fbeta_score, make_scorer
>>> ftwo_scorer = make_scorer(fbeta_score, beta=2)
>>> ftwo_scorer
make_scorer(fbeta_score, beta=2)
>>> from sklearn.model_selection import GridSearchCV
>>> from sklearn.svm import LinearSVC
>>> grid = GridSearchCV(LinearSVC(), param_grid={'C': [1, 10]},
...                     scoring=ftwo_scorer)


sklearn.metrics.matthews_corrcoef(y_true, y_pred, *, sample_weight=None) [source]

Compute the Matthews correlation coefficient (MCC). The Matthews correlation coefficient is used in machine learning as a measure of the quality of binary and multiclass classifications. It takes into account true and false positives and negatives and is generally regarded as a balanced measure which can be used even if the classes are of very different sizes. The MCC is in essence a correlation coefficient value between -1 and +1. A coefficient of +1 represents a perfect prediction, 0 an average random prediction and -1 an inverse prediction. The statistic is also known as the phi coefficient. [source: Wikipedia] Binary and multiclass labels are supported. Only in the binary case does this relate to information about true and false positives and negatives. See references below. Read more in the User Guide.  Parameters

y_truearray, shape = [n_samples]

Ground truth (correct) target values.
y_predarray, shape = [n_samples]

Estimated targets as returned by a classifier.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.  New in version 0.18.     Returns

mccfloat

The Matthews correlation coefficient (+1 represents a perfect prediction, 0 an average random prediction and -1 and inverse prediction).     References
1
Baldi, Brunak, Chauvin, Andersen and Nielsen, (2000). Assessing the accuracy of prediction algorithms for classification: an overview.
2
Wikipedia entry for the Matthews Correlation Coefficient.
3
Gorodkin, (2004). Comparing two K-category assignments by a K-category correlation coefficient.
4
Jurman, Riccadonna, Furlanello, (2012). A Comparison of MCC and CEN Error Measures in MultiClass Prediction.   Examples >>> from sklearn.metrics import matthews_corrcoef
>>> y_true = [+1, +1, +1, -1]
>>> y_pred = [+1, -1, +1, +1]
>>> matthews_corrcoef(y_true, y_pred)
-0.33...


sklearn.metrics.max_error(y_true, y_pred) [source]

max_error metric calculates the maximum residual error. Read more in the User Guide.  Parameters

y_truearray-like of shape (n_samples,)

Ground truth (correct) target values.
y_predarray-like of shape (n_samples,)

Estimated target values.    Returns

max_errorfloat

A positive floating point value (the best value is 0.0).     Examples >>> from sklearn.metrics import max_error
>>> y_true = [3, 2, 7, 1]
>>> y_pred = [4, 2, 7, 1]
>>> max_error(y_true, y_pred)
1


sklearn.metrics.mean_absolute_error(y_true, y_pred, *, sample_weight=None, multioutput='uniform_average') [source]

Mean absolute error regression loss. Read more in the User Guide.  Parameters

y_truearray-like of shape (n_samples,) or (n_samples, n_outputs)

Ground truth (correct) target values.
y_predarray-like of shape (n_samples,) or (n_samples, n_outputs)

Estimated target values.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
multioutput{‘raw_values’, ‘uniform_average’} or array-like of shape (n_outputs,), default=’uniform_average’

Defines aggregating of multiple output values. Array-like value defines weights used to average errors.  ‘raw_values’ :

Returns a full set of errors in case of multioutput input.  ‘uniform_average’ :

Errors of all outputs are averaged with uniform weight.      Returns

lossfloat or ndarray of floats

If multioutput is ‘raw_values’, then mean absolute error is returned for each output separately. If multioutput is ‘uniform_average’ or an ndarray of weights, then the weighted average of all output errors is returned. MAE output is non-negative floating point. The best value is 0.0.     Examples >>> from sklearn.metrics import mean_absolute_error
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> mean_absolute_error(y_true, y_pred)
0.5
>>> y_true = [[0.5, 1], [-1, 1], [7, -6]]
>>> y_pred = [[0, 2], [-1, 2], [8, -5]]
>>> mean_absolute_error(y_true, y_pred)
0.75
>>> mean_absolute_error(y_true, y_pred, multioutput='raw_values')
array([0.5, 1. ])
>>> mean_absolute_error(y_true, y_pred, multioutput=[0.3, 0.7])
0.85...


sklearn.metrics.mean_absolute_percentage_error(y_true, y_pred, sample_weight=None, multioutput='uniform_average') [source]

Mean absolute percentage error regression loss. Note here that we do not represent the output as a percentage in range [0, 100]. Instead, we represent it in range [0, 1/eps]. Read more in the User Guide.  New in version 0.24.   Parameters

y_truearray-like of shape (n_samples,) or (n_samples, n_outputs)

Ground truth (correct) target values.
y_predarray-like of shape (n_samples,) or (n_samples, n_outputs)

Estimated target values.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
multioutput{‘raw_values’, ‘uniform_average’} or array-like

Defines aggregating of multiple output values. Array-like value defines weights used to average errors. If input is list then the shape must be (n_outputs,).  ‘raw_values’ :

Returns a full set of errors in case of multioutput input.  ‘uniform_average’ :

Errors of all outputs are averaged with uniform weight.      Returns

lossfloat or ndarray of floats in the range [0, 1/eps]

If multioutput is ‘raw_values’, then mean absolute percentage error is returned for each output separately. If multioutput is ‘uniform_average’ or an ndarray of weights, then the weighted average of all output errors is returned. MAPE output is non-negative floating point. The best value is 0.0. But note the fact that bad predictions can lead to arbitarily large MAPE values, especially if some y_true values are very close to zero. Note that we return a large value instead of inf when y_true is zero.     Examples >>> from sklearn.metrics import mean_absolute_percentage_error
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> mean_absolute_percentage_error(y_true, y_pred)
0.3273...
>>> y_true = [[0.5, 1], [-1, 1], [7, -6]]
>>> y_pred = [[0, 2], [-1, 2], [8, -5]]
>>> mean_absolute_percentage_error(y_true, y_pred)
0.5515...
>>> mean_absolute_percentage_error(y_true, y_pred, multioutput=[0.3, 0.7])
0.6198...


sklearn.metrics.mean_gamma_deviance(y_true, y_pred, *, sample_weight=None) [source]

Mean Gamma deviance regression loss. Gamma deviance is equivalent to the Tweedie deviance with the power parameter power=2. It is invariant to scaling of the target variable, and measures relative errors. Read more in the User Guide.  Parameters

y_truearray-like of shape (n_samples,)

Ground truth (correct) target values. Requires y_true > 0.
y_predarray-like of shape (n_samples,)

Estimated target values. Requires y_pred > 0.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

lossfloat

A non-negative floating point value (the best value is 0.0).     Examples >>> from sklearn.metrics import mean_gamma_deviance
>>> y_true = [2, 0.5, 1, 4]
>>> y_pred = [0.5, 0.5, 2., 2.]
>>> mean_gamma_deviance(y_true, y_pred)
1.0568...


sklearn.metrics.mean_poisson_deviance(y_true, y_pred, *, sample_weight=None) [source]

Mean Poisson deviance regression loss. Poisson deviance is equivalent to the Tweedie deviance with the power parameter power=1. Read more in the User Guide.  Parameters

y_truearray-like of shape (n_samples,)

Ground truth (correct) target values. Requires y_true >= 0.
y_predarray-like of shape (n_samples,)

Estimated target values. Requires y_pred > 0.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

lossfloat

A non-negative floating point value (the best value is 0.0).     Examples >>> from sklearn.metrics import mean_poisson_deviance
>>> y_true = [2, 0, 1, 4]
>>> y_pred = [0.5, 0.5, 2., 2.]
>>> mean_poisson_deviance(y_true, y_pred)
1.4260...


sklearn.metrics.mean_squared_error(y_true, y_pred, *, sample_weight=None, multioutput='uniform_average', squared=True) [source]

Mean squared error regression loss. Read more in the User Guide.  Parameters

y_truearray-like of shape (n_samples,) or (n_samples, n_outputs)

Ground truth (correct) target values.
y_predarray-like of shape (n_samples,) or (n_samples, n_outputs)

Estimated target values.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
multioutput{‘raw_values’, ‘uniform_average’} or array-like of shape (n_outputs,), default=’uniform_average’

Defines aggregating of multiple output values. Array-like value defines weights used to average errors.  ‘raw_values’ :

Returns a full set of errors in case of multioutput input.  ‘uniform_average’ :

Errors of all outputs are averaged with uniform weight.
squaredbool, default=True

If True returns MSE value, if False returns RMSE value.    Returns

lossfloat or ndarray of floats

A non-negative floating point value (the best value is 0.0), or an array of floating point values, one for each individual target.     Examples >>> from sklearn.metrics import mean_squared_error
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> mean_squared_error(y_true, y_pred)
0.375
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> mean_squared_error(y_true, y_pred, squared=False)
0.612...
>>> y_true = [[0.5, 1],[-1, 1],[7, -6]]
>>> y_pred = [[0, 2],[-1, 2],[8, -5]]
>>> mean_squared_error(y_true, y_pred)
0.708...
>>> mean_squared_error(y_true, y_pred, squared=False)
0.822...
>>> mean_squared_error(y_true, y_pred, multioutput='raw_values')
array([0.41666667, 1.        ])
>>> mean_squared_error(y_true, y_pred, multioutput=[0.3, 0.7])
0.825...


sklearn.metrics.mean_squared_log_error(y_true, y_pred, *, sample_weight=None, multioutput='uniform_average') [source]

Mean squared logarithmic error regression loss. Read more in the User Guide.  Parameters

y_truearray-like of shape (n_samples,) or (n_samples, n_outputs)

Ground truth (correct) target values.
y_predarray-like of shape (n_samples,) or (n_samples, n_outputs)

Estimated target values.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
multioutput{‘raw_values’, ‘uniform_average’} or array-like of shape (n_outputs,), default=’uniform_average’

Defines aggregating of multiple output values. Array-like value defines weights used to average errors.  ‘raw_values’ :

Returns a full set of errors when the input is of multioutput format.  ‘uniform_average’ :

Errors of all outputs are averaged with uniform weight.      Returns

lossfloat or ndarray of floats

A non-negative floating point value (the best value is 0.0), or an array of floating point values, one for each individual target.     Examples >>> from sklearn.metrics import mean_squared_log_error
>>> y_true = [3, 5, 2.5, 7]
>>> y_pred = [2.5, 5, 4, 8]
>>> mean_squared_log_error(y_true, y_pred)
0.039...
>>> y_true = [[0.5, 1], [1, 2], [7, 6]]
>>> y_pred = [[0.5, 2], [1, 2.5], [8, 8]]
>>> mean_squared_log_error(y_true, y_pred)
0.044...
>>> mean_squared_log_error(y_true, y_pred, multioutput='raw_values')
array([0.00462428, 0.08377444])
>>> mean_squared_log_error(y_true, y_pred, multioutput=[0.3, 0.7])
0.060...


sklearn.metrics.mean_tweedie_deviance(y_true, y_pred, *, sample_weight=None, power=0) [source]

Mean Tweedie deviance regression loss. Read more in the User Guide.  Parameters

y_truearray-like of shape (n_samples,)

Ground truth (correct) target values.
y_predarray-like of shape (n_samples,)

Estimated target values.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
powerfloat, default=0

Tweedie power parameter. Either power <= 0 or power >= 1. The higher p the less weight is given to extreme deviations between true and predicted targets.  power < 0: Extreme stable distribution. Requires: y_pred > 0. power = 0 : Normal distribution, output corresponds to mean_squared_error. y_true and y_pred can be any real numbers. power = 1 : Poisson distribution. Requires: y_true >= 0 and y_pred > 0. 1 < p < 2 : Compound Poisson distribution. Requires: y_true >= 0 and y_pred > 0. power = 2 : Gamma distribution. Requires: y_true > 0 and y_pred > 0. power = 3 : Inverse Gaussian distribution. Requires: y_true > 0 and y_pred > 0. otherwise : Positive stable distribution. Requires: y_true > 0 and y_pred > 0.     Returns

lossfloat

A non-negative floating point value (the best value is 0.0).     Examples >>> from sklearn.metrics import mean_tweedie_deviance
>>> y_true = [2, 0, 1, 4]
>>> y_pred = [0.5, 0.5, 2., 2.]
>>> mean_tweedie_deviance(y_true, y_pred, power=1)
1.4260...


sklearn.metrics.median_absolute_error(y_true, y_pred, *, multioutput='uniform_average', sample_weight=None) [source]

Median absolute error regression loss. Median absolute error output is non-negative floating point. The best value is 0.0. Read more in the User Guide.  Parameters

y_truearray-like of shape = (n_samples) or (n_samples, n_outputs)

Ground truth (correct) target values.
y_predarray-like of shape = (n_samples) or (n_samples, n_outputs)

Estimated target values.
multioutput{‘raw_values’, ‘uniform_average’} or array-like of shape (n_outputs,), default=’uniform_average’

Defines aggregating of multiple output values. Array-like value defines weights used to average errors.  ‘raw_values’ :

Returns a full set of errors in case of multioutput input.  ‘uniform_average’ :

Errors of all outputs are averaged with uniform weight.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.  New in version 0.24.     Returns

lossfloat or ndarray of floats

If multioutput is ‘raw_values’, then mean absolute error is returned for each output separately. If multioutput is ‘uniform_average’ or an ndarray of weights, then the weighted average of all output errors is returned.     Examples >>> from sklearn.metrics import median_absolute_error
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> median_absolute_error(y_true, y_pred)
0.5
>>> y_true = [[0.5, 1], [-1, 1], [7, -6]]
>>> y_pred = [[0, 2], [-1, 2], [8, -5]]
>>> median_absolute_error(y_true, y_pred)
0.75
>>> median_absolute_error(y_true, y_pred, multioutput='raw_values')
array([0.5, 1. ])
>>> median_absolute_error(y_true, y_pred, multioutput=[0.3, 0.7])
0.85


sklearn.metrics.multilabel_confusion_matrix(y_true, y_pred, *, sample_weight=None, labels=None, samplewise=False) [source]

Compute a confusion matrix for each class or sample.  New in version 0.21.  Compute class-wise (default) or sample-wise (samplewise=True) multilabel confusion matrix to evaluate the accuracy of a classification, and output confusion matrices for each class or sample. In multilabel confusion matrix \(MCM\), the count of true negatives is \(MCM_{:,0,0}\), false negatives is \(MCM_{:,1,0}\), true positives is \(MCM_{:,1,1}\) and false positives is \(MCM_{:,0,1}\). Multiclass data will be treated as if binarized under a one-vs-rest transformation. Returned confusion matrices will be in the order of sorted unique labels in the union of (y_true, y_pred). Read more in the User Guide.  Parameters

y_true{array-like, sparse matrix} of shape (n_samples, n_outputs) or (n_samples,)

Ground truth (correct) target values.
y_pred{array-like, sparse matrix} of shape (n_samples, n_outputs) or (n_samples,)

Estimated targets as returned by a classifier.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.
labelsarray-like of shape (n_classes,), default=None

A list of classes or column indices to select some (or to force inclusion of classes absent from the data).
samplewisebool, default=False

In the multilabel case, this calculates a confusion matrix per sample.    Returns

multi_confusionndarray of shape (n_outputs, 2, 2)

A 2x2 confusion matrix corresponding to each output in the input. When calculating class-wise multi_confusion (default), then n_outputs = n_labels; when calculating sample-wise multi_confusion (samplewise=True), n_outputs = n_samples. If labels is defined, the results will be returned in the order specified in labels, otherwise the results will be returned in sorted order by default.      See also
confusion_matrix

  Notes The multilabel_confusion_matrix calculates class-wise or sample-wise multilabel confusion matrices, and in multiclass tasks, labels are binarized under a one-vs-rest way; while confusion_matrix calculates one confusion matrix for confusion between every two classes. Examples Multilabel-indicator case: >>> import numpy as np
>>> from sklearn.metrics import multilabel_confusion_matrix
>>> y_true = np.array([[1, 0, 1],
...                    [0, 1, 0]])
>>> y_pred = np.array([[1, 0, 0],
...                    [0, 1, 1]])
>>> multilabel_confusion_matrix(y_true, y_pred)
array([[[1, 0],
        [0, 1]],

       [[1, 0],
        [0, 1]],

       [[0, 1],
        [1, 0]]])
 Multiclass case: >>> y_true = ["cat", "ant", "cat", "cat", "ant", "bird"]
>>> y_pred = ["ant", "ant", "cat", "cat", "ant", "cat"]
>>> multilabel_confusion_matrix(y_true, y_pred,
...                             labels=["ant", "bird", "cat"])
array([[[3, 1],
        [0, 2]],

       [[5, 0],
        [1, 0]],

       [[2, 1],
        [1, 2]]])


sklearn.metrics.mutual_info_score(labels_true, labels_pred, *, contingency=None) [source]

Mutual Information between two clusterings. The Mutual Information is a measure of the similarity between two labels of the same data. Where \(|U_i|\) is the number of the samples in cluster \(U_i\) and \(|V_j|\) is the number of the samples in cluster \(V_j\), the Mutual Information between clusterings \(U\) and \(V\) is given as:  \[MI(U,V)=\sum_{i=1}^{|U|} \sum_{j=1}^{|V|} \frac{|U_i\cap V_j|}{N} \log\frac{N|U_i \cap V_j|}{|U_i||V_j|}\] This metric is independent of the absolute values of the labels: a permutation of the class or cluster label values won’t change the score value in any way. This metric is furthermore symmetric: switching label_true with label_pred will return the same score value. This can be useful to measure the agreement of two independent label assignments strategies on the same dataset when the real ground truth is not known. Read more in the User Guide.  Parameters

labels_trueint array, shape = [n_samples]

A clustering of the data into disjoint subsets.
labels_predint array-like of shape (n_samples,)

A clustering of the data into disjoint subsets.
contingency{ndarray, sparse matrix} of shape (n_classes_true, n_classes_pred), default=None

A contingency matrix given by the contingency_matrix function. If value is None, it will be computed, otherwise the given value is used, with labels_true and labels_pred ignored.    Returns

mifloat

Mutual information, a non-negative value      See also
adjusted_mutual_info_score

Adjusted against chance Mutual Information.
normalized_mutual_info_score

Normalized Mutual Information.    Notes The logarithm used is the natural logarithm (base-e).


sklearn.metrics.ndcg_score(y_true, y_score, *, k=None, sample_weight=None, ignore_ties=False) [source]

Compute Normalized Discounted Cumulative Gain. Sum the true scores ranked in the order induced by the predicted scores, after applying a logarithmic discount. Then divide by the best possible score (Ideal DCG, obtained for a perfect ranking) to obtain a score between 0 and 1. This ranking metric yields a high value if true labels are ranked high by y_score.  Parameters

y_truendarray of shape (n_samples, n_labels)

True targets of multilabel classification, or true scores of entities to be ranked.
y_scorendarray of shape (n_samples, n_labels)

Target scores, can either be probability estimates, confidence values, or non-thresholded measure of decisions (as returned by “decision_function” on some classifiers).
kint, default=None

Only consider the highest k scores in the ranking. If None, use all outputs.
sample_weightndarray of shape (n_samples,), default=None

Sample weights. If None, all samples are given the same weight.
ignore_tiesbool, default=False

Assume that there are no ties in y_score (which is likely to be the case if y_score is continuous) for efficiency gains.    Returns

normalized_discounted_cumulative_gainfloat in [0., 1.]

The averaged NDCG scores for all samples.      See also
dcg_score

Discounted Cumulative Gain (not normalized).    References Wikipedia entry for Discounted Cumulative Gain Jarvelin, K., & Kekalainen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS), 20(4), 422-446. Wang, Y., Wang, L., Li, Y., He, D., Chen, W., & Liu, T. Y. (2013, May). A theoretical analysis of NDCG ranking measures. In Proceedings of the 26th Annual Conference on Learning Theory (COLT 2013) McSherry, F., & Najork, M. (2008, March). Computing information retrieval performance measures efficiently in the presence of tied scores. In European conference on information retrieval (pp. 414-421). Springer, Berlin, Heidelberg. Examples >>> from sklearn.metrics import ndcg_score
>>> # we have groud-truth relevance of some answers to a query:
>>> true_relevance = np.asarray([[10, 0, 0, 1, 5]])
>>> # we predict some scores (relevance) for the answers
>>> scores = np.asarray([[.1, .2, .3, 4, 70]])
>>> ndcg_score(true_relevance, scores)
0.69...
>>> scores = np.asarray([[.05, 1.1, 1., .5, .0]])
>>> ndcg_score(true_relevance, scores)
0.49...
>>> # we can set k to truncate the sum; only top k answers contribute.
>>> ndcg_score(true_relevance, scores, k=4)
0.35...
>>> # the normalization takes k into account so a perfect answer
>>> # would still get 1.0
>>> ndcg_score(true_relevance, true_relevance, k=4)
1.0
>>> # now we have some ties in our prediction
>>> scores = np.asarray([[1, 0, 0, 0, 1]])
>>> # by default ties are averaged, so here we get the average (normalized)
>>> # true relevance of our top predictions: (10 / 10 + 5 / 10) / 2 = .75
>>> ndcg_score(true_relevance, scores, k=1)
0.75
>>> # we can choose to ignore ties for faster results, but only
>>> # if we know there aren't ties in our scores, otherwise we get
>>> # wrong results:
>>> ndcg_score(true_relevance,
...           scores, k=1, ignore_ties=True)
0.5