neulab/docprompting-conala huggingface dataset:
sklearn.metrics.plot_det_curve  
sklearn.metrics.plot_det_curve(estimator, X, y, *, sample_weight=None, response_method='auto', name=None, ax=None, pos_label=None, **kwargs) [source]
 
Plot detection error tradeoff (DET) curve. Extra keyword arguments will be passed to matplotlib’s plot. Read more in the User Guide.  New in version 0.24.   Parameters 
 
estimatorestimator instance 

Fitted classifier or a fitted Pipeline in which the last estimator is a classifier.  
X{array-like, sparse matrix} of shape (n_samples, n_features) 

Input values.  
yarray-like of shape (n_samples,) 

Target values.  
sample_weightarray-like of shape (n_samples,), default=None 

Sample weights.  
response_method{‘predict_proba’, ‘decision_function’, ‘auto’} default=’auto’ 

Specifies whether to use predict_proba or decision_function as the predicted target response. If set to ‘auto’, predict_proba is tried first and if it does not exist decision_function is tried next.  
namestr, default=None 

Name of DET curve for labeling. If None, use the name of the estimator.  
axmatplotlib axes, default=None 

Axes object to plot on. If None, a new figure and axes is created.  
pos_labelstr or int, default=None 

The label of the positive class. When pos_label=None, if y_true is in {-1, 1} or {0, 1}, pos_label is set to 1, otherwise an error will be raised.    Returns 
 
displayDetCurveDisplay 

Object that stores computed values.      See also  
det_curve


Compute error rates for different probability thresholds.  
DetCurveDisplay


DET curve visualization.  
plot_roc_curve


Plot Receiver operating characteristic (ROC) curve.    Examples >>> import matplotlib.pyplot as plt  
>>> from sklearn import datasets, metrics, model_selection, svm
>>> X, y = datasets.make_classification(random_state=0)
>>> X_train, X_test, y_train, y_test = model_selection.train_test_split(
...     X, y, random_state=0)
>>> clf = svm.SVC(random_state=0)
>>> clf.fit(X_train, y_train)
SVC(random_state=0)
>>> metrics.plot_det_curve(clf, X_test, y_test)  
>>> plt.show()                                   
 
 Examples using sklearn.metrics.plot_det_curve
 
  Detection error tradeoff (DET) curve
sklearn.metrics.plot_precision_recall_curve  
sklearn.metrics.plot_precision_recall_curve(estimator, X, y, *, sample_weight=None, response_method='auto', name=None, ax=None, pos_label=None, **kwargs) [source]
 
Plot Precision Recall Curve for binary classifiers. Extra keyword arguments will be passed to matplotlib’s plot. Read more in the User Guide.  Parameters 
 
estimatorestimator instance 

Fitted classifier or a fitted Pipeline in which the last estimator is a classifier.  
X{array-like, sparse matrix} of shape (n_samples, n_features) 

Input values.  
yarray-like of shape (n_samples,) 

Binary target values.  
sample_weightarray-like of shape (n_samples,), default=None 

Sample weights.  
response_method{‘predict_proba’, ‘decision_function’, ‘auto’}, default=’auto’ 

Specifies whether to use predict_proba or decision_function as the target response. If set to ‘auto’, predict_proba is tried first and if it does not exist decision_function is tried next.  
namestr, default=None 

Name for labeling curve. If None, the name of the estimator is used.  
axmatplotlib axes, default=None 

Axes object to plot on. If None, a new figure and axes is created.  
pos_labelstr or int, default=None 

The class considered as the positive class when computing the precision and recall metrics. By default, estimators.classes_[1] is considered as the positive class.  New in version 0.24.   
**kwargsdict 

Keyword arguments to be passed to matplotlib’s plot.    Returns 
 
displayPrecisionRecallDisplay 

Object that stores computed values.      See also  
precision_recall_curve


Compute precision-recall pairs for different probability thresholds.  
PrecisionRecallDisplay


Precision Recall visualization.    
 Examples using sklearn.metrics.plot_precision_recall_curve
 
  Precision-Recall
sklearn.metrics.plot_roc_curve  
sklearn.metrics.plot_roc_curve(estimator, X, y, *, sample_weight=None, drop_intermediate=True, response_method='auto', name=None, ax=None, pos_label=None, **kwargs) [source]
 
Plot Receiver operating characteristic (ROC) curve. Extra keyword arguments will be passed to matplotlib’s plot. Read more in the User Guide.  Parameters 
 
estimatorestimator instance 

Fitted classifier or a fitted Pipeline in which the last estimator is a classifier.  
X{array-like, sparse matrix} of shape (n_samples, n_features) 

Input values.  
yarray-like of shape (n_samples,) 

Target values.  
sample_weightarray-like of shape (n_samples,), default=None 

Sample weights.  
drop_intermediateboolean, default=True 

Whether to drop some suboptimal thresholds which would not appear on a plotted ROC curve. This is useful in order to create lighter ROC curves.  
response_method{‘predict_proba’, ‘decision_function’, ‘auto’} default=’auto’ 

Specifies whether to use predict_proba or decision_function as the target response. If set to ‘auto’, predict_proba is tried first and if it does not exist decision_function is tried next.  
namestr, default=None 

Name of ROC Curve for labeling. If None, use the name of the estimator.  
axmatplotlib axes, default=None 

Axes object to plot on. If None, a new figure and axes is created.  
pos_labelstr or int, default=None 

The class considered as the positive class when computing the roc auc metrics. By default, estimators.classes_[1] is considered as the positive class.  New in version 0.24.     Returns 
 
displayRocCurveDisplay 

Object that stores computed values.      See also  
roc_curve


Compute Receiver operating characteristic (ROC) curve.  
RocCurveDisplay


ROC Curve visualization.  
roc_auc_score


Compute the area under the ROC curve.    Examples >>> import matplotlib.pyplot as plt  
>>> from sklearn import datasets, metrics, model_selection, svm
>>> X, y = datasets.make_classification(random_state=0)
>>> X_train, X_test, y_train, y_test = model_selection.train_test_split(
...     X, y, random_state=0)
>>> clf = svm.SVC(random_state=0)
>>> clf.fit(X_train, y_train)
SVC(random_state=0)
>>> metrics.plot_roc_curve(clf, X_test, y_test)  
>>> plt.show()                                   
 
 Examples using sklearn.metrics.plot_roc_curve
 
  Release Highlights for scikit-learn 0.22  

  ROC Curve with Visualization API  

  Detection error tradeoff (DET) curve  

  Receiver Operating Characteristic (ROC) with cross validation
sklearn.metrics.precision_recall_curve  
sklearn.metrics.precision_recall_curve(y_true, probas_pred, *, pos_label=None, sample_weight=None) [source]
 
Compute precision-recall pairs for different probability thresholds. Note: this implementation is restricted to the binary classification task. The precision is the ratio tp / (tp + fp) where tp is the number of true positives and fp the number of false positives. The precision is intuitively the ability of the classifier not to label as positive a sample that is negative. The recall is the ratio tp / (tp + fn) where tp is the number of true positives and fn the number of false negatives. The recall is intuitively the ability of the classifier to find all the positive samples. The last precision and recall values are 1. and 0. respectively and do not have a corresponding threshold. This ensures that the graph starts on the y axis. Read more in the User Guide.  Parameters 
 
y_truendarray of shape (n_samples,) 

True binary labels. If labels are not either {-1, 1} or {0, 1}, then pos_label should be explicitly given.  
probas_predndarray of shape (n_samples,) 

Estimated probabilities or output of a decision function.  
pos_labelint or str, default=None 

The label of the positive class. When pos_label=None, if y_true is in {-1, 1} or {0, 1}, pos_label is set to 1, otherwise an error will be raised.  
sample_weightarray-like of shape (n_samples,), default=None 

Sample weights.    Returns 
 
precisionndarray of shape (n_thresholds + 1,) 

Precision values such that element i is the precision of predictions with score >= thresholds[i] and the last element is 1.  
recallndarray of shape (n_thresholds + 1,) 

Decreasing recall values such that element i is the recall of predictions with score >= thresholds[i] and the last element is 0.  
thresholdsndarray of shape (n_thresholds,) 

Increasing thresholds on the decision function used to compute precision and recall. n_thresholds <= len(np.unique(probas_pred)).      See also  
plot_precision_recall_curve


Plot Precision Recall Curve for binary classifiers.  
PrecisionRecallDisplay


Precision Recall visualization.  
average_precision_score


Compute average precision from prediction scores.  
det_curve


Compute error rates for different probability thresholds.  
roc_curve


Compute Receiver operating characteristic (ROC) curve.    Examples >>> import numpy as np
>>> from sklearn.metrics import precision_recall_curve
>>> y_true = np.array([0, 0, 1, 1])
>>> y_scores = np.array([0.1, 0.4, 0.35, 0.8])
>>> precision, recall, thresholds = precision_recall_curve(
...     y_true, y_scores)
>>> precision
array([0.66666667, 0.5       , 1.        , 1.        ])
>>> recall
array([1. , 0.5, 0.5, 0. ])
>>> thresholds
array([0.35, 0.4 , 0.8 ])
 
 Examples using sklearn.metrics.precision_recall_curve
 
  Visualizations with Display Objects  

  Precision-Recall
sklearn.metrics.r2_score  
sklearn.metrics.r2_score(y_true, y_pred, *, sample_weight=None, multioutput='uniform_average') [source]
 
R^2 (coefficient of determination) regression score function. Best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a R^2 score of 0.0. Read more in the User Guide.  Parameters 
 
y_truearray-like of shape (n_samples,) or (n_samples, n_outputs) 

Ground truth (correct) target values.  
y_predarray-like of shape (n_samples,) or (n_samples, n_outputs) 

Estimated target values.  
sample_weightarray-like of shape (n_samples,), default=None 

Sample weights.  
multioutput{‘raw_values’, ‘uniform_average’, ‘variance_weighted’}, array-like of shape (n_outputs,) or None, default=’uniform_average’ 

Defines aggregating of multiple output scores. Array-like value defines weights used to average scores. Default is “uniform_average”.  ‘raw_values’ :

Returns a full set of scores in case of multioutput input.  ‘uniform_average’ :

Scores of all outputs are averaged with uniform weight.  ‘variance_weighted’ :

Scores of all outputs are averaged, weighted by the variances of each individual output.    Changed in version 0.19: Default value of multioutput is ‘uniform_average’.     Returns 
 
zfloat or ndarray of floats 

The R^2 score or ndarray of scores if ‘multioutput’ is ‘raw_values’.     Notes This is not a symmetric function. Unlike most other scores, R^2 score may be negative (it need not actually be the square of a quantity R). This metric is not well-defined for single samples and will return a NaN value if n_samples is less than two. References  
1  
Wikipedia entry on the Coefficient of determination   Examples >>> from sklearn.metrics import r2_score
>>> y_true = [3, -0.5, 2, 7]
>>> y_pred = [2.5, 0.0, 2, 8]
>>> r2_score(y_true, y_pred)
0.948...
>>> y_true = [[0.5, 1], [-1, 1], [7, -6]]
>>> y_pred = [[0, 2], [-1, 2], [8, -5]]
>>> r2_score(y_true, y_pred,
...          multioutput='variance_weighted')
0.938...
>>> y_true = [1, 2, 3]
>>> y_pred = [1, 2, 3]
>>> r2_score(y_true, y_pred)
1.0
>>> y_true = [1, 2, 3]
>>> y_pred = [2, 2, 2]
>>> r2_score(y_true, y_pred)
0.0
>>> y_true = [1, 2, 3]
>>> y_pred = [3, 2, 1]
>>> r2_score(y_true, y_pred)
-3.0
 
 Examples using sklearn.metrics.r2_score
 
  Non-negative least squares  

  Linear Regression Example  

  Lasso and Elastic Net for Sparse Signals  

  Effect of transforming the targets in regression model
sklearn.metrics.rand_score  
sklearn.metrics.rand_score(labels_true, labels_pred) [source]
 
Rand index. The Rand Index computes a similarity measure between two clusterings by considering all pairs of samples and counting pairs that are assigned in the same or different clusters in the predicted and true clusterings. The raw RI score is: RI = (number of agreeing pairs) / (number of pairs) Read more in the User Guide.  Parameters 
 
labels_truearray-like of shape (n_samples,), dtype=integral 

Ground truth class labels to be used as a reference.  
labels_predarray-like of shape (n_samples,), dtype=integral 

Cluster labels to evaluate.    Returns 
 
RIfloat 

Similarity score between 0.0 and 1.0, inclusive, 1.0 stands for perfect match.      See also  
adjusted_rand_score


Adjusted Rand Score  
adjusted_mutual_info_score


Adjusted Mutual Information    References Examples Perfectly matching labelings have a score of 1 even >>> from sklearn.metrics.cluster import rand_score
>>> rand_score([0, 0, 1, 1], [1, 1, 0, 0])
1.0
 Labelings that assign all classes members to the same clusters are complete but may not always be pure, hence penalized: >>> rand_score([0, 0, 1, 2], [0, 0, 1, 1])
0.83...
sklearn.metrics.roc_curve  
sklearn.metrics.roc_curve(y_true, y_score, *, pos_label=None, sample_weight=None, drop_intermediate=True) [source]
 
Compute Receiver operating characteristic (ROC). Note: this implementation is restricted to the binary classification task. Read more in the User Guide.  Parameters 
 
y_truendarray of shape (n_samples,) 

True binary labels. If labels are not either {-1, 1} or {0, 1}, then pos_label should be explicitly given.  
y_scorendarray of shape (n_samples,) 

Target scores, can either be probability estimates of the positive class, confidence values, or non-thresholded measure of decisions (as returned by “decision_function” on some classifiers).  
pos_labelint or str, default=None 

The label of the positive class. When pos_label=None, if y_true is in {-1, 1} or {0, 1}, pos_label is set to 1, otherwise an error will be raised.  
sample_weightarray-like of shape (n_samples,), default=None 

Sample weights.  
drop_intermediatebool, default=True 

Whether to drop some suboptimal thresholds which would not appear on a plotted ROC curve. This is useful in order to create lighter ROC curves.  New in version 0.17: parameter drop_intermediate.     Returns 
 
fprndarray of shape (>2,) 

Increasing false positive rates such that element i is the false positive rate of predictions with score >= thresholds[i].  
tprndarray of shape (>2,) 

Increasing true positive rates such that element i is the true positive rate of predictions with score >= thresholds[i].  
thresholdsndarray of shape = (n_thresholds,) 

Decreasing thresholds on the decision function used to compute fpr and tpr. thresholds[0] represents no instances being predicted and is arbitrarily set to max(y_score) + 1.      See also  
plot_roc_curve


Plot Receiver operating characteristic (ROC) curve.  
RocCurveDisplay


ROC Curve visualization.  
det_curve


Compute error rates for different probability thresholds.  
roc_auc_score


Compute the area under the ROC curve.    Notes Since the thresholds are sorted from low to high values, they are reversed upon returning them to ensure they correspond to both fpr and tpr, which are sorted in reversed order during their calculation. References  
1  
Wikipedia entry for the Receiver operating characteristic  
2  
Fawcett T. An introduction to ROC analysis[J]. Pattern Recognition Letters, 2006, 27(8):861-874.   Examples >>> import numpy as np
>>> from sklearn import metrics
>>> y = np.array([1, 1, 2, 2])
>>> scores = np.array([0.1, 0.4, 0.35, 0.8])
>>> fpr, tpr, thresholds = metrics.roc_curve(y, scores, pos_label=2)
>>> fpr
array([0. , 0. , 0.5, 0.5, 1. ])
>>> tpr
array([0. , 0.5, 0.5, 1. , 1. ])
>>> thresholds
array([1.8 , 0.8 , 0.4 , 0.35, 0.1 ])
 
 Examples using sklearn.metrics.roc_curve
 
  Feature transformations with ensembles of trees  

  Species distribution modeling  

  Visualizations with Display Objects  

  Detection error tradeoff (DET) curve  

  Receiver Operating Characteristic (ROC)
sklearn.metrics.silhouette_samples  
sklearn.metrics.silhouette_samples(X, labels, *, metric='euclidean', **kwds) [source]
 
Compute the Silhouette Coefficient for each sample. The Silhouette Coefficient is a measure of how well samples are clustered with samples that are similar to themselves. Clustering models with a high Silhouette Coefficient are said to be dense, where samples in the same cluster are similar to each other, and well separated, where samples in different clusters are not very similar to each other. The Silhouette Coefficient is calculated using the mean intra-cluster distance (a) and the mean nearest-cluster distance (b) for each sample. The Silhouette Coefficient for a sample is (b - a) / max(a,
b). Note that Silhouette Coefficient is only defined if number of labels is 2 <= n_labels <= n_samples - 1. This function returns the Silhouette Coefficient for each sample. The best value is 1 and the worst value is -1. Values near 0 indicate overlapping clusters. Read more in the User Guide.  Parameters 
 
Xarray-like of shape (n_samples_a, n_samples_a) if metric == “precomputed” or (n_samples_a, n_features) otherwise 

An array of pairwise distances between samples, or a feature array.  
labelsarray-like of shape (n_samples,) 

Label values for each sample.  
metricstr or callable, default=’euclidean’ 

The metric to use when calculating distance between instances in a feature array. If metric is a string, it must be one of the options allowed by sklearn.metrics.pairwise.pairwise_distances. If X is the distance array itself, use “precomputed” as the metric. Precomputed distance matrices must have 0 along the diagonal.  
`**kwds`optional keyword parameters 

Any further parameters are passed directly to the distance function. If using a scipy.spatial.distance metric, the parameters are still metric dependent. See the scipy docs for usage examples.    Returns 
 
silhouettearray-like of shape (n_samples,) 

Silhouette Coefficients for each sample.     References  
1  
Peter J. Rousseeuw (1987). “Silhouettes: a Graphical Aid to the Interpretation and Validation of Cluster Analysis”. Computational and Applied Mathematics 20: 53-65.  
2  
Wikipedia entry on the Silhouette Coefficient   
 Examples using sklearn.metrics.silhouette_samples
 
  Selecting the number of clusters with silhouette analysis on KMeans clustering
sklearn.metrics.silhouette_score  
sklearn.metrics.silhouette_score(X, labels, *, metric='euclidean', sample_size=None, random_state=None, **kwds) [source]
 
Compute the mean Silhouette Coefficient of all samples. The Silhouette Coefficient is calculated using the mean intra-cluster distance (a) and the mean nearest-cluster distance (b) for each sample. The Silhouette Coefficient for a sample is (b - a) / max(a,
b). To clarify, b is the distance between a sample and the nearest cluster that the sample is not a part of. Note that Silhouette Coefficient is only defined if number of labels is 2 <= n_labels <= n_samples - 1. This function returns the mean Silhouette Coefficient over all samples. To obtain the values for each sample, use silhouette_samples. The best value is 1 and the worst value is -1. Values near 0 indicate overlapping clusters. Negative values generally indicate that a sample has been assigned to the wrong cluster, as a different cluster is more similar. Read more in the User Guide.  Parameters 
 
Xarray-like of shape (n_samples_a, n_samples_a) if metric == “precomputed” or (n_samples_a, n_features) otherwise 

An array of pairwise distances between samples, or a feature array.  
labelsarray-like of shape (n_samples,) 

Predicted labels for each sample.  
metricstr or callable, default=’euclidean’ 

The metric to use when calculating distance between instances in a feature array. If metric is a string, it must be one of the options allowed by metrics.pairwise.pairwise_distances. If X is the distance array itself, use metric="precomputed".  
sample_sizeint, default=None 

The size of the sample to use when computing the Silhouette Coefficient on a random subset of the data. If sample_size is None, no sampling is used.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for selecting a subset of samples. Used when sample_size is not None. Pass an int for reproducible results across multiple function calls. See Glossary.  
**kwdsoptional keyword parameters 

Any further parameters are passed directly to the distance function. If using a scipy.spatial.distance metric, the parameters are still metric dependent. See the scipy docs for usage examples.    Returns 
 
silhouettefloat 

Mean Silhouette Coefficient for all samples.     References  
1  
Peter J. Rousseeuw (1987). “Silhouettes: a Graphical Aid to the Interpretation and Validation of Cluster Analysis”. Computational and Applied Mathematics 20: 53-65.  
2  
Wikipedia entry on the Silhouette Coefficient   
 Examples using sklearn.metrics.silhouette_score
 
  Demo of affinity propagation clustering algorithm  

  Demo of DBSCAN clustering algorithm  

  A demo of K-Means clustering on the handwritten digits data  

  Selecting the number of clusters with silhouette analysis on KMeans clustering  

  Clustering text documents using k-means
sklearn.metrics.top_k_accuracy_score  
sklearn.metrics.top_k_accuracy_score(y_true, y_score, *, k=2, normalize=True, sample_weight=None, labels=None) [source]
 
Top-k Accuracy classification score. This metric computes the number of times where the correct label is among the top k labels predicted (ranked by predicted scores). Note that the multilabel case isn’t covered here. Read more in the User Guide  Parameters 
 
y_truearray-like of shape (n_samples,) 

True labels.  
y_scorearray-like of shape (n_samples,) or (n_samples, n_classes) 

Target scores. These can be either probability estimates or non-thresholded decision values (as returned by decision_function on some classifiers). The binary case expects scores with shape (n_samples,) while the multiclass case expects scores with shape (n_samples, n_classes). In the nulticlass case, the order of the class scores must correspond to the order of labels, if provided, or else to the numerical or lexicographical order of the labels in y_true.  
kint, default=2 

Number of most likely outcomes considered to find the correct label.  
normalizebool, default=True 

If True, return the fraction of correctly classified samples. Otherwise, return the number of correctly classified samples.  
sample_weightarray-like of shape (n_samples,), default=None 

Sample weights. If None, all samples are given the same weight.  
labelsarray-like of shape (n_classes,), default=None 

Multiclass only. List of labels that index the classes in y_score. If None, the numerical or lexicographical order of the labels in y_true is used.    Returns 
 
scorefloat 

The top-k accuracy score. The best performance is 1 with normalize == True and the number of samples with normalize == False.      See also  
accuracy_score

  Notes In cases where two or more labels are assigned equal predicted scores, the labels with the highest indices will be chosen first. This might impact the result if the correct label falls after the threshold because of that. Examples >>> import numpy as np
>>> from sklearn.metrics import top_k_accuracy_score
>>> y_true = np.array([0, 1, 2, 2])
>>> y_score = np.array([[0.5, 0.2, 0.2],  # 0 is in top 2
...                     [0.3, 0.4, 0.2],  # 1 is in top 2
...                     [0.2, 0.4, 0.3],  # 2 is in top 2
...                     [0.7, 0.2, 0.1]]) # 2 isn't in top 2
>>> top_k_accuracy_score(y_true, y_score, k=2)
0.75
>>> # Not normalizing gives the number of "correctly" classified samples
>>> top_k_accuracy_score(y_true, y_score, k=2, normalize=False)
3