neulab/docprompting-conala huggingface dataset:
partial_fit(X, y=None, sample_weight=None) [source]
 
Online computation of mean and std on X for later scaling. All of X is processed as a single batch. This is intended for cases when fit is not feasible due to very large number of n_samples or because X is read from a continuous stream. The algorithm for incremental mean and std is given in Equation 1.5a,b in Chan, Tony F., Gene H. Golub, and Randall J. LeVeque. “Algorithms for computing the sample variance: Analysis and recommendations.” The American Statistician 37.3 (1983): 242-247:  Parameters 
 
X{array-like, sparse matrix} of shape (n_samples, n_features) 

The data used to compute the mean and standard deviation used for later scaling along the features axis.  
yNone 

Ignored.  
sample_weightarray-like of shape (n_samples,), default=None 

Individual weights for each sample.  New in version 0.24: parameter sample_weight support to StandardScaler.     Returns 
 
selfobject 

Fitted scaler.
transform(X, copy=None) [source]
 
Perform standardization by centering and scaling  Parameters 
 
X{array-like, sparse matrix of shape (n_samples, n_features) 

The data used to scale along the features axis.  
copybool, default=None 

Copy the input X or not.    Returns 
 
X_tr{ndarray, sparse matrix} of shape (n_samples, n_features) 

Transformed array.
fit(X, y=None) [source]
 
Generate a sparse random projection matrix.  Parameters 
 
X{ndarray, sparse matrix} of shape (n_samples, n_features) 

Training set: only the shape is used to find optimal random matrix dimensions based on the theory referenced in the afore mentioned papers.  y

Ignored    Returns 
 self
transform(X) [source]
 
Project the data by using matrix product with the random matrix  Parameters 
 
X{ndarray, sparse matrix} of shape (n_samples, n_features) 

The input data to project into a smaller dimensional space.    Returns 
 
X_new{ndarray, sparse matrix} of shape (n_samples, n_components) 

Projected array.
sklearn.random_projection.johnson_lindenstrauss_min_dim(n_samples, *, eps=0.1) [source]
 
Find a ‘safe’ number of components to randomly project to. The distortion introduced by a random projection p only changes the distance between two points by a factor (1 +- eps) in an euclidean space with good probability. The projection p is an eps-embedding as defined by: (1 - eps) ||u - v||^2 < ||p(u) - p(v)||^2 < (1 + eps) ||u - v||^2 Where u and v are any rows taken from a dataset of shape (n_samples, n_features), eps is in ]0, 1[ and p is a projection by a random Gaussian N(0, 1) matrix of shape (n_components, n_features) (or a sparse Achlioptas matrix). The minimum number of components to guarantee the eps-embedding is given by: n_components >= 4 log(n_samples) / (eps^2 / 2 - eps^3 / 3) Note that the number of dimensions is independent of the original number of features but instead depends on the size of the dataset: the larger the dataset, the higher is the minimal dimensionality of an eps-embedding. Read more in the User Guide.  Parameters 
 
n_samplesint or array-like of int 

Number of samples that should be a integer greater than 0. If an array is given, it will compute a safe number of components array-wise.  
epsfloat or ndarray of shape (n_components,), dtype=float, default=0.1 

Maximum distortion rate in the range (0,1 ) as defined by the Johnson-Lindenstrauss lemma. If an array is given, it will compute a safe number of components array-wise.    Returns 
 
n_componentsint or ndarray of int 

The minimal number of components to guarantee with good probability an eps-embedding with n_samples.     References  
1  
https://en.wikipedia.org/wiki/Johnson%E2%80%93Lindenstrauss_lemma  
2  
Sanjoy Dasgupta and Anupam Gupta, 1999, “An elementary proof of the Johnson-Lindenstrauss Lemma.” http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.45.3654   Examples >>> johnson_lindenstrauss_min_dim(1e6, eps=0.5)
663
 >>> johnson_lindenstrauss_min_dim(1e6, eps=[0.5, 0.1, 0.01])
array([    663,   11841, 1112658])
 >>> johnson_lindenstrauss_min_dim([1e4, 1e5, 1e6], eps=0.1)
array([ 7894,  9868, 11841])
fit(X, y) [source]
 
Fit a semi-supervised label propagation model based All the input data is provided matrix X (labeled and unlabeled) and corresponding label matrix y with a dedicated marker value for unlabeled samples.  Parameters 
 
Xarray-like of shape (n_samples, n_features) 

A matrix of shape (n_samples, n_samples) will be created from this.  
yarray-like of shape (n_samples,) 

n_labeled_samples (unlabeled points are marked as -1) All unlabeled samples will be transductively assigned labels.    Returns 
 
selfobject
predict(X) [source]
 
Performs inductive inference across the model.  Parameters 
 
Xarray-like of shape (n_samples, n_features) 

The data matrix.    Returns 
 
yndarray of shape (n_samples,) 

Predictions for input data.
predict_proba(X) [source]
 
Predict probability for each possible outcome. Compute the probability estimates for each single sample in X and each possible outcome seen during training (categorical distribution).  Parameters 
 
Xarray-like of shape (n_samples, n_features) 

The data matrix.    Returns 
 
probabilitiesndarray of shape (n_samples, n_classes) 

Normalized probability distributions across class labels.
decision_function(X) [source]
 
Calls decision function of the base_estimator.  Parameters 
 
X{array-like, sparse matrix} of shape (n_samples, n_features) 

Array representing the data.    Returns 
 
yndarray of shape (n_samples, n_features) 

Result of the decision function of the base_estimator.
fit(X, y) [source]
 
Fits this SelfTrainingClassifier to a dataset.  Parameters 
 
X{array-like, sparse matrix} of shape (n_samples, n_features) 

Array representing the data.  
y{array-like, sparse matrix} of shape (n_samples,) 

Array representing the labels. Unlabeled samples should have the label -1.    Returns 
 
selfobject 

Returns an instance of self.
predict(X) [source]
 
Predict the classes of X.  Parameters 
 
X{array-like, sparse matrix} of shape (n_samples, n_features) 

Array representing the data.    Returns 
 
yndarray of shape (n_samples,) 

Array with predicted labels.
predict_log_proba(X) [source]
 
Predict log probability for each possible outcome.  Parameters 
 
X{array-like, sparse matrix} of shape (n_samples, n_features) 

Array representing the data.    Returns 
 
yndarray of shape (n_samples, n_features) 

Array with log prediction probabilities.
predict_proba(X) [source]
 
Predict probability for each possible outcome.  Parameters 
 
X{array-like, sparse matrix} of shape (n_samples, n_features) 

Array representing the data.    Returns 
 
yndarray of shape (n_samples, n_features) 

Array with prediction probabilities.
score(X, y) [source]
 
Calls score on the base_estimator.  Parameters 
 
X{array-like, sparse matrix} of shape (n_samples, n_features) 

Array representing the data.  
yarray-like of shape (n_samples,) 

Array representing the labels.    Returns 
 
scorefloat 

Result of calling score on the base_estimator.
sklearn.set_config(assume_finite=None, working_memory=None, print_changed_only=None, display=None) [source]
 
Set global scikit-learn configuration  New in version 0.19.   Parameters 
 
assume_finitebool, default=None 

If True, validation for finiteness will be skipped, saving time, but leading to potential crashes. If False, validation for finiteness will be performed, avoiding error. Global default: False.  New in version 0.19.   
working_memoryint, default=None 

If set, scikit-learn will attempt to limit the size of temporary arrays to this number of MiB (per job when parallelised), often saving both computation time and memory on expensive operations that can be performed in chunks. Global default: 1024.  New in version 0.20.   
print_changed_onlybool, default=None 

If True, only the parameters that were set to non-default values will be printed when printing an estimator. For example, print(SVC()) while True will only print ‘SVC()’ while the default behaviour would be to print ‘SVC(C=1.0, cache_size=200, …)’ with all the non-changed parameters.  New in version 0.21.   
display{‘text’, ‘diagram’}, default=None 

If ‘diagram’, estimators will be displayed as a diagram in a Jupyter lab or notebook context. If ‘text’, estimators will be displayed as text. Default is ‘text’.  New in version 0.23.       See also  
config_context


Context manager for global scikit-learn configuration.  
get_config


Retrieve current values of the global configuration.
sklearn.show_versions() [source]
 
Print useful debugging information”  New in version 0.20.
sklearn.base.clone  
sklearn.base.clone(estimator, *, safe=True) [source]
 
Constructs a new unfitted estimator with the same parameters. Clone does a deep copy of the model in an estimator without actually copying attached data. It yields a new estimator with the same parameters that has not been fitted on any data. If the estimator’s random_state parameter is an integer (or if the estimator doesn’t have a random_state parameter), an exact clone is returned: the clone and the original estimator will give the exact same results. Otherwise, statistical clone is returned: the clone might yield different results from the original estimator. More details can be found in Controlling randomness.  Parameters 
 
estimator{list, tuple, set} of estimator instance or a single estimator instance 

The estimator or group of estimators to be cloned.  
safebool, default=True 

If safe is False, clone will fall back to a deep copy on objects that are not estimators.
sklearn.base.is_classifier  
sklearn.base.is_classifier(estimator) [source]
 
Return True if the given estimator is (probably) a classifier.  Parameters 
 
estimatorobject 

Estimator object to test.    Returns 
 
outbool 

True if estimator is a classifier and False otherwise.
sklearn.base.is_regressor  
sklearn.base.is_regressor(estimator) [source]
 
Return True if the given estimator is (probably) a regressor.  Parameters 
 
estimatorestimator instance 

Estimator object to test.    Returns 
 
outbool 

True if estimator is a regressor and False otherwise.
sklearn.calibration.calibration_curve  
sklearn.calibration.calibration_curve(y_true, y_prob, *, normalize=False, n_bins=5, strategy='uniform') [source]
 
Compute true and predicted probabilities for a calibration curve. The method assumes the inputs come from a binary classifier, and discretize the [0, 1] interval into bins. Calibration curves may also be referred to as reliability diagrams. Read more in the User Guide.  Parameters 
 
y_truearray-like of shape (n_samples,) 

True targets.  
y_probarray-like of shape (n_samples,) 

Probabilities of the positive class.  
normalizebool, default=False 

Whether y_prob needs to be normalized into the [0, 1] interval, i.e. is not a proper probability. If True, the smallest value in y_prob is linearly mapped onto 0 and the largest one onto 1.  
n_binsint, default=5 

Number of bins to discretize the [0, 1] interval. A bigger number requires more data. Bins with no samples (i.e. without corresponding values in y_prob) will not be returned, thus the returned arrays may have less than n_bins values.  
strategy{‘uniform’, ‘quantile’}, default=’uniform’ 

Strategy used to define the widths of the bins.  uniform

The bins have identical widths.  quantile

The bins have the same number of samples and depend on y_prob.      Returns 
 
prob_truendarray of shape (n_bins,) or smaller 

The proportion of samples whose class is the positive class, in each bin (fraction of positives).  
prob_predndarray of shape (n_bins,) or smaller 

The mean predicted probability in each bin.     References Alexandru Niculescu-Mizil and Rich Caruana (2005) Predicting Good Probabilities With Supervised Learning, in Proceedings of the 22nd International Conference on Machine Learning (ICML). See section 4 (Qualitative Analysis of Predictions). Examples >>> import numpy as np
>>> from sklearn.calibration import calibration_curve
>>> y_true = np.array([0, 0, 0, 0, 1, 1, 1, 1, 1])
>>> y_pred = np.array([0.1, 0.2, 0.3, 0.4, 0.65, 0.7, 0.8, 0.9,  1.])
>>> prob_true, prob_pred = calibration_curve(y_true, y_pred, n_bins=3)
>>> prob_true
array([0. , 0.5, 1. ])
>>> prob_pred
array([0.2  , 0.525, 0.85 ])
 
 Examples using sklearn.calibration.calibration_curve
 
  Comparison of Calibration of Classifiers  

  Probability Calibration curves
sklearn.cluster.affinity_propagation  
sklearn.cluster.affinity_propagation(S, *, preference=None, convergence_iter=15, max_iter=200, damping=0.5, copy=True, verbose=False, return_n_iter=False, random_state='warn') [source]
 
Perform Affinity Propagation Clustering of data. Read more in the User Guide.  Parameters 
 
Sarray-like of shape (n_samples, n_samples) 

Matrix of similarities between points.  
preferencearray-like of shape (n_samples,) or float, default=None 

Preferences for each point - points with larger values of preferences are more likely to be chosen as exemplars. The number of exemplars, i.e. of clusters, is influenced by the input preferences value. If the preferences are not passed as arguments, they will be set to the median of the input similarities (resulting in a moderate number of clusters). For a smaller amount of clusters, this can be set to the minimum value of the similarities.  
convergence_iterint, default=15 

Number of iterations with no change in the number of estimated clusters that stops the convergence.  
max_iterint, default=200 

Maximum number of iterations  
dampingfloat, default=0.5 

Damping factor between 0.5 and 1.  
copybool, default=True 

If copy is False, the affinity matrix is modified inplace by the algorithm, for memory efficiency.  
verbosebool, default=False 

The verbosity level.  
return_n_iterbool, default=False 

Whether or not to return the number of iterations.  
random_stateint, RandomState instance or None, default=0 

Pseudo-random number generator to control the starting state. Use an int for reproducible results across function calls. See the Glossary.  New in version 0.23: this parameter was previously hardcoded as 0.     Returns 
 
cluster_centers_indicesndarray of shape (n_clusters,) 

Index of clusters centers.  
labelsndarray of shape (n_samples,) 

Cluster labels for each point.  
n_iterint 

Number of iterations run. Returned only if return_n_iter is set to True.     Notes For an example, see examples/cluster/plot_affinity_propagation.py. When the algorithm does not converge, it returns an empty array as cluster_center_indices and -1 as label for each training sample. When all training samples have equal similarities and equal preferences, the assignment of cluster centers and labels depends on the preference. If the preference is smaller than the similarities, a single cluster center and label 0 for every sample will be returned. Otherwise, every training sample becomes its own cluster center and is assigned a unique label. References Brendan J. Frey and Delbert Dueck, “Clustering by Passing Messages Between Data Points”, Science Feb. 2007 
 Examples using sklearn.cluster.affinity_propagation
 
  Visualizing the stock market structure
sklearn.cluster.cluster_optics_dbscan  
sklearn.cluster.cluster_optics_dbscan(*, reachability, core_distances, ordering, eps) [source]
 
Performs DBSCAN extraction for an arbitrary epsilon. Extracting the clusters runs in linear time. Note that this results in labels_ which are close to a DBSCAN with similar settings and eps, only if eps is close to max_eps.  Parameters 
 
reachabilityarray of shape (n_samples,) 

Reachability distances calculated by OPTICS (reachability_)  
core_distancesarray of shape (n_samples,) 

Distances at which points become core (core_distances_)  
orderingarray of shape (n_samples,) 

OPTICS ordered point indices (ordering_)  
epsfloat 

DBSCAN eps parameter. Must be set to < max_eps. Results will be close to DBSCAN algorithm if eps and max_eps are close to one another.    Returns 
 
labels_array of shape (n_samples,) 

The estimated labels.     
 Examples using sklearn.cluster.cluster_optics_dbscan
 
  Demo of OPTICS clustering algorithm
sklearn.cluster.cluster_optics_xi  
sklearn.cluster.cluster_optics_xi(*, reachability, predecessor, ordering, min_samples, min_cluster_size=None, xi=0.05, predecessor_correction=True) [source]
 
Automatically extract clusters according to the Xi-steep method.  Parameters 
 
reachabilityndarray of shape (n_samples,) 

Reachability distances calculated by OPTICS (reachability_)  
predecessorndarray of shape (n_samples,) 

Predecessors calculated by OPTICS.  
orderingndarray of shape (n_samples,) 

OPTICS ordered point indices (ordering_)  
min_samplesint > 1 or float between 0 and 1 

The same as the min_samples given to OPTICS. Up and down steep regions can’t have more then min_samples consecutive non-steep points. Expressed as an absolute number or a fraction of the number of samples (rounded to be at least 2).  
min_cluster_sizeint > 1 or float between 0 and 1, default=None 

Minimum number of samples in an OPTICS cluster, expressed as an absolute number or a fraction of the number of samples (rounded to be at least 2). If None, the value of min_samples is used instead.  
xifloat between 0 and 1, default=0.05 

Determines the minimum steepness on the reachability plot that constitutes a cluster boundary. For example, an upwards point in the reachability plot is defined by the ratio from one point to its successor being at most 1-xi.  
predecessor_correctionbool, default=True 

Correct clusters based on the calculated predecessors.    Returns 
 
labelsndarray of shape (n_samples,) 

The labels assigned to samples. Points which are not included in any cluster are labeled as -1.  
clustersndarray of shape (n_clusters, 2) 

The list of clusters in the form of [start, end] in each row, with all indices inclusive. The clusters are ordered according to (end,
-start) (ascending) so that larger clusters encompassing smaller clusters come after such nested smaller clusters. Since labels does not reflect the hierarchy, usually len(clusters) >
np.unique(labels).
sklearn.cluster.estimate_bandwidth  
sklearn.cluster.estimate_bandwidth(X, *, quantile=0.3, n_samples=None, random_state=0, n_jobs=None) [source]
 
Estimate the bandwidth to use with the mean-shift algorithm. That this function takes time at least quadratic in n_samples. For large datasets, it’s wise to set that parameter to a small value.  Parameters 
 
Xarray-like of shape (n_samples, n_features) 

Input points.  
quantilefloat, default=0.3 

should be between [0, 1] 0.5 means that the median of all pairwise distances is used.  
n_samplesint, default=None 

The number of samples to use. If not given, all samples are used.  
random_stateint, RandomState instance, default=None 

The generator used to randomly select the samples from input points for bandwidth estimation. Use an int to make the randomness deterministic. See Glossary.  
n_jobsint, default=None 

The number of parallel jobs to run for neighbors search. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.    Returns 
 
bandwidthfloat 

The bandwidth parameter.     
 Examples using sklearn.cluster.estimate_bandwidth
 
  A demo of the mean-shift clustering algorithm  

  Comparing different clustering algorithms on toy datasets
sklearn.cluster.kmeans_plusplus  
sklearn.cluster.kmeans_plusplus(X, n_clusters, *, x_squared_norms=None, random_state=None, n_local_trials=None) [source]
 
Init n_clusters seeds according to k-means++  New in version 0.24.   Parameters 
 
X{array-like, sparse matrix} of shape (n_samples, n_features) 

The data to pick seeds from.  
n_clustersint 

The number of centroids to initialize  
x_squared_normsarray-like of shape (n_samples,), default=None 

Squared Euclidean norm of each data point.  
random_stateint or RandomState instance, default=None 

Determines random number generation for centroid initialization. Pass an int for reproducible output across multiple function calls. See Glossary.  
n_local_trialsint, default=None 

The number of seeding trials for each center (except the first), of which the one reducing inertia the most is greedily chosen. Set to None to make the number of trials depend logarithmically on the number of seeds (2+log(k)).    Returns 
 
centersndarray of shape (n_clusters, n_features) 

The inital centers for k-means.  
indicesndarray of shape (n_clusters,) 

The index location of the chosen centers in the data array X. For a given index and center, X[index] = center.     Notes Selects initial cluster centers for k-mean clustering in a smart way to speed up convergence. see: Arthur, D. and Vassilvitskii, S. “k-means++: the advantages of careful seeding”. ACM-SIAM symposium on Discrete algorithms. 2007 Examples >>> from sklearn.cluster import kmeans_plusplus
>>> import numpy as np
>>> X = np.array([[1, 2], [1, 4], [1, 0],
...               [10, 2], [10, 4], [10, 0]])
>>> centers, indices = kmeans_plusplus(X, n_clusters=2, random_state=0)
>>> centers
array([[10,  4],
       [ 1,  0]])
>>> indices
array([4, 2])
 
 Examples using sklearn.cluster.kmeans_plusplus
 
  An example of K-Means++ initialization
sklearn.cluster.mean_shift  
sklearn.cluster.mean_shift(X, *, bandwidth=None, seeds=None, bin_seeding=False, min_bin_freq=1, cluster_all=True, max_iter=300, n_jobs=None) [source]
 
Perform mean shift clustering of data using a flat kernel. Read more in the User Guide.  Parameters 
 
Xarray-like of shape (n_samples, n_features) 

Input data.  
bandwidthfloat, default=None 

Kernel bandwidth. If bandwidth is not given, it is determined using a heuristic based on the median of all pairwise distances. This will take quadratic time in the number of samples. The sklearn.cluster.estimate_bandwidth function can be used to do this more efficiently.  
seedsarray-like of shape (n_seeds, n_features) or None 

Point used as initial kernel locations. If None and bin_seeding=False, each data point is used as a seed. If None and bin_seeding=True, see bin_seeding.  
bin_seedingbool, default=False 

If true, initial kernel locations are not locations of all points, but rather the location of the discretized version of points, where points are binned onto a grid whose coarseness corresponds to the bandwidth. Setting this option to True will speed up the algorithm because fewer seeds will be initialized. Ignored if seeds argument is not None.  
min_bin_freqint, default=1 

To speed up the algorithm, accept only those bins with at least min_bin_freq points as seeds.  
cluster_allbool, default=True 

If true, then all points are clustered, even those orphans that are not within any kernel. Orphans are assigned to the nearest kernel. If false, then orphans are given cluster label -1.  
max_iterint, default=300 

Maximum number of iterations, per seed point before the clustering operation terminates (for that seed point), if has not converged yet.  
n_jobsint, default=None 

The number of jobs to use for the computation. This works by computing each of the n_init runs in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.  New in version 0.17: Parallel Execution using n_jobs.     Returns 
 
cluster_centersndarray of shape (n_clusters, n_features) 

Coordinates of cluster centers.  
labelsndarray of shape (n_samples,) 

Cluster labels for each point.     Notes For an example, see examples/cluster/plot_mean_shift.py.