neulab/docprompting-conala huggingface dataset:
sklearn.cluster.ward_tree  
sklearn.cluster.ward_tree(X, *, connectivity=None, n_clusters=None, return_distance=False) [source]
 
Ward clustering based on a Feature matrix. Recursively merges the pair of clusters that minimally increases within-cluster variance. The inertia matrix uses a Heapq-based representation. This is the structured version, that takes into account some topological structure between samples. Read more in the User Guide.  Parameters 
 
Xarray-like of shape (n_samples, n_features) 

feature matrix representing n_samples samples to be clustered  
connectivitysparse matrix, default=None 

connectivity matrix. Defines for each sample the neighboring samples following a given structure of the data. The matrix is assumed to be symmetric and only the upper triangular half is used. Default is None, i.e, the Ward algorithm is unstructured.  
n_clustersint, default=None 

Stop early the construction of the tree at n_clusters. This is useful to decrease computation time if the number of clusters is not small compared to the number of samples. In this case, the complete tree is not computed, thus the ‘children’ output is of limited use, and the ‘parents’ output should rather be used. This option is valid only when specifying a connectivity matrix.  
return_distancebool, default=None 

If True, return the distance between the clusters.    Returns 
 
childrenndarray of shape (n_nodes-1, 2) 

The children of each non-leaf node. Values less than n_samples correspond to leaves of the tree which are the original samples. A node i greater than or equal to n_samples is a non-leaf node and has children children_[i - n_samples]. Alternatively at the i-th iteration, children[i][0] and children[i][1] are merged to form node n_samples + i  
n_connected_componentsint 

The number of connected components in the graph.  
n_leavesint 

The number of leaves in the tree  
parentsndarray of shape (n_nodes,) or None 

The parent of each node. Only returned when a connectivity matrix is specified, elsewhere ‘None’ is returned.  
distancesndarray of shape (n_nodes-1,) 

Only returned if return_distance is set to True (for compatibility). The distances between the centers of the nodes. distances[i] corresponds to a weighted euclidean distance between the nodes children[i, 1] and children[i, 2]. If the nodes refer to leaves of the tree, then distances[i] is their unweighted euclidean distance. Distances are updated in the following way (from scipy.hierarchy.linkage): The new entry \(d(u,v)\) is computed as follows,  \[d(u,v) = \sqrt{\frac{|v|+|s|} {T}d(v,s)^2 + \frac{|v|+|t|} {T}d(v,t)^2 - \frac{|v|} {T}d(s,t)^2}\] where \(u\) is the newly joined cluster consisting of clusters \(s\) and \(t\), \(v\) is an unused cluster in the forest, \(T=|v|+|s|+|t|\), and \(|*|\) is the cardinality of its argument. This is also known as the incremental algorithm.
sklearn.compose.make_column_selector  
sklearn.compose.make_column_selector(pattern=None, *, dtype_include=None, dtype_exclude=None) [source]
 
Create a callable to select columns to be used with ColumnTransformer. make_column_selector can select columns based on datatype or the columns name with a regex. When using multiple selection criteria, all criteria must match for a column to be selected.  Parameters 
 
patternstr, default=None 

Name of columns containing this regex pattern will be included. If None, column selection will not be selected based on pattern.  
dtype_includecolumn dtype or list of column dtypes, default=None 

A selection of dtypes to include. For more details, see pandas.DataFrame.select_dtypes.  
dtype_excludecolumn dtype or list of column dtypes, default=None 

A selection of dtypes to exclude. For more details, see pandas.DataFrame.select_dtypes.    Returns 
 
selectorcallable 

Callable for column selection to be used by a ColumnTransformer.      See also  
ColumnTransformer


Class that allows combining the outputs of multiple transformer objects used on column subsets of the data into a single feature space.    Examples >>> from sklearn.preprocessing import StandardScaler, OneHotEncoder
>>> from sklearn.compose import make_column_transformer
>>> from sklearn.compose import make_column_selector
>>> import pandas as pd  
>>> X = pd.DataFrame({'city': ['London', 'London', 'Paris', 'Sallisaw'],
...                   'rating': [5, 3, 4, 5]})  
>>> ct = make_column_transformer(
...       (StandardScaler(),
...        make_column_selector(dtype_include=np.number)),  # rating
...       (OneHotEncoder(),
...        make_column_selector(dtype_include=object)))  # city
>>> ct.fit_transform(X)  
array([[ 0.90453403,  1.        ,  0.        ,  0.        ],
       [-1.50755672,  1.        ,  0.        ,  0.        ],
       [-0.30151134,  0.        ,  1.        ,  0.        ],
       [ 0.90453403,  0.        ,  0.        ,  1.        ]])
 
 Examples using sklearn.compose.make_column_selector
 
  Categorical Feature Support in Gradient Boosting  

  Column Transformer with Mixed Types
sklearn.config_context  
sklearn.config_context(**new_config) [source]
 
Context manager for global scikit-learn configuration  Parameters 
 
assume_finitebool, default=False 

If True, validation for finiteness will be skipped, saving time, but leading to potential crashes. If False, validation for finiteness will be performed, avoiding error. Global default: False.  
working_memoryint, default=1024 

If set, scikit-learn will attempt to limit the size of temporary arrays to this number of MiB (per job when parallelised), often saving both computation time and memory on expensive operations that can be performed in chunks. Global default: 1024.  
print_changed_onlybool, default=True 

If True, only the parameters that were set to non-default values will be printed when printing an estimator. For example, print(SVC()) while True will only print ‘SVC()’, but would print ‘SVC(C=1.0, cache_size=200, …)’ with all the non-changed parameters when False. Default is True.  Changed in version 0.23: Default changed from False to True.   
display{‘text’, ‘diagram’}, default=’text’ 

If ‘diagram’, estimators will be displayed as a diagram in a Jupyter lab or notebook context. If ‘text’, estimators will be displayed as text. Default is ‘text’.  New in version 0.23.       See also  
set_config


Set global scikit-learn configuration.  
get_config


Retrieve current values of the global configuration.    Notes All settings, not just those presently modified, will be returned to their previous values when the context manager is exited. This is not thread-safe. Examples >>> import sklearn
>>> from sklearn.utils.validation import assert_all_finite
>>> with sklearn.config_context(assume_finite=True):
...     assert_all_finite([float('nan')])
>>> with sklearn.config_context(assume_finite=True):
...     with sklearn.config_context(assume_finite=False):
...         assert_all_finite([float('nan')])
Traceback (most recent call last):
...
ValueError: Input contains NaN, ...
sklearn.covariance.empirical_covariance  
sklearn.covariance.empirical_covariance(X, *, assume_centered=False) [source]
 
Computes the Maximum likelihood covariance estimator  Parameters 
 
Xndarray of shape (n_samples, n_features) 

Data from which to compute the covariance estimate  
assume_centeredbool, default=False 

If True, data will not be centered before computation. Useful when working with data whose mean is almost, but not exactly zero. If False, data will be centered before computation.    Returns 
 
covariancendarray of shape (n_features, n_features) 

Empirical covariance (Maximum Likelihood Estimator).     Examples >>> from sklearn.covariance import empirical_covariance
>>> X = [[1,1,1],[1,1,1],[1,1,1],
...      [0,0,0],[0,0,0],[0,0,0]]
>>> empirical_covariance(X)
array([[0.25, 0.25, 0.25],
       [0.25, 0.25, 0.25],
       [0.25, 0.25, 0.25]])
 
 Examples using sklearn.covariance.empirical_covariance
 
  Shrinkage covariance estimation: LedoitWolf vs OAS and max-likelihood
sklearn.covariance.graphical_lasso  
sklearn.covariance.graphical_lasso(emp_cov, alpha, *, cov_init=None, mode='cd', tol=0.0001, enet_tol=0.0001, max_iter=100, verbose=False, return_costs=False, eps=2.220446049250313e-16, return_n_iter=False) [source]
 
l1-penalized covariance estimator Read more in the User Guide.  Changed in version v0.20: graph_lasso has been renamed to graphical_lasso   Parameters 
 
emp_covndarray of shape (n_features, n_features) 

Empirical covariance from which to compute the covariance estimate.  
alphafloat 

The regularization parameter: the higher alpha, the more regularization, the sparser the inverse covariance. Range is (0, inf].  
cov_initarray of shape (n_features, n_features), default=None 

The initial guess for the covariance. If None, then the empirical covariance is used.  
mode{‘cd’, ‘lars’}, default=’cd’ 

The Lasso solver to use: coordinate descent or LARS. Use LARS for very sparse underlying graphs, where p > n. Elsewhere prefer cd which is more numerically stable.  
tolfloat, default=1e-4 

The tolerance to declare convergence: if the dual gap goes below this value, iterations are stopped. Range is (0, inf].  
enet_tolfloat, default=1e-4 

The tolerance for the elastic net solver used to calculate the descent direction. This parameter controls the accuracy of the search direction for a given column update, not of the overall parameter estimate. Only used for mode=’cd’. Range is (0, inf].  
max_iterint, default=100 

The maximum number of iterations.  
verbosebool, default=False 

If verbose is True, the objective function and dual gap are printed at each iteration.  
return_costsbool, default=Flase 

If return_costs is True, the objective function and dual gap at each iteration are returned.  
epsfloat, default=eps 

The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. Default is np.finfo(np.float64).eps.  
return_n_iterbool, default=False 

Whether or not to return the number of iterations.    Returns 
 
covariancendarray of shape (n_features, n_features) 

The estimated covariance matrix.  
precisionndarray of shape (n_features, n_features) 

The estimated (sparse) precision matrix.  
costslist of (objective, dual_gap) pairs 

The list of values of the objective function and the dual gap at each iteration. Returned only if return_costs is True.  
n_iterint 

Number of iterations. Returned only if return_n_iter is set to True.      See also  
GraphicalLasso, GraphicalLassoCV

  Notes The algorithm employed to solve this problem is the GLasso algorithm, from the Friedman 2008 Biostatistics paper. It is the same algorithm as in the R glasso package. One possible difference with the glasso R package is that the diagonal coefficients are not penalized.
sklearn.covariance.ledoit_wolf  
sklearn.covariance.ledoit_wolf(X, *, assume_centered=False, block_size=1000) [source]
 
Estimates the shrunk Ledoit-Wolf covariance matrix. Read more in the User Guide.  Parameters 
 
Xarray-like of shape (n_samples, n_features) 

Data from which to compute the covariance estimate  
assume_centeredbool, default=False 

If True, data will not be centered before computation. Useful to work with data whose mean is significantly equal to zero but is not exactly zero. If False, data will be centered before computation.  
block_sizeint, default=1000 

Size of blocks into which the covariance matrix will be split. This is purely a memory optimization and does not affect results.    Returns 
 
shrunk_covndarray of shape (n_features, n_features) 

Shrunk covariance.  
shrinkagefloat 

Coefficient in the convex combination used for the computation of the shrunk estimate.     Notes The regularized (shrunk) covariance is: (1 - shrinkage) * cov + shrinkage * mu * np.identity(n_features) where mu = trace(cov) / n_features 
 Examples using sklearn.covariance.ledoit_wolf
 
  Sparse inverse covariance estimation
sklearn.covariance.oas  
sklearn.covariance.oas(X, *, assume_centered=False) [source]
 
Estimate covariance with the Oracle Approximating Shrinkage algorithm.  Parameters 
 
Xarray-like of shape (n_samples, n_features) 

Data from which to compute the covariance estimate.  
assume_centeredbool, default=False 

If True, data will not be centered before computation. Useful to work with data whose mean is significantly equal to zero but is not exactly zero. If False, data will be centered before computation.    Returns 
 
shrunk_covarray-like of shape (n_features, n_features) 

Shrunk covariance.  
shrinkagefloat 

Coefficient in the convex combination used for the computation of the shrunk estimate.     Notes The regularised (shrunk) covariance is: (1 - shrinkage) * cov + shrinkage * mu * np.identity(n_features) where mu = trace(cov) / n_features The formula we used to implement the OAS is slightly modified compared to the one given in the article. See OAS for more details.
sklearn.covariance.shrunk_covariance  
sklearn.covariance.shrunk_covariance(emp_cov, shrinkage=0.1) [source]
 
Calculates a covariance matrix shrunk on the diagonal Read more in the User Guide.  Parameters 
 
emp_covarray-like of shape (n_features, n_features) 

Covariance matrix to be shrunk  
shrinkagefloat, default=0.1 

Coefficient in the convex combination used for the computation of the shrunk estimate. Range is [0, 1].    Returns 
 
shrunk_covndarray of shape (n_features, n_features) 

Shrunk covariance.     Notes The regularized (shrunk) covariance is given by: (1 - shrinkage) * cov + shrinkage * mu * np.identity(n_features) where mu = trace(cov) / n_features
sklearn.datasets.clear_data_home  
sklearn.datasets.clear_data_home(data_home=None) [source]
 
Delete all the content of the data home cache.  Parameters 
 
data_homestr, default=None 

The path to scikit-learn data directory. If None, the default path is ~/sklearn_learn_data.
sklearn.datasets.dump_svmlight_file  
sklearn.datasets.dump_svmlight_file(X, y, f, *, zero_based=True, comment=None, query_id=None, multilabel=False) [source]
 
Dump the dataset in svmlight / libsvm file format. This format is a text-based format, with one sample per line. It does not store zero valued features hence is suitable for sparse dataset. The first element of each line can be used to store a target variable to predict.  Parameters 
 
X{array-like, sparse matrix} of shape (n_samples, n_features) 

Training vectors, where n_samples is the number of samples and n_features is the number of features.  
y{array-like, sparse matrix}, shape = [n_samples (, n_labels)] 

Target values. Class labels must be an integer or float, or array-like objects of integer or float for multilabel classifications.  
fstring or file-like in binary mode 

If string, specifies the path that will contain the data. If file-like, data will be written to f. f should be opened in binary mode.  
zero_basedboolean, default=True 

Whether column indices should be written zero-based (True) or one-based (False).  
commentstring, default=None 

Comment to insert at the top of the file. This should be either a Unicode string, which will be encoded as UTF-8, or an ASCII byte string. If a comment is given, then it will be preceded by one that identifies the file as having been dumped by scikit-learn. Note that not all tools grok comments in SVMlight files.  
query_idarray-like of shape (n_samples,), default=None 

Array containing pairwise preference constraints (qid in svmlight format).  
multilabelboolean, default=False 

Samples may have several labels each (see https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/multilabel.html)  New in version 0.17: parameter multilabel to support multilabel datasets.      
 Examples using sklearn.datasets.dump_svmlight_file
 
  Libsvm GUI
sklearn.datasets.fetch_20newsgroups  
sklearn.datasets.fetch_20newsgroups(*, data_home=None, subset='train', categories=None, shuffle=True, random_state=42, remove=(), download_if_missing=True, return_X_y=False) [source]
 
Load the filenames and data from the 20 newsgroups dataset (classification). Download it if necessary.  
Classes 20  
Samples total 18846  
Dimensionality 1  
Features text   Read more in the User Guide.  Parameters 
 
data_homestr, default=None 

Specify a download and cache folder for the datasets. If None, all scikit-learn data is stored in ‘~/scikit_learn_data’ subfolders.  
subset{‘train’, ‘test’, ‘all’}, default=’train’ 

Select the dataset to load: ‘train’ for the training set, ‘test’ for the test set, ‘all’ for both, with shuffled ordering.  
categoriesarray-like, dtype=str or unicode, default=None 

If None (default), load all the categories. If not None, list of category names to load (other categories ignored).  
shufflebool, default=True 

Whether or not to shuffle the data: might be important for models that make the assumption that the samples are independent and identically distributed (i.i.d.), such as stochastic gradient descent.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for dataset shuffling. Pass an int for reproducible output across multiple function calls. See Glossary.  
removetuple, default=() 

May contain any subset of (‘headers’, ‘footers’, ‘quotes’). Each of these are kinds of text that will be detected and removed from the newsgroup posts, preventing classifiers from overfitting on metadata. ‘headers’ removes newsgroup headers, ‘footers’ removes blocks at the ends of posts that look like signatures, and ‘quotes’ removes lines that appear to be quoting another post. ‘headers’ follows an exact standard; the other filters are not always correct.  
download_if_missingbool, default=True 

If False, raise an IOError if the data is not locally available instead of trying to download the data from the source site.  
return_X_ybool, default=False 

If True, returns (data.data, data.target) instead of a Bunch object.  New in version 0.22.     Returns 
 
bunchBunch 

Dictionary-like object, with the following attributes.  
datalist of shape (n_samples,) 

The data list to learn.  target: ndarray of shape (n_samples,)

The target labels.  filenames: list of shape (n_samples,)

The path to the location of the data.  DESCR: str

The full description of the dataset.  target_names: list of shape (n_classes,)

The names of target classes.    
(data, target)tuple if return_X_y=True 

 New in version 0.22.      
 Examples using sklearn.datasets.fetch_20newsgroups
 
  Biclustering documents with the Spectral Co-clustering algorithm  

  Topic extraction with Non-negative Matrix Factorization and Latent Dirichlet Allocation  

  Sample pipeline for text feature extraction and evaluation  

  Column Transformer with Heterogeneous Data Sources  

  Semi-supervised Classification on a Text Dataset  

  FeatureHasher and DictVectorizer Comparison  

  Clustering text documents using k-means  

  Classification of text documents using sparse features
sklearn.datasets.fetch_california_housing  
sklearn.datasets.fetch_california_housing(*, data_home=None, download_if_missing=True, return_X_y=False, as_frame=False) [source]
 
Load the California housing dataset (regression).  
Samples total 20640  
Dimensionality 8  
Features real  
Target real 0.15 - 5.   Read more in the User Guide.  Parameters 
 
data_homestr, default=None 

Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in ‘~/scikit_learn_data’ subfolders.  
download_if_missingbool, default=True 

If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site.  
return_X_ybool, default=False. 

If True, returns (data.data, data.target) instead of a Bunch object.  New in version 0.20.   
as_framebool, default=False 

If True, the data is a pandas DataFrame including columns with appropriate dtypes (numeric, string or categorical). The target is a pandas DataFrame or Series depending on the number of target_columns.  New in version 0.23.     Returns 
 
datasetBunch 

Dictionary-like object, with the following attributes.  
datandarray, shape (20640, 8) 

Each row corresponding to the 8 feature values in order. If as_frame is True, data is a pandas object.  
targetnumpy array of shape (20640,) 

Each value corresponds to the average house value in units of 100,000. If as_frame is True, target is a pandas object.  
feature_nameslist of length 8 

Array of ordered feature names used in the dataset.  
DESCRstring 

Description of the California housing dataset.  
framepandas DataFrame 

Only present when as_frame=True. DataFrame with data and target.  New in version 0.23.     
(data, target)tuple if return_X_y is True 

 New in version 0.20.      Notes This dataset consists of 20,640 samples and 9 features. 
 Examples using sklearn.datasets.fetch_california_housing
 
  Release Highlights for scikit-learn 0.24  

  Partial Dependence and Individual Conditional Expectation Plots  

  Imputing missing values with variants of IterativeImputer  

  Imputing missing values before building an estimator  

  Compare the effect of different scalers on data with outliers
sklearn.datasets.fetch_covtype  
sklearn.datasets.fetch_covtype(*, data_home=None, download_if_missing=True, random_state=None, shuffle=False, return_X_y=False, as_frame=False) [source]
 
Load the covertype dataset (classification). Download it if necessary.  
Classes 7  
Samples total 581012  
Dimensionality 54  
Features int   Read more in the User Guide.  Parameters 
 
data_homestr, default=None 

Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in ‘~/scikit_learn_data’ subfolders.  
download_if_missingbool, default=True 

If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for dataset shuffling. Pass an int for reproducible output across multiple function calls. See Glossary.  
shufflebool, default=False 

Whether to shuffle dataset.  
return_X_ybool, default=False 

If True, returns (data.data, data.target) instead of a Bunch object.  New in version 0.20.   
as_framebool, default=False 

If True, the data is a pandas DataFrame including columns with appropriate dtypes (numeric). The target is a pandas DataFrame or Series depending on the number of target columns. If return_X_y is True, then (data, target) will be pandas DataFrames or Series as described below.  New in version 0.24.     Returns 
 
datasetBunch 

Dictionary-like object, with the following attributes.  
datandarray of shape (581012, 54) 

Each row corresponds to the 54 features in the dataset.  
targetndarray of shape (581012,) 

Each value corresponds to one of the 7 forest covertypes with values ranging between 1 to 7.  
framedataframe of shape (581012, 53) 

Only present when as_frame=True. Contains data and target.  
DESCRstr 

Description of the forest covertype dataset.  
feature_nameslist 

The names of the dataset columns.  target_names: list

The names of the target columns.    
(data, target)tuple if return_X_y is True 

 New in version 0.20.      
 Examples using sklearn.datasets.fetch_covtype
 
  Release Highlights for scikit-learn 0.24  

  Scalable learning with polynomial kernel aproximation