neulab/docprompting-conala huggingface dataset:
sklearn.feature_extraction.image.img_to_graph(img, *, mask=None, return_as=<class 'scipy.sparse.coo.coo_matrix'>, dtype=None) [source]
 
Graph of the pixel-to-pixel gradient connections Edges are weighted with the gradient values. Read more in the User Guide.  Parameters 
 
imgndarray of shape (height, width) or (height, width, channel) 

2D or 3D image.  
maskndarray of shape (height, width) or (height, width, channel), dtype=bool, default=None 

An optional mask of the image, to consider only part of the pixels.  
return_asnp.ndarray or a sparse matrix class, default=sparse.coo_matrix 

The class to use to build the returned adjacency matrix.  
dtypedtype, default=None 

The data of the returned sparse matrix. By default it is the dtype of img     Notes For scikit-learn versions 0.14.1 and prior, return_as=np.ndarray was handled by returning a dense np.matrix instance. Going forward, np.ndarray returns an np.ndarray, as expected. For compatibility, user code relying on this method should wrap its calls in np.asarray to avoid type issues.
class sklearn.feature_extraction.image.PatchExtractor(*, patch_size=None, max_patches=None, random_state=None) [source]
 
Extracts patches from a collection of images Read more in the User Guide.  New in version 0.9.   Parameters 
 
patch_sizetuple of int (patch_height, patch_width), default=None 

The dimensions of one patch.  
max_patchesint or float, default=None 

The maximum number of patches per image to extract. If max_patches is a float in (0, 1), it is taken to mean a proportion of the total number of patches.  
random_stateint, RandomState instance, default=None 

Determines the random number generator used for random sampling when max_patches is not None. Use an int to make the randomness deterministic. See Glossary.     Examples >>> from sklearn.datasets import load_sample_images
>>> from sklearn.feature_extraction import image
>>> # Use the array data from the second image in this dataset:
>>> X = load_sample_images().images[1]
>>> print('Image shape: {}'.format(X.shape))
Image shape: (427, 640, 3)
>>> pe = image.PatchExtractor(patch_size=(2, 2))
>>> pe_fit = pe.fit(X)
>>> pe_trans = pe.transform(X)
>>> print('Patches shape: {}'.format(pe_trans.shape))
Patches shape: (545706, 2, 2)
 Methods  
fit(X[, y]) Do nothing and return the estimator unchanged.  
get_params([deep]) Get parameters for this estimator.  
set_params(**params) Set the parameters of this estimator.  
transform(X) Transforms the image samples in X into a matrix of patch data.    
fit(X, y=None) [source]
 
Do nothing and return the estimator unchanged. This method is just there to implement the usual API and hence work in pipelines.  Parameters 
 
Xarray-like of shape (n_samples, n_features) 

Training data.     
  
get_params(deep=True) [source]
 
Get parameters for this estimator.  Parameters 
 
deepbool, default=True 

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns 
 
paramsdict 

Parameter names mapped to their values.     
  
set_params(**params) [source]
 
Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters 
 
**paramsdict 

Estimator parameters.    Returns 
 
selfestimator instance 

Estimator instance.     
  
transform(X) [source]
 
Transforms the image samples in X into a matrix of patch data.  Parameters 
 
Xndarray of shape (n_samples, image_height, image_width) or (n_samples, image_height, image_width, n_channels) 

Array of images from which to extract patches. For color images, the last dimension specifies the channel: a RGB image would have n_channels=3.    Returns 
 
patchesarray of shape (n_patches, patch_height, patch_width) or (n_patches, patch_height, patch_width, n_channels) 

The collection of patches extracted from the images, where n_patches is either n_samples * max_patches or the total number of patches that can be extracted.
sklearn.feature_extraction.image.PatchExtractor  
class sklearn.feature_extraction.image.PatchExtractor(*, patch_size=None, max_patches=None, random_state=None) [source]
 
Extracts patches from a collection of images Read more in the User Guide.  New in version 0.9.   Parameters 
 
patch_sizetuple of int (patch_height, patch_width), default=None 

The dimensions of one patch.  
max_patchesint or float, default=None 

The maximum number of patches per image to extract. If max_patches is a float in (0, 1), it is taken to mean a proportion of the total number of patches.  
random_stateint, RandomState instance, default=None 

Determines the random number generator used for random sampling when max_patches is not None. Use an int to make the randomness deterministic. See Glossary.     Examples >>> from sklearn.datasets import load_sample_images
>>> from sklearn.feature_extraction import image
>>> # Use the array data from the second image in this dataset:
>>> X = load_sample_images().images[1]
>>> print('Image shape: {}'.format(X.shape))
Image shape: (427, 640, 3)
>>> pe = image.PatchExtractor(patch_size=(2, 2))
>>> pe_fit = pe.fit(X)
>>> pe_trans = pe.transform(X)
>>> print('Patches shape: {}'.format(pe_trans.shape))
Patches shape: (545706, 2, 2)
 Methods  
fit(X[, y]) Do nothing and return the estimator unchanged.  
get_params([deep]) Get parameters for this estimator.  
set_params(**params) Set the parameters of this estimator.  
transform(X) Transforms the image samples in X into a matrix of patch data.    
fit(X, y=None) [source]
 
Do nothing and return the estimator unchanged. This method is just there to implement the usual API and hence work in pipelines.  Parameters 
 
Xarray-like of shape (n_samples, n_features) 

Training data.     
  
get_params(deep=True) [source]
 
Get parameters for this estimator.  Parameters 
 
deepbool, default=True 

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns 
 
paramsdict 

Parameter names mapped to their values.     
  
set_params(**params) [source]
 
Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters 
 
**paramsdict 

Estimator parameters.    Returns 
 
selfestimator instance 

Estimator instance.     
  
transform(X) [source]
 
Transforms the image samples in X into a matrix of patch data.  Parameters 
 
Xndarray of shape (n_samples, image_height, image_width) or (n_samples, image_height, image_width, n_channels) 

Array of images from which to extract patches. For color images, the last dimension specifies the channel: a RGB image would have n_channels=3.    Returns 
 
patchesarray of shape (n_patches, patch_height, patch_width) or (n_patches, patch_height, patch_width, n_channels) 

The collection of patches extracted from the images, where n_patches is either n_samples * max_patches or the total number of patches that can be extracted.
fit(X, y=None) [source]
 
Do nothing and return the estimator unchanged. This method is just there to implement the usual API and hence work in pipelines.  Parameters 
 
Xarray-like of shape (n_samples, n_features) 

Training data.
transform(X) [source]
 
Transforms the image samples in X into a matrix of patch data.  Parameters 
 
Xndarray of shape (n_samples, image_height, image_width) or (n_samples, image_height, image_width, n_channels) 

Array of images from which to extract patches. For color images, the last dimension specifies the channel: a RGB image would have n_channels=3.    Returns 
 
patchesarray of shape (n_patches, patch_height, patch_width) or (n_patches, patch_height, patch_width, n_channels) 

The collection of patches extracted from the images, where n_patches is either n_samples * max_patches or the total number of patches that can be extracted.
sklearn.feature_extraction.image.reconstruct_from_patches_2d(patches, image_size) [source]
 
Reconstruct the image from all of its patches. Patches are assumed to overlap and the image is constructed by filling in the patches from left to right, top to bottom, averaging the overlapping regions. Read more in the User Guide.  Parameters 
 
patchesndarray of shape (n_patches, patch_height, patch_width) or (n_patches, patch_height, patch_width, n_channels) 

The complete set of patches. If the patches contain colour information, channels are indexed along the last dimension: RGB patches would have n_channels=3.  
image_sizetuple of int (image_height, image_width) or (image_height, image_width, n_channels) 

The size of the image that will be reconstructed.    Returns 
 
imagendarray of shape image_size 

The reconstructed image.
build_analyzer() [source]
 
Return a callable that handles preprocessing, tokenization and n-grams generation.  Returns 
 analyzer: callable

A function to handle preprocessing, tokenization and n-grams generation.
build_preprocessor() [source]
 
Return a function to preprocess the text before tokenization.  Returns 
 preprocessor: callable

A function to preprocess the text before tokenization.
build_tokenizer() [source]
 
Return a function that splits a string into a sequence of tokens.  Returns 
 tokenizer: callable

A function to split a string into a sequence of tokens.
decode(doc) [source]
 
Decode the input into a string of unicode symbols. The decoding strategy depends on the vectorizer parameters.  Parameters 
 
docstr 

The string to decode.    Returns 
 doc: str

A string of unicode symbols.
fit(raw_documents, y=None) [source]
 
Learn a vocabulary dictionary of all tokens in the raw documents.  Parameters 
 
raw_documentsiterable 

An iterable which yields either str, unicode or file objects.    Returns 
 self
fit_transform(raw_documents, y=None) [source]
 
Learn the vocabulary dictionary and return document-term matrix. This is equivalent to fit followed by transform, but more efficiently implemented.  Parameters 
 
raw_documentsiterable 

An iterable which yields either str, unicode or file objects.    Returns 
 
Xarray of shape (n_samples, n_features) 

Document-term matrix.
get_feature_names() [source]
 
Array mapping from feature integer indices to feature name.  Returns 
 
feature_nameslist 

A list of feature names.
get_stop_words() [source]
 
Build or fetch the effective stop words list.  Returns 
 stop_words: list or None

A list of stop words.
inverse_transform(X) [source]
 
Return terms per document with nonzero entries in X.  Parameters 
 
X{array-like, sparse matrix} of shape (n_samples, n_features) 

Document-term matrix.    Returns 
 
X_invlist of arrays of shape (n_samples,) 

List of arrays of terms.
transform(raw_documents) [source]
 
Transform documents to document-term matrix. Extract token counts out of raw text documents using the vocabulary fitted with fit or the one provided to the constructor.  Parameters 
 
raw_documentsiterable 

An iterable which yields either str, unicode or file objects.    Returns 
 
Xsparse matrix of shape (n_samples, n_features) 

Document-term matrix.
fit(X, y=None) [source]
 
Does nothing: this transformer is stateless.  Parameters 
 
Xndarray of shape [n_samples, n_features] 

Training data.
fit_transform(X, y=None) [source]
 
Transform a sequence of documents to a document-term matrix.  Parameters 
 
Xiterable over raw text documents, length = n_samples 

Samples. Each sample must be a text document (either bytes or unicode strings, file name or file object depending on the constructor argument) which will be tokenized and hashed.  
yany 

Ignored. This parameter exists only for compatibility with sklearn.pipeline.Pipeline.    Returns 
 
Xsparse matrix of shape (n_samples, n_features) 

Document-term matrix.
partial_fit(X, y=None) [source]
 
Does nothing: this transformer is stateless. This method is just there to mark the fact that this transformer can work in a streaming setup.  Parameters 
 
Xndarray of shape [n_samples, n_features] 

Training data.
transform(X) [source]
 
Transform a sequence of documents to a document-term matrix.  Parameters 
 
Xiterable over raw text documents, length = n_samples 

Samples. Each sample must be a text document (either bytes or unicode strings, file name or file object depending on the constructor argument) which will be tokenized and hashed.    Returns 
 
Xsparse matrix of shape (n_samples, n_features) 

Document-term matrix.
fit(X, y=None) [source]
 
Learn the idf vector (global term weights).  Parameters 
 
Xsparse matrix of shape n_samples, n_features) 

A matrix of term/token counts.
transform(X, copy=True) [source]
 
Transform a count matrix to a tf or tf-idf representation  Parameters 
 
Xsparse matrix of (n_samples, n_features) 

a matrix of term/token counts  
copybool, default=True 

Whether to copy X and operate on the copy or perform in-place operations.    Returns 
 
vectorssparse matrix of shape (n_samples, n_features)
fit(raw_documents, y=None) [source]
 
Learn vocabulary and idf from training set.  Parameters 
 
raw_documentsiterable 

An iterable which yields either str, unicode or file objects.  
yNone 

This parameter is not needed to compute tfidf.    Returns 
 
selfobject 

Fitted vectorizer.
fit_transform(raw_documents, y=None) [source]
 
Learn vocabulary and idf, return document-term matrix. This is equivalent to fit followed by transform, but more efficiently implemented.  Parameters 
 
raw_documentsiterable 

An iterable which yields either str, unicode or file objects.  
yNone 

This parameter is ignored.    Returns 
 
Xsparse matrix of (n_samples, n_features) 

Tf-idf-weighted document-term matrix.
transform(raw_documents) [source]
 
Transform documents to document-term matrix. Uses the vocabulary and document frequencies (df) learned by fit (or fit_transform).  Parameters 
 
raw_documentsiterable 

An iterable which yields either str, unicode or file objects.    Returns 
 
Xsparse matrix of (n_samples, n_features) 

Tf-idf-weighted document-term matrix.
sklearn.feature_selection.chi2(X, y) [source]
 
Compute chi-squared stats between each non-negative feature and class. This score can be used to select the n_features features with the highest values for the test chi-squared statistic from X, which must contain only non-negative features such as booleans or frequencies (e.g., term counts in document classification), relative to the classes. Recall that the chi-square test measures dependence between stochastic variables, so using this function “weeds out” the features that are the most likely to be independent of class and therefore irrelevant for classification. Read more in the User Guide.  Parameters 
 
X{array-like, sparse matrix} of shape (n_samples, n_features) 

Sample vectors.  
yarray-like of shape (n_samples,) 

Target vector (class labels).    Returns 
 
chi2array, shape = (n_features,) 

chi2 statistics of each feature.  
pvalarray, shape = (n_features,) 

p-values of each feature.      See also  
f_classif


ANOVA F-value between label/feature for classification tasks.  
f_regression


F-value between label/feature for regression tasks.    Notes Complexity of this algorithm is O(n_classes * n_features).
sklearn.feature_selection.f_classif(X, y) [source]
 
Compute the ANOVA F-value for the provided sample. Read more in the User Guide.  Parameters 
 
X{array-like, sparse matrix} shape = [n_samples, n_features] 

The set of regressors that will be tested sequentially.  
yarray of shape(n_samples) 

The data matrix.    Returns 
 
Farray, shape = [n_features,] 

The set of F values.  
pvalarray, shape = [n_features,] 

The set of p-values.      See also  
chi2


Chi-squared stats of non-negative features for classification tasks.  
f_regression


F-value between label/feature for regression tasks.
sklearn.feature_selection.f_regression(X, y, *, center=True) [source]
 
Univariate linear regression tests. Linear model for testing the individual effect of each of many regressors. This is a scoring function to be used in a feature selection procedure, not a free standing feature selection procedure. This is done in 2 steps:  The correlation between each regressor and the target is computed, that is, ((X[:, i] - mean(X[:, i])) * (y - mean_y)) / (std(X[:, i]) * std(y)). It is converted to an F score then to a p-value.  For more on usage see the User Guide.  Parameters 
 
X{array-like, sparse matrix} shape = (n_samples, n_features) 

The set of regressors that will be tested sequentially.  
yarray of shape(n_samples). 

The data matrix  
centerbool, default=True 

If true, X and y will be centered.    Returns 
 
Farray, shape=(n_features,) 

F values of features.  
pvalarray, shape=(n_features,) 

p-values of F-scores.      See also  
mutual_info_regression


Mutual information for a continuous target.  
f_classif


ANOVA F-value between label/feature for classification tasks.  
chi2


Chi-squared stats of non-negative features for classification tasks.  
SelectKBest


Select features based on the k highest scores.  
SelectFpr


Select features based on a false positive rate test.  
SelectFdr


Select features based on an estimated false discovery rate.  
SelectFwe


Select features based on family-wise error rate.  
SelectPercentile


Select features based on percentile of the highest scores.
fit(X, y) [source]
 
Run score function on (X, y) and get the appropriate features.  Parameters 
 
Xarray-like of shape (n_samples, n_features) 

The training input samples.  
yarray-like of shape (n_samples,) 

The target values (class labels in classification, real numbers in regression).    Returns 
 
selfobject
get_support(indices=False) [source]
 
Get a mask, or integer index, of the features selected  Parameters 
 
indicesbool, default=False 

If True, the return value will be an array of integers, rather than a boolean mask.    Returns 
 
supportarray 

An index that selects the retained features from a feature vector. If indices is False, this is a boolean array of shape [# input features], in which an element is True iff its corresponding feature is selected for retention. If indices is True, this is an integer array of shape [# output features] whose values are indices into the input feature vector.
inverse_transform(X) [source]
 
Reverse the transformation operation  Parameters 
 
Xarray of shape [n_samples, n_selected_features] 

The input samples.    Returns 
 
X_rarray of shape [n_samples, n_original_features] 

X with columns of zeros inserted where features would have been removed by transform.
transform(X) [source]
 
Reduce X to the selected features.  Parameters 
 
Xarray of shape [n_samples, n_features] 

The input samples.    Returns 
 
X_rarray of shape [n_samples, n_selected_features] 

The input samples with only the selected features.
sklearn.feature_selection.mutual_info_classif(X, y, *, discrete_features='auto', n_neighbors=3, copy=True, random_state=None) [source]
 
Estimate mutual information for a discrete target variable. Mutual information (MI) [1] between two random variables is a non-negative value, which measures the dependency between the variables. It is equal to zero if and only if two random variables are independent, and higher values mean higher dependency. The function relies on nonparametric methods based on entropy estimation from k-nearest neighbors distances as described in [2] and [3]. Both methods are based on the idea originally proposed in [4]. It can be used for univariate features selection, read more in the User Guide.  Parameters 
 
Xarray-like or sparse matrix, shape (n_samples, n_features) 

Feature matrix.  
yarray-like of shape (n_samples,) 

Target vector.  
discrete_features{‘auto’, bool, array-like}, default=’auto’ 

If bool, then determines whether to consider all features discrete or continuous. If array, then it should be either a boolean mask with shape (n_features,) or array with indices of discrete features. If ‘auto’, it is assigned to False for dense X and to True for sparse X.  
n_neighborsint, default=3 

Number of neighbors to use for MI estimation for continuous variables, see [2] and [3]. Higher values reduce variance of the estimation, but could introduce a bias.  
copybool, default=True 

Whether to make a copy of the given data. If set to False, the initial data will be overwritten.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for adding small noise to continuous variables in order to remove repeated values. Pass an int for reproducible results across multiple function calls. See Glossary.    Returns 
 
mindarray, shape (n_features,) 

Estimated mutual information between each feature and the target.     Notes  The term “discrete features” is used instead of naming them “categorical”, because it describes the essence more accurately. For example, pixel intensities of an image are discrete features (but hardly categorical) and you will get better results if mark them as such. Also note, that treating a continuous variable as discrete and vice versa will usually give incorrect results, so be attentive about that. True mutual information can’t be negative. If its estimate turns out to be negative, it is replaced by zero.  References  
1  
Mutual Information on Wikipedia.  
2(1,2)  
A. Kraskov, H. Stogbauer and P. Grassberger, “Estimating mutual information”. Phys. Rev. E 69, 2004.  
3(1,2)  
B. C. Ross “Mutual Information between Discrete and Continuous Data Sets”. PLoS ONE 9(2), 2014.  
4  
L. F. Kozachenko, N. N. Leonenko, “Sample Estimate of the Entropy of a Random Vector:, Probl. Peredachi Inf., 23:2 (1987), 9-16