neulab/docprompting-conala huggingface dataset:
sklearn.datasets.make_sparse_spd_matrix  
sklearn.datasets.make_sparse_spd_matrix(dim=1, *, alpha=0.95, norm_diag=False, smallest_coef=0.1, largest_coef=0.9, random_state=None) [source]
 
Generate a sparse symmetric definite positive matrix. Read more in the User Guide.  Parameters 
 
dimint, default=1 

The size of the random matrix to generate.  
alphafloat, default=0.95 

The probability that a coefficient is zero (see notes). Larger values enforce more sparsity. The value should be in the range 0 and 1.  
norm_diagbool, default=False 

Whether to normalize the output matrix to make the leading diagonal elements all 1  
smallest_coeffloat, default=0.1 

The value of the smallest coefficient between 0 and 1.  
largest_coeffloat, default=0.9 

The value of the largest coefficient between 0 and 1.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns 
 
precsparse matrix of shape (dim, dim) 

The generated matrix.      See also  
make_spd_matrix

  Notes The sparsity is actually imposed on the cholesky factor of the matrix. Thus alpha does not translate directly into the filling fraction of the matrix itself. 
 Examples using sklearn.datasets.make_sparse_spd_matrix
 
  Sparse inverse covariance estimation
sklearn.datasets.make_sparse_uncorrelated  
sklearn.datasets.make_sparse_uncorrelated(n_samples=100, n_features=10, *, random_state=None) [source]
 
Generate a random regression problem with sparse uncorrelated design. This dataset is described in Celeux et al [1]. as: X ~ N(0, 1)
y(X) = X[:, 0] + 2 * X[:, 1] - 2 * X[:, 2] - 1.5 * X[:, 3]
 Only the first 4 features are informative. The remaining features are useless. Read more in the User Guide.  Parameters 
 
n_samplesint, default=100 

The number of samples.  
n_featuresint, default=10 

The number of features.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns 
 
Xndarray of shape (n_samples, n_features) 

The input samples.  
yndarray of shape (n_samples,) 

The output values.     References  
1  
G. Celeux, M. El Anbari, J.-M. Marin, C. P. Robert, “Regularization in regression: comparing Bayesian and frequentist methods in a poorly informative situation”, 2009.
sklearn.datasets.make_spd_matrix  
sklearn.datasets.make_spd_matrix(n_dim, *, random_state=None) [source]
 
Generate a random symmetric, positive-definite matrix. Read more in the User Guide.  Parameters 
 
n_dimint 

The matrix dimension.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns 
 
Xndarray of shape (n_dim, n_dim) 

The random symmetric, positive-definite matrix.      See also  
make_sparse_spd_matrix
sklearn.datasets.make_swiss_roll  
sklearn.datasets.make_swiss_roll(n_samples=100, *, noise=0.0, random_state=None) [source]
 
Generate a swiss roll dataset. Read more in the User Guide.  Parameters 
 
n_samplesint, default=100 

The number of sample points on the S curve.  
noisefloat, default=0.0 

The standard deviation of the gaussian noise.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns 
 
Xndarray of shape (n_samples, 3) 

The points.  
tndarray of shape (n_samples,) 

The univariate position of the sample according to the main dimension of the points in the manifold.     Notes The algorithm is from Marsland [1]. References  
1  
S. Marsland, “Machine Learning: An Algorithmic Perspective”, Chapter 10, 2009. http://seat.massey.ac.nz/personal/s.r.marsland/Code/10/lle.py   
 Examples using sklearn.datasets.make_swiss_roll
 
  Hierarchical clustering: structured vs unstructured ward  

  Swiss Roll reduction with LLE
sklearn.datasets.make_s_curve  
sklearn.datasets.make_s_curve(n_samples=100, *, noise=0.0, random_state=None) [source]
 
Generate an S curve dataset. Read more in the User Guide.  Parameters 
 
n_samplesint, default=100 

The number of sample points on the S curve.  
noisefloat, default=0.0 

The standard deviation of the gaussian noise.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns 
 
Xndarray of shape (n_samples, 3) 

The points.  
tndarray of shape (n_samples,) 

The univariate position of the sample according to the main dimension of the points in the manifold.     
 Examples using sklearn.datasets.make_s_curve
 
  Comparison of Manifold Learning methods  

  t-SNE: The effect of various perplexity values on the shape
sklearn.decomposition.dict_learning  
sklearn.decomposition.dict_learning(X, n_components, *, alpha, max_iter=100, tol=1e-08, method='lars', n_jobs=None, dict_init=None, code_init=None, callback=None, verbose=False, random_state=None, return_n_iter=False, positive_dict=False, positive_code=False, method_max_iter=1000) [source]
 
Solves a dictionary learning matrix factorization problem. Finds the best dictionary and the corresponding sparse code for approximating the data matrix X by solving: (U^*, V^*) = argmin 0.5 || X - U V ||_2^2 + alpha * || U ||_1
             (U,V)
            with || V_k ||_2 = 1 for all  0 <= k < n_components
 where V is the dictionary and U is the sparse code. Read more in the User Guide.  Parameters 
 
Xndarray of shape (n_samples, n_features) 

Data matrix.  
n_componentsint 

Number of dictionary atoms to extract.  
alphaint 

Sparsity controlling parameter.  
max_iterint, default=100 

Maximum number of iterations to perform.  
tolfloat, default=1e-8 

Tolerance for the stopping condition.  
method{‘lars’, ‘cd’}, default=’lars’ 

The method used:  
 
'lars': uses the least angle regression method to solve the lasso 

problem (linear_model.lars_path);    
'cd': uses the coordinate descent method to compute the Lasso solution (linear_model.Lasso). Lars will be faster if the estimated components are sparse.   
n_jobsint, default=None 

Number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.  
dict_initndarray of shape (n_components, n_features), default=None 

Initial value for the dictionary for warm restart scenarios.  
code_initndarray of shape (n_samples, n_components), default=None 

Initial value for the sparse code for warm restart scenarios.  
callbackcallable, default=None 

Callable that gets invoked every five iterations  
verbosebool, default=False 

To control the verbosity of the procedure.  
random_stateint, RandomState instance or None, default=None 

Used for randomly initializing the dictionary. Pass an int for reproducible results across multiple function calls. See Glossary.  
return_n_iterbool, default=False 

Whether or not to return the number of iterations.  
positive_dictbool, default=False 

Whether to enforce positivity when finding the dictionary.  New in version 0.20.   
positive_codebool, default=False 

Whether to enforce positivity when finding the code.  New in version 0.20.   
method_max_iterint, default=1000 

Maximum number of iterations to perform.  New in version 0.22.     Returns 
 
codendarray of shape (n_samples, n_components) 

The sparse code factor in the matrix factorization.  
dictionaryndarray of shape (n_components, n_features), 

The dictionary factor in the matrix factorization.  
errorsarray 

Vector of errors at each iteration.  
n_iterint 

Number of iterations run. Returned only if return_n_iter is set to True.      See also  
dict_learning_online


DictionaryLearning


MiniBatchDictionaryLearning


SparsePCA


MiniBatchSparsePCA
sklearn.feature_extraction.image.extract_patches_2d  
sklearn.feature_extraction.image.extract_patches_2d(image, patch_size, *, max_patches=None, random_state=None) [source]
 
Reshape a 2D image into a collection of patches The resulting patches are allocated in a dedicated array. Read more in the User Guide.  Parameters 
 
imagendarray of shape (image_height, image_width) or (image_height, image_width, n_channels) 

The original image data. For color images, the last dimension specifies the channel: a RGB image would have n_channels=3.  
patch_sizetuple of int (patch_height, patch_width) 

The dimensions of one patch.  
max_patchesint or float, default=None 

The maximum number of patches to extract. If max_patches is a float between 0 and 1, it is taken to be a proportion of the total number of patches.  
random_stateint, RandomState instance, default=None 

Determines the random number generator used for random sampling when max_patches is not None. Use an int to make the randomness deterministic. See Glossary.    Returns 
 
patchesarray of shape (n_patches, patch_height, patch_width) or (n_patches, patch_height, patch_width, n_channels) 

The collection of patches extracted from the image, where n_patches is either max_patches or the total number of patches that can be extracted.     Examples >>> from sklearn.datasets import load_sample_image
>>> from sklearn.feature_extraction import image
>>> # Use the array data from the first image in this dataset:
>>> one_image = load_sample_image("china.jpg")
>>> print('Image shape: {}'.format(one_image.shape))
Image shape: (427, 640, 3)
>>> patches = image.extract_patches_2d(one_image, (2, 2))
>>> print('Patches shape: {}'.format(patches.shape))
Patches shape: (272214, 2, 2, 3)
>>> # Here are just two of these patches:
>>> print(patches[1])
[[[174 201 231]
  [174 201 231]]
 [[173 200 230]
  [173 200 230]]]
>>> print(patches[800])
[[[187 214 243]
  [188 215 244]]
 [[187 214 243]
  [188 215 244]]]
 
 Examples using sklearn.feature_extraction.image.extract_patches_2d
 
  Online learning of a dictionary of parts of faces  

  Image denoising using dictionary learning
sklearn.feature_extraction.image.grid_to_graph  
sklearn.feature_extraction.image.grid_to_graph(n_x, n_y, n_z=1, *, mask=None, return_as=<class 'scipy.sparse.coo.coo_matrix'>, dtype=<class 'int'>) [source]
 
Graph of the pixel-to-pixel connections Edges exist if 2 voxels are connected.  Parameters 
 
n_xint 

Dimension in x axis  
n_yint 

Dimension in y axis  
n_zint, default=1 

Dimension in z axis  
maskndarray of shape (n_x, n_y, n_z), dtype=bool, default=None 

An optional mask of the image, to consider only part of the pixels.  
return_asnp.ndarray or a sparse matrix class, default=sparse.coo_matrix 

The class to use to build the returned adjacency matrix.  
dtypedtype, default=int 

The data of the returned sparse matrix. By default it is int     Notes For scikit-learn versions 0.14.1 and prior, return_as=np.ndarray was handled by returning a dense np.matrix instance. Going forward, np.ndarray returns an np.ndarray, as expected. For compatibility, user code relying on this method should wrap its calls in np.asarray to avoid type issues.
sklearn.feature_extraction.image.img_to_graph  
sklearn.feature_extraction.image.img_to_graph(img, *, mask=None, return_as=<class 'scipy.sparse.coo.coo_matrix'>, dtype=None) [source]
 
Graph of the pixel-to-pixel gradient connections Edges are weighted with the gradient values. Read more in the User Guide.  Parameters 
 
imgndarray of shape (height, width) or (height, width, channel) 

2D or 3D image.  
maskndarray of shape (height, width) or (height, width, channel), dtype=bool, default=None 

An optional mask of the image, to consider only part of the pixels.  
return_asnp.ndarray or a sparse matrix class, default=sparse.coo_matrix 

The class to use to build the returned adjacency matrix.  
dtypedtype, default=None 

The data of the returned sparse matrix. By default it is the dtype of img     Notes For scikit-learn versions 0.14.1 and prior, return_as=np.ndarray was handled by returning a dense np.matrix instance. Going forward, np.ndarray returns an np.ndarray, as expected. For compatibility, user code relying on this method should wrap its calls in np.asarray to avoid type issues.
sklearn.feature_extraction.image.reconstruct_from_patches_2d  
sklearn.feature_extraction.image.reconstruct_from_patches_2d(patches, image_size) [source]
 
Reconstruct the image from all of its patches. Patches are assumed to overlap and the image is constructed by filling in the patches from left to right, top to bottom, averaging the overlapping regions. Read more in the User Guide.  Parameters 
 
patchesndarray of shape (n_patches, patch_height, patch_width) or (n_patches, patch_height, patch_width, n_channels) 

The complete set of patches. If the patches contain colour information, channels are indexed along the last dimension: RGB patches would have n_channels=3.  
image_sizetuple of int (image_height, image_width) or (image_height, image_width, n_channels) 

The size of the image that will be reconstructed.    Returns 
 
imagendarray of shape image_size 

The reconstructed image.     
 Examples using sklearn.feature_extraction.image.reconstruct_from_patches_2d
 
  Image denoising using dictionary learning
sklearn.feature_selection.chi2  
sklearn.feature_selection.chi2(X, y) [source]
 
Compute chi-squared stats between each non-negative feature and class. This score can be used to select the n_features features with the highest values for the test chi-squared statistic from X, which must contain only non-negative features such as booleans or frequencies (e.g., term counts in document classification), relative to the classes. Recall that the chi-square test measures dependence between stochastic variables, so using this function “weeds out” the features that are the most likely to be independent of class and therefore irrelevant for classification. Read more in the User Guide.  Parameters 
 
X{array-like, sparse matrix} of shape (n_samples, n_features) 

Sample vectors.  
yarray-like of shape (n_samples,) 

Target vector (class labels).    Returns 
 
chi2array, shape = (n_features,) 

chi2 statistics of each feature.  
pvalarray, shape = (n_features,) 

p-values of each feature.      See also  
f_classif


ANOVA F-value between label/feature for classification tasks.  
f_regression


F-value between label/feature for regression tasks.    Notes Complexity of this algorithm is O(n_classes * n_features). 
 Examples using sklearn.feature_selection.chi2
 
  Selecting dimensionality reduction with Pipeline and GridSearchCV  

  SVM-Anova: SVM with univariate feature selection  

  Classification of text documents using sparse features
sklearn.feature_selection.f_classif  
sklearn.feature_selection.f_classif(X, y) [source]
 
Compute the ANOVA F-value for the provided sample. Read more in the User Guide.  Parameters 
 
X{array-like, sparse matrix} shape = [n_samples, n_features] 

The set of regressors that will be tested sequentially.  
yarray of shape(n_samples) 

The data matrix.    Returns 
 
Farray, shape = [n_features,] 

The set of F values.  
pvalarray, shape = [n_features,] 

The set of p-values.      See also  
chi2


Chi-squared stats of non-negative features for classification tasks.  
f_regression


F-value between label/feature for regression tasks.    
 Examples using sklearn.feature_selection.f_classif
 
  Pipeline Anova SVM  

  Univariate Feature Selection
sklearn.feature_selection.f_regression  
sklearn.feature_selection.f_regression(X, y, *, center=True) [source]
 
Univariate linear regression tests. Linear model for testing the individual effect of each of many regressors. This is a scoring function to be used in a feature selection procedure, not a free standing feature selection procedure. This is done in 2 steps:  The correlation between each regressor and the target is computed, that is, ((X[:, i] - mean(X[:, i])) * (y - mean_y)) / (std(X[:, i]) * std(y)). It is converted to an F score then to a p-value.  For more on usage see the User Guide.  Parameters 
 
X{array-like, sparse matrix} shape = (n_samples, n_features) 

The set of regressors that will be tested sequentially.  
yarray of shape(n_samples). 

The data matrix  
centerbool, default=True 

If true, X and y will be centered.    Returns 
 
Farray, shape=(n_features,) 

F values of features.  
pvalarray, shape=(n_features,) 

p-values of F-scores.      See also  
mutual_info_regression


Mutual information for a continuous target.  
f_classif


ANOVA F-value between label/feature for classification tasks.  
chi2


Chi-squared stats of non-negative features for classification tasks.  
SelectKBest


Select features based on the k highest scores.  
SelectFpr


Select features based on a false positive rate test.  
SelectFdr


Select features based on an estimated false discovery rate.  
SelectFwe


Select features based on family-wise error rate.  
SelectPercentile


Select features based on percentile of the highest scores.    
 Examples using sklearn.feature_selection.f_regression
 
  Feature agglomeration vs. univariate selection  

  Comparison of F-test and mutual information
sklearn.feature_selection.mutual_info_classif  
sklearn.feature_selection.mutual_info_classif(X, y, *, discrete_features='auto', n_neighbors=3, copy=True, random_state=None) [source]
 
Estimate mutual information for a discrete target variable. Mutual information (MI) [1] between two random variables is a non-negative value, which measures the dependency between the variables. It is equal to zero if and only if two random variables are independent, and higher values mean higher dependency. The function relies on nonparametric methods based on entropy estimation from k-nearest neighbors distances as described in [2] and [3]. Both methods are based on the idea originally proposed in [4]. It can be used for univariate features selection, read more in the User Guide.  Parameters 
 
Xarray-like or sparse matrix, shape (n_samples, n_features) 

Feature matrix.  
yarray-like of shape (n_samples,) 

Target vector.  
discrete_features{‘auto’, bool, array-like}, default=’auto’ 

If bool, then determines whether to consider all features discrete or continuous. If array, then it should be either a boolean mask with shape (n_features,) or array with indices of discrete features. If ‘auto’, it is assigned to False for dense X and to True for sparse X.  
n_neighborsint, default=3 

Number of neighbors to use for MI estimation for continuous variables, see [2] and [3]. Higher values reduce variance of the estimation, but could introduce a bias.  
copybool, default=True 

Whether to make a copy of the given data. If set to False, the initial data will be overwritten.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for adding small noise to continuous variables in order to remove repeated values. Pass an int for reproducible results across multiple function calls. See Glossary.    Returns 
 
mindarray, shape (n_features,) 

Estimated mutual information between each feature and the target.     Notes  The term “discrete features” is used instead of naming them “categorical”, because it describes the essence more accurately. For example, pixel intensities of an image are discrete features (but hardly categorical) and you will get better results if mark them as such. Also note, that treating a continuous variable as discrete and vice versa will usually give incorrect results, so be attentive about that. True mutual information can’t be negative. If its estimate turns out to be negative, it is replaced by zero.  References  
1  
Mutual Information on Wikipedia.  
2(1,2)  
A. Kraskov, H. Stogbauer and P. Grassberger, “Estimating mutual information”. Phys. Rev. E 69, 2004.  
3(1,2)  
B. C. Ross “Mutual Information between Discrete and Continuous Data Sets”. PLoS ONE 9(2), 2014.  
4  
L. F. Kozachenko, N. N. Leonenko, “Sample Estimate of the Entropy of a Random Vector:, Probl. Peredachi Inf., 23:2 (1987), 9-16
sklearn.feature_selection.mutual_info_regression  
sklearn.feature_selection.mutual_info_regression(X, y, *, discrete_features='auto', n_neighbors=3, copy=True, random_state=None) [source]
 
Estimate mutual information for a continuous target variable. Mutual information (MI) [1] between two random variables is a non-negative value, which measures the dependency between the variables. It is equal to zero if and only if two random variables are independent, and higher values mean higher dependency. The function relies on nonparametric methods based on entropy estimation from k-nearest neighbors distances as described in [2] and [3]. Both methods are based on the idea originally proposed in [4]. It can be used for univariate features selection, read more in the User Guide.  Parameters 
 
Xarray-like or sparse matrix, shape (n_samples, n_features) 

Feature matrix.  
yarray-like of shape (n_samples,) 

Target vector.  
discrete_features{‘auto’, bool, array-like}, default=’auto’ 

If bool, then determines whether to consider all features discrete or continuous. If array, then it should be either a boolean mask with shape (n_features,) or array with indices of discrete features. If ‘auto’, it is assigned to False for dense X and to True for sparse X.  
n_neighborsint, default=3 

Number of neighbors to use for MI estimation for continuous variables, see [2] and [3]. Higher values reduce variance of the estimation, but could introduce a bias.  
copybool, default=True 

Whether to make a copy of the given data. If set to False, the initial data will be overwritten.  
random_stateint, RandomState instance or None, default=None 

Determines random number generation for adding small noise to continuous variables in order to remove repeated values. Pass an int for reproducible results across multiple function calls. See Glossary.    Returns 
 
mindarray, shape (n_features,) 

Estimated mutual information between each feature and the target.     Notes  The term “discrete features” is used instead of naming them “categorical”, because it describes the essence more accurately. For example, pixel intensities of an image are discrete features (but hardly categorical) and you will get better results if mark them as such. Also note, that treating a continuous variable as discrete and vice versa will usually give incorrect results, so be attentive about that. True mutual information can’t be negative. If its estimate turns out to be negative, it is replaced by zero.  References  
1  
Mutual Information on Wikipedia.  
2(1,2)  
A. Kraskov, H. Stogbauer and P. Grassberger, “Estimating mutual information”. Phys. Rev. E 69, 2004.  
3(1,2)  
B. C. Ross “Mutual Information between Discrete and Continuous Data Sets”. PLoS ONE 9(2), 2014.  
4  
L. F. Kozachenko, N. N. Leonenko, “Sample Estimate of the Entropy of a Random Vector”, Probl. Peredachi Inf., 23:2 (1987), 9-16   
 Examples using sklearn.feature_selection.mutual_info_regression
 
  Comparison of F-test and mutual information
sklearn.get_config  
sklearn.get_config() [source]
 
Retrieve current values for configuration set by set_config  Returns 
 
configdict 

Keys are parameter names that can be passed to set_config.      See also  
config_context


Context manager for global scikit-learn configuration.  
set_config


Set global scikit-learn configuration.
sklearn.isotonic.check_increasing  
sklearn.isotonic.check_increasing(x, y) [source]
 
Determine whether y is monotonically correlated with x. y is found increasing or decreasing with respect to x based on a Spearman correlation test.  Parameters 
 
xarray-like of shape (n_samples,) 

Training data.  
yarray-like of shape (n_samples,) 

Training target.    Returns 
 
increasing_boolboolean 

Whether the relationship is increasing or decreasing.     Notes The Spearman correlation coefficient is estimated from the data, and the sign of the resulting estimate is used as the result. In the event that the 95% confidence interval based on Fisher transform spans zero, a warning is raised. References Fisher transformation. Wikipedia. https://en.wikipedia.org/wiki/Fisher_transformation