sklearn.datasets.load_svmlight_file
sklearn.datasets.load_svmlight_file(f, *, n_features=None, dtype=<class 'numpy.float64'>, multilabel=False, zero_based='auto', query_id=False, offset=0, length=-1) [source]

Load datasets in the svmlight / libsvm format into sparse CSR matrix This format is a text-based format, with one sample per line. It does not store zero valued features hence is suitable for sparse dataset. The first element of each line can be used to store a target variable to predict. This format is used as the default format for both svmlight and the libsvm command line programs. Parsing a text based source can be expensive. When working on repeatedly on the same dataset, it is recommended to wrap this loader with joblib.Memory.cache to store a memmapped backup of the CSR results of the first call and benefit from the near instantaneous loading of memmapped structures for the subsequent calls. In case the file contains a pairwise preference constraint (known as “qid” in the svmlight format) these are ignored unless the query_id parameter is set to True. These pairwise preference constraints can be used to constraint the combination of samples when using pairwise loss functions (as is the case in some learning to rank problems) so that only pairs with the same query_id value are considered. This implementation is written in Cython and is reasonably fast. However, a faster API-compatible loader is also available at: https://github.com/mblondel/svmlight-loader  Parameters

fstr, file-like or int

(Path to) a file to load. If a path ends in “.gz” or “.bz2”, it will be uncompressed on the fly. If an integer is passed, it is assumed to be a file descriptor. A file-like or file descriptor will not be closed by this function. A file-like object must be opened in binary mode.
n_featuresint, default=None

The number of features to use. If None, it will be inferred. This argument is useful to load several files that are subsets of a bigger sliced dataset: each subset might not have examples of every feature, hence the inferred shape might vary from one slice to another. n_features is only required if offset or length are passed a non-default value.
dtypenumpy data type, default=np.float64

Data type of dataset to be loaded. This will be the data type of the output numpy arrays X and y.
multilabelbool, default=False

Samples may have several labels each (see https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/multilabel.html)
zero_basedbool or “auto”, default=”auto”

Whether column indices in f are zero-based (True) or one-based (False). If column indices are one-based, they are transformed to zero-based to match Python/NumPy conventions. If set to “auto”, a heuristic check is applied to determine this from the file contents. Both kinds of files occur “in the wild”, but they are unfortunately not self-identifying. Using “auto” or True should always be safe when no offset or length is passed. If offset or length are passed, the “auto” mode falls back to zero_based=True to avoid having the heuristic check yield inconsistent results on different segments of the file.
query_idbool, default=False

If True, will return the query_id array for each file.
offsetint, default=0

Ignore the offset first bytes by seeking forward, then discarding the following bytes up until the next new line character.
lengthint, default=-1

If strictly positive, stop reading any new line of data once the position in the file has reached the (offset + length) bytes threshold.    Returns

Xscipy.sparse matrix of shape (n_samples, n_features)

yndarray of shape (n_samples,), or, in the multilabel a list of

tuples of length n_samples.
query_idarray of shape (n_samples,)

query_id for each sample. Only returned when query_id is set to True.      See also
load_svmlight_files

Similar function for loading multiple files in this format, enforcing the same number of features/columns on all of them.    Examples To use joblib.Memory to cache the svmlight file: from joblib import Memory
from .datasets import load_svmlight_file
mem = Memory("./mycache")

@mem.cache
def get_data():
    data = load_svmlight_file("mysvmlightfile")
    return data[0], data[1]

X, y = get_data()


sklearn.datasets.load_svmlight_files
sklearn.datasets.load_svmlight_files(files, *, n_features=None, dtype=<class 'numpy.float64'>, multilabel=False, zero_based='auto', query_id=False, offset=0, length=-1) [source]

Load dataset from multiple files in SVMlight format This function is equivalent to mapping load_svmlight_file over a list of files, except that the results are concatenated into a single, flat list and the samples vectors are constrained to all have the same number of features. In case the file contains a pairwise preference constraint (known as “qid” in the svmlight format) these are ignored unless the query_id parameter is set to True. These pairwise preference constraints can be used to constraint the combination of samples when using pairwise loss functions (as is the case in some learning to rank problems) so that only pairs with the same query_id value are considered.  Parameters

filesarray-like, dtype=str, file-like or int

(Paths of) files to load. If a path ends in “.gz” or “.bz2”, it will be uncompressed on the fly. If an integer is passed, it is assumed to be a file descriptor. File-likes and file descriptors will not be closed by this function. File-like objects must be opened in binary mode.
n_featuresint, default=None

The number of features to use. If None, it will be inferred from the maximum column index occurring in any of the files. This can be set to a higher value than the actual number of features in any of the input files, but setting it to a lower value will cause an exception to be raised.
dtypenumpy data type, default=np.float64

Data type of dataset to be loaded. This will be the data type of the output numpy arrays X and y.
multilabelbool, default=False

Samples may have several labels each (see https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/multilabel.html)
zero_basedbool or “auto”, default=”auto”

Whether column indices in f are zero-based (True) or one-based (False). If column indices are one-based, they are transformed to zero-based to match Python/NumPy conventions. If set to “auto”, a heuristic check is applied to determine this from the file contents. Both kinds of files occur “in the wild”, but they are unfortunately not self-identifying. Using “auto” or True should always be safe when no offset or length is passed. If offset or length are passed, the “auto” mode falls back to zero_based=True to avoid having the heuristic check yield inconsistent results on different segments of the file.
query_idbool, default=False

If True, will return the query_id array for each file.
offsetint, default=0

Ignore the offset first bytes by seeking forward, then discarding the following bytes up until the next new line character.
lengthint, default=-1

If strictly positive, stop reading any new line of data once the position in the file has reached the (offset + length) bytes threshold.    Returns
 [X1, y1, …, Xn, yn]
where each (Xi, yi) pair is the result from load_svmlight_file(files[i]).
If query_id is set to True, this will return instead [X1, y1, q1,
…, Xn, yn, qn] where (Xi, yi, qi) is the result from
load_svmlight_file(files[i])
    See also
load_svmlight_file

  Notes When fitting a model to a matrix X_train and evaluating it against a matrix X_test, it is essential that X_train and X_test have the same number of features (X_train.shape[1] == X_test.shape[1]). This may not be the case if you load the files individually with load_svmlight_file.


sklearn.datasets.load_wine
sklearn.datasets.load_wine(*, return_X_y=False, as_frame=False) [source]

Load and return the wine dataset (classification).  New in version 0.18.  The wine dataset is a classic and very easy multi-class classification dataset.
Classes 3
Samples per class [59,71,48]
Samples total 178
Dimensionality 13
Features real, positive   Read more in the User Guide.  Parameters

return_X_ybool, default=False

If True, returns (data, target) instead of a Bunch object. See below for more information about the data and target object.
as_framebool, default=False

If True, the data is a pandas DataFrame including columns with appropriate dtypes (numeric). The target is a pandas DataFrame or Series depending on the number of target columns. If return_X_y is True, then (data, target) will be pandas DataFrames or Series as described below.  New in version 0.23.     Returns

dataBunch

Dictionary-like object, with the following attributes.
data{ndarray, dataframe} of shape (178, 13)

The data matrix. If as_frame=True, data will be a pandas DataFrame.  target: {ndarray, Series} of shape (178,)

The classification target. If as_frame=True, target will be a pandas Series.  feature_names: list

The names of the dataset columns.  target_names: list

The names of target classes.  frame: DataFrame of shape (178, 14)

Only present when as_frame=True. DataFrame with data and target.  New in version 0.23.   DESCR: str

The full description of the dataset.
(data, target)tuple if return_X_y is True
The copy of UCI ML Wine Data Set dataset is downloaded and modified to fit
standard format from:
https://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data
   Examples Let’s say you are interested in the samples 10, 80, and 140, and want to know their class name. >>> from sklearn.datasets import load_wine
>>> data = load_wine()
>>> data.target[[10, 80, 140]]
array([0, 1, 2])
>>> list(data.target_names)
['class_0', 'class_1', 'class_2']

 Examples using sklearn.datasets.load_wine

  Outlier detection on a real data set

  ROC Curve with Visualization API

  Importance of Feature Scaling


sklearn.datasets.make_biclusters
sklearn.datasets.make_biclusters(shape, n_clusters, *, noise=0.0, minval=10, maxval=100, shuffle=True, random_state=None) [source]

Generate an array with constant block diagonal structure for biclustering. Read more in the User Guide.  Parameters

shapeiterable of shape (n_rows, n_cols)

The shape of the result.
n_clustersint

The number of biclusters.
noisefloat, default=0.0

The standard deviation of the gaussian noise.
minvalint, default=10

Minimum value of a bicluster.
maxvalint, default=100

Maximum value of a bicluster.
shufflebool, default=True

Shuffle the samples.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape shape

The generated array.
rowsndarray of shape (n_clusters, X.shape[0])

The indicators for cluster membership of each row.
colsndarray of shape (n_clusters, X.shape[1])

The indicators for cluster membership of each column.      See also
make_checkerboard

  References
1
Dhillon, I. S. (2001, August). Co-clustering documents and words using bipartite spectral graph partitioning. In Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 269-274). ACM.
 Examples using sklearn.datasets.make_biclusters

  A demo of the Spectral Co-Clustering algorithm


sklearn.datasets.make_blobs
sklearn.datasets.make_blobs(n_samples=100, n_features=2, *, centers=None, cluster_std=1.0, center_box=- 10.0, 10.0, shuffle=True, random_state=None, return_centers=False) [source]

Generate isotropic Gaussian blobs for clustering. Read more in the User Guide.  Parameters

n_samplesint or array-like, default=100

If int, it is the total number of points equally divided among clusters. If array-like, each element of the sequence indicates the number of samples per cluster.  Changed in version v0.20: one can now pass an array-like to the n_samples parameter
n_featuresint, default=2

The number of features for each sample.
centersint or ndarray of shape (n_centers, n_features), default=None

The number of centers to generate, or the fixed center locations. If n_samples is an int and centers is None, 3 centers are generated. If n_samples is array-like, centers must be either None or an array of length equal to the length of n_samples.
cluster_stdfloat or array-like of float, default=1.0

The standard deviation of the clusters.
center_boxtuple of float (min, max), default=(-10.0, 10.0)

The bounding box for each cluster center when centers are generated at random.
shufflebool, default=True

Shuffle the samples.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.
return_centersbool, default=False

If True, then return the centers of each cluster  New in version 0.23.     Returns

Xndarray of shape (n_samples, n_features)

The generated samples.
yndarray of shape (n_samples,)

The integer labels for cluster membership of each sample.
centersndarray of shape (n_centers, n_features)

The centers of each cluster. Only returned if return_centers=True.      See also
make_classification

A more intricate variant.    Examples >>> from sklearn.datasets import make_blobs
>>> X, y = make_blobs(n_samples=10, centers=3, n_features=2,
...                   random_state=0)
>>> print(X.shape)
(10, 2)
>>> y
array([0, 0, 1, 0, 2, 2, 2, 1, 1, 0])
>>> X, y = make_blobs(n_samples=[3, 3, 4], centers=None, n_features=2,
...                   random_state=0)
>>> print(X.shape)
(10, 2)
>>> y
array([0, 1, 2, 0, 2, 2, 2, 1, 1, 0])

 Examples using sklearn.datasets.make_blobs

  Release Highlights for scikit-learn 0.23

  Probability calibration of classifiers

  Probability Calibration for 3-class classification

  Normal, Ledoit-Wolf and OAS Linear Discriminant Analysis for classification

  An example of K-Means++ initialization

  A demo of the mean-shift clustering algorithm

  Demonstration of k-means assumptions

  Demo of affinity propagation clustering algorithm

  Demo of DBSCAN clustering algorithm

  Inductive Clustering

  Compare BIRCH and MiniBatchKMeans

  Comparison of the K-Means and MiniBatchKMeans clustering algorithms

  Comparing different hierarchical linkage methods on toy datasets

  Selecting the number of clusters with silhouette analysis on KMeans clustering

  Comparing different clustering algorithms on toy datasets

  Plot randomly generated classification dataset

  SGD: Maximum margin separating hyperplane

  Plot multinomial and One-vs-Rest Logistic Regression

  Comparing anomaly detection algorithms for outlier detection on toy datasets

  Demonstrating the different strategies of KBinsDiscretizer

  SVM: Maximum margin separating hyperplane

  SVM Tie Breaking Example

  Plot the support vectors in LinearSVC

  SVM: Separating hyperplane for unbalanced classes


sklearn.datasets.make_checkerboard
sklearn.datasets.make_checkerboard(shape, n_clusters, *, noise=0.0, minval=10, maxval=100, shuffle=True, random_state=None) [source]

Generate an array with block checkerboard structure for biclustering. Read more in the User Guide.  Parameters

shapetuple of shape (n_rows, n_cols)

The shape of the result.
n_clustersint or array-like or shape (n_row_clusters, n_column_clusters)

The number of row and column clusters.
noisefloat, default=0.0

The standard deviation of the gaussian noise.
minvalint, default=10

Minimum value of a bicluster.
maxvalint, default=100

Maximum value of a bicluster.
shufflebool, default=True

Shuffle the samples.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape shape

The generated array.
rowsndarray of shape (n_clusters, X.shape[0])

The indicators for cluster membership of each row.
colsndarray of shape (n_clusters, X.shape[1])

The indicators for cluster membership of each column.      See also
make_biclusters

  References
1
Kluger, Y., Basri, R., Chang, J. T., & Gerstein, M. (2003). Spectral biclustering of microarray data: coclustering genes and conditions. Genome research, 13(4), 703-716.
 Examples using sklearn.datasets.make_checkerboard

  A demo of the Spectral Biclustering algorithm


sklearn.datasets.make_circles
sklearn.datasets.make_circles(n_samples=100, *, shuffle=True, noise=None, random_state=None, factor=0.8) [source]

Make a large circle containing a smaller circle in 2d. A simple toy dataset to visualize clustering and classification algorithms. Read more in the User Guide.  Parameters

n_samplesint or tuple of shape (2,), dtype=int, default=100

If int, it is the total number of points generated. For odd numbers, the inner circle will have one point more than the outer circle. If two-element tuple, number of points in outer circle and inner circle.  Changed in version 0.23: Added two-element tuple.
shufflebool, default=True

Whether to shuffle the samples.
noisefloat, default=None

Standard deviation of Gaussian noise added to the data.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset shuffling and noise. Pass an int for reproducible output across multiple function calls. See Glossary.
factorfloat, default=.8

Scale factor between inner and outer circle in the range (0, 1).    Returns

Xndarray of shape (n_samples, 2)

The generated samples.
yndarray of shape (n_samples,)

The integer labels (0 or 1) for class membership of each sample.
 Examples using sklearn.datasets.make_circles

  Classifier comparison

  Comparing different hierarchical linkage methods on toy datasets

  Comparing different clustering algorithms on toy datasets

  Kernel PCA

  Hashing feature transformation using Totally Random Trees

  t-SNE: The effect of various perplexity values on the shape

  Compare Stochastic learning strategies for MLPClassifier

  Varying regularization in Multi-layer Perceptron

  Feature discretization

  Label Propagation learning a complex structure


sklearn.datasets.make_classification
sklearn.datasets.make_classification(n_samples=100, n_features=20, *, n_informative=2, n_redundant=2, n_repeated=0, n_classes=2, n_clusters_per_class=2, weights=None, flip_y=0.01, class_sep=1.0, hypercube=True, shift=0.0, scale=1.0, shuffle=True, random_state=None) [source]

Generate a random n-class classification problem. This initially creates clusters of points normally distributed (std=1) about vertices of an n_informative-dimensional hypercube with sides of length 2*class_sep and assigns an equal number of clusters to each class. It introduces interdependence between these features and adds various types of further noise to the data. Without shuffling, X horizontally stacks features in the following order: the primary n_informative features, followed by n_redundant linear combinations of the informative features, followed by n_repeated duplicates, drawn randomly with replacement from the informative and redundant features. The remaining features are filled with random noise. Thus, without shuffling, all useful features are contained in the columns X[:, :n_informative + n_redundant + n_repeated]. Read more in the User Guide.  Parameters

n_samplesint, default=100

The number of samples.
n_featuresint, default=20

The total number of features. These comprise n_informative informative features, n_redundant redundant features, n_repeated duplicated features and n_features-n_informative-n_redundant-n_repeated useless features drawn at random.
n_informativeint, default=2

The number of informative features. Each class is composed of a number of gaussian clusters each located around the vertices of a hypercube in a subspace of dimension n_informative. For each cluster, informative features are drawn independently from N(0, 1) and then randomly linearly combined within each cluster in order to add covariance. The clusters are then placed on the vertices of the hypercube.
n_redundantint, default=2

The number of redundant features. These features are generated as random linear combinations of the informative features.
n_repeatedint, default=0

The number of duplicated features, drawn randomly from the informative and the redundant features.
n_classesint, default=2

The number of classes (or labels) of the classification problem.
n_clusters_per_classint, default=2

The number of clusters per class.
weightsarray-like of shape (n_classes,) or (n_classes - 1,), default=None

The proportions of samples assigned to each class. If None, then classes are balanced. Note that if len(weights) == n_classes - 1, then the last class weight is automatically inferred. More than n_samples samples may be returned if the sum of weights exceeds 1. Note that the actual class proportions will not exactly match weights when flip_y isn’t 0.
flip_yfloat, default=0.01

The fraction of samples whose class is assigned randomly. Larger values introduce noise in the labels and make the classification task harder. Note that the default setting flip_y > 0 might lead to less than n_classes in y in some cases.
class_sepfloat, default=1.0

The factor multiplying the hypercube size. Larger values spread out the clusters/classes and make the classification task easier.
hypercubebool, default=True

If True, the clusters are put on the vertices of a hypercube. If False, the clusters are put on the vertices of a random polytope.
shiftfloat, ndarray of shape (n_features,) or None, default=0.0

Shift features by the specified value. If None, then features are shifted by a random value drawn in [-class_sep, class_sep].
scalefloat, ndarray of shape (n_features,) or None, default=1.0

Multiply features by the specified value. If None, then features are scaled by a random value drawn in [1, 100]. Note that scaling happens after shifting.
shufflebool, default=True

Shuffle the samples and the features.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape (n_samples, n_features)

The generated samples.
yndarray of shape (n_samples,)

The integer labels for class membership of each sample.      See also
make_blobs

Simplified variant.
make_multilabel_classification

Unrelated generator for multilabel tasks.    Notes The algorithm is adapted from Guyon [1] and was designed to generate the “Madelon” dataset. References
1
I. Guyon, “Design of experiments for the NIPS 2003 variable selection benchmark”, 2003.
 Examples using sklearn.datasets.make_classification

  Release Highlights for scikit-learn 0.24

  Release Highlights for scikit-learn 0.22

  Comparison of Calibration of Classifiers

  Probability Calibration curves

  Classifier comparison

  Plot randomly generated classification dataset

  Feature importances with forests of trees

  OOB Errors for Random Forests

  Feature transformations with ensembles of trees

  Pipeline Anova SVM

  Recursive feature elimination with cross-validation

  Detection error tradeoff (DET) curve

  Successive Halving Iterations

  Comparison between grid search and successive halving

  Neighborhood Components Analysis Illustration

  Varying regularization in Multi-layer Perceptron

  Feature discretization

  Scaling the regularization parameter for SVCs


sklearn.datasets.make_friedman1
sklearn.datasets.make_friedman1(n_samples=100, n_features=10, *, noise=0.0, random_state=None) [source]

Generate the “Friedman #1” regression problem. This dataset is described in Friedman [1] and Breiman [2]. Inputs X are independent features uniformly distributed on the interval [0, 1]. The output y is created according to the formula: y(X) = 10 * sin(pi * X[:, 0] * X[:, 1]) + 20 * (X[:, 2] - 0.5) ** 2 + 10 * X[:, 3] + 5 * X[:, 4] + noise * N(0, 1).
 Out of the n_features features, only 5 are actually used to compute y. The remaining features are independent of y. The number of features has to be >= 5. Read more in the User Guide.  Parameters

n_samplesint, default=100

The number of samples.
n_featuresint, default=10

The number of features. Should be at least 5.
noisefloat, default=0.0

The standard deviation of the gaussian noise applied to the output.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset noise. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape (n_samples, n_features)

The input samples.
yndarray of shape (n_samples,)

The output values.     References
1
J. Friedman, “Multivariate adaptive regression splines”, The Annals of Statistics 19 (1), pages 1-67, 1991.
2
L. Breiman, “Bagging predictors”, Machine Learning 24, pages 123-140, 1996.


sklearn.datasets.make_friedman2
sklearn.datasets.make_friedman2(n_samples=100, *, noise=0.0, random_state=None) [source]

Generate the “Friedman #2” regression problem. This dataset is described in Friedman [1] and Breiman [2]. Inputs X are 4 independent features uniformly distributed on the intervals: 0 <= X[:, 0] <= 100,
40 * pi <= X[:, 1] <= 560 * pi,
0 <= X[:, 2] <= 1,
1 <= X[:, 3] <= 11.
 The output y is created according to the formula: y(X) = (X[:, 0] ** 2 + (X[:, 1] * X[:, 2]  - 1 / (X[:, 1] * X[:, 3])) ** 2) ** 0.5 + noise * N(0, 1).
 Read more in the User Guide.  Parameters

n_samplesint, default=100

The number of samples.
noisefloat, default=0.0

The standard deviation of the gaussian noise applied to the output.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset noise. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape (n_samples, 4)

The input samples.
yndarray of shape (n_samples,)

The output values.     References
1
J. Friedman, “Multivariate adaptive regression splines”, The Annals of Statistics 19 (1), pages 1-67, 1991.
2
L. Breiman, “Bagging predictors”, Machine Learning 24, pages 123-140, 1996.


sklearn.datasets.make_friedman3
sklearn.datasets.make_friedman3(n_samples=100, *, noise=0.0, random_state=None) [source]

Generate the “Friedman #3” regression problem. This dataset is described in Friedman [1] and Breiman [2]. Inputs X are 4 independent features uniformly distributed on the intervals: 0 <= X[:, 0] <= 100,
40 * pi <= X[:, 1] <= 560 * pi,
0 <= X[:, 2] <= 1,
1 <= X[:, 3] <= 11.
 The output y is created according to the formula: y(X) = arctan((X[:, 1] * X[:, 2] - 1 / (X[:, 1] * X[:, 3])) / X[:, 0]) + noise * N(0, 1).
 Read more in the User Guide.  Parameters

n_samplesint, default=100

The number of samples.
noisefloat, default=0.0

The standard deviation of the gaussian noise applied to the output.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset noise. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape (n_samples, 4)

The input samples.
yndarray of shape (n_samples,)

The output values.     References
1
J. Friedman, “Multivariate adaptive regression splines”, The Annals of Statistics 19 (1), pages 1-67, 1991.
2
L. Breiman, “Bagging predictors”, Machine Learning 24, pages 123-140, 1996.


sklearn.datasets.make_gaussian_quantiles
sklearn.datasets.make_gaussian_quantiles(*, mean=None, cov=1.0, n_samples=100, n_features=2, n_classes=3, shuffle=True, random_state=None) [source]

Generate isotropic Gaussian and label samples by quantile. This classification dataset is constructed by taking a multi-dimensional standard normal distribution and defining classes separated by nested concentric multi-dimensional spheres such that roughly equal numbers of samples are in each class (quantiles of the \(\chi^2\) distribution). Read more in the User Guide.  Parameters

meanndarray of shape (n_features,), default=None

The mean of the multi-dimensional normal distribution. If None then use the origin (0, 0, …).
covfloat, default=1.0

The covariance matrix will be this value times the unit matrix. This dataset only produces symmetric normal distributions.
n_samplesint, default=100

The total number of points equally divided among classes.
n_featuresint, default=2

The number of features for each sample.
n_classesint, default=3

The number of classes
shufflebool, default=True

Shuffle the samples.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape (n_samples, n_features)

The generated samples.
yndarray of shape (n_samples,)

The integer labels for quantile membership of each sample.     Notes The dataset is from Zhu et al [1]. References
1
 Zhu, H. Zou, S. Rosset, T. Hastie, “Multi-class AdaBoost”, 2009.
 Examples using sklearn.datasets.make_gaussian_quantiles

  Plot randomly generated classification dataset

  Two-class AdaBoost

  Multi-class AdaBoosted Decision Trees


sklearn.datasets.make_hastie_10_2
sklearn.datasets.make_hastie_10_2(n_samples=12000, *, random_state=None) [source]

Generates data for binary classification used in Hastie et al. 2009, Example 10.2. The ten features are standard independent Gaussian and the target y is defined by: y[i] = 1 if np.sum(X[i] ** 2) > 9.34 else -1
 Read more in the User Guide.  Parameters

n_samplesint, default=12000

The number of samples.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape (n_samples, 10)

The input samples.
yndarray of shape (n_samples,)

The output values.      See also
make_gaussian_quantiles

A generalization of this dataset approach.    References
1
T. Hastie, R. Tibshirani and J. Friedman, “Elements of Statistical Learning Ed. 2”, Springer, 2009.
 Examples using sklearn.datasets.make_hastie_10_2

  Gradient Boosting regularization

  Discrete versus Real AdaBoost

  Early stopping of Gradient Boosting

  Demonstration of multi-metric evaluation on cross_val_score and GridSearchCV


sklearn.datasets.make_low_rank_matrix
sklearn.datasets.make_low_rank_matrix(n_samples=100, n_features=100, *, effective_rank=10, tail_strength=0.5, random_state=None) [source]

Generate a mostly low rank matrix with bell-shaped singular values. Most of the variance can be explained by a bell-shaped curve of width effective_rank: the low rank part of the singular values profile is: (1 - tail_strength) * exp(-1.0 * (i / effective_rank) ** 2)
 The remaining singular values’ tail is fat, decreasing as: tail_strength * exp(-0.1 * i / effective_rank).
 The low rank part of the profile can be considered the structured signal part of the data while the tail can be considered the noisy part of the data that cannot be summarized by a low number of linear components (singular vectors).  This kind of singular profiles is often seen in practice, for instance:

 gray level pictures of faces TF-IDF vectors of text documents crawled from the web    Read more in the User Guide.  Parameters

n_samplesint, default=100

The number of samples.
n_featuresint, default=100

The number of features.
effective_rankint, default=10

The approximate number of singular vectors required to explain most of the data by linear combinations.
tail_strengthfloat, default=0.5

The relative importance of the fat noisy tail of the singular values profile. The value should be between 0 and 1.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape (n_samples, n_features)

The matrix.


sklearn.datasets.make_moons
sklearn.datasets.make_moons(n_samples=100, *, shuffle=True, noise=None, random_state=None) [source]

Make two interleaving half circles. A simple toy dataset to visualize clustering and classification algorithms. Read more in the User Guide.  Parameters

n_samplesint or tuple of shape (2,), dtype=int, default=100

If int, the total number of points generated. If two-element tuple, number of points in each of two moons.  Changed in version 0.23: Added two-element tuple.
shufflebool, default=True

Whether to shuffle the samples.
noisefloat, default=None

Standard deviation of Gaussian noise added to the data.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset shuffling and noise. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape (n_samples, 2)

The generated samples.
yndarray of shape (n_samples,)

The integer labels (0 or 1) for class membership of each sample.
 Examples using sklearn.datasets.make_moons

  Classifier comparison

  Comparing different hierarchical linkage methods on toy datasets

  Comparing different clustering algorithms on toy datasets

  Comparing anomaly detection algorithms for outlier detection on toy datasets

  Statistical comparison of models using grid search

  Compare Stochastic learning strategies for MLPClassifier

  Varying regularization in Multi-layer Perceptron

  Feature discretization


sklearn.datasets.make_multilabel_classification
sklearn.datasets.make_multilabel_classification(n_samples=100, n_features=20, *, n_classes=5, n_labels=2, length=50, allow_unlabeled=True, sparse=False, return_indicator='dense', return_distributions=False, random_state=None) [source]

Generate a random multilabel classification problem.  For each sample, the generative process is:

 pick the number of labels: n ~ Poisson(n_labels) n times, choose a class c: c ~ Multinomial(theta) pick the document length: k ~ Poisson(length) k times, choose a word: w ~ Multinomial(theta_c)    In the above process, rejection sampling is used to make sure that n is never zero or more than n_classes, and that the document length is never zero. Likewise, we reject classes which have already been chosen. Read more in the User Guide.  Parameters

n_samplesint, default=100

The number of samples.
n_featuresint, default=20

The total number of features.
n_classesint, default=5

The number of classes of the classification problem.
n_labelsint, default=2

The average number of labels per instance. More precisely, the number of labels per sample is drawn from a Poisson distribution with n_labels as its expected value, but samples are bounded (using rejection sampling) by n_classes, and must be nonzero if allow_unlabeled is False.
lengthint, default=50

The sum of the features (number of words if documents) is drawn from a Poisson distribution with this expected value.
allow_unlabeledbool, default=True

If True, some instances might not belong to any class.
sparsebool, default=False

If True, return a sparse feature matrix  New in version 0.17: parameter to allow sparse output.
return_indicator{‘dense’, ‘sparse’} or False, default=’dense’

If 'dense' return Y in the dense binary indicator format. If 'sparse' return Y in the sparse binary indicator format. False returns a list of lists of labels.
return_distributionsbool, default=False

If True, return the prior class probability and conditional probabilities of features given classes, from which the data was drawn.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape (n_samples, n_features)

The generated samples.
Y{ndarray, sparse matrix} of shape (n_samples, n_classes)

The label sets. Sparse matrix should be of CSR format.
p_cndarray of shape (n_classes,)

The probability of each class being drawn. Only returned if return_distributions=True.
p_w_cndarray of shape (n_features, n_classes)

The probability of each feature being drawn given each class. Only returned if return_distributions=True.
 Examples using sklearn.datasets.make_multilabel_classification

  Plot randomly generated multilabel dataset

  Multilabel classification


sklearn.datasets.make_regression
sklearn.datasets.make_regression(n_samples=100, n_features=100, *, n_informative=10, n_targets=1, bias=0.0, effective_rank=None, tail_strength=0.5, noise=0.0, shuffle=True, coef=False, random_state=None) [source]

Generate a random regression problem. The input set can either be well conditioned (by default) or have a low rank-fat tail singular profile. See make_low_rank_matrix for more details. The output is generated by applying a (potentially biased) random linear regression model with n_informative nonzero regressors to the previously generated input and some gaussian centered noise with some adjustable scale. Read more in the User Guide.  Parameters

n_samplesint, default=100

The number of samples.
n_featuresint, default=100

The number of features.
n_informativeint, default=10

The number of informative features, i.e., the number of features used to build the linear model used to generate the output.
n_targetsint, default=1

The number of regression targets, i.e., the dimension of the y output vector associated with a sample. By default, the output is a scalar.
biasfloat, default=0.0

The bias term in the underlying linear model.
effective_rankint, default=None

 if not None:

The approximate number of singular vectors required to explain most of the input data by linear combinations. Using this kind of singular spectrum in the input allows the generator to reproduce the correlations often observed in practice.  if None:

The input set is well conditioned, centered and gaussian with unit variance.
tail_strengthfloat, default=0.5

The relative importance of the fat noisy tail of the singular values profile if effective_rank is not None. When a float, it should be between 0 and 1.
noisefloat, default=0.0

The standard deviation of the gaussian noise applied to the output.
shufflebool, default=True

Shuffle the samples and the features.
coefbool, default=False

If True, the coefficients of the underlying linear model are returned.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape (n_samples, n_features)

The input samples.
yndarray of shape (n_samples,) or (n_samples, n_targets)

The output values.
coefndarray of shape (n_features,) or (n_features, n_targets)

The coefficient of the underlying linear model. It is returned only if coef is True.
 Examples using sklearn.datasets.make_regression

  Release Highlights for scikit-learn 0.23

  Prediction Latency

  Plot Ridge coefficients as a function of the L2 regularization

  Robust linear model estimation using RANSAC

  HuberRegressor vs Ridge on dataset with strong outliers

  Lasso on dense and sparse data

  Effect of transforming the targets in regression model


sklearn.datasets.make_sparse_coded_signal
sklearn.datasets.make_sparse_coded_signal(n_samples, *, n_components, n_features, n_nonzero_coefs, random_state=None) [source]

Generate a signal as a sparse combination of dictionary elements. Returns a matrix Y = DX, such as D is (n_features, n_components), X is (n_components, n_samples) and each column of X has exactly n_nonzero_coefs non-zero elements. Read more in the User Guide.  Parameters

n_samplesint

Number of samples to generate
n_componentsint

Number of components in the dictionary
n_featuresint

Number of features of the dataset to generate
n_nonzero_coefsint

Number of active (non-zero) coefficients in each sample
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

datandarray of shape (n_features, n_samples)

The encoded signal (Y).
dictionaryndarray of shape (n_features, n_components)

The dictionary with normalized components (D).
codendarray of shape (n_components, n_samples)

The sparse code such that each column of this matrix has exactly n_nonzero_coefs non-zero items (X).
 Examples using sklearn.datasets.make_sparse_coded_signal

  Orthogonal Matching Pursuit


sklearn.datasets.make_sparse_spd_matrix
sklearn.datasets.make_sparse_spd_matrix(dim=1, *, alpha=0.95, norm_diag=False, smallest_coef=0.1, largest_coef=0.9, random_state=None) [source]

Generate a sparse symmetric definite positive matrix. Read more in the User Guide.  Parameters

dimint, default=1

The size of the random matrix to generate.
alphafloat, default=0.95

The probability that a coefficient is zero (see notes). Larger values enforce more sparsity. The value should be in the range 0 and 1.
norm_diagbool, default=False

Whether to normalize the output matrix to make the leading diagonal elements all 1
smallest_coeffloat, default=0.1

The value of the smallest coefficient between 0 and 1.
largest_coeffloat, default=0.9

The value of the largest coefficient between 0 and 1.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

precsparse matrix of shape (dim, dim)

The generated matrix.      See also
make_spd_matrix

  Notes The sparsity is actually imposed on the cholesky factor of the matrix. Thus alpha does not translate directly into the filling fraction of the matrix itself.
 Examples using sklearn.datasets.make_sparse_spd_matrix

  Sparse inverse covariance estimation


sklearn.datasets.make_sparse_uncorrelated
sklearn.datasets.make_sparse_uncorrelated(n_samples=100, n_features=10, *, random_state=None) [source]

Generate a random regression problem with sparse uncorrelated design. This dataset is described in Celeux et al [1]. as: X ~ N(0, 1)
y(X) = X[:, 0] + 2 * X[:, 1] - 2 * X[:, 2] - 1.5 * X[:, 3]
 Only the first 4 features are informative. The remaining features are useless. Read more in the User Guide.  Parameters

n_samplesint, default=100

The number of samples.
n_featuresint, default=10

The number of features.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape (n_samples, n_features)

The input samples.
yndarray of shape (n_samples,)

The output values.     References
1
G. Celeux, M. El Anbari, J.-M. Marin, C. P. Robert, “Regularization in regression: comparing Bayesian and frequentist methods in a poorly informative situation”, 2009.


sklearn.datasets.make_spd_matrix
sklearn.datasets.make_spd_matrix(n_dim, *, random_state=None) [source]

Generate a random symmetric, positive-definite matrix. Read more in the User Guide.  Parameters

n_dimint

The matrix dimension.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape (n_dim, n_dim)

The random symmetric, positive-definite matrix.      See also
make_sparse_spd_matrix


sklearn.datasets.make_swiss_roll
sklearn.datasets.make_swiss_roll(n_samples=100, *, noise=0.0, random_state=None) [source]

Generate a swiss roll dataset. Read more in the User Guide.  Parameters

n_samplesint, default=100

The number of sample points on the S curve.
noisefloat, default=0.0

The standard deviation of the gaussian noise.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape (n_samples, 3)

The points.
tndarray of shape (n_samples,)

The univariate position of the sample according to the main dimension of the points in the manifold.     Notes The algorithm is from Marsland [1]. References
1
S. Marsland, “Machine Learning: An Algorithmic Perspective”, Chapter 10, 2009. http://seat.massey.ac.nz/personal/s.r.marsland/Code/10/lle.py
 Examples using sklearn.datasets.make_swiss_roll

  Hierarchical clustering: structured vs unstructured ward

  Swiss Roll reduction with LLE


sklearn.datasets.make_s_curve
sklearn.datasets.make_s_curve(n_samples=100, *, noise=0.0, random_state=None) [source]

Generate an S curve dataset. Read more in the User Guide.  Parameters

n_samplesint, default=100

The number of sample points on the S curve.
noisefloat, default=0.0

The standard deviation of the gaussian noise.
random_stateint, RandomState instance or None, default=None

Determines random number generation for dataset creation. Pass an int for reproducible output across multiple function calls. See Glossary.    Returns

Xndarray of shape (n_samples, 3)

The points.
tndarray of shape (n_samples,)

The univariate position of the sample according to the main dimension of the points in the manifold.
 Examples using sklearn.datasets.make_s_curve

  Comparison of Manifold Learning methods

  t-SNE: The effect of various perplexity values on the shape


sklearn.decomposition.dict_learning
sklearn.decomposition.dict_learning(X, n_components, *, alpha, max_iter=100, tol=1e-08, method='lars', n_jobs=None, dict_init=None, code_init=None, callback=None, verbose=False, random_state=None, return_n_iter=False, positive_dict=False, positive_code=False, method_max_iter=1000) [source]

Solves a dictionary learning matrix factorization problem. Finds the best dictionary and the corresponding sparse code for approximating the data matrix X by solving: (U^*, V^*) = argmin 0.5 || X - U V ||_2^2 + alpha * || U ||_1
             (U,V)
            with || V_k ||_2 = 1 for all  0 <= k < n_components
 where V is the dictionary and U is the sparse code. Read more in the User Guide.  Parameters

Xndarray of shape (n_samples, n_features)

Data matrix.
n_componentsint

Number of dictionary atoms to extract.
alphaint

Sparsity controlling parameter.
max_iterint, default=100

Maximum number of iterations to perform.
tolfloat, default=1e-8

Tolerance for the stopping condition.
method{‘lars’, ‘cd’}, default=’lars’

The method used:

'lars': uses the least angle regression method to solve the lasso

problem (linear_model.lars_path);
'cd': uses the coordinate descent method to compute the Lasso solution (linear_model.Lasso). Lars will be faster if the estimated components are sparse.
n_jobsint, default=None

Number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
dict_initndarray of shape (n_components, n_features), default=None

Initial value for the dictionary for warm restart scenarios.
code_initndarray of shape (n_samples, n_components), default=None

Initial value for the sparse code for warm restart scenarios.
callbackcallable, default=None

Callable that gets invoked every five iterations
verbosebool, default=False

To control the verbosity of the procedure.
random_stateint, RandomState instance or None, default=None

Used for randomly initializing the dictionary. Pass an int for reproducible results across multiple function calls. See Glossary.
return_n_iterbool, default=False

Whether or not to return the number of iterations.
positive_dictbool, default=False

Whether to enforce positivity when finding the dictionary.  New in version 0.20.
positive_codebool, default=False

Whether to enforce positivity when finding the code.  New in version 0.20.
method_max_iterint, default=1000

Maximum number of iterations to perform.  New in version 0.22.     Returns

codendarray of shape (n_samples, n_components)

The sparse code factor in the matrix factorization.
dictionaryndarray of shape (n_components, n_features),

The dictionary factor in the matrix factorization.
errorsarray

Vector of errors at each iteration.
n_iterint

Number of iterations run. Returned only if return_n_iter is set to True.      See also
dict_learning_online

DictionaryLearning

MiniBatchDictionaryLearning

SparsePCA

MiniBatchSparsePCA


sklearn.decomposition.dict_learning_online
sklearn.decomposition.dict_learning_online(X, n_components=2, *, alpha=1, n_iter=100, return_code=True, dict_init=None, callback=None, batch_size=3, verbose=False, shuffle=True, n_jobs=None, method='lars', iter_offset=0, random_state=None, return_inner_stats=False, inner_stats=None, return_n_iter=False, positive_dict=False, positive_code=False, method_max_iter=1000) [source]

Solves a dictionary learning matrix factorization problem online. Finds the best dictionary and the corresponding sparse code for approximating the data matrix X by solving: (U^*, V^*) = argmin 0.5 || X - U V ||_2^2 + alpha * || U ||_1
             (U,V)
             with || V_k ||_2 = 1 for all  0 <= k < n_components
 where V is the dictionary and U is the sparse code. This is accomplished by repeatedly iterating over mini-batches by slicing the input data. Read more in the User Guide.  Parameters

Xndarray of shape (n_samples, n_features)

Data matrix.
n_componentsint, default=2

Number of dictionary atoms to extract.
alphafloat, default=1

Sparsity controlling parameter.
n_iterint, default=100

Number of mini-batch iterations to perform.
return_codebool, default=True

Whether to also return the code U or just the dictionary V.
dict_initndarray of shape (n_components, n_features), default=None

Initial value for the dictionary for warm restart scenarios.
callbackcallable, default=None

callable that gets invoked every five iterations.
batch_sizeint, default=3

The number of samples to take in each batch.
verbosebool, default=False

To control the verbosity of the procedure.
shufflebool, default=True

Whether to shuffle the data before splitting it in batches.
n_jobsint, default=None

Number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
method{‘lars’, ‘cd’}, default=’lars’

'lars': uses the least angle regression method to solve the lasso problem (linear_model.lars_path);
'cd': uses the coordinate descent method to compute the Lasso solution (linear_model.Lasso). Lars will be faster if the estimated components are sparse.
iter_offsetint, default=0

Number of previous iterations completed on the dictionary used for initialization.
random_stateint, RandomState instance or None, default=None

Used for initializing the dictionary when dict_init is not specified, randomly shuffling the data when shuffle is set to True, and updating the dictionary. Pass an int for reproducible results across multiple function calls. See Glossary.
return_inner_statsbool, default=False

Return the inner statistics A (dictionary covariance) and B (data approximation). Useful to restart the algorithm in an online setting. If return_inner_stats is True, return_code is ignored.
inner_statstuple of (A, B) ndarrays, default=None

Inner sufficient statistics that are kept by the algorithm. Passing them at initialization is useful in online settings, to avoid losing the history of the evolution. A (n_components, n_components) is the dictionary covariance matrix. B (n_features, n_components) is the data approximation matrix.
return_n_iterbool, default=False

Whether or not to return the number of iterations.
positive_dictbool, default=False

Whether to enforce positivity when finding the dictionary.  New in version 0.20.
positive_codebool, default=False

Whether to enforce positivity when finding the code.  New in version 0.20.
method_max_iterint, default=1000

Maximum number of iterations to perform when solving the lasso problem.  New in version 0.22.     Returns

codendarray of shape (n_samples, n_components),

The sparse code (only returned if return_code=True).
dictionaryndarray of shape (n_components, n_features),

The solutions to the dictionary learning problem.
n_iterint

Number of iterations run. Returned only if return_n_iter is set to True.      See also
dict_learning

DictionaryLearning

MiniBatchDictionaryLearning

SparsePCA

MiniBatchSparsePCA


sklearn.decomposition.fastica
sklearn.decomposition.fastica(X, n_components=None, *, algorithm='parallel', whiten=True, fun='logcosh', fun_args=None, max_iter=200, tol=0.0001, w_init=None, random_state=None, return_X_mean=False, compute_sources=True, return_n_iter=False) [source]

Perform Fast Independent Component Analysis. Read more in the User Guide.  Parameters

Xarray-like of shape (n_samples, n_features)

Training vector, where n_samples is the number of samples and n_features is the number of features.
n_componentsint, default=None

Number of components to extract. If None no dimension reduction is performed.
algorithm{‘parallel’, ‘deflation’}, default=’parallel’

Apply a parallel or deflational FASTICA algorithm.
whitenbool, default=True

If True perform an initial whitening of the data. If False, the data is assumed to have already been preprocessed: it should be centered, normed and white. Otherwise you will get incorrect results. In this case the parameter n_components will be ignored.
fun{‘logcosh’, ‘exp’, ‘cube’} or callable, default=’logcosh’

The functional form of the G function used in the approximation to neg-entropy. Could be either ‘logcosh’, ‘exp’, or ‘cube’. You can also provide your own function. It should return a tuple containing the value of the function, and of its derivative, in the point. The derivative should be averaged along its last dimension. Example:  def my_g(x):

return x ** 3, np.mean(3 * x ** 2, axis=-1)
fun_argsdict, default=None

Arguments to send to the functional form. If empty or None and if fun=’logcosh’, fun_args will take value {‘alpha’ : 1.0}
max_iterint, default=200

Maximum number of iterations to perform.
tolfloat, default=1e-04

A positive scalar giving the tolerance at which the un-mixing matrix is considered to have converged.
w_initndarray of shape (n_components, n_components), default=None

Initial un-mixing array of dimension (n.comp,n.comp). If None (default) then an array of normal r.v.’s is used.
random_stateint, RandomState instance or None, default=None

Used to initialize w_init when not specified, with a normal distribution. Pass an int, for reproducible results across multiple function calls. See Glossary.
return_X_meanbool, default=False

If True, X_mean is returned too.
compute_sourcesbool, default=True

If False, sources are not computed, but only the rotation matrix. This can save memory when working with big data. Defaults to True.
return_n_iterbool, default=False

Whether or not to return the number of iterations.    Returns

Kndarray of shape (n_components, n_features) or None

If whiten is ‘True’, K is the pre-whitening matrix that projects data onto the first n_components principal components. If whiten is ‘False’, K is ‘None’.
Wndarray of shape (n_components, n_components)

The square matrix that unmixes the data after whitening. The mixing matrix is the pseudo-inverse of matrix W K if K is not None, else it is the inverse of W.
Sndarray of shape (n_samples, n_components) or None

Estimated source matrix
X_meanndarray of shape (n_features,)

The mean over features. Returned only if return_X_mean is True.
n_iterint

If the algorithm is “deflation”, n_iter is the maximum number of iterations run across all components. Else they are just the number of iterations taken to converge. This is returned only when return_n_iter is set to True.     Notes The data matrix X is considered to be a linear combination of non-Gaussian (independent) components i.e. X = AS where columns of S contain the independent components and A is a linear mixing matrix. In short ICA attempts to un-mix' the data by estimating an
un-mixing matrix W where ``S = W K X.` While FastICA was proposed to estimate as many sources as features, it is possible to estimate less by setting n_components < n_features. It this case K is not a square matrix and the estimated A is the pseudo-inverse of W K. This implementation was originally made for data of shape [n_features, n_samples]. Now the input is transposed before the algorithm is applied. This makes it slightly faster for Fortran-ordered input. Implemented using FastICA: A. Hyvarinen and E. Oja, Independent Component Analysis: Algorithms and Applications, Neural Networks, 13(4-5), 2000, pp. 411-430


sklearn.decomposition.non_negative_factorization
sklearn.decomposition.non_negative_factorization(X, W=None, H=None, n_components=None, *, init='warn', update_H=True, solver='cd', beta_loss='frobenius', tol=0.0001, max_iter=200, alpha=0.0, l1_ratio=0.0, regularization=None, random_state=None, verbose=0, shuffle=False) [source]

Compute Non-negative Matrix Factorization (NMF). Find two non-negative matrices (W, H) whose product approximates the non- negative matrix X. This factorization can be used for example for dimensionality reduction, source separation or topic extraction. The objective function is:  \[ \begin{align}\begin{aligned}0.5 * ||X - WH||_{Fro}^2 + alpha * l1_{ratio} * ||vec(W)||_1\\+ alpha * l1_{ratio} * ||vec(H)||_1\\+ 0.5 * alpha * (1 - l1_{ratio}) * ||W||_{Fro}^2\\+ 0.5 * alpha * (1 - l1_{ratio}) * ||H||_{Fro}^2\end{aligned}\end{align} \] Where: \(||A||_{Fro}^2 = \sum_{i,j} A_{ij}^2\) (Frobenius norm) \(||vec(A)||_1 = \sum_{i,j} abs(A_{ij})\) (Elementwise L1 norm) For multiplicative-update (‘mu’) solver, the Frobenius norm \((0.5 * ||X - WH||_{Fro}^2)\) can be changed into another beta-divergence loss, by changing the beta_loss parameter. The objective function is minimized with an alternating minimization of W and H. If H is given and update_H=False, it solves for W only.  Parameters

Xarray-like of shape (n_samples, n_features)

Constant matrix.
Warray-like of shape (n_samples, n_components), default=None

If init=’custom’, it is used as initial guess for the solution.
Harray-like of shape (n_components, n_features), default=None

If init=’custom’, it is used as initial guess for the solution. If update_H=False, it is used as a constant, to solve for W only.
n_componentsint, default=None

Number of components, if n_components is not set all features are kept.
init{‘random’, ‘nndsvd’, ‘nndsvda’, ‘nndsvdar’, ‘custom’}, default=None

Method used to initialize the procedure. Valid options:  None: ‘nndsvd’ if n_components < n_features, otherwise ‘random’.
 ‘random’: non-negative random matrices, scaled with:

sqrt(X.mean() / n_components)
 ‘nndsvd’: Nonnegative Double Singular Value Decomposition (NNDSVD)

initialization (better for sparseness)
 ‘nndsvda’: NNDSVD with zeros filled with the average of X

(better when sparsity is not desired)
 ‘nndsvdar’: NNDSVD with zeros filled with small random values

(generally faster, less accurate alternative to NNDSVDa for when sparsity is not desired)    ‘custom’: use custom matrices W and H if update_H=True. If update_H=False, then only custom matrix H is used.   Changed in version 0.23: The default value of init changed from ‘random’ to None in 0.23.
update_Hbool, default=True

Set to True, both W and H will be estimated from initial guesses. Set to False, only W will be estimated.
solver{‘cd’, ‘mu’}, default=’cd’

Numerical solver to use:
 ‘cd’ is a Coordinate Descent solver that uses Fast Hierarchical

Alternating Least Squares (Fast HALS).    ‘mu’ is a Multiplicative Update solver.   New in version 0.17: Coordinate Descent solver.   New in version 0.19: Multiplicative Update solver.
beta_lossfloat or {‘frobenius’, ‘kullback-leibler’, ‘itakura-saito’}, default=’frobenius’

Beta divergence to be minimized, measuring the distance between X and the dot product WH. Note that values different from ‘frobenius’ (or 2) and ‘kullback-leibler’ (or 1) lead to significantly slower fits. Note that for beta_loss <= 0 (or ‘itakura-saito’), the input matrix X cannot contain zeros. Used only in ‘mu’ solver.  New in version 0.19.
tolfloat, default=1e-4

Tolerance of the stopping condition.
max_iterint, default=200

Maximum number of iterations before timing out.
alphafloat, default=0.

Constant that multiplies the regularization terms.
l1_ratiofloat, default=0.

The regularization mixing parameter, with 0 <= l1_ratio <= 1. For l1_ratio = 0 the penalty is an elementwise L2 penalty (aka Frobenius Norm). For l1_ratio = 1 it is an elementwise L1 penalty. For 0 < l1_ratio < 1, the penalty is a combination of L1 and L2.
regularization{‘both’, ‘components’, ‘transformation’}, default=None

Select whether the regularization affects the components (H), the transformation (W), both or none of them.
random_stateint, RandomState instance or None, default=None

Used for NMF initialisation (when init == ‘nndsvdar’ or ‘random’), and in Coordinate Descent. Pass an int for reproducible results across multiple function calls. See Glossary.
verboseint, default=0

The verbosity level.
shufflebool, default=False

If true, randomize the order of coordinates in the CD solver.    Returns

Wndarray of shape (n_samples, n_components)

Solution to the non-negative least squares problem.
Hndarray of shape (n_components, n_features)

Solution to the non-negative least squares problem.
n_iterint

Actual number of iterations.     References Cichocki, Andrzej, and P. H. A. N. Anh-Huy. “Fast local algorithms for large scale nonnegative matrix and tensor factorizations.” IEICE transactions on fundamentals of electronics, communications and computer sciences 92.3: 708-721, 2009. Fevotte, C., & Idier, J. (2011). Algorithms for nonnegative matrix factorization with the beta-divergence. Neural Computation, 23(9). Examples >>> import numpy as np
>>> X = np.array([[1,1], [2, 1], [3, 1.2], [4, 1], [5, 0.8], [6, 1]])
>>> from sklearn.decomposition import non_negative_factorization
>>> W, H, n_iter = non_negative_factorization(X, n_components=2,
... init='random', random_state=0)


sklearn.decomposition.sparse_encode
sklearn.decomposition.sparse_encode(X, dictionary, *, gram=None, cov=None, algorithm='lasso_lars', n_nonzero_coefs=None, alpha=None, copy_cov=True, init=None, max_iter=1000, n_jobs=None, check_input=True, verbose=0, positive=False) [source]

Sparse coding Each row of the result is the solution to a sparse coding problem. The goal is to find a sparse array code such that: X ~= code * dictionary
 Read more in the User Guide.  Parameters

Xndarray of shape (n_samples, n_features)

Data matrix.
dictionaryndarray of shape (n_components, n_features)

The dictionary matrix against which to solve the sparse coding of the data. Some of the algorithms assume normalized rows for meaningful output.
gramndarray of shape (n_components, n_components), default=None

Precomputed Gram matrix, dictionary * dictionary'.
covndarray of shape (n_components, n_samples), default=None

Precomputed covariance, dictionary' * X.
algorithm{‘lasso_lars’, ‘lasso_cd’, ‘lars’, ‘omp’, ‘threshold’}, default=’lasso_lars’

The algorithm used:
'lars': uses the least angle regression method (linear_model.lars_path);
'lasso_lars': uses Lars to compute the Lasso solution;
'lasso_cd': uses the coordinate descent method to compute the Lasso solution (linear_model.Lasso). lasso_lars will be faster if the estimated components are sparse;
'omp': uses orthogonal matching pursuit to estimate the sparse solution;
'threshold': squashes to zero all coefficients less than regularization from the projection dictionary * data'.
n_nonzero_coefsint, default=None

Number of nonzero coefficients to target in each column of the solution. This is only used by algorithm='lars' and algorithm='omp' and is overridden by alpha in the omp case. If None, then n_nonzero_coefs=int(n_features / 10).
alphafloat, default=None

If algorithm='lasso_lars' or algorithm='lasso_cd', alpha is the penalty applied to the L1 norm. If algorithm='threshold', alpha is the absolute value of the threshold below which coefficients will be squashed to zero. If algorithm='omp', alpha is the tolerance parameter: the value of the reconstruction error targeted. In this case, it overrides n_nonzero_coefs. If None, default to 1.
copy_covbool, default=True

Whether to copy the precomputed covariance matrix; if False, it may be overwritten.
initndarray of shape (n_samples, n_components), default=None

Initialization value of the sparse codes. Only used if algorithm='lasso_cd'.
max_iterint, default=1000

Maximum number of iterations to perform if algorithm='lasso_cd' or 'lasso_lars'.
n_jobsint, default=None

Number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
check_inputbool, default=True

If False, the input arrays X and dictionary will not be checked.
verboseint, default=0

Controls the verbosity; the higher, the more messages.
positivebool, default=False

Whether to enforce positivity when finding the encoding.  New in version 0.20.     Returns

codendarray of shape (n_samples, n_components)

The sparse codes      See also
sklearn.linear_model.lars_path

sklearn.linear_model.orthogonal_mp

sklearn.linear_model.Lasso

SparseCoder


sklearn.feature_extraction.image.extract_patches_2d
sklearn.feature_extraction.image.extract_patches_2d(image, patch_size, *, max_patches=None, random_state=None) [source]

Reshape a 2D image into a collection of patches The resulting patches are allocated in a dedicated array. Read more in the User Guide.  Parameters

imagendarray of shape (image_height, image_width) or (image_height, image_width, n_channels)

The original image data. For color images, the last dimension specifies the channel: a RGB image would have n_channels=3.
patch_sizetuple of int (patch_height, patch_width)

The dimensions of one patch.
max_patchesint or float, default=None

The maximum number of patches to extract. If max_patches is a float between 0 and 1, it is taken to be a proportion of the total number of patches.
random_stateint, RandomState instance, default=None

Determines the random number generator used for random sampling when max_patches is not None. Use an int to make the randomness deterministic. See Glossary.    Returns

patchesarray of shape (n_patches, patch_height, patch_width) or (n_patches, patch_height, patch_width, n_channels)

The collection of patches extracted from the image, where n_patches is either max_patches or the total number of patches that can be extracted.     Examples >>> from sklearn.datasets import load_sample_image
>>> from sklearn.feature_extraction import image
>>> # Use the array data from the first image in this dataset:
>>> one_image = load_sample_image("china.jpg")
>>> print('Image shape: {}'.format(one_image.shape))
Image shape: (427, 640, 3)
>>> patches = image.extract_patches_2d(one_image, (2, 2))
>>> print('Patches shape: {}'.format(patches.shape))
Patches shape: (272214, 2, 2, 3)
>>> # Here are just two of these patches:
>>> print(patches[1])
[[[174 201 231]
  [174 201 231]]
 [[173 200 230]
  [173 200 230]]]
>>> print(patches[800])
[[[187 214 243]
  [188 215 244]]
 [[187 214 243]
  [188 215 244]]]

 Examples using sklearn.feature_extraction.image.extract_patches_2d

  Online learning of a dictionary of parts of faces

  Image denoising using dictionary learning


sklearn.feature_extraction.image.grid_to_graph
sklearn.feature_extraction.image.grid_to_graph(n_x, n_y, n_z=1, *, mask=None, return_as=<class 'scipy.sparse.coo.coo_matrix'>, dtype=<class 'int'>) [source]

Graph of the pixel-to-pixel connections Edges exist if 2 voxels are connected.  Parameters

n_xint

Dimension in x axis
n_yint

Dimension in y axis
n_zint, default=1

Dimension in z axis
maskndarray of shape (n_x, n_y, n_z), dtype=bool, default=None

An optional mask of the image, to consider only part of the pixels.
return_asnp.ndarray or a sparse matrix class, default=sparse.coo_matrix

The class to use to build the returned adjacency matrix.
dtypedtype, default=int

The data of the returned sparse matrix. By default it is int     Notes For scikit-learn versions 0.14.1 and prior, return_as=np.ndarray was handled by returning a dense np.matrix instance. Going forward, np.ndarray returns an np.ndarray, as expected. For compatibility, user code relying on this method should wrap its calls in np.asarray to avoid type issues.


sklearn.feature_extraction.image.img_to_graph
sklearn.feature_extraction.image.img_to_graph(img, *, mask=None, return_as=<class 'scipy.sparse.coo.coo_matrix'>, dtype=None) [source]

Graph of the pixel-to-pixel gradient connections Edges are weighted with the gradient values. Read more in the User Guide.  Parameters

imgndarray of shape (height, width) or (height, width, channel)

2D or 3D image.
maskndarray of shape (height, width) or (height, width, channel), dtype=bool, default=None

An optional mask of the image, to consider only part of the pixels.
return_asnp.ndarray or a sparse matrix class, default=sparse.coo_matrix

The class to use to build the returned adjacency matrix.
dtypedtype, default=None

The data of the returned sparse matrix. By default it is the dtype of img     Notes For scikit-learn versions 0.14.1 and prior, return_as=np.ndarray was handled by returning a dense np.matrix instance. Going forward, np.ndarray returns an np.ndarray, as expected. For compatibility, user code relying on this method should wrap its calls in np.asarray to avoid type issues.


sklearn.feature_extraction.image.reconstruct_from_patches_2d
sklearn.feature_extraction.image.reconstruct_from_patches_2d(patches, image_size) [source]

Reconstruct the image from all of its patches. Patches are assumed to overlap and the image is constructed by filling in the patches from left to right, top to bottom, averaging the overlapping regions. Read more in the User Guide.  Parameters

patchesndarray of shape (n_patches, patch_height, patch_width) or (n_patches, patch_height, patch_width, n_channels)

The complete set of patches. If the patches contain colour information, channels are indexed along the last dimension: RGB patches would have n_channels=3.
image_sizetuple of int (image_height, image_width) or (image_height, image_width, n_channels)

The size of the image that will be reconstructed.    Returns

imagendarray of shape image_size

The reconstructed image.
 Examples using sklearn.feature_extraction.image.reconstruct_from_patches_2d

  Image denoising using dictionary learning


sklearn.feature_selection.chi2
sklearn.feature_selection.chi2(X, y) [source]

Compute chi-squared stats between each non-negative feature and class. This score can be used to select the n_features features with the highest values for the test chi-squared statistic from X, which must contain only non-negative features such as booleans or frequencies (e.g., term counts in document classification), relative to the classes. Recall that the chi-square test measures dependence between stochastic variables, so using this function “weeds out” the features that are the most likely to be independent of class and therefore irrelevant for classification. Read more in the User Guide.  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Sample vectors.
yarray-like of shape (n_samples,)

Target vector (class labels).    Returns

chi2array, shape = (n_features,)

chi2 statistics of each feature.
pvalarray, shape = (n_features,)

p-values of each feature.      See also
f_classif

ANOVA F-value between label/feature for classification tasks.
f_regression

F-value between label/feature for regression tasks.    Notes Complexity of this algorithm is O(n_classes * n_features).
 Examples using sklearn.feature_selection.chi2

  Selecting dimensionality reduction with Pipeline and GridSearchCV

  SVM-Anova: SVM with univariate feature selection

  Classification of text documents using sparse features


sklearn.feature_selection.f_classif
sklearn.feature_selection.f_classif(X, y) [source]

Compute the ANOVA F-value for the provided sample. Read more in the User Guide.  Parameters

X{array-like, sparse matrix} shape = [n_samples, n_features]

The set of regressors that will be tested sequentially.
yarray of shape(n_samples)

The data matrix.    Returns

Farray, shape = [n_features,]

The set of F values.
pvalarray, shape = [n_features,]

The set of p-values.      See also
chi2

Chi-squared stats of non-negative features for classification tasks.
f_regression

F-value between label/feature for regression tasks.
 Examples using sklearn.feature_selection.f_classif

  Pipeline Anova SVM

  Univariate Feature Selection


sklearn.feature_selection.f_regression
sklearn.feature_selection.f_regression(X, y, *, center=True) [source]

Univariate linear regression tests. Linear model for testing the individual effect of each of many regressors. This is a scoring function to be used in a feature selection procedure, not a free standing feature selection procedure. This is done in 2 steps:  The correlation between each regressor and the target is computed, that is, ((X[:, i] - mean(X[:, i])) * (y - mean_y)) / (std(X[:, i]) * std(y)). It is converted to an F score then to a p-value.  For more on usage see the User Guide.  Parameters

X{array-like, sparse matrix} shape = (n_samples, n_features)

The set of regressors that will be tested sequentially.
yarray of shape(n_samples).

The data matrix
centerbool, default=True

If true, X and y will be centered.    Returns

Farray, shape=(n_features,)

F values of features.
pvalarray, shape=(n_features,)

p-values of F-scores.      See also
mutual_info_regression

Mutual information for a continuous target.
f_classif

ANOVA F-value between label/feature for classification tasks.
chi2

Chi-squared stats of non-negative features for classification tasks.
SelectKBest

Select features based on the k highest scores.
SelectFpr

Select features based on a false positive rate test.
SelectFdr

Select features based on an estimated false discovery rate.
SelectFwe

Select features based on family-wise error rate.
SelectPercentile

Select features based on percentile of the highest scores.
 Examples using sklearn.feature_selection.f_regression

  Feature agglomeration vs. univariate selection

  Comparison of F-test and mutual information


sklearn.feature_selection.mutual_info_classif
sklearn.feature_selection.mutual_info_classif(X, y, *, discrete_features='auto', n_neighbors=3, copy=True, random_state=None) [source]

Estimate mutual information for a discrete target variable. Mutual information (MI) [1] between two random variables is a non-negative value, which measures the dependency between the variables. It is equal to zero if and only if two random variables are independent, and higher values mean higher dependency. The function relies on nonparametric methods based on entropy estimation from k-nearest neighbors distances as described in [2] and [3]. Both methods are based on the idea originally proposed in [4]. It can be used for univariate features selection, read more in the User Guide.  Parameters

Xarray-like or sparse matrix, shape (n_samples, n_features)

Feature matrix.
yarray-like of shape (n_samples,)

Target vector.
discrete_features{‘auto’, bool, array-like}, default=’auto’

If bool, then determines whether to consider all features discrete or continuous. If array, then it should be either a boolean mask with shape (n_features,) or array with indices of discrete features. If ‘auto’, it is assigned to False for dense X and to True for sparse X.
n_neighborsint, default=3

Number of neighbors to use for MI estimation for continuous variables, see [2] and [3]. Higher values reduce variance of the estimation, but could introduce a bias.
copybool, default=True

Whether to make a copy of the given data. If set to False, the initial data will be overwritten.
random_stateint, RandomState instance or None, default=None

Determines random number generation for adding small noise to continuous variables in order to remove repeated values. Pass an int for reproducible results across multiple function calls. See Glossary.    Returns

mindarray, shape (n_features,)

Estimated mutual information between each feature and the target.     Notes  The term “discrete features” is used instead of naming them “categorical”, because it describes the essence more accurately. For example, pixel intensities of an image are discrete features (but hardly categorical) and you will get better results if mark them as such. Also note, that treating a continuous variable as discrete and vice versa will usually give incorrect results, so be attentive about that. True mutual information can’t be negative. If its estimate turns out to be negative, it is replaced by zero.  References
1
Mutual Information on Wikipedia.
2(1,2)
A. Kraskov, H. Stogbauer and P. Grassberger, “Estimating mutual information”. Phys. Rev. E 69, 2004.
3(1,2)
B. C. Ross “Mutual Information between Discrete and Continuous Data Sets”. PLoS ONE 9(2), 2014.
4
L. F. Kozachenko, N. N. Leonenko, “Sample Estimate of the Entropy of a Random Vector:, Probl. Peredachi Inf., 23:2 (1987), 9-16


sklearn.feature_selection.mutual_info_regression
sklearn.feature_selection.mutual_info_regression(X, y, *, discrete_features='auto', n_neighbors=3, copy=True, random_state=None) [source]

Estimate mutual information for a continuous target variable. Mutual information (MI) [1] between two random variables is a non-negative value, which measures the dependency between the variables. It is equal to zero if and only if two random variables are independent, and higher values mean higher dependency. The function relies on nonparametric methods based on entropy estimation from k-nearest neighbors distances as described in [2] and [3]. Both methods are based on the idea originally proposed in [4]. It can be used for univariate features selection, read more in the User Guide.  Parameters

Xarray-like or sparse matrix, shape (n_samples, n_features)

Feature matrix.
yarray-like of shape (n_samples,)

Target vector.
discrete_features{‘auto’, bool, array-like}, default=’auto’

If bool, then determines whether to consider all features discrete or continuous. If array, then it should be either a boolean mask with shape (n_features,) or array with indices of discrete features. If ‘auto’, it is assigned to False for dense X and to True for sparse X.
n_neighborsint, default=3

Number of neighbors to use for MI estimation for continuous variables, see [2] and [3]. Higher values reduce variance of the estimation, but could introduce a bias.
copybool, default=True

Whether to make a copy of the given data. If set to False, the initial data will be overwritten.
random_stateint, RandomState instance or None, default=None

Determines random number generation for adding small noise to continuous variables in order to remove repeated values. Pass an int for reproducible results across multiple function calls. See Glossary.    Returns

mindarray, shape (n_features,)

Estimated mutual information between each feature and the target.     Notes  The term “discrete features” is used instead of naming them “categorical”, because it describes the essence more accurately. For example, pixel intensities of an image are discrete features (but hardly categorical) and you will get better results if mark them as such. Also note, that treating a continuous variable as discrete and vice versa will usually give incorrect results, so be attentive about that. True mutual information can’t be negative. If its estimate turns out to be negative, it is replaced by zero.  References
1
Mutual Information on Wikipedia.
2(1,2)
A. Kraskov, H. Stogbauer and P. Grassberger, “Estimating mutual information”. Phys. Rev. E 69, 2004.
3(1,2)
B. C. Ross “Mutual Information between Discrete and Continuous Data Sets”. PLoS ONE 9(2), 2014.
4
L. F. Kozachenko, N. N. Leonenko, “Sample Estimate of the Entropy of a Random Vector”, Probl. Peredachi Inf., 23:2 (1987), 9-16
 Examples using sklearn.feature_selection.mutual_info_regression

  Comparison of F-test and mutual information


sklearn.get_config
sklearn.get_config() [source]

Retrieve current values for configuration set by set_config  Returns

configdict

Keys are parameter names that can be passed to set_config.      See also
config_context

Context manager for global scikit-learn configuration.
set_config

Set global scikit-learn configuration.


sklearn.inspection.partial_dependence
sklearn.inspection.partial_dependence(estimator, X, features, *, response_method='auto', percentiles=0.05, 0.95, grid_resolution=100, method='auto', kind='legacy') [source]

Partial dependence of features. Partial dependence of a feature (or a set of features) corresponds to the average response of an estimator for each possible value of the feature. Read more in the User Guide.  Warning For GradientBoostingClassifier and GradientBoostingRegressor, the 'recursion' method (used by default) will not account for the init predictor of the boosting process. In practice, this will produce the same values as 'brute' up to a constant offset in the target response, provided that init is a constant estimator (which is the default). However, if init is not a constant estimator, the partial dependence values are incorrect for 'recursion' because the offset will be sample-dependent. It is preferable to use the 'brute' method. Note that this only applies to GradientBoostingClassifier and GradientBoostingRegressor, not to HistGradientBoostingClassifier and HistGradientBoostingRegressor.   Parameters

estimatorBaseEstimator

A fitted estimator object implementing predict, predict_proba, or decision_function. Multioutput-multiclass classifiers are not supported.
X{array-like or dataframe} of shape (n_samples, n_features)

X is used to generate a grid of values for the target features (where the partial dependence will be evaluated), and also to generate values for the complement features when the method is ‘brute’.
featuresarray-like of {int, str}

The feature (e.g. [0]) or pair of interacting features (e.g. [(0, 1)]) for which the partial dependency should be computed.
response_method{‘auto’, ‘predict_proba’, ‘decision_function’}, default=’auto’

Specifies whether to use predict_proba or decision_function as the target response. For regressors this parameter is ignored and the response is always the output of predict. By default, predict_proba is tried first and we revert to decision_function if it doesn’t exist. If method is ‘recursion’, the response is always the output of decision_function.
percentilestuple of float, default=(0.05, 0.95)

The lower and upper percentile used to create the extreme values for the grid. Must be in [0, 1].
grid_resolutionint, default=100

The number of equally spaced points on the grid, for each target feature.
method{‘auto’, ‘recursion’, ‘brute’}, default=’auto’

The method used to calculate the averaged predictions:
'recursion' is only supported for some tree-based estimators (namely GradientBoostingClassifier, GradientBoostingRegressor, HistGradientBoostingClassifier, HistGradientBoostingRegressor, DecisionTreeRegressor, RandomForestRegressor, ) when kind='average'. This is more efficient in terms of speed. With this method, the target response of a classifier is always the decision function, not the predicted probabilities. Since the 'recursion' method implicitely computes the average of the Individual Conditional Expectation (ICE) by design, it is not compatible with ICE and thus kind must be 'average'.
'brute' is supported for any estimator, but is more computationally intensive.
'auto': the 'recursion' is used for estimators that support it, and 'brute' is used otherwise.  Please see this note for differences between the 'brute' and 'recursion' method.
kind{‘legacy’, ‘average’, ‘individual’, ‘both’}, default=’legacy’

Whether to return the partial dependence averaged across all the samples in the dataset or one line per sample or both. See Returns below. Note that the fast method='recursion' option is only available for kind='average'. Plotting individual dependencies requires using the slower method='brute' option.  New in version 0.24.   Deprecated since version 0.24: kind='legacy' is deprecated and will be removed in version 1.1. kind='average' will be the new default. It is intended to migrate from the ndarray output to Bunch output.     Returns

predictionsndarray or Bunch

if kind='legacy', return value is ndarray of shape (n_outputs, len(values[0]), len(values[1]), …)

The predictions for all the points in the grid, averaged over all samples in X (or over the training data if method is ‘recursion’).

if kind='individual', 'average' or 'both', return value is Bunch

Dictionary-like object, with the following attributes.
individualndarray of shape (n_outputs, n_instances, len(values[0]), len(values[1]), …)

The predictions for all the points in the grid for all samples in X. This is also known as Individual Conditional Expectation (ICE)
averagendarray of shape (n_outputs, len(values[0]), len(values[1]), …)

The predictions for all the points in the grid, averaged over all samples in X (or over the training data if method is ‘recursion’). Only available when kind=’both’.
valuesseq of 1d ndarrays

The values with which the grid has been created. The generated grid is a cartesian product of the arrays in values. len(values) == len(features). The size of each array values[j] is either grid_resolution, or the number of unique values in X[:, j], whichever is smaller.       n_outputs corresponds to the number of classes in a multi-class setting, or to the number of tasks for multi-output regression. For classical regression and binary classification n_outputs==1. n_values_feature_j corresponds to the size values[j].
valuesseq of 1d ndarrays

The values with which the grid has been created. The generated grid is a cartesian product of the arrays in values. len(values) ==
len(features). The size of each array values[j] is either grid_resolution, or the number of unique values in X[:, j], whichever is smaller. Only available when kind="legacy".      See also
plot_partial_dependence

Plot Partial Dependence.
PartialDependenceDisplay

Partial Dependence visualization.    Examples >>> X = [[0, 0, 2], [1, 0, 0]]
>>> y = [0, 1]
>>> from sklearn.ensemble import GradientBoostingClassifier
>>> gb = GradientBoostingClassifier(random_state=0).fit(X, y)
>>> partial_dependence(gb, features=[0], X=X, percentiles=(0, 1),
...                    grid_resolution=2)
(array([[-4.52...,  4.52...]]), [array([ 0.,  1.])])

 Examples using sklearn.inspection.partial_dependence

  Partial Dependence and Individual Conditional Expectation Plots


sklearn.inspection.permutation_importance
sklearn.inspection.permutation_importance(estimator, X, y, *, scoring=None, n_repeats=5, n_jobs=None, random_state=None, sample_weight=None) [source]

Permutation importance for feature evaluation [BRE]. The estimator is required to be a fitted estimator. X can be the data set used to train the estimator or a hold-out set. The permutation importance of a feature is calculated as follows. First, a baseline metric, defined by scoring, is evaluated on a (potentially different) dataset defined by the X. Next, a feature column from the validation set is permuted and the metric is evaluated again. The permutation importance is defined to be the difference between the baseline metric and metric from permutating the feature column. Read more in the User Guide.  Parameters

estimatorobject

An estimator that has already been fitted and is compatible with scorer.
Xndarray or DataFrame, shape (n_samples, n_features)

Data on which permutation importance will be computed.
yarray-like or None, shape (n_samples, ) or (n_samples, n_classes)

Targets for supervised or None for unsupervised.
scoringstring, callable or None, default=None

Scorer to use. It can be a single string (see The scoring parameter: defining model evaluation rules) or a callable (see Defining your scoring strategy from metric functions). If None, the estimator’s default scorer is used.
n_repeatsint, default=5

Number of times to permute a feature.
n_jobsint or None, default=None

Number of jobs to run in parallel. The computation is done by computing permutation score for each columns and parallelized over the columns. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
random_stateint, RandomState instance, default=None

Pseudo-random number generator to control the permutations of each feature. Pass an int to get reproducible results across function calls. See :term: Glossary <random_state>.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights used in scoring.  New in version 0.24.     Returns

resultBunch

Dictionary-like object, with the following attributes.
importances_meanndarray, shape (n_features, )

Mean of feature importance over n_repeats.
importances_stdndarray, shape (n_features, )

Standard deviation over n_repeats.
importancesndarray, shape (n_features, n_repeats)

Raw permutation importance scores.       References
BRE
L. Breiman, “Random Forests”, Machine Learning, 45(1), 5-32, 2001. https://doi.org/10.1023/A:1010933404324   Examples >>> from sklearn.linear_model import LogisticRegression
>>> from sklearn.inspection import permutation_importance
>>> X = [[1, 9, 9],[1, 9, 9],[1, 9, 9],
...      [0, 9, 9],[0, 9, 9],[0, 9, 9]]
>>> y = [1, 1, 1, 0, 0, 0]
>>> clf = LogisticRegression().fit(X, y)
>>> result = permutation_importance(clf, X, y, n_repeats=10,
...                                 random_state=0)
>>> result.importances_mean
array([0.4666..., 0.       , 0.       ])
>>> result.importances_std
array([0.2211..., 0.       , 0.       ])

 Examples using sklearn.inspection.permutation_importance

  Release Highlights for scikit-learn 0.22

  Feature importances with forests of trees

  Gradient Boosting regression

  Permutation Importance with Multicollinear or Correlated Features

  Permutation Importance vs Random Forest Feature Importance (MDI)


sklearn.inspection.plot_partial_dependence
sklearn.inspection.plot_partial_dependence(estimator, X, features, *, feature_names=None, target=None, response_method='auto', n_cols=3, grid_resolution=100, percentiles=0.05, 0.95, method='auto', n_jobs=None, verbose=0, line_kw=None, contour_kw=None, ax=None, kind='average', subsample=1000, random_state=None) [source]

Partial dependence (PD) and individual conditional expectation (ICE) plots. Partial dependence plots, individual conditional expectation plots or an overlay of both of them can be plotted by setting the kind parameter. The len(features) plots are arranged in a grid with n_cols columns. Two-way partial dependence plots are plotted as contour plots. The deciles of the feature values will be shown with tick marks on the x-axes for one-way plots, and on both axes for two-way plots. Read more in the User Guide.  Note plot_partial_dependence does not support using the same axes with multiple calls. To plot the the partial dependence for multiple estimators, please pass the axes created by the first call to the second call: >>> from sklearn.inspection import plot_partial_dependence
>>> from sklearn.datasets import make_friedman1
>>> from sklearn.linear_model import LinearRegression
>>> from sklearn.ensemble import RandomForestRegressor
>>> X, y = make_friedman1()
>>> est1 = LinearRegression().fit(X, y)
>>> est2 = RandomForestRegressor().fit(X, y)
>>> disp1 = plot_partial_dependence(est1, X,
...                                 [1, 2])
>>> disp2 = plot_partial_dependence(est2, X, [1, 2],
...                                 ax=disp1.axes_)
   Warning For GradientBoostingClassifier and GradientBoostingRegressor, the 'recursion' method (used by default) will not account for the init predictor of the boosting process. In practice, this will produce the same values as 'brute' up to a constant offset in the target response, provided that init is a constant estimator (which is the default). However, if init is not a constant estimator, the partial dependence values are incorrect for 'recursion' because the offset will be sample-dependent. It is preferable to use the 'brute' method. Note that this only applies to GradientBoostingClassifier and GradientBoostingRegressor, not to HistGradientBoostingClassifier and HistGradientBoostingRegressor.   Parameters

estimatorBaseEstimator

A fitted estimator object implementing predict, predict_proba, or decision_function. Multioutput-multiclass classifiers are not supported.
X{array-like or dataframe} of shape (n_samples, n_features)

X is used to generate a grid of values for the target features (where the partial dependence will be evaluated), and also to generate values for the complement features when the method is 'brute'.
featureslist of {int, str, pair of int, pair of str}

The target features for which to create the PDPs. If features[i] is an integer or a string, a one-way PDP is created; if features[i] is a tuple, a two-way PDP is created (only supported with kind='average'). Each tuple must be of size 2. if any entry is a string, then it must be in feature_names.
feature_namesarray-like of shape (n_features,), dtype=str, default=None

Name of each feature; feature_names[i] holds the name of the feature with index i. By default, the name of the feature corresponds to their numerical index for NumPy array and their column name for pandas dataframe.
targetint, default=None

 In a multiclass setting, specifies the class for which the PDPs should be computed. Note that for binary classification, the positive class (index 1) is always used. In a multioutput setting, specifies the task for which the PDPs should be computed.  Ignored in binary classification or classical regression settings.
response_method{‘auto’, ‘predict_proba’, ‘decision_function’}, default=’auto’

Specifies whether to use predict_proba or decision_function as the target response. For regressors this parameter is ignored and the response is always the output of predict. By default, predict_proba is tried first and we revert to decision_function if it doesn’t exist. If method is 'recursion', the response is always the output of decision_function.
n_colsint, default=3

The maximum number of columns in the grid plot. Only active when ax is a single axis or None.
grid_resolutionint, default=100

The number of equally spaced points on the axes of the plots, for each target feature.
percentilestuple of float, default=(0.05, 0.95)

The lower and upper percentile used to create the extreme values for the PDP axes. Must be in [0, 1].
methodstr, default=’auto’

The method used to calculate the averaged predictions:
'recursion' is only supported for some tree-based estimators (namely GradientBoostingClassifier, GradientBoostingRegressor, HistGradientBoostingClassifier, HistGradientBoostingRegressor, DecisionTreeRegressor, RandomForestRegressor but is more efficient in terms of speed. With this method, the target response of a classifier is always the decision function, not the predicted probabilities. Since the 'recursion' method implicitely computes the average of the ICEs by design, it is not compatible with ICE and thus kind must be 'average'.
'brute' is supported for any estimator, but is more computationally intensive.
'auto': the 'recursion' is used for estimators that support it, and 'brute' is used otherwise.  Please see this note for differences between the 'brute' and 'recursion' method.
n_jobsint, default=None

The number of CPUs to use to compute the partial dependences. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
verboseint, default=0

Verbose output during PD computations.
line_kwdict, default=None

Dict with keywords passed to the matplotlib.pyplot.plot call. For one-way partial dependence plots.
contour_kwdict, default=None

Dict with keywords passed to the matplotlib.pyplot.contourf call. For two-way partial dependence plots.
axMatplotlib axes or array-like of Matplotlib axes, default=None

 If a single axis is passed in, it is treated as a bounding axes and a grid of partial dependence plots will be drawn within these bounds. The n_cols parameter controls the number of columns in the grid. If an array-like of axes are passed in, the partial dependence plots will be drawn directly into these axes. If None, a figure and a bounding axes is created and treated as the single axes case.   New in version 0.22.
kind{‘average’, ‘individual’, ‘both’}, default=’average’
 Whether to plot the partial dependence averaged across all the samples in the dataset or one line per sample or both.
kind='average' results in the traditional PD plot;
kind='individual' results in the ICE plot.  Note that the fast method='recursion' option is only available for kind='average'. Plotting individual dependencies requires using the slower method='brute' option.  New in version 0.24.
subsamplefloat, int or None, default=1000

Sampling for ICE curves when kind is ‘individual’ or ‘both’. If float, should be between 0.0 and 1.0 and represent the proportion of the dataset to be used to plot ICE curves. If int, represents the absolute number samples to use. Note that the full dataset is still used to calculate averaged partial dependence when kind='both'.  New in version 0.24.
random_stateint, RandomState instance or None, default=None

Controls the randomness of the selected samples when subsamples is not None and kind is either 'both' or 'individual'. See Glossary for details.  New in version 0.24.     Returns

displayPartialDependenceDisplay
    See also
partial_dependence

Compute Partial Dependence values.
PartialDependenceDisplay

Partial Dependence visualization.    Examples >>> from sklearn.datasets import make_friedman1
>>> from sklearn.ensemble import GradientBoostingRegressor
>>> X, y = make_friedman1()
>>> clf = GradientBoostingRegressor(n_estimators=10).fit(X, y)
>>> plot_partial_dependence(clf, X, [0, (0, 1)])

 Examples using sklearn.inspection.plot_partial_dependence

  Release Highlights for scikit-learn 0.23

  Release Highlights for scikit-learn 0.24

  Monotonic Constraints

  Partial Dependence and Individual Conditional Expectation Plots

  Advanced Plotting With Partial Dependence


sklearn.isotonic.check_increasing
sklearn.isotonic.check_increasing(x, y) [source]

Determine whether y is monotonically correlated with x. y is found increasing or decreasing with respect to x based on a Spearman correlation test.  Parameters

xarray-like of shape (n_samples,)

Training data.
yarray-like of shape (n_samples,)

Training target.    Returns

increasing_boolboolean

Whether the relationship is increasing or decreasing.     Notes The Spearman correlation coefficient is estimated from the data, and the sign of the resulting estimate is used as the result. In the event that the 95% confidence interval based on Fisher transform spans zero, a warning is raised. References Fisher transformation. Wikipedia. https://en.wikipedia.org/wiki/Fisher_transformation