Incremental principal components analysis (IPCA).
Linear dimensionality reduction using Singular Value Decomposition of the data, keeping only the most significant singular vectors to project the data to a lower dimensional space. The input data is centered but not scaled for each feature before applying the SVD.
Depending on the size of the input data, this algorithm can be much more memory efficient than a PCA, and allows sparse input.
This algorithm has constant memory complexity, on the order of batch_size * n_features, enabling use of np.memmap files without loading the entire file into memory. For sparse matrices, the input is converted to dense in batches (in order to be able to subtract the mean) which avoids storing the entire dense matrix at any one time.
The computational overhead of each SVD is O(batch_size * n_features ** 2), but only 2 * batch_size samples remain in memory at a time. There will be n_samples / batch_size SVD computations to get the principal components, versus 1 large SVD of complexity O(n_samples * n_features ** 2) for PCA.
Read more in the User Guide.
New in version 0.16.
Number of components to keep. If n_components is None, then n_components is set to min(n_samples, n_features).
When True (False by default) the components_ vectors are divided by n_samples times components_ to ensure uncorrelated outputs with unit component-wise variances.
Whitening will remove some information from the transformed signal (the relative variance scales of the components) but can sometimes improve the predictive accuracy of the downstream estimators by making data respect some hard-wired assumptions.
If False, X will be overwritten. copy=False can be used to save memory but is unsafe for general use.
The number of samples to use for each batch. Only used when calling fit. If batch_size is None, then batch_size is inferred from the data and set to 5 * n_features, to provide a balance between approximation accuracy and memory consumption.
Components with maximum variance.
Variance explained by each of the selected components.
Percentage of variance explained by each of the selected components. If all components are stored, the sum of explained variances is equal to 1.0.
The singular values corresponding to each of the selected components. The singular values are equal to the 2-norms of the n_components variables in the lower-dimensional space.
Per-feature empirical mean, aggregate over calls to partial_fit.
Per-feature empirical variance, aggregate over calls to partial_fit.
The estimated noise covariance following the Probabilistic PCA model from Tipping and Bishop 1999. See “Pattern Recognition and Machine Learning” by C. Bishop, 12.2.1 p. 574 or http://www.miketipping.com/papers/met-mppca.pdf.
The estimated number of components. Relevant when n_components=None.
The number of samples processed by the estimator. Will be reset on new calls to fit, but increments across partial_fit calls.
Inferred batch size from batch_size.
See also
Implements the incremental PCA model from: D. Ross, J. Lim, R. Lin, M. Yang, Incremental Learning for Robust Visual Tracking, International Journal of Computer Vision, Volume 77, Issue 1-3, pp. 125-141, May 2008. See https://www.cs.toronto.edu/~dross/ivt/RossLimLinYang_ijcv.pdf
This model is an extension of the Sequential Karhunen-Loeve Transform from: A. Levy and M. Lindenbaum, Sequential Karhunen-Loeve Basis Extraction and its Application to Images, IEEE Transactions on Image Processing, Volume 9, Number 8, pp. 1371-1374, August 2000. See https://www.cs.technion.ac.il/~mic/doc/skl-ip.pdf
We have specifically abstained from an optimization used by authors of both papers, a QR decomposition used in specific situations to reduce the algorithmic complexity of the SVD. The source for this technique is Matrix Computations, Third Edition, G. Holub and C. Van Loan, Chapter 5, section 5.4.4, pp 252-253.. This technique has been omitted because it is advantageous only when decomposing a matrix with n_samples (rows) >= 5/3 * n_features (columns), and hurts the readability of the implemented algorithm. This would be a good opportunity for future optimization, if it is deemed necessary.
D. Ross, J. Lim, R. Lin, M. Yang. Incremental Learning for Robust Visual Tracking, International Journal of Computer Vision, Volume 77, Issue 1-3, pp. 125-141, May 2008.
G. Golub and C. Van Loan. Matrix Computations, Third Edition, Chapter 5, Section 5.4.4, pp. 252-253.
fit(X[, y])
Fit the model with X, using minibatches of size batch_size.
fit_transform(X[, y])
Fit to data, then transform it.
get_covariance()
Compute data covariance with the generative model.
get_params([deep])
Get parameters for this estimator.
get_precision()
Compute data precision matrix with the generative model.
inverse_transform(X)
Transform data back to its original space.
partial_fit(X[, y, check_input])
Incremental fit with X.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Apply dimensionality reduction to X.
Fit the model with X, using minibatches of size batch_size.
Training data, where n_samples is the number of samples and n_features is the number of features.
Returns the instance itself.
Fit to data, then transform it.
Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.
Input samples.
Target values (None for unsupervised transformations).
Additional fit parameters.
Transformed array.
Compute data covariance with the generative model.
cov = components_.T * S**2 * components_ + sigma2 * eye(n_features) where S**2 contains the explained variances, and sigma2 contains the noise variances.
Estimated covariance of data.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Compute data precision matrix with the generative model.
Equals the inverse of the covariance but computed with the matrix inversion lemma for efficiency.
Estimated precision of data.
Transform data back to its original space.
In other words, return an input X_original whose transform would be X.
New data, where n_samples is the number of samples and n_components is the number of components.
If whitening is enabled, inverse_transform will compute the exact inverse operation, which includes reversing whitening.
Incremental fit with X. All of X is processed as a single batch.
Training data, where n_samples is the number of samples and n_features is the number of features.
Run check_array on X.
Returns the instance itself.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Apply dimensionality reduction to X.
X is projected on the first principal components previously extracted from a training set, using minibatches of size batch_size if X is sparse.
New data, where n_samples is the number of samples and n_features is the number of features.
Incremental PCA


Fit the model with X, using minibatches of size batch_size.
Training data, where n_samples is the number of samples and n_features is the number of features.
Returns the instance itself.


Fit to data, then transform it.
Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.
Input samples.
Target values (None for unsupervised transformations).
Additional fit parameters.
Transformed array.


Compute data covariance with the generative model.
cov = components_.T * S**2 * components_ + sigma2 * eye(n_features) where S**2 contains the explained variances, and sigma2 contains the noise variances.
Estimated covariance of data.


Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.


Compute data precision matrix with the generative model.
Equals the inverse of the covariance but computed with the matrix inversion lemma for efficiency.
Estimated precision of data.


Transform data back to its original space.
In other words, return an input X_original whose transform would be X.
New data, where n_samples is the number of samples and n_components is the number of components.
If whitening is enabled, inverse_transform will compute the exact inverse operation, which includes reversing whitening.


Incremental fit with X. All of X is processed as a single batch.
Training data, where n_samples is the number of samples and n_features is the number of features.
Run check_array on X.
Returns the instance itself.


Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.


Apply dimensionality reduction to X.
X is projected on the first principal components previously extracted from a training set, using minibatches of size batch_size if X is sparse.
New data, where n_samples is the number of samples and n_features is the number of features.


Kernel Principal component analysis (KPCA).
Non-linear dimensionality reduction through the use of kernels (see Pairwise metrics, Affinities and Kernels).
Read more in the User Guide.
Number of components. If None, all non-zero components are kept.
Kernel used for PCA.
Kernel coefficient for rbf, poly and sigmoid kernels. Ignored by other kernels. If gamma is None, then it is set to 1/n_features.
Degree for poly kernels. Ignored by other kernels.
Independent term in poly and sigmoid kernels. Ignored by other kernels.
Parameters (keyword arguments) and values for kernel passed as callable object. Ignored by other kernels.
Hyperparameter of the ridge regression that learns the inverse transform (when fit_inverse_transform=True).
Learn the inverse transform for non-precomputed kernels. (i.e. learn to find the pre-image of a point)
Select eigensolver to use. If n_components is much less than the number of training samples, arpack may be more efficient than the dense eigensolver.
Convergence tolerance for arpack. If 0, optimal value will be chosen by arpack.
Maximum number of iterations for arpack. If None, optimal value will be chosen by arpack.
If True, then all components with zero eigenvalues are removed, so that the number of components in the output may be < n_components (and sometimes even zero due to numerical instability). When n_components is None, this parameter is ignored and components with zero eigenvalues are removed regardless.
Used when eigen_solver == ‘arpack’. Pass an int for reproducible results across multiple function calls. See Glossary.
New in version 0.18.
If True, input X is copied and stored by the model in the X_fit_ attribute. If no further changes will be done to X, setting copy_X=False saves memory by storing a reference.
New in version 0.18.
The number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
New in version 0.18.
Eigenvalues of the centered kernel matrix in decreasing order. If n_components and remove_zero_eig are not set, then all values are stored.
Eigenvectors of the centered kernel matrix. If n_components and remove_zero_eig are not set, then all components are stored.
Inverse transform matrix. Only available when fit_inverse_transform is True.
Projection of the fitted data on the kernel principal components. Only available when fit_inverse_transform is True.
The data used to fit the model. If copy_X=False, then X_fit_ is a reference. This attribute is used for the calls to transform.
Bernhard Schoelkopf, Alexander J. Smola, and Klaus-Robert Mueller. 1999. Kernel principal component analysis. In Advances in kernel methods, MIT Press, Cambridge, MA, USA 327-352.
fit(X[, y])
Fit the model from data in X.
fit_transform(X[, y])
Fit the model from data in X and transform X.
get_params([deep])
Get parameters for this estimator.
inverse_transform(X)
Transform X back to original space.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Transform X.
Fit the model from data in X.
Training vector, where n_samples in the number of samples and n_features is the number of features.
Returns the instance itself.
Fit the model from data in X and transform X.
Training vector, where n_samples in the number of samples and n_features is the number of features.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Transform X back to original space.
“Learning to Find Pre-Images”, G BakIr et al, 2004.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Transform X.


Kernel Principal component analysis (KPCA).
Non-linear dimensionality reduction through the use of kernels (see Pairwise metrics, Affinities and Kernels).
Read more in the User Guide.
Number of components. If None, all non-zero components are kept.
Kernel used for PCA.
Kernel coefficient for rbf, poly and sigmoid kernels. Ignored by other kernels. If gamma is None, then it is set to 1/n_features.
Degree for poly kernels. Ignored by other kernels.
Independent term in poly and sigmoid kernels. Ignored by other kernels.
Parameters (keyword arguments) and values for kernel passed as callable object. Ignored by other kernels.
Hyperparameter of the ridge regression that learns the inverse transform (when fit_inverse_transform=True).
Learn the inverse transform for non-precomputed kernels. (i.e. learn to find the pre-image of a point)
Select eigensolver to use. If n_components is much less than the number of training samples, arpack may be more efficient than the dense eigensolver.
Convergence tolerance for arpack. If 0, optimal value will be chosen by arpack.
Maximum number of iterations for arpack. If None, optimal value will be chosen by arpack.
If True, then all components with zero eigenvalues are removed, so that the number of components in the output may be < n_components (and sometimes even zero due to numerical instability). When n_components is None, this parameter is ignored and components with zero eigenvalues are removed regardless.
Used when eigen_solver == ‘arpack’. Pass an int for reproducible results across multiple function calls. See Glossary.
New in version 0.18.
If True, input X is copied and stored by the model in the X_fit_ attribute. If no further changes will be done to X, setting copy_X=False saves memory by storing a reference.
New in version 0.18.
The number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
New in version 0.18.
Eigenvalues of the centered kernel matrix in decreasing order. If n_components and remove_zero_eig are not set, then all values are stored.
Eigenvectors of the centered kernel matrix. If n_components and remove_zero_eig are not set, then all components are stored.
Inverse transform matrix. Only available when fit_inverse_transform is True.
Projection of the fitted data on the kernel principal components. Only available when fit_inverse_transform is True.
The data used to fit the model. If copy_X=False, then X_fit_ is a reference. This attribute is used for the calls to transform.
Bernhard Schoelkopf, Alexander J. Smola, and Klaus-Robert Mueller. 1999. Kernel principal component analysis. In Advances in kernel methods, MIT Press, Cambridge, MA, USA 327-352.
fit(X[, y])
Fit the model from data in X.
fit_transform(X[, y])
Fit the model from data in X and transform X.
get_params([deep])
Get parameters for this estimator.
inverse_transform(X)
Transform X back to original space.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Transform X.
Fit the model from data in X.
Training vector, where n_samples in the number of samples and n_features is the number of features.
Returns the instance itself.
Fit the model from data in X and transform X.
Training vector, where n_samples in the number of samples and n_features is the number of features.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Transform X back to original space.
“Learning to Find Pre-Images”, G BakIr et al, 2004.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Transform X.
Kernel PCA


Fit the model from data in X.
Training vector, where n_samples in the number of samples and n_features is the number of features.
Returns the instance itself.


Fit the model from data in X and transform X.
Training vector, where n_samples in the number of samples and n_features is the number of features.


Transform X back to original space.
“Learning to Find Pre-Images”, G BakIr et al, 2004.


Latent Dirichlet Allocation with online variational Bayes algorithm
New in version 0.17.
Read more in the User Guide.
Number of topics.
Changed in version 0.19: n_topics was renamed to n_components
Prior of document topic distribution theta. If the value is None, defaults to 1 / n_components. In [1], this is called alpha.
Prior of topic word distribution beta. If the value is None, defaults to 1 / n_components. In [1], this is called eta.
Method used to update _component. Only used in fit method. In general, if the data size is large, the online update will be much faster than the batch update.
Valid options:
Changed in version 0.20: The default learning method is now "batch".
It is a parameter that control learning rate in the online learning method. The value should be set between (0.5, 1.0] to guarantee asymptotic convergence. When the value is 0.0 and batch_size is n_samples, the update method is same as batch learning. In the literature, this is called kappa.
A (positive) parameter that downweights early iterations in online learning. It should be greater than 1.0. In the literature, this is called tau_0.
The maximum number of iterations.
Number of documents to use in each EM iteration. Only used in online learning.
How often to evaluate perplexity. Only used in fit method. set it to 0 or negative number to not evaluate perplexity in training at all. Evaluating perplexity can help you check convergence in training process, but it will also increase total training time. Evaluating perplexity in every iteration might increase training time up to two-fold.
Total number of documents. Only used in the partial_fit method.
Perplexity tolerance in batch learning. Only used when evaluate_every is greater than 0.
Stopping tolerance for updating document topic distribution in E-step.
Max number of iterations for updating document topic distribution in the E-step.
The number of jobs to use in the E-step. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Verbosity level.
Pass an int for reproducible results across multiple function calls. See Glossary.
Variational parameters for topic word distribution. Since the complete conditional for topic word distribution is a Dirichlet, components_[i, j] can be viewed as pseudocount that represents the number of times word j was assigned to topic i. It can also be viewed as distribution over the words for each topic after normalization: model.components_ / model.components_.sum(axis=1)[:, np.newaxis].
Exponential value of expectation of log topic word distribution. In the literature, this is exp(E[log(beta)]).
Number of iterations of the EM step.
Number of passes over the dataset.
Final perplexity score on training set.
Prior of document topic distribution theta. If the value is None, it is 1 / n_components.
RandomState instance that is generated either from a seed, the random number generator or by np.random.
Prior of topic word distribution beta. If the value is None, it is 1 / n_components.
“Online Learning for Latent Dirichlet Allocation”, Matthew D. Hoffman, David M. Blei, Francis Bach, 2010
Chong Wang, John Paisley, 2013
https://github.com/blei-lab/onlineldavb
fit(X[, y])
Learn model for the data X with variational Bayes method.
fit_transform(X[, y])
Fit to data, then transform it.
get_params([deep])
Get parameters for this estimator.
partial_fit(X[, y])
Online VB with Mini-Batch update.
perplexity(X[, sub_sampling])
Calculate approximate perplexity for data X.
score(X[, y])
Calculate approximate log-likelihood as score.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Transform data X according to the fitted model.
Learn model for the data X with variational Bayes method.
When learning_method is ‘online’, use mini-batch update. Otherwise, use batch update.
Document word matrix.
Fit to data, then transform it.
Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.
Input samples.
Target values (None for unsupervised transformations).
Additional fit parameters.
Transformed array.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Online VB with Mini-Batch update.
Document word matrix.
Calculate approximate perplexity for data X.
Perplexity is defined as exp(-1. * log-likelihood per word)
Changed in version 0.19: doc_topic_distr argument has been deprecated and is ignored because user no longer has access to unnormalized distribution
Document word matrix.
Do sub-sampling or not.
Perplexity score.
Calculate approximate log-likelihood as score.
Document word matrix.
Use approximate bound as score.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Transform data X according to the fitted model.
Changed in version 0.18: doc_topic_distr is now normalized
Document word matrix.
Document topic distribution for X.


Latent Dirichlet Allocation with online variational Bayes algorithm
New in version 0.17.
Read more in the User Guide.
Number of topics.
Changed in version 0.19: n_topics was renamed to n_components
Prior of document topic distribution theta. If the value is None, defaults to 1 / n_components. In [1], this is called alpha.
Prior of topic word distribution beta. If the value is None, defaults to 1 / n_components. In [1], this is called eta.
Method used to update _component. Only used in fit method. In general, if the data size is large, the online update will be much faster than the batch update.
Valid options:
Changed in version 0.20: The default learning method is now "batch".
It is a parameter that control learning rate in the online learning method. The value should be set between (0.5, 1.0] to guarantee asymptotic convergence. When the value is 0.0 and batch_size is n_samples, the update method is same as batch learning. In the literature, this is called kappa.
A (positive) parameter that downweights early iterations in online learning. It should be greater than 1.0. In the literature, this is called tau_0.
The maximum number of iterations.
Number of documents to use in each EM iteration. Only used in online learning.
How often to evaluate perplexity. Only used in fit method. set it to 0 or negative number to not evaluate perplexity in training at all. Evaluating perplexity can help you check convergence in training process, but it will also increase total training time. Evaluating perplexity in every iteration might increase training time up to two-fold.
Total number of documents. Only used in the partial_fit method.
Perplexity tolerance in batch learning. Only used when evaluate_every is greater than 0.
Stopping tolerance for updating document topic distribution in E-step.
Max number of iterations for updating document topic distribution in the E-step.
The number of jobs to use in the E-step. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Verbosity level.
Pass an int for reproducible results across multiple function calls. See Glossary.
Variational parameters for topic word distribution. Since the complete conditional for topic word distribution is a Dirichlet, components_[i, j] can be viewed as pseudocount that represents the number of times word j was assigned to topic i. It can also be viewed as distribution over the words for each topic after normalization: model.components_ / model.components_.sum(axis=1)[:, np.newaxis].
Exponential value of expectation of log topic word distribution. In the literature, this is exp(E[log(beta)]).
Number of iterations of the EM step.
Number of passes over the dataset.
Final perplexity score on training set.
Prior of document topic distribution theta. If the value is None, it is 1 / n_components.
RandomState instance that is generated either from a seed, the random number generator or by np.random.
Prior of topic word distribution beta. If the value is None, it is 1 / n_components.
“Online Learning for Latent Dirichlet Allocation”, Matthew D. Hoffman, David M. Blei, Francis Bach, 2010
Chong Wang, John Paisley, 2013
https://github.com/blei-lab/onlineldavb
fit(X[, y])
Learn model for the data X with variational Bayes method.
fit_transform(X[, y])
Fit to data, then transform it.
get_params([deep])
Get parameters for this estimator.
partial_fit(X[, y])
Online VB with Mini-Batch update.
perplexity(X[, sub_sampling])
Calculate approximate perplexity for data X.
score(X[, y])
Calculate approximate log-likelihood as score.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Transform data X according to the fitted model.
Learn model for the data X with variational Bayes method.
When learning_method is ‘online’, use mini-batch update. Otherwise, use batch update.
Document word matrix.
Fit to data, then transform it.
Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.
Input samples.
Target values (None for unsupervised transformations).
Additional fit parameters.
Transformed array.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Online VB with Mini-Batch update.
Document word matrix.
Calculate approximate perplexity for data X.
Perplexity is defined as exp(-1. * log-likelihood per word)
Changed in version 0.19: doc_topic_distr argument has been deprecated and is ignored because user no longer has access to unnormalized distribution
Document word matrix.
Do sub-sampling or not.
Perplexity score.
Calculate approximate log-likelihood as score.
Document word matrix.
Use approximate bound as score.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Transform data X according to the fitted model.
Changed in version 0.18: doc_topic_distr is now normalized
Document word matrix.
Document topic distribution for X.
Topic extraction with Non-negative Matrix Factorization and Latent Dirichlet Allocation


Learn model for the data X with variational Bayes method.
When learning_method is ‘online’, use mini-batch update. Otherwise, use batch update.
Document word matrix.


Online VB with Mini-Batch update.
Document word matrix.


Calculate approximate perplexity for data X.
Perplexity is defined as exp(-1. * log-likelihood per word)
Changed in version 0.19: doc_topic_distr argument has been deprecated and is ignored because user no longer has access to unnormalized distribution
Document word matrix.
Do sub-sampling or not.
Perplexity score.


Calculate approximate log-likelihood as score.
Document word matrix.
Use approximate bound as score.


Transform data X according to the fitted model.
Changed in version 0.18: doc_topic_distr is now normalized
Document word matrix.
Document topic distribution for X.


Mini-batch dictionary learning
Finds a dictionary (a set of atoms) that can best be used to represent data using a sparse code.
Solves the optimization problem:
Read more in the User Guide.
Number of dictionary elements to extract.
Sparsity controlling parameter.
Total number of iterations to perform.
The algorithm used:
Number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Number of samples in each mini-batch.
Whether to shuffle the samples before forming batches.
initial value of the dictionary for warm restart scenarios
Algorithm used to transform the data:
Number of nonzero coefficients to target in each column of the solution. This is only used by algorithm='lars' and algorithm='omp' and is overridden by alpha in the omp case. If None, then transform_n_nonzero_coefs=int(n_features / 10).
If algorithm='lasso_lars' or algorithm='lasso_cd', alpha is the penalty applied to the L1 norm. If algorithm='threshold', alpha is the absolute value of the threshold below which coefficients will be squashed to zero. If algorithm='omp', alpha is the tolerance parameter: the value of the reconstruction error targeted. In this case, it overrides n_nonzero_coefs. If None, default to 1.
To control the verbosity of the procedure.
Whether to split the sparse feature vector into the concatenation of its negative part and its positive part. This can improve the performance of downstream classifiers.
Used for initializing the dictionary when dict_init is not specified, randomly shuffling the data when shuffle is set to True, and updating the dictionary. Pass an int for reproducible results across multiple function calls. See Glossary.
Whether to enforce positivity when finding the code.
New in version 0.20.
Whether to enforce positivity when finding the dictionary.
New in version 0.20.
Maximum number of iterations to perform if algorithm='lasso_cd' or 'lasso_lars'.
New in version 0.22.
Components extracted from the data.
Internal sufficient statistics that are kept by the algorithm. Keeping them is useful in online settings, to avoid losing the history of the evolution, but they shouldn’t have any use for the end user. A (n_components, n_components) is the dictionary covariance matrix. B (n_features, n_components) is the data approximation matrix.
Number of iterations run.
The number of iteration on data batches that has been performed before.
RandomState instance that is generated either from a seed, the random number generattor or by np.random.
See also
References:
J. Mairal, F. Bach, J. Ponce, G. Sapiro, 2009: Online dictionary learning for sparse coding (https://www.di.ens.fr/sierra/pdfs/icml09.pdf)
We can check the level of sparsity of X_transformed:
We can compare the average squared euclidean norm of the reconstruction error of the sparse coded signal relative to the squared euclidean norm of the original signal:
fit(X[, y])
Fit the model from data in X.
fit_transform(X[, y])
Fit to data, then transform it.
get_params([deep])
Get parameters for this estimator.
partial_fit(X[, y, iter_offset])
Updates the model using the data in X as a mini-batch.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Encode the data as a sparse combination of the dictionary atoms.
Fit the model from data in X.
Training vector, where n_samples in the number of samples and n_features is the number of features.
Returns the instance itself.
Fit to data, then transform it.
Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.
Input samples.
Target values (None for unsupervised transformations).
Additional fit parameters.
Transformed array.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Updates the model using the data in X as a mini-batch.
Training vector, where n_samples in the number of samples and n_features is the number of features.
The number of iteration on data batches that has been performed before this call to partial_fit. This is optional: if no number is passed, the memory of the object is used.
Returns the instance itself.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Encode the data as a sparse combination of the dictionary atoms.
Coding method is determined by the object parameter transform_algorithm.
Test data to be transformed, must have the same number of features as the data used to train the model.
Transformed data.


Mini-batch dictionary learning
Finds a dictionary (a set of atoms) that can best be used to represent data using a sparse code.
Solves the optimization problem:
Read more in the User Guide.
Number of dictionary elements to extract.
Sparsity controlling parameter.
Total number of iterations to perform.
The algorithm used:
Number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Number of samples in each mini-batch.
Whether to shuffle the samples before forming batches.
initial value of the dictionary for warm restart scenarios
Algorithm used to transform the data:
Number of nonzero coefficients to target in each column of the solution. This is only used by algorithm='lars' and algorithm='omp' and is overridden by alpha in the omp case. If None, then transform_n_nonzero_coefs=int(n_features / 10).
If algorithm='lasso_lars' or algorithm='lasso_cd', alpha is the penalty applied to the L1 norm. If algorithm='threshold', alpha is the absolute value of the threshold below which coefficients will be squashed to zero. If algorithm='omp', alpha is the tolerance parameter: the value of the reconstruction error targeted. In this case, it overrides n_nonzero_coefs. If None, default to 1.
To control the verbosity of the procedure.
Whether to split the sparse feature vector into the concatenation of its negative part and its positive part. This can improve the performance of downstream classifiers.
Used for initializing the dictionary when dict_init is not specified, randomly shuffling the data when shuffle is set to True, and updating the dictionary. Pass an int for reproducible results across multiple function calls. See Glossary.
Whether to enforce positivity when finding the code.
New in version 0.20.
Whether to enforce positivity when finding the dictionary.
New in version 0.20.
Maximum number of iterations to perform if algorithm='lasso_cd' or 'lasso_lars'.
New in version 0.22.
Components extracted from the data.
Internal sufficient statistics that are kept by the algorithm. Keeping them is useful in online settings, to avoid losing the history of the evolution, but they shouldn’t have any use for the end user. A (n_components, n_components) is the dictionary covariance matrix. B (n_features, n_components) is the data approximation matrix.
Number of iterations run.
The number of iteration on data batches that has been performed before.
RandomState instance that is generated either from a seed, the random number generattor or by np.random.
See also
References:
J. Mairal, F. Bach, J. Ponce, G. Sapiro, 2009: Online dictionary learning for sparse coding (https://www.di.ens.fr/sierra/pdfs/icml09.pdf)
We can check the level of sparsity of X_transformed:
We can compare the average squared euclidean norm of the reconstruction error of the sparse coded signal relative to the squared euclidean norm of the original signal:
fit(X[, y])
Fit the model from data in X.
fit_transform(X[, y])
Fit to data, then transform it.
get_params([deep])
Get parameters for this estimator.
partial_fit(X[, y, iter_offset])
Updates the model using the data in X as a mini-batch.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Encode the data as a sparse combination of the dictionary atoms.
Fit the model from data in X.
Training vector, where n_samples in the number of samples and n_features is the number of features.
Returns the instance itself.
Fit to data, then transform it.
Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.
Input samples.
Target values (None for unsupervised transformations).
Additional fit parameters.
Transformed array.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Updates the model using the data in X as a mini-batch.
Training vector, where n_samples in the number of samples and n_features is the number of features.
The number of iteration on data batches that has been performed before this call to partial_fit. This is optional: if no number is passed, the memory of the object is used.
Returns the instance itself.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Encode the data as a sparse combination of the dictionary atoms.
Coding method is determined by the object parameter transform_algorithm.
Test data to be transformed, must have the same number of features as the data used to train the model.
Transformed data.
Image denoising using dictionary learning
Faces dataset decompositions


Fit to data, then transform it.
Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.
Input samples.
Target values (None for unsupervised transformations).
Additional fit parameters.
Transformed array.


Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.


Updates the model using the data in X as a mini-batch.
Training vector, where n_samples in the number of samples and n_features is the number of features.
The number of iteration on data batches that has been performed before this call to partial_fit. This is optional: if no number is passed, the memory of the object is used.
Returns the instance itself.


Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.


Encode the data as a sparse combination of the dictionary atoms.
Coding method is determined by the object parameter transform_algorithm.
Test data to be transformed, must have the same number of features as the data used to train the model.
Transformed data.


Mini-batch Sparse Principal Components Analysis
Finds the set of sparse components that can optimally reconstruct the data. The amount of sparseness is controllable by the coefficient of the L1 penalty, given by the parameter alpha.
Read more in the User Guide.
number of sparse atoms to extract
Sparsity controlling parameter. Higher values lead to sparser components.
Amount of ridge shrinkage to apply in order to improve conditioning when calling the transform method.
number of iterations to perform for each mini batch
callable that gets invoked every five iterations
the number of features to take in each mini batch
Controls the verbosity; the higher, the more messages. Defaults to 0.
whether to shuffle the data before splitting it in batches
Number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
lars: uses the least angle regression method to solve the lasso problem (linear_model.lars_path) cd: uses the coordinate descent method to compute the Lasso solution (linear_model.Lasso). Lars will be faster if the estimated components are sparse.
Used for random shuffling when shuffle is set to True, during online dictionary learning. Pass an int for reproducible results across multiple function calls. See Glossary.
Sparse components extracted from the data.
Estimated number of components.
New in version 0.23.
Number of iterations run.
Per-feature empirical mean, estimated from the training set. Equal to X.mean(axis=0).
See also
fit(X[, y])
Fit the model from data in X.
fit_transform(X[, y])
Fit to data, then transform it.
get_params([deep])
Get parameters for this estimator.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Least Squares projection of the data onto the sparse components.
Fit the model from data in X.
Training vector, where n_samples in the number of samples and n_features is the number of features.
Returns the instance itself.
Fit to data, then transform it.
Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.
Input samples.
Target values (None for unsupervised transformations).
Additional fit parameters.
Transformed array.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Least Squares projection of the data onto the sparse components.
To avoid instability issues in case the system is under-determined, regularization can be applied (Ridge regression) via the ridge_alpha parameter.
Note that Sparse PCA components orthogonality is not enforced as in PCA hence one cannot use a simple linear projection.
Test data to be transformed, must have the same number of features as the data used to train the model.
Transformed data.


Mini-batch Sparse Principal Components Analysis
Finds the set of sparse components that can optimally reconstruct the data. The amount of sparseness is controllable by the coefficient of the L1 penalty, given by the parameter alpha.
Read more in the User Guide.
number of sparse atoms to extract
Sparsity controlling parameter. Higher values lead to sparser components.
Amount of ridge shrinkage to apply in order to improve conditioning when calling the transform method.
number of iterations to perform for each mini batch
callable that gets invoked every five iterations
the number of features to take in each mini batch
Controls the verbosity; the higher, the more messages. Defaults to 0.
whether to shuffle the data before splitting it in batches
Number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
lars: uses the least angle regression method to solve the lasso problem (linear_model.lars_path) cd: uses the coordinate descent method to compute the Lasso solution (linear_model.Lasso). Lars will be faster if the estimated components are sparse.
Used for random shuffling when shuffle is set to True, during online dictionary learning. Pass an int for reproducible results across multiple function calls. See Glossary.
Sparse components extracted from the data.
Estimated number of components.
New in version 0.23.
Number of iterations run.
Per-feature empirical mean, estimated from the training set. Equal to X.mean(axis=0).
See also
fit(X[, y])
Fit the model from data in X.
fit_transform(X[, y])
Fit to data, then transform it.
get_params([deep])
Get parameters for this estimator.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Least Squares projection of the data onto the sparse components.
Fit the model from data in X.
Training vector, where n_samples in the number of samples and n_features is the number of features.
Returns the instance itself.
Fit to data, then transform it.
Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.
Input samples.
Target values (None for unsupervised transformations).
Additional fit parameters.
Transformed array.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Least Squares projection of the data onto the sparse components.
To avoid instability issues in case the system is under-determined, regularization can be applied (Ridge regression) via the ridge_alpha parameter.
Note that Sparse PCA components orthogonality is not enforced as in PCA hence one cannot use a simple linear projection.
Test data to be transformed, must have the same number of features as the data used to train the model.
Transformed data.
Faces dataset decompositions


Fit the model from data in X.
Training vector, where n_samples in the number of samples and n_features is the number of features.
Returns the instance itself.


Least Squares projection of the data onto the sparse components.
To avoid instability issues in case the system is under-determined, regularization can be applied (Ridge regression) via the ridge_alpha parameter.
Note that Sparse PCA components orthogonality is not enforced as in PCA hence one cannot use a simple linear projection.
Test data to be transformed, must have the same number of features as the data used to train the model.
Transformed data.


Non-Negative Matrix Factorization (NMF).
Find two non-negative matrices (W, H) whose product approximates the non- negative matrix X. This factorization can be used for example for dimensionality reduction, source separation or topic extraction.
The objective function is:
Where:
\(||A||_{Fro}^2 = \sum_{i,j} A_{ij}^2\) (Frobenius norm)
\(||vec(A)||_1 = \sum_{i,j} abs(A_{ij})\) (Elementwise L1 norm)
For multiplicative-update (‘mu’) solver, the Frobenius norm (\(0.5 * ||X - WH||_{Fro}^2\)) can be changed into another beta-divergence loss, by changing the beta_loss parameter.
The objective function is minimized with an alternating minimization of W and H.
Read more in the User Guide.
Number of components, if n_components is not set all features are kept.
Method used to initialize the procedure. Default: None. Valid options:
Numerical solver to use: ‘cd’ is a Coordinate Descent solver. ‘mu’ is a Multiplicative Update solver.
New in version 0.17: Coordinate Descent solver.
New in version 0.19: Multiplicative Update solver.
Beta divergence to be minimized, measuring the distance between X and the dot product WH. Note that values different from ‘frobenius’ (or 2) and ‘kullback-leibler’ (or 1) lead to significantly slower fits. Note that for beta_loss <= 0 (or ‘itakura-saito’), the input matrix X cannot contain zeros. Used only in ‘mu’ solver.
New in version 0.19.
Tolerance of the stopping condition.
Maximum number of iterations before timing out.
Used for initialisation (when init == ‘nndsvdar’ or ‘random’), and in Coordinate Descent. Pass an int for reproducible results across multiple function calls. See Glossary.
Constant that multiplies the regularization terms. Set it to zero to have no regularization.
New in version 0.17: alpha used in the Coordinate Descent solver.
The regularization mixing parameter, with 0 <= l1_ratio <= 1. For l1_ratio = 0 the penalty is an elementwise L2 penalty (aka Frobenius Norm). For l1_ratio = 1 it is an elementwise L1 penalty. For 0 < l1_ratio < 1, the penalty is a combination of L1 and L2.
New in version 0.17: Regularization parameter l1_ratio used in the Coordinate Descent solver.
Whether to be verbose.
If true, randomize the order of coordinates in the CD solver.
New in version 0.17: shuffle parameter used in the Coordinate Descent solver.
Select whether the regularization affects the components (H), the transformation (W), both or none of them.
New in version 0.24.
Factorization matrix, sometimes called ‘dictionary’.
The number of components. It is same as the n_components parameter if it was given. Otherwise, it will be same as the number of features.
Frobenius norm of the matrix difference, or beta-divergence, between the training data X and the reconstructed data WH from the fitted model.
Actual number of iterations.
Cichocki, Andrzej, and P. H. A. N. Anh-Huy. “Fast local algorithms for large scale nonnegative matrix and tensor factorizations.” IEICE transactions on fundamentals of electronics, communications and computer sciences 92.3: 708-721, 2009.
Fevotte, C., & Idier, J. (2011). Algorithms for nonnegative matrix factorization with the beta-divergence. Neural Computation, 23(9).
fit(X[, y])
Learn a NMF model for the data X.
fit_transform(X[, y, W, H])
Learn a NMF model for the data X and returns the transformed data.
get_params([deep])
Get parameters for this estimator.
inverse_transform(W)
Transform data back to its original space.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Transform the data X according to the fitted NMF model.
Learn a NMF model for the data X.
Data matrix to be decomposed
Learn a NMF model for the data X and returns the transformed data.
This is more efficient than calling fit followed by transform.
Data matrix to be decomposed
If init=’custom’, it is used as initial guess for the solution.
If init=’custom’, it is used as initial guess for the solution.
Transformed data.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Transform data back to its original space.
Transformed data matrix.
Data matrix of original shape.
New in version 0.18: ..
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Transform the data X according to the fitted NMF model.
Data matrix to be transformed by the model.
Transformed data.


Non-Negative Matrix Factorization (NMF).
Find two non-negative matrices (W, H) whose product approximates the non- negative matrix X. This factorization can be used for example for dimensionality reduction, source separation or topic extraction.
The objective function is:
Where:
\(||A||_{Fro}^2 = \sum_{i,j} A_{ij}^2\) (Frobenius norm)
\(||vec(A)||_1 = \sum_{i,j} abs(A_{ij})\) (Elementwise L1 norm)
For multiplicative-update (‘mu’) solver, the Frobenius norm (\(0.5 * ||X - WH||_{Fro}^2\)) can be changed into another beta-divergence loss, by changing the beta_loss parameter.
The objective function is minimized with an alternating minimization of W and H.
Read more in the User Guide.
Number of components, if n_components is not set all features are kept.
Method used to initialize the procedure. Default: None. Valid options:
Numerical solver to use: ‘cd’ is a Coordinate Descent solver. ‘mu’ is a Multiplicative Update solver.
New in version 0.17: Coordinate Descent solver.
New in version 0.19: Multiplicative Update solver.
Beta divergence to be minimized, measuring the distance between X and the dot product WH. Note that values different from ‘frobenius’ (or 2) and ‘kullback-leibler’ (or 1) lead to significantly slower fits. Note that for beta_loss <= 0 (or ‘itakura-saito’), the input matrix X cannot contain zeros. Used only in ‘mu’ solver.
New in version 0.19.
Tolerance of the stopping condition.
Maximum number of iterations before timing out.
Used for initialisation (when init == ‘nndsvdar’ or ‘random’), and in Coordinate Descent. Pass an int for reproducible results across multiple function calls. See Glossary.
Constant that multiplies the regularization terms. Set it to zero to have no regularization.
New in version 0.17: alpha used in the Coordinate Descent solver.
The regularization mixing parameter, with 0 <= l1_ratio <= 1. For l1_ratio = 0 the penalty is an elementwise L2 penalty (aka Frobenius Norm). For l1_ratio = 1 it is an elementwise L1 penalty. For 0 < l1_ratio < 1, the penalty is a combination of L1 and L2.
New in version 0.17: Regularization parameter l1_ratio used in the Coordinate Descent solver.
Whether to be verbose.
If true, randomize the order of coordinates in the CD solver.
New in version 0.17: shuffle parameter used in the Coordinate Descent solver.
Select whether the regularization affects the components (H), the transformation (W), both or none of them.
New in version 0.24.
Factorization matrix, sometimes called ‘dictionary’.
The number of components. It is same as the n_components parameter if it was given. Otherwise, it will be same as the number of features.
Frobenius norm of the matrix difference, or beta-divergence, between the training data X and the reconstructed data WH from the fitted model.
Actual number of iterations.
Cichocki, Andrzej, and P. H. A. N. Anh-Huy. “Fast local algorithms for large scale nonnegative matrix and tensor factorizations.” IEICE transactions on fundamentals of electronics, communications and computer sciences 92.3: 708-721, 2009.
Fevotte, C., & Idier, J. (2011). Algorithms for nonnegative matrix factorization with the beta-divergence. Neural Computation, 23(9).
fit(X[, y])
Learn a NMF model for the data X.
fit_transform(X[, y, W, H])
Learn a NMF model for the data X and returns the transformed data.
get_params([deep])
Get parameters for this estimator.
inverse_transform(W)
Transform data back to its original space.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Transform the data X according to the fitted NMF model.
Learn a NMF model for the data X.
Data matrix to be decomposed
Learn a NMF model for the data X and returns the transformed data.
This is more efficient than calling fit followed by transform.
Data matrix to be decomposed
If init=’custom’, it is used as initial guess for the solution.
If init=’custom’, it is used as initial guess for the solution.
Transformed data.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Transform data back to its original space.
Transformed data matrix.
Data matrix of original shape.
New in version 0.18: ..
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Transform the data X according to the fitted NMF model.
Data matrix to be transformed by the model.
Transformed data.
Beta-divergence loss functions
Faces dataset decompositions
Topic extraction with Non-negative Matrix Factorization and Latent Dirichlet Allocation
Selecting dimensionality reduction with Pipeline and GridSearchCV


Learn a NMF model for the data X.
Data matrix to be decomposed


Learn a NMF model for the data X and returns the transformed data.
This is more efficient than calling fit followed by transform.
Data matrix to be decomposed
If init=’custom’, it is used as initial guess for the solution.
If init=’custom’, it is used as initial guess for the solution.
Transformed data.


Transform data back to its original space.
Transformed data matrix.
Data matrix of original shape.
New in version 0.18: ..


Transform the data X according to the fitted NMF model.
Data matrix to be transformed by the model.
Transformed data.


Compute Non-negative Matrix Factorization (NMF).
Find two non-negative matrices (W, H) whose product approximates the non- negative matrix X. This factorization can be used for example for dimensionality reduction, source separation or topic extraction.
The objective function is:
Where:
\(||A||_{Fro}^2 = \sum_{i,j} A_{ij}^2\) (Frobenius norm)
\(||vec(A)||_1 = \sum_{i,j} abs(A_{ij})\) (Elementwise L1 norm)
For multiplicative-update (‘mu’) solver, the Frobenius norm \((0.5 * ||X - WH||_{Fro}^2)\) can be changed into another beta-divergence loss, by changing the beta_loss parameter.
The objective function is minimized with an alternating minimization of W and H. If H is given and update_H=False, it solves for W only.
Constant matrix.
If init=’custom’, it is used as initial guess for the solution.
If init=’custom’, it is used as initial guess for the solution. If update_H=False, it is used as a constant, to solve for W only.
Number of components, if n_components is not set all features are kept.
Method used to initialize the procedure.
Valid options:
sqrt(X.mean() / n_components)
initialization (better for sparseness)
(better when sparsity is not desired)
(generally faster, less accurate alternative to NNDSVDa for when sparsity is not desired)
Changed in version 0.23: The default value of init changed from ‘random’ to None in 0.23.
Set to True, both W and H will be estimated from initial guesses. Set to False, only W will be estimated.
Numerical solver to use:
Alternating Least Squares (Fast HALS).
New in version 0.17: Coordinate Descent solver.
New in version 0.19: Multiplicative Update solver.
Beta divergence to be minimized, measuring the distance between X and the dot product WH. Note that values different from ‘frobenius’ (or 2) and ‘kullback-leibler’ (or 1) lead to significantly slower fits. Note that for beta_loss <= 0 (or ‘itakura-saito’), the input matrix X cannot contain zeros. Used only in ‘mu’ solver.
New in version 0.19.
Tolerance of the stopping condition.
Maximum number of iterations before timing out.
Constant that multiplies the regularization terms.
The regularization mixing parameter, with 0 <= l1_ratio <= 1. For l1_ratio = 0 the penalty is an elementwise L2 penalty (aka Frobenius Norm). For l1_ratio = 1 it is an elementwise L1 penalty. For 0 < l1_ratio < 1, the penalty is a combination of L1 and L2.
Select whether the regularization affects the components (H), the transformation (W), both or none of them.
Used for NMF initialisation (when init == ‘nndsvdar’ or ‘random’), and in Coordinate Descent. Pass an int for reproducible results across multiple function calls. See Glossary.
The verbosity level.
If true, randomize the order of coordinates in the CD solver.
Solution to the non-negative least squares problem.
Solution to the non-negative least squares problem.
Actual number of iterations.
Cichocki, Andrzej, and P. H. A. N. Anh-Huy. “Fast local algorithms for large scale nonnegative matrix and tensor factorizations.” IEICE transactions on fundamentals of electronics, communications and computer sciences 92.3: 708-721, 2009.
Fevotte, C., & Idier, J. (2011). Algorithms for nonnegative matrix factorization with the beta-divergence. Neural Computation, 23(9).


Principal component analysis (PCA).
Linear dimensionality reduction using Singular Value Decomposition of the data to project it to a lower dimensional space. The input data is centered but not scaled for each feature before applying the SVD.
It uses the LAPACK implementation of the full SVD or a randomized truncated SVD by the method of Halko et al. 2009, depending on the shape of the input data and the number of components to extract.
It can also use the scipy.sparse.linalg ARPACK implementation of the truncated SVD.
Notice that this class does not support sparse input. See TruncatedSVD for an alternative with sparse data.
Read more in the User Guide.
Number of components to keep. if n_components is not set all components are kept:
If n_components == 'mle' and svd_solver == 'full', Minka’s MLE is used to guess the dimension. Use of n_components == 'mle' will interpret svd_solver == 'auto' as svd_solver == 'full'.
If 0 < n_components < 1 and svd_solver == 'full', select the number of components such that the amount of variance that needs to be explained is greater than the percentage specified by n_components.
If svd_solver == 'arpack', the number of components must be strictly less than the minimum of n_features and n_samples.
Hence, the None case results in:
If False, data passed to fit are overwritten and running fit(X).transform(X) will not yield the expected results, use fit_transform(X) instead.
When True (False by default) the components_ vectors are multiplied by the square root of n_samples and then divided by the singular values to ensure uncorrelated outputs with unit component-wise variances.
Whitening will remove some information from the transformed signal (the relative variance scales of the components) but can sometime improve the predictive accuracy of the downstream estimators by making their data respect some hard-wired assumptions.
The solver is selected by a default policy based on X.shape and n_components: if the input data is larger than 500x500 and the number of components to extract is lower than 80% of the smallest dimension of the data, then the more efficient ‘randomized’ method is enabled. Otherwise the exact full SVD is computed and optionally truncated afterwards.
run exact full SVD calling the standard LAPACK solver via scipy.linalg.svd and select the components by postprocessing
run SVD truncated to n_components calling ARPACK solver via scipy.sparse.linalg.svds. It requires strictly 0 < n_components < min(X.shape)
run randomized SVD by the method of Halko et al.
New in version 0.18.0.
Tolerance for singular values computed by svd_solver == ‘arpack’. Must be of range [0.0, infinity).
New in version 0.18.0.
Number of iterations for the power method computed by svd_solver == ‘randomized’. Must be of range [0, infinity).
New in version 0.18.0.
Used when the ‘arpack’ or ‘randomized’ solvers are used. Pass an int for reproducible results across multiple function calls. See Glossary.
New in version 0.18.0.
Principal axes in feature space, representing the directions of maximum variance in the data. The components are sorted by explained_variance_.
The amount of variance explained by each of the selected components.
Equal to n_components largest eigenvalues of the covariance matrix of X.
New in version 0.18.
Percentage of variance explained by each of the selected components.
If n_components is not set then all components are stored and the sum of the ratios is equal to 1.0.
The singular values corresponding to each of the selected components. The singular values are equal to the 2-norms of the n_components variables in the lower-dimensional space.
New in version 0.19.
Per-feature empirical mean, estimated from the training set.
Equal to X.mean(axis=0).
The estimated number of components. When n_components is set to ‘mle’ or a number between 0 and 1 (with svd_solver == ‘full’) this number is estimated from input data. Otherwise it equals the parameter n_components, or the lesser value of n_features and n_samples if n_components is None.
Number of features in the training data.
Number of samples in the training data.
The estimated noise covariance following the Probabilistic PCA model from Tipping and Bishop 1999. See “Pattern Recognition and Machine Learning” by C. Bishop, 12.2.1 p. 574 or http://www.miketipping.com/papers/met-mppca.pdf. It is required to compute the estimated data covariance and score samples.
Equal to the average of (min(n_features, n_samples) - n_components) smallest eigenvalues of the covariance matrix of X.
See also
Kernel Principal Component Analysis.
Sparse Principal Component Analysis.
Dimensionality reduction using truncated SVD.
Incremental Principal Component Analysis.
For n_components == ‘mle’, this class uses the method of Minka, T. P. “Automatic choice of dimensionality for PCA”. In NIPS, pp. 598-604
Implements the probabilistic PCA model from: Tipping, M. E., and Bishop, C. M. (1999). “Probabilistic principal component analysis”. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 61(3), 611-622. via the score and score_samples methods. See http://www.miketipping.com/papers/met-mppca.pdf
For svd_solver == ‘arpack’, refer to scipy.sparse.linalg.svds.
For svd_solver == ‘randomized’, see: Halko, N., Martinsson, P. G., and Tropp, J. A. (2011). “Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions”. SIAM review, 53(2), 217-288. and also Martinsson, P. G., Rokhlin, V., and Tygert, M. (2011). “A randomized algorithm for the decomposition of matrices”. Applied and Computational Harmonic Analysis, 30(1), 47-68.
fit(X[, y])
Fit the model with X.
fit_transform(X[, y])
Fit the model with X and apply the dimensionality reduction on X.
get_covariance()
Compute data covariance with the generative model.
get_params([deep])
Get parameters for this estimator.
get_precision()
Compute data precision matrix with the generative model.
inverse_transform(X)
Transform data back to its original space.
score(X[, y])
Return the average log-likelihood of all samples.
score_samples(X)
Return the log-likelihood of each sample.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Apply dimensionality reduction to X.
Fit the model with X.
Training data, where n_samples is the number of samples and n_features is the number of features.
Returns the instance itself.
Fit the model with X and apply the dimensionality reduction on X.
Training data, where n_samples is the number of samples and n_features is the number of features.
Transformed values.
This method returns a Fortran-ordered array. To convert it to a C-ordered array, use ‘np.ascontiguousarray’.
Compute data covariance with the generative model.
cov = components_.T * S**2 * components_ + sigma2 * eye(n_features) where S**2 contains the explained variances, and sigma2 contains the noise variances.
Estimated covariance of data.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Compute data precision matrix with the generative model.
Equals the inverse of the covariance but computed with the matrix inversion lemma for efficiency.
Estimated precision of data.
Transform data back to its original space.
In other words, return an input X_original whose transform would be X.
New data, where n_samples is the number of samples and n_components is the number of components.
If whitening is enabled, inverse_transform will compute the exact inverse operation, which includes reversing whitening.
Return the average log-likelihood of all samples.
See. “Pattern Recognition and Machine Learning” by C. Bishop, 12.2.1 p. 574 or http://www.miketipping.com/papers/met-mppca.pdf
The data.
Average log-likelihood of the samples under the current model.
Return the log-likelihood of each sample.
See. “Pattern Recognition and Machine Learning” by C. Bishop, 12.2.1 p. 574 or http://www.miketipping.com/papers/met-mppca.pdf
The data.
Log-likelihood of each sample under the current model.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Apply dimensionality reduction to X.
X is projected on the first principal components previously extracted from a training set.
New data, where n_samples is the number of samples and n_features is the number of features.


Principal component analysis (PCA).
Linear dimensionality reduction using Singular Value Decomposition of the data to project it to a lower dimensional space. The input data is centered but not scaled for each feature before applying the SVD.
It uses the LAPACK implementation of the full SVD or a randomized truncated SVD by the method of Halko et al. 2009, depending on the shape of the input data and the number of components to extract.
It can also use the scipy.sparse.linalg ARPACK implementation of the truncated SVD.
Notice that this class does not support sparse input. See TruncatedSVD for an alternative with sparse data.
Read more in the User Guide.
Number of components to keep. if n_components is not set all components are kept:
If n_components == 'mle' and svd_solver == 'full', Minka’s MLE is used to guess the dimension. Use of n_components == 'mle' will interpret svd_solver == 'auto' as svd_solver == 'full'.
If 0 < n_components < 1 and svd_solver == 'full', select the number of components such that the amount of variance that needs to be explained is greater than the percentage specified by n_components.
If svd_solver == 'arpack', the number of components must be strictly less than the minimum of n_features and n_samples.
Hence, the None case results in:
If False, data passed to fit are overwritten and running fit(X).transform(X) will not yield the expected results, use fit_transform(X) instead.
When True (False by default) the components_ vectors are multiplied by the square root of n_samples and then divided by the singular values to ensure uncorrelated outputs with unit component-wise variances.
Whitening will remove some information from the transformed signal (the relative variance scales of the components) but can sometime improve the predictive accuracy of the downstream estimators by making their data respect some hard-wired assumptions.
The solver is selected by a default policy based on X.shape and n_components: if the input data is larger than 500x500 and the number of components to extract is lower than 80% of the smallest dimension of the data, then the more efficient ‘randomized’ method is enabled. Otherwise the exact full SVD is computed and optionally truncated afterwards.
run exact full SVD calling the standard LAPACK solver via scipy.linalg.svd and select the components by postprocessing
run SVD truncated to n_components calling ARPACK solver via scipy.sparse.linalg.svds. It requires strictly 0 < n_components < min(X.shape)
run randomized SVD by the method of Halko et al.
New in version 0.18.0.
Tolerance for singular values computed by svd_solver == ‘arpack’. Must be of range [0.0, infinity).
New in version 0.18.0.
Number of iterations for the power method computed by svd_solver == ‘randomized’. Must be of range [0, infinity).
New in version 0.18.0.
Used when the ‘arpack’ or ‘randomized’ solvers are used. Pass an int for reproducible results across multiple function calls. See Glossary.
New in version 0.18.0.
Principal axes in feature space, representing the directions of maximum variance in the data. The components are sorted by explained_variance_.
The amount of variance explained by each of the selected components.
Equal to n_components largest eigenvalues of the covariance matrix of X.
New in version 0.18.
Percentage of variance explained by each of the selected components.
If n_components is not set then all components are stored and the sum of the ratios is equal to 1.0.
The singular values corresponding to each of the selected components. The singular values are equal to the 2-norms of the n_components variables in the lower-dimensional space.
New in version 0.19.
Per-feature empirical mean, estimated from the training set.
Equal to X.mean(axis=0).
The estimated number of components. When n_components is set to ‘mle’ or a number between 0 and 1 (with svd_solver == ‘full’) this number is estimated from input data. Otherwise it equals the parameter n_components, or the lesser value of n_features and n_samples if n_components is None.
Number of features in the training data.
Number of samples in the training data.
The estimated noise covariance following the Probabilistic PCA model from Tipping and Bishop 1999. See “Pattern Recognition and Machine Learning” by C. Bishop, 12.2.1 p. 574 or http://www.miketipping.com/papers/met-mppca.pdf. It is required to compute the estimated data covariance and score samples.
Equal to the average of (min(n_features, n_samples) - n_components) smallest eigenvalues of the covariance matrix of X.
See also
Kernel Principal Component Analysis.
Sparse Principal Component Analysis.
Dimensionality reduction using truncated SVD.
Incremental Principal Component Analysis.
For n_components == ‘mle’, this class uses the method of Minka, T. P. “Automatic choice of dimensionality for PCA”. In NIPS, pp. 598-604
Implements the probabilistic PCA model from: Tipping, M. E., and Bishop, C. M. (1999). “Probabilistic principal component analysis”. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 61(3), 611-622. via the score and score_samples methods. See http://www.miketipping.com/papers/met-mppca.pdf
For svd_solver == ‘arpack’, refer to scipy.sparse.linalg.svds.
For svd_solver == ‘randomized’, see: Halko, N., Martinsson, P. G., and Tropp, J. A. (2011). “Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions”. SIAM review, 53(2), 217-288. and also Martinsson, P. G., Rokhlin, V., and Tygert, M. (2011). “A randomized algorithm for the decomposition of matrices”. Applied and Computational Harmonic Analysis, 30(1), 47-68.
fit(X[, y])
Fit the model with X.
fit_transform(X[, y])
Fit the model with X and apply the dimensionality reduction on X.
get_covariance()
Compute data covariance with the generative model.
get_params([deep])
Get parameters for this estimator.
get_precision()
Compute data precision matrix with the generative model.
inverse_transform(X)
Transform data back to its original space.
score(X[, y])
Return the average log-likelihood of all samples.
score_samples(X)
Return the log-likelihood of each sample.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Apply dimensionality reduction to X.
Fit the model with X.
Training data, where n_samples is the number of samples and n_features is the number of features.
Returns the instance itself.
Fit the model with X and apply the dimensionality reduction on X.
Training data, where n_samples is the number of samples and n_features is the number of features.
Transformed values.
This method returns a Fortran-ordered array. To convert it to a C-ordered array, use ‘np.ascontiguousarray’.
Compute data covariance with the generative model.
cov = components_.T * S**2 * components_ + sigma2 * eye(n_features) where S**2 contains the explained variances, and sigma2 contains the noise variances.
Estimated covariance of data.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Compute data precision matrix with the generative model.
Equals the inverse of the covariance but computed with the matrix inversion lemma for efficiency.
Estimated precision of data.
Transform data back to its original space.
In other words, return an input X_original whose transform would be X.
New data, where n_samples is the number of samples and n_components is the number of components.
If whitening is enabled, inverse_transform will compute the exact inverse operation, which includes reversing whitening.
Return the average log-likelihood of all samples.
See. “Pattern Recognition and Machine Learning” by C. Bishop, 12.2.1 p. 574 or http://www.miketipping.com/papers/met-mppca.pdf
The data.
Average log-likelihood of the samples under the current model.
Return the log-likelihood of each sample.
See. “Pattern Recognition and Machine Learning” by C. Bishop, 12.2.1 p. 574 or http://www.miketipping.com/papers/met-mppca.pdf
The data.
Log-likelihood of each sample under the current model.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Apply dimensionality reduction to X.
X is projected on the first principal components previously extracted from a training set.
New data, where n_samples is the number of samples and n_features is the number of features.
A demo of K-Means clustering on the handwritten digits data
Principal Component Regression vs Partial Least Squares Regression
The Iris Dataset
PCA example with Iris Data-set
Incremental PCA
Comparison of LDA and PCA 2D projection of Iris dataset
Factor Analysis (with rotation) to visualize patterns
Blind source separation using FastICA
Principal components analysis (PCA)
FastICA on 2D point clouds
Kernel PCA
Model selection with Probabilistic PCA and Factor Analysis (FA)
Faces dataset decompositions
Faces recognition example using eigenfaces and SVMs
Multi-dimensional scaling
Multilabel classification
Explicit feature map approximation for RBF kernels
Balance model complexity and cross-validated score
Kernel Density Estimation
Dimensionality Reduction with Neighborhood Components Analysis
Concatenating multiple feature extraction methods
Pipelining: chaining a PCA and a logistic regression
Selecting dimensionality reduction with Pipeline and GridSearchCV
Importance of Feature Scaling


Fit the model with X.
Training data, where n_samples is the number of samples and n_features is the number of features.
Returns the instance itself.


Fit the model with X and apply the dimensionality reduction on X.
Training data, where n_samples is the number of samples and n_features is the number of features.
Transformed values.
This method returns a Fortran-ordered array. To convert it to a C-ordered array, use ‘np.ascontiguousarray’.


Compute data covariance with the generative model.
cov = components_.T * S**2 * components_ + sigma2 * eye(n_features) where S**2 contains the explained variances, and sigma2 contains the noise variances.
Estimated covariance of data.


Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.


Compute data precision matrix with the generative model.
Equals the inverse of the covariance but computed with the matrix inversion lemma for efficiency.
Estimated precision of data.


Transform data back to its original space.
In other words, return an input X_original whose transform would be X.
New data, where n_samples is the number of samples and n_components is the number of components.
If whitening is enabled, inverse_transform will compute the exact inverse operation, which includes reversing whitening.


Return the average log-likelihood of all samples.
See. “Pattern Recognition and Machine Learning” by C. Bishop, 12.2.1 p. 574 or http://www.miketipping.com/papers/met-mppca.pdf
The data.
Average log-likelihood of the samples under the current model.


Return the log-likelihood of each sample.
See. “Pattern Recognition and Machine Learning” by C. Bishop, 12.2.1 p. 574 or http://www.miketipping.com/papers/met-mppca.pdf
The data.
Log-likelihood of each sample under the current model.


Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.


Apply dimensionality reduction to X.
X is projected on the first principal components previously extracted from a training set.
New data, where n_samples is the number of samples and n_features is the number of features.


Sparse coding
Finds a sparse representation of data against a fixed, precomputed dictionary.
Each row of the result is the solution to a sparse coding problem. The goal is to find a sparse array code such that:
Read more in the User Guide.
The dictionary atoms used for sparse coding. Lines are assumed to be normalized to unit norm.
Algorithm used to transform the data:
Number of nonzero coefficients to target in each column of the solution. This is only used by algorithm='lars' and algorithm='omp' and is overridden by alpha in the omp case. If None, then transform_n_nonzero_coefs=int(n_features / 10).
If algorithm='lasso_lars' or algorithm='lasso_cd', alpha is the penalty applied to the L1 norm. If algorithm='threshold', alpha is the absolute value of the threshold below which coefficients will be squashed to zero. If algorithm='omp', alpha is the tolerance parameter: the value of the reconstruction error targeted. In this case, it overrides n_nonzero_coefs. If None, default to 1.
Whether to split the sparse feature vector into the concatenation of its negative part and its positive part. This can improve the performance of downstream classifiers.
Number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Whether to enforce positivity when finding the code.
New in version 0.20.
Maximum number of iterations to perform if algorithm='lasso_cd' or lasso_lars.
New in version 0.22.
The unchanged dictionary atoms.
Deprecated since version 0.24: This attribute is deprecated in 0.24 and will be removed in 1.1 (renaming of 0.26). Use dictionary instead.
See also
fit(X[, y])
Do nothing and return the estimator unchanged.
fit_transform(X[, y])
Fit to data, then transform it.
get_params([deep])
Get parameters for this estimator.
set_params(**params)
Set the parameters of this estimator.
transform(X[, y])
Encode the data as a sparse combination of the dictionary atoms.
Do nothing and return the estimator unchanged.
This method is just there to implement the usual API and hence work in pipelines.
Fit to data, then transform it.
Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.
Input samples.
Target values (None for unsupervised transformations).
Additional fit parameters.
Transformed array.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Encode the data as a sparse combination of the dictionary atoms.
Coding method is determined by the object parameter transform_algorithm.
Test data to be transformed, must have the same number of features as the data used to train the model.
Transformed data.


Sparse coding
Finds a sparse representation of data against a fixed, precomputed dictionary.
Each row of the result is the solution to a sparse coding problem. The goal is to find a sparse array code such that:
Read more in the User Guide.
The dictionary atoms used for sparse coding. Lines are assumed to be normalized to unit norm.
Algorithm used to transform the data:
Number of nonzero coefficients to target in each column of the solution. This is only used by algorithm='lars' and algorithm='omp' and is overridden by alpha in the omp case. If None, then transform_n_nonzero_coefs=int(n_features / 10).
If algorithm='lasso_lars' or algorithm='lasso_cd', alpha is the penalty applied to the L1 norm. If algorithm='threshold', alpha is the absolute value of the threshold below which coefficients will be squashed to zero. If algorithm='omp', alpha is the tolerance parameter: the value of the reconstruction error targeted. In this case, it overrides n_nonzero_coefs. If None, default to 1.
Whether to split the sparse feature vector into the concatenation of its negative part and its positive part. This can improve the performance of downstream classifiers.
Number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Whether to enforce positivity when finding the code.
New in version 0.20.
Maximum number of iterations to perform if algorithm='lasso_cd' or lasso_lars.
New in version 0.22.
The unchanged dictionary atoms.
Deprecated since version 0.24: This attribute is deprecated in 0.24 and will be removed in 1.1 (renaming of 0.26). Use dictionary instead.
See also
fit(X[, y])
Do nothing and return the estimator unchanged.
fit_transform(X[, y])
Fit to data, then transform it.
get_params([deep])
Get parameters for this estimator.
set_params(**params)
Set the parameters of this estimator.
transform(X[, y])
Encode the data as a sparse combination of the dictionary atoms.
Do nothing and return the estimator unchanged.
This method is just there to implement the usual API and hence work in pipelines.
Fit to data, then transform it.
Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.
Input samples.
Target values (None for unsupervised transformations).
Additional fit parameters.
Transformed array.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Encode the data as a sparse combination of the dictionary atoms.
Coding method is determined by the object parameter transform_algorithm.
Test data to be transformed, must have the same number of features as the data used to train the model.
Transformed data.
Sparse coding with a precomputed dictionary


Do nothing and return the estimator unchanged.
This method is just there to implement the usual API and hence work in pipelines.


Fit to data, then transform it.
Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.
Input samples.
Target values (None for unsupervised transformations).
Additional fit parameters.
Transformed array.


Encode the data as a sparse combination of the dictionary atoms.
Coding method is determined by the object parameter transform_algorithm.
Test data to be transformed, must have the same number of features as the data used to train the model.
Transformed data.


Sparse Principal Components Analysis (SparsePCA).
Finds the set of sparse components that can optimally reconstruct the data. The amount of sparseness is controllable by the coefficient of the L1 penalty, given by the parameter alpha.
Read more in the User Guide.
Number of sparse atoms to extract.
Sparsity controlling parameter. Higher values lead to sparser components.
Amount of ridge shrinkage to apply in order to improve conditioning when calling the transform method.
Maximum number of iterations to perform.
Tolerance for the stopping condition.
lars: uses the least angle regression method to solve the lasso problem (linear_model.lars_path) cd: uses the coordinate descent method to compute the Lasso solution (linear_model.Lasso). Lars will be faster if the estimated components are sparse.
Number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
Initial values for the loadings for warm restart scenarios.
Initial values for the components for warm restart scenarios.
Controls the verbosity; the higher, the more messages. Defaults to 0.
Used during dictionary learning. Pass an int for reproducible results across multiple function calls. See Glossary.
Sparse components extracted from the data.
Vector of errors at each iteration.
Estimated number of components.
New in version 0.23.
Number of iterations run.
Per-feature empirical mean, estimated from the training set. Equal to X.mean(axis=0).
See also
fit(X[, y])
Fit the model from data in X.
fit_transform(X[, y])
Fit to data, then transform it.
get_params([deep])
Get parameters for this estimator.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Least Squares projection of the data onto the sparse components.
Fit the model from data in X.
Training vector, where n_samples in the number of samples and n_features is the number of features.
Returns the instance itself.
Fit to data, then transform it.
Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.
Input samples.
Target values (None for unsupervised transformations).
Additional fit parameters.
Transformed array.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Least Squares projection of the data onto the sparse components.
To avoid instability issues in case the system is under-determined, regularization can be applied (Ridge regression) via the ridge_alpha parameter.
Note that Sparse PCA components orthogonality is not enforced as in PCA hence one cannot use a simple linear projection.
Test data to be transformed, must have the same number of features as the data used to train the model.
Transformed data.


Fit the model from data in X.
Training vector, where n_samples in the number of samples and n_features is the number of features.
Returns the instance itself.


Least Squares projection of the data onto the sparse components.
To avoid instability issues in case the system is under-determined, regularization can be applied (Ridge regression) via the ridge_alpha parameter.
Note that Sparse PCA components orthogonality is not enforced as in PCA hence one cannot use a simple linear projection.
Test data to be transformed, must have the same number of features as the data used to train the model.
Transformed data.


Sparse coding
Each row of the result is the solution to a sparse coding problem. The goal is to find a sparse array code such that:
Read more in the User Guide.
Data matrix.
The dictionary matrix against which to solve the sparse coding of the data. Some of the algorithms assume normalized rows for meaningful output.
Precomputed Gram matrix, dictionary * dictionary'.
Precomputed covariance, dictionary' * X.
The algorithm used:
Number of nonzero coefficients to target in each column of the solution. This is only used by algorithm='lars' and algorithm='omp' and is overridden by alpha in the omp case. If None, then n_nonzero_coefs=int(n_features / 10).
If algorithm='lasso_lars' or algorithm='lasso_cd', alpha is the penalty applied to the L1 norm. If algorithm='threshold', alpha is the absolute value of the threshold below which coefficients will be squashed to zero. If algorithm='omp', alpha is the tolerance parameter: the value of the reconstruction error targeted. In this case, it overrides n_nonzero_coefs. If None, default to 1.
Whether to copy the precomputed covariance matrix; if False, it may be overwritten.
Initialization value of the sparse codes. Only used if algorithm='lasso_cd'.
Maximum number of iterations to perform if algorithm='lasso_cd' or 'lasso_lars'.
Number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
If False, the input arrays X and dictionary will not be checked.
Controls the verbosity; the higher, the more messages.
Whether to enforce positivity when finding the encoding.
New in version 0.20.
The sparse codes
See also


Dimensionality reduction using truncated SVD (aka LSA).
This transformer performs linear dimensionality reduction by means of truncated singular value decomposition (SVD). Contrary to PCA, this estimator does not center the data before computing the singular value decomposition. This means it can work with sparse matrices efficiently.
In particular, truncated SVD works on term count/tf-idf matrices as returned by the vectorizers in sklearn.feature_extraction.text. In that context, it is known as latent semantic analysis (LSA).
This estimator supports two algorithms: a fast randomized SVD solver, and a “naive” algorithm that uses ARPACK as an eigensolver on X * X.T or X.T * X, whichever is more efficient.
Read more in the User Guide.
Desired dimensionality of output data. Must be strictly less than the number of features. The default value is useful for visualisation. For LSA, a value of 100 is recommended.
SVD solver to use. Either “arpack” for the ARPACK wrapper in SciPy (scipy.sparse.linalg.svds), or “randomized” for the randomized algorithm due to Halko (2009).
Number of iterations for randomized SVD solver. Not used by ARPACK. The default is larger than the default in randomized_svd to handle sparse matrices that may have large slowly decaying spectrum.
Used during randomized svd. Pass an int for reproducible results across multiple function calls. See Glossary.
Tolerance for ARPACK. 0 means machine precision. Ignored by randomized SVD solver.
The variance of the training samples transformed by a projection to each component.
Percentage of variance explained by each of the selected components.
The singular values corresponding to each of the selected components. The singular values are equal to the 2-norms of the n_components variables in the lower-dimensional space.
See also
SVD suffers from a problem called “sign indeterminacy”, which means the sign of the components_ and the output from transform depend on the algorithm and random state. To work around this, fit instances of this class to data once, then keep the instance around to do transformations.
Finding structure with randomness: Stochastic algorithms for constructing approximate matrix decompositions Halko, et al., 2009 (arXiv:909) https://arxiv.org/pdf/0909.4061.pdf
fit(X[, y])
Fit model on training data X.
fit_transform(X[, y])
Fit model to X and perform dimensionality reduction on X.
get_params([deep])
Get parameters for this estimator.
inverse_transform(X)
Transform X back to its original space.
set_params(**params)
Set the parameters of this estimator.
transform(X)
Perform dimensionality reduction on X.
Fit model on training data X.
Training data.
Returns the transformer object.
Fit model to X and perform dimensionality reduction on X.
Training data.
Reduced version of X. This will always be a dense array.
Get parameters for this estimator.
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Parameter names mapped to their values.
Transform X back to its original space.
Returns an array X_original whose transform would be X.
New data.
Note that this is always a dense array.
Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.
Estimator parameters.
Estimator instance.
Perform dimensionality reduction on X.
New data.
Reduced version of X. This will always be a dense array.