sklearn.linear_model.TheilSenRegressor
class sklearn.linear_model.TheilSenRegressor(*, fit_intercept=True, copy_X=True, max_subpopulation=10000.0, n_subsamples=None, max_iter=300, tol=0.001, random_state=None, n_jobs=None, verbose=False) [source]

Theil-Sen Estimator: robust multivariate regression model. The algorithm calculates least square solutions on subsets with size n_subsamples of the samples in X. Any value of n_subsamples between the number of features and samples leads to an estimator with a compromise between robustness and efficiency. Since the number of least square solutions is “n_samples choose n_subsamples”, it can be extremely large and can therefore be limited with max_subpopulation. If this limit is reached, the subsets are chosen randomly. In a final step, the spatial median (or L1 median) is calculated of all least square solutions. Read more in the User Guide.  Parameters

fit_interceptbool, default=True

Whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations.
copy_Xbool, default=True

If True, X will be copied; else, it may be overwritten.
max_subpopulationint, default=1e4

Instead of computing with a set of cardinality ‘n choose k’, where n is the number of samples and k is the number of subsamples (at least number of features), consider only a stochastic subpopulation of a given maximal size if ‘n choose k’ is larger than max_subpopulation. For other than small problem sizes this parameter will determine memory usage and runtime if n_subsamples is not changed.
n_subsamplesint, default=None

Number of samples to calculate the parameters. This is at least the number of features (plus 1 if fit_intercept=True) and the number of samples as a maximum. A lower number leads to a higher breakdown point and a low efficiency while a high number leads to a low breakdown point and a high efficiency. If None, take the minimum number of subsamples leading to maximal robustness. If n_subsamples is set to n_samples, Theil-Sen is identical to least squares.
max_iterint, default=300

Maximum number of iterations for the calculation of spatial median.
tolfloat, default=1.e-3

Tolerance when calculating spatial median.
random_stateint, RandomState instance or None, default=None

A random number generator instance to define the state of the random permutations generator. Pass an int for reproducible output across multiple function calls. See Glossary
n_jobsint, default=None

Number of CPUs to use during the cross validation. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
verbosebool, default=False

Verbose mode when fitting the model.    Attributes

coef_ndarray of shape (n_features,)

Coefficients of the regression model (median of distribution).
intercept_float

Estimated intercept of regression model.
breakdown_float

Approximated breakdown point.
n_iter_int

Number of iterations needed for the spatial median.
n_subpopulation_int

Number of combinations taken into account from ‘n choose k’, where n is the number of samples and k is the number of subsamples.     References  Theil-Sen Estimators in a Multiple Linear Regression Model, 2009 Xin Dang, Hanxiang Peng, Xueqin Wang and Heping Zhang http://home.olemiss.edu/~xdang/papers/MTSE.pdf
  Examples >>> from sklearn.linear_model import TheilSenRegressor
>>> from sklearn.datasets import make_regression
>>> X, y = make_regression(
...     n_samples=200, n_features=2, noise=4.0, random_state=0)
>>> reg = TheilSenRegressor(random_state=0).fit(X, y)
>>> reg.score(X, y)
0.9884...
>>> reg.predict(X[:1,])
array([-31.5871...])
 Methods
fit(X, y) Fit linear model.
get_params([deep]) Get parameters for this estimator.
predict(X) Predict using the linear model.
score(X, y[, sample_weight]) Return the coefficient of determination \(R^2\) of the prediction.
set_params(**params) Set the parameters of this estimator.
fit(X, y) [source]

Fit linear model.  Parameters

Xndarray of shape (n_samples, n_features)

Training data.
yndarray of shape (n_samples,)

Target values.    Returns

selfreturns an instance of self.

get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.

predict(X) [source]

Predict using the linear model.  Parameters

Xarray-like or sparse matrix, shape (n_samples, n_features)

Samples.    Returns

Carray, shape (n_samples,)

Returns predicted values.

score(X, y, sample_weight=None) [source]

Return the coefficient of determination \(R^2\) of the prediction. The coefficient \(R^2\) is defined as \((1 - \frac{u}{v})\), where \(u\) is the residual sum of squares ((y_true - y_pred)
** 2).sum() and \(v\) is the total sum of squares ((y_true -
y_true.mean()) ** 2).sum(). The best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a \(R^2\) score of 0.0.  Parameters

Xarray-like of shape (n_samples, n_features)

Test samples. For some estimators this may be a precomputed kernel matrix or a list of generic objects instead with shape (n_samples, n_samples_fitted), where n_samples_fitted is the number of samples used in the fitting for the estimator.
yarray-like of shape (n_samples,) or (n_samples, n_outputs)

True values for X.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

scorefloat

\(R^2\) of self.predict(X) wrt. y.     Notes The \(R^2\) score used when calling score on a regressor uses multioutput='uniform_average' from version 0.23 to keep consistent with default value of r2_score. This influences the score method of all the multioutput regressors (except for MultiOutputRegressor).

set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.

 Examples using sklearn.linear_model.TheilSenRegressor

  Theil-Sen Regression

  Robust linear estimator fitting


fit(X, y) [source]

Fit linear model.  Parameters

Xndarray of shape (n_samples, n_features)

Training data.
yndarray of shape (n_samples,)

Target values.    Returns

selfreturns an instance of self.


get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.


predict(X) [source]

Predict using the linear model.  Parameters

Xarray-like or sparse matrix, shape (n_samples, n_features)

Samples.    Returns

Carray, shape (n_samples,)

Returns predicted values.


score(X, y, sample_weight=None) [source]

Return the coefficient of determination \(R^2\) of the prediction. The coefficient \(R^2\) is defined as \((1 - \frac{u}{v})\), where \(u\) is the residual sum of squares ((y_true - y_pred)
** 2).sum() and \(v\) is the total sum of squares ((y_true -
y_true.mean()) ** 2).sum(). The best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a \(R^2\) score of 0.0.  Parameters

Xarray-like of shape (n_samples, n_features)

Test samples. For some estimators this may be a precomputed kernel matrix or a list of generic objects instead with shape (n_samples, n_samples_fitted), where n_samples_fitted is the number of samples used in the fitting for the estimator.
yarray-like of shape (n_samples,) or (n_samples, n_outputs)

True values for X.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

scorefloat

\(R^2\) of self.predict(X) wrt. y.     Notes The \(R^2\) score used when calling score on a regressor uses multioutput='uniform_average' from version 0.23 to keep consistent with default value of r2_score. This influences the score method of all the multioutput regressors (except for MultiOutputRegressor).


set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.


class sklearn.linear_model.TweedieRegressor(*, power=0.0, alpha=1.0, fit_intercept=True, link='auto', max_iter=100, tol=0.0001, warm_start=False, verbose=0) [source]

Generalized Linear Model with a Tweedie distribution. This estimator can be used to model different GLMs depending on the power parameter, which determines the underlying distribution. Read more in the User Guide.  New in version 0.23.   Parameters

powerfloat, default=0

The power determines the underlying target distribution according to the following table:
Power Distribution
0 Normal
1 Poisson
(1,2) Compound Poisson Gamma
2 Gamma
3 Inverse Gaussian   For 0 < power < 1, no distribution exists.
alphafloat, default=1

Constant that multiplies the penalty term and thus determines the regularization strength. alpha = 0 is equivalent to unpenalized GLMs. In this case, the design matrix X must have full column rank (no collinearities).
link{‘auto’, ‘identity’, ‘log’}, default=’auto’

The link function of the GLM, i.e. mapping from linear predictor X @ coeff + intercept to prediction y_pred. Option ‘auto’ sets the link depending on the chosen family as follows:  ‘identity’ for Normal distribution ‘log’ for Poisson, Gamma and Inverse Gaussian distributions
fit_interceptbool, default=True

Specifies if a constant (a.k.a. bias or intercept) should be added to the linear predictor (X @ coef + intercept).
max_iterint, default=100

The maximal number of iterations for the solver.
tolfloat, default=1e-4

Stopping criterion. For the lbfgs solver, the iteration will stop when max{|g_j|, j = 1, ..., d} <= tol where g_j is the j-th component of the gradient (derivative) of the objective function.
warm_startbool, default=False

If set to True, reuse the solution of the previous call to fit as initialization for coef_ and intercept_ .
verboseint, default=0

For the lbfgs solver set verbose to any positive number for verbosity.    Attributes

coef_array of shape (n_features,)

Estimated coefficients for the linear predictor (X @ coef_ +
intercept_) in the GLM.
intercept_float

Intercept (a.k.a. bias) added to linear predictor.
n_iter_int

Actual number of iterations used in the solver.     Examples >>> from sklearn import linear_model
>>> clf = linear_model.TweedieRegressor()
>>> X = [[1, 2], [2, 3], [3, 4], [4, 3]]
>>> y = [2, 3.5, 5, 5.5]
>>> clf.fit(X, y)
TweedieRegressor()
>>> clf.score(X, y)
0.839...
>>> clf.coef_
array([0.599..., 0.299...])
>>> clf.intercept_
1.600...
>>> clf.predict([[1, 1], [3, 4]])
array([2.500..., 4.599...])
 Methods
fit(X, y[, sample_weight]) Fit a Generalized Linear Model.
get_params([deep]) Get parameters for this estimator.
predict(X) Predict using GLM with feature matrix X.
score(X, y[, sample_weight]) Compute D^2, the percentage of deviance explained.
set_params(**params) Set the parameters of this estimator.
fit(X, y, sample_weight=None) [source]

Fit a Generalized Linear Model.  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Training data.
yarray-like of shape (n_samples,)

Target values.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

selfreturns an instance of self.

get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.

predict(X) [source]

Predict using GLM with feature matrix X.  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Samples.    Returns

y_predarray of shape (n_samples,)

Returns predicted values.

score(X, y, sample_weight=None) [source]

Compute D^2, the percentage of deviance explained. D^2 is a generalization of the coefficient of determination R^2. R^2 uses squared error and D^2 deviance. Note that those two are equal for family='normal'. D^2 is defined as \(D^2 = 1-\frac{D(y_{true},y_{pred})}{D_{null}}\), \(D_{null}\) is the null deviance, i.e. the deviance of a model with intercept alone, which corresponds to \(y_{pred} = \bar{y}\). The mean \(\bar{y}\) is averaged by sample_weight. Best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse).  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Test samples.
yarray-like of shape (n_samples,)

True values of target.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

scorefloat

D^2 of self.predict(X) w.r.t. y.

set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.


sklearn.linear_model.TweedieRegressor
class sklearn.linear_model.TweedieRegressor(*, power=0.0, alpha=1.0, fit_intercept=True, link='auto', max_iter=100, tol=0.0001, warm_start=False, verbose=0) [source]

Generalized Linear Model with a Tweedie distribution. This estimator can be used to model different GLMs depending on the power parameter, which determines the underlying distribution. Read more in the User Guide.  New in version 0.23.   Parameters

powerfloat, default=0

The power determines the underlying target distribution according to the following table:
Power Distribution
0 Normal
1 Poisson
(1,2) Compound Poisson Gamma
2 Gamma
3 Inverse Gaussian   For 0 < power < 1, no distribution exists.
alphafloat, default=1

Constant that multiplies the penalty term and thus determines the regularization strength. alpha = 0 is equivalent to unpenalized GLMs. In this case, the design matrix X must have full column rank (no collinearities).
link{‘auto’, ‘identity’, ‘log’}, default=’auto’

The link function of the GLM, i.e. mapping from linear predictor X @ coeff + intercept to prediction y_pred. Option ‘auto’ sets the link depending on the chosen family as follows:  ‘identity’ for Normal distribution ‘log’ for Poisson, Gamma and Inverse Gaussian distributions
fit_interceptbool, default=True

Specifies if a constant (a.k.a. bias or intercept) should be added to the linear predictor (X @ coef + intercept).
max_iterint, default=100

The maximal number of iterations for the solver.
tolfloat, default=1e-4

Stopping criterion. For the lbfgs solver, the iteration will stop when max{|g_j|, j = 1, ..., d} <= tol where g_j is the j-th component of the gradient (derivative) of the objective function.
warm_startbool, default=False

If set to True, reuse the solution of the previous call to fit as initialization for coef_ and intercept_ .
verboseint, default=0

For the lbfgs solver set verbose to any positive number for verbosity.    Attributes

coef_array of shape (n_features,)

Estimated coefficients for the linear predictor (X @ coef_ +
intercept_) in the GLM.
intercept_float

Intercept (a.k.a. bias) added to linear predictor.
n_iter_int

Actual number of iterations used in the solver.     Examples >>> from sklearn import linear_model
>>> clf = linear_model.TweedieRegressor()
>>> X = [[1, 2], [2, 3], [3, 4], [4, 3]]
>>> y = [2, 3.5, 5, 5.5]
>>> clf.fit(X, y)
TweedieRegressor()
>>> clf.score(X, y)
0.839...
>>> clf.coef_
array([0.599..., 0.299...])
>>> clf.intercept_
1.600...
>>> clf.predict([[1, 1], [3, 4]])
array([2.500..., 4.599...])
 Methods
fit(X, y[, sample_weight]) Fit a Generalized Linear Model.
get_params([deep]) Get parameters for this estimator.
predict(X) Predict using GLM with feature matrix X.
score(X, y[, sample_weight]) Compute D^2, the percentage of deviance explained.
set_params(**params) Set the parameters of this estimator.
fit(X, y, sample_weight=None) [source]

Fit a Generalized Linear Model.  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Training data.
yarray-like of shape (n_samples,)

Target values.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

selfreturns an instance of self.

get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.

predict(X) [source]

Predict using GLM with feature matrix X.  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Samples.    Returns

y_predarray of shape (n_samples,)

Returns predicted values.

score(X, y, sample_weight=None) [source]

Compute D^2, the percentage of deviance explained. D^2 is a generalization of the coefficient of determination R^2. R^2 uses squared error and D^2 deviance. Note that those two are equal for family='normal'. D^2 is defined as \(D^2 = 1-\frac{D(y_{true},y_{pred})}{D_{null}}\), \(D_{null}\) is the null deviance, i.e. the deviance of a model with intercept alone, which corresponds to \(y_{pred} = \bar{y}\). The mean \(\bar{y}\) is averaged by sample_weight. Best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse).  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Test samples.
yarray-like of shape (n_samples,)

True values of target.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

scorefloat

D^2 of self.predict(X) w.r.t. y.

set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.

 Examples using sklearn.linear_model.TweedieRegressor

  Release Highlights for scikit-learn 0.23

  Tweedie regression on insurance claims


fit(X, y, sample_weight=None) [source]

Fit a Generalized Linear Model.  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Training data.
yarray-like of shape (n_samples,)

Target values.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

selfreturns an instance of self.


predict(X) [source]

Predict using GLM with feature matrix X.  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Samples.    Returns

y_predarray of shape (n_samples,)

Returns predicted values.


score(X, y, sample_weight=None) [source]

Compute D^2, the percentage of deviance explained. D^2 is a generalization of the coefficient of determination R^2. R^2 uses squared error and D^2 deviance. Note that those two are equal for family='normal'. D^2 is defined as \(D^2 = 1-\frac{D(y_{true},y_{pred})}{D_{null}}\), \(D_{null}\) is the null deviance, i.e. the deviance of a model with intercept alone, which corresponds to \(y_{pred} = \bar{y}\). The mean \(\bar{y}\) is averaged by sample_weight. Best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse).  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Test samples.
yarray-like of shape (n_samples,)

True values of target.
sample_weightarray-like of shape (n_samples,), default=None

Sample weights.    Returns

scorefloat

D^2 of self.predict(X) w.r.t. y.


class sklearn.manifold.Isomap(*, n_neighbors=5, n_components=2, eigen_solver='auto', tol=0, max_iter=None, path_method='auto', neighbors_algorithm='auto', n_jobs=None, metric='minkowski', p=2, metric_params=None) [source]

Isomap Embedding Non-linear dimensionality reduction through Isometric Mapping Read more in the User Guide.  Parameters

n_neighborsint, default=5

number of neighbors to consider for each point.
n_componentsint, default=2

number of coordinates for the manifold
eigen_solver{‘auto’, ‘arpack’, ‘dense’}, default=’auto’

‘auto’ : Attempt to choose the most efficient solver for the given problem. ‘arpack’ : Use Arnoldi decomposition to find the eigenvalues and eigenvectors. ‘dense’ : Use a direct solver (i.e. LAPACK) for the eigenvalue decomposition.
tolfloat, default=0

Convergence tolerance passed to arpack or lobpcg. not used if eigen_solver == ‘dense’.
max_iterint, default=None

Maximum number of iterations for the arpack solver. not used if eigen_solver == ‘dense’.
path_method{‘auto’, ‘FW’, ‘D’}, default=’auto’

Method to use in finding shortest path. ‘auto’ : attempt to choose the best algorithm automatically. ‘FW’ : Floyd-Warshall algorithm. ‘D’ : Dijkstra’s algorithm.
neighbors_algorithm{‘auto’, ‘brute’, ‘kd_tree’, ‘ball_tree’}, default=’auto’

Algorithm to use for nearest neighbors search, passed to neighbors.NearestNeighbors instance.
n_jobsint or None, default=None

The number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
metricstring, or callable, default=”minkowski”

The metric to use when calculating distance between instances in a feature array. If metric is a string or callable, it must be one of the options allowed by sklearn.metrics.pairwise_distances for its metric parameter. If metric is “precomputed”, X is assumed to be a distance matrix and must be square. X may be a Glossary.  New in version 0.22.
pint, default=2

Parameter for the Minkowski metric from sklearn.metrics.pairwise.pairwise_distances. When p = 1, this is equivalent to using manhattan_distance (l1), and euclidean_distance (l2) for p = 2. For arbitrary p, minkowski_distance (l_p) is used.  New in version 0.22.
metric_paramsdict, default=None

Additional keyword arguments for the metric function.  New in version 0.22.     Attributes

embedding_array-like, shape (n_samples, n_components)

Stores the embedding vectors.
kernel_pca_object

KernelPCA object used to implement the embedding.
nbrs_sklearn.neighbors.NearestNeighbors instance

Stores nearest neighbors instance, including BallTree or KDtree if applicable.
dist_matrix_array-like, shape (n_samples, n_samples)

Stores the geodesic distance matrix of training data.     References
1
Tenenbaum, J.B.; De Silva, V.; & Langford, J.C. A global geometric framework for nonlinear dimensionality reduction. Science 290 (5500)   Examples >>> from sklearn.datasets import load_digits
>>> from sklearn.manifold import Isomap
>>> X, _ = load_digits(return_X_y=True)
>>> X.shape
(1797, 64)
>>> embedding = Isomap(n_components=2)
>>> X_transformed = embedding.fit_transform(X[:100])
>>> X_transformed.shape
(100, 2)
 Methods
fit(X[, y]) Compute the embedding vectors for data X
fit_transform(X[, y]) Fit the model from data in X and transform X.
get_params([deep]) Get parameters for this estimator.
reconstruction_error() Compute the reconstruction error for the embedding.
set_params(**params) Set the parameters of this estimator.
transform(X) Transform X.
fit(X, y=None) [source]

Compute the embedding vectors for data X  Parameters

X{array-like, sparse graph, BallTree, KDTree, NearestNeighbors}

Sample data, shape = (n_samples, n_features), in the form of a numpy array, sparse graph, precomputed tree, or NearestNeighbors object.
yIgnored
  Returns

selfreturns an instance of self.

fit_transform(X, y=None) [source]

Fit the model from data in X and transform X.  Parameters

X{array-like, sparse graph, BallTree, KDTree}

Training vector, where n_samples in the number of samples and n_features is the number of features.
yIgnored
  Returns

X_newarray-like, shape (n_samples, n_components)

get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.

reconstruction_error() [source]

Compute the reconstruction error for the embedding.  Returns

reconstruction_errorfloat
   Notes The cost function of an isomap embedding is E = frobenius_norm[K(D) - K(D_fit)] / n_samples Where D is the matrix of distances for the input data X, D_fit is the matrix of distances for the output embedding X_fit, and K is the isomap kernel: K(D) = -0.5 * (I - 1/n_samples) * D^2 * (I - 1/n_samples)

set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.

transform(X) [source]

Transform X. This is implemented by linking the points X into the graph of geodesic distances of the training data. First the n_neighbors nearest neighbors of X are found in the training data, and from these the shortest geodesic distances from each point in X to each point in the training data are computed in order to construct the kernel. The embedding of X is the projection of this kernel onto the embedding vectors of the training set.  Parameters

Xarray-like, shape (n_queries, n_features)

If neighbors_algorithm=’precomputed’, X is assumed to be a distance matrix or a sparse graph of shape (n_queries, n_samples_fit).    Returns

X_newarray-like, shape (n_queries, n_components)


sklearn.manifold.Isomap
class sklearn.manifold.Isomap(*, n_neighbors=5, n_components=2, eigen_solver='auto', tol=0, max_iter=None, path_method='auto', neighbors_algorithm='auto', n_jobs=None, metric='minkowski', p=2, metric_params=None) [source]

Isomap Embedding Non-linear dimensionality reduction through Isometric Mapping Read more in the User Guide.  Parameters

n_neighborsint, default=5

number of neighbors to consider for each point.
n_componentsint, default=2

number of coordinates for the manifold
eigen_solver{‘auto’, ‘arpack’, ‘dense’}, default=’auto’

‘auto’ : Attempt to choose the most efficient solver for the given problem. ‘arpack’ : Use Arnoldi decomposition to find the eigenvalues and eigenvectors. ‘dense’ : Use a direct solver (i.e. LAPACK) for the eigenvalue decomposition.
tolfloat, default=0

Convergence tolerance passed to arpack or lobpcg. not used if eigen_solver == ‘dense’.
max_iterint, default=None

Maximum number of iterations for the arpack solver. not used if eigen_solver == ‘dense’.
path_method{‘auto’, ‘FW’, ‘D’}, default=’auto’

Method to use in finding shortest path. ‘auto’ : attempt to choose the best algorithm automatically. ‘FW’ : Floyd-Warshall algorithm. ‘D’ : Dijkstra’s algorithm.
neighbors_algorithm{‘auto’, ‘brute’, ‘kd_tree’, ‘ball_tree’}, default=’auto’

Algorithm to use for nearest neighbors search, passed to neighbors.NearestNeighbors instance.
n_jobsint or None, default=None

The number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
metricstring, or callable, default=”minkowski”

The metric to use when calculating distance between instances in a feature array. If metric is a string or callable, it must be one of the options allowed by sklearn.metrics.pairwise_distances for its metric parameter. If metric is “precomputed”, X is assumed to be a distance matrix and must be square. X may be a Glossary.  New in version 0.22.
pint, default=2

Parameter for the Minkowski metric from sklearn.metrics.pairwise.pairwise_distances. When p = 1, this is equivalent to using manhattan_distance (l1), and euclidean_distance (l2) for p = 2. For arbitrary p, minkowski_distance (l_p) is used.  New in version 0.22.
metric_paramsdict, default=None

Additional keyword arguments for the metric function.  New in version 0.22.     Attributes

embedding_array-like, shape (n_samples, n_components)

Stores the embedding vectors.
kernel_pca_object

KernelPCA object used to implement the embedding.
nbrs_sklearn.neighbors.NearestNeighbors instance

Stores nearest neighbors instance, including BallTree or KDtree if applicable.
dist_matrix_array-like, shape (n_samples, n_samples)

Stores the geodesic distance matrix of training data.     References
1
Tenenbaum, J.B.; De Silva, V.; & Langford, J.C. A global geometric framework for nonlinear dimensionality reduction. Science 290 (5500)   Examples >>> from sklearn.datasets import load_digits
>>> from sklearn.manifold import Isomap
>>> X, _ = load_digits(return_X_y=True)
>>> X.shape
(1797, 64)
>>> embedding = Isomap(n_components=2)
>>> X_transformed = embedding.fit_transform(X[:100])
>>> X_transformed.shape
(100, 2)
 Methods
fit(X[, y]) Compute the embedding vectors for data X
fit_transform(X[, y]) Fit the model from data in X and transform X.
get_params([deep]) Get parameters for this estimator.
reconstruction_error() Compute the reconstruction error for the embedding.
set_params(**params) Set the parameters of this estimator.
transform(X) Transform X.
fit(X, y=None) [source]

Compute the embedding vectors for data X  Parameters

X{array-like, sparse graph, BallTree, KDTree, NearestNeighbors}

Sample data, shape = (n_samples, n_features), in the form of a numpy array, sparse graph, precomputed tree, or NearestNeighbors object.
yIgnored
  Returns

selfreturns an instance of self.

fit_transform(X, y=None) [source]

Fit the model from data in X and transform X.  Parameters

X{array-like, sparse graph, BallTree, KDTree}

Training vector, where n_samples in the number of samples and n_features is the number of features.
yIgnored
  Returns

X_newarray-like, shape (n_samples, n_components)

get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.

reconstruction_error() [source]

Compute the reconstruction error for the embedding.  Returns

reconstruction_errorfloat
   Notes The cost function of an isomap embedding is E = frobenius_norm[K(D) - K(D_fit)] / n_samples Where D is the matrix of distances for the input data X, D_fit is the matrix of distances for the output embedding X_fit, and K is the isomap kernel: K(D) = -0.5 * (I - 1/n_samples) * D^2 * (I - 1/n_samples)

set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.

transform(X) [source]

Transform X. This is implemented by linking the points X into the graph of geodesic distances of the training data. First the n_neighbors nearest neighbors of X are found in the training data, and from these the shortest geodesic distances from each point in X to each point in the training data are computed in order to construct the kernel. The embedding of X is the projection of this kernel onto the embedding vectors of the training set.  Parameters

Xarray-like, shape (n_queries, n_features)

If neighbors_algorithm=’precomputed’, X is assumed to be a distance matrix or a sparse graph of shape (n_queries, n_samples_fit).    Returns

X_newarray-like, shape (n_queries, n_components)

 Examples using sklearn.manifold.Isomap

  Release Highlights for scikit-learn 0.22

  Comparison of Manifold Learning methods

  Manifold Learning methods on a severed sphere

  Manifold learning on handwritten digits: Locally Linear Embedding, Isomap…


fit(X, y=None) [source]

Compute the embedding vectors for data X  Parameters

X{array-like, sparse graph, BallTree, KDTree, NearestNeighbors}

Sample data, shape = (n_samples, n_features), in the form of a numpy array, sparse graph, precomputed tree, or NearestNeighbors object.
yIgnored
  Returns

selfreturns an instance of self.


fit_transform(X, y=None) [source]

Fit the model from data in X and transform X.  Parameters

X{array-like, sparse graph, BallTree, KDTree}

Training vector, where n_samples in the number of samples and n_features is the number of features.
yIgnored
  Returns

X_newarray-like, shape (n_samples, n_components)


reconstruction_error() [source]

Compute the reconstruction error for the embedding.  Returns

reconstruction_errorfloat
   Notes The cost function of an isomap embedding is E = frobenius_norm[K(D) - K(D_fit)] / n_samples Where D is the matrix of distances for the input data X, D_fit is the matrix of distances for the output embedding X_fit, and K is the isomap kernel: K(D) = -0.5 * (I - 1/n_samples) * D^2 * (I - 1/n_samples)


transform(X) [source]

Transform X. This is implemented by linking the points X into the graph of geodesic distances of the training data. First the n_neighbors nearest neighbors of X are found in the training data, and from these the shortest geodesic distances from each point in X to each point in the training data are computed in order to construct the kernel. The embedding of X is the projection of this kernel onto the embedding vectors of the training set.  Parameters

Xarray-like, shape (n_queries, n_features)

If neighbors_algorithm=’precomputed’, X is assumed to be a distance matrix or a sparse graph of shape (n_queries, n_samples_fit).    Returns

X_newarray-like, shape (n_queries, n_components)


class sklearn.manifold.LocallyLinearEmbedding(*, n_neighbors=5, n_components=2, reg=0.001, eigen_solver='auto', tol=1e-06, max_iter=100, method='standard', hessian_tol=0.0001, modified_tol=1e-12, neighbors_algorithm='auto', random_state=None, n_jobs=None) [source]

Locally Linear Embedding Read more in the User Guide.  Parameters

n_neighborsint, default=5

number of neighbors to consider for each point.
n_componentsint, default=2

number of coordinates for the manifold
regfloat, default=1e-3

regularization constant, multiplies the trace of the local covariance matrix of the distances.
eigen_solver{‘auto’, ‘arpack’, ‘dense’}, default=’auto’

auto : algorithm will attempt to choose the best method for input data
arpackuse arnoldi iteration in shift-invert mode.

For this method, M may be a dense matrix, sparse matrix, or general linear operator. Warning: ARPACK can be unstable for some problems. It is best to try several random seeds in order to check results.
denseuse standard dense matrix operations for the eigenvalue

decomposition. For this method, M must be an array or matrix type. This method should be avoided for large problems.
tolfloat, default=1e-6

Tolerance for ‘arpack’ method Not used if eigen_solver==’dense’.
max_iterint, default=100

maximum number of iterations for the arpack solver. Not used if eigen_solver==’dense’.
method{‘standard’, ‘hessian’, ‘modified’, ‘ltsa’}, default=’standard’

standarduse the standard locally linear embedding algorithm. see

reference [1]
hessianuse the Hessian eigenmap method. This method requires

n_neighbors > n_components * (1 + (n_components + 1) / 2 see reference [2]
modifieduse the modified locally linear embedding algorithm.

see reference [3]
ltsause local tangent space alignment algorithm

see reference [4]
hessian_tolfloat, default=1e-4

Tolerance for Hessian eigenmapping method. Only used if method == 'hessian'
modified_tolfloat, default=1e-12

Tolerance for modified LLE method. Only used if method == 'modified'
neighbors_algorithm{‘auto’, ‘brute’, ‘kd_tree’, ‘ball_tree’}, default=’auto’

algorithm to use for nearest neighbors search, passed to neighbors.NearestNeighbors instance
random_stateint, RandomState instance, default=None

Determines the random number generator when eigen_solver == ‘arpack’. Pass an int for reproducible results across multiple function calls. See :term: Glossary <random_state>.
n_jobsint or None, default=None

The number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.    Attributes

embedding_array-like, shape [n_samples, n_components]

Stores the embedding vectors
reconstruction_error_float

Reconstruction error associated with embedding_
nbrs_NearestNeighbors object

Stores nearest neighbors instance, including BallTree or KDtree if applicable.     References
1
Roweis, S. & Saul, L. Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323 (2000).
2
Donoho, D. & Grimes, C. Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. Proc Natl Acad Sci U S A. 100:5591 (2003).
3
Zhang, Z. & Wang, J. MLLE: Modified Locally Linear Embedding Using Multiple Weights. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.70.382
4
Zhang, Z. & Zha, H. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. Journal of Shanghai Univ. 8:406 (2004)   Examples >>> from sklearn.datasets import load_digits
>>> from sklearn.manifold import LocallyLinearEmbedding
>>> X, _ = load_digits(return_X_y=True)
>>> X.shape
(1797, 64)
>>> embedding = LocallyLinearEmbedding(n_components=2)
>>> X_transformed = embedding.fit_transform(X[:100])
>>> X_transformed.shape
(100, 2)
 Methods
fit(X[, y]) Compute the embedding vectors for data X
fit_transform(X[, y]) Compute the embedding vectors for data X and transform X.
get_params([deep]) Get parameters for this estimator.
set_params(**params) Set the parameters of this estimator.
transform(X) Transform new points into embedding space.
fit(X, y=None) [source]

Compute the embedding vectors for data X  Parameters

Xarray-like of shape [n_samples, n_features]

training set.
yIgnored
  Returns

selfreturns an instance of self.

fit_transform(X, y=None) [source]

Compute the embedding vectors for data X and transform X.  Parameters

Xarray-like of shape [n_samples, n_features]

training set.
yIgnored
  Returns

X_newarray-like, shape (n_samples, n_components)

get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.

set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.

transform(X) [source]

Transform new points into embedding space.  Parameters

Xarray-like of shape (n_samples, n_features)
  Returns

X_newarray, shape = [n_samples, n_components]
   Notes Because of scaling performed by this method, it is discouraged to use it together with methods that are not scale-invariant (like SVMs)


sklearn.manifold.LocallyLinearEmbedding
class sklearn.manifold.LocallyLinearEmbedding(*, n_neighbors=5, n_components=2, reg=0.001, eigen_solver='auto', tol=1e-06, max_iter=100, method='standard', hessian_tol=0.0001, modified_tol=1e-12, neighbors_algorithm='auto', random_state=None, n_jobs=None) [source]

Locally Linear Embedding Read more in the User Guide.  Parameters

n_neighborsint, default=5

number of neighbors to consider for each point.
n_componentsint, default=2

number of coordinates for the manifold
regfloat, default=1e-3

regularization constant, multiplies the trace of the local covariance matrix of the distances.
eigen_solver{‘auto’, ‘arpack’, ‘dense’}, default=’auto’

auto : algorithm will attempt to choose the best method for input data
arpackuse arnoldi iteration in shift-invert mode.

For this method, M may be a dense matrix, sparse matrix, or general linear operator. Warning: ARPACK can be unstable for some problems. It is best to try several random seeds in order to check results.
denseuse standard dense matrix operations for the eigenvalue

decomposition. For this method, M must be an array or matrix type. This method should be avoided for large problems.
tolfloat, default=1e-6

Tolerance for ‘arpack’ method Not used if eigen_solver==’dense’.
max_iterint, default=100

maximum number of iterations for the arpack solver. Not used if eigen_solver==’dense’.
method{‘standard’, ‘hessian’, ‘modified’, ‘ltsa’}, default=’standard’

standarduse the standard locally linear embedding algorithm. see

reference [1]
hessianuse the Hessian eigenmap method. This method requires

n_neighbors > n_components * (1 + (n_components + 1) / 2 see reference [2]
modifieduse the modified locally linear embedding algorithm.

see reference [3]
ltsause local tangent space alignment algorithm

see reference [4]
hessian_tolfloat, default=1e-4

Tolerance for Hessian eigenmapping method. Only used if method == 'hessian'
modified_tolfloat, default=1e-12

Tolerance for modified LLE method. Only used if method == 'modified'
neighbors_algorithm{‘auto’, ‘brute’, ‘kd_tree’, ‘ball_tree’}, default=’auto’

algorithm to use for nearest neighbors search, passed to neighbors.NearestNeighbors instance
random_stateint, RandomState instance, default=None

Determines the random number generator when eigen_solver == ‘arpack’. Pass an int for reproducible results across multiple function calls. See :term: Glossary <random_state>.
n_jobsint or None, default=None

The number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.    Attributes

embedding_array-like, shape [n_samples, n_components]

Stores the embedding vectors
reconstruction_error_float

Reconstruction error associated with embedding_
nbrs_NearestNeighbors object

Stores nearest neighbors instance, including BallTree or KDtree if applicable.     References
1
Roweis, S. & Saul, L. Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323 (2000).
2
Donoho, D. & Grimes, C. Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. Proc Natl Acad Sci U S A. 100:5591 (2003).
3
Zhang, Z. & Wang, J. MLLE: Modified Locally Linear Embedding Using Multiple Weights. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.70.382
4
Zhang, Z. & Zha, H. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. Journal of Shanghai Univ. 8:406 (2004)   Examples >>> from sklearn.datasets import load_digits
>>> from sklearn.manifold import LocallyLinearEmbedding
>>> X, _ = load_digits(return_X_y=True)
>>> X.shape
(1797, 64)
>>> embedding = LocallyLinearEmbedding(n_components=2)
>>> X_transformed = embedding.fit_transform(X[:100])
>>> X_transformed.shape
(100, 2)
 Methods
fit(X[, y]) Compute the embedding vectors for data X
fit_transform(X[, y]) Compute the embedding vectors for data X and transform X.
get_params([deep]) Get parameters for this estimator.
set_params(**params) Set the parameters of this estimator.
transform(X) Transform new points into embedding space.
fit(X, y=None) [source]

Compute the embedding vectors for data X  Parameters

Xarray-like of shape [n_samples, n_features]

training set.
yIgnored
  Returns

selfreturns an instance of self.

fit_transform(X, y=None) [source]

Compute the embedding vectors for data X and transform X.  Parameters

Xarray-like of shape [n_samples, n_features]

training set.
yIgnored
  Returns

X_newarray-like, shape (n_samples, n_components)

get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.

set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.

transform(X) [source]

Transform new points into embedding space.  Parameters

Xarray-like of shape (n_samples, n_features)
  Returns

X_newarray, shape = [n_samples, n_components]
   Notes Because of scaling performed by this method, it is discouraged to use it together with methods that are not scale-invariant (like SVMs)

 Examples using sklearn.manifold.LocallyLinearEmbedding

  Visualizing the stock market structure

  Comparison of Manifold Learning methods

  Manifold Learning methods on a severed sphere

  Manifold learning on handwritten digits: Locally Linear Embedding, Isomap…


fit(X, y=None) [source]

Compute the embedding vectors for data X  Parameters

Xarray-like of shape [n_samples, n_features]

training set.
yIgnored
  Returns

selfreturns an instance of self.


fit_transform(X, y=None) [source]

Compute the embedding vectors for data X and transform X.  Parameters

Xarray-like of shape [n_samples, n_features]

training set.
yIgnored
  Returns

X_newarray-like, shape (n_samples, n_components)


get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.


set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.


transform(X) [source]

Transform new points into embedding space.  Parameters

Xarray-like of shape (n_samples, n_features)
  Returns

X_newarray, shape = [n_samples, n_components]
   Notes Because of scaling performed by this method, it is discouraged to use it together with methods that are not scale-invariant (like SVMs)


sklearn.manifold.locally_linear_embedding(X, *, n_neighbors, n_components, reg=0.001, eigen_solver='auto', tol=1e-06, max_iter=100, method='standard', hessian_tol=0.0001, modified_tol=1e-12, random_state=None, n_jobs=None) [source]

Perform a Locally Linear Embedding analysis on the data. Read more in the User Guide.  Parameters

X{array-like, NearestNeighbors}

Sample data, shape = (n_samples, n_features), in the form of a numpy array or a NearestNeighbors object.
n_neighborsint

number of neighbors to consider for each point.
n_componentsint

number of coordinates for the manifold.
regfloat, default=1e-3

regularization constant, multiplies the trace of the local covariance matrix of the distances.
eigen_solver{‘auto’, ‘arpack’, ‘dense’}, default=’auto’

auto : algorithm will attempt to choose the best method for input data
arpackuse arnoldi iteration in shift-invert mode.

For this method, M may be a dense matrix, sparse matrix, or general linear operator. Warning: ARPACK can be unstable for some problems. It is best to try several random seeds in order to check results.
denseuse standard dense matrix operations for the eigenvalue

decomposition. For this method, M must be an array or matrix type. This method should be avoided for large problems.
tolfloat, default=1e-6

Tolerance for ‘arpack’ method Not used if eigen_solver==’dense’.
max_iterint, default=100

maximum number of iterations for the arpack solver.
method{‘standard’, ‘hessian’, ‘modified’, ‘ltsa’}, default=’standard’

standarduse the standard locally linear embedding algorithm.

see reference [1]
hessianuse the Hessian eigenmap method. This method requires

n_neighbors > n_components * (1 + (n_components + 1) / 2. see reference [2]
modifieduse the modified locally linear embedding algorithm.

see reference [3]
ltsause local tangent space alignment algorithm

see reference [4]
hessian_tolfloat, default=1e-4

Tolerance for Hessian eigenmapping method. Only used if method == ‘hessian’
modified_tolfloat, default=1e-12

Tolerance for modified LLE method. Only used if method == ‘modified’
random_stateint, RandomState instance, default=None

Determines the random number generator when solver == ‘arpack’. Pass an int for reproducible results across multiple function calls. See :term: Glossary <random_state>.
n_jobsint or None, default=None

The number of parallel jobs to run for neighbors search. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.    Returns

Yarray-like, shape [n_samples, n_components]

Embedding vectors.
squared_errorfloat

Reconstruction error for the embedding vectors. Equivalent to norm(Y - W Y, 'fro')**2, where W are the reconstruction weights.     References
1
Roweis, S. & Saul, L. Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323 (2000).
2
Donoho, D. & Grimes, C. Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. Proc Natl Acad Sci U S A. 100:5591 (2003).
3
Zhang, Z. & Wang, J. MLLE: Modified Locally Linear Embedding Using Multiple Weights. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.70.382
4
Zhang, Z. & Zha, H. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. Journal of Shanghai Univ. 8:406 (2004)


class sklearn.manifold.MDS(n_components=2, *, metric=True, n_init=4, max_iter=300, verbose=0, eps=0.001, n_jobs=None, random_state=None, dissimilarity='euclidean') [source]

Multidimensional scaling. Read more in the User Guide.  Parameters

n_componentsint, default=2

Number of dimensions in which to immerse the dissimilarities.
metricbool, default=True

If True, perform metric MDS; otherwise, perform nonmetric MDS.
n_initint, default=4

Number of times the SMACOF algorithm will be run with different initializations. The final results will be the best output of the runs, determined by the run with the smallest final stress.
max_iterint, default=300

Maximum number of iterations of the SMACOF algorithm for a single run.
verboseint, default=0

Level of verbosity.
epsfloat, default=1e-3

Relative tolerance with respect to stress at which to declare convergence.
n_jobsint, default=None

The number of jobs to use for the computation. If multiple initializations are used (n_init), each run of the algorithm is computed in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
random_stateint, RandomState instance or None, default=None

Determines the random number generator used to initialize the centers. Pass an int for reproducible results across multiple function calls. See :term: Glossary <random_state>.
dissimilarity{‘euclidean’, ‘precomputed’}, default=’euclidean’

Dissimilarity measure to use:
 ‘euclidean’:

Pairwise Euclidean distances between points in the dataset.
 ‘precomputed’:

Pre-computed dissimilarities are passed directly to fit and fit_transform.        Attributes

embedding_ndarray of shape (n_samples, n_components)

Stores the position of the dataset in the embedding space.
stress_float

The final value of the stress (sum of squared distance of the disparities and the distances for all constrained points).
dissimilarity_matrix_ndarray of shape (n_samples, n_samples)

Pairwise dissimilarities between the points. Symmetric matrix that:  either uses a custom dissimilarity matrix by setting dissimilarity to ‘precomputed’; or constructs a dissimilarity matrix from data using Euclidean distances.
n_iter_int

The number of iterations corresponding to the best stress.     References “Modern Multidimensional Scaling - Theory and Applications” Borg, I.; Groenen P. Springer Series in Statistics (1997) “Nonmetric multidimensional scaling: a numerical method” Kruskal, J. Psychometrika, 29 (1964) “Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis” Kruskal, J. Psychometrika, 29, (1964) Examples >>> from sklearn.datasets import load_digits
>>> from sklearn.manifold import MDS
>>> X, _ = load_digits(return_X_y=True)
>>> X.shape
(1797, 64)
>>> embedding = MDS(n_components=2)
>>> X_transformed = embedding.fit_transform(X[:100])
>>> X_transformed.shape
(100, 2)
 Methods
fit(X[, y, init]) Computes the position of the points in the embedding space.
fit_transform(X[, y, init]) Fit the data from X, and returns the embedded coordinates.
get_params([deep]) Get parameters for this estimator.
set_params(**params) Set the parameters of this estimator.
fit(X, y=None, init=None) [source]

Computes the position of the points in the embedding space.  Parameters

Xarray-like of shape (n_samples, n_features) or (n_samples, n_samples)

Input data. If dissimilarity=='precomputed', the input should be the dissimilarity matrix.
yIgnored

initndarray of shape (n_samples,), default=None

Starting configuration of the embedding to initialize the SMACOF algorithm. By default, the algorithm is initialized with a randomly chosen array.

fit_transform(X, y=None, init=None) [source]

Fit the data from X, and returns the embedded coordinates.  Parameters

Xarray-like of shape (n_samples, n_features) or (n_samples, n_samples)

Input data. If dissimilarity=='precomputed', the input should be the dissimilarity matrix.
yIgnored

initndarray of shape (n_samples,), default=None

Starting configuration of the embedding to initialize the SMACOF algorithm. By default, the algorithm is initialized with a randomly chosen array.

get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.

set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.


sklearn.manifold.MDS
class sklearn.manifold.MDS(n_components=2, *, metric=True, n_init=4, max_iter=300, verbose=0, eps=0.001, n_jobs=None, random_state=None, dissimilarity='euclidean') [source]

Multidimensional scaling. Read more in the User Guide.  Parameters

n_componentsint, default=2

Number of dimensions in which to immerse the dissimilarities.
metricbool, default=True

If True, perform metric MDS; otherwise, perform nonmetric MDS.
n_initint, default=4

Number of times the SMACOF algorithm will be run with different initializations. The final results will be the best output of the runs, determined by the run with the smallest final stress.
max_iterint, default=300

Maximum number of iterations of the SMACOF algorithm for a single run.
verboseint, default=0

Level of verbosity.
epsfloat, default=1e-3

Relative tolerance with respect to stress at which to declare convergence.
n_jobsint, default=None

The number of jobs to use for the computation. If multiple initializations are used (n_init), each run of the algorithm is computed in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
random_stateint, RandomState instance or None, default=None

Determines the random number generator used to initialize the centers. Pass an int for reproducible results across multiple function calls. See :term: Glossary <random_state>.
dissimilarity{‘euclidean’, ‘precomputed’}, default=’euclidean’

Dissimilarity measure to use:
 ‘euclidean’:

Pairwise Euclidean distances between points in the dataset.
 ‘precomputed’:

Pre-computed dissimilarities are passed directly to fit and fit_transform.        Attributes

embedding_ndarray of shape (n_samples, n_components)

Stores the position of the dataset in the embedding space.
stress_float

The final value of the stress (sum of squared distance of the disparities and the distances for all constrained points).
dissimilarity_matrix_ndarray of shape (n_samples, n_samples)

Pairwise dissimilarities between the points. Symmetric matrix that:  either uses a custom dissimilarity matrix by setting dissimilarity to ‘precomputed’; or constructs a dissimilarity matrix from data using Euclidean distances.
n_iter_int

The number of iterations corresponding to the best stress.     References “Modern Multidimensional Scaling - Theory and Applications” Borg, I.; Groenen P. Springer Series in Statistics (1997) “Nonmetric multidimensional scaling: a numerical method” Kruskal, J. Psychometrika, 29 (1964) “Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis” Kruskal, J. Psychometrika, 29, (1964) Examples >>> from sklearn.datasets import load_digits
>>> from sklearn.manifold import MDS
>>> X, _ = load_digits(return_X_y=True)
>>> X.shape
(1797, 64)
>>> embedding = MDS(n_components=2)
>>> X_transformed = embedding.fit_transform(X[:100])
>>> X_transformed.shape
(100, 2)
 Methods
fit(X[, y, init]) Computes the position of the points in the embedding space.
fit_transform(X[, y, init]) Fit the data from X, and returns the embedded coordinates.
get_params([deep]) Get parameters for this estimator.
set_params(**params) Set the parameters of this estimator.
fit(X, y=None, init=None) [source]

Computes the position of the points in the embedding space.  Parameters

Xarray-like of shape (n_samples, n_features) or (n_samples, n_samples)

Input data. If dissimilarity=='precomputed', the input should be the dissimilarity matrix.
yIgnored

initndarray of shape (n_samples,), default=None

Starting configuration of the embedding to initialize the SMACOF algorithm. By default, the algorithm is initialized with a randomly chosen array.

fit_transform(X, y=None, init=None) [source]

Fit the data from X, and returns the embedded coordinates.  Parameters

Xarray-like of shape (n_samples, n_features) or (n_samples, n_samples)

Input data. If dissimilarity=='precomputed', the input should be the dissimilarity matrix.
yIgnored

initndarray of shape (n_samples,), default=None

Starting configuration of the embedding to initialize the SMACOF algorithm. By default, the algorithm is initialized with a randomly chosen array.

get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.

set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.

 Examples using sklearn.manifold.MDS

  Comparison of Manifold Learning methods

  Multi-dimensional scaling

  Manifold Learning methods on a severed sphere

  Manifold learning on handwritten digits: Locally Linear Embedding, Isomap…


fit(X, y=None, init=None) [source]

Computes the position of the points in the embedding space.  Parameters

Xarray-like of shape (n_samples, n_features) or (n_samples, n_samples)

Input data. If dissimilarity=='precomputed', the input should be the dissimilarity matrix.
yIgnored

initndarray of shape (n_samples,), default=None

Starting configuration of the embedding to initialize the SMACOF algorithm. By default, the algorithm is initialized with a randomly chosen array.


fit_transform(X, y=None, init=None) [source]

Fit the data from X, and returns the embedded coordinates.  Parameters

Xarray-like of shape (n_samples, n_features) or (n_samples, n_samples)

Input data. If dissimilarity=='precomputed', the input should be the dissimilarity matrix.
yIgnored

initndarray of shape (n_samples,), default=None

Starting configuration of the embedding to initialize the SMACOF algorithm. By default, the algorithm is initialized with a randomly chosen array.


sklearn.manifold.smacof(dissimilarities, *, metric=True, n_components=2, init=None, n_init=8, n_jobs=None, max_iter=300, verbose=0, eps=0.001, random_state=None, return_n_iter=False) [source]

Computes multidimensional scaling using the SMACOF algorithm. The SMACOF (Scaling by MAjorizing a COmplicated Function) algorithm is a multidimensional scaling algorithm which minimizes an objective function (the stress) using a majorization technique. Stress majorization, also known as the Guttman Transform, guarantees a monotone convergence of stress, and is more powerful than traditional techniques such as gradient descent. The SMACOF algorithm for metric MDS can summarized by the following steps:  Set an initial start configuration, randomly or not. Compute the stress Compute the Guttman Transform Iterate 2 and 3 until convergence.  The nonmetric algorithm adds a monotonic regression step before computing the stress.  Parameters

dissimilaritiesndarray of shape (n_samples, n_samples)

Pairwise dissimilarities between the points. Must be symmetric.
metricbool, default=True

Compute metric or nonmetric SMACOF algorithm.
n_componentsint, default=2

Number of dimensions in which to immerse the dissimilarities. If an init array is provided, this option is overridden and the shape of init is used to determine the dimensionality of the embedding space.
initndarray of shape (n_samples, n_components), default=None

Starting configuration of the embedding to initialize the algorithm. By default, the algorithm is initialized with a randomly chosen array.
n_initint, default=8

Number of times the SMACOF algorithm will be run with different initializations. The final results will be the best output of the runs, determined by the run with the smallest final stress. If init is provided, this option is overridden and a single run is performed.
n_jobsint, default=None

The number of jobs to use for the computation. If multiple initializations are used (n_init), each run of the algorithm is computed in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
max_iterint, default=300

Maximum number of iterations of the SMACOF algorithm for a single run.
verboseint, default=0

Level of verbosity.
epsfloat, default=1e-3

Relative tolerance with respect to stress at which to declare convergence.
random_stateint, RandomState instance or None, default=None

Determines the random number generator used to initialize the centers. Pass an int for reproducible results across multiple function calls. See :term: Glossary <random_state>.
return_n_iterbool, default=False

Whether or not to return the number of iterations.    Returns

Xndarray of shape (n_samples, n_components)

Coordinates of the points in a n_components-space.
stressfloat

The final value of the stress (sum of squared distance of the disparities and the distances for all constrained points).
n_iterint

The number of iterations corresponding to the best stress. Returned only if return_n_iter is set to True.     Notes “Modern Multidimensional Scaling - Theory and Applications” Borg, I.; Groenen P. Springer Series in Statistics (1997) “Nonmetric multidimensional scaling: a numerical method” Kruskal, J. Psychometrika, 29 (1964) “Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis” Kruskal, J. Psychometrika, 29, (1964)


class sklearn.manifold.SpectralEmbedding(n_components=2, *, affinity='nearest_neighbors', gamma=None, random_state=None, eigen_solver=None, n_neighbors=None, n_jobs=None) [source]

Spectral embedding for non-linear dimensionality reduction. Forms an affinity matrix given by the specified function and applies spectral decomposition to the corresponding graph laplacian. The resulting transformation is given by the value of the eigenvectors for each data point. Note : Laplacian Eigenmaps is the actual algorithm implemented here. Read more in the User Guide.  Parameters

n_componentsint, default=2

The dimension of the projected subspace.
affinity{‘nearest_neighbors’, ‘rbf’, ‘precomputed’, ‘precomputed_nearest_neighbors’} or callable, default=’nearest_neighbors’

 How to construct the affinity matrix.

 ‘nearest_neighbors’ : construct the affinity matrix by computing a graph of nearest neighbors. ‘rbf’ : construct the affinity matrix by computing a radial basis function (RBF) kernel. ‘precomputed’ : interpret X as a precomputed affinity matrix. ‘precomputed_nearest_neighbors’ : interpret X as a sparse graph of precomputed nearest neighbors, and constructs the affinity matrix by selecting the n_neighbors nearest neighbors. callable : use passed in function as affinity the function takes in data matrix (n_samples, n_features) and return affinity matrix (n_samples, n_samples).
gammafloat, default=None

Kernel coefficient for rbf kernel. If None, gamma will be set to 1/n_features.
random_stateint, RandomState instance or None, default=None

Determines the random number generator used for the initialization of the lobpcg eigenvectors when solver == ‘amg’. Pass an int for reproducible results across multiple function calls. See :term: Glossary <random_state>.
eigen_solver{‘arpack’, ‘lobpcg’, ‘amg’}, default=None

The eigenvalue decomposition strategy to use. AMG requires pyamg to be installed. It can be faster on very large, sparse problems. If None, then 'arpack' is used.
n_neighborsint, default=None

Number of nearest neighbors for nearest_neighbors graph building. If None, n_neighbors will be set to max(n_samples/10, 1).
n_jobsint, default=None

The number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.    Attributes

embedding_ndarray of shape (n_samples, n_components)

Spectral embedding of the training matrix.
affinity_matrix_ndarray of shape (n_samples, n_samples)

Affinity_matrix constructed from samples or precomputed.
n_neighbors_int

Number of nearest neighbors effectively used.     References  A Tutorial on Spectral Clustering, 2007 Ulrike von Luxburg http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.165.9323
 On Spectral Clustering: Analysis and an algorithm, 2001 Andrew Y. Ng, Michael I. Jordan, Yair Weiss http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.19.8100
 Normalized cuts and image segmentation, 2000 Jianbo Shi, Jitendra Malik http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.160.2324
  Examples >>> from sklearn.datasets import load_digits
>>> from sklearn.manifold import SpectralEmbedding
>>> X, _ = load_digits(return_X_y=True)
>>> X.shape
(1797, 64)
>>> embedding = SpectralEmbedding(n_components=2)
>>> X_transformed = embedding.fit_transform(X[:100])
>>> X_transformed.shape
(100, 2)
 Methods
fit(X[, y]) Fit the model from data in X.
fit_transform(X[, y]) Fit the model from data in X and transform X.
get_params([deep]) Get parameters for this estimator.
set_params(**params) Set the parameters of this estimator.
fit(X, y=None) [source]

Fit the model from data in X.  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Training vector, where n_samples is the number of samples and n_features is the number of features. If affinity is “precomputed” X : {array-like, sparse matrix}, shape (n_samples, n_samples), Interpret X as precomputed adjacency graph computed from samples.
yIgnored
  Returns

selfobject

Returns the instance itself.

fit_transform(X, y=None) [source]

Fit the model from data in X and transform X.  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Training vector, where n_samples is the number of samples and n_features is the number of features. If affinity is “precomputed” X : {array-like, sparse matrix} of shape (n_samples, n_samples), Interpret X as precomputed adjacency graph computed from samples.
yIgnored
  Returns

X_newarray-like of shape (n_samples, n_components)

get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.

set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.


sklearn.manifold.SpectralEmbedding
class sklearn.manifold.SpectralEmbedding(n_components=2, *, affinity='nearest_neighbors', gamma=None, random_state=None, eigen_solver=None, n_neighbors=None, n_jobs=None) [source]

Spectral embedding for non-linear dimensionality reduction. Forms an affinity matrix given by the specified function and applies spectral decomposition to the corresponding graph laplacian. The resulting transformation is given by the value of the eigenvectors for each data point. Note : Laplacian Eigenmaps is the actual algorithm implemented here. Read more in the User Guide.  Parameters

n_componentsint, default=2

The dimension of the projected subspace.
affinity{‘nearest_neighbors’, ‘rbf’, ‘precomputed’, ‘precomputed_nearest_neighbors’} or callable, default=’nearest_neighbors’

 How to construct the affinity matrix.

 ‘nearest_neighbors’ : construct the affinity matrix by computing a graph of nearest neighbors. ‘rbf’ : construct the affinity matrix by computing a radial basis function (RBF) kernel. ‘precomputed’ : interpret X as a precomputed affinity matrix. ‘precomputed_nearest_neighbors’ : interpret X as a sparse graph of precomputed nearest neighbors, and constructs the affinity matrix by selecting the n_neighbors nearest neighbors. callable : use passed in function as affinity the function takes in data matrix (n_samples, n_features) and return affinity matrix (n_samples, n_samples).
gammafloat, default=None

Kernel coefficient for rbf kernel. If None, gamma will be set to 1/n_features.
random_stateint, RandomState instance or None, default=None

Determines the random number generator used for the initialization of the lobpcg eigenvectors when solver == ‘amg’. Pass an int for reproducible results across multiple function calls. See :term: Glossary <random_state>.
eigen_solver{‘arpack’, ‘lobpcg’, ‘amg’}, default=None

The eigenvalue decomposition strategy to use. AMG requires pyamg to be installed. It can be faster on very large, sparse problems. If None, then 'arpack' is used.
n_neighborsint, default=None

Number of nearest neighbors for nearest_neighbors graph building. If None, n_neighbors will be set to max(n_samples/10, 1).
n_jobsint, default=None

The number of parallel jobs to run. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.    Attributes

embedding_ndarray of shape (n_samples, n_components)

Spectral embedding of the training matrix.
affinity_matrix_ndarray of shape (n_samples, n_samples)

Affinity_matrix constructed from samples or precomputed.
n_neighbors_int

Number of nearest neighbors effectively used.     References  A Tutorial on Spectral Clustering, 2007 Ulrike von Luxburg http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.165.9323
 On Spectral Clustering: Analysis and an algorithm, 2001 Andrew Y. Ng, Michael I. Jordan, Yair Weiss http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.19.8100
 Normalized cuts and image segmentation, 2000 Jianbo Shi, Jitendra Malik http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.160.2324
  Examples >>> from sklearn.datasets import load_digits
>>> from sklearn.manifold import SpectralEmbedding
>>> X, _ = load_digits(return_X_y=True)
>>> X.shape
(1797, 64)
>>> embedding = SpectralEmbedding(n_components=2)
>>> X_transformed = embedding.fit_transform(X[:100])
>>> X_transformed.shape
(100, 2)
 Methods
fit(X[, y]) Fit the model from data in X.
fit_transform(X[, y]) Fit the model from data in X and transform X.
get_params([deep]) Get parameters for this estimator.
set_params(**params) Set the parameters of this estimator.
fit(X, y=None) [source]

Fit the model from data in X.  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Training vector, where n_samples is the number of samples and n_features is the number of features. If affinity is “precomputed” X : {array-like, sparse matrix}, shape (n_samples, n_samples), Interpret X as precomputed adjacency graph computed from samples.
yIgnored
  Returns

selfobject

Returns the instance itself.

fit_transform(X, y=None) [source]

Fit the model from data in X and transform X.  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Training vector, where n_samples is the number of samples and n_features is the number of features. If affinity is “precomputed” X : {array-like, sparse matrix} of shape (n_samples, n_samples), Interpret X as precomputed adjacency graph computed from samples.
yIgnored
  Returns

X_newarray-like of shape (n_samples, n_components)

get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.

set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.

 Examples using sklearn.manifold.SpectralEmbedding

  Various Agglomerative Clustering on a 2D embedding of digits

  Comparison of Manifold Learning methods

  Manifold Learning methods on a severed sphere

  Manifold learning on handwritten digits: Locally Linear Embedding, Isomap…


fit(X, y=None) [source]

Fit the model from data in X.  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Training vector, where n_samples is the number of samples and n_features is the number of features. If affinity is “precomputed” X : {array-like, sparse matrix}, shape (n_samples, n_samples), Interpret X as precomputed adjacency graph computed from samples.
yIgnored
  Returns

selfobject

Returns the instance itself.


fit_transform(X, y=None) [source]

Fit the model from data in X and transform X.  Parameters

X{array-like, sparse matrix} of shape (n_samples, n_features)

Training vector, where n_samples is the number of samples and n_features is the number of features. If affinity is “precomputed” X : {array-like, sparse matrix} of shape (n_samples, n_samples), Interpret X as precomputed adjacency graph computed from samples.
yIgnored
  Returns

X_newarray-like of shape (n_samples, n_components)


sklearn.manifold.spectral_embedding(adjacency, *, n_components=8, eigen_solver=None, random_state=None, eigen_tol=0.0, norm_laplacian=True, drop_first=True) [source]

Project the sample on the first eigenvectors of the graph Laplacian. The adjacency matrix is used to compute a normalized graph Laplacian whose spectrum (especially the eigenvectors associated to the smallest eigenvalues) has an interpretation in terms of minimal number of cuts necessary to split the graph into comparably sized components. This embedding can also ‘work’ even if the adjacency variable is not strictly the adjacency matrix of a graph but more generally an affinity or similarity matrix between samples (for instance the heat kernel of a euclidean distance matrix or a k-NN matrix). However care must taken to always make the affinity matrix symmetric so that the eigenvector decomposition works as expected. Note : Laplacian Eigenmaps is the actual algorithm implemented here. Read more in the User Guide.  Parameters

adjacency{array-like, sparse graph} of shape (n_samples, n_samples)

The adjacency matrix of the graph to embed.
n_componentsint, default=8

The dimension of the projection subspace.
eigen_solver{‘arpack’, ‘lobpcg’, ‘amg’}, default=None

The eigenvalue decomposition strategy to use. AMG requires pyamg to be installed. It can be faster on very large, sparse problems, but may also lead to instabilities. If None, then 'arpack' is used.
random_stateint, RandomState instance or None, default=None

Determines the random number generator used for the initialization of the lobpcg eigenvectors decomposition when solver == ‘amg’. Pass an int for reproducible results across multiple function calls. See :term: Glossary <random_state>.
eigen_tolfloat, default=0.0

Stopping criterion for eigendecomposition of the Laplacian matrix when using arpack eigen_solver.
norm_laplacianbool, default=True

If True, then compute normalized Laplacian.
drop_firstbool, default=True

Whether to drop the first eigenvector. For spectral embedding, this should be True as the first eigenvector should be constant vector for connected graph, but for spectral clustering, this should be kept as False to retain the first eigenvector.    Returns

embeddingndarray of shape (n_samples, n_components)

The reduced samples.     Notes Spectral Embedding (Laplacian Eigenmaps) is most useful when the graph has one connected component. If there graph has many components, the first few eigenvectors will simply uncover the connected components of the graph. References  https://en.wikipedia.org/wiki/LOBPCG Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method Andrew V. Knyazev https://doi.org/10.1137%2FS1064827500366124


sklearn.manifold.trustworthiness(X, X_embedded, *, n_neighbors=5, metric='euclidean') [source]

Expresses to what extent the local structure is retained. The trustworthiness is within [0, 1]. It is defined as  \[T(k) = 1 - \frac{2}{nk (2n - 3k - 1)} \sum^n_{i=1} \sum_{j \in \mathcal{N}_{i}^{k}} \max(0, (r(i, j) - k))\] where for each sample i, \(\mathcal{N}_{i}^{k}\) are its k nearest neighbors in the output space, and every sample j is its \(r(i, j)\)-th nearest neighbor in the input space. In other words, any unexpected nearest neighbors in the output space are penalised in proportion to their rank in the input space.  “Neighborhood Preservation in Nonlinear Projection Methods: An Experimental Study” J. Venna, S. Kaski “Learning a Parametric Embedding by Preserving Local Structure” L.J.P. van der Maaten   Parameters

Xndarray of shape (n_samples, n_features) or (n_samples, n_samples)

If the metric is ‘precomputed’ X must be a square distance matrix. Otherwise it contains a sample per row.
X_embeddedndarray of shape (n_samples, n_components)

Embedding of the training data in low-dimensional space.
n_neighborsint, default=5

Number of neighbors k that will be considered.
metricstr or callable, default=’euclidean’

Which metric to use for computing pairwise distances between samples from the original input space. If metric is ‘precomputed’, X must be a matrix of pairwise distances or squared distances. Otherwise, see the documentation of argument metric in sklearn.pairwise.pairwise_distances for a list of available metrics.  New in version 0.20.     Returns

trustworthinessfloat

Trustworthiness of the low-dimensional embedding.


class sklearn.manifold.TSNE(n_components=2, *, perplexity=30.0, early_exaggeration=12.0, learning_rate=200.0, n_iter=1000, n_iter_without_progress=300, min_grad_norm=1e-07, metric='euclidean', init='random', verbose=0, random_state=None, method='barnes_hut', angle=0.5, n_jobs=None, square_distances='legacy') [source]

t-distributed Stochastic Neighbor Embedding. t-SNE [1] is a tool to visualize high-dimensional data. It converts similarities between data points to joint probabilities and tries to minimize the Kullback-Leibler divergence between the joint probabilities of the low-dimensional embedding and the high-dimensional data. t-SNE has a cost function that is not convex, i.e. with different initializations we can get different results. It is highly recommended to use another dimensionality reduction method (e.g. PCA for dense data or TruncatedSVD for sparse data) to reduce the number of dimensions to a reasonable amount (e.g. 50) if the number of features is very high. This will suppress some noise and speed up the computation of pairwise distances between samples. For more tips see Laurens van der Maaten’s FAQ [2]. Read more in the User Guide.  Parameters

n_componentsint, default=2

Dimension of the embedded space.
perplexityfloat, default=30.0

The perplexity is related to the number of nearest neighbors that is used in other manifold learning algorithms. Larger datasets usually require a larger perplexity. Consider selecting a value between 5 and 50. Different values can result in significantly different results.
early_exaggerationfloat, default=12.0

Controls how tight natural clusters in the original space are in the embedded space and how much space will be between them. For larger values, the space between natural clusters will be larger in the embedded space. Again, the choice of this parameter is not very critical. If the cost function increases during initial optimization, the early exaggeration factor or the learning rate might be too high.
learning_ratefloat, default=200.0

The learning rate for t-SNE is usually in the range [10.0, 1000.0]. If the learning rate is too high, the data may look like a ‘ball’ with any point approximately equidistant from its nearest neighbours. If the learning rate is too low, most points may look compressed in a dense cloud with few outliers. If the cost function gets stuck in a bad local minimum increasing the learning rate may help.
n_iterint, default=1000

Maximum number of iterations for the optimization. Should be at least 250.
n_iter_without_progressint, default=300

Maximum number of iterations without progress before we abort the optimization, used after 250 initial iterations with early exaggeration. Note that progress is only checked every 50 iterations so this value is rounded to the next multiple of 50.  New in version 0.17: parameter n_iter_without_progress to control stopping criteria.
min_grad_normfloat, default=1e-7

If the gradient norm is below this threshold, the optimization will be stopped.
metricstr or callable, default=’euclidean’

The metric to use when calculating distance between instances in a feature array. If metric is a string, it must be one of the options allowed by scipy.spatial.distance.pdist for its metric parameter, or a metric listed in pairwise.PAIRWISE_DISTANCE_FUNCTIONS. If metric is “precomputed”, X is assumed to be a distance matrix. Alternatively, if metric is a callable function, it is called on each pair of instances (rows) and the resulting value recorded. The callable should take two arrays from X as input and return a value indicating the distance between them. The default is “euclidean” which is interpreted as squared euclidean distance.
init{‘random’, ‘pca’} or ndarray of shape (n_samples, n_components), default=’random’

Initialization of embedding. Possible options are ‘random’, ‘pca’, and a numpy array of shape (n_samples, n_components). PCA initialization cannot be used with precomputed distances and is usually more globally stable than random initialization.
verboseint, default=0

Verbosity level.
random_stateint, RandomState instance or None, default=None

Determines the random number generator. Pass an int for reproducible results across multiple function calls. Note that different initializations might result in different local minima of the cost function. See :term: Glossary <random_state>.
methodstr, default=’barnes_hut’

By default the gradient calculation algorithm uses Barnes-Hut approximation running in O(NlogN) time. method=’exact’ will run on the slower, but exact, algorithm in O(N^2) time. The exact algorithm should be used when nearest-neighbor errors need to be better than 3%. However, the exact method cannot scale to millions of examples.  New in version 0.17: Approximate optimization method via the Barnes-Hut.
anglefloat, default=0.5

Only used if method=’barnes_hut’ This is the trade-off between speed and accuracy for Barnes-Hut T-SNE. ‘angle’ is the angular size (referred to as theta in [3]) of a distant node as measured from a point. If this size is below ‘angle’ then it is used as a summary node of all points contained within it. This method is not very sensitive to changes in this parameter in the range of 0.2 - 0.8. Angle less than 0.2 has quickly increasing computation time and angle greater 0.8 has quickly increasing error.
n_jobsint, default=None

The number of parallel jobs to run for neighbors search. This parameter has no impact when metric="precomputed" or (metric="euclidean" and method="exact"). None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.  New in version 0.22.
square_distancesTrue or ‘legacy’, default=’legacy’

Whether TSNE should square the distance values. 'legacy' means that distance values are squared only when metric="euclidean". True means that distance values are squared for all metrics.  New in version 0.24: Added to provide backward compatibility during deprecation of legacy squaring behavior.   Deprecated since version 0.24: Legacy squaring behavior was deprecated in 0.24. The 'legacy' value will be removed in 1.1 (renaming of 0.26), at which point the default value will change to True.     Attributes

embedding_array-like of shape (n_samples, n_components)

Stores the embedding vectors.
kl_divergence_float

Kullback-Leibler divergence after optimization.
n_iter_int

Number of iterations run.     References  [1] van der Maaten, L.J.P.; Hinton, G.E. Visualizing High-Dimensional Data

Using t-SNE. Journal of Machine Learning Research 9:2579-2605, 2008.  [2] van der Maaten, L.J.P. t-Distributed Stochastic Neighbor Embedding

https://lvdmaaten.github.io/tsne/  [3] L.J.P. van der Maaten. Accelerating t-SNE using Tree-Based Algorithms.

Journal of Machine Learning Research 15(Oct):3221-3245, 2014. https://lvdmaaten.github.io/publications/papers/JMLR_2014.pdf   Examples >>> import numpy as np
>>> from sklearn.manifold import TSNE
>>> X = np.array([[0, 0, 0], [0, 1, 1], [1, 0, 1], [1, 1, 1]])
>>> X_embedded = TSNE(n_components=2).fit_transform(X)
>>> X_embedded.shape
(4, 2)
 Methods
fit(X[, y]) Fit X into an embedded space.
fit_transform(X[, y]) Fit X into an embedded space and return that transformed output.
get_params([deep]) Get parameters for this estimator.
set_params(**params) Set the parameters of this estimator.
fit(X, y=None) [source]

Fit X into an embedded space.  Parameters

Xndarray of shape (n_samples, n_features) or (n_samples, n_samples)

If the metric is ‘precomputed’ X must be a square distance matrix. Otherwise it contains a sample per row. If the method is ‘exact’, X may be a sparse matrix of type ‘csr’, ‘csc’ or ‘coo’. If the method is ‘barnes_hut’ and the metric is ‘precomputed’, X may be a precomputed sparse graph.
yIgnored

fit_transform(X, y=None) [source]

Fit X into an embedded space and return that transformed output.  Parameters

Xndarray of shape (n_samples, n_features) or (n_samples, n_samples)

If the metric is ‘precomputed’ X must be a square distance matrix. Otherwise it contains a sample per row. If the method is ‘exact’, X may be a sparse matrix of type ‘csr’, ‘csc’ or ‘coo’. If the method is ‘barnes_hut’ and the metric is ‘precomputed’, X may be a precomputed sparse graph.
yIgnored
  Returns

X_newndarray of shape (n_samples, n_components)

Embedding of the training data in low-dimensional space.

get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.

set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.


sklearn.manifold.TSNE
class sklearn.manifold.TSNE(n_components=2, *, perplexity=30.0, early_exaggeration=12.0, learning_rate=200.0, n_iter=1000, n_iter_without_progress=300, min_grad_norm=1e-07, metric='euclidean', init='random', verbose=0, random_state=None, method='barnes_hut', angle=0.5, n_jobs=None, square_distances='legacy') [source]

t-distributed Stochastic Neighbor Embedding. t-SNE [1] is a tool to visualize high-dimensional data. It converts similarities between data points to joint probabilities and tries to minimize the Kullback-Leibler divergence between the joint probabilities of the low-dimensional embedding and the high-dimensional data. t-SNE has a cost function that is not convex, i.e. with different initializations we can get different results. It is highly recommended to use another dimensionality reduction method (e.g. PCA for dense data or TruncatedSVD for sparse data) to reduce the number of dimensions to a reasonable amount (e.g. 50) if the number of features is very high. This will suppress some noise and speed up the computation of pairwise distances between samples. For more tips see Laurens van der Maaten’s FAQ [2]. Read more in the User Guide.  Parameters

n_componentsint, default=2

Dimension of the embedded space.
perplexityfloat, default=30.0

The perplexity is related to the number of nearest neighbors that is used in other manifold learning algorithms. Larger datasets usually require a larger perplexity. Consider selecting a value between 5 and 50. Different values can result in significantly different results.
early_exaggerationfloat, default=12.0

Controls how tight natural clusters in the original space are in the embedded space and how much space will be between them. For larger values, the space between natural clusters will be larger in the embedded space. Again, the choice of this parameter is not very critical. If the cost function increases during initial optimization, the early exaggeration factor or the learning rate might be too high.
learning_ratefloat, default=200.0

The learning rate for t-SNE is usually in the range [10.0, 1000.0]. If the learning rate is too high, the data may look like a ‘ball’ with any point approximately equidistant from its nearest neighbours. If the learning rate is too low, most points may look compressed in a dense cloud with few outliers. If the cost function gets stuck in a bad local minimum increasing the learning rate may help.
n_iterint, default=1000

Maximum number of iterations for the optimization. Should be at least 250.
n_iter_without_progressint, default=300

Maximum number of iterations without progress before we abort the optimization, used after 250 initial iterations with early exaggeration. Note that progress is only checked every 50 iterations so this value is rounded to the next multiple of 50.  New in version 0.17: parameter n_iter_without_progress to control stopping criteria.
min_grad_normfloat, default=1e-7

If the gradient norm is below this threshold, the optimization will be stopped.
metricstr or callable, default=’euclidean’

The metric to use when calculating distance between instances in a feature array. If metric is a string, it must be one of the options allowed by scipy.spatial.distance.pdist for its metric parameter, or a metric listed in pairwise.PAIRWISE_DISTANCE_FUNCTIONS. If metric is “precomputed”, X is assumed to be a distance matrix. Alternatively, if metric is a callable function, it is called on each pair of instances (rows) and the resulting value recorded. The callable should take two arrays from X as input and return a value indicating the distance between them. The default is “euclidean” which is interpreted as squared euclidean distance.
init{‘random’, ‘pca’} or ndarray of shape (n_samples, n_components), default=’random’

Initialization of embedding. Possible options are ‘random’, ‘pca’, and a numpy array of shape (n_samples, n_components). PCA initialization cannot be used with precomputed distances and is usually more globally stable than random initialization.
verboseint, default=0

Verbosity level.
random_stateint, RandomState instance or None, default=None

Determines the random number generator. Pass an int for reproducible results across multiple function calls. Note that different initializations might result in different local minima of the cost function. See :term: Glossary <random_state>.
methodstr, default=’barnes_hut’

By default the gradient calculation algorithm uses Barnes-Hut approximation running in O(NlogN) time. method=’exact’ will run on the slower, but exact, algorithm in O(N^2) time. The exact algorithm should be used when nearest-neighbor errors need to be better than 3%. However, the exact method cannot scale to millions of examples.  New in version 0.17: Approximate optimization method via the Barnes-Hut.
anglefloat, default=0.5

Only used if method=’barnes_hut’ This is the trade-off between speed and accuracy for Barnes-Hut T-SNE. ‘angle’ is the angular size (referred to as theta in [3]) of a distant node as measured from a point. If this size is below ‘angle’ then it is used as a summary node of all points contained within it. This method is not very sensitive to changes in this parameter in the range of 0.2 - 0.8. Angle less than 0.2 has quickly increasing computation time and angle greater 0.8 has quickly increasing error.
n_jobsint, default=None

The number of parallel jobs to run for neighbors search. This parameter has no impact when metric="precomputed" or (metric="euclidean" and method="exact"). None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.  New in version 0.22.
square_distancesTrue or ‘legacy’, default=’legacy’

Whether TSNE should square the distance values. 'legacy' means that distance values are squared only when metric="euclidean". True means that distance values are squared for all metrics.  New in version 0.24: Added to provide backward compatibility during deprecation of legacy squaring behavior.   Deprecated since version 0.24: Legacy squaring behavior was deprecated in 0.24. The 'legacy' value will be removed in 1.1 (renaming of 0.26), at which point the default value will change to True.     Attributes

embedding_array-like of shape (n_samples, n_components)

Stores the embedding vectors.
kl_divergence_float

Kullback-Leibler divergence after optimization.
n_iter_int

Number of iterations run.     References  [1] van der Maaten, L.J.P.; Hinton, G.E. Visualizing High-Dimensional Data

Using t-SNE. Journal of Machine Learning Research 9:2579-2605, 2008.  [2] van der Maaten, L.J.P. t-Distributed Stochastic Neighbor Embedding

https://lvdmaaten.github.io/tsne/  [3] L.J.P. van der Maaten. Accelerating t-SNE using Tree-Based Algorithms.

Journal of Machine Learning Research 15(Oct):3221-3245, 2014. https://lvdmaaten.github.io/publications/papers/JMLR_2014.pdf   Examples >>> import numpy as np
>>> from sklearn.manifold import TSNE
>>> X = np.array([[0, 0, 0], [0, 1, 1], [1, 0, 1], [1, 1, 1]])
>>> X_embedded = TSNE(n_components=2).fit_transform(X)
>>> X_embedded.shape
(4, 2)
 Methods
fit(X[, y]) Fit X into an embedded space.
fit_transform(X[, y]) Fit X into an embedded space and return that transformed output.
get_params([deep]) Get parameters for this estimator.
set_params(**params) Set the parameters of this estimator.
fit(X, y=None) [source]

Fit X into an embedded space.  Parameters

Xndarray of shape (n_samples, n_features) or (n_samples, n_samples)

If the metric is ‘precomputed’ X must be a square distance matrix. Otherwise it contains a sample per row. If the method is ‘exact’, X may be a sparse matrix of type ‘csr’, ‘csc’ or ‘coo’. If the method is ‘barnes_hut’ and the metric is ‘precomputed’, X may be a precomputed sparse graph.
yIgnored

fit_transform(X, y=None) [source]

Fit X into an embedded space and return that transformed output.  Parameters

Xndarray of shape (n_samples, n_features) or (n_samples, n_samples)

If the metric is ‘precomputed’ X must be a square distance matrix. Otherwise it contains a sample per row. If the method is ‘exact’, X may be a sparse matrix of type ‘csr’, ‘csc’ or ‘coo’. If the method is ‘barnes_hut’ and the metric is ‘precomputed’, X may be a precomputed sparse graph.
yIgnored
  Returns

X_newndarray of shape (n_samples, n_components)

Embedding of the training data in low-dimensional space.

get_params(deep=True) [source]

Get parameters for this estimator.  Parameters

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.    Returns

paramsdict

Parameter names mapped to their values.

set_params(**params) [source]

Set the parameters of this estimator. The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.  Parameters

**paramsdict

Estimator parameters.    Returns

selfestimator instance

Estimator instance.

 Examples using sklearn.manifold.TSNE

  Comparison of Manifold Learning methods

  t-SNE: The effect of various perplexity values on the shape

  Manifold Learning methods on a severed sphere

  Manifold learning on handwritten digits: Locally Linear Embedding, Isomap…

  Approximate nearest neighbors in TSNE


fit(X, y=None) [source]

Fit X into an embedded space.  Parameters

Xndarray of shape (n_samples, n_features) or (n_samples, n_samples)

If the metric is ‘precomputed’ X must be a square distance matrix. Otherwise it contains a sample per row. If the method is ‘exact’, X may be a sparse matrix of type ‘csr’, ‘csc’ or ‘coo’. If the method is ‘barnes_hut’ and the metric is ‘precomputed’, X may be a precomputed sparse graph.
yIgnored