  Locally Linear Embedding (LLE) is a nonlinear spectral dimensionality
reduction and manifold learning method. It has two main steps which are linear
reconstruction and linear embedding of points in the input space and embedding
space, respectively. In this work, we propose two novel generative versions of
LLE, named Generative LLE (GLLE), whose linear reconstruction steps are
stochastic rather than deterministic. GLLE assumes that every data point is
caused by its linear reconstruction weights as latent factors. The proposed
GLLE algorithms can generate various LLE embeddings stochastically while all
the generated embeddings relate to the original LLE embedding. We propose two
versions for stochastic linear reconstruction, one using expectation
maximization and another with direct sampling from a derived distribution by
optimization. The proposed GLLE methods are closely related to and inspired by
variational inference, factor analysis, and probabilistic principal component
analysis. Our simulations show that the proposed GLLE methods work effectively
in unfolding and generating submanifolds of data.
