  Sparse manifold learning algorithms combine techniques in manifold learning
and sparse optimization to learn features that could be utilized for downstream
tasks. The standard setting of compressive sensing can not be immediately
applied to this setup. Due to the intrinsic geometric structure of data,
dictionary atoms might be redundant and do not satisfy the restricted isometry
property or coherence condition. In addition, manifold learning emphasizes
learning local geometry which is not reflected in a standard $\ell_1$
minimization problem. We propose weighted $\ell_0$ and weighted $\ell_1$
metrics that encourage representation via neighborhood atoms suited for
dictionary based manifold learning. Assuming that the data is generated from
Delaunay triangulation, we show the equivalence of weighted $\ell_1$ and
weighted $\ell_0$. We discuss an optimization program that learns the
dictionaries and sparse coefficients and demonstrate the utility of our
regularization on synthetic and real datasets.
