In dictionary learning, also known as sparse coding, the algorithm is given samples of the form y = Ax where x ∈ R m is an unknown random sparse vector and A is an unknown dictionary matrix in R n×m (usually m > n, which is the overcomplete case). The goal is to learn A and x. This problem has been studied in neuroscience, machine learning, visions, and image processing. In practice it is solved by heuristic algorithms and provable algorithms seemed hard to find. Recently, provable algorithms were found that work if the unknown feature vector x is √ n-sparse or even sparser. Spielman et al. This raised the problem of designing provable algorithms that allow sparsity ≫ √ n in the hidden vector x. The current paper designs algorithms that allow sparsity up to n/poly(log n). It works for a class of matrices where features are individually recoverable, a new notion identified in this paper that may motivate further work. The algorithm runs in quasipolynomial time because they use limited enumeration.
Similarly, Arora et al propose a framework that allows sparsity up to O(d/ log d) but runs in quasi-polynomial time REF .
More Algorithms for Provable Dictionary Learning
ArXiv
ArXiv
Mathematics Computer Science