Approximation of signals and functions in high dimensions with low dimensional structure: finite-valued sparse signals and generalized ridge functions

In this thesis, we consider the following class of high dimensional functions f : X → R, f (x) = g(dist(x, L)), where L is a linear subspace of a usually high dimensional space X = RN and g : Ω ⊂ R → R is a function with certain properties. We study different reconstruction problems under additional assumptions on g and L. In the papers [54, 41, 56] (see Appendix A – C), we assume that g is a multiple of the identity map and L is some (N − 1)-dimensional subspace. We further consider the signed distance, so that f becomes f (x) = 〈x, a〉, where a ∈ RN is orthogonal to L. If a is a sparse vector, this problem is well-known from compressed sensing. In this context we study different reconstruction problems. In Appendix A, [54], we assume that a ∈ RN is sparse and finite-valued. We present an approach that incorporates a finite values prior into basis pursuit, which is one classical reconstruction strategy in compressed sensing. In particular, we address unipolar binary and bipolar ternary sparse signals, i.e., sparse signals with entries in {0, 1}, or in {−1, 0, 1} respectively. We show that phase transition takes place earlier than using the classical basis pursuit approach and that, independently of the sparsity of the signal, at most N/2, respectively 3N/4, measurements are necessary to recover a unipolar binary, and a bipolar ternary signal uniquely. We further discuss robustness with respect to noisy measurements and generalizations to signals with entries in larger alphabets. In this work we consider Gaussian measurements yi = 〈xi, a〉, i = 1, . , m, where [x1, . , xm] ∈ Rm,N is a Gaussian matrix. In Appendix B, [41], we concentrate on the recovery of sparse, (unipolar) binary signals through box-constrained basis pursuit. In contrast to the work in Appendix A we use biased measurement matrices, whose entries have a nonzero expected value. This enables us, using a probabilistic model, to provide conditions under which the recovery of both s-sparse and saturated, i.e., (N − s)-sparse binary signals, is very likely. In fact, we also show that under the same conditions, the solution of the boxed-constrained basis pursuit program can be found using boxed-constrained least squares, which has some practical impact. This allows for example to establish stability without a-priori knowledge on the noise level. In Appendix C, [56], we also study the reconstruction of binary sparse signals from biased measurements. In contrast to the work in Appendix B, however, we consider biased partial random circulant instead of fully random measurements. We again show that the reconstruction via the least-squares strategy is as good as the reconstruction via the usually used program basis pursuit. We further show that we need as many measurements to recover an s-sparse signal x0 ∈ RN as we need to recover a saturated signal, moreprecisely an (N − s)-sparse signal x0 ∈ RN . We further establish stability with respect to noisy measurements. Then, in [55], we consider the case that L ⊆ RN is an unknown linear subspace of X and g : R → R is also an unknown map. Note that in the case that L is an (N − 1)dimensional subspace, this problem is known in the context of so-called ridge functions. Thus, in Appendix D, [55], "we study the approximation of generalized ridge func-