SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Junren Chen*
University of Hong Kong chenjr58@connect.hku.hk
Jonathan Scarlett
National University of Singapore scarlett@comp.nus.edu.sg
Michael K. Ng
Hong Kong Baptist University  
michael-ng@hkbu.edu.hk
Zhaoqiang Liu*
UESTC
zqliu12@gmail.com
In generative compressed sensing (GCS), we want to recover a signal  $\pmb{x}^{*} \in \mathbb{R}^{n}$  from  $m$  measurements ( $m \ll n$ ) using a generative prior  $\pmb{x}^{*} \in G(\mathbb{B}_{2}^{k}(r))$ , where  $G$  is typically an  $L$ -Lipschitz continuous generative model and  $\mathbb{B}_2^k (r)$  represents the radius-  $r$ $\ell_2$ -ball in  $\mathbb{R}^k$ . Under nonlinear measurements, most prior results are non-uniform, i.e., they hold with high probability for a fixed  $\pmb{x}^{*}$  rather than for all  $\pmb{x}^{*}$  simultaneously. In this paper, we build a unified framework to derive uniform recovery guarantees for nonlinear GCS where the observation model is nonlinear and possibly discontinuous or unknown. Our framework accommodates GCS with 1-bit/uniformly quantized observations and single index models as canonical examples. Specifically, using a single realization of the sensing ensemble and generalized Lasso, all  $\pmb{x}^{*} \in G(\mathbb{B}_{2}^{k}(r))$  can be recovered up to an  $\ell_{2}$ -error at most  $\epsilon$  using roughly  $\tilde{O} (k / \epsilon^2)$  samples, with omitted logarithmic factors typically being dominated by  $\log L$ . Notably, this almost coincides with existing non-uniform guarantees up to logarithmic factors, hence the uniformity costs very little. As part of our technical contributions, we introduce the Lipschitz approximation to handle discontinuous observation models. We also develop a concentration inequality that produces tighter bounds for product processes whose index sets have low metric entropy. Experimental results are presented to corroborate our theory.
In compressed sensing (CS) that concerns the reconstruction of low-complexity signals (typically sparse signals) [5, 6, 15], it is standard to employ a random measurement ensemble, i.e., a random sensing matrix and other randomness that produces the observations. Thus, a recovery guarantee involving a single draw of the measurement ensemble could be non-uniform or uniform — the non-uniform one ensures the accurate recovery of any fixed signal with high probability, while the uniform one states that one realization of the measurements works simultaneously for all structured signals of interest. Uniformity is a highly desired property in CS, since in applications the measurement ensemble is typically fixed and should work for all signals [17]. Besides, the derivation of a uniform guarantee is often significantly harder than a non-uniform one, making uniformity an interesting theoretical problem in its own right.
Inspired by the tremendous success of deep generative models in different applications, it was recently proposed to use a generative prior to replace the commonly used sparse prior in CS [2], which led to numerical success such as a significant reduction of the measurement number. This new perspective for CS, which we call generative compressed sensing (GCS), has attracted a large volume of research interest, e.g., nonlinear GCS [29, 33, 45], MRI applications [24, 46], and information-theoretic bounds [27, 34], among others. This paper focuses on the uniform recovery problem for nonlinear GCS, which is formally stated below. Our main goal is to build a unified framework that can produce uniform recovery guarantees for various nonlinear measurement models.
Problem: Let  $\mathbb{B}_2^k (r)$  be the  $\ell_2$ -ball with radius  $r$  in  $\mathbb{R}^k$ . Suppose that  $G:\mathbb{B}_2^k (r)\to \mathbb{R}^n$  is an  $L$ -Lipschitz continuous generative model,  $\pmb {a}_1,\dots,\pmb {a}_m\in \mathbb{R}^n$  are the sensing vectors,  $\pmb{x}^{*}\in \mathcal{K}:= G(\mathbb{B}_2^k (r))$  is the underlying signal, and we have the observations  $y_{i} = f_{i}(a_{i}^{\top}\pmb{x}^{*})$ ,  $i = 1,\ldots ,m$ , where  $f_{1}(\cdot),\ldots ,f_{m}(\cdot)$  are possibly unknown, possibly random non-linearities. Given a single realization of  $\{\pmb {a}_i,f_i\}_{i = 1}^m$ , under what conditions we can uniformly recover all  $\pmb{x}^{*}\in \mathcal{K}$  from the corresponding  $\{\pmb {a}_i,y_i\}_{i = 1}^m$  up to an  $\ell_2$ -norm error of  $\epsilon$ ?
We divide the related works into nonlinear CS (based on traditional structures like sparsity) and nonlinear GCS.
Nonlinear CS: Beyond the standard linear CS model where one observes  $y_{i} = \pmb{a}_{i}^{\top}\pmb{x}^{*}$ , recent years have witnessed rapidly increasing literature on nonlinear CS. An important nonlinear CS model is 1-bit CS that only retains the sign  $y_{i} = \mathrm{sign}(\pmb{a}_{i}^{\top}\pmb{x}^{*})$  [3,22,41,42]. Subsequent works also considered 1-bit CS with dithering  $y_{i} = \mathrm{sign}(\pmb{a}_{i}^{\top}\pmb{x}^{*} + \tau_{i})$  to achieve norm reconstruction under sub-Gaussian sensing vectors [9, 14, 48]. Besides, the benefit of using dithering was found in uniformly quantized CS with observation  $y_{i} = \mathcal{Q}_{\delta}(\pmb{a}_{i}^{\top}\pmb{x}^{*} + \tau_{i})$ , where  $\mathcal{Q}_{\delta}(\cdot) = \delta \left( \left\lfloor \frac{\cdot}{\delta} \right\rfloor + \frac{1}{2} \right)$  is the uniform quantizer with resolution  $\delta$  [8, 48, 52]. Moreover, the authors of [16, 43, 44] studied the more general single index model (SIM) where the observation  $y_{i} = f_{i}(\pmb{a}_{i}^{\top}\pmb{x}^{*})$  involves (possibly) unknown nonlinearity  $f_{i}$ .
While the restricted isometry property (RIP) of the sensing matrix  $\mathbf{A} = [a_{1},\dots,a_{m}]^{\top}$  leads to uniform recovery in linear CS [4, 15, 49], this is not true in nonlinear CS. In fact, many existing results are nonuniform [9, 16, 21, 41, 43, 44, 48], and some uniform guarantees can be found in [7, 8, 14, 17, 41, 42, 52]. Most of these uniform guarantees suffer from a slower error rate.
The most relevant work to this paper is the recent work [17] that described a unified approach to uniform signal recovery for nonlinear CS. The authors of [17] showed that in the aforementioned models with  $k$ -sparse  $\pmb{x}^*$ , a uniform  $\ell_2$ -norm recovery error of  $\epsilon$  could be achieved via generalized Lasso using roughly  $k / \epsilon^4$  measurements [17, Section 4]. In this work, we build a unified framework for uniform signal recovery in nonlinear GCS. To achieve a uniform  $\ell_2$ -norm error of  $\epsilon$  in the above models with the generative prior  $\pmb{x}^* \in G(\mathbb{B}_2^k(r))$ , our framework only requires a number of samples proportional to  $k / \epsilon^2$ . Unlike [17] that used the technical results [36] to bound the product process, we develop a concentration inequality that produces a tighter bound in the setting of generative prior, thus allowing us to derive a sharper uniform error rate.
Nonlinear GCS: Building on the seminal work by Bora et al. [2], numerous works have investigated linear or nonlinear GCS [1, 11, 12, 19, 20, 23, 25, 30, 39, 40, 51], with a recent survey [47] providing a comprehensive overview. Particularly for nonlinear GCS, 1-bit CS with generative models has been studied in [26, 31, 45], and generative priors have been used for SIM in [29, 32, 33]. In addition, score-based generative models have been applied to nonlinear CS in [10, 38].
The majority of research for nonlinear GCS focuses on non-uniform recovery, with only a few exceptions [33, 45]. Specifically, under a generative prior, [33, Section 5] presented uniform recovery guarantees for SIM where  $y_{i} = f_{i}(a_{i}^{\top}\pmb{x}^{*})$  with deterministic Lipschitz  $f_{i}$  or  $f_{i}(x) = \mathrm{sign}(x)$ . Their proof technique is based on the local embedding property developed in [31], which is a geometric property that is often problem-dependent and currently only known for 1-bit measurements and deterministic Lipschitz link functions. In contrast, our proof technique does not rely on such
In order to establish a unified framework, our recovery method (2.1) involves a parameter  $T$  that should be chosen according to  $f_{i}$ . For the specific single index model with possibly unknown  $f_{i}$ , we can follow prior works [33,43] to assume that  $Tx^{*} \in \mathcal{K}$ , and recover  $x^{*}$  without using  $T$ . See Remark 5 for more details.
geometric properties and yields a unified framework with more generality. Furthermore, [33] did not consider dithering, which limits their ability to estimate the norm of the signal.
The authors of [45] derived a uniform guarantee from dithered 1-bit measurements under bias-free ReLU neural network generative models, while we obtain a uniform guarantee with the comparable rate for more general Lipschitz generative models. Additionally, their recovery program differs from the generalized Lasso approach (cf. Section 2.1) used in our work. Specifically, they minimize an  $\ell_2$  loss with  $\| \pmb{x}\| _2^2$  as the quadratic term, while generalized Lasso uses  $\| Ax\| _2^2$  that depends on the sensing vector. As a result, our approach can be readily generalized to sensing vectors with an unknown covariance matrix [33, Section 4.2], unlike [45] that is restricted to isotropic sensing vectors. Under random dithering, while [45] only considered 1-bit measurements, we also present new results for uniformly quantized measurements (also referred to as multi-bit quantizer in some works [13]).
1.2 Contributions
In this paper, we build a unified framework for uniform signal recovery in nonlinear GCS. We summarize the paper structure and our main contributions as follows:
- We present Theorem 1 as our main result in Section 2. Under rather general observation models that can be discontinuous or unknown, Theorem 1 states that the uniform recovery of all  $\boldsymbol{x}^{*} \in G(\mathbb{B}_{2}^{k}(r))$  up to an  $\ell_{2}$ -norm error of  $\epsilon$  can be achieved using roughly  $O\left(\frac{k \log L}{\epsilon^{2}}\right)$  samples. Specifically, we obtain uniform recovery guarantees for 1-bit GCS, 1-bit GCS with dithering, Lipschitz-continuous SIM, and uniformly quantized GCS with dithering.
- We provide a proof sketch in Section 3. Without using the embedding property as in [33], we handle the discontinuous observation model by constructing a Lipschitz approximation. Compared to [17], we develop a new concentration inequality (Theorem 2) to derive tighter bounds for the product processes arising in the proof.
We also perform proof-of-concept experiments on the MNIST [28] and CelebA [35] datasets for various nonlinear models to demonstrate that by using a single realization of  $\{a_i, f_i\}_{i=1}^m$ , we can obtain reasonably accurate reconstruction for multiple signals. Due to the page limit, the experimental results and detailed proofs are provided in the supplementary material.
1.3 Notation
We use boldface letters to denote vectors and matrices, while regular letters are used for scalars. For a vector  $\pmb{x}$ , we let  $\| \pmb{x} \|_q$  ( $1 \leq q \leq \infty$ ) denote its  $\ell_q$ -norm. We use  $\mathbb{B}_q^n(r) := \{ \pmb{z} \in \mathbb{R}^n : \| \pmb{z} \|_q \leq r \}$  to denote the  $\ell_q$  ball in  $\mathbb{R}^n$ , and  $(\mathbb{B}_q^n(r))^c$  represents its complement. The unit Euclidean sphere is denoted by  $\mathbb{S}^{n-1} := \{ \pmb{x} \in \mathbb{R}^n : \| \pmb{x} \|_2 = 1 \}$ . We use  $C, C_i, c_i, c$  to denote absolute constants whose values may differ from line to line. We write  $A = O(B)$  or  $A \lesssim B$  (resp.  $A = \Omega(B)$  or  $A \gtrsim B$ ) if  $A \leq CB$  for some  $C$  (resp.  $A \geq cB$  for some  $c$ ). We write  $A \asymp B$  if  $A = O(B)$  and  $A = \Omega(B)$  simultaneously hold. We sometimes use  $\tilde{O}(\cdot)$  to further hide logarithmic factors, where the hidden factors are typically dominated by  $\log L$  in GCS, or  $\log n$  in CS. We let  $\mathcal{N}(\pmb{\mu}, \pmb{\Sigma})$  be the Gaussian distribution with mean  $\pmb{\mu}$  and covariance matrix  $\pmb{\Sigma}$ . Given  $\mathcal{K}_1, \mathcal{K}_2 \subset \mathbb{R}^n$ ,  $\pmb{a} \in \mathbb{R}^n$  and some  $a \in \mathbb{R}$ , we define  $\mathcal{K}_1 \pm \mathcal{K}_2 := \{ x_1 \pm x_2 : x_1 \in \mathcal{K}_1, x_2 \in \mathcal{K}_2 \}$ ,  $\pmb{a} + \mathcal{K}_1 := \{ a \} + \mathcal{K}_1$ , and  $a\mathcal{K}_1 := \{ a\pmb{x} : x \in \mathcal{K}_1 \}$ . We also adopt the conventions of  $a \wedge b = \min\{a, b\}$ , and  $a \vee b = \max\{a, b\}$ .
2 Main Results
We first give some preliminaries.
Definition 1. For a random variable  $X$ , we define the sub-Gaussian norm  $\| X\|_{\psi_2} \coloneqq \inf \{t > 0 : \mathbb{E}\exp (X^2 /t^2)\leq 2\}$  and the sub-exponential norm  $\| X\|_{\psi_1} \coloneqq \inf \{t > 0 : \mathbb{E}\exp (|X| / t)\leq 2\}$ .  $X$  is sub-Gaussian (resp. sub-exponential) if  $\| X\|_{\psi_2} < \infty$  (resp.  $\| X\|_{\psi_1} < \infty$ ). For a random vector  $\pmb{x} \in \mathbb{R}^n$ , we let  $\| \pmb{x}\|_{\psi_2} \coloneqq \sup_{\pmb{v} \in \mathbb{S}^{n - 1}}\| \pmb{v}^\top \pmb{x}\|_{\psi_2}$ .
Definition 2. Let  $S$  be a subset of  $\mathbb{R}^n$ . We say that a subset  $S_0 \subset S$  is an  $\eta$ -net of  $S$  if every point in  $S$  is at most  $\eta$  distance away from some point in  $S_0$ , i.e.,  $S \subset S_0 + \mathbb{B}_2^n(\eta)$ . Given a radius  $\eta$ , we
define the covering number  $\mathcal{N}(\mathcal{S},\eta)$  as the minimal cardinality of an  $\eta$ -net of  $\mathcal{S}$ . The metric entropy of  $\mathcal{S}$  with respect to radius  $\eta$  is defined as  $\mathcal{H}(\mathcal{S},\eta) = \log \mathcal{N}(\mathcal{S},\eta)$ .
2.1 Problem Setup
We make the following assumptions on the observation model.
Assumption 1. Let  $\mathbf{a} \sim \mathcal{N}(0, I_n)$  and let  $f$  be a possibly unknown, possibly random non-linearity that is independent of  $\mathbf{a}$ . Let  $(\mathbf{a}_i, f_i)_{i=1}^m$  be i.i.d. copies of  $(\mathbf{a}, f)$ . With a single draw of  $(\mathbf{a}_i, f_i)_{i=1}^m$ , for  $\mathbf{x}^* \in \mathcal{K} = G(\mathbb{B}_2^k(r))$ , where  $G: \mathbb{B}_2^k(r) \to \mathbb{R}^n$  is an L-Lipschitz generative model, we observe  $\{y_i := f_i(\mathbf{a}_i^\top \mathbf{x}^*)\}_{i=1}^m$ . We can express the model more compactly as  $\mathbf{y} = \mathbf{f}(\mathbf{A}\mathbf{x}^*)$ , where  $\mathbf{A} = [a_1, \dots, a_m]^\top \in \mathbb{R}^{m \times n}$ ,  $\mathbf{f} = (f_1, \dots, f_m)^\top$  and  $\mathbf{y} = (y_1, \dots, y_m)^\top \in \mathbb{R}^m$ .
In this work, we consider the generalized Lasso as the recovery method [16, 33, 43], whose core idea is to ignore the non-linearity and minimize the regular  $\ell_2$  loss. In addition, we need to specify a constraint that reflects the low-complexity nature of  $x^{*}$ , and specifically, we introduce a problem-dependent scaling factor  $T\in \mathbb{R}$  and use the constraint " $x\in T\mathcal{K}$ ". Note that this is necessary even if the problem is linear; for example, with observations  $\pmb {y} = 2\pmb {A}\pmb{x}^*$ , one needs to minimize the  $\ell_{2}$  loss over " $x\in 2\mathcal{K}$ ". Also, when the generative prior is given by  $T\pmb{x}^{*}\in \mathcal{K} = G(\mathbb{B}_{2}^{k}(r))$ , we should simply use " $x\in \mathcal{K}$ " as constraint; this is technically equivalent to the treatment adopted in [33] (see more discussions in Remark 5 below). Taken collectively, we consider
$$
\hat {\boldsymbol {x}} = \arg \min  _ {\boldsymbol {x} \in T \mathcal {K}} \| \boldsymbol {y} - \boldsymbol {A} \boldsymbol {x} \| _ {2}. \tag {2.1}
$$
Importantly, we want to achieve uniform recovery of all  $\pmb{x}^{*} \in \mathcal{K}$  with a single realization of  $(\pmb{A}, \pmb{f})$ .
2.2 Assumptions
Let  $f$  be the function that characterizes our nonlinear measurements. We introduce several assumptions on  $f$  here, and then verify them for specific models in Section 2.3. We define the set of discontinuities as
$$
\mathscr {D} _ {f} = \{a \in \mathbb {R}: f \text {i s d i s c o n t i n u o u s a t} a \}.
$$
We define the notion of jump discontinuity as follows.
Definition 3. (Jump discontinuity). A function  $f: \mathbb{R} \to \mathbb{R}$  has a jump discontinuity at  $x_0$  if both  $L^{-} := \lim_{x \to x_0^-} f(x)$  and  $L^{+} := \lim_{x \to x_0^+} f(x)$  exist but  $L^{-} \neq L^{+}$ . We simply call the oscillation at  $x_0$ , i.e.,  $|L^{+} - L^{-}|$ , the jump.
Roughly put, our framework applies to piece-wise Lipschitz continuous  $f_{i}$  with (at most) countably infinite jump discontinuities, which have bounded jumps and are well separated. The precise statement is given below.
Assumption 2. For some  $(B_0, L_0, \beta_0)$ , the following statement unconditionally holds true for any realization of  $f$  (specifically,  $f_1, \ldots, f_m$  in our observations):
-  $\mathcal{D}_f$  is one of the following:  $\varnothing$ , a finite set, or a countably infinite set;
- All discontinuities of  $f$  (if any) are jump discontinuities with the jump bounded by  $B_0$ ;
-  $f$  is  $L_0$ -Lipschitz on any interval  $(a, b)$  satisfying  $(a, b) \cap \mathcal{D}_f = \varnothing$ .
-  $|a - b| \geq \beta_0$  holds for any  $a, b \in \mathcal{D}_f$ ,  $a \neq b$  (we set  $\beta_0 = \infty$  if  $|\mathcal{D}_f| \leq 1$ ).
For simplicity, we assume  $f(x_0) = \lim_{x \to x_0^+} f(x)$  for  $x_0 \in \mathcal{D}_f$ .<sup>3</sup>
We note that Assumption 2 is satisfied by  $L$ -Lipschitz  $f$  with  $(B_0, L_0, \beta_0) = (0, L, \infty)$ , 1-bit quantized observation  $f(\cdot) = \mathrm{sign}(\cdot + \tau)$  ( $\tau$  is the potential dither, similarly below) with  $(B_0, L_0, \beta_0) = (2, 0, \infty)$ , and uniformly quantized observation  $f(\cdot) = \delta \left( \left\lfloor \frac{\cdot + \tau}{\delta} \right\rfloor + \frac{1}{2} \right)$  with  $(B_0, L_0, \beta_0) = (\delta, 0, \delta)$ .
This is very mild because the observations are  $f_{i}(\pmb{a}_{i}^{\top}\pmb{x})$ , while  $\mathbb{P}(\pmb{a}^{\top}\pmb{x} \in \mathcal{D}_{f_i}) = 0$  (as  $\mathcal{D}_{f_i}$  is at most countably infinite and  $\pmb{a} \sim \mathcal{N}(0, I_n)$ ).
Under Assumption 2, for any  $\beta \in [0,\frac{\beta_0}{2})$  we construct  $f_{i,\beta}$  as the Lipschitz approximation of  $f_{i}$  to deal with the potential discontinuity of  $f_{i}$  (i.e.,  $\mathcal{D}_{f_i}\neq \emptyset$ ). Specifically,  $f_{i,\beta}$  modifies  $f_{i}$  in  $\mathcal{D}_{f_i} + [-\frac{\beta}{2},\frac{\beta}{2} ]$  to be piece-wise linear and Lipschitz continuous; see its precise definition in (3.4).
We develop Theorem 2 to bound certain product processes appearing in the analysis, which produces bounds tighter than [36] when the index sets have low metric entropy. To make Theorem 2 applicable, we further make the following Assumption 3, which can be checked case-by-case by estimating the sub-Gaussian norm and probability tail. Also,  $U_g^{(1)}$  and  $U_g^{(2)}$  can even be a bit crude because the measurement number in Theorem 1 depends on them in a logarithmic manner.
Assumption 3. Let  $\mathbf{a} \sim \mathcal{N}(0, I_n)$ , under Assumptions 1-2, we define the Lipschitz approximation  $f_{i,\beta}$  as in (3.4). We let
$$
\xi_ {i, \beta} (a) := f _ {i, \beta} (a) - T a, \varepsilon_ {i, \beta} (a) := f _ {i, \beta} (a) - f _ {i} (a). \tag {2.2}
$$
For all  $\beta \in (0, \frac{\beta_0}{2})$ , we assume the following holds with some parameters  $(A_g^{(1)}, U_g^{(1)}, P_0^{(1)})$  and  $(A_g^{(2)}, U_g^{(2)}, P_0^{(2)})$ :
-  $\sup_{\pmb{x} \in \mathcal{K}} \| \xi_{i,\beta} (\pmb{a}^\top \pmb{x}) \|_{\psi_2} \leq A_g^{(1)}$ ,  $\mathbb{P}\big(\sup_{\pmb{x} \in \mathcal{K}} |\xi_{i,\beta} (\pmb{a}^\top \pmb{x})| \leq U_g^{(1)}\big) \geq 1 - P_0^{(1)}$ ;
-  $\sup_{\pmb{x} \in \mathcal{K}} \| \varepsilon_{i,\beta} (\pmb{a}^\top \pmb{x}) \|_{\psi_2} \leq A_g^{(2)}$ ,  $\mathbb{P}\left(\sup_{\pmb{x} \in \mathcal{K}} |\varepsilon_{i,\beta} (\pmb{a}^\top \pmb{x})| \leq U_g^{(2)}\right) \geq 1 - P_0^{(2)}$ .
To build a more complete theory we further introduce two useful quantities. For some  $\pmb{x} \in \mathcal{K}$ , we define the target mismatch  $\rho(\pmb{x})$  as in [17, Definition 1]:
$$
\rho (\boldsymbol {x}) = \left\| \mathbb {E} \left[ f _ {i} \left(\boldsymbol {a} _ {i} ^ {\top} \boldsymbol {x}\right) \boldsymbol {a} _ {i} \right] - T \boldsymbol {x} \right\| _ {2}. \tag {2.3}
$$
It is easy to see that  $\mathbb{E}\big[f_i(\pmb {a}_i^\top \pmb {x})\pmb {a}_i\big]$  minimizes the expected  $\ell_2$  loss  $\mathbb{E}\big[\| \pmb {y} - \pmb {A}\pmb {x}\| _2^2\big]$ , thus one can roughly understand  $\mathbb{E}\big[f_i(\pmb {a}_i^\top \pmb {x})\pmb {a}_i\big]$  as the expectation of  $\hat{\pmb{x}}$ . Since  $Tx$  is the desired ground truth, a small  $\rho (\pmb {x})$  is intuitively an important ingredient for generalized Lasso to succeed. Fortunately, in many models,  $\rho (\pmb {x})$  with a suitably chosen  $T$  will vanish (e.g., linear model [2], single index model [33], 1-bit model [31]) or at least be sufficiently small (e.g., 1-bit model with dithering [45]).
As mentioned before, our method to deal with discontinuity of  $f_{i}$  is to introduce its approximation  $f_{i,\beta}$ , which differs from  $f_{i}$  only in  $\mathcal{D}_{f_i} + \left[-\frac{\beta}{2},\frac{\beta}{2}\right]$ . This will produce some bias because the actual observation is  $f_{i}(\pmb{a}_{i}^{\top}\pmb{x}^{*})$  rather than  $f_{i,\beta}(\pmb{a}_i^\top \pmb{x}^*)$ . Hence, for some  $\pmb{x} \in \mathcal{K}$  we define the following quantity to measure the bias induced by  $f_{i,\beta}$ :
$$
\mu_ {\beta} (\boldsymbol {x}) = \mathbb {P} \left(\boldsymbol {a} ^ {\top} \boldsymbol {x} \in \mathcal {D} _ {f _ {i}} + \left[ - \frac {\beta}{2}, \frac {\beta}{2} \right]\right), \quad \boldsymbol {a} \sim \mathcal {N} (0, I _ {n}). \tag {2.4}
$$
The following assumption can often be satisfied by choosing suitable  $T$  and sufficiently small  $\beta_{1}$
Assumption 4. Suppose Assumptions 1-3 hold true with parameters  $B_0, L_0, \beta_0, A_g^{(1)}, A_g^{(2)}$ . For the  $T$  used in (2.1),  $\rho(\pmb{x})$  defined in (2.3) satisfies
$$
\sup  _ {\boldsymbol {x} \in \mathcal {K}} \rho (\boldsymbol {x}) \lesssim \left(A _ {g} ^ {(1)} \vee A _ {g} ^ {(2)}\right) \sqrt {\frac {k}{m}}. \tag {2.5}
$$
Moreover, there exists some  $0 < \beta_{1} < \frac{\beta_{0}}{2}$  such that
$$
\left(L _ {0} \beta_ {1} + B _ {0}\right) \sup  _ {\boldsymbol {x} \in \mathcal {K}} \sqrt {\mu_ {\beta_ {1}} (\boldsymbol {x})} \lesssim \left(A _ {g} ^ {(1)} \vee A _ {g} ^ {(2)}\right) \sqrt {\frac {k}{m}}. \tag {2.6}
$$
In the proof, the estimation error  $\| \hat{\pmb{x}} - T\pmb{x}^* \|$  is contributed by a concentration term of scaling  $\tilde{O}\big((A_g^{(1)} \vee A_g^{(2)})\sqrt{k / m}\big)$  and some bias terms. The main aim of Assumption 4 is to pull down the bias terms so that the concentration term is dominant.
2.3 Main Theorem and its Implications
We now present our general theorem and apply it to some specific models.
Theorem 1. Under Assumptions 1-4, given any recovery accuracy  $\epsilon \in (0,1)$ , if it holds that  $m \gtrsim (A_g^{(1)} \vee A_g^{(2)})^2 \frac{k\mathcal{L}}{\epsilon^2}$ , then with probability at least  $1 - m(P_0^{(1)} + P_0^{(2)}) - m\exp(-\Omega(n)) - C\exp(-\Omega(k))$  on a single realization of  $(\mathbf{A}, \mathbf{f}) \coloneqq (\mathbf{a}_i, f_i)_{i=1}^m$ , we have the uniform signal recovery guarantee  $\|\hat{\mathbf{x}} - T\mathbf{x}^*\|_2 \leq \epsilon$  for all  $\mathbf{x}^* \in \mathcal{K}$ , where  $\hat{\mathbf{x}}$  is the solution to (2.1) with  $\mathbf{y} = \mathbf{f}(\mathbf{A}\mathbf{x}^*)$ , and  $\mathcal{L} = \log \widetilde{P}$  is a logarithmic factor with  $\widetilde{P}$  being polynomial in  $(L,n)$  and other parameters that typically scale as  $O(L + n)$ . See (C.11) for the precise expression of  $\mathcal{L}$ .