SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. Since  $\pmb{x} \in \mathbb{S}^{n-1}$ , for some orthogonal matrix  $\pmb{P}$  we have  $\pmb{P}\pmb{x} = \pmb{e}_1$  (the first column of  $I_n$ ). Since  $\tilde{\pmb{a}} := \pmb{P}\pmb{a} = [\tilde{a}_i]$  has the same distribution as  $\pmb{a}$ , we have
$$
\begin{array}{l} \mathbb {E} [ \operatorname {s i g n} (\boldsymbol {a} ^ {\top} \boldsymbol {x}) \boldsymbol {a} ] = \mathbb {E} [ \operatorname {s i g n} (\tilde {\boldsymbol {a}} ^ {\top} \boldsymbol {e} _ {1}) \boldsymbol {P} ^ {\top} \tilde {\boldsymbol {a}} ] = \boldsymbol {P} ^ {\top} \mathbb {E} [ \operatorname {s i g n} (\tilde {\boldsymbol {a}} _ {1}) \tilde {\boldsymbol {a}} ] \\ = \boldsymbol {P} ^ {\top} \sqrt {\frac {2}{\pi}} \boldsymbol {e} _ {1} = \sqrt {\frac {2}{\pi}} \boldsymbol {x}. \\ \end{array}
$$
D.4 Proof of Lemma 9 (Choice of  $T$  in 1-bit GCS with dithering).
Proof. We first note that
$$
\left\| \mathbb {E} [ \operatorname {s i g n} (\boldsymbol {a} ^ {\top} \boldsymbol {x} + \tau) \boldsymbol {a} ] - \frac {\boldsymbol {x}}{\lambda} \right\| _ {2} = \frac {1}{\lambda} \sup  _ {\boldsymbol {v} \in \mathbb {S} ^ {n - 1}} \left(\mathbb {E} \left[ \lambda \cdot \operatorname {s i g n} (\boldsymbol {a} ^ {\top} \boldsymbol {x} + \tau) \boldsymbol {a} ^ {\top} \boldsymbol {v} \right] - \boldsymbol {x} ^ {\top} \boldsymbol {v}\right). \tag {D.1}
$$
We first fix  $\pmb{a}$  and expect over  $\pmb{\tau} \sim \mathcal{U}[-\lambda, \lambda]$  to obtain
$$
\begin{array}{l} \mathbb {E} _ {\tau} \left[ \lambda \operatorname {s i g n} \left(\boldsymbol {a} ^ {\top} \boldsymbol {x} + \tau\right) \boldsymbol {a} ^ {\top} \boldsymbol {v} \right] \\ = (\lambda \boldsymbol {a} ^ {\top} \boldsymbol {v}) \left(\mathbb {1} (| \boldsymbol {a} ^ {\top} \boldsymbol {x} | > \lambda) \operatorname {s i g n} (\boldsymbol {a} ^ {\top} \boldsymbol {x}) + \mathbb {1} (| \boldsymbol {a} ^ {\top} \boldsymbol {x} | \leq \lambda) \cdot \left(\frac {\lambda + \boldsymbol {a} ^ {\top} \boldsymbol {x}}{2 \lambda} - \frac {\lambda - \boldsymbol {a} ^ {\top} \boldsymbol {x}}{2 \lambda}\right)\right) \\ = (\boldsymbol {a} ^ {\top} \boldsymbol {x}) (\boldsymbol {a} ^ {\top} \boldsymbol {v}) \mathbb {1} (| \boldsymbol {a} ^ {\top} \boldsymbol {x} | \leq \lambda) + (\lambda \boldsymbol {a} ^ {\top} \boldsymbol {v}) \operatorname {s i g n} (\boldsymbol {a} ^ {\top} \boldsymbol {x}) \mathbb {1} (| \boldsymbol {a} ^ {\top} \boldsymbol {x} | > \lambda). \\ \end{array}
$$
We plug this into (D.1), and note that  $\pmb{x}^{\top}\pmb{v} = \mathbb{E}[(\pmb{a}^{\top}\pmb{x})(\pmb{a}^{\top}\pmb{v})]$ , which gives
$$
\begin{array}{l} \left\| \mathbb {E} [ \operatorname {s i g n} (\boldsymbol {a} ^ {\top} \boldsymbol {x} + \tau) \boldsymbol {a} ] - \frac {\boldsymbol {x}}{\lambda} \right\| _ {2} \\ = \frac {1}{\lambda} \sup  _ {\boldsymbol {v} \in \mathbb {S} ^ {n - 1}} \mathbb {E} \left(\left[ (\lambda \boldsymbol {a} ^ {\top} \boldsymbol {v}) \operatorname {s i g n} (\boldsymbol {a} ^ {\top} \boldsymbol {x}) - (\boldsymbol {a} ^ {\top} \boldsymbol {x}) (\boldsymbol {a} ^ {\top} \boldsymbol {v}) \right] \mathbb {1} (| \boldsymbol {a} ^ {\top} \boldsymbol {x} | > \lambda)\right) \tag {D.2} \\ \leq \frac {1}{\lambda} \sup  _ {\boldsymbol {v} \in \mathbb {S} ^ {n - 1}} \mathbb {E} \left(\left[ \lambda | \boldsymbol {a} ^ {\top} \boldsymbol {v} | + | \boldsymbol {a} ^ {\top} \boldsymbol {x} | | \boldsymbol {a} ^ {\top} \boldsymbol {v} | \right] \mathbb {1} \left(| \boldsymbol {a} ^ {\top} \boldsymbol {x} | > \lambda\right)\right) \\ \end{array}
$$
For any  $\pmb{x} \in \mathbb{B}_2^n(R)$  and  $\pmb{v} \in \mathbb{S}^{n-1}$ , we have  $\| \pmb{a}^\top \pmb{x} \|_{\psi_2} = O(R)$  and  $\| \pmb{a}^\top \pmb{v} \|_{\psi_2} = O(1)$ . Applying the Cauchy-Schwarz inequality, we obtain
$$
\begin{array}{l} \mathbb {E} \left(\left[ \lambda | \boldsymbol {a} ^ {\top} \boldsymbol {v} | + | \boldsymbol {a} ^ {\top} \boldsymbol {x} | | \boldsymbol {a} ^ {\top} \boldsymbol {v} | \right] \mathbb {1} \left(| \boldsymbol {a} ^ {\top} \boldsymbol {x} | > \lambda\right)\right) \\ \leq \sqrt {\mathbb {E} [ (\lambda | \boldsymbol {a} ^ {\top} \boldsymbol {v} | + | \boldsymbol {x} ^ {\top} \boldsymbol {a} \boldsymbol {a} ^ {\top} \boldsymbol {v} |) ^ {2} ]} \sqrt {\mathbb {P} (| \boldsymbol {a} ^ {\top} \boldsymbol {x} | > \lambda)} \\ \leq \sqrt {2 \left(\lambda^ {2} \mathbb {E} [ | \boldsymbol {a} ^ {\top} \boldsymbol {v} | ^ {2} ] + \mathbb {E} [ (\boldsymbol {a} ^ {\top} \boldsymbol {x}) ^ {2} (\boldsymbol {a} ^ {\top} \boldsymbol {v}) ^ {2} ]\right)} \sqrt {2 \exp (- c \lambda^ {2} / R ^ {2})} \\ \lesssim \lambda \exp \left(- \frac {c \lambda^ {2}}{R ^ {2}}\right) \lesssim \frac {\lambda}{m ^ {9}} \\ \end{array}
$$
Note that in the third line, we use the probability tail bound of the sub-Gaussian  $|\pmb{a}^{\top}\pmb{x}|$ , and in the last line, we use  $\lambda = CR\sqrt{\log m}$  with some sufficiently large  $C$ . The proof is completed by substituting this into (D.2).
D.5 Proof of Lemma 10 (Choice of  $T$  in SIM).
Proof. This lemma slightly generalizes that of Lemma 8. We again choose an orthogonal matrix  $\pmb{P}$  such that  $P\pmb{x} = e_1$ , where  $e_1$  represents the first column of  $I_n$ . Since  $\pmb{a}$  and  $P\pmb{a}$  have the same distribution, we have
$$
\begin{array}{l} \mathbb {E} [ f (\boldsymbol {a} ^ {\top} \boldsymbol {x}) \boldsymbol {a} ] = \boldsymbol {P} ^ {\top} \mathbb {E} [ f ((\boldsymbol {P a}) ^ {\top} \boldsymbol {e} _ {1}) \boldsymbol {P a} ] \\ = \boldsymbol {P} ^ {\top} \mathbb {E} [ f (\boldsymbol {a} ^ {\top} \boldsymbol {e} _ {1}) \boldsymbol {a} ] = \boldsymbol {P} ^ {\top} (\mu \boldsymbol {e} _ {1}) = \mu \boldsymbol {x}. \\ \end{array}
$$
D.6 Proof of Lemma 11 (Choice of  $T$  in uniformly quantized GCS with dithering).
Proof. In the theorem, the statement before "In particular" can be found in [18, Theorem 1]. Based on this, we have  $\mathbb{E}[\mathcal{Q}_{\delta}(\pmb{a}^{\top}\pmb{x} + \tau)\pmb{a}] = \mathbb{E}_{\pmb{a}}\mathbb{E}_{\tau}[\mathcal{Q}_{\delta}(\pmb{a}^{\top}\pmb{x} + \tau)\pmb{a}] = \mathbb{E}_{\pmb{a}}(\pmb{a}\pmb{a}^{\top}\pmb{x}) = \pmb{x}$ .
D.7 Proof of Lemma 12. (Bounds on  $|\xi_{i,\beta}|$  and  $|\varepsilon_{i,\beta}|$  for the uniform quantizer).
Proof. By the definition of  $f_{i,\beta}$  in (3.4), we have  $|\varepsilon_{i,\beta}(a)| = |f_{i,\beta}(a) - f_i(a)| \leq \delta$ . It follows that  $f_{i}(\cdot) = \mathcal{Q}_{\delta}(\cdot + \tau)$  with  $\mathcal{Q}_{\delta}(a) = \delta \left( \left\lfloor \frac{a}{\delta} \right\rfloor + \frac{1}{2} \right)$ , and  $|\mathcal{Q}_{\delta}(a) - a| \leq \frac{\delta}{2}$  holds for any  $a \in \mathbb{R}$ . Hence, we have  $|f_{i}(a) - a| = |\mathcal{Q}_{\delta}(a + \tau) - (a + \tau) + \tau| \leq |\mathcal{Q}_{\delta}(a + \tau) - (a + \tau)| + |\tau| \leq \frac{\delta}{2} + \frac{\delta}{2} = \delta$ . To complete the proof, we use the inequalities  $|\xi_{i,\beta}(a)| \leq |f_{i,\beta}(a) - f_i(a)| + |f_i(a) - a| \leq \delta + \delta = 2\delta$ .
D.8 Proof of Lemma 13. (Bound on the approximation error  $|\varepsilon_{i,\beta}|$ )
Proof. For any  $a \notin \mathcal{D}_{f_i} + \left[-\frac{\beta}{2}, \frac{\beta}{2}\right]$ , by the definition in (3.4) we have  $\varepsilon_{i,\beta}(a) = 0$ . If  $a \in [x_0 - \frac{\beta}{2}, x_0]$  for some  $x_0 \in \mathcal{D}_{f_i}$ , then we have
$$
\begin{array}{l} \left| \varepsilon_ {i, \beta} (a) \right| = \left| f _ {i, \beta} (a) - f _ {i} (a) \right| \\ \leq \left| f _ {i, \beta} (a) - f _ {i, \beta} \left(x _ {0}\right) \right| + \left| f _ {i, \beta} \left(x _ {0}\right) - f _ {i} ^ {-} \left(x _ {0}\right) \right| + \left| f _ {i} ^ {-} \left(x _ {0}\right) - f _ {i} (a) \right| \\ \leq \left(2 L _ {0} + \frac {B _ {0}}{\beta}\right) | a - x _ {0} | + | f _ {i} ^ {a} (x _ {0}) - f _ {i} ^ {-} (x _ {0}) | + L _ {0} | x _ {0} - a | \\ \leq \left(3 L _ {0} + \frac {B _ {0}}{\beta}\right) \cdot \frac {\beta}{2} + \frac {1}{2} \left| f _ {i} ^ {+} \left(x _ {0}\right) - f _ {i} ^ {-} \left(x _ {0}\right) \right| \leq \frac {3 L _ {0} \beta}{2} + B _ {0}, \\ \end{array}
$$
where we use Lemma 1 and Assumption 2 in the third line, and use  $|a - x_0| \leq \frac{\beta}{2}$  and  $f_i^a(x_0) = \frac{1}{2} (f_i^-(x_0) + f_i^+(x_0))$  in the fourth line.
E Parameter Selection for Specific Models
E.1 1-bit GCS
To specialize Theorem 1 to this model, we select the parameters as follows:
- Assumption 2. Under the 1-bit observation model  $y_{i} = \mathrm{sign}(a_{i}^{\top}\pmb{x}^{*})$ , the function  $f_{i}(\cdot) = f(\cdot) = \mathrm{sign}(\cdot)$  satisfies Assumption 2 with  $(B_0, L_0, \beta_0) = (2, 0, \infty)$ .
(2.5) in Assumption 4. Recall that  $\mathcal{K} \subset \mathbb{S}^{n-1}$ . Under the assumption  $\|x^*\|_2 = 1$ , we set  $T = \sqrt{2/\pi}$  so that  $\rho(x) = 0$  holds for all  $x \in \mathcal{X}$  (Lemma 8), which provides (2.5).
- Assumption 3. By Lemma 7, we have  $\mathbb{P}(\| \pmb{a}\|_2 = O(\sqrt{n})) \geq 1 - 2\exp(-\Omega(n))$ , and we suppose that this high-probability event holds. Also note that  $|f_{i,\beta}| \leq 1$ . Hence, we have
$$
\begin{array}{l} \left\| \xi_ {i, \beta} (\boldsymbol {a} ^ {\top} \boldsymbol {x}) \right\| _ {\psi_ {2}} \leq \left\| f _ {i, \beta} (\boldsymbol {a} ^ {\top} \boldsymbol {x}) \right\| _ {\psi_ {2}} + \left\| T \boldsymbol {a} ^ {\top} \boldsymbol {x} \right\| _ {\psi_ {2}} = O (1), \\ \sup  _ {\boldsymbol {x} \in \mathcal {K}} | \xi_ {i, \beta} (\boldsymbol {a} ^ {\top} \boldsymbol {x}) | \leq | f _ {i, \beta} (\boldsymbol {a} ^ {\top} \boldsymbol {x}) | + | T \boldsymbol {a} ^ {\top} \boldsymbol {x} | \leq 1 + T \| \boldsymbol {a} \| _ {2} = O (\sqrt {n}). \\ \end{array}
$$
Because  $\varepsilon_{i,\beta} = f_{i,\beta} - f_i$ , we have  $\| \varepsilon_{i,\beta}(\pmb{a}^{\top}\pmb{x}) \|_{\psi_2} = O(1)$ , and  $|\varepsilon_{i,\beta}(\pmb{a}^{\top}\pmb{x})| \leq 2$  holds deterministically. Hence, regarding the parameters in Assumption 3, we can take
$$
A _ {g} ^ {(1)} \asymp 1, U _ {g} ^ {(1)} \asymp \sqrt {n}, P _ {0} ^ {(1)} \asymp \exp (- \Omega (n)), A _ {g} ^ {(2)} \asymp 1, U _ {g} ^ {(2)} \asymp 1, P _ {0} ^ {(2)} = 0.
$$
(2.6) in Assumption 4. It remains to pick  $\beta_{1}$  that satisfies (2.6). Note that  $\mathcal{D}_{f_i} = \{0\}$ , and for any  $\pmb {x}\in \mathcal{K}$ ,  $\pmb{a}^{\top}\pmb {x}\sim \mathcal{N}(0,1)$ , so we have
$$
\mu_ {\beta} (\pmb {x}) = \mathbb {P} \left(\pmb {a} ^ {\top} \pmb {x} \in \left[ - \frac {\beta}{2}, \frac {\beta}{2} \right]\right) = O (\beta).
$$
Thus, we take  $\beta = \beta_{1} \asymp \frac{k}{m}$  to guarantee (2.6).
E.2 1-bit GCS with dithering
- Assumption 2. The observation function can be written as  $f(\cdot) = \mathrm{sign}(\cdot + \tau)$  with  $\tau \sim \mathcal{U}[-\lambda, \lambda]$ , which satisfies Assumption 2 with  $(B_0, L_0, \beta_0) = (2, 0, \infty)$ .
(2.5) in Assumption 4. We set  $\lambda = CR\sqrt{\log m}$  with  $C$  large enough, so that Lemma 9 justifies (2.5).
- Assumption 3. By Lemma 7, we have  $\mathbb{P}(\| \pmb{a}\|_2 = O(\sqrt{n})) \geq 1 - 2\exp(-\Omega(n))$ . Assume this event holds, and note that  $f_{i,\beta}$  is still bounded by 1, we have
$$
\begin{array}{l} \left\| \xi_ {i, \beta} (\boldsymbol {a} ^ {\top} \boldsymbol {x}) \right\| _ {\psi_ {2}} \leq \left\| f _ {i, \beta} (\boldsymbol {a} ^ {\top} \boldsymbol {x}) \right\| _ {\psi_ {2}} + \left\| \lambda^ {- 1} \boldsymbol {a} ^ {\top} \boldsymbol {x} \right\| _ {\psi_ {2}} = O (R / \lambda) + O (1) = O (1), \\ \sup  _ {\boldsymbol {x} \in \mathcal {K}} | \xi_ {i, \beta} (\boldsymbol {a} ^ {\top} \boldsymbol {x}) | \leq 1 + \sup  _ {\boldsymbol {x} \in \mathcal {K}} | \lambda^ {- 1} \boldsymbol {a} ^ {\top} \boldsymbol {x} | \leq 1 + \sup  _ {\boldsymbol {x} \in \mathcal {K}} \lambda^ {- 1} \| \boldsymbol {a} \| _ {2} \| \boldsymbol {x} \| _ {2} = O (\sqrt {n}). \\ \end{array}
$$
Moreover, because  $\varepsilon_{i,\beta} = f_{i,\beta} - f_i$ , the following hold deterministically:  $\| \varepsilon_{i,\beta}(\pmb{a}^{\top}\pmb{x})\|_{\psi_2} = O(1)$ ,  $\sup_{\pmb{x}\in \mathcal{K}}|\varepsilon_{i,\beta}(\pmb{a}^{\top}\pmb{x})| = O(1)$ . Thus, regarding the parameters in Assumption 3 we can take
(2.6) in Assumption 4. It remains to confirm (2.6) for suitable  $\beta_{1}$ . For any  $\beta$ , note that  $\mathcal{D}_{f_i} + [-\frac{\beta}{2}, \frac{\beta}{2}] = [\tau - \frac{\beta}{2}, \tau + \frac{\beta}{2}]$ , and hence for any  $\pmb{x} \in \mathcal{K} \subset \mathbb{B}_2^n(R)$  we have
$$
\mu_ {\beta} (\boldsymbol {x}) = \mathbb {P} \left(\boldsymbol {a} ^ {\top} \boldsymbol {x} \in \left[ - \tau - \frac {\beta}{2}, - \tau + \frac {\beta}{2} \right]\right) = \mathbb {P} \left(\boldsymbol {a} ^ {\top} \boldsymbol {x} + \tau \in \left[ - \frac {\beta}{2}, \frac {\beta}{2} \right]\right) \leq \frac {\beta}{\lambda},
$$
which can be seen by conditioning on  $\mathbf{a}$ . Hence, we can take  $\beta = \beta_{1} = \frac{\lambda k}{m}$  to guarantee (2.6).
E.3 Lipschitz-continuous SIM with generative prior
- Assumption 2. Since  $f$  is  $\hat{L}$ -Lipschitz by assumption, it satisfies Assumption 2 with  $(B_0, L_0, \beta_0) = (0, \hat{L}, \infty)$ .
(2.5) in Assumption 4. Recall that we have defined the quantities  $\mu = \mathbb{E}[f(g)g],\psi = \| f(g)\|_{\psi_2}$ , where  $g\sim \mathcal{N}(0,1)$ . Then, we choose  $T = \mu$  so that  $\rho (\pmb {x}) = 0$  holds for any  $\pmb{x}$  (Lemma 10), thus justifying (2.5).
- Assumption 3. Because  $f_{i}$  is  $\hat{L}$ -Lipschitz and does not contain any discontinuity, there is no need to construct the Lipschitz approximation  $f_{i,\beta}$  for some  $\beta > 0$ , while we simply use  $\beta = 0$  which implies  $f_{i,\beta} = f_{i}$  and  $\varepsilon_{i,\beta} = 0$ . Note that  $\xi_{i,\beta}(a) = f_{i}(a) - \mu a$ , and so we have
$$
\left\| f _ {i} \left(\boldsymbol {a} ^ {\top} \boldsymbol {x}\right) - \mu \boldsymbol {a} ^ {\top} \boldsymbol {x} \right\| _ {\psi_ {2}} \leq \left\| f _ {i} \left(\boldsymbol {a} ^ {\top} \boldsymbol {x}\right) \right\| _ {\psi_ {2}} + \left\| \mu \boldsymbol {a} ^ {\top} \boldsymbol {x} \right\| _ {\psi_ {2}} = O (\psi + \mu).
$$
We suppose  $\| \boldsymbol{a}\| _2 = O(\sqrt{n})$ , which holds with probability at least  $1 - 2\exp (-\Omega (n))$  (Lemma 7); we also suppose  $f_{i}(0)\leq \hat{B}$ , which holds with probability at least  $1 - P_0^{\prime}$  by assumption. On these two events, we have
$$
\begin{array}{l} \left| f _ {i} \left(\boldsymbol {a} ^ {\top} \boldsymbol {x}\right) - \mu \boldsymbol {a} ^ {\top} \boldsymbol {x} \right| \leq \left| f _ {i} \left(\boldsymbol {a} ^ {\top} \boldsymbol {x}\right) - f _ {i} (0) \right| + \left| f _ {i} (0) \right| + \mu \| \boldsymbol {a} \| _ {2} \| \boldsymbol {x} \| _ {2} \\ \le \hat {L} \| \boldsymbol {a} \| _ {2} + \hat {B} + \mu \| \boldsymbol {a} \| _ {2} \lesssim (\hat {L} + \mu) \sqrt {n} + \hat {B}. \\ \end{array}
$$
Combined with  $\varepsilon_{i,\beta} = 0$ , we can set the parameters in Assumption 3 as follows:
$$
\begin{array}{l} A _ {g} ^ {(1)} \asymp \psi + \mu , U _ {g} ^ {(1)} \asymp (\hat {L} + \mu) \sqrt {n} + \hat {B}, P _ {0} ^ {(1)} \asymp P _ {0} ^ {\prime} + \exp (- \Omega (n)), \\ A _ {g} ^ {(2)} \asymp \psi + \mu , U _ {g} ^ {(2)} = 0, P _ {0} ^ {(2)} = 0. \\ \end{array}
$$
(2.6) in Assumption 4. Because  $\beta = 0$  and  $\mathcal{D}_{f_i} = \varnothing$ , (2.6) is trivially satisfied.
E.4 Uniformly quantized GCS with dithering
- Assumption 2. The uniform quantizer with resolution  $\delta > 0$  is defined as  $\mathcal{Q}_{\delta}(a) = \delta \left( \left\lfloor \frac{a}{\delta} \right\rfloor + \frac{1}{2} \right)$  for  $a \in \mathbb{R}$ . We consider this quantizer with dithering  $\tau_{i} \sim \mathcal{U}[-\frac{\delta}{2}, \frac{\delta}{2}]$ . Specifically, we observe  $y_{i} = \mathcal{Q}_{\delta} (\boldsymbol{a}_{i}^{\top} \boldsymbol{x}^{*} + \tau_{i})$ , so the observation function is  $f(\cdot) = \mathcal{Q}_{\delta}(\cdot + \tau)$  with  $\tau \sim \mathcal{U}[-\frac{\delta}{2}, \frac{\delta}{2}]$ . Hence, Assumption 2 is satisfied with  $(B_0, L_0, \beta_0) = (\delta, 0, \delta)$ .
(2.5) in Assumption 4. The benefit of dithering is to whiten the quantization noise. With  $T = 1$ , for any  $\pmb{x} \in \mathcal{K}$ , Lemma 11 implies  $\rho(\pmb{x}) = \| \mathbb{E}[\mathcal{Q}_{\delta}(\pmb{a}_i^\top \pmb{x} + \tau_i)\pmb{a}_i] - \pmb{x}\|_2 = 0$ , thus justifying (2.5).
- Assumption 3. Note that for any  $\beta \in (0, \frac{\delta}{2})$ , by Lemma 12, we can take the parameters for Assumption 3 as follows:
$$
A _ {g} ^ {(1)}, U _ {g} ^ {(1)}, A _ {g} ^ {(2)}, U _ {g} ^ {(2)} \asymp \delta , P _ {0} ^ {(1)} = P _ {0} ^ {(2)} = 0.
$$
- (2.6) in Assumption 4. All that remains is to pick  $\beta = \beta_{1}$  that satisfies (2.6). Because  $\mathcal{D}_{f_i} = -\tau_i + \delta \mathbb{Z}$ , hence for any  $\pmb{x} \in \mathcal{K}$  we have
$$
\mu_ {\beta} (\boldsymbol {x}) = \mathbb {P} \left(\boldsymbol {a} ^ {\top} \boldsymbol {x} \in - \tau + \delta \mathbb {Z} + \left[ - \frac {\beta}{2}, \frac {\beta}{2} \right]\right) = \mathbb {P} \left(\boldsymbol {a} ^ {\top} \boldsymbol {x} + \tau \in \delta \mathbb {Z} + \left[ - \frac {\beta}{2}, \frac {\beta}{2} \right]\right) = O \left(\frac {\beta}{\bar {\delta}}\right),
$$
which can be seen by using the randomness of  $\tau \sim \mathcal{U}\left[-\frac{\delta}{2}, \frac{\delta}{2}\right]$  conditionally on  $\pmb{x}$ . Hence, we take  $\beta = \beta_{1} \asymp \frac{k\delta}{m}$ , which provides (2.6).
F Handling Sub-Gaussian Additive Noise
In this appendix, we describe how our results can be extended to the noisy model  $\pmb{y} = \pmb{f}(\pmb{A}\pmb{x}^{*}) + \pmb{\eta}$ , where  $\pmb{\eta} \in \mathbb{R}^m$  is the noise vector that is independent of  $(\pmb{A},\pmb{f})$  and has i.i.d. sub-Gaussian entries  $\eta_{i}$  satisfying  $\| \eta_i\|_{\psi_2} = O(1)$ . Along similar lines as in (3.1)-(3.3), we find that  $\pmb{\eta}$  gives rise to an additional term  $\frac{2}{m}\langle \pmb{\eta},A(\hat{\pmb{x}} -T\pmb{x}^{*})\rangle$  to the right-hand side of (3.1), which is bounded by
$2\|\hat{\boldsymbol{x}}-Tx^{*}\|_{2} \cdot \sup_{\boldsymbol{v} \in (\mathcal{K}_{\epsilon}^{-})^{*}} \frac{1}{m}\langle \boldsymbol{\eta}, \boldsymbol{A}\boldsymbol{v} \rangle$ , with the constraint set  $(\mathcal{K}_{\epsilon}^{-})^{*}$  defined in (3.2). Thus, in (3.3), in addition to  $\mathcal{R}_u$ , in the noisy setting we need to bound the additional term
$$
\mathcal {R} _ {u} ^ {\prime} := \sup  _ {\boldsymbol {v} \in (\mathcal {K} _ {\epsilon} ^ {-}) ^ {*}} \frac {1}{m} \langle \boldsymbol {\eta}, \boldsymbol {A} \boldsymbol {v} \rangle = \sup  _ {\boldsymbol {v} \in (\mathcal {K} _ {\epsilon} ^ {-}) ^ {*}} \frac {1}{m} \sum_ {i = 1} ^ {m} \eta_ {i} \boldsymbol {a} _ {i} ^ {\top} \boldsymbol {v}.
$$
This can be done by the following lemma, which indicates that the sharp (uniform) rate in Theorem 1 can be retained in the presence of noise  $\pmb{\eta}$ .
Lemma 15. (Bounding the additional term  $\mathcal{R}_u'$ ). In the noisy setting described above, with probability at least  $1 - C_1\exp (-\Omega (k\log \frac{T L r}{\epsilon})) - C_2\exp (-\Omega (m))$ , we have  $\mathcal{R}_u' \lesssim \sqrt{\frac{k\log\frac{T L r}{\epsilon}}{m}}$ .
Proof. Conditioning on  $\eta$ , the randomness of  $\pmb{a}_i$ 's gives  $\frac{1}{m}\sum_{i=1}^{m}\eta_i\pmb{a}_i \sim \mathcal{N}(0,\frac{\|\pmb{\eta}\|_2^2}{m^2}\pmb{I}_n)$ , and so  $\|( \frac{1}{m}\sum_{i=1}^{m}\eta_i\pmb{a}_i)^\top \pmb{v}_1 - (\frac{1}{m}\sum_{i=1}^{m}\eta_i\pmb{a}_i)^\top \pmb{v}_2 \|_{\psi_2} \leq \frac{C_0\|\pmb{\eta}\|_2\|\pmb{v}_1 - \pmb{v}_2\|_2}{m}$  holds for any  $\pmb{v}_1, \pmb{v}_2 \in \mathbb{R}^n$ . Let  $\omega(\cdot)$  be the Gaussian width as defined in Lemma 5. Then, using the randomness of  $\pmb{a}_i$ 's, Talagrand's comparison inequality [50, Exercise 8.6.5] yields that for any  $t \geq 0$ , we have
$$
\mathbb {P} \left(\mathcal {R} _ {u} ^ {\prime} \leq \frac {C _ {1} \| \boldsymbol {\eta} \| _ {2} \cdot [ \omega ((\mathcal {K} _ {\epsilon} ^ {-}) ^ {*}) + t ]}{m}\right) \geq 1 - 2 \exp (- t ^ {2}). \tag {F.1}
$$
Next, we bound the Gaussian width  $\omega((\mathcal{K}_{\epsilon}^{-})^{*})$ . Recall that  $(\mathcal{K}_{\epsilon}^{-})^{*}$  is defined in (3.2), and Lemma 6 bounds its metric entropy as  $\mathcal{H}((\mathcal{K}_{\epsilon}^{-})^{*}, \eta) \leq 2k \log \frac{12TLr}{\epsilon\eta}$ . Thus, we can invoke Dudley's integral inequality [50, Theorem 8.1.3] to obtain
$$
\omega \left(\left(\mathcal {K} _ {\epsilon} ^ {-}\right) ^ {*}\right) \leq C _ {2} \int_ {0} ^ {2} \sqrt {2 k \log \frac {1 2 T L r}{\epsilon \eta}} d \eta \lesssim \sqrt {k \log \frac {T L r}{\epsilon}}.
$$
Now, we further let  $t = \sqrt{k\log\frac{T L r}{\epsilon}}$  in (F.1) to obtain that  $\mathcal{R}_u' \lesssim \frac{\|\eta\|_2\sqrt{k\log\frac{T L r}{\epsilon}}}{m}$  holds with probability at least  $1 - 2\exp(-k\log\frac{T L r}{\epsilon})$ . It remains to deal with the randomness of  $\pmb{\eta}$  and bound  $\| \pmb{\eta}\|_2$ . Because  $\pmb{\eta}$  has i.i.d. entries with  $\| \eta_i\|_{\psi_2} = O(1)$ , by [50, Theorem 3.1.1] we can obtain that  $\| \pmb{\eta}\|_2 \leq C_3\sqrt{m}$  with probability at least  $1 - 2\exp(-c_3m)$ . Substituting this bound into  $\mathcal{R}_u' \lesssim \frac{\|\eta\|_2\sqrt{k\log\frac{T L r}{\epsilon}}}{m}$ , the result follows.
To close this appendix, we briefly state how to adapt the proof of Theorem 1 to explicitly include the additive noise  $\pmb{\eta}$ . Specifically, the left-hand side of (3.1) and its uniform lower bound  $\Omega (\| \hat{\pmb{x}} -T\pmb{x}^{*}\|_{2}^{2})$  remain unchanged, while the right-hand side of (3.1) is now bounded by  $2\| \hat{\pmb{x}} -T\pmb{x}^{*}\|_{2}\cdot (\mathcal{R}_{u1}+$ $\mathcal{R}_{u2} + \mathcal{R}_u^{\prime})$  (with  $2\| \hat{\pmb{x}} -T\pmb{x}^{*}\|_{2}\mathcal{R}_{u}^{\prime}$  being the additional term); thus, combining the bound (C.12) on  $2\| \hat{\pmb{x}} -T\pmb{x}^{*}\|_{2}\cdot (\mathcal{R}_{u1} + \mathcal{R}_{u2})$  and Lemma 15, we establish a uniform upper bound
$$
O \left(\| \hat {\boldsymbol {x}} - T \boldsymbol {x} ^ {*} \| _ {2} \cdot \left[ \left(A _ {g} ^ {(1)} \vee A _ {g} ^ {(2)}\right) \sqrt {\frac {k \mathscr {L}}{m}} + \sqrt {\frac {k \log \frac {T L r}{\epsilon}}{m}} \right]\right)
$$
for the right-hand side of (3.1). Therefore, to ensure uniform recovery up to the  $\ell_2$ -norm accuracy of  $\epsilon$  under the sub-Gaussian noise  $\eta$ , it suffices to have a sample complexity
$$
m \gtrsim \left(A _ {g} ^ {(1)} \vee A _ {g} ^ {(2)}\right) ^ {2} \frac {k \mathscr {L}}{\epsilon^ {2}} + \frac {k \log \frac {T L r}{\epsilon}}{\epsilon^ {2}}. \tag {F.2}
$$
Since the logarithmic factors  $\mathcal{L}$  in (C.11) dominates  $\log \frac{T L r}{\epsilon}$ , (F.2) indeed coincides with the sample complexity  $m \gtrsim (A_g^{(1)} \vee A_g^{(2)})^2 \frac{k \mathcal{L}}{\epsilon^2}$  in Theorem 1 under the mild condition of  $A_g^{(1)} \vee A_g^{(2)} = \Omega(1)$ .
G Experimental Results for the MNIST dataset
G.1 Details of the Settings
In this section, we conduct experiments on the MNIST dataset [28] to support our theoretical framework. We use various nonlinear measurement models, including 1-bit, dithered 1-bit, ReLU,
Figure 2: Reconstructed images of the MNIST dataset for the noiseless 1-bit measurements with  $m = 150$ .
and uniformly quantized CS with dithering (UQD). We select 30 images from the MNIST testing set, ensuring that there are three images from each of the 10 classes for maximum variability. A single measurement matrix  $\mathbf{A}$  is generated and used for all 30 test images. All the experiments are repeated for 10 random trials. All the experiments are run using Python 3.10.6 and PyTorch 2.0.0, with an NVIDIA RTX 3060 Laptop 6GB GPU.
We train a variational autoencoder (VAE) on the training set of the MNIST dataset, which has 60,000 images, each of size 784. The decoder of the VAE is a fully connected neural network with ReLU activations, with input dimension  $k = 20$  and output dimension  $n = 784$ , and two hidden layers with 500 neurons each. We train the VAE using the Adam optimizer with a mini-batch size of 100 and a learning rate of 0.001.