SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Since our contributions are primarily theoretical, we only provide simple proof-of-concept experimental results. In particular, since (2.1) is intractable to solve exactly, to estimate the underlying signal, we choose to use the algorithm proposed in [2] (referred to as CSGM) to approximate it. CSGM performs a gradient descent algorithm in the latent space in  $\mathbb{R}^k$  with random restarts. In addition, we compare with the Lasso program that is solved by the iterative shrinkage thresholding algorithm.
For CSGM, we follow the setting in [2] and perform 10 random restarts with 1000 gradient descent steps per restart and pick the reconstruction with the best measurement error.
G.2 Experimental Results for Noiseless 1-bit Measurements and Uniformly Quantized Measurements with Dithering
In this subsection, we present the numerical results for 1-bit measurements and uniformly quantized measurements with dithering, while the results for dithered 1-bit measurements and the Lipschitz SIM where the nonlinear link function is ReLU are similarly provided in Appendix G.3. For 1-bit measurements, since the underlying signal is assumed to be a unit vector and we aim to recover the direction of the signal, we use cosine similarity that is calculated as  $\hat{\boldsymbol{x}}^T\boldsymbol{x}^* / (\|\hat{\boldsymbol{x}}\|_2 \cdot \| \boldsymbol{x}^*\|_2)$  with  $\hat{\boldsymbol{x}}$  being the estimated vector to measure the reconstruction performance. For uniformly quantized measurements with dithering, we use the relative  $\ell_2$ -norm distance between the underlying signal and the estimated vector, i.e.,  $\| \hat{\boldsymbol{x}} - \boldsymbol{x}^* \| / \| \boldsymbol{x}^*\|_2$ , to measure the reconstruction performance.
Since this paper is concerned with uniform recovery performance, in each trial, we record the worst-case reconstruction performance (i.e., the smallest cosine similarity or the largest relative error) over the 30 test images, and the worst-case cosine similarity or relative error is averaged over 10 trials.
Figures 2, 3, and 4 show that for noiseless 1-bit measurements and uniformly quantized measurements with dithering with  $\delta = 3$ , the CSGM approach can produce reasonably accurate reconstruction for all the test images when the number of measurements  $m$  is as small as 150 and 100 respectively.
G.3 Experimental Results for ReLU and Dithered 1-bit Measurements
We present the experimental results for the ReLU link function and dithered 1-bit measurements in Figures 5, 6, and 7. For dithered 1-bit measurements, we set  $\lambda = R\sqrt{\log m}$  with  $R > 0$  being a tuning parameter. For the case of using the ReLU link function, similarly to noiseless 1-bit measurements, we calculate the cosine similarity to measure the reconstruction performance. For dithered 1-bit measurements, similarly to uniformly quantized measurements with dithering, we calculate the relative  $\ell_2$ -norm distance. We observe that for these two nonlinear measurement models
Figure 3: Reconstructed images of the MNIST dataset for UQD with  $m = 100$  and  $\delta = 3$ .
(a) 1-bit with varying  $m$
(b) UQD with fixed  $\delta = 3$
Figure 4: Quantitative results of the performance of CSGM for 1-bit and UQD measurements on the MNIST dataset.
(c) UQD with fixed  $m = 200$
with a single realization of the random measurement ensemble, CSGM can also lead to reasonably good reconstruction for all the test images when the number of measurements is small compared to the ambient dimension.
H Experimental Results for the CelebA dataset
In this section, we present numerical results for the CelebA dataset [35], which contains more than 200,000 face images for celebrities with an ambient dimension of  $n = 12288$ . We train a deep convolutional generative adversarial network (DCGAN) following the settings in https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html. The latent dimension of the generator is  $k = 100$  and the number of epochs for training is 20. Since the experiments for CelebA are more time-consuming than those of MNIST, we select 20 images from the test set of CelebA and perform 5 random trials. Other settings are the same as those for the MNIST dataset.
Since we have observed from the numerical results for MNIST that the experiments for the ReLU link function and dithered 1-bit measurements are similar, we only present the results for noiseless 1-bit measurements and uniformly quantized observations with dithering.
From Figures 8 and 10, we observe that for noiseless 1-bit measurements with 1500 samples, a single measurement matrix  $A$  can lead to reasonably accurate reconstruction for all the 20 test images. In addition, from Figures 9 and 10, we observe that for uniformly quantized measurements with
Figure 5: Reconstructed images of the MNIST dataset for the ReLU link function with  $m = 150$  and  $\sigma = 0.2$ .
Figure 6: Examples of reconstructed images of the MNIST dataset for dithered 1-bit measurements with  $m = 250$  and  $R = 5$ .
(a) ReLU with fixed  $\sigma = 0.1$
(c) Dithered 1-bit with fixed  $R = 5$
Figure 7: Quantitative results of the performance of CSGM for the ReLU link function and dithered 1-bit measurements on the MNIST dataset.
(b) ReLU with fixed  $m = 150$
dithering, a single realization of the measurement matrix and random dither is sufficient for the reasonably accurate recovery of the 20 test images when  $m = 1000$  and  $\delta = 20$ .
Figure 8: Reconstructed images of the CelebA dataset for the noiseless 1-bit measurements with  $m = 1500$ .
CSGM Original
Figure 9: Reconstructed images of the CelebA dataset for UQD with  $m = 1000$  and  $\delta = 20$ .
(b) UQD with fixed  $\delta = 20$
Figure 10: Quantitative results of the performance of CSGM for 1-bit and UQD measurements on the CelebA dataset.
(c) UQD with fixed  $m = 500$