Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Motivated by the power of score-based generative models (SGM), this work studies quantized compressed sensing with score-based generative models and proposed an unsupervised data-driven approach that is denoted by QCS-SGM. One key to QCS-SGM is proposing an annealed likelihood score (noise-perturbed likelihood score), which incorporates information from the measurements while performing the posterior sampling. In particular, if $\mathbf{A}\mathbf{A}^T$ is a diagonal matrix, QCS-SGM reduces to a form similar to that of Jalal et al. (2021a), and corresponding guarantees are provided in Theorem 1 and Corollary 1.1. Numerical results are provided to show the superiority of QCS-SGM. 
strength_and_weaknesses: Strength:

As the authors mentioned, their work is the first one that utilizes SGM quantized CS. Such an idea is interesting, and the experimental results are promising. 

Weaknesses:

(a). SGM has been used for linear CS in some recent works, including Jalal et al., 2021a;b and 

Daras, G., Dagan, Y., Dimakis, A. and Daskalakis, C., 2022, June. Score-Guided Intermediate Level Optimization: Fast Langevin Mixing for Inverse Problems. In International Conference on Machine Learning (pp. 4722-4753). PMLR.

I hope the authors can provide more discussions on these closely relevant works and provide intuition on why the QCS-SGM can outperform the methods proposed in these works. Ideally, similar to Jalal et al., 2021a, also perform the experiments for some MRI datasets.

(b). In Theorem 1, the assumption that $\mathbf{A}\mathbf{A}^T$ is diagonal is restrictive. Although this assumption holds approximately for some popular measurement matrices such as i.i.d. Gaussian and discrete cosine transform matrices, it is more desired to provide guarantees for these measurement matrices directly (i.e., for the case that $\mathbf{A}\mathbf{A}^T$ is approximately diagonal ).
clarity,_quality,_novelty_and_reproducibility: This paper is well-written, and the main idea of this work is natural and clearly presented. This work may seem to be incremental compared to Jalal et al., 2021a and Liu & Liu, 2022. 

As for reproducibility, I hope that the code has been submitted along with the main paper, instead of saying that "Our code will be open-sourced after acceptance". 
summary_of_the_review: Although this work seems to be incremental compared to some closely relevant prior works, the main idea is interesting and the experimental results are promising. Therefore, I am inclined to acceptance.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes an approach for signal recovery in quantized compressed sensing (CS) that leverages score-based generative models. This addresses a specific setting for the first time, as previous approaches to SGM-based CS do not assume quantization, and quantized CS approaches with generative models have used other options such as GANs and variational autoencoders. A new score is proposed that leverages knowledge of the measurements when assessing the likelihood of a particular candidate signal.

Sample candidates for the recovered signal are generated using a Angevin dynamics formulation where the probability term becomes conditioned on the measurements observed. A Bayes rule is leveraged to allow for the score training process to be independent of the measurement matrix.

The paper sets up a recovery method that leverages a score-based generative model and derives a closed form solution under an orthogonal-row measurement matrix assumption. This result is specialized to the one-bit and non-quantized cases. 

strength_and_weaknesses: The numerical results use real-world datasets of varying complexity and show significant performance improvements vs. other generative model-based approaches to quantized CS recovery. However, I would argue that the use of Daubechies-1 (i.e., Haar) wavelets for images does not provide a fair comparison, as other wavelet or patch DCT bases are widely known to be better suited for images. Other results show good scaling to multi-bit quantization and out-of-distribution samples. 

The method opens some questions that the authors may want to briefly address, or consider in the future:

It would be interesting to comment on the robustness of these results when the rows of the measurement matrix are only approximately orthogonal, or perhaps validate it numerically.

It would be interesting to consider the effect of noise in the recovery, and how the different methods would perform on a "fixed budget basis" setting - e.g, M one-bit measurements vs. M/B B-bit measurements.
clarity,_quality,_novelty_and_reproducibility: There are a handful of points that can be clarified/corrected:

Theorem 1 is somewhat confusing as phrased. Can one instead start with the assumption of orthogonality for the rows of A, and then make the "if gaussian then closed form solution" statement after?

In Corollary 1.2, it is not clear what a_m corresponds to in (17) - it appears at first look that this equation is meant to be for entry m of the score?

Page 5, typo "expliciitly"
Algorithm 1, suggest changing the reference to eq (25) to be (11-13) so that the reader does not have to delve into the appendix.
summary_of_the_review: An interesting approach to quantized CS recovery with good analytical treatment and compelling experimental results.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors present a novel method for reconstruction from quantized compressive sensing measurements. The proposed method is a modification of score-based models to incorporate information about the forward measurement process and quantization noise. Overall, the method is interesting and generally well-explained. However, the experimental assessment seems somewhat incomplete. As a minor point, the quality of figures is poor and they are often hard to read.
strength_and_weaknesses: Strengths:
- Novel and interesting method to adapt score-based generative models to compressive sensing reconstruction in presence of quantization
- Good performance in the specific settings that are presented and on "toy" datasets like MNIST and CIFAR

Weaknesses:
- Experimental evaluation seems rather limited. Some of the alternative methods shown in the comparisons are not quite mainstream, while important and widely-used baselines for deep-learning CS image reconstruction are missing (for example, ISTA-net Zhang 2017, and Neumann networks Gilton 2019) 
- Authors only report comparisons with other techniques for the 1-bit case but it is unclear how the method is positioned with respect to the state-of-the-art for a wider range of quantization rates
- For the purpose of quality comparisons of image reconstructions, I feel cosine similarity is not very useful. The most used metrics are just PSNR and SSIM. The space saved removing cosine similarity could be better used expanding the set of comparisons. 
- Training SGM models is known to be very computationally expensive. The need for a pretrained one limits the applicability of the proposed method to a wider range of data for which no pretrained SGM is available, or it would be too expensive to train or not enough data are available. The authors need to discuss this limitation of the approach with respect to existing techniques, especially the ones not relying on generative models.
- Related to the previous point, results are presented on toy datasets for which pretrained SGMs are easily available, but it is unclear how the method generalizes to higher resolution images. 
clarity,_quality,_novelty_and_reproducibility: The work is generally clear and the idea is novel. Reproducibility seems good from the textual description, and the authors pledged to release the code.
summary_of_the_review: The authors present a generally novel and interesting idea to use score-based models for CS reconstruction in presence of quantization. However, I feel the idea is not thoroughly validated from the experimental standpoint, making it unclear how it is positioned in the wider literature on CS reconstruction.

-----

Post-rebuttal: the authors significantly improved the original submission. I raised my score accordingly.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The authors introduce a new method for reconstructing images from quantized compressed sensing measurements using score-based generative models; they call their method QCS-SGM. The basic idea is to calculate a quantity called the noise-perturbed pseudo-likelihood score, which can be computed in closed form under certain assumptions. This enables them to implement annealed Langevin Dynamics, leading to a natural reconstruction algorithm. Conceptually, the method extends (and strengthens) previous works in this direction that consider the linear (unquantized) case. Results on a number of standard benchmark datasets confirm the validity of the approach.

Overall, reviews for this paper were positive. While the proposed technique is a combination of well-established ideas, the new pseudo-likelihood calculation is clean and theoretically supports certain algorithmic choices made in a related approach (Jalal et al, 2021). Moreover, the setting of (highly) quantized observations is practically compelling, and the authors do a good job of validating the proposed method on benchmarks.  Concerns were raised about novelty compared to these earlier papers, but I feel that on the whole the paper merits acceptance.

One comment (not raised by any reviewers): the paper does not emphasize this point too much, but it may be helpful to illuminate the role of pre-quantization noise, and how that interacts with the reconstruction procedure. From earlier results in 1-bit CS, it is known that some amount of noise may be necessary for recovery since it provides a "dithering"/regularization effect; however, too much of this may affect the pseudo-likelihood estimates. Consider exploring this tradeoff while preparing future revisions.
justification_for_why_not_higher_score: While the paper is generally of high quality and the implementation of the ideas is well done, the reconstruction techniques themselves are a tweak of well-known score-baed generative modeling (SGM).
justification_for_why_not_lower_score: There is consensus that the paper makes interesting contributions, and as such merits acceptance.
Decision: Accept: poster