Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper appears to beat the SOTA on inverse halftoning  with an approach that uses diffusion models. The details are not very transparent.
strengths_and_weaknesses: The artifact reduction in inverse results is significant. In addition, the generated halftone images have a nice quality to them, so that might be useful, too. But the description of the details is difficult to read. Not enough is illustrates with images and the math is a it too abstract. When I read the title I assumed that the noising process in a diffusion model would reflect the dithering (going from the original image to a noise one by adjusting intensities until they are binary), but the method instead is using a traditional diffusion models to capture image distribution and a series of adjustments happens with meta-halftones (a new idea), which are not well illustrated. Nevertheless, the results look very good.
questions: See above
limitations: No comment
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a new method for inverse halftoning (recovering realistic images from ancient prints). The proposed method is a generative method that uses reverse diffusion process to produce desired halftone dithering patterns. A meta-halftone guided network is proposed to use the blue noise to guide the halftone diffusion process for handling redundant patterns and artifacts. The paper also proposes to condition the inverse halftoning diffusion on the initial state of halftone dithering diffusion, in order to learn a more robust mapping. Experiments demonstrate its state-of-the-art performance.
strengths_and_weaknesses: Strengths.

1. A generative halftoning method is proposed.
2. A meta-halftone guided network is proposed to reduce redundant patterns and undesired artifacts.
3. A learning dataset is constructed.
4. The motivation of this work is clear.
5. Code is provided.

Weaknesses.
1. The split of the proposed dataset (7857 for training, 400 for validation and 400 for test) seems not reasonable.
2. The paper compres to only four methods chosen/adapted from previous halftoning/super-resolution methods, which is not convincing. For example, diffusion models that are proposed for image denoising can be used for comparisons.     
questions: Please see above section.
limitations: Yes.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: In this paper, the authors propose a generative halftoning method, which regards the black pixels in halftones as physically moving
particles, and makes the randomly distributed particles move under some certain guidance through the reverse diffusion process. The proposed Conditional Diffusion model for image  Halftoning (CDH)  consists of a halftone dithering process and an inverse halftoning process. Specifically, to avoid introducing redundant patterns and undesired artifacts during halftone generation, they propose
a meta-halftone guided network to incorporate the blue noise guidance into the diffusion process and train an inverse halftoning diffusion model to learn the mapping function from the halftone distribution to the continuous-tone distribution. They claim that this helps the model to learn a  more robust mapping to perform inverse halftoning and improve the generalization ability to unseen samples.

strengths_and_weaknesses: Strength: This is the first attempt to use diffusion models for the half-tone generation and inverse half-toning.

Weakness:  The writing is quite confusing. The reviewer has tried to understand the link between the proposed method and the score-based diffusion model, but could not find any close link.  Why does the complicated forward diffusion process be necessary? In figure 2, why there are two paths for half-tone dithering and inverse half-toning? If I understand correctly,  a better way of inverse halftoning using diffusion would be a noising process of the dithered image followed by the reverse diffusion process to obtain the continuous image. But the main focus of this paper appears to be the dither pattern generation, which appears quite far distance from the diffusion model. I have to say that I am not an expert on half-toning, but still, the way of describing the algorithm is quite confusing to general NeuRIPS readers.
questions: The authors claimed that the limitation of the existing deep learning approaches for inverse halftoning is the lack of generalization due to the use of U-Net.  However, to my understanding, the particular architecture of the U-Net is not the main culprit, but the way of supervised training is the problem. The authors should clarify this.
limitations: The slow speed of the diffusion model is stated as the limitation.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper introduces an inverse halftoning method based on conditional diffusion processes. The goal is to restore halftoned images generated using any halftoning algorithm, by leveraging recent advances on generative models. Previous work did not generalize properly to halftoning algorithms outside of those used for training the model, while the proposed method performs the inverse halftoning operation through conditional diffusion models. To improve the capabilities of the model, the paper ontroduces a meta-halftone guiding framework for introducing blue noise into the diffusion process. Quantitative results are provided that show improved results with respect to supervised learning and image filtering frameworks. 
strengths_and_weaknesses: *Summary* 

This paper provides valuable ideas for designing diffusion models for inverse halftoning. The paper is in general well written, but lacks sufficient validation, appropriate discussion of previous work and the significance of the findings of this paper are limited in the overall literature of diffusion models. 


Strenghts:
- This paper provides specific ideas on how to leverage diffusion models for the particular problem of inverse halftoning.
- The paper is, in general, well written and is very easy to follow. The authors present their ideas in a concise manner.
- Code is provided for enhanced reproducibility. 
- Provided analysis show significant improvements with respect to previous work.


Weaknesses:
- The paper lacks sufficient validation, it is not clear why some decisions were taken. Please see "questions" for details. 
- The design of the Meta-halftone guided network is not adequately justified. Quantitatevely, it is shown that this module enhances the result of the model, but it is not clear to the reader why this design was chosen. 
- Previous work is not adequately studied. I suggest the authors to consider analyzing Palette: Image-to-Image Diffusion Models, Saharia et al. and Deep Image Prior, Ulyanov et al. 
- The analysis of previous work is also limited. Following the authors wording, it is claimed that "U-net architecture has poor generalization in practical applications". Reading this, it may be understood that this paper provides alternatives to the U-net architecture, but it does not. I suggest the authors to change the wording to "supervised learning" or "pixel-wise regression" instead of "mainstream U-net architecture", as this represents more accurately the baselines that the authors want to improve. 
- Limitations need further analysis.


questions: - What is the impact of the design of the meta-halftone module? Why is this design chosen? Could you provide a baseline for this design and some ablations on why this design is better than other methods? I am concerned about the use of a pretraiend VGG network for this purpose, as the features on its last layer may be too closely related to ImageNet classification. Why is pre-training needed? 
- What does it mean that the "mainstream U-net architecture has poor generalization ability in practical applications"? Is it a problem of the U-net architecture by itself? If so, what does this paper do to mitigate these problems? Looking at the code, it seems that the diffusion model is a U-net with attention, similar to those used in Palette or other DDPM models. Can the authors provide more details on this? Is it really an architectural problem, or are more factors involved (learning formulation, loss function, etc.)? 
- On the validation side, I think PSNR and SSIM are not sufficient for analyzing the results of generative models. What are the scores using perceptual metrics (FiD, Inception Score, LPIPS)? 
- On Table 1, why are standard deviations only provided for CDH? Why is the standard deviation for SSIM so low?
- What happens with RGB images? Does this method only work for grayscale images?
- On Figure 1 and Figure 5, what is the input to the model? It would be interesting to see the halftoned inputs used throughout the paper. 
- Does the model work better with some halftoning algorithms than others? A quantitative analysis on this would be benefitial.
- Could the authors compare this paper to Palette: Image-to-Image Diffusion Models, Saharia et al. and Deep Image Prior, Ulyanov et al. ?  I think the image restoration ideas on these papers are relevant for this CDH paper and should be at least mentioned. 
- What happens when the images are larger to the 256x256 pixels used in training?
- Can the authors provide an illustration on the "blue noise" that can be introduced by the proposed metahalftone module? 
limitations: The authors have adressed some limitations of the proposed method, particularly issues of computational cost at inference time with respect to other work. They also point out that their work may be used to "generate fake images for some certain purpose", and that the dataset used for training "may contain some bias". 

I suggest the authors extend this analysis. In particular, I think it would be interesting to give particular examples of how this work may be used for unethical purposes, or which harmful biases are introduced with the dataset. Also, I suggest the authors to extend the limitation analysis, in terms of: 
- What happens with RGB images?
- What happens with resolutions outside of the specified 256x256 pixels?
review_title: null

Meta-review: metareview: Most reviewers are positive about the paper. After rebuttal, a number of issues were addressed, and the additional evaluations provided in the authors' response should be published in the paper or supplemental. The clarity of the technical exposition and details can still be improved, but reviewers found the results better than SOTA and the ideas worth publishing
confidence: Certain
Decision: Accept