Conditional Diffusion Process for Inverse Halftoning
Authors: Hao Jiang
Yadong MU
Venue: NeurIPS 2022 Accept (2022)

Abstract: Inverse halftoning is a technique used to recover realistic images from ancient prints (\textit{e.g.}, photographs, newspapers, books). The rise of deep learning has led to the gradual incorporation of neural network designs into inverse halftoning methods. Most of existing inverse halftoning approaches adopt the U-net architecture, which uses an encoder to encode halftone prints, followed by a decoder for image reconstruction. However, the mainstream supervised learning paradigm with element-wise regression commonly adopted in U-net based methods has poor generalization ability in practical applications. Specifically, when there is a large gap between the dithering patterns of the training and test halftones, the reconstructed continuous-tone images have obvious artifacts. This is an important issue in practical applications, since the algorithms for generating halftones are ever-evolving. Even for the same algorithm, different parameter choices will result in different halftone dithering patterns. In this paper, we propose the first generative halftoning method in the literature, which regards the black pixels in halftones as physically moving particles, and makes the randomly distributed particles move under some certain guidance through reverse diffusion process, so as to obtain desired halftone patterns. In particular, we propose a Conditional Diffusion model for image Halftoning (CDH), which consists of a halftone dithering process and an inverse halftoning process. By changing the initial state of the diffusion model, our method can generate visually plausible halftones with different dithering patterns under the condition of image gray level and Laplacian prior. To avoid introducing redundant patterns and undesired artifacts, we propose a meta-halftone guided network to incorporate blue noise guidance in the diffusion process. In this way, halftone images subject to more diverse distributions are fed into the inverse halftoning model, which helps the model to learn a more robust mapping from halftone distributions to continuous-tone distributions, thereby improving the generalization ability to unseen samples. Quantitative and qualitative experimental results demonstrate that the proposed method achieves state-of-the-art results.

Conditional Diffusion Process for Inverse Halftoning
Hao Jiang Peking University j jianghaodstu. pku. edu.
Yadong Mu" Peking University Peng Cheng Laboratory myd@pku edu. cn
Abstract
Inverse halftoning is a technique used to recover realistic images from ancient prints (e.g., photographs, newspapers, books). The rise of deep learning has led to the gradual incorporation of neural network designs into inverse halftoning methods. Most of existing inverse halftoning approaches adopt the U-net archi- tecture, which uses encoder to encode halftone prints, followed by a decoder for image reconstruction. However, the mainstream supervised learning paradign with element- wise regression commonly adopted in U-net based methods has poor generalization ability practical applications. Specifically, when there is a large gap between the dithering patterns of the training and testing halftones, the constructed continuous-tone images have obvious artifacts. This is an important issue in practical applications, since the algorithms for generating halftones ever-evolving. Even for the same algorithm, different parameter choices will re- sult in different halftone dithering patterns. In this paper, we propose the first generative halftoning method in the literature, which regards the black pixels halftones physically moving particles, and makes the randomly distributed parti- cles move under some certain guidance through reverse diffusion process, so as to obtain desired halftone patterns. In particular, we propose Conditional Diffusion model for image Halftoning (CDH), which consists of a halftone dithering process and inverse halftoning process. By changing the initial state of the diffusion model, our method can generate visually plausible halftones with different dither- ing patterns under the condition of image gray level and Laplacian prior. To avoid introducing redundant patterns and undesired artifacts, we propose meta-halftone guided network to incorporate blue noise guidance in the diffusion process. In this way. halftone images subject to more diverse distributions are fed into the inverse halftoning model, which helps the model learn more robust mapping from halftone distributions to continuous-tone distributions, thereby improving the generalization ability to unseen samples. Quantitative and qualitative experimental results demonstrate that the proposed method achieves state-of-the-art results.
Introduction
Halftoning refers to the task of simulating the brightness change of a continuous-tone image by changing the size frequency of halftone dots (such ink dots). In the last century, halftoning technology has been widely used in the printing industry to store precious image data through old newspapers, books, photographs, etc. At the same time, inverse halftoning technology emerged to recover stored continuous-tome image from vintage materials. The goal of the inverse halftoning technique is to minimize the loss of information in the restoration process, that the restored print has the highest possible visual quality.
Corresponding author.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Figure 1: Illustration of the presence of artifacts in restored images by previous inverse halftoning method (Xia and Wong, 2018), where the gap exists between the dithering patterns of training and testing halftones. The top row shows inverse halftoning results for some testing samples, and the bottom row demonstrates corresponding ground-truth continuous-tom images.
Traditional inverse halftoning methods are mainly based on image filtering (e.g., wavelet domain filtering (Xiong al., 1999), edge-preserving filtering (Kite al., 2000), SUSAN filtering (Siddiqui and Bouman, 2007)), bilateral filtering (Sun et al., 2014) and statistical learning methods (e.g., least mean square filtering (Chen and Hang, 1997), maximum posteriori (Stevenson, 1997), look-up table (Mese and Vaidyanathan, 2001; Chung and Wu. 2005), dictionary learning (Zhang et al., 2018)). With the revival of deep learning technology. inverse halftoning methods based on deep neural networks have made great progress and attracted more and more attention (Hou and Qiu, 2017; Xiao al., 2017; Kim and Park, 2018). The most representative method the U-net (Ronneberger et al., 2015) based architecture, which has an encoder to learn the hidden encoding of the halftone printing, followed by decoder to reconstruct the image (Xia and Wong. 2018; Gao al., 2019).
However, the paradigm of supervised learning with element- regression commonly adopted in the U-net based methods suffers from poor generalization in practical applications. Specifically, when there is a large gap between the dithering patterns of training and testing halftones, the restored continuous-tome images often have obvious artifacts. This an important issue in applications. since the algorithms used for generating halftones evolve with time. Even for the same algorithm. different parameter choices will result different halftone dithering patterns. Taking frequency modulation (FM) halftoning as an example, we select classical error diffusion processes with different dithering patterns, namely Floyd-Steinberg Dithering,
JussJJuddi--inkk Dithering, Stucki Dithering, Atkinson Dithering, Burkes Dithering, Sierra Dithering, and several of their variants (Lau and Arce, 2018). We train the U-net model on the halftones generated by 5 of the algorithms, and test on the halftones obtained with the rest 4 algorithms. The experimental results are shown in Figure 1. Artifacts can be clearly observed in the restored continuous-tone images.
To address this problem, in this paper, we propose a Conditional Diffusion model for image Halftoning (CDH), as shown in Figure which consists of halftone dithering process and inverse halftoning process. We regard the black pixels in halftones as physically moving particles, and make the randomly distributed particles move under some certain guidance through the reverse diffusion process, so to obtain the desired halftone distribution. Specifically, for the halftone dithering process, we train conditional diffusion model to generate halftones with different dithering patterns under the condition of image gray level and Laplacian prior. By changing the initial state of the diffusion model, it can simulate different dithering processes to generate diverse halftone images. To avoid introducing redundant patterns and undesired artifacts during halftone generation, we propose a meta-halftone guided network to incorporate the blue noise guidance into the diffusion process. For inverse halftoning, train inverse halftoning diffusion model to learn the mapping function from

---

the halftone distribution to the continuous- distribution. In this way, halftones subject to more diverse distributions are input to the inverse halftonnng model, which helps the model learn more robust mapping and improve the generalization ability to unseen samples.
Our contributions are summarized follows:
This is the first work to propose a generative halftonning method, which regards the black pixels in halftones as physically moving particles, and makes the randomly distributed particles move under some certain guidance through the reverse diffusion process, so as to obtain the desired halftone dithering patterns.
To avoid introducing redundant patterns and undesired artifacts during halftone generation, we propose a meta-halftone guided network to incorporate the blue noise guidance into the halftone diffusion process.
To obtain better generalization ability, we use the Zo state of halftone dithering diffusion as the condition for inverse halftoning diffusion, so that the inverse halftoning diffusion model benefits from a wider range of dithering patterns and learns a more robust mapping.
We conduct experiments on the dataset consisting of halftoning algorithms, and quantitative and qualitative experiments demonstrate that the proposed method achieves state-of-the-aa results.
2
Related Work
Inverse Halftoning. Traditional inverse halftoning methods are mainly based on image filtering (e.g.. pige-ppreeervng filtering (Kite et al., 2000), wavelet domain filtering (Xiong et al., 1999), bilateral filtering (Sun et al., 2014), SUSAN filtering (Siddiqui and Bouman, 2007)) and statistical learning methods (e.g., look-up table (Chung and Wu, 2005; Mese and Vaidyanathan, 2001), least mean square filtering (Chen and Hang, 1997), maximum posteriori (Stevenson, 1997), dictionary learning (Zhang et al., 2018)). For example. Sun et al. (Sun et al., 2014) proposed to use an anisotropic Gaussian filter and edge-preserving filter for inverse halftoning. With the revival of deep learning technology, some researchers have tried to use deep neural networks to accomplish inverse halftoning, e.g., U-net based models (Hou and Qiu, 2017; Xiao et al., 2017: Gao al., 2019), residual learning based models (Xia and Wong, 2018) and contextual learning based models (Kim and Park, 2018). For example, Xia al. (Xia and Wong, 2018) proposed progressively residual based U-net that synthesizes the global tone and subtle details to generate inverse halftones. However, these approaches suffer from unacceptable artifacts the restored continuous-tone images when faced with dithering pattern gaps.
Diffusion models are class of deep generative models developed from non- equilibrium thermodynamics (Sohl-Dickstein et al., 2015). They define a Markovian process for the diffusion steps, incrementally adding random noises the original data, and then learn the reverse diffusion process to resample the data from noises (Sohl-Dicksted) et al., 2015; Ho et al., 2020; Luo and Hu, 2021). Diffusion models are closely related score-based generative models, which generate samples by Langevin dynamics based on estimated gradients of the data distribution (Song and Ermon, 2019, 2020; Song et al., 2020b, 2021). Some improved techniques are proposed to help the diffusion model converge to lower negative log-likelihood or to speed up sampling (Nichol and Dhariwal, 2021: Dhariwal and Nichol, 2021; Song et al., 2020a). For example, Song et al. (Song et al., 2020a) generalizes denoising diffusion probabilistic models via a class of non-Markovian processes, which can correspond to deterministic generative processes and gives rise to implicit models that produce high quality samples much faster. However, these diffusion models cannot be directly applied to halftone generation, since they do not take into account the blue noise properties of halftone prints, making it difficult to generate visually pleasing dithering patterns.
Diffusion Models.
3 Method
3.1 Halftone Dithering Diffusion Conditioned on Gray-scale and Laplacian Prior
Traditional halftone dithering methods are mainly based on techniques of amplitude modulation (Blat- ner and Roth, 1993; Campbell et al., 1966) and frequency modulation (Eschbach, 1997; Floyd, 1976; Meşe and Vaidyanathan, 1999). With the development of deep learning, some modern approaches try
3

---

Halftone Dithering Diffusion
Meta halftone Guidance
Po1
x
X
x $1
0
qut
(x--1)
P02
22
(x |x,2 h)
X
X
x
$2 x
|x|-1]
Inverse Halftoning Diffusion
Figure 2: Illustration of the proposed Conditional Diffusion model for image Halftoning (CDH), which consists of a halftone dithering diffusion process and an inverse halftoning diffusion process.
to generate halftone images using convolutional neural networks (Kim and Park, 2018: Xia al., 2021). However, these methods are limited to specific dithering patterns and cannot achieve flexible halftone generation. To address this problem, considering that the halftone prints are only composed of black and white pixels, we regard the black pixels in halftones as moving particles, and make these particles move from random Gaussian distributions halftone distributions under reverse diffusion processes, so to achieve a generative halftoning approach.
Given the distribution ge (x=)) ) of halftone prints, the diffusion process get is Markovian noising process (Ho et al., 2020) that gradually adds noise to æ to obtain "ir where denotes the halftone dithering diffusion stage. Specifically, each step t. the diffusion step adds the random Gaussian noise with a f--oontroleed variance:
II 1=1
nth (a
(1)
(x ( = Naxx'; /1- 3
(2)
where B, E (0,1), T. With the reparameterization trick (Kingma and Welling, 2013), we can sample n. from any time step / closed form: x²" varag + vT 046, A/ I), that is
geh (2 (|
= Na Vara -1-10 (1-)))
where a =1 =1 3r, ā = I[ ag In this way, we can directly derive x; by q"((x)" (x)) without repeatedly applying the Markovian process q²1 and calculating g²1l (x=²² (x"11).
In the halftone dithering scenario, we would like to obtain the halftone sample x via the reverse diffusion process g²d (x=",læ;") from random Gaussian distribution, i.e., the state x ~N(0 I). Considering that the particles tend to rely on the gradient information of images when forming halftone patterns, we use the image gray level and the Laplacian prior i conditions to guide the reverse diffusion process. Formally, the reverse halftoning diffusion process is defined on the distribution Po; (r 1), which is Markov chain with start state p" (x [xx I,
po (r 1 (x
I
(a 1a r.
(4)
16 (0 1, = (m 11) (ar rr,, Eg (a r,,,t)
(5)
Different from previous methods like using class labels (Dhariwal and Nichol, 2021) or shape latents (Luo and Hu, 2021) as conditions for the diffusion model, we use the pixcl-wise image priors

---

Mela-halftone Guided
Feature Extractor
Hidden Feature Affine Leaming
A Affine Factors
Xt
X
t
Output
Meta- halltonee Set
Depth- wise Aggregation
Spatial-wis wise Aggregation
Figure 3: Illustration of the proposed Meta-halftone Guided Network.
as conditions. The conditions are performed in a simple way, i.e., we concatenate the state vector x;"" at diffusion step with and Z along the channel dimension to provide pixel- mixel-wise guidance:
L + 1,
(6)
where @ denotes the concatenate operation. Due to the properties of halftones, local dithering patterns are more important for generating high-gn high-quality halftones than global semantic information. We also observe in experiments that excessive introduction of global information can be harmful to dithering results.
We would like to learn the model po to approximate the conditional probabilities g (2 |x,', az))
With Bayes' theorem, it has
(a |x, a = =q= (x;"""" x6¹ qer (xx"" 2x
and q²1 (x.",|>""
x can be represented as a Gaussian distribution:
q²t (x |z,," xx)
(2 (xx x 3,I),
(8)
/ / 13 xx +
Va,
1 ā 32.
1. (m 2
Õz. The simplified objective function (Ho al., 2020) is used to train the model during the halftone dithering diffusion process:
5t
L ET- un (m (o."). |Ike c0) (Vāna + + -xc,t) I..
(10)
3.2 Meta-halftone Guided Network
Blue noise properties are critical for generating high-quality halftones. It avoids noticeable low- frequency visual artifacts in the resulting halftones by forcing random pixel dithering (Lau and Arce, 2018). Some traditional methods have been proposed derive blue noise dithering patterns, such as simulated annealing based methods (Sullivan et al., 1991), void-and-cluster techniques (Ulichney, 1993), power spectrum manipulation algorithms (Yao and Parker, 1994), and dither pattern ordering methods (Lau et al., 1999). However, most of these approaches are based on statistical methods or manually designed blue-noise dithering matrices, which are difficult to directly apply to learning- based neural frameworks. Recently, some researchers have tried to inject blue noise properties into the I.I norm for learning objectives (Xia et al., 2021), however, since calculation the I.1 norm requires known halftone generation results, it cannot be used in generative models. So far, how to
5

---

incorporate the blue-noise dithering properties into halftone generative models remains an unexplored problem.
To address this issue, we propose a meta-halftone guided network (as shown in Figure 3), which introduces blue noise guidance into halftone dithering diffusion process and avoids dithering results containing artifacts or redundant patterns. Formally, for step t of the halftone dithering diffusion process, we consider the state vector æ; and take input the meta-halftone guided network. We first feed x;" into a feature extraction network E to obtain the extracted hidden feature f(x;) ) E(x;") ). We use the pre- pre-trained VGG network (Simonyan and Zisserman, 2015) as E in the experiment, other networks such as InceptionNet (Szegedy et al., 2016) or ResNet (He et al., 2016) are also available. The purpose of using pre- training to save the consumption of computing resources and training time. Next, we define the meta-halftone set M, which consists of set of meta-halftone vectors m S i S k): M m1, m2, ma, mz),
i... f( f(xx")
(11)
where & represents the number of meta halftones. We obtain m; by the halftone dithering diffusion states of a of low-frequency images. Specifically, we construct & images I1, I2. I3..., I& with large range of low-frequency regions (e.g., images with constant gray levels), and obtain the halftones H1, H2, H.,,............... H2 accordingly using conventional halftone algorithms (e.g., Floyd-Steinberg Algo- rithm (Lau and Arce, 2018)). We first train for a certain step using the traditional diffusion model alone, where the model uses U-net architecture common in (Ho al., 2020; Dhariwal and Nichol, 2021) without adding the proposed meta halftone guided network. We have
Poj ( ( H. H N (x p (x, H. H.. HH. t), 2 (a) H. H. H.. t)),
(12)
where the superseript H; denotes the constructed halftone sample from H, E (H,, H2. ..., H₂ and H. and H. can be derived from I {I,, I2,,.,, Lt} accordingly. The superscript s1 of F.. x #1 is omitted here for brevity. We can get 60] (Vorat + VIV 0,6, 1) from model predictions and calculate :
H:
Mo (at
H. ()
1 H.
(at
B.
H. 8. (a)
,)).
(13)
Var
vI
We use m, in Equation 11 to represent
$1 and construct meta- maett-halftooee set M accordingly:
m: Mè, (Vā H. + vI Sig
(14)
Next, the model learns affine relationships between the hidden feature f(x ) and the meta-halftone set M through an affine learning layer, and then obtains affine factors g (a);;)
g Wg-i f fax²') + by-i.
(15)
where Wg-i and b represent learnable weights and biases, respectively. With the calculated g( we perform a depth-wise aggregation of the meta-halftone set M to learn the refined representations depth channels:
m
m: g(
(16)
Besides depth information, spatial information is also crucial for meta-halftone guidance. Meta- halftones can provide guidance for the generation of new dithering patterns in local areas. In light of this, next perform spatial-wist wise aggregation of the meta- meta-halftone refined representation m and the dithering diffusion state vector x; ; and denote the result as m':
Σ d (m @ x:+))
(17)
where + represents the element- wise concatenation in the spatial dimension, denotes the spatial convolution operation, and d represents the convolution kernel. The output o of the meta-halftone guided network is determined by the meta-halftone guidance mn and dithering diffusion state vector a;
x
(18)
and o is subsequently fed into U-net as in previous work (Dhariwal and Nichol, 2021; Nichol and Dhariwal, 2021) for model predictions.

---

3.3 Inverse Halftoning Diffusion Conditioned on 2.
The goal of inverse halftoning process is to learn a mapping from halftonn distributions to continuous- tone distributions and reduce the loss of information in the process. It is similar to halftone dithering diffusion process in Section 3.1, but with different diffusion conditions. For the inverse halftoning diffusion process, we deal with the process naa (m""" (=g") that gradually adds noise to the data 2 from a continuous-tone distribution (=x")) and the process (x= [mx²) of gradually denoising from Gaussian noise to obtain desired samples.
We use a model 22 (xf" h)) estimate ("< x0 which is conditioned on the halftone distribution h (i.e., sampling aq from halftone dithering diffusion model):
2 II
Po, (o Ih) = (m
Po (2:2 12 h),
(19)
(2 h) Ma p (a h,t), "3 h,,",,)t))
(20)
Pixel-wise guidance between the state at" and the halftone condition h is applied in the inverse halftoning diffusion process:
x
h.
(21)
where @ denotes the concatenate operation. The rest parts (such as the training objective) are similar to the halftone dithering diffusion process, and we omit them here to save space.
4 Experiments
4.1 Experimental Setup
Datasets. In order to evaluate the generalization ability of the proposed model CDH to different halftoning methods, we construct dataset with relatively strong domain shifts. The domain shifts mainly arise from two aspects: the halftone algorithm and the image semantic. We construct the training set and validation set based on the UTKFace dataset (Zhang et al., 2017) and the test set based on the VOC2012 dataset (Everingham et al., 2010). We collect 9 different halftone dithering algorithms, namely Atkinson Dithering, Burkes Dithering, Sierra Dithering, and several of their variants (Lau and Arce, 2018). We use halftones generated by 5 of the algorithms in training and validation sets, and halftones generated by remaining 4 algorithms are used test sets. There are 7, 857 images in the training set, 400 images in the validation set, and 400 images in the test set.
Dithering,
Dithering. Stucki Dithering,
We choose the baselines that are commonly used in inverse halftoning tasks, which include the U-net based methods, generative adversarial network based methods, and super-resolution based methods: PRL (Xia and Wong, 2018) proposes inverse halftoning network with progressively residual learning, which synthesizes the global tone and subtle details from halftone images in a progressive manner. Dhariwal et al. (Dhariwal and Nichol, 2021) show that diffusion models can achieve superior image sampling quality than existing generative models, and they use an improved U-net model architecture to achieve high-quality image synthesis. Nichol et al. (Nichol and Dhariwal, 2021) show that with some simple modifications, denoising diffusion probabilistic models can also achieve competitive log-likelihood while maintaining high sample quality. They learn variances of reverse diffusion process using reparameterizations and a hybrid learning objective that combines the variational lower-bound with the simplified objective from (Ho et al., 2020). In additon, they improve the noise schedule from a linear noise schedule to a cosine noise schedule. Song et al. (Song al., 2020a) introduces denoising diffusion implicit models, giving rise to implicit models that produce high quality samples much faster. ESRGAN (Wang et al., 2018) is used as baselines in previous inverse halftoning work, which introduces the residual-in--reiidual dense block generate realistic textures and avoid unpleasant artifacts. GLEAN (Chan et al., 2021) is applied to the super- resolution task, using pre-trained generative adversarial networks to solve ill-posed problems in image restoration. We first smooth the halftone images with Gaussian kernels of different sizes, and then use GLEAN for image restoration. RealESRGAN (Wang et al., 2021) is extended to a practical image restoration application, which introduces high-order degradation modeling process to better simulate complex real-world degradations and employs U-net discriminator with spectral normalization to increase discriminator cappriiil
Baselines.
7

---

Table 1: Model performance comparison of the proposed method CDH and baseline methods in terms
of PSNR and SSIM.
Method
Variants
PSNR
SSIM
PRL (Xia and Wong, 2018)
22.82
0.698
Dhariwal et al. (Dhariwal and Nichol, 2021)
23.35
0.693
Nichol et al. (Nichol and Dhariwal, 2021)
Cosine Noise Schedule
22.40
0.683
Nichol et al. (Nichol and Dhariwal, 2021)
Learn Sigma
22.45
0.702
Nichol et al. (Nichol and Dhariwal, 2021)
Importance Sampled VLB
19.99
0.615
Song el al. (Song et al., 2020a)
DDIM
18.20
0,571
ESRGAN (Wang et al., 2018)
20.20
0.428
ESRGAN (Wang et al., 2018)
Ikernel
22.47
0.645
GLEAN (Chan et al., 2021)
20.11
0,491
GLEAN (Chan et al., 2021)
Ikernel
21.74
0,607
RealESRGAN (Wang et al., 2021)
21.04
0.626
RealESRGAN (Wang et al., 2021)
Ikernel
20.94
0.619
CDH (Ours)
24.24
0.727
Evaluation Metrics. Following previous inverse halftoning work (Xia and Wong, 2018), we use two metrics to evaluate the performance of different models: Peak Signal to Noise Ratio (PSNR) and Structural Similarity (SSIM). PSNR expresses the ratio between the power of peak signals and the power of noises, and SSIM evaluates the similarity between images in terms of luminance, contrast, and structure.
Implementation Details. The image size for halftone dithering and inverse halftoning is 256 256. The channel number of input halftones is 1. For halftone dithering diffusion model, the learning rate set to 0.0001, and set 20. We use 200 diffusion steps in training and testing phases. The number of model channels is 64 and the linear noise schedule is adopted throughout diffusion. We adopt A (Loshchilov and Hutter, 2018) optimizer train the halftone dithering diffusion model. For inverse halftoning diffusion model, we set the learning rate to 0.0001 and use 800 steps the diffusion process. We set the number of model attention heads Adam W (Loshchilov and Hutter, 2018) is also used for optimization.
4.2 Experimental Results
We evaluate the performance of the proposed method CDH and baseline methods in terms of PSNR and SSIM, and the experimental results are shown in Table 1. We evaluate the effect of different Gaussian kernel sizes on baseline performance, where ESRGAN (Ikernel), GLEAN (Ikernel), RealESRGAN (Ikernel) denote the use of Gaussian kernels of size 7, and ESRGAN, GLEAN, RealESRGAN represent the use of Gaussian kernels of size 3, respectively. The experimental results show that our method achieves the best performance on both metrics. Previous methods such as PRL (Xia and Wong, 2018), due to the lack of considering the generalization ability of the model on different halftoning algorithms, show sub-optimal performance on test data with relatively strong domain shifts. Compared with traditional diffusion models (Dhariwal and Nichol, 2021; Nichol and Dhariwal, 2021: Song al., 2020a), the proposed method CDH achieves better results by taking into account the blue noise characteristics in the halftone dithering process, which enables the model to learn from a more realistic and diverse halftone distribution. Different Gaussian kernel sizes affect the performance of image restoration baselines, such as ( (Wang et al., 2018; Chan et al., 2021; Wang et al., 2021). A larger Gaussian kernel crases dot dithering patterns in halftones to a greater extent, resulting in restored images with less halftonn artifacts, but also loss of high frequency information. Our method performs better than the image restoration basclines with different Gaussian kernel sizes. This shows that simply using traditional image restoration approaches is not suitable for the inverse halftoning task, since they do not take into account diverse pixel dithering patterns unique to halftone images.
To verify the effectiveness of the proposed meta-halftone guided network, we conduct experiments to observe the effect on generated halftone dithering qualities with and without the meta- meta-halftone guided network. Experimental results are demonstrated in Figure From the results shown in Figure 4, it can be observed that when the meta hala--alf guided network is removed, the generated
8

---

wo baltoone w/ heltonneltone Guided Network Guided Network
Groundruth Dithering
w/ hala-aee w/ Meta helftone
Groundruuth Dithering
Guided Network Guided Network
Figure 4: Experimental results of the meta-halftone guided network. We show the generated halftone dithering results w/ and w/o the proposed meta-halftone guided network, as well as the halftone dithering groundtruth.
Table 2:
Model performance comparison on halftones generated by different halftoning algorithms in
terms of PSNR and SSIM.
Halftoning Algorithms
PSNR
SSIM
Floyd-Steinberg Dithering
24.46
0.735
Simple Floyd-Steinberg Dithering
24.01
0.692
Jarvis-Judice-Nime Dithering
24.42
0.749
Stucki Dithering
24.53
0.749
Atkinson Dithering
23.08
0.710
Burkes Dithering
24.69
0.746
Sierra Dithering
24.49
0.750
Sierra Lite Dithering
24.40
0.733
Two Sierra Dithering
24.54
0.741
halftones have more obvious dithering artifacts, resulting poorer visual quality. When using the meta-halftone guided network, the visual quality of generated halftones is improved and more similar to the groundtruth. The experimental results demonstrate the effectiveness the proposed meta-halftone guided network.
4.3 Study the Effect of Different Halftoning Algorithms
In order to explore the performance of CDH on different halftoning algorithms, we conduct ex- periments on halftones generated by halftoning algorithms, namely Floyd-Steinberg Dithering. Jarvis-Judice-Ninks Dithering, Stucki Dithering, Atkinson Dithering, Burkes Dithering, Sierra Dither- ing, and several of their variants (Lau and Arce, 2018). The experimental results are shown in Table 2. From the experimental results we observe that the proposed method achieves similar results on different halftoning algorithms, which also verifies good generalization abilities of our method to different halftoning algorithms.
4.4 Qualitative Results
To further verify the effectiveness of the proposed method, we also conduct qualitative experiments to compare the performance of our methods and baselines, which are illustrated in Figure 5. From the qualitative experimental results, it can observed that continuous-tone images generated by baseline methods still have some redundant artifacts, which are dot dithering patterns that are not completely removed during the inverse halftoning process. contrast, our model removes the artifacts well without introducing redundant patterns, validating the effectiveness of the proposed method.

---

PRL
Ours
Groundtruth
PRL
Ours
Groundiruth
Figure 5: Illustration of the qualitative experimental results. We show the inverse halftoning results generated by baseline methods and our method, as well as the groundtruth.
5
Conclusion
In this work, we propose Conditional Diffusion model for image Halftoning (CDH), which con- sists halftone dithering process and an inverse halftoning process. We introduce a generative halftoning method, which regards the black pixels in halftones physically moving particles, and makes the randomly distributed particles move under some certain guidance through the reverse diffusion process,
to obtain the desired halftone dithering patterns. A meta- veta-halftone guided network is introduced to avoid redundant patterns and undesired artifacts in generated halftones. By adopting more diverse halftones from halftone dithering diffusion process, further improve the generalization ability inverse halftoning model. We conduct experiments on the dataset consisting of halftoning algorithms, and quantitative and qualitative experiments demonstrate the effectiveness of the proposed method.
Ackmowledgement: The research is supported by Science and Technology Innovation 2030 New Generation Artificial Intelligence (2000AAA0104401) Beijing Natural Science Foundation (Z190001), and Peng Cheng Laboratory Key Research Project PCL222222.
Broader Impact
A potential negative side effect of this work is that the sample generation model may be used to generate fake images in the halftoning or inverse halftoning process for some certain purpose. In addition, most of the images we used in the dataset are collected from the Internet and may contain some bias. The biases are preserved the learning the model and may reflected the generated image samples (e.g., induce the model produce undesired results). An example of this work being used for unethical purposes to first train the model with biased or discriminatory data and then induce the model to produce unfaithful results when inverse halftoning images. This may misinterpret the original meaning of some ancient prints and mislead people.
Limitation Analysis
limitation of our model is that the diffusion process is relatively slow and thus requires more inference time than traditional inverse halftoning models, which is limited when the model is applied to mobile devices. One way to address this limitation to design parallel inverse halftoning methods, such restoring low-frequency and high-frequency components of images in parallel, thereby increasing the sampling speed.
References
David Blatner and Stephen F Roth. Real world scanning and halftones. Peachpit Press, 1993.
10

---

Fergus W Campbell, Janus J Kulikowski, and Levinson. The effect of orientation the visual resolution of gratings. The Journal of Physiology, 187(2)427-436, 1966.
Kelvin CK Chan, Xintao Wang, Xiangyu Xu, Jinwei Gu, and Chen Change Lay. Glean: Generative latent bank for large-factor image super-resolution. In Proceedings of the IEFF Conference on Computer Vision and Pattern Recognition, pages 14245-14254, 2021.
Li-Ming Chen and Hsuch-Ming Hang. An adaptive inverse halftoning algorithm. IEEE Transactions on Image Processing, 6(8): 1209, 1997.
Kuo-Liang Chung and Shih-Tung Wu. Inverse halftoning algorithm using edge-based lookup table approach. IEEE Trunsactions on Image Processing. 14(10):1583-1589 2005.
Prafulla Dhariwal Alexander Nichol. Diffusion models beat gans on image synthesis. In Advances in Neural Information Processing Systems, pages 8780-8794, 2021.
Reiner Eschbach. Error diffusion algorithm with homogenous response highlight and shadow arcas, Journal of Electronic Imaging, 6(3):348-38) -86666 1997.
Mark Everingham, Lue Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88(2):303-3355 2010.
Robert W Floyd. An adaptive algorithm for spatial gray-scale. In Proceedings of the Society of Information Display, volume 17, pages 75-77, 1976.
Qifan Gao, Xiao Shu, and Xiaolin Wu. Deep restoration of vintage photographs from scanned halftone prints. In Proceedings of the IEEE International Conference Computer Vision, pages 41 4120-4129, 2019.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 770 778,7, 2016.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, pages 6840-6851, 2020.
Xianxu Hou and Guoping Qiu. Image companding and inverse halftoning using deep convolutional neural networks. arXiv preprint arXiv: 16, 2017.
Tac-Hoon Kim and Sang II Park. Deep context-aware descrcening and rescreening of halftone images. ACM Transactions Graphics, 37(4):1-12, 2018.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114. 2013.
Thomas D Kite, Niranjan Damera-Venkann Brian L Evans, and Alan C Bovik. A fast, high-quality inverse halftoning algorithm for error diffused halftones. IEEE Transactions on Image Processing. 153:15553 2000.
Daniel Lau and Gonzalo R Arce. Modern digital halftooning CRC Press, 2018
Daniel L Lau, Gonzalo R Arce, and Neal C Gallagher. Digital halftoning by means of green-noise masks. Journal of the Optical Society of America A, 167551157) 1999.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. International Conference on Learning Representations, 2018.
Shitong Luo and Wei Hu. Diffusion probabilistic models 3d point cloud generation. In Proceedings of the IEBE Conference on Computer Vision and Pattern Recognition, pages 1837-2843 2021.
Murat Mese and Palghat P Vaidyanathan. Look-up table (lut) method for inverse halftoning. IEEE Transactions on Image Processing, 10(10):1566-1578 2001.
Murat Meşe and PP Vaidyanathan. Improved dot diffusion for image halftoning. NIP & Digital Fabrication Conference, volume 1999, pages 350-353. Society for Imaging Science and Technology. 1999.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162-8171, 2021.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and cosputer-assisted intervention, pages 2015.

---

Hasib Siddiqui and Charles A Bouman. Training-based descreening. IEEE Trunsactions on Image P Processing. 16(3):789-802 2007.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015.
Jascha Siil-Dickstein, Eric Weiss, Niru Maheswaranathan. and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. International Conference on Machine Learning, pages 2266-2265, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2020a.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems, pages 11918-11938 2019.
Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. In Advances in Neural Information Processing Systems, pages 12438 12448, 2020.
Yang Song. Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020b.
Yang Song, Conor Durkan, lain Murray, and Stefano Ermon. Maximum likelihood training of score-basee diffusion models. In Advances in Neural Information Processing Systems, pages 1415-1428, 2021.
Robert L Stevenson. Inverse halftoning via map estimation. IEEE Transactions on Image Processing, 6(4): 374-583, 1997.
James R Sullivan, Lawrence A Ray, and Rodney Miller. Design of minimum visual modulation halftone patterns.
IEEE Trunsactions on Syxtems. Man, and Cybernetics, 2 (1(1):33-33, 1991.
Bin Sun, Shutao Li, and Jun Sun. Scanned image descreening with image redundancy and adaptive filtering.
IFFE Transactions on Image Processing, 23(8)3698-337 2014.
Christian Szegedy. Vincent Vanhoucke, Sergey loffe, Jon Shlens. and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 2818-2826, 2016.
Robert A Ulichney. Void-and mlu-ccuster method for dither aray generation. In Human Vision, Visual Processing, and Digital Display IV, volume 1913, pages 332-343 International Society for Optics and Photonics, 1993.
Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. Esrgan: Enhanced super-resolution generative adversarial networks. In Proceedings of the European Conference on Computer Vision Workshops, pages 63-79, 2018.
Xintao Wang, Liangbin Xie, Chao Dong, and Ying Shan. Real-esrgan: Training real-world blind super- resolutionn with pure synthetic data. International Conference on Computer Vision Workshops, pages 1905 1914, 2021.
Menghan Xia and Tien-Tsin Wong. Deep inverse halftoning via progressively residual learning. In Asian Conference on Computer Vision, pages 523 539, 2018.
Menghan Xia, Wenbo Hu, Xueting Liu, Tien-Tsin Wong. Deep halftoning with reversible binary pattern. In Proceedings of the IEEE International Conference on Computer Vision, pages 14000-14009, 2021.
Yi Xiao, Chao Pan, Xianyi Zhu, Hai Jiang, and Yan Zheng. Deep neural inverse halftoning. In International Conference on Virtual Reality and Visualization, pages 213-218, 2017.
Zixiang Xiong, Michael Orchard, and Kannan Ramchandran. Inverse halftoning using wavelets. IEEE
Transactions Image Processing, 8(10):1479-1488 1999.
Meng Yao and Kevin Parker. Modified approach to the construction of blue noise mask. Journal of Electronic Imaging, 3(1):92-91 1994.
Yan Zhang, Erhu Zhang. Wanjun Chen, Yajun Chen, and Jinghong Duan. Sparsity- based inverse halftoning via semilcoupled multi-dictionary learning and structural clustering. Engineering Applications of Artificial Intelligence, 72:43-53, 2018.
Zhifei Zhang, Yang Song, and Hairong Qi. Age progressionrregressonn by conditional adversarial autoencoder. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5810-5818, 2017.
12


Reviews:
[{"review_id": "Ya6IWPaXJoT", "reviewer": "Reviewer_FAj2", "summary": "This paper proposes a new method for inverse halftoning (recovering realistic images from ancient prints). The proposed method is a generative method that uses reverse diffusion process to produce desired halftone dithering patterns. A meta-halftone guided network is proposed to use the blue noise to guide the halftone diffusion process for handling redundant patterns and artifacts. The paper also proposes to condition the inverse halftoning diffusion on the initial state of halftone dithering diffusion, in order to learn a more robust mapping. Experiments demonstrate its state-of-the-art performance.", "questions": "Please see above section.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths.\n\n1. A generative halftoning method is proposed.\n2. A meta-halftone guided network is proposed to reduce redundant patterns and undesired artifacts.\n3. A learning dataset is constructed.\n4. The motivation of this work is clear.\n5. Code is provided.\n\nWeaknesses.\n1. The split of the proposed dataset (7857 for training, 400 for validation and 400 for test) seems not reasonable.\n2. The paper compres to only four methods chosen/adapted from previous halftoning/super-resolution methods, which is not convincing. For example, diffusion models that are proposed for image denoising can be used for comparisons.     ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "8gHSpATCYo0", "reviewer": "Reviewer_kkXU", "summary": "The paper appears to beat the SOTA on inverse halftoning  with an approach that uses diffusion models. The details are not very transparent.", "questions": "See above", "limitations": "No comment", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The artifact reduction in inverse results is significant. In addition, the generated halftone images have a nice quality to them, so that might be useful, too. But the description of the details is difficult to read. Not enough is illustrates with images and the math is a it too abstract. When I read the title I assumed that the noising process in a diffusion model would reflect the dithering (going from the original image to a noise one by adjusting intensities until they are binary), but the method instead is using a traditional diffusion models to capture image distribution and a series of adjustments happens with meta-halftones (a new idea), which are not well illustrated. Nevertheless, the results look very good.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "6w5Wk0QslX", "reviewer": "Reviewer_Ptjr", "summary": "In this paper, the authors propose a generative halftoning method, which regards the black pixels in halftones as physically moving\nparticles, and makes the randomly distributed particles move under some certain guidance through the reverse diffusion process. The proposed Conditional Diffusion model for image  Halftoning (CDH)  consists of a halftone dithering process and an inverse halftoning process. Specifically, to avoid introducing redundant patterns and undesired artifacts during halftone generation, they propose\na meta-halftone guided network to incorporate the blue noise guidance into the diffusion process and train an inverse halftoning diffusion model to learn the mapping function from the halftone distribution to the continuous-tone distribution. They claim that this helps the model to learn a  more robust mapping to perform inverse halftoning and improve the generalization ability to unseen samples.\n", "questions": "The authors claimed that the limitation of the existing deep learning approaches for inverse halftoning is the lack of generalization due to the use of U-Net.  However, to my understanding, the particular architecture of the U-Net is not the main culprit, but the way of supervised training is the problem. The authors should clarify this.", "limitations": "The slow speed of the diffusion model is stated as the limitation.", "rating": 3, "confidence": 1, "soundness": 1, "presentation": 1, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: This is the first attempt to use diffusion models for the half-tone generation and inverse half-toning.\n\nWeakness:  The writing is quite confusing. The reviewer has tried to understand the link between the proposed method and the score-based diffusion model, but could not find any close link.  Why does the complicated forward diffusion process be necessary? In figure 2, why there are two paths for half-tone dithering and inverse half-toning? If I understand correctly,  a better way of inverse halftoning using diffusion would be a noising process of the dithered image followed by the reverse diffusion process to obtain the continuous image. But the main focus of this paper appears to be the dither pattern generation, which appears quite far distance from the diffusion model. I have to say that I am not an expert on half-toning, but still, the way of describing the algorithm is quite confusing to general NeuRIPS readers.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "13bJQsMbWah", "reviewer": "Reviewer_6UuY", "summary": "This paper introduces an inverse halftoning method based on conditional diffusion processes. The goal is to restore halftoned images generated using any halftoning algorithm, by leveraging recent advances on generative models. Previous work did not generalize properly to halftoning algorithms outside of those used for training the model, while the proposed method performs the inverse halftoning operation through conditional diffusion models. To improve the capabilities of the model, the paper ontroduces a meta-halftone guiding framework for introducing blue noise into the diffusion process. Quantitative results are provided that show improved results with respect to supervised learning and image filtering frameworks. ", "questions": "- What is the impact of the design of the meta-halftone module? Why is this design chosen? Could you provide a baseline for this design and some ablations on why this design is better than other methods? I am concerned about the use of a pretraiend VGG network for this purpose, as the features on its last layer may be too closely related to ImageNet classification. Why is pre-training needed? \n- What does it mean that the \"mainstream U-net architecture has poor generalization ability in practical applications\"? Is it a problem of the U-net architecture by itself? If so, what does this paper do to mitigate these problems? Looking at the code, it seems that the diffusion model is a U-net with attention, similar to those used in Palette or other DDPM models. Can the authors provide more details on this? Is it really an architectural problem, or are more factors involved (learning formulation, loss function, etc.)? \n- On the validation side, I think PSNR and SSIM are not sufficient for analyzing the results of generative models. What are the scores using perceptual metrics (FiD, Inception Score, LPIPS)? \n- On Table 1, why are standard deviations only provided for CDH? Why is the standard deviation for SSIM so low?\n- What happens with RGB images? Does this method only work for grayscale images?\n- On Figure 1 and Figure 5, what is the input to the model? It would be interesting to see the halftoned inputs used throughout the paper. \n- Does the model work better with some halftoning algorithms than others? A quantitative analysis on this would be benefitial.\n- Could the authors compare this paper to Palette: Image-to-Image Diffusion Models, Saharia et al. and Deep Image Prior, Ulyanov et al. ?  I think the image restoration ideas on these papers are relevant for this CDH paper and should be at least mentioned. \n- What happens when the images are larger to the 256x256 pixels used in training?\n- Can the authors provide an illustration on the \"blue noise\" that can be introduced by the proposed metahalftone module? ", "limitations": "The authors have adressed some limitations of the proposed method, particularly issues of computational cost at inference time with respect to other work. They also point out that their work may be used to \"generate fake images for some certain purpose\", and that the dataset used for training \"may contain some bias\". \n\nI suggest the authors extend this analysis. In particular, I think it would be interesting to give particular examples of how this work may be used for unethical purposes, or which harmful biases are introduced with the dataset. Also, I suggest the authors to extend the limitation analysis, in terms of: \n- What happens with RGB images?\n- What happens with resolutions outside of the specified 256x256 pixels? ", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "*Summary* \n\nThis paper provides valuable ideas for designing diffusion models for inverse halftoning. The paper is in general well written, but lacks sufficient validation, appropriate discussion of previous work and the significance of the findings of this paper are limited in the overall literature of diffusion models. \n\n\nStrenghts:\n- This paper provides specific ideas on how to leverage diffusion models for the particular problem of inverse halftoning.\n- The paper is, in general, well written and is very easy to follow. The authors present their ideas in a concise manner.\n- Code is provided for enhanced reproducibility. \n- Provided analysis show significant improvements with respect to previous work.\n\n\nWeaknesses:\n- The paper lacks sufficient validation, it is not clear why some decisions were taken. Please see \"questions\" for details. \n- The design of the Meta-halftone guided network is not adequately justified. Quantitatevely, it is shown that this module enhances the result of the model, but it is not clear to the reader why this design was chosen. \n- Previous work is not adequately studied. I suggest the authors to consider analyzing Palette: Image-to-Image Diffusion Models, Saharia et al. and Deep Image Prior, Ulyanov et al. \n- The analysis of previous work is also limited. Following the authors wording, it is claimed that \"U-net architecture has poor generalization in practical applications\". Reading this, it may be understood that this paper provides alternatives to the U-net architecture, but it does not. I suggest the authors to change the wording to \"supervised learning\" or \"pixel-wise regression\" instead of \"mainstream U-net architecture\", as this represents more accurately the baselines that the authors want to improve. \n- Limitations need further analysis.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept