# Inverse Halftoning Methods Based on Deep Learning and Their Evaluation Metrics: A Review
## 3 Inverse Halftoning Technology for Scanned Halftone Images
### Inverse Halftoning Based on General Purpose DCNNs for Scanned Halftone Images  
Super-resolution reconstruction is a typical image processing problem that is used to restore high resolution images from low resolution images. Similarly, inverse halftoning can also be considered as an image super-resolution reconstruction problem. Thus, the DCNN designed for super-resolution reconstruction can also be used for inverse halftoning. In reference [24], the authors [PERSON] et al. used the Enhanced Super-Resolution Generative Adversarial Networks (ESRGAN method) proposed by [PERSON] et al. [25] for comparison with their proposed inverse halftoning method. The architecture of ESRGAN is shown in Figure 10, which is the optimized version of Super-Resolution Generative Adversarial Network (SRGAN) [34]. In ESRGAN, the discriminator of Relativistic GAN (RaGAN) [35] and dense block [32] are adopted to replace the discriminator of SRGAN and residual block [31], and the batch normalization is also removed. Image style transfer is a technique that converts an image into another style image. On the basis of the idea that halftone images and continuous-tone images are two different style images, the DCNN designed for style transfer can be retrained for inverse halftoning. Considering the limitations of no paired dataset for scanned halftone images, DCNNs based on unsupervised GAN network proposed by [PERSON] [36] is a better choice, such as CycleGAN [26] and XNet [27]. Figure 10: The architecture of ESRGAN. The generator is used to produce an inverse halftone image from its low-resolution halftone image, and the RaD is a discriminator for judging whether the inverse halftone image is a continuous-tone image. Figure 9: The network architecture in reference [23]. The core technique of CycleGAN is the two mapping generators, with which the whole process can be performed as a loop as shown in Figure 11. The two mapping generators denote the generator of converting halftone images to inverse halftone images and the generator of transferring continuous-tone images to halftone images. Meanwhile, two discriminators are used to promote the generation of the optimal inverse halftone image. One is used to reduce the difference between the generated halftone image and the original halftone image, and the other is used to narrow the gap between the inverse halftone image and the continuous-tone image. The cycle structure with two discriminators realizes the conversion of halftone and continuous-tone style smoothly. In 2019, [PERSON] [27] proposed the concept of latent space on the basis of CycleGAN, and designed XNet by fusing two common GAN networks as shown in Figure 12. Accordingly, cross identity loss, latent cycle-consistency loss, and latent cross-translation consistency loss were proposed, which are similar to the principle in CycleGAN. These loss functions make constraints on training more accurate and are prone to achieving better performance. To sum up, the GAN-based network can indeed be applied to inverse halftoning, but the image performance is not ideal because of the uncertainty of information learning. To alleviate the problem, [PERSON] et al. [28] proposed a phased inverse halftoning method, as shown in Figure 13, which consists of three sub-networks with a three-layer. The whole process is divided into dataset generation and network training. For dataset generation, the rough image and guidance image are generated by processing the scanned halftone image with previous methods. For network training, the training dataset is firstly sent to the first two sub-networks for feature extraction, and then the extracted features are inputted into the filter network. The smaller the residual, the closer the restored image and original continuous-tone image will be. Although the above general purpose DCNN methods can be retrained for inverse halftoning, the restored image quality is poor compared with those of purposefully designed inverse halftoning. Figure 11: The architecture of CycleGAN.