# Inverse Halftoning Methods Based on Deep Learning and Their Evaluation Metrics: A Review
## 2 Inverse Halftoning Technology for Digital Halftoning
### End-to-End Learning Method Based on A Single Network (EEBSN)  
EEBSN means that the whole deep neural network consists of only one network. The specific process of EEBSN is to design the network structure, generate the training dataset, and input the dataset into the network for training.  
In 2017, [PERSON] et al. [17] first proposed an Inverse Halftoning Method based on UNet (called IH-UNet). The network structure is shown in Figure 5, where the U-Net structure combined with batch normalization (BN) were applied as the basic network structure. In order to merge the low-level features into the high-level features, the skip connection structure was added on the basis of U-Net. Following this, the U-Net structure was also used by [PERSON] et al. in reference [18], in which the equal-size feature graph in up-sampling and down-sampling were fused by a concatenating method instead of using skip connection, and average pooling (AP) was used for down-sampling as shown in Figure 5. Apart from the U-Net structure, the fully convolution neural network proposed in reference [19] can also be used for inverse halftoning, which consists of two types of layers: the convolution layers and Rectified Linear Unit (ReLU) layers, where they are arranged in a sequence. On the basis of the above basic principle of image transformation with DCNN, we think the ResNet proposed by [PERSON], [PERSON] et al. [31] and the DenseNet proposed by [PERSON], [PERSON] et al. [32] can also be taken as a basic network structure for inverse halftoning.  
Figure 5: The network architectures in references [17; 18]. The networks in references [17; 18] are shown in the dotted box and solid box, respectively.  
The loss function is another important problem for inverse halftoning based on DCNN. The common loss function used in reference [18] was the mean square error (MSE) of the restored continuous-tone image \(I^{th}\) and its corresponding real continuous-tone image \(I^{c}\), which is a per-pixel loss function as follows:  
\[L_{MSE}=\frac{1}{N}\sum_{i=1}^{N}\left(I^{c}_{i}-I^{th}_{i}\right)^{2} \tag{2}\]  
where \(N\) is the pixel number of image \(I^{th}\) or image \(I^{c}\).  
Due to easy producing blurry outputs or visual artifacts by per-pixel losses, the perceptual loss was used in reference [17], which was used to measure the difference between two images at different deep feature levels on the basis of pre-trained deep convolutional neural networks. The perceptual loss function is defined as Equation (3).  
\[L_{p}=\frac{1}{N}\sum_{i=1}^{N}\left(\Phi_{j}(I^{c}_{i})-\Phi_{j}(I^{th}_{i}) \right)^{2} \tag{3}\]  
where \(\Phi_{j}\) represents the extracted features at the \(j^{th}\) convolutional layer of the pre-trained deep convolutional neural network. The VGG-Net [33] is commonly used as a typical pre-trained deep convolutional neural network.  
By analyzing the above methods, we found that the method in [17] was better than other methods based on a single network. The advantage of this method is that using U-Net structure and skip connection can incorporate low level features in the decoding process. In addition, the perceptual loss is a good choice for improving the quality of restored images. However, the image details are still not satisfactory.