# Inverse Halftoning Methods Based on Deep Learning and Their Evaluation Metrics: A Review
## 2 Inverse Halftoning Technology for Digital Halftoning
### End-to-End Learning Method Based on Training of Multiple Sub-Networks (EEBMN)
#### 2.2.2 Deep Learning with Pre-Training (DLWPT)  
In order to further improve the accuracy of information reproduction, the researchers put forward the idea of firstly pre-training a network layer by layer and then training comprehensively. The process could be divided into designing the network, generating datasets according to the functions of sub-networks, pre-training the sub-network, and training the whole network in sequence. [PERSON] [19] proposed an inverse halftoning method through structure-aware DCNN, with the network being composed of three sub-networks as shown in Figure 7, in which each sub-network is constructed by a different number of layers including convolution and ReLU operation. The first sub-network aims to generate the initial continuous-tone image from an input halftone image, whereas the second sub-network connected to the output of the first sub-network is designed to extract local image structures such as lines, curves, or textures. The input halftone image, the output of the first sub-network, and the second sub-network are all concatenated and then fed into the third sub-network for reconstructing the final continuous-tone image. As shown in Figure 7, the entire network is trained in stages with each sub-network. The first sub-network is primarily trained using the input halftone and its corresponding continuous-tone image, where the MSE loss function is employed to guide the training process. Once the first sub-network has been trained well, the second sub-network begins to be trained. For the training of the second sub-network, the Sobel gradient operator is applied to generate the gradient image, and the MSE loss between the gradient images generated by the output of the second sub-network and the original continuous-tone image is used to train it. Ultimately, the halftone images and continuous-tone images are applied to train the whole network, including the third sub-network in an end-to-end manner. The advantage of this method is that the image structure map predictor (ISMP) is introduced in the network, which can predict image structure for improving the quality of restored images. The disadvantage of the method is that the experiment is only evaluated in halftone images generated by the error diffusion method. Figure 6: The network architecture in reference [20]. Figure 7: The network architecture in reference [19]. The same as reference [19], [PERSON] et al. [21] also used gradient information in their proposed network to boost details in the recovered continuous-tone image. In contrast to the method in reference [19], the proposed network as shown in the dashed box of Figure 8 firstly extracts the vertical and horizontal gradient information directly from the input halftone image by sub-network V1 and sub-network V2, respectively. Then, the input halftone image and two gradient maps generated by the sub-network V1 and sub-network V2 are concatenated as the input of the third sub-network IH. The third sub-network IH is used to restore continuous-tone images. All the three sub-networks have the same architecture with a U-shaped encoder-decoder and skip connections. The training of the network is divided into two stages. In the first stage, the sub-network V1 and V2 are pre-trained, respectively, by the MSE loss between gradient maps obtained from ground truth images and predicated gradient maps generated by the sub-network V1 and V2. In the second stage, the loss function for training the third sub-network IH is composed of two parts: the first part is the MSE loss between the final output image and the ground truth image, and the second part is the gradient loss term. To further enhance the detail and texture information, [PERSON] et al. [22] proposed an improved method based on the method in the reference [21], which takes the effectiveness of residual sub-network into account, as shown in Figure 8. For the training of the residual sub-network, the \(L_{1}\) loss between the predicted continuous-tone image by the residual sub-network and the ground truth image is used.