# Progress of the Computer-Generated Holography Based on Deep Learning
## 4 The Learning-Based CGH
### ResNet and U-Net Based Deep CGH  
The U-Net neural network has few layers and parameters and is easy to be trained. ResNet and U-Net-based deep CGHs have been developed. In 2018, [PERSON] et al. used a deep ResNet for calculating holograms [23]. Their training dataset was composed of uniform random phase patterns and their Fresnel propagating patterns at a distance. The number of training speckle pairs was 100,000. Then, for less computation and faster generation speed, [PERSON] et al. proposed a U-Net-based architecture to generate binary holograms [24]. The DenseNet architecture was used to find a better solution. The output gradient of each block in the front was skip-connected to the output of each block in the back. A similar selective structure was added to the same layer to further improve the effect. In this method, random amplitude binary patterns and their Fresnel propagating intensity patterns were used for the network training. The pixel number of the hologram and target pattern was 128 \(\times\) 128. The number of the training dataset was 10,000. Numerical and optical experiments were carried out, and the proposed method was compared with the Fresnel ping-pong method in terms of quality and calculation time. The training of the proposed method took 6000 s for preparation and 0.03 s for each hologram, while the ping-pong method took 0.7 s per hologram. The algorithms above realized better and faster reconstruction of the two-dimensional image through deep learning networks. To realize the reconstruction of 3D images, [PERSON] et al. presented a deep neural network for generating a multi-depth hologram in 2020, as shown in Figure 6[29]. The proposed multi-depth hologram generation network (MDHGN) receives multiple images with different depths as input and generates complex holograms that can reconstruct the input images. According to each depth of d\({}_{1}\) to d\({}_{5}\), five different images of randomly positioned dots were generated and superimposed as a 3D target. The complex hologram calculated by the ASM algorithm was divided into a real hologram and an imaginary hologram, and stored as the ground truth. NVIDIA's Quadro P5000 GPU was used to train the network. The numerical and optical reconstructions were carried out, which proved that it is possible to reconstruct multiple images at different depths [29]. \begin{table}
\begin{tabular}{c c c} \hline \hline
**CGH Methods Based on Deep Learning** & **DNN Related** & **Year** \\ \hline Deep-learning–generated holography [23] & ResNet & 2018 \\ Deep neural network for multi-depth hologram generation [29] & ResNet & 2020 \\ Deep-learning–based binary hologram [24] & U–Net & 2020 \\ DeepCGH [49] & U–Net & 2020 \\ CGH using an autoencoder–based deep neural network [10] & U–Net & 2021 \\ Three–dimensional generated holography [5] & U–Net & 2021 \\ GAN–Holo [27] & GAN & 2021 \\ Deep–learning–based hologram using a generative model [25] & GAN & 2021 \\ Real–time photorealistic 3D holography [50] & ResNet & 2021 \\ Compensation of spatially varying aberrations [26] & GAN and U–Net & 2022 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: DNN used in CGH based on deep learning. Figure 5: Development and application of CGH based on DNN in recent years. In 2020, [PERSON] et al. proposed a new algorithm, DeepCGH, for hologram synthesis [49]. The concept map of DeepCGH is shown in Figure 7. The unsupervised training method was employed in order to overcome the limitation of the training dataset. The network architecture of DeepCGH was U-Net with five convolutional blocks. The proposed method was compared with the GS and NOVO-CGH algorithms. DeepCGH was implemented in the Python programming language using TensorFlow 2.0 deep learning framework. The models were trained with NVIDIA GeForce RTX 2080 Ti GPUs, while the GS and NOVO-CGH algorithms were implemented with MATLAB and CUDA GPU libraries. It was presented that the 3D holograms synthesized with DeepCGH had greater accuracy, and the processing speed was higher than that of the existing methods.