# High-Precision Depth Map Estimation from Missing Viewpoints for 360-Degree Digital Holography
## 2 Previous Research  
Many previous studies have estimated depth maps based on three distinct approaches. The first is the monocular approach. [PERSON] et al. [3] proposed the generation of depth maps using image classification. In this work, digital images are classified as indoor, outdoor, and outdoor with geometric objects, with low computational costs. [PERSON] et al. [4] proposed a convolutional neural network (CNN)-based model consisting of two different networks. One estimates the global structure of the scene, whereas the other estimates the local information. [PERSON] et al. [5] studied the preservation of edges and planar regions, depth consistency, and absolute distance accuracy from a single image. Other studies adopting conditional random fields [6; 7; 8], generative adversarial networks [9; 10], and U-nets with an encoder-decoder structure have been introduced [11]. [PERSON] et al. [12] applied transfer learning to a high-resolution depth map estimation, which we refer to as conventional dense depth (CDD) for the remainder of this paper. The second method is a stereo-view approach, which uses a pair of left and right color images as input [13; 14]. [PERSON] et al. [15] recently applied self-supervised learning to single depth map estimation based on stereoscopic color images. Their model used a pair of left and right color images to synthesize a middle-view-pointed color image and estimated its corresponding depth map. The stereo-view approach is different from the proposed multi-view method as it can estimate the depth map of only a single-viewed image located between the left and right images. The third approach estimates a single depth map using multi-view images (more than two RGB images). Many of them use the plane-sweep method [16], which is a basic geometry algorithm for finding intersecting line segments. [PERSON] et al. [17] used CNNs for multi-view stereo matching, which combines the cost volumes with the depth hypothesis in multi-view images. [PERSON] et al. [18] proposed an end-to-end model that learns a full plane-sweep process, including the construction of cost volumes. Recently, [PERSON] et al. [19] proposed an asymmetric encoder-decoder model that has improved accuracy for outdoor environments. [PERSON] et al. 's work features a CNN for solving the depth estimation problem on several image-pose pairs that are taken continuously while the camera is moving [20]. All of the above studies can generate only a single depth image using multi-view RGB images. To provide realistic holographic metaverse [21; 22], Augmented Reality (AR) [23], or Virtual Reality (VR) [24] content (see Figure 1b) to users, it is essential to estimate as many highly precise depth maps as possible in a short time for each given narrow angular range. Therefore, in this study, although we use the input data of multi-view RGB images, we do not adopt either conventional stereo-view or multi-view methods because these methods  
Figure 1: **(a)** Depth map estimation at missing viewpoints (II) using multi-viewed RGB-Depth map set (I). Only RGB color image exists at each missing viewpoint. **(b)** The proposed HDD model provides holographic 3D content (CGHs) with even missing viewpoints (II) covered to the full so that the user can watch much richer 360-degree holographic AR movies (3D model of the deer in figure was provided from TurboSquid, [[https://www.turbosquid.com/ko/3d-models/free-caribou-3d-model/591531](https://www.turbosquid.com/ko/3d-models/free-caribou-3d-model/591531)]([https://www.turbosquid.com/ko/3d-models/free-caribou-3d-model/591531](https://www.turbosquid.com/ko/3d-models/free-caribou-3d-model/591531)), accessed on 17 September 2022). use multiple RGB images as input to output only a single depth map. Our model can generate new depth maps derived from new RGB images of the missing viewpoints, as illustrated in (II) of Figure 1a. As a result, we were able to obtain 360-degree depth maps of objects for realistic holographic 3D or AR/VR content.