# High-Precision Depth Map Estimation from Missing Viewpoints for 360-Degree Digital Holography
## 3 Proposed Method
### Proposed Model  
The proposed depth map estimation model HDD (Holographic Dense Depth) consists of two components: the encoder and decoder, as shown in Figure 2 and Table 3. We experimented with several encoders as seen in previous studies [4; 8; 12]. Then, we found that deeper layers and more convolution operations do not necessarily lead to better depth map estimation results and DenseNet-161 [26] is an optimal model for our purpose. The encoder performs feature extraction and down-sampling of the input RGB images. Each feature maps are connected by skip-connections to correspond the up-sampling layer in the decoder. At this point, both layers are calculated as concatenation. The decoder performs up-sampling by concatenating the extracted features based on the size of the RGB image. The weights for both components are optimized by a loss function that minimizes the discrepancy between the ground truth and the estimated depth map. CDD learns and estimates the depth from a single viewpoint. On the other hand, the proposed model, HDD, learns depth from multiple viewpoints and estimates the depth of viewpoints that are not used for training (new viewpoints). We also adopted bilinear interpolation in the up-sampling layer and ReLU as the activation function, of which are different from CDD. Consequently, we obtained better depth estimation results. Since [PERSON] et al. 's research [4]--which was an early CNN-based depth map estimation study--in general, the loss function of depth map estimation has been based on a difference between ground truth depth map and estimated depth map. Furthermore, in the comparative model CDD [12], _MSE_ (Mean Squared Error) and _SSIM_ (Structural Similarity) [27] were used together for loss function. They optimized coefficient of loss function for CDD's purpose (general depth map estimation). In this study, we experimentally explored the best coefficient of loss function that gives the optimal depth estimation for our purpose (depth map estimation for 360-degree holographic content). As a result from this experiment, we concluded that the coefficient of 100% _MSE_ and 0% _SSIM_ is optimal for depth map estimation for our data, of which are suitable for 360-degree holographic content. The details of the loss function coefficient are shown in Figure S1. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multicolumn{4}{c}{**Data Set 1 (Depth Map Estimation for Objects Used in Train)**} \\ \hline \multicolumn{2}{c}{Generated sample} & \multicolumn{2}{c}{Shape} & \multicolumn{2}{c}{Training set/Test set} \\ \hline \multirow{4}{*}{RGB images} & \multirow{2}{*}{4096} & Torus & 1024 & \multirow{2}{*}{\begin{tabular}{c} 614 \\ \end{tabular} } & \multirow{2}{*}{\begin{tabular}{c} 410 \\ \end{tabular} } & \multirow{2}{*}{\begin{tabular}{c} 410 \\ \end{tabular} } \\ \cline{3-3} \cline{5-6}  & & Cube & 1024 & & & 614 \\ \cline{2-6}  & & Cone & 1024 & & 614 \\ \cline{2-6}  & & Sphere & 1024 & & 614 \\ \hline \multirow{4}{*}{Depth map images} & \multirow{4}{*}{4096} & Torus & 1024 & \multirow{2}{*}{\
\begin{tabular}{c} 614 \\ \end{tabular} } & \multirow{2}{*}{
\begin{tabular}{c} 410 \\ \end{tabular} } \\ \cline{3-3} \cline{5-6}  & & Cube & 1024 & & 614 \\ \cline{1-1} \cline{2-6}  & & Cone & 1024 & & 614 \\ \cline{1-1} \cline{2-6}  & & Sphere & 1024 & & 614 \\ \hline \multicolumn{6}{c}{**Data set 2 (Depth map estimation for objects not used in train)**} \\ \hline \multicolumn{2}{c}{Generated sample} & \multicolumn{2}{c}{Shape} & \multicolumn{2}{c}{Training set/Test set} \\ \hline \multirow{2}{*}{RGB images} & \multirow{2}{*}{820} & Dodecahedron & 410 & Set for & - & Set for & 410 \\ \cline{2-6}  & & Icosahedron & 410 & training & - & test & 410 \\ \hline \multirow{2}{*}{Depth map images} & \multirow{2}{*}{820} & Dodecahedron & 410 & Set for & - & Set for & 410 \\ \cline{2-6}  & & Icosahedron & 410 & training & - & test & 410 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Data sets in which RGB images and depth maps are prepared for training and test in the research.