# A Sparsity-Invariant Model via Unifying Depth Prediction and Completion
## 5 Experiment
### Experimental Results in Real-World Application
#### 5.5.2 Construction of Trainset  
As obtaining corresponding ground-truth depth maps of Spot-iTOF for training is infeasible, we turn to virtual datasets. Among them, we specifically chose the Hypersim dataset, which contains scenes most similar to real-world scenarios and color images closest to actual images. The Hypersim dataset [80] is a large-scale synthetic dataset for computer vision and machine learning research, providing highly realistic indoor scenes. This dataset encompasses a variety of residential, commercial, and office environments, comprising 77,400 datasets across 461 indoor scenes, and it provides comprehensive scene information, including geometry, materials, depth, lighting, and semantic segmentation. In this experimental section, we employed the Hypersim dataset for training and tested the model on the dataset acquired from OPPO's Spot-iToF316 prototype. To achieve this, we preprocessed the Hypersim data as follows. Firstly, from the 77,400 datasets in Hypersim, we selected 38,542 sets of indoor scene data as the training set. Then, for the generation of color images, we utilized the lighting information in Hypersim and simulated different exposure settings to generate color images. The depth information provided by Hypersim represents the distance of object surfaces from the optical center; therefore, for the generation of dense depth maps, we utilized the camera intrinsic parameters to convert it into depth maps in terms of distances from objects to the image plane. Finally, to match the resolution of the test set, we downsampled both the color images and depth maps, adjusting them to the same resolution as OPPO's, i.e., 256 \(\times\) 192. Bilinear downsampling \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Methods** & **500** & **200** & **100** & **50** & **5** & **0** \\ \hline pNCNN [79] & 0.170 & 0.237 & 0.338 & 0.568 & 2.412 & - \\ \hline CSPN [68] & 0118 & 0.177 & 0.338 & 0.884 & 2.063 & - \\ \hline NLSPN [6] & **0.101** & **0.142** & 0.246 & 0.423 & 1.033 & - \\ \hline NConv-CNN [22] & 0.129 & 0.173 & - & - & - & - \\ \hline G2-MonoDepth [73] & 0118 & - & - & _0.248_ & 1.321 & - \\ \hline SpAgNet [38] & _0.114_ & _0.155_ & _0.209_ & 0.272 & _0.469_ & - \\ \hline Ours & 0.122 & 0.162 & **0.197** & **0.239** & **0.393** & **0.470** \\ \hline \hline \end{tabular} The optimal numerical results correspond to sparse points, highlighted in bold; the next best are indicated with underlines; the third best are shown in italics.  
\end{table} 
Table 4: Comparison with state-of-the-art methods (RMSE). was applied to the color images, while nearest-neighbor downsampling was applied to the depth maps to preserve depth values rather than generating pseudo-depth values. Following the preliminary processing of the Hypersim dataset, we acquired a dataset comprising color images and corresponding ground-truth dense depth maps. To generate the sparse depth maps needed for training, we aimed to derive them from the ground-truth dense depth maps while ensuring that the resulting sparse depth maps closely resemble those obtained from iTOF depth sensors in real-world scenarios. Based on our observations of the basic characteristics of the depth value distribution of iTOF cameras in real-world scenes, we followed these steps to generate sparse depth maps. * Step 1: The sparse depth map obtained from the iTOF sensor exhibits distinct distribution characteristics in rows and columns; thus, we first obtain a basic binary mask based on the fundamental row-column positions to represent the positions of valid depth points. * Step 2: Considering that the sparse depth map generated by the iTOF sensor may not strictly adhere to the regular pattern of row-column positions due to inherent errors, we introduce random offsets to the positions in the basic binary mask to obtain a preliminary sparse mask. * Step 3: We use the preliminary sparse mask to sample the dense depth map truth values, resulting in a preliminary sparse depth map.