## 4 Dataset Development
### Depth Map Standardization
Most significantly, there is no discernible presence of the orthogonal fringe pattern that would have been strikingly present in the input image, highlighting VUNet's ability to reconstruct the underlying surface geometry without retaining or passing through the pattern information. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model** & **MAE** & **RMSE** & **LogError** & **Inference Time (ms)** \\ \hline VUDNet (On Deformation) & 0.0443 & 0.0600 & 0.0122 & 19 \\ \hline VUDNet (On Geometric) & 0.0168 & 0.0218 & 0.0038 & 19 \\ \hline VUDNet (General) & **0.0355** & **0.0470** & **0.0090** & 19 \\ \hline DYNet++ * (General) & 0.1607 & 0.1987 & 0.0790 & 26 \\ \hline D-UNet ** (General)** & 0.2052 & 0.2245 & 0.0451 & 21 \\ \hline \hline \end{tabular}

* DYNet++ plus reconstructing the surface by zonal integration for reconstructing the surfaces as mentioned in [17]. Since there are no publicly available data on the DYNET++ dataset, we were unable to test on their dataset, so we implemented their method on our dataset. **For D-UNet [29], as there are no publicly available data or code, we implemented it based on [29] and tested it on our proposed dataset.

\end{table} 
Table 1: Performance comparison of different models on our generated dataset. VUDNet has reconstruction errors approximately one fourth that of competing methods, under all three criteria of Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Log Error, and with an inference time (computational complexity) similar to or less than competing methods. Bolded numbers indicate the best performance on the General data set. Figure 5 presents a comparison between the ground truth and predicted depth maps for a randomly selected sample. The left images display the ground truth depth map, while the right images show the depth map predicted by our VUDNet model. The predicted depth map is smoother than the original, due to the regularization term. Nevertheless, the estimates still retain a high degree of fine detail. The visual comparison suggests that the model effectively captured the essential features of the surface, aligning well with the expected characteristics observed in the ground truth data. This smoothing effect helps to manage the noise and artifacts typically associated with specular surfaces. The t-SNE (t-distributed Stochastic Neighbor Embedding) method [53] is a powerful dimensionality reduction technique commonly used to visualize high-dimensional data in a two- or three-dimensional space. By applying t-SNE, we can effectively observe how well-separated the latent space is, learned by the VAE. Figure 6 demonstrates such a separation of the representation space, whereby the clustering effect indicates that the VAE successfully formed a coherent representation space where intrinsic features of different surfaces are well captured and recognizable. In this context, a cluster in the VAE's representation space refers to a grouping of data points (input images) that share similar underlying features. Ideally, cluster separation is evidence of meaningful feature learning, as it implies that the VAE is able to effectively distinguish between various surface characteristics. In Figure 6, the well-defined clusters suggest that the VAE can identify and learn the underlying patterns in the data, leading to potent feature extraction. This capability is essential for generalizing the network's performance across diverse and complex surface geometries. Figure 7 builds on Figure 6, but now based on a network trained exclusively on the deformation data as opposed to the entire dataset in Figure 6. The scattering of the samples based on their features shows that similar data points (similar surface shapes) are located near to each other in the latent space. This proximity indicates that the VAE can effectively group similar surfaces, enhancing the network's ability to perform accurate depth estimation.