## 4 Dataset Development
### Depth Map Standardization
The close grouping of similar samples demonstrates the VAE's proficiency in feature extraction, ensuring that surfaces with similar characteristics are represented similarly. Figure 4: Mean absolute difference between the ground truth depth map and the predicted depth map from our VUDNet model for a selected sample. The error pattern demonstrates smoothness, effective regularization, and an overall minimal presence of outliers. The top region contains localized patterns of higher error values (illustrated as collection of red pixels), however despite these localized artifacts, the remainder of the image demonstrates the model’s accuracy with no visible orthogonal fringe patterns. The VAE's ability to develop such clusters and similarity behaviours signifies its strength in capturing and representing the essential features of the input data. This representation plays a critical role in the subsequent stages of depth estimation, where accurate and reliable feature extraction is paramount. Figure 5: Comparison of ground truth (**left**) and VUDNet-estimated depth maps (**right**), showing effective noise reduction and fine detail retention. The top images showcase a result for a deformation sample, while the bottom images represent an example of the geometric case. Figure 6: VAE representation (**left**) showcasing distinct separation of clusters in the latent space, visualized as the first and second components of t-SNE. The clustering indicates effective differentiation of surface characteristics and potent feature extraction. The four images on the right correspond to the selected points in the latent space (**right**). It is evident that images from a given cluster share related surface characteristics, highlighting the network’s ability to identify underlying similarities despite variations in surface characteristics. Although our method employs an ensemble of a VAE and U-Net, we focus on visualizing the latent representation of the VAE. As highlighted in prior research [32], VAEs are designed to learn compact, informative latent spaces [54; 55; 56] that capture the underlying structure of the data. In contrast, U-Net primarily performs pixel-level refinement, working directly in the image space without explicitly learning a latent representation [33]. Therefore, the U-Net's contribution is more appropriately assessed through its impact on the final depth maps, rather than through latent space visualization. Table 1 demonstrates the remarkable performance of VUDNet. The quantitative metrics presented in the table demonstrate the network's capability to achieve high accuracy and consistency in depth estimation. Specifically, the VUDNet model shows significantly lower Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and LogError compared to the comparator DYNet++ [17] and D-UNet [29] models. The lower errors across all metrics, at the same time at reduced computational complexity, highlight the superior performance of the VUDNet's hybrid architecture in providing precise, reliable, and efficient 3D surface reconstructions. The effective combination of VAE and U-Net components, along with the MLP ensemble, allows VUDNet to capture intricate surface details while maintaining overall structural coherence. The distinct separation in the VAE representation space indicates successful feature extraction, validating the model's ability to generalize across diverse and complex surface geometries. ## 6 Conclusions

In this paper, we introduced VUDNet, an innovative deep neural network designed for the end-to-end 3D reconstruction of specular free-form surfaces using single-shot deflectometry. Our approach uniquely combines the strengths of both generative and discriminative models by integrating a Variational Autoencoder (VAE) and a modified U-Net. This hybrid architecture excels in depth estimation and detail refinement, providing high-fidelity reconstructions that surpass the performance of existing methods.