# Toward Robust Neural Reconstruction from Sparse Point Sets

Amine Ouasfi, Shubhendu Jena, Eric Marchand, Adnane Boukhayma

► To cite this version:

Amine Ouasfi, Shubhendu Jena, Eric Marchand, Adnane Boukhayma. Toward Robust Neural Reconstruction from Sparse Point Sets. CVPR 2025 - IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun 2025, Nashville, United States. pp.6552-6562, 10.1109/cvpr52734.2025.00614. hal-05006233

HAL Id: hal-05006233

https://hal.science/hal-05006233v1

Submitted on 26 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Toward Robust Neural Reconstruction from Sparse Point Sets

Amine Ouasfi, Shubhendu Jena, Eric Marchand, Adnane Boukhayma
Inria, Univ. Rennes, CNRS, IRISA

## Abstract

We consider the challenging problem of learning Signed Distance Functions (SDF) from sparse and noisy 3D point clouds. In contrast to recent methods that depend on smoothness priors, our method, rooted in a distributionally robust optimization (DRO) framework, incorporates a regularization term that leverages samples from the uncertainty regions of the model to improve the learned SDFs. Thanks to tractable dual formulations, we show that this framework enables a stable and efficient optimization of SDFs in the absence of ground truth supervision. Using a variety of synthetic and real data evaluations from different modalities, we show that our DRO based learning framework can improve SDF learning with respect to baselines and the state-of-the-art methods.

## 1. Introduction

3D reconstruction from point clouds remains a longstanding problem at the intersection of computer vision, graphics, and machine learning. While traditional optimization techniques like Poisson Reconstruction [28, 37] and Moving Least Squares [27] perform well on dense, clean point clouds with accurate normal estimations, recent deep learning-based approaches offer improved robustness, especially when handling noisy or sparse inputs. These methods often eliminate the need for normal data. Many existing approaches rely on deep priors learned from large, fully labeled 3D datasets such as ShapeNet [18], but this strategy requires expensive training, and the resulting models are still susceptible to generalization issues when exposed to out-of-distribution data—whether due to changes in input density or domain shifts, as noted by [20, 64]. Indeed, as demonstrated in Tab. 2, our unsupervised approach outperforms supervised generalizable models when tested on data that is sparser and diverges from the training data. This highlights the importance of developing learning frameworks that can ensure robust reconstruction under these challenging conditions.

Recent work [62] shows that strategies that can successfully recover SDF representations from dense point clouds, such as Neural-Pull (NP) [52], often struggle when the point

cloud is sparse and noisy due to overfitting. As a consequence, the extracted shapes have missing parts and hallucinations (cf. Fig. 4, Fig. 1). Instead of relying on smoothness priors, [62] shifts the focus on how training distributions of spatial queries affect the performance of the SDF network. It introduces a special case of distributionally robust optimization (DRO) [71, 80] for SDF learning. Within this framework, the network is trained by considering the worst-case distribution in terms of the loss function in a neighborhood around the observed training distribution. To tackle the challenging task of finding worst-case distributions, the training strategy proposed by [62] that we dub here NAP for Neural Adversarial Pull, relies on a first-order taylor approximation of the loss to find query-pointwise adversarial samples that are used to regularize the training. These are defined as query points that maximize the loss. In this paper, instead of relying on pointwise adversaries, we leverage recent advances in DRO literature to explore a tractable formulation

<figure><img src="image_2.png" /><figcaption>Figure 1. We fit a neural SDF to a sparse noisy point cloud solely, using a distributionally robust loss function. Compared to the state-of-the-art, our method provides more faithful and robust reconstructions, as can be seen in these detailed and thin structures of ShapeNet [18] objects.</figcaption></figure>
for finding actual worst-case distributions for the first time in
the context of reconstruction from point cloud. Our solution
also proves to be more resilient to noise in the sparse input
setting.

One key design choice that determines the type and the
level of input noise that can be mitigated as well as the
tractability of the problem is the uncertainty set. That is
the set of distributions where the worst-case distribution can
be found. This is usually defined as a neighborhood of the
initial training distribution. To measure the distance between
distributions, various metrics have been explored in DRO
literature, including f-divergence [7, 58, 60], alongside the
Wasserstein distance [10, 59]. The latter has demonstrated
notable advantages in terms of efficiency and simplicity,
in addition to being widely adopted in computer vision and
graphics downstream applications [68, 73, 77, 78], as it takes
into account the geometry of the sample space, in contrast to
other metrics.

In order to learn a neural SDF from a sparse noisy point
cloud within a DRO framework, we proceed in this work
as follows. • We first present a tractable implementation
for this problem (SDF WDRO) benefiting from the dual
reformulation [10] of the DRO problem with the Wasser-
stein distribution metric [10, 14, 59, 75]. We build on NP
[52], but instead of using their predefined empirical spatial
query distribution (sampling normally around each of the
input points), we rely on queries from the worst-case distri-
bution in the Wasserstein ball around the empirical distribu-
tion. While this reduces overfitting and leads to more robust
reconstructions thanks to using more informative samples
throughout training instead of overfitting on easy ones, this
improvement comes at the cost of additional training time
compared to the NP baseline as shown in Fig. 7. • Further-
more, by interpreting the Wasserstein distance computation
as a mass transportation problem, recent advances in Opti-
mal Transport shows that it is possible to obtain theoretically
grounded approximations by regularizing the original mass
transportation problem with a relative entropy penalty on
the transport plan (e.g. [23]). The resulting distance is re-
ferred to as Sinkhorn distance. Thus, we show subsequently
that substituting the Wasserstein distance with the Sinkhorn
one in our SDF DRO problem results in a computationally
efficient dual formulation [81] that significantly improves
the convergence time of our first baseline SDF WDRO. The
training algorithm of the resulting SDF SDRO is outlined
in Algorithm 1. Thanks to the entropic regularization, SDF
SDRO produces more diffused spatial adversaries by smooth-
ing the worst-case distribution [5, 9, 81]. As a result, errors
in the SDF approximation are better distributed across the
shape, improving overall performance.

Through extensive quantitative and qualitative evaluation
under several real and synthetic benchmarks for object, non
rigid and scene level shape reconstruction, our results show

that our final method (SDF SDRO) outperforms SDF WDRO,
the baseline NP, as well as the most relevant competition,
notably the current state-of-the-art in unsupervised learning
of SDFs from sparse point cloud, such as NTPS [20], NAP
[62] and SparseOcc [65].

**Summary of intuition and contribution** We understand the approach proposed in [62] as a means of distributing the SDF approximation errors evenly throughout the 3D shape. Point cloud low-density and noisy areas are where this SDF error tends to concentrate. While in [62] the query points are independently perturbed within a local radius, our key idea is to construct a distribution of the most challenging query samples around the shape in terms of the loss function by “perturbing” the initial distribution of query points. The cost of this perturbation is controlled globally through an optimal transport distance. Minimizing the expected loss over this distribution flattens the landscape of the loss spatially, ensuring that the implicit model behaves consistently in the 3D space. As demonstrated by [79], not only does this generalize the approach proposed in [62], but also provides stronger adversaries that can be used to regularize the training, which justifies our superior results.

## 2. Related Work

**Reconstruction from Point Clouds** Traditional methods for reconstructing shapes include combinatorical techniques that divide the input point cloud into parts, such as using alpha shapes [8], Voronoi diagrams [2], or triangulation [16, 49, 72]. An alternative approach, is to define implicit functions, whose zero level set represents the target shape, using the input samples. This can be achieved by incorporating global smoothing priors [45, 64, 85, 86], such as radial basis functions [15] and Gaussian kernel fitting [74], or local smoothing priors like moving least squares [27, 40, 50, 55]. A different approach consists of solving a Poisson equation with boundary conditions [37]. In recent years, there has been a shift towards representing these implicit functions using deep neural networks, with parameters learned via gradient descent, either in a supervised (e.g. [12, 22, 31, 46, 61, 63, 69, 70, 86]) or unsupervised manner. These implicit representations [56, 67] aleviate many of the shortcomings of explicit ones (e.g. meshes [33, 36, 83] and point clouds [1, 24, 38]) in modelling shape, radiance and light fields (e.g. [17, 32, 34, 42, 43, 57, 84, 88]), as they allow to model arbitrary topologies at virtually infinite resolution.

**Unsupervised Implicit Neural Reconstruction** A neural network is used to fit a single point cloud without additional supervision in this setting. Regularizations, such as the spatial gradient constraint based on the Eikonal equation proposed by Gropp et al. [26], the spatial Laplacian constraint introduced in [6], and Lipschitz regularization on the network [48] are used to constrain the learned SDF,
<figure><img src="image_3.png" /><figcaption>Figure 2. We learn a neural SDF $f_{\theta}$ from a point cloud (black dots) by minimizing the error between projection of spatial queries {$q$} on the level set of the field (gray curve) and their nearest input point $p$. Instead of learning with a standard predefined distribution of queries $Q$, we optimize for the worst-case query distribution $Q'$ within a ball of distributions around $Q$.</figcaption></figure>

leading to performance improvement. [76] introduced periodic activations. [47] shows that an occupancy function can be learned such that its log transform converges to a distance function. [3] learns SDF from unsigned distances, with normal supervision on the spatial gradient of the function [4]. [52] express the nearest point on the surface as a function of the neural signed distance and its gradient. Several state-of-the-art reconstruction methods (e.g. [19, 20, 29, 52–54, 62, 89]) build on this strategy. Self-supervised local priors are used to handle very sparse inputs [53] or enhance generalization [54]. [13] proposed to learn an occupancy function by assuming that needle end points near the surface are statistically located on opposite sides of the surface. [85] solved a kernel ridge regression problem using points and their normals. [70] proposed a differentiable Poisson solving layer to efficiently obtain an indicator function grid from predicted normals. [41] learns an implicit field using Octree-based labeling as a guiding mechanism. [20] provides additional coarse surface supervision to the shape network using a learned surface parametrization. However, when the input is sparse and noisy, most of the methods mentioned above continue to face challenges in generating accurate reconstructions due to insufficient supervision. [65] learns an occupancy function by sampling from its uncertainty field and stabilizes the optimization by biasing the occupancy function towards minimal entropy fields. [62] augments the training with adversarial samples around the input point cloud. Differently from this literature, we explore here a new paradigm for learning unsupervised neural SDFs for the first time, namely through DRO with Wasserstein uncertainty sets.

## 3. Method

Let $\Xi$ be a subset of $\mathbb{R}^3$, and let $\mathcal{M}(\Xi)$, and $\mathcal{P}(\Xi)$ represent the set of measures and the set of probability measures on $\Xi$, respectively. Given a noisy, sparse unoriented point cloud

$\mathbf{P} \subset \Xi^{N_p}$, our objective is to reconstruct a corresponding watertight 3D shape reconstruction, i.e. the shape surface $\mathcal{S}$ that best fits the point cloud $\mathbf{P}$. To achieve this, we parameterise the shape function $f$ to be learned with an MLP $f_{\theta}$ that implicitly represents the signed distance field to the target shape $\mathcal{S}$. The reconstructed shape $\hat{\mathcal{S}}$ is represented as the zero level set of the SDF (signed distance function) $f_{\theta}: \hat{\mathcal{S}} = \{q \in \mathbb{R}^3 | f_{\theta}(q) = 0\}$. In practice, we use the Marching Cubes algorithm [51] to extract an explicit triangle mesh for $\hat{\mathcal{S}}$ by querying neural network $f_{\theta}$.

### 3.1. Learning an SDF by Query Neural Pulling

Neural Pull (NP) [52] approximates a signed distance function by pulling query points to their nearest input point cloud sample using the gradient of the SDF network. The normalized gradient is multiplied by the negated signed distance predicted by the network in order to pull both inside and outside queries to the surface. Query points are drawn from normal distributions centered at input samples {$p$}, with local standard deviations {$\sigma_p$} defined as the maximum euclidean distance to the $K$ nearest points to $p$ in $\mathbf{P}$:

$$ \mathcal{Q} := \bigcup_{p \in \mathbf{P}} \{q \sim \mathcal{N}(p, \sigma_p \mathbf{I}_3)\}, \quad (1) $$

The neural SDF $f_{\theta}$ is trained in [52] with empirical risk minimization (ERM) using the following objective:

$$ \mathcal{L}(\theta, q) = \|q - f_{\theta}(q)\|_{\|\nabla f_{\theta}(q)\|_{2}} - p_{\|\nabla f_{\theta}(q)\|_{2}}^{2}, \quad (2) $$

where $p$ is the closest point to $q$ in $\mathbf{P}$. By minimizing the expected loss under the empirical distribution $Q = \sum_{q \in \mathcal{Q}} \delta_q$ where $\delta_q$ is the dirac distribution or the unit mass on $q$, this objective ensures that the samples in $\mathbf{P}$ are on zero level set of the neural SDF $f_{\theta}$.

### 3.2. Neural SDF DRO

Inspired by [62], we focus on how to distribute the SDF approximation errors evenly throughout the shape as without regularization these errors tends to concentrate in low-density and noisy areas. NAP [62] introduces the following regularization term:

$$ \mathcal{L}_{\text{NAP}}(\theta, Q) = \mathbb{E}_{q \sim Q} \max_{\delta, \|\delta\|_{2} < \rho} \mathcal{L}(\theta, q + \delta), \quad (3) $$

Where the perturbation radius $\rho$ controls the distance to the spatial adversaries. Using a first order Taylor expansion of the loss, this problem is solved efficiently in [62] by deriving individual perturbations on the query points $q$ as follows:

$$ \hat{\delta} = \rho \frac{\nabla_q \mathcal{L}(\theta, q)}{\|\nabla_q \mathcal{L}(\theta, q)\|_2} \quad (4) $$

We consider the DRO problem introduced by NAP with Wasserstein uncertainty sets (Eq. (5)). We optimize the parameters of the SDF network $\theta$ under the worst-case expected
loss among a ball of distributions $Q'$ in this uncertainty set [10, 25],:

$$ 
\begin{gathered}
\inf_{\theta} \sup_{Q':W_c(Q',Q)<\epsilon q'\sim Q'} \mathbb{E} \mathcal{L}(\theta, q'), \\
\text{where } W_c(Q', Q) := \inf_{\gamma \in \Gamma(Q', Q)} \int c d\gamma.
\end{gathered}
\quad (5) $$

Here, $\epsilon > 0$ and $W_c$ denotes the optimal transport (OT) or a Wasserstein distance for a cost function $c$, defined as the infimum over the set $\Gamma(Q', Q)$ of couplings whose marginals are $Q'$ and $Q$. We refer the reader to the body of work in e.g. [10, 25] for more background.

**Neural SDF Wasserstein DRO (WDRO)** A tractable reformulation of the optimization problem defined in Equation (5) is made possible thanks to the following duality result [10]. For upper semi-continuous loss functions and non-negative lower semi-continuous costs satisfying $c(z, z') = 0$ iff $z = z'$, the optimization problem (Eq. (5)) is equivalent to:

$$ 
\begin{gathered}
\inf_{\theta, \lambda \ge 0} \{ \lambda \epsilon + \mathcal{L}_{\text{WDRO}}(\theta, Q) \}, \\
\text{where } \mathcal{L}_{\text{WDRO}}(\theta, Q) = \mathbb{E}_{q \sim Q} \left[ \sup_{q'} \{ \mathcal{L}(\theta, q') - \lambda c(q', q) \} \right].
\end{gathered}
\quad (6) $$

As shown in [14], solving the optimization above with a fixed dual variable $\lambda$ yields inferior results to the case where $\lambda$ is updated. In fact, optimizing $\lambda$ allows to capture global information when solving the outer minimization, whilst only local information (local worst-case spatial queries) is considered when minimizing $\mathcal{L}_{\text{WDRO}}$ solely.

Following [14], the optimization in Equation Eq. (6) can be carried as follows: Given the current model parameters $\theta$ and the dual variable $\lambda$, the worst-case spatial query $q'$ corresponding to a query $q$ drawn from the empirical distribution $Q$ can be obtained through a perturbation of $q$ followed by a few steps of iterative gradient ascent over $\mathcal{L}(\theta, q') - \lambda c(q', q)$. Subsequently, inspired by the Danskin's theorem, $\lambda$ can be updated accordingly $\lambda \leftarrow \lambda - \eta_{\lambda} \left( \epsilon - \frac{1}{N_b} \sum_{i=1}^{N_b} c(q'_i, q_i) \right)$, where $N_b$ represents the query batch size, and $\eta_{\lambda} > 0$ symbolizes a learning rate [14]. The current batch loss $\mathcal{L}_{\text{WDRO}}$ can then be backpropagated. We provide an Algorithm in supplemental material recapitulating this training.

While NAP consists of a hard-ball projection with locally adaptive radii, WDRO samples from the worst case distribution around the shape, (Equation Eq. (6)) through a soft-ball projection controlled by the parameter $\lambda$ that is adjusted throughout the training. The $\lambda$ update rule ensures that it grows when the worst-case sample distance from the initial

queries exceeds the Wasserstein ball radius $\epsilon$. While this approach provides promising results, it suffers from rather slow convergence, as shown in Figure Fig. 7. Furthermore, because our nominal distribution $Q$ is finitely supported, the worst-case distribution generated with WDRO is proven to be a discrete distribution [25], even while the underlying actual distribution is continuous. As pointed out in [81], this questions whether WDRO hedges the right family of distributions or generates too conservative solutions. In the next section, we show how these limitations can be addressed by taking inspiration from recent advances in Optimal Transport.

**Neural SDF Wasserstein DRO with entropic regularization (SDRO)** One key technical aspect underpinning the recent achievements of Optimal Transport in various applications lies in the use of regularization, particularly entropic regularization [5]. This approach has paved the way for efficient computational methodologies (e.g. [23]) to obtain theoretically-grounded approximations of Wasserstein distances. Building upon these advancements, recent work [5, 81] extend the framework of Wasserstein Distributionally Robust Optimization with entropic regularization by substituting the Wasserstein distance in Equation Eq. (5) with the Sinkhorn distance [81].

For $P, Q \in \mathcal{P}(\Xi)$, the Sinkhorn distance is defined as:

$$ W_{\rho}(P, Q) = \inf_{\gamma \in \Gamma(P, Q)} \left\{ \mathbb{E}_{(x,y) \sim \gamma} [c(x, y)] + \rho H(\gamma | \mu \otimes \nu) \right\}, \quad (7) $$

where $\rho \ge 0$ is a regularization parameter. $\mu$ and $\nu$ are two reference measures in $\mathcal{M}(\Xi)$ such that $P$ and $Q$ are absolutely continuous w.r.t. to $\mu$ and $\nu$ respectively, $H(\gamma | \mu \otimes \nu)$ denotes the relative entropy of $\gamma$ with respect to the product measure $\mu \otimes \nu$:

$$ H(\gamma | \mu \otimes \nu) = \mathbb{E}_{(x,y) \sim \gamma} \left[ \log \left( \frac{d\gamma(x, y)}{d\mu(x)d\nu(y)} \right) \right], \quad (8) $$

where $\frac{d\gamma(x,y)}{d\mu(x)d\nu(y)}$ stands for the density ratio of $\gamma$ with respect to $\mu \otimes \nu$ evaluated at $(x, y)$.

Compared to the Wasserstein distance, Sinkhorn distance regularizes the original mass transportation problem with relative entropy penalty on the transport plan. The choice of the reference measures $\mu$ and $\nu$ acts as a prior on the DRO problem. Following [81], we fix $\mu$ as our empirical distribution $Q$ and $\nu$ as the Lebesgue measure. Consequently, optimization problem in Equation Eq. (5) with the Sinkhorn distance admits the following dual form:

$$ \inf_{\theta, \lambda \ge 0} \left\{ \lambda \bar{\epsilon} + \lambda \rho \mathbb{E}_{q \sim Q} \left[ \log \mathbb{E}_{q' \sim Q_{q, \rho}} \left[ e^{\mathcal{L}(\theta, q') / (\lambda \rho)} \right] \right] \right\}, \quad (9) $$

where $\bar{\epsilon}$ is a constant that depends on $\rho$ and $\epsilon$ ([81]). Additionally, distribution $Q_{q, \rho}$ is defined through:

$$ d\mathbb{Q}_{x, \rho}(z) := \frac{e^{-c(x, z)/\rho}}{\mathbb{E}_{u \sim \nu} [e^{-c(x, u)/\rho}]} d\nu(z). \quad (10) $$
As discussed in [81], optimizing $\lambda$ within problem Eq. (9) leads to instability. Hence, for a given fixed $\lambda > 0$, optimization Eq. (9) can be carried practically by sampling a set of $N_s$ samples $q' \sim \mathbb{Q}_{q,\rho}$ for each query $q$, then backpropagating the following distributionally robust loss:

$$ \mathcal{L}_{\text{SDRO}}(\theta, Q) = \lambda \rho \mathbb{E}_{q \sim Q} \left[ \log \mathbb{E}_{q' \sim \mathbb{Q}_{q,\rho}} \left[ e^{\mathcal{L}(\theta, q')/(\lambda \rho)} \right] \right]. \quad (11) $$

Algorithm 1 summarizes the training of our SDRO based method.

### 3.3. Training Objective

Similar to [62] we train using the strategy of [44] which combines the original objective and the distributionally robust one:

$$ \mathcal{L}(\theta, q) = \frac{1}{2\lambda_1} \mathcal{L}(\theta, q) + \frac{1}{2\lambda_2} \mathcal{L}_{\text{DRO}}(\theta, q) + \ln(1+\lambda_1) + \ln(1+\lambda_2). \quad (12) $$

where $\lambda_1$ and $\lambda_2$ are learnable weights and $\mathcal{L}_{\text{SDRO}}$ or $\mathcal{L}_{\text{WDRO}}$ is either $\mathcal{L}_{\text{SDRO}}$ or $\mathcal{L}_{\text{WDRO}}$.

**Algorithm 1** The training procedure of our method with SDRO.

**Input:** Point cloud $\mathbf{P}$, learning rate $\alpha$, number of iterations $N_{it}$, batch size $N_b$.

SDRO hyperparameters: $\rho$, $\lambda$, $N_s$.

**Output:** Optimal parameters $\theta^*$.

Compute local st. devs. $\{\sigma_p\}$ ($\sigma_p = \max_{t \in K_{nn}(p, \mathbf{P})} ||t - p||_2$). $\Omega \leftarrow \text{sample}(\mathbf{P}, \{\sigma_p\})$ (Equ. Eq. (1)).

Compute nearest points in $\mathbf{P}$ for all samples in $\in \Omega$.

Initialize $\lambda_1 = \lambda_2 = 1$.

**for** $N_{it}$ **times** **do**

Sample $N_b$ query points $\{q, q \sim Q\}$.

For each $q$, sample $N_s$ points $\{q', q' \sim \mathbb{Q}_{q,\rho}\}$ (Equ.Eq. (10)).

Compute SDRO losses $\{\mathcal{L}_{\text{SDRO}}(\theta, q)\}$ (Equ. Eq. (11)).

Compute combined losses $\{\mathcal{L}(\theta, q)\}$ (Equ. Eq. (12)).

$$ (\theta, \lambda_1, \lambda_2) \leftarrow (\theta, \lambda_1, \lambda_2) - \alpha \nabla_{\theta, \lambda_1, \lambda_2} \sum_q \mathcal{L}(\theta, q) $$

**end for**

## 4. Results

To assess the performance of our approach, we conducted evaluations on widely used 3D reconstruction benchmarks. In line with previous research, we assess the accuracy of 3D meshes generated by our MLPs after convergence. We benchmark our method against SOTA methods for sparse unsupervised reconstruction, including NP [52], NAP [62], SparseOcc [65], and NTPS [20]. Further, we extend our comparisons to include approaches such as SAP [70], DIGS [6], NDrop [13], and NSpline [85], as well as hybrid methods that incorporate both explicit and implicit representations, such as OG-INR [41] and GridPull (GP) [21]. We also

evaluate our approach relative to SOTA supervised methods. These include robust, generalizable feed-forward methods such as POCO [12], CONet [69], and NKSR [30], in addition to prior-based optimization strategies tailored for sparse data, like On-Surf [53]. Following the settings adopted by NAP, our experiments employ point clouds with $N_p = 1024$ points.

### 4.1. Metrics

We evaluate our method using standard metrics commonly employed for 3D reconstruction tasks. Specifically, we compute the L1 Chamfer Distance (CD1) and L2 Chamfer Distance (CD2), both scaled by a factor of $10^2$. Additionally, we compute the F-Score (FS), based on Euclidean distance, and the Normal Consistency (NC) between the meshes generated by our approach and the ground-truth. Detailed mathematical formulations for these metrics are provided in the supplementary material.

### 4.2. Datasets and input definitions

We evaluate our approach on several benchmark datasets representing a variety of 3D data.

ShapeNet [18] provides a diverse set of synthetic 3D models across 13 distinct categories. In line with previous work, we report results on the Table, Chair, and Lamp classes, utilizing the train/test splits specified in [85]. For each mesh, we generate noisy input point clouds by sampling 1024 points and adding Gaussian noise with a standard deviation of 0.005, as done in [12, 62, 69]. Faust [11] includes real 3D scans of 10 different human body identities, each captured in 10 distinct poses. We sample 1024 points from these scans to serve as input for our method. 3D Scene [90] contains large-scale, real-world scenes acquired with a handheld commodity range sensor. We follow the protocols in [20, 35, 52, 62] to generate sparse point clouds with a density of 100 points per m³ and present results for several scenes, including Burghers, Copyroom, Lounge, Stonewall, and Totempole. SemanticPOSS [66] contains LiDAR data collected from 6 sequences of road scenes. Each scan captures a 51.2m range ahead, 25.6m on either side, and 6.4m vertically. We provide qualitative results from each of these sequences. Finally, we extend our evaluation to challenging scenes from the BlendedMVS [87] dataset, which is a multi-view stereo dataset, with scenes consisting of architecture, sculptures, and small objects with complex backgrounds as well as Tanks and Temples [39] dataset, which consists of large-scale indoor and outdoor scenes, with high-resolution images captured by a handheld monocular RGB camera. For both the datasets, sparse views are used with VGGSFM [82] to generate sparse, noisy input point clouds for our setup.

### 4.3. Implementation details

Our MLP model, denoted as $f_\theta$, follows the architecture outlined in Neural Pull (NP) [52]. We train the model using the Adam optimizer with a batch size of $N_b = 5000$ and set
<figure><img src="image_4.png" /><figcaption>Figure 3. Faust [11] reconstructions. **CONet and POCO use data priors.**</figcaption></figure>

$K = 51$ to compute the local standard deviations $\sigma_p$, in line with NP. Training is performed on a single NVIDIA RTX A6000 GPU. For fair and practical comparison, we select the optimal evaluation epoch for each method based on Chamfer distance between the reconstructed and input point clouds, choosing the epoch that minimizes this metric. We also conduct a hyperparameter search on the SRB benchmark to determine the best parameters for our method.

For the Wasserstein Robust DRO (WRDO) approach, we perform two gradient ascent steps ($N_{it}^{wdro} = 2$) with a learning rate $\alpha_{wdro} = 10^{-3}$ in the inner loop. The dual variable is initialized as $\lambda = 80$, and the Wasserstein ball radius is fixed at $\epsilon = 10^{-4}$. For Standard DRO (SDRO), we use $N_s = 5$ samples per query point $q \sim Q$, with $\lambda = 20$ in our experiments. The transport cost is defined as $c(\cdot, \cdot) = \frac{1}{2}\|\cdot - \cdot\|^2$, implying that sampling from $Q_{q,\rho}$ follows a Gaussian distribution $N(q, \rho I_3)$.

## 4.4. Object level reconstruction

We evaluate the reconstruction of ShapeNet [18] objects from sparse and noisy point clouds. A quantitative comparison is presented in Tab. 1, while Fig. 1 offers a qualitative assessment of the results. Our approach, based on Wasserstein Robust DRO (WDRO), outperforms existing methods in terms of reconstruction accuracy, as measured by $CD_1$ and $CD_2$. When combined with the SDRO loss, our method further improves across all evaluation metrics. This is reflected in the visually enhanced reconstruction quality, which demonstrates superior detail and structure preservation. While NTPS produces generally acceptable coarse reconstructions, its use of thin plate spline smoothing limits its ability to capture finer details. NAP and SparseOcc are able to produce better reconstructions but struggle under high levels of noise. Additionally, we find that OG-INR struggles to achieve satisfactory convergence under sparse and noisy conditions, despite its success in denser scenarios aided by Octree-based sign fields.

<table><caption>Table 1. ShapeNet [18] reconstructions from sparse noisy unoriented point clouds.</caption><thead><tr><th></th><th>CD1</th><th>CD2</th><th>NC</th><th>FS</th></tr></thead><tbody><tr><td>SPSR [37]</td><td>2.34</td><td>0.224</td><td>0.74</td><td>0.50</td></tr><tr><td>OG-INR [41]</td><td>1.36</td><td>0.051</td><td>0.55</td><td>0.55</td></tr><tr><td>NP [52]</td><td>1.16</td><td>0.074</td><td>0.84</td><td>0.75</td></tr><tr><td>GP [21]</td><td>1.07</td><td>0.032</td><td>0.70</td><td>0.74</td></tr><tr><td>NTPS [20]</td><td>1.11</td><td>0.067</td><td>0.88</td><td>0.74</td></tr><tr><td>NAP [62]</td><td>0.76</td><td>0.020</td><td>0.87</td><td>0.83</td></tr><tr><td>SparseOcc [65]</td><td>0.76</td><td>0.020</td><td>0.88</td><td>0.83</td></tr><tr><td>Ours (WDRO)</td><td>0.77</td><td>0.015</td><td>0.87</td><td>0.83</td></tr><tr><td>Ours (SDRO)</td><td>0.63</td><td>0.012</td><td>0.90</td><td>0.86</td></tr></tbody></table>

<table><caption>Table 2. Faust [11] reconstructions from sparse noisy unoriented point clouds. **POCO, CONet, On-Surf and NKSR use data priors.**</caption><thead><tr><th></th><th>CD1</th><th>CD2</th><th>NC</th><th>FS</th></tr></thead><tbody><tr><td>POCO [12]</td><td>0.308</td><td>0.002</td><td>0.934</td><td>0.981</td></tr><tr><td>CONet [69]</td><td>1.260</td><td>0.048</td><td>0.829</td><td>0.599</td></tr><tr><td>On-Surf [53]</td><td>0.584</td><td>0.012</td><td>0.936</td><td>0.915</td></tr><tr><td>NKSR [30]</td><td>0.274</td><td>0.002</td><td>0.945</td><td>0.981</td></tr><tr><td>SPSR [37]</td><td>0.751</td><td>0.028</td><td>0.871</td><td>0.839</td></tr><tr><td>GP [21]</td><td>0.495</td><td>0.005</td><td>0.887</td><td>0.945</td></tr><tr><td>NTPS [20]</td><td>0.737</td><td>0.015</td><td>0.943</td><td>0.844</td></tr><tr><td>NAP [62]</td><td>0.220</td><td>0.001</td><td>0.956</td><td>0.981</td></tr><tr><td>SparseOcc [65]</td><td>0.260</td><td>0.002</td><td>0.952</td><td>0.974</td></tr><tr><td>Ours (WDRO)</td><td>0.255</td><td>0.002</td><td>0.953</td><td>0.977</td></tr><tr><td>Ours (SDRO)</td><td>0.251</td><td>0.002</td><td>0.955</td><td>0.979</td></tr></tbody></table>

This is reflected in the visually enhanced reconstruction quality, which demonstrates superior detail and structure preservation. While NTPS produces generally acceptable coarse reconstructions, its use of thin plate spline smoothing limits its ability to capture finer details. NAP and SparseOcc are able to produce better reconstructions but struggle under high levels of noise. Additionally, we find that OG-INR struggles to achieve satisfactory convergence under sparse and noisy conditions, despite its success in denser scenarios aided by Octree-based sign fields.

## 4.5. Real articulated shape reconstruction

We evaluate the reconstruction of human shapes from the Faust dataset [11] using sparse, noisy point clouds. Quantitative and qualitative comparisons with competing methods are provided in Tab. 2 and Fig. 3, respectively. Across all metrics, our distributionally robust training procedures demonstrate superior performance, with SDRO achieving slightly better accuracy and faster convergence than WDRO. Visually, our reconstructions exhibit a significant improvement, especially in capturing finer details at body extremities, which pose challenges due to sparse input data and can lead to ambiguous shape predictions, similar to the fine structures observed in ShapeNet experiments. Notably, NAP outperforms our approach in this setting, and our method is
<table><caption>Table 3. 3D Scene [90] reconstructions from sparse point clouds.</caption><thead><tr><th rowspan="2"></th><th colspan="3">Burghers</th><th colspan="3">Copyroom</th><th colspan="3">Lounge</th><th colspan="3">Stonewall</th><th colspan="3">Totemple</th><th colspan="3">Mean</th></tr><tr><th>CD1</th><th>CD2</th><th>NC</th><th>CD1</th><th>CD2</th><th>NC</th><th>CD1</th><th>CD2</th><th>NC</th><th>CD1</th><th>CD2</th><th>NC</th><th>CD1</th><th>CD2</th><th>NC</th><th>CD1</th><th>CD2</th><th>NC</th></tr></thead><tbody><tr><td>SPSR [37]</td><td>0.178</td><td>0.2050</td><td>0.874</td><td>0.225</td><td>0.2860</td><td>0.861</td><td>0.280</td><td>0.3650</td><td>0.869</td><td>0.300</td><td>0.4800</td><td>0.866</td><td>0.588</td><td>1.6730</td><td>0.879</td><td>0.314</td><td>0.6024</td><td>0.870</td></tr><tr><td>NDrop [13]</td><td>0.200</td><td>0.1140</td><td>0.825</td><td>0.168</td><td>0.0630</td><td>0.696</td><td>0.156</td><td>0.0500</td><td>0.663</td><td>0.150</td><td>0.0810</td><td>0.815</td><td>0.203</td><td>0.1390</td><td>0.844</td><td>0.175</td><td>0.0894</td><td>0.769</td></tr><tr><td>NP [52]</td><td>0.064</td><td>0.0080</td><td>0.898</td><td>0.049</td><td>0.0050</td><td>0.828</td><td>0.133</td><td>0.0380</td><td>0.847</td><td>0.060</td><td>0.0050</td><td>0.910</td><td>0.178</td><td>0.0240</td><td>0.908</td><td>0.097</td><td>0.0160</td><td>0.878</td></tr><tr><td>SAP [70]</td><td>0.153</td><td>0.1010</td><td>0.807</td><td>0.053</td><td>0.0090</td><td>0.771</td><td>0.134</td><td>0.0330</td><td>0.813</td><td>0.070</td><td>0.0070</td><td>0.867</td><td>0.474</td><td>0.3820</td><td>0.725</td><td>0.151</td><td>0.1064</td><td>0.797</td></tr><tr><td>NSpline [85]</td><td>0.135</td><td>0.1230</td><td>0.891</td><td>0.056</td><td>0.0230</td><td>0.855</td><td>0.063</td><td>0.0390</td><td>0.827</td><td>0.124</td><td>0.0910</td><td>0.897</td><td>0.378</td><td>0.7680</td><td>0.892</td><td>0.151</td><td>0.2088</td><td>0.872</td></tr><tr><td>NTPS [20]</td><td>0.055</td><td>0.0050</td><td>0.909</td><td>0.045</td><td>0.0030</td><td>0.892</td><td>0.129</td><td>0.0220</td><td>0.872</td><td>0.054</td><td>0.0040</td><td>0.939</td><td>0.103</td><td>0.0170</td><td>0.935</td><td>0.077</td><td>0.0102</td><td>0.897</td></tr><tr><td>NAP [62]</td><td>0.051</td><td>0.006</td><td>0.881</td><td>0.037</td><td>0.002</td><td>0.833</td><td>0.044</td><td>0.011</td><td>0.862</td><td>0.035</td><td>0.003</td><td>0.912</td><td>0.042</td><td>0.002</td><td>0.925</td><td>0.041</td><td>0.004</td><td>0.881</td></tr><tr><td>SparseOcc [65]</td><td>0.022</td><td>0.001</td><td>0.871</td><td>0.041</td><td>0.012</td><td>0.812</td><td>0.021</td><td>0.001</td><td>0.870</td><td>0.028</td><td>0.003</td><td>0.931</td><td>0.026</td><td>0.001</td><td>0.936</td><td>0.027</td><td>0.003</td><td>0.886</td></tr><tr><td>Ours (WDRO)</td><td>0.014</td><td>0.0006</td><td>0.871</td><td>0.028</td><td>0.0036</td><td>0.820</td><td>0.038</td><td>0.0051</td><td>0.803</td><td>0.019</td><td>0.0005</td><td>0.930</td><td>0.009</td><td>0.0003</td><td>0.936</td><td>0.022</td><td>0.0020</td><td>0.872</td></tr><tr><td>Ours (SDRO)</td><td>0.015</td><td>0.0006</td><td>0.873</td><td>0.021</td><td>0.0017</td><td>0.823</td><td>0.027</td><td>0.0032</td><td>0.842</td><td>0.021</td><td>0.0006</td><td>0.932</td><td>0.020</td><td>0.0005</td><td>0.934</td><td>0.020</td><td>0.0013</td><td>0.881</td></tr></tbody></table>

<figure><img src="image_5.png" /><figcaption>Figure 4. 3D Scene [90] reconstructions from sparse unoriented point clouds.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Figure 5. SemanticPOSS [66] reconstruction from road scene LiDAR data.</figcaption></figure>

comparable to SparseOcc, as these methods tend to perform better under low noise conditions, whereas ours is specifically designed for robustness under higher noise levels. In contrast, NTPS reconstructions tend to be coarser with fewer details. It is also worth mentioning that several generalizable methods, particularly those trained on ShapeNet (seen in the upper section of the table), show limited effectiveness in this experiment.

## 4.6. Real scene level reconstruction

We report reconstruction results on the 3D Scene dataset [90] from sparse point clouds, following in [20]. Comparative results for state-of-the-art methods, including NTPS, NP, SAP, NDrop, and NSpline, are obtained from NTPS, while the performances of NAP and SparseOcc are cited from their respective publications and summarized in Tab. 3. Our method demonstrates superior performance in this setting, attributable to our loss function's capacity to handle high levels of noise, unlike NAP. Qualitative comparisons with our NP baseline and SPSR are shown in Fig. 4, where specific regions highlighted by colored boxes illustrate areas where our approach achieves notably high detail and reconstruction fidelity.

Additionally, we conduct qualitative comparisons on BlendedMVS [87] and large-scale scenes from the Tanks & Temples dataset [39] using sparse views. VGGSfM [82], a recent state-of-the-art fully differentiable structure-from-

motion pipeline, is used to generate the sparse point cloud inputs for this experiment. Although VGGSfM effectively generates point clouds by triangulating 2D point trajectories and learned camera poses, the sparse input views result in sparse and noisy point clouds, making SDF-based reconstruction challenging. To illustrate the strength of our method, we compare 3 examples from each dataset against SparseOcc and NAP in Fig. 6, demonstrating sharper details, especially on large-scale scenes from Tanks & Temples, where other methods struggle due to noise in VGGSfM's point clouds.

To further evaluate the robustness of our method, we present reconstruction results on the SemanticPOSS dataset [66] and provide qualitative comparisons with SparseOcc, NAP, and NP in Fig. 5. The visualizations use the dataset's color-coded semantic segmentations, which were not utilized during training. Our approach achieves marked improvements in reconstruction quality, largely due to our DRO framework. In particular, elements such as cars, trees, and pedestrians are reconstructed with significantly greater detail and precision, whereas baseline methods often blend these object categories into indistinct forms. Addition-
<figure><img src="image_7.png" /><figcaption>Figure 6. Reconstructions from VGGSM point clouds of sparse views from BlendedMVS [87] and Tanks & Temples datasets [39].</figcaption></figure>

ally, our SDRO method is notably effective in preserving the broader scene structure. Although SparseOcc and NAP demonstrate solid performance under low-noise conditions, their accuracy deteriorates sharply under higher noise levels. Additional qualitative examples are provided in the supplementary materials.

## 5. Ablation studies

**Noise ablation** To examine the influence of input noise (displacement from the surface) and sparsity on our method's performance in comparison to the NP baseline, we conduct an ablation study across different noise levels, as shown in Tab. 4. The results consistently indicate that our method outperforms the baseline at various noise levels. This suggests that our distributionally robust training approach is effective in reducing noise due to both sparse inputs and displacement. Additionally, under high-noise conditions, our method demonstrates superior performance over both NAP and SparseOcc.

**Training time** To evaluate the computational efficiency of our approach, we present a performance analysis over training time in Fig. 7, comparing our DRO methods with the NP baseline. The plot demonstrates the performance attained after specific training durations. Notably, WDRO achieves

<figure><img src="image_8.png" /><figcaption>Figure 7. Performance over training time on Shapenet [18] class Tables.</figcaption></figure>

<table><caption>Table 4. Ablation of our method under varying levels of noise on Shapenet [18] class Tables.</caption><thead><tr><th rowspan="2"></th><th colspan="2">σ = 0.0</th><th colspan="2">σ = 0.005</th><th colspan="2">σ = 0.025</th></tr><tr><th>CD1</th><th>NC</th><th>CD1</th><th>NC</th><th>CD1</th><th>NC</th></tr></thead><tbody><tr><td>NP (baseline) [52]</td><td>0.73</td><td>0.906</td><td>1.07</td><td>0.847</td><td>2.45</td><td>0.668</td></tr><tr><td>NAP [62]</td><td>0.63</td><td>0.926</td><td>0.75</td><td>0.86</td><td>2.21</td><td>0.67</td></tr><tr><td>SparseOcc [65]</td><td>0.56</td><td>0.931</td><td>0.77</td><td>0.89</td><td>2.16</td><td>0.68</td></tr><tr><td>Ours(SDRO)</td><td>0.43</td><td>0.945</td><td>0.65</td><td>0.91</td><td>1.54</td><td>0.702</td></tr></tbody></table>

baseline performance after approximately 3 minutes and reaches its peak in 10 minutes. On the other hand, SDRO shows an advantage over the NP baseline after just 2 minutes of training, attaining optimal performance in under 6 minutes, matching the baseline's convergence time while outperforming both the baseline and WDRO. This observation underscores the computational advantages of using the Sinkhorn distance in our distributionally robust optimization formulation (Eq. (5)), as opposed to the Wasserstein distance. Further ablation results can be found in the supplementary material.

## 6. Limitations

Our method is able to significantly improve over recent state of the art methods such as NAP and SparseOcc in very challenging scenarios. However, when the input points are clean and dense, NAP can show competitive or even better performance as shown in Tab. 2 and in the density ablation in the supplementary material. An interesting direction for future research could be to explore strategies that combine local adaptive radii control in NAP with the global control on the worst-case distribution in SDRO. We aim to investigate this as part of our future work.

## 7. Conclusion

We have shown that regularizing implicit shape representation learning from sparse unoriented point clouds through distributionally robust optimization with wasserstein uncertainty sets can lead to superior reconstructions. We believe these new findings can usher in a new body of work incorporating distributional robustness in learning neural implicit functions, which in turn can potentially have a larger impact beyond the specific scope of this paper.
## References

[1] Kara-Ali Aliev, Artem Sevastopolsky, Maria Kolos, Dmitry Ulyanov, and Victor Lempitsky. Neural point-based graphics. In *Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXII 16*, 2020. 2

[2] Nina Amenta, Sunghee Choi, and Ravi Krishna Kolluri. The power crust, unions of balls, and the medial axis transform. *CG*, 2001. 2

[3] Matan Atzmon and Yaron Lipman. Sal: Sign agnostic learning of shapes from raw data. In *CVPR*, 2020. 3

[4] Matan Atzmon and Yaron Lipman. Sald: Sign agnostic learning with derivatives. In *ICML*, 2020. 3

[5] Waïss Azizian, Franck Iutzeler, and Jérôme Malick. Regularization for wasserstein distributionally robust optimization. *ESAIM: Control, Optimisation and Calculus of Variations*, 29:33, 2023. 2, 4

[6] Yizhak Ben-Shabat, Chamin Hewa Koneputugodage, and Stephen Gould. Digs: Divergence guided shape implicit neural representation for unoriented point clouds. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 19323–19332, 2022. 2, 5

[7] Aharon Ben-Tal, Dick Den Hertog, Anja De Waegenaere, Bertrand Melenberg, and Gijs Rennen. Robust solutions of optimization problems affected by uncertain probabilities. *Management Science*, 59(2):341–357, 2013. 2

[8] Fausto Bernardini, Joshua Mittleman, Holly Rushmeier, Claudio Silva, and Gabriel Taubin. The ball-pivoting algorithm for surface reconstruction. *TVCG*, 1999. 2

[9] Jose Blanchet and Yang Kang. Semi-supervised learning based on distributionally robust optimization, 2020. 2

[10] Jose Blanchet and Karthyek Murthy. Quantifying distributional model risk via optimal transport. *Mathematics of Operations Research*, 44(2):565–600, 2019. 2, 4

[11] Federica Bogo, Javier Romero, Matthew Loper, and Michael J. Black. FAUST: Dataset and evaluation for 3D mesh registration. In *CVPR*, 2014. 5, 6

[12] Alexandre Boulch and Renaud Marlet. Poco: Point convolution for surface reconstruction. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 6302–6314, 2022. 2, 5, 6

[13] Alexandre Boulch, Pierre-Alain Langlois, Gilles Puy, and Renaud Marlet. Needrop: Self-supervised shape representation from sparse point clouds using needle dropping. In *2021 International Conference on 3D Vision (3DV)*, pages 940–950. IEEE, 2021. 3, 5, 7

[14] Tuan Anh Bui, Trung Le, Quan Tran, He Zhao, and Dinh Phung. A unified wasserstein distributional robustness framework for adversarial training. *arXiv preprint arXiv:2202.13437*, 2022. 2, 4

[15] Jonathan C Carr, Richard K Beatson, Jon B Cherrie, Tim J Mitchell, W Richard Fright, Bruce C McCallum, and Tim R Evans. Reconstruction and representation of 3d objects with radial basis functions. In *SIGGRAPH*, 2001. 2

[16] Frédéric Cazals and Joachim Giesen. *Effective Computational Geometry for Curves and Surfaces*. 2006. 2

[17] Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al. Efficient geometry-aware 3d generative adversarial networks. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 1612–16133, 2022. 2

[18] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. *arXiv preprint arXiv:1512.03012*, 2015. 1, 5, 6, 8

[19] Chao Chen, Yu-Shen Liu, and Zhizhong Han. Latent partition implicit with surface codes for 3d representation. In *European Conference on Computer Vision (ECCV)*, 2022. 3

[20] Chao Chen, Zhizhong Han, and Yu-Shen Liu. Unsupervised inference of signed distance functions from single sparse point clouds without learning priors. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, 2023. 1, 2, 3, 5, 6, 7

[21] Chao Chen, Yu-Shen Liu, and Zhizhong Han. Gridpull: Towards scalability in learning implicit representations from 3d point clouds. In *Proceedings of the IEEE/CVF International Conference on Computer Vision*, pages 18322–18334, 2023. 5, 6

[22] Julian Chibane and Gerard Pons-Moll. Implicit feature networks for texture completion from partial 3d data. In *European Conference on Computer Vision*, pages 717–725. Springer, 2020. 2

[23] Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. *Advances in neural information processing systems*, 26, 2013. 2, 4

[24] Haoqiang Fan, Hao Su, and Leonidas J Guibas. A point set generation network for 3d object reconstruction from a single image. In *CVPR*, 2017. 2

[25] Rui Gao and Anton Kleywegt. Distributionally robust stochastic optimization with wasserstein distance. *Mathematics of Operations Research*, 48(2):603–655, 2023. 4

[26] Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. Implicit geometric regularization for learning shapes. In *ICML*, 2020. 2

[27] Gaël Guennebaud and Markus Gross. Algebraic point set surfaces. In *ACM siggraph 2007 papers*, pages 23–es. 2007. 1, 2

[28] Fei Hou, Chiyu Wang, Wencheng Wang, Hong Qin, Chen Qian, and Ying He. Iterative poisson surface reconstruction (ipsr) for unoriented points. *arXiv preprint arXiv:2209.09510*, 2022. 1

[29] Han Huang, Yulun Wu, Junsheng Zhou, Ge Gao, Ming Gu, and Yushen Liu. Neusurf: On-surface priors for neural surface reconstruction from sparse input views. In *AAAI*, 2024. 3

[30] Jiahui Huang, Zan Gojcic, Matan Atzmon, Or Litany, Sanja Fidler, and Francis Williams. Neural kernel surface reconstruction. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pages 4369–4379, 2023. 5, 6

[31] Jiahui Huang, Zan Gojcic, Matan Atzmon, Or Litany, Sanja Fidler, and Francis Williams. Neural kernel surface reconstruction. In *Proceedings of the IEEE/CVF Conference on*
*Computer Vision and Pattern Recognition*, pages 4369–4379, 2023. 2

[32] Ajay Jain, Ben Mildenhall, Jonathan T. Barron, Pieter Abbeel, and Ben Poole. Zero-shot text-guided object generation with dream fields. 2022. 2

[33] Shubhendu Jena, Franck Multon, and Adnane Boukhayma. Neural mesh-based graphics. In *European Conference on Computer Vision*, 2022. 2

[34] Shubhendu Jena, Franck Multon, and Adnane Boukhayma. Geotransfer: Generalizable few-shot multi-view reconstruction via transfer learning. In *European Conference on Computer Vision*, 2024. 2

[35] Chiyu Jiang, Avneesh Sud, Ameesh Makadia, Jingwei Huang, Matthias Nießner, Thomas Funkhouser, et al. Local implicit grid representations for 3d scenes. In *CVPR*, 2020. 5

[36] Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada. Neural 3d mesh renderer. In *CVPR*, 2018. 2

[37] Michael Kazhdan and Hugues Hoppe. Screened poisson surface reconstruction. *TOG*, 2013. 1, 2, 6, 7

[38] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. *ACM Transactions on Graphics*, 2023. 2

[39] Arno Knapitsch, Jaesik Park, Qian-Yi Zhou, and Vladlen Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. *ACM Transactions on Graphics (ToG)*, 36(4): 1–13, 2017. 5, 7, 8

[40] Ravikrishna Kolluri. Provably good moving least squares. *TALG*, 2008. 2

[41] Chamin Hewa Koneputugodage, Yizhak Ben-Shabat, and Stephen Gould. Octree guided unoriented surface reconstruction. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 16717–16726, 2023. 3, 5, 6

[42] Qian Li, Franck Multon, and Adnane Boukhayma. Learning generalizable light field networks from few images. In *ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)*, pages 1–5. IEEE, 2023. 2

[43] Qian Li, Franck Multon, and Adnane Boukhayma. Regularizing neural radiance fields from sparse rgb-d inputs. In *2023 IEEE International Conference on Image Processing (ICIP)*, pages 2320–2324. IEEE, 2023. 2

[44] Lukas Liebel and Marco Körner. Auxiliary tasks in multi-task learning. *arXiv preprint arXiv:1805.06334*, 2018. 5

[45] Siyou Lin, Dong Xiao, Zuoqiang Shi, and Bin Wang. Surface reconstruction from point clouds without normals by parametrizing the gauss formula. *ACM Transactions on Graphics*, 42(2):1–19, 2022. 2

[46] Stefan Lionar, Daniil Emtsev, Dusan Svilarkovic, and Songyou Peng. Dynamic plane convolutional occupancy networks. In *Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision*, pages 1829–1838, 2021. 2

[47] Yaron Lipman. Phase transitions, distance functions, and implicit neural representations. In *ICML*, 2021. 3

[48] Hsueh-Ti Derek Liu, Francis Williams, Alec Jacobson, Sanja Fidler, and Or Litany. Learning smooth neural functions via

lipschitz regularization. *arXiv preprint arXiv:2202.08345*, 2022. 2

[49] Minghua Liu, Xiaoshuai Zhang, and Hao Su. Meshing point clouds with predicted intrinsic-extrinsic ratio guidance. In *ECCV*, 2020. 2

[50] Shi-Lin Liu, Hao-Xiang Guo, Hao Pan, Peng-Shuai Wang, Xin Tong, and Yang Liu. Deep implicit moving least-squares functions for 3d reconstruction. In *CVPR*, 2021. 2

[51] William E Lorensen and Harvey E Cline. Marching cubes: A high resolution 3d surface construction algorithm. In *SIGGRAPH*, 1987. 3

[52] Baorui Ma, Zhizhong Han, Yu-Shen Liu, and Matthias Zwicker. Neural-pull: Learning signed distance functions from point clouds by learning to pull space onto surfaces. In *ICML*, 2021. 1, 2, 3, 5, 6, 7, 8

[53] Baorui Ma, Yu-Shen Liu, and Zhizhong Han. Reconstructing surfaces for sparse point clouds with on-surface priors. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 6315–6325, 2022. 3, 5, 6

[54] Baorui Ma, Yu-Shen Liu, Matthias Zwicker, and Zhizhong Han. Surface reconstruction from point clouds by learning predictive context priors. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 6326–6337, 2022. 3

[55] Corentin Mercier, Thibault Lescoat, Pierre Roussillon, Tamy Boubekeur, and Jean-Marc Thiery. Moving level-of-detail surfaces. *ACM Transactions on Graphics (TOG)*, 41(4):1–10, 2022. 2

[56] Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 4460–4470, 2019. 2

[57] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In *ECCV*, 2020. 2

[58] Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, Ken Nakae, and Shin Ishii. Distributional smoothing with virtual adversarial training. *arXiv preprint arXiv:1507.00677*, 2015. 2

[59] Peyman Mohajerin Esfahani and Daniel Kuhn. Data-driven distributionally robust optimization using the wasserstein metric: performance guarantees and tractable reformulations. *Mathematical Programming*, 171(1-2):115–166, 2018. 2

[60] Hongseok Namkoong and John C Duchi. Stochastic gradient methods for distributionally robust optimization with f-divergences. In *NIPS*, pages 2208–2216, 2016. 2

[61] Amine Ouasfi and Adnane Boukhayma. Few’zero level set’-shot learning of shape signed distance functions in feature space. In *ECCV*, 2022. 2

[62] Amine Ouasfi and Adnane Boukhayma. Few-shot unsupervised implicit neural shape representation learning with spatial adversaries. *arXiv preprint arXiv:2408.15114*, 2024. 1, 2, 3, 5, 6, 7, 8

[63] Amine Ouasfi and Adnane Boukhayma. Mixing-denoising generalizable occupancy networks. *3DV*, 2024. 2
[64] Amine Ouasfi and Adnane Boukhayma. Robustifying generalizable implicit shape networks with a tunable non-parametric model. *Advances in Neural Information Processing Systems*, 36, 2024. 1, 2

[65] Amine Ouasfi and Adnane Boukhayma. Unsupervised occupancy learning from sparse point cloud. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 21729–21739, 2024. 2, 3, 5, 6, 7, 8

[66] Yancheng Pan, Biao Gao, Jilin Mei, Sibo Geng, Chengkun Li, and Huijing Zhao. Semanticposs: A point cloud dataset with large quantity of dynamic instances. In *2020 IEEE Intelligent Vehicles Symposium (IV)*, pages 687–693, 2020. 5, 7

[67] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In *CVPR*, 2019. 2

[68] Ofir Pele and Michael Werman. A linear time histogram metric for improved sift matching. In *European Conference on Computer Vision*, pages 495–508, 2008. 2

[69] Songyou Peng, Michael Niemeyer, Lars Mescheder, Marc Pollefeys, and Andreas Geiger. Convolutional occupancy networks. In *European Conference on Computer Vision*, pages 523–540. Springer, 2020. 2, 5, 6

[70] Songyou Peng, Chiyu Jiang, Yiyi Liao, Michael Niemeyer, Marc Pollefeys, and Andreas Geiger. Shape as points: A differentiable poisson solver. *Advances in Neural Information Processing Systems*, 34:13032–13044, 2021. 2, 3, 5, 7

[71] Hamed Rahimian and Sanjay Mehrotra. Distributionally robust optimization: A review. *arXiv preprint arXiv:1908.05659*, 2019. 1

[72] Marie-Julie Rakotosaona, Noam Aigerman, Niloy Mitra, Maks Ovsjanikov, and Paul Guerrero. Differentiable surface triangulation. In *SIGGRAPH Asia*, 2021. 2

[73] Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. The earth mover’s distance as a metric for image retrieval. *International Journal of Computer Vision*, 40(2):99–121, 2000. 2

[74] Bernhard Schölkopf, Joachim Giesen, and Simon Spalinger. Kernel methods for implicit surface modeling. In *NeurIPS*, 2004. 2

[75] Aman Sinha, Hongseok Namkoong, Riccardo Volpi, and John Duchi. Certifying some distributional robustness with principled adversarial training. *arXiv preprint arXiv:1710.10571*, 2017. 2

[76] Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Implicit neural representations with periodic activation functions. In *NeurIPS*, 2020. 3

[77] Justin Solomon, Raif Rustamov, Leonidas Guibas, and Adrian Butscher. Earth mover’s distances on discrete surfaces. *ACM Transactions on Graphics*, 33(4):67, 2014. 2

[78] Justin Solomon, Fernando De Goes, Gabriel Peyré, Marco Cuturi, Adrian Butscher, Andy Nguyen, Taegyu Du, and Leonidas Guibas. Convolutional wasserstein distances: Efficient optimal transportation on geometric domains. *ACM Transactions on Graphics*, 34(4):66, 2015. 2

[79] Matthew Staib and Stefanie Jegelka. Distributionally robust deep learning as a generalization of adversarial training. In *NIPS workshop on Machine Learning and Computer Security*, page 4, 2017. 2

[80] Riccardo Volpi, Hongseok Namkoong, Ozan Sener, John C Duchi, Vittorio Murino, and Silvio Savarese. Generalizing to unseen domains via adversarial data augmentation. *Advances in neural information processing systems*, 31, 2018. 1

[81] Jie Wang, Rui Gao, and Yao Xie. Sinkhorn distributionally robust optimization. *arXiv preprint arXiv:2109.11926*, 2021. 2, 4, 5

[82] Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny. Vggsfm: Visual geometry grounded deep structure from motion. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 21686–21697, 2024. 5, 7

[83] Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. Pixel2mesh: Generating 3d mesh models from single rgb images. In *ECCV*, 2018. 2

[84] Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. *arXiv preprint arXiv:2106.10689*, 2021. 2

[85] Francis Williams, Matthew Trager, Joan Bruna, and Denis Zorin. Neural splines: Fitting 3d surfaces with infinitely-wide neural networks. In *CVPR*, 2021. 2, 3, 5, 7

[86] Francis Williams, Zan Gojcic, Sameh Khamis, Denis Zorin, Joan Bruna, Sanja Fidler, and Or Litany. Neural fields as learnable kernels for 3d reconstruction. In *CVPR*, 2022. 2

[87] Yao Yao, Zixin Luo, Shiwei Li, Jingyang Zhang, Yufan Ren, Lei Zhou, Tian Fang, and Long Quan. Blendedmvs: A large-scale dataset for generalized multi-view stereo networks. In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pages 1790–1799, 2020. 5, 7, 8

[88] Lior Yariv, Jiatao Gu, Yoni Kasten, and Yaron Lipman. Volume rendering of neural implicit surfaces. *Advances in Neural Information Processing Systems*, 34:4805–4815, 2021. 2

[89] Mae Younes, Amine Ouasfi, and Adnane Boukhayma. Sparsecraft: Few-shot neural reconstruction through stereopsis guided geometric linearization. In *European Conference on Computer Vision*. 3

[90] Qian-Yi Zhou and Vladlen Koltun. Dense scene reconstruction with points of interest. *ACM Transactions on Graphics (ToG)*, 32(4):1–8, 2013. 5, 7
# Toward Robust Neural Reconstruction from Sparse Point Sets
## Supplementary Material

Amine Ouasfi, Shubhendu Jena, Eric Marchand, Adnane Boukhayma
Inria, Univ. Rennes, CNRS, IRISA

## 1. Background on Distributionally Robust Optimization

Distributionally Robust Optimization (DRO) was initially introduced by [21] and has since become a significant framework for addressing uncertainty in decision-making [4, 9]. The DRO framework operates by defining an uncertainty set $U$, typically modeled as a ball of radius $\epsilon$ around an empirical distribution $\hat{Q}_n$, such that $U = \{Q : d(Q, \hat{Q}_n) \le \epsilon\}$. The specific choice of the divergence measure greatly influences both the required size of $\epsilon$ and the tractability of the resulting optimization problem. The loss function is minimized under the worst-case distribution $Q \in U$ in terms of the expected loss.

In machine learning, two primary divergence measures are widely adopted: f-divergences and the Wasserstein distance. With f-divergences [3, 14, 16] convex optimization techniques are usually leveraged to define tractable uncertainty sets. Alternatively, the Wasserstein distance [5, 15] is based on a metric over the data space, enabling the inclusion of distributions with supports different from the empirical distribution, thereby offering robustness to unseen data. However, the computational complexity of Wasserstein-based DRO makes it more challenging to handle. To address these challenges, various studies have proposed tractable methods for specific uncertainty sets and loss functions. For instance, [5, 15, 22] provide practical approaches for solving DRO problems with uncertainty regions defined by Wasserstein balls. For smooth loss functions, [23] proposes an efficient formulation for certifying robustness under Wasserstein uncertainty sets. Furthermore, the Unified DRO framework (UDR) introduced by [7] establishes a connection between Wasserstein DRO and adversarial training (AT) methods, offering a novel approach where the dual variable of the DRO problem is adaptively learned during training. This contrasts with [23], where this parameter is fixed. [1, 24] study the DRO problem using the Sinkhorn Distance instead of the Wasserstein Distance providing efficient dual formulations.

<table><caption>Table 1. Ablation of point cloud density</caption><thead><tr><th></th><th>Sparse</th><th>Dense</th></tr></thead><tbody><tr><td>SPSR [10]</td><td>2.27</td><td>1.25</td></tr><tr><td>DIGS [2]</td><td>0.68</td><td>0.19</td></tr><tr><td>OG-INR [11]</td><td>0.85</td><td>0.20</td></tr><tr><td>NTPS [8]</td><td>0.73</td><td>-</td></tr><tr><td>NP [12]</td><td>0.58</td><td>0.23</td></tr><tr><td>SparseOcc [18]</td><td>0.49</td><td>0.20</td></tr><tr><td>NAP [17]</td><td>0.49</td><td>0.19</td></tr><tr><td>Ours (WDRO)</td><td>0.51</td><td>0.20</td></tr><tr><td>Ours (SDRO)</td><td>0.48</td><td>0.21</td></tr></tbody></table>

## 2. Additional Ablative Analysis

### 2.1. Varying the point cloud density

In order to assess the performance of our method under various point cloud densities we perform an ablative analysis on the SRB benchmark [25]. We present quantitative results for both 1024-sized and dense input point clouds. In the dense setting, we report results from OG-INR. Our distributionally robust training strategy outperforms competitors in the sparse case and performs on par with the state-of-the-art in the dense case. Importantly, we observe considerable improvement over our baseline (NP) in both scenarios. Fig. 1 visually supports these results, illustrating reconstructions for sparse and dense inputs. In the dense setting, our method captures finer details, emphasized by the red boxes. These results highlight the practical advantages of our approach, even for dense inputs. Interestingly, our ablative analysis reveals that for dense inputs, WDRO may exhibit slightly better performance compared to SDRO. This result is not surprising, given that WDRO is certified to effectively hedge against small perturbations [23]. Consequently, as the input becomes denser, the noise on the labels due to input sparsity diminishes, thereby favoring WDRO.

### 2.2. Hyperparameter Analysis

In order to determine the hyperparameters of our proposed approach (SDRO), We performed a hyperparameter search on the SRB [25] benchmark utilizing the chamfer distance between the reconstruction and the input point cloud as a
<figure><img src="image_9.png" /><figcaption>Figure 1. SRB [25] unsupervised reconstructions from sparse (1024 pts) unoriented point clouds without data priors.</figcaption></figure>

validation metric. For the remaining datasets, we employed
the same hyperparameters.

We carry out here an ablation study where we vary each one of the hyperparameters $\lambda$ and $\rho$ while fixing the remaining ones in order to better understand the behavior of our approach (SDRO) and its sensitivity to the choice of these hyperparameters.

**Regularization parameter** $\lambda$. This parameter controls how close the worst-case distribution $Q'$ is to the nominal distribution. Fig. 3 illustrates how a very high value for this parameter minimizes the regularization impacts of SDRO by maintaining the worst-case samples around the nominal samples. Conversely, excessively low values lead to overly pessimistic estimations over-smoothing the results, despite greatly improving over the NP baseline.

**Regularization parameter** $\rho$. This parameter is responsible for the strength of the entropic regularization: it controls how the SDRO worst case distribution is concentrated around the support points of WDRO worst case distribution [24]. Consequently, it has to be defined such that it facilitates finding challenging distributions around the surface while maintaining a useful supervision signal. According to Fig. 2, it is important to utilize a sufficiently high $\rho$ value in order to hedge against the right family of distributions. Contrastively, very high values can result in increased variance. Notice that $\rho_{avg}$ here corresponds to average $\sigma_p$ over the input points **P**.

<figure><img src="image_10.png" /><figcaption>Figure 2. Ablation of the regularization parameter $\rho$.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Figure 3. Ablation of the regularization parameter $\lambda$.</figcaption></figure>

**Algorithm 1** The training procedure of our method with WDRO.

**Input:** Point cloud **P**, learning rate $\alpha$, number of iterations $N_{it}$, batch size $N_b$.

WDRO hyperparameters: $\epsilon, \sigma_0, \alpha_{wdro}, N_{it}^{wdro}, \eta_{\lambda}$.

**Output:** Optimal parameters $\theta^*$.

Compute local st. devs. $\{\sigma_p\}$ ($\sigma_p = \max_{t \in K_{nn}(p, \mathbf{P})} ||t - p||_2$).

$\mathcal{Q} \leftarrow \text{sample}(\mathbf{P}, \{\sigma_p\})$. (Equ. ??)

Compute nearest points in **P** for all samples in $\mathcal{Q}$.

Initialize $\lambda_1 = \lambda_2 = 1$.

Initialize $\lambda$.

**for** $N_{it}$ **times do**

Sample $N_b$ query points $\{q, q \sim Q\}$.

Initialize $N_b$ points $\{q'\}$, $(q' \sim \mathcal{N}(q, \sigma_0 \mathbf{I}_3))$.

**for** $N_{it}^{wdro}$ **times do**

$$q' \leftarrow q' + \alpha_{wdro} \nabla_{q'} [\mathcal{L}(\theta, q') - \lambda c(q, q')]$$

**end for**

$$\lambda \leftarrow \lambda - \eta_{\lambda} \left( \epsilon - \frac{1}{N_b} \sum_{i=1}^{N_b} c(q'_i, q_i) \right)$$

Compute WDRO losses $\{\mathcal{L}_{WDRO}(\theta, q)\}$ (Equ. ??)

Compute combined losses $\{\mathcal{L}(\theta, q)\}$ (Equ. ??)

$$(\theta, \lambda_1, \lambda_2) \leftarrow (\theta, \lambda_1, \lambda_2) - \alpha \nabla_{\theta, \lambda_1, \lambda_2} \sum_q \mathcal{L}(\theta, q)$$

**end for**

### 3. Training algorithm for WDRO

We provide in Algorithm 1 the detailed training procedure for WDRO.
<figure><img src="image_12.png" /><figcaption>Figure 4. SemanticPOSS [19] reconstructions from road scene LiDAR data.</figcaption></figure>

## 4. Additional Qualitative Results

We provide additional qualitative comparisons using SemanticPOSS road scene LiDAR data. Fig. 4 highlights the superior
priority of our method in this challenging scenario compared to NAP and SparseOcc. This is particularly evident in highly noisy regions, such as trees, where these methods struggle, whereas our SDRO approach demonstrates robust performance.

## 5. Evaluation Metrics

Building on the definitions provided in [6] and [25], we present the formal definitions of the metrics used for evaluation in the main submission. Let $S$ and $\hat{S}$ denote the ground truth and predicted meshes, respectively. Following [8], all metrics are approximated using 100k samples drawn from $S$ and $\hat{S}$ for ShapeNet and Faust, and 1M samples for 3DScene. For SRB, we also utilize 1M samples, as suggested by [2] and [11].

**Chamfer Distance (CD1)** The $L_1$ Chamfer Distance is computed using the two-way nearest-neighbor distance:

$$CD_1 = \frac{1}{2|S|} \sum_{v \in S} \min_{\hat{v} \in \hat{S}} \|v - \hat{v}\|_2 + \frac{1}{2|\hat{S}|} \sum_{\hat{v} \in \hat{S}} \min_{v \in S} \|\hat{v} - v\|_2$$

**Chamfer Distance (CD2)** The $L_2$ Chamfer Distance is computed using the two-way nearest-neighbor squared distance:

$$CD_2 = \frac{1}{2|S|} \sum_{v \in S} \min_{\hat{v} \in \hat{S}} \|v - \hat{v}\|_2^2 + \frac{1}{2|\hat{S}|} \sum_{\hat{v} \in \hat{S}} \min_{v \in S} \|\hat{v} - v\|_2^2$$

**F-Score (FS)** For a given threshold $\tau$, the F-Score between the ground truth mesh $S$ and the predicted mesh $\hat{S}$ is defined as:

$$FS(\tau, S, \hat{S}) = \frac{2 \text{ Recall} \cdot \text{Precision}}{\text{Recall} + \text{Precision}},$$

where

$$\text{Recall}(\tau, S, \hat{S}) = |\{v \in S, \text{s.t. } \min_{\hat{v} \in \hat{S}} \|v - \hat{v}\|_2 \langle \tau \rangle\}|, \\ \text{Precision}(\tau, S, \hat{S}) = |\{\hat{v} \in \hat{S}, \text{s.t. } \min_{v \in S} \|v - \hat{v}\|_2 \langle \tau \rangle\}|.$$

Following [13] and [20], we set $\tau$ to 0.01.

**Normal consistency (NC)** measures the alignment of surface normals between two meshes $S$ (ground truth) and $\hat{S}$ (prediction). Denoting the normal at a point $v$ in $S$ by $n_v$, it is defined as

$$NC = \frac{1}{2|S|} \sum_{v \in S} n_v \cdot n_{\text{closest}(v, \hat{S})} + \frac{1}{2|\hat{S}|} \sum_{\hat{v} \in \hat{S}} n_{\hat{v}} \cdot n_{\text{closest}(\hat{v}, S)},$$

where

$$\text{closest}(v, \hat{S}) = \underset{\hat{v} \in \hat{S}}{\operatorname{argmin}} \|v - \hat{v}\|_2.$$

## References

[1] Waïss Azizian, Franck Iutzeler, and Jérôme Malick. Regularization for wasserstein distributionally robust optimization. *ESAIM: Control, Optimisation and Calculus of Variations*, 29:33, 2023. 1

[2] Yizhak Ben-Shabat, Chamin Hewa Koneputugodage, and Stephen Gould. Digs: Divergence guided shape implicit neural representation for unoriented point clouds. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 19323–19332, 2022. 1, 4

[3] Aharon Ben-Tal, Dick Den Hertog, Anja De Waegenaere, Bertrand Melenberg, and Gijs Rennen. Robust solutions of optimization problems affected by uncertain probabilities. *Management Science*, 59(2):341–357, 2013. 1

[4] Dimitris Bertsimas, Vishal Gupta, and Nathan Kallus. Data-driven robust optimization. *Mathematical Programming*, 167: 235–292, 2018. 1

[5] Jose Blanchet and Karthyek Murthy. Quantifying distributional model risk via optimal transport. *Mathematics of Operations Research*, 44(2):565–600, 2019. 1

[6] Alexandre Boulch and Renaud Marlet. Poco: Point convolution for surface reconstruction. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 6302–6314, 2022. 4

[7] Tuan Anh Bui, Trung Le, Quan Tran, He Zhao, and Dinh Phung. A unified wasserstein distributional robustness framework for adversarial training. *arXiv preprint arXiv:2202.13437*, 2022. 1

[8] Chao Chen, Zhizhong Han, and Yu-Shen Liu. Unsupervised inference of signed distance functions from single sparse point clouds without learning priors. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, 2023. 1, 4

[9] Joel Goh and Melvyn Sim. Distributionally robust optimization and its tractable approximations. *Operations research*, 58(4-part-1):902–917, 2010. 1

[10] Michael Kazhdan and Hugues Hoppe. Screened poisson surface reconstruction. *TOG*, 2013. 1

[11] Chamin Hewa Koneputugodage, Yizhak Ben-Shabat, and Stephen Gould. Octree guided unoriented surface reconstruction. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 16717–16726, 2023. 1, 4

[12] Baorui Ma, Zhizhong Han, Yu-Shen Liu, and Matthias Zwicker. Neural-pull: Learning signed distance functions from point clouds by learning to pull space onto surfaces. In *ICML*, 2021. 1

[13] Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 4460–4470, 2019. 4

[14] Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, Ken Nakae, and Shin Ishii. Distributional smoothing with virtual adversarial training. *arXiv preprint arXiv:1507.00677*, 2015. 1
[15] Peyman Mohajerin Esfahani and Daniel Kuhn. Data-driven distributionally robust optimization using the wasserstein metric: performance guarantees and tractable reformulations. *Mathematical Programming*, 171(1-2):115–166, 2018. 1

[16] Hongseok Namkoong and John C Duchi. Stochastic gradient methods for distributionally robust optimization with f-divergences. In *NIPS*, pages 2208–2216, 2016. 1

[17] Amine Ouasfi and Adnane Boukhayma. Few-shot unsupervised implicit neural shape representation learning with spatial adversaries. *arXiv preprint arXiv:2408.15114*, 2024. 1

[18] Amine Ouasfi and Adnane Boukhayma. Unsupervised occupancy learning from sparse point cloud. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pages 21729–21739, 2024. 1

[19] Yancheng Pan, Biao Gao, Jilin Mei, Sibo Geng, Chengkun Li, and Huijing Zhao. Semanticposs: A point cloud dataset with large quantity of dynamic instances. In *2020 IEEE Intelligent Vehicles Symposium (IV)*, pages 687–693, 2020. 3

[20] Songyou Peng, Michael Niemeyer, Lars Mescheder, Marc Pollefeys, and Andreas Geiger. Convolutional occupancy networks. In *European Conference on Computer Vision*, pages 523–540. Springer, 2020. 4

[21] Herbert E Scarf, KJ Arrow, and S Karlin. *A min-max solution of an inventory problem*. Rand Corporation Santa Monica, 1957. 1

[22] Soroosh Shafieezadeh Abadeh, Peyman M Mohajerin Esfahani, and Daniel Kuhn. Distributionally robust logistic regression. *Advances in neural information processing systems*, 28, 2015. 1

[23] Aman Sinha, Hongseok Namkoong, Riccardo Volpi, and John Duchi. Certifying some distributional robustness with principled adversarial training. *arXiv preprint arXiv:1710.10571*, 2017. 1

[24] Jie Wang, Rui Gao, and Yao Xie. Sinkhorn distributionally robust optimization. *arXiv preprint arXiv:2109.11926*, 2021. 1, 2

[25] Francis Williams, Teseo Schneider, Claudio Silva, Denis Zorin, Joan Bruna, and Daniele Panozzo. Deep geometric prior for surface reconstruction. In *CVPR*, 2019. 1, 2, 4