Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes
Authors: Sidhanth Holalkere
David Bindel
Silvia Sellán
Alexander Terenin
Venue: ICML 2025 poster (2025)
Keywords: stochastic poisson surface reconstruction
geometric gaussian processes
computer graphics
Primary area: probabilistic_methods->gaussian_processes

Abstract: Poisson Surface Reconstruction is a widely-used algorithm for reconstructing a surface from an oriented point cloud. To facilitate applications where only partial surface information is available, or scanning is performed sequentially, a recent line of work proposes to incorporate uncertainty into the reconstructed surface via Gaussian process models. The resulting algorithms first perform Gaussian process interpolation, then solve a set of volumetric partial differential equations globally in space, resulting in a computationally expensive two-stage procedure. In this work, we apply recently-developed techniques from geometric Gaussian processes to combine interpolation and surface reconstruction into a single stage, requiring only one linear solve per sample. The resulting reconstructed surface samples can be queried locally in space, without the use of problem-dependent volumetric meshes or grids. These capabilities enable one to (a) perform probabilistic collision detection locally around the region of interest, (b) perform ray casting without evaluating points not on the ray's trajectory, and (c) perform next-view planning on a per-ray basis. They also do not requiring one to approximate kernel matrix inverses with diagonal matrices as part of intermediate computations, unlike prior methods. Results show that our approach provides a cleaner, more-principled, and more-flexible stochastic surface reconstruction pipeline.

Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes
Sidhanth Holalkere David S. Bindel
Silvia Sellán 23 Alexander Terenin'
Abstract
world three-dimensional scans are captured- into other downstream-ussle formats, such as meshes or other surface representations. Such algorithms are key computational primitive used in computer graphics pipelines.
Poisson Surface Reconstruction is a widely-used algorithm for reconstructing a surface from an ori- ented point cloud. To facilitate applications where only partial surface information is available, or scanning is performed sequentially, recent line of work proposes to incorporate uncertainty into the reconstructed surface via Gaussian process models. The resulting algorithms first perform Gaussian process interpolation, then solve a set of volumetric partial differential equations glob- ally in space, resulting in a computationally ex- pensive two-stage procedure. In this work, we apply recently-developed techniques from geo- metric Gaussian processes to combine interpo- lation and surface reconstruction into a single stage, requiring only one linear solve per sam- ple. The resulting reconstructed surface samples can be queried locally space, without the use of problem-dependen volumetric meshes grids. These capabilities enable one to (a) perform prob- abilistic collision detection locally around the re- gion of interest, (b) perform ray casting without evaluating points not on the ray's trajectory, and (c) perform next-view planning on a per-ray ba- sis. They also do not requiring one to approxi- mate kernel matrix inverses with diagonal matri- ces part of intermediate computations, unlike prior methods. Results show that our approach provides a cleaner, pore-principled, and more- flexible stochastic surface reconstruction pipeline.
The most widely-used approach is Poisson Surface Recon- struction [17] [6]. It works by solving a partial differential equation to compute an implicit surface representation- function where positive (resp. negative) values denote loca- tions inside (resp. outside) the surface, with zeroes denoting the surface. Its computation relies standard numerical techniques- most commonly the finite element method- making it simple, reliable, and efficient on compute devices well-suited to sparse linear algebra.
Real-world scans are necessarily imperfect: they involve noise, measurement error, and may even need gracefully handle situations where parts of the object occluded or otherwise not observed by the camera. As consequence, surface reconstruction algorithms must both interpolate and extrapolate from noisy and potentially incomplete data as part of their computations. Poisson surface reconstruction typically does this by blurring the data using a smooth ker- nel: while simple and effective, this process does not quan- tify uncertainty about interpolation or extrapolation.
To provide richer characterization of the information con- tained in a scan, recent line of work proposes Stochastic Poisson Surface Reconstruction 12]-a Bayesian formal- ism which incorporates uncertainty about the scan through the use of Gaussian process priors. The implicit surface representation then recovered by applying Bayes' Rule in the solution space of the respective Poisson equation's stochastic analog. This approach not only quantifies certainty, it provides a potential path for selecting optimal scan directions in technically principled manner, mirror- ing data acquisition techniques from areas such as Bayesian optimization
1. Introduction
Surface reconstruction algorithms are used to process point cloud data most-common format in which real-
and probabilistic numerics [13
A key challenge in stochastic Poisson surface reconstruc- tion is that it requires more complex computations than its non-stochastic analogs. The simplest approach, proposed by Sellán and Jacobson (27] is to first condition the Gaussian process on point-cloud observations, then solve the required Poisson equation to obtain the surface representation. Un-
Code: IITTPS://GITTUB
'Coreell University *Columbia University 'MIT, Correspon- dence to: Sidhanth Holalkere <sh844 comell.duu.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes
fortunately, this approach suffers from the classically-cubic scalability of Gaussian processes: the authors handle this by approximating kernel matrix inverses with diagonal ma- trices. Then, finite elements are used to solve the Poisson equation-a potentially expensive operation in its own right due to reliance on a volumetric mesh or grid.
Mean
Variance
P(z E Q)
Input point cloud
This work: stochastic reconstruction
Statistical query
In this work, we study the following question: can one compute the posterior of the implicit surface directly, with- out solving separate linear systems for interpolating the data and calculating the implicit surface? We answer this affirmatively using methods from inter-domain Gaussian processes, geometric Gaussian processes, and related areas. Our techniques:
Figure I. Like prior work in stochastic reconstruction choose to represent the space of shapes determined by an input point cloud (left) through volumetric Gaussian Process (middle) from which standard statistical quantities can be extracted (right).
27
26 we
a volumetric finite element mesh to produce a vector field v IR 1 for which v(z;) s ug. Then, one reconstructs f. the implicit representation of an, by numerically solving the Poisson equation
1. Produce posterior means, posterior covariances, and posterior random function samples, up to principled set of approximations, using only kernel-matrix- based linear solves. These are handled at graphics-scale with standard scalable Gaussian process methods, without involving the Poisson equation.
Af = V v(x)
(1)
2. Avoid the use of mesh-hased or otherwise global Pois- son solves, allowing one to obtain the surface from purely-local evaluations.
over a hypercube 10, subject to Neumann boundary conditions V - n = 0 at the hypercube's boundary, where n are the normals. Given the surface is a priori unknown, the most common approach to choose the mesh to be a regular grid, with v obtained using trilinear interpolation- see Kazhdan et al. [17 for further details.
3. Eliminate the coupling between the discretization struc- ture of the enclosed bounding box and the length scale hyperparameter used in interpolation.
To enable the surface reconstruction algorithm to assess and propagate uncertainty for downstream use, Sellán and Jacobson [27, 26] propose Stochastic Poisson Surface Re- construction: instead of a deterministic v, they place a Gaus- sian process prior v ~ GP k) over the vector field, and calculate the respective posterior distribution in order to interpolate the point cloud data in an uncertainty- manner. Then, they solve a stochastic analog of ( obtain- ing a random function f v, which can be interpreted as the posterior distribution of the implicit surface given the point cloud data. This can be seen in Figure [
4. Analytically describe the Gaussian process induced by the Poisson solve, and provide user control over length scales and other hyperparameters which is retained in solution space.
2. Stochastic Surface Reconstruction
The goal of implicit surface reconstruction algorithm is to produce function f R + R representing a solid shape Ω C TR3, given data consisting of an oriented point cloud (z,, o where x; E Qe and i E R² are surface normals to 02, both potentially noisy. To represent the surface, / should take positive values inside of Ω negative values outside of it, so that the zero level set dn is the reconstructed surface. Throughout this work, we use bold italic letters such as a, to refer to vectors corresponding to batches of data, and bold upface letters such as A, B to refer to corresponding matrices, with the convention that functions act on such terms component-wise.
From computational perspective, the resulting stochastic formulation is significantly more involved than ordinary Poisson reconstruction. Note that, since the Poisson equa- tion is linear, f v is itself a Gaussian process-Vv we review this and other relevant properties of Gaussian processes in the next section. Leveraging this, Sellán and Jacobson (27 propose an approach which computes the mean Myf and covariance kyjo using the respective finite-clement Lapla- cian L and divergence matrix Z, as well as the kernel ma- trices K. and K.v for the training and test points, re- spectively. To facilitate computation at point cloud sizes typical in graphics applications, Sellán and Jacobson (27) rely on the fact that L and are sparse, and approximate Kil as diag(",P.)) -1, where og is a hyperparameter and p is a vector measuring local sampling density.
2.1. Stochastic Poisson Surface Reconstruction
Poisson surface reconstraction is the most widely-used sur- face reconstruction algorithm, with numerous applications in computer graphics [28 15 and beyond 12 24). It
works as follows: first, the point cloud is interpolated onto
2

---

Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes
SPSR variance with decreasing length scale
This work: variance with decreasing length scale
Inpul point cloud
13 seconds
80 seconds
650 seconds
seconds
5 seconds
scconds
Figure Our method removes the interpolation length scale's dependence on any grid resolution, therefore the cost to evaluate the posterior is independent of the length scale. Further, our method enables local evaluation leading to faster computations when we do not require global results. contrast, SPSR baseline grid-dependent and unable represent the correct variance lower length scales,
The resulting method enables one to perform a number of queries that are useful in computer graphics and vision ap-
Pk. zd + R called the spectral measure of are
plications, like casting rays against the uncertain surface, deciding on best
2 x'1-
$1/2(1, = cosh
(2)
view and computing collision like- lihoods. It also comes with number of limitations. (i) One needs to solve the Poisson equation globally on the full domain, even if one only interested in queries in a small region of space. Recent work has proposed ways to avoid this for non-stochastic Poisson surface reconstruction [8). (ii) This solution is only obtained on fixed grid, and must either perform interpolation for non-grid points, or learn their values using neural network (26). at the cost of efficiency and convergence guarantees. This artificially ties the length scale of the GP kernel to the resolution of the utilized grid-see Figure 2 (iii) The approximations used, such as for the inverse covariance, can make uncertainty behavior difficult control. One of our goals, in this work, will be reduce these limitations.
(1, = sinh
4m
2x
x
We will make use of number of properties of such kernels in our work. In particular, periodic Matérn kernels are stationary, in the sense that k: satisfies k(x++,, x' ec) for any 2, x', E Té, where addition is defined in terms of angles. Under mild regularity conditions, one can show-se see for instance Borovitskiy et al. (6], Theorem 5 and Appendix B that such admit the Mercer's expansion
k(x, x²)
Σ (m) REZ4
(4)
where each Jn is either a sine cosine, with frequency determined by n. Using orthonormality of sines and cosines, can factor this expression to find an analogous representation for Gaussian process prior f: this gives the Karhunen- Loève expansion
2.2. Geometric Gaussian Processes and Periodic Kernels
random function (x1,-- NN), the output (x) ~ N(u, K ) is multivariate normal with mean μ ((xx) and covariance K k(æ, æ). A key property of Gaussian processes is that their conditional dis- tributions f y- given data f(x) = y. are also Gaussian pro- cesses with explicit means and covariances. For an overview of these properties, see Rasmussen and Williams 25
Gaussian process f GP k))
where, given finite of input points
f(x) Σ (enln)enfaaa)
En id N(O.1).
(5)
For a formal treatment, see Borovitskiy et 6). These expansions will play central role in computations.
The techniques we develop will rely on ideas from geomet- ric Gaussian processes that is, Gaussian processes whose domain is not I but instead has geometric structure. Specif- ically, we work with Gaussian processes f : T - IR over the d-dimensional torus T The kernels of such Gaussian processes can be viewed as periodic functions on Idd; we thus refer to them as periodic kernels.
3. Stochastic Poisson Surface Reconstruction
Stochastic Poisson Surface Reconstruction offers techni- cal capabilities that go beyond the classical, deterministic Poisson reconstruction algorithm- this stochasticity greatly complicates the computational pipeline needed efficiently run it. The original work by Sellán and Jacob- son (27 proposes pipeline that mimics classical Poisson reconstruction, in the sense that once Gaussian process in- terpolation is performed, traditional knite-element computation used to calculate the implicit surface. While our goal will be broadly improve the technical capabilities
We work with the class of periodic Matérn kernels of Borovitskiy al. (6). denoted ky, nels admit analytic expressions for d = 1 and half-integer. For / = 1/2, the kernel-and a closely-related expression
R+. These ker-
3

---

Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes
of SPSR, will start by asking: within Gaussian process formalism, can calculate the implicit surface directly, without solving more than one linear system?
Input
L= 7
L = 19*
L = 29'
L 39'
Mean
3.1. Bypassing the Poisson Solve using Geometric Gaussian Processes
Variance
We now proceed to answer the above initial guiding question affirmatively. The mean and covariance of our Gaussian process, assuming centered prior, are
Piz E
1*]((-) K 56 (Kev +Z)= ky)o(-,-) ((kk))))))>>>>)))) (Ko)oKov + -)-KKKKCe Kof())
(6)
Figure 3. The number of Fourier basis functions used, L, is a key hyperparameter in algorithm: experimentally, find that using too few results in a loss of high frequency detail and observe diminishing returns beyond 20
(7)
Alteraatvely, rather than calculate means and covariances, one can work with Monte Carlo samples from the posterior. This avoids needing to store posterior covariances, which contain O(N2) entries, in memory. We can sample from the posterior using the inter-domain Gaussian process [18]
These assumptions enable us apply the geometric Gaus- sian process machinery introduced previously: one can view both the vector field and implicit surface as functions on the torus, namely v : T R2d and ʃ TV + R. Be- fore proceeding, note that compared to the default setup, we have not lost anything substantive: the original Neu- mann boundary conditions of Kazhdan et al. (17) and Sellán and Jacobson (27) were selected primarily for convenience in the first place, and in practice one places the surface sufficiently-away from the bounding box to limit its impact.
form of pathwise conditioning 32 33), which is
+)))) - {())KKK)) (0) K + +) (v-v(x)) -
(8)
where equality holds in distribution. Note that none of the quantities arising from either the mean and covariance, or the pathwise conditioning expression, are grid-dependent. To compute the posterior, beyond efficiently solving stan- dard linear system of the Gaussian process type, which we will return to in the sequel, we need to compute either one or two additional kinds of expressions:
With the Fourier domain in place, we can apply the Mer- cer's and Karhunen- Loève decompositions of Section 2.2 to express the cross-covariance as Fourier series.
Proposition 1. Let GP k) where le is product of sufficiently-smooth one-dimensional stationary periodic kernll,, and define f by Af = V. with periodic boundary conditions. Then
(b) The cross-covariance term (2,0(2,1'))
(a) If computing posterior samples, joint samples of the prior vector field and its induced implicit surface f defined by As V c.
14 Pk. (n)
f. (I, = Σ
sin((n, 1222=x))) z')).
(9)
||]||
At first, this appears challenging: both quantities we need to compute fundamentally involve the Poisson equation, which in general one must solve numerically. However, if are able do so, the Poisson solve is removed: it suf- fices solve the random linear system once-the Poisson solve is, effectively, replaced by grid-independent matrix multiplication.
mEZ!
All proofs given in Appendix Using this expression, we can compute the cross-covariance matrix K Ky() approx- imately by truncating ( We now show that the same tech- niques also allow us to efficiently jointly sample f and v. Proposition Under the same assumptions as Proposi- tion the random functions f and v can jointly be written
To achieve this, the key idea will be to leverage the fact that both the Poisson equation, and uppropriately-coosee Gaussian processes, are soey-well-behave in the Fourier domain 10
Ex.,1 cos((n,)))
v(()) = Ve,
(10)
For this, we make two assumptions:
Si,,, sin (n,
nezd
n, / Pk.
Pk. (n)
sin((n,
1. The prior covariance kernel k is product of stationary periodic kernels.
f(.) = Σ
(11)
|||
Eim 2 (n)(()))))
DEZd im1 n
2. The Poisson equation defining the implicit surface has periodic boundary conditions.
lid N(O. 1).
where Eims

---

Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes
Fourier truncation
Amortized
Mean-sq error
10 10-5
40
Compute time (s)
10-8 88
20
Input point cloud
Increasingly conservative hitbox
10-11
0
Figure 5. A stochastic formalism for surface reconstruction allows us to construct conservative hitboxes or cages for collision detec- tion that account geometric uncertainty.
Approximation quality
Figure We compare direct covariance computation by trun- cating the Fourier series with its amortized analog, in terms of ac- curacy (kernel matrix MSE evaluated at random batch points) and runtime. Approximation quality is determined by number of Fourier coefficients and amortization grid size, respectively. Amor- tization can substantively improvement runtimes when its error level and initial training cost are acceptable.
of our additional machinery kicks into play only once the solve done.
With this in mind, the question becomes how to select an appropriate linear-time Gaussian process approximation. Since correctly capturing the fine details of the object is key goal of surface reconstruction, we restrict ourselves to approximations which are designed to perform well in large-domain regimes [19 30 where the kernel's length scale small relative to the domain covered by the data, In such regimes, inducing point methods [31 14 which are otherwise popular and efficient-arc known to perform poorly [19 30). Where Cholesky factorization is not viable, we therefore apply the sampling- based stochastic gradient descent approach of Lin et al. [19 20).. which was shown in those works to perform well at data sizes up to ss 20m
Together, these expressions give us way, in principle, to obtain either means and covariances, random samples, from f the latter via the pathwise conditioning ex- pression 8 so long as we can scalably solve the linear system, and compute all of the respective Fourier series. The computational cost of assembling the cross covariccve OLMN), and, for sampling, the cost of computing the necessary set of prior samples is D(L'N)), where and I' are the number of Fourier basis functions used for the cross-covariance and prior, respectively, N is the size of the training data, . is the size of the test data, and P is the number of Monte Carlo samples needed. A visualization of how the resulting approximation error affects reconstruction fidelity is given in Figure Following Wilson et [32] 33
in these regimes.
3.3. Amortized Cross-Covariance Computation
We now re-examine the cross-covariance computation, which costs O(LMN) to assemble, if computed via trun- cation. In contrast with the prior samples for f and v- where, by 1.2-optimality of the Fourier basis for represent- ing stationary Gaussian processes, a relatively-small number of Fourier basis functions suffice 32 covariance term, the number of Fourier basis functions needed to ensure accurate computation can become large, even dimension three. To avoid this computation be- coming the most-expensive part of the overall pipeline, we propose amortization scheme built follows.
one can expect to need < L basis functions, due to the Fourier basis' efficiency at representing random functions drawn from the prior. We will address computations costs of the cross-covariance term further in the sequel, but first proceed to handling the linear system.
33
for the cross-
3.2. Scalability: Training Data Size
A key challenge of working with Gaussian process methods in computer graphics applications is that they require one to solve dense linear systems, typically at cost in our setting, N scales according to the number of points in the point cloud, While this may seem as major challenge, observe that the (random) linear system of interest, namely
Observe that, owing to stationarity, the cross-covariance k x') depends only on the distance x - x' R Moreover, this term does not depend on any hyperpa- rameters except the model's length scale- and, thus, is problem-inddependen. Furthermore, it smooth, since lower- fower-frequency Fourier terms in the Mereer's expansion have larger coefficients.
(Krv + Σ) (0 v(x) =)
(12)
identical in form to the linear system which appears in ordinary Gaussian process eegression-ant indeed, in ker- nel ridge regression. The situation for computing posterior covariances is similar. This means that, even though we are in an interdomain setting which involves the Poisson equa- tion and other machinery, we can apply standard large-scale Gaussian process techniques in an unmodified manner-al all
Taking advantage of these observations, we propose to pre- compute k on a grid in [0, 2mjdd and apply simple linear interpolation to evaluate ky,, at other input locations. While this ostensibly makes our approach dededdeennnnt--- part of what we wanted to avoid the first place note again
5

---

Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes
Large length scale
Variance
Medium length scale
Variance
Small length scale
Variance
Figure Through nallwise-conddiionnnng. based approach, our algorithm allows us to sample different possible surfaces from the Gaussian process determined by the input point cloud. This is be contrasted with SPSR (27. which produces irregular samples due to various (including potentially diagonal) approximations covariance-1 a set samples shown Appendix
that f is smooth and problem-inddepeddent. This should be contrasted with the Poisson equation's 's solution, which by definition aabeeetddeeendent and whose smoothness completely dependent on properties of the surface being can expect grid-based
object, then simplify its geometry, One can generalize this to an uncertainty- -ware approach by scaling the object more in regions with higher uncertainty, for instance by defining the hitbox according the implicit surface given by function of the form g (f) D//k/p (ar, x)), defined using the posterior mean and standard deviation. Here, for differ- ent values of 1). the higher the value, the more conservative the hitbox. Extracting the zero level set as a triangle mesh can then be done using standard contouring algorithms such as marching cubes [22). as shown in Figure
reconstructed. As consequence, amortization of ky.u to be substantially-less lefffiiiuu than the original, grid-based Poisson solve we sought to avoid. We examine its efficiency quantitatively in Figure 4
3.4. Supported Statistical Queries
A key difference in our computational pipeline, compared to ordinary Stochastic Poisson Surface Reconstruction, is that we also allow Monte Carlo sampling, rather than only supporting computations using means and covariances. We
Next-view planning. A key advantage of having prob- abilistic formalism is that, following approaches in areas like Bayesian optimization (II] and probabilistic numeries 13]. one can use the obtained uncertainty to adaptively se- lect where to gather data next. For scanning, Sellán and Jacobson [27] propose to adaptively select camera angles by optimizing a quantity termed the total uncertainty, which defined be the expression
thus detail how various posterior queries can be handled.
Collision detection. This involves computing the proba- bility the reconstructed surface intersects given known object. We handle this by sampling M points on the surface of the known object, denoted F;, and computing (x;( 2 0, Vi I,,,, M) from the samples.
/ (0.5 - [P(x E 2 0.5l) dr.
(13)
This formulation involves computing the uncertainty over the entire volume evaluate the score of a single ray, mak- ing it potentially expensive. Note, however, that this limita- tion does not affect SPSR, which requires one query the GP over the entire volume in order to perform the Poisson solve in the first place.
Ray-casting. This works similarly collision detection: compute the transmittance, which amounts to the joint probability the ray has not hit the object. Letting He be the path of the ray, this is (x,) < 0. Vt = O....)..
incertainty-awarg hitbox generation. The standard ap- proach for generating hitboxes is to uniformly scale-up the
Since our single-solve strategy allows us to more-efficiently query the GP smaller set of points compared to the whole
6

---

Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes
0% collision chance
55% chance
100% chance
Colli-sion Pmobahiiity
Transmittance
Transmittance
256 Data size
512
0
Distance along
Distance along
Figure 7. By querying joint probabilitics from the Gaussian pro- cess produced with our algorithm. we can compute the collision probability of the three cats with the partially observed dragon.
Randomly chosen next views
ovew 896
Best next view candidates
Transmittance
Transmittance
Input point cloud
Bl000
Figure& 8. Our method's support for joint probability queries enable us to perform sequential scanning by adaptively selecting the next view in order minimize total uncertainty.
Distance along ray
Distance along ray
Figure 9, Our statistical formalism allows us to compute joint col- lision probabilities along specific direction. In graphics, this equates to the transmittance of a cast ray, quantity that can be used to score potential future camera positions (greener is hetter).
volume-we we show this in Figure and discuss further in Section $ we propose moditied approach which only relies on computing function over the desired ray. For given camera position and angle, let xe be sequence of points along the center ray, where Ê refers to the total travel distance. Using these points, we compute the transmittance
though the computational pipeline used is almost completely different. In Figure T and Figure mirroring Figure 11 of Sellán and Jacobson 277,. we see that the mean and variance produced by our approach, as well as probability queries, behave in similar qualitative manner to those of that work.
T(L) = P({(x) > 0, VT < l).
(14)
We then compute the total distance where the center ray is not approximately equal to zero or one, namely
Next, we demonstrate that our approach can provide func- tionality which is either not supported by SPSR, or which SPSR scales poorly. Of the capabilities described in Sec- tion 3 we focus first on the property that it avoids the need to compute finite element solve over a volumetric mesh or grid. A consequence of this requirement for the SPSR base- line is that the model's 's ability to resolve fine details is not controlled purely by the Gaussian process' length scale, and is instead also limited to the refinement level of the finite ele- ment mesh which in turn is limited by the system's 'ss memory. Figure 2 shows that, in contrast, our model's 's predictions continue to become sharper as length scales decrease.
Lo BesT(nsi-ede dt
(15)
for a given > O. Intuitively, this measures the range of possible distances the surface can be from the ray origin.
4. Experiments and Applications
We now demonstrate the proposed approach empirically on suite of example problems. Given its identical problem statement and similar theoretical backdrop. use the origi- nal Stochastic Poisson Surface Reconstruction (SPSR) work by Sellán and Jacobson 27 as our main evaluation baseline, for which we utilize the authors' open-source implemen- tation (29.. Implementation details and other experiment information given Appendix B
By avoiding a finite-llement-based Poisson solve, our algo- rithm is also output-sensitive Figure I shows that, unlike SPSR, our proposed algorithm's 's runtime scales with the number of queried points, and not with the size of an entire volumetric grid enclosing the point cloud. This is a critical advantage in many common graphics applications, like ray casting or collision detec- tion, where the stochastic surface need only be queried at a smaller, potentially liwer-dimeniionl number of points in
computer
4.1. Comparisons
To begin, we verify that our proposed algorithm output qualitatively performs at least as well as the baseline, even

---

Stochastic Poisson Surface Reconstruction with One Solve using Geometrie Gaussian Processes
Length scale 0.02
Length scale 0.01
Length scale 0.005
Ground truth
Input point cloud
Distance ulong ray
Distance ulong ray
Distance along ray
Figure 10. Using formalism, showcase random samples drawn from ff reduced of points here, ray query the Gaussian process at a large volumetric bounding box.
without the need
space-for instance, along line or plane, rather than over a three-dimensional volume.
SPSR
Querying on a single line: min
Finally, both SPSR and its neural-network-i based follow-up (66 discard all correlations between the input data by ap- proximating the sample covariance with diagonal matrix. This approximation-whic, is not theoretically justified beyond very specific configurationssmaleee to effectively answer statistical queries that rely on corre- lations. As we show in Figure 6 our approach directly generates smooth samples from the reconstructed surface, in contrast with samples arising from SPSR's s diagonal ap- proximation to the inverse kernel matrix, where smoothness comes purely from the Poisson solve performed the end. This capability combines with output-sensitivity from Fig- ure 10 shows the proposed method is able to sample along a set of points given by single ray without requiring any global Poisson-solve-lite volumetric computation. While here we limit ourselves to qualitative demonstrations, to con- clude, we note that handling correlations and smoothness well critical part of principled tata-acquisition algorithms such as, for instance, in Bayesian optimization: we hope the capabilities showcased potentially pave the way for similar tools in computer graphics applications.
Querying GP on a plane: min
SPSR unable
Querying GP entire volume: 10 min
This work
Querying GP single line: 4 GPU: 0.8
Querying GP plane: 33 GPU:
Querying GP on entire volume: 7.5 min in GPU: 25
Figure 11. Our single-solve stochastic reconstruction algorithm output sensitive: its complexity scales lincarly with the number of test points (bottom). This is contrast to SPSR, which requires a global volumetric FEM solve regardless of the required test
In summary, examining the aforementioned figures, we find that our algorithm qualitatively matches the outputs pro- duced by SPSR while alleviating number of its limitations. We now examine how this affeets downstream applications of stochastic surface reconstruction.
Figure - through joint probabilities when these one, or siieeddimensioaal--ee Figure 7 and Figure 11 In comparison with the SPSR baseline, the output-sensitivity of our method make it ideal for containment tasks like this one, in which the captured scene-such as an entire streetscape or room- can be significantly larger than the relevant queried surface-such car parked on the street, or a robotic gripper.
4.2. Applications
The most basic stochastic reconstruction task relevant in computer graphics, vision and robotics applications is the computation of containment queries: for example, to find if a projectile hits a target, if an autonomous car collides with an obstacle, or if a grasping pose is free of intersections. Our Gaussian process formulation allows to answer these queries through the computation of marginal probabilities when the location is sero-dimensional see Figure and
One of our method's limitations that, even with our improved, output-sensitive performance, formally comput- ing collision likelihoods may be beyond the computational requirements of real-time applications. Figure 5 shows that, for these cases, our method can be used to generate uncertainty- aware hitboxes for shapes in the form tri- angle meshes, such that they can be efficiently tested for
8

---

Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes
intersection, using the uncertainty-aware approach outlined
inho's Sculpture of the Prophet Joel. Journal of Cul- tural Heritage, 2012. Cited on page 2
in Section 3.4
[2] I. Azangulov, A. Smolensky, A. Terenin, and V. Borovitskiy. Stationary Kernels and Gaussian Pro- cesses on Lie Groups and their Homogeneous Spaces
In Figures to 10 we show how our algorithm can be
applied to the prototypical stochastic reconstruction applica- tion of ray casting and next best view planning. In particular, Figure 2 inspired by Figure 26 of Sellán and Jacobson (27]. combines computing the shape's transmittance with scoring set of potential future camera positions to select the op- timal next view, outlined further in Section 3.4 for details. Unlike SPSR, using our approach to compute the score of each camera position requires querying the Gaussian pro- cess along the proposed one-dimensional camera ray, and not an entire volumetric grid surrounding the point cloud.
I: the compact case. Journal of Machine Learning Research, 2024. Cited on page [T
[3] I. Azangulov, A. Smolensky, A. Terenin, and V. Borovitskiy. Stationary Kernels and Gaussian Pro- cesses on Lie Groups and their Homogeneous Spaces II: non-compact symmetric spaces. Journal of Ma- chine Learning Research, 2024. Cited on pages 4
[4]
M. Berger, A. Tagliasacchi, I., M. Seversky, Alliez, G. Guennebaud, J. A. Levine, A. Sharf, and C. T. Silva. A Survey of Surface Reconstruction from Point Clouds. Computer Graphics Forum, 2017. Cited on
5. Conclusion
In this paper, we introduced a reformulation of stochastic Poisson surface reconstruction which requires only one lin- car solve, for performing Gaussian process interpolation. This is in contrast with prior work, which requires addi- tional linear solves arising from its use of a volumetric finite element method. This was achieved by applying tools from geometric Gaussian processes in Fourier space, which ef- fectively replaces the solve with multiplication by matrix whose entries are given by a certain Fourier series. As result of this formulation, the proposed method's s compu- tational costs are output-ennsitve, in the sense that they scale with the number of points at which the posterior Gaus- sian process is evaluated, and not on the size of volumetric meshes or analogous quantities which depend on the space enclosing the object. The proposed method was shown to support the same set of statistical queries as prior work, and additionally provide new, aundom-samplin based queries which take into account smoothness properties captured by the kernel's correlations. Our work constitutes a first step to incorporating sample-efficient data acquisition techniques from the Gaussian process and Bayesian optimization litera- tures into surface reconstruction and computer graphics.
page
2
[5]
V. Borovitskiy, Azangulov, A. Terenin, P. Mostowsky, M. P. Deisenroth, and N. Durrande. Matérn Gaussian Processes Graphs. Artificial In- telligence and Statistics, 2021. Cited on page 4
[6] V. Borovitskiy, A. Terenin, P. Mostowsky, and M. P. Deisenroth. Matern Gaussian processes on Rieman- nian manifolds. Advances in Neural Information Pro- cessing Systems, 2020. Cited on pages
[7] J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. Van- derPlas, S. Wanderman-
and Q. Zhang. JAX: composable transformations of Python+NumPy pro- grams, 2018. URL: //TTP://GINII GITHIUB COM/
ML/JAX Cited on page 12
[8] H. Chen, B. Miller, and Gkioulekas. 3D Recon-
struction with Fast Dipole Sums. ACM Transactions
on Graphics, 2024. Cited page 3
[9]
E. De Vito. N. Mücke, and L. Rosasco. Reproducing Kernel Hilbert Spaces on Manifolds: Sobolev and Diffusion Spaces. Analysis and Applications, 2020.
Cited on pages [1 12
Impact Statement
[10]
L. C. Evans. Partial Differential Equations. American Mathematical Society, 2010. Cited on page
This paper presents work whose goal is to apply recent advances in machine learning in order to advance the field of computer graphics. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
[11] R. Garnett, Bayesian Optimization. Cambridge Uni- versity Press, 2023. Cited on pages] 6
[12] S. K. Gupta and D. Shukla. 3D reconstruction of a landslide by application of UAV and structure from motion. AGILE Conference on Geographic Informa- tion Science, 2017. Cited on page 2
References
[13] P. Hennig, M. A. Osborne, and H. P. Kersting. Proba- bilistic Numerics: Computation as Machine Learning. Cambridge University Press, 2022. Cited on pages]
[1] B. T. Andrade, C. M. Mendes, J. de Oliveira San- tos Jr, O. R. P. Bellon, and L. Silva. 3D Preserving XVIII Century Barroque Masterpiece: Challenges and Results on the Digital Preservation of Aleijad-
6
9

---

Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes
[14] J. Hensman, N. Fusi, and N. D. Lawrence. Gaussian
[28]
S. Sellán, Y. Ren, C. Batty, and O. Stein. Reach for the Ares: Reconstructing Surfaces from SDFs via Tangent Points. ACM SIGGRAPH, 2024, Cited on page 2
Processes for Big Data. Uncertainty in Artificial In-
telligence, 2013. Cited on page|5
[15] F. Hou, C. Wang, W. Wang, H. Qin, C. Qian, and Y.
He. Iterative Poisson Surface Reconstruction (iPSR) for Unoriented Points. ACM Transactions on Graph-
[29]
S. Sellán and O. Stein. gpytoolbox: Python Ge- ometry Processing Toolbox, 2023. URL: HTTPS://
ics, 2022. Cited page
GPYTOOLBOX.ORG/ Cited on pages
[16] M. Kazhdan and H. Hoppe. Screened Poisson Sur- face Reconstruction. ACM Transactions on Graphics, 2013. Cited on page
[30] A. Terenin, D. R. Burt, A. Artemev, S. Flaxman, M. van der Wilk, C. E. Rasmussen, and H. Ge. Numeri- cally Stable Sparse Gaussian Processes via Minimum Separation using Cover Trees. Journal of Machine Learning Research, 2024. Cited on page
[17] M. M. Kazhdan, M. Bolitho, and H. Hoppe. Poisson Surface Reconstruction. Eurographics Symposium on Geometry Processing, 2006. Cited on pages I
[31] M. Titsias. Variational Learning of Inducing Vari- ables in Sparse Gaussian Processes. Artificial Intelli- gence and Statistics, 2009. Cited on page 5
[18] M. Lázaro-Gredilla and A. Figueiras-Vidal Vidal. Inter- domain Gaussian Processes for Sparse Inference us- ing Inducing Features. Advances in Neural Informa- tion Processing Systems, 2009. Cited on page g
[32] J. T. Wilson, V. Borovitskiy, A. Terenin, P.
Mostowsky, and M. P. Deisenroth. Efficiently Sam- pling Functions from Gaussian Process Posteriors. In- ternational Conference on Machine Learning, 2020.
[19] J. A. Lin, Antorán, S. Padhy, D. Janz, J. M. Hernández-Lobato, and A. Terenin. Sampling from Gaussian Process Posteriors using Stochastic Gradi- ent Descent. Advances in Neural Information Pro-
Cited on pages
[33] J. T. Wilson, V. Borovitskiy, A. Terenin, P. Mostowsky, and M. P. Deisenroth. Pathwise Con- ditioning of Gaussian Processes. Journal of Machine Learning Research, 2020. Cited on pages
cessing Systems, 2023. Cited on page 5
[20] J. A. Lin, S. Padhy, J. Antorán, A. Terenin, J. M. Hernández-Loobtto, and D. Janz. Stochastic Gradient Descent for Gaussian Processes Done Right. Inter- national Conference on Learning Representations,
2024. Cited on pages] 5 14
[21] H.-T. D. Liu. BlenderToolbox.
GITHUB.COM/HTDDI Toolbox. 2023. URL: HTTPS:// BOLBOX
Cited on page 12
[22] W. E. Lorensen and H. E. Cline. Marching Cubes: A High Resolution 3D Surface Construction Algorithm, Seminal Graphics: Pioneering Efforts that Shaped the Field. 1998. Cited on page 6
[23] S. V. Lototsky and B. L. Rozovsky, Stochastic Par- tial Differential Equations. Springer, 2017. Cited on pages
12
[24] R. Palomar, F. A. Cheikh, B. Edwin, A. Beghdadhi, and O. J. Elle. Surface Reconstruction for Plan- ning and Navigation of Liver Resections. Computer- ized Medical Imaging and Graphies, 2016. Cited
page
[25] C. E. Rasmussen and C. K. Williams. Gaussian Pro- cesses for Machine Learning. MIT Press, 2006. Cited on 3
[26] S. Sellán and A. Jacobson. Neural Stochastic Pois- son Surface Reconstruction. ACM SIGGRAPH Asia,
2023. Cited on pages 3
[27] S. Sellán and A. Jacobson. Stochastic Poisson Sur- face Reconstruction. ACM Transactions Graphics,
41, 2022. Cited on pages
10

---

Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes
A. Theory
Here we prove the propositions which state the cross-covariance and prior sample expressions for the Gaussian process prior and its respective Poisson equation solution. In what follows, by sufficiently smooth we mean that each & lies in the Sobolev space 1 1 (Td TV; R): for the periodic Matérn kernels used in this work, with 2 3. this is guaranteed by Azangulov also De Vito et al.
[9] and Borovitskiy al. (6]. By standard stochastic PDE theory [23] this condition ensures that the Poisson equation Aʃ = V admits an almost sure weak solution which is unique up to constant shifts. Taking the constant shift part to be zero, by the same theory, this solution can be viewed as Gaussian process which is almost surely continuous. It also follows that Ps: defines a finite measure, meaning Enezs (a(n) is finite. We first derive the prior sampling formulas, as we will use them in the cross-covariance calculation.
al. 3 Theorem 20
Proposition 2. Under the same assumptions as Proposition 7 the random functions I and u can jointly be written
fim,,1 cos((n,)))
0((+) = Σ
Vpk.(n)
(10)
+ En, sin((n, -))
nEZ"
10 10= = Σ Σ
14
Pr. pr.(n) (n)
Ei,,, sin (m, -))
(11)
||]|
2 cos( (n)
nEZ 1=1
11300
iid where Einj ~ N(O, 1).
Proof. We first describe a formal calculation which gives the desired expression, deferring discussion on regularity to the end. Note that each k; is stationary and continuous by assumption: hence, each i; admits Karhunen-Loève expansion
va (x(x))
(n)(En) cos( (n),))))))) 2 sin,(n, nEZ
Σ
Ei,.. N(O,1)
(16)
where Pk. (n) are the respective spectral measures of the kernels k,. On the other hand, note that since both the Laplacian and divergence operators are shift-equivariant, follows that f is also stationary. Since is almost surely continuous, its covariance is as well, and it admits a Karhunen-Loèwe expansion
f(z) = Σ Vo(n)(Gn,1 cos( (n, x)) + Sn, sin((n,
Snj ~ N(0, 1).
(17)
nEZa
By definition, V vツ= Af: we now formally calculate the gradient and Laplacian of the respective expansions. These are
(V v)(x)) Σ
Σ
niy Pk. (n) sin((n,2")) + Ei,,, cos( x²)))
(18)
RE2d ==1
and
(A)())= I-I (ol)) comm cos((n, + Su,2 sin((n,
(19)
nEZ.
where, since under - regularity these expressions cannot be understood as actual random functions in the obvious sense, defer rigorous definition of their precise meaning to later. Continuing the calculation, since sines and cosines linearly independent, these series must be equal term-by-term. This means
V Pk. (n)e.... ||m||] Vo(n)Gn.2
ne naVee, Pe, (n) ||m|] 2
(20)
where d(n) and En,y must be chosen such that the latter are IID standard normal. Choosing
Φ(n) Σd n Pk. (n)
Sn.1 - Ei ns / Ph. (n)
Sn.2 = Σde n, Pk. En)..
(21)
|n||
V E pK (p (n)
V Ei p2 (n)

---

Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes
produces coupling which satisfies these requirements. To complete the proof, we must give rigorous meaning the formal computations performed above. To do so, following an approach mirroring Borovitskiy et al. (6]. reinterpret V and Af as generalized Gaussian fields in the sense of Lototsky and Rozovsky [23 -that is, we interpret differentiation acting on the respective covariance operators defined by the Mercer's ' expansion of both random fields. Under this interpretation, the maps A, A and V v-(+) (-) can be lifted to act on the respective covariances instead of the stochastic processes themselves: following De Vito et al. [9. when restricted to the appropriate Sobolev spaces, these maps are continuous and bijective up to appropriate constant shifts. Using this, one can calculate the generalized Gaussian field corresponding to the image of u under the Poisson equation's 's solution map, and from obtain a Mercer's expansion. From this, one directly obtains the respective Karhunen-Loève expansion: equating this to the expansion of f above produces calculation whose steps- up working with modified definitions- and result match what is given above.
With these expressions derived, we are now ready calculate the respective cross-covariance. Proposition 1. Let v ~ GP(O, k) where is a product of sufficienly-smooth one-dimensional stationary periodic kernels, and define f by A ff V u, with periodic boundary conditions. Then
nay Pke (n)
kf..v. (2,
sin((n, 1022-x)) x')).
(9)
||]|
nE24
Proof. Applying Proposition 2 this follows by direct calculation:
(((f((),,,(x')) (((()))((z)))
(22)
EE Σ (G, cos((n, Gn,2 sin(n,,)))))))
(23)
nEZa
Σ
VPl. pr, (ni
+ Sin.2nnn(((n
(24)
REZS
E
Σ
V (n) cos((n, x)) + Cn, (n, x) (Ei.,. cos((n, x')) + Ei., sin sin ((in((2xxx))
(25)
4
14 Pke (n) cos (n, 2)) xn)((,,,')) + no Pk. (n)
E
sin((n, x)) (ns((n,,))))
(26)
||||
||n|
nEZd
Σ nEZ
T4V
PR. (n)
cos( x))) sin( sin(,,=))))
14 Ph. (n)
sin( (n, cos( (n,))))
(27)
||2|12
||n|]
14V Ph. (n) |n]
sin (n, - x²))
(28)
nEZ
where we pass the expectation inside the infinite series using Fubini's Theorem, for which we now verify that absolute integrability holds. To see this, first note that || S and |sin(-) cos(!') < 1. then write Eneza / (n)5.. = EnEzy V Pk. (n) S VEneza Pks (n) < oo, where the second-to-final step is by Jensen's inequality, and finiteness of the sum follows from the assumed smoothness of k. We conclude that all necessary sums and expectations are finite and do not depend on the order of integration, from which the claim follows.
B. Experimental Details
Implementation. We implement our algorithm in Python using GPYTOOLBOX 29 for common geometry processing subroutines, JAX (7) for numerical computations, and render our results in Blender using BLENDERTOOLBON (2T). All reported timings are calculated on a machine running Ubuntu 20.04 with an Intel Xeon Silver 4316 CPU, 256GB RAM, and an Nvidia RTX A6000 GPU. All of our results use the Matérn kernel with / = 3/2 and length scales between 4 10 and 1 10-2, depending on the specific mesh. Unless specified otherwise, we use 100 functions for the cross-covariance
12

---

Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes
and I,' = 40 functions for the prior samples. The cross-covariance amortized using total of 50 points. To generate an amortization grid, we first evenly sample [-1, 1] with 50 points, raise them to the 5th power in order concentrate values near the origin, multiply by to obtain points in [-T,, x]. then take the Cartesian product for each dimension.
Meshes. The Armadillo, Bunny, Falcon, Scorpion, Springer, Tree, and Well meshes are from Oded Stein's 's repository, at: ODEDSTBIN.COM//EERT The Dragon mesh is originally from the Stanford 3D Scanning Repository we use the version from Alec Jacobson's 's repository, at: GITTHB. htwp:///w...AAAAA.SS///OOOOOO...
Kernel numerical stability. The geometric Matérn-3/2 kernel on T given by Borovitskiy et al. can be numerically unstable, particularly for small length scales, say on the order of 10 2 This occurs due to the specific form of various hyperbolic functions in the formula. To improve stability in floating point arithmetic, we equivalently rewrite the kernel as
0²
v
2=² sinh(u))
63/2(2, x')
2x + coth
cosh(u)
(29)
C3/2
2x
0² T
v3
cosh (w)
2k
2w sinh
(30)
3C3/2
tanh
K
2x
0²22
v3
w (
w
2k +
2w tanh
- k
cosh
(31)
303/2
tanh
v3
2x
0²T2N K
v/
exp
v3
+ exp
-w-
2k
2w tanh
(32)
6C3/2
tanh
/3
K
exp
v3 2
2x
where distances run from to 1, u= = v3 and x is the length scale parameter. Our implementation uses the unnormalized version of this expression.
v - x'1
3
C3/2 is a constant which ensures k( x)) 02.
2
C. Additiona Results
Visualization of samples from SPSR baseline. Here provide visual illustration of samples generated by the SPSR baseline, using its provided mean and covariance. We use total of 253 points, then compute the mesh via marching cubes. Given the interplay between the grid spacing and kernel length scale imposed by SPSR, this coarse grid equates to large length scale which produces over-smoothing In turn, unfortunately, using finer grid-specificalll the 1003 grid from Figure iis is not feasible given the memory available on our system, due to the need to form a large covariance matrix. The obtained samples are shown in Figure 12] and are seen to fail to capture a substantial portion of the reconstructed surface's 's fidelity due to over-smoothing. This the case even well-sampled regions such as the front of the surface, due the aforementioned interaction between length scale and grid spacing. In comparison, samples obtained from the correct posterior, when computed using the techniques of our work shown in Figure are qualitatively much sharper. This is in part because our approach allows for much-larger set of 100s points, which is feasible because it provides samples directly, without requiring the formation of a large covariance matrix.
Figure 12. Random samples drawn using the SPSR baseline.
13

---

Stochastic Poisson Surface Reconstruction with One Solve using Geometrie Gaussian Processes
Comparison of SGD Cholesky. Here examine the behavior of using SGD for the kernel matrix solve and compare it the standard method of using Cholesky decomposition. Throughout this work, by Lin et al. 20). We observe in Figure 13 that SGD is able to provide reasonable approximation after only 10 iterations and converges to solution not visually distinguishable from that found by Cholesky factorization within 1000 iterations. Figure [6] of the sequel additionally shows that the amount iterations is similar across length scales-inn contrast to Cholesky factorization, which will fail under sufficiently-large length scales due to ill-conditioning.
the dual SGD variant proposed
Stochastic Gradient Descent
Cholesky
Input point cloud
iteration
10 iterations
100 iterations
1000 iterations
Figure SGD converges to qualitatively comparable solution to found by Cholesky factorization thousand iterations.
Effect of amortization grid density. We investigate how the grid density used for amortizing the cross-covariance function (x, x²), using the procedure described previously in Appendix B affects the posterior and reconstruction quality. Figure shows that using an N that is too small, for instance N 5, results in poor performance with contain strong artifacting. On the other hand, increasing N leads to higher-fidelity reconstructions with more high-frequency details.
Mean
Variance
Input point cloud
N N=5t
N 9
N 17
N 33?
Figure The size of amortization grid determines the reconstruction's high- high-frequency level detail.
Runtime sensitivity input and output size. Figure 15 shows how our algorithm's wall-clock runtime scales with input size that is, with the number of observed data points.
Runtime scalability with the size of the input data...
... the output data
Time (seconds)
103
Time (scconds)
10a
Meun reconstraccton
10a
10a
10a
10
26
2
216
244
216
218
220
Input data size increases
Data size
Data size
Figure 15. An illustration of our algorithm's 's input and output sensitivity.
14

---

Stochastic Poisson Surface Reconstruction with One Solve using Geometrie Gaussian Processes
Runtime independence of length scale. SPSR, due only being able evaluate the posterior finite element mesh, couples the kernel length scale with the output density, and thus runtime well. In this example, attempt match the interpolation length scale between SPSR and our method much possible, given their different kernels. Figure 16 gives comparison, where each row has half the length scale compared to the row above. We observe in the first row that using large length scales with SPSR leads to too much interpolation post-solve which produces blurry results. Our method instead does not require any interpolation post-solve since by construction we can evaluate the posterior anywhere, leading to a sharper reconstruction with larger length scales, Furthermore, practice, SPSR's s runtime scales exponentially with (inverse) length scale, in the sense that dividing the length scale by increases the runtime by roughly 2 On the other hand, method's runtime is empirically constant with respect to the length scale. For comparison, in this example the amount of time to compute the mean-which is simply (a variant of) ordinary non-stochastic PSR-i is 0.053, 0.54, and 9.1 seconds, respectively, for the three length scales considered, in increasing order.
SPSR
This work (Chol.)
This work (SGD)
Lengith sccalle decreasess
12.4
1.44
6.5
83.8 s
1.44 s
6,5
Input
664
1,44
6.5
Pigure 16. Our runtime not sensitive the kernel's length scale, both with Cholesky and SGD.
15


Reviews:
[{"review_id": "sn4hAt8PqJ", "reviewer": "Reviewer_otmx", "summary": "In the paper, the authors reformulated the stochastic Poisson surface reconstruction by introducing geometric Gaussian processes and periodic kernels. Their proposed method achieves similar results while addressing a number of limitations to increase computational efficiency.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review! We appreciate your recognition that our approach **“increase[s] computational efficiency”** and that our paper has **“little weakness”** and are **delighted by these comments**! We address your key comments below:\n\n> “no quantitative evaluations and comparisons of reconstruction quality”  \n> “However, as a method focused on reconstructing surfaces from point clouds, the paper does not directly compare the reconstruction quality and runtime with other approaches”  \n> “Additionally, the paper only compares its approach with a single baseline, which appears to be somewhat limited”  \n\nThese comments, while brief, are also quite important: let us address them in some detail. First, let us **draw attention to the following subtle distinction between tasks**:\n\n1. **“Surface reconstruction”**: given a point cloud, produce a reconstructed surface\n2. **“Uncertainty quantification for surface reconstruction”**: given an underlying surface reconstruction algorithm, and noisy or otherwise imperfect inputs, produce an estimate of uncertainty for the reconstructed surface\n\nFrom this viewpoint, **our proposed algorithm’s purpose is 2 - that is, uncertainty quantification for surface reconstruction**, where the “underlying surface reconstruction algorithm” is classical PSR (or more precisely, a minor variant thereof, given our slightly-different boundary conditions). \n\nAs such - and given that algorithms of this kind are rather new - **SPSR is the only appropriate baseline we are aware of**. Mirroring the standard used in that work - and more broadly in graphics - our results focus on qualitative behavior, though we agree more quantitative comparisons would make the paper stronger. \n\nTo achieve this, in addition to the quantitative results that we currently have - such as runtime performance benchmarks - we have **performed additional evaluations such as hyperparameter comparisons** (as requested by other reviewers and described in our responses there). In addition, inspired by the need to quantitatively evaluate uncertainty in a manner fitting the above framing, we will also **add a quantitative benchmark of our next-view planning heuristic** (compared to SPSR baseline), following the setup in Figure 15 of \"Neural Stochastic Screened Poisson Reconstruction\", Sellán and Jacobson, SIGGRAPH Asia 2023 (which studies the more-general screened Poisson surface reconstruction setting). We anticipate to complete this by the next manuscript draft.\n\n---\n\n# Summary\n\nOverall, your suggestions have led us to **significant improvements, especially in clarifying appropriate positioning** for this paper, but also in the right way to handle quantitative comparisons. On behalf of these additions - including extra experimental results described in our responses to other referees, which were inspired in part by points that became apparent to us through your review - we would gently like to ask whether you would consider increasing your score.", "claims_and_evidence": "The claims made in the submission should have been supported by evidence.", "methods_and_evaluation_criteria": "Not sure. As a paper introducing a new Poisson surface reconstruction method, there are no quantitative evaluations and comparisons of reconstruction quality.", "theoretical_claims": "The proof for theoretical claims should be correct.", "experimental_designs_or_analyses": "Overall, the experimental designs and analyses are sound.", "relation_to_broader_scientific_literature": "The paper introduces an advanced surface reconstruction methodology.", "essential_references_not_discussed": "All essential references should have been discussed.", "other_strengths_and_weaknesses": "I find little weakness in the paper overall. It is appreciated that the authors also demonstrate various applications of their method. However, as a method focused on reconstructing surfaces from point clouds, the paper does not directly compare the reconstruction quality and runtime with other approaches. Additionally, the paper only compares its approach with a single baseline, which appears to be somewhat limited. It fails to provide a clearer understanding of the method's efficiency and accuracy relative to existing techniques.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NVSQgg67fY", "reviewer": "Reviewer_L7yZ", "summary": "Poisson surface reconstruction (Khazdan et al.) is the task of fitting a function $v(x)$ to point cloud data $(x_i, v_i)_i$ and solving $\\Delta f = \\nabla \\cdot v$ for $f$ (subject to, e.g., Neumann boundary conditions). Then, the zero-level set of $f$ is the desired surface.\nStochastic poisson surface reconstruction (SPSR; Sellan and Jacobsen) proceeds similarly but models $v$ as a Gaussian process (GP), which makes $f \\mid v$ a GP with tractable mean and covariance. But this is costly because (loosely speaking) point clouds contain many points, and because solving PDEs is expensive.\nThe submission proposes ways to make stochastic Poisson surface reconstruction more efficient by using geometric Gaussian processes, pathwise conditioning, and SGD-based Gaussian process training.\n\nThe paper proposes a set of refinements to make SPSR more efficient. These refinements focus on leveraging geometric Gaussian processes, pathwise conditioning, and efficient linear solvers. Specifically, the contributions include:\n- Make $v$ a Gaussian process that automatically satisfies the boundary condition of the PDE. Choosing a periodic boundary implies Matern processes on the $d$-dimensional Torus do precisely that. Such processes admit known Karhunen-Loeve expansions (Borovitskiy et al.). \n- Deduce a Karhunen-Loeve expansion for $f$ from $v$ and $\\Delta f = \\nabla \\cdot v$. This is possible because operations like $\\nabla$ and $\\Delta$ can be computed in closed form for the Fourier-like terms in the expansions for $v$.\n- Avoid working with conditional covariance matrices, and only ever sample from the posterior via pathwise conditioning (Wilson et al.), implementing joint samples from $p(f, v)$ and the cross-covariance between $f$ and $v$ via the expansions. The cross-covariance is expensive to evaluate, so an amortisation scheme is proposed. This avoids an explicit (typically, FEM-based) Poisson solve. What remains in terms of linear algebra is that pathwise conditioning requires solving a linear system involving a Gram matrix.\n- To solve this linear system, the submission uses Lin et al.'s SGD-based algorithm instead of, for example, inducing points.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review! We are very happy that you recognized our approach as **“more efficient”** than prior ones and that our **“claims are supported relatively well by evidence.”** We address your questions below:\n\n> “manipulation of Fourier coefficients … something like a linear solve.”  \n> “Appendix B suggests … relatively expensive”\n\nThis is a fair point - but let us add a critical distinction: **our Fourier coefficients depend only on the kernel and not on the point cloud,** unlike the linear solves in SPSR. This property enables amortization, so one can precompute Fourier-coefficient manipulations once in advance, rather than on a per-point-cloud basis.\n\n> “how many samples … generating the queries in Figures 1 and 3”\n\nThese are done using analytic expressions derived from eqn. (6) rather than sampling: in hindsight, this point came out somewhat-hidden in our text, as we wanted to emphasize sampling as a new capability compared to baselines. We will modify the next draft to improve this.\n\n> “A first step in incorporating … one such scheme (Sellan and Jacobsen, Figure 15).”\n\nWe largely agree, with one tiny exception: **we do introduce a modified score function in Sec. 3.4**, which can be evaluated much faster than the one from the original SPSR paper, but is otherwise similar. Due to space constraints, we cut this in favor of future work: for completeness, we will **add a comparison between the two in the appendix of the next draft**, following the setup presented in Figure 15 of \"Neural Stochastic Screened Poisson Reconstruction\", Sellán and Jacobson, SIGGRAPH Asia 2023 (which studies the more-general screened Poisson surface reconstruction setting), and will modify our claims as you suggest.\n\n> “could benefit from additional quantitative analysis”\n\nWe acknowledge that our results are less-quantitative than many ML works: this is similar to prior work such as SPSR, and reflects norms in the graphics community. We nonetheless agree quantitative evals would strengthen the work, and will add a number of such comparisons to the appendix, described in further detail in our response to Reviewer vrrd.\n\n> “…unclear how the reconstruction reacts to changing $L$ … Figure 3”  \n> “…role of $L$ on the reconstruction quality”\n\nThank you for this idea. We’ve done some preliminary tests, and **found that setting $L$ too small results in loss of high-frequency details**. The same holds for using too-coarse a grid for amortization. We will add this to the next draft’s appendix.\n\n> “Figure 2 shows … (induced?) hyperparameter for “f”.”\n\nIn both cases, **this is a hyperparameter**, specifically the number $\\kappa$ distances are scaled by before going into the kernel, $k(x,x') = k(\\frac{x-x'}{\\kappa})$, which determines both $v$ and in turn $f$. We will make this clearer.\n\n> “… mechanism to calibrate hyperparameters … more convincing”\n\nThis is an excellent question. Since our kernel-matrix solve is identical to that of an ordinary GP, one way to do this would be to apply **standard maximum marginal likelihood techniques**. We will add discussion on this.\n\n> “Figure 10 … number of query points … Or have I misunderstood something?”\n\nYou are correct that $L$ is fixed and sufficiently large (we use $L = 100^3$ as mentioned in Appendix B). Here, allow us to again emphasize that our cross-covariance is independent of the input point cloud. While one may be able to optimize FEM meshes for specific point clouds, **we can compute our cross covariance once and reuse it for any point cloud.** \n\n> “Another criticism is that the proposed… either component isn't enough” and “Independent benchmark studies”\n\nThis is a good idea. We ran additional comparisons on the difference between SGD and Cholesky factorization, which **show that performance (provided Cholesky works) is comparable**, and will add them to the appendix. In the case of pathwise conditioning, we mainly view this as new functionality, since SPSR’s global nature limits the attainable resolution of samples.\n\n> “A more quantitative version… linear complexity gain”\n\nGood idea! We will add this to the next draft’s appendix.\n\n> “Equation (11) .. clarification”\n\nHere, $P(x \\in \\Omega) = P(f(x) \\leq 0)$ as our implicit surface representation takes on negative values inside $\\Omega$, zero on the boundary of $\\Omega$, and positive values outside of $\\Omega$. We will make this more clear in the camera ready.\n\n> “term \"formal\" .. consistently”  \n> “term \"problem-(in)dependent\"\n\nThank you for these - we will fix them.\n\n> “How close is it … to non-stochastic [PSR]?”\n\nThis is a good point and might help readers get a sense of how expensive the method is  - we will add numbers to the next draft.\n\n---\n\n# Summary\n\nYour suggestions have led to **improved clarity** (via the many points - thank you for them!), as well as **strength of evaluations** of this work. Given these additions and above clarifications, we would gently like to ask whether you would consider increasing your score.", "claims_and_evidence": "The submission makes the following claims (using the formulations from the paper's conclusion):\n- **A single linear solve (for interpolation), as opposed to two linear solves in SPSR (one for interpolation, one for PDE solving).** This is accurate, even though the manipulation of Fourier coefficients of $f$ and $v$ could be regarded as something like a linear solve, too. (Appendix B suggests that one million summands are used for representing the cross-covariance, so evaluating the terms sounds relatively expensive, too.) But from a linear algebra perspective, there is no explicit PDE solver. \n- **The computational cost of the proposed method depends on where the solver is queried, not the size of finite element meshes.** This is also accurate, even though it disregards the computational complexity of interpolation via pathwise conditioning, which solves a linear system that involves a Gram matrix with as many rows and columns as the point cloud has data points. However, this cost is the same for both SPSR and the proposal, so it's fair to ignore it.\n- **The same set of statistical queries as in prior work are supported.** This is more or less accurate, because even though the queries are available, they all rely on evaluating densities, means, or standard deviations based on samples from the Gaussian, which leads to approximations. Prior work evaluates them exactly, but based on a model of reduced complexity (diagonal covariances). Qualitative results suggest that queries can be evaluated reasonably well by using samples, but there are no quantitative results (more on quantitative results in \"Methods and Evaluation Criteria\"). Further, I could not find information on how many samples are used for generating the queries in Figures 1 and 3, which means there are some open questions.\n- **A first step in incorporating sample-efficient data acquisition schemes.** There is a mention of this point in Section 3, but no concrete suggestion and no (theoretical or empirical) evidence is provided. This seems to be more future work than a contribution (which is fair, as there is a page limit), but since both the abstract and the introduction also mention it, perhaps these claims could be weakened. For reference, the main prior work (Sellan and Jacobsen) proposes and benchmarks one such scheme (Sellan and Jacobsen, Figure 15).\n\nIn evaluating the claims, I find that the submission provides strong qualitative evidence but could benefit from additional quantitative analysis.\nOverall, I think the submission's claims are supported relatively well by evidence, and even though qualitative results would make the improvements more convincing, I think this is a nice paper, and I lean towards recommending acceptance. \nI discuss the lack of quantitative evidence under \"Methods and Evaluation Criteria\" below.", "methods_and_evaluation_criteria": "I appreciate the Figure's focus on readability and on visually explaining the similarities and differences between existing SPSR and the proposal. I like the numerous qualitative results. That said, the submission would be stronger with quantitative results. The absence of quantitative analysis is why I only give a borderline score.\n\nMore specifically, the experiments discuss the following scenarios: \n\n 1. **Figures 1 and 3** show that the mean and standard deviation of the samples yield \"good-looking\" results. However, there is no mention of quantitative results, like calibration metrics, or at least ratios of standard deviation and mean error. It's also unclear how the reconstruction reacts to changing $L$ (or any of the other hyperparameters) beyond one sentence in the caption of Figure 3. Is there a way of taking an exactly known shape (a sphere?), generating a point cloud, and seeing how the reconstruction error is affected by parameters like $L$ or the number of data points? \n 2. **Figure 2** shows how the proposed algorithm can resolve small lengthscales because it's not limited by the memory demands of a finite element mesh. It's a bit unclear what ``lengthscale'' means here. According to Section 1, it's a hyperparameter of the prior over $v$. But in Figure 2, it seems to be an (induced?) hyperparameter for $f$. I think I understand the high-level point in Figure 2, but if possible, some more precision in the terminology for \"lengthscale\" would be nice. And since we're talking about hyperparameters of Gaussian process models, does the proposed algorithm offer a mechanism to calibrate hyperparameters, e.g. via marginal likelihoods? And again, if there were some more quantitative error analysis, the point of SPSR not capturing these points would be more convincing than looking at a single example.\n 3. **Figure 10** demonstrates that the algorithm's runtime scales with the number of query points, not the number of FEM points. This is all under the assumption that $L$ is fixed (and sufficiently large), using the amortisation from Section 3.3, and that the term $K^{-1} (\\mathbf{v} - v(x))$ has been computed, correct? I am asking, because precomputing all $L$ terms and amortising the covariance feels like there is a corresponding performance gain in the FEM solver to be explored (something along the lines of precomputing the inverse of the matrix and amortising that result). I like the result in Figure 10, but perhaps there is some nuance to provide in the analysis. Or have I misunderstood something?\n4. Another criticism is that the proposed algorithm contains multiple new components that are only benchmarked on surface reconstruction in combination, never on their own. Currently, it seems that the combination of approaches works well; however, for example, the combination of pathwise conditioning and SGD-based solves seems to be applicable to the baseline SPSR algorithm, too, with perhaps notable performance improvements. Same (but maybe to a lesser extent) for geometric Gaussian processes. To be fair, the paper only claims that the combination is helpful, but it would be nice to see that using either component isn't enough.\n\n\n\n\nIn summary, I like the demonstrations. But I think the submission would gain clarity with (some of) the following investigations:\n- Investigating the role of $L$ on the reconstruction quality (on a toy example, if necessary). \n- Investigating the number of samples needed for reasonable results in statistical queries. With these kinds of results, I think the submission would be strengthened\n- A more quantitative version of Figure 10 (eg a \"No. points queried\" vs \"Runtime\" plot) that more clearly shows the linear complexity gain.\n\nIdeally, there would also be independent benchmark studies for the different components. Mainly, to demonstrate whether it's the combination of geometric GPs and pathwise conditioning with SGD that leads to the good performance, or whether either of those two components suffices. That said, I understand it's a big ask, so I'm okay with this change not happening (even though I'd like to see it).", "theoretical_claims": "I have checked the proofs of Propositions 1 and 2 in Appendix A. I appreciate the thorough derivation. \nHowever, Equation (11) could benefit from more clarification.\nBeyond Propositions 1 and 2, all theoretical claims are known.", "experimental_designs_or_analyses": "Covered by \"Methods and evaluation criteria\" above.", "relation_to_broader_scientific_literature": "The paper extends prior work on stochastic Poisson surface reconstruction (SPSR) with a number of computational considerations. Most of these techniques are known (and cited in the paper):\n- Geometric Gaussian processes\n- Pathwise conditioning\n- SGD-style linear-system solving\n\nThe combination of these techniques and the derivation of the Karhunen-Loeve expansion of $f$ based on that of $v$ are new, to the best of my knowledge. As such, I think the submission embeds well into related work, but also provides a series of new results.", "essential_references_not_discussed": "All essential references are discussed.", "other_strengths_and_weaknesses": "See the other sections.", "other_comments_or_suggestions": "- It might help readability if the term \"formal\" is used more consistently. In the sentence after Equation (5), it means \"rigorous\". In Appendix A, it means \"non-rigorous\".\n- It might also help readability if the term \"problem-(in)dependent\" is replaced by something more accurate. For example, Section 3.3 says that $k_{f, v}$ is problem-independent\" which means that $k_{f, v}$ is independent of the point cloud (i.e. the surface being constructed). There are many subproblems in this algorithm (interpolation, Fourier coefficients respectively, PDE solving, amortisation, sampling, etc.), and I got confused by \"problem-dependent\" regularly. But this is my subjective opinion, and my recommendation doesn't depend on this change.", "questions_for_authors": "- The proposed algorithm seems to be much faster than previous approaches to stochastic Poisson surface reconstruction. How close is it (in runtime and memory demands) to non-stochastic Poisson surface reconstruction?\n- Figure 3 mentions diminishing results beyond $L=20^3$, and Appendix B mentions that the experiments use $L=100^3$. The latter is unexpected, given the former. Where does this discrepancy come from?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XPXkvfh48c", "reviewer": "Reviewer_vrrd", "summary": "The paper uses techniques from geometry Gaussian process to speed up the stochastic Poisson surface reconstruction method.\n\n## Update after rebuttal\n\nI appreciate the authors' efforts in providing a more nuanced discussion and additional comprehensive results. Given this, I keep my score which is already positive.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review! We appreciate that you mentioned our method **“has the potential for broad applicability”** and that we have a **“mathematically principled approach”** which was indeed part of our motivation for this work.\n\n> “I'm not convinced by the claim that the proposed method qualitatively matches the outputs of SPSR in Section 4.1”  \n> “Despite the paper’s contribution to accelerating SPSR… diminished if the method does not maintain the original reconstruction quality”\n\nThese are very good points: but addressing them involves some nuance. First, let us **draw attention to the following distinction between tasks** (also mentioned in our response to Reviewer otmx):\n\n1. **“Surface reconstruction”**: given a point cloud, produce a reconstructed surface\n2. **“Uncertainty quantification for surface reconstruction”**: given an underlying surface reconstruction algorithm, and noisy or otherwise imperfect inputs, produce an estimate of uncertainty for the reconstructed surface\n\nFrom this viewpoint, **our proposed algorithm’s purpose is 2 - that is, uncertainty quantification for surface reconstruction**, where the “underlying surface reconstruction algorithm” is classical PSR (or more precisely, a minor variant thereof, given our slightly-different boundary conditions).\n\nWith this framing, we **agree improved evaluations would strengthen** our paper, but where the **focus should be on stochastic aspects to do with uncertainty** as opposed to the mean reconstructed surface, which is essentially the same as in classical methods. Mirroring the standard in graphics and in the SPSR paper (our baseline), our focus has been on qualitative properties, though we agree quantitative evaluations would strengthen our paper further. To this end, we have decided to add additional experiments to the next manuscript’s draft in the form of an expanded appendix:\n\n1. This includes a **comprehensive examination of how hyperparameters affect results**: we find that (a) too-small an amortization grid density leads to over-smoothing, (b) SGD and Cholesky perform similarly (assuming the latter succeeds), (c) for sufficiently-small length scales, the SPSR baseline can result in over-smoothing, whereas our approach works well.\n2. We have also **performed runtime comparisons**, which show our approach to be faster than SPSR as long the length scale is not too large (note that the small length scale regime is the interesting one, as this allows the algorithm to capture fine surface details). \n3. We are additionally working on a **quantitative evaluation examining how the produced uncertainty affects next-view planning**, as a way to test how our algorithm’s numerics affect situations where one needs to use the produced uncertainty for a downstream purpose.\n\nWe hope that these additions - of which the first two are complete - will help strengthen our evaluations and therefore alleviate your concerns.\n\n\n> “A key contribution of the paper is replacing the computation of means and covariances with Monte Carlo sampling from the posterior.”  \n> “I would appreciate it if the authors could provide a more in-depth discussion on Monte Carlo sampling”\n\nThank you for raising these points: due to their importance, let us respond in two parts.\n\nFirst, in situations where this is viable, we utilize analytic expressions derived from eqn. (6) to compute means and covariances rather than sampling: in hindsight, this point came out somewhat-hidden in our text, as we wanted to emphasize sampling as a new capability compared to baselines. We will therefore **modify the next draft to make this clearer** - thank you for drawing our attention here!\n\nSecond, in situations where sampling is needed, we agree that further evaluation of the number of Monte Carlo samples needed is appropriate. Since the random functions we are sampling do come from Gaussian process posteriors, we expect the number of samples needed, in most cases, to be similar to other situations where Gaussian process sample paths are used - such questions are explored to some degree in the pathwise conditioning papers, though for different purposes. To address this, we will add **additional comparisons which show how transmittance calculations and collision detection performance varies with the number of Monte Carlo samples** to the next manuscript draft.\n\n---\n\n# Summary\n\nOverall, your suggestions have led us to **significant improvements, both in terms clarity and especially the evaluations we present**, which we believe, based on those parts we were able to complete so far, will significantly strengthen our results. On behalf of these additions, we would gently like to ask whether you would consider increasing your score.", "claims_and_evidence": "I'm not convinced by the claim that the proposed method qualitatively matches the outputs of SPSR in Section 4.1. In the first paragraph, the authors support this claim by comparing Figures 1 and 3 in this paper with Figure 11 in the SPSR paper. However, these comparisons are performed on different objects which cannot be directly compared. Since it is straightforward to generate results on the same object using identical slices, I expect a side-by-side comparison with SPSR on the same objects (including mean, variance, and probability).", "methods_and_evaluation_criteria": "Despite the paper’s contribution to accelerating SPSR, supported by both theoretical analysis and quantitative results, it lacks qualitative comparisons with SPSR. In contrast, SPSR provides extensive qualitative comparisons with PSR, as seen in Figures 1, 3, 7, 8, 18, and 20. These comparisons are crucial, as the contribution of speeding up SPSR is diminished if the method does not maintain the original reconstruction quality. I elaborate more on this in other sections.", "theoretical_claims": "The theoretical claims look good to me.", "experimental_designs_or_analyses": "I'm convinced by the theoretical analysis and results demonstrating the improvement in terms of the speed over SPSR, despite that, there are some limitations:\n\n1. A key contribution of the paper is replacing the computation of means and covariances with Monte Carlo sampling from the posterior, avoiding the need to store posterior covariances. However, the paper lacks discussion on how the number of Monte Carlo samples impacts both reconstruction speed and quality.\n\n2. A more fair qualitative comparison with SPSR is necessary to justify the quality claims, as mentioned in my comments under Claims and Evidence.", "relation_to_broader_scientific_literature": "The proposed method has the potential for broad applicability in accelerating various tasks, including surface reconstruction and ray casting in computer graphics, as well as collision detection in autonomous driving and human-robot interaction.", "essential_references_not_discussed": "To the best of my knowledge the essential related works are cited.", "other_strengths_and_weaknesses": "Overall, I acknowledge the paper's contribution to the community as a sped-up version of SPSR, with the local querying capabilities and mathematically principled approach.", "other_comments_or_suggestions": "-", "questions_for_authors": "I would appreciate it if the authors could provide a more in-depth discussion on Monte Carlo sampling, particularly its impact on reconstruction speed and quality. Additionally, a more comprehensive qualitative comparison with SPSR would help better support the claims made in the paper.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WBmrpJZ5Jn", "reviewer": "Reviewer_WF3M", "summary": "The paper improves the stochastic Poisson surface reconstruction [25], which combines the interpolation and surface reconstruction into a single stage. The method avoids the complicated finite element method and makes use of Fourier transformation. It also proposes to use Monte Carlo samples from the posterior to reduce memory cost. The paper also presents several applications, e.g., collision detection and ray-casting.\n\n## update after rebuttal\nI appreciate that the authors' rebuttal addressed some of my concerns. I maintain my score, which is positive for the paper.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your review! Let us address your points below:\n\n> \"Fourier domain analysis … limit the practical applicability of the methods?”  \n> “Why torus? Is there any topology constraint?”\n\nIn short: **no, it does not limit applicability**. This is because the periodic boundary conditions needed to leverage our Fourier methods apply to the **bounding box inside of which the point cloud sits, not to the point cloud or reconstructed surface** - the point cloud and surface do not need to be periodic in any way.\n\nMore specifically, to avoid periodic boundary conditions affecting reconstruction, we situate the point cloud sufficiently-far from the boundary to limit their effect: **this mirrors what is done in ordinary PSR**, which imposes other types of boundary conditions (say, Dirichlet or Neumann), and moves the point cloud far-enough away from the boundary so that it is largely unaffected by them.\n\nFrom a mathematical standpoint, the main reason for using periodic boundary conditions - and, therefore, the torus - is because (a) its Fourier analysis is much simpler compared to alternatives and (b) it works. One could instead try to develop a method using, say, properties of the sphere, but this would necessitate computations in terms of spherical harmonics, which are more complicated than the sines and cosines we use. We therefore think extensions like this are better left to future work.\n\n\n> “The evaluation criteria in the current work are rather simplistic … I suggest to design more evaluations and compare with [25] comprehensively, including, accuracy, scalable etc”  \n> “Since approximations are used during Gaussian process, the quantitative accuracies are expected in the experiments.”\n\nWe agree that improved evaluations would strengthen the paper (though with the caveat that we have followed the SPSR baseline paper’s lead and focused on qualitative evaluations because they are important in graphics applications), and to this end have **completed a number of additional evaluations looking at things like the effect of hyperparameters** on results, and timing experiments. We are additionally **aiming to add a quantitative next-view planning comparison** to the final manuscript draft. We describe these additional experiments to be added to the appendix - both the ones that are complete with results, and those still in progress - in detail in our response to Reviewer vrrd.\n\n\n> “Does the grid occupy too much space? What is the resolution?”\n\nWe use a grid of size $50^d$. As all of the experiments are done for $d = 3$, the grid size is ~500 KB of memory. More broadly, we agree with the importance of comprehensively evaluating how grid size affects performance, and have performed **additional experiments showing a too-small amortization grid results in over-smoothing**, which we will add to next manuscript version’s appendix.\n\n\n> “The experiments should valid the time and space on large point cloud comprehensively”\n\nThis is a good idea: **we have added an experiment, which uses points sampled from the Stanford Dragon mesh, and examines runtime**: we find that using 64 points takes a 15 seconds (much of it we suspect due to Python overhead), 4096 points takes about 30 seconds, and using 65536 points takes about 5 minutes. We will add a plot showing this to the next manuscript version.\n\n\n> “When Monte Carlo sampling is used to reduce memory consumption, how is the representativeness of the sampling results to the posterior distribution ensured? Is there statistical bias due to insufficient sampling times?”\n\nSince our posterior is a Gaussian process, we can use direct Monte Carlo sampling, which is by definition unbiased (as opposed to, say, other settings which require Markov chain Monte Carlo methods or similar). However, one might be concerned about variance of posterior functionals: here, it is a good question to ask how many samples are enough in practice, as is requested by Reviewer vrrd. \nTo address this, we will add **additional comparisons which show how transmittance calculations and collision detection performance varies with the number of Monte Carlo samples** to the next manuscript draft.\n\n\n> “It is possible to generalize the the method to screened Poisson surface reconstruction”\n\nThis is an interesting question! It seems highly plausible that an extension of our approach would work, but it would require analytically re-computing the relationship between the Karhunen-Loeve decompositions, which would be sufficiently-involved mathematically that we believe it is best to defer it to future work.\n\n---\n\n# Summary\n\nOverall, your suggestions have led us to **improve the writing** - in particular, to emphasize that our Fourier formulation does not restrict the kinds of surfaces we can reconstruct - as well as the **strength of evaluations** used for this work. On behalf of these additions, we would gently like to ask whether you would consider increasing your score.", "claims_and_evidence": "Fourier domain analysis relies on periodic kernel functions and boundary conditions, but actual point cloud data are often non-periodic. Does this assumption limit the practical applicability of the methods?\n\n\"one can view both the vector field and implicit surface as functions on the torus\". Why torus? Is there any topology constraint? Can the method deal with high genus models?", "methods_and_evaluation_criteria": "The evaluation criteria in the current work are rather simplistic, and as a result, they do not provide sufficient support for effectively assessing the methods in question. I suggest to design more evaluations and compare with [25] comprehensively, including, accuracy, scalable etc.", "theoretical_claims": "The paper proposed complex theories for stochastic Poisson reconstruction. During Amortized cross-covariance, $f_{k,v}$ is pre-computed on a grid and simple linear interpolation is applied to evaluate $f_{k,v}$. Does the grid occupy too much space? What is the resolution?", "experimental_designs_or_analyses": "The paper improves the time and space efficiencies of the original SPSR [25]. The experiments should valid the time and space on large point cloud comprehensively. However, only a few examples are demonstrated. The numbers of points are not presented. Since approximations are used during Gaussian process, the quantitative accuracies are expected in the experiments. Currently, the evaluations cannot support the claims very well.", "relation_to_broader_scientific_literature": "This is a very theoretical paper that uses the Gaussian process for Poisson surface reconstruction. I am not sure about the practical applicability.", "essential_references_not_discussed": "N.A", "other_strengths_and_weaknesses": "N.A.", "other_comments_or_suggestions": "N.A.", "questions_for_authors": "When Monte Carlo sampling is used to reduce memory consumption, how is the representativeness of the sampling results to the posterior distribution ensured? Is there statistical bias due to insufficient sampling times?\n\nIt is possible to generalize the the method to screened Poisson surface reconstruction?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper introduces a more efficient variant of Poisson Surface Reconstruction. It was well-received by the reviewers, converging to unanimous 4xWA. The evaluation was already positive pre-rebuttal, and mentioned concerns were mostly regarding clarity in writing, which have been addressed by the authors in the rebuttal, according to the reviewers.
I congratulate the authors to the great work and follow with an accept recommendation accordingly.
Decision: Accept (poster)