Paper: Implicit Gaussian process representation of vector fields over arbitrary latent manifolds (ICLR-2024)

Reviewer comment (Writing):
V is defined multiply: for vectors and for the nodes of the graph. One of the important statements "While G approximates M it will not restrict the domain to V" becomes unclear.

Author response:
Thank you for spotting this. We have removed the definition of V relating to vectors as it was unnecessary.

Reviewer comment (Theory):
The shape of $O_{ij}$ is unclear. It seems to be $m \times m$ for eq. 11, but it seems to be $d \times d$ for eq.12. (The rank of $O_{ij}$ will be m.) The shape of $L_{c}$ is also unclear. It is clearly defined as $nd \times nd$ in eq. 12, but it is inconsistent with $\Lambda_{c},U_{c} \in R^{nm\times nm}$. Maybe something is wrong.

Author response:
Thank you for spotting this inconsistency. The shape of the matrix Lc is nm x nm, which was a typo. Correspondingly, O_ij is m x m. Both of these operators involve the manifold dimension m because they act intrinsically over the manifold.

Reviewer comment (Evaluation):
A quantitative evaluation and analysis is missing for 5.1. I could not judge whether "smoothly resolved the singularity by gradually reducing the vector amplitudes to zero" is okay or not. Does this mean that the result is different from the ground truth? A quantitative evaluation is also appreciated.

Author response:
Thank you for requesting further clarifications on this matter. In Fig 2B of the revised version, we now include a quantitative comparison against a method that uses a channel-wise encoding of the vector signal with a radial basis function kernel. We find out our method provides overall higher accuracy and robustness to variation in the data quantity. Furthermore, this accuracy is achieved with fewer eigenvectors, which confirms the efficiency of our positional encoding.

With this additional quantification, Fig 2C now provides a pictorial illustration of the errors stemming from an incomplete satisfaction of the tangency and smoothness constraint by the competing channel-wise RBF method.

Reviewer comment (Writing):
Something wrong: {($x_i$}), $\hat{v} = ∥_{i=0}^{n}\hat{v}_{i}$.

Author response:
Thank you for spotting the lack of clarity in this paragraph in the previous two bullet points. We have slightly rewritten the paragraph to clarify the construction of the smooth vector field and fixed two minor typos.

Reviewer comment (Writing):
I could not understand what the authors were doing: "We then sampled corresponding vectors {$v_{i}$} from a uniform distribution on the sphere".

Author response:
Thank you for spotting the lack of clarity in this paragraph in the previous two bullet points. We have slightly rewritten the paragraph to clarify the construction of the smooth vector field and fixed two minor typos.

Reviewer comment (Writing):
$S^{3}$ should be $S^{2}$.

Author response:
Thank you, corrected.

Reviewer comment (Experiments):
The reviewer has some doubts about the term "unknown manifold". For 5.1, the training data is dense enough to approximate the manifold well. For 5.2, the authors "constructed RVGP kernels using the connection Laplacian eigenvectors derived from a k-nearest neighbour graph (k = 5) fit to mesh vertices". It seems that the Laplacian was computed from 256 vertices, not n=61. If I understand correctly, this violates the prerequisite of "unknown manifold". Is there no value in analyzing the relationship between the coverage of the data for the manifold and the accuracy of the vector field prediction? I'm also curious about the analysis of the predicted vector field for out of the manifold to check for regularity.

Author response:
We thank the Reviewer for bringing up this important point. By ‘unknown manifold’, we mean that the manifold is latent, and we only have access to it through a number of sample points, and there is no a priori parameterization. Our method will implicitly build a manifold irrespective of the sampling density. If the sampling density is too low, the manifold will be an approximation based on the thresholding of the spectral decomposition of the connection Laplacian. We have clarified this in the Discussion.
For 5.2, the authors "constructed RVGP kernels using the connection Laplacian eigenvectors derived from a k-nearest neighbour graph (k = 5) fit to mesh vertices". It seems that the Laplacian was computed from 256 vertices, not n=61. If I understand correctly, this violates the prerequisite of "unknown manifold". 

As the Reviewer correctly points out, in example 5.2, the manifold graph was constructed with the known 256-channel xyz locations, from which our method constructs the underlying latent manifold. The manifold is still unknown because the tangent spaces and connections need to be approximated based on the available data and are not given by an explicit parameterisation (which would exist for canonical manifolds such as the sphere or torus).

Reviewer comment (Theory):
I found the section "Vector-field GP on arbitrary latent manifolds" difficult to follow. For example it is not clear to me what $(U_c)_i$ is. To me this would denote the $i'th$ row, but it clearly is not as it is the wrong shape. Also in equation 15, $\Phi(\Lambda_C)^{-2}$ is a $\mathbb{R}^{mn \times mn}$ matrix, but is being producted with $P_v$, a $\mathbb{R}^{d\times k}$ matrix?

Author response:
We thank the Reviewer for pointing out the lack of clarity in this section. We believe the Reviewer’s confusion is due to points in the tangent bundle representing vector spaces of dimension m rather than points. Therefore, the positional encoding will be a matrix rather than a vector. The index i can be considered a slice of an n x m x k tensor. We have revised the manuscript to clarify this.

Thank you for pointing out this typo. \Phi(Lambda_c) should be \Phi((Lambda_c_{1:k,1:k})) since we are taking the first k eigenvalues.

Reviewer comment (Reproducibility):
Can you explain the procedure of constructing the kernel in "Vector-field GP on arbitrary latent manifolds"?

Author response:
We construct the kernel over the tangent bundle in Equation 15 by direct analogy to the kernel over the manifold in Equation 5-6 by replacing the Laplace Beltrami operator acting on scalar signals on the manifold with the higher-order connection Laplacian operator acting on vector fields over the manifold. This provides a constructive and fully intrinsic definition, i.e., relying on the local Riemannian metric. We will elaborate on this below in the next response.

Reviewer comment (Novelty):
How does this kernel differ from using the method of Hutchinson et. al. more directly? I.e. Using the scalar kernel defined by the graph laplacian, $k(i,j)$, from this creating a diagonal kernel $ K(i,j) = k(i,j) * I_{d\times d}$, and then restricting this to the estimated tangent spaces, $\mathbb{K}(i, j) = \mathbb{T}_i K(i,j)  \mathbb{T}_j$

Author response:
Thank you for drawing our attention to a more detailed comparison to Hutchinson et al. First, let us point out that our kernel cannot be obtained by the procedure suggested by the reviewer, i.e., expanding the kernel defined by the graph laplacian, k(i,j), and then restricting to the tangent spaces. This is because the graph Laplacian and the connection Laplacian are not related by linear transformations. Instead, they share a non-linear relationship given by the Weitzenbock inequality, L_c - L = A, where A is an operator depending on the Ricci curvature of the manifold (note that we used Lc and L to define continuous operators obtained as limiting objects of our discrete formulation).

Regarding the comparison to Hutchinson et al., while their construction relies on a non-linear but isometric projection to Euclidean space and defining the GP therein, our method builds the GP intrinsically over the manifold. By intrinsically, we mean using the connection Laplacian operator, which is defined in terms of the Riemannian metric.

Thus, our method is more robust in practice since the connection Laplacian operator can always be constructed from local connections, whereas a global Euclidean isometric projection may be less reliable to approximate.

We have added additional explanations in the text.

Reviewer comment (Experiments):
One advantage of GP is data efficiency. In section 5.1's superresolution experiment RVGP is trained using vectors over 50% of the nodes, I would be interested to see the results when trained with less data.

Author response:
We thank the Reviewer for highlighting this point. In Fig 2B, we already vary the density of data points, hence the amount of training data, by changing the resolution of the surface representation. We showed that there is no significant drop in accuracy when we vary the pairwise distance of sample points relative to the object (bunny) diameter. Increasing from 1% to 8% effectively uses less data and results in a smoother bunny surface due to the lower resolution. 

To address the Reviewer’s concern, we have now added a benchmarking of our method, which is based on a vectorial representation using the connection Laplacian, against an alternative GP method based on a channel-wise representation using the graph Laplacian and a radial basis function kernel. Unlike the benchmark, our method shows no significant variation against resolution and can represent the vector field with significantly fewer eigenvectors (Fig 2B).

Reviewer comment (Experiments):
In section 5.2 the authors compare RVGP with interpolation methods, which might be a too simple baseline comparison. The RVGP's performance is also close to linear prediction. I would be interested to see the comparison against a stronger baseline method.

Author response:
Thank you for pushing us to perform further benchmarking of RVGP. We chose linear and spline interpolation as benchmarks since there are broadly adopted approaches in current experimental and clinical practice.

At the request of the reviewer, we now include a comparison of our vector-based method against a channel-wise radial basis function (RBF) kernel. We show in Fig. 3D,E that this method offers higher performance than linear and spline interpolation but is subpar compared to RVGP.

Reviewer comment (Presentation):
A section discussing the limitations of the proposed method would be good.

Author response:
In the revised version, we now include an expanded Discussion section detailing the limitations and future possibilities.