Improving Neural Network Surface Processing with Principal Curvatures

Authors: josquin  harrison
james  benn
maxime  sermesant
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 15:55:59

Abstract: The modern study and use of surfaces is a research topic grounded in centuries of mathematical and empirical inquiry. From a mathematical point of view, curvature is an invariant that characterises the intrinsic geometry and the extrinsic shape of a surface. Yet, in modern applications the focus has shifted away from finding expressive representations of surfaces, and towards the design of efficient neural network architectures to process them. The literature suggests a tendency to either overlook the representation of the processed surface, or use overcomplicated representations whose ability to capture the essential features of a surface is opaque. We propose using curvature as the input of neural network architectures for surface processing, and explore this proposition through experiments making use of the shape operator. Our results show that using curvature as input leads to significant a increase in performance on segmentation and classification tasks, while allowing far less computational overhead than current methods.

Main paper: 
 introduction: 
 Surfaces are a natural representation for many real world objects ranging from organs and organisms to archaeological artefacts. They are also a central tool in virtual environments such as computer games, or computer-aided design. This ubiquity has resulted in a large body of work dedicated to mathematical methods developed for the efficient use of surfaces, as well as their analysis.The goal of traditional computational surface analysis is to find a representation of a surface that is expressive enough to capture details relevant for the problem or task at hand, while being computationally light-weight. However, the effectiveness of Convolutional Neural Network (CNN) in image processing opened new doors to surface processing. The design of efficient convolution-like operations to adapt neural networks (NN) to surfaces alleviated the need for complex and detailed representations, to the point where most state of the art architectures use extrinsic vertex coordinates as input, letting the NN models learn the surface structure at multiple scales. While some attempts were made to use well known representations as inputs, yielding some increase in performance [38, 35] , the general consensus is that the model should be able to learn it by itself [25] . While it is true that neural networks are efficient at capturing surface features at multiple scales, the use of a local surface representation that is more expressive and more natural to interpret than extrinsic coordinates should naturally improve the performance of the network. The optimal choice of representation should be somewhere between coarse extrinsic vertex coordinates, and more complex representations.In Riemannian geometry, the shape operator is the main tool linking the intrinsic geometry of a surface with its bending and curving in ambient space. Its eigenvalues are the principal curvatures. Their product and evenly divided sum give precisely the Gauss and mean curvature of the surface, respectively. We hypothesize that the optimal choice for a local surface representation that meets the requirements of surface processing is the set of principal curvatures: they characterize the surface up to isometry (location and orientation in space); they are purely local, which allows the neural network to decide on more general surface features; and they are lightweight, leaving very little computational overhead in any scenario. This work tests our hypothesis against two widely used representations of surfaces in three state of the art NN architectures.In the next section we give an overview of surface processing, and introduce shape representations and learning methods. In section 3 we give an introduction to the Shape Operator, although we should remark now that this is not the first appearance of the shape operator in the surface processing literature. The shape operator has already become an efficient tool in surface processing and is, among other things, used to define local tangent frames and compute surface features like creases [31] . Following these introductions, we then conduct extensive experiments in section 4, comparing principal curvature with three other representations in conjunction with three different NN architectures, on two segmentation datasets and one classification dataset, that shows how principal curvature enhances any state of the art model in different tasks. In addition to outperforming other methods, we show that this more concise representation is faster to compute, leaving minimal computational overhead when added to a pipeline. 
related work: 
 The first step to surface processing is usually its discretisation as a mesh or point cloud, which is particularly useful for visualisation or rendering. From this starting point, novel representations have been derived in an effort to provide tools for different surface related tasks. These tasks include surface matching, semantic segmentation, classification, or even shape retrieval.Historically, the general trend has been to find compact descriptors of a shape which could be then compared within a dataset. A long list of such descriptors exist, among them signaturebased descriptors, such as Heat Kernel Signature (HKS) [42] or wave kernel signature [4] , proved to be particularly efficient. Closely related are histogram methods, which are often combined with signatures to provide expressive representations such as the SHOT [37] or the Echo [26] descriptors. Geometric measure theory has also been a source of inspiration for developing efficient representations, such as geometric currents [5] leveraging on finite elements, or kernel-based currents [43] and varifolds [10] tailored for shape deformation. Such representations can be used in conjunction with classical statistical analysis tools, e.g [26, 37, 5] , although they are often building blocks for specialised methods on surfaces such as LDDMM [49] , functional maps [29] or spectral-based analysis [45] .With the advent of deep learning, many methods previously stated were re-written with the help of neural networks resulting in Deep functional maps [20] , and ResNet-LDDMM [2] , to name but a few. Representations of surfaces themselves were proposed as neural networks, such as DeepSDF [32] or DeepCurrents [30] . As convolutions proved particularly effective when learning on images, i.e structured grids, some work proposed voxel-based solutions to the study of surfaces [23] . Others suggested representing surfaces as geometric images [40] , on which convolutions can be applied. A second generation of geometric deep learning has focused on building network architecture specifically tailored to work directly on surfaces, i.e meshes or point clouds. From a point cloud perspective, Point Net [34] and its extension Point Net ++ [35] consider the surface as a set of points by applying set operations on them. Among others, DGCNN [46] applies a convolution-like operation on dynamic graphs constructed layer-wise. MeshCNN [16] on the other hand fully leverages the mesh structure to develop operation unique to triangulations. Transformer based architectures have also appeared for the specific purpose of surface processing [18] . Among them, and in a similar vein as before, GaTr [8] proposes to represent geometric data in an algebra of choice and designs an architecture with operations belonging to this algebra. An effort to have efficient generalisation of convolutions on surfaces was proposed by [22] , although the lack of global coordinates creates ambiguity in local operations. To alleviate this problem, a large body of work has proposed rotation equivariant operations, namely GemCNN [12] , augmenting graph NNs, or field convolutions [25] . Finally, recent models propose to bypass the problem of generalising convolutions by focusing on well defined operations on surfaces, such as discrete exterior calculus in Hodge Net [41] , heat diffusion in Diffusion Net [38] or a suit of known operators in Delta Net [48] .As model architectures include more and more knowledge of shapes, the need for a better representation of the input to these models has decreased. Outside of models that contain operations proper to the structure of choice (e.g [16, 8] ), most models naturally accept as input the coordinates at every point. Some papers propose to augment the model by inputing higher dimensional descriptors initially designed for a more direct analysis, such as the ones previously mentioned (e.g HKS, WKS, SHOT). Such proposals can be seen in [38] , where HKS interacts well with the diffusion part of the architecture, or [35] were they combine HKS, WKS, Gaussian curvature through concatenation and PCA. However, recent work has dismissed this idea [25] , citing the results of Diffusion Net [38] which show no great improvement when moving from coordinates to HKS.As methods for learning on surfaces have evolved, we suggest that a better input representation is a simpler one, yet is more expressive than coordinates. We suggest that we can scale back to the simplest differentiable invariant of a surface: its curvature.The Gauss and mean curvatures can be equivalently interpreted as the determinant and half the trace of [S p ], respectively.The importance of these quantities is two-fold: (1) two surfaces differ only in location and orientation in space if and only if they have the same principal curvatures (Theorems 9.1 and 9.2 in [28] ) -that is, the shape operator completely characterizes the shape of a surface; and (2) the Gauss and mean curvature generate all possible differential invariants of a surface (see Guggenheim [15] , Olver [27] )in particular, Gauss and mean curvature are fundamental characteristics of the shape of a surface, and the inclusion of the higher order invariants they generate into a representation could even improve the results shown here.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: We perform a comparative study and experimentally highlight what we believe to be the best methodology.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
the shape operator: 
 Here we give a conceptual introduction to the Shape Operator and describe how its eigenvalues completely characterize the surface to which it belongs (section 3.1). In section 3.2 we describe an explicit calculation of the Shape Operator which igl's implementation of the principal curvatures is based on -this is the implementation we use in our experiments (see section 4.1). Those already familiar with the differential geometry of curves and surfaces may skip ahead to section 4; for others this section serves as a concise introduction -although, we do rely on a basic understanding of functions of several variables and their derivatives, and surfaces and their tangent spaces.Surfaces in R 3 will be denoted by S and S, points in surfaces by p's and q's, and the tangent space to S at a point p ∈ S by T p S. Maps from R 3 to itself will be denoted by F : R 3 → R 3 , and their derivatives at a point p by DF p -the Jacobian matrix. A parameterisation X of a smooth surface S is a diffeomorphism between an open set U ⊂ R 2 and an open set V ⊂ S, and provides a mathematical description of S as it lies in R 3 . The standard Euclidean inner product on R 3 will be signified by ⟨•, •⟩, and it's restriction to a surface S and its tangent bundle T S = p∈S T p S by g S (•, •), which we call the induced metric. A normal vector to S at p is one which is orthogonal to every vector v in T p S (measured in ⟨•, •⟩) and will be denoted by N p ; if we have a field of normal vectors in an open set around p then this field will be denoted simply by N .The Shape Operator of a surface S at a point p ∈ S measures the rate at which surface normal vectors N separate around p, which is precisely the bending of the surface in space: Definition 1 Given a point p on a surface S ⊂ R 3 , and the unit normal vector N defined on a neighbourhood U of p, the shape operator is the linear map S p : T p S -→ T p S v -→ -∇ v N, where T p S denotes the tangent space of S at point p In other words, the shape operator S p tells us how the normal vector changes as we move in S, in the direction of v from p. One possible way to visualise the shape operator is through the Gauss map, which identifies each point p ∈ S ⊂ R 3 with its unit normal vector N p , now thought of as a point in S 2 . The shape operator is then the differential of the Gauss map at p and is a tangent vector to S 2 at the image N p of p, as illustrated in figure 1 .The operator S p is linear for each p ∈ S, and self-adjoint in the Euclidean inner product ⟨•, •⟩: ⟨S p (v), w⟩ = ⟨v, S p (w)⟩.(1) It can therefore be represented by a symmetric 2 × 2 matrix [S p ] : T p S → T p S at each point p ∈ S. It is well-known that symmetric matrices admit a complete system of orthonormal eigenvectors (e 1 , e 2 ) spanning the space on which they act. The matrix representation [S p ] with respect to the basis (e 1 , e 2 ) has the simple form:EQUATION)Figure 1 : The shape operator may be visualised via the Gauss map.Where κ 1 and κ 2 are the eigenvalues of S p .Definition 2 Let S be a surface in R 3 , p a point in S, S p the shape operator at p and [S p ] its matrix representation.1. The eigenvalues κ 1 (p) and κ 2 (p) of [S p ] at p are the principal curvatures of S at p, and their corresponding eigenvectors e 1 and e 2 are the principal directions;2. The Gauss curvature κ of S at p is the product κ 1 (p) • κ 2 (p) of the principal curvatures;3. The mean curvature H p is the average κ1(p)+κ2(p)To explain how Gauss and mean curvature completely describe the shape of a surface we need a few more definitions.An isometry of R 3 is a map F : R 3 → R 3 whose differential preserves the angles between tangent vectors at every point of R n :EQUATIONIf g S is the Riemannian inner product induced on T S by the Euclidean inner product ⟨•, •⟩ then an isometry between two surfaces is a map η : S → S whose differential preserves the angles between tangent vectors to S:EQUATIONEvery isometry F of R 3 restricts to an isometry of surfaces F | S = η : S → F (S), but the converse need not be true, unless an additional hypothesis on the shape operators is satisfied.Two surfaces S and S are congruent if there exists an isometry F : R 3 → R 3 such that F (S) = S; that is, congruent surfaces are surfaces which differ only in their location and orientation in space. It is clear that the shape operators S and S of two congruent surfaces are related byEQUATIONin particular, the matrices [S p ] and [S p ] are conjugate to one another via [DF p ]. As per Theorem 9.2 of [28] , if there exists an isometry η : S → S such thatEQUATIONi.e. such that the matrices [S p ] and [S p ] are conjugate to one another via the matrix representation [Dη p ], then there exists an isometry F : R 3 → R 3 such that F | S (S) = η(S) = S, and the two surfaces are congruent. The conclusion of this brief mathematical digression is that two congruent surfaces have the same intrinsic geometry and shape in space, and two surfaces with the same intrinsic geometry and shape in space are congruent. This is what is sought after when representing shapes with intrinsic quantities.As well as being an important theoretical tool, curvature is a central notion in mesh processing. A large body of work has been dedicated to estimating its discrete counterpart. Among them, many methods propose to infer Gaussian curvature directly, such as in [24] , or involve the use of geometric measure theory [13] , as in [11, 17] . Interestingly, many efficient methods propose to first discretize the shape operator in order to compute the Gaussian curvature from it. This is done either directly on the mesh triangles, such as in [36] , or by first locally fitting a function to the surface, and then computing explicitly the shape operator. To get a better feel for why this is a natural construction of the shape operator, consider a surface S, given a point p ∈ S. Then the surface around p can be parameterized asX(u, v) with (u, v) ∈ R 2 .The inner product at T p S, also called the first fundamental form, is then given for any two tangent vectors v, w by:EQUATIONwhereE = ⟨∂ u X, ∂ u X⟩, F = ⟨∂ u X, ∂ v X⟩ and G = ⟨∂ v X, ∂ v X⟩.And the surface normal at p can be defined asEQUATIONWe can now define the second partial derivatives of X in the normal direction n, a quantity called the second fundamental form, noted II:EQUATIONwhere L = ⟨∂ uu X, n⟩, M = ⟨∂ uv X, n⟩, and N = ⟨∂ vv X, n⟩. The partial derivatives of the surface normal can then be expressed via the Weingarten equations, in terms of the components of the first and second fundamental form:∂ u n = F M -GL EG -F 2 ∂ u X + F L -EM EG -F 2 ∂ v X ∂ v n = F N -GM EG -F 2 ∂ u X + F M -EN EG -F 2 ∂ v X.This enables us to write the matrix form of the shape operator at p as:EQUATIONFrom these derivations, it becomes interesting to find good local parametrisation of surfaces, that is, a bi-variate scalar function f such that:EQUATIONThe shape operator can then be easily derived from the first and second derivatives of f . An efficient way to find such functions is via osculating jets, proposed in [9] . For the following experiments, we use a multi-scale version of this, proposed in [31] , in which the shape operator is computed by using neighbourhoods of varying size around a point, yielding a robust method for estimating curvature on a mesh.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? 
experiments: 
 We test the representation of surfaces by the principal curvatures κ 1 , κ 2 and Gaussian curvature κ against the three most commonly used representations: the HKS [42] , the SHOT descriptor [37] , and All representations are tested with three different architectures. We regard Diffusion Net [38] as the state of the art in NN architectures, as it shows the most promising results on general benchmark tasks. In addition, it shows very little difference in performance when changing the input from coordinates to HKS, making it the hardest test for our representation. Point Net ++ [35] has been designed as a general method to process shapes arising in many situations, including controlled environments -as in our case -but also from segmented images encountered in the autonomous driving field [35] . As such Point Net ++ uses the least geometric structure to describe a surface: all one needs is a point set. We believe that in this case, using better surface information for the input will greatly enhance the performance of the model. The authors of PointNet++ have already touched on this subject, recommending a linear combination of HKS, WKS and Gaussian curvature, followed by a PCA projection, leading to a 64 dimensional feature per point. We aim to show that a 2d (or even 1d) input of curvature information is more relevant for a smaller computational cost. Delta Net [48] proposes an architecture intrinsic to surfaces by design, by combining four operators defined on the surface: Laplacian, divergence, curl and norm. Most papers that propose other surface descriptors rather than coordinates as input, do it solely to have an intrinsic representation of the surface. Curvature gives isometry invariance (section 3), and we further believe it is also more robust, numerically. Better performance from a curvature based representation in this architecture would support this belief.Finally, we pick three tasks of varying complexity to measure the impact of each method: human segmentation [21] , molecular segmentation [6] , and shape classification [19] . Examples from each dataset are shown in figure 3 .The performance of each representation is strongly dependent on the chosen implementation. We have tried to be as fair as possible by not developing our own implementations of existing work and instead using implementations which have already been tried, tested, and validated in the literature.For calculating the discrete principal curvatures via quadratic surface fitting, we have used igl's implementation with a fixed neighbourhood radius of 5; the Gaussian curvature κ is then computed directly as the product of κ 1 , κ 2 . HKS depends on the Laplacian, and we have used the method implemented in robust-laplacian based on [39] -this is also consistent with what is used in Diffusion Net. The eigendecomposition of the Laplacian is then performed with scipy. For the Regarding the neural networks, we use the implementations made publicly available by the authors, modifying only when needed to accommodate more than just coordinates as input. We also use the same parameters proposed in each paper when they are known, which we detail for every task below. We make all our code and experiments available at https://github.com/Inria-Asclepios/ shape-netsAs a first experiment, we compute1 for each representation method, the computation time as a function of the number of points in a surface. The performances are reported in figure 4 . HKS and curvature are both efficient for meshes with up to 100k points. However, curvature is consistently faster, even for larger meshes (up to 500k points), displaying the very small overhead incurred by the use of curvature in surface models.Figure 4 : Time of computation for each representation with respect to the number of points in a mesh.We first segment the human parts from the composite dataset proposed in [21] , containing samples from other human dataset, namely FAUST [7] , SCAPE [3] , Adobe [1] , MIT [44] , and SHREC07 [14] .As in the original paper, we use the SHREC07 dataset as test set. Similar to [47] , we differ from [21] by evaluating on vertices rather than faces. For Point Net ++ and Delta Net we resample each shape to 1024 points, and we leave the meshes untouched for Diffusion Net, as per the experiments conducted in each paper. We optimise the negative log-likelihood for 100 epochs, with the ADAM optimiser and a scheduler step every 20 epochs. We ran the experiment 5 times and have reported the mean test accuracy in table 1 . The results highlight the assumption that better representations lead to better performance. PointNet++ showed the greatest improvement when moving away from coordinates: this is due to its architecture having the least amount of geometric information at baseline. The better results come from the principal curvatures, and show how expressive this representation is. The effects of the principal curvatures are even more pronounced in the Delta Net experiment, where κ 1 , κ 2 greatly outperform all other methods. It's interesting to note that in this experiment the coordinate representation performs better than the other more complex representations. Diffusion Net may show that it is more robust to the type of input, as long as it loosely describes the shape, however the improvement brought by the principal curvature is still significant. To further demonstrate the impact of a good representation, even in the case of Diffusion Net, we show in 5 the worst cases for xyz and κ 1 , κ 2 inputs. The clear improvement in this case may be even more important than general accuracy in some cases, e.g with human-in-the-loop type corrections. xyzThe molecular dataset made available by [6] and first proposed in [33] , can be considered a harder segmentation task then the Human part dataset: it proposes a wider range of shapes in the form of RNA molecules, and a 260-way part segmentation task. We resample all meshes to 2048 points, except in the case of Diffusion Net where we kept the original discretisation. We evaluate all our baselines on 5 random splits with a train-test ratio of 80-20. We run the models for 200 epochs, and report the mean test accuracies in table Again, we see a significant improvement when using a better representation of the surface in the case of PointNet++, going from failing in the case of coordinates to outperforming Delta Net -with principal curvatures giving the best performances. Diffusion Net shows a non-negligable jump in performance as well. Although the SHOT descriptor outperforms other representations in the case of Delta Net, the general performance of this architecture is underwhelming. We believe this is due to the accumulation of errors in the discretisation of surface operators used. Indeed, one layer computes a chain of 6 operators on the surface: since the RNA shapes are very irregular, the error for each operator could be significant.In addition to segmentation tasks, we propose to compare representations in the context of classification. This experiment should show whether or not geometrically informative inputs interact well with pooling-type operations. We choose the widely adopted baseline Shrec11, proposed in [19] . It is a 30-way classification dataset with 20 shapes per category. We choose the simplified mesh dataset and the harder version of training, using only 10 samples per class and evaluating on the test. We perform our experiments on 5 random splits. We train our baselines for 100 epochs with a scheduler step at epoch 50 and optimise the cross-entropy loss with a label smoothing factor of 0. Yet again we observe a significant improvement when turning to better representations, even more so when using Gaussian curvature κ. Additionally, figure 6 shows that all geometric representations yield less variability across each folds. In addition, HKS, Gaussian curvature κ, and principal curvature κ 1 , κ 2 converge much faster than all others. The fact that gaussian curvature, closely followed by HKS outperform principal curvature in this classification task seems to indicate that Gaussian curvature interacts better with pooling operations present in classification architectures. Interestingly, all three architectures tested here have different ways of performing the pooling operation. Although it is hard to give any analytical reasoning to this behavior, we believe it is simply the fact that gaussian curvature is already an aggregation of the principal curvatures, that it shows better performance in classification tasks.For each experiment, additional metrics can be found in Appendix A.1.We propose one final experiment to highlight the robustness of input features to noisy data. We focus on three representations: HKS, known to be robust to noise as it computes a representation at multiple scale; extrinsic coordinates that are directly impacted by the noise; and principal curvatures, known to be less robust to noise as a purely local descriptor. To compare these representations we pick the diffusion net trained on the human pose dataset, and we add noise to the dataset at inference time. Specifically, we add gaussian noise with a standard deviation of 1%, 3%, 5%, 7% and 10% of the diagonal length of the bounding box of each shape. Examples of the noisy data can be seen in Appendix A.2. Results show that the accuracy for all three features worsen at the same rate, as shown in figure 7 , showing that principal curvature can be a viable choice even in the presence of noisy data. Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: See section 4. Moreover, a repository with all the code and experiments will be made available after the review process. 
conclusion: 
 In this work we have shown that curvature should be the representation of choice when it comes to processing surfaces with neural networks. In almost all experiments the principal and Gaussian curvatures performed better than any other choice of input, both qualitatively and quantitatively. In particular, this representation can be obtained with minimal computational overhead. Its combination with PointNet++, the architecture that has the least prior information about the surface, showed that it can help the network better understand the surface structure. When combined with Delta Net, which contains only intrinsic operations, the improvement indicates that curvature gives more than just a rigid transformation invariance. Even in the case of Diffusion Net, where the diffusion operation seems to interact nicely with any representation, curvature as input showed significant amelioration. For these reasons, we believe curvature should become the standard practice when using models to learn on surfaces. Finally, although experiments have shown that gaussian curvature outperforms principal curvatures on classification tasks, we would like to further define those guidelines in future work, as well as compare representations in a wider range of tasks and architectures.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: Link will be available after the review process.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: See section 4Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: See Appendix A.1.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [No] Justification: Not completely. Section 4.2 shows the time complexity of each representation. We do not provide the computational cost of other experiments. However, we point out that they take very similar time to the one indicated in papers describing each architecture.Guidelines:• The answer NA means that the paper does not include experiments. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? 
 
 Appendix: We present below the complete results for each experiment. For each dataset, and each neural network architecture, we show the Accuracy, Balanced accuracy, F1 score, and Specificity. They were all measured using 5-fold cross validation, and we give the results on the test sets in the form of mean ± standard deviation. NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Justification: There are no theoretical results.Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced.• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.Justification: There is no societal impact to discuss.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.38th Conference on Neural Information Processing Systems (NeurIPS 2024).Computed on an Apple M2 chip 


Decision: To characterize shape properties the authors proposed to use the principal curvatures and/or mean curvature. They demonstrated that these descriptors can be efficiently computed and leads to a significant increase in performance on segmentation and classification tasks for three very different neural network pipelines when used as input. The proposed recipe is very easy to implement.

The main achievement of the article is a very simple practical recipe. Since the approach is empirical, its validity should be assessed under different conditions as much as possible. Despite the positive reviews, I would insist on adding additional important experiments, see comments below.

In fact, 
- the authors performed testing using relatively old and not very representative datasets such as shrek
- they did no study how robust the results are with respect to changing the hyperparameters of the considered neural network architectures
- the authors did not study to what extent the results change in the presence of noise in the data, how this effect depends on the noise level. It is important to discuss how to deal with noisy data and to which extent the noise corrupts the observed effects
- is this approach suitable only for 3d shapes? will it work efficiently for 3D models of mechanical parts?

I personally would recommend to consider more datasets to verify the proposed approach, e.g. using so-called ABC dataset (collection of one million Computer-Aided Design (CAD) models) to predict sharp features (see the follow up paper DEF of the same authors).

Reviews: review: paper_summary: paper_summary: The paper proposes to use curvature instead of previous surface descriptors in neural networks that process shapes. They show that the principal curvatures and/or mean curvature are better surface descriptors for three very different neural network pipelines, and is much faster to compute. They point out previously known properties of curvature for why curvature makes for such good shape representation.
strength_weakness: strength_weakness: - The reduction in performance for the principle curvatures for classification is a bit puzzling, especially since Gaussian curvature can trivially be computed from the principal curvatures. Surely the complexity of Delta Net and Diffision Net can cope with that, but there is a large performance reduction compared to HKS. The paper mentions that it indicates that Gaussian curvature interacts better with pooling, but that is not really justified. In the task HKS performs a lot better than shot16/64 indicating that there is some difference in the task. Given that this paper is more of a discovery of the benefit of a well known quantity, I would expect more analysis into this. Especially since for the other two tasks, Gaussian curvature performs worse than the principle curvatures, and intuitively this made sense as it decreased the amount of information, but then discarding this argument for the classification task is not appropriate.
- It would be nice to also benchmark against the pipeline PointNet++ recommended, the "linear combination of HKS, WKS and Gaussian curvature, followed by a PCA projection, leading to a 64 dimensional feature per point"
questions: questions: -
limitations: limitations: None given, and in the checklist have said it is not necessary given it is a comparative study. However given they also try to motivate why curvature is approriate theoretically and it depends whether to use principle curvature or Gaussian curvature, it would be nice for them to either try to identify when to use principal or Gaussian curvature(s) (theoretically or hypothesis-wise)
paper_summary: paper_summary: This paper proposes to use surface curvatures as input to neural networks to improve performance on 3D tasks. The main hypothesis is that as the curvatures are intrinsic properties of the surface, they will enable more effective learning for the relevant tasks. The paper carries out some evaluation comparing performance of network with curvatures as input against existing approach that uses point coordinates or other intrinsic properties as input. The proposed approach achieves better results on a few category of tasks.
strength_weakness: strength_weakness: - The use of surface curvature in neural networks is not really new, so the novelty is limited.

- The paper fails to discuss some inherent limitations when using curvatures as input. e.g.
   *) Surface curvature requires 2D manifold structure. In a lot of applications, we are dealing with point clouds rather than a well-defined surface structure such as meshes. On point clouds with thin structures, some points not in the neighborhood of a point on the manifold may be closed actually close to the point in 3D space, causing difficulty in correcting identifying the neighbors and affecting the accuracy of curvature computation.
   *) As a high-order differential property, curvature is sensitive to noises.
   *) The signs of curvatures are dependent on the orientation of the surface (i.e., among the two possible normal directions, which one points to the outside and which one points to the inside). Given a point cloud, it is non-trivial to properly orient the normals to enforce a globally consistent orientation. If some points on the point cloud have incorrect orientation, their curvatures will have negated signs and may mislead the neural network.
   These real-world scenarios can cause difficulties when using curvature rather than point coordinates as input to neural networks. There is insufficient evaluation on such real-world cases.

- In recent years, transformers have shown better performance than convolution networks on many tasks. There is a growing body of transformer-based networks for 3D tasks (see https://github.com/lahoud/3d-vision-transformers), but there is no comparison with such methods in this paper.
questions: questions: - How well does the method perform when dealing with noisy data?
- How well does the method perform when compared against transformer-based architectures?
limitations: limitations: There is no discussion on limitations. In particular, the dependencies on orientation, the difficulty of orienting the surface globally, and the sensitivity to noises are not discussed sufficiently.
paper_summary: paper_summary: The paper promote to use the principal curvature as the surface presentation that can be better used in the modern neural network architectures. To support their hypothesis, the paper compares three different representations: HKS, SHOT descriptor and extrinsic coordinates in the use by PointNet++, Delta Net and Diffusion Net. Among all the providing experiment figures, principal curvatures provides the best performance.
strength_weakness: strength_weakness: - Quality: 
  - Although section 3 provides the way to calculate the principal curvature, it takes up too much space but does not adding equivalently significant value to the paper, and thus appear to be a bit redundant.
  - The mathematical presentation in the paper is quit disorganized. Most of the formulation are not indexed. The definitions are given without a term [line 104, line 118]. 

- Incompliteness:
  -  Section 6 Acknowledgement is empty, which becomes another major drawback regarding to the presentation quality.

- Limited Contributions:
  -  The surface representations being analyzed in the paper is relatively limited. The task and the methods are not new, not does the paper provide an intelligence-stimulated combination of the existing method.
questions: questions: Refer to the weaknesses.
limitations: limitations: Refer to the weaknesses.

Comments: title: Response and additional comment
Additional comments for the review?
Any comments?
Keeping my original score, but hope the authors are interested in improving their paper.
comment: We thank the reviewer for taking into consideration our rebuttal, and are happy they raised their rating.
We thank the reviewer for this additional feedback. Their concerns now appear clearer although we might have misunderstood some claims. We are happy to address them below: 

Novelty and contributions of the work:
- ⁠Laplace mesh processing has been addressed in the paper; in fact, the HKS representation is the most successful representation stemming from this line of work, and in our paper we show that the principal, Gaussian, and mean curvatures outperform the HKS representation. 
- While the Laplacian is linked to curvature (albeit through an equality relating two 2nd order differential operators), we are specifically suggesting in our work that curvature alone is a better suited input to neural networks processing shapes, which is strongly backed by our experiments. To the best of our knowledge, this has never been addressed in existing literature. Therefore (this could be a misunderstanding from our part) we do not agree with the claim that our work has entirely been done in the realm of spectral mesh processing. On the contrary, we believe our work is novel and contributes to the field of shapes processing.
- We thank the reviewer for the references they have provided. We will be happy to extend the contextualization of our work by adding a paragraph on transformer-based methods and including the suggested references.

Notation and the mathematical aspect of the paper:
- The concerns of the reviewer about notations are now clearer. We will be happy to address them here (and in the revision of our paper) to facilitate the understanding of the mathematical aspects of our paper. Nonetheless, we would like to point out that we have used common notations surrounding the well-studied subject of curvature, and have also suggested Guggenheim [15], Olver [26], and O'Neill [27], for further reading. These are classical texts on the subject and are widely used in undergraduate and graduate courses. The definitions used are standard and can be found in many textbooks, including the three previously mentioned. This also makes it hard for us to understand what the reviewer means by 'lack of rigor' -- the mathematical section does not contain heuristics or sketches, we have given precise definitions which serve to simply define our tools.

To answer the specific questions of the reviewer:
- (1) We believed definitions 1 and 2 did not need to be “titled”, as their content is rather explicit and straightforward: def. 1 introduces the shape operator, while def. 2 introduces the principal, Gauss and mean curvatures, which are nothing more than eignenvalues, determinants, and traces. 
- (2) k1 and k2 are the eigenvalues associated to the orthonormal eigenvectors e1, e2. While it is explicitly mentioned in definition 2, we will specify these terms directly at the end of definition 1 in the revisions, to avoid any confusions.
- ⁠(3) DF is a standard notation denoting the differential of a map F. We use the same notation throughout the paper, as so, p is a point in S, and the subscript p refers to the map at point p -- standard notation. However, we are willing explicate this in revisions of our paper.
- (4) TS is the standard notation for the tangent bundle of S. We are willing to explicate this in the revisions of our paper. S_hat is introduced as one of the two surfaces referred to on the same line 139. We struggle to see how else it could be introduced. 
- ⁠(5) Although we fail to see how the indexing of equations is a big problem (given that no equation is referencing another), we will address this in revisions of our work to facilitate reading. Regarding ⁠g_s, it is introduced and defined on line 138, as the Riemannian inner product induced on TS by the euclidean inner product. 
- (6) We believe F and eta have been introduced in a way that the difference between them is clear: F is an isometry of R^3, while eta is an isometry between surfaces. We already introduce them both in writing and formula -- we struggle to understand what more is needed here.
Overall, we believe all objects have been properly introduced. However, we will happily explicate some common notations used in our work to help readers less familiar with them. 
We hope we answered the reviewer’s questions more specifically in this comment
We thank the reviewer for this additional feedback. 
First, we would like to apologize if the reviewer was under the impression that the questions they have raised have not been considered. On the contrary, they have stimulated discussion amongst us and we agree the suggestions will improve the paper. We we will be happy to include a discussion section on the points raised by the reviewer and summarised below:
 
Benefits of Gaussian curvature: 
- ⁠We believe the initial comments made about better understanding why gaussian curvature could be better than principal curvature are interesting and relevant; it is an important point that will add value to the paper's thesis, and we thank you for highlighting it. 
- We could also hypothesize that the size of meshes - rather than the task of classification - could be an important factor, since the SHREC dataset we have used in the paper also happens to be the one that contains the smallest sizes of shapes.
- As we do not have a strong analytical understanding of neural network behaviour, let alone an understanding of their action on shapes, it is hard to give a guideline as to when should we use gaussian against principal curvature or to give a scientific discussion that would not be purely hypothetical/speculative, without any experiments and hypothesis testing as support. 
- However, we will be happy to discuss potential experiments that could help tackle this question, e.g. benchmarking both curvatures on different classification datasets with ranging sample size and size of meshes.

Representation ability of the neural networks:
- ⁠We believe that, at the minimum, delta net and diffusion net are flexible enough to "easily reach" gaussian curvature from principal curvature. In fact, they can probably reach it from the extrinsic coordinates representation: in terms of shape information, each representation is complete in a sense; that is, they contain all the geometric information needed to fully describe the shape. We do not create more information when using curvature, it is just "presented" in a compact, yet information rich, form which the neural network uses to process the shape. In addition, principal curvature and gaussian curvature are indeed very close to each other, but the HKS is as well, it is essentially computed from the shape Laplacian, which is related to curvature.  

Noisy data:
- First, we thank the reviewer for emphasizing both the fact that added experiments on noisy data would be beside the point (since we work with known shapes), and that there is no need for comparing with transformers based architectures. 
- As for integrating curvature in a noisy data scenario: curvature has a long history of research, with numerous discretisation methods and implementations, each designed to be robust to different scenarios. We believe that noisy data mostly implies a scenario where the orientation of the shape is missing, and in this case we point out that curvature as defined, and implemented, from geometric measure theory should be the best option, as it doesn't rely on the normals. We have only slightly touched upon this point in the paper but agree that elaborating on it, specifically in section 3.2, would improve our paper.

We hope we answered the reviewer’s questions more specifically in this comment.
The rebuttal has addressed my concerns. I am happy to raise my rating to 6.
## I.Regardsing to the novelty...
I had spent quit a long period in reading this paper, and trying to understand the ideas it convey. I really appologies that makes the authors think the review does not contribute much to a constructive discussion, it is also because the shape descriptor appears to be an unfamiliar topics to me. During the whole reviewing period, I keep thinking about how principal curvature contribute to the geometric learning scheme. There has been intensive research upon mesh and point cloud laplacian [1][2], mentioned about laplacian beltrami operator via its descrite forms of matrices contains important curvature, angles information. Also, a concurrent work "Manifold Diffusion Fields"[3] use the first k eigen vectors of the laplacian beltrami operator as intrinsic coordinates to replace x,y,z on mesh vertices or point cloud. They used it as positional encoder and use a transformer directly, got great improvement. These line of research are also conluded as the spectral methods in geometric deep learning, ref [4] (Section V)

Another line of geometric deep learning on processing and learning the manifold surface, utilising the tangent bundle of surfaces. One of the most recent work in this line of research is [5]. Although they're task is to learning a texture genertion on single mesh, surface geometry are still being learned and can also be generalized to mesh deformation task.

It could because my knowledge base are dominant by these evidence, that I did not fully recognise the contribution by the submission 17689. It appears to me that the paper performs experiment with the use of two surface representations with different neural networks. I recognized the overall experiments that being conducted in the paper, and the effort made by the authos, but think that the contribution is limited, also a major part in section 3 being vagues and hinder me to follow the paper. When the authors stating they are filling the research gaps, I was more expecting to see that the aforementioned research fields being addressed. Also it could also because my limited knowledge in the surface processing or shape operators, that did not realize the significance as much as authors and reviewer qKsN.

- [1] Sorkine, Olga. "Laplacian mesh processing." Eurographics (State of the Art Reports) 4.4 (2005): 1.
- [2] Lévy, Bruno, and Hao Zhang. "Spectral mesh processing." ACM SIGGRAPH 2010 Courses. 2010. 1-312.
- [3] Elhag, Ahmed AA, et al. "Manifold Diffusion Fields." The Twelfth International Conference on Learning Representations.(2023).
- [4] Bronstein, Michael M., Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. "Geometric deep learning: going beyond Euclidean data." arXiv preprint arXiv:1611.08097 (2016).
- [5] Mitchel, Thomas W., Carlos Esteves, and Ameesh Makadia. "Single Mesh Diffusion Models with Field Latents for Texture Generation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

## II. Confusing points in the paper, especially section 3 Operator.

Carrying with the questions, I kept reading the paper's method part, and hope to develop a better understanding. 
- (1) Two definitions are given without a terms, so I guess definition 1 (line 104) defines the shape operator,  definition 2 (line 118) defines the principal curvatures. If so, it would be better if the authors could root the definitions with some references. 
- (2) It is also confusing when the paper mentioned (e1,e2) in kune 115, what are their relationships to (k1,k2), the paper does not mention clearly, leaving an not rigorous impression to this part.
- (3) line 137, (as the equation index not given), I do not find a explaintion to the operator DF that being used in the innert product <.,.>, so as the subscript p
- (4) line 138, what is TS, the tangent bundel on surface S? Also S_hat is not proporly introduced.
- (5) Based on (3), (4), the equations after line 140 is vague in the meaning, again the equations not indexed is a big problem. Does g_s refers to gaussian curvatures? What it g_s(.), definition or description not clearly given/easily found. 
- (6) line 139, to tell the difference between \ita and F, requires a prior knowledge in eulcidean space and manifold when reading.

In a nutshell, section 3, especially section 3.1 is not rigourous in writting.

## III. Fianally... 
After reading the reviews and reply from reviewer qKsN, I realise that I am not fully recognise the contribution the submission 17689 has made to the surface processing and I was not very confidence about my understanding to the paper, while I also agree with reviewer LGgy with the weaknesses. Also the paper's writting for section 3 still being a big confusion to me, this could either be my rather limited understanding, or the paper need a revision to have a more rigorous way to better informed the audience. Thus, I will adjust my rating to boderline reject and lower down my confidence scroes as dicussed.
Dear ifJ1. 

The end of the discussion period is close.
It looks like your review contains many general comments concerning the structure of the text, but not about the technical properties of the proposed method. Could you please provide in more detail any additional technical comments concerning the proposed method itself?
Dear LGgy. The end of the discussion period is close. I would be grateful if you provide a feedback regarding authors’ answers to your review.
I'm not sure I agree with the rebuttal from the authors in regards to my weaknesses. Yes it is important that discoveries are shared with the community and circulated, such as the one presented in the paper, but that does not mean only a surface level investigation of the discovery is needed before circulation, and not being interested to investigate or at least discuss further is not a scientific strength. 

While I did not expect the authors to necessarily run more experiments based on the weaknesses I listed, I was hoping for at least some discussion or thoughts about them. In regards to the first weakness, do you think that a) the complexity of Delta Net and Diffision Net do not have the ability to compute Gaussian curvature from principal curvatures, or b) they do have the representation ability but they never find Gaussian curvature or some equally useful information derived from principal curvatures? 

I also had a look at the other reviews. Reviewer LGgy has a point about the difficulty of estimating curvature robustly, and it makes sense that at least a short discussion about this would be beneficial to the paper, and I hope the authors add this. However the experimental results show that at least for the experiments given, current standard methods (like the one from [30] used in the paper) are sufficient for various interesting datasets. I don't think that it is necessary to specifically give results with noisy data, and especially not necessary to give results with transformers. 

I am keeping my original score of 7, but hope the authors are interested in improving their paper.