Learning 3D Equivariant Implicit Function with Patch-Level Pose-Invariant Representation
Authors: Xin Hu
Xiaole Tang
Ruixuan Yu
Jian Sun
Venue: NeurIPS 2024 poster (2024)
Keywords: Equivariant implicit neural representation; Pose-invariant representation; Generalizability to 3D objects; Robustness to transformation
Primary area: machine_vision
TL;DR: We design the patch-level pose-invariant 3D feature representation to represent the 3D shape, resulting in the implicit displacement estimation of 3D query points based on the local patch-level pose-invariant representation.

Abstract: Implicit neural representation gains popularity in modeling the continuous 3D surface for 3D representation and reconstruction. In this work, we are motivated by the fact that the local 3D patches repeatedly appear on 3D shapes/surfaces if the factor of poses is removed. Based on this observation, we propose the 3D patch-level equivariant implicit function (PEIF) based on the 3D patch-level pose-invariant representation, allowing us to reconstruct 3D surfaces by estimating equivariant displacement vector fields for query points. Specifically, our model is based on the pose-normalized query/patch pairs and enhanced by the proposed intrinsic patch geometry representation, modeling the intrinsic 3D patch geometry feature by learnable multi-head memory banks. Extensive experiments show that our model achieves state-of-the-art performance on multiple surface reconstruction datasets, and also exhibits better generalization to crossdataset shapes and robustness to arbitrary rotations. Our code will be available at https://github.com/mathXin112/PEIF.git.

Learning 3D Equivariant Implicit Function with
Patch-Level Pose-Invariant Representation
Xin Hu', Xiaole Tang', Ruixuan Yu², Jian Sun(Ez)
Xi' an Jiaotong University, Xi' an, China Shandong University. Weihai, China Pazhou Laboratory (Huangpu), Guangzhou, China (huxin7020, tangx1]@stu. tut.. edu. en, yuruixuan@sdu. edu. cn, j jiansun@x tu. edu. en
Abstract
Implicit neural representation gains popularity in modeling the continuous 3D surface for 3D representation and reconstruction. In this work, we are motivated by the fact that the local 3D patches repeatedly appear on 3D shapes/surfaces if the factor of poses is removed. Based on this observation, we propose the 3D patch-level equivariant implicit function (PEIF) based on the patch-level pose-invariant representation, allowing us to reconstruct 3D surfaces by estimat- ing equivariant displacement vector fields for query points. Specifically, our model is based the nose-normalized query/patch pairs and enhanced by the proposed intrinsic patch geometry representation, modeling the intrinsic 3D patch geometry feature by learnable multi-head memory banks. Extensive experi- ments show that our model achieves state-of-the-ase performance on multiple surface reconstruction datasets, and also exhibits better generalization to cross- dataset shapes and robustness to arbitrary rotations. Our code will be available at https//giihuu..co/mmah... 12/PEIF.git.
Introduction
Surface reconstruction aims generating continuous surfaces from discrete point clouds. It is fundamental and challenging task current robotics and vision applications [ 2 B). Recently, deep learning-base based implicit neural representations (INRs) have emerged as powerful tool for this task, such as signed distance fields (SDFs) 4 5) unsigned distance fields (UDFs) 6 g and
neural vector fields (NVF) [9. INRs benefit from its continuity, and the ability to handle complicated topology, showing promising performance on surface reconstruction.
Although current INRs-based methods have achieved promising performance in reconstructing surfaces, they suffer from two main challenges. First, most methods 9 TO IT deal with the distinct local regions as geometry elements to estimate the query point values, e.g., signed/unsigned distance. However, different local regions may exhibit different poses but with similar intrinsic geometry. The extrinsic poses of these 3D patches prevent the models from capturing the intrinsic geometry of 3D shape patches. Second. INRs 6 7 9 12 without considering equivalence commonly learn the representation of the points using a fixed coordinate frame, implying that if the input points are rotated, the original coordinate mapping may no longer accurately predict the desired output, leading to distortions or inaccuracies. These properties of INRs hinder their applicability to complex 3D scenarios, in particular with regard to their cross-domain generalization ability and robustness to arbitrary transformations like rotations.
To tackle these challenges, we try to eliminate the redundant factor of poses and more focus the learning of the intrinsic geometric representation of local regions, yielding a patch-level pose-
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

invariant representation (PPIR) of 3D objects, Based on this representation, we develop a patch-level equivariant implicit function (PEIF), allowing us achieve the equivariance patch-wisely while effectively encoding arbitrary topology. Specifically, in the PEIF framework, the query/patch pairs are first normalized via a unique pose normalization. Then the query/patch features are extracted and processed via learnable multi-head memory banks to acquire the intrinsic patch geometry representation, which is aggregated with the spatial relation representation, resulting in the patch- level pose-invariant representation. PPIR is then utilized for displacement prediction, which can be proven to be equivariant for (3) transformations. These designs enhance the expressive power of INRs with PPIR and enable the PEIF to flexibly adapt to 3D domain gaps well as arbitrary SE( transformations.
Our contributions can be summarized as follows. First, we propose patch-based equivariant implicit function based on the pose-invariant feature learning. facilitating 3D reconstruction robust to 3D shapes S (3) transformations. Second, we design intrinsic patch geometry representation module encoding rich patch-level pose-invariant features leveraging similar geometric patches. Third, the effectiveness of PEIF for surface reconstruction is demonstrated on four datasets including two CAD object datasets, a synthetic scene-level dataset, and a real sean dataset. Experiments show that our method outperforms baseline methods and can effectively reconstruct fine geometric structures, particularly performing well in cross cooss-d generalization and the robustness to arbitrary rotations.
2 Related Work
2.1 Implicit Representation for 3D Shape Reconstruction
Deep learning-based implicit representations have achieved significant advancements, due their continuity and ability to handle complex geometry structures. Implicit representation for 3D surface reconstruction commonly learns to assign specific values for query points in 3D space. For example, occupancy field (occ) based methods [13 [4 [5]
I6 [8) enable the 3D reconstruction as a binary classification problem. The Occupancy Network [4] introduces predictions of spatial point occu- pancy, while advancements like ConvONet [19 and POCO [00] integrate grid-oriented convolutional or transformer frameworks to enhance performance. Recently, ALTO (20] iteratively refines features from both points and grids, deploying attention-driven interpolation from adjacent grids to decode occupancy values for query points. GridFormer (IT] introduces transformer architecture to integrate the advantages of both points and grids for the prediction of occupancy.
SDF/UDF provides a continuous value to each spatial point, indicating the corresponding signed or unsigned distance to the surface. UDF with unsigned distance overcomes the limitations of SDF in handling non- watertight geometries. DeepSDF leverages Multi- Multi-Layer Perceptron (MLP) to globally model SDF for entire 3D shape. while DeepLS [5], Instant-NGP [1] and NKSR (22 design more detailed operations to predict the SDF UDF locally or hierarchically with MIP, kernel function or transformers, etc. GIFS 12 represents general shapes with multi-layer surfaces based on the spatial relationship between points. CAP-UDF [ employs a field consistency constraint to get consistency-awant UDF. GcoUDF 77 adaptively approximates the UDF and its gradient of a point cloud by leveraging local geometry in a decoupled manner. However, separate learning of UDF values and gradients for points may result in accurate UDF but with the inverted direction problem. proposes an explicit approach to learning implicit representations based displacement vectors, which ensures both accuracy and correct directional information. In this paper, we adopt this representation, predicting a displacement vector for each point in 3D space. Compared to NVF D.
To address this issue, NVF [
we design our PEIF over the pose- normaazed 3D patches and obtain the S FF(3 -quuivraaa implicit function.
2.2 SE Equivarana Network
SE -eeu)v)ruvaaaancc has been extensively studied in both 2D images 23 and 3D point clouds 24
25
26 27 28). Given 3D point cloud X and transformation VC E SE(3), model f said be SE
equivariant when it satisfies f o ((X) C f(X.. Various works have been proposed to achieve SE(())-quivariin)
based on PCA 29
30 25 spherical harmonies [3] [1
equivariant message
passing (34 35 36. or Vector Neuron 226 27 28.. SF(3) -qquiv-aqu networks are particularly
useful for 3D point analysis tasks, such as molecular property or trajectory modeling [
35
36 37
protein structure prediction [38] 39 40) 3D shape recognition [26] 27 28). and robotics
42
43

---

Puickes extraction
Geumetry grouping
Point
ppormulization
Figure I: Local 3D patches may exhibit geometric similarity, but with different poses. When the pose removed, these local regions appear repeatedly.
Introducing -euiiiiqqie to build an orientation robusst implicit field is one of the motivations of this work. There are few works involving equivariance in the implicit field. EFEM 44 uses Vector Neuron 27 to learn equivariant shape representations hefore shape segmentation. E-GrrphONNet [24] utilizes basic Vector Neuron 2 layers design graph networks, achieving locally (0(3)-invar))) features for implicit function learning. E-GraphONeet
(24) is the most related work to ours, which extends neurons from ID scalars to 3D vectors for each point. In comparison, our PEIF employs lightweight PCA to achieve pose-invariant patch-level representation and leverages multi-head memory bank for intrinsic geometry representation, achieving state-of-the-art 3D reconstruction performance.
Problem Statement for Equivariant Neural Vector Field
In this section, we first introduce the implicit representation, namely the neural vector fields (NVF) 91. and then introduce the equivariant implicit function of this representation.
Given sparse point cloud X E RN sampled shape X, and query set Q E R&, x3 sampled near the surface of X, where N₂ and Ng represent the number of input points and query points respectively. A shape X' is defined as the zero displacement of the implicit function F
X
R - =
(1)
where r point in point cloud X, containing its spatial coordinate. Ō represents the zero displace- ment of the point For a query point
RR, the implicit function F is formulated by
F(g) = Aq i q, where & argmin exxl=x all+
(2)
and å is the nearest point of query q on the X.
Definition (Equivariant Implicit Function). Given an abstract group G, the implicit function F based NVF is equivariant with regard to G,
F( q) ( (F())) Aq, E
where q a query point near or the surface of shape X.. In this work, the group G SE(3).
4 Equivariant Neural Implicit Function
In this work, we aim to develop an equivariant implicit function model grounded on neural vector field representations. We achieve this goal by firstly learning patch-level pose-invariant representation (PPIR), and then designing shape-level equivariant implicit representation. The overview of our method is introduced in Section 4.1 with the detailed designs presented in Sections 4.2 and4.3
4.1 Overview of the Basic Idea
Given point cloud X, implicit representations conventionally involve sampling a set of query points Q and employing implicit function JF to compute their associated implicit values. Typically, the

---

Patch-level Equivariant Implicit Function 3
Displacment Predictor
Spattal Beletion Modile
Diplacements
Marehing Gite
query
Puich Featurg Extrction Vegule
PPR
TM
Incress Patch Beometry Batrogtor level invaratant Representaion Extractor 6
Patch-based Pose Normalization
Pose
Denormalization
Figure 2: Overview of the proposed PEIF. Given query points, the local patches are selected using KNN. The query/patch pairs normalized by pose transformations. The displacements of query points to the surface are predicted by displacement predictor D. The implicit function is equivariant under the (3) transformations of the input. Finally, the mesh is generated by marching cubes algorithm.
depiction of a query point E Q depends on its K-nearest neighbors (KNN) in X. As shown in [ is observed that some local KNN patches exhibit identical geometric structures if ignoring their pose variations in S (3), and the local patches across 3D objects also repeatedly appear. Based on this observation, we design an equivariant implicit function based on patch-level pose-invariant representation, capturing recurring geometric patterns invariant to pose transformation.
Before delving into the specific details of our approach, we present the overall framework as shown in Figure 2 Given query set Q {q.}. the corresponding patch for q on point cloud is Euclidean distance. The point patch P. and query point 9; are firstly normalized by patch-based basded pose normalization t;, achieving invariant ones under SE(3) transformation of patch P.. We then feed {7, (P.), (:(4)) to the displacement predictor D for (5E((3-invarin)) representation learning and displacement prediction. Finally, this predicted displacement transformed back to the pose P. with T, The overall displacement prediction can be written
P. = {pr [p= i.e., the KNN of q, based
K
Aq. F(a) -,
{
This framework is -qqiivaaan for patch P; and point cloud X. The detailed design of the patch-based based nose-normalizatinn and displacement predictor D are presented in the following Sections 4.2 and respectivel, We remove index for brevity and denote the query point, point patch, and pose-normalization as q IR3, P RRAR Kx3 and respectively in the following paragraphs.
4.2 Pose Normalization
Geometrically identical patches are expected to maintain consistency across various pose transfor- mations, enabling their representations to complement and reinforce each other. Accordingly, we employ Principal Component Analysis (PCA) extract the pose-invariant information for patch P.
We first decenter the patch P by subtracting the points center μ4, then obtaín the rotation matrix U by computing the Singular Value Decomposition (SVD) (45) over the covariance matrix (P-1) T (P- 11). The pose-normalized patch and query point are derived as
(5) The pose-porsean nossmaormllized patch P and query point q are invariant under SE(3) transformation of P, and take them input to displacement predictor D. The prediction D o (r)) is also invariant as proven in following Lemma 1. Note that we uniquely determine as [46] to solve the direction uncertainty problem brought by PCA.
P P = - UU, q _ T(g) = (q 1-)
Lemma With as our pose normalization, and D as displacement predictor, Do (T(P), invariant under SE(3) transformation of P.

---

Please refer to the Appendix for proof. The displacement predictor will be introduced as follows.
4.3 Displacement Predictor Design on Normalized Patches
Taking the pose-normalized patch P and query input, the displacement predictor D is designed to predict the displacement Ag. As shown Figure 2
predictor D comprises pose-invariant feature extractor @ and a MLP n Specifically, the feature extractor @ is composed of three modules: the Spatial Relation Module (SRM) for query point feature learning. which models the spatial relative relationship between g and P: the Patch Feature Extraction Module (PFEM) for patch feature learning, which extracts patch feature leveraging correlation in feature space; the Intrinsic Patch Geometry Extractor (IPGE), which learns meemory-augmentte patch representation.
Spatial Relation Module. We design SRM to learn query point features based on spatial relation within query ạ and patch P = [p: f=1 Specifically, the point-wise representation of point E P is firstly computed as
Yo, (p:. 4) i 1,
(6)
yo, (-) is set MLP. Taking query point position and relative offset inputs, 2; is expected to directly capture the geometric patterns. Then we aggregate 2; with simple concatenation operator + by
hg 21
zk.
Feature hg E RXX contains the relative feature of query point the patch P. We take it as representation for the query point q.
Patch Feature Extraction Module. We further design PFEM to learn the patch feature for P. Taking the point positions of : and p {pr J as inputs, we first lift them from Fuclidean space feature space via two MLPs 70, (-) and 70, n0, (-)
fq = 70, (g).
= =
i = 1,2, 1.2,.... K.
(8)
where Jp. RIID are the learned point-wise features. Then, transformer is designed to obtain the patch feature, by encoding the feature attention between point p; and query point g as
f Σ a. aa--aaa af. Wv), where {a,) Softmax (f
where represents the attention score between query point and patch points p:. The matrices Wo, Wo, Wv E DxD are learnable parameters. Patch feature fp, is aggregated from all the points features in patch P. while different patches may have diverse point distributions. To mitigate the effects point density patches, we further design the importance-aware patch feature Sp, by selecting the top-Ka important points, and aggregating their features as
K
SP.
bufne
(10)
where {b,) is the selected top-K, Ka attention scores from {a The final patch feature fp from this PFEM is
=p= > + A2ff.
(11)
where \1, A2 are learnable combination coefficients.
Intrinsic Patch Geometry Extractor. As dis- cussed in Section [4.1 the normalized point patches can be grouped into different geometric patterns across patches or shapes. To learn the intrinsic features hidden behind those geometric patterns, we propose IPGE to enhance the patch features. Specifically, learnable multi-head memory bank M = [M is constructed, and it is shared across the whole dataset to im- plicitly model the patch patterns. Then the patch feature fp is enhanced by querying and aggregating each memory item as
Patch Suature
Firresic Posch Geometry Degoomentation
Memory
Multi-hadd memory bank 3
Figure 3: Feature enhancement with IPGE.
Nu
gp
wi
wi = Softmax(fpMM)
(12)
5

---

The memory weight wi; is defined as softmax normaazzed similarity vectors between query feature f and the entries of M, E RCXP, Figure illlusrates the procedures of this module.
Displacement Prediction. Based on query point features (hg. fg). noint-wise patch feature (sp, }.K enhanced patch feature Pp, the patch-level pose-invariant (PPIR) representation is achieved by
fPPIR y0. (ha @ fa @ (fr.h) @ gp.
(13)
Then the displacement for query point yj can then be derived by
A No, (frrIR).
(14)
Both Teo, and Te. are set as MLPs. Finally, Aς is transformed back with pose denormalization,
i.e., the inverse transformation of 7, achieving the final -qqiivrana displacement estimation ( (iiivaannee of fPPIR can be found in Lemma 1, and the (E(3)-
Ag = - (Ag). The
equivariance of learned implicit representation can be found the following Theorem 1, please refer to Appendix for proof.
Theorem
Given query point q and patch P, implicit function F(g) SE(()) -qquivvrrvv
4.4 Network Training and Inference
Sections 4.2 and 4.3
illustrate how to obtain the displacement for a query point, where the trained
parameters include the parameters 0z, Pp 0g On. 0a of five MLPs, the multi-head memory bank M and parameters Å₁ Ag. To optimize the implicit function F, we design joint loss function over the query set Q train our method in an end-to-end manner.
Displacement Optimization Loss. We compute -loss between the predicted displacement åq; for each query point di Q and its ground-truth displacement Ai; as
La
Ag Aqil
(15)
N
Patch Discrimination Loss. The items in memory should be apart from each other to enhance the representativeness of the memory items. To ensure this, we design the patch discrimination loss as
NM
NM.
(MM] 11.0)
Σ mogiimm
(16)
NAs, (Na. - 1)
which is similar to cosine embedding loss 47 with a margin set to O. NM, is the number of the items
memory bank M;, The overall loss function finally written as La + 3L,s where /3 is hyper-parameter for balancing the two terms.
3D Reconstruction in Inference Stage. We employ the Marching Cubes (MC) algorithm proposed by MeshUDF [48], which can reconstruct surfaces on UDFs. We first discretize the 3D volume into 3D grid with resolution of NR. resulting in N grid points the query set Q. Then, we use the implicit function F to predict the displacement Âg of each query point Similar to [9], we get the UDF value and gradient of as d = || and V,
Based on d and Vg. the MC in MeshUDF 48 can reconstruct the surface of the input point cloud as mesh.
Ag
5 Experiments
Implementation Details. We implement our PEIF in Pytorch [49] using Adam optimizer [50]. The learning rate is 8 10-4, For each query point, the size of the neighborhood is set as K = 32 for ShapeNet [5]] and ABC [52] datasets, K = 54 for Synthetic Rooms [19] dataset. We set 3 = 0.1 in the training loss and N,, 4 for the memory bank. Please refer to the Appendix for details on the structures of involved MLPs, and the effect of different values of 3. We conducted all experiments on one NVIDIA RTX 4090 GPU.
Datasets. We experiment on four datasets including ShapeNet [TI, ABC 52 Synthetic Rooms [19], MGN [53]. (1) ShapeNet [51), pre-processed by [7], contains watertight meshes of shapes in

---

Table 1: The reconstruction results of ShapeNet [51]. All models are trained on the base classes and evaluated on both the base classes and novel classes. Note that E-GraphONet is the equivariant version of GraphONet.
Base
Novel
Method
CD
EMD
NCT
F-Score
CD
EMD
NCT
F-Score
POCO
10
0,395
3,937
0.929
0.970
0.520
4,941
0.906
0.954
GIFS
12
0,385
3,859
0.932
0.962
0.422
4,923
0.917
0,942
ALTO
20
0,352
3,851
0,930
0.963
0,357
4,924
0,920
0,929
NVF
0.255
3.766
0.938
0,983
0.266
4.721
0.921
0,982
GeoUDF
0.225
3,761
0.957
0.997
4.735
0.997
GridForer
||
0.284
3.768
0.942
0.984
0.289
4.725
0.928
0,985
GraphONet
x
0.389
3.868
0
0.932
0.461
4.733
0,917
0.952
E-GruphhONet
24
0.479
3.834
0.917
0.927
0.508
4.743
0.911
0,942
OEIF(Ours)
0.215
3.755
0.956
0.998
0.209
4.725
0.941
0.997
POCO
GIFS
ALTO
NVF
GeoUDI
GridFoomer
PEIF
GT
Figure 4: The qualitative results of ShapeNet [5T] dataset. The object is selected from meshes used for class-unseen reconstruction (novel classes in Table T
13 classes. Following the experimental setting in 19, we select cars, chairs, planes, and tables as base classes in Table and speakers, bench, lamps, and watercraft as novel classes in Table [
for category-unseen reconstruction, only for testing.
(2) ABC [52] has one million CAD models,
mainly mechanical objects. We use the splits from 154 and select watertight meshes for experi- ments: 3599/883/98 shapes for rrainingvaliddationnesstt
(3) Synthetic Rooms [19] contains 5k synthetic room scenes composed of random walls, floors, and ShapeNet objects. We adopt the same train/validation/eest division in [19]. (4) MGN (53) is a real scanned dataset containing clothing categories. To generate watertight surfaces,
employ the method [55] for preprocessing. Specifically, we sample 3k points on the surface as input points for ShapeNet and ABC datasets, while 10k input points for Synthetic Rooms. Then 2048 query points sampled near the surface for ShapeNet, ABC, and Synthetic Rooms. All experiments are tested on 10k points.
Evaluation Metries. We use the Chamfer-LI distance (CD, x x10 Earth Mover Distance (EMD, x10-2) 2), Normal Consistency (NC), and F-Score (with threshold value 1%) metrics for our evaluation.
Baselines. To evaluate the effectiveness of our methods, baselines used for comparison include the cquivariant network E-GraphONet [24), and six non-equivariant networks, including POCO (IO), GIFS [1]. ALTO (20). NVF [9], GcoUDF (7]. GridFormer([I] For fairness, we trained these networks from scratch under the same traiiing/aliddtion/eesingg dataset splitting.
5.1 Results and Comparisons
3D Object Datasets Reconstruction. We first report the results of the 3D object reconstruction on the object datasets: ShapeNet (51) and ABC (52). The quantitative results on base and novel classes of ShapeNet [51] are shown in Table T our PEIF achieves better results both in base and novel classes, especially in terms of the CD and F-Score metrics. Qualitative comparisons are provided in
Figure 4 Compared with other competitors, our method can capture fine-graine grained details, and the
7

---

Table Comparison of different methods on ABC [22 and Synthetic Rooms [9 datasets.
ABC
SyntheticRoom
Method
CD
EMD 4
NC
F-Score
CD
EMD 4
NC1
F. F-Scure
POCO
10
0.475
2.785
0.957
0.941
0.512
2.624
0.896
0.973
GIFS
P2
0,339
2.765
0.950
0.985
0.425
2.658
0.913
0.984
ALTO
0.451
2.739
0.943
0.958
0.492
2,426
0.901
0.975
NVF
0.245
2.685
0.963
0.996
0.504
2,052
0,925
0.979
Ge
0.245
2,688
0.964
0.997
0.383
2.182
0.921
0.98%
GridFormer
0.299
0.964
0.981
2.252
0.913
E-GraphONet
0.432
2,688
0.910
0.906
0.485
2.534
0.903
0.980
PEIF (Ours)
0.241
2,672
0.969
0.998
0.314
2.045
0.925
0.994
overall topology of the shape is more consistent. Additional instances are provided in the Appendix (Figure [). We further evaluate the compared methods on the ABC [52] dataset. The quantitative results in Table demonstrate that our PEIF achieves competitive performance compared to both equivariant and non-equívariant methods. Visualizations are provided in Figure of the Appendix.
3D Scene Datasets Reconstruction. Table Table the quantitative results on the Synthetic Rooms dataset. Our PEIF shows state-of-the-art performance under all quantitative metrics. The competitive competitors such GridFormer [TT] and POCO [T] produce smooth but incomplete surfaces. Other methods like GcoUDF [7] and NVF (9) produce results with rough surfaces. In contrast, our method reconstructs relatively smooth surfaces with fewer issues of completeness and consistency. Visualizations are provided in Figure 10 of the Appendix.
GT
GeoUDF
NVF
GridFormer
Ours
Figure 5: The visual example of cross-domain evaluation on the real scanned dataset MGN where the model is pre-trained on Synthetic Rooms dataset 19
53
Cross-domain Evaluation on Real- world Dataset. We test and com- pare our method with the state-of- the-art methods NVF, GoeUDF, and GridFormer on MGN (53] dataset us-
Table 3: The cross-domain evaluation MNG dataset [53
Method
CD |
EMD L
NC f
F-Score f
NVF
9
0.272
4.329
0.847
0.991
GeoUDF
t
0.249
4.269
0.891
0.995
ing the trained models Synthetic Rooms [9). Table shows that the compared methods generally exhibit a declined performance in the pres- ence of a synthetic-real domain gap. However, in the presence of such a domain gap, PEIF still achieves notable performance under all metrics. Figure dispplas the visual comparison. The competitors either produce rough surface suffer from shape incompleteness. As comparison, our PEIF reconstructs a complete surface with fine-grained details. These results show that our PEIF trained on the synthetic data can be well generalized real scenarios. The reason might be that the nose-normalized patches are the basic elements for composing different shapes, and our PEIF is based on the pose-invariant patch representations.
GridFormer
11]
0.281
4.675
0.916
0.969
E-GraphONet 24
0.433
3.817
0.863
0.920
PEIF (Ours)
0.241
2.672
0.969
0.998
Table Performance under Arbitrary SO(3) rotations.
Methods
CD
EMD | NC 1 F-Score 1
NVF
9
0.245
2.685
0.963
0.997
w/o rotations
GeoUDE
?
0.245
2.688
0,964
0.997
B-GraphONet
A4
0.432
2.688
0.910
0.906
PEIF (Ours)
0.241
2.672
0,969
0,998
GeoUDF
NVF
E-GraphONet
PEIF
NVF
9
0.261
2.699
0.946
0.993
Figure 6: Visual results before (top) and after (down) arbitrary rotations.
w/ rotations
GeoUDF
0,253
2.698
0.957
0.994
E- O-GraphONet
0,433
2.689
0,912
0,909
PEIF (Ours)
0.241
2.675
0.968
0.998

---

Table Ablation study on ABC
52
dataset.
Table Model size and inference time.
Setting
CD L
EMD
NC
F-Score
Method
Para (M)
Time (6)
Posc normalization
wio 0.261
2,677
0,933
0.993
POCO [0]
12.19
45.10
Nu =
0.275
2.713
0,957
0.985
GIFS 12
3.51
16.68
Nu =1 0.244
2,705
0.962
0.997
ALTO
2.64
20.78
Memory bank
Ny 0.244
2.696
0.962
0.997
NVF 0
10.29
73.90
Nu 0.243
2.694
0.964
0.998
GeoUDF f
0.74
124.73
Na 0.244
2.691
0.961
0.997
GridFormer
13.32
KNN
K = 54
0.245
2,692
0.962
0.996
GraphONet
24
0.06
1.72
K 20
0.249
2.691
0.996
E-GraphONet
0.07
1.92
0.241
2,672
0.969
0.998
Ours
7.65
40.98
( and
Robustness to rotations. We compare the robustness of two equivariant networks (PEIF and E- GraphONet), and two non-equivariant networks (NVF and GeoUDF) arbitrary rotations. All methods are trained with canonical pose and tested with arbitrary rotations on the ABC [52) dataset. The quantitative and qualitative results are reported in Table 4 and Figure respectively. In Table 4. "w/ rotation" and "w/o rotation" represent that the testing input point cloud is with and without arbitrary rotation, respectively. The visual results presented Figure iillustrate iiluutatee that our PEIF can retain stable performance under arbitrary rotations, which is consistent with the numerical results presented in Table These results justify the robustness of our PEIF arbitrary rotations.
5.2 Ablation Study and Model Analysis
We conduct ablation studies and present the model size and inference time on the ABC [52] dataset.
Effeet of Pose Normalization. As shown in the 2nd row in Table 5 after removing the patch-level pose normalization, all metrics decline. Particularly, the NC metric notably affected.
Effect of the Multi-head Memory Bank. As demonstrated Table 5 the performance of our PEIF deteriorates significantly when the multi-head memory bank, i.e., the intrinsic patch geometry extractor, is removed. The model performs better with NM increase from 1 to 4. When NM = the performance of the model starts to deteriorate.
Number of Neighbour Points K. The size of KNN determines the number of points in each patch. We report the performance of our PEIF with different patch sizes in Table The results show that our method is relatively stable the size of KNN.
Model Size and Computational Time. We compare model size and inference time on the ABC [52 dataset. In Table 6 our network is comparable to other methods in the number of parameters. The computation time of our approach for 3D reconstruction of one point cloud is lower than the of the-art models NVF, and GeoUDF, but higher than GrridFormer and E GraphONet. However, method achieves the best accuracy for 3D reconstruction as shown in experiments.
6 Conclusion
In this work, we propose a patch-level equivariant implicit function, based on the patch-level pose invariant feature representation over the nosemnorallized query points and corresponding neighboring patches. The proposed representation achieves promising results in 3D reconstruction both quantitatively and qualitatively, and generates shapes with better geometry details and robustness to SE(() transforms. Due to the flexibility of the patch-based representation, in the future, we plan to extend this approach to larger-scale 3D reconstruction real scans. Additionally, our patch- basch pose invariant representation can be taken foundation network for pre- mre-training, followed by fine-tuning on few-shot examples.
Limitation. As an implicit network, one limitation is that PEIF relies on the query points and estimating the displacement point-wisely. For scaling up to a larger scale, we plan to utilize multi-scale technique and importance sampling of query points for efficient displacement field estimation.

---

Impact Statement. This work aims to advance the field of equivariant deep learning with applica- tions in 3D surface reconstruction. It may be valuable to the research of equivariant implicit neural representation and geometric modeling of 3D shapes and has no ethical concerns as far as we know.
Acknowledgments
This work was supported by the National Key R&D Program 2021 YFA1003002, NSFC 121251 104, U20B2075, 12326615, 62306167, Shandong Province Natural Science Foundation ZR2024QA161.
References
[1] William E Lorensen and Harvey Cline. Marching cubes: A high resolution 3d surface construction algorithm. In Seminal graphics: pioneering efforts that shaped the field, pages 347-353, 1998.
[2] Hugues Hoppe. Poisson surface reconstruction and its applications. Proceedings of the 2008 ACM symposium Solid and physical modeling, pages 10-10, 2008
[3] Philipp Mittendorfer and Gordon Cheng. 3d surface reconstruction for robotic body parts with artificial skins. In 2012 IEEE/RSJ International Conference Intelligent Robots and Systems, pages 4505-4510. IEEE, 2012.
[4] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 165-174, 2019.
[5] Rohan Chabra, Jan E Lenssen, Eddy llg, Tanner Schmidt, Julian Straub, Steven Lovegrove, and Richard Newcombe. Deep local shapes: Learning local sdf priors for detailed 3d reconstruction. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28,
2020, Proceedings, Part XXIX 16, pages 608-625, 2020.
[6] Julian Chibane, Gerard Pons-Moll, et al. Neural unsigned distance fields for implicit function learning, Advances in Neural Information Processing Systems, 33:2 $3:21638-21652. 2020.
[7] Siyu Ren, Junhui Hou, Xiaodong Chen, Ying He, and Wenping Wang. Geoudf: Surface reconstruction from 3d point clouds via geometry-guides distance representation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14214-14224, 2023.
[8] Junsheng Zhou, Baorui Ma, Yu-Shen Liu, Yi Fang, and Zhizhong Han. Learning consistency- aware unsigned distance functions progressively from raw point clouds. Advances in Neural Information Processing Systems, 15:16411116494, 2022.
[9] Xianghui Yang. Guosheng Lin, Zhenghao Chen, and Luping Zhou. Neural vector fields: Implicit representation by explicit learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16727-16738, 2023.
[10] Alexandre Boulch and Renaud Marlet. Poco: Point convolution for surface reconstruction. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 6302-6314, 2022.
[11] Shengtao Li. Ge Gao, Yudong Liu, Yu-Shen Liu, and Ming Gu. Gridformer: Point-grid transformer for surface reconstruction. arXiv preprint arXiv:240..02292. 2024.
[12] Jianglong Ye, Yuntao Chen, Naiyan Wang. and Xiaolong Wang. Gifs: Neural implicit function for general shape representation. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12829-12839 2022.
[13] Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. In Proceedings of the JEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 5939-5948, 2019.
10

---

[14] Lars Mescheder, Michael Oechsle. Michael Niemeyer. Sebastian Nowozin, and Andreas Geiger. Oceupancy networks: Learning 3d reconstruction function space. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 4460-4477 2019.
[15] Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang. Jie Zhou, and Jiwen Lu. Tri-perspective view for vision-based 3d semantic occupancy prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2223-223,, 2023.
[16] Yiming Li, Zhiding Yu, Christopher Choy, Chaowei Xiao, Jose M Alvarez. Sanja Fidler, Chen Feng, and Anima Anandkumar. Voxformer: Sparse voxel transformer for camera-based 3d semantic scene completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9087-9098, 2023.
[17] Yi Wei, Linqing Zhao, Wenzhao Zheng, Zheng Zhu, Jie Zhou, and Jiwen Lu. Surroundoce: Multi-camera 3d occupancy prediction for autonomous driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 21729-21740, 2023.
[18] Yunpeng Zhang. Zheng Zhu, and Dalong Du Oceformer: Dual-path transformer for vision- based 3d semantic occupancy prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9433-9443, 2023.
[19] Songyou Peng, Michael Niemeyer, Lars Mescheder, Mare Pollefeys, and Andreas Geiger. Con- volutional occupaney networks. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part IIII 16, pages 523-540. Springer, 2020.
[20] Zhen Wang, Shijie Zhou, Jeong Joon Park, Despoina Paschalidou, Suya You, Gordon Wetzstein, Leonidas Guibas, and Achuta Kadambi. Alto: Alternttig latent topologies for implicit 3d reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 259-270, 2023.
[21] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1- 15, 2022.
[22] Jiahui Huang, Zan Gojcic, Matan Atzmon, Or Litany, Sanja Fidler, and Francis Williams. Neural kernel surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 4369-4379, 2023.
[23] Jongmin Lee, Byungjin Kim, Seungwook Kim, and Minsu Cho. Learning cotation-equivarian features for visual correspondence. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, pages 21887-21897, 2023.
[24] Yunlu Chen, Basura Fernando, Hakan Bilen, Matthias Nießner, and Efstratios Gavves. 3d cquivariant graph implicit functions. In European Conference on Computer Vision, pages 485-502. Springer, 2022.
[25] Ruixuan Yu and Jian Sun. Pose-transformed equivariant network for 3d point trajectory prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[26] Yiyang Chen, Lunhao Duan, Shanshan Zhao, Changxing Ding, and Dacheng Tao. Local- consistent transformation learning for rotation-invarian point cloud analysis. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[27] Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas J Guibas. Vector neurons: A general framework for so (3)-equivariant networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12200-12209, 2021.
[28] Chunghyun Park, Seungwook Kim, Jaesik Park, and Minsu Cho. Learning so(3)-invariant cor- respondence via point-wise local shape transform. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, 2024.

---

[29] Alexandre Agm Duval, Victor Schmidt, Alex Hernández-Garcia, Santiago Miret, Fragkiskos D Malliaros, Yoshua Bengio, and David Rolnick. Faenet: Frame averaging equivariant gnn for materials modeling. In International Conference Machine Learning, pages 9013-9033. PMIR, 2023.
[30] Omri Puny, Matan Atzmon, Edward J Smith, Ishan Misra, Aditya Grover, Heli Ben-Hamu, and Yaron Lipman. Frame averaging for invariant and equivariant network design. In International Conference on Learning Representations, 20211
[31] Carlos Esteves, Christine Allen Blanchete,,,,,e, Ameesh Makadia, and Kostas Daniilidis. Learn- ing so (3) equivariant representations with spherical cnns. In Proceedings of the European Conference on Computer Vision (ECCV), pages 28, 2018.
[32] Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation-equiiarraan neural networks for 3d point clouds. arXiv preprint arXiv:1802.082/39 2018.
[33] Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling, Se (3)-transformers: 3d roto-translation equivariant attention networks. Advances in Neural Information Processing Systems, 33:1970-19999 191000--81,.210 2020.
[34] Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. International Conference on Machine Learning, pages 9323-992299 93332,93,2222 2021.
[35] Wenbing Huang, Jiaqi Han, Yu Rong, Tingyang Xu, Fuchun Sun, and Junzhou Huang. Equiv- ariant graph mechanics networks with constraints. In International Conference on Learning Representations, 2021.
[36] Kristof Schütt, Oliver Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In International Conference Machine Learning, pages 9377-9388, 2021.
[37] Fang Wu and Stan Li. Diffmn: a geometric diffusion model for molecular dynamics simu- lations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 5321-5329, 2023.
[38] Yangtian Zhang, Huiyu Cai, Chence Shi, and Jian Tang, E3bind: An end-to-end equivariant network for protein-ligand docking. The Eleventh International Conference on Learning Representations, 2022.
[39] Chen Chen, Xiao Chen, Alex Morehead, Tianqi Wu, and Jianlin Cheng. 3d-equivariant graph neural networks for protein model quality assessment. Bioinformatics, (bbaadda, 2023.
[40] Rahmatullah Roche, Bernard Moussad, Md Hossain Shuvo, and Debswapna Bhattacharya. (3) equivariant graph neural networks for robust and accurate protein-protein interaction site prediction. PLoS Computational Biology, ((8)):1011433: 2023.
[41] Haojie Huang, Dian Wang, Robin Walters, and Robert Platt. Equivariant transporter network. In Robotics: Science and Systems, 2022.
[42] Dian Wang, Mingxi Jia, Xupeng Zhu, Robin Walters, and Robert Platt. On-robot learning with equivariant models. In Conference on Robot Learning, pages 1345-1354. PMLR, 2023.
[43] Xupeng Zhu, Dian Wang, Guanang Su, Ondrej Biza, Robin Walters, and Robert Platt. On robot grasp learning using equivariant models. Autonomous Robots, 47(8):1175-11999 2023.
[44] Jiahui Lei. Congyue Deng, Karl Schmeckpeper, Leonidas Guibas, and Kostas Danilidis. Efem: Equivariant neural field expectation maximization for 3d object segmentation without scene supervision. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 4902-4912, 2023.
[45] Gene H Golub and Christian Reinsch. Singular value decomposition and least squares solutions. In Handbook for Automatic Computation: Volume II Linear Algebra, pages 134-151. Springer, 1971.
12

---

[46] Chen Zhao, Jiaqi Yang, Xin Xiong, Angfan Zhu, Zhiguo Cao, and Xin Li. Rotation invariant point cloud analysis: Where local geometry meets global topology. Pattern Recognition. 127:108626, 2022.
[47] Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 5265-5274, 2018.
[48] Benoit Guillard, Federico Stella, and Pascal Fua. Meshudf: Fast and differentiable meshing of unsigned distance field networks, In European Conference on Computer Vision, pages 576-592. Springer, 2022.
[49] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-pes tigh-performance deep learning library. Advances in Neural Information Processing Systems, 32, 2019.
[50] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[51] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint 151.00005 2015.
[52] Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Marc Alexa, Denis Zorin, and Daniele Panozzo. Abe: A big cad model dataset for geometric deep learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9601-9611, 2019.
[53] Bharat Lal Bhatnagar, Garvita Tiwari, Christian Theobalt, and Gerard Pons-Moll. Multi-garment net: Learning to dress 3d people from images. In Proceedings of the IEFF/CVF International Conference on Computer Vision, pages 5420-5430, 2019.
[54] Yizhi Wang, Zeyu Huang, Ariel Shamir, Hui Huang. Hao Zhang, and Ruizhen Hu. Aro-net: Learning implicit fields from anchored radial observations. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3572-3581, 2023.
[55] Jingwei Huang, Hao Su, and Leonidas Guibas. Robust watertight manifold surface generation method for shapenet models. arXiv preprint arXiv:1802.01696 2018.
13

---

Appendix
A Proofs
A.1 Proof of Lemma
Lemma 1. With as our pose normalization, and D as displacement predictor, D (T(P), r(4)} invariant under (3) transformation of P.
Assume that P, Y E I Rxx3 are two point patches with Y = =(P),, < E SE and <(P) = PR T is the S3) transformation of P with rotation matrix R and translation vector T. Denote the patch centers of P.Y are μ. v respectively, and their corresponding PCA-normalization are (P) (P μ (Y) = (Y v)V. To prove Lemma 1, we firstly prove the uniqueness of pose-normalization then prove the S (E(3) of D o (T(P), r(g)}.
Step 1. Uniqueness of PCA-normalization т. According to the definition of TP(P) (P- u)U,, the patch center ys uniquely computed as patch centroid, while rotation matrix is computed by SVD the covariance matrix (P² μu) (P- μ4). The vector elements of matrix U = [u,}=1 may change their directions and result in eight rotation matrix Ù (tuu:)=z. which bring uncertainty for pose-normalizaaion Tp. To uniquely determine U, we follow [46] to determine a single direction for every {u: = by estimating their angle with predefined anchor point y (the vector from the farthest point of the patch to the patch center). The direction of u, should be flipped if the corresponding angle larger than 90% Specifically, if (u,, y) > O, we take u, as one vector of U. If (w,, y) we take -u, instead. If (u;, y) = 0, we take another point y' (e.g., the second farthest point from patch center) that satisfies (ut, 3/) * the rotation matrix U is uniquely determined and the PCA-normalized TP(() is uniquely determined.
new anchor point determine the U. By this strategy.
Step 2. -inaarnine of D {TP(P), TP(g)). For point patch P {p:) and its SE(3) transformed patch Y (yil Wi = PuR + T), with their corresponding centers v computed by
K
# =
Pin
V =
K
Vis
(17)
we have
K
K
1 K
V/
1
V1
(p,R + +T) =
K Σ Pi
R+T==RR+ +T.
(18)
K
K
1=1
1=1
Then it is obvious
Y-v= P
(19)
which means Y is an orthogonal transformation - thus Y - y/ has same singular values as P - 11 when we conduct SVD their corresponding covariance matrices, i.e..
UAU = (P- (P-)) (P-))
VAV T (Y vv) -v)
[(P-+) w)]l] [(P [)P]][((- p)
(20)
RTT(PP - T(P - H)lR
= (UAU
(R URTUAARRTT""
Eqn. 20 holds for any rotation matrix Rt and recalling that we uniquely conduct SVD as proved in Step I, we derive
= R.U.
(21)
Finally, according to the definition of our PCA-normalization and Eqns. 1111).. obvious that
(Y)) = (Y-r)V' (P- (0)--)RRTUU (Pppp)- TP))
(22)
which means (P) is SE(3)-invrriant The SE(3)-innariancc of Tp(g) can be proven similarly. Taking the -ivaa-iinarrant Tp(P), TP(g) as input, our displacement predictor D will produce SE(3) -iivvarraa output, i.e., Do (PP))), TP(g)} Do (T(())(((P)), T<())((()))} Jς SE(3).
14

---

Table The time process 10,000 points on the ABC [52] dataset using one NVIDIA 4090 GPU.
Operator
SVD
PE
SRM
PFEM
IPGE
Others
Total
Time(s)
0.4473
0.1773
0.0297
0.0004
0.0008
0.02
0.6688
A.2 Proof of Theorem
Theorem Given query point q and patch P, implicit function F(q) is -equiveruiariiatt.
Following the denotations in the proof of Lemma 1, we denote the query point of patch P, as P1,P2: P2, and we further denote the predicted (E -iivaraatt displacements for query point P1: P2 as Pi, P2, and the final prediction of p1 P2 are respectively
F(p)) = pp (pt) = PÍ + +W, F(P2) = T -VV + v.
(23)
According to Eqns 19 we can derive
F(p2) =V²V +U
=
+ (TR+T)
(24)
.PijU' + + =
which means F(P1) is equivariant under SE(3) transformation of point patch P.
B Architecture Details
MLPs. "e, Eqn. 7 consists four 1 x convolution layers with 6, 32, 64, and 128 hidden units. "9, and ye, in Equ. 88 consists of units while these for yop are 3, 32, 64, and 128. For na, in Eqn. ((44), the unit numbers are 256, 512, 256, and 384. For yo.. the unit numbers
convolution layers with 3, 32, 64, and 128 hidden
256, 256, 256, and 256. All feature dimensions are 128. For the multi-head memory bank M, the number the memory bank is set as NM = 4, with each memory bank containing 596 items.
C
Additional Results
C.1
The visualization of the learned memory bank
In Figure 7
we provided two approaches to visualize the learned memory bank.
(1) We visualize the set of point patches with the highest weights the corresponding element of the memory bank (the weights are computed by Eqn. (12). These patches are highlighted by colors in these examples. It shows that the patches with high weights each element of memory have similar geometry structures.
(2) We further visualize (by (-SNE) the features of point patches with the highest weights (Eqn. 12 to different elements of the learned memory bank, rendered by different colors. shows that the patches with high weights assigned different elements of the learned memory bank have clustered features the feature space.
C.2 A detailed analysis of the inference time
In Table we report the time consumption of each operator in PEIF to process 10,000 query points. Specifically, the operations include SVD (Singular Value Decomposition), PE (Point-wise Feature Extraction), SRM (Spatial Relation Module), PFEM (Patch Feature Extraction Module), IPGE (Intrinsic Patch Geometry Extractor) and Others (other Conv layers).
C.3 Computational cost
We report the computational cost in Table including the training time per epoch, training memory, testing time per 3D shape, and testing memory cost the ABC dataset. Methods of GeoUDF and
15

---

Figure 7: The visualization of the learned memory bank. (a)-(c) Visualization of patches correspond- ing the same memory item with the highest weights (Different colors represent different items). (d) t-SNE visualization of patch features. The patches with the highest weights to different memory items are highlighted different colors.
Table The comparison of computational cost.
Cost NVF
Training Time (s)
Training Mem (G)
Testing Time (s)
Testing Mem (G)
28.06
6.70
73,9
0.66
GeoUDF
61.20+58.78
14.99+14.99
124.73
GridFormer
26.48-26.77
6.59+6.59
13.32
0.31
E-GraphONet
37.82
16.11
1.92
1,49
PEIF (Ours) 34.56
17.75
40.98
1.56
GridFormer include two stages of upsampiing/reconstuuiionn and reconstruction/refineeeet We report the computation cost them in each table cell with two values (denoted as + +),, respectively representing the costs for each stage.
C.4 Additional results on degraded data
We evaluate the performance of our PEIF on different data degradations (sparse, noisy, and partial) on the ABC [52) dataset. the following experiments, the test input point clouds with different degradations, and the results reported in Tables 0
Sparse point cloud. In previous experiments, all the compared methods use the same number of input points (10k) for each shape in testing, as NVE. We randomly select a subset input points as input, and the results are in Table
Table 9: The reconstruction results of sparse data on the ABC [52) dataset.
N
5k
N
Method
CD |
EMD
NC1
F-Score
CD
EMD |
NC
F-Score f
NVF
0.297
2.706
0.935
0.979
0.409
2.725
0.932
0.946
GeoUDF 2
0.306
2.726
0.940
0.985
0.399
2.711
0.935
0.952
GridFormer II]
0.292
2.694
0,952
0.982
0.369
2.703
0.945
0.956
PEIr(Ouus)
0.269
2.679
0.945
0.988
0.360
2.685
0.938
0.960
16

---

Noisy point cloud. We plugged Gaussian noise with standard deviation (a) as 0.005 and 0.01 to the input points. The results are reported in Table I0
Table 10: The reconstruction results from noisy input on the ABC
S2
dataset.
0.005
0.01
Method
CD 4
EMD
NC
F-Score
CD
EMD 4
NC
F-Score f
NVF
9
0.512
3.257
0.712
0.924
0.792
3.687
0.723
0.693
GeoLiDF F
0.496
3.268
0.732
0,911
0.785
3.428
0.710
0.655
GridFormer
0.839
3.321
0.793
0.805
1.132
3.379
0.759
0.510
PHIF(Ours)
0.480
3.132
0.745
0.952
0.773
3,358
0.715
0.702
Partial point cloud. cloud. Specifically, we use the farthest point sampling to select set of center points and remove their KNN points to ensure the sampling fraction. The results are reported in Table
We remove a fraction (with ratio p) of the input points to form a partial point
Table 11: The reconstruction results from partial points on the ABC [22 dataset.
10%
20%
Method
CD |
EMD
NC
F-Score
CD
EMD |
NC
F-Score f
NVF 9
0.264
2.697
0,943
0.992
0,274
2.710
0.940
0.990
GooUDF
0.268
2.695
0.959
0,994
0,275
2.745
0.947
0.991
GridFormer
0.267
2.706
0.982
0.298
2.746
0.987
(Elr(Ours)
0.246
2.692
0,960
0.996
0.249
2.697
0.956
0.995
C.5
Additional ablation study
Ablation study ABC [52]. In Table 13 we present the additional results of our ablation studies with respect the coefficient B of the joint loss function in Section 4.4 the impact of keypoint- boosted feature representation fr, in Section 4.3
Ablation study in MGN (53]. In Table [2, we present the additional ablation results of K on MGN dataset. The testing results in Ta- ble B on the MNG dataset using the trained model with K = 54 on the Synthetic Rooms dataset. As shown in Table 12 when changing K to 48 and 32, the test results using the corresponding K on MGN are stable.
Table 12: The impact of K when training on Synthetic Rooms and testing on MGN (33
K-
CD L
EMD J
NC 1
F-Score f
48
0.247
2.724
0.961
0.998
32
0.252
2.735
0.959
0.991
C.6 Additional Visualizations
In this section, we provide more visual results of our PEIF across four datasets: ShapeNet, ABC, Synthetic Rooms, and MGN.
ShapeNet [51]. We present three examples further illustrate the performance of our PEIF The 1st row in Figure 8 are results of examples selected from base classes. In contrast, results in the 2nd and 3rd rows come from the novel classes in this dataset. The visualization results show that our PEIF not only preserves local structures but also demonstrates remarkable generalization capability.
We validate the efficacy of our multi-head memory bank on this dataset, with visualizations depicted as shown in Figure 9 By using the intrinsic patch geometry extractor with a multi-head memory bank our PEIF reconstructs better 3D surfaces which more smooth and complete.
ABC (22).
Synthetic Rooms 19). Beyond the reconstruction of CAD objects, we also validate the performance of our PEIF on synthesized scene dataset, thereby extending its applicability to more complex environmental contexts. The results in Figure [0 demonstrate that our PEIF achieves competitive results in terms of structural completeness and surface smoothness.
17

---

Table 13: Additional ablation study in ABC [52]]
Setting
CD |
EMD 4
NC
F-Score f
w/o 0.247
2.684
0.961
0.997
Feature fP,
w/
0.241
2.672
0.969
0.998
0.0
0.244
2.705
0.962
0.997
8
0.01
0.245
2.683
0.960
0.996
0.241
2.672
0.969
0.998
0.5
0.248
2.732
0.961
0.996
POCO
GIFS
ALTO
NVF
GeoUDF GridFormer
PEIF
GT
Figure 8: Results on ShapeNet [5T] dataset. The results from the 1st row are selected from base classes in T Objects in the 2nd to 3rd rows are selected from meshes used for class-unseen reconstruction (novel classes Table T
Figure 9: Examples of results on ABC [52 dataset with (below) and without (top) intrinsic patch geometry extractor using multi-head memory bank.
MGN 153.
To evaluate the generalization of competing methods, we conducted tests on the real scanned data. and the corresponding results are presented in Figure [T] The results indicate that our PEIF yields reconstruction results with better completeness and surface smoothness.
18

---

GT
POCO
GIFS
ALTO
NVF
GeoUDF
GridFormer
PEIF
Figure 10: Examples of results on Synthetic Room dataset.
GT
GeoUDF
NVF
GridFormer
Ours
Figure 11: The visual example of cross-domain evaluation on the real scanned dataset MGN where the model trained on Synthetic Room dataset 19
53
19

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility. transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS paper checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
See abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See section Conlusion.
20

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: See Appendix
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
See the implementation details of section Experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
21

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. w
For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We will release the codes at https///ithub..com/maabbt 1 12/PEIF.git
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (nttps://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ 7/nips. /ttppp.cc/pblic/guideeee http://oeessmmiiiooooooooooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
22

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: See section Experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: error bars are not reported because it would be too computationally expensive.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
See section Experiments.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
23

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttp::/neuurpps http::/p.....p.bbiiecccccccciaae
Answer:
[Yes]
Justification: We have read the code of ethics carefully and ensure there is violation.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See section Conclusion.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification: There are no contents concerning safeguards.
Guidelines:
The answer NA means that the paper poses such risks.

---

Released models that have high risk for misuse or dual-use use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: We were unable to find the license for the datasets used.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: There are no new assets introduced in the paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: There is Crowdsourcing and Research with Human Subjects in this paper.
25

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: There are no potential risks as far we are concerned.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
26


Reviews:
[{"review_id": "IQ9L6EPzf6", "reviewer": "Reviewer_dfBj", "summary": "In this paper, the authors address the task of surface reconstruction. They propose a patch-level pose-invariant representation of 3D objects, which is employed in the design of a patch-level equivariant implicit function. The proposed PEIF framework is composed of three modules: the spatial relation module, the patch feature extraction module, and the intrinsic patch geometry extractor. They authors demonstrate the effectiveness of the proposed framework for the surface reconstruction task through comprehensive experimental evaluations.", "questions": "- Could the authors provide a visualization of the learned memory bank? This would aid in comprehending the proposed intrinsic patch geometry extractor.\n- What value of K is used in the MGN experiments?\n- Minor comments\n    - L57: 3D construction -> 3D reconstruction\n    - L62: introduces Transformer -> introduces transformer\n    - Figure2: displacement Predictor -> displacement predictor\n    - L180: multi-head memory bank index starts from 1, but it starts from 0 in Figure 3\n    - L191: displacement estimate -> displacement estimation\n    - L234: distance(CD -> distance (CD", "limitations": "The authors have adequately addressed the limitations and societal impact in the main paper.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The authors introduce a novel pose normalization scheme and a displacement predictor that employs the proposed pose normalization scheme, accompanied by rigorous proofs\n- The proposed method shows the state-of-the-art performance in the surface reconstruction task, surpassing other equivariant method (i.e. E-GraphONet)\n- The proposed method shows the state-of-the-art performance in the cross-domain evaluation setting", "weaknesses": "- The proposed method exhibits a significantly longer inference time compared to other equivariant methods (E-GraphONet). It appears that the majority of thie increased inference time results from the computation of the SVD. A detailed analysis of the inference time would be beneficial for a more comprehensive understanding of the proposed method\n- The proposed method shows comparable performance compared to GeoUDF (which is not an equivariant method). This raises questions about the necessity of using an equivariant method for the surface reconstruction task\n- The authors claim that the proposed pose-invariant property is intended to enhance the cross-domain generalization ability. To validate this, the cross-domain experiment should include the result of E-GraphONet (which is also pose-invariant)\n- In table 4, it seems that other algorithms also quite robust to rotation changes. The authors are encouraged to provide further explanation on this observation.\n- An ablation study concerning the three modules (i.e., the spatial relation module, the patch feature extraction module, and the intrinsic patch geometry extractor) is not provided. The inclusion of such a study would be valuable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for these comments. We address the concerns and questions as follows.\n\n**Q1: The visualization of the learned memory bank.**\n\nWe provided two approaches for visualizations of the learned memory bank. Please refer to Figure 1 in the attached PDF file uploaded in the top “general response”.\n\n(1) We visualize the set of point patches with the highest weights to the corresponding element of the memory bank (the weights are computed by Eqn. (12)). These patches are highlighted by colors in these examples. It shows that the patches with high weights to each element of memory have similar geometry structures.\n\n(2) We further visualize (by t-SNE) the features of point patches with the highest weights (Eqn. (12)) to different elements of the learned memory, rendered by different colors. It shows that the patches with high weights assigned to different elements of learned memory have clustered features in the feature space. \n\nWe will include these visualizations in the appendix of our paper.\n\n**Q2: What value of $K$ is used in the MGN experiments?**\n\n$K$ is set to 54 in testing on the MNG dataset, using the trained model with $K=54$ on Synthetic Rooms dataset. As shown in Table R1-1, when changing $K$ to 48 and 32 on the Synthetic Rooms in training, the test results using the corresponding $K$ on MGN are stable. We also presented the ablation studies on $K$ in Table 5 of the manuscript.\n\nTable R1-1. The impact of $K$ when training on Synthetic Rooms and testing on MGN.\nK|CD&darr;|EMD&darr;|NC&uarr;|F-Score&uarr;\n-|-|-|-|-\n48|0.247|2.724|0.961|0.998\n32|0.252|2.735|0.959|0.991\n\n**Q3: Minor comments on typos.**\n\nThanks and we will  correct them.\n\n**Q4: A detailed analysis of the inference time.**\n\nIn Table R1-2, we report the time consumption of each operator in PEIF to process 10,000 query points. Specifically, the operations include SVD (Singular Value Decomposition), PE (Point-wise Feature Extraction), SRM (Spatial Relation Module), PFEM (Patch Feature Extraction Module), IPGE (Intrinsic Patch Geometry Extractor) and Others (other Conv layers).\n\nTable R1-2. The time to process 10,000 points on the ABC dataset using one NVIDIA 4090 GPU.\nOperator|SVD|FE|SRM|PFEM|IPGE|Others|Total\n-|-|-|-|-|-|-|-\nTime (s)|0.4473|0.1773|0.0297|0.0004|0.0008|0.02|0.6688\n\nWe will add them in the main body or appendix of our paper.\n\n**Q5: Comparison with GeoUDF and necessity of equivariance.**\n\nAs shown in Tables 1, 2 of the manuscript and Tables R3-1 to R3-3 in response to Reviewer H6Ua, our results are better than GeoUF, especially for degraded data. Moreover, Table R2-1 in response to Reviewer 6vbf shows that our time/memory cost is lower than GeoUDF.\n\n(1) Learning the equivariant implicit representation ensures that the reconstructed surfaces are robust to the rotation of the input points. Figure 2 of the uploaded PDF in “general response” shows that GeoUDF generated 3D surfaces with some noise artifacts after rotation. While our results are more smooth and stable to rotations.\n\n(2) The equivariance/invariance is important to our model's performance. We learn the equivariant implicit vector field based on patch intrinsic geometric features by removing patch poses. It is inspired by the observation that local patches are repetitively appearing on 3D shapes if removing patch poses. This novel idea is essential for achieving improved reconstruction accuracy, while robust to rotations. As shown in Table 5, if removing the equivariance design (i.e., w/o pose normalization), the 3D reconstruction accuracy apparently decreases.\n\n**Q6: The comparison of cross-domain generalization ability with E-GraphONet.**\n\nAs suggested, we compare with E-GraphONet in the cross-domain experiment as reported in Table R1-3. Our PEIF achieves better quantitative results on the MGN dataset. These results will be included in Table 3 of our paper.\n\nTable R1-3. The cross-domain evaluation on the real MGN dataset, where the model is pre-trained on the Synthetic Room dataset.\nMethod|CD&darr;|EMD&darr;|NC&uarr;|F-Score&uarr;\n-|-|-|-|-\nE-GraphONet|0.433|3.817|0.863|0.920\nPEIF (Ours)|0.241|2.672|0.969|0.998\n\n**Q7: Further clarification on the robustness of the competed algorithms to rotation changes.**\n\nThanks for the question. We have further evaluated the rotation robustness of the methods in Table 4 to different rotation angles, which is reported in Table R1-4.  Figures 2, 3 of the uploaded PDF in \"general response\" demonstrate that NVF and GeoUDF exhibit noise and holes after rotation, while our results are more smooth and robust to rotations. In Q5, we also discussed the importance of equivalence/invariance of vector field/patch features to our model for achieving good performance.\n\nTable R1-4. The results for different rotation angles (0/90/180/270) on the ABC dataset.\nMethod|CD&darr;|EMD&darr;|NC&uarr;|F-Score&uarr;\n-|-|-|-|-\nNVF|0.245/0.260/0.263/0.262|2.685/2.683/2.685/2.687|0.963/0.950/0.944/0.952|0.996/0.994/0.993/0.993\nGeoUDF|0.245/0.256/0.253/0.263|2.688/2.691/2.698/2.683|0.9640.956/0.958/0.966|0.997/0.993/0.996/0.994\nE-GraphONet|0.432/0.441/0.436/0.445|2.688/2.696/2.702/2.690|0.910/0.906/0.906/0.897|0.906/0.896/0.909/0.906\nPEIF(Ours)|0.241/0.249/0.247/0.243|2.672/2.675/2.678/2.676|0.969/0.966/0.964/0.968|0.998/0.996/0.998/0.998\n\n**Q8: An ablation study of the three modules.**\n\nWe conduct an ablation study of the three modules (e.g., SRM, PFEM, IPGE) in our PEIF on the ABC dataset. Table R1-5 reports the quantitative measures of our PEIF without these modules. The results show that the intrinsic patch geometry extractor (IPGE) contributes more to the performance of PEIF.\n\nTable R1-5. Ablation study of the three main modules proposed in PEIF.\nSetting|CD&darr;|EMD&darr;|NC&uarr;|F-Score&uarr;\n-|-|-|-|-\nw/o SRM|0.243|2.686|0.962|0.997\nw/o PFEM|0.244|2.699|0.961|0.996\nw/o IPGE|0.276|2.715|0.959|0.992\nPEIF (Full)|0.241|2.672|0.969|0.998", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LC8zmgvGM9", "reviewer": "Reviewer_6vbf", "summary": "The authors introduce the 3D Patch-level Equivariant Implicit Function (PEIF), leveraging a 3D Patch-level Pose-Invariant Representation (PPIR) to address the surface reconstruction task. To overcome the limitation that existing Implicit Neural Representations (INRs) are not equivariant to 3D rotation, they develop PEIF to encode both equivariant and invariant information, thereby enhancing generalization to unseen 3D rotations. The SE(3)-equivariant implicit function is optimized using displacement optimization loss and patch discrimination loss with ground-truth 3D models. Experimental results on surface reconstruction datasets validate the effectiveness of PEIF.", "questions": "Further Research Direction: The concept introduced could potentially be extended to few-shot training scenarios, where the local embedding might capture various types of 3D rotations. Did the authors explore this direction?", "limitations": "I recommend a weak accept score for this paper due to its strong motivation and technical novelty in addressing the limitations of existing INRs with respect to 3D rotation equivariance. The introduction of the 3D Patch-level Equivariant Implicit Function (PEIF) and its verification of rotational robustness demonstrate a significant advancement in 3D surface reconstruction, achieving state-of-the-art performance on multiple datasets. \n\nHowever, there are some limitations that should be addressed. The experimental settings regarding rotational robustness need further clarification. Additionally, the paper lacks citations to related works in 2D pixel-level correspondence that employ similar techniques, which would strengthen the discussion of novelty and prior art. Finally, the computational cost of PEIF, in terms of computation time and memory consumption, is not discussed, leaving questions about its practical applicability compared to existing methods. Addressing these points would enhance the overall contribution and clarity of the paper.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. Motivation: The study is well-motivated, addressing the redundancy in existing INR-based methods concerning local orientation-normalized patches. Moreover, the current methods are weak against unseen rotations of local shapes.\n\n2. Technical Novelty and Soundness: The introduction of local pose-invariant representation for SE(3) equivariant implicit function is novel for 3D surface reconstruction. Patch-based pose normalization facilitates efficient training without the need for rotation augmentation.\n\n3. Verification of Rotational Robustness: The authors demonstrate the rotational robustness of the proposed method, as shown in Table 4.\n\n4. Performance Improvement: The proposed PEIF achieves superior performance compared to both equivariant and non-equivariant surface reconstruction methods on the ShapeNet, ABC, and SyntheticRooms datasets, as indicated in Tables 1 and 2.", "weaknesses": "1. Rotational Robustness: Further clarification is needed regarding the experimental settings in Table 4. Specifically, it is unclear whether \"w/o rotation\" and \"w/ rotation\" refer to rotation augmentation during training or testing.\n\n1. Missing Citations: A similar approach exists in 2D pixel-level correspondence, as detailed in the work by Lee et al. (CVPR 2023). This study also utilizes local-level dominant orientation from rotation-equivariant features and normalizes the equivariant feature using the dominant orientation for an invariant descriptor. It would be beneficial to cite this work and discuss the similarities and differences. \n\n[A] Learning Rotation-Equivariant Features for Visual Correspondence (Lee et al., CVPR 2023)\n\n1. Computational Cost: There is a lack of discussion regarding the computational cost of the proposed PEIF. Information on computation time and memory consumption, and a comparison with E-GraphONet would be valuable.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive comment that our approach is well-motivated and novel. Please see below for responses.\n\n**Q1: The experimental settings of \"w/o rotation\" and \"w/ rotation\" in Table 4.**\n\nIn Table 4, \"w/ rotation\" and \"w/o rotation\" represent that the testing input point cloud is with and without arbitrary rotation respectively. This will be clarified in paper.\n\n**Q2: Missing citations of related works in 2D pixel-level.**\n\nThanks for this comment. RELF [1] uses group equivariant CNNs to extract discriminative rotation-invariant descriptors for 2D images. Differently, our framework is designed for 3D reconstruction, and we design equivariant implicit function learning for 3D reconstruction. We will include the reference of equivariant network in 2D and discussion of RELF [1] in our paper.\n\n**Q3: Computational cost.**\n\nWe report the computational cost, including the training time per epoch, training memory, testing time per 3D shape, and testing memory cost on the ABC dataset, in Table R2-1. Methods of GeoUDF and GridFormer include two stages of upsampling/reconstruction and reconstruction/refinement. We report the computation cost of them in each table cell with two values (denoted as · + ·), respectively representing the costs for each stage. We will include these details in the appendix. \n\nTable R2-1. The comparison of computational cost. \nCost|Training Time (s)|Training Mem (G)|Testing Time (s)|Testing Mem (G)\n|-|-|-|-|-|\nNVF|28.06|6.70|73.9|0.66\nGeoUDF|61.20+58.78|14.99+14.99|124.73|2.27\nGridFormer|26.48+26.78|6.59+6.59|13.32|0.31\nE-GraphONet|37.82|16.11|1.92|1.49\nPEIF (Ours)|34.56|17.75|40.98|1.56\n\n**Q4: Further research direction.**\n\nIt is a good suggestion to extend our approach to few-shot training scenarios. Along this direction, our patch-based pose invariant representation can be taken as a foundation network for pre-training, followed by fine-tuning on few-shot examples. In the pre-training step, we may learn a general representation of intrinsic 3D patch features, and the fine-tuning may adapt these representations to the given few-shot examples. We will include this direction as a future work in the conclusion section.\n\n[1] Lee et al. Learning rotation-equivariant features for visual correspondence. CVPR. 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OKhWT6Vf0t", "reviewer": "Reviewer_H6Ua", "summary": "This paper studies a simple task: input dense point cloud and output the implicit surface reconstruction of the geometry. To achieve this goal, the model uses an \"equivariant\" network to predict the displacement field. Since the input point cloud is dense, this paper crops the nearest patch on the surface point cloud to the query space point and uses PCA to canonicalize the patch. Once aligned, a transformer will predict the query points displacement vector to the surface. Since the PCA canonicalization is known, the displacement can be transformed back. This simple task is evaluated on shape and scene data.", "questions": "Please see weakness", "limitations": "Some limitations are discussed in the end.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The insight of reusing elementary shapes with different poses is good.\n- This paper, although straightforward, considers using equivariance to model such elementary shapes/local intrinsic patterns.", "weaknesses": "- Heuristic baseline: I have a feeling that the model may depend very much on the dense KNN queries of the surface point cloud, in other words, the network is learning a potentially too easy task, just find the nearest point (or interpolate the nearest point) in the patch and compute a displacement to it if the point patch is too dense. A heuristic baseline could be just fitting a small parametric (polynomial, or even plane) to the nearest patch, and analytically computing, or directly finding the nearest point to produce the displacement vector.\n- Noise/Sparse/Partial data? The task might be too easy in the current literature. Since the input is a completely dense point cloud, the geometry is almost given, and this work still depends on the surface point KNN queries to do the canonicalization, so what happens if the input observation is partial, sparse, or noisy?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the valuable comments and suggestions. Please see below for the responses.\n\n**Q1: A heuristic baseline.**\n\nWe aim to learn the equivariant implicit function that outputs the vector of each query point to its nearest point on the unknown continuous 3D surface. Since only discrete points on the surface are observed, it is challenging to infer the vector of the query point to the continuous 3D surface based on these discrete points. Instead of using local knn points to regress a local surface function (e.g., polynomial functions) for each query point, our neural network-based approach infers the vector of the query point to the surface by learning the intrinsic geometry feature of the local patch to estimate the vector of each query point. We also design and take advantage of the rotation-invariant patch features for achieving equivariant implicit function learning. This design enables us to achieve sota performance for 3D reconstruction. In Q2, we also evaluate the performance of the degradations of input point clouds.\n\n**Q2: The performance of PEIF on sparse/noisy/partial data.**\n\nAs suggested, we evaluate the performance of our PEIF on different data degradations (sparse, noisy, and partial) on the ABC dataset.\n\nIn the following experiments, the test input point clouds with different degradations, and the results are reported in Tables R3-1 to R3-3. In Tables 1-3 of the uploaded PDF file in “general response”, we also report the results when both the training and test point clouds are degraded.\n\n(1) Sparse point cloud. In experiments of the original submission, all the compared methods use the same number of input points (10k) for each shape in testing, as NVF. We randomly select a subset of input points as input, and the results are in Table R3-1.\n\nTable R3-1. The reconstruction results of sparse data on the ABC dataset. \nMethods|$N$|CD&darr;|EMD&darr;|NC&uarr;|F-Score&uarr;\n-|-|-|-|-|-\nNVF|5k|0.297|2.706|0.935|0.979\nGeoUDF|5k|0.306|2.726|0.940|0.985\nGridFormer|5k|0.292|2.694|**0.952**|0.982\nPEIF (Ours)|5k|**0.269**|**2.679**|0.945|**0.988**\n-----|-----|-----|-----|-----|-----|-----\nNVF|2k|0.409|2.725|0.932|0.946\nGeoUDF|2k|0.399|2.711|0.935|0.952\nGridFormer|2k|0.369|2.703|**0.945**|0.956\nPEIF (Ours)|2k|**0.360**|**2.685**|0.938|**0.960**\n\n(2) Noisy point cloud. We plugged Gaussian noise with standard deviation ($\\sigma$) as 0.005 and 0.01 to the input points.\n\nTable R3-2. The reconstruction results from noisy input on the ABC dataset. \nMethods|$\\sigma$|CD&darr;|EMD&darr;|NC&uarr;|F-Score&uarr;\n-|-|-|-|-|-\nNVF|0.005|0.512|3.257|0.712|0.924\nGeoUDF|0.005|0.496|3.268|0.732|0.911\nGridFormer|0.005|0.839|3.321|**0.793**|0.805\nPEIF (Ours)|0.005|**0.480**|**3.132**|0.745| **0.952**\n-----|-----|-----|-----|-----|-----|-----\nNVF|0.01|0.792|3.687|0.723|0.693\nGeoUDF|0.01|0.785|3.428|0.710|0.655\nGridFormer|0.01|1.132|3.379|**0.759**|0.510\nPEIF (Ours)|0.01|**0.773**|**3.358**|0.715|**0.702**\n\n(3) Partial point cloud. We remove a fraction (with ratio $p$) of the input points to form a partial point cloud. Specifically, we use the farthest point sampling to select a set of center points and remove their K-NN points to ensure the sampling fraction. \n\nTable R3-3. The reconstruction results from partial points on the ABC dataset. \nMethods|$p$|CD&darr;|EMD&darr;|NC&uarr;|F-Score&uarr;\n-|-|-|-|-|-\nNVF|10%|0.264|2.697|0.943|0.992\nGeoUDF|10%|0.268|2.695|**0.959**|0.994\nGridFormer|10%|0.267|2.706|0.964|0.982\nPEIF (Ours)|10%|**0.246**|**2.692**|0.960|**0.996**\n-----|-----|-----|-----|-----|-----|-----\nNVF|20%|0.274|2.710|0.940|0.990\nGeoUDF|20%|0.275|2.745|0.947|0.991\nGridFormer|20%|0.298|2.746|0.946|0.987\nPEIF (Ours)|20%|**0.249**|**2.697**|**0.956**|**0.995**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qhaW474rwo", "reviewer": "Reviewer_FF8y", "summary": "The paper addresses 3D surface reconstruction from point clouds. It proposes a patchwise rotation equivariant neural network to map query points to their 3D displacement to the surface. The local rotation equivariance allows weight-sharing between similar patches at different orientation, and  displacement fields have been shown to outperform occupancy and distance fields. Experiments show the method outperforms the baselines on several datasets.", "questions": "Q1) Poisson surface reconstruction [6] is a classical method for surface reconstruction that is quite robust, how does the proposed method compare to it and similar follow-up works? \n\n## Typos:\n\nL123: P -> P_i\n\nL48, L77: PEFI -> PEIF\n\n## References:\n\n[6] Kazhdan et al, \"Poisson Surface Reconstruction.\", SGP'06.", "limitations": "The limitation regarding robustness of PCA-alignment should be addressed more clearly (see W2).", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "S1) Using equivariant models to promote better use of model capacity is a great idea. Point clouds often present patches that are similar up to rotation so this design choice makes a lot of sense.\n\nS2) Results are strong and seems state-of-the-art on surface reconstruction from point clouds.", "weaknesses": "W1) As far as I understand, the ideas in the paper are not novel, so the contributions are around  combining existing ideas. a) NVF [1] introduced the idea of using vector instead of distance fields, b) E-GraphONet [2] uses the idea of rotation-equivariant models for implicit surface representation, c) Zhao et al [3] uses the particular way of achieving equivariance through SVD on point sets. This might not be a deal-breaker since the results are good but more novel ideas would make for a stronger submission.\n\nW2) I think the PCA-based alignment is not very robust. While it is perfectly rotation equivariant given the exact same point cloud patch on a different orientation, I think in practice we would see slightly different patches so the alignment is not guaranteed. Moreover, the way the ambiguity on the axis orientation is resolved seems to rely on the furthest point position so a moving a single point slightly might change the orientation drastically. There are other methods that are use equivariant layers which seem more appropriate such as Vector-Neurons [4] and SE(3)-transformers [5], why weren't they considered?\n\nW3) Given W2 I found the design decision of using the PCA-alignment quite arbitrary. Given that the most related works are NVF and E-GraphONet, I believe a more natural choice would be to modify E-GraphONet to predict vector fields instead of occupancy fields, would it perform better than the proposed method? If the goal is to show that the PCA-alignment can be better than equivariant layers, a comparison against E-GraphONet on occupancy field prediction should have been performed. \n\n## References:\n\n[1] Yang et al, \"Neural Vector Fields: Implicit Representation by Explicit Learning\", CVPR'23.\n\n[2] Chen et al, \"3D Equivariant Graph Implicit Functions\", ECCV'22.\n\n[3] Zhao, \"Rotation invariant point cloud classification: where local geometry meets global\ntopology.\", 2021.\n\n[4] Deng et al, \"Vector neurons: A general framework for so (3)-equivariant networks\", ICCV'21.\n\n[5] Fuchs et al, \"SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks\", NeurIPS'20.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the questions and comments. Please see the following responses.\n\n**Q1: Novelty and relevant methods discussion.**\n\nThanks for this question. The major novelty of our approach is to learn the equivariant implicit vector fields for 3D reconstruction. Our novelty lies in the motivation and idea to design the patch-based intrinsic feature learning network. Our framework is inspired by the observation that local patches of 3D shapes repeatedly appear on 3D surfaces if removing the poses, we thus conduct pose normalization, and design the patch feature learning and learnable memory bank for learning the intrinsic patch geometry features. With them, the learned implicit vector fields are equivariant. These designs enable us to achieve state-of-the-art reconstruction results while being robust to the rotations of input points, as shown in the experiments.\n\nCompared with [1], our approach achieves equivariance for the vector field prediction. Compared with [2], our model is equivariant by proposing intrinsic patch geometric feature learning, instead of using vector neurons in [2]. Compared with [3], we tackle different tasks (classification vs. 3D reconstruction), and the model designs are significantly different. We will more clearly discuss the relation and novel contributions compared with these related works in the paper. \n\n**Q2: The Robustness of PCA-based alignment and comparison with other alternatives, such as Vector-Neurons[4] and SE(3)-transformers [5].**\n\nThanks for this good question. In our approach, we compute the PCA over local patches of local points. As suggested, we conduct experiments to test the robustness of PCA-based alignment, and compare with other alternatives using Vector-Neurons[4] and SE(3)-transformers [5].\n\n(1) To test the robustness of PCA, when computing the PCA over patches, we randomly perturbed the patch point coordinates by Gaussian noises with $\\sigma=0.001$ (the average distance from knn points to patch center point is about 0.004), resulting in perturbed PCA matrices. The results on the ABC dataset in Table R4-1 show the robustness of PEIF to PCA perturbations.\n\nTable R4-1. Comparison of results for random perturbations of rotation matrices. \nMethods|CD&darr;|EMD&darr;|NC&uarr;|F-Score&uarr;\n-|-|-|-|-\nw/ perturbation|0.250|2.680|0.960|0.990\nw/o perturbation|0.241|2.672|0.969|0.998\n\n(2) Using the same GPU, we substitute the layers of our network with the Vector-Neurons[4]-based layers, and set hyper-parameters fitting the GPU memory. The results are in Table R4-2. Table R4-3 provides the hyper-parameters and resource consumption of our model and Vector-Neurons[4]-based implementation.\n\nTable R4-2. The results of PEIF with Vector-Neurons [4] equivariant layers. \nMethods|CD&darr;|EMD&darr;|NC&uarr;|F-Score&uarr;\n-|-|-|-|-\nVector-Neurons|0.379|2.769|0.895|0.955\n\nTable R4-3. The hyper-parameter comparison of Vector-Neurons [4] and Pose-normalization based PEIF on ABC dataset with a single NVIDIA 4090 GPU.\nCost|Vector-Neurons [4]|PCA-based\n-|-|-\nFeature dim|64|128\nHidden feature dim|32|256\nPara (M)|0.71|7.65\nTraining time (s, per epoch)|283.59|34.56\nTraining memory (G)|15.97|17.75\nTesting time (s, per shape)|664.96|40.98\nTesting memory (G)|3.54|1.56\n\n(3) We also attempt to use the SE(3)-transformers [5] to replace the patch feature extraction (SRM, PFEM) in our model. Using the same GPU, even by setting the batch size to 1 and the feature dimension of 64, we experienced out-of-memory issues when training the SE(3)-transformer based implementation.\n\nAs a summary, the PCA-based patch pose normalization enables us to design a light-weight network achieving equivariant implicit function learning, and achieves sota results shown in experiments.\n\n**Q3: Further comparison with E-GraphONet [2].**\n\nFor a fair comparison, we tried to use PEIF to predict the occupancy field (OF) by only changing its loss to regress OF. In such a setting, we use the same training/test dataset as E-GraphONet. The reconstruction results are reported in Table R4-4. We also trained E-GraphONet to predict the vector field, we tried our best but the learned model cannot reasonably reconstruct object surfaces. The integration of training based on both occupancy and vector field prediction deserves us to try in the future.\n\nTable R4-4. The reconstruction results of PEIF predicting occupancy field on ABC dataset. \nMethods|CD&darr;|EMD&darr;|NC&uarr;|F-Score&uarr;\n-|-|-|-|-\nPEIF (OF)|0.874|4.7199|0.709|0.675\n\n**Q4: Comparison of PSR [6] and similar follow-up works on the ABC dataset.**\n\nAs suggested, we compare with PSR [6], OccNet [7], and SAP [8] on the ABC dataset in Table R4-5. Our PEIF achieves the best results.\n\nTable R4-5. The reconstruction results on ABC dataset. \nMethods|CD&darr;|EMD&darr;|NC&uarr;|F-Score&uarr;\n-|-|-|-|-\nPSR [6]|1.207|4.137|0.535|0.584\nOccNet [7]|0.691|3.261|0.797|0.636\nSAP [8]|0.376|2.765|0.946|0.962\nPEIF (Ours)|0.241|2.672|0.969|0.998\n\n**Q5: Typos.**\n\nThanks and we will fix these typos.\n\n[1] Yang et al. Neural vector fields: Implicit representation by explicit learning. CVPR. 2023. \\\n[2] Chen et al. 3D Equivariant Graph Implicit Functions. ECCV. 2022. \\\n[3] Zhao et al. Rotation invariant point cloud classification: where local geometry meets global topology. Pattern Recognition. 2021. \\\n[4] Deng et al. Vector neurons: A general framework for so (3)-equivariant networks. ICCV. 2021. \\\n[5] Fuchs et al. SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks, NeurIPS. 2020. \\\n[6] Kazhdan et al. Poisson surface reconstruction. Proceedings of the fourth Eurographics symposium on Geometry processing. 2006. \\\n[7] Mescheder et al. Occupancy networks: Learning 3d reconstruction in function space. CVPR. 2019. \\\n[8] Peng et al. Shape as points: A differentiable poisson solver. NeurIPS. 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# General Response
We appreciate the reviewers' positive comments on the novelty (especially Reviewers dfBj, 6vbf, H6Ua), motivation (especially Reviewers 6vbf, FF8y, H6Ua), and performance gain (especially Reviewers 6vbf, H6Ua, FF8y). We have responded to these questions/suggestions of reviewers, and will incorporate the corresponding revisions into our paper's main body or appendix. 

Following this general response, we uploaded a PDF file containing the figures/additional tables, referred to in the responses to each reviewer's comments.

Meta-review: Most reviewers have agreed on the soundness and good performance of the proposed work. There was some differences among the reviewers about the significance of the novelty, and the issue was that the main idea could be seen as somewhat combinatorial. The AC reviewed the paper in detail  to yield the following conclusion:

It is true that the concepts of neural vector fields and rotation equivariance are borrowed from other works, and the use of SVD for rotation calculation is common in the field. However, these are either more of a larger framework or a basic tool. On the other hand, designing an implicit function based on patch-wise representation (for rotation equivariance) is interesting and novel. This representation is not straightforward and is not a direct combination of building blocks, so there is a non-trivial contribution. Accordingly, an accept decision is recommended.
Decision: Accept (poster)