Title: Learning Generalizable Shape Completion with SIM(3) Equivariance
Venue: NeurIPS 2025
Accepted: yes
Keywords: Shape completion

Point cloud completion

Equivariance

Generalizability

3D representation
Abstract: 3D shape completion methods typically assume scans are pre-aligned to a canonical frame. This leaks pose and scale cues that networks may exploit to memorize absolute positions rather than inferring intrinsic geometry. When such alignment is absent in real data, performance collapses. We argue that robust generalization demands architectural equivariance to the similarity group, SIM(3), so the model remains agnostic to pose and scale. Following this principle, we introduce the first SIM(3)-equivariant shape completion network, whose modular layers successively canonicalize features, reason over similarity-invariant geometry, and restore the original frame. Under a de-biased evaluation protocol that removes the hidden cues, our model outperforms both equivariant and augmentation baselines on the PCN benchmark. It also sets new cross-domain records on real driving and indoor scans, lowering minimal matching distance on KITTI by 17\% and Chamfer distance $\ell1$ on OmniObject3D by 14\%. Perhaps surprisingly, ours under the stricter protocol still outperforms competitors under their biased settings. These results establish full SIM(3) equivariance as an effective route to truly generalizable shape completion.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a SIM(3)-equivariant shape completion network architecture based on an adaptation of the AdaPoinTr backbone. The original Transformer layers are replaced with novel SIM(3)-equivariant modules.\n\nThe proposed SIM(3)-equivariant module consists of three subcomponents:\n1. a canonicalization  module (layer norm extension) that normalizes scale and translation, while being rotation equivariant,\n2. a shape reasoning module (VN-transformer) that is rotation invariant, to obtain a fully SIM(3)-invariant representation,\n3. and a transform restoration module to restore scale and translation, to obtain a SIM(3)-equivariant output.\n\nThe resulting model, SIMECO, achieves significant improvements over state-of-the-art methods trained with SIM(3) data augmentation. On the PCN benchmark, it outperforms AdaPoinTr by 10% in CD-L1 and 8% in F1-score. However, SIMECO’s forward pass latency is higher (76 ms vs. 16 ms for AdaPoinTr).\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n1. The paper proposes the first SIM(3)-equivariant network architecture for shape completion.\n2. The proposed SIM(3)-equivariant module is intuitive and simple.\n3. The paper is well illustrated, especially with the use of color-coded circles and triangles to clearly highlight different types of equivariances and invariances.\n4. An extensive ablation study supports the architectural and design choices.\n\n\n## Weaknesses\n1. The motivation for introducing the relaxed problem in Equation (4) is unclear. Since Equation (2) appears sufficient, the use of a relaxed version where $g^* \\neq I$ seems to unnecessarily complicate the formulation. Clarification or justification would strengthen this point.\n\n2. References to the supplementary material are missing in several places. Explicitly pointing to the supplement would help readers, especially since important derivations and explanations are located there.\n\n3. Inconsistencies in notation: below Equation (9), the terms $E_{D^l}$​ and $E_i$​ appear to denote expectations, yet the standard expectation notation $\\mathbb{E}$ is not used (even though it is sometimes used in the supp. mat., albeit inconsistently).\n\n4. SIMECO is much slower (76ms vs 16ms) than AdaPoinTr and only increases the performance by 10%. The trade-off between accuracy and efficiency could be more thoroughly discussed.\n\nQUESTIONS:\n1. **Training Convergence** - All methods are reported to be trained for 200 epochs. However, did all of them fully converge? It would be helpful to include training and validation loss curves—especially for AdaPoinTr vs. SIMECO. I suspect that SIMECO may converge faster since it does not rely on SIM(3) data augmentation, but it's not entirely clear from the current presentation.\n\n2. **Table 2 – Possible Typo** - In Table 2, PoinTR is shown with a Fidelity score of 0.000. Is this a typo, or is there an explanation?\n\n3. **Clarification on Figure 10** - If I understood Figure 10 correctly, the model is trained with $N_{in}=2018$ input points, but evaluated at test time with fewer points. Is that interpretation correct?\n\nLIMITATIONS:\nThe limitations are discussed in the supp. mat.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, to address the bias learning caused by leaked pose and scale cues in most 3D shape completion methods, the authors proposed a more robust shape completion network that ensures SIM(3)-equivariant. With this novel shape completion network, a completed 3D shape can be inferred without the need for any ground-truth alignment, which makes the proposed method more available in practice use. The main contribution of the paper is stating the bias learning caused by leaked pose and scale cues in most traditional 3D shape completion methods; meanwhile, to address this issue, the authors decompose the optimizing objective into three parts: geometric reasoning, equivariance, and transform alignment. With this designed fully SIM(3)-equivariant network for shape completion, the inference 3D shapes follow arbitrary similarity transforms applied to inputs. The experiments show the effectiveness of the proposed method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The motivation and idea of this paper are reasonable, and the problem that poses and scale bias existing in shape completion methods is clearly stated, which shows the importance of the proposed method.\n2. A fully SIM(3)-equivariant network for shape completion is designed to address the identified problem, which includes feature canonicalization to remove translation and scale variation, shape reasoning to eliminate rotation transformations, and transform restoration to recover the absolute alignment. With all of these designs, the proposed network can learn representations agnostic to pose and scale in the training stage, and any transformations to inputs can be applied to the completed outputs.\n3. The visualizations in Fig. 5 and Fig.6 show the effectiveness of the proposed network, which can generate completed 3D shapes with more details.\n\n\nWeaknesses:\n1. The training time and datasets used for training are not clearly stated in the paper.\n2. It would be better if the authors could show the outputs given the same input partial shape with different poses and scales, which could better illustrate the proposed network is agnostic with pose and scale.\n3. I am curious about the performance of the proposed network if the GT completed shape with many thin structures.\n\nQUESTIONS:\nPlease refer to the weakness part.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

comments: SUMMARY:
This paper aims to address the issues that many point cloud completion methods leak pose and scale cues to the model so that networks may exploit to memorize absolute positions rather than inferring intrinsic geometry. The authors proposed the first shape completion architecture whose modules are SIM(3)-equivariant by design. The experiment shows the good performance of this method.

STRENGTHS AND WEAKNESSES:
Strengths:
1. The problem setting is very interesting.
2. The formulation is clear and intetersting.

Weaknesses:

1. SIM(3)-equivalence does not convey any meaning if the reader does not know the concept. The author may need to give formal definition, clear introduction earlier or reference for readers to understand what the method aims to do and what was proposed. Similar to SO(3) and SE(3)

2. The compared baseline are out-of-date. The authors need to compare with more recent baselines.

3. Insufficient experiment: For De-biased benchmark evaluation, this paper only provided results on the PCN dataset, which is insufficient as most recent point cloud completion paper will also present results on other ShapeNet-derived datasets to comprehensively evaluate their method.

4. Concern about zero-shot concern: It is invalid to claim it works on zero-shot case. It was tested on KITTI cars, while cars are included in the PCN dataset, therefore it is totally invalid to claim it is zero-shot. Also, many previous methods were also tested on KITTI, but, to my best knowledge, no one claimed that this is zero-shot.

QUESTIONS:
1. what is the computation cost of the proposed method

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper claims that they developed a SIM(3)-equivariant point cloud completion network. It aims to address the problem of shape completion in any reference frame, which is a very important problem, in my opinion. The method is through translation and scale normalization, building on top of the Vector Neurons feature representation. Experiments are conducted under a \"debiased protocol\", meaning that the bounding box and center information are not given during inference, which aligns much better with the practical use case.\n\nSTRENGTHS AND WEAKNESSES:\nI'm glad that this paper aimed to address the issue of shape completion from arbitrary reference frames without privileged ground-truth information in terms of the center or scale. This is the real challenge in practice, and solving it adds real value. \n\nIn 3D shape completion, I think we need to be careful about what equivariance we are talking about. The challenge of applying equivariance in 3D point clouds mainly comes from the partial observation. It means that the observations from different perspectives are no longer simply connected by a transformation, and it also brings ambiguities in terms of the reference frame. In this context, discussing equivariance to the transformation of a single input partial-view point cloud itself is not very useful, because you can always centralize and scale-normalize the point cloud, leaving only rotation variations to be concerned with, in which case, only rotation equivariance is needed. However, what's more important is finding the internal symmetries among different partial observations. While they can be associated using a common center, the underlying center is not in the data, and centering each input individually breaks the association across observations. Furthermore, the scale-normalization step, which is common in point cloud learning, leads to further discrepancy across observations, because different parts have different scales. The above calls for translation and scaling equivariance for the shape completion task. \n\nI don't think any work, including this work, has a perfect solution to the equivariance discussed above, because the centering and scaling information are not directly observable in a single input point cloud. However, most previous work (cited in this paper) only attempted to approximate translation equivariance (for example, using anchor points as in ESCAPE [10]), but left scaling out. This paper does not do any better in dealing with translation equivariance (it still relies on centering the input point cloud), but the scale normalization in the layer-normalization step, done for the features of each point, addressed the scaling variations in the partial view point clouds after the unit-ball normalization. I think this is the key contribution and differentiator of this work. \n\nThe proposed strategy showed its practical value in the experiments under the realistic evaluation protocol, which I appreciate. \n\nWeaknesses: \nAs in the previous discussion, this paper also followed the notation of equivariance of a single point cloud, and thus, the proof of equivariance is equally trivial and impractical as previous work. Specifically, the problem still lies in the translation. For different partial observations, the centering step subtracts a different offset, resulting in different outcomes, thus breaking the symmetry. \n\nYou may argue that the per-point feature canonicalization removed the dependency of perfect centralization. However, if I understand correctly, the vector features for each point in Vector Neurons are initialized with the absolute coordinate of each point and the local differences with other points. Global translation will change the global component but not the local component, resulting in different overall vector features, which cannot be cancelled by point-wise canonicalization. \n\nThe notations in the method explanation are unclear, especially in terms of dimensions. For example, the canonicalization step is written with a lot of ambiguities. What are \"channel-wise mean\", \"row-wise norm\", and \"modified layer normalization\"? I need to think very carefully to make sense of these descriptions. Use letters to disambiguate the dimensions. Figure 3 also didn't help too much.\n\nQUESTIONS:\nExplain how the vector features are initialized for each point, and discuss the equivariance property at this step. \n\nThe layer normalization seems to be done per point, but why is the scale restoration common across all points? Does that create a discrepancy? \n\nClarify the dimensions in the canonicalization step. \n\nExplain equation 47 in the appendix. Why is it designed this way? It seems to be different from typical VN-DGCNNs. What is the dimension of $\\bar{V}_i^l$, and what's its meaning? The per-point mean across the feature dimension? \n\nIs the definition of $mu^l$ wrong? The $E_i$ should be outside of the norm? Is $E$ expectation?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}