Title: Coarse-to-Fine 3D Part Assembly via Semantic Super-Parts and Symmetry-Aware Pose Estimation
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3D part assembly; semantic super-parts; dual-range feature propagation; symmetry-aware supervision
Abstract: We propose a novel two-stage framework, Coarse-to-Fine Part Assembly (CFPA), for 3D shape assembly from basic parts. Effective part assembly demands precise local geometric reasoning for accurate component assembly, as well as global structural understanding to ensure semantic coherence and plausible configurations. CFPA addresses this challenge by integrating semantic abstraction and symmetry-aware reasoning into a unified pose prediction process. In the first stage, semantic super-parts are constructed via an optimal transport formulation to capture high-level object structure, which is then propagated to individual parts through a dual-range feature propagation mechanism. The second stage refines part poses via cross-stage feature interaction and instance-level geometric encoding, improving spatial precision and coherence. To enable diverse yet valid assemblies, we introduce a symmetry-aware loss that jointly models both self-symmetry and inter-part geometric similarity, allowing for diverse but structurally consistent assemblies. Extensive experiments on the PartNet benchmark demonstrate that CFPA achieves state-of-the-art performance in assembly accuracy, structural consistency, and diversity across multiple categories.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes CFPA (Coarse-to-Fine Part Assembly), a novel two-stage framework for 3D part assembly that jointly addresses semantic structuring and geometric symmetry challenges. The method begins with a coarse pose estimation stage, where semantic super-parts are constructed using entropy-regularized optimal transport, providing high-level guidance via dual-range feature propagation. Then it takes a refinement stage that adjusts part poses through cross-stage attention and instance-level geometric encoding. Besides, this paper introduces a symmetry-aware loss that models both self-symmetry and inter-part geometric similarity. Extensive experiments on the PartNet benchmark show its effectiveness.\n\nSTRENGTHS AND WEAKNESSES:\nGoodness:\n\n1.\tThis paper presents a novel Coarse-to-Fine Part Assembly framework, integrates global structure and local geometric reasoning. Specifically, it takes a coarse pose estimation stage and a refinement stage to enhance accurate and diverse part assemblies.\n\n2.\tThis paper accounts for pose ambiguity caused by symmetry, enabling accurate and diverse configurations.\n\n3.\tExtensive experiments on PartNet demonstrate its effectiveness.\n\n4.\tThis paper is written and organized well.\n\n\nWeakness:\n\n1.\tSince the proposed method has the ability to learn the base part-level shapes based on the semantic guidance, it will be interesting to evaluate its performance on unseen categories.\n\n2.\tThe framework involves multiple attention layers, message passing, and OT computation. It will be interesting to report on the computational cost impact of these components.\n\n3.\tIt will be interesting to add visualization or analysis of learned super-parts or their semantic consistency across shapes.\n\n4.\tIt will be interesting to evaluate the generalization of the proposed symmetry-aware loss on other baselines, for that it is  a problem often overlooked or only heuristically addressed in prior work.\n\n5.\tLack comparison with Imagine [1] in Tab.1.\n\n[1] Imagine: Image-Guided 3D Part Assembly with Structure Knowledge Graph. AAAI 2025.\n\nQUESTIONS:\nRefer to the Weakness. I will improve my rates when the weaknesses are solved.\n\nLIMITATIONS:\nThe author analyzes the limitations and societal impacts adequately.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-02"}

comments: SUMMARY:
This paper is about 3D part assembly. The pipeline is clear, including *Coarse Pose Estimation with Super-Part*, *Pose Refinement*, and *Loss Function*. This paper introduces a dual-range feature propagation strategy to integrate both local and global structural cues. The Pose Refinement part applies a cross-stage attention and instant encoding. The Loss function integrates structural priors, geometric reasoning, and symmetry-aware supervision.

STRENGTHS AND WEAKNESSES:
Strength:
1. The method is easy to understand, and the experiment in the paper is SOTA
2. This task is interesting, and the comparison with other methods also proves the effectiveness of this method.

Weakness：
1. Some parts do not have a clear explanation process, like how the semantic super part $ \{h_{j}\} _ {j=1}^M $ comes from $\{h_{j}\} _ {j=1}^N$ and the geometrically similar parts in line 159.
2. The ablation study should contain the baseline method; simply removing any single part is not convincing enough.
3. The method part needs more explanation about how this module solves the issues.

QUESTIONS:
1. I would like to see more explanation of the approach, including how the designed method addresses the needs.
2. More ablation experiments are encouraged.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-06-30

comments: SUMMARY:
This paper introduces a method to assemble parts togehter. It uses optimal transport to produce a coarse assembly, before refining in the second stage. They train and test on PartNet.

STRENGTHS AND WEAKNESSES:
+ Strengths
  Thorough comparisons with prior works, substantial improvements demonstrated quantitatively. The methods seem non-trivial, and takes into account many nuanced problems of the part assembly task.

  Additionally, substantial ablations have been done to show the effectiveness of different design decisions.

+ Weaknesses

  The method was only demonstrated on 3 categories of Partnet, hindering how convincing the results are. Moreover, the qualitative results demonstrated in Figure 4 feels a bit underwhelming, especially given the complexity of the method.

  As far as I can tell (please correct if wrong), the models introduced are not conditional on some input image/pointcloud/mesh of the desired final assembly at test time. This strikes me as a little weird, since it's possible to assemble the parts to produce many different, though equality valid, final assemblies. For instance, in the first Column of tables in Figure 4, I would say that "Score", "Ours" and "Groundtruth" all corresponded to equally likely assemblies.
  `

QUESTIONS:
Why are only 3 categories of shapes shown in Table 1?

  How do you account for the various possibilities that the same set of parts can assemble to? Is there a way to communicate the desired final assembly through some form of conditioning?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces CFPA , a two-stage deep learning framework for 3D shape assembly that predicts  poses for individual parts to reconstruct coherent 3D shapes. The key innovation lies in constructing semantic super-parts through optimal transport to capture high-level object structure, followed by dual-range feature propagation that integrates both local and global structural information. The method addresses critical challenges in part assembly including complex geometric dependencies, symmetry handling, and the need for both precise local reasoning and global structural understanding. Experimental results on the PartNet benchmark demonstrate state-of-the-art performance across Chair, Table, and Lamp categories, achieving superior assembly accuracy, structural consistency, and assembly diversity compared to existing methods.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths And Weaknesses\n\n#### Strengths\n\n- **Novel Two-Stage Framework with Semantic Abstraction** The paper introduces a principled coarse-to-fine architecture that first constructs *semantic super-parts* via an entropy-regularized optimal transport formulation and then refines part poses through cross-stage attention and instance-level encoding. This dual-stage design effectively balances global structural understanding with precise local geometric reasoning\n- **Comprehensive Experimental Validation** Extensive experiments on the PartNet benchmark across Chair, Table, and Lamp categories demonstrate SOTA performance in assembly accuracy (SCD, PA, CA), structural consistency, and assembly diversity (QDS, WQDS). The ablation studies systematically quantify the contribution of each component, highlighting the author’s reasons for choosing each module.\n\n#### Weaknesses\n\n- **Insufficient Theoretical Foundations for Design Choices** While the authors provide comprehensive empirical validation demonstrating the effectiveness of each proposed module, the paper would benefit significantly from theoretical analysis or intuitive explanations underlying the design principles and mechanisms. For instance, the optimal transport formulation for semantic super-part construction, the dual-range feature propagation mechanism, and the cross-stage attention design would all benefit from theoretical justification or principled intuition beyond empirical effectiveness. This theoretical gap potentially constrains the generalizability and interpretability of the proposed framework to new scenarios or domains.\n- **Limited Novelty Claims Regarding Symmetry-Aware Loss Terms**  The symmetry-aware loss formulation presented in this work appears to have been explored in prior literature. For example, works such as Manual-PA[1] and other contemporary studies have previously addressed the handling of part self-symmetry and inter-part geometric similarity through specialized loss terms. The reflective symmetry loss and geometric similarity constraints have been incorporated into part assembly frameworks to address pose ambiguities arising from symmetric components. To substantiate the claimed contributions in symmetry-aware modeling, the authors should provide more rigorous comparative analysis or experimental validation demonstrating the specific advantages and innovations of their proposed symmetry-aware loss formulation compared to these existing symmetry-handling approaches. This could include detailed ablation studies comparing different symmetry loss formulations, theoretical analysis of the proposed approach's superiority, or empirical evidence showing improved performance over prior symmetry-aware methods.\n\n- **Computational and Memory Overhead** The construction of semantic super-parts via Sinkhorn’s algorithm, dual-range propagation, and cross-stage attention introduces multiple heavyweight modules and hyperparameters (e.g., number of super-parts, entropy regularization), potentially limiting scalability to larger part sets or high-resolution meshes. Additionally, in the self symmetry part, the proposed method need to compute 8 times of forward propagation, which seems quite heavy.\n\n[1] Zhang, Jiahao, et al. \"Manual-PA: Learning 3D Part Assembly from Instruction Diagrams.\" *arXiv preprint arXiv:2411.18011* (2024).\n\nQUESTIONS:\n- What is the physical meaning and intuitive interpretation of the super-part features constructed via optimal transport formulation? How do these semantic super-parts conceptually differ from simpler global feature aggregation approaches[1] or utilize graph neural network-based feature aggregation and message passing between different parts[2], what are the fundamental theoretical advantages and principled design motivations behind constructing these intermediate semantic super-parts?\n\n- While the empirical results demonstrate the effectiveness of optimal transport-based super-part construction, could the authors provide theoretical analysis or intuitive explanations for why the entropy-regularized optimal transport formulation is particularly well-suited for this hierarchical part grouping task? And why inner product can serve as a great cost function ? A deeper theoretical understanding of the underlying mechanisms and design principles would significantly strengthen the paper's contribution beyond purely empirical validation.\n\n- The proposed symmetry-aware loss enumerates all eight axis-aligned reflection transformations and selects the one with minimal pose error. However, since reflections are not elements of the SE(3) group, this exhaustive search may introduce unintended chirality issues, potentially producing assemblies with inverted handedness. Moreover, real-world part point clouds can assume arbitrary orientations and need not align with the canonical axes; it is unclear whether the fixed set of eight reflections suffices to capture all symmetry variants under arbitrary poses. Finally, this approach incurs additional computational overhead due to repeated network forward passes for each candidate reflection. Could the authors provide:\n\n  1. A theoretical discussion on how reflecting parts outside SE(3) affects the preservation of chirality and whether any mechanisms ensure consistency of handedness across the assembled shape?\n\n  2. Empirical or analytical justification that the eight axis-aligned reflections are sufficient to cover the full space of self-symmetries for arbitrarily oriented part point clouds?\n\n  3. An evaluation of the added computational cost from multiple forward passes, and potential strategies (e.g., grouping symmetric variants or embedding symmetry into the architecture) to mitigate this overhead while preserving symmetry-aware supervision?\n\n- In Eq.14, there may exists a typo, the subscript $j$ should be $g$\n\n- In Eq.14, could the author clarify why your training objective minimizes the pose differences between geometrically similar parts, rather than encouraging them to occupy distinct positions? Intuitively, one would expect interchangeable components—such as the four legs of a chair—to be driven toward different spatial locations instead of collapsing into the same pose. What is the motivation for favoring minimal pose discrepancy among symmetric parts, and have you considered an alternative formulation that explicitly promotes diversity in their placements?\n\n\n\n[1] Wu, Ruihai, et al. \"Leveraging se (3) equivariance for learning 3d geometric shape assembly.\" *Proceedings of the IEEE/CVF International Conference on Computer Vision*. 2023.\n\n[2] Scarpellini, Gianluca, et al. \"Diffassemble: A unified graph-diffusion model for 2d and 3d reassembly.\" *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*. 2024.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}