Title: Rectified Point Flow: Generic Point Cloud Pose Estimation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Point Cloud Registration

Rectified Flow

Multi-Part Assembly

Shape Assembly
Abstract: We present Rectified Point Flow, a unified parameterization that formulates pairwise point cloud registration and multi-part shape assembly as a single conditional generative problem. Given unposed point clouds, our method learns a continuous point-wise velocity field that transports noisy points toward their target positions, from which part poses are recovered. In contrast to prior work that regresses part-wise poses with ad-hoc symmetry handling, our method intrinsically learns assembly symmetries without symmetry labels. Together with an overlap-aware encoder focused on inter-part contacts, Rectified Point Flow achieves a new state-of-the-art performance on six benchmarks spanning pairwise registration and shape assembly. Notably, our unified formulation enables effective joint training on diverse datasets, facilitating the learning of shared geometric priors and consequently boosting accuracy. Our code and models are available at https://rectified-pointflow.github.io/.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper is addressing the problem of registration of point clouds. More specifically, the main focus is registration of multiple point clouds to each other resulting in an assembled object. This problem is not solving generic multi point cloud registration, but rather a specific problem, of part assembling, where the input are object parts in arbitrary poses and they can be aligned to make an object. A side problem the paper addresses, is a registration of two scans. The problem  addressed in this paper is interesting and relevant.\n\nThe paper approaches this problem using generative modelling, notably, it leverages a diffusion model. The diffusion model is basically conditioned on the unposed object parts and outputs a vector field for each point in each part, pointing towards its location in the target or assembled object. The rigid transformations are easily computed from known dense point correspondences using a SVD. For such generative models to work reasonably it is necessary to train them on a large amount of data. Authors use Objaverse dataset for initialisation, where the objects were segmented into the parts using available off the shelf methods. Later their diffusion model is trained specifically on each dataset they use in evaluation, resulting in one model per dataset. Alternatively, they trained a single model for all available datasets they use in the evaluation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: The paper proposes an interesting approach for multi part registration. I find  it clearly formulated and presented. I find using a diffusion model for addressing this problem intriguing, but the formulation is nice and appealing. Experimental evaluation is substantial and contains a number of datasets and related works the paper compares too. Finally, I find the discussion useful and appreciate showing failure cases and their causes.\n\nWeaknesses: Even though I like the approach and the formulation, I think it has several weaknesses. \n1) Questionable generalisation capablities. Primarily, generative models, like diffusion models, are still bound to the datasets they are trained on. They excel in the generative tasks, like generating new shapes, interpolated between the existing one from the training. However, for the registration, they are bound to the datasets they use for training. Unlike, classical methods, e.g. keypoint matching based, which can handle any objects in any configurations, generative models, cannot be applied to completely new objects. I believe that the same applies here. The model is bound to the datasets they are trained on, even if they are trained on all datasets used in this paper. For example, the paper, follows, train/validation/test splits. This means that part of the dataset has been seen during the training of the diffusion model. \n\n2) Self-supervised pretraining. Training the encoder and overlap head on a large dataset, as Objaverse, using cross entropy loss, where all object parts and their overlaps are known, doesn’t seem to be self-supervised, but rather supervised. Generating labels(object part IDs) and their overlaps automatically, cannot be considered as past of self-supervision. I would appreciate your comment on this.\n\n3) Related works. The discussion about the related work is valid and well structured. However, it is hard to see how different the proposed approach is from the related diffusion models:DiffAssembly, PuzzleFusion and GARF. Claiming that others handle symmetries in an ad-hoc manner, while the proposed approach does it intrinsically is not clear. \n\n4) Experimental evaluation. Even though the evaluation is exhaustive, I would like to see comparisons, where test objects have never been used for training. For example, a cross validation kind of evaluation would be interesting: the model is trained on all datasets except one, on which it is tested. Why there is no comparison to DiffAssembly, which seems to be quite related to the current approach? Symmetry handling example in Figure 6 is not sufficient. It would be great to have a cylindrical toy object, e.g. bottle, and show how multiple symmetric scans could be registered. Training on one set of cylindrical objects and testing on  completely different one would be beneficial to see if the method generalises in case of symmetries.\n\nQUESTIONS:\nThe questions below are related to weaknesses, so when addressing them, please refer to corresponding weaknesses.\n1) Generalization. The question is if the objects from the test split belong to the same classes as those used for training. What would be the results if the model was trained on one dataset and evaluated on the other.\n\n2) Self-supervision. It would be good to get more details here and explain, why the authors call pre-training approach self-supervised.\n\n3) Related work especially wrt. symmetry handling. It would be good to give an intuition about this intrinsic symmetry handling. My feeling is that the diffusion model “memorises” how to from different parts to the canonical shape, even in case of symmetries. I would be surprised if a new object with novel type of symmetry can easily be handled.\n\nLIMITATIONS:\nNot applicable", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes rectified point flow, a framework that can treat point cloud registration and multi-part shape assembly in a unified manner. The key idea is to adopt the recently popular flow matching technique into the classic point cloud pose estimation problem, which remarkably accommodates both pair-wise and multiple components settings. Overall the idea is interesting, and the experimental validation suggests sufficient advantage over the competing baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\n1. Point pose estimation task is fundamental in processing 3D data. The reported method demonstrates promising results on rather challenging setting, e.g., shown in the last two columns of Fig. 4, which is potentially of great utility in various down-stream applications. \n\n2. The proof of Thm. 1 looks reasonable to me. It provides good basis from the theoretical perspective. \n\n3. The idea of incorporating rectified flow framework into point pose estimation is interesting, and I appreciate the authors' efforts to make this seemingly intuitive idea work in practice (e.g., the conditioning design as well as the ablation in Tab.4 suggest the efforts). \n\nWeaknesses: \n\nI have some questions, please refer to the *Question* part.\n\nQUESTIONS:\n- The reported models are mainly trained on instance-level man-made datasets. Some important applications of shape assembly come from medical tasks. I am wondering 1) the current method relies on an overlap estimation module, which requires semantic segmentation. can the framework deal with more random segments such as bone fragments due to fracture? 2) following 1), if one generates some bone fragments, which are likely unseen by the models, how well the models can generate in assembling bones? 3) if one has to train from scratch, is there any insight on the minimal requirements of training set? \n\n- Another challenging scenario (also with strong medical background) is non-rigid shape assembly. Is there any insight on how the proposed framework can be extended to the more challenging setting?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Rectified Point Flow (RPF), which is a unified framework for 3D point cloud pose estimation, addressing both pairwise registration and multi-part shape assembly problems as a single conditional generative task. RPF learns a continuous point-wise velocity field to transport noisy points to their target positions, from which poses are then recovered. This approach inherently learns and handles assembly symmetries without explicit labels. Coupled with a self-supervised encoder that focuses on overlapping points, RPF achieves state-of-the-art performance across six benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n- Both pairwise registration and multi-part assembly tasks are tackled in a unified framework of per-part pose estimation with the proposed point flow based point cloud pose estimation method.\n- A explicit function of overlap prediction between parts is formulated to facilitate the learning of inter-part relationships.\n- The comparison with state-of-the-art shows performance improvement across the evaluated metrics.\n\nWeaknesses\n\n- The experimental comparison against baselines may be biased due to the usage of the pre-trained dataset.\n- time consumption. Time complexity comparison is missing. \n- others as listed in the questions.\n\nQUESTIONS:\nExperiments: \n- The trade-off between time complexity and quality. What the results will be with different iterations of conditional generation.\n- The size of the point cloud. Are the point clouds sampled to the same fixed number of point clouds? What is the value of M_i? Will it be different for parts?\n- Baseline. Do the baselines also use the pre-trained dataset used in RPF?\n- The straight-line interpolation is used for each part (line 153). Does this linear interpolation matches the flow process, which is unlikely to be linear?\n- It would be good to see the denoising intermediate results in the main text.\n- What is the effect of the overlap self-supervised pre-training? It would be good to see the results with and without the self-supervised task in the main text.\n\nLIMITATIONS:\nYes. The failure cases were discussed (Fig. 7).", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a unified formulation of point cloud registration and multi-part shape assembly as a conditional generative modeling problem. To address this, the authors propose Rectified Point Flow, a flow-based model that explicitly handles symmetry and part interchangeability—challenges commonly faced in assembly tasks. The method demonstrates strong performance on both shape assembly and registration benchmarks, outperforming prior approaches that rely on deterministic or SE(3)-based optimization.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- Originality: The paper is the first to model point cloud registration and multi-part shape assembly under a shared conditional generation framework. This is a notable departure from prior works that treat the two tasks separately. In contrast to methods like GARF, Rectified Point Flow can naturally handle ambiguous or interchangeable parts.\n- Quality & Clarity: The proposed model is thoroughly evaluated through multiple ablations and visualizations. For instance, Figure 4 illustrates how the model scales to larger assemblies, and Table 2 compares both per-task and joint training strategies, clearly showing the benefits of joint learning. Supplementary experiments (e.g., Fig. 11 on cross-category generalization) further strengthen the motivation for using generative modeling for assembly.\n- Significance: Addressing part ambiguity and interchangeability is central to robust 3D assembly. This work provides a practical solution through generative modeling in Euclidean space, potentially influencing downstream applications such as robotic assembly or 3D scene synthesis.\n\n**Weaknesses**\n- Clarity: While the paper proposes a pretraining approach using overlap prediction, it does not compare this to standard self-supervised pretraining techniques (e.g., PointBERT), leaving unclear whether the proposed strategy is necessary or optimal.\n- Clarity: Although the method is motivated by ambiguity and interchangeability, most evaluation metrics in the main paper focus on pose accuracy. Measuring the Chamfer Distance (CD) of the assembled shape would better reflect success under these challenges. While CD is reported in the supplementary (Tab. 5), it is missing from the main tables (Tabs. 2 and 4), making comparisons less complete.\n\nQUESTIONS:\n- Chamfer Distance in Main Results: Could the authors include the shape Chamfer Distance (CD) in Tables 2 and 4, alongside pose metrics? This would provide a more comprehensive evaluation aligned with the method’s stated goals.\n- Comparison with Self-supervised Pretraining: In Table 4, only no pretraining and supervised methods are compared. Please consider including results with standard self-supervised methods (e.g., PointBERT or similar baselines) to contextualize the benefit of the proposed overlap-aware pretraining.\n- Overlap Label Symmetry: The overlap supervision seems to assign binary labels to points based on spatial proximity (e.g., only the top region of chair legs is marked as overlapping in Fig. 2). Given that parts like chair legs are symmetric and can be assembled in reverse, does the current labeling overlook valid symmetrical configurations? How does the model handle such ambiguity?\n- Anchor Part Selection: Line 118 states that the anchor part is chosen as the first part in the input. Could the authors clarify how the input order is determined? If the order is based on part size or another heuristic, is the assembly result sensitive to different anchor choices?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces point-cloud registration and part assembly at the object level. The authors propose the Rectified Point Flow to achieve a generalizable point cloud pose estimation scheme and show good results across multiple benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1) The numerical results are good; the proposed method achieves better performance than other counterparts on different datasets.\n2) The idea of considering point cloud registration and multi-part shape assembly as a conditional generative problem seems to work on sythentic object-level datasets.\n\nWeaknesses:\n1) Method: The overlapping training strategy seems to work only for objects with clear and determinative parts to be assembled, which may be suitable for synthetic toy datasets but not sufficient for most in-the-wild real objects. The claimed 'generic' nature of the method should be toned down.\n2) Evaluation: The current evaluation scheme does not fully support the claimed 'generic' performance, as all evaluations are conducted on an in-domain basis. This setup only demonstrates that the Rectified Flow performs better than other baselines in terms of 'remembering' the dataset, rather than validating the proposed method's generalization capability.\n\nQUESTIONS:\n1) How efficient is the overlapping training strategy? All successful results in the paper seem to involve relatively simple parts to be assembled, e.g., the dining table, lamp, etc. However, in the failure case, when the parts become less determinative (e.g., the Wassily Chair), the method fails to assemble them.\n2) Is there any performance evaluation on out-of-domain data? I understand that the authors trained across different datasets and used a train/test split, yet there is no demonstration of how distinct the train and test sets are. Especially, there is no out-of-domain evaluation.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}