Title: Point-MaDi: Masked Autoencoding with Diffusion for Point Cloud Pre-training
Venue: NeurIPS 2025
Accepted: yes
Keywords: 3D computer vision

self-supervised learning

diffusion
Abstract: Self-supervised pre-training is essential for 3D point cloud representation learning, as annotating their irregular, topology-free structures is costly and labor-intensive. Masked autoencoders (MAEs) offer a promising framework but rely on explicit positional embeddings, such as patch center coordinates, which leak geometric information and limit data-driven structural learning. In this work, we propose Point-MaDi, a novel Point cloud Masked autoencoding Diffusion framework for pre-training that integrates a dual-diffusion pretext task into an MAE architecture to address this issue. Specifically, we introduce a center diffusion mechanism in the encoder, noising and predicting the coordinates of both visible and masked patch centers without ground-truth positional embeddings. These predicted centers are processed using a transformer with self-attention and cross-attention to capture intra- and inter-patch relationships. In the decoder, we design a conditional patch diffusion process, guided by the encoder's latent features and predicted centers to reconstruct masked patches directly from noise. This dual-diffusion design drives comprehensive global semantic and local geometric representations during pre-training, eliminating external geometric priors. Extensive experiments on ScanObjectNN, ModelNet40, ShapeNetPart, S3DIS, and ScanNet demonstrate that Point-MaDi achieves superior performance across downstream tasks, surpassing Point-MAE by 5.51\% on OBJ-BG, 5.17\% on OBJ-ONLY, and 4.34\% on PB-T50-RS for 3D object classification on the ScanObjectNN dataset.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe proposed Point-MaDi is a novel masked autoencoding diffusion framework for self-supervised 3D point cloud representation learning, which introduces a dual-diffusion mechanism to eliminate reliance on explicit positional embeddings. Its center diffusion encoder predicts patch coordinates without ground-truth geometric priors, while the conditional patch diffusion decoder reconstructs masked patches directly from noise using latent features, enabling pure data-driven learning of both global semantics and local geometry. Experiments show state-of-the-art performance and demonstrate superior generalization across multiple 3D tasks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n1. The proposed Center Diffusion avoids using real position embedding by predicting the coordinates of image blocks, effectively preventing the leakage of geometric information, thereby improving the model's ability to autonomously locate local areas.\n2. This method uses latent features to reconstruct obscured image blocks from noise, enhancing the model's understanding and generation of local structures, enabling it to more accurately restore missing content.\n3. This method abandons external geometric priors and relies only on the data itself for learning, thereby optimizing both global and local representations and improving the generalization and adaptability of the model.\n\n**Weaknesses**:\n1. Lack of model efficiency comparison. In the main experimental results table 1, please provide a comparative analysis of different methods in terms of computational complexity, and inference speed, so as to have a more comprehensive understanding of model performance.\n2. Lack of visual analysis of segmentation tasks. Lack of visual analysis on segmentation tasks, as well as visual comparative analysis with other methods.\n\nQUESTIONS:\nIn addition to the ones mentioned in weaknesses, please solve the following confusions:\n1. In the visualization in Appendix C, it seems that the reconstruction of the center point of some samples is relatively poor. Can authors provide the performance of center point prediction of different categories, such as CD distance, etc., and put it together with the accuracy of different categories to conduct a more in-depth analysis of the center diffusion process?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors propose a point cloud pre-training method that combines masked auto-encoding and diffusion. They claim that this can help to avoid geometric information leakage in point-MAE-style pre-training. They conduct extensive experiments and ablations to verify the effectiveness of the pre-training method.\n\nSTRENGTHS AND WEAKNESSES:\n#### Strengths:\n\n1. The motivation to resolve the geometric information leakage problem is crucial to the point cloud community.\n\n2. The paper is overall well-written and easy to follow.\n\n3. The experiment results convince the effectiveness of the proposed method.\n\n#### Weaknesses:\n\n1. The technical contribution of this paper is limited. It simply combines point-MAE with the diffusion model.\n\n2. Whether the geometric information leakage problem is resolved is questionable.\n\nQUESTIONS:\nQuestion 2 is the most crucial one and I'm expecting the authors' rebuttal. \n\n1. What is the implementation detail of the PointNet embedding process of visible point patches? Do you strictly avoid masked information leakage in this encoding process?\n\n2. Since the positional embedding of masked patches is used as conditional latent to the denoising decoder, I think the masked center information is also leaked in the reconstruction process. Is there any misunderstanding? From my perspective, if you want to claim that geometric information leakage is avoided, then no masked center features should be provided. The diffusion model should learn how to denoise masked regions only based on visible patches. If not, I think this paper would be overclaimed. \n\n3. Why can you use MSE loss instead of CHamfer distance on center diffusion loss? Is the center point order kept during the diffusion process?\n\n4. More recent related works should be included in the results comparison: Point-CMAE[1], PCP-MAE[2].\n\n5. Whether this method is applicable to scene-level model pre-training? For example, directly pre-training on ScanNet dataset like PiMAE[3]?\n\n#### References:\n\n[1] Bin Ren, Guofeng Mei, Danda Pani Paudel, Weijie Wang, Yawei Li, Mengyuan Liu, Rita Cucchiara, Luc Van Gool, and Nicu Sebe. Bringing masked autoencoders explicit contrastive properties for point cloud self-supervised learning. arXiv preprint arXiv:2407.05862, 2024.\n\n[2] Xiangdong Zhang, Shaofeng Zhang, and Junchi Yan. Pcp-mae: Learning to predict centers for point masked autoencoders. arXiv preprint arXiv:2408.08753, 2024.\n\n[3] Chen, Anthony, et al. Pimae: Point cloud and image interactive masked autoencoders for 3d object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposes a novel point cloud pre-training algorithm, Point-MaDi, which integrates masked autoencoding and diffusion models for representation learning. Specifically, the method introduces two diffusion mechanisms: a center diffusion mechanism and a patch diffusion mechanism, where the latter is primarily used to prevent geometric information leakage. Extensive experiments conducted on diverse datasets demonstrate the effectiveness of the proposed approach, surpassing previous state-of-the-art methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1.A simple yet effective design that facilitates engineering extensibility.\n\n2.The paper is clearly written and easy to follow.\n\nWeaknesses:\n\n1.Insufficient qualitative comparisons with existing approaches.\n\n2.Limited interpretability and ablation analysis for the individual components.\n\nQUESTIONS:\n1. In Figure 1, the comparison with contrastive learning and masked autoencoder paradigms fails to clearly highlight the unique contribution of the proposed method. To improve this, the authors are encouraged to include comparisons with more relevant baselines such as DiffPMAE and PointDiff, and clarify the distinct advantages of Point-MaDi.\n\n2. The authors should provide deeper qualitative and quantitative analysis to compare the effectiveness of using noised center point embeddings versus original (non-noised) center point embeddings.\n\n3. Figure 2 is not referenced in the main text. Additionally, since the figure contains multiple subplots, it is recommended to add (a), (b), (c) labels to improve clarity. Notations such as E and PE in Eq. (7) should also be clearly indicated within the figure for better alignment with the text.\n\n4. The reviewer suggests that Section 3 should include an overview of the entire pipeline, along with a clear explanation of how Point-MaDi is related to or differs from PointDiff.\n\n5. In Eq. (3), it appears that the center points are already perturbed with noise, but the authors do not provide sufficient explanation or corresponding notations to clarify this process.\n\n6. Time embedding is not discussed within the center diffusion process. The authors should provide more explanation and ideally support it with ablation experiments.\n\n7. Are center diffusion and patch diffusion sampled simultaneously? If they follow different sampling schedules (i.e., two different time steps t), what is the effect? The authors are encouraged to add comparative experiments to explore this question.\n\n8. It remains unclear what specific types of information are learned by the encoder and decoder through the center and patch diffusion mechanisms, respectively. Providing visualizations of the learned representations would help the reader better appreciate the contributions of this work.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper, \"Point-MaDi: Masked Autoencoding with Diffusion for Point Cloud Pre-training,\" proposes a novel self-supervised pretraining framework for point clouds called Point-MaDi. Its core innovation lies in integrating a dual-diffusion process into the Masked AutoEncoder (MAE) architecture. This design eliminates the reliance on geometric priors—such as explicit center position embeddings—found in traditional methods, while enhancing the model’s ability to learn both local geometric details and global structural representations directly from the data.\n\nSTRENGTHS AND WEAKNESSES:\nThe main contribution of this paper is the introduction of a patch position diffusion process prior to patch-level MAE modeling on point clouds. While this design sounds more reasonable, it seems somewhat similar to a form of data augmentation on patch positions. However, I am not entirely convinced by the authors’ claim that this design helps the model better understand structural information.\n\nThis is because patch positions only represent very coarse-grained information about the point cloud. How different can the predicted patch centers obtained from the center diffusion be compared to the ground-truth patch center positions? If the difference is minimal, then this might be equivalent to simply adding small perturbations to the ground-truth patch centers. On the other hand, if the predicted centers deviate significantly, they might no longer be suitable as positional embeddings (PE) for the patch diffusion decoder.\n\nEven if the model does learn certain geometric structures, such knowledge is acquired within the center diffusion process itself, not necessarily transferred downstream.\n\nQUESTIONS:\nIt seems that the center diffusion module is not utilized during downstream inference, and I couldn't find a clear description of this in the paper.\n\nI recommend that the authors explicitly clarify how the pretrained model is used for downstream tasks. If I have overlooked this detail, please kindly point out where it is mentioned in the paper.\n\nLIMITATIONS:\nI believe the theoretical motivation behind this paper's contribution is not particularly strong. Although the method may show some empirical improvements, these could be attributed to factors such as model design or the amount of training data used. Overall, I find the authors’ justification for the methodological motivation unconvincing.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-21"}