Point-MaDi: Masked Autoencoding with Diffusion for Point Cloud Pre-training
Authors: Xiaoyang Xiao
Runzhao Yao
Zhiqiang Tian
Shaoyi Du
Venue: NeurIPS 2025 poster ()
Keywords: 3D computer vision
self-supervised learning
diffusion
Primary area: deep_learning
TL;DR: A dual-diffusion pretraining framework for point clouds that predicts patch centers and masked geometry to learn robust semantic and geometric representations.

Abstract: Self-supervised pre-training is essential for 3D point cloud representation learning, as annotating their irregular, topology-free structures is costly and labor-intensive. Masked autoencoders (MAEs) offer a promising framework but rely on explicit positional embeddings, such as patch center coordinates, which leak geometric information and limit data-driven structural learning. In this work, we propose Point-MaDi, a novel Point cloud Masked autoencoding Diffusion framework for pre-training that integrates a dual-diffusion pretext task into an MAE architecture to address this issue. Specifically, we introduce a center diffusion mechanism in the encoder, noising and predicting the coordinates of both visible and masked patch centers without ground-truth positional embeddings. These predicted centers are processed using a transformer with self-attention and cross-attention to capture intra- and inter-patch relationships. In the decoder, we design a conditional patch diffusion process, guided by the encoder's latent features and predicted centers to reconstruct masked patches directly from noise. This dual-diffusion design drives comprehensive global semantic and local geometric representations during pre-training, eliminating external geometric priors. Extensive experiments on ScanObjectNN, ModelNet40, ShapeNetPart, S3DIS, and ScanNet demonstrate that Point-MaDi achieves superior performance across downstream tasks, surpassing Point-MAE by 5.51\% on OBJ-BG, 5.17\% on OBJ-ONLY, and 4.34\% on PB-T50-RS for 3D object classification on the ScanObjectNN dataset.

Point-MaDi: Masked Autoencoding with Diffusion for Point Cloud Pre-training
Xiaoyang Xiao', Runzhao Yao', Zhiqiang Tian?, Shaoyi Du State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, National Engineering Research Center for Visual Information and Applications, and Institute of Artificial Intelligence and Robotics, Xi an Jiaotong University
2 School of Software Engineering, Xi' an Jiaotong University lopeture@stu. xjtu edu, cn, rzy33208163. com zhi thiqiangtian8xy tu. edu. en, dushaoyi8x] tu. edu. cn
Abstract
Self-supervised pre- pre-training is essential for 3D point cloud representation learning, annotating their irregular, topology-free structures is costly and labor-intensive. Masked autoencoders (MAEs) offer a promising framework but rely on explicit positional embeddings, such as patch center coordinates, which Ieak geometric information and limit data-driven structural learning. In this work, we propose Point-MaDi, novel Point cloud Masked autoencoding Diffusion framework for pre-training that integrates a dual-diffusion pretext task into an MAE architecture address this issue. Specifically, we introduce center diffusion mechanism in the encoder, noising and predicting the coordinates of both visible and masked patch centers without ground-truth positional embeddings. These predicted centers are processed using transformer with self- and cross- cross-attention to capture intra- and inter-patch relationships. In the decoder, we design a conditional patch diffusion process, guided by the encoder's latent features and predicted centers reconstruct masked patches directly from noise. This dual-diffusion design drives comprehensive global semantic and local geometric representations during pre-training, eliminating external geometric priors. Extensive experiments on ScanObjectNN, ModelNet40, ShapeNetPart, S3DIS, and ScanNet demonstrate that Point-MaDi achieves superior performance across downstream tasks, surpassing Point-MAE by 5.50% OBJ-BG, 5.17% OBJ-ONLY, and 4.34% PB-T50- RS for 3D object classification on the ScanObjectNN dataset. Codes are available at https://github.com/ wwwaa//aaannaa/PiiitttWttaa
1 Introduction
Driven by advances in LiDAR and depth-sensing technologies, point clouds have become a fundamen- tal data representation in applications such as autonomous driving. robotics, and virtual reality, owing to their ability to capture fine-grained geometric details objects and environments. More recently, supervised learning has significantly advanced 3D computer vision by introducing 3D-centric meth- ods that directly operate on raw point clouds for tasks such as object classification [34]
35 [1
semantic segmentation 34 25 and object detection B3 26). These approaches typically rely large-scale annotated datasets to achieve high performance. However, unlike 2D images arranged in regular grids, point clouds lack consistent topology, making the annotation process both expensive and labor-intensive. Labeling 3D data 4 16 60
47 often requires expert knowledge to accu- rately capture complex geometrical structures, which limits the scalability and generalization ability of supervised approaches. Self-supervised learning (SSL) (2 7 54 [4] has emerged as a promising
CCorrespnniigg author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

reconstructed point
denoised point
denoised point
denoised position
Generator
Model
Model
Encoder
masked point
masked point
noised point
masked point
noised position
(a) Masked autoencoder
(b) PointDif
(c) Ours
Figure I: Comparison between different pretext tasks. (a) Masked autoencoders reconstruct masked point patches. (b) PointDif uses a conditional point generator to guide the point-to-point generation from noisy input. (c) Our Point-MaDi denoises noisy masked patches and reconstruct their centers.
alternative, enabling the extraction of generalizable representations from unlabeled point clouds through the design of various pretext tasks, including generative- and contrastive-based objectives.
28 32 have recently emerged as a powerful paradigm for 3D point cloud representation learning. owing their ability to model complex data distributions through iterative denoising processes. Unlike contrastive learning [57] 69 61 aligns views of point cloud via maximizing the similarities of positive pairs to capture global
Among them, diffusion probabilistic models (DPMs) [3
semantic consistency, or reconstruction-bart -assed methods [3 65
13
36 37 like masked
autoencoders (MAEs) [4 31, which mask patches and reconstruct geometry using an encoder- decoder architecture. DPMs generate data step-wise from Gaussian noise and learn to reverse this process, potentially capturing rich semantic and geometric structures across scales. Despite their strengths, existing diffusion-based methods mainly rely on global context aggregation or predefined conditioning mechanisms, such class labels auxiliary features, to guide the denoising process. Recent studies [9] have begun to address these challenges by integrating diffusion frameworks into MAEs; this structure naturally complements diffusion models: the encoder can operate on partially observed data, while the decoder can progressively recover masked content from latent noise, aligning well with the denoising objective of DPMs. Nonetheless, directly combining MAE and diffusion remains nontrivial, as current MAEs inject geometric priors, such as patch center embeddings, that leak explicit positional information into the encoder, hindering the objective of learning structure purely from data-driven cues 42 49 3]. This motivates our central question: How can we design a diffusion- based pre-training framework that mitigates geometric information leakage while enhancing the modeling of local geometric structures for faithful reconstruction?
Intuitively, if positional embeddings are particularly provided noised, the model is forced to infer global spatial relationships solely from the visual and contextual content of the point cloud, encouraging deeper understanding of its structure and semantics. Considering this, we propose Point-MaDi, a novel Point cloud Masked autoencoding Diffusion framework. It introduces a dual- diffusion pretext task that applies diffusion both centers and local patches, predicting centers and reconstructing patches to enforce robust modeling of comprehensive representations without external geometric cues. Fig. [
illustratess how Point-MalD contrasts with existing pretext tasks: while MAEs reconstruct masked patches using provided positional embeddings. PointDif advances this paradigm by formulating pre-training as a conditional point-to-point generation task. Our proposed Point-MaDi further instantiates this by performing center denoising in the encoder and patch reconstruction in the decoder, enabling acometry-awart supf-supe learning without positional cues.
To this end, we first group the point cloud into patches and apply random masking strategies to create visible and masked regions, in traditional MAEs. In the encoder, a center diffusion process applies noise to visible and masked patch centers and tasks the model with denoising them, eliminating reliance on ground-trui -trutthttuhhhh positional embeddings. This process, implemented via iterative sampling, forces the encoder to model global spatial relationships by inferring center positions from partial observations. Visible patches undergo self-attention to capture intra-patch relationships, while masked patches leverage pross-attention with visible patches to model inter-patch dependencies, refining the predicted centers. the decoder, a patch diffusion process reconstructs masked patches from latent noise, conditioned on the encoder's latent representations of visible patches and the predicted centers.

---

This reconstruction is optimized using Chamfer Distance, ensuring high-fidelity recovery of local structures, particularly in sparse point clouds. By integrating center diffusion for global modeling and patch diffusion for local reconstruction, Point-MaDi encourages the encoder to learn robust, context-aware representations while enabling the decoder to focus on fine-grained geometric details.
Our main contributions of this work are as follows:
We propose Point-MaDi, novel self-supervised pre-training framework for point clouds that disentangles positional encoding and geometry modeling via two dedicated diffusion processes, effectively mitigating positional shortcut leakage.
The center diffusion process adds noise to both visible and masked patch centers and predicts clean coordinates, replacing explicit positional embeddings and promoting high-level spatial understanding: the patch diffusion process, conditioned visible features and predicted centers, reconstructs noisy masked patches via tise denoising to recover local geometry.
Extensive experiments on ScanObjectNN, ModelNet40, ShapeNet, S3DIS, and ScanNet demonstrate that Point-MaDi significantly outperforms existing methods in classification, segmentation, and detection tasks, validating its effectiveness and generalizability.
2 Related Work
Self-supervised point cloud representation learning. Self-supervised Learning (SSL) has achieved remarkable success in many fields such as NLP and computer vision. It aims to learn useful representations from the massive unlabeled data by defining a pre-text task 29 17 image/patch operations, with no external labels. Recent methods have explored the potential and strengths of SSL. in the point cloud domain. prominent approach in SSL (57 69 is contrastive learning, which encourages representations of augmented versions of an instance to be more similar compared to different inputs. PointContrast
through
[57 pioneers contrastive learning by performing point- level invariant mapping learning. Several recent works integrate point cloud representations with other modalities, such as vision 1 20 46 61 and language [66
23 to facilitate the learning of transferable 3D point cloud representations. CrossPoint [1] combines intra-modal and cross-modal contrastive learning, enforcing invariance to point cloud augmentations and aligning 2D image features with point cloud prototypes. More recently, masked prediction [14] has re-attracted attention trying to recover the original input from a masked version since the introduction of the Vision Transformer (VIT) [10. Point-Bert [62] extends the BERT-style 77 pre-training strategy point cloud transformers via discrete Variational Autoencoder 33). Point-MAE (31] brings the masked autoencoder idea to point cloud tasks by randomly masking input point patches and reconstructing them. Afterward, PointM2AE (65) and 12P-MAE 667] adopt hierarchical MAE transformers to extract fine- grained and higher-level semantic features of 3D shapes. Subsequent works 9 mainly enrich the model's comprehension of point cloud geometric structures using cross-modal knowledge. For instance, ACT (9] utilized pre-trained ViT as teacher model to acquire knowledge from other modalities. MAin [13] proposes a 2D-3D joint MAE framework to reconstruct the masked two modalities. ReCon++ (37] is similar ReCon (36). both employing a generative framework (MAE-based) and incorporating contrastive learning through ensemble distillation. Our Point-MaDi leverages masked autoencoding with diffusion, and by progressively denoising noised positional centers, it reduces reliance on positional shortcuts while enhancing geometric awareness.
63 59
Diffusion probabilistic models. Diffusion probabilistic models [15 R] also known as score-based models 43 44 have gained significant attention in computer vision for their ability to generate high-fidelity images. DPMs begin by using an evolving Stochastic Differential Equation (SDE) to gradually add Gaussian noise to real data, turning complex data into a Gaussian distribution. Then, a time-reversed SDE maps Gaussian noise back into high-quality samples, guided by network using the score function [45] with multiple steps. It has demonstrated superior performance in various generative fields, including image generation (28) 40 32 7T I8 64], video generation 41 [2].
and speech generation 21 However, when adopted in the 3D domain, number of works focus on 3D generation [24
55 19. Applying DPMs to 3D point cloud pre-training remains underexplored due to the challenges posed by the irregular sampling patterns of point clouds in 3D space. (24 proposed an autoencoder architecture with a DPM as decoder. DifiPMAE [19] introduces a two-stage architecture that denoises the masked regions conditioned on the latent representation from the first stage. The most related work to ours PointDif (70, which aggregates latent features via condition aggregation network to guide iterative denoising of noisy point
58 27
30
3

---

Enwoder
Encoder
Encoder
Encoder
Layer
Diffusion
FC Layer
Decoder
Decoder
Decoder
Decender
Chamfer
Distinne
(b) Patch Diffusion Process
Pox Eimbol Layer
Shared Layer Norm
Black
Trandormer
Encoder
Block Log
Transfornes
SelfAklt
Choss-Ali
Shaned
(c) Encoder Module
(d) Transformer Block
Figure 2: The pipeline of our Point-MaDi framework. The encoder adopts a center diffusion process, where noise is added to the centers of both visible and masked patches. A multilayer perceptron (MLP) maps the noisy centers to positional embeddings, and both these and visible patch embeddings input transformer to predict the clean centers. The decoder performs patch diffusion process, conditioned on the visible tokens and the predicted centers, to progressively denoise and reconstruct the masked patches. The chamfer distance guides the patch reconstruction, while the stop gradient (SG) operation prevents leakage of ground-truth masked patch positions during pre-training.
clouds for pre-training. However, Point-MaDi differs in several key aspects. First, unlike PointDif's holistic point-to-point denoising, we introduce a joint diffusion process that simultancously models patch centers and coordinates, capturing both structural and local geometrie priors for enhanced robustness. Second, instead of relying on complex modules like PointDif's separate encoder, aggregator, and diffusion model, we unify the point encoding and decoding process with bi-jective network design, improving efficiency and reconstruction fidelity.
3
Point-MaDi
3.1 Overview of Point-MaDi
The proposed framework contains two modules, each driven by distinct diffusion models for specific tasks. Fig. jillustrates the end-to-end diffusion process of Point-MaDi. For given clean point cloud, the encoder module partitions the input into patches and trains diffusion model that iteratively adds noise to the visible and masked patch centers. Then denoising process predicts clean centers for all patches. The decoder module, conditioned on the encoder's 's visible tokens and predicted patch centers, aims to reconstruct the noisy masked patches supervised by Chamfer Distance. This dual-diffusion design enables Point-MaDi to learn both sparse structural and dense geometric representations.
3.2 Point cloud processing
31 we divide point clouds into overlap- ping point patches via Farthest Point Sampling (FPS) and K Neearest Neighbors (KNN) algorithm. Formally, given the input point cloud E R with three-dimensional (x. y, coordinates) num- ber of points, we first choose the centers C' E Rex3 for g number of groups through FPS. For centers C, we adopt the KNN to select & nearest points and obtain local geometric groups P RR9xXx3
Point patches generation. Following previous works 62
C FPS(X),., P KNN(X, C),
(I)
To eliminate global location bias, each patch P is normalized by subtracting its corresponding center coordinates. The resulting centered patches treated as sub-clouds and served sequence of localized point representations treated like words in NLP or image patches in vision.
Patch masking. For the point patches, we select predefined mask ratio m (0, 1) and deploy random masking, outputting the visible point patches Pa x and masked point patches

---

pm E R where r [g m is the number of masked patches, g is the number of visible patches, and |:] denotes the floor function. The corresponding centers are denoted as C?" e (3 and C E IR xx respectively. We also obtain a binary indicator of whether the patch is masked. Embedding. The visible point patches P are embedded through a simplified PointNet which employs 1xl convolutions and max pooling to generate the visible tokens E
E Point.eet(P"). E E g(s-r)xd
(2)
where d is the hidden dimension of the network. To incorporate spatial information, we compute positional embeddings for both visible and masked patches by applying shared- -eight MLP with GELU activation to their respective center points:
PE = MLP(C"), PE = ML
(3)
The position embeddings PE and PE" added every transformer block. For the encoder, these positional embeddings are fixed, derived directly from the input patch centers to provide stable spatial cues during the masking and diffusion process. In contrast, the decoder leverages predicted center positions estimated from the encoder's 's output. Note that we use the same MLP for the encoder and decoder in autoencoder to compute positional embeddings.
3.3 Center diffusion process
After obtaining visible and masked point patches E", FT We adopt a 12-layer standard transformer encoder the diffusion backbone, with each block containing multi-head self-attention (MSA) layer and feed-forward network (FFN). The encoder applies self-attention to E to extract conditioning features that guide the diffusion dynamics, and then performs cross-attention using ETd the query to generate masked token representations conditioned on visible context.
Forward diffusion process. To model geometric corruption, we apply forward diffusion process to the centers of visible and masked point patches. At each of the time steps, Gaussian noise is incrementally added to C' and C following a Markov chain:
q( CT-) = NCC
3,0%-1.
where l. E 11, is the time step of center diffusion process and 3, is the variance of the noise at step lc. which controls the amount of noise added at each step. Since all transition kernels of the diffusion process are Gaussian, samples from the intermediate distributions can be directly formulated single step by applying the reparameterization trick:
C( Ci) N(C, VārCa (1 -.)
(5)
with a₁ Br and ā. IIT ag Similarly, the forward process for masked centers can be sampled directly:
(6) We linear variance schedule with 3r increasing from 0.0001 0.02 over 2000 steps. As time goes by, the centers gradually diffuse into a chaotic set of points. To ensure stability and consistency, we use the same variance schedule for both visible and masked centers.
q( | = Var (1 a..
Conditional reverse process. Providing strong conditioning information e is usually helpful to reduce the number of inference steps and improve the generation quality. For visible centers and corresponding tokens, we imput the latent space E" from the token layer and the corresponding noised position embedding PEY to generate the conditional latent visible patches.
T = Encoder(E" (E", T = Encoder(E"", EE,"""))
where T serves the conditioning feature for visible centers, guiding the reverse transition, while Tm serves as the conditioning feature for masked centers, integrating visible and masked patch information to guide the reverse diffusion process. It performed via self-attention and cross- attention within each transformer block, which includes both MSA and FFN layers. The self-attention and cross-attention for a single block can be formulated as:
(8) where Q", K V are the Query, Key, Value of the visible features, and Qm,, km++e Vm+v are the masked query, concatenated key, and value, respectively. These attention outputs (Z", " are
z SelfAttn(Q". K" VV). TTh CrossAAtnnQ",,, "aassAttt"",KKttt vm+t),
5

---

passed through the FFN and normalization layers with shared parameters to produce the final encoder outputs T and T The self-attention and cross-attention share the parameters of transformer encoder blocks to avoid increasing parameters.
Unlike conventional diffusion models predicting additive noise, our reverse process aims recover the visible and masked centers C' and Cn" by gradually removing the noise under the condition of both the visible and masked representations. However, it is non-trivial approximate q( C²-1 C., IT) without knowing the entire diffusion process. Therefore, we train the Encoder module with trans- formers to learn po C²-1 reverse diffusion process. The reverse transition for the visible centers can be formulated as:
Cr IT') for approximating the conditional probabilities to infer the entire
( V - ā=-1) Or + V0t-18t Cτ) + tor. c~
CT-1
(9)
1 - āt
1 1 at
Here, x E {u, m² indicates visible or masked centers. Cg = f-) denotes the predicted clean center, inferred by a saaared-arramterr encoder. For the visible case (x = v), the prediction is conditioned on CY, E", PEY, and t.; for the masked case (I = m). additional inputs En and PET" are included.
3.4 Patch diffusion process
Similar to the encoder's s center diffusion process, the decoder in Point-MaDi leverages diffusion- based approach to pre-train robust point cloud representations by denoising masked patches. However, it differs in its target and conditioning mechanism. While the encoder diffuses both visible and masked group centers (C", C employing self-attention for visible centers and cross-attention for masked ones, the decoder exclusively denoises masked point cloud patches pm In the forward diffusion process, Gaussian noise is added to masked patches P over T time steps, formulated as (P P = (Pm V 0r Pm (1 T,) where ã₁ IT Og following the same lin- ear variance schedule as in Sec. 3.3 The reverse process approximates the denoising transition Po P. P. T,", :)) = W(P!.. no (P... T, f), of I). with the transformer decoder predicting clean patches. To this end, we train a diffusion model as the decoder, which conditions on the encoder's visible encoded features T concatenated with learnable mask tokens X", utilizing a transformer- based architecture with self-attention to process these integrated inputs. This design, coupled with specialized modules for token generation and reconstruction, enables the recovery of dense local geometry, complementing the encoder's ' sparse center denoising, with key components as follows.
Mask token layer. The mask token layer maps noisy masked patches PT from the diffusion process to a latent representation X," E R ensuring a constant number of output samples and aligning the masked and visible patches. It consists of 1D convolutional layer processing input patches of shape r x k x 3. The model processes a point cloud with 1024 input points with a mask ratio m 0.6, resulting in |1024- 0.6] masked points. The output from the mask token layer is size x d.
Time embedding. To provide unique embedding for each time step in the diffusion sequence, allowing the decoder transformer to learn the temporal relation and handle the time sequence. We construct a 384-dimensional frequency embedding TE, followed by a two-layer MLP with dimensionality equal to the transformer's 's hidden size and SiLU activations.
Transformer decoder. The structure of the decoder contains fewer transformer blocks compared to the encoder. Empirically, the depth of the network affects its diffusion performance. Hence, we explore fine-tuning with a different number of layers. Unlike existing SSL methods such as Point-MAE (3I) and ReCon (36), which rely on ground-trut -ruth positional embeddings for both visible and masked patches alongside visible tokens (C", Cmm) and learnable mask tokens X"m, our decoder processes a concatenated input of encoded visible tokens and noisy mask tokens, paired with positional embeddings that combine ground -ttth positions for visible patches P and predicted positions pmm. for masked patches. This can be expressed as:
HThh Decoder(T",, X"" PE", SG(PEm..rrdd TE),
(10)
where Hm Rd denotes the decoder's output, SG is the ttop-gradiem operation. By using ground-truth PE", we leverage accurate spatial context for visible patches, while P prr. derived from the encoder's predicted centers Cm via stop-gradient, prevents leakage of ground-trutt masked patch positions and encourages the decoder reliance on the encoder's 's learned representations. The stop-gradient further ensures that decoder gradients do not disrupt the encoder's 's center diffusion task, preserving the encoder's 's robust feature representations, Subsequently, H is passed through

---

MLP layer for masked coordinate reconstruction, producing pm R This hybrid approach enhances the robustness and generalization of patch reconstruction, complementing the encoder's sparse center denoising objective.
3.5 Training objective
Center diffusion loss. The encoder predicts clean group centers from noisy counterparts by leveraging latent features of visible and masked tokens in the forward diffusion process. It predicts visible group centers by feeding their latent features Z" into dedicated MLP. yielding C MLP,, Similarly, predicts masked group centers by processing zn through a separate MLP, giving CT" (Z for visible and masked tokens, respectively. These two MLP networks do not share parameters, ensuring that each group (visible and masked) is processed independently. The loss function for this task is computed as follows:
Ceenter
Σ c
c
- + 5=1
1 -
(11)
9
where Ö2, C. and CY, C) are the predicted and ground-truth visible/masked centers, respectively.
Patch diffusion loss. Unlike the encoder, which predicts the center positions using MSE, the decoder reconstructs masked point cloud patches from noisy inputs. The L2 Chamfer Distance is adopted as the decoder loss function.
Lyatch
min |Pp pll pe
min ||p 211
(12)
|rmnl
Pm
pé pm Pe
de pm
Final loss. Therefore, we have final loss with a weighting factor:
L= >Loenter
Epatch:
(13)
where y adjusts the relative importance of the center denoising task. We set it to 0.1 by default. Intuitively, the training process encourages the encoder to leam geometric features from the corrupted inputs and encourages the decoder to reconstruct the original point cloud. The diffusion process introduces structured perturbations geometric details and global context, thus enhancing its capacity beyond the original Point-MAE. After pre-training, abandon the decoder and only keep the encoder for downstream tasks.
the data and promotes the encoder capture both local
4
Experiments
4.1
Downstream tasks
Linear evaluation for real-world classification. We first fine-tune the proposed method on real- world scenes for 3D object classification. Rotation is applied for data augmentation during fine-tuning. We take the overall accuracy (OA) on ScanObjectNN [4]] subsets as the evaluation metric and summarize experiment results as in Tab. [ Our Point-MaDi achieves superior performance subsets, reaching 95.52%, 93. 33.46, and 89.52% accuracies, respectively. Compared to the previous Point-MAE [31,, diffusion-basse based Point-MaDi yields consistent improvements of 5.50%, 5.17%, and 4.34% on OBJ-BG, OBJ-ONLY, and PB-T50-RS, respectively. Furthermore, the performance is competitive with recent modsl methods (e.g., ReCon (36), I2P-MAE (67), without requiring additional modalities or complex pre-training pipelines.
all
Linear evaluation for synthetic classification. We also conduct experiments on the classification of the synthetic ModelNet40 dataset (56). Standard random scaling and translation are applied for data augmentation during training. While ceatusiionbbseee methods like PointDif may not consistently dominate on the relatively clean and less diverse ModelNet40 dataset, our Point-MaDi still achieves 93.8% accuracy, demonstrating strong generalization without relying additional modalities or elaborate architectures.
Part semantic segmentation. We conduct part segmentation on the ShapeNetPart [60] dataset. Following previous research, we randomly sample 2,048 points from each input instance and adopt the same segmentation head for the fair comparison, which concatenates the global features with each local feature from the 4th, 8th, and 12th layers of the transformer block, and a shared MI.P predicts part label for each point. Both category mloU and instance mloU are computed and presented in
7

---

Table 1: Classification accuracy (%) on three variants of ScanObjectNN and ModelNet4O, Parameters of inference models #P (M) are listed. We report ScanObjectNN results without voting. ModeINNe40 results are shown without and with voting. Not all papers report Acc, indicated as in our table.
ScanObjectNN
ModelNet40
Method
Reference
#P
OBJ-BG
OBJ-ONLY
PB-T50-RS
w/o Vote
w/ Vote
Supervised Learning Only
PointNet
CVPR 2017
3.5
73.3
79.2
68.0
89.2
PointNel++
NeurIPS 2017
1.5
82.3
84.3
77.9
90.7
DGCNN 51
TOG 2019
1.8
82.8
86.2
78.1
92.9
SimpleView
11
ICML 2021
80.510.3
93.9
PointMLP 25
ICLR 2022
12.6
85,4=0.3
94.5
P2P-IHorNet
32
NeurIPS 2022
195.8
89.3
94.0
with Single-Modal Self-supervised Learning
Point-BERT 62
CVPR 2022
22.1
87.43
88, 12
83.07
92.7
93.2
MaskPoint
22
ECCV 2022
89.30
88.10
84,30
93.8
Point-MAE
BCCV 2022
22.1
90.02
88.29
85.18
93.2
93.8
Point-M2AF 665
NeurIPS 2022
15.3
91,22
88.81
86.43
93.4
94.0
PointGPT S
NeurIPS 2022
19.5
91.60
90.00
86.90
94.0
Point-CMAE 58
ACCV 2024
22.1
93.46
91.05
88.75
93.6
PCP-MAE 688
NeurlPS 2024
22.1
95.52
94.32
90.35
94.0
94.2
PointDif 70
CVPR 2024
22.3
91.91
93.29
87.61
Point-MaDi (Ours)
22.1
95.52
93.46
89.52
93.8
94.1
Improve (over Point-MAE)
+5.50
+5.17
+4.34
+0.6
+0.3
with Cross-Modal Self-Supervised Learning
ACT 9
ICLR 2023
22.1
93.29
91.91
88.21
93.2
93.7
Joint-MAE 13
UCAI 2023
90,94
88.86
86.07
94.0
12P-MAR 67
CVPR 2023
15.3
94,14
91.57
90.11
93.7
94.1
ReCon 36
ICML 2023
43.6
95.18
93.63
90.63
94.1
94.5
Table 2: Part segmentation ShapeNetPart and semantic segmentation S3DIS Area 5. The mean intersection over union (mloU) for all classes (CIs.) and for all instances (Inst.) are reported for Part Segmentation. Mean accuracy (mAcc) and mloU reported for Semantic Segmentation.
Part Seg.
Semantic Seg.
Method
Reference
Cls. mloU
Inst. mloU
mAce
mloU
Supervised Learning Only
PointNet 64
CVPR 2017
80.4
83.7
49.0
41.1
DGCNN 51
TOG 2019
82.3
85.2
PointMLP 25
ICLR 2022
Self-SSperrised Representation Learning
Transformer 48
NeurIPS 2017
83.4
84.7
68.6
60.0
PoiT 62
CVPR 2022
84.1
85.6
MaskPoint 22
ECCV 2022
84.4
86.0
70.1
61.0
Point-MAE
ECCV 2022
84.2
69.9
60.8
PointGPT 5
NeurlPS 2022
84.1
86.2
PoinDDff 70
CVPR 2024
84.4
85.8
69.5
60.2
Point-MaDi (Ours)
84.8
86.3
71.0
61.2
Improve (over Point-MAE)
+0.6
+0.2
+1.1
40.4
Tab. Our Point-MaDi achieves state-of-the-an performance, with a category mloU 84.8% and an instance mloU of 86.3%, improving over Point-MAE by 0.6% and 0.2%, respectively.
3D scene segmentation. We validate our model on the indoor S3DIS dataset to demonstrate the ability of the models to comprehend contextual semantics and intricate local geometric relationships.
Tab. 2 demonstrates the performance of our proposed method. Our Point-MaDi achieves superior performance Area 5, with mAce of 71.0% and mloU of 61 2%, improving over Point-MAF by 0.2% and 0.4%, respectively. These results underscore the effectiveness of Point-MaDi's dual- diffusion pre- pre-training in capturing complex scene semantics and fine- fine-grained geometric details.

---

Table 3: Object detection results on ScanNet. We report average precision (9)). "Pre Dataset" refers
the pre-training dataset. ScanNet-Medium is subset of ScanNet.
Method
Reference
[P]
Pre Dataset
AP50
VoteNet 133
ICCV 2019
x
33.5
STRL [6
ICCV 2021
ScanNet
38,4
PointContrast
57
ECCV 2020
ScanNet
38.0
3DETR 20
ICCV 2021
X
37.9
Point-BERT 62
CVPR 2022
ScanNet-Medium
38.3
Mask-Point
22
ECCV 2022
Medinu
42.1
Point-Man
31
ECCV 2022
ShapeNet
42.8
TAP 53
ICCV 2023
ShapeNet
41.4
PiintDif
70
CVPR 2024
ShapeNet
43.7
Point-MaDi (Ours)
ShapeNet
43.7
Improve (over Point-MAE)
+0.9
Table Classification accuracy (%) of decoder architectures on ScanObjectNN variants. The configurations differ in how attention applied between visible and masked tokens.
Table 5: Classification accuracy (%) of masking strategies on ScanObjectNN variants. "Random" is random masking, "Block" is block masking, "Rand & Block" combines both.
Configuration
Masking Stratogy
OBJ-BG
OBJ-ONLY
PB-T30-RS
Joint decoder
95.52
93.46
89.52
Rand
93.98
92.77
88.45
Cross decoder
94.66
92.60
88,69
Block
91.57
Cross-self
Rand & Block
95.52
93.46
89.52
3D object detection. To further demonstrate the scene understanding ability of the proposed method, fine-tune our Point-MaDi the more challenging indoor dataset ScanNetV2 [6]. Following MaskPoint, we utilize 3DETR [26] as the baseline and replace the encoder with our Point-MaDi backbone. For evaluation purposes, with 0.5 thresholds for loU. Tab. presents the results. Our method, along with Point-MAE and TAP, is pre-trained Our Point-MaDi, pretrained on ShapeNet, achieves a state-of-the- AP50 of 43.7%, improving over Point-MAE by 0.9%. Despite the domain gap Point-MaDi outperforms methods pretrained on ScanNet-Medium, such as MaskPoint (42. 1) and Point-BERT (38.3%), highlighting the robustness of its dual-diffusion pre- pre-traing teeettaiiing in capturing complex scene semantics and geometric structures.
measure the Average Precision (AP) of 3D bounding boxes
NetpeNet in a different domain compared the ScanNet-Mediun dataset.
4.2 Ablation studies
Decoder architecture. We discuss the effect of different decoder designs, exploring three configura- tions that vary in how attention modules are applied to visible latent tokens T and the noise tokens X" The joint decoder applies transformer blocks on the concatenated sequence of the visible latent and the noise tokens, enabling self-attention across all tokens to capture global interactions. The Cross decoder takes 7" queries and X keys and values in cross-attention. mapping noise tokens to reconstructed patches within visible context. The cross-self decoder combines cross-attention where visible tokens T" serve context, with self-attention tokens to interact before querying visible tokens. We conduct experiments with the same depth to investigate the quality of predicted representations. As shown in Tab. 4 the joint decoder achieves the best overall performance. We make the joint decoder the default option for pre-training.
noise tokens X allowing noise
Masking strategy. We assess the impact of different masking strategies on the classification tasks. We trained our model with a 60% mask ratio on three masking settings: Rand: randomly seleeting masked and visible parts; Block: masking a large block that contains multiple continuous and consecutive patches; and Rand & Block: which denotes that we feed both masked inputs sequentially through the same network and employ shared weight prediction head, ensuring additional parameters are introduced during training. As illustrated in Tab, 5 the Rand & Block strategy achieves the best performance under the which encourages the model to learn more robust and generalized representations.
masking ratio. It introduces more spatial diversity corrupted regions,
Effective of component. We conduct a comprehensive ablation study focusing the components of our dual-diffusion framework Tab. 6 To ensure a fair comparison, we maintained the core Point-MaDi framework. The baseline uses clean patch centers for both visible and masked patches in

---

Table 6: Classification accuracy (%) of different component configurations on three variants of ScanObjectNN.
Center (Vis) Center (Mask)
Patch
Time Embedding
OBJ-BG
OBJ-ONLY
PB-T50-RS
93.97
92.60
88.83
93.63
92.43
88.13
94.32
92.08
88.79
94.32
92.94
89.17
94.66
93.11
89.17
94.49
92.43
88.83
95.52
93.46
89.52
the encoder and employs learnable mask tokens in the decoder. The key variable is the activation deactivation of the diffusion processes. The analysis reveals several key points: firstly, the baseline achieves a performance of 93.97%. 92.60%, 88. 83.3% on OBJ-BG, OBJ-ONLY, PB-T50-RS. When only one set of centers is noised while the other remains clean, the performance compared to the baseline either slightly degraded shows mixed results, suggesting that providing clean positional information for only subset of patches creates an inconsistent learning signal for the encoder; secondly, when both visible and masked centers are noised in the encoder, performance improves to 94.32%, 92.94%, 89.17%. demonstrating that our full center diffusion mechanism, which removes all ground-truth positional shortcuts and forces the encoder to predict all clean centers from noisy inputs, is more effective than partial noising or relying amplified when combined with the patch diffusion task in the decoder: the configuration with full center diffusson and patch diffusion enabled achieves the best overall performance, outperforming the scenario where center diffusion disabled but patch diffusion active. Additionally, the analysis highlights the positive impact of incorporating time embeddings for patch diffusion processes.
clean centers. Furthermore, this benefit is
Conclusions
In this work, we present novel self-supervised pre-training framework for point cloud analysis that integrates dual-diffusion paradigm into a masked autoencoding architecture. By jointly modeling the denoising of both patch centers and masked patches, our method mitigates the risk of geometric information leakage and encourages the learning of more robust semantic and geometric representations. The proposed framework leverages a center diffusion module in the encoder to eliminate reliance on positional embeddings. while conditional patch diffusion module in the decoder facilitates fine-grained reconstruction guided by visible context. Through extensive experiments across multiple downstream tasks, our approach consistently demonstrates superior performance and generalization. These results validate our initial motivation to force the model to infer global spatial relationships, enhancing its ability capture comprehensive 3D structural understanding.
Acknowledgements
This work was supported by the National Natural Science Foundation of China under Grant Nos. U24A20252, and 62327808, the Key Research and Development Program of Shaanxi Province of China under Grant Nos. 2024PT-ZCK 66 and 2024CY2-GJHX-49 Guangdong Major Project of Basic and Applied Basic Research under Grant No. 2023B0303000009, and the Fundamental Research Funds for the Central Universities under Grant No. xzy022024007.
References
[1] M. Afham, I. Dissanayake, D. Dissanayake, A. Dharmasiri, K. Thilakarathna, and R. Rodrigo. Crosspoint: Self-supervised cross-modal contrastive learning for 3d point cloud understanding. In Proceedings of the IEEE/CVF conference on compuler vision and pattern recognition, pages 9902-9912, 2022.
[2] I. Armeni, O. Sener, A. R. Zamir, H. Jiang, I. Brilakis, M. Fischer, and S. Savarese. 3d semantic parsing of large-scale indoor spaces. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1534-1543,
2016. 2016.
10

---

[3] M. Caron, N. Houlsby, and C. Schmid. Location-avee aware self-supervised transformers for semantic segmentation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 11 17-1277 2024.
[4] A. X. Chang. T. Funkhouser, L. Guibas, P. Hanrahan. Q. Huang, Z. Li, S. Savarese, M. Savva,
S. Song, H. Su, al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:
2015.
[5] G. Chen, M. Wang, Y. Yang, K. Yu, L. Yuan, and Y. Yue. Pointgpt: Auto-regressively generative pre-training from point clouds. Advances in Neural Information Processing Systems, 36:29667- 29679, 2023.
[6] A. Dai, A. X. Chang, M. Savva, M. Halber, T. Funkhouser, and M. Nießner. Scannet: annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference computer vision and pattern recognition, pages
Richly-
2017.
[7] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Fre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computationnl linguistics: human language technolo- gies, volume I (long and short papers), pages 4171-4186, 2019.
[8] P. Dhariwal and A. Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
[9] R. Dong, Qi, L. Zhang, Zhang, Sun, Z. Ge, Yi, and K. Ma. Autoencoders as cross- modal teachers: Can pretrained 2d image transformers help 3d representation learning? arXiv preprint ar Xiv: 2. 083320, 2022.
[10] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 2020.
[1]] A. Goyal, H. Law, B. Liu, A. Newell, and J. Deng. Revisiting point cloud shape classification with a simple and effective baseline. In International conference on machine learning, pages 3809-3820. PMLR, 2021.
[12] Y. Guo, C. Yang. A. Rao, M. Agrawala, D. Lin, and B. Dai. Sparsectrl: Adding sparse controls to text-to-video diffusion models. In European Conference on Computer Vision, pages 330-348. Springer, 2024.
[13] Z. Guo, R. Zhang, L. Qiu, X. Li, and P.-A. Heng. Joint-mae: 2d-3d joint masked autoencoders for 3d point cloud pre-training. arXiv preprint wVXiv22302.14003 2023.
[14] K. He, Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000-16009, 2022.
[15] Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6855 2021,
[16] S. Huang, Y. Xie, S.-C. Zhu, and Y. Zhu. Spatio-temporal self-supervised representation learning for 3d point clouds. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6535-6545, 2021.
[17] G. Larsson, M. Maire, and G. Shakhnarovich. Learning representations for automatic col- orization. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 1 11-14, 2016, Proceedings, Part IV 14, pages 577-593. Springer, 2016.
[18] Y. Li, H. Liu, Q. Wu, F. Mu, J. Yang, J. Gao, C. Li, and Y. J. Lee. Gligen: Open-set grounded soxt-to-image generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22511 1-22221, 2023.
[19] Y. Li, C. Madarasingha, and K. Thilakarathna. Diffpmae: Diffusion masked autoencoders for point cloud reconstruction. In European Conference Computer Vision, pages 362-380. Springer, 2024.
[20] Z. Li, Z. Chen, A. Li, L. Fang, Q. Jiang, X. Liu, J. Jiang, B. Zhou, and H. Zhao. Simipu: Simple 2d image and 3d point cloud unsupervised pre-training for spatial-aware visual representations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 1500-1508.
2022.
11

---

[21] Z. Lin, Y. Gong, Y. Shen, T. Wu. Z. Fan, C. Lin, N. Duan, and W. Chen. Text generation with diffusion language models: A pre-training approach with continuous paragraph denoise. In International Conference on Machine Learning, pages 21051-21064. PMLR, 2023.
[22] H. Liu, M. Cai, and Y. J. Lee. Masked discrimination for self-supervised learning on point clouds. In European Conference on Computer Vision, pages 657-675. Springer, 2022.
[23] Y. Liu, C. Chen, Z. Wang, and L. Yi. Crossvideo: Self-supervised cross-modal contrastive learning for point cloud video understanding. In 2024 IEEE International Conference on Robotics and Automation (ICRA). pages 12436-12442. IEEE, 2024.
[24] S. Luo and W. Hu. Diffusion probabilistic models for 3d point cloud generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2837-2845, 2021.
[25] X. Ma, C. Qin, H. You, H. Ran, and Y. Fu. Rethinking network design and local geometry in point cloud: A simple residual mlp framework. arXiv preprint WrXiv:220..07123 2022.
[26] I. Misra, R. Girdhar, and A. Joulin. An end-to-end transformer model for 3d object detection. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2917, 2021.
[27] S. Mo, E. Xie, R. Chu, L. Hong, M. Niessner, and Z. Li. Dit-3d: Exploring plain diffusion transformers for 3d shape generation. Advances in neural information processing systems, 36:67960-67977 2023.
[28] A. Q. Nichol and Dhariwal. Improved denoising diffusion probabilistic models. In Interna- tional conference on machine learning, pages 1162-8171. PMLR, 2021.
[29] M. Noroozi and P. Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. European conference on computer vision, pages 69-84. Springer, 2016.
[30] L. Nunes, R. Marcuzzi, B. Mersch, Behley, and C. Stachniss. Scaling diffusion models to real-world 3d lidar scene completion. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 17770-1470.) 2024.
[31] Y. Pang, W. Wang. F. E. Tay, W. Liu, Y. Tian, and L. Yuan. Masked autoencoders for point cloud self-supervised learning. In European conference on computer vision, pages 604 6211.22. Springer, 2022.
[32] W. Peebles and S. Xic. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages
2023.
[33] C. R. Qi, O. Litany, K. He, and L. J. Guibas. Deep hough voting for 3d object detection in point clouds. In proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9277-9286, 2019.
[34] C. R. Qi, H. Su, K. Mo, and L. J. Guibas. Pointnet: classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652-660, 2017.
Deep learning on point sets for 3d
[35] C. R. Qi, L. Yi, H. Su, and L. J. Guibas. Pointnet++: Deep hierarchical feature learning on point sets in metric space. Advances neural information processing systems, 30, 2017.
[36] Z. Qi, R. Dong, G. Fan, Z. Ge, X. Zhang, K. Ma, and L. Yi. Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining. In International Conference on Machine Learning, pages 28223-28243. PMLR, 2023.
[37] Z. Qi, R. Dong, S. Zhang, H. Geng, C. Han, Z. Ge, L. Yi, and K. Ma. Shapellm: Universal 3d object understanding for embodied interaction. In European Conference on Computer Vision, pages 214-238. Springer, 2024.
[38] B. Ren, G. Mei, D. P. Paudel, W. Wang. Y. Li, M. Liu, R. Cucchiara, L. Van Gool, and N. Sebe. Bringing masked autoencoders explicit contrastive properties for point cloud sulf-super learning. In Proceedings of the Asian Conference Computer Vision, pages
2024. [39] J. Rolfe. Discrete variational autoencoders, arXiv preprint 1rXiv:1609.02200 2016.
[40] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
12

---

[41] L. Ruan, Y. Ma, H. Yang, H. He, B. J. Fu, N. J. Yuan, Q. Jin, and B. Guo. Mm-diffusion: Learning multi-modal diffusion models for joint audio and video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10219-10228, 2023.
[42] S. Sameni, S. Jenni, and P. Favaro. Representation learning by detecting incorrect location embeddings. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 7704-9715 2023.
[43] Y. Song and S. Ermon. Generative modeling by estimating gradients of the data distribution. Advances neural information processing systems, 32, 2019.
[44] Y. Song and S. Ermon. Improved techniques for training score-based generative models. Advances in neural information processing systems, 33:12438-1223
2020.
[45] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 2020.
[46] B. Tran, B.-S. Hua, A. T. Tran, and M. Hoai. Self-supervised learning with multi-view rendering for 3d point cloud analysis. In Proceedings of the Asian Conference on Computer Vision, pages 3086-3103, 2022.
[47] M. A. Uy Q.-. Pham, B.-S. Hua, T. Nguyen, and S.-K. Yeung. Revisiting point cloud classification: new benchmark dataset and classification model on real-world data. In Proceedings of the IEEE/CVE international conference on computer vision, pages 1588-1597, 2019.
[48] A. Vaswani, Shazeer, N. Parmar, Uszkoreit, L. Jones, A. N. Gomez, Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[49] H. Wang, J. Fan, Y. Wang, K. Song, T. Wang, and Z.-X. ZHANG. Droppos: Pre-training vision transformers by reconstructing dropped positions. Advances in Neural Information Processing Systems, 36:46134 1614111446111, 2023.
[50] H. Wang, Q. Liu, X. Yue, J. Lasenby, and M. J. Kusner. Unsupervised point cloud pre- training via occlusion completion. In Proceedings the on computer vision, pages 9782-9792, 2021.
the IEEE/CVF international conference
[51] Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (tog). 38(5):1-12, 2019.
[52] Z. Wang, X. Yu, Y. Rao, J. Zhou, and J. Lu. P2p: Tuning pre-trained image models for point cloud analysis with point-to-pixel prompting. Advances in neural information processing systems, 35: 11148-1184022 2022.
[53] Z. Wang. X. Yu, Y. Rao, J. Zhou, and J. Lu. Take-a-photo: 3d-to-2d generative pre-training of point cloud models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5640-5650, 2023.
[54] C. Wei, L. Xie, X. Ren, Y. Xia, C. Su, J. Liu, Q. Tian, and A. L. Yuille. Iterative reorganization with weak spatial constraints: Solving arbitrary jigsaw puzzles for unsupervised representa- tion learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1910-1919, 2019.
[55] S. Wu, Y. Lin, F. Zhang, Y. Zeng, J. Xu, P. Torr, X. Cao, and Y. Yao. Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer. arXiv preprint arXiv:2-405.44822 2024.
[56] Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1912-1920, 2015.
[57] S. Xie, J. Gu, D. Guo, C. R. Qi, L. Guibas, and O. Litany. Pointcontrast: Unsupervised pre- training for 3d point cloud understanding. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16, pages 574-591. Springer, 2020.
13

---

[58] J. Xu, X. Wang, W. Cheng, Y.-P. Cao. Y. Shan, X. Qie, and S. Gao. Dream3d: Zero-shot text-to-3d synthesis using 3d shape prior and text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11908-20911 2023.
[59] I.. Xue, M. Gao, C. Xing, R. Martin-Martin, Wu, C. Xiong, R. Xu, J. C. Niebles, and S. Savarese. Ulip: Learning a unified representation of language. images, and point clouds for 3d understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1179-1189, 2023.
[60] L. Yi, V. G. Kim, D. Ceylan, I.-C. Shen, M. Yan, H. Su, C. Lu, Q. Huang, A. Sheffer, and L. Guibas. A scalable active framework for region annotation in 3d shape collections. ACM Transactions on Graphics (1oG), 35 2016.
[61] H.-T. Yu and M. Song. Mm-point: Multi-view information-cnhanee multi-modal self- supervised 3d point cloud understanding. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 6773-6781, 2024.
[62] X. Yu, L. Tang, Y. Rao, T. Huang. Zhou, and Lu. Point-bert: Pre-training 3d point cloud transformers with masked point modeling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19313 19322, 2022.
[63] Y. Zeng, C. Jiang, J. Mao, J. Han, C. Ye, Huang, D.-Y. Yeung. Z. Yang, X. Liang, and H. Xu. Clip2: Contrastive language-inage-oing pretraining from real-world point cloud data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
15244-15253, 2023.
[64] L. Zhang, A. Rao, and M. Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference computer vision, pages 3836-3847, 2023.
[65] R. Zhang, Z. Guo, P. Gao, R. Fang, B. Zhao, D. Wang. Y. Qiao, and H. Li. Point-m2ae: multi-scale masked autoencoders for hierarchical point cloud pre-training, Advances in neural information processing systems, 35:27061-27074. 2022.
[66] R. Zhang, Z. Guo, W. Zhang. K. Li, X. Miao, B. Cui, Y. Qiao, P. Gao, and H. Li. Pointelip: Point cloud understanding by clip. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5552-8562, 2022.
[67] R. Zhang, L. Wang, Y. Qiao, P. Gao, and H. Li. Learning 3d representations from 2d pre-trained models via image-to-point masked autoencoders. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21769-21780, 2023.
[68] X. Zhang, S. Zhang, and J. Yan. Pep-mae: Learning to predict centers for point masked autoencoders. Advances in Neural Information Processing Systems, 01880033-80217 2024.
[69] 7. Zhang, R. Girdhar, A. Joulin, and I. Misra. Self-supervised pretraining of 3d features on any point-cloud. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10252-10263, 2021.
[70] X. Zheng, X. Huang, G. Mei, Y. Hou, Z. Lyu, B. Dai, W. Ouyang, and Y. Gong. Point cloud pre-training with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22935-22945, 2024.
[71] Y. Zhou, B. Liu, Y. Zhu, X. Yang, C. Chen, and J. Xu. Shifted diffusion for text-to-image generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10157-10166, 2023.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The Abstract and Introduction sections clearly and accurately summarize the paper's ' core contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Our paper includes a dedicated discussion of limitations and future directions in Appendix F
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sellsspeciiiationn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer:
[NA]
Justification:
This paper does not include theoretical results that require a full proof.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proof's provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide comprehensive details to ensure the reproducibility of our experi- mental results. This includes clear descriptions of the model architecture, training settings, dataset splits, and data preprocessing steps. For each downstream task, we specify the exact hyperparameters and configurations used. All results are obtained using standard publicly available datasets, and the experimental setups are described in Appendix A
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
16

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We have already open sourced method. Experiments can be replicated using our open source Py PyTorch optimizer implementation together with existing open source code bases implementing each method.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
httips..ccppullcguuu.eeee
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have detailed all the training and evaluation settings before the main results in the experimental part.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We have reported the experimental results averaged over multiple random seeds.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
17

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide information of GPU type and number of GPUs used for running our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:///waarrrp.ccppppaaccaaaaaa
Answer: [Yes]
Justification: We have read the NeurlPS Code of Ethics and confirm that this research conforms to it.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., - if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
We foresee no potential societal impact of this work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
18

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: This paper does not involve data or models with high risk for misuse. Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All assets used in this work have been properly credited, and the licenses and terms of use are explicitly mentioned and respected.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This research does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This research does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
20

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core methods of this research not involve LLMs as important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
21

---

Table 7: Training details for pre-training and downstream fine-tuning.
Contig
ShapeNet
ScanObjectNN
ModelNet
ShapeNedPart
S3DIS
Optimizer
AdamW
Adam W
Adam
AdamW
Adam
Learning rate
5c-4
2c-5
lo-5
2c-4
2c-4
Weight decay
5e-2
5e-2
5e-2
5c-2
5e-2
Learning rate scheduler
cosine
cosine
cosine
cosine
cosine
Training epochs
300
300
300
300
60
Warmup epochs
10
10
Batch size
32
16
Drop path rate
0.1
0.1
Number points
1024
2048
1024
2048
2048
Number of point patches
128
128
128
Diffusion step
2000
Beta start
le-4
Beta end
le-2
Augmentation
Scalc&TTran4Rottation
Rotation RTX 3090
Scale&Trans RTX 3090
GPU device
RTX 3090
RTX 3090
RTX 3090
Appendix
A Experimental Settings Details
Pre-training details. We sample each input 1,024 points and divide them into 64 groups, each containing 32 points. We apply scale and translation operations, followed by rotation for data augmentation. The model is pre-trained with batch size of 128 for 300 total epochs. Following Point-BERT (621. we set the hidden dimension of each encoder block to 384, the number of heads 6, and the FFN expansion ratio 4. The depth of the transformer decoder is set to During pre-training, we adopt the AdamW optimizer with a weight decay of 0.05 and an initial learning rate of 5 10-4 with the cosine decay. All experiments are conducted on single GeForce RTX 3090. To ensure fair comparison, we employed identical experimental settings to the default fine-tuning. More details are provided in Tab. 7
Dataset details. The ShapeNet [4] is used as our pre-training dataset; covers over 50,000 unique 3D models from 55 common categories. ScanObjectNN 47 contains 15K unique 3D point cloud objects spanning 15 diverse categories, scanned from indoor scenes obtained by scanning, often characterized by cluttered backgrounds and occlusions. We evaluate our Point-MaDi on three variants: OBJ-BG, OBJ-ONLY, and PB-T50-RS, each with increasing complexity. ModelNet40 (56) includes 12,311 clean 3D CAD objects with 40 different categories; these objects are split into 9,843 samples in the official training set and 2,468 in the test set. ShapeNetPart (60) dataset contains 14,007 and 2,874 samples with 16 object categories and 50 semantic parts for training and validation. S3DIS consists of 3D scan data from 271 1 rooms across 6 different indoor spaces, which are annotated into 13 classes. We evaluate our model on Area 5, while the other areas are used for fine-tuning our model.
B Model Efficiency Comparison
We also have conducted additional experiments comparing the pre-training efficiency of our method, Point-MaDi, against both single-modal and cross- masked autoencoding approaches, including the most relevant recent work PointDif. As shown in Tab. [8] we report four key metrics: the number of parameters, GFLOPs, pre-training time (hours), and downstream classification performance on ScanObjectNN and ModelNet40. Point-MaDi achieves similar or faster pre-training time compared to PointDif (14.4h vs. 14.8h). while avoiding complex multi-stage training. Compared to other MAF-based methods, Point-MaDi introduces only a slight increase in parameters and pre-training cost, while maintaining high efficiency.
22

---

Table 8: Comparison of existing singlc-modal and cross-modal MAE methods in terms of pre- pre-training efficiency and representation capability on standard SSL benchmarks.
Pre- Pre-training efficiency
Performance
Method
Reference
Single/Cross Modal
# Params
GFLOPS
Time (h)
ScanObjectNN
ModelNee40
Point-MAE
2022
Single
29.0
85.18
93.8
Point-MZAE 65
NeuriPS 2022
Single
15.3
3.7
29.1
86.43
94.0
PointDif 70
CVPR 2024
Single
25.5
14.8
87.61
ACT 9
ICLR
Cross
135.5
88.21
12P-MAE 67
CVPR 2023
Cross
74.9
16.8
64.4
90.11
94.1
ReCon 36
ICML 2023
Cross
140.9
20.9
28.3
90.63
94,5
Point-MaDi
Single
29.0
89.52
94.1
Table 9: Few-shot classification results on ModelNet40. We perform ten separate trials for each experimental setting and the mean accuracy (%) and standard deviation are reported.
5-way
10-way
Method
10-shot
20-shot
10-shot
20-shot
Supervised Learning Only
PointNet 34
52.0.13.8
57.814.9
46.614.33
35.2.14.8
DGCNN 51
31.6#2.88
40.8
19.9#2.1
16.9
OcCo 50
90.6:t..8
9.
82.9
86.5#222
with Single-Modal Self-Supervised Representation Learning
Point-BERT [62
94.6#1.11
96.3+2.7
91.015.4
92.7=5,11
MaskPoint 22
95.0=3.77
97.2+1.7
91.41:.0
93.4
Point-MAE 31
96.3+2.5
97.811.8
92.6#4.1
95.0:33.0
Point-M2AF 65
9..8+1188
98.3+114
92.3++455
95.0++.0
PointGPT 5
96.8+-2.0
98.6+1.11
92.6+4.66
95.2+3.4
Point-MaDi (Ours)
9.
9.9
93.5+4..
95.7+2.3
Improve (over Point MAAE
+1.2
+0,9
+0.7
with Cross-Modal Sele-Super Representation Learning
ACT
96.8+22.3
98.0+1..4
93.3++4.0
95.6
Joint-MAE
96.7.1.22
97.911.9
92.613.7
95.1
12P-MAE 67
97.0.11.8
98.3111.
92.61..00
95.5.13.0
TAP 53
97.3
97.8111.
93.112.6
95.8-11.0
ReCon
36
97.3
98.9
93.3#3.9
95.8+t3.0
Additional Experimental Results
C.1 Additional downstream tasks
3D object few-shot classification. We conduct few-shot classification experiments on the Model- Net40 dataset to evaluate our model's ability to generalize to new categories with limited labeled data. In the N-way K-shot setting, N classes randomly selected, and K instances per class used for training. We evaluate configurations with E (5, 10} and K {10, 20). For each class, K samples utilized for fine-tuning the model, while 20 unseen samples per class are reserved for testing. We perform 10 independent trials for each configuration following previous works [50] 62 The mean accuracy (%) and standard deviation across these trials reported, as shown in Tab. The results demonstrate that our method achieves superior performance compared to recent of approaches across various settings.
C.2 Additional ablation studies
Mask ratio. We evaluate the effect of varying the mask ratio to assess its impact on downstream performance. shown in Tab. 10 the optimum mask ratio of our Point-MaDi 60%. our framework, Point-MaDi learns features from the ground truth of masked patches and uses visible patches and predicted centers as guidance to reconstruct masked patches only. Hence, a lower mask ratio provides excessive visible context, which simplifies center denoising and weakens the
23

---

Table 10: Performance of our model on different masking strategies. The accuracies (%) are reported on three variants of ScanOhjectNN.
Mask Ratio
OBJ-BG
OBJ-ONLY
PB-T50-RS
40%
93,98
91.91
88.03
50%
93.63
92.43
88.51
60%
95.52
93.46
89.52
65%
94.66
92.25
88.31
70%
94.32
92.43
88.24
75%
93.12
92.43
88.72
80%
92.77
92.08
88.51
Table 11: Effect of different loss functions for Ccenter and patch.. The accuracies (%) are reported on three variants of ScanObjectNN.
Loss Function
OBJ-BG OBJ-ONLY
PB-T5O-RS
MSE, MSE CDL2, CDL2
94.49
92.43
87.89
91.91
90.71
86.47
MSE, CDL2
95.52
93.46
89.52
Smooth LI, CDL2
94.84
92.25
88.83
Table 12: The number of decoder depth. ScanObjectNN,
The accuracies (%) are reported on three variants of
Decoder Depth
# P (M)
OBJ-BG
OBJ-ONLY
PB-T50-RS
24.36
94.49
91.57
87.75
4
29.68
95.52
93.46
89.52
8
36.78
93.98
90.88
87.54
12
43.87
93.12
92.60
88.58
difficulty of learning meaningful spatial correlations, while higher ratios leave too few visible patches, hindering the encoder's ability infer accurate centers.
Loss function. We explore different loss function settings to evaluate the impact on the pre-training objective by varying the loss functions for Leenter and Cpatch- We test four combinations: (1) MSE for both Loenter and Cpatehs (2) L2 Chamfer Distance for both, (3) MSE for Ceenter and CDL2 for Lpatch. and (4) Smooth LI for Loenter and CDL2 for Lpatchh- The results are shown in Tab. 11 The combination of MSE for Leenter and CDL2 achieves the best performance across all benchmarks. This suggests that MSE effectively aligns sparse group centers ground truth by penalizing coordinate eттогs directly. Using CDL2 for both yields the lowest performance. Smooth L1 for Ceenter mitigates outlier effects but provides less precise center predictions than MSE, Consequently, we adopt the MSE loss for center prediction and CDL2 for patch reconstruction.
Effect of decoder depth. We investigate the impact of decoder depth in Point-MaDi to assess whether increasing the number of transformer layers enhances the quality of patch reconstruction. The results presented in Tab. 12] The performance improves as the depth increases from 1 to 4, indicating that a moderate number of layers helps the decoder better process the predicted representations. Nevertheless, Point-MaDi does not benefit from larger depth, excessive layers (8-12) lead to overfitting, focusing on overly specific patch details.
Loss weighting. The final loss involves weighted combination of the center denoising loss and the patch reconstruction loss. We evaluate different weighting values to examine their influence on performance. As shown in Tab. 13 setting 7 yields the best overall results across all datasets. We also observe a consistent decline in performance as increases, indicating that excessive emphasis on patch reconstruction may distract the model from learning robust and generalizable structural representations through center denoising.
Diffusion timestep. To measure the impact of times I, on the patch diffusion process, pre-train the model with different values of 1p E {20, 100, 200, 400, 1000, 2000), keeping other hyperparameters fixed. The results are shown in Tab. [4 We observe that performance generally improves with

---

Table 13: Performance with different 7 values. The accuracies (%) are reported on three variants of ScanObjectNN.
OBI-BG
OBJ-ONLY
PB-T50-RS
0.1
95.52
93.46
89,52
0.2
93.98
92.60
88.10
93.98
91.91
88.72
0.6
94.66
92.08
88.17
0.8
94.15
91.91
88.45
Table 14: The impact of pre-trat pre-training with different timestep lp. The accuracies (%) are reported on three variants of ScanObjectNN.
tp
OBJ-BG
OBJ-ONLY
PB-T50-RS
20
94.49
92.25
88.17
100
94,66
93.29
88.65
200
94.84
92.77
88.72
400
93.98
92.77
88.65
1000
93.98
92.94
88.38
2000
95.52
93.46
89.52
Table 15: The impact of timestep schedules for center t. and patch tp diffusion. The accuracies (%) are reported on three variants of ScanObjecINN,
te
tp
OBJ-BG BG
OBJ-ONLY
PB-T50-FF RS
200
2000
94.15
92.60
88.90
1000
2000
94.15
92.60
88.72
2000
2000
95.52
93.46
89.52
200
1000
93,80
92.94
88.97
2000
200
94.66
91.91
88.72
increasing fp. peaking at t, = 2000. This trend suggests that larger timestep introduces a broader noise range, forcing the encoder to learn robust center predictions across diverse noise levels.
Timestep schedule. In our default setting, center diffusion and patch diffusion sampled simulta- neously using the same number of time steps L. lp 2000 with a shared linear variance schedule (3, from 0.0001 0.02). We conducted ablation experiments by varying and t, independently. Results are shown in Tab. [5] We observe that desynchronizing the diffusion schedules leads to performance degradation all cases. Interestingly, the model is more sensitive changes in the decoder-side patch diffusion tp compared to variations in the encoder-side center diffusion t... When tp is reduced while keeping te fixed (e.g.. tp 200), the performance drops more significantly, likely due to insufficient corruption during training, which weakens the decoder's ' ability to reconstruct complex local geometry. In contrast, reducing & while keeping tp fixed leads smaller, though still noticeable, performance drop.
Time embedding of the encoder. the transformer encoder during the center diffusion process. The motivation behind this decision is to align the encoder's architecture with downstream tasks, where no diffusion steps or time conditioning are present. By avoiding the introduction of time embeddings during pre-training, we ensure that the encoder learns time-agnostic, task-agnostic features that generalize better across downstream domains. To validate this design decision, we conducted an ablation study comparing two variants in Tab. [6 1) With time embedding added the encoder; 2) Without time embedding, as used in our default implementation. The results on the ScanObjectNN benchmark are summarized below. Incorporating time embeddings led to consistent performance degradation across all three variants. We hypothesize that the time embeddings may introduce unnecessary conditioning noise reduce the encoder's s ability to generalize to downstream inputs, which are always clean and have no associated time-step semantics.
In framework, intentionally omit time-step embeddings in
25

---

Table 16:
The effect of time embedding in the encoder. The accuracies (%) are reported on three
variants of ScanObjectNN.
Time Embedding
OBJ-BG
OBJ-ONLY
PB-TSO-RS
94.49
92.59 93.46
88.83
95.52
89.52
D Additional Visualization Results
We present additional visualizations on the ShapeNet test split to evaluate the denoising perfor- mance of our Point-MaDi model. The encoder independently predicts visible and masked center points from 3D input coordinates, which are concatenated form the denoised centers. Concurrently, the decoder reconstructs the full point cloud from encoded features. As illustrated in Figures [ the denoised centers closely align with the GT centers across diverse categories, demonstrating the encoder's 's ability to capture global structure effectively. Meanwhile, the decoder reconstructs both global shape and local geometry the denoised points. These results highlight the model's strengths in global representation and reconstruction, with ongoing challenges in detailed local recovery.
E
Limitations
This work primarily focuses on learning robust geometric and semantic representations from point clouds in purely self-supervised and unimodal setting. While the proposed dual-diffusion framework achieves consistent improvements across various downstream tasks, several directions remain open for future exploration. One natural extension is to incorporate multi-modal information, such as images or language, to further enrich the learned representations and enhance scene understanding. Additionally, while encoder effectively predicts denoised centers that capture the global shape of point clouds, struggles to preserve fine-grained local details, limiting the precision of local geometry in the predicted centers.
26

---

(a) GT Points
(b) GT Centers
(c) Denoised Points
(d) Denoised Centers
Figure 3: Visualization of point cloud denoising by Point-MaDi. (a) GT Points: original point
cloud on ShapeNet test split. (b) GT Centers: FPS-sampled centers. (c) Denoised Points: decoder- reconstructed point cloud. (d) Denoised Centers: encoder preddcctded center points.
27

---

(a) GT Points
(b) GT Centers
(c) Denoised Points
(d) Denoised Centers
Figure 4: Visualization of point cloud denoising by Point-MaDi. (a) GT Points: original point
cloud on ShapeNet test split. (b) GT Centers: FPS-sampled centers. (c) Denoised Points: decoder-
reconstructed point cloud. (d) Denoised Centers: encoder-predicted center points.
28

---

(a) GT Points
(b) GT Centers
(c) Denoised Points
(d) Denoised Centers
Figure 5: Visualization of point cloud denoising by Point-MaDi. (a) GT Points: original point
cloud on ShapeNet test split. (b) GT Centers: FPS-sampled centers. (c) Denoised Points: decoder-
reconstructed point cloud. (d) Denoised Centers: encoder-predicte center points.
29


Reviews:
[{"review_id": "zefjDBuWfR", "reviewer": "Reviewer_tX9U", "summary": "This paper, \"Point-MaDi: Masked Autoencoding with Diffusion for Point Cloud Pre-training,\" proposes a novel self-supervised pretraining framework for point clouds called Point-MaDi. Its core innovation lies in integrating a dual-diffusion process into the Masked AutoEncoder (MAE) architecture. This design eliminates the reliance on geometric priors—such as explicit center position embeddings—found in traditional methods, while enhancing the model’s ability to learn both local geometric details and global structural representations directly from the data.", "questions": "It seems that the center diffusion module is not utilized during downstream inference, and I couldn't find a clear description of this in the paper.\n\nI recommend that the authors explicitly clarify how the pretrained model is used for downstream tasks. If I have overlooked this detail, please kindly point out where it is mentioned in the paper.", "limitations": "I believe the theoretical motivation behind this paper's contribution is not particularly strong. Although the method may show some empirical improvements, these could be attributed to factors such as model design or the amount of training data used. Overall, I find the authors’ justification for the methodological motivation unconvincing.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The main contribution of this paper is the introduction of a patch position diffusion process prior to patch-level MAE modeling on point clouds. While this design sounds more reasonable, it seems somewhat similar to a form of data augmentation on patch positions. However, I am not entirely convinced by the authors’ claim that this design helps the model better understand structural information.\n\nThis is because patch positions only represent very coarse-grained information about the point cloud. How different can the predicted patch centers obtained from the center diffusion be compared to the ground-truth patch center positions? If the difference is minimal, then this might be equivalent to simply adding small perturbations to the ground-truth patch centers. On the other hand, if the predicted centers deviate significantly, they might no longer be suitable as positional embeddings (PE) for the patch diffusion decoder.\n\nEven if the model does learn certain geometric structures, such knowledge is acquired within the center diffusion process itself, not necessarily transferred downstream.", "was_revised": true, "final_justification": "I still stand by my opinion that patch positional encoding represents only very coarse-level information of the point cloud, and performing diffusion on top of it is unnecessary. Moreover, if you downsample the point cloud, those points may become so coarse that they resemble noise and may not follow any specific distribution, making diffusion on them even less reasonable.\nIn short, I consider this to be a borderline-below paper. I don’t understand why other reviewers have given scores that exceed expectations. I will keep my original score, leaving the final decision to the AC.", "rebuttal": "We sincerely thank the reviewer for the constructive feedback and careful evaluation. The concerns raised regarding the center diffusion mechanism and its impact on downstream transfer have prompted us to revisit and clarify several key aspects of our design. We address each point below with detailed explanations and supporting results.\n\n**\\<W1\\> Concern about the effectiveness and necessity of the center diffusion process.** To address this point, we organize our response into three aspects:\n\n**1). Regarding the distinction from data augmentation.** We fully agree that patch center coordinates represent coarse-grained spatial information. However, the core innovation of Point-MaDi's center diffusion lies not in perturbing these coordinates (as typical data augmentation does), but in removing the reliance on ground-truth positional embeddings and forcing the encoder to infer global structure from partial, noisy observations.\n\nStandard data augmentation (e.g., adding noise to coordinates) aims to improve model robustness by creating slightly varied inputs while preserving the original semantics. The perturbations are usually mild and do not fundamentally alter the data's structure. Moreover, data augmentation does not require learning; it's non-parametric and uncontrolled. In contrast, our center diffusion process, over 2000 steps, progressively degrades the original patch center coordinates $C_{0}^{v}$, $C_{0}^{m}$ into near-Gaussian noise $C_{t}^{v}$, $C_{t}^{m}$. This drastic corruption means the encoder cannot rely on any form of original positional information. It is then tasked with predicting the clean centers solely based on the visual content and contextual relationships derived from the visible and masked patches. This constitutes a \"location-free\" learning paradigm, compelling the model to develop an internal understanding of spatial layout, rather than merely being robust to minor input variations.\n\n**2). Regarding learning structural information.** Building on the above, once ground-truth center positions are no longer accessible due to the heavy corruption in diffusion, the encoder must leverage self-attention among visible patches and cross-attention from masked to visible patches to reason about the likely positions of all patches. Predicting centers from high noise is not a trivial interpolation but a complex inference based on learned geometric priors and relationships. The predicted centers, even if not pixel-perfect, represent the model's synthesized understanding of the global structure.\n\n**3). Regarding difference between predicted and ground-truth center.** We acknowledge the predicted centers $\\hat{C}$ may differ from the ground-truth $C$. However, our experiments (e.g., superior downstream performance, visualizations in Appendix Fig 6-8 showing alignment) demonstrate that these predictions are meaningful and capture essential structural aspects. Even with deviations, they provide far more contextual guidance than random or grossly incorrect positions.\n\nTo further validate this, we report per-category Chamfer Distance ($×10^2$) on ShapeNe test. Lower errors in rigid categories (e.g., guitar, rifle, airplane) indicate that the predicted centers remain close to the ground-truth and preserve their positional relevance.\n\n| Category     | microphone | earphone | guitar | laptop | table | chair | airplane | car  | rifle | motorbike | pistol |\n| :----------- | :--------: | :------: | :----: | :----: | :---: | :---: | :------: | :--: | :---: | :-------: | :----: |\n| CD ↓         |    2.60    |   4.78   |  0.73  |  2.83  | 4.07  | 2.67  |   1.43   | 2.29 | 1.04  |   2.68    | 0.02 |\n\n**\\<W2\\> Concern about whether the learned structural knowledge is transferred to downstream tasks.** To clarify how the structural knowledge learned during pretraining benefits downstream tasks, we elaborate on the encoder design and provide empirical evidence as follows:\n\n**1). Unified encoder design enables joint learning of geometry and representations.** The encoder in our framework serves a dual role during pretraining: **it performs the center denoising task by predicting clean patch centers $C^v$, $C^m$, and it simultaneously extracts feature representations $T^v$ for masked patch reconstruction. This shared transformer encoder is the core of both pretext objectives.**\n\nLearning to accurately recover patch center positions from their noisy counterparts requires the encoder to capture global spatial context and inter-patch geometric relationships, rather than relying on absolute positions or local clues. As such, the learned structure is not isolated to a separate branch but becomes an intrinsic part of the encoder’s output representations.\n\n**2). Encoder-only downstream protocol ensures full reliance on learned representations.** Once pretraining is complete, we discard the decoder entirely, and only the pretrained encoder is retained for downstream tasks such as classification, segmentation, and detection. Therefore, any performance in downstream tasks relies entirely on the learned encoder features, not on decoder-specific logic or intermediate reconstruction tricks.\n\n**3). Empirical and ablation evidence confirms transferability of structural knowledge.** We present two lines of evidence to validate that the geometric knowledge acquired during center diffusion is effectively transferred to downstream tasks:\n\n- **Ablation study on positional shortcuts**:\n   We conduct a control experiment in which the encoder is provided with clean patch centers during pretraining instead of learning to denoise them. This simulates a positional shortcut akin to standard positional embeddings. We find that this variant performs worse across multiple downstream tasks. In contrast, our diffusion-based center corruption and recovery removes the positional prior and forces the model to internalize structural reasoning, leading to stronger generalization. This ablation directly supports our motivation and validates the necessity of the center diffusion design.\n\n- **Performance on structure-sensitive benchmarks**:\n   Point-MaDi outperforms Point-MAE across a wide range of downstream tasks. For instance, we observe a +5.51% gain on ScanObjectNN (OBJ-BG), +0.6% mIoU on ShapeNetPart, and +0.9% AP@50 on ScanNet 3D detection — all tasks that require strong structural awareness. These consistent gains indicate that the encoder’s learned features capture transferrable geometric information beyond appearance or local patterns.\n\n**\\<Q1\\>  Concern about whether the center diffusion module is used during downstream inference.**\n\nIndeed, the center diffusion module is only employed during the pretraining phase as part of the self-supervised pretext task. During downstream inference and fine-tuning, we do not include the center diffusion process, nor do we perform any additional denoising steps. Instead, we follow the standard protocol adopted by prior masked autoencoder works (e.g., Point-MAE, Point-BERT) and retain only the pretrained encoder to extract representations from clean, uncorrupted input point clouds.\n\n**\\<Q2\\> Suggestion to clarify how the pretrained model is used for downstream tasks.**\n\nWe sincerely apologize for the confusion and appreciate you raising this important point, and acknowledge that the paper could have been clearer on this matter. Following the standard protocol established by prior work (e.g., Point-MAE, Point-BERT), after the pretraining phase is complete: 1. The decoder (responsible for patch diffusion reconstruction) is discarded. 2. The encoder, which has learned rich representations through the dual-diffusion pretext task, is retained. 3. The pretrained encoder is then directly fine-tuned (or used with a task-specific head for linear evaluation) for various downstream tasks (classification, segmentation, detection).\n\nWe thank the reviewer once again for pointing out this potential source of confusion. We will explicitly clarify this standard procedure in the revised version of the manuscript, likely in the dedicated section on implementation details for downstream tasks."}, {"review_id": "Bs7yftrvkD", "reviewer": "Reviewer_EQeP", "summary": "This work proposes a novel point cloud pre-training algorithm, Point-MaDi, which integrates masked autoencoding and diffusion models for representation learning. Specifically, the method introduces two diffusion mechanisms: a center diffusion mechanism and a patch diffusion mechanism, where the latter is primarily used to prevent geometric information leakage. Extensive experiments conducted on diverse datasets demonstrate the effectiveness of the proposed approach, surpassing previous state-of-the-art methods.", "questions": "1. In Figure 1, the comparison with contrastive learning and masked autoencoder paradigms fails to clearly highlight the unique contribution of the proposed method. To improve this, the authors are encouraged to include comparisons with more relevant baselines such as DiffPMAE and PointDiff, and clarify the distinct advantages of Point-MaDi.\n\n2. The authors should provide deeper qualitative and quantitative analysis to compare the effectiveness of using noised center point embeddings versus original (non-noised) center point embeddings.\n\n3. Figure 2 is not referenced in the main text. Additionally, since the figure contains multiple subplots, it is recommended to add (a), (b), (c) labels to improve clarity. Notations such as E and PE in Eq. (7) should also be clearly indicated within the figure for better alignment with the text.\n\n4. The reviewer suggests that Section 3 should include an overview of the entire pipeline, along with a clear explanation of how Point-MaDi is related to or differs from PointDiff.\n\n5. In Eq. (3), it appears that the center points are already perturbed with noise, but the authors do not provide sufficient explanation or corresponding notations to clarify this process.\n\n6. Time embedding is not discussed within the center diffusion process. The authors should provide more explanation and ideally support it with ablation experiments.\n\n7. Are center diffusion and patch diffusion sampled simultaneously? If they follow different sampling schedules (i.e., two different time steps t), what is the effect? The authors are encouraged to add comparative experiments to explore this question.\n\n8. It remains unclear what specific types of information are learned by the encoder and decoder through the center and patch diffusion mechanisms, respectively. Providing visualizations of the learned representations would help the reader better appreciate the contributions of this work.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1.A simple yet effective design that facilitates engineering extensibility.\n\n2.The paper is clearly written and easy to follow.\n\nWeaknesses:\n\n1.Insufficient qualitative comparisons with existing approaches.\n\n2.Limited interpretability and ablation analysis for the individual components.", "was_revised": true, "final_justification": "I appreciate the authors' rebuttal, which has addressed most of my concerns. At this stage, I intend to maintain my original score.", "rebuttal": "We are grateful to the reviewer for the thoughtful and encouraging comments. The provided suggestions have led us to refine our presentation and deepen the analysis of our method’s contributions. Below, we respond to each concern with detailed clarifications and additional experiments.\n\n**\\<Q1\\> Lack of clear comparison with relevant diffusion-based baselines.** In the revised version, we will update Figure 1 by replacing the schematic representations of contrastive learning and MAE with two more pertinent methods: DiffPMAE and PointDiff. We will also briefly describe the characteristics of these methods.\n\n**Revised version:** “Figure 1 illustrates how Point-MaDi contrasts with existing pretext diffusion tasks: while DiffPMAE applies diffusion to masked tokens only within a masked autoencoding framework, PointDiff treats pre-training as conditional generation over full point clouds via diffusion. In contrast, our Point-MaDi introduces a dual diffusion framework that denoises both visible centers and masked patches through aligned encoder-decoder objectives, enabling robust and geometry-aware representation learning without relying on positional priors.”\n\n**\\<Q2\\> Concern about the effectiveness of using noised centers versus clean ones.** We conducted a comprehensive ablation study focusing on the components of our dual-diffusion framework as below. To ensure a fair comparison, we maintained the core Point-MaDi framework. The baseline configuration uses clean patch centers for both visible and masked patches in the encoder and employs learnable mask tokens in the decoder. The key variable is the activation or deactivation of the diffusion processes. The analysis reveals several key points: firstly, the baseline achieves a performance of 93.97, 92.60, 88.83 on OBJ_BG, OBJ_ONLY, PB_T50_RS. When only one set of centers is noised while the other remains clean, the performance compared to the baseline is either slightly degraded or shows mixed results, suggesting that providing clean positional information for only a subset of patches creates an inconsistent learning signal for the encoder; secondly, when both visible and masked centers are noised in the encoder, performance improves to 94.32, 92.94, 89.17, demonstrating that our full center diffusion mechanism, which removes all ground-truth positional shortcuts and forces the encoder to predict all clean centers from noisy inputs, is more effective than partial noising or relying on clean centers. Furthermore, this benefit is amplified when combined with the patch diffusion task in the decoder: the configuration with full center diffusion and patch diffusion enabled achieves our best overall performance, significantly outperforming the scenario where center diffusion is disabled but patch diffusion is active. Additionally, the analysis highlights the positive impact of incorporating time embeddings for patch diffusion processes.\n\n| Center (Vis) | Center (Mask) | Patch | Time Embedding | OBJ\\_BG | OBJ\\_ONLY | PB\\_T50\\_RS |\n| ------------ | ------------- | ----- | -------------- | ------- | --------- | ----------- |\n| ✗            | ✗             | ✗     | ✗              | 93.97   | 92.60     | 88.83       |\n| ✓            | ✗             | ✗     | ✗              | 93.63   | 92.43     | 88.13       |\n| ✗            | ✓             | ✗     | ✗              | 94.32   | 92.08     | 88.79       |\n| ✓            | ✓             | ✗     | ✗              | 94.32   | 92.94     | 89.17       |\n| ✗            | ✗             | ✓     | ✓              | 94.66   | 93.11     | 89.17       |\n| ✓            | ✓             | ✓     | ✗              | 94.49   | 92.43     | 88.83       |\n| ✓            | ✓             | ✓     | ✓              | 95.53   | 93.46     | 89.52       |\n\n**\\<Q3\\> Figure 2 is not referenced and lacks proper subfigure labels and alignment with equations.** We sincerely thank the reviewer's constructive suggestions, which are indeed very helpful for improving the clarity and overall presentation of our paper. We are committed to implementing the suggested changes: Specifically, we will explicitly reference Figure 2 in the main text, particularly within the newly added subsection that introduces the overall Point-MaDi pipeline. We will also revise Figure 2 to include (a), (b), and (c) labels for its subplots to enhance clarity. In addition, we will refine the figure’s legend and annotations to clearly indicate key notations such as E and PE, ensuring consistency with the corresponding text. Due to the constraints of the rebuttal process, we are unable to include the revised figure at this stage. However, we have already prepared the necessary updates and will incorporate them into the revised version of the paper to improve its clarity and completeness.\n\n**\\<Q4\\> Missing overview of the full pipeline.** We have revised the corresponding section to include a concise pipeline description and highlight the key distinctions from PointDiff.\n\n**Revised version:** “3.1 Point-MaDi Framework\nPoint-MaDi comprises two diffusion-driven modules tailored for structural and geometric representation learning. As illustrated in Fig.1, given a clean point cloud, the encoder partitions it into $g$ patches and applies a diffusion model to progressively noise the centers of both visible and masked patches. The model then learns to denoise these centers. Conditioned on the visible tokens and predicted centers, the decoder reconstructs the masked patches under Chamfer Distance supervision. Unlike PointDiff, which performs point-wise generation via a multi-module architecture, Point-MaDi unifies center and patch diffusion within a masked autoencoding framework, enabling data-driven pretraining without reliance on ground-truth positional embeddings.”\n\n**\\<Q5\\> Insufficient explanation of center perturbation process.** We apologize for the unclear presentation of the notation in the initial submission. To address this confusion, in the revised manuscript, we will explicitly clarify that the center coordinates ${C^v}$ and ${C^m}$ used in the positional embedding calculation.\n\n**Revised version:** “where $d$ is the hidden dimension of the network. To incorporate spatial information, we compute positional embeddings for both visible and masked patches. The original centers ${C^v}$ and ${C^m}$ at  are progressively noised into $C_t^v$, $C_t^m$ in the subsequent center diffusion process, which are then used as inputs to the MLP for computing positional embeddings:”\n\n**\\<Q6\\> Missing discussion and ablation of time embeddings in the center diffusion process.** In our framework, we intentionally omit time-step embeddings in the transformer encoder during the center diffusion process. The motivation behind this decision is to align the encoder's architecture with downstream tasks, where no diffusion steps or time conditioning are present. By avoiding the introduction of time embeddings during pretraining, we ensure that the encoder learns time-agnostic, task-agnostic features that generalize better across downstream domains. To validate this design decision, we conducted an ablation study comparing two variants: 1) With time embedding added to the encoder; 2) Without time embedding, as used in our default implementation. The results on the ScanObjectNN benchmark are summarized below. Incorporating time embeddings led to consistent performance degradation across all three variants. We hypothesize that the time embeddings may introduce unnecessary conditioning noise or reduce the encoder’s ability to generalize to downstream inputs, which are always clean and have no associated time-step semantics.\n\n| Time Embedding | OBJ\\_BG | OBJ\\_ONLY | PB\\_T50\\_RS |\n| -------------- | ------- | --------- | ----------- |\n| ✗              | 94.49   | 92.59     | 88.83       |\n| ✓              | 95.53   | 93.46     | 89.52       |\n\n**\\<Q7\\>  Lack of clarification on whether center and patch diffusion follow joint or separate schedules.** In our default setting, center diffusion and patch diffusion are sampled simultaneously using the same number of time steps ${t_c} = {t_p} = 2000$ with a shared linear variance schedule (${\\beta_t}$ from 0.0001 to 0.02).  We conducted ablation experiments by varying ${t_c}$ and ${t_p}$ independently. We observe that desynchronizing the diffusion schedules leads to performance degradation in all cases. Interestingly, the model is more sensitive to changes in the decoder-side patch diffusion ${t_p}$ compared to variations in the encoder-side center diffusion ${t_c}$. When ${t_p}$ is reduced while keeping ${t_c}$ fixed (e.g., ${t_p} = 200$), the performance drops more significantly, likely due to insufficient corruption during training, which weakens the decoder’s ability to reconstruct complex local geometry. In contrast, reducing ${t_c}$ while keeping ${t_p}$ fixed leads to a smaller, though still noticeable, performance drop.\n\n| \\$t\\_c\\$ | \\$t\\_p\\$ | OBJ\\_BG | OBJ\\_ONLY | PB\\_T50\\_RS |\n| -------- | -------- | ------- | --------- | ----------- |\n| 200      | 2000     | 94.15   | 92.60     | 88.90       |\n| 1000     | 2000     | 94.15   | 92.60     | 88.72       |\n| 2000     | 2000     | 95.53   | 93.46     | 89.52       |\n| 2000     | 1000     | 93.80   | 92.94     | 88.97       |\n| 2000     | 200      | 94.66   | 91.91     | 88.72       |\n\n**\\<Q8\\> Lack of analysis or visualization of encoder and decoder representations.** The encoder, through the center diffusion task, is encouraged to capture global geometric structure and contextual dependencies, the decoder, via the patch diffusion task, focuses on reconstructing fine-grained local geometry, learning to recover detailed point distributions within each patch.To further enhance interpretability, we will include additional t-SNE plots of intermediate representations and segmentation-based visualizations in the final version."}, {"review_id": "Ozd6cEJSGp", "reviewer": "Reviewer_wMGd", "summary": "In this paper, the authors propose a point cloud pre-training method that combines masked auto-encoding and diffusion. They claim that this can help to avoid geometric information leakage in point-MAE-style pre-training. They conduct extensive experiments and ablations to verify the effectiveness of the pre-training method.", "questions": "Question 2 is the most crucial one and I'm expecting the authors' rebuttal. \n\n1. What is the implementation detail of the PointNet embedding process of visible point patches? Do you strictly avoid masked information leakage in this encoding process?\n\n2. Since the positional embedding of masked patches is used as conditional latent to the denoising decoder, I think the masked center information is also leaked in the reconstruction process. Is there any misunderstanding? From my perspective, if you want to claim that geometric information leakage is avoided, then no masked center features should be provided. The diffusion model should learn how to denoise masked regions only based on visible patches. If not, I think this paper would be overclaimed. \n\n3. Why can you use MSE loss instead of CHamfer distance on center diffusion loss? Is the center point order kept during the diffusion process?\n\n4. More recent related works should be included in the results comparison: Point-CMAE[1], PCP-MAE[2].\n\n5. Whether this method is applicable to scene-level model pre-training? For example, directly pre-training on ScanNet dataset like PiMAE[3]?\n\n#### References:\n\n[1] Bin Ren, Guofeng Mei, Danda Pani Paudel, Weijie Wang, Yawei Li, Mengyuan Liu, Rita Cucchiara, Luc Van Gool, and Nicu Sebe. Bringing masked autoencoders explicit contrastive properties for point cloud self-supervised learning. arXiv preprint arXiv:2407.05862, 2024.\n\n[2] Xiangdong Zhang, Shaofeng Zhang, and Junchi Yan. Pcp-mae: Learning to predict centers for point masked autoencoders. arXiv preprint arXiv:2408.08753, 2024.\n\n[3] Chen, Anthony, et al. Pimae: Point cloud and image interactive masked autoencoders for 3d object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "#### Strengths:\n\n1. The motivation to resolve the geometric information leakage problem is crucial to the point cloud community.\n\n2. The paper is overall well-written and easy to follow.\n\n3. The experiment results convince the effectiveness of the proposed method.\n\n#### Weaknesses:\n\n1. The technical contribution of this paper is limited. It simply combines point-MAE with the diffusion model.\n\n2. Whether the geometric information leakage problem is resolved is questionable.", "was_revised": true, "final_justification": "The authors have addressed my concerns in the rebuttal, particularly regarding their explicit approach to preventing information leakage. As this is an important issue for the community, I’m glad to see the paper tackling it explicitly. I therefore recommend acceptance of this paper.", "rebuttal": "We deeply appreciate the reviewer’s insightful questions and in-depth observations. The issues raised prompted a deeper reflection on the core design of our method and offered us an opportunity to clarify important aspects. We provide comprehensive responses and supporting evidence to each point below.\n\n**\\<Q1\\>** We provide the implementation details and clarify the design rationale.\n\n1). Our embedding module adopts a standard PointNet-like architecture, following previous works on patch-level point cloud encoding. The encoder consists of two sequential 1×1 convolutional blocks with BatchNorm and ReLU activations, followed by global max pooling to produce fixed-size per-patch representations. Each patch ${{P}^{v}}$, ${{P}^{m}}$ is encoded independently using only its own points, with no cross-patch aggregation.\n\n2). While we do encode both visible and masked patches in the encoder, we emphasize that this design is not a source of masked information leakage, but rather a preventive mechanism that enables spatially informed yet leakage-free learning. This choice is motivated by three key aspects: (1) The center denoising task is a self-supervised pretext objective, analogous to masked language modeling or masked image modeling, where masked inputs are still processed during learning. (2) The decoder, which performs reconstruction, is the module where leakage must be strictly avoided. We ensure that the decoder has no access to masked patch features or ground-truth positions. Positional embeddings are derived from predicted centers with stop-gradient applied, ensuring that gradients cannot flow back to the encoder. (3) During encoder training, gradients from center denoising do backpropagate through masked patch features, but this occurs only in the context of pretext learning, not reconstruction. The model is not using masked features to predict visible outputs or reconstruct their geometry.\n\nAn additional question might be whether it is feasible to predict all centers using only visible patch tokens. While this may work for visible centers, it becomes problematic for masked ones due to two main reasons: (1) Masked patches lack corresponding tokens, making it difficult to associate visible features with spatially distant masked centers; (2) The number and identity of masked centers do not align with visible tokens, introducing ambiguity and instability in training targets.\n\n**\\<Q2\\>** We thank the reviewer for raising this important and nuanced concern regarding potential geometric information leakage and the proper conditioning in our masked patch denoising process, and we appreciate the opportunity to clarify our design.\n\n**1). Regarding the concern of geometric information leakage in our encoder-decoder architecture.** In our design, we explicitly prevent geometric leakage of masked centers by the following strategies:\n\nIn the encoder, we apply a center diffusion process that perturbs both visible and masked patch centers via Gaussian noise. The encoder is then trained to denoise both sets of centers. This dual prediction setup ensures that center positions are approximated, not copied, thereby preserving the self-supervised nature of the task. In the decoder, positional embeddings for masked patches are derived from the predicted centers, and a stop-gradient operation is applied. This ensures that the encoder's predicted centers do not receive gradient signals from the decoder, further preventing any potential shortcut or information leakage via backpropagation.\n\nThus, while both visible and masked patch features are used to predict masked centers, these centers are never used in their ground-truth form during reconstruction, and no leakage path exists between the input and output. Our design differs from traditional MAE approaches where ground-truth patch center coordinates are directly provided as positional embeddings to the encoder or decoder.\n\n**2). Regarding whether masked regions are denoised solely based on visible patches.** We affirm that our decoder strictly follows this principle. During decoding, the reconstruction of masked patches is conditioned only on: (1) The visible latent features (Tᵛ) produced by the encoder; (2) The masked tokens, which are initialized as Gaussian noise and progressively denoised; (3) The positional embeddings derived from the encoder’s predicted centers, with stop-gradient applied.\n\n**3). Regarding the use of positional information in the decoder and overclaiming concerns.** We totally understand the reviewer’s concern that the presence of positional information in the decoder may seem to contradict our claim of avoiding geometric priors. However, we respectfully clarify that:\n\nOur method does not eliminate positional embeddings altogether; rather, we avoid explicit ground-truth positional shortcuts. In the decoder, we employ positional embeddings derived from predicted centers, not ground-truth ones. These predicted positions are learned through the encoder’s center diffusion process and are not tied to any external priors. Furthermore, using predicted positional embeddings is a necessary design choice, both theoretically and empirically. Transformer architectures lack built-in inductive bias for spatial structure. Without position encodings, tokens become permutation-invariant, and the network is unable to distinguish between geometrically distinct patches with similar appearance. This makes reconstruction ill-posed and ambiguous, especially in 3D point clouds where spatial relations are critical.\n\nMore importantly, in our framework, the model is tasked with learning two intertwined objectives during pre-training: (1) Predict the center positions. (2) Reconstruct the geometric structure of those masked patches. These are mutually dependent tasks. If the model is denied both ground-truth and predicted positional cues, it must simultaneously infer what to reconstruct and where, without any explicit spatial reference. This constitutes a significantly under-constrained problem, leading to poor convergence and inferior downstream transferability. Additionally, excluding all positional embeddings introduces a misalignment between pre-training and fine-tuning: downstream tasks such as classification or segmentation inevitably rely on positional context, and models pretrained without any form of positional grounding tend to generalize poorly.\n\nTherefore, we believe that the use of predicted, detached positional embeddings strikes a principled balance between geometric integrity and task feasibility, and should not be interpreted as an overclaim.\n\n**\\<Q3\\>** We provide a detailed response to each part of this question as follows:\n\n**1). Regarding using MSE loss instead of Chamfer distance.** We adopt MSE loss for the center diffusion task due to the structured nature of the prediction targets. The center diffusion process aims to predict the clean coordinates of patch centers from their noisy counterparts. These centers, obtained via FPS as described in Section 3.1, have a fixed and unique correspondence to specific patch indices in the point cloud. Unlike the patch diffusion task, which reconstructs entire point patches and must account for unordered point sets, the center diffusion task operates on a structured set of 3D coordinates with a one-to-one mapping to the ground-truth centers. MSE is well-suited for this task because it directly measures the Euclidean distance between predicted and ground-truth center coordinates, ensuring precise alignment without requiring permutation invariance, which Chamfer Distance is designed to handle. As shown in Table 6, using MSE for the center diffusion loss outperforms Chamfer Distance, achieving higher accuracies across all ScanObjectNN variants. This suggests that MSE effectively encourages the encoder to learn robust spatial relationships for center prediction.\n\n**2). Regarding the concern of center point order.** Yes, the order of the center points is preserved throughout the diffusion process. In Section 3.1, we describe how the input point cloud is divided into g patches using FPS to select patch centers, followed by KNN to form patches. These centers are assigned a fixed order based on the FPS algorithm. During the forward diffusion process, Gaussian noise is added to the center coordinates independently for each center, preserving their original indexing. Similarly, in the reverse diffusion process, the encoder predicts clean centers while maintaining their correspondence to the original patch indices. The fixed order ensures that the predicted centers $\\hat{C}_i^v$ and $\\hat{C}_i^m$ align directly with their ground-truth counterparts $C_i^v$ and $C_i^m$, enabling the use of MSE for accurate loss computation.\n\n**\\<Q4\\>** We agree that their inclusion improves the completeness of our evaluation and better contextualizes our contributions. Accordingly, we have updated the comparisons in Section 4.1 to incorporate both methods.\n\n**\\<Q5\\>** To validate this, we conducted experiments on the SUN RGB-D dataset following the official PiMAE pipeline. We incorporate our proposed Point-MaDi into the PiMAE framework (cetner diffusion & patch diffusion process) The model is pre-trained for 400 epochs with a batch size of 16. Due to GPU resource constraints and limited rebuttal time, the fine-tuning was performed with 300 epochs.\n\nOur fine-tuned model slightly surpasses the reproduced PiMAE under the same conditions, validating the generalizability of our approach to scene-level, multi-modal pre-training. We will release our code to support further validation and reproducibility.\n\n| Method         | Pre-trained   | Epoch | AP25 | AP50 |\n|----------------|----------------|--------|------|------|\n| PiMAE| SUN RGB-D     | 1080| 59.4| 33.2 |\n| PiMAE-reproduce | SUN RGB-D     | 300    | 58.6| 31.0 |\n| PiMAE-ours      | SUN RGB-D     | 300    | 58.7| 31.2 |"}, {"review_id": "DZ6XBmUq1p", "reviewer": "Reviewer_LUo6", "summary": "The proposed Point-MaDi is a novel masked autoencoding diffusion framework for self-supervised 3D point cloud representation learning, which introduces a dual-diffusion mechanism to eliminate reliance on explicit positional embeddings. Its center diffusion encoder predicts patch coordinates without ground-truth geometric priors, while the conditional patch diffusion decoder reconstructs masked patches directly from noise using latent features, enabling pure data-driven learning of both global semantics and local geometry. Experiments show state-of-the-art performance and demonstrate superior generalization across multiple 3D tasks.", "questions": "In addition to the ones mentioned in weaknesses, please solve the following confusions:\n1. In the visualization in Appendix C, it seems that the reconstruction of the center point of some samples is relatively poor. Can authors provide the performance of center point prediction of different categories, such as CD distance, etc., and put it together with the accuracy of different categories to conduct a more in-depth analysis of the center diffusion process?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "**Strengths**:\n1. The proposed Center Diffusion avoids using real position embedding by predicting the coordinates of image blocks, effectively preventing the leakage of geometric information, thereby improving the model's ability to autonomously locate local areas.\n2. This method uses latent features to reconstruct obscured image blocks from noise, enhancing the model's understanding and generation of local structures, enabling it to more accurately restore missing content.\n3. This method abandons external geometric priors and relies only on the data itself for learning, thereby optimizing both global and local representations and improving the generalization and adaptability of the model.\n\n**Weaknesses**:\n1. Lack of model efficiency comparison. In the main experimental results table 1, please provide a comparative analysis of different methods in terms of computational complexity, and inference speed, so as to have a more comprehensive understanding of model performance.\n2. Lack of visual analysis of segmentation tasks. Lack of visual analysis on segmentation tasks, as well as visual comparative analysis with other methods.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the helpful suggestions.  Below, we provide point-by-point responses to the raised concerns and describe the corresponding improvements made in the revised manuscript.\n\n**\\<W1\\> Lack of model efficiency comparison.**\n\nWe sincerely thank the reviewer for the insightful suggestion regarding model efficiency. To provide a more comprehensive understanding of Point-MaDi’s performance, we have conducted additional experiments comparing the pre-training efficiency of our method, Point-MaDi, against both single-modal and cross-modal masked autoencoding approaches, including the most relevant recent work Point-Diff. We report four key metrics: the number of parameters, GFLOPs, pre-training time (hours), and downstream classification performance on ScanObjectNN and ModelNet40. Point-MaDi achieves a similar or faster pre-training time compared to Point-Diff (14.4h vs. 14.8h). Compared to other MAE-based methods, Point-MaDi introduces only a slight increase in parameters and pre-training cost, while maintaining high efficiency.\n\n| Method         | Reference    | Modal Type  | # Params | GFLOPs  | Time (h) | ScanObjectNN | ModelNet40 |\n| -------------- | ------------ | ----------- | -------------------- | ------- | -------- | ------------ | ---------- |\n| Point-MAE      | ECCV 2022    | Single      | 29.0       | 2.3   | 13.1    | 85.18        | 93.8       |\n| Point-M2AE     | NeurIPS 2022 | Single      | 15.3                 | 3.7     | 29.1       | 86.43        | 94.0       |\n| Point-Diff     | CVPR 2024    | Single      | 25.5                 | 4.7       | 14.8       | 87.61        | –          |\n| ACT            | ICLR 2023    | Cross| 135.5                | 31.0    | 52.8       | 88.21        | 93.7       |\n| I2P-MAE        | CVPR 2023    | Cross| 74.9                 | 16.8    | 64.4       | 90.11        | 94.1       |\n| ReCon          | ICML 2023    | Cross| 140.9                | 20.9    | 28.3       | 90.63        | 94.5       |\n| Point-MaDi | Ours     | Single  | 29.5             | 5.4 | 14.4   | 89.52    | 94.1   |\n\n\n**\\<W2\\> Lack of visual analysis of segmentation tasks.**\n\nWe thank the reviewer for the constructive suggestion regarding visual analysis on segmentation tasks. We acknowledge that the current version lacks qualitative comparisons in this aspect, and we agree that such visualizations would provide valuable insight into the effectiveness of the learned representations. Due to the restrictions on adding figures during the rebuttal phase, we are unable to include segmentation visualizations at this time. However, we will provide detailed qualitative comparisons on the S3DIS dataset in the appendix of the final version. These results will highlight the spatial coherence and boundary precision of our segmentation outputs, as well as visual differences compared to existing baseline methods.\n\n**\\<Q1\\> Can authors provide the performance of center point prediction of different categories？**\n\nWe thank the reviewer for the careful observation. We agree that some samples in Appendix exhibit relatively poor center reconstruction quality, and we appreciate the suggestion to conduct a more fine-grained analysis based on object categories. \n\nWe computed the per-category Chamfer Distance * $10^2$ (CD) over a subset of representative categories from the ShapeNet test set. The results reveal that categories with clearer and more rigid structures, such as guitar, rifle, and airplane, achieve significantly better reconstruction accuracy. In contrast, categories with less consistent shapes or small-scale geometry, such as earphone and table, show larger prediction errors. These findings align well with the visual examples in Appendix, where certain object types (e.g., earphones or tables) exhibit noisier center outputs. This analysis supports our interpretation that the center diffusion process captures global geometric priors, but may be challenged by fine-grained or ambiguous shapes. We will include this category-level CD analysis in the final version to complement the visual results and provide further insight into the behavior of center diffusion across different object types.\n\n| Category     | microphone | earphone | guitar | laptop | table | chair | airplane | car  | rifle | motorbike | pistol |\n| :----------- | :--------: | :------: | :----: | :----: | :---: | :---: | :------: | :--: | :---: | :-------: | :----: |\n| CD ↓         |    2.60    |   4.78   |  0.73  |  2.83  | 4.07  | 2.67  |   1.43   | 2.29 | 1.04  |   2.68    | 0.02 |"}]

Author rebuttal:

Meta-review: This paper proposes a diffusion-based point cloud pre-training algorithm, Point-MaDi, Specifically, the method introduces two diffusion mechanisms: a center diffusion mechanism and a patch diffusion mechanism, where the latter is primarily used to prevent geometric information leakage. Extensive experiments conducted on diverse datasets demonstrate the effectiveness of the proposed approach, surpassing previous state-of-the-art methods.

Three reviewers are positive about the paper before/after the rebuttal. One reviewer remains skeptical about the paper, claiming patch-based encoding in the paper is sub-optimal. The authors did provide experimental results that justify the empirical performance gains. The AC recommends acceptance due to the majority vote among the reviewers. It is recommended that the authors provide more intuitions about the proposed patch encoding scheme in the final version.
Decision: Accept (poster)