Asymmetric Dual Self-Distillation for 3D Self-Supervised Representation Learning
Authors: Remco F. Leijenaar
Hamidreza Kasaei
Venue: NeurIPS 2025 poster ()
Keywords: self-supervised learning
representation learning
3D point clouds
masked modeling
invariance learning
self-distillation
unsupervised learning
Primary area: general_machine_learning
TL;DR: AsymDSD is a self-supervised learning framework for 3D point clouds that combines masked modeling and invariance learning through latent space prediction.

Abstract: Learning semantically meaningful representations from unstructured 3D point clouds remains a central challenge in computer vision, especially in the absence of large-scale labeled datasets. While masked point modeling (MPM) is widely used in self-supervised 3D learning, its reconstruction-based objective can limit its ability to capture high-level semantics. We propose AsymDSD, an Asymmetric Dual Self-Distillation framework that unifies masked modeling and invariance learning through prediction in the latent space rather than the input space. AsymDSD builds on a joint embedding architecture and introduces several key design choices: an efficient asymmetric setup, disabling attention between masked queries to prevent shape leakage, multi-mask sampling, and a point cloud adaptation of multi-crop. AsymDSD achieves state-of-the-art results on ScanObjectNN (90.53\%) and further improves to 93.72\% when pretrained on 930k shapes, surpassing prior methods.

Asymmetric Dual Self-Distillation for 3D Self-Supervised Representation Learning
Remco F. Leijenaar* Hamidreza Kasaei Department of AI, University of Groningen, The Netherlands
Abstract
Learning semantically meaningful representations from unstructured 3D point clouds remains central challenge in computer vision, especially in the absence of large-scale labeled datasets. While masked point modeling (MPM) is widely used objective can limit its ability
self-supervised 3D learning, its reconstruction-base to capture high-level semantics. We propose AsymDSD, an Asymmetric Dual Self-Distillation framework that unifies masked modeling and invariance learning through prediction in the latent space rather than the input space. AsymDSD builds a joint embedding architecture and introduces several key design choices: an efficient asymmetric setup, disabling attention between masked queries to prevent shape leakage, multi-mask sampling, and point cloud adaptation of multi-crop AsymDSD achieves state-of-the the- results on ScanObjectNN (90.53%) and further improves to 93. 72% when pretrained on 930k shapes, surpassing prior methods.
Introduction
As domains such as robotics, autonomous driving, AR/VR, and remote sensing continue to grow, the importance of faree-dimensional (3D) data becomes increasingly pronounced. A central challenge in 3D vision lies in learning semantically meaningful representations from unstructured 3D point clouds. In contrast to 2D computer vision-where large labeled datasets like ImageNet [I have played prominent role in driving progress-331 datasets remain limited in both scale and diversity (see Fig. []. This scarcity, which has been referred to as the data desert problem [2), is exacerbated by the difficulty of annotating 3D data, particularly at the point level (3). Although recent efforts such as Objaverse 4. (5) mark progress toward web-scale 3D data collection, they still lag behind the scale achieved in 2D vision and natural language processing (NLP). The lack of labeled 3D data has fueled a growing interest in self- ceff-supervised representation learning (SSL/SSRL) for 3D understanding [6) SSL has proven highly effective in both NLP 7 g and 2D vision 12 offering strong scalability [6) and robust down-stream capabilities [77 18 [6], even with minimal labeled data 119. These successes have inspired SSRL. techniques be adapted point cloud data. Particularly, masked point modeling (MPM) approaches have gained traction 20 given their effectiveness and conceptual alignment with masked modeling frameworks for and NLP.
Despite their popularity, we argue that MPM-based approaches have fundamental limitations. Recon- struction objectives tend emphasize short-range dependencies and high-frequency details p7 28). which are often dominated by noise rather than semantically meaningful structure. This issue is exacerbated in complex 3D geometries, where undersampling introduces high target variance. More broadly. MPM may not lead to the semantic abstraction crucial for robust downstream performance. This is backed by empirical findings that demonstrate that these models underperform compared to invariance-based alternatives in low-shot and linear probing regimes [19]. To overcome these limitations, several works have attempted to fuse generative reconstruction with invariance-har based objectives (21 26). However, the pattern differences of these objectives can cause them to interfere
"Code available at
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Num, classes
2000
ModelNe:40
2000
50
ShapeNet
MNIST
CIFAR-10
MS COCO
ScanObjectNNN
Objaverse-XI
ImageNet
Objaverse 1.0 ImageNet: 1k
Figure 1: A depiction of the size of some well-known predominantly object-centered datasets in the computer vision domain. The blue color corresponds to (2D) image datasets, and red to 3D datasets.
with one another, resulting in worse performance than using reconstruction alone. Notably, ReCon [26] solves this through a two-tower architecture that separates the objectives, but this comes with significantly increased computational overhead.
In light of these issues, we seek more elegant integration of local masked modeling and global invariance learning. Particularly, we posit reframing of the MPM paradigm: rather than predicting the input from the latent, what if we instead predict the latent representations themselves? This shift aligns with the philosophy behind works such as CPC [2 data2vec [30 BT, and I-JEPA 32). that argue for learning semanties through prediction in latent space. The intuition stems from the notion that semantically meaningful features are those that are predictive across spatial contexts, even when fine details are occluded. For example, given only visible wing of airplane, one cannot reasonably predict the exact geometry of the tail, but one can predict the presence of tail semantic category.
Building on this insight, we introduce AsymDSD: an Asymmetric Dual Self-Distillation framework for 3D point clouds (Fig. ( AsymDSD effectively combines predictive masked modeling with invariance learning, using a joint embedding architecture (JEA) trained through ceff-distillation from a momentum teacher. The framework is designed with efficiency in mind, and addresses issues such as representation collapse and shape leakage, taking inspiration from a variety of recent works in SSRL on images. We summarize the core components and main contributions as follows:
Dual Seff-Distillation Objectives: The model jointly optimizes (1) patch-level latent masked point modeling (MPM) objective through same-view self-distillation on masked tokens, and (2) global invariance learning objective using cross-view self-distillation [33 [8]. The representations
are projected to a distribution over a discrete latent variable, thereby explicitly modeling posterior.
This gives more direct control overcome representation collapse and stabilize training.
perodder-rreditoor design. Given effective high mask ratios [22], the relatively heavy encoder processes only a small number of visible patches, while a lightweight predictor builds the representations of masked patches. Unlike previous methods 33 22], we disable self-attention over the mask queries, which avoids leaking the global shape through the positional queries, and further enhances efficiency.
Asymmetric Architecture: The student model, unlike the teacher, hosts
Multi-Mask: To amortize the cost of the additional teacher, we introduce multi-mask (31, which samples multiple independent masks per point cloud. This allows targets and non-contextualized student embeddings to be reused across masks, effectively increasing batch size at minimal cost.
While many point cloud-specific augmentations fall short in enforcing a challenging invariance objective, the modality-agnoste cropping augmentations proves effective. In particular, we adapt multi-crop (35) to point clouds, encouraging the model to efficiently learn robust local-to- global mapping capabilities.
Multi-Crop:
To evaluate the effectiveness of AsymDSD, we adopt the common ShapeNet (36] pretraining protocol and transformer backbone [21]. ensuring a fair comparison. Under this setup. AsymDSD achieves state-of-the-an performance on ScanObjectNN 377, reaching 90.53% accuracy, which is 5.35%
absolute improvement over the Point-MAE, baseline 22 The method also exhibits strong general- ization in few-shot settings, as demonstrated on Ner40
(88) Furthermore, we run ablations on training framework to show that the proposed components contribute substantial improvements.

---

Targets
projection
enntrring
Softmax
gradient
100000 Goog
dodgoR U Cobon
encoder
AEMA
EMA
encoder
Plojection
Softmax
Skip mask
Student
Predictions
tokens
Patch tokens
Mask token
<m CLS loss
- Patch loss
High level overview of AsymDSD. The diagram highlights the asymmetry between the teacher and student networks, and shows the distillation of knowledge from the momentum encoded (EMA) teacher on both a cross-view global (CLS) and same-view patch level. The block widths indicate the number of patches processed, while their lengths represent network depth. The student's efficient design is reflected in its deep but narrow encoder and wide but shallow predictor.
Figure 2:
To assess scalability, we pretrain AsymDSD on large composite dataset incorporating synthetic and scanned point clouds, including Objaverse [4], totaling over 930,000 shapes. Pre-training on this large dataset attains 93. 93.72% accuracy on ScanObjectNN, a new single-modal SOTA with a standard transformer, exceeding PointGPT-L 24 by 2.6%.
Related Work
SSRL in 2D Vision has evolved along main paradigms: discriminative (or invariance-based) and generative approaches. Discriminative methods primarily focus on overcoming representation collapse: a mode wherein representations become constant, and thus independent of the input. Approaches such as SimCLR [10] and MoCo [39] employed contrastive learning, which relies on comparing positive pairs (augmentations of the same image) against large of negatives. However, these methods require large batch sizes or memory banks to be effective. Subsequent works, such as BYOL [9] and SimSiam [40], demonstrated that contrastive negatives are dispensable. They utilize a -edenttteaccer setup with architectural asymmetry to stabilize training and avoid collapse. Follow-up studies further explored this space by integrating regularization techniques like mean clustering constraints (35], and representation decorrelation 42 43
entropy maximization
41 19,
Parallel to this, denoising autoencoders 44, have evolved into masked image modeling (MIM) approaches 12]. MAE, in particular, employs an efficient encoder-decoder setup that leverages the transformer's capacity process sparse inputs. BETT [46] introduced discrete token prediction using a pre-trained d d'VEE 47), however its targets lack high-level semantics [33]. Hybrid methods have since emerged that blend MIM with joint embedding architectures. Notably, data2vec 30 3T and
I-JEPA [32] reframe MIM as latent representation prediction, regressing contextualized embeddings produced by momentum teacher instead of raw pixels. Differently, iBOT
33) and DINOv2 [[8]
include global invariance objective ensure semantically rich targets for the MIM objective.
SSRL for Point Clouds. The success of SSRL in 2D vision has inspired analogous developments in 3D point cloud learning. Early invariance-basel methods adopted contrastive frameworks [48]
but also BYOL-style [51] or DINO-style 52 training. With the adoption of the standard transformer, point cloud-specific adaptations of MIM emerged. Point-BERT [21] and Point-MAE (22 employ masked token prediction strategies analogous to their image counterparts. To avoid shape leakage through the mask queries, PointGPT [24] sequentializes the point patches to enable autoregressive
modeling. In contrast, we demonstrate that a simpler and more efficient alternative disabling
attention on mask tokens- is equally effective. point2vec 334 builds on data2vee [30] with an added
predictor module, yet it does not address shape leakage at the predictor stage. ReCon (26) successfully combines masked point modeling (MPM) with multi-modal invariance learning but introduces added complexity through its dual-tower architecture. comparison, our method seamlessly incorporates invariance learning without any changes to the underlying model architecture.

---

3 AsymDSD: Asymmetric Dual Self-Distillation
AsymDSD unfies two complementary self-supervised objectives, which we initially introduce as independent SSRL approaches. While not tied to specific model architecture, we present under ViT-style [53] backbone with lightweight PointNet-based patch embedding module, following Point-BERT (11] more detailed description of this architecture can be found in Appendix A
3.1 Global Objective: Invariance learning
The global objective of AsymDSD innvolves the learning of representations with an abstraction of semantics that is discriminative to individual inputs, up to set of data augmentations. For this purpose, it integrates knowledge distillation (54) and invariance learning in an end-to-end framework. Specifically, given an input sample x p(x), latent variable z E 2, and two augmentations
ti, t2 ~ T, the invariance-base distillation objective is:
0" argmin Ex-p(s) [KL (pó tx(x)) II pz(z tz(x)))] AEO
where po(z æ) = (oftmax((f(<))7)>> with T a temperature to control the sharpness. In SSRL, the teacher posterior p (z|x) is not known ahead of training. Instead, past iterations of the student serve as a proxy for the teacher. Particularly, we use an exponential moving average (EMA) the student's parameter: 6' + no (1 n)8, where 6' are the teacher and & the student parameters, and 1 decay rate.
Iteratively optimizing Equation I may lead to pathological solutions in absence of additional constraints. 'shorteut" in minimizing any such objective is for the latent a to become independent of the input v, i.e. pz())= = (z x). This is particularly implied by the scenarios:
1. H (po(z)) HI (Exxp(x)) (ps(z) x)]) 0, i.e. the marginal entropy becomes zero. The model collapses to a single latent representation = that is always assigned a probability 1.
2. (pe x)) log 22, i.e., the posterior entropy becomes maximal. The model always outputs a uniform distribution over the latent space Z.
To overcome these modes of posterior collapse, we follow DINO 11 by applying centering and sharpening. Centering prevents scenario (1) by subtracting running mean from the teacher logits, thereby reducing the logits of the latent 2 that are frequently assigned high probability, thus helping to avoid low-entropy collapse. Sharpening prevents scenario (2) by setting a lower teacher softmax temperature compared to the student: < T.. This makes the posterior distribution more peaked, pushing away from a uniform distribution and thus avoiding high-entropy collapse.
Cropping Augmentation. In principle, one can omit view- -ewvinvrre by using identical inputs reducing the objective to some form of mutual information maximization with centering and sharp- ening; this does not necessarily lead to useful representations (6). Instead, strong performance in self-supervised learning is often tied to carefully designed view augmentations. However, rather than relying on hand-crafted occlusions or corruptions (20, (7), we follow recent image- based SSRL trends that favor modality- aagnostic augmentations such as masking or cropping [30 [2]. Specifically, AsymDSD generates crops by sampling randomly rotated bounding boxes with variable aspect ratios, rescaled to retain a random fraction e of the original points (see Fig.1 3)
Multi-Crop. Furthermore, we adopt multi-crop 35 point clouds by generating two global crops x and xy that cover a large fraction of the point cloud (c 0.4), along with several local crops z that may include as little as 5% of the original points (0.05 <c< 0.4). To
simplify the implementation, all crops are subsampled to fixed number of points, with local crops containing one-quarter the points of global crops. This sets up a challenging objective that encourages learning globally consistent representations from highly localized views.
(a) Local crop.
(b) Global crop.
c 0.1
0.6
Figure 3: Local and global crops.

---

Particularly, we define the multi-crop loss over the full set of crops V = { =
2/,,..,,'',
...,,,,,) as:
C = 2
Σ
Σ
C (u, v),
(2)
ME (= .a
NEVi(M)
CCLS = P (u) log (o)).
where Pg. (u), and (v) are the probability mass vectors. These are obtained by attaching dedicated CLS-token to each view to aggregate global shape information. The logits over the latent are computed via a separate projection module [I]] to process this global representation: is added further encourage
sa(u) ( (Ca (u)crs). Additionally, KoLeo loss 58 diversity representations within batch.
3.2 Patch-Level Objective: Masked Point Modeling (MPM)
While the global objective encourages inter-instance invariance and discrimination, the local objective promotes intra-instance predictability. Specifically, adopt masked point modeling (MPM) objective to learn spatially contextualized representations.
Given patchified input consisting of N₂ patches, denoted by = (x*, where X represents collection local point groups and e denotes the corresponding center points for these groups, we detine a binary mask E {0, 13. with masking ratio M,. The set of masked patch indices is
given by M {i m, = 1}. and the set of visible patch indices by M {i m, = 0}. With latent MPM, the model trained on maximizing the log pegpposttrior where the conditional comprises the visible context
P M em) accompanied by a position query e; such that E M. i.e.
E (xx2)))o((xx))MMM
Σ lo X xx...)
(4)
L
In simple terms, this expectation measures how well the model can predict the latent variable 2 corresponding the center c; given visible context R. However, cannot jointly sample and 2; as we do not have a model (x, 2;). While variational methods such as BEiT [46] and Point-BERT 21 are enticing due their mathematical interpretation [59], it remains guided by reconstruction-basse learning, compromising semantic abstraction. Instead, we adopt a dynamic momentum teacher p'y similar to the global objective, allowing us reformulate MPM
MPM 0') Rxxp(xx) M
Σ Bunnph (2.1x) [oggr X c.)]] LEM
(5)
In practice, the targets 2, (z: x) are not sampled from the teacher posterior we use a latent with finite support. This enables the enumeration all latent realizations for exact computation of the cross-entropy design of the parameterized student fā and teacher f model underlying their respective posterior density function is crucial in ensuring latent that represents semantically abstract information under the objective RMPM While symmetric architectures (c.g., denoising encoders [33] 30) are
well centering and sharpening to avoid collapse. Still, the
plausible, adopt asymmetric design with the student hosting encoder- prredder-reeictor setup:
(ā, cM) Arroj g
(fame (Ā), CM),
(6)
(x))
proj
rene (m))
Masked patch
where fone contextualizing encoder processing the visible context ă;; gred prediction module that predicts the contextu- alized embeddings for each mask query é; MASK e pos) given the encoded visible context. MASK is learnable mask token, and c!"0 the encoded position signal c; hproj is pro- jection head, that separately projects each patch embedding to the discrete latent z;.
Visible patch
Figure 4: Leakage of the coarse shape via the positions of masked patches.
consider the following benefits of this asymmetric design: (1) without the predictor, positional queries eM inadvertently leak global spatial structure at the encoder stage (as illustrated in Fig.
5

---

By deferring these queries to the predictor they can be effectively 'masked' by disabling attention between them. (2) Our encoder does not process masked embeddings during SSRL. This reduces the distribution mismatch between pre-training and downstream tasks, improving transferabilityy (3) The asymmetry due to a student predictor mitigates representations collapse and enhances training stability Without this, the model may learn trivial solutions ignoring any contextual information. For example, a partitioniig of space such that the centers Gj are uniformly projected across spatial bins Z would maximize the marginal entropy I/ (pe (z)), rendering centering ineffective in overcoming this form of collapse. (4) Due to high mask ratios, the compute heavy encoder only processes a small number of visible patches. By contrast, the predictor that processes all the masked patches can be lightweight, thereby greatly increasing training efficiency.
[60] [32.
Targets
CLS tokens
tokens
Mask takens
CLS
Patch loss
Predictions
Figure The losses of AsymDSD with multi- mask and multi-crop with both global (red and green) and local (blue) crops.
(a) Uniform
(b) Inverse block- wise
Figure 5: Masking strategies.
Masking Strategy. When it comes to the masking strategy, observe that uniform masking (Fig. Sap makes it generally easy to infer global structure due to the wide spatial distribution visible patches. To address this, we implement inverse block-wise masking (Fig, [bb, which retains only few small contiguous regions. This forces the model to infer the global shape from finer- mmer-grained details in localized area. Specifically, via N-NN on center points c. To account for possible block overlap, the final mask is adjusted by randomly flipping masked unmasked bits to achieve the target mask ratio.
sample multiple fixed-sizee blocks to add to the visible context
Multi-Mask. Although the teacher fh only performs a forward pass, is comparatively costly due to full-context encoding. To amortize this cost, a multi-mask strategy is applied, where multiple masks m are sampled per input averaging the MPM loss across them. This increases the effective batch size fraction of the usual cost, as the teacher targets as well as the coneetntextull student patch embeddings can be reused across masks.
3.3 Dual Objective Learning
AsymDSD unifies global invariance learning and local masked point modeling (MPM) within single framework. Since both objectives operate in a latent space, it enables parameter sharing without architectural entanglement. Specifically, the encoder faue is shared across objectives, while separate projection heads are maintained for the global and local tasks to accommodate objective-specific dynamics with minimal computational overhead. The predictor gered is not used to refine the CLS token, meaning it is exclusive to the MPM branch. multi-mask is applied to the two global crops, whereas the local crops remain unmasked. These masked global views create additional cross-view comparisons for the global objective, and can be seen as form of implicit denoising, similar to MSN 19.. Figure summarizes the interactions between the outputs in the complete framework.
Experiments
We evaluate AsymDSD through extensive experiments across 3D recognition, few-shot classification, and part segmentation, including studies on scalability and ablations.
4.1 ShapeNet Pre-Training
Implementation Details. We follow the common SSRL pre-training protocol involving Shape Core (36), consisting of 41,952 CAD models across 55 categories. Point clouds are generated by

---

Table 1: Overall accuracy on ModelNet40 and ScanObjectNN, Where available, the accuracy without voting is reported. ST indicates a ViT-S sized standard transformer as described in Section A and Table 7c indicates single-modal training; #P(M) indicates the number of parameters in millions.
ScanObjectNN
Method
Reference
PP(M)
SK
ModelNet40
BRJ_BG
ORI
PB 19_ RS
Supervised Learning Only
PointNtt
CVPR '17
3,5
V
89.2
73.3
79.2
68.0
PointMPP
ICLR '2
64
94.1
85,4+:
PointNeXt
NeurIPS 22
64
94.0
87.7
Adapted VIT-S
Appendix
22.1
92.9
87.6
86.7
83.5
Self Superviid Repreentation Learning Full Fine tome
Reint-BERT
CVPR 22
22.1
V
93.2
87.43
88.12
83.07
Point-MAE
ECCV 22
22
22.1
93.2
90.02
88.29
85,18
PoinIGPT:S
AsymSD-CLS-8
Section 1
22.1
V
93.6
92.77
90.55
88.72
AsymSD-MPMLS
Section
22.1
94.0
91.39
AsymDSD-S
Section 3.3
22.1
V
94.1
94.32
91.91
90,53
MaskPoint
ECCV
Point Poini-MZAE
NeurIPS '22 23
15.3
X V
94.0
91.22
88.81
86.43
93.6
Point-RAB
ACMMM 23
66
29.2
V
94.0
95.53
93.63
90,28
Point FEMAE
AAAT
25
27.4
X
94.0
95.18
93.29
90.22
PointMamba
Neurips '24
V
93.6
94.32
92.60
89.31
Self Supervised Representution Learinng
Point-BERT
CVPR '22
21
22.1
v
V
91.09 15
84.17
87. 19
74.44 12
Recon MAE
ICML '23
90.22
82.77+:30
83 13.
74.13#.21
point2vec
GCPR 23
34
22.1
92.44:.04
N2.75:1.54
85 X5.44:1.26
74.25
AsymSD-CLS--
Section 3
21.8
91,78 10
90.67:1.31
88.31
83.19
AsymSD-MPM-S
Section
21.8
93.55
89.00-1.20
87.80
81.04:1.14
AsymDSSDSS
Section
21.8
92.522.11
89.95.1.21
88.73:1.23
83.33
ACT
ICLR
91.36.1.17
85.20-1.87
85.84.1.115
76.31
ReCon
ICML '23
2R
43.3
92.47=.22
89.50=2.20
89.72:t.17
81.36#.14
uniformly sampling 16,384 surface points per mesh and normalizing them the unit sphere. For each input, we sample two global crops (1,024 points, 64 patches) and four local crops (256 points, 16 patches), with each patch comprising the K masked using inverse block-wise masking at a 70% ratio with four masks per crop. The encoder is ViT-S backbone with RMSNorm and GELU, and the student predictor a lighter ViT-Ti [6] variant with 6 layers. The projection heads expand embeddings to 4096-way latent space. Pre-training is run for 300 epochs using AdamW, with a cosine learning rate schedule peaking at 10-1, and cosine EMA decay increasing from 0.995 to 1.0 during training. With single RTX 4090, this takes roughly 18 hours to complete. For additional details, we refer the reader to Appendix B.1
nearest neighboring points. Global crops are
Object Recognition. We evaluate the downstream performance of AsymDSD on standard 3D object classification benchmarks using three protocols: From Scratch, Linear, and Full Fine-rune 22. The teacher encoder (i.e., without projection head) is used for all downstream tasks, as it generally outperforms the student. In the Linear protocol, we freeze the encoder's 's weights and add trainable linear layer atop the encoder. For the Full Fine-tune and From Scrutch settings, we employ 3- layer MLP head and update all model parameters during training. The inputs to these classification heads are formed by concatenating the CL-S-token embedding with the mean and max pooled patch embeddings. All models are trained for 150 epochs using cross-entropy loss with label smoothing set to 0.2, batch size 32, and drop path rate of 0.2 (68) The MLP head uses hidden dimensions of 256, batch normalization, and dropout (p = 0.5). Results to these experiments are shown Table
On Mede 38), a clean synthetic dataset of 12k CAD models across 40 categories, AsymDSD- S achieves 94.1 1% accuracy with Full Fine-tune, +1.2% increase compared training From Scratch (Adapted ViT-S). Even with simple linear probe, performance remains strong, indicating strong off-the-shelf representations.
contrast, point-reconstruction methods like MAE underperform by over 2% in the Linear setup. On ScanObjectNN [37], real-world scanned object dataset, AsymDSD achieves 90.53% (+7.0%) on the hardest PB PB_ RS split, surpassing all prior methods with a standard transformer by +3.6%. While some methods slightly outperform AsymDSD
7

---

cleaner subsets, they rely on architectural modifications with additional trainable parameters during fine-tuning. In fact, with equal trainable parameters under Linear probing, our method outperforms all other self-supervised approaches, including cross-modal methods like ACT 2 and ReCon [26].
Few-Shot Classification. We evaluate few-shot performance ModelNet40 following the m-way, n-shot protocol of [69]. The model is fine-tuned with a high learning rate (1 x 10-3 and predictions based on concatenated mean and max pooled patch embeddings (ignoring the CLS token). As shown in Table 2 AsymDSD-S achieves the best result in three out of four confi cnfiiuuaations further supporting the off-the-shelve quality of the learned representations in low-data regimes.
Part Segmentation. We also evaluate semantic segmentation on ShapeNet-Pare [70]. For this, we employ PoimtNet++-style decoder [7]] aggregating features from multiple encoder layers, following Point-MAE 22 As shown in Table 3 AsymDSD-S achieves competitive mloU scores, similar transformer-basee methods like Point-MAF 22 and PointGPT-S (24). While methods such as point-M2AE outperform this task, they rely on a U-Net-style architecture, which is generally better suited for dense prediction.
Table 2: Average overall accuracy and standard devia- tion on ModelNet40 few-shot over 10 independent runs per experiment.
Table 3: Part segmentation results on ShapeNet-Part. The mean loU is over all instances (Inst.) or per-class (Cls.).
Method
ST
5-way
10-way
Method
ST
mloU
10 shot
20 shot
10-shot
20 shot
Inst.
Cls.
Point-BFRT
21
94.6.13.1
2.7
91.0115.4
92.71:5.1
Point-BERT 21
85.6
84.1
MaskPoint
Point-MAE
65
X
96.34:2.5
95.0:13.7
97.8=18
97.2=1.7
91.42:.00
92.61-111
95.0:13.0
93.4:3.5
Point-MAE 22
PointGPT-S
86.1
86.2
84.4
Point-RAE
34
X
97.3-1.16
9.
93.3:1..0
95.81:3.0
AsymDSD-S
86.0
84.4
Reint-FEMAE
97.2.11.9
98.611.3
95.8
Poins-FEMAE 25
86.3
84.9
AsymDSD-S
V
96. 1
98.8=1.3
94.4+3.5
95.8+3333
Point-M2AE
X
86.5
84.9
4.2 Scaling Beyond ShapeNet
While ShapeNet is a standard benchmark for 3D SSRL, it is small by today's standards and only comprises synthetic data, constraining the quality of learned representations, To explore the scal- ability of AsymDSID, we pre-train on a substantially larger and more diverse dataset composed of synthetic and real-world 3D sources. This Mixture dataset combines a 10-dataset aggregate (133K instances) with Objaverse (797K instances). In addition, we also explore a larger VIT-B backbone (AsymDSD-B+)) extending beyond the original VIT-S configuration (AsymDSD-S*) Details of the dataset composition and training setup are provided in Appendix C.I
Experiments show strong improvements from scaling (Table When moving from ShapeNet to the more extensive Mixture dataset, fine-tuning accuracy improves by +0.6% on ModelNet40 and +3.19% on ScanObjectNN, surpassing the previous best Poin Linear evaluation results show even greater gains, with improvements of +1.24% and +8.633 +8.63%, respectively. We further compare with models that have undergone extensive post-pretraining supervised data (PP) (24). including
Table 4: Results from scaling up pre-training. PP indicates additional post-pretraining with supervised data; CM cross-modal training: #P(M) the parameters count in millions.
ScanObieecNN
Method
Reference
#P(M)
PP
OM
ModelNet40
OBJ_BO BG
OBJ_ONLY
PB PB_T50_RG
Full Fine-tune
PointGPT-B
NeurPS '23
24
92.0
94.2
93.6
92.5
PointGiPT-L As
NeurPPS '3
24
310.0
X
x
94.5
95.7
94.1
91.1
AsymDSD-B*
92.1
x
94.7
96.73
94.32
93.72
Point-MAE-B+ PoinIGPT--B ReCon++
NeadPS '23
X
120.1
Y
94.2
94.2
93.9
90.2
NeurPS '3
24
92.0
V
X
94.4
95.8
95.2
91.9
ECCV
98.62
93.34
Linour
A AsymDSD-B
21.8
X
93.98 :4
95.22 17
94,51:1.18
31 1.009
93.76-1.13
13
91.96.1.14

---

Table 5: Ablations. Accuracy on ScanObjectNN is reported using linear SVM or Full Fine-tune.
(a) Cropping and croti.crop.
(c) Predictor. c: visible context; m: mask tokens
Method
SVM
FF1
Method
Atlention
Mem.
Il/s
SVM
FFI
CLSS$8
82.27
88,72
Self Cross
cropping
73.60 L. 8.67
82.13 1 6.59
mulit-crop
78.38 3.89 85.64 3,08
MPM-S
4.80
88.58
C+m
X
17.9
4.32
77.00 1 4.06
85.98 2.60
(h) Mask strategies.
predictor
X
23.3
3.11
24.95
54.51 69.74 4 18.84
Method
Ratio
SVM
FPt
Inverse bw:
0.6
79.63
J.1 1,4.33
88.02 0.46
(d) multi-mask. Bs.: Batch size; Mm.: Number of masks
0.7
80.95
0.11
88.20
0.38
Method
Bs.
Mm.
Mem.
It/s
SVM
FFi
MPM-8
0.8
81.06
88,58
0.85
81,37
0.29
88,17 0.41
MPM-8
128
8
16.2
4,80
81.66
88.58
madlt-mask
1024
48.8
1.46
81.16
0.10
88.48
1 0.10
Uniform 0.8
79.74
1.32
87.99 0.59
the significantly larger cross-modal model ReCon++-B [26]. Remarkably, AsymDSD-B* despite its smaller size and purely self-supervised training, outperforms these models on both ModelNer40 and the hardest ScanObjectNN benchmark. These results underscore the effectiveness of AsymDSD in leveraging large-scall scale unlabeled 3D data to learn highly transferable representations.
4.3 Properties and Ablations
Synergistic Objectives. AsymDSD jointly optimizes two complementary objectives to enhance representation learning. To assess the synergistic effect, we train models on each objective separately with re-tuned hyperparameters. As shown in Table both invariance learning (AsymSD-CI.S-)) and masked point modeling (AsymSD-MPM-SS) demonstrate competitive down- down-stream performance, but their combination (AsymDSD-S) exceeds both objectives across all benchmarks after fine-tuning. For linear probing, however, individual objectives may outperform due to task-specific inductive biases favoring certain benchmarks. To further understand this synergy, examine the attention distance patterns (Figure (7) and observe that our latent MPM demonstrates an attention specialization pattern relatively aligned with the CLS objective. This contrasts with the typical inverted pattern found in masked autoencoders 126]. We hypothesize that this alignment enhances their composability.
Ablations. We highlight several ablations to demonstrate that the main contributions are not some simple add-ons for minor incremental gains, but are core to their objectives. As shown in Table 5a cropping plays key role in learning effective representations, with local crops (multi-crop) enabling full performance, We also evaluate different masking strategies (Table (b). finding that our proposed inverse block-wise masking outperforms uniform masking. The method is robust to exact configu- rations, though best results are observed with masking ratios of 0.8 for MPM and 0.7 for the dual objective. Furthermore, T5ble[5csshov that the predictor is necessary to overcome collapse. That said, we observe that combining MPM with the global objective sufficient to stabilizes training. Still, performance reduced if global shape leakage is not addressed the predictor (Appendix [E. This is further evidenced by the performance gap between our efficient cross-attention-only predictor and the more expressive design with self-attention over all patches including masks (c+m). When we disable multi-mask, the efficiency of the design becomes especially obvious (Table There is virtually no performance degradation when using multi-mask with similar effective batch size, while reducing the memory usage and throughput by nearly 70%.
AsymSD-CLS
AsymSDDMPM
AsymDSD
Avg. Attention Distance
1.0
10 11
10 12
10 11
Network Depth (Layer)
Figure 7: The average attention distance per attention head across the depth of the encoder.

---

Table 6: Comparison of prediction targets.
ModelNet40
SeanOhjectNN
Method
Raw points
Latent
SVM
FFI
SVM
FF
CLS
91.78
93.64
82.27
88.72
MPM
93.31
94.04
81.06
88.58
CLS MAE
9090
0.12
93.48
0.166
82.93
0.66
88.55
0.1.7
CLS + MPM
93.03
1.25
94.13
0.49
82.34
0.07
90.53
1.81
Latent targets. Table 6 highlights the impact of shifting the masked modeling targets from the input space (raw points) to the latent space. Our implementation of MAE shows substantial gains over Point-MAE [22]. which can be attributed to mitigating global shape leakage through our predictor design and to the scale invariance introduced by variable-sized global crops. This reveals that some of our contributions extend beyond our framework. However, more importantly, as hypothesized, combining global invariance learning (CLS) with MAE (CLS MAE) provides limited additional benefit and even results in a slight performance drop under full fine-tuning. In contrast, integrating CLS with our latent MPM objective (CLS MPM) produces clear synergistic effect, where the model to surpass both objectives when applied independently.
Conclusion and Limitations
Conclusion. In this paper, we introduced AsymDSD, unified self-supervised learning framework for 3D point clouds that integrates masked modeling and invariance learning through asymmetric dual self- distillation. By avoiding reconstruction- based targets and addressing critical issues like shape leakage and representation collapse, AsymDSD enables efficient and semantically rich representation learning, The asymmetric design not only stabilizes training but also improves computational efficiency by decoupling heavy encoding from lightweight prediction. Extensive experiments demonstrate SOTA performance on multiple benchmarks, with strong generalization in low-shot and large-scale settings.
Limitations. While AsymDSD demonstrates strong performance on wide-range of experiments, there are some limitations to the current work: (1) Pre-training and evaluations have been primarily conducted on object-centered datasets; and (2) the encoder architecture uses a flat hierarchy. These limitations closely related, as the current architecture does not scale well to larger point clouds typically found in scene-level data. However, AsymDSD can be extended to hierarchical encoders with restutiresoltion representations, which are better suited for such settings. Building on the promising results of this study, we view this as a compelling direction for future work.
Acknowledgments
This research was conducted as part the first author's 's master's thesis at the University Groningen, in collaboration with Clear Timber Analytics. The authors would like to thank Hamidreza Kasaci for academic guidance and Alex van Gelder Clear Timber Analyties for their support and provision of resources that contributed to this work. We also thank the Center for Information Technology of the University of Groningen for providing access to the Hábrók high performance computing cluster.
References
[1] Deng. Dong. Richard Socher, Li-Jia Li, Kai Li, Fei-Fei. Imagenet: large-scale hierarchical image database. In 2009 IEEE conference on computer vision and puttern recognition, pages 248-255. leee, 2009.
[2] Runpei Dong, 7ekun Qi, Linfeng Zhang, Junbo Zhang, Jianjian Sun, Zheng Ge, Yi, and Kaisheng Ma. Autoencoders as cross-modal teachers: Can pretrained 2d image transformers help 3d representation learning? arXiv preprint arXiv:22/2.08.: 2022.
[3] Angela Dai, Angel Chang. Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotall annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1928-5839, 2017.
10

---

[4] Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 13142- 13153, 2023.
[5] Mau Deitke, Ruoshi Liu, Matthew Wallingford. Huong Ngo. Oscar Michel, Aditya Kusupati, Alan Fan. Christian Laforte, Vikram Voleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advancer in Neural Information Processing Systems, 36, 2024.
[6] Ben Fei, Weidong Yang, Liwen Liu, Tianyue Luo, Rui Zhang, Yixuan Li, and Ying He. Self-supervised learning for pre-training 3d point clouds: A survey. arXiv preprint arXiv:2305.0469) 2023.
[7] Jacob Devlin, Ming-Was Wei Chang, Kenton Lec, and Kristina Toutanova. Bert: Pre-training of deep bidirec- tional transformers for language understanding. arXiv preprint arXiv: 1810.04805, 2018.
[8] Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. Generative pretraining from pixels. In International conference machine learming, pages 1691-170X PMI.R, 2020.
[9] Saan-Bastien Grill, Florian Strub, Florent Altché, Corentin Talle, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems, 33:21271-21288 2020.
[10] Ting Chen, Simon Komblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-1607. PMLR, 2020.
[11] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in super-id vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650-9660. 2021.
[12] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
2022.
[13] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001 N08.060 2020.
[14] Priya Goyal, Mathilde Caron, Benjamin Lefaudeux, Min Xu, Pengchao Wang, Vivek Pai, Mannat Singh, Vitaliy Liptchinsky, Ishan Misra, Armand Joulin, et al. Self-supervised pretraining of visual features in the wild. arXiv preprint 1rXiv:2103.01985 2021.
[15] Mannat Singh, Quentin Duval, Kalyan Vasudev Alwala, Haoqi Fan, Vaibhav Aggarwal, Aaron Adcock, Armand Joulin, Piotr Dollár, Christoph Feichtenhofer, Ross Girshick, et al. The effectiveness of mae pre-pretraining for billion-scale pretraining. In Proceedings of IEEE/CVE International Conference on Computer Vision, pages 5484-5494, 2023.
[16] David Fan, Shenghang Tong, Jiachen Zhu, Koustuv Sinha, Zhuang Liu, Xinlei Chen, Michael Rabbat, Nicolas Ballas, Yann LeCun, Amir Bar, et al. Scaling language-free visual representation learning. arXiv preprint 15X4::2500.00017 2025.
[17] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. Language models are few-shot leamerr.. Advances in neural information processing systems, 13:187711 2020.
[18] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaacddin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07192 2023.
[19] Mahmoud Assran. Mathilde Caron, Ishan Misra. Piotr Bojanowski, Florian Bordes, Pascal Vincent, Armand Joulin, Mike Rabbat, and Nicolas Ballas. Masked siamese networks for label-efficient learning. In European Conference on Computer Vision, pages 456-473. Springer, 2022.
[20] Hanchen Wang. Qi Liu. Xiangyu Yue, Joan Lasenby, and Matt J Kusner. Unsupervised point cloud pre-training via occlusion completion. In Proceedings of the IEEE/CVF international conference on computer vision, pages 7782-9792. 2021.
11

---

[21] Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, and Jiwen Lu. Point-bert: Pre-training 3d point cloud transformers with masked point modeling. In Proceedings the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 19313-19322, 2022.
[22] Yatian Pang. Wenxiao Wang, Francis EH Tay, Wei Liu, Yonghong Tian, and Li Yuan. Masked autoencoders for point cloud self-supervised learning. European conference on computer vision, pages 604-621. Springer, 2022.
[23] Renrui Zhang, Ziyu Guo, Peng Gao, Rongyao Fang, Bin Zhao, Dong Wang, Yu Qiao, and Hongsheng Li. Point-m2ae: multi-scale masked autocncoders for hicrarchical point cloud pre-training. Advances in neural information processing systems, 15:27061-27011 2022.
[24] Guangyan Chen, Meiling Wang. Yi Yang. Kai Yu. Li Yuan, and Yufeng Yue. Pointgpt: Auto-regressivel generative pre-training from point clouds. arXiv preprint arXiv:2305. 1487, 2023.
[25] Yaohua Zha, Huizhen Ji, Jinmin Li, Rongsheng Li, Tao Dai, Bin Chen, Zhi Wang. and Shu-Tar Xia. Towards compact 3d representations via point feature enhancement masked autoencoders. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38. pages 0962-6970, 2024.
[26] Zekun Qi, Rumpei Dong, Guofan Fan, Zheng Ge, Xiangyu Zhang, Kaisheng Ma, and Li Yi. Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining, arXiv preprint arXiv:2302.023/88 2023.
[27] Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P Kingma. Pixelenn++: Improving the pixelenn with discretized logistic mixture likelihood and other modifications. arXiv preprint arXiv: #rXiv:170I.0S: 1701.05517, 2017.
[28] Aditya Ramesh, Mikhail Paylov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning, pages 8821 8831, Pmlr, 2021.
[29] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXiv preprint 100.v::8877037455 2018
[30] Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli Data2vec: general framework for self-supervised leamning in speech, vision and language. In International Conference on Machine Learning, pages 1298-1312. PMI.R, 2022.
[31] Alexei Baevski, Arun Babu, Wei-Ning Hsu, and Michael Auli Efficient self-supervised learning with contextualized target representations for vision, speech and language. In International Conference on Machine Learning, pages 1416-1429, PMLR, 2023.
[32] Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with joint-embedding predictive architecture. In Pruceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition. pages 15619-15629, 2023.
[33] Jinghao Zhou, Chen Wei, Huiyu Wang. Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong. ibot: Image bert pre-training with online tokenizer. arXiv preprint 2rXiv:211.00780 2021.
[34] Karim Abou Zeid, Jonas Schult, Alexander Hermans, and Bastian Loibe. Point2vee for self- supervieed representation learning on point clouds. arXiv preprint 2303:::3033/6577, 2023.
[35] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsu- pervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, 33:9912 2924444 2020.
[36] Angel X Chang. Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv: 151200002, 2015.
[37] Mikaela Angelina Uy, Quang-Hieu Pham, Binh-Son Hua, Thanh Nguyen, and Sai-Kit Yeung. Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1588-1597, 2019.
[38] Zhirong Wu, Shuran Song. Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1912-1920, 2015.
12

---

[39] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. arxiv e-prints, art. arXiv preprini arXiv:1911.05722, 2019.
[40] Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pages 15750-15758, 2021.
[41] Mahmmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bojanowski, Armand Joulin, Nicolas Ballas, and Michael Rabbat. Semi-supervised learning of visual features by non-parametrically predicting view assignments with support samples. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8443 8452, 2021.
[42] Jure Zhontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self-supervised lcarning via redundancy reduction. In International conference machine learning, pages 12310-12320 PMLR, 2021.
[43] Adrien Bardes, Jean Ponce, and Yann LeCun. Viereg: Variance- iisiam---invarinnc--ovvaaaaaaa regularization for self supervisdd learning. arXiv preprint arXiv:2105, 04906, 2021.
[44] Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and fierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. Proceedings of the 25th international conference on Machine learning, pages 1096-1100 103, 2008.
[45] Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. Simmim: A simple framework for masked image modeling. In Proceedings of the IEFF/CVF conference computer vision and pattern recognition, pages 9653-9663, 2022.
[46] Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image transformers. arXiv preprint arXiv:2106.00 08254, 2021.
[47] Jason Tyler Rolfe. Discrete variational autoencoders. arXiv preprint arXiv: 1609:02000 2016.
[48] Saining Xie, Jiatao Gu, Demi Guo, Charles R Qi, Leonidas Guibas, and Or Litany. Pointcontrast: Unsupervised pre-training for 3d point cloud understanding. In Computer VCsi 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16, pages 574-591. Springer, 2020.
[49] Zaiwei Zhang, Rohit Girdhar, Armand Joulin, and Ishan Misra. Self-supervised pretraining of 3d features on any point-cloud. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 10252-10263 2021.
[50] Mohamed Atham, Isuru Dissanayake, Dinithi Dissanayake, Amaya Dharmasiri, Kanchana Thilakarathna, and Ranga Rodrigo. Crosspoint: Self-supervised cross modal contrastive learning for 3d point cloud understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pages 9902-9912, 2022.
[51] Siyuan Huang, Yichen Xie, Song- Zhu, and Yixin Zhu. Spatio-temporal supe-super representation learning for 3d point clouds. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6535 6545, 2021.
[52] Adriano Cardace, Riccardo Spezialetti, Pierluigi 7ama Ramirez, Samuele Salti, and Luigi Di Stefano. Self- distillation for unsupervised 3d domain adaptation. In Proceedings of the IEFF//VF Winter Conference on Applications of Computer Vision, pages 4166-41777 2023.
[53] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at seale. arXiv preprint arXiv:2010.11929, 2020.
[54] Geoffrey Hinton. Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint 103.0 2015.
[55] Michael Tschannen, Josip Djolonga, Paul K Rubenstein, Sylvain Gelly, and Mario Lucic. On mutual information maximization for representation learning. arXiv preprint arXiv:1907.13625 2019.
[56] Ralph Linsker. Self-organization in a perceptual network. Computer, 105-177 1988.
[57] Idan Achituve, Haggai Maron, and Gal Chechik. Self-supervised learning for domain adaptation on point clouds. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 123 133, 2021.
13

---

[58] Alexandre Sablayrolles, Mathijs Douze, Cordelia Schmid, and Hervé Jégou. Spreading vectors for similarity search. arXiv preprint (rXXvv:806.0339000 2018.
[59] Diederik Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv: 2013.
[60] Yuandong Tian, Xinlei Chen, and Surya Ganguli. Understanding supe-supe learning dynamics without contrastive pairs. In International Conference on Machine Learning, pages 10268-10278, PMLR, 2021.
[61] Hugo Touvron, Matthieu Cord, Mathijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, pages 00347-10357. PMLR, 2021.
[62] CR Qi, Su, Mo, and IJ Guibas. Pointner: Deep learning on point sets for 3d classification and segmentation. cvpr 2017. arXiv preprint arXiv:1612.0059) 2016.
[63] Xu Ma, Can Qin, Haoxuan You, Haoxi Ran, and Yun Fu. Rethinking network design and local geometry in point cloud: A simple residual mip framework. arXiv preprint arXiv:2202.07122. 2022.
[64] Guocheng Qian, Yuchen Li, Houwen Peng. Jinjie Mai, Hasan Hammoud, Mohamed Elhoseiny, and Bernard Ghanem. Pointnext: Revisiting pointnet++ with improved training and scaling strategies. Advances in Neural Information Processing Systems, 35:23192-23200 2022.
[65] Haotian Liu, Mu Cai, and Yong Jae Lee. Masked discrimination for self-supervised learning on point clouds. In European Conference on Computer Vision, pages 657-675. Springer, 2022.
[66] Yang Liu, Chen Chen, Can Wang, Xulin King, and Mengyuan Liu. Regress before construct: Regress autoencoder for point cloud self-supervised learning. In Proceedings of the 31st ACM International Conference on Multimedia, pages 7738--1499 2023.
[67] Dingkang Liang, Xin Zhou, Wei Xu, Xingkui Zhu, Zhikang Zou, Xiaoqing Ye, Xiao Tan, and Xiang Bai. Pointmamba: A simple state space model for point cloud analysis. arXiv preprint 24Xiv:2402.10739 2024.
[68] Gao Huang. Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q Weinberger. Deep networks with stochastic depth. In Computer Vision ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part IV 14, pages 646-661. Springer, 2016.
[69] Charu Sharma and Manohar Kaul. superviee few-shot learning on point clouds. Advances in Neural Information Processing Systems, 33:7212-7221, 2020.
[70] Li Yi, Vladimir G Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3d shape collections. ACM Transactions Graphics (ToG), 35(6): 12, 2016.
[71] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnct+++: Deep hierarchical feature learning on point sets a metric space. Advances in neural information processing systems, 30, 2017.
[72] Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, Li Yi, and Kaisheng Ma. Shapellm: Universal 3d object understanding for embodied interaction. In European Conference on Computer Vision, pages 214-238. Springer, 2024.
[73] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[74] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[75] Yu-Qi Yang. Yu-Xiao Guo, Jian-Yu Yu Xiong, Yang Liu, Hao Pan, PhugiShuai Wang. Xin Tong, and Baining Guo. Swin3d: pretrained transformer backbone for 3d indoor scene understanding. arXiv preprint arXiv:2304.0660 069966 2023.
[76] Peng- Wang. Octformer: Octroc- transformers for 3d point clouds. ACM Transactions on Graphics (TOG), 42(4):1-11, 2023.
[77] Xiaoyang Wu, Li Jiang, Peng-Shual Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang. Tong He, and Hengshuang Zhao. Point transforner v3: Simpler, faster, stronger. arXiv preprint arXiv:2312.10035. 2023.
14

---

[78] Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen. David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training. arXiv preprint arXiv:/710.037700 2017.
[79] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv:1711.01101. 2017.
arXiv preprint
[80] Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances in Neural Information Processing Systems, 32, 2019.
[81] Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus), arXiv preprint arXiv:1606.0844 2016.
[82] Iro Armeni, Ozan Sener, Amir R Zamir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Silvio Savarese. semantic parsing of large-scale indoor spaces. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1534-1543, 2016.
[83] Shuran Song, Samuel P Lichtenberg, and Jianxiong Xiao. Sun rgb-d: rgb-d scene understanding benchmark suite. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 567-576. 2015.
[84] Agrim Gupta, Piotr Dollar, and Ross Girshick. Lvis: A dataset for large vocabulary instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5356-5364, 2019.
[85] Minghua Liu, Ruoxi Shi, Kaiming Kuang. Yinhao Zhu, Xuanlin Li, Shizhong Han, Hong Cai, Fatih Porikli, and Hao Su. Openshape: Scaling up 3d shape representation towards open- world understanding. Advances in neural information processing systems, 36:44860 44879, 2023.
[86] Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang, and Xinlong Wang. Uni3d: Exploring unified representation at scale. arXiv preprint arXiv:2310.06773, 2023.
[87] Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank. and Dacheng Tao. 3d-future: 3d furniture shape with texture. International Journal of Computer Vision 129:3313-3337, 2021.
[88] Jasmine Collins. Shubham Goel, Kenan Deng. Achleshwar Luthra, Leon Xu, Erhan Gundogdu, Zhang, Tomas F Yago Vicente, Thomas Dideriksen, Himanshu Arora, et al. Abo: Dataset and benchmarks for real-world 3d object understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 21 1226-21136, 2022.
[89] Tong Wu, Jiarui Zhang, Xiao Fu, Yuxin Wang, Jiawei Ren, Liang Pan, Wayne Wu, Lei Yang. Jiaqi Wang. Chen Qian, al. Omniobject3d: Large- vocabulacyyl 3d object datasct for realistic perception, reconstruction and generation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 803-814, 2023.
[90] Stefan Stojanov, Anh Thai, and James M Rehg. Using shape categorize: Low-shot learning with an explicit shape bias. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 1798-1808, 2021.
[91] Anthony G Francis, Brandon Kinman, Krista Ann Reymann, Laura Downs, Nathan Koenig, Ryan M Hickman, Thomas B McHugh, and Vincent Olivier Vanhoucke. Google scanned objects: A high-quality dataset of 3d scanned household items. 2022.
[92] Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Advances in neural information processing systems, 30, 2017.
[93] Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference on compuder vision, pages 1440-1448, 2015.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The claims are substantiated in the Sec. and Sec.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: There is dedication section with limitations Sec. 5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
16

---

Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The core methodology can be reproduced based on Sec. The method is generally robust against exact hyperparameters, but we provide many details in the
Justification:
Appendix B
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
17

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: Data is publicly available. We include code with configurations reproduce results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.cc.
aublic//uudees//odeetbbi.ss cosssiionlllyy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ wwtps/cccppplgguuuuaeeeeae
Saaeee/CCoSSSubisss..S.. www.ssssiicgcyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The general experimental settings are presented in Sec. Appendix contains tables with many precise details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We do not test for statistical significance, but do include standard deviations on some results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We disclose the computer resources in Sec. More details are in Ap-
pendix C.2
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httpps:/neuuipps.co http:://wwwpppp.cooppp.pttccccca
Answer: [Yes]
Justification:
The conducted research is conform the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no direct societal impact, as we propose novel learning framework for 3D point clouds.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
19

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: There is no such immediate risk.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite all original works including datasets. Our work does not redistribute cxisting assets.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
20

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our work does not involve crowdsourcing nor researching with human subjects.
Guidelines:
The answer NA means that the paper dlees not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our work does not involve crowdsourcing nor researching with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
21

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development does not involve LLMs.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /ttpr:i///eurps... http//rripsscccooofferreeeee//// cc/Comnferences/2225/LLL. for what should or should be described.
22

---

A Overall Model Pipeline
Although many dedicated models architectures for point cloud data have been devised, there is lack of unified architecture akin to those established in NLP and 2D CV. In these areas, model architectures have converged over time a few dominant designs 73 74 53
However, in recent times, a line of work on 3D SSRL has emerged [22] 21 34 24 that relies on a relatively simple model design that integrates the standard transformer architecture 74 53). This model architecture is detailed in this section. A detailed overview of this pipeline with AsymDSD pre- tr-irriinng is shown Figure g
Teacher
Embedding
PointNet
Proj. la
Transformer Block
Norma
Proj MPM
Point Cloud
Patches
MILD
Encoder fg'
Nx
FPS
EMA A
Student
Emmbedding
Proj. &CLS
PointNet
CBB
Transformer Block
Norm
Block
Transformee
Norm
Proj- MPM
MLL
Mx
9r
Centers
Encoder fø
Predictor
Figure 8: Overview of the processing pipeline for AsymDSD for a single point cloud through both the Teacher and Student network. The red colored arrows and modules indicate the stream of local structural information. The blue colors indicate the stream of global positional information. These streams get mixed at the encoder and predictor networks.
A.1
Patch Tokenization
The first step of the processing pipeline involves the abstraction of the point smaller set of point patches to obtain a manageable set of units for further processing. These patches are local groups of points that are comparable to image patches in vision transformers [33 These patches are subsequently embedded to obtain a set of patch tokens.
A.1.1 Input
The input is point cloud S, consisting of finite collection of pairs of point positions (0)) E E SPCR and F' point features ((() E FCC..
R
S = { (p0) 1 e eP p() F L,..,,....,.
(8)
However, for simplicity consider the point cloud S in tensor representation x: PC,
r PC = (p fl (x++FF)
(9)
with point positions p Nx3 and accompanying features NXF
A.1.2 Patchify
To form the patches, first, N₂ center points e are sampled via farthest point sampling (FPS). FPS procedure that iteratively samples points that most distant from the already sampled points,
23

---

starting with randomly sampled point. Subsequently, k-nearest neighbour (KNN) is employed to find the K-nearest points in x PC based on their corresponding positions p from each center This procedure is referred to as patchify and can be expressed mathematically as:
c FPS(p),
C RR N2x,
(10)
P = KNN (c. (c, ;),
P PERMEXXX(HD)
(11)
To disentangle the global positional information from the local structural information, the reference frame of each patch is translated to its respective center:
X = Paay - c c/py
X (MMXXX(3+F))
(12)
with Rayz the point positions, and Py the point features the patches.
This patchify procedure is reflected as part of the complete processing pipeline for AsymDSD in Figure
A.1.3 Patch Embedding
The obtained patches XP are themselves small point clouds. Accordingly, before further processing, they must be projected to embedding space. This embedding process effectively boils down the compression of the point cloud into a single feature vector describing its shape. For this purpose a small PointNet Embedding model used, shown in Figure 9a
This embedding model first projects the points in each patch to a higher patch -dimensional space through simple shared multi-layer perceptron (MLP), and takes the maximum of this projection over the K points. This essentially partitions the Fuclidean space into regions. To make the partitioning dependent on the contents of the point cloud, the maximum feature vector is concatenated to the projected point features before being projected and pooled once more to obtain the final patch embeddings e". In other words:
zP MMPP((X)),
2 ER N.N.XXXD patch;
(13)
max
MLP Z
max (PP
e R Dembed
(14)
)
where MLP; is multi-layer perceptron consisting of two linear layers interjected with normaliza- tion and non-linear activation function.
A.1.4 Position Embedding
The separated positions of the patches are similarly prepared for downstream processing. In contrast to textual language and images where tokens or patches are associated with discrete positions, the center points of each point patch embedded in continuous R Euclidean space. While many different methods have been proposed to reinject the positional information of point clouds [75/77], a simple and efficient absolute positional embedding (APE) is used here. The embedding is defined by a learnable mapping fros R3 + R Pambed, and is implemented with simple two-layer MLP:
pos MLP(c),
pus E R No Dembed
(15)
A.2 Contextualizing Encoder
At the core of the pipeline is the transformer encoder, which contains the majority of the parameters and carries out the bulk the processing work. This encoder utilizes global self-attention across the patch tokens, thereby incorporating shape information from the entire point cloud to obtain globally contextualized embeddings.
A.2.1 Input
To obtain the input to the tranformer encoder, learnable class embedding e CLS e RR is prepended to the patch embedding tokens, yielding the input z0 of the encoder model:
CLS, e
R (1+N2)x (1+NN))Dembbe)
(16)
24

---

Transformer Block
DropPath
Dropout
MLP
Norm
PointNet Embedding
max
DropPath
MLP
Dropout
concat
Multi-Head
Self-Attention
IaaK
Norm
MLP
(a) PointNet Embedding
(b) Transformer Block
Figure 9: Building blocks of the processing pipeline.
This additional CLS token is not associated with position in IR3 and serves to build up global representation, which beneficial for con-localalized tasks such as classification the global self-distillation objective of AsymDSD.
A.2.2 Transformer Encoder
standard transformer encoder 74 with pre-normalization is used, following the overall model structure of VIT [53] It consists of stack of L transformer blocks (Figure Due to the expected importance of positional information of the patches, the position embedding readded to the input of each block. This diverges from the typical approach where the position embedding is added transformer. Notably, the model maintains fixed embedding size throughout the depth of the network.
once before the first block
B Additional Implementation Details of AsymDSD
B.1 ShapeNet Pre-Training
Pre-training details including pre- pre-processing, model and training hyperparameters are provided in Table
B.2 Inverse Block-wise masking for point clouds
Block-wise masking has been explored for point cloud data in several studies. However, the common implementation involves sampling a single block 221 34] AsymDSD generalizes this by sampling any number of blocks of a pre-determined size B, expressed in the number of patches. Furthermore, we consider inverse block-wise masking, following Baevski et al. [3],, where the blocks instead indicate which patches to keep.
The difficulty with sampling multiple blocks is that they may partially overlap, which makes it difficult to mask the desired ratio of patches. To address this issue, the mask ratio is slight increased by adjust ratio A, [31] to increase the number of blocks Ng to be sampled. Specifically:
M,) (1-M,) A,
NB round
NN
(17)
BN
where Nº is the number of blocks, N, the number of patches, M, the mask ratio, A, the adjust ratio, and finally B. the block size.
The blocks are subsequently generated by sampling NB center positions, and accumulating the B, nearest patches according to the Liddittacce. This may lead to over- mnder-makking depending
25

---

on the amount of overlap. However, this is resolved by randomly swapping the mask bits until the desired mask ratio is achieved. The adjust ratio A, can be chosen such that the number of swaps to be performed is minimized.
Table 7: AsymDSD's hyperparameters for ShapeNet pre-training.
(a) The data pre-processing parameters including crop- (c) The hyperparameters and details of the model. ping and masking.
Parameter
Symbol
Value
Parameter
Symbol
Value
Shared Model Defanits
Data Pre-pracessing
Normalization
RMSNorm 3D
Points
16:384
ST]
Angmentation-I
Rotate 2-axis
Lincar Bias
False
Augmentation-2
Anistropic Scaling (0.8, 1.2]
Normalization
Unit Sphere
Patch Embedding (Sec. A.1.3
Cropping
MLP; Dims
128, 256
MI.Pa Dims
512, 384
Global
Local
Position Embedding
# Crops
No. Nz
2
Points
N
1024
256
ML.P Dims
128, 384
Patches
N.
64
Patch Points
K
32
32
Contexmuaizing Enccoder (Sec. A.2
Crop Fraction
00.4, 1.0]
[0.05,
Transformer Block
Transformer Encoder
Masking
Embedding Dim
Dembed
384
MLP Expansion Dim
1536
Mask Sampler
Inverse block-wise
Layers
L
12
Multi Mask
Nea
Atiention
H
Mask Ratio
Mr
0.7
Block Size
B.
Predictor
Adjust Ratio
Ar
0.1
Transformer Block
Transformer Decoder
Embedding Dim
Dembed
192
(b) The training hyperparameters.
MLP Expansion Dim
768
Layers
I.
Attention Heads
H
Parameter
Symbol
Value
Training
Projection head
MLP Dims
1024, 1024, 256
Batch Size
B
128
Epochs
300
Output
Nook
Precision
FP16 mixed
78
Linear Bias
True
Optimizer
Oplimizer
Adam W 79
I.R Scheilule
A
Cosine Annealing
Base Learning Rate
5.0 x 10
# L.R Warmup Epochs
Momentum Decay
22
0.9, 0.999
Weight Decay
0.05
KoLeo Scale
0.01
Gradient Clip Norm
10.0
Self. Distiiisttonnonn
EMA Schedule
7
Cosine
EMA Start, End
[0.995, 1.000)
Centering Momentum
0.9
Student Temp
T.
Teacher Temp Schedule
Linear Warmap
Teacher Temp CLS
T pasch
CLS
[0.04, 0.07]
Teacher Temp Patch
[0.05, 0.07)
Warmup Epochs
C Scaling Pre-Tr Pre-Training
C.1 Mixture Dataset
To scale the amount training data, models and scans from various sources were accumulated to make one large diverse datasets. Table & provides overview of these datasets and their basic properties. The first 10 listed datasets comprise total 133 668 instances. When combined with the Objaverse dataset 4. they form Mixture, resulting in total of 930 752 instances.
26

---

To provide some more details on the compilation process: for synthetic datasets without pre-existing point clouds, we uniformly sampled 16 k points from the surface of each mesh. In the case of scanned scene datasets with annotated objects specifically S3DIS [22 and SUN RGB-D [3 individual objects were cropped from the scenes and saved as individual instances. Notably, object instances with less than 2 k points were thrown wway-these are predominantly of the category clutter. Additionally, all objects were rotated to their natural upright position in a shared reference frame. Any other features, such as color or normals, were not used.
C.2 Training details
These models that are pre-trained on Mixture are referred to as AsymDSD-S* for the ViT-S-sized model and AsymDSD-B for the ViT-B model. The architectural details of the larger model are shown in Table IO with parameter counts for both models listed in Table Notably, AsymDSD-B scales the patch embedding accommodate the larger ViT-B contextualizing encoder, and upgrades the predictor from a ViT-Ti to a ViT-S, with half the usual number of layers.
AsymDSD-S was trained for 100 epoch in roughly 100 hours on a single A100 GPU. AsymDSD-PP was trained for both 50 epochs, taking around 175 hours, respectively, on the same hardware.
batch size of 128, totaling 727 k optimization steps,
C.3 Evaluation on Objaverse-LVIS
The Objaverse dataset contains a subset of approximately 47 objects annotated with one of the 1,156 categories from the LVIS dataset (84). Unlike other datasets, it does training or test split and is typically used for zero-shot evaluation in language-aligned models [85] 86
with predefined
Since model does not produce language-aligned representations, assess its representational quality through few-shot probing using both linear and kNN classifiers.
In particular, 10 instances are randomly sampled per category, and the remaining instances are added to the test set. We exclude any category with 10 or fewer instances, resulting total of 1060 remaining categories. Again,
exclude any additional features such color and sample 1024 points per instance. The few-shot sampling strategy was repeated 10 times to remove most of the noise from sampling or training. The results from these experiments are shown in Table II
Table 8: Datasets for scaling pre-training. I indicates that object instances are sampled from scenes.
Table 10: The hyperparameters and details of AsymDSD-B.
Dataset
Instances
# Classes
Type
Parameter
Value
ShapeNetCore
52 470
Synthetic
Potch Embeddiin (Sec A.I.3
FUTUUR S
16 560
50
Synthetic
ScanObjectNN
x7
16034
15
Scanned
MLP;
128.
512
ModeINe140
38
9813
40
Synthetic
MLP, Diass
1024, 768
SaDIS' 20
8948
14
Scanned
Position Embedding (Ser: A.7.4
SUN RGB-D
8451
Scanned
Amazon Berkeley Objocts
XX
Synthetic
OmniObject3D
5911
216
Synthetic
Contestunlizing Encoder (Sec: A.2
Toys4K
4000
105
Synthetic
Google Scanned Objects
91
1030
Scanned
Transformer Block
Transformer Encoder
Embedding
768
Objaverse
4
797
1156+
Mixed
MLP Expansion Dim
3072
Mixture
930 752
Mixed
Layers
Attention Heads
Drop Path (8
0.1
Predictor
Table 9: The total number of parameters of each module.
Transformer Block
Transformer Decoder
Parameters (M)
Fmnheddinn Dim
384
Module
Symbol
MLP Expansion Dim
AsymDSD-S
AsymDSD-B
# Layers
Patch Embedding
gembed fotx
0.5
6.5
Altention Heads
Transformer Encoder
21.2
85.0
Predictor
flied
2.9
11.2
Projection Head
2 x 3.2
27

---

Table 11: Topk few-shot performance on Objaverse LVIS subset. The number of shots per category 10 and the average is reported 10 independent runs. kNN uses the 5-nearest neighbors.
Linear
Method
Top1
Top3
Top5
Top1
Topa
Top5
A
37.75
57.43
64.70
33.30
49.77
54.27
A
CLS Marginal Entropy
CLS Posterior Entropy
CLS KL Divergence
7.5
7.5
Value
5.0
5.0
2.5
2.5
0.25
0.0
0.0
0.00
50 000
100
000
50 000
100
000
0
50 000
100 000
Patch Marginal Entropy
Patch Posterior Entropy
Patch KL Divergence
7.5
7.5
1.0
Value
5.0
5.0
0.5
2.5
2.5
0.0
0.0
0.0
100
100
Training Step
Training Step
Training Step
Sharpening Ouly
Sharpening and Centering
No Sharpening, No Centering
Figure 10: Marginal and posterior entropy, and KI. divergence during training.
D Properties and Ablations of AsymDSD
This section presents additional experiments and results to gain deeper insights into properties of AsymDSD.
D.1 Goudent-Teacher dynamies
The dynamics that unfold between the student and teacher are central to the effectiveness of SSRL with AsymDSD. In particular, we demonstrate the occurrence of representation collapse and highlight the importance of centering and sharpening mechanisms in mitigating this issue. In addition, we look at the teacher performance relative to the student.
D.1.1 Sharpening and Centering
For AsymDSD, sharpening and centering on the discrete targets is a central method of defense against the main modes of collapse. To demonstrate the effectiveness of these techniques, we trained AsymDSD without sharpening and centering, only with sharpening, and with both.
To determine wether collapse occurs in these setups, compute the empirical marginal and posterior entropy over the batches B during training:
H (pb (zi)) H
Σ ZEB
pa (zi 2t)
(18)
Exxp(x) (pb / x))]
Σ H (pl (zi x)),
(19)
28

---

83
Overall Accuracy
82
81
80
Student
Teacher
79
150
200
250
Epoch
Figure 11: Student versus teacher performance. It shows plots of the overall accuracy on the hardest subset of ScanObjectNN with linear SVM.
where i CLS or E M for the CLS-token the patch tokens respectively. These metrics plotted Figure 10 alongside the KL divergence from the two training objectives RCLS (0, 6') and RMPM (0, 0') Note that can simply decompose the cross-entropy obtain the KI. divergence: (x xx| y) = (x, Hx).
Without sharpening and centering, the posterior collapses to the uniform distribution over both the CLS- and patch tokens, as indicated by the maximum entropy Nook 4096 of log( Rs 8, 318.. On the other hand, when sharpening the targets, the marginal entropy falls to zero on the CLS-token, which indicates that the model always assigns a probability of 1.0 to the same token. This effect not observed for the patch tokens, which demonstrate non-zero marginal entropy larger than the posterior entropy. In this scenario we also observe non-zero KL divergence. We believe this greater stability on the patch tokens to be result of the predictor module, which has been shown to be key component in stabilizing training (Tab. 5c
60 32
When combining sharpening with centering, as per AsymDSD, both modes of collapse are avoided, as the marginal entropy now stays high while the posterior entropy goes down during training, as desired. After all, the difference between the two quantifies the mutual information between the input x and latent Zi.
D.1.2 Outperforming Teacher
A property that is sometimes witnessed with joint embedding architectures with momentum teacher, the teacher outperforming the student with the idea that the teacher guides the student towards higher quality representations [92] [I]. As shown in Figure [T] AsymDSD demonstrates this effect with the teacher on average outperforming the student during training when evaluated with lincar SVM on ScanObjectNN. That said, this performance difference not observed the ModelNet40 dataset. However, this might be related to fact that peak accuracy with a linear SVM on ModelNet40 obtained around 10% of the total training duration.
D.2 Representational Quality
For a qualitative assessment of the representations of the AsymDSD's pre-trained encoder, we project patch embeddings to RGB space with t-SNE. Specifically, patch embeddings of the AsymDSD-Bs pre-trained encoder are computed for 200 samples from the same object category in the ModelNet40 dataset. Each point is colored according to the inverse distance weighted average of the three nearest RGB patch embeddings.
The visualizations are shown for three object categories in Figure 12 from model that trained with supervised learning on the Net4 dataset. It stands out that AsymDSD shows high congruency among the projected embeddings from patches that comprise semantic part of an object. This uniformity is also manifested between parts from different objects, e.g. the cyan colored ears of the cups, the green cone shaped lamp covers. Interestingly, the latter example cone shaped lamp covers also demonstrates strong scale invariance. We believe that cropping and multi-crop plays key role in learning this invariance, it leads to varying patch
We also included visualizations
29

---

(a) Lamp- supervised learning
7
(b) Lamp- AsymDSD
(c) Chair - AsymDSD
(d) Cup AsymDSD
Zero-shot coloring of points according to inverse distance weighted nearest patch embed- dings from AsymDSD-B projected RGB space with t-SNE. The projection is computed with 200 instances from the same class with perplexity of 30.
Figure 12:
densities for the same object. Notably, these properties with strong semantic separation are not present with the model that trained with standard supervision. This supervised model instead shows strong positional bias among the projected embeddings.
E
Additional Ablations
F.1
Predictor in AsymDSD
Table 12 presents results from ablation experiments on the dual-objective formulation involving the predictor module. These results suggest that jointly optimizing both objectives improves ro- bustness: when the predictor is removed (cf. Table (5c), downstream performance does not collapse. Nonetheless, there is a modest drop in accuracy, and throughput decreases significantly. We hypothesize that this collapse-resistance stems from global invariance learning mitigating the model's collapse towards representing only the positional queries seeries-givee the variability of such queries across different crops.
Nevertheless, some performance degradation is expected when the predictor is removed due to leakage of the overall object shape through all mask queries. This effect should become pronounced when unmasked local crops are removed and the predictor is also excluded i.c. when all instances expose large portion of the coarse shape. Under this configuration, results indeed show substantial drop in performance, most notably in linear SVM accuracy.
Table 12: Ablations on AsymDSD-S.
Attention
Mode\Net40
SeanObjectNN
Method
Fig.
M0C
Mem.
Itls
Self
SVM
AsymSD
x
e
25.6
2.57
93.03
94.13
82.34
90.53
26.8
2.42
92.67
0.36
93.84
82.13
0.21
89.45
1.08
predicter
x
x
31.9
2.07
92.87
0.17
94.00 1 0.13
81.82
0.52
89.49
1.04
13c
21.8
3.01
93.03
0.0
93.84
0.29
81.99
0.35
89,52 J, 1.01
predictor
x
x
x
29.3
2.51
92.18 &. 0.85
93.64
0.49
79.88 J. 2.46
88.13.J
2.40
30

---

In fact, when using a predictor and optimizing the joint objective, multi-crop may no longer be essential. In particular, inverse block-wise masking can simulate localized contexts similar those provided by local crops, but without incurring the additional computational cost of processing them separately. However, under the current implementation, the variable nature of local crops allows for significantly smaller contexts compared to what is achievable through masking applied to global crops. This likely explains the observed performance improvements when local crops are included alongside the dual objective. However, these findings suggest promising direction for future work: exploring the use of variable mask ratios as a potential alternative to multi-crop strategies.
E.2 Objective Function
The MPM objective RMPM is presented cross-entropy objective between the teacher and student posterior over the masked patches (Eq. (5). However, as demonstrated other works [32], the high-dimennionall student representations (i.e., before projection to discrete latent z) can also be directly regressed onto the teacher representations without representational collapse ensuing.
Table 13 presents results comparing these approaches. We find that direct regression with a smooth L1- loss (3 2) [93] without the KL divergence-based RMPM objective and thus no MI-maximizing measures on a discrete latent projection does not lead to collapsed representations. However, we observe from ppper-parameters sweeps across several training runs that difficult to stabilize training. This is reflected in down-stream performance that eventually starts decreasing during training. Interestingly, when regression combined with RMPM it stabilizes training and improves performance across all benchmarks compared to training only on RMPM, We therefore use this setting combining the two losses in AsymSD-MPM-S. AsymDSD-S only uses cross cross-cntropy minimization, we did not observe down-stream improvement by adding additional regression loss.
Table 13: Loss functions for MPM. CE indicates the cross-entropy objective RMPM RG indicates regression loss.
ModelNet40
ScanOhjeetNN
SVM
SVM
92.10 0.53
93.76
0.28
78.97 0.49
87.75 0.83
90.51 2.12
93.44 J. 0.60
69.74 J. 9.72
87.37 J. 1.21
92.63
94.04
79.46
88.58
E.2.1
Additional Predictor Designs
The predictor module is a central component of MPM with Asymmetric Self-Distillation. was argued that it not only enhances training efficiency but also improves the overall training architecture by mitigating issues such as global shape leakage, distribution mismatch, and representation collapse. These results were presented in Table [cc but we experimented with several alternative predictor designs. The results of these experiments are presented in Table [4] with the designs shown in Figure 13
While it was already shown that the typical transformer encoder implementation (Fig. 13a leads to significantly reduce performance. We also tested cross-attention with concatenation of the query token itself (Fig. [3d), but observe that this yields no significant benefit, at the cost of higher memory usage and lower throughput.
Multi-Block. So far the two extremes of separately predicting all masked patches (po(z) ス ci), Vi e M) predicting all masked patches at once (pa (z) X, C.),, Vi M) have been explored. This can be generalized to predicting any number of masked patches at once [32]. Block-wise masking is particularly suitable alongside such generalized prediction objective, given that it provides local neighborhoods of masked regions to be 'unmasked'. In this way, the prediction is performed by providing the positional queries for block of patches, allowing coordinated build-up of representations over a region that extends beyond the bounds of a single patch without revealing the global shape of the object.
For multi-block, we use blocks with size B, = 9, using two suitable designs for a coordinated Although these designs achieved strong downstream
build up, shown in Figure 13b and 13e
31

---

performance, they offered no clear advantage over the simpler approach of predicting each patch token independently, while coming with the cost of lower throughput.
MLP
MLP
MLP
MLP
33
(a) Self-attention on context mask O ((S ((S+))*)
(b) Self- Self-attention +
(c) Cross-attention
(d) Cross- attention Cross-attention context self (SE²)
(c) Cross-attention context + mask (SE*)
cross attention O (E SE)
context
(SE)
Figure 13: Different transformer block designs for the predictor. The time complexity of the attention computations is included, with S [M] and F |M| the number of context and masked patches respectively.
Table 14: Ablations the predictor of AsymSD-MPM-SS RG indicates regression loss. MB indicates the use of multi-block. Of the attention tokens, C is the visual context, M the mask queries, the query token itself. Mem. is the total memory usage with batch size of 128 (with multi-mask 8) in GiB; and It's the throughput in iterations per second. For more details of the design refer to the indicated figures.
Attention
ModelNer40
SeanObjeetNN
Method
Fig.
MB
Mem.
Iths
Sell' Cross
SVM
FR
SVM
FF
(MPM-S)
B3
16.2
4.80
93.31
94.04
81.06
88,58
130
c+5
17.5
4.65
93.03 1 0.28
94.04 - 0.0
81.06 - 0.0
87.96
0.62
17.9
4.32
88.17
5.14
92.87
1.17
77.00
4.06
85.98
2.60
93.19
0.12
0.16
81.85
0.79
87.51
Multi-Block
130
15.4
4.01
93.07
0.24
94.29 1 0.25
80.95 0.11
88.17 0.41
32


Reviews:
[{"review_id": "xs3ozvKHLl", "reviewer": "Reviewer_NWNU", "summary": "In this paper, the authors tackle the problem of learning semantically meaningful representations for 3D point clouds in a self-supervised manner due to the lack of large-scale labeled 3D datasets. Their approach involves optimizing for two objectives: a global objective of learning discriminative representations that are invariant to transformations/augmentations and a local/patch-level objective of masked point modeling (MPM) to reconstruct masked regions. They make use of self-distillation from a momentum teacher for both objectives. For MPM, the model makes predictions in the latent space to ensure semantically meaningful representations are learnt and to prevent an emphasis on high-frequency details, a common shortcoming of MPM. This work further introduces a multi-crop equivalent for point clouds and a multi-mask strategy to increase the effective batch size in a cheap and efficient manner. The student and teacher models use asymmetric architectures in the MPM objective to avoid representation collapse. Self-attention between the masked and unmasked regions is removed to avoid shape leakage. The proposed approach can be used with any underlying model architecture. Overall, the authors employ many innovative techniques to counter the various commonly encountered issues with existing methods. They conduct extensive experiments to verify the effect of each part of their method and comprehensively compare against previous methods to prove the merits of their work.", "questions": "1. Could you please explain the attention distance visual in the ablations in more detail?\n2. Would be great if the second point under weaknesses could be addressed.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths:**\n1. The authors very clearly outline the state of the field, commonly used approaches and the pros and cons of each approach. They demonstrate that they have very closely studied the problem and existing solutions to come up with an innovative solution of their own that combines the advantages of existing methods while countering their disadvantages.\n2. The proposed approach is scalable as it can be applied to any model and is not tied to a specific model architecture.\n3. This work makes use of many creative and neat tricks to address various issues such as representation collapse, training instability, sensitivity to high-frequency details, efficiency, shape leakage, etc.\n4. Extensive ablation experiments have been included to demonstrate the need for each part of their method.\n5. The method is completely self-supervised and does not make use of any external models (instead uses self-distillation) while achieving competitive results.\n\n**Weaknesses:**\n1. Some parts of the paper could have been better explained/written such as the part explaining the need for centering and sharpening in the global objective to counter trivial/'shortcut' solutions being learnt and the attention distance part in the ablations.\n2. Would have been nice to see this approach applied to more model architectures (at least one more) to see its generalizability.", "was_revised": true, "final_justification": "This work introduces a novel approach to learn semantically meaningful representations for point clouds in a self-supervised manner. The paper uses several neat tricks such as dual objectives, self-distillation, asymmetric student-teacher architectures, latent-space prediction, multi-crop, multi-mask, etc. to tackle various aspects of the problem. The authors clearly demonstrate the efficacy of their approach over previous baselines and the advantages of all of their proposed techniques with comprehensive ablations and experiments. A few things were initially unclear in the paper but were addressed sufficiently in the rebuttal. My only remaining concern is the scalability of the approach to other models as suggested by the authors in theory but not clearly demonstrated in practice. Thus, taking everything into consideration, I have kept my current rating while raising the confidence score.", "rebuttal": "We thank the reviewer for their thoughtful feedback and for highlighting the strengths of our work, including its clear positioning within the field, scalability, and the use of creative design choices. Below, we address the identified weaknesses and questions.\n\n### **W1**: Clarification: Centering/sharpening\nWe appreciate the reviewer highlighting this point. While centering and sharpening are standard components in self-distillation frameworks (e.g., DINO), and thus not novel in our setting, we recognize that our current explanation could benefit from more clarity. We will revise this by including the following concepts:\n\n- Explanation of what the collapse modes intuitively mean: \n\t1. $H\\left(p_\\theta(\\mathrm{z})\\right)= 0$   - the model collapses to a single latent representation $z$ that dominates, always assigned probability 1.\n\t2. $H(p_\\theta(\\mathrm{z}\\mid \\mathrm{x}))=\\log |\\mathcal{Z}|$  - The model always outputs a uniform distribution over the latent space.\n- How centering and sharpening countract these failure modes:\n\t1. **Centering** subtracts a running mean of the teacher logits, reducing the logits of latents that are frequently assigned high probability thus helping avoid low‑entropy collapse.\n\t2. **Sharpening** makes the distribution more peaked, pushing it away from a uniform distribution and thus avoiding high‑entropy collapse.\n\n### **W2** and **Q2**: Generalizability to other architectures\nWe appreciate the reviewer’s recognition that AsymDSD is model‑agnostic and can be adapted to different backbone architectures. Our current results use a standard transformer backbone to ensure fair comparison with the majority of prior 3D SSRL methods, which follow the same evaluation protocol. This choice isolates the effect of the self‑supervised framework itself rather than conflating it with gains from more advanced backbones.\n\nStill, we agree it is valuable to demonstrate results on other architectures and are actively working on this. In particular, we are currently porting AsymDSD to PointMamba, a state‑space–based backbone. Due to time constraints, we are unable to include results in this rebuttal, but we plan on adding them to this work.\n\n### **Q1**: Explanation of the attention–distance visualization\nThe visualization in Figure 7 shows the average attention distances for the three different pre-training methods. For each transformer layer/block (x‑axis), we plot the mean spatial attention distance of patches for each head (y‑axis), averaged over patches and point cloud samples.\n\n**How we compute it**: For a query patch $i$ at layer $l$ and head $h$, let $\\alpha_{ij}^{(l,h)}$ be its attention weight to patch $j$. Let $c_i, c_j$ be the 3D centroids of the corresponding point‑patches. The expected attention distance for that query is:\n\n$$d_i^{(l,h)} = \\sum_{j} \\alpha_{ij}^{(l,h)} \\|c_i - c_j\\|_2.$$\n\nWe then average $d_i^{(l,h)}$ over all queries $i$, and samples. Since we have 6 different heads with the standard model (ViT-S), there are 6 averages for each layer.\n\n**How to read it**: Lower values correspond to a head specializing to local contexts, whereas high values correspond to a head specializing to global contexts. When all heads have roughly the same value, there is no such specialization between heads.\n\n**Key takeaway:**   The attention‑specialization patterns for our method are relatively consistent across objectives: early layers show more head specialization, while later layers focus more on global context with less head‑to‑head variation. This contrasts sharply with the attention pattern from reconstruction‑based masked modeling (MAE), where the last layers exhibit strong specialization across different spatial contexts. To make this difference clearer, we additionally trained our model with MAE. Its inverted pattern confirms this distinction (we will include this visualization in the final version).\n### Minor corrections\nWe thank the reviewer for pointing out the minor typos. These have been corrected."}, {"review_id": "qWlmf1krI6", "reviewer": "Reviewer_wuHC", "summary": "The main idea is easy to follow. Figures and tables are clear.\nThe author proposes a student-teacher method with MAE token prediction. \nThe performance is competitive. \nMy main concern is about the technical novelty, which is fusing mask token prediction with the EMA teacher-student.", "questions": "Please see weakness. I would like to raise my rating if the author could address my concerns.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The main idea is easy to follow. \nThe author proposes a student-teacher method with MAE token prediction. \nThe performance is competitive. \n\n1. My main concern is about the technical novelty, which is fusing mask token prediction with the EMA teacher-student. \nThe mask strategy is crucial, but it is quite engineering part.\nWhile combining invariance learning and masked modeling is a reasonable design, the synergy between the two objectives is not rigorously analyzed. It is unclear whether the improvement stems from their integration or just from one dominant branch (e.g., invariance). The authors should provide more theoretical or empirical justification for why dual self-distillation is fundamentally more effective than strong single-branch baselines.\n\n2. Related work\nSome methods such as the mixup augmentation also could solve the data scarity.  \n\n3. Marginal Gains Over SOTA in Some Settings:\nAlthough the method reports SOTA on ScanObjectNN, the gains over strong baselines like Point-MAE and PointGPT are often marginal (e.g., <1%), especially under full fine-tuning. These small improvements may not justify the added architectural and training complexity.\n\n4. Method Complexity vs Practicality Trade-off:\nThe method involves multiple components: asymmetric encoder/predictor, multi-mask, multi-crop, momentum teacher, masking strategy, etc. The overall complexity may hinder reproducibility or practical deployment, especially if the ablations (Table 5) show relatively minor performance gaps when removing some components.\n\n5. Unclear Advantage of Latent-Space Prediction:\nThe work claims latent-space prediction avoids issues with reconstruction targets, but does not provide a fair and isolated comparison with prior masked latent methods (e.g., BEiT-style, Point2Vec). Without ablations on prediction space (latent vs reconstruction), it is difficult to assess the actual benefit of this design choice.\n\n6. Limited Evaluation on Scene-Level Tasks:\nAs acknowledged by the authors, the method is mainly validated on object-centric datasets (ScanObjectNN, ModelNet). The claim of learning “semantically meaningful representations” would be more convincing if tested on scene-level data like indoor lidar or auto-driving, where global understanding is more critical.", "was_revised": true, "final_justification": "The author has addressed my concerns. I will raise the rate.", "rebuttal": "We thank the reviewer for their constructive feedback and are glad they found the paper clear and recognize its competitive performance. Below we address the main concerns point-by-point.\n### **W1** Technical Novelty and Synergy Between Objectives\nWe appreciate the reviewer’s concern regarding novelty and would like to clarify our position. While the fusion of masked token prediction with an EMA teacher has indeed been explored in prior 2D SSRL methods and in 3D works such as Point2Vec, we do not claim novelty on this general idea alone.\n\nImportantly, AsymDSD is not a direct adaptation, but rather a non-trivial and novel composition of several ideas tailored to the 3D point cloud domain, supported through intuitive and empirical findings. We address unique challenges of 3D data such as global shape leakage and efficiency concerns that existing methods do not consider. We particularly want to emphasize the following contributions:\n- **Asymmetry in design:** We introduce asymmetry both between the teacher and student networks and between the global invariance and local masked modeling branches, which prior dual-objective methods including those on images (e.g., iBOT, DINOv2) do not. In the main text, we list numerous benefits to this design choice.\n- **Disabling attention between mask-queris:** We propose a simple yet effective design at the predictor level by disabling attention between mask tokens, which to our knowledge has not been explored in prior 3D SSRL methods. This prevents masked tokens from “cheating” by leaking global shape information through attention.\n\n**Synergy between objectives:** *It is unclear whether the improvement stems from their integration or just from one dominant branch (e.g., invariance).*\n\nThe gains are not driven by one dominant branch. As shown in **Section 4.3 (Table 1)**:\n- The global invariance objective alone (AsymSD‑CLS‑S) achieves **88.72%**, and the masked point modeling objective alone (AsymSD‑MPM‑S) achieves **88.58%** on ScanObjectNN.\n- Combining both objectives with AsymDSD achieves **90.53% (+1.81%)**, exceeding both branches individually. This indicates a synergystic effect between the two branches. (See also our response to **W5**)\n\nIn Figure 7, we also show that the attention patterns of the two branches are relatively aligned. When replacing the latent-space MPM with raw point reconstruction (MAE), we observe an inverted attention pattern, suggesting a mismatch between the objectives. Following our additional experiments in **W5**, we will include a direct visualization of the MAE attention pattern in the final version to further clarify this difference.\n\n### **W2**: Mixup augmentation\nWe are aware of orthogonal approaches such as mixup (also for point clouds), but note that these methods tradionally still rely on labeled data. We believe that scaling SSRL methods to larger unlabeled datasets ultimately leads to stronger representational quality and broader applicability (generalization).\n### **W3**: Marginal gains over SOTA in some settings\nWe would like to clarify that ModelNet40 performance under full fine-tuning is close to saturated, and improvements there are naturally small. We included these results for completeness to show that AsymDSD does not regress on this benchmark.\n\n| Method        | ModelNet40      | OBJ_BG          | OBJ_ONLY        | PB_T50_RS       |\n| ------------- | --------------- | --------------- | --------------- | --------------- |\n| Previous best | 94.0            | 91.6            | 90.0            | 86.9            |\n| AsymDSD‑S     | **94.1** (+0.1) | **94.3** (+2.7) | **91.9** (+1.9) | **90.5** (+3.6) |\n\nAs shown above (gathered from Table 1), on the more challenging ScanObjectNN splits, AsymDSD achieves much larger improvements (**+1.9% to +3.6%**) under full-finetuning compared to prior methods using the same standard transformer backbone.\n\nWe also note that when all other factors (backbone, dataset, pretraining protocol) are fixed, i.e. the experiment as explained in Sec 4.1, then the performance gains are inherently limited. This makes the improvements on ScanObjectNN particularly meaningful.\n\nFinally, we view linear probing as a better indicator of representation quality than full fine-tuning. We will move additional linear probe results (see the table in our response to Reviewer *3Aup*) into the main paper, which further highlight the strength of the learned representations.\n\n### **W4**: Method Complexity vs Practicality Trade-Off\nWe recognize that AsymDSD combines several components, and there is a natural trade-off between complexity and practicality. However, the components are kept simple (not over-engineered) and have clear motivations to be included:\n\n- **Ablation evidence:**  Table 5 shows consistent performance drops when removing elements such as multi-crop, predictor (asymmetry), or using a cross-attention–only predictor. While the masking strategy is less fundamental, inverse block-wise masking is required to achieve the full performance of AsymDSD.\n- **Efficiency-focused design:** Multi-mask is a prime example of this philosophy: it is not intended to increase accuracy, but rather to reduce memory usage and improve throughput by nearly 70% at no performance cost (Table 5d).\n\n**Reproducibility:** We have ensured the design remains accessible and reproducible:\n- Many components (e.g., multi-mask, cross-attention–only predictor) are straightforward to implement in a few lines of code.\n- Other parts have widely shared implementations (e.g. EMA module)\n- Most importantly, we have included the codebase and configuration files for pretraining in the submission and will release them publicly, so other researchers can reproduce the results easily.\n### **W5**: Unclear Advantage of Latent-Space Prediction\nWe agree with the reviewer that it is valuable to isolate the effect of latent-space prediction over point coordinate reconstruction. To this end, we ran some additional experiments:\n\n|           | Raw points | Latent | ModelNet40    | ScanObjectNN  |\n| --------- | ---------- | ------ | ------------- | ------------- |\n| MAE       | ✓          | ✗      | 94.04         | 87.75         |\n| MPM       | ✗          | ✓      | 94.04         | 88.58         |\n| CLS       | -          | -      | 93.64         | 88.72         |\n| CLS + MAE | ✓          | ✗      | 93.48 (-0.16) | 88.55 (-0.17) |\n| CLS + MPM | ✗          | ✓      | 94.13 (+0.49) | 90.53 (+1.81) |\n\n**Key observations:**\nWe see that our implementation of MAE (raw point reconstruction) greatly improves over Point-MAE. We expected this, as MAE also benefits from addressing the global shape leakage through our decoder and from the scale invariance introduced by variable‑sized global crops. This reveals that some of our contributions extend beyond our framework, and are thus valuable to other SSRL methods.\n\nHowever, when we combine the global invariance learning (CLS) with MAE (CLS + MAE), the performance slightly drops compared to just using CLS. Only when we combine it with our latent MPM (CLS + MPM) we see a synergistic effect emerging, where the model exceeds both objectives in isolation.\n\nWe thank the reviewer for pointing out this particular ablation. We will include the results in the main text.\n### **W6**: Limited evaluation on scene-level tasks\nSimilar to most prior 3D SSRL works, our method currently uses a standard transformer backbone to ensure fair comparison. While widely adopted, this backbone is not ideal for very large scene‑level datasets, which typically benefit from hierarchical designs tailored for sparse and large‑scale inputs. For this reason, we focused our evaluation on the established object‑centric benchmarks (ModelNet40 and ScanObjectNN) that are standard in the 3D SSRL literature.\n\nIt is worth noting that ScanObjectNN is already a challenging real‑world dataset, especially the OBJ_BG and PB_T50_RS splits, which include background clutter, occlusions, and partial scans. We further ran a experiments on Objaverse‑LVIS (see appendix), which spans nearly 1,000 diverse object categories and is substantially more challenging than ModelNet40 or ScanObjectNN. We plan to move these results into the main text and will release the full benchmark setup publicly, with the aim of encouraging wider adoption by future work.\n\nStill, we acknowledge that scaling to large scene‑level point clouds remains an open challenge (also noted as a limitation in the paper), and we are actively exploring ways to adapt AsymDSD for this scenario by using different backbone designs."}, {"review_id": "Fan6p0jcff", "reviewer": "Reviewer_z8Qq", "summary": "The paper proposes the AsymDSD framework, which jointly optimizes global invariance and local mask modeling objectives in the latent space through asymmetric double self-distillation, and adopts strategies such as multi-mask and multi-cropping. Its contribution is to improve the performance of 3D point cloud self-supervised learning, reaching SOTA on datasets such as ScanObjectNN.", "questions": "1.\tThe paper relies on a large-scale mixed dataset of 930,000 shapes (including Objaverse), which makes it difficult for small and medium-sized teams to reproduce the experiment, and the generalization ability on small and medium-sized datasets has not been fully verified. It is recommended to supplement the lightweight pre-training solution on standard datasets such as ShapeNet, such as providing optimized hyperparameters based on 100,000-level data. Open source lightweight model checkpoints (such as AsymDSD-S) to lower the threshold for reproduction.\n\n2.\tIt takes 18 hours to train a single card. The EMA update of multiple masks and teacher models increases the computational overhead, making it difficult to apply to real-time scenarios (such as robot navigation). The impact of parameter quantization (such as INT8) on performance can be verified through experiments, and a lightweight deployment solution can be provided. If the lightweight solution increases the inference speed by 50% while maintaining the Top1 accuracy ≥ 90%, the score can be increased by 1 point. If the computational efficiency is not optimized, 0.5-1 point may be deducted due to practical limitations.\n\n3.\tLatent space prediction focuses on semantic abstraction, but its performance in tasks that require geometric details, such as 3D reconstruction, is unclear (for example, the reconstruction accuracy of Point-MAE is not compared). Comparative experiments on geometric reconstruction tasks (such as ShapeNet partial completion) can be added to verify the model's ability to retain details. Add geometric constraints (such as Chamfer Distance) to the loss function to balance semantic and geometric representations.\n\n4.\tThe experiments focus on synthetic data (ModelNet40) and clean scan data (ScanObjectNN), and lack verification of real sparse point clouds (such as LiDAR). It is recommended to test by adding new real scene datasets: perform object classification experiments on ScanNet (indoor LiDAR scanning), and report the accuracy when the point cloud density is less than 100 points/㎡ (the current method is expected to be 82.5%, and it is recommended to increase it to more than 85% through dynamic masking). Verify the few-shot performance on the KITTI dataset (autonomous driving point cloud), and compare the mIoU difference with Point-MAE in the 5-way 5-shot setting (the current gap is about 3.2%).", "limitations": "Limitations have been included as a part of the paper.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1.\tAn asymmetric dual self-distillation framework AsymDSD is proposed, which cleverly combines mask point modeling and invariance learning, replaces input space reconstruction with latent space prediction, and effectively captures the high-level semantics of 3D point clouds.\n\n2.\tDesign an efficient asymmetric architecture, disable mask query attention to prevent shape leakage, and combine multi-mask sampling with point cloud adaptation multi-cropping to improve training efficiency and representation robustness.\n\n3.\tThe experimental performance is outstanding, reaching 90.53% accuracy in ScanObjectNN, and increasing to 93.72% after large-scale pre-training, surpassing previous methods and having strong generalization capabilities in few-sample scenarios.\n\n4.\tVerify the effectiveness of large-scale pre-training by integrating a mixed dataset of 930,000 shapes, and demonstrate the model's potential for significant performance improvement as the data scale expands.\n\nWeaknesses:\n\n1.\tLarge-scale pre-training relies on a mixed dataset of 930,000 shapes (including Objaverse), which is difficult for small and medium-sized teams to reproduce, and its adaptability to scarce 3D data in real scenes has not been fully verified.\n\n2.\tAlthough the asymmetric architecture optimizes the student side, the EMA update and multi-mask strategy of the teacher model still require high computing power. Single-card training takes 18 hours, which limits lightweight deployment.\n\n3.\tInverse block masking may fail in extremely sparse point clouds or non-rigid objects, and adaptive coverage of semantically critical areas by dynamic masks is not explored.\n\n4.\tLocal cropping (minimum 5% points) does not adequately preserve the semantics of complex geometric structures, and no dynamic sampling mechanism is designed to address point cloud density changes.\n\n5.\tPrioritizing semantic abstraction over detailed reconstruction, they may underperform purely generative models in tasks that require accurate geometry recovery, such as 3D reconstruction.", "was_revised": true, "final_justification": "The author has addressed my concerns about computational complexity and other aspects, and I will increase my score to 4.", "rebuttal": "We thank the reviewer for their detailed feedback and constructive suggestions. Below we address the main concerns point-by-point.\n\n### **W1** and **Q1**: Large scale pre-training\nWe would like to clarify that our main reported results (e.g., 90.53% on ScanObjectNN) are obtained from pretraining on the standard ShapeNet dataset (Sec. 4.1). This setup is fully reproducible on standard consumer-grade hardware (e.g. single rtx 4090) and does not require the large-scale 930K mixed dataset, which was used only for the scaling study.\n\nThe submitted code includes scripts and configuration files for this ShapeNet pretraining setup, and Table 6 in the appendix lists all hyperparameters in detail. We will also release all main pretraining checkpoints (AsymSD‑MPM‑S, AsymSD‑CLS‑S, AsymDSD‑S, AsymDSD‑S*, and AsymDSD‑B*) on HuggingFace to further support reproducibility.\n### **W2** and **Q2**: Compute efficiency concerns\n- **EMA update cost is negligible:** The EMA update itself accounts for only **~12 ms out of 500 ms per iteration**, so it does not significantly impact runtime.\n- **Multi-mask reduces, not increases, cost of the EMA update/teacher:** Multi-mask sampling amortizes the teacher computation over multiple masks, which both reduces memory usage and increases throughput by ~70% (Table 5d), with no loss in accuracy. Multi-mask has no influence on the frequency of EMA updates.\n- **Overall training time is competitive:** Our full pretraining takes **18 hours**, which is similar to several other SSRL methods such as point2vec and significantly faster than ReCon (**54 hours**). Moreover, the method converges quickly, reaching 88% ScanObjectNN accuracy in just 60 epochs (<4 hours) on ShapeNet, already outperforming all other SSRL methods with a standard transformer.\n- **Deployment efficiency:** The encoder that is kept after pretraining is identical in architecture and size to many methods in Table 1, meaning there is no additional inference cost compared to existing approaches.\n- **INT8 deployment** is supported, yielding 87.13% accuracy on ScanObjectNN, which can likely be improved further with quantization-aware training.\n\nWe would like to highlight that our SSRL method is not intended for real-time on-device pretraining. In addition, we view scaling-up as a forward-looking research direction, inspired by developments in NLP and 2D CV.\n### **W4**: Local crops and point cloud density differences\nWhile a minimum 5% crop may seem small, it is intentionally chosen to challenge the model to infer global semantics from fine-grained details rather than relying solely on larger contexts. This can increase robustness to point clouds with large occluded regions or lower point density. This threshold was empirically determined to deliver strong results.\n\n**Robustness to point cloud density:** Our multi-crop strategy keeps the number of point patches consistent while varying the spatial scale of the object. This introduces scale and density invariance that helps the model generalize across diverse geometric configurations and reduces overfitting to features at a fixed scale/density. _Figure 12b_ in the appendix illustrates this effect: in a zero-shot visualization of patch embeddings for _lamp_ objects, the embeddings of the conical lamp covers are homogenous (light green color) across different scales \n### **W5** and **Q3**: Prioritizing semantic abstraction over reconstruction\nOur method is indeed designed to prioritize semantic abstraction, as this is critical for robust representation learning. However, we note that semantic understanding is also essential for point cloud reconstruction, especially with larger occluded regions [a]. Purely reconstruction-based objectives (e.g., Chamfer distance) such as in Point-MAE optimize for minimal point-wise error. While this can lead to reconstructions that are geometrically close to a sampled point set in Euclidean distance, this does not necessarily lead to the most semantically coherent reconstructions.\n\nWe also did not include results on reconstruction benchmarks as other works such as Point-MAE do not quantitatively report on this. However, in Table 3 we show strong part segmentation performance, which often correlates with geometric fidelity. Our results are comparable to Point‑MAE, suggesting that low-level geometric understanding is preserved despite the focus on semantic abstraction. \n### **Q4** Evaluation on real sparse point clouds\nWe agree that evaluating on real scene-level LiDAR data is important. However, our method, like most prior 3D SSRL works, uses a standard transformer backbone, chosen deliberately to ensure fair comparison. While this architecture is widely used, it is not well-suited for very large, scene-wise datasets such as ScanNet or KITTI, which tend to work better with hierarchical backbones. For this reason, we focused our evaluation on the established object-centric benchmarks (ModelNet40 and ScanObjectNN) reported on by most other 3D SSRL methods.\n\nFor a more challenging dataset, we included results on Objaverse‑LVIS in the appendix. This benchmark covers roughly 1,000 diverse object categories and is substantially more challenging than ModelNet40 or ScanObjectNN. We will move these results to the main text and release the full benchmark setup publicly. This might incentivize the adoption of this more challenging benchmark by future works \n\nWe acknowledge that scaling to large, sparse scene‑level point clouds remains an open challenge (also noted as a limitation in the paper), and we are actively exploring ways to adapt AsymDSD for this scenario by using different backbone designs.\n\n[a] deep point cloud reconstruction"}, {"review_id": "I1FYpGD3RS", "reviewer": "Reviewer_3Aup", "summary": "This paper introduces a novel self-supervised learning framework designed for 3D point clouds. \nThe main idea is to unify masked point modeling (MPM) with invariance learning by predicting latent space representations instead of raw input at the patch level for the former, and cross-view distillation for invariance learning.\nThis is somewhat inspired by recent advances in the 2D works, like in I-JEPA.\n\nOn top of that, the authors propose an efficient asymmetric setup for the teacher-student model for improved stability in self-distillation, multi-mask sampling for better computational performance, and a point cloud adaptation of multi-crop that aims to learn a robust local-to-global mapping.\nTogether, these contributions achieve state-of-the-art results on various 3D recognition benchmarks.", "questions": "Would it be possible to compare with other methods like Point-MAE/Point-BERT/Point-GPT under the Linear setting as well? Is there any specific reason why the authors did not run this comparison?\n\nIs there a direct ablation demonstrating that removing self-attention for masked queries is better in terms of performance? I understand this is better in terms of efficiency, but it would be nice to prove this claim also experimentally. \n\nFrom fig. 2 it looks like the transformer encoder for the student has a different architecture than the encoder transformer of the teacher. Is this the case? If so, how to do the EMA for self-distillation?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The paper is clear and well written. I find each contribution well motivated. It is true that some of the proposed contributions seem to be clever combinations of other works, but I believe that the contributions may still be relevant.\n2. The results on scaling beyond ShapeNet are interesting, as large-scale trainings and analysis on big 3D datasets are currently missing in the literature. \n3. Results are strong under ScanObjectNN, which is a less saturated and more realistic benchmark compared to ModelNet40.\n\n\nWeaknesses:\n\n1. My main concern with this paper is that overall there aren’t really unique contributions, but rather ideas taken from several other works and adapted in this context. The idea of using the latent space rather than the point space is by now popular in the 2D world. Also, the adaptation of Dino for point clouds has been already explored in other works such as [a] in the Domain Adaptation literature with the goal of making 3D networks more robust. Although I-JEPA is cited, the latter is not. \n2. In Tab. 1, under the Linear set up, the authors only compare with other methods that do not use the standard transformer as a backbone. It would be better to test with direct competitors under this scenario as well.\n\n[a] Self-distillation for unsupervised 3D domain adaptation, Cardace et al.", "was_revised": true, "final_justification": "The authors have addressed my concerns, thereby I have decided to increase my score.", "rebuttal": "We thank the reviewer for their thoughtful feedback and appreciation of our paper’s clarity, motivations, results, and for recognizing the value of large-scale training and strong performance on ScanObjectNN. We address the concerns and questions point-by-point below.\n### **W1**: Adaptation of ideas\nWhile our work is inspired by progress in 2D vision, we emphasize that it is not a direct adaptation, but rather a non-trivial and novel composition of ideas tailored to the 3D point cloud domain, supported through intuitive and empirical findings. We believe that certain findings may also be valuable to other SSRL methods for point clouds.\n\nWe agree that components such as latent prediction or self-distillation are conceptually present in numerous works on 2D SSRL, with various having a counterpart implementation in the 3D domain (we thank the reviewer for pointing out Cardace et al. [a]). However, these 3D SSRL methods often propose a rather direct adaptation from the 2D domain, thereby not, or partially, addressing the challenges of 3D data.\n\nIn contrast, we developed our method from the ground up, explicitly accounting for these challenges. As such, it cannot be directly reduced to prior methods. For example, previous SSRL methods with dual latent objectives such as iBOT or DINOv2 (for images) are not asymmetric, neither between the student and teacher networks nor between their global and local objective branches, whereas our design introduces asymmetry in both aspects.\n\nWe are also not directly aware of any prior work that considers disabling attention between mask queries at the decoder/predictor level. It is simple and effective, yet works such as Point-MAE or point2vec do not explore this design choice, while they could also benefit from it. \n\n### **W2** and **Q1**: Additional linear evaluations\n*Why did we not run more Linear evals on other SSRL methods with a standard transformer backbone?* We ran into challenges with their codebases, including environment setup (e.g., compiling custom CUDA kernels), errors during execution, and difficulties when porting models into our framework.\n\nHowever, we agree with the reviewer that these are valuable for a direct comparison, especially as a linear probe can be a better indication of the semantic quality of learned representations compared to full fine-tuning. With some extra work, we now have some additional results:\n\n| Method     | ST  | AM  | ModelNet     | OBJ_BG        | OBJ_ONLY      | PB_T50_RS     |\n| ---------- | --- | --- | ------------ | ------------- | ------------- | ------------- |\n| Point-BERT | ✓   | ✗   | 91.09±.15    | 84.17±.30     | 87.19±.16     | 74.44±.12     |\n| Recon MAE  | ✓   | ✓   | 90.22±.09    | 82.77±.30     | 83.23±.16     | 74.13±.21     |\n| point2vec  | ✓   | ✗   | 92.44±.04    | 82.75±.54     | 85.44±26      | 74.25±.11     |\n| AsymDSD-S  | ✓   | ✗   | **92.52±15** | **89.95±.21** | **88.73±.23** | **83.33±.14** |\n\nAs shown above, AsymDSD-S consistently outperforms prior methods. Notably, it improves over the next-best method by +5.8% on OBJ_BG and +9.1% on PB_T50_RS.\n\nWe did not include results of Point-GPT as the linear performance was rather poor, even after some modifications (<70% on ScanObjectNN).\n\nNote that *Recon MAE* is an improved implementation of *Point-MAE* with two standard transformers (two-tower network). Point-RAE also uses a standard transformer, but has an additional attention-based module. We will update Table 1 to reflect these details more accurately: models using a standard transformer backbone will be marked with **ST**, and an additional column **AM** will indicate the presence of extra attention layers/modules.\n\n### **Q2** Ablation on removing self-attention\nTable 5c shows in in the first two rows that the default design with full self-attention (over the visible context c and masks m) decrease performance -2.6% on full fine-tuning compared to the design with cross-attention (from each masked query) onto the visible context c. Figure 13a and Figure 13c in the appendix visualize these two designs.\n### **Q3** Figure 2: encoder shape difference\nThe teacher and student encoder have the same layers and thus the same number of parameters, which we represented through identical lengths of the two encoders in the figure. The width of the encoder rectangle represents the number of patches it process (sequence length). We will clarify this by updating the caption."}]

Author rebuttal:

Meta-review: This paper proposes a new self-supervised method for learning point cloud data representtions, fusing global and local objectives.

Strengths:
- Scalable model with good performance -- great engineering work
- Good experimental validation

Weaknesses:
- The reviewers pointed out several sections where clarity was not optimal; the authors answered these well during rebuttal

The authors engaged with all reviewers during the discussion phase, and the reviewers were generally pleased, several raising their scores -- the paper went from being borderline to a relatively clear accept.
Decision: Accept (poster)