GeoMask3D: Geometrically Informed Mask Selection for Self-Supervised Point Cloud Learning in 3D
Authors: Ali Bahri
Moslem Yazdanpanah
Mehrdad Noori
Milad Cheraghalikhani
Gustavo Adolfo Vargas Hakim
David OSOWIECHI
Farzad Beizaee
Ismail Ben Ayed
Christian Desrosiers
Venue: Accepted by TMLR ()

Abstract: We introduce a novel approach to self-supervised learning for point clouds, employing a geometrically informed mask selection strategy called GeoMask3D (GM3D) to boost the efficiency of Masked Auto Encoders (MAE). Unlike the conventional method of random
masking, our technique utilizes a teacher-student model to focus on intricate areas within the data, guiding the model’s focus toward regions with higher geometric complexity. This strategy is grounded in the hypothesis that concentrating on harder patches yields a more
robust feature representation, as evidenced by the improved performance on downstream tasks. Our method also presents a feature-level knowledge distillation technique designed to guide the prediction of geometric complexity, which utilizes a comprehensive context from
feature-level information. Extensive experiments confirm our method’s superiority over State-Of-The-Art (SOTA) baselines, demonstrating marked improvements in classification, segmentation, and few-shot tasks.

Published in Transactions on Machine Learning Research (03/2025)
GeoMask3D: Geometrically Informed Mask Selection for Self-Supervised Point Cloud Learning in 3D
Ali B2hri Moslem Yazdanpanah 12, Mehrdad Noori!,, Milad Cheraghalikhani Gustavo
Adolfo Vargas Hakim! David Osowicchi Farzad Beizaee122 Ismail Ben Ayed!, Christian Desrosiers
École de technologie supérieure (ÉTS)
International Laboratory on Learning Systems (ILLS)
Reviewed on OpenReview:
Abstract
We introduce a novel approach to self-supervised learning for point clouds, employing a geometrically informed mask selection strategy called GeoMask3D (GM3D) to boost the efficiency of Masked Auto Encoders (MAE). Unlike the conventional method of random masking, technique utilizes teacher-student model to focus intricate areas within the data, guiding the model's focus toward regions with higher geometric complexity. This strategy is grounded in the hypothesis that concentrating on harder patches yields more robust feature representation, as evidenced by the improved performance on downstream tasks. Our method also presents a feature-level knowledge distillation technique designed to guide the prediction geometric complexity, which utilizes a comprehensive context from feature-level information. Extensive experiments confirm our method's superiority over State-OF-The-Ar (SOTA) baselines, demonstrating marked improvements in classification, segmentation, and few-shot tasks. Code is available our GitHub
1 Introduction
086-86
The advent of large-scale 3D datasets (Deitke al. 2023 Slim et al.] 2023) has propelled research deep learning for point clouds, lcading to notable improvements in complex 3D tasks. However, the tine-consuming nature of data collection, com- pounded by the complexity of 3D view variations and the mis- match between human perception and point cloud representation, significantly hinders the development of effective deep networks for this type of data. (Xiao et al. 2023) In response to this chal- lenge, Self-Supervised Learning (SSL) has emerged as promising solution, facilitating the learning of representalions without rely- ing on manual annotations. SSL not only circumvents the issues of costly and error-pronc labeling but also improves the model's generalizatinn ability, offering a pivotal advancement the field of point cloud- based deep learning (Fei et al. 2023).
FS_10-20
ORI-ONLY
FS_10.00
75-750-85
FS_5-30
ModelNet
F5_5-10
Transfomerr
OcCo
Point-MZAF GM3D(our)
REiRt-BERT
Point-MAE
MAEs, simple yet effective self-supervised learners, have gained prominence by learning to recover masked parts of data. This approach has significantly advanced NLP models and has resulted in exceptional vision-based representation learners (Ken- ton & Toutanova,
A relative comparison of the SOTA point cloud MAE methods dif- ferent tasks. Here, the center and the outer circles represent the lowest and highest values on each lask, respectively.
Figure 1:
2019 He et al. 2022 when applied to vision
Correspondence ali.bahri. 1eahri.1@eens.com ".ens.etsmtl.e.

---

Published in Transactions on Machine Learning Research (03/2025)
tasks. Building on their success, MAEs have recently been adapted for point cloud representation learning, leading to SOTA methods including MaskPoint (Liu et al. 2022), Point-Bert Yuu et al. 2022,, Point-MAF. Paan et al. 2022). Point-M2AE (Zhang et al. 2022), 12P-MAE (Zhang et al. 2023), and MAE3D (Jiang ct al. 2023) However, these methods share a common limitation stemming from the random masking strat- egy they use, where masked regions of the point cloud are selected arbilrarily without taking into account their informativeness. As demonstrated 2023), employing a selection strategy that prioritizes informative regions over background areas can signif- icantly enhance the robustness of the learned representation. While such strategy has shown promising results for image processing, its application to point clouds has not been explored so far.
recent work on Masked Image Modeling (MTM) (Wang et al.,
To bridge this gap, we study the use of a targeted masking strategy for point clouds within the MAE framework, applicable across both single and multi-scale methods. We introduce GeoMask3D (GM3D), a novel goometrically-iformeed mask selection strategy for object point clouds. Due to the lack of background data in the object's point clouds, GM3D enables models to concentrate on more complex areas, such as canonical ones with higher connections to the other areas, and pay less attention to the geometrically simple areas like smooth surfaces, as depicted in Fig. 2 To showcase the effectiveness of our method, we integrate it into the pretraining process of both singlo-scale Point-MAE and multi-scale Point-M2AE, the leading MAE methods for point clouds. As illustrated in Fig. I our method exhibits notable enhancements over the earlier SOTA approaches across a range of challenging tasks.
To the best of our knowledge, this represents the first attempt to implement a masking strategy for point clouds, independent of additional modalities such as milti-view images. Our contributions are summarized follows:
1. We propose a novel masking approach for point cloud MAEs, which selects patches based on their geometric complexity rather than selecting them randomly. This approach employs an easy-to- hard curriculum learning strategy where the ratio of patches selected using geometric complexity is gradually increased during training.
2. We also introduce a fcature-level knowledge distillation technique to further guide the prediction of geometric complexily. Instead of relying on the noisy and incomplete information of 3D points, this efficient technique transfers latent features from a frozen teacher model, encoding higher-level information the geometry, to the student model learning the point cloud representation.
3. We integrate these mechanisms into the pretraining process of single-scale Point-MAE and multi- scale PointM2AE, both of which are SOTA point cloud MAEs, significantly enhancing their perfor- mance in diverse downstream tasks.
2 Related Works
Point Cloud Learning. PointNet (Qi et al., 2017ab established point cloud processing as key method in 3D geometric data analysis by addressing the permutation issue of point clouds with a max-pooling layer. To further enhance performance and capture both local and global features, +ointNet+++ (Qi et al., 2017bb introduced a hierarchical structure, expanding the receptive fields of its kernels recursively for improved results over PointNet. Another study (Jaritz et al.) 2019) focused on point cloud scene processing, where multi-view image fcatures are combined with point clouds. In this study, 2D image features are aggrogated into 3D point clouds and a point-based based network fuses these features in 3D space for semantic labeling, demonstrating the substantial benefits of multi-view information in point cloud analysis.
MAE for Representation Learning. Leveraging the success of MAEs in text and image modalities, Point-BERT (Yu et al. 2022) introduced an approach inspired by BERT (Devlin et al. 2018) adapting Trans- formers to 3D point clouds. This approach creates a Masked Point Modeling task, partitioning point clouds into patches and using a Tokenizer with a discrete Variational AutoEncoder (dVAE) to produce localized point tokens, with the goal of recovering original tokens at masked points. Similarly, Point-GPT (Chen et al., 2024) introduced an auto-regressive generative pretraining (GPT) approach to address the unordered nature and low information density of point clouds. ACT (Dong et al.
2022 proposed modal knowledge
2

---

Published in Transactions on Machine Learning Research (03/2025)
highCC
low GC
pgo
100
Figure 2: Visualization of estimated Geometric Complexity (GC) progression throughout training is depicted. The color spectrum denotes GC, ranging from low (Blue) to high (Red), GC values are normalized per object to reflect relative complexity across patches within each object's point cloud. As training progresses (from left to right), initial GC rankings display a random distribution (a). After 100 epochs, the model learns to assign lower complexity rankings to smooth areas (b) and higher rankings complex regions (c). Through GC guided masking, the model increasingly focuses on complex areas from epochs 200 to 300, resulting in a reduction of GC ranking (d) and smoothing of the complexity ranking distribution, accompanied by decrease in total complexity loss LGC (e). Eventually, the model converges to low _ value, consistently targeting canonical patches while maintaining a smoother GC distribution (f).
transfer method using pretrained 2D natural language Transformers as teachers for 3D representation learning. MaskPoint (Liu et al.) 2022), a discriminative mask pretraining framework for point clouds, rep- resents the point cloud with discrete occupancy values and performs binary classification between object points and noise points, showing resilience to point sampling variance. Point-MAE (Pang et al. 2022) adapted MAF-style pretraining to 3D point clouds, employing a specialized Transformer-basel autoencoder reconstruct masked irregular patches and demonstrating strong generalization in various tasks. Following this, MAE3D (Jiang et al., 2023) used Patch Embedding Module for feature extraction from unmasked patches. Point-M2AE (Zhang et al. 2022 introduced Multi-scale MAE framework with a pyramid ar- chitecture for self-supervised learning, focusing on fine-grained and high-level semantics. 12P-MAE (Zhang et al. 2023) further improved the self-supervised point cloud learning process by leveraging pretrained 2D models through an Image-to-Point transformation.
Our proposed method, which can be integrated into any point cloud MAE architecture, differs from previous approaches like Point-MaskPooie Poin, MAE, MAE3D and Point-M2AE that are based on random patch selection. Moreover, unlike recent point cloud learning approaches such as I2P-MAE (Zhang et al. 2023), which rely on image information and 2D backbones, our method only requires 3D coordinates as input.
3 Method
3.1 Preliminaries
Masked Auto Encoders. Autoencoders use an encoder & to map an input X to a latent representation
Z = (X) and a decoder to reconstruct the input as X = D(Z). Masked Auto Encoders (MAE) are a special type of autoencoder that receive a masked-patod input composed of a set of visible patches (with positional encoding) X" and the index set of masked patches M, and reconstruct the masked patches as follows:
[X Xm MAE(X",
M) ,
(((((X"),
(1)
The encoder e and decoder D are both wransfrrme--based networks. The encoder only transforms the visible patches to their latent representations Z" {(X"). On the other hand, the decoder takes input 2" and set of tokens Tw { (t Eposss) i E M } where tmakk is global learnable mask token and (p) is the positional embedding of masked patch i E M. Let = + Nm be the number of patches in the input. with N" = | and N = [M), the masking ratio is defined as m NNNNN.
MAE for Point Clouds. A 3D point cloud P is a set of N points py E 3 For this type of data, patches correspond to possibly overlapping subsets of K points in P. While there are various ways to generate
3

---

Published in Transactions on Machine Learning Research (03/2025)
Input oinnt Cloud
Teacher
Predicted
Selecting patches predicted N
far
Updato
spoch
#poch of
Student
Masked Input based on GC
Peatire-space
D
Prodical
Selected Patches
E
1,00
PatchNumber
FDK Update
Cloud
Selectio of Patches based N
Knowledge Teacher
Backward
sponential Moving Average (E MA)
L
E
Stop Gradient
Distiled
Frozen Network
Used Downstream Tasks
Figure 3: Overview of the GeoMask3D (GM3D) method for self-supervised representation learning in point clouds. The Teacher network predicts Geometric Complexity (GC), and patches with the highest GC. denoted by N selected for masking. The Student network is then trained to reconstruct these masked tokens while simultaneously learning GC through the loss CGC, The reconstruction loss is defined as L ree, + rrey The Teacher network's weights are updated using the Exponential Moving Average (FMA) of the Student's weights, while the Knowledge Teacher remains frozen and is used for gencrating encoder Teatures essential for the Student's training with Creey
patches from point cloud, we follow the strategy employed by several related methods (Pang et al 2022 Zhang et al. 2022) where each patch I; is defined as center point and the set P: C P of K-Nearest Neighbours (KNN) to this center. To uniformly represent the whole point cloud, patch centers are obtained using a Farthest Point Sampling (FPS) algorithm, where first center is randomly chosen from and then the next one is selected as the point in P furthest away from previously selected centers. Assuming that e is inchuded as the first element of its nearest- nearest-neighhor list P., we can represent the patchified version of the point cloud tensor X E NxKx3
In autoencoders for images, pixel-wise L2 reconstruction loss is typically used for training the MAE. In our case, since patches E X are points sets, we instead employ the Chamfer distance to measure the reconstruction eтror eeecy of masked patches:
Prop
Nm Σ
Chamfer(z:", (T).
(2)
where the Chamfer distance between two sets of points S and S' is defined as
Chamfer(S,S)
Σ min IP p'112 + Σ
min
p 1/122.
(3)
P'ES'
PES
PES
E'S
In the next section, we build on these definitions and present our GeoMask3D method for self-supervised point cloud representation lcarning.
3.2 GeoMask3D
In the originally proposed MAE, masked patches are selected randomly during cach itcration without con- sidering the varying impacts that different, patches may have on the training process. This random masking approach may not efficient, as patches in a point cloud can exhibit varying levels Geometric Complexity (GC), which pose different degrees of challenge to the learning network. Inspired by the principles of human Jearning-where where repeatedly tackling challenging tasks enhances performance over time- we propose prior- itizing geometrically complex patches during the pre-training phase of MAE network. A MAE network
4

---

Published in Transactions on Machine Learning Research (03/2025)
achieve more efficient and effective learning by shifting from random masking to strategy focusing complex patches.
This strategy raises a fundamental question: what is Geometric Complexity (GC), and how can it be mea- sured? We define GC for patch as the relative difficulty of reconstructing that patch using an MAE network. Specifically, a patch is considered complex the MAF network demonstrates difficulty reconstructing it, as indicated by higher reconstruction loss 2
To this end, propose GeoMask33 (GM3D) modular component that integrates with any point cloud MAE backbone. This component is incorporated into the pretraining phase of chosen method, shifting from basic naive random masking approach to a selective focus on geometrically complex patches for masking, The architecture of GM3D employs an auxiliary head Dac for predicting the geometric complexity GC E IRN across the input patches, which is trained with a loss LCC,
A tcacher-student framework is utilized to integrate GM3D with a target network. We denote as GM3D" (E*, D*, &ec) the student and as GM3D (E', D P(c) the leacher, both of them having their own encoder, decoder and auxiliary head. In line with (He et al. 2020 apply momentun update method to maintain consistent teacher, updating it in each itcration,
GM3D = G-GM3D' + (1-1).GM3D*
(4)
where μ represents the momentum um coefficient. Both networks predict the GC based on the patch's informa- tional content, as elaborated in Section 3.2.1 We employ the prediction of GC in the masking strategy of the method during its pretraining stage. The GC of the student network (GC*) is predicted based the masked input X", while the GC of the teacher (GC+) is calculated in inference mode using the complete input X:
GC (E(()( Dac (IE(X"),TM))
if t (teacher) if a (student)
The overview of our method for self-supervieed representation learning in the point cloud is depicted in Fig. 3 The GooMask3D (GM3D) approach involves three interconnected steps, which will be explained in the following sections. Additionally, we provide a detailed explanation of the Knowledge- GC strategy in Section 3.3
Knowledge-Dsstilaaionn-- Detis-iiiiatinn-uudeeee
3.2.1 Prediction of Geometric Complexity (GC)
During this stage, our goal is to evaluate GC of cach patch Xm, relative to the others within the same set. We achieve this by using a Dense Relation Comparison (DRC) loss (Wang el al. 2023) which enforces the GC of masked patch pairs (k, I), predicted by the student (i.e., GC$ and GOT), to follow the same relative order their loss values L C
N N
CGC
I log (a(GC) GCF)) In log (1 (GC) -
(6)
where + (C CTe)), In 1 < C,ee), o(-) is the sigmoid function, and ree is detailed in Scction 3.
This loss function enforces consistency between the predicted GC values and re as the ground truth, effectively guiding the student model to learn meaningful ranking of geometric complexities for the masked patches. By comparing all pairs of patches, the loss ensures robust evaluation of relative complexity within _
3.2.2 Geometric- Guided Masking
Patches with a high GC score are typically those that the model struggles to reconstruct accurately (see Fig. 2 This difficulty often stems from their complex geometry, compounded by the absence of color and
5

---

Published in Transactions on Machine Learning Research (03/2025)
background information. While choosing those patches for masking might seem straightforward, there are two challenges to this approach. First, during training, the GC is evaluated by the student for masked patches X yet we need to pick candidate patches from the entire set. Second, the student's 's GC estimation can be noisy, making the training unstable. We address both these challenges by instead selecting patches based on the teacher's score (GC+). Thus, at each iteration, the teacher predicts the GC for all patches X, including unmasked ones. Thanks to the momentum update Fig. 4 the teacher's predictions are more consistent across different training iterations.
3.2.3 Curriculum Mask Selection
During the initial phases of training, the model may struggle to reconstruct fine details and often over- whelmed by the complexity of the point cloud structure. To mitigate this problem, we follow a curriculum easy-to-hard mask selection strategy by starting from pure random masking al the initial training epoch and gradually increasing the portion of geometric-guidee masking until the maximun epoch emaz. Let A E [0, 1] be the maximum ratio of patches that can be selected using GC At each epoch e,, we select the N Lez/emaz x A x N patches with highest GC value, and the remaining N - N ones are selected randomly based on a uniform distribution.
3.3
GC
Instead of relying exclusively on point geometry, our approach employs a knowledge distillation strategy to also learn from latent features. This strategy involves transferring geometric knowledge from a frozen teacher network = (81, D that processes the full set of patches to the student GM3D* observing unmasked patches. This encourages the student GM3D to replicate the feature activations of the knowledge teacher F, indirectly learning from the full structure of data. This unique setup enables the student network to benefit from the global geometric context provided by the teacher network, which is constructed from the complete point cloud. As result, this process facilitates the learning of robust and meaningful representations, which improve performance on downstream tasks. The complexity of patches in the feature space is determined by employing the Mean Square Error loss between the output of gl and the output of D before converting back to point space:
N
C
Σ [x), - D'(LE" (X") (x"))TT)))ll
N
This loss, combined with the Chamfer loss e applied in the point space, serves as the ground-trut loss re for the prediction of GC:
re = roc, + ree,
(8)
The total training loss & is calculated
L a0 GC + rec + Lecc
(9)
where a, 8, and y are apper-parameters.
4 Experiments
Several experiments are carried out to evaluate the proposed method. First, we pretrain both Point-MAE, and Point-M2AE networks utilizing our GM3D approach on the ShapeNet (Chang al., 20155) training datasot. Moreover, we assess the performance of these pretrained models across range of standard benchmarks, such object classification, few-shot learning, and part segmentation. It is important to note that, to maintain a completely fair comparison, we exclusively utilize the encoder of the student network for downstream tasks, ensuring it is identical to the encoder used in the method of interest.
In our approach, we adopt network configurations consistent with those used in the Point-MAE and Point- M2AE models to guarantce fully fair comparison, notably using masking ratios of 60% for Point-MAE and 80% for Point-M2AE. This involves the technique of dividing point clouds into patches, along with employing the KNN algorithm with predetermined parameters for consistent patch uniformity. While our

---

Published in Transactions on Machine Learning Research (03/2025)
autoencoder architecture, including the configuration of Transformer blocks in both encoder and decoder, generally follows the patterns established in these models, we have uniquely tailored the decoder's 's design specifically for the GC estimation purposes. Moreover, the specifies of our network's hyperparameters for the pretraining and fino-tuning phases are comprehensively detailed in the Supplementary Materials.
4.1 Pretraining Setup
We adopt the ShapeNet dataset (Chang et al. 2015) for the pretraining of our technique, in line with the practices established by Point-MAE and Point-M2AE. This datasct, known for its diverse and extensive collection of 3D models across various categories, provides a robust basis for training and evaluation. It contains 57,448 synthetic 3D shapes of 55 categories.
After this pretraining phase, we assess the quality of 3D representations produced by our approach through linear evaluation on the ModelNet40 dataset (Wu et al. 2015). We extract 1,024 points from every 3D model in ModelNet40 and then pass them through our encoder, which remains unchanged during this phase to preserve the learned features. The linear evaluation is performed by a Support Vector Machine (SVM) fitted on these features. This classification performance is quantified by the accuracy metrics detailed in Table [ The results clearly indicate that our technique, when applied to Point-MAE and Point-M2AE, enhances the network's performance.
4.2 Downstream Tasks
Object Classification on Real-World Datasct. In self-supervised learning for point clouds, it is crucial to create a model that exhibits strong generalization abilities across various scenarios. The ShapeNet dataset, which is favored for pretraining, contains clean, isolated object models, lacking any intricate scenes or background details. Inspired by this limitation, and building on prior approaches, we put our mothods the test on the ScanObjectNN dataset (Uy al., 2019), more demanding dataset that represents about 15,000 real-world objects across 15 categories. This dataset presents a realistic challenge, with objects that are embedded in cluttered backgrounds, making it an ideal benchmark for assessing our model's robustness and generalization in real-world scenarios.
We carry out tests on three different variants: OBJ-BG, OBJ-ONLY, and PBT50-RS. It is important to note that we do not employ any voting techniques or data augmentation during the testing phase. The outcomes of these experiments can be found in Table These results demonstrate that integrating the GM3D module with Point-MAE and Point-M2AF, significantly boosts their object classification accuracy this dataset. These findings underscore our method's effectiveness in complex real-world scenarios.
Object Classification on Clean Objects Dataset. For the task of object classification on the Mod- elNet40 dataset (Wu et al. 2015), we evaluated our pretrained models using the same protocols and con- figurations as the Point-MAE approach. ModelNe140,, featuring 12,311 pristine 3D CAD models across 40 categories, was divided into training set of 9,843 models and a testing set of 2,468 models, adhering established norms. Throughout the training, employed common data augmentation stratogics, including random scaling and shifting. To ensure fair comparisons, the standard voling method (Liu et al.] 2019) was also applied during the testing phase. According to Table 4 integrating our GM3D module with Point- MAE has yielded a classification accuracy of 94.20% which surpasses the performance of the standalone Point-MAE and even the more complex Point-M2AE on this dataset.
Few-shot Learning. Following the protocols of earlier studies (Yu et al., 2022 Sharma & Kaul 2020; Wang et al., 2021), we conduct few-shot learning experiments on ModelNeta0 et al. 2015). using an n-way, m-shot configuration. Here, is the number of classes randomly chosen from the dataset, and m is the count of objects randomly selected for each class. The n X m objects are utilized for training. During the test phase, we randomly sample 20 additional unseen objects from each of the n classes for evaluation.
The results of our few-shot learning experiments are summarized in Fig. G. In this highly saturated bench- mark, the combination of the GM3D module exhibits outstanding performance across all tested scenarios. It is worth noting that 12P-MAB(Zhamg et al. 2023) which additionally benefits from multiple 2D views provides
7

---

Published in Transactions on Machine Learning Research (03/2025)
Table 1: Linear evaluation on Model-
Table 2: Ablation study on different maximum hard patch ratios (A). The highest performance is observed at 50% where the OBJ-ONLY score reaches 90.36%.
Net40 (Wu et al., 2015) by SVM.
Method
SVM
MAP-VAR Wang
2019
88.4
VIP-GAN Guo et 2021
90.2
Model
A
OBJ-ONLY
DGCNN Jasaw Yu et al.
2022
90.6
Original Point-MAE
88.29
DGCNN + OcCo
Yu et al.
2022
90.7
Point-MAE+
0.4
89.67
DGCNN + Cross Points oint
Yu
et ai 2022
91.2
Poin--MAE+GM33D
0.5
90.36
Transformer + OcCo
Yu et al.
2022
89.6
Poin-MMAE+GM3D
0.7
89.84
Point-BERT
2022
87.4
Point-MAE Pang
91.05
Point-MAE GM3D
92.30
Point-M2AE (hhang et al.
2022
92.90
Point-M2AE GM3D
93.15
Table Lincar evaluation on Modcl- Net40 (Wu et al., 2015). 'points' and 'Acc' de- note the number of points for training and overall accuracy. grepresents self-supervised pretraining.
ShapeNet- Part (Yi et al., 2016). mloU, (%) and mloU; (%) denote the mean IoU across all part categories and all instances in the dataset, respectively. rep- resents self-supervised pertaining.
Table 3: Part segmentation
Method
Points
Ace (%)
Method
mIoU, mloU;
Point.et
Q et al.
2017a
1k
89.2
PointNet
QU
et al.
2017a
1k
90.5
PointNet Qi al.
2017a
80.39
83.70
&SO-Net
Li et al.
20
5k
92.5
PointNet
Qi et
2017a
81.85
85.10
DGCNN Wang et al.
2019
1k
92.9
DGCNN Wang
82.33
85.20
Point Transtormer
Zhao al
2021
93.7
Transformer
Yu et
2022
83.42
85.10
Transformer
Yu
2022
1k
91.4
Transformer
OcCo
Yu al.
2022
83.42
85.10
& Transformer
OcCo
Yu al.
2022
1k
92.1
Point--BERT
Yu et al.
2022
84.11
85.60
X PPoint-BERR
Yu al.
2022
1k
93.2
812P-MAE
Zhang et al
2023
85.15
86.76
& &PointBBERT
Yu et al.
2022
4k
93.4
Point-GPP-S Chen et al.
2024
84.10
86.2
# Point-BERT
Yu et al.
2022
8k
93.8
BACT Dong 2022
84.66
86.16
& Point-MMAB
Zhang et al.
2022
1k
94.00
Point-MAB
Pang al. GM3D
2022
84.19
86.10
8 Point-
-Point-GPTSS
Chen et
2021
1k
94.00
Point-MMEE
84.49
86.04
E AACT
Dong et al. 2022
1k
93.5
8 412P-MAR Zhang
1k
94.1
Point-MMAE
Zhang
2022
84.86
86.51
0 Pointt
Pang al.
2022
1k
93.80
Point-MMAE
GM33
84.91
86.52
# PPint-MMAE GM33
1k
94.20
only marginal improvements in results. Furthermore, Point-GPT Chen el al. 2024 and ACT (Dong el al. 2022), despite being SOTA and complex methods, show only slight improvements compared to each other and other SOTA methods. Our findings highlight the effectiveness of our method as our Point-MAE+GMMI model has already achieved higher accuracy than single-scale Point-MAE and multi-scale Point-M2AE.
Part Segmentation. Our method's capacity for representation learning was assessed using the ShapeNet- Part dataset (Yi et al. 2016), which includes 16,881 objects across 16 different categories. In alignment with the approaches taken in prior studies (Qi et al. 2017a Yu et al. 2022 sampled 2,048 points from each object to serve as input.
For this highly competitive benchmark, our GM3D method achieves a slight improvement on both the Point- MAE and Point-M2AE networks, as detailed Table Considering that our approach exclusively utilizes 3D information, the observed improvement over methods like I2P-MAE(Zhang et al. 2023) that supplement 3D with additional 2D dala is reasonable, especially considering the slight enhancements achieved by T2P-
8

---

Published in Transactions on Machine Learning Research (03/2025)
Table 5: Object classification on real-world ScanObjectNI dataset (Uy et al., 2019). We evaluate our approach three variants, among which PB-T50-RS is the hardest setting. Accuracy (%) for each variant is reported. @represents self-supervised pretraining.
Method
OB.--GG
OBJ-ONLY
PB-T50-RS
PointNet QQ et al.
20178
73.3
79.2
68.0
SpiderCNN Xu ct al.
2018
77.1
79.5
73.7
PointNet +
Qi et al. 70
82.3
84.3
77.9
DGCNN Wang et al.
2019
82.8
86.2
78.1
PointCNN LL
2018b
86.1
85.5
78.5
BGA-DGCNN
79.7
BGA-PN +
Uy
2019
80.2
GBNet Qiu et al. 2021
80.5
PRANet Cheng et al.
2021
81.0
Transformer
Yu
2022
79.86
80.55
77.24
# "Transformer-O
-eCo OcCo
84.85
85.54
78.79
@ Point-BERT
Yu al.
2022
87.43
88.12
83.07
8
12PPMMAF
Zhang al.
2023
94.15
91.57
90.11
Point-GPP-S CCen et al.
2024
91.6
90.0
86.9
8 ACT
Dong et al. 2022)
93.29
91.91
88.21
8 Point-MAE
Pang al. Point-MAB GM3D
2022
90.02
88.29
85.18
93.11
90.36
88.30
8 Point-M2A1
Zhang et
2022
91.22
88.81
86.43
Point-M2AE GM3D
94.14
90.70
87.64
MAE. Furthermore, Point-GPT (Chen et al. 2024 and ACT (Dong el al. 2022), despite being SOTA and complex methods, show only slight improvements over each other and other SOTA methods. Based on the results of SOTA methods presented in Table 3 it is evident that this datasct is highly challenging and competitive. This highlights the effectiveness our masking strategy in enhancing the understanding of detailed, point-wise 3D patterns.
Table 6: Few-shot classification MedelNet40.
We report the average accuracy (%) and standard deviation (%) of 10 independent experiments. Brepresents self-supervised pretraining.
5-way
10-way
Method
10-shot
20-shot
10-shot
20-shot
DGCNN Wang all
2019
91.8 13.7
93.4 23.2
86.3 26.2
90.9 #s.1
8 DGCNN + OcCo
Wang al.
202
91.9 +3.3
93.9 =3.1
86.4 =5.4
91.3444
Transformer Yu al.
2022
87.8 #5.2
93.3 44.
84.6 45.5
89.4 +6.3
# Transformer
+ OcCo
Yu et al.
2022
94.0 13.66
95.9 12.3
89.4 15.1
92.4 1:4.6
8 Point-oBEEI BERT
Yu et al.
2022
94.6 =3.1
96.3
91.0=54
92.7 +5.1
12P-MAE (Zhang et al.
2023
97.0 =1.8
98.3
92.6 15.0
95.5 13300
8 Point-GGP-S Chen
96.8+24
98.6
92.6
95.2 43.4
8 A.TT Doog 2022
96.8 +2.3
98.0=14
93.3
8 PPonnttNii M2AE Zhang al.
2022
96.8 11.8
98.3 11.4
92.3 14.5
95.0 43.0
8 Poo
96.3 +2.5
97.8 =1.8
92.6 -4.1
95.0 43.00
Point-MAE + GM3D
97.0
98.3 +1.3
93.1 +4.00
95.2 43.6

---

Published in Transactions on Machine Learning Research (03/2025)
Figure 4: Visualization of GC values on diverse point clouds from the ShapeNet dataset CCang al.
2015
4.3 Additional Visualization
Geometric Complexity. Fig. illustrates the GC of randomly selected point clouds from the ShapeNet dataset. This illustration highlights the model's capability to assess GC al the patch level, where the red points denote areas of high GC and blue ones indicate areas of low GC. As mentioned in our methodology section, the model bases the masking process the predicted GC of the patches. Consequently, patches representing regions with higher GC are preferentially masked. This strategic masking induces the model to focus intensively on intricate point cloud regions containing salient geometric information, thereby enhancing its overall performance in tasks requiring nuanced geometric understanding. It is important to note that the provided normalized GC scores are computed relative to the individual patches within each point cloud sample from the ShapeNet dataset. This normalization ensures that the GC scores are reflection of the variation in complexity within given sample, enabling the model internally assess and compare different regions of the same point cloud.
4.4 Additional Analyses
The convergence rates shown in Fig. 5 cleearly highlight the efficiency of our proposed modules. Among the models, the Point-MAE+GMBD model stands out for its fast convergence, reaching high SVM accuracy with fewer epochs compared to the other methods. This quick convergence suggests that the GM3D module helps the model focus on important features in the data more effectively, speeding the learning process.
Pretraining Phase.
The Point-MAE model, while still effective, achieves a lower accuracy and takes more epochs to get there, indicating that it learns more slowly. On the other hand, the GoMs+ version (integrating the GM3D method with Point-MAE alongside eree and CGC) also effective but doesn't converge as quickly the Point- ManttMMAEGM3DD showing the importance of knowledge distillation alongside GM3D module. These findings highlight the practical benefits of adding the GM3D module to the Point-MAE framework. By helping the model learn faster and more reliably, the GM3D module not only improves the model's overall performance but also reduces the time needed to achieve high accuracy.
Fine-tuning Phase. Fig. 6 displays the fine-tuning accuracy on the OBJBG dataset, providing clear evi- dence of the benefits brought by integrating our GM3D module with Point-MAF. The results reveal that the
10

---

Published in Transactions on Machine Learning Research (03/2025)
Accuracy vs Epochs for Different Models
Fine-tuning Accuracy (%) on OBJBG
0.0
0.90
95.45
0.85
SVM Accuracy
Accuracy
0.20
0.49
Point-MAE
From Scratch
DointMAA++MMBR
75
Point-MAE
Seint-MAE+GIIN GM3D
Point-MAF +GM3D
Epochs
Epochs
Figure 5: Comparison of convergence speed during the training phase (Point-MAE).
Figure 6: Fine-tuning vs. Training from Scratch SacnObjectNN (Point-MAE).
MaantMME++aA GM3D model not only achieves the highest accuracy but also maintains this improvement con- sistently over the course of 400 epochs. This consistent performance highlights the stability and effectiveness of the GM3D module in guiding the model to learn more relevant features from the data.
4.5 Ablation Study
Our ablation study focuses the incremental improvements offered by our proposed method GM3D when integrated with the original Point-MAE framework. The original Point-MAE serves as our baseline, using the Chamfer loss for self-supervised learning and setting a performance benchmark on subsequent pretraining and fine-tuning tasks.
GM3D. Initially, integrate the GM3D method with Point-MAE alongside n and LGC_ As reported in Table 7 this combination, termed Montt-MAEGGMBD shows clear improvement over the baseline model by achieving higher pretraining SVM evaluation metrics on ModelNet400 and better fine-tuning results on ScanObjectNN (OBJ-ONLY), This supports the idea that a training focus on more geometrically complex patches contributes to improved model generalization.
Building on this structure, we enhance the performance by incorporating knowledge distillation alongside the GM3D module. The improved model, Point-MAE++MME only outperforms the baseline Point-MAE but also shows further improvement over the Point-MAE- GM3D" approach that utilizes only Tee, and LGC_ This advancement validates the effectiveness of our knowledge distillation strategy, which focuses on accurate reconstruction whilc also capturing the complex geometric interrelations in the data. The various impacts of knowledge distillation are further explored in Table S
which employs three distinct loss functions,
Maximum Hard Patch Ratios. The data presented in Table offer insights into the ablation study focusing different hardness ratios, denoted by A, within the context of point cloud modeling. It is noteworthy that the inclusion of GM3D enhances the performance across different A settings when compared to the original model, with the highest performance observed at a 50% hardness ratio, where the OBJ-ONLY score reaches 90.36%.
Table 7: Comparison of Point-MAE, and Point-MAE-GGM3D on Protraining (SVM) and Fine-tuning (OBJ- ONLY) Tasks. ** stands for our mothod without ree,
SVM Med4
Model
Loss Function
OBJ-ONLY
Point-MAE
Creey
91.05
88,29
Point-MAE GM3D*
L LGC
91.45
89.50
Point-MAE GM3D
Leep + freey CGC
92.30
90.36
11

---

Published in Transactions on Machine Learning Research (03/2025)
Table 8: Ablation study on different components of our method based on Point-MAL
Input -
C
L
GM3D
rreyy
OBJ-ONLY
X
X
a
E1 + D E1 D E D E1 D
89.32
V
90.18
89.67
d
V
V
89.50
V
E1 - D
89.33
f
V
V
E1 E"
89.15
V
A
E1 - D
90.36
Additional Configurations. In Table which details our ablation study, investigate the various configurations of our proposed method. The 'Imput' column pertains to the input utilized by the knowledge- teacher network; it specifies whether complete data X is provided or only partial dała X are used. The second column, denoted by L. encompasses both Cree and _ representing the reconstruction loss func- tions in two spaces. In the third column, we analyze the impact of the chosen loss functions serving as the ground truth for LGC, which is our geometric complexity loss. As can be seen, the performance is enhanced by the geometric complexity guidance, which informed by the feature-level knowledge distillation (rows c, and g). The subsequent column considers the influence of momentum, a parameter linked to the perfor- mance of GM3D'. In the fifth column, we evaluate the impact of implementing L on the interactions between various components of GM3D" and F. As evidenced by the results, each setting has been systemat- ically varied to assess its effect on the final performance metric, OBJ-ONLY, demonstrating the significant contributions of each component to the model's ' learning eflicacy.
5 Conclusion
We presented a ggoometrically-inforeee masked selection strategy for point cloud representation learning. Our GeoMask3D (GM3D) approach leverages teacher-student model to find complex-to-rroonstinee patches in the point cloud, which are more informative for learning robust representations. A knowledge distillation is further proposed to transfer rich geometric information from the teacher to the student, thereby improving the student's reconstruction of masked point clouds. Comprehensive experiments on several datasets and downstream tasks show our method's ability to boost the performance of point cloud learners.
References
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. ShapeNet: An information-rich 3D model repository. arXiv preprint 1eXii:1512.0533 2015.
Guangyan Chen, Meiling Wang, Yi Yang, Kai Yu, Li Yuan, and Yufeng Yue. PointGPT: Auto-regressively generative pre-training from point clouds. Advances in Neural Information Processing Systems, 36, 2024.
Silin Cheng, Xiwu Chen, Xinwei He, Zhe Liu, and Xiang Bai. PRA-Net: Point relation-aware network for 3d point cloud analysis. IEEE Transactions on Image Processing. 30:4436-4448, 2021.
Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram Volcti, Samir Yitzhak Gadre, et al. Objaverso-XL: universe of 10M+ 3D objects. Xixiv prepriat wxXx:23307006000 2023.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirec- tional transformers for language understanding. arXio preprint 1800::1800.044 2018.
12

---

Published in Transactions on Machine Learning Research (03/2025)
Runpei Dong, Zekun Qi, Linfeng Zhang, Junbo Zhang, Jianjian Sun, Zheng Ge, Li Yi, and Kaisheng Ma. Autoencoders as cross-modal teachers: Can pretrained 2D image transformers help 3D representation learning? arxiv preprint arXic:2212.0833 2022.
Ben Fei, Weidong Yang, Liwen Liu, Tianyue Luo, Rui Zhang, Yixuan Li, and Ying He. Self-supervised learning for pro-training 3D point clouds: A survey. arXiv preprint ar (rXi:2905.44000 2023.
Meng-Hao Guo, Jwn- Xiong Cai, Zheng-Ning Liu, Tai-Jiang Mu, Ralph R Martin, and Shi-Min Hu. PCT: Point cloud transformer. Computational Visual Media, 7:187-199, 2021.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVP conference on compuler vision and pallern recognition, pp. 9729 9738,9778--88 2020.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/ /CV conference computer vision and pattern recognition, pp. 16000 16009, 2022.
Maximilian Jaritz, Jiayuan Gu, and Hao Su. Multi-view pointnet for 3D scene understanding. In Proceedings of the IEEE/CVF International Conference Computer Vision Workshops, pp. 0-0, 2019.
Jincen Jiang, Xuequan Lu, Lizhi Zhao, Richard Dazaley, and Meili Wang. Masked autoencoders in 3D point cloud representation learning. IEEE Transactions on Multimedia, 2023.
Jacob Devlin Ming- Wei Chang Kenton and Lee Kristina Toutanova. BERT: Pre-training of deep bidirectional
transformers for language understanding. In Proceedings of naacI-HLLT, volume 1, pp- 2, 2019.
Jiaxin Li, Ben M Chen, and Gim Hee Lee. SO-net: Self-organizing network for point cloud analysis. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9397-9406, 2018a.
Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. PointCNN: Convolution on x-transformed points. Advances in neural information processing systems, 31, 2018b.
Haotian Liu, Mu Cai, and Yong Jae Lec. Masked discrimination for self-supervised learning on point clouds. In European Conference on Computer Vision, pp. 657-675. Springer, 2022.
Yongcheng Liu, Bin Fan, Shiming Xiang, and Chunhong Pan. Relation-shape convolutional neural network for point cloud analysis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
2019.
Yatian Pang, Wenxiao Wang, Francis EH Tay, Wei Liu, Yonghong Tian, and Li Yuan. Masked autoencoders for point cloud superviee learning, In European conference computer vision, pp. 604-621. Springer, 2022.
Charles R Qi, Hao Su, Kaichun Mo, and Lconidas J Guíbas. PointNet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on compuler vision and pallern recognition, pp. 652-600 2206 2017a.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas Guibas. PointNet++:: Deep hierarchical feature learning on point sets in a metric space. Advances neural information processing systems, 30, 2017b.
Shi Qiu, Saeed Anwar, and Nick Barnes. Geometric back-prooection network for point cloud classification. JEEE Transactions on Multimedia, 14:1943-1955, 2021.
Charu Sharma and Manohar Kaul. Self-supervised few-shot learning point clouds. Advances in Neural Information Processing Systems, 33:7212 7221, 2020.
Habib Slim, Xiang Li, Yuchen Li, Mahmoud Ahmed, Mohamed Ayman, Ujjwal Upadhyay, Ahmod Abdel- reheem, Arpit Prajapati, Suhail Pothigara, Peter Wonka, and Mohamed Elhoseiny. 3DCoMPaT++: An improved large-scale 3d vision dataset for compositional recognition. 2023.
13

---

Published in Transactions on Machine Learning Research (03/2025)
Mikaela Angelina Uy, Quang-Hien Pham, Binh-Son Hua, Thanh Nguyen, and Sai-Kit Yeung. Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1588 1597, 2019.
Hanchen Wang, Qi Liu, Xiangyu Yue, Joan Lasenby, and Matt J Kusner. Unsupervised point cloud pre- training via occlusion completion. In Proceedings of the IEEE/CVF international conference computer vision, pp. 9782 9792, 2021.
Haochen Wang. Kaiyou Song, Junsong Fan, Yuxi Wang, Jin Xie, and Zhaoxiang Zhang. Hard patches mining for masked image modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10375-10385, 2023.
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (tog), 3855):1-12, 2019.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3D ShapeNets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1922, 2015.
Aoran Xiao, Jiaxing Huang, Dayan Guan, Xiaoqin Zhang, Shijian Lu, and Ling Shao. Unsupervised point cloud representation learning with deep neural networks: A survey, IEEE Transactions on Pottern Analysis and Machine Intelligence, 2023.
Yifan Xu, Tianqi Fan, Mingye Xu, Long Zeng, and Yu Qiao. SpiderCNN: Deep learning on point sets with parameterized convolutional filters. In Proceedings of the European conference computer vision (ECCV), pp. 87-102, 2018.
Li Yi, Vladimir G Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3D shape collections. ACM Trunsactions Grophics (ToG), 35(6):1-11) 12, 2016.
Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, and Jiwen Lu. Point-BERT: Pre-training 3d point cloud transformers with masked point modeling. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pallern Recognition, pp. 19313 19322, 2022.
Renrui Zhang, Ziyu Guo, Peng Gao, Rongyao Fang, Bin Zhao Dong Wang, Yu Qiao, and Hongsheng Li. P2in- multi-scale masked autoencoders for hierarchical point cloud pre-training. Advances in neural information processing systems, 35:27061-27074, 2022.
Renrui Zhang, Liuhui Wang, Yu Qiao, Peng Gao, and Hongsheng Li. Learning 3D representations from 2D pre-trained models via imagc-to-point masked autocncoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pallern Recognition, pp. 21769 21780, 2023.
Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. Point transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 16259-162222 2021.
14


Reviews:
[{"review_id": "00LLZmAPAg", "reviewer": "Reviewer_LqZs", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. GeoMask3D proposes the geometrically informed mask selection strategy to achieve better performance.\n2. GeoMask3D achieves better performance compared with baseline.\n\nWeaknees:\n1. The written about proposed geometrically informed mask selection strategy should be more detailed, which is the core of this paper.\n2. What does Geometric Complexity represent and how is its GT generated?\n3. The proposed feature-level knowledge distillation is a feature distillation technique, which is widely used in many works. I don't think there are any innovation.\n4. GeoMask3D only conduct all experiments on only simple classification datasets (ModelNet40, and ScanObjectNN) I think more datasets (e.g., S3DIS, ScanNet, KITTI, Waymo) are needed to prove the effectiveness of the proposed method.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper aims to boost the MAE learning of point clouds by the proposed geometrically informed mask selection strategy. Specifically, GeoMask3D first focus on the regions with higher geometric complexity and adopts teacher-student framework to achieve more robust feature representation. GeoMask3D improves the performance of MAE baseline on classification, segmentation, and few-shot tasks", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. The written should be improved to present the proposed geometrically informed mask selection strategy and does Geometric Complexity.\n2.  Authors should provide more experiments on more datasets (e.g., S3DIS, ScanNet, KITTI, Waymo) to prove the effectiveness of the proposed method.\n3. See the part of weaknesses", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Jnuq6Rrhe6", "reviewer": "Reviewer_o5sU", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**:\n1. GeoMask3D foucs on the regions with higher geometric complexity, which makes sense.\n2. GeoMask3D achives superior performance compared with SOTA baseline.\n\n**Weaknees**:\n1. The definition of GT and GC GT is not clear, which makes it difficult to understand.\n2. I suggest that authors provide more detailed ablation studies about Geometric-Guided Masking to prove that the performance gain does not brougth by GC prediction task but by Geometric-Guided Masking.\n3. I think it is not sufficient to compare only PointMAE and PointM2AE. There are many improved MIM methods (e.g., Point-FEMAE [1]).\n4. The feature-level knowledge distillation is a common technique (e.g., MVP [2]). More discussion and experiments are needed to compare MIM of MVP.\n\n[1] Towards Compact 3D Representations via Point Feature Enhancement Masked Autoencoders\n\n[2] MVP: Multimodality-guided Visual Pre-training", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "**Summary**:\nThis paper proposes the geometrically informed mask selection strategy (GeoMask3D) to improve the performance of MAE for point clouds. Specifically, GeoMask3D adopts a teacher-student model to focus on the regions with higher geometric complexity, leading to a more robust feature representation. Finally, GeoMask3D achieves marked improvements in classification, segmentation, and few-shot tasks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Provide more detailed ablation studies about Geometric-Guided Masking to prove that the performance gain does not brougth by GC prediction task but by Geometric-Guided Masking.\n2. Compare more improved MIM methods (e.g., Point-FEMAE).\n3. More discussion and experiments are needed to compare MIM of MVP.", "broader_impact_concerns": "No any concerns on the ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "LetxE9ku6e", "reviewer": "Reviewer_JYxP", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The motivation of this paper is clear. By utilizing 3D geometry information, the paper designs a novel masking strategy. From experiments, this masking strategy seems useful.\n2. The proposed teacher-student framework also seems effective.\n3. The paper is well-formatted and easy to follow.\n4. Extensive experiments are conducted to show the effectiveness of the proposed method.\n\nWeaknesses:\n1. In the experiment part, the description of the teacher model is missing. Which model is used as a teacher?\n2. It seems there are unfair comparisons between the proposed method and Point-MAE. The authors pre-train & fine-tune the model for 400&500 epochs respectively, while Point-MAE trains 300&300 epochs. \nI understand that the proposed method uses an easy-to-hard masking strategy and might take a longer time to converge during pre-training, but I think it is better to finetune the same epochs while making comparisons.\n3. I was curious about the computational resource comparison with baseline methods during pre-training as an additional teacher model is used.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a novel self-supervised method for point cloud processing. The main idea is to mask point patches in an easy-to-hard way and use a teacher-student framework to additionally \"supervise\" the student model. Extensive experiments are conducted to validate the proposed method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see weaknesses 1,2,3.\n\nIn Figure 1, it is better to add scale/number indications.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper focuses on improving the application of the MAE method in 3D perception tasks. The core motivation is that existing random masking does not effectively utilize data information. The proposed GeoMask3D learns better features and enhances the performance of downstream tasks. The clear motivation behind leveraging 3D geometry information is well-executed, and the extensive experiments conducted validate the effectiveness of the proposed method. The teacher-student framework is also a strong addition to the approach. The paper does have some weaknesses, including insufficient detail on the core contributions, such as the geometrically informed mask selection strategy and the definition of key terms like Geometric Complexity and its ground truth. Despite these shortcomings, the paper's overall presentation is clear, and its proposed method shows promise in improving 3D perception tasks. The authors should carefully revise their paper with respect to the above issue in the minor revision so that the paper can reach the standard of publishment.
Decision: Accept (poster)