GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
Authors: Zixiang Ai
Zichen Liu
Yuanhang Lei
Zhenyu Cui
Xu Zou
Jiahuan Zhou
Venue: ICML 2025 poster (2025)
Keywords: Point Cloud Model
Parameter-Efficient Fine-tuning
Prompt Learning
Primary area: deep_learning->foundation_models
TL;DR: We propose a novel Geometry-Aware Point Cloud Prompt that leverages geometric cues to enhance the adaptability of 3D vision models.

Abstract: Pre-trained 3D vision models have gained significant attention for their promising performance on point cloud data. However, fully fine-tuning these models for downstream tasks is computationally expensive and storage-intensive. Existing parameter-efficient fine-tuning (PEFT) approaches, which focus primarily on input token prompting, struggle to achieve competitive performance due to their limited ability to capture the geometric information inherent in point clouds. To address this challenge, we propose a novel Geometry-Aware Point Cloud Prompt (GAPrompt) that leverages geometric cues to enhance the adaptability of 3D vision models. First, we introduce a Point Prompt that serves as an auxiliary input alongside the original point cloud, explicitly guiding the model to capture fine-grained geometric details. Additionally, we present a Point Shift Prompter designed to extract global shape information from the point cloud, enabling instance-specific geometric adjustments at the input level. Moreover, our proposed Prompt Propagation mechanism incorporates the shape information into the model's feature extraction process, further strengthening its ability to capture essential geometric characteristics. Extensive experiments demonstrate that GAPrompt significantly outperforms state-of-the-art PEFT methods and achieves competitive results compared to full fine-tuning on various benchmarks, while utilizing only 2.19\% of trainable parameters.

GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
Zixiang Ai Zichen Liu
Yuanhang Lei?
Zhenyu Cui
Xu Zou
Jiahuan Zhou
Abstract
91
GAPrompi (Ours)
Pre-trained 3D vision models have gained significant attention for their promising per- formance point cloud data. However, fully fine-tuning these models for downstream tasks is computationally expensive and skorage-intensive Existing parameter-efficient fine-tuning (PEFT) approaches, which focus primarily on input token prompting, struggle to achieve competitive performance due to their limited ability capture the geometric information inherent in point clouds. To address this challenge, we propose novel Geometry-Aware Point Cloud Prompt (GAPrompt) that leverages geometric cues to enhance the adaptability of 3D vision models. First, we introduce a Point Prompt that serves as auxiliary input alongside the original point cloud, explicitly guiding the model to capture fine-grained geometric details. Additionally, we present Point Shift Prompter designed to extract global shape information from the point cloud, enabling instance-specific geometric adjustments at the input level. Moreover, our proposed Prompt Propagation mechanism incorporates the shape information into the model's feature extraction process, further strengthening its ability to capture essential geometric character- istics. Extensive experiments demonstrate that GAPrompt significantly outperforms state-of- the-art PEFT methods and achieves competitive results compared to full fine-tuning on various benchmarks, while utilizing only 2.19% of train- able parameters. Our code is available at https: //github. com/ zhou waw.jaaannnne 991/ ICML2025-GAProom
Overall Accurney (%)
Full Fine- Tuming
90
O
89
Point-PEFT
88
IDPT
87
0
1% 2%
3%
4% 5%
6%
100%
Learnable Parameters Total Parameters
Figure I. Our GAPrompt compares to full fine-tuning and existing PEFT methods. We compare the classification accuracy the hardest variant of ScanObjectNN (Uy et al., 2019) based on pre- trained Point-FEMAE (Zha al., 2024).
1. Introduction
The advent of scanning sensor devices has significantly fa- cilitated the acquisition of 3D point cloud data, an inherently irregular and unstructured geometric representation. This advancement has propelled the development of various 3D vision applications, including 3D reconstruction (Xu et al., 2022; Lu al., 2024) and autonomous driving (Zhao et al., 2024). Recently, pre-trained 3D vision models (Yu et al., 2022; Zhang et al., 2022; Zha et al., 2024) have shown remarkable performance in processing point cloud data, en- abling their direct application to a variety of downstream 3D tasks through full fine-tuning. However, fine-tuning the entire pre-traue pre-trained model incurs substantial computational costs and necessitates large quantity of labeled data. More- over, without freezing the pre-trained model, there exists considerable risk of catastrophic forgetting, potentially resulting in the loss of critical pre-trained knowledge.
To address these challenges, parameter-efficien fine-tuning (PEFT) methods have been introduced, particularly 2D vision, improve the efficiency and effectiveness of adapt- ing pre-trained models. The core concept behind PEFT is freeze the pre- rrairri model and only fine-tune newly added modules, thereby bridging the distribution gap between pre-training tasks and downstream tasks while preserving the original knowledge. Two notable PEFT strategies are Prompt Tuning (Lester et al., 2021; Li Liang, 2021; Liu et al., 2024) and Adapter Tuning (Houlsby al., 2019:
Wangxuan Institute of Computer Technology, Peking Univer- sity, Beijing, China "State Key Laboratory of CAD&CG, Zhe- jiang University, Hangzhou, China "School of Artificial Intel- ligence and Automation, Huazhong University of Science and Technology, Wuhan, China. Correspondence to: Jiahuan Zhou <jiahuanzhouG hrttrrhho@@kuu.cc... pku.edu.cn.>.
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
Self-supervised
Frozen
Pre- Pre-training
(a) Fine-tuning
(b) Prompt
(c) GAPrompt (Ours)
Tunable
Point Cloud Inputs
38
Enhance
Tokens
Pre-truining Model
Pre-trmined Model
Pre-trained Model
Pre-truined Model
Backbone
Pre-training Tasks
Downstream Tasks
Outputs
Figure 2. Methods for adapting pre-trained 3D vision models. (a) Fine-tuning updates entire model parameters. (b) Prompt-based methods adapt the model to downstream tasks by reformulating the input at the token level. (c) Our proposed GAPrompt adapts the model by tuning explicit learnable point clouds and token prompts, enhanced with instance-specific shape fcatures extracted by geometry-aware prompter.
He et al., 2021). However, the transition of these PEFT methods from 2D to 3D vision poses sigmificant challenges due to the inherent sparsity and irregularity of point clouds. Specifically, token prompts initialized randomly often fail to align well with point cloud data, leading to difficulties in convergence when downstream tasks are supervised solely by prediction loss. Similarly, Adapter Tuning, which pri- marily focuses on token features, struggles to capture the critical geometric information embedded in the distribution of discrete points within 3D space.
we propose a Prompt Propagation mechanism, which seam- lessly integrates the instance shape information extracted by the Point Shift Prompter into the model's feature extraction process. This design enhances the model's ability to capture and utilize critical geometric information from the point cloud, leading to more accurate and efficient processing.
In summary, the key contributions of this work are: (1) We propose GAPrompt, novel geometry- prompt learning method tailored for pre-trained 3D vision models. GAPrompt achieves competitive performance comparable to full fine-tuning, while significantly reducing computational and storage overhead. (2) We introduce three key algorithm designs including Point Prompt, Point Shift Prompter, and the Prompt Propagation mechanism, which together enable the model to effectively capture and utilize geometric in- formation inherent in point clouds, thereby enhancing its representational capacity. (3) Extensive experiments on var- ious benchmarks demonstrate the superior efficiency and effectiveness of GAPrompt, outperforming existing methods in both accuracy and resource utilization.
Recently, several studies (Zha et al., 2023; Zhou et al., 2024) have recognized the critical need for 3D-specific PEFT methods and have initially tried to design networks that are better suited for adaptation within the 3D domain. These approaches typically focus either constructing complex networks to model token interactions and dynamically gen- erate token prompts, or employing dynamic adapters that simultaneously produce both prompt tokens and scaling fac- tors for adapter tuning. However, by concentrating primarily on encoded input tokens, these methods fail capture the rich geometrie information intrinsic to point clouds, which severely limits their ability to achieve competitive perfor- mance, as illustrated in Figure 1.
2. Related Work
2.1. Pre-trained 3D Vision Model
To this end, we propose a novel Geometry-Aware Point Cloud Prompt (GAPrompt), specifically designed for parfmeter-efficient fine-tunn totnttuning of 3D models. Our approach begins with the introduction of Point Prompt, which ex- plicitly incorporates point cloud data as input, allowing the model to better capture subtle geometric features. To further enhance the capacity of leveraging instance-specific geome- try information, we introduce a Point Shift Prompter. This module extracts global shape information from the original point cloud and shifts the points accordingly, thereby en- riching the geometric features at the input level. In addition,
Pre-training on 3D datasets has become a prominent re- search area, particularly with the use of vision transform- ers (Dosovitskiy et al., 2021). Two principal pretext task paradigms have been developed for 3D pre-training: con- trastive learning and mask modeling. Methods based on contrastive learning (Huang et al., 2023: Dong et al., 2022; Qi et al., 2024; Zhu et al., 2023) have demonstrated re- markable performance in zero/few-shot learning, largely due to the inherent power of multi-modality. Mask mod- eling (Yu et al., 2022; Zha et al., 2024) typically relies on
2

---

GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
autoencoders to learn the latent features by reconstructing the original input as illustrated in Figure 2. For instance, Point-MAE (Pang et al., 2022) employs autoencoder to learn high-level latent features from unmasked patches and reconstruct the masked point patches. ReCon (Qi et al., 2023) utilizes an ensemble distillation approach, drawing upon both generative modeling teachers and single/cross- modal contrastive teachers. These pre-trained 3D vision models have demonstrated exceptional capabilities when fully fine-tuned for various downstream 3D tasks.
and bias tuning (Zaken et al., 2021) to transfer pre-trained models. Nevertheless, it still relies on point priors extracted from pre-training data, further adding computational over- head.
While these pioneering approaches have made significant strides, they often lack an explicit understanding of the geo- metric characteristics inherent to point clouds, which limits their overall performance. To address this, we propose a novel 3D-specific PEFT method with geometry awareness, termed GA Grorompt. Our approach leverages a lightweight prompter to extract instance-specifee shape information from point clouds. Furthermore, we integrate explicit learnable point cloud prompts with token prompts to enhance the model's geometric awareness and improve its ability to cap- ture subtle geometric details, as illustrated in Figure 2.
While full fine-tuning can yield promising performance, it is computationally expensive and inefficient, as it requires up- dating the entire backbone of the model. This has motivated the exploration of more efficient methods for adapting pre- trained models to downstream tasks with minimal parameter updates. In this work, we address this challenge by propos- ing geometry-awan prompting approach that leverages instance-speciie shape information extracted from input point clouds, enabling efficient transfer of pre-trained mod- els to downstream tasks with significantly fewer trainable parameters.
3. The Proposed Method
We illustrate our GAPrompt method for efficiently fine- tuning pre-trained 3D vision models in detail, which con- tains three geometry-aware prompt modules: the Point Prompt, the Point Shift Prompter, and the Prompt Propaga- tion mechanism. As shown in Figure 3, given a pre-trained 3D transformer with N blocks and a specific downstream task, we freeze the backbone and solely update newly intro- duced GAPrompt modules and the classification head.
2.2. Parameter- Efficcien Fine-Tuning
As deep learning technology advances, both the perfor- mance and size of models have steadily increased, mak- ing full fine-tuning of these models for downstream tasks computationally expensive. To address these challenges, researchers in 2D computer vision have developed various Parameter-Efficient Tuning (PEFT) methods. One pop- ular approach, prompt tuning (Jia et al., 2022; Li & Zhou, 2025), introduces learnable latent tokens as prompts to task- specific inputs, allowing pre-trained models to adapt within the latent feature space. Adapter tuning methods (Houlsby et al., 2019) complement this by inserting lightweight mod- ules into the pre-trained model blocks and adjusting the latent feature distribution. Building on these foundations, numerous variations (Jie & Deng, 2023; Karimi Mahabadi et al., 2021) have been proposed, achieving performance levels comparable to full fine-tuning in 2D vision tasks.
3.1. Point Prompt
To facilitate explicit awareness of point clouds and assist models in capturing subtle geometry, we design the Point Prompt which explicitly utilizes learnable point clouds as prompt. It has stronger inherent relevance with point cloud data than prompt tokens, assisting models in capturing ge- ometry from inherently irregular and unstructured point clouds. The Point Prompt P RPx3 is initialized in uni- form distribution:
2 ~ U(-r,+T).
(I)
where 2 is the coordinates on the z-axis, and the other two dimensions are the same distribution. r represents the range of a single dimension of coordinates.
However, due to the inherent sparsity and irregularity of point clouds, these 2D vision PEFT methods struggle to generalize effectively to 3D vision tasks, underscoring the need for 3D-spccffic approaches. For example, IDPT (Zha et al.. 2023) utilized a heavy EdgeConv (Phan et al., 2018) network to capture local interactions between tokens, dy- namically generating token prompts. While this method narrows the performance gap with full fine-tuning, it signif- icantly increases computational costs. DAPT (Zhou al., 2024) introduced dynamic adapters for transfer learning in 3D vision models, extending standard adapters to produce both scale factors and prompts d dynamically. Additionally, Point-PEFT (Tang et al., 2024) combined prompts, adapters,
Given raw input point cloud E Sx3 with S points, firstly we hybrid Point Prompt P E Rx3 into its 3D space, denoted as [x; Pl E (RSS++)) P)x3, where "[]" indicaaes con- catenation and P is the number of learnable points. Ad- ditionally, we modify the shape of x with our Point Shift Prompter, producing a shifted point cloud E Rx3 This module also generates instance-speeific informative shape features f E DR, where D is the embedding dimension of transformers, formulated as:
x. f = Point-Shift-Per (hiiittt-rrrmpte((a)))
(2)
3

---

GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
Point Prompt
Shifted Input
Prompted Input
Encoder Block
XN
Frozen
Prompt Propagation
Tunable
Token Embedding
0000
Downstrean Head
Self Allentinn
Adapter
Raw Input
Shift Head
Raw Prompt Token
FFN
?
C
Shape Feature
Enhanced
Prompt Token
Enhance
FPS&KNN
Prompl Inject
Peature Propagation
Multi-resolution Grouping
Feature
Propagation
Eg Eq 14
E
Eq.16
Remupping
Geometry-Aware Point Shift Prompter
Geometry-Awan Prompt Propagation
Input Token
O O Enhanced Prompt Token
Propagated Input Token
O Local Conter
Local Neighbor
Figure The overall pipeline GAPrompt. raw input point clouds are processed by Point Shift Prompter, generating instance- specific shape features and shifted points. These shifted points then combined with the Point Prompt and embedded into input tokens. The shape features are further utilized to enhance the prompt tokens, which are concatenated with the input tokens. Subsequently, the concatenated tokens fed into Prompt Propagation, incorporating the instance shape information into the model's feature extraction process. Finally, the propagated tokens are passed through pre-trained model blocks to produce results.
Then the hybrid point cloud (z;; P] becomes prompted input point cloud [ P| E ()x+3
3.2. Point Shift Prompter
In addition to employing Point Prompt capture subtle ge- ometric details, utilize Point Shift Prompter to extract informative shape features from raw input point clouds. As described above, to adjust the point cloud shape accordingly, we pass point-level shape features through a shallow shift head to generate a coordinate shift for each input point. Fur- thermore, the shape features can be utilized to enhance the geometry awareness of prompt tokens and adapters.
Following the original architecture of the pre-trained model, the prompted point cloud is encoded into L. point tokens h1 by the token embedding module. We denote input tokens of i-th block as h, E RL.DD, After that, input tokens h; concatenated with Lp prompt tokens p, RR DD hanced by shape feature f, formulated [h,; P:]. Then feed these tokens into our Prompt Propagation mechanism, injecting prompt tokens into the feature extraction process:
Shape Feature. Specifically, acquire global shape in- formation of point clouds without much computational cost, we utilize hierarchical downsampling strategy. As shown in Figure 3, the raw input point cloud æ is sampled by nuutti-resolution grouping referring to PointNet++ (Qi et al., 2017b), which iteratively finds out C, center points xy C x3 via farthest point sampling (FPS) at j-th reso- lution level. Then respectively find out neighbor points nj K3 corresponding to each center with K- nearest neighbor (KNN) algorithm:
h PrompptPropaggtioon(t..
where fú; E xL the propagated input tokens. Finally, feed h, with enhanced prompt tokens p, into vision transformer attention layers, consisting of self-attention and feed-forward layers. Furthermore, we adjust the tokens with adapters enhanced by shape feature f.
h4 P. (h.,
(4)
Fj+1 = FPS(2;))
(6)
hi+1 h; Ah; f Ba),
(5)
n, LNN(m,,.25+1)
where fi, P, E XD are intermediate outputs of attention layer and 3, is scale factor for enhancing adapters. In the following parts, we will demonstrate the details of Point Shift Prompter and Prompt Propagation respectively.
Obtaining such hierarchical spatial information, we em- bed point coordinates xy to features d, RC xD, with a lightweight pointnet (Qi et al., 2017a), which is combina-

---

GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
tion of convolutional layers, formulated as:
2014), the prompt injection process makes our model more robust while adapting downstream. A variety of replacement methods for alternation will be discussed in detail in the Ablation Study. We formulate this process as:
d Pointnet(zz)).
(8)
After & levels of downsampling, we obtain center point fea- tures dg E x where C& D2 D and concatenate
h h = Inject (hi, h..
(15)
them as shape feature f.
where h E RCXD K E RCxX*D denote tokens indexed by centers and neighbors and h, h, denote tokens after injection. Then we propagate features from local centers to all input tokens, formulated as:
f Reshape(d,) E R
(9)
which is the overall shape information of input point clouds. Shifted Input. Furthermore, to generate instance-specific shifts for each point, we need to obtain features for all the original points æ. Firstly, an upsampling strategy is em- ployed to propagate features from center points to neighbor points. Then we further process the features with another pointnet:
h; = Propagate(h""),
(16)
where hi is propagated input tokens. More details of Eq. 15& 16 can be found in Appendix.
The core of our method lies in the strategic injection of enhanced prompt tokens. Without this enhancement, the propagation process among imput tokens would result in triv- ial solutions, yielding minimal performance improvements. Specifically, when the features of center points are provided, and the goal is to compute the features of neighboring points, our propagation mechanism performs feature interpolation. This interpolation leverages the spatial distances between the center and neighboring points, allowing the model to effectively transfer and refine feature information across the tokens, thereby ensuring performance gains.
dj Pointne((ropagaaee(]]))
(10)
where dÿ E C, is features of neighbor points ny in j-th level. After upsampling from level & to level 1, the
feature propagation process yields d". We concatenate d" and d, and send into small MLP, namely shift head, to generate the z:
T Shift-Head((dd d.))-
(11)
Besides, the shape feature f E R can be used enhance prompt tokens, imparting rich geometric awareness:
3.4. Analysis and Discussion
The objective of our method is to facilitate task-specific model adaptation through the integration of geometric- aware prompt mechanisms. In contrast to previous ap- proaches, our method not only incorporates prompt tokens but also effectively captures fine- aee-grained point-level geo- metric information. The attention mechanism with prompt integration can be formally expressed as follows:
P = p/- + f 3p.
(12)
where pt E RL,xD denotes raw prompt tokens, Pt E R xD is enhanced prompt tokens, 3p is scale factor.
3.3. Prompt Propagation
O AWah, Wghh,, Wyhh),
(17)
With prompt tokens enhanced by global shape features from the Point Shift Prompter, we further leverage these tokens to enhance the model's geometry awareness. To achieve this, we design a Prompt Propagation mechanism that integrates instance-specific shape information into the model's feature extraction process.
6. = Attn (Wih., Wx Wx(p.,hd) Wv hi),
(18)
where o, and ô; represent the attention outputs without and with prompt integration.
Building upon the theory established by (He et al., 2021), we can derive an equivalent transformation of Eq. 18 as:
Given a set of input tokens P,] E (x(++)))))) in the i-th block, we can find the geometric relationship between input tokens via FPS and KNN. We formulate it as follows:
6, Σ AuWwp' (11- E Anjo,
(19)
P*EP:
h = FP.().))
(13)
where Ay denotes the attention weight assigned to prompt pd for query h; by the transformer. This formulation reveals two critical aspects: the soft prompt mechanism induces a linear interpolation of the head's position- output, and the bias term facilitates adaptation within an offiset subspace.
h KNN(H,, h.))
(14)
Subsequently, we randomly replace the enhanced prompt tokens p, into input tokens in h and h," namely prompt injection. c and n denote the index for center and neighbor points. Inspired by the thought of dropout (Srivastava et al.,
The key distinction of our approach lies in the point-level operation, addressing the limitations of previous prompting
5

---

GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
Table 1. Classification on three variants of the ScanObjectNN and the ModelNet40, including the number of trainable parameters (Param) and overall accuracy (Acc). We report SeanObjectNN and ModelNet40 results without voting.
ScanObjectNN
ModelNet40
Method
Reference
Param.(M)
FL.OPs(G) 4
ORJ BG
OBJ ONLY
PB T50 RS
Acc. (%) 1
Full Fine-Tuning
OcCo
ICCV 21
22.1
4.8
84.85
85.54
78.79
92.1
Point-BERT
CVPR 22
22.1
4.8
87.43
88.12
83.07
92.7
MaskPoint
ECCV 22
22.1
89.70
89.30
84.60
93.8
Point-MAE
ECCV 22
22.1
4.8
90.02
88.29
85.18
93.2
Point-M2AE
NeurIPS 22
15.3
3.6
91.22
88.81
86.43
93.4
ReCon
ICML 23
43.6
5.3
94.15
93.12
89.73
93.9
PointGPT-L
Ne 23
360.5
67.7
97.20
96.60
93.40
94.1
Point-FEMAE
AAAI 24
27.4
5.0
95.18
93.29
90.22
94.0
Efficient Fine- Tuming
Point-MAE
CVPR 22
22.1()00%)
4.8
90.02
88.29
85.18
93.2
+IDPT
ICCV 23
1.7(7.69%)
7.2
91.22(+1.20)
90.02(+1.73)
84.94(-0.24)
93.3(+0.1)
+DAPT
CVPR 24
1.1(4.97%
5.0
90.88(+0.86)
90.19(+1.90))
85.08(-0.10)
93.5(+0.3)
+Point-PEFT
AAAI 24
0.7(3.17%)
7.0
89.33(-0.69)
88. (8.98(+0.69)
84.42(-0.76)
94.2(+1.0)
+GAPrompt
This Paper
0.6(2.711
5.0
91.91(+1.89)
90.19(+1.90)
85.57
04.2(+1.00)
ReCon
ICML 23
43.6(100%)
5.3
94.15
93.12
89.73
93.9
+IDPT
ICCV 23
1.7(3.90%)
7.2
93.29(-0.86)
91.57(-1.55)
87.27(-2.46)
93.4(-0.5)
+DAPT
CVPR 24
(2.52.)%)
5.0
94.32(+0.17)
92.43(-0.69)
89.38(-0.35)
93.5(-0.4)
+Point-PEFT
AAAI 24
0.7(1.61%
7.0
92.94(-1.21)
91.57(-1.55)
89.07(-0.66)
93.8(-0.1)
+GAPrompt
This Paper
0.6(1.38%
5.0
94.49(+0.34)
92.60(-0.52)
89.76(+0.03)
94.0(+0.1)
PointGPT-L
NeurIPS 23
360.5(100%)
67.7
97.20
96.60
93.40
94.1
+IDPT
ICCV 23
10.0(2.77%)
75.2
38.11(40.911
96.04(-0.56)
92.99(-0.41)
93.4(-0.7)
+DAPT
CVPR 24
4.2(1.17%)
71.6
98.11(+0.91)
96.21(-0339
93.02(-0.38)
94.2(+0.1)
*Point-PEFT
AAAT
3.1(0.86%
73.2
97.76(+0.56)
96.21(-0.39)
93.11(-0.29)
93.9(-0.2)
+GAPrompt
This Paper
2.0(0.55%)
71.8
98.97(+1.77)
96.73(+0.13)
94.31(+0.91)
96.2
Point-FEMAB
AAAI
27.4(100%)
5.0
95.18
93.29
90.22
94.0
+IDPT
ICCV
1.7(6.20%)
7.2
92.94(-2.24)
90.88(-2.41)
88.38(-1.84)
93.4(-0.6)
+DAPT
CVPR 24
1.1(4.01%
5.0
93.98(-1.20)
92.25(-1.04)
88.51(-1.71)
93.2(-0..)
-Point-PEFT
AAAI 24
0.7(2.55%)
7.0
94.32(-0.86)
92.94(-0.35)
89.35(-0.87)
94.3(+0.3)
+GAPrompt
This Paper
0.6(2.19%
5.0
95.53(+0.35)
(3.63(+
90.67(+0.45)
94.5(+0.5)
methods that primarily operate at the token level. While ex- isting methods often struggle to capture fine-grained geomet- ric features due their token-level adaptation constraints, our Point Prompt and Point Shift Prompter mechanisms directly modulate h; in Eq. 17 and Eq. 18, enabling precise latent space adjustments at the point level. This fundamental difference in operational granularity contributes to enhanced geometric feature representation and more effective model adaptation for point cloud models.
FEMAE (Zha et al., 2024) as baselines. For a fair com- parison, we employ identical data augmentation to the full fine-tuning method for each baseline. The hyperparameters are set as 3, 0.5, 3p 0.5 and P 20. Additional analyses of 3, and P can be found in the Appendix.
4.1. Experimental Settings
ScanObjectNN. The ScanObjectNN (Uy et al., 2019) is highly challenging 3D dataset comprising 15K real-world objects across 15 categories. These objects consist of indoor scene data obtained by scanning, exhibiting characteristics such as cluttered backgrounds and occlusions. As demon- strated Table 1, we conducted experiments on three vari- of ScanObeectNN, each with increasing complexity. Note that experiments on dataset ScanObjectNN sample
4. Experiments
We evaluate the performance of our proposed GAPrompt on the point cloud classification task. We utilize four pre- trained models Point-MAE (Pang et al., 2022), ReCon (Qi et al., 2023), Point-GPT (Chen et al., 2024) and Point-
6

---

GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
Table Comparisons PEFT methods from NLP and 2D Vision on the hardest variant of ScanObjectNN,
Table 3. The effect of components our GAPrompt.
Point Prompt
PS-Prompter
Prompt Propagation
Acc.
Method
Param. (M)
Acc.
87.85
89.34
Point-MAE
22.1
85.18
90.67
Linear Probing
0.3
75.99
Table 4.
Ablation study on Point Shift Prompter.
Prefix Tuning
0.7
77.72
VPT
0.4
81.09
Shift Head
Prompt Enhance
Adapter Enhance
Acc.
Adapter Tuning
0.9
83.93
88.23
LoRA
0.9
81.74
89.71
SSF
0.4
82.58
90.67
AdapterFormer
0.9
83.45
even with less than 3% trainable parameters and less than 0.IG FLOPs of computational increment, which verifies the efficacy and efficiency of adopting geometry informa- tion for prompting even on noiseless point clouds. Notably, our GAPrompt achieves the state-of-the-art performance of 96.2% % based on Point-GPT, with basic scale-and salle-and-translan translate augmentation and no voting.
GAPrompt
0.6
85.57
2048 points as input for each point cloud, consistent with previous works (Wang et al., 2021; Liu et al., 2022).
ModelNet40 (Wu et al., 2015) comprises 12,311 pristine 3D CAD models across 40 categories, with complete, uniform, and noise-free point clouds that simplify the task. Following baselines, we sample 1024 points per instance. Since voting (Liu et al., 2019) is time-consuming, we focus on reporting overall accuracy without it.
ModelNet40.
Comparison to PEFT Methods. To further illustrate our superiority, we compare GAPrompt with several PEFT ap- proaches (Chen et al., 2022; Lian et al., 2022) from NLP and 2D vision. As shown in Table 2, we select the most sophisticated PB_T50_RS variant with Point-MAE as the baseline. Although these basic methods show different im- provements over linear probing, there is still a considerable performance gap to full fine-tuning due to the irregularity and complexity of point cloud structures. Our method with 3D 3D-specific designs excels VPT and Adapter respectively by 4.48 % and 1.64% % attributed to instance-specie shape features integration into the feature extraction of pre-trained models. Besides, our GAPrompt even has comparable train- able parameters against these basic methods, attributed to the compact design of Point Shift Prompter which directly extracts geometry features from point clouds
4.2. Quantitative Analysis
Performance ScanObjectNN, As shown in Table 1, our method GAPrompt achieves the highest accuracy among all the parameter-efficient fine-tuning methods for 3D vision models. Furthermore, we even surpass the full fine-tuning of Point-MAE, ReCon, Point-GPT, and Point-FEMAE by 1.89%, 9 0.34% 4 1.77%, 0.35% on OBJBBG variant of ScanObjectNN respectively, and reduce over 97% train- able parameters. It is attributed to our GAPrompt, which captures instance-specifee geometry information of original point clouds and effectively integrates it into the feature extraction process of pre-trained models. In contrast, IDPT, DAPT, and Point-PEFT fall short of full fine-tuning perfor- mance due to their limited ability to capture geometric infor- mation from point clouds. Moreover, our method stands out in both efficiency and computational cost. With just 0.6M trainable parameters, our GAPrompt requires far fewer than IDPT and DAPT. In terms of FLOPs, our approach adds vir- tually no extra computational burden compared to baselines, significantly outperforming IDPT and Point-PEFT. This can be credited to our lightweight Point Shift Prompter and the parameter-free Prompt Propagation mechanism.
4.3. Ablation Study
We conduct ablation studies on the most challenging PB_T50_RS variant based on Point-FEMAF investigate the rationalization and effectiveness of our GAPrompt.
Analysis on Main Components. As shown in Table 3, to quantify the contribution of each main component, we incre- mentally conduct ablation experiments until the complete GAPrompt scheme. Solely introducing the Point Prompt, the performance can reach 87. 87.85%, for facilitating explicit awareness of point clouds and assisting models capturing subtle geometry. Then appending Point Shift Prompter leads to a 1. 1.89% increment due to capturing instance-speciff ge- ometry information. Finally, the utilization of Prompt Prop- agation mechanism boosts the result to 90.67%, surpassing the full fine-tuning, which is attributed to the integrating en- hanced prompts into feature extraction of pre-trate pre-trained model.
Performance on ModelNet40. As shown in Table 1, although the result in ModelNet40 is almost saturated, our GAPrompt still excels over previous works due to the rastance-specifie shape features extracted from point clouds and explicit awareness of point clouds. Moreover, GA GAPrompt gains positive increments on all four baselines

---

GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
91.0
Full Fine-tuning
90.67
90.5
98-48t
Aceuracy (%)
90.0
90.13
91.22
00.02
90.22
$9.83
19.75
89.77
89.56
89.5
W9:71
49.44
89.52
89.0
Before Self-am..
89.20
$8.44
GAPrompt
58.51
Rep. After Self atn..
088662
88.5
Per. Before Self-atm.
*
Per. Afer Self-atm.
A88.31
Airplane
Plant
Lamp
Flower Pot
Prompt Enhancing Factor 3p
Figure 5. Visualization of attention position of GA Prompt and full fine-tuning. We visualize the attention scores of the [CLS] token to other point cloud tokens.
Figure 4. Ablation study of Prompt Propagation mechanism and prompt enhancing factor/ Sp.
Effect of Point Shift Prompter Components. As shown in Table 4, we evaluate the effectiveness of each component in our Point Shift Prompter in an incremental manner. Basi- cally, we only use the Shift Head to produce shifted point clouds as input for the encoder, attaining 88.23% accuracy. Furthermore, respectively enhancing Prompt and Adapter with extracted shape features boosts the performance to the peak. This increment is attributed to the instance-specifi shape features improving the geometry awareness of token prompts and adapters.
Raw Point Cloud
Shifted Point Cloud
Analysis on Settings of Prompt Propagation and 3p. As shown Figure 4, we conduct ablation experiments on prompt propagation settings and prompt enhancing factor 3p- Our prompt propagation mechanism has two alternations, operating before or after self-attention layers, denoted as 'Before Self-attn.' and 'After Self-attn.'. Furthermore, we design two different prompt injection options, which are realized by directly replacing and indirectly permutation. The combination of these choices produces the four curves in Figure 4. It is clear that utilizing permutation after self- attention layers and 0.5 3, obtains the peak performance. Intuitively, it is because this setting brings more randomness and results in more robust convergence.
Toilet
Bed
Table
Figure 6. Visualization of point clouds before and after transfor- mation by the Point Shift Prompter. The samples drawn from the test split of the ScanOhjeatNN dataset, demonstrating its broad generalization capability across unseen data.
We can see that point clouds shifted by Point Shift Prompter tend to be more compact and exhibit sharper boundaries, making them easier recognize, especially in the regions highlighted by the rectangle. This suggests that the Point Shift Prompter can enhance the geometric features of the point cloud at the input level, thereby contributing to im- proved performance. Additionally, the point cloud contains learnable point prompts, which tend to move the inner space of point clouds during training.
Visualization of A Attention Position. In Figure 5, we respec- tively visualize the attention positions of the [CLS] token of GAPrompt and full fine-tuning, where the warm color indicates higher values. As illustrated, the [CLS] token of GAPrompt captures more essential 3D semantics, such the head and vertical stabilizer of the plane, the stand of the lamp, and the pot of the flower pot. But the [CLS) token of full fine-tuning merely grasps the vertical tails and the base of the lamp. It indicates that with inherent geometric clues in point clouds, GAPrompt effectively grasps the critical information and further benefits point cloud understanding.
5. Conclusion
In this paper, we introduce Geometry-Aware Point Cloud Prompt (GAPrompt), parameter-efffcient fine-tuning spe- cific to pre-trained 3D vision models. We find that capturing instance-speciffe shape features an effective way en- hance geometry awareness of prompting. To capture and utilize inherent geometric clues in point clouds, we readily develop the Point Prompt, the Point Shift Prompter, and the Prompt Propagation mechanism, greatly boosting the representational ability of prompting. Our approach outper- forms other state-of-the of-the-arrrr art parameter-efffiient fine-tuning methods and reduces trainable parameters significantly.
Visualization of Shifted Point Clouds. Figure 6 depicts the raw and shifted point clouds from the test split of ScanOb- jectNN. The raw point clouds are noisy and scattered, re- flecting the inherent complexity real-world data. Note the green color mapping merely for convenient recognition.
8

---

GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
Acknowledgments
Jie, S. and Deng, Z.-H. Fact: Factor-tuning for lightweight adaptation on vision transformer. In Proceedings of the AAAI conference artificial intelligence, volume 37, pp. 1060-1068, 2023.
This work was supported by the National Natural Science Foundation of China (62376011) and the National Key R&D Program of China (204YFAA140000).
Karimi Mahabadi, R., Henderson, J., and Ruder, S. Com- pacter: Efficient low-rank hypercomplex adapter layers. Advances in Neural Information Processing Systems, 34: 1022-1035, 2021.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
Lester, B., Al-Rfou, R., and Constant, N. The power of scale for parameter-efficient prompt tuning. arXiv preprint urXiv:2104.08691, 2021.
References
Li, Q. and Zhou, J. Caprompt: Cyclic prompt aggregation for pre-trained model based class incremental learning. In Proceedings of the AAAI Conference on Artificial In- telligence, volume 39, pp. 18421-18429, 2025.
Chen, G., Wang. M., Yang, Y., Yu, K., Yuan, L., and Yue, Y. Pointgpt: Auto-regressively generative pre-training from point clouds. Advances in Neural Information Processing Systems, 36, 2024.
Li, X. L. and Liang, P. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arX1v:2101.00190 2021.
Chen, S., Ge, C., Tong, Z.,, Wang, J., Song, Y., Wang, J., and Luo, P. Adaptformer: Adapting vision transform- for scalable visual recognition. Advances in Neural Information Processing Systems, 35:16664-16678, 2022.
Lian, D., Zhou, D., Feng. J., and Wang. X. Scaling & shift- ing your features: A new baseline for efficient model tuning. Advances in Neural Information Processing Sys- tems, 15:109-1 2022.
Dong, R., Qi, Z., Zhang, L., Zhang, J., Sun, J., Ge, Z., Yi, L., and Ma, K. Autoencoders as cross-modal teachers: Can pretrained 2d image transformers help 3d representation learning? arXiv preprint arXiv:2212.088: 083200, 2022.
Liu, H., Cai, M., and Lee, Y. J. Masked discrimination for self-supervised learning on point clouds. In European Conference on Computer Vision, pp. 657-675. Springer, 2022.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x words: Transformers for image recognition scale. ICLR, 2021.
Liu, Y., Fan, B., Xiang, S., and Pan, C. Relation-shape convolutional neural network for point cloud analysis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
He, J., Zhou, C., Ma, X., Berg-Kirkpatrick, T., and Neubig, G. Towards a unified view of parameter-efficient transfer learning. arXiv preprint arXiv:2110.04366, 2021.
2019.
Liu, Z.., Peng, Y., and Zhou, J. Insvp: Efficient instance visual prompting from image itself. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 2024.
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pp. 2790- 2799. PMLR, 2019.
Loshchilov, I. and Hutter, F. Decoupled weight decay reg- ularization. In International Conference Learning Representations, 2019.
Huang, T., Dong, B., Yang, Y., Huang, X., Lau, R. W., Ouyang, W., and Zuo, W. Clip2point: Transfer clip to point cloud classification with image-depth pre-training. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision (ICCV), pp. 22157-22167, Oc- tober 2023.
Loshchilov, I. and Hutter, F. Sgdr: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations, 2022.
Lu, T., Yu, M., Xu, L., Xiangli, Y., Wang, L.,, Lin, D., and Dai, B. Scaffold-gs: Structured 3d gaussians for view- adaptive rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 00654-20666 June 2024.
Jia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., and Lim, S.-N. Visual prompt tuning. In European Conference on Computer Vision, pp. 709-727. Springer, 2022.

---

GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
Pang, Y., Wang, W., Tay, F. E., Liu, W., Tian, Y., and Yuan, L.. Masked autoencoders for point cloud self-supervised learning. In European conference computer vision, pp. 604-62 621. Springer, 2022.
on computer vision and pattern recognition, pp. 1912- 1920, 2015.
Xu, Q. Xu, Z., Philip, J., Bi, S., Shu, Z., Sunkavalli, K., and Neumann, U. Point-nerf: Point-based neural radiance fields. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5438-5448, 2022.
Phan, A. V., Le Nguyen, M., Nguyen, Y. L. H., and Bui, L. T. Dgenn: A convolutional neural network over large-scale labeled graphs. Neural Networks, 108:533-543, 2018.
Qi R., Su, H., Mo, K., and Guibas, I.. Pointnet: Deep learning on point sets for 3d classification and segmenta- tion. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652-660, 2017a.
Yu, X., Tang, L.., Rao, Y., Huang, T., Zhou, J., and Lu, J. Point-bert: Pre-training 3d point cloud transformers with masked point modeling. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 19313- 2022.
Qi C. R., Yi, L.,, Su, H., and Guibas, L. J. Pointnet++: Deep hierarchical feature learning on point sets a met- ric space. Advances in neural information processing systems, 30, 2017b.
Zaken, E. B., Ravfogel, S., and Goldberg, Y. Bitfit: Simple parameter-efficient fine-tuning for transformer- based masked language-modes models. arXiv preprint arXiv:2 NNNi:2/06//0/9900 10/99, 2021.
Qi. Z., Dong, R., Fan, G., Ge, Z., Zhang, X., Ma, K., and Yi, L.. Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining. In International Conference Machine Learning, pp. 28223-28243. PMLR, 2023.
Zha, Y.. Wang, J., Dai, T., Chen, B., Wang, Z., and Xia, S.-T. Instance-aware dynamic prompt tuning for pre-trained point cloud models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14161- 14170, 2023.
Qi. Z., Dong, R., Zhang. S., Geng, H., Han, C., Ge, Z., Yi, L., and Ma, K. Shapellm: Universal 3d object un- derstanding for embodied interaction. arXiv preprint wrXiv:2402.17766, 2024.
Zha, Y.. Ji, Ji, Dai, T., Chen, B., Wang. Z., and Xia, S.-T. Towards compact 3d representations via point feature enhancement masked autoencoders. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 6962-6970, 2024.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. Dropout: simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929 1958, 2014.
Zhang, R., Guo, Z.,, Gao, P., Fang, R., Zhao, B., Wang, D., Qiao, Y., and Li, H. Point-m2ae: multi-scale masked autoencoders for hierarchical point cloud pre-training, Advances in neural information processing systems, 35: 27061-27074, 2022.
Tang, Y., Zhang, R., Guo, Z., Ma. X., Zhao, B., Wang. Z., Wang, D., and Li, X. Point-peft: gerameter-efficien fine- tuning for 3d pre-trained models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 5171-5179, 2024.
Zhao, B., Yang, L., Mao, M., Bao, H., and Cui, Z. Pner- floc: Visual localization with point- neural radiance fields. In Proceedings of the AAAI Conference on Artifi- cial Intelligence, volume 38, pp. 7450-7459, 2024.
Uy, M. A., Pham, Q.-.., Hua, B.-S., Nguyen, T., and Yeung, S.-K. Revisiting point cloud classification: A new bench- mark dataset and classification model on real-world data. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1588-15977 2019.
Zhou, X., Liang, D., Xu, W., Zhu, X., Xu, Y., Zou, Z., and Bai, X. Dynamic adapter meets prompt tuning: Parameter-efficient transfer learning for point cloud anal- ysis. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 14707- 14717, 2024.
Van der Maaten, L. and Hinton, G. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008
Wang, H., Liu, Q. Yue, X., Lasenby, J., and Kusner, M. Unsupervised point cloud pre-training via occlusion com- pletion. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9782-9792, 2021.
Zhu, X., Zhang, R., He, B., Guo, Z., Zeng Z., Qin, Z., Zhang, S., and Gao, P. Pointclip v2: Prompting clip and gpt for powerful 3d open-world learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2639-2650, 2023.
Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., and Xiao, J. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEFE conference
10

---

GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
Appendix
A. Training Detail
We adopt downstream fine-tuning configurations in alignment with the pioneering work Point-MAE (Pang et al., 2022). The detailed configurations are provided in Table 5. For example, when fine-tuning on ScanObjeetNN (Uy et al., 2019), the training process spans 400 epochs, using cosine learning rate scheduler (Loshchilov & Hutter, 2022) that starts at 5e-4, with 10-epoch warm-up period. The AdamW optimizer (Loshchilov & Hutter, 2019) is employed. Given that ReCon (Qi et al., 2023) and of DAPT (Zhou et al., 2024) by only loading pre-trained weights into a Point-MAE model for efficient fine-tuning, while excluding the residual components of ReCon and
(Zha et al., 2024) extend Point-MAE with several additional modules, we follow the approach
This option also leads to a sight computational saving, as shown in Table I. Indeed, the computation overhead should be calculated by subtracting FL.PSs of fine-tuning Point-MAE. All experiments are conducted on single GeForce RTX 4090 using PyTorch version 1.13.1.
Table Training details for downstream fine-tuning.
ScanObjecINN
ModelNet
Dataset
OBJ BG
OBJ ONLY
PB T50 RS
1K
Optimizer
AdamW
AdamW
AdamW
Adam W
Learning rate
5e-4
5c-4
5c-4
5c-4
Weight decay
5e-2
5e-2
5e-2
5e-2
Learning rate scheduler
cosine
cosine
cosine
cosine
Training epochs
400
400
400
400
Warmup epochs
10
10
10
10
Batch size
32
32
32
32
Point Prompt number
20
10
20
5
Prompt enhancing factor
0.5
0.5
0.5
0.5
Adapter enhancing factor
0.5
0.5
0.5
0.5
Number of points
2048
2048
2048
1024
Number point patches
128
128
128
64
Point patch size
32
32
32
32
B. Implementation Detail
B.1. Prompt Injection
We provide a comprehensive elucidation of the Prompt Injection process, as delineated in Eq. 15 of the main paper.
Given set of input tokens and prompt tokens [h, R(L++L,)xD in the i-th block, we can find geometric relationship between input tokens via FPS and KNN algorithm. We use hf RCDD and h E RCCKXD to denote center tokens and local neighboring tokens, where and represent the index for center and neighboring tokens. L, is the input tokens number, Lp is the prompt tokens number, and C' and K represent the number of local centers and local neighbors respectively.
Inspired by the thought of dropout (Srivastava et al., 2014), the prompt injection process randomly replaces the center and neighboring tokens with enhanced prompt tokens, which are informative of global shape.
Considering that FPS randomly selects initial points, although the resulting sets are consistent, the permutation of the resulting point sets may vary due to inherent randomness. Leveraging this randomness, replace the last Lp tokens in h with P.. and similarly, replace the last Lp tokens in h² with P,. injecting prompt tokens into input tokens. This prompt injection configuration is referred to as 'Replacement'.
Alternativlly, another prompt injection method, termed 'Permutation', can be employed. Given that h and h² are subsets of h, indexed by c and n, we can insert Lp prompt tokens p. before h; and remove the last Lp tokens of h,. Subsequently, by indexing the mixed set of p, and h; using e and n, we obtain the tokens h and h with prompt tokens injected.

---

GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
B.2. Feature Propagation
In this paper, we extensively employ the Feature Propagation operation introduced in PointNet++ (Qi al., 2017b). Below, provide detailed formulation.
Given the features of the center points, the objective is to compute the features of neighboring points through propagation mechanism that performs feature interpolation. This interpolation utilizes the spatial distances between the center and neighboring points, enabling the model to efficiently transfer and refine feature information across the tokens.
denote the set of center point coordinates as C E IR3 where L...., C, and the coordinates of any neighboring point as x E R where C represents the number of center points. The corresponding feature of given point denoted as f(-). Our objective is to compute (xx based æ; Ct and ((x).
First, we compute the Euclidean distance from to each center point c;
d(z, c.) = I cill-
(20)
Next, we calculate the weight by taking the inverse of the spatial distance:
w(r cz)
(21)
d(x, c:)
where p is typically set to 2. This results in set of weights w(z, c,) for I..... C. We then select only the top-K weights for interpolation:
[w(x,e)) = Top-K aa)),
(22)
where j ..... K, and K is typically set to 32. Subsequently, the interpolation of features is based on the weighted distances, formulated as:
f(x) Σド= w(1,,))({s)) Σ w(r, cÿ)
ΣK
(23)
Finally, this procedure is repeated for each neighboring point to obtain their features for further utilization.
91.0
90667
00.77
90.5
90.5
90.41
M1:11
90.24
10.477
90.38
Accuracy (%)
90.0
(%)
$9.70
99.79
s0.85
90.59
99.42
39.98
50,00
19.14
69.80
Adiqver Tuning
68.5
89.5
Claster
$8.43
0.10
0.20
0.30
0.50
0.70
1.40
Point Number
Adapter Enhancing Factor 3
Figure 7. Ablation study of Point Prompt number P and ini- tialization distribution.
Figure 8. Ablation study on the impact the adapter enhane- ing factor Sa when augmented with shape fcatures.
C. Additional Experiments
C.1. Analysis on Point Prompt Number P and Initialization.
We perform quantitative analysis Point Prompt about two additional different initialization distributions and learnable point number P. We respectively test initializing all points in uniform distribution and in cluster forms, where each cluster follows Gaussian distribution in 3D space. And we explore the impact of different point numbers. As shown in Figure 7, The result turns out that uniform distribution generally outperforms the cluster distribution, and with 20 learnable points, the model achieves the best results. This suggests that appropriate additional point prompts help to better capture subtle geometric features and to acquire explicit awareness of geometry.
12

---

GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model
Param: 0.3M Ace: 79.23%
Param:: 27.4M Ace: 90.22%
Param: 1.7M Ace: 88.38%
Param: 1.IM Acc: 88.51%
Param: 0.7M Ace: 89.35% Param: 0.6M Acc: 90.67%
(4) Limear Probing
(b) Full Fine-tuning
IDPT
(4) DAPT
Point-PEET
GAPrompt (Ours)
Figure 9. The t-SNE visualizations from the test sets of ScanObjectNN (PB_T50.RS RSR) using pre-trained Point-FEMAE with different tuning strategies. We extract the final classification features from the top linear layer I-SNE visualizations.
Dowretrcam Taak Head
Downstraem
Dowrstream
Downrzoo: Task Flead
50.7
Max Pooling
Max Pooling
Max Pooling
Max Pooling
50.84
Tokens
Prompts
Proneees
Tokess
Prompis
Patch Tokors
Shaps Poutidee
Last Transfommer Block
Last Transformer Block
Last T'mesfommer Block
Last Tamssoomer Block
(d)
[CIS])Patch Token
[CISS)+Pacch Telen-Prompe
(d) (CLsS)-Patch Token-Shaape Feature
Figure 10. Ablation study on different input for downstream head.
C.2. Analysis on Adapter Enhancing Factor Ba.
As shown in Figure 8, we conduct further ablation study on 3a, which is the factor when enhancing adapters with global shape features. It turns out that 3e set to 0.5 leads to peak performance. This indicates that properly enhancing adapters with global shape features can boost the model's performance. We attribute it to that global shape features help adjust and refine feature distributions and improve geometry awareness.
C.3. t-SNE Visualization of [CLS] Feature.
In Figure 9, the t-SNE (Van der Maaten & Hinton, 2008) feature manifold visualization displays the methods following linear probing, full fine-tuning, IDPT, DAPT, Point-PEFT and our GAPrompt on the ScanObjectNN PB T50 RS dataset. From Figure 9 (a), it is evident that the feature distribution extracted by the pre-trained model appears less discriminative. We contend that this is mainly due to the significant domain gap between the synthetic pre-training ShapeNet dataset and the real-world ScanObjectNN dataset, demonstrating the necessity for adapting downstream tasks. With full fine-tuning in Figure (b), the feature distribution becomes more discriminative all parameters are tuned. Figure (c-f) confirms that our GAPrompt helps the pre-trained model generate more distinguishable representations, with fewer learnable parameters and higher accuracy than other PEFT methods.
C.4. Ablation Downstream Head Input.
As demonstrated in Figure 10, examine the impact of different input choices for the downstream head. We consider four options: the [CLSS] token, the maximum input patch token, the maximum prompt token, and the global shape feature extracted by our Point Shift Prompter. Empirically, we find that the combination of the [CLS] token, the maximum patch token, and the shape feature yields the best results, demonstrated in Figure 10 (d). This can be attributed to the fact that the [CLS] token and the maximum patch token provide valuable knowledge extracted from the pre- pre-trained model, while the shape feature introduces instance-specifie global geometry information. However, Figure 10 (c), the knowledge embedded in the maximum prompt token largely overlaps with that of the maximum patch token. Consequently, our shape feature complements the traditional outputs from the pre-trained model effectively, thereby enhancing the performance of the downstream head.
13


Reviews:
[{"review_id": "2nmGWdRzVX", "reviewer": "Reviewer_4ih6", "summary": "The authors propose a parameter fine-tuning method for point cloud models, called GAPrompt. The main motivation of this method is to inject geometric information into the point cloud model. To achieve this goal, the authors propose three components: point prompts, which are used to increase the number of points; point shift prompter, which are used to extract global features; and prompt propagation mechanisms, which input global information into different blocks of the model. The effectiveness of this method is verified in the point cloud classification task.", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### R1. comparison with PPT and PointGST\nAlthough PPT and PointGST are currently available only as preprints on **arXiv** and have not been officially published, we supplement comparison experiments as shown below. \n\nThe results across four datasets and four representative backbones illustrate that **GAPrompt achieves 12 SOTA results** while PPT and PointGST each attain 2. Furthermore, our approach requires only **0.6M** trainable parameters, the highest parameter efficiency.\n\n|Method|Ref.|Param.|OBJ_BG|OBJ_ONLY|PB_T50_RS|ModelNet|\n|:-------:|:--------:|:-----:|:-----------:|:-------:|:-------:|:------:|\n||||*Point-MAE*||||\n|+PPT|arXiv|1.1|89.33|88.81|84.87|93.7|\n|+PointGST|arXiv|0.6|91.74|90.19|85.29|93.5|\n|+GAPrompt|ThisPaper|**0.6**|**91.91**|**90.19**|**85.57**|**94.2**|\n||||*ReCon*||||\n|+PPT|arXiv|1.1|**95.01**|**93.28**|89.52|93.8|\n|+PointGST|arXiv|0.6|94.49|92.94|89.49|93.6|\n|+GAPrompt|ThisPaper|**0.6**|94.49|92.60|**89.76**|**94.0**|\n||||*PointGPT-L*||||\n|+PPT|arXiv|3.6|98.28|96.21|94.10|95.1|\n|+PointGST|arXiv|2.4|98.97|**97.59**|**94.83**|94.8|\n|+GAPrompt|ThisPaper|**2.0**|**98.97**|96.73|94.31|**96.2**|\n||||*Point-FEMAE*||||\n|+PPT|arXiv|1.1|93.98|92.08|88.79|93.3|\n|+PointGST|arXiv|0.6|94.66|92.94|90.22|93.8|\n|+GAPrompt|ThisPaper|**0.6**|**95.53**|**93.63**|**90.67**|**94.5**|\n\n### W1. innovation and difference against PPT and PointGST\nOur GAPrompt differs from these methods in two key aspects. \n\n1. **Finer-grained prompting:** PPT operates on **position encodings** of point tokens, as a **coarse-grained** token-level prompting approach. In contrast, GAPrompt introduces **fine-grained point-level** prompting via the Point Prompt and Point Shift Prompter, allowing more precise and adaptive feature modulation at the individual point level.\n\n2. **Interpretability and geometric awareness:** PointGST introduces a **Spectral Adapter**, transforming point tokens from spatial domain to spectral domain, which belongs to **adapter** methods. But GAPrompt belongs to prompt methods, avoiding **obscure spectral domain transformations** and ensuring stronger **interpretability and geometric awareness**.\n\n### W2. part or semantic segmentation tasks\nWe supplement additional segmentation results on both ShapeNetPart and S3DIS datasets.\n\nIt can be seen that our GAPrompt excels other methods including the arXiv twos. We achieve **six** SOTA metrics across 2 datasets and 2 backbones with highest parameter efficiency, only **0.3M** additional parameters beyond 5.2M of downstream head.\n\nResults in *ShapeNetPart*\n\n|Method|Ref.|Param.|Cls.mIoU|Ins.mIoU|\n|:-------:|:---------:|:------:|:-------:|:-------:|\n||*Point-MAE*||||\n|+DAPT|CVPR24|5.65|84.01|85.7|\n|+PPT|arXiv|5.62|84.07|85.7|\n|+PointGST|arXiv|5.59|83.81|85.8|\n|+GAPrompt|ThisPaper|**5.55**|**84.10**|**85.8**|\n||*ReCon*||||\n|+DAPT|CVPR24|5.65|83.87|85.7|\n|+PPT|arXiv|5.62|**84.23**|85.6|\n|+PointGST|arXiv|5.59|83.98|85.8|\n|+GAPrompt|ThisPaper|**5.55**|83.90|**85.8**|\n\nResults in *S3DIS*\n\n|Method|Ref.|Param.|mAcc|mIoU|\n|:-------:|:---------:|:------:|:------:|:------:|\n||*Point-MAE*||||\n|+DAPT|CVPR24|5.61|67.2|56.2|\n|+PPT|arXiv|5.58|67.6|57.9|\n|+PointGST|arXiv|5.59|68.4|**58.6**|\n|+GAPrompt|ThisPaper|**5.51**|**68.5**|58.4|\n||*ReCon*||||\n|+DAPT|CVPR24|5.61|66.3|56.3|\n|+PPT|arXiv24|5.58|67.4|57.3|\n|+PointGST|arXiv24|5.59|67.8|57.9|\n|+GAPrompt|ThisPaper|**5.51**|**68.0**|**58.0**|\n\n### W3. a disguised way of increasing dataset size, data augment, unfair\nOur Point Prompt is not data augmentation and the setting is absolutely fair, cause the inputs are same for all methods. The reason is three-folded:\n\nFirst, the Point Prompts are **randomly initialized**, rather than additional sample points from each instance. So, no training nor testing data leakage. \n\nSecond, Point Prompts are **fixed after trained**, adapting to a specific domain and intensifying discriminant information, not varying for each instance.\n\nFinally, comparing to the 1024 points input, 20 prompts are less than **2%**. We provide results on ModelNet40 at 1044 resolution. Nothing changes from 1024 points.\n\n|points|Point-FEMAE|+IDPT|+DAPT|+Point-PEFT|+GAPrompt|\n|:-----:|:---------:|:---:|:---:|:---------:|:-------:|\n|1024|94.0|93.4|93.2|94.3|94.5|\n|1024+20|94.0|93.4|93.2|94.3|94.5|\n\n### W4. point prompt number as zero in Figure 7\nWe add more ablation on point prompt number $P$ as below. When $P$ is zero, it equals to drop Point Prompt module. However, we still attains **89.65%** with only other two modules, surpassing DAPT's 88.51% and Point-PEFT's 89.35%. This is attributed to the point-level adaptation of Point Shift Prompter and prompt enhancement of Prompt Propagation, while further gain can be achieved with incorporation of Point Prompt.\n\n|$P$|0|5|10|20|30|\n|:------:|:---:|:---:|:---:|:---:|:---:|\n|Acc.(%)|89.65|89.76|90.41|90.67|90.24|\n\n### S1. data symbols\nThanks for advice, we will detail the symbols in Sec. 3.2 and 3.3 and check out other symbols.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "This article lacks some theoretical innovations.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "The author mentioned the most advanced work in the introduction, related work and experimental comparison. However, the author did not mention some of the most advanced work such as PPT[1] and PointGST[2].\n[1] Parameter-efficient Prompt Learning for 3D Point Cloud Understanding\n[2] Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learning", "essential_references_not_discussed": "[1] Parameter-efficient Prompt Learning for 3D Point Cloud Understanding\n[2] Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learning", "other_strengths_and_weaknesses": "Strengths:\n1、The author proposes GAPrompt, a novel geometry-aware prompt learning method tailored for pre-trained 3D vision models.\n2、The author ntroduces three key algorithm designs including Point Prompt, Point Shift Prompter, and the Prompt Propagation mechanism.\nWeaknesses: \n1、Although the proposed method has achieved advanced results, I think its innovation lies in the combination of ideas that have been published so far. Specifically, PPT[1] has proposed to fine-tune the 3D model using the position information of points. At the same time, the method proposed in this paper is very similar to the results of PointGST[2], but in some methods it is still lower than PointGST.\n[1] Parameter-efficient Prompt Learning for 3D Point Cloud Understanding\n[2] Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learning\n2、The experiments conducted by the author only in the point cloud classification task are insufficient and need to be further verified in the point cloud part segmentation or semantic segmentation tasks.\n3、I think that the hint increases the number of input points, which is a disguised way of increasing the size of the dataset, which is similar to a kind of data augmentation. I think this is unfair to other methods.\n4、In Figure 7, the author should show the case where the point prompt number is zero. Based on Figure 7, I guess the corresponding result should be lower than 89.76%. This is not significantly different from other methods. Does this mean that the method proposed by the author is mainly caused by increasing the number of point cloud inputs?", "other_comments_or_suggestions": "The text lacks some explanations of data symbols. Especially in Sections 3.2 and 3.3.", "questions_for_authors": "See Weakness.\n-----------------------------------------------------------------\nAfter reading the author's reply carefully and thinking deeply, I still think that the manuscript lacks innovation, the experimental data is single, and the writing is confusing. I think the manuscript fails to meet the publication requirements of the conference.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pECpkJTcR3", "reviewer": "Reviewer_WVi3", "summary": "This paper proposes GAPrompt, a geometry-aware prompt learning method for parameter-efficient fine-tuning (PEFT) of pre-trained 3D vision models. Existing PEFT approaches in 3D vision struggle to capture geometric information from sparse and irregular point clouds. To address this, GAPrompt introduces three key innovations:\n\nPoint Prompt: Explicitly incorporates learnable point clouds as auxiliary input to enhance geometric awareness.\n\nPoint Shift Prompter: Dynamically adjusts point cloud positions using global shape features extracted via hierarchical downsampling.\n\nPrompt Propagation: Integrates shape information into feature extraction through token replacement and interpolation.\nExperiments on ScanObjectNN and ModelNet40 show GAPrompt achieves competitive performance with full fine-tuning (e.g., 90.67% vs. 90.22% on PB-T50-RS) while using only 2.19% trainable parameters.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Q1,W1. computational cost of multi-resolution grouping\n\nMulti-resolution grouping of Point Shift Prompter can hardly become a bottleneck for real-time applications. \n\nAlthough multi-resolution FPS/KNN operation has $O(N^2)$ complexity, its overhead remains minimal compared to $O(N^2)$ attention mechanism and expensive MLPs, because the feature size of FPS/KNN is **3**, far less than the **384**-dimensional model features.\n\nWe supplement a breakdown of computational costs. As seen, our three modules collectively account for **less than 2%** of total computational cost, with majority stemming from encoder and attention mechanism.\n\n||Point Prompt|Point Shift Prompter|Prompt Propagation|Encoder|Attn. Layers|FFN layers|Downstream head|\n|---------|:----------:|:------------------:|:----------------:|:-----:|:----------:|:--------:|:-------------:|\n|**FLOPs**|0.0005G|0.045G|0.20G|2.03G|0.082G×12|0.164G×12|0.001G|\n|**Ratio**|0.01%|0.9%|0.4%|39.9%|19.3%|39.5%|0.01%|\n\n### Q2. pretraining dependency\n\nIn addition to backbones already introduced (mask-based Point-MAE and **contrastive** pre-trained **ReCon**), we supplement additional **Point-BERT**. Notably, in Table 1 of the paper, **ReCon** is exactly pre-trained via contrastive learning and our method **surpasses full fine-tuning** with only **1.38%** trainable parameters.\n\nBased on Point-BERT, GAPrompt still attains highest performance with lowest trainable parameters, verifying its generalizability to pre-trained backbones.\n\n|Method|Ref.|Param.|OBJ_BG|OBJ_ONLY|PB_T50_RS|ModelNet|\n|:-------:|:--------:|:-----:|:----------:|:-------:|:-------:|:------:|\n||||*Point-BERT*||||\n|+IDPT|ICCV23|1.7|88.12|88.30|83.69|92.6|\n|+DAPT|CVPR24|1.1|91.05|89.67|85.43|93.1|\n|+GAPrompt|ThisPaper|**0.6**|**91.22**|**89.85**|**85.64**|**93.5**|\n\n### Q3. density variations\n\nWe supplement experiments in extremely sparse inputs condition, at 128 input resolution on ModelNet40. While all methods faces a performance drop,  GAPrompt **consistently outperforms** other PEFT methods, demonstrating robustness to varying input densities.\n\n||Resolution|*Point-MAE*|+IDPT|+DAPT|+Point-PEFT|+GAPrompt|\n|:--------:|:--------:|:---------:|:---:|:---:|:---------:|:-------:|\n|ModelNet40|128points|86.2%|84.4%|85.2%|85.6%|**86.0%**|\n\n### Q4. latency analysis\n\nWe measure inference time on ScanObjectNN using an RTX 4090, as shown below. GAPrompt incurs only **0.9ms** additional latency, accounting for less than **9%** of the base 10.2ms required by Point-MAE. Furthermore, our inference time is less than other PEFT methods, benefiting from lightweight point-level prompting design.\n\n||*Point-MAE*|+IDPT|+DAPT|+Point-PEFT|+GAPrompt|\n|:------------------------:|:---------:|:---:|:---:|:---------:|:-------:|\n|inferencetime(ms/sample)|10.2|16.2|12.0|13.5|**11.1**|\n\n### W2. task specificity\n\nWe supplement additional results on part segmentation dataset *ShapeNetPart* and semantic segmentation dataset *S3DIS*. Even though such dense prediction tasks are challenging, our method excels previous SOTA methods including DAPT[CVPR24] and PointPEFT[AAAI24] in both efficiency and efficacy.\n\nResults in *ShapeNetPart*\n\n|Method|Ref.|Param.|Cls.mIoU|Ins.mIoU|\n|:---------:|:---------:|:------:|:-------:|:-------:|\n||*Point-MAE*||||\n|+DAPT|CVPR24|5.65|84.01|85.7|\n|+Point-PEFT|AAAI24|5.62|83.41|85.4|\n|+GAPrompt|ThisPaper|**5.55**|**84.10**|**85.8**|\n||*ReCon*||||\n|+DAPT|CVPR24|5.65|83.87|85.7|\n|+Point-PEFT|AAAI24|5.62|83.23|85.3|\n|+GAPrompt|ThisPaper|**5.55**|**83.90**|**85.8**|\n\nResults in *S3DIS*\n\n|Method|Ref.|Param.|mAcc|mIoU|\n|:---------:|:---------:|:------:|:------:|:------:|\n||*Point-MAE*||||\n|+DAPT|CVPR24|5.61|67.2|56.2|\n|+Point-PEFT|AAAI24|5.58|66.5|56.0|\n|+GAPrompt|ThisPaper|**5.51**|**68.5**|**58.4**|\n||*ReCon*||||\n|+DAPT|CVPR24|5.61|66.3|56.3|\n|+Point-PEFT|AAAI24|5.58|65.8|55.8|\n|+GAPrompt|ThisPaper|**5.51**|**68.0**|**58.0**|\n\n### W3. initialization sensitivity\n\nOur point prompt initialization exhibits robustness on non-uniform LiDAR data. In our Table 1 experiments, the **three ScanObjectNN variants** originate from real-world **LiDAR scans**, which inherently contain **non-uniform distributions** and **background fragments**. Despite this, GAPrompt still achieves SOTA results, even surpassing full fine-tuning, using less than **2%** trainable parameters.\n\n### S1. potential method enhancements\n\nThanks for your insightful suggestions! \n\nAs classic algorithms with $O(N^2)$ complexity, replacing FPS/KNN with sparse point cloud operations can further enhance efficiency.\n\nAs for adaptive weighting between shape features and prompts, current hyperparameter setting relies human expertise. We agree adaptive weighting would be a better choice, deserving future exploration. \n\n### S2. theoretical analysis advice\n\nThanks for your valuable advice. We will provide additional discussions on robustness of point shifts against adversarial perturbations and Point Prompt initialization strategies in final version.", "claims_and_evidence": "I think they are clear.", "methods_and_evaluation_criteria": "They are reasonable for me.", "theoretical_claims": "I have checked.\nBut I’m not familiar with this topic at all. \nSo I will read the comments from other reviewers carefully.", "experimental_designs_or_analyses": "Yes, I have checked. They are sound to me.", "relation_to_broader_scientific_literature": "The key contributions of GAPrompt build upon and extend several critical strands of research in 3D vision, parameter-efficient fine-tuning (PEFT), and geometric deep learning.", "essential_references_not_discussed": "Sorry. I'm not familiar with this topic.", "other_strengths_and_weaknesses": "Strengths:\n\nGeometry Integration: First work to explicitly leverage point-level geometric cues for 3D PEFT, addressing a critical gap.\n\nParameter Efficiency: Achieves SOTA performance with <3% tunable parameters, outperforming adapter-based methods.\n\nInterpretability: Visualizations validate the role of shape features in guiding attention.\n\n\nWeaknesses:\n\nComputational Overhead: Multi-resolution FPS/KNN operations may introduce latency (unquantified in the paper).\n\nTask Specificity: Evaluated only on classification; generalization to segmentation/detection remains unproven.\n\nInitialization Sensitivity: Uniform point prompt initialization may underperform on non-uniform LiDAR data.", "other_comments_or_suggestions": "Method Enhancements:\n\nExplore sparse point cloud optimizations to reduce FPS/KNN costs.\nInvestigate adaptive weighting between shape features and prompts.\n\nTheoretical Analysis:\n\nFormalize the robustness of point shifts against adversarial perturbations.\nCompare initialization strategies (e.g., clustered vs. uniform prompts).", "questions_for_authors": "Computational Cost: Does the Point Shift Prompter’s multi-resolution grouping become a bottleneck for real-time applications (e.g., robotics)?\n\nPretraining Dependency: How does GAPrompt perform with contrastive pre-trained models (e.g., Point-BERT) versus mask-based ones (Point-MAE)?\n\nDensity Variations: Can the current design handle extremely sparse inputs (e.g., 100 points) without performance degradation?\n\nLatency Metrics: What is the actual inference time increase compared to full fine-tuning (e.g., milliseconds per sample)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "m6LBKhm88g", "reviewer": "Reviewer_LHwj", "summary": "This is a paper on efficient point cloud fine-tuning, where the author added a geometric perception structure to the point cloud embedding part for efficient fine-tuning and achieved good results.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your appreciation and valuable advice!\n\n### Q1. inference time and FLOPs comparison\n\nWe test the inference time and FLOPs of our method and other SOTA methods, including IDPT[ICCV23], DAPT[CVPR24] and Point-PEFT[AAAI24]. All experiments are conducted on a RTX 4090.\n\nAs table shown below, our method attains the fastest inference time with only **11.1** millisecond per sample and fewest FLOPs with **5.0G** MACs among parameter-efficient works. Notably, our method merely introduces **4%** computational overhead but brings over **97%** reduction in trainable parameters.\n\n|                            | *Point-MAE* | +IDPT | +DAPT | +Point-PEFT | +GAPrompt |\n| :------------------------: | :---------: | :---: | :---: | :---------: | :-------: |\n| inference time (ms/sample) |    10.2     | 16.2  | 12.0  |    13.5     | **11.1**  |\n|         FLOPs (G)          |     4.8     |  7.2  |  5.0  |     7.0     |  **5.0**  |\n\n### Q2. number of the enhanced prompt tokens\n\nThe number of enhanced prompt tokens in our experiments is set to **10** as a hyperparameter. We provide additional ablation experiment results below and will add it into final version for better interpretability. \n\nWe find that our method produces peak performance when adopting 10 enhanced prompt tokens. A smaller number leads to suboptimal prompting effects due to insufficient guidance, while an excessive number does not yield further improvements but incurring additional computational cost.\n\n| Prompt Token Number |   0   |   3   |   6   |    10     |  15   |  20   |  30   |\n| ------------------- | :---: | :---: | :---: | :-------: | :---: | :---: | :---: |\n| Acc. on PB_T50_RS   | 89.34 | 89.98 | 90.34 | **90.67** | 90.55 | 90.47 | 90.33 |\n\n### C1. citation to PointGST and PPT\n\nThanks for your reminder and we will cite them in final version. Notably, at the time of our submission and even at present, both works remain as preprints on arXiv and have not been officially published. However, our method still excels both in terms of accuracy and efficiency. \n\nWe compare with them on four dataset based on four representative backbones, as shown below. GAPrompt achieve **12** SOTA results while PPT and PointGST get 2 SOTA results respectively. And our method has only **0.6M** trainable parameters, attaining highest parameter efficiency, attributed to our geometry-aware point-level prompting design.\n\n|  Method   |    Ref.    | Param.  |    OBJ_BG     | OBJ_ONLY  | PB_T50_RS | ModelNet |\n| :-------: | :--------: | :-----: | :-----------: | :-------: | :-------: | :------: |\n|           |            |         |  *Point-MAE*  |           |           |          |\n|   +PPT    |   arXiv    |   1.1   |     89.33     |   88.81   |   84.87   |   93.7   |\n| +PointGST |   arXiv    |   0.6   |     91.74     |   90.19   |   85.29   |   93.5   |\n| +GAPrompt | This Paper | **0.6** |   **91.91**   | **90.19** | **85.57** | **94.2** |\n|           |            |         |    *ReCon*    |           |           |          |\n|   +PPT    |   arXiv    |   1.1   |   **95.01**   | **93.28** |   89.52   |   93.8   |\n| +PointGST |   arXiv    |   0.6   |     94.49     |   92.94   |   89.49   |   93.6   |\n| +GAPrompt | This Paper | **0.6** |     94.49     |   92.60   | **89.76** | **94.0** |\n|           |            |         | *PointGPT-L*  |           |           |          |\n|   +PPT    |   arXiv    |   3.6   |     98.28     |   96.21   |   94.10   |   95.1   |\n| +PointGST |   arXiv    |   2.4   |     98.97     | **97.59** | **94.83** |   94.8   |\n| +GAPrompt | This Paper | **2.0** |   **98.97**   |   96.73   |   94.31   | **96.2** |\n|           |            |         | *Point-FEMAE* |           |           |          |\n|   +PPT    |   arXiv    |   1.1   |     93.98     |   92.08   |   88.79   |   93.3   |\n| +PointGST |   arXiv    |   0.6   |     94.66     |   92.94   |   90.22   |   93.8   |\n| +GAPrompt | This Paper | **0.6** |   **95.53**   | **93.63** | **90.67** | **94.5** |", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes, but this paper does not contain complex proofs.", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "Previous point cloud PEFT methods, such as DAPT, IDPT, and PPT, mainly rely on prompt learning and adapter for feature extraction.", "essential_references_not_discussed": "Some papers have not been compared and cited. The author should add these discussions, which will not affect the contribution of this paper.\n\nParameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learning\nPositional Prompt Tuning for Efficient 3D Representation Learning", "other_strengths_and_weaknesses": "This work is solid and worth accepting, which has many ablation studies and visualization.", "other_comments_or_suggestions": "None", "questions_for_authors": "1. How about the inference time and flops compared to the other works?\n2. How many tokens of the Enhanced Prompt Token are concatenated to the original tokens in sequence? Is there any ablation study?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Sd9B5udG70", "reviewer": "Reviewer_7QpT", "summary": "The paper focuses on parameter-efficient fine-tuning for 3D vision models and introduces a geometry-aware prompt learning method, named GAPrompt. GAPrompt incorporates three key designs to effectively capture geometric information, including Point Prompt, Point Shift Prompter, and the Prompt Propagation mechanism. Experimental results on the ScanObjectNN and ModelNet40 datasets demonstrate the effectiveness of GAPrompt.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Q1. working mechanism of Point Prompt\nAs for working mechanism of Point Prompt, it can be analyzed through Equation 18 and 19 in paper. Previous prompting methods operate at **token level**, which corresponds to local patches, failing to adjust exact points within patches. \n\nIn contrast, our Point Prompt directly prompts within patches, operating at **point-level grains**. When adapting to downstream, it is optimized via task loss to focus on critical points that encode subtle yet essential geometric information.\n\nAs for visualization, the learned Point Prompts are already present in Figures 5 and 6. However, as there are only **20 prompts** within **2048 points**, may difficult to see. In final version, we will explicitly highlight learned Point Prompts in visualization for better clarity.\n\n### W1. ablation study on Point Prompt in Table 3\nWe supplement additional ablation results on \"GAPrompt without Point Prompt\" as shown below. Incorporating the learnable Point Prompt yields an **1.02%** gain in accuracy. \n\n|Point Prompt|Point Shift Prompter|Prompt Propagation|Acc.(%)|\n|:----------:|:------------------:|:----------------:|:------:|\n|-|-|-|86.10|\n|√|-|-|87.85|\n|√|√|-|89.34|\n|-|√|√|89.65|\n|√|√|√|90.67|\n\n### Q2,W2. comparison with PPT \nThanks for your advice, and we will add it to main table in final version. The primary reason we did not initially include this comparison is that PPT hasn't been officially published and only available on **arXiv**. Even though, we compare on 4 different representative backbones across 4 datasets, and our GAPrompt achieves **14** higher performances in 16 experiments with fewer trainable parameters(**0.6M**), due to our lightweight geometry-aware prompting design. \n\n|Method|Ref.|Param.|OBJ_BG|OBJ_ONLY|PB_T50_RS|ModelNet|\n|:-------:|:--------:|:-----:|:-----------:|:-------:|:-------:|:------:|\n||||*Point-MAE*||||\n|+PPT|arXiv|1.1|89.33|88.81|84.87|93.7|\n|+GAPrompt|ThisPaper|**0.6**|**91.91**|**90.19**|**85.57**|**94.2**|\n||||*ReCon*||||\n|+PPT|arXiv|1.1|**95.01**|**93.28**|89.52|93.8|\n|+GAPrompt|ThisPaper|**0.6**|94.49|92.60|**89.76**|**94.0**|\n||||*PointGPT-L*||||\n|+PPT|arXiv|3.6|98.28|96.21|94.10|95.1|\n|+GAPrompt|ThisPaper|**2.0**|**98.97**|**96.73**|**94.31**|**96.2**|\n||||*Point-FEMAE*||||\n|+PPT|arXiv|1.1|93.98|92.08|88.79|93.3|\n|+GAPrompt|ThisPaper|**0.6**|**95.53**|**93.63**|**90.67**|**94.5**|\n\n### W3,Q3. few-shot and part segmentation experiments\nWe supplement few-shot experiments on ModelNet40 and segmentation experiments on ShapeNetPart below. \nWe compare our method with other SOTA works based on pre-trained Point-FEMAE. It can be seen that our method performs well in few-shot setting and can generalize to segmentation tasks well, verifying the efficacy of GAPrompt.\n\n*Few-shot on ModelNet40*\n\n||Ref.|5-way||10-way||\n|:---------:|:--------:|:----------:|:----------:|:----------:|:----------:|\n|||10-shot|20-shot|10-shot|20-shot|\n|+DAPT|CVPR24|96.6±2.1|97.9±2.7|92.1±3.4|94.9±3.3|\n|+Point-PEFT|AAAI24|96.8±2.6|98.1±2.5|92.4±3.2|95.0±.3.1|\n|+PPT|arXiv|96.9±1.9|98.0±2.9|91.9±3.6|95.2±3.0|\n|+GAPrompt|ThisPaper|**97.2**±1.7|**98.4**±2.1|**92.7**±2.9|**95.7**±2.8|\n\nResult in *ShapeNetPart*\n\n|Method|Ref.|Param.|Cls. mIoU|Ins. mIoU|\n|:-------:|:--------:|:---------:|:-------:|:-------:|\n|||*Point-MAE*|||\n|+DAPT|CVPR24|5.65|84.01|85.7|\n|+PPT|arXiv|5.62|84.07|85.7|\n|+GAPrompt|ThisPaper|**5.55**|**84.10**|**85.8**|\n|||*ReCon*|||\n|+DAPT|CVPR24|5.65|83.87|85.7|\n|+PPT|arXiv|5.62|**84.23**|85.6|\n|+GAPrompt|ThisPaper|**5.55**|83.90|**85.8**|\n\n### Q4. difference before and after Self-Attn in Fig. 4\nThe difference before and after Self-Attention **arises from distinct inference orders**. This is similar to choosing between **\"Attn→FFN\"** or **\"FFN→Attn\"** in Transformer models. The Prompt Propagation mechanism enhances spatial information, further facilitating interactions between prompts and point tokens. When applied **after Self-Attention**, it provides stronger performance improvements.\n\n### Q5a.  determination of Point Prompt number\nThe number of Point Prompts is set **proportionally** to the input point cloud resolution, typically **0.5%-1%** of the dataset resolution. \nFor **clean datasets** (e.g., ModelNet40, OBJ_ONLY), **0.5%** yields slightly better results (+0.1%), corresponding to 5 and 10 prompts for 1024 and 2048 points, respectively. While for **noisier datasets** (e.g., OBJ_BG, PB_RS_T50), **1%** provides a small gain (+0.2%), resulting in 20 prompts for 2048 points.\n\n### Q5b. setting of $L_p$ and $p_i^{\\prime}$\n $L_p$ is set to **10**, as shown in the ablation study below. \nWhen $L_p$ is too small, the prompting effect is insufficient, leading to suboptimal performance. Conversely, an excessive $L_p$ does not yield further improvements while incurring additional parameter overhead. As for $p_i^{\\prime}$, we use Kaiming Uniform initialization.\n\n|$L_p$|0|3|6|10|15|20|\n|-----|:---:|:---:|:---:|:-------:|:---:|:---:|\n|Acc.|89.34|89.98|90.34|**90.67**|90.55|90.47|", "claims_and_evidence": "This paper emphasizes the importance of geometric information in parameter-efficient fine-tuning of 3D vision models. Experimental results demonstrate the effectiveness of the three proposed geometry-aware designs on benchmark datasets.", "methods_and_evaluation_criteria": "Yes. The three proposed designs focus on capturing the geometric information, and ScanObjectNN and ModelNet40 are standard benchmark datasets to evaluate fine-tuning on 3D vision models.", "theoretical_claims": "Yes, I have checked the theoretical analysis in Sec. 3.4.", "experimental_designs_or_analyses": "I have reviewed all the experiments. \n\nAblation studies in Tables 3 and 4 assess the effectiveness of each design. However, the effectiveness of Point Prompt is not explicitly verified in Table 3.", "relation_to_broader_scientific_literature": "Prior studies have also acknowledged the importance of 3D-aware parameter-efficient fine-tuning. For example, IDPT'23 generates dynamic prompt tokens for each point cloud instance to capture semantic prior features, while DAPT'24 combines dynamic scale adapters with internal prompts for effective point cloud transfer learning.", "essential_references_not_discussed": "The Positional Prompt Tuning [PPT'22] method is designed for the same task as this work and has released models and code. However, it is neither cited nor compared in the paper.\n\n[PPT'22] Positional Prompt Tuning for Efficient 3D Representation Learning.", "other_strengths_and_weaknesses": "### Strengths\n- GAPrompt introduces a geometry-aware point cloud prompt for parameter-efficient fine-tuning of 3D vision models, incorporating three designs to effectively capture geometric information.\n- Experimental comparisons with existing methods, along with ablation studies on individual designs, demonstrate the effectiveness of GAPrompt.\n- The paper is well-written and easy to follow.\n\n### Weaknesses\n- Insufficient ablation study on Point Prompt in Table 3.\n- Missing comparisons with [PPT'22] (Positional Prompt Tuning for Efficient 3D Representation Learning).\n- Lack of few-shot learning experiments on ModelNet40 and part segmentation experiments on ShapeNetPart.", "other_comments_or_suggestions": "None.", "questions_for_authors": "### Major\n- 1. The working mechanism of Point Prompt in capturing subtle geometric features is not clearly explained. In Table 3, results for \"GAPrompt without Point Prompt\" should be included. Additionally, besides analyzing the effect of different numbers of Point Prompts, visualizations of the trained Point Prompt should be provided for better understanding.\n\n- 2. A comparison with [PPT'22] (Positional Prompt Tuning for Efficient 3D Representation Learning) should be added to Table 1.\n\n- 3. It is recommended to include few-shot learning experiments on ModelNet40 and part segmentation experiments on ShapeNetPart.\n\n### Minor\n\n- 4. In Fig. 4, the results before and after Self-Attention are inconsistent across different prompt injection options. Further clarification is needed.\n\n- 5. The number of Point Prompts varies across datasets in Table 5. A justification for this choice is needed. Additionally, details on how $L_p$ is set and how $p_i^{\\prime}$ is initialized should be provided.\n\nMy evaluation is primarily based on the main weaknesses outlined in the previous section, which correspond to the major problems. Addressing these concerns with substantial evidence and clarifications could potentially influence my rating.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces GAPrompt, a geometry-aware, parameter-efficient fine-tuning approach for 3D point cloud models. To effectively capture geometric information from sparse and irregular point clouds, GAPrompt incorporates three novel components: Point Prompts, which serve as learnable auxiliary points; a Point Shift Prompter, which refines point positions based on global shape features; and a Prompt Propagation mechanism, which transmits global information throughout the model’s layers. Experiments on ScanObjectNN and ModelNet40 demonstrate that GAPrompt achieves performance comparable to full model fine-tuning while requiring only a small fraction of trainable parameters.

This paper received divergent scores. The reviewers 7QpT, WVi3, and LHwj gave a positive score, but the reviewer 4ih6 gave a rejection score. AC reads the paper, review, and the rebuttal carefully, and agrees with the positive comments stated by the reviewers 7QpT, WVi3, and LHwj. In particular, the authors provided extensive explanations and feedback so that the reviewer 7QpT could revert the scores to the positive side. Although the major concern of reviewer 4ih6 is technical novelty and the paper writing, AC thinks that judging the submitted paper based on unpublished work is not proper, and the authors' clarification on the difference in method is reasonable. However, since the experiments are done with a small-scale dataset, the applicability of the proposed approach has room for improvement. As a result, AC recommends weak acceptance of this paper. AC recommends that the authors improve the paper based on the feedback from the authors and AC.
Decision: Accept (poster)