On Geometry-Enhanced Parameter-Efficient Fine-Tuning for 3D Scene Segmentation
Authors: Liyao Tang
Zhe Chen
Dacheng Tao
Venue: NeurIPS 2025 poster ()
Keywords: scene understanding
semantic segmentation
3D vision
Primary area: deep_learning

Abstract: The emergence of large-scale pre-trained point cloud models has significantly advanced 3D scene understanding, but adapting these models to specific downstream tasks typically demands full fine-tuning, incurring high computational and storage costs. Parameter-efficient fine-tuning (PEFT) techniques, successful in natural language processing and 2D vision tasks, would underperform when naively applied to 3D point cloud models due to significant geometric and spatial distribution shifts.
Existing PEFT methods commonly treat points as orderless tokens, neglecting important local spatial structures and global geometric contexts in 3D modeling.
To bridge this gap, we introduce the Geometric Encoding Mixer (GEM), a novel geometry-aware PEFT module specifically designed for 3D point cloud transformers. GEM explicitly integrates fine-grained local positional encodings with a lightweight latent attention mechanism to capture comprehensive global context, thereby effectively addressing the spatial and geometric distribution mismatch.
Extensive experiments demonstrate that GEM achieves performance comparable to or sometimes even exceeding full fine-tuning, while only updating 1.6% of the model's parameters, fewer than other PEFT methods.
With significantly reduced training time and memory requirements, our approach thus sets a new benchmark for efficient, scalable, and geometry-aware fine-tuning of large-scale 3D point cloud models.
Code is available at https://github.com/LiyaoTang/GEM.

On Geometry-Enhanced Parameter-Effiient Fine-Tuning for 3D Scene Segmentation
Liyao Tang The University of Sydney Itan9687 duni sydneyy edu. au
Zhe Chen
Dacheng Tao
La Trobe University
Nanyang Technological University dacheng. tao @gmail com
zhe chen@latrobe edu..a
Abstract
The emergence of large-scale pre-trained point cloud models has significantly advanced 3D scene understanding, but adapting these models to specific down- stream tasks typically demands full fine-tuning, incurring high computational and storage costs. Parameter--fffiient fine-tuning (PEFT) techniques, successful in natural language processing and 2D vision tasks, would underperform when naively applied to 3D point cloud models due significant geometric and spatial distri- bution shifts. Existing PEFT methods commonly treat points as orderless tokens, neglecting important local spatial structures and global geometric contexts 3D modeling. To bridge this gap. we introduce the Geometric Encoding Mixer (GEM), a novel geometry-ave -waree PEFT module specifically designed for 3D point cloud transformers. GEM explicitly integrates fine-grad fine-grained local positional encodings with a lightweight latent attention mechanism capture comprehensive global con- text, thereby effectively addressing the spatial and geometric distribution mismatch. Extensive experiments demonstrate that GEM achieves performance comparable to or sometimes even exceeding full fine-tuning, while only updating 1.6% of the model's parameters. fewer than other PEFT methods. With significantly reduced training time and memory requirements, our approach thus sets a new benchmark for efficient, scalable, and gcometry-aware fine-tuning of large-scale 3D point cloud models. Code is available at https://gitubb.com//iy Tang/GGM..
Introduction
Point cloud semantic segmentation is fundamental task for scene understanding that underpins many real-world applications, from autonomous driving and unmanned acrial vehicles augmented reality [20, 83, 46]. Recent breakthroughs in large-scale models for language and 2D vision [16, 50, 59, 14, 58, 49] have spurred an analogous trend toward training powerful point cloud backbones capable of capturing rich semantics from 3D scenes [76, 77, 79].
Despite significant capabilities, adapting large pre-trained backbone models to specific downstream target domains is non-trivial. The typical strategy heavily relies on full fine-tuning, a process that both computationally expensive and storage-intensive, since each downstream task demands independent copy of all model parameters. Furthermore, fine-tuning large models typically requires extensive datasets prevent overfitting and catastrophic forgetting [4, 51], while large datasets are usually inaccessible when adapted to smaller, specialized tasks. To address these issues, parameter- efficient fine-tuning (PEFT) methods offer a promising performance in reducing the cost of adaptation while reaching the performance of full fine-tuning with significantly fewer parameters.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).
"Corresponding authors. Also affiliated with Cisco La Trobe Centre for Al and loT. "College of Computing and Data Science, NTU, Singapore 639798.

---

Adapter
Full fine-tuning
78
Feed Ferward
Not
77-
Worm
Add Nem
Maglt
76
Procept
Seli-Aneriion
Adapter
mloU
75
Lincar probing
74
LoRA
Spatial
73
Atlaptor
(ouns)
Adaptart Prongt Turning
MAM
Adapter
72
Goometry Encoding Mixer (GEM)
3
4
5
Existing PEFT methods
Parameters (%)
(a)
(b)
Figure 1. (a) Existing PEFT methods, such adapters, prompt tuning, and LoRA, focus adaptations inside attention and feed-forward layers. (b) contrast, Geometry Encoding Mixer (GEM) explicitly encodes geometric cues and mixes them into the pre-trained model, by the Spatial Adapter refining the pre-trained positional encoding and the Context Adapter complementing the local attention. (c) By capturing 3D spatial details and scene-wide wide geometry context, GEM reaches full fine-tuning performance while tuning < 2% parameters, outperforming existing PEFT methods.
While PEFT has been extensively validated in language processing and 2D vision tasks, its effec- tiveness on large-scale 3D point clouds remains largely unexplored and inadequately addressed. As demonstrated empirically in Fig. 1, existing PEFT methods only yield limited performance gains directly applied 3D point cloud backbone. We tend attribute this limitation key challenge that is often overlooked by current PEFT methods when using for 3D point cloud: unlike structured data such as text or images, point clouds inherently unordered sets of coordinates in R They exhibit strong irregularity, sparsity, and structural variability, shaped by different sensing protocols and scene geometries. These factors lead significant geometric and spatial distribution shifts between large-scale pre-training datasets and downstream domains, which existing PEFT methods fail account for. In particular, representative PEFT methods, such LoRA [26], adapters [25] and prompt tuning [37, 36], either adapt models at an isolated, per-point level or insert fixed external tokens the global level, thus failing to adequately capture important geometric and spatial contexts inherent in 3D scenes. Moreover, to avoid the prohibitive computational cost of global attention over a large number of points, current 3D transformers predominantly employ local attention mechanisms without explicitly modeling global contexts, which further restricts the potential of current PEFT approaches in performance.
To address the above challenges, we re-examine PEFT for 3D scene understanding and hypothesize that effective PEFT on 3D scenes could take advantage of explicitly modeling both fine-graine mnaiirai local spatial patterns and global geometric contexts. We propose that neglecting either aspect would degrade fine-tuning performance: local-only adaptations may be prone to noise due the lack of broader context consideration, whereas global-only methods would miss local details and lose precision. This motivates novel PEFT adaptation framework tailored specifically for 3D scenes to bridge these local and global scales.
We introduce Geometry Encoding Mixer (GEM), geometry-aware module for parameter-efffiient fine-tuning of point cloud transformer on 3D scenes. It comprises two complementary components: a spatial adapter local neighborhood and a context adapter capturing the scene context. In the spatial adapter, we employ a lightweight 3D convolutional bottleneck that operates on points neighborhood, enriching the pre-trained positional encoding by learning fine-grained local spatial details at target domains. In the context adapter, we introduce a set of learned latent tokens to serve as global context vectors. These latent tokens interact with the full point cloud through efficient attention, forming bottleneck at the token dimension and bypassing the constraints of local attention, and thus aggregating scene-specific context from across the entire point cloud. By fusing these two paths,

---

the Geometry Encoding Mixer effectively bridges local and global representations, providing the fine-tuned model with richer understanding of the 3D scene than either component alone.
Empirically, we validate our approach on large-scale 3D scene datasets, including both indoor [12, 61, 2, 84] and outdoor [5] scenes. Our results indicate that models equipped with GEM consistently achieve performance matching or sometimes surpassing full fine-tunin siue-tuning methods, updating merely -1.6% of model parameters, while being fewer than existing PEFT alternatives. These findings underscore the importance of explicitly modeling geometric and spatial contexts for efficient and effective adaptation in 3D scene understanding. To the best of our knowledge, our work represents the first exploration and validation of PEFT approaches tailored explicitly for large point cloud transformer under large-scale 3D scenes, hoping establish a foundation for future research and practical deployments.
Related Work
Point cloud segmentation. Point clouds serve efficient representation for large-scale 3D scenes. Consequently, point cloud segmentation becomes a fundamental task that drives the design of 3D backbone architectures. Since the seminal work PointNet [53, 54], numerous 3D backbone architectures have been proposed for this task. These backbones either project the points onto a grid-like structure, e.g. 3D voxel grid, to exploit 3D convolutional networks [10, 70, 65, 21, 19, 13], or directly process the raw unordered point sets [73, 38, 75, 27, 66, 45]. Recently, inspired by the success of large transformer models natural language and 2D vision [68, 16, 7, 9]. researchers have started training increasingly large and powerful point cloud backbones [56, 91, 69, 57, 34, 78, 77J.
Although these models achieve strong performance, they typically need to be trained from scratch for each new dataset and often require specialized training recipes to reach optimal results.
Following the paradigm of large-scale pre- treitrrining that has proven successful in vision and language domains [59, 33, 50, 14, 49, 23], researchers have begun exploring similar strategies for 3D point clouds. In particular, self-supervised learning (SSL) large 3D scene datasets has demonstrated promising results [82, 44, 24, 80]. However, most such efforts focus on modest convolutional backbones like SNerse [10] rather than transformer-batt wowffoomme--bseeee 3D backbone: Sonata [76] introduced SSL for large point transformer encoder [77], but even this method still requires full fine-tuning on each downstream dataset to achieve optimal performance.
architectures. Only recently has an
SSL approach been applied to
In this work, explore effective PEFT methods for large pre-trained point cloud backbones, with the goal of reducing the computational and storage overhead when adapting them to downstream datasets, while matching the performance of full fine-tuning.
carameter-efficien fine-tuning (PEFT). The ever-growing size of transformer-based foundation 10 33, 67] makes full fine-tuning for downstream task prohibitively expensive in both memory and computation. PEFT methods address this challenge by adapting large models while updating only small fraction of their parameters. Existing approaches fall into four broad categories.
Selective mue-t methods update a carefully chosen subset of the original weights. One simple variant is linear probing [50, 15] that trains only the classification head. Other strategies restrict training to specific parts of the network, e-g., tuning only bias terms [6] and selecting a subset of parameters based on gradient magnitude criteria [90].
Adapter-based tuning inserts lightweight bottleneck modules into otherwise frozen backbone. First explored for CNNs [60] and later extended to transformers, adapters may be placed sequentially [25, 62] or in parallel [22. 93, 17, 8] the original modules. Beyond single task, adapters can be composed or fused, promoting knowledge sharing without catastrophic forgetting [72, 51].
based tuning techniques, including prompt tuning [36] and prefix tuning [37], extend to the prompting paradigm in large language models [41]. Instead of hand-crafted prompts, these methods leamn task-specifi vectors that are prepended to the input sequence [3, 36] injected as extra tokens in transformer layers [37, 43, 31], thereby steering the model without modifying its original weights
[36, 41].
Low-rank adaptation (LoRA) constrains the weight updates to a low-dimensional subspace by learning pair of low-rank matrices that are added to each pre-trained weight, typically in the
3

---

attention layers [26]. This design approximates the effect of full fine-tuning while introducing only small number of additional parameters. Successors further enhance the approximation ability, robustness, or rank allocation [32, 42, 88, 81,4
Recent works also propose hybrid schemes that highlight common design principles [28], for instance, integrating their respective strengths under a unified adapter framework [22].
Although PEFT has proved effective in language and 2D vision, direct transfer to 3D scene understanding is often sub-optimal, largely due to the unordered nature of point clouds and the prominence of geometric cues. Our work thus investigates PEFT strategies tailored to large-scale 3D
scenes.
PEFT for 3D point cloud. In the context of 3D point clouds, PEFT remains relatively underexplored. Existing methods thus far have focused mostly on object-level inputs with limited spatial scale. Some approaches introduce prompt tokens that adapt to 3D data, where the prefix tokens are dynamically generated from intermediate features [87, 92, 1] or from the spatial centers of local patches [89]. Several works also introduce auxiliary side networks, operating either in the spectral domain [39] or the spatial domain [18, 71J, to provide geometric context. Other methods construct banks of prompt vectors using domain-specific dataset to inject 3D prior knowledge [64], or combine language-side prompt tuning with point-cloud adapters to handle open-vocabulary recognition [63].
Compared to object-level datasets [85, 48], 3D scene understanding involves much larger inputs containing on the order of millions of points [12, 2]. This scale amplifies computational challenges and underscores the need for PEFT techniques expressly designed for large 3D scene models, which however a research direction that remains largely unexplored.
3 Methodology
We propose the Geometry Encoding Mixer (GEM) as lightweight parameter-efficiet module for fine-tuning point-cloud transformer. Fig. summarises the overall architecture.
In particular, GEM consists of two complementary adaptations: a Spatial Adapter that refines the positional encoding of each point, injecting local geometry information overlooked by generic adapters; and a Context Adapter that distills compact of latent tokens that broadcast global scene-level cues. Both components follow the residual, bottleneck design of classic adapters, yet emphasize and operate on spatial rather than channel space.
3.1 Preliminaries
We first consider the direct application of existing PEFT methods on current point-cloud transformers.
Challenges under 3D. Following transformer [68, 16), point-cloud transformers also build upon self-attention layer (Attn) followed by feed-forward network (FFN). However, due to the large scale input points at million scale, global self-attention over all points would demand prohibitively large GPU memory and compute resource, if ever possible, due to the quadratic complexity of the attention operations: (Attn) n where n denotes the number of input points. In this regard, state-of-the -otth--art point-cloud transformers propose various designs of local attentions [91, 78, 77, 34, 69] to cap the complexity, where each point can only attend to points within the same patch. With a patch size of p. the complexity of local attention is then ((Attnse) np. For example, it is set to p = 16 in PT [91] and p 1024 in the larger PTv3 [77].
The irregular and sparse nature of 3D point cloud, together with these architecture designs, greatly impede the application of standard PEFT methods such adapter, LoRA, and prompt tuning. As shown in Fig. 1, directly applying existing methods yields only marginal gains. We attribute this to their lack of spatial structure awareness of 3D geometry and the inability to capture scene-wide context under the local attention regimes of modern point cloud backbones.
Adapters. The adapter- based methods [25, 62] insert small modules after the attention or FFN layers:
fadapter(x)) + (x W Wdown)
up:
where W down E dxxr and W.p Rexd are down and up projections that form a bottleneck structure with < d and activation in the mid, and also surrounded by residual connection. As generic

---

Wup
Globel Attention
Wup
pre-tained
Nonilnsarity
loral Atlention
Global Attention
Positional
Conv Inc
Enooding
K
V
Witoon
W Wasws
Latents
X
Lc
Spatial Adapter
Context Context.Adapeer
Figure 2. Geometry Encoding Mixed. We propose the spatial adapter to enhance the pre-trained positional encoding, and the context adapter to overeome the local attention mechanism, thus enhancing the efficient adaptation large-scale 3D scenes with explicit geometry encoding.
MLP bottlenecks, adapters adapt the model on per-point basis, overlooking the important spatial cues in 3D point cloud.
LoRA. LoRA methods [26] typically adapt the attention projection layers and approximate the full fine-tuning with bottleneck in weight space, transforming the attention into:
LutA = Attn(Q + AQ, + AK, V);
where AQ = x: W down W Wap with Wdown E dx and W E B'xd and similarly for A K with different pair of weights. While LoRA leverage the strong ability of the attention layers, note that all attentions in point cloud transformer are constrained within local patches. LoRA methods are thus inherently limited by the prevalent design of local attention. With only limited rank << d, could fail to provide global context of the target dataset.
Pompt tuning. Prompt tuning methods [6] explicitly introduce global tokens into the attention:
Attmpsomps = Attn Q, PP K]. [Pv: VI),
(3)
where PK, Pv E mx of the constraint of local patches, prompt tokens can be duplicated into each patch. However, with m << n, it can be hard for few static external tokens capture scene-specific context that can vary across point clouds within the same dataset. In addition, it also overlooks the spatial pattern of the point cloud, lacking the adaptation to local point features.
sets of tokens prepending to the original key-value pair. In spite
Others. There are more methods that explore other aspects. Seleetive tunings propose to tune a small subset of the frozen model parameters, such BitFit [6] that tunes only bias terms. Hybrid methods are also explored, such as MAM [22] that couples FFN adapter with prompt tuning for attention layer. These methods still inherit the constraints from those representative PEFT methods, which hindered by the local attention and fail to recognize the spatial structure.
In summary, existing methods overlook 3D geometry and remain confined to local patches, motivating a PEFT design that incorporates spatial awareness with scene-wide context, introduced next.
3.2 Our Methodology: Geometry Encoding Mixer (GEM)
To address these limitations, we present the Geometry Encoding Mixer (GEM). It refines positional encodings and injects scene context for efficient adaptation on large-scale 3D scenes:
pos((r) (z) S (2),
x <- x Attn (x: + fcuntex(
(5)
where æ denotes input points and pos(-) the pre-trained positional embedding. Fig. 2 illustrats the overall structure.
5

---

Spatial Adapter (SA). Point clouds are sparse, irregular samples in 3D. Adapting to their fine- grained geometry requires explicitly modeling local neighborhoods, which prior PEFT methods fail to address.
To this end, the spatial adapter refines per-point positional encoding through lightweight 3D> convolutional bottleneck. Concretely, for each point æ, we consider a 3D grid and gather points in the vicinity voxels as neighbors, denoted by .N:
Espatial (x) = + a
Σ sEN
W (a down) W up
Ioc
(6)
where down E Rd and Wip E R composes the spatial kernel weights for i-th neighboring voxel and o--) is the nonlinearity, such as ReLU.
R are down and up projectsion with r < d. Wi E R
With common kernel dimension k = 3. the spatial adapter touches at most k3 neighbours per point and adds 2rd + y² parameters, rendering a complexity of ((ndrr + nk* (nd) given r < d. It thus functions as efficient convolution-base positional encoding [11, 29] in parallel the pre-trained positional encoding, thereby capturing fine spatial layout to match the target distribution.
Context Adapter (CA). Due to the local attention, any local and channel-wise adaptation would be constrained from acquiring scene-wide context, hindering the adaptation performance on downstream tasks like semantic segmentation.
To inject global context without breaking the On²) barrier, we introduce m latent tokens L E mm where n, that attend to all points once:
Le Attn LW ? K, V),
(7)
Scontexi (x: = x + Autn Q. L.W.K L.WV) Wap.
(8)
where mR and Q,K, V
x W a: Z.K
down: xWV are down-projected point features.
down:
As both attention costs (nm) with m < n, global aggregation is affordable and at the same level of complexity the conventional prompt tuning [37, 36]. Furthermore, update L L + L, each adapter insertion to share across layers, yielding dynamic prompts that capture the context of current scene, unlike static prefixes in prompt tuning.
Discussion. In comparison to existing PEFT methods, GEM explicitly biases the adaptation to 3D geometry and context. The spatial adapter captures localized variations that generic adapters, such as LoRA and basic adapters, would miss, since those ignore spatial structure and use the positional encodings from pre-training. The context adapter circumvents the local attention design provide beene-wide context for efficient adaptation with few parameters. By combining these two components, approach enables scene-awer aware adaptation of 3D transformers, where each point is tuned with respect to both its neighbors and the entire point cloud. It thus results in significantly improved performance on 3D scene datasets with only a lightweight adaptation overhead!
4 Experiments
We present the results of our proposed GEM on semantic segmentation of large-scale 3D scene datasets and experiment with both self-supervised pre-trained backbone, Sonata [76], and supervised pretrained one, PTv3-PPT [79], for investigation.. We also provide ablation studies to reveal the detailed effects of different components better.
4.1 Experimental Setup
We primarily follow the comprehensive protocols proposed in Sonata [76], covering ScanNet [12], ScanNet200 [61], ScanNet++ [84] and S3DIS [2].
'We study the empirical overheads the supplementary Sec.
Foor more generalization and comparisons in other settings, such convolutional model and 3D shape analysis. please refer to the supplementary Sec. B.

---

Semantic Sog.
Params
ScanNet Val [12)
ScanNer200 Val [61] ScanNet++ Val [84]
S3DIS Area [2]
S3DIS 6-fold [2]
Methods
Learn.
Pet.
mioU mAce allAce miol mAce allAce mloU mAce allAce mioU mAce allAcc mloU mAce all
Thuining from Sparsel/Net [10)
39.2M
72.3
80.2
90.0
25.0
32.9
28.8
38.4
80.1
72.5
72.4
80.9
89.9
PTv3 (77)
124.8M
100%
77.6
85.0
92.0
35.3
46.0
83.4
42.1
53.4
85.6
73.4
78.9
91.7
77.7
85.3
91.5
Full fine-tuning
Sonata (full) [76]
124.8M
100%
79.4
86.1
92.5
36.8
46.5
84.4
43.7
55.8
86.6
76.0
81.6
93.0
82.3
89.9
93.3
Sonata (f.)
108.5M
100%:
78.3
85.9
92.3
37.3
47.8
83.7
49.8
61.2
87.6
72.4
79.0
92.2
79.5
87.3
92.3
PEFT methods
Sonata (lin.)
0.02M
0.02%
72.5
83.1
89.7
29.3
41.6
81.2
37.3
50.9
84.3
72.3
81.2
90.9
76.5
87.4
90.8
RidFit
0.2M
0.2%
74.7
84.7
90.8
32.5
45.5
82.0
42.4
56.5
85.7
73.9
82.0
91.5
76.1
87.1
91.0
- -Aaapter [25]
2.8M
2.5%
77.0
85.4
91.8
33.6
45.7
82.5
42.6
57.5
85.6
13.8
82.9
91.5
76.4
87.5
91.4
-LoRA [26]
76.7
85.7
91.7
33.6
45.5
58.0
77.4
87.8
91.2
- -Prompt Tunning [37]
5.5M
4.8%
74.3
84.1
90.5
31.4
44.4
81.6
41.2
56.2
84.8
73.4
82.5
91.0
73.7
86,5
90.5
-GEM (ours)
1.8M
1.6%
78.3
86.6
92.3
35.6
46.9
83.3
46.6
60.3
86.3
75.1
83.0
92.1
77.9
88.2
92.1
Table 1. Semantic segmentation.
We adopt two representative pre- pre-trained backbones, the Sonata model [76] with large-scale self- supervised training and the PTv3-PPT [79] with supervised pre-training on multiple large-scate curated datasets. We compare GEM with the existing popular PEFT methods, including bias-tuning from BitFit [6], Adapter [25]. LoRA [26]. and Prompt Tuning [37]. We consider linear probing as a baseline and the full fine-tuned model as our target strong reference. In addition, we note that the Sonata (full.) specifically introduces a task-specific decoder for semantic segmentation during the fine-tuning, which may not reflect the fair comparison. Therefore, we revive to the standard full fine-tuning without additional decoder network, which denoted by Sonata (ft.).
For training, we follow the widely accepted fine-tuning setups to update only the inserted or selected weights with the pre-trained backbone weights remaining frozen. All PEFT baselines follow the implementations from released code, adopt the suggested common practice [47, 52], and are tuned to their best validation setting in Fig. 1(c). Specifically. we set the default rank be 32 and the number of learnable tokens to be m 4. More details are given in the supplementary.
4.2 Performance Comparison
Main results. Tab. shows that GEM consistently surpasses all representative PEFT methods across datasets. It matches the performance of full fine-tuning on most datasets and even exceeds it on ScanNet++[84J.. ScanNet++ comprises large and diverse scenes captured at mibbiiiiimtte resolution, diverging greatly from the spatial distribution of common pre- tre-training datasets such as ScanNet [12]. Under this domain gap, the joint modeling of local spatial cues and global scene context introduced by GEM proves especially beneficial.
Interestingly, LoRA [26] and adapter [25] deliver similar scores, despite acting on different trans- former components, This observation implies that, when updates remain local and geometric is not modeled explicitly, the precise choice of adaptation target can be secondary. In addition, prompt tuning [37] underperforms even the linear probing baseline in the S3DIS 6-fold evaluation, revealing the penalty of ignoring spatial structure during fine-tuning.
Data efficiency. We assess PEFT methods under the scenarios limited data and annotations in Tab. 2. The results demonstrate the exceptional data efficiency of GEM, which outperforms both other PEFT methods and the full fine-tuning counterparts. Notably, under extreme data scarcity, such 1% of the labeled scenes and limited annotations (20 points per scene), GEM surpasses both Sonata (full.) and Sonata (ft.), highlighting its superiority in low-data regimes.
Supervised pre-training. While self-supervised methods dominate the language domain, supervised pre-training remains competitive for vision [33]. To probe the limits of PEFT under large-scale 3D supervision, we adopt recent advances in 3D supervised pre-training [79]. which achieve leading performance by training on a large collection of curated, labeled scene datasets. As shown in Tab. 3, existing PEFT methods In contrast, GEM improves upon supervised backbone, further outperforming the larger PTv3 [77] with only 1.6% additional parameters.
degrade performance, likely suffering from negative transfer [86, 74].
PEFT with decoder. A dedicated segmentation decoder (13% of the total parameters) known to lift fully fine-tuned baselines. We therefore ask whether PEFT can still add value in the presence of
7

---

Data Efficiency
Limited Scenes (Pct.)
Limited Anmotation (Pts.)
Supervised Pre-train.
Params
ScanNet Val (12)
Methods
20% Fall
50
100 200
Full
Methods
Learn.
Pet.
mloU mAce allAce
Training from scratch
Training from scratch
SperseUNet
26.047.8
56.7
62.9
53.9
65.5
SparseUNet [10]
39.2M 100%
72.3
80.2
90.0
PTv3 [77]
48.9
61.0
67.0
77.2 60.1
72.7 77.2
PTv3 [77]
124.8M 100%
77.6
85.0
92.0
Full fine-tuning
Full fine-twinn
Sonata (full) [76]
45.3
72.4
72.8
79.4 70.5
76.
77.0
79.4
PTv3-PPT (ft) [79]
97.4M
100%
78.6
86.0
92.5
Sonata
63.4
72.3
75.3
PEFT methods
PEPT methods
Sonata (lin.)
43.662.5 68.6
69.8
72.5
69.0
711
71.5
72.5
PTv3-PPT (li..)
0.IM
0.1%
78.6
85.9
92.5
-BitFit [6]
46.5.64.9
69.9
71.7 74.7
71.0
72.3
72.9
73.6
74.7
BitFis [6]
0.2M
0.2%
78.2
85.6
92.3
-Adapter [25]
46.4.64.2
72.2
-Adapter [25]
1.8M
1.8%
78.5
85.9
92.4
-LoRA
63.6
70.1
72.6
76.7
72.1
73.6
75.2
76.7
LoRA [26]
78.4
86.0
92.4
PPrompt Tunning [37]
68.9
71.1
72.5
-Prompt Tonning [37]
4.8M
4.8%
78.3
85,9
92.4
-GEM (ours)
47.5 65.6 71.0 73.3 78.3 74.7
76.2
76.6
78.3
-GEM (ours)
1.8M
1.6%
79.1
86.6
92.6
Table 2. Data efficiency.
Table 3. Supervised pre-training.
with Decoder
Params
ScanNet Val [12]
Outdoor Seg.
Params
Sem.KITTI [5]
Methods
Learn.
Pet.
mioU mAce all
Methods
Learn.
Pet.
mloU mAcc allAce
Training from scratch
Training from scratch
PTv3 [77]
124.8M 100%
77.6
85.0
92.0
PTv3 [77]
124.8M
100% 69.1
76.1
92.6
Full fine-turing
Full fine-tuming
Sonata (full) [76]
124.8M
100%
79.4
86.1
92.5
Sonata (full.) [76]'
124.8M
100%
72.6
77.9
93.4
Sonata (full.)
124.8M
100%
78.5
86.3
92.4
Sonata (ft.)
108.5M
100%
68.8
75.2
92.6
PEFT methods with decoder
PEFT methods
Sonata (dec.)
16.3M 13.1%
79.1
86.6
92.7
Sonata (lin.)'
0.02M 0.02%
62.0
72.5
91.0
Sonata (dec.)"
16.3M 13.1%
77.2
85.9
91.9
Sonata (lin.)
0.02M 0.02%
52.0
63.3
87.1
BitFiit [6]
16.4M 13.2%
78.1
86.1
92.3
-BitFiit [6]
0.2M
0.2%
59.9
70.6
91.0
.Adapter [25]
19.1M 15.0%
78.2
86.5
92.3
1 Adapter [25]
2.8M
2.5%
62.5
72.2
92.0
-LoRA [26]
18.2M 14.3%
78.9
87.0
92.5
LLRA [26]
1.9M
1.7%
63.8
74.0
92.0
-Prompt Tunning [37]
22.6M 17.2%
77.4
86.1
92.1
+ -Prompt Tunning [37]
5.5M
4.8%
59.1
71.0
90.8
.GEM (ours)
M8.1M 14.3% 79.5
87.3
92.7
-EM (ours)
1.8M
1.6%
67.7
75.5
93.1
Table 4. PEFT with segmentation decoder. Methods
Table 5. Outdoor semantic segmentation. Methods use outdoor datasets for pre-training.
with reports our re-produced results.
with
such a task-specific head. Tab. answers in the affirmative: GEM outperforms all competing PEFT variants and even surpasses its own fully fine-tuned counterpart. setting new state-of- state-of-the--rd on ScanNet. The result underscores the merit jointly modeling local geometry and scene-level context, even when the decoder already provides task-specific capacity.
Outdoor segmentations. During the original evaluation of Sonata [76], it employs separate pre- trained models for indoor and outdoor scenarios, due to the significant domain gaps between indoor and outdoor scenes. Here, we test stronger setting: transferring an indoor pre-trained backbone an outdoor autonomous-driving benchmark. We replace the input layer with a randomly initialized counterpart match dimensionality and freeze all remaining weights.
Tab. 5 shows that GEM narrows the gap to a model trained from scratch on outdoor data, yet a noticeable performance gap persists. Closing this gap remains interesting direction for future work.
4.3 Ablations and Analsys
We mainly consider the linear probing, Sonata (lin.), as baseline and ablate on ScanNet [12] to better investigate the key factors for effective PEFT in 3D scenes, shown in Tab. 6. For more studies on the effect of available parameters, please refer to the supplement.
Individual effectiveness of SA and CA. To validate our initial hypothesis, we study the individual effectiveness of the SA and CA in Tab. 6a. We find that both adapters can already superior to baseline and competitive to the existing PEFT methods. We notice that the SA takes the majority part of the introduced parameters, which partially due the inherently expensive convolution with 3 kernels. Such challenge is also 3D specific and urges us to develop the CA, rather than solely
8

---

CA Params mloU
m mloU mAce allAcc
strategy
mioU mAce All.cc
Sonata (lin.)
0.02%
72.5
78.1
86.4
92.1
N/A.
77.8
86.1
92.1
1.0%
77.2
78.3
86.6
92.3
per-stage.
78.0
86.1
92.0
GEM
V
0.6%
77.3
78.2
86.8
92.3
global
78.3
86.6
92.3
1.6%
78.3
(a) SA and CA compose of effective PEFT for 3D scenes, individually and jointly.
(b) GEM robust available la- tent tokens.
(c) GEM provide better capabili- ties when sharing latent tokens.
Table Ablations GEM. not specified. the backbone is the self-supervised pre-trained Sonata [76] with no additional task-specific decoder. evaluated on ScanNet [12]. Default settings are marked gray.
relying on the positional encoding to capture 3D geometry. By combining the two form of spatial adapters, we obtain GEM that reaches the best performance by comprehensively exploring the 3D scene while remaining parameter-efficien.
Understanding the latent tokens. In Tab. 6b, we are motivated to further investigate how efficient and effective the proposed CA could be. We thus study how few tokens it could afford before failing to capture the global context. Surprisingly, we find that CA can use as few as one single token to effectively express the global scene context, obtaining clear improvement over SA-only approach. We consider this to be the effectiveness of weight decomposition, where we are akin to approximating the n² global self- atffeattentoon weights with two n X matrices, In comparison LoRA that decomposes in the weight space, we decompose the spatial attention weight matrices and form spatial bottleneck at token dimension. addition, while we are aware that related topics have been similarly explored for efficient attentions [30, 35], we are, to the best of our knowledge, the first to motivate their effective application as PEFT methods for 3D scenes understanding.
Sharing the latent tokens. The last experiment, Tab. 6c probes whether the latent tokens learned by CA should remain layer-local be shared across the hierarchy, assessing its ability capture dynamic and scene-specific context. When each layer keeps its own bank of tokens (N/A), GEM already surpasses SA-only, confirming that CA supplies complementary global cues. Re-using the same tokens within each encoder stage (per-stage) yields further yet modest gain, suggesting that coherent context inside each resolution level helps the network reconcile local geometry with mid-range structure.
The strongest result emerges when single of latent tokens is shared by all transformer blocks. We attribute this improvement to its consistency with the fact that 3D point clouds represent the same underlying 3D geometry in spite of their irregularity and sparsity. From this perspective, such globally shared latent tokens confine the model to adapt to the actual underlying scene, rather than overfitting on irrelevant patterns due the noisy sampled point cloud.
Revealing the geometry cues in latent tokens. Under the above assumption that latent tokens regularize the model to capture the underlying 3D scene, visualize the attention weights and compare them to those produced by other PEFT methods that also inject global context, such as prompt tuning.
As shown in Fig. 3, starting from the first stage, our latent tokens learn to produce crisper attention weights that follow geometric cues, such as the simple geometric primitives of floor surface. In deeper stages, the tokens progressively attend more salient geometric objects, including clutter on the table and sofa. In contrast, the static tokens introduced by prompt tuning produce yield blurrier attention patterns that often span across object and region boundaries.
These observations suggest that the proposed latent tokens effectively capture and broadcast scene- wide geometric context. Combined with the local spatial structure extracted by 3D convolutions, this leads to superior performance, establishing our method as an effective PEFT approach for 3D scene understanding.
5
Conclusion
This study presents GEM, a geometry-aware fine-tuning module tailored for large-scale 3D scene segmentation. By combining local spatial refinement and global context modeling, GEM addresses key challenges inherent adapting pre-trained 3D models to new domains. Experimental results across multiple benchmarks confirm its ability to achieve competitive performance with minimal

---

Input/ GT
Stage
Stage 2
Stage3
Stage4
Pron
GEM
Figure 3. We visualize the attention weights of our latent tokens, comparing to the attentional weights with the prompt tuning, showing the enhanced geometry cues produced by GEM.
parameter updates. These findings highlight the value of incorporating geometric priors into efficient model adaptation and offer a promising direction for scalable deployment in 3D scene understanding tasks.
Limitation and future work. While GEM achieves promising performance with little overhead in complexity, we notice it relies on the assumption that the model can be fully parallelized. For example, it would incur noticeable overhead if using large batch sizes, as the device forced to process the input sequentially, in spite of the parallel design in GEM. In addition, we are able to successfully train our PEFT methods with a fraction of epochs than the full fine-tuning, e.g. 100 rather than 800 epochs, we realize that the gradients need to be back-propagated into early layers and could hinder the training efficiency, especially with dense point clouds like S3DIS [2]. We thus suggest that it is required a systematic examination on the additional overhead that PEFT methods would experience in 3D scene understanding.
Acknowledgement. Dr. Tao's research is partially supported by NTU RSR and Start Up Grants.
References
[1] Zixiang Ai, Zichen Liu, Yuanhang Lci, Zhenyu Cui, Xu Zou, and Jiahuan Zhou. Gaprompt: Geometry-
aware point cloud prompt for 3d vision model, 2025. 4
[2] Tro Armeni, Sasha Sax, Amir R 7amir, and Silvio Savarese. Joint 2d-3d-semantic data for indoor scene understanding. arXiv preprint
(tf::1702.01000 2017. 3. 3.4. 6. 7, 10
[3] Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, and Phillip Isola. Exploring visual prompts for
adapting large-scale models, 2022. 3
Ankur Bapna and Orhan Firat. Simple, scalable adaptation for neural machine translation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP- (INNLP. Association for Computational Linguistics, 2019.
[5] J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, and J. Gall. SemanticKITTi: A Dataset for Semantic Scene Understanding of LiDAR Sequences. In Proc. of the IEEE/CVF International
Conf. Computer Vision (ICCV), 2019. 3.8
[6] Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. BitFit:
Simple parameter-effiieen fine-tuning for transformer- based masked language-models. In Smaranda Muresan, Preslav Nakov, and Aline Villavi- cencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1-9, Dublin, Ireland, May 2022. Association for Computational Linguis- tics. 3, 5. 8, 23
[7] Nicolas Carion, Francisco Massa, Gabriel Synnaeve. Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV, 2020. 3
[8] Shoufa Chen, Chongjian Ge. Zhan Tong. Jiangliu Wang. Yibing Song, Jue Wang, and Ping Luo. Adapt-
former: Adapting vision transformers for scalable visual recognition. arXiv preprint arXiv:2205.13535.
2022. 3
[9] Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexander Kirillov, and Rohit Girdhar. Masked- attention mask transformer universal image segmentation. CVPR, 2022.
[10] Christopher Choy, Jun Young Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski
convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Patter
Recognition, pages 3075-3084, 2019. 3, 7, 8, 22, 23
[11] Xiangxiang Chu, Zhi Tian, Bo Zhang, Xinlong Wang, and Chunhua Shen. Conditional positional encodings for vision transformers. In The Eleventh International Conference Learning Representations, 2023. 6 [12] Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias NieBner. Scannet: Richly-ane annottaad 3d reconstructions of indoor scenes. In Proc. Computer Vision and Pattenn
10

---

Recognition (CVPR), IEEE, 2017. 6,
[13] Angela Dai, Daniel Ritchic, Martin Bokeloh, Scott Reed, Jürgen Sturm, and Matthias NieBner. Scancom- plete: Large-scale seene completion and semantic segmentation for 3d scans. CoRR, abs/1712.10215,, 2017.
[14]
DeepSeek-AL Deepseek-v3 technical report, 2025. 1.3
[15] Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. Decaf: A deep convolutional activation feature for generie visual recognition. In Eric P. Xing and Tony Jebara, editors, Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, pages 647-655, Bejing, China, 22-24 Jun 2014. PMLR. 3 [16] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition scale. ICLR, 2021. 3,3,4 4
[17] Chin-Lun Fu, Zih-Ching Chen, Yun-Ru Lee, and Hung-yi Lee. AdapterBias: Parameter-efffiient token- dependent representation shift for adapters in NLP tasks. Marine Carpuat, Marie-Catherine Marnelfe. and Ivan Vladimir Meza Ruiz, editors, Findings of the Association for Computational Linguistics: NAACL 2022, pages 2608-2621 Seattle, United States, July 2022. Association for Computational Linguistics. 3 [18] Takahiko Furuya. Token adaptation via side graph convolution for efficient fine-tuning of 3d point cloud transformers, 2025. 4
[19] Benjamin Graham, Martin Engelcke, and Laurens van der Maaten. 3d semantic segmentation with submanifold sparse convolutional networks. 2018 /EEE/CVF Conference on Computer Vision and Pattern
Recognition, Jun 2018.
[20] Yulan Guo, Hanyun Wang, Qingyong Hu, Hao Liu, Li Liu, and Mohammed Bennamoun. Deep learning for point clouds: A survey. IEEE transactions on pattern analysis and machine intelligence, 2020. [21] Lei Han, Tian Zheng, Lan Xu, and Lu Fang. Occuseg: Occupancy-awan
3d instance segmentation. 2020
IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2937-2946, 2020.
[22] Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg- Kirg-Kirkaatrick, and Graham Neubig. Towards a unified
view of parameter-efficien transfer learning. In International Conference on Learning Representations.
2022. 3, 5,5
[23] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15979-15988, 2022. 3
[24] Ji Hou. Benjamin Graham, Matthias Nießnner and Saining Xie. Exploring data-efficient 3d scene under- standing with contrastive scene contexts. CoRR, abs/2012.09 6.5 2020. 3
[25] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski. Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter--fficient transfer learning for NLP. In Proceedings of the 36th International Conference on Machine Learning, 2019, 2, 3,4, 7, 8, 23
[26] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on
Learning Representations, 2022. 2, 4. 7,6 8, 23
[27] Qingyong Hu, Bo Yang. Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, and Andrew
Markham. Randla-net: Efficient semantic segmentation of large-scale point clouds. CoRR, abs/191 1.111.12222 1236,
2019. 3
[28] Zhiqiang Hu, Yihuai Lan, Lei Wang, Wanyu Xu, Ee-Peng Lim, Roy Ka-Wei Lee, Lidong Bing, and Soujanya Poria. Lim-adapters: An adapter family for parameter-effiiient fine-tuning of large language
models. arXiv preprint (2ti:22044019333 2023, 4
[29] Md Amirul Islam*, Sen Jia", and Neil D. B. Bruce How much position information do convolutional
neural networks encode? In International Conference on Learning Representations, 2020. 6
[30] Andrew Jaegle, Felix Gimeno, Andrew Brock, Andrew Zisserman. Oriol Vinyals, and Joao Carreira.
Perceiver: General perception with itcrative attention, 2021. 9
[31] Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and
Ser-Nam Lim. Visual prompt tuning. In European Conference Computer Vision (ECCV), 2022. 3
[32] Damjan Kalajdzievski. A rank stabilization scaling factor for fine-tuning with lora, 2023. 4
[33] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tcte Xiao, Spencer Whitehead, Alexander C. Berg, Wan- Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, Oct. 2023. 3. 7
[34] Xin Lai. Jianhui Liu, Li Jiang, Liwei Wang. Hengshuang Zhao, Shu Liu, Xiaojuan Qi, and Jiaya Jia. Stratified transformer for 3d point cloud segmentation. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pages 8490-8499, 2022. 3, 4
[35] Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Kosiorek, Seungjin Choi, and Yee Whye Teh. Set transformer: neural networks. In Kamalika Chaudhuri and
A framework for attention-base Ruslan Salakhutdinov, editors. Proceedings of the 36th International Conference on Machine Learning. volume 97 of Proceedings of Machine Learning Research, pages 3744-3753 PMLR, 09-15 Jun 2019. 9 [36] Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-effiiintt prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2021. 5,6
sermutation-invarian

---

[37] Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting the Association for Compulational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Pupers). Association for Computational Linguisties, 2021. 2, 5, 6, 7, 23
[38] Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. pointenn: Convolution on x-transformed points. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc.,
2018. 3
[39] Dingkang Liang, Tianrui Feng, Xin Zhou, Yumeng Zhang. Zhikang Zou, and Xiang Bai. Parameter- efficient fine-tuning in spectral domain for point cloud learning. arXiv preprint arXiv:2410.08// 14, 2024, 4,
23
[40] Haokun Liu, Derek Tam, Mugeeth Mohammed, Jay Mohta, Tenghao Huang. Mohit Bansal, and Colin Raffel. Few-shot parameter- efficieete fine- fine-tuning is better and cheaper than in-context learning. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. 4
[41] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveyx, 55(9):1-35, Jan. 2023.
[42] Shih- Liu, Chien-Yi Wang. Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang- -ing Cheng, and Min-Hung Chen. Dora: weight-decomposed low-rank adaptation. In Proceedings of the 41st International Conference on Machine Learning, (C.L'244 JMLR.org, 2024. 4
[43] Xiao Liu, Kaixuan Ji. Yicheng Fu, Weng Tam, Zhengxiao Du Zhilin Yang. and Jie Tang. P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 61-68, Dublin, Ireland, May 2022. Association for Computational Linguistics. 3
[44] Yunze Li Yi. Shanghang Zhang, Qingnan Fan, Thomas A. Funkhouser, and Hao Dong P4contrast: Contrastive learning with pairs of point-pixel pairs for RGB-D D scene understanding. CoRR, abs/2012.130883
2020,
3
[45] Ze Liu, Han Hu, Yue Cao, Zheng Zhang, and Xin Tong. A closer look at local aggregation operators in point cloud analysis. ECCV, 2020. 3
[46] Dening Lu, Qian Xie, Mingqiang Wei, Kyle Gao, Linlin Xu, and Jonathan Li. Transformers in 3d point clouds: A survey, 2022.
[47]
Sourab Mangruikar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, and Benjamin Bossan. Peft: of-te-of-the-arr art parfieter-efffcientt fine tne-ttning methods. https: / thub. com/hugu ingface/
peft, 2022. 7
[48]
Kaichun Mo, Shilin Zhu, Angel X. Chang, Li Yi, Subarna Tripathi, Lconidas J. Guibas, and Hao Su. PartNet: A large-scale benchmark for fine-graing ain-ggrined and hicrarchical part-level 3D object understanding. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019. 4
[49]
OpenAL. Hello gpt-4o, 2024. 1,
[50]
Maxime Oquab, Timothée Darcet, Theo Moutakanni, Huy V. Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Russell Howes, Yao Huang. Hu Xu, Vasu Sharma, Shang-Wen Li, Wojeiech Galuba, Mike Rabbat, Mido Assran, Nicolas Ballas, Gabriel Synnaeve, Ishan Misra, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski.
Dinov2: Learning robust visual features without supervision, 2023. 1,
[51] Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. Adaptcrfusion: -on-destructive task composition for transfer learning. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. Association for Computational Linguistics, 2021. 1,3
[52] Clifton Poth, Hannah Sterz, Indraneil Paul, Sukannya Purkayastha, Leon Engländer, Timo Imhof, Ivan Vulić, Sebastian Ruder, Tryna Gurevych, and Jonas Pfeiffer. Adapters: A unified library for parameter- efficient and modular transfer learning. In Proceedings of the 2023 Conference on Empirical Methods Natural Language Processing: System Demonstrations, pages 149-160, Singapore, Dec. 2023. Association for Computational Linguistics. 7
[53] Charles Ruizhongtai Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. Pointnet: Deep learning on point
sets for 3d classification and segmentation. CoRR, abs/1612.00593, 2016. 3
[54]
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J. Guibas. Pointnet++:: Deep hierarchical feature
learning point sets in a metric space. CoRR, abs/1706.02413,, 2017.
[55]
Zekun Qi, Kunpei Dong, Guofan Fan, Zheng Ge, Xiangyu Zhang, Kaisheng Ma, and Li Yi. Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining. In International
Conference on Machine Learning (ICML), 2023. 2023. 22, 23
[56]
Guocheng Qian, Yuchen Li, Houwen Peng, Jinjie Mai, Hasan Hammoud, Mohamed Elhoseiny, and Bernard Ghanem. Pointnext: Revisiting pointnet++ with improved training and scaling strategics. In Advances in
Neural Information Processing Systems (NeurIPS), 2022. 3
[57] Haibo Qiu, Baosheng Yu, and Dacheng Tao. bollect-and-distriien transformer for 3d point cloud analysis.
arXiv preprint arXiv:2306.00 235,0 2023. 3
[58] Qwen. Qwen2.5 technical report, 2025.
12

---

[59]
Alec Radford, Jong Wook Kim. Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. 1.3
[60] Sylvestre-Alvise Rebuffi, Hakan Bilen,
Andrea Vedaldi. Learning multiple visual domains with residual adapters. In Proceedings of the 31 International Conference on Neural Information Processing
Systems, NIPS' 17, page 506-516, Red Hook, NY, USA, 2017. Curran Associates Inc. 3
[61] David Rozenberszki, Or Litany, and Angela Dai. tangguage-groundee indoor 3d semantic segmentation in the wild. In Proceedings of the European Conference on Computer Vision ( (ECCV), 2022. 3, 6, 7, 22, 23
[62] Jan-Martin O. Steitz and Stefan Roth. Adapters strike back. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 23449-23459, IECE, June 2024. 3.4
[63] Hongyu Sun, Yongcai Wang, Wang Chen, Haoran Deng, and Deying Li. Parameter-efficient prompt learning for 3d point cloud understanding. In IEEE International Conference on Robotics and Automation (ICRA), 2024. 4
[64]
Yiwen Tang, Ray Zhang, Zoey Guo, Xianzheng Ma, Bin Zhao. Zhigang Wang, Dong Wang, and Xuelong Li. Point-peft: Parameter-efficiet line-tuning for 3d pre-trained models. In Proceedings of the AAAI
Conference on Artificial Intelligence, volume 38, pages 5171-5179, 2024, 4
[65] Lyne Tchapmi, Christopher Choy, Iro Armeni, Jun Young Gwak, and Silvio Savarese. Segcloud: Semantic segmentation of 3d point clouds. 2017 International Conference on 3D Vision (3DV), Oct 2017. 3 [66] Hugues Thomas, Charles R. Qi. Yran-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J. Guibas. Kpconv: Flexible and deformable convolution for point clouds. CoRR, abs/1904, 1904.0889.
2019. 3
[67] Hugo Touvron. Thibaut Lavril, Gautier Izacard, Xavier Martinet, Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models, 2023.
[68] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L. ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.. 2017. 3.4
[69]
Peng- Song-Shuai Wang. Octformer: Octree-based transformers for 3d point clouds. ACM Transactions on Graphics, 42(4):1-11, July 2023. 3. 4
[70] Peng-Shuai Wang, Yang Liu, Yu-Xiao Guo, Chun-Yu Yu Sun, and Xin Tong. O-CNN: Octree-based Convo-
lutional Neural Networks for 3D Shape Analysis. ACM Transactions on Graphics (SIGGRAPH), 36(4),
2017.3 3
[71] Song Wang, Xiaolu Liu, Lingdong Kong, Jianyun Xu, Chunyong Hu, Gongfan Fang. Wentong Li, Jianke Zhu, and Xinchao Wang. Pointlora: Low-rank adaptation with token selection for point cloud learning. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 6605-6615, 2025. 4, 23 (72) Yaqing Wang. Sahaj Agarwal, Subhabrata Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Hassan Awadallah, and Jianfeng Gao. Adamix: Mixture-of of adaptations for crrameter-efffcien model tuning. arXiv preprint
arXiv:2205. 12410. 2022. 3
[73] Yue Wang. Yonghin Sun, Ziwei Liu, Sanjay F. Sarma, Michael M. Bronstein, and Justin M. Solomon.
Dynamic graph CNN for learning on point clouds. CoRR, abs/1801.07829, 2018. 3
[74] Zirui Wang, Zihang Dai, Barnabas Poczos, and Jaime Carbonell. Characterizing and avoiding negative transfer. In 2019 TEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page
11294, IEEE, June 2019. 7
[75] Wenxuan Wu, Zhongang Qi, and Fuxin Li. Pointconv: Deep convolutional networks on 3d point clouds.
CoRR, 1.07246. 2018. 3
[76] Xiaoyang Wu, Daniel De DeTone, Duncan Frost, Tianwei Shen, Chris Xie, Nan Yang, Jakob Engel, Richard
Newcomhe, Hengshuang Zhao and Julian Strauh. Sonata: Self-supervised learning of reliable point
representations. In CVPR. 2025. 1. 3, 6, 7, 8, 9, 22, 23
[77] Xiaoyang Wu, Li Jiang, Peng-Shuai Wang. Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang, Tong Hc, and Hengshuang Zhao. Point transformer v3: Simpler, faster, stronger. In CVPR, 2024. 1, 4, 7, 8, 22, 23
[78] Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, and Hengshuang Zhao. Point transformer v2: Grouped
vector attention and partition-hased pooling. In NeurlPS, 2022. 3,4
[79] Xiaoyang Wu, Zhuotao Tian, Xin Wen, Bohao Peng. Xihui Liu, Kaicheng Yu, and Hengshuang Zhao. Towards large-scale 3d representation learning with multi-dataset point prompt training. In CVPR, 2024. 1,
6, 7, 8, 22
[80] Xiaoyang Wu, Xin Wen, Xihui Liu, and Hengshuang Zhao. Masked scene contrast: A scalable framework for unsupervised 3d representation learning. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9415 9424. IEEE, June 2023. 3, 22, 23
[81] Zhengxuan Wa, Aryaman Arora, Zheng Wang. Atticus Geiger, Dan Jurafsky, Christopher D Manning, and Christopher Potts. ReFT: Representation finetuning for language models. In The Thirty-eighth Annual
Conference on Neural Information Processing Systems. 2024. 4
[82] Saining Xie, Jiatao Gu, Demi Guo, Charles R. Qi, Lconidas J. Guibas, and Or Litany. Pointcontrast: Unsupervised pre-training for 3d point cloud understanding. CoRR, abs/2007. 10900,, 2020. 3
[83] Y. Xie, J. Tian, and X. X. Zhu. Linking points with labels in 3d: A review of point cloud semantic segmentation. IEEE Geoscience and Remote Sensing Magazine, 8(4):38-59 2020.
13

---

[84]
Chandan Yeshwanth, Yueh-Cheng Liu, Matthias NieBner, and Angela Dai. Scannet++: A high-fidelity dataset of 3d indoor scenes. In Proceedings of the IEEE/CVF International Conference Computer
Vision, pages 12-22, 2023. 6.7
[85] Li Yi, Vladimir G. Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu Lu, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3d shape collections.
ACM Trans. Graph., 35(6), Nov. 2016. 4, 22, 23
[86] Jason Yosinski. Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In Proceedings of the 28th International Conference on Neural Information Processing Systems Volume 2. NIPS 14, page 3320-3328, Cambridge, MA, USA, 2014. MIT Press. 7
[87] Yaohua Zha, Jinpeng Wang, Tao Dai, Bin Chen, Zhi Wang, and Shu-Tao Xia. Instance-aware dynamic prompt tuning for pre-trained point cloud models. Proceedings of the IEEE/CVF International Confer-
ence on Computer Vision (ICCV), 2023,
[88]
Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo 7hao. Adaptive budget allocation for paramcter-effiient fine-tuning. The Eleventh International
Conference on Learning Representations, 2023. 4
[89]
Shaochen Zhang, Zekun Qi, Runpei Dong, Xiuxiu Bai, and Xing Wei. Positional prompt tuning for efficient 3d representation lcarning, 2024. 4
[90] Zhi Zhang. Qizhe Zhang, Zijun Gao, Renrui Zhang, Ekaterina Shutova, Shiji Zhou, and Shanghang Zhang. IEEE/CVF Conference Computer Gradient-based parameter selection for efficient fine-tuning. In 2024
Vision and Pottern Recognition (CVPR), pages 28566 28577. IEEE, June 2024. 3
[91] Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip Torr, and Vladlen Koltun. Point transformer, 2021. 3,
[92] Xin Zhou, Dingkang Liang. Wei Xu, Xingkui Zhu, Yihan Xu, Zhikang Zou, and Xiang Bai. Dynamic adapter meets prompt tuning: Parameter-efficient transfer learning for point cloud analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 14707-14717, 2024.
[93] Yaoming Zhu, Jiangtao Feng, Chengqi Zhao, Mingxuan Wang. and Lei Li. Counter-interference adapter for multilingual machine translation. Findings of the Association for Computational Linguistics: EMNLP 2021, page 281 12-2283.. Association for Computational Linguistics, 2021. 3
14

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility. transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (I-2 sentence) justificcton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable answer "[No] provided proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[N] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write justification elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
15

---

Justification: The paper should point out any strong assumptions and how robust the results are violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: no theoretical proof needed
Guidelines:
The answer NA means that the paper does include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
16

---

Justification:
all publicly available
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
will open-source for reproduction
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips- cc/
Beaiii/guudde//odeeiii wttttbmmmissiiiii aesionoooicy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (httpa: /nipp. ctaaps.cc/public/// hwtpliic/ggiiisee SCooeSubmiiieeecccece onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
17

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: publicly available Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: following the established protocols
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula,
call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: indicated by the public codebase
Guidelines:
The answer NA means that the paper does not include experiments.
18

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics /t ctwth//eeetiiieeeeleeelaeaeeee ines?
Answer: [Yes]
Justification: follow
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: not related
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: not related
19

---

Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All are open-source and are properly cited
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thcode com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
will be integrated into open-sourced codebase
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
20

---

Answer:
[NA]
Justification: not related
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification:not related
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: not related
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non non-standard components.
Please refer to LLM policy (https: /neuripss.... ct/ /20/ for what should or should not be described.
21

---

Supplementary
In this supplementary, we provide more materials as follows,
Sec. A details the implementation, training, and inference;
Sec. B presents additional experiments and analyses with tight parameter budgets
Sec. C offers more qualitative visualizations.
A Details of Implementation, Training, and Inference
Our implementation based on the open-source codebase Pointcept (here) and follows the official implementations for Sonata [76], PPT [79], well PTv3 [77].
Leveraging the parameter efficiency of our method, we train on single 4090 GPU for much fewer epochs to obtain the reported performance. For example, we train on ScanNet for 100 epochs, contrast to the 800 epochs the released Sonata configuration. Our code is available at https://gituub.com/iiaa/ TannaaaLL..TT
Tab. summarizes empirical latency and memory usage. We intentionally refrain from deployment- specific optimizations such as LoRA weight merging. Although the global attention and local convolution modules introduce extra cost, the runtime and memory footprint stay within the same order of magnitude as other PEFT baselines.
B More Experiments and Analysis
In spite of the promising results shown in Sec. 3, we suggest that it can better demonstrate the full potential of PEFT methods under broader and more challenging settings.
PEFT for shape analysis. While GEM is motivated by PEFT for large-scale 3D scenes, most existing 3D PEFT methods target object-level understanding, as discussed in Sec. 2. To assess cross- cross-domain generalization and compare against these existing 3D-specific PEFT methods, we further evaluate GEM on 3D shape datasets, ShapeNetPart [85]. As shown in Tab. 8, GEM remains competitive and achieves the best performance. We also observe that common backbones for shape analysis, such as ReCon [55], attach large MIP heads that account for approximately 20% of all parameters, rendering fine-tuning inefficient. To broaden the applicability of backbone models for 3D shapes, an interesting direction for future work is to develop architectures with more parameter-efficient heads that are better suited to fine-tuning.
PEFT for 3D convolutional networks. While our primary experiments apply GEM to state-of-the-art backbones, mainly transformer models, we also verify its generality on convolutional architectures. In particular, we integrate GEM into SparseUNe [10]. a 3D convolutional model, following the MSC [80] protocol to pre-train on ScanNet [12] and fine-tune on Sea200 [61]. As reported in Tab. 9, GEM improves performance by +4.6 mloU over linear probing and outperforms other PEFT baselines, demonstrating robustness beyond transformer models. We notice the gains are narrower than those transformer backbones, largely due to the limited capacity of convolutions, as also indicated by a near-collapse linear probing performance this setting.
PEFT under tight budgets. To stress parameter efficiency, we constrain all methods to the same limited learnable- leamable-parameter parameter budgets, including enforcing rank r 1. allowing 0.1 % parameters, and % parameters. As reported in Tab. 10, GEM delivers consistently higher accuracy, whereas several baselines fail when the budget becomes extremely stringent.
C More Visualizations
We provide more qualitative results regarding the attention map generated by our latent tokens in Fig. 4. Despite shared across the stages, the latent tokens appear to focus on different parts of the scenes, providing different scene context to the backbone models at different stages.
22

---

Methods
Params, Memory Latency
Shape-Pane Seg.
Params
ShapeNetPart [85]
Base model
Methods
Learn. Cls. mloU Inst. mloU
Sonata (full.) [76]
124.8M
8.2G
61.8mm 18ms
Full fine-tuning
Sonata (ft.)
108.5M
6.4G
50.2ms
ReCon (ft.) [55]
27.06
84.52
86.1
PEFT methods
PEFT methods
Sonata (lin.)
0.02M
6.4G
50.2ms
*Adapter [25]
2.8M
6.7G
50.6ms
ReCon (lin.) [55]
5.23M
83.06
85.2
-LoRA [26]
1.9M
7.9G
52.8ms
PoointLoRA [71]
5.63M
83.98
85,4
PPrompt Tunning [37]
5.5M
7.3G
51.5ms
PointGST [39]
5.59M
83.98
85.8
-GEM (ours)
1.8M
7.1G
57.8ms
GGM (ours)
5.58M
84.02
85.8
Table 7. Inference efficiency of PEFT methods.
Table 8. Generalizing to 3D shapes.
We benchmark with the batch size fixed to I.
Tight Budgets
Params
ScanNer200 Val [61]
Tight Budgets
Params
ScanNet Val [12]
Methods
Learn. Pet.
mloU mAce allAcc
Methods
Learn
Pet.
mIoU
allAce
Training from scratch
Training from scratch
SparseUnet [10)
39.2M 100% 25.0
32.9
80.4
PTv3 [77]
124.8M
100%
77.6
85.0
92.0
Full fine-tuning
Full fine-tuning
SparseUnet (ft.) [80] 0.02M 0.05%
32.0
41.6
82.3
Sonata (full.) [76]
124.8M
100%
79.4
86.1
Sonata (ft.)
108.5M
100%
78.3
85.9
92.3
PEFT methods with rank=/
PEFT methods with rank=1
SnarseUnet (lin.)
0.02M 0.05%
1.5
2.5
53.6
Sonata (lin.)
0.02M 0.02%
72.5
BitFii [6]
0.03M 0.07%
4.8
6.9
62.7
Adapter [25]
0.05M 0.04%
74.0
90.5
-Adupter [25]
0.6M 1.5%
9.5
13.4
69.5
LoRA [26]
0.05M 0.05%
42.5
55,4
75.1
-LoRA [26]
0.8M 2.0%
13.2
17.6
74.7
-Prompt Tunning [37]
0.05M 0.05%
72.9
-GEM (ours)
0.6M
1.5%
15.2
22.9
75.6
-GEM (ours)
0.07M 0.06%
75.2
84.8
Table
Generalizing to convolutional networks.
PEFT methods with 0.1 params.
Sonata (lin.)
Adapter [25]
0.1M
0.1%
75.1
84.7
LoRA [26]
0.1M
0.1%
75.0
84.4
.Prompt Tunning [37]
0.1M
0.1%
73.5
83.9
90.3
.GEM (ours)
0.IM
0.1%
76.5
85.5
PEFT methods with 1% params.
Sonata (lin.)
Adapter [25]
1.1M
1.0%
76.6
85.3
LoRRA [26]
1.IM
1.0%
76.7
85.6
+Prompt Tunning [37]
L.IM
1.0%
73.8
84.2
90.4
.GEM (ours)
1.IM
1.0%
78.2
86.3
92.2
Table 10. Performance with tight budgets.
23

---

Epp
Stage1 1
Saage2
Stepe3
Stagel
GT
Figure 4. Attention maps of our latent tokens in context adapter.
24


Reviews:
[{"review_id": "CIJfFLn8Oq", "reviewer": "Reviewer_gyor", "summary": "The authors introduce the Geometric Encoding Mixer (GEM), a novel geometry-aware PEFT module specifically designed for 3D point cloud transformers. GEM explicitly integrates fine-grained local positional encodings with a lightweight latent attention mechanism to capture comprehensive global context, thereby effectively addressing the spatial and geometric distribution mismatch. Extensive experiments demonstrate that GEM achieves performance comparable to or sometimes even exceeding full fine-tuning, while only updating 1.6% of the model’s parameters, fewer than other PEFT methods.", "questions": "Please refer to the weakness", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The writing is logical and easy to follow\n- The experimental part of the article shows competitive results\n- While achieving a significant performance improvement, it does not increase the computational burden.\n\nWeakness\n- Please add experiments on other datasets similar to Tab3\n- Regarding the design of the Spatial adapter, I am curious whether using a simple 3D convolution instead can achieve the same effect?", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable suggestions. We provide our feedback as follows.\n\n---\n>**Q1: More experiments on other datasets similar to Tab3.**\n\n**A1:** We appreciate the suggestion. As you may have noticed, due to the space limit, we include the extended experiments, with additional datasets and settings, in the supplementary material (Tab. 6–8).\nWe will incorporate both these experiments and their discussion into the main paper in revision.\n\n>**Q2: Simple 3D convolution instead of Spatial Adapter (SA).**\n\n**A2:** We follow the setting of our ablations and conduct a controlled experiment on ScanNet dataset to explore your suggestion.\nWith our spatial adapter (SA) replaced with simple 3D convolution (denoted SA-conv), we summarize the results below:\n| Methods | Params. |  Pct. | mIoU |\n|-|-:|-:|-|\n| Sonata (lin.)| 0.02M | 0.02% | 72.5 |\n| + SA      |  1.2M |  1.0% | 77.2 |\n| + SA-conv | 72.9M | 40.2% | 71.1 |\n| + GEM     |  1.8M |  1.6% | 78.3 |\n| + GEM (SA-conv) | 73.6M | 40.4% | 71.6 |\n||||\n\nAs shown in the table, using a simple 3D convolution drastically increases the number of trainable parameters (~40% vs. 1%) and degrades performance (71.1 vs. 77.2 mIoU for SA; 71.6 vs. 78.3 for GEM). This supports our design choice.\n\nWe analyze that the core issue is that 3D kernels (*eg* a common 3x3x3 kernel) introduce $27C^2$ parameters, where $C$ denotes the backbone channel width. This is significantly more than a transformer block with qkv-projection, out-projection, and the feed‑forward network, requiring only a total of $(3 + 1 + 8)C^2 = 12C^2$ parameters.\nAt the fine-tuning stage, especially with limited data, 3D convolution would thus further trigger overfitting and optimization difficulties, negatively impacting the adaptation of backbone models.\n\nInstead, SA circumvents this issue with the lightweight bottleneck design.\nIt thus captures local geometry, with a much better balance between expressiveness and parameter efficiency.\nThis allows GEM to be performant within a ~1.6% parameter budget, and converge faster with fewer training epochs.\n\n---\nThank you again for your thoughtful review; we welcome any further questions or suggestions."}, {"review_id": "MZOdMM0L7i", "reviewer": "Reviewer_A47U", "summary": "This paper presents GEM (Geometry Encoding Mixer), a parameter-efficient fine-tuning (PEFT) method for 3D scene segmentation using pretrained point cloud transformers. Recognizing the limitations of existing PEFT techniques (e.g., LoRA, adapters, prompt tuning) in modeling spatial and geometric structures inherent in 3D point clouds, the authors propose two key components: a Spatial Adapter (SA) leveraging 3D convolution to capture local geometry and a Context Adapter (CA) using latent tokens to inject global scene context. Evaluated on ScanNet, S3DIS, and ScanNet200, GEM achieves performance comparable to full fine-tuning while updating only 1.6% of parameters.", "questions": "- The experiment only compared classic fine-tuning methods such as prompt tuning and LORA. Can they be compared with 3D PEFT related work, and what is the effect?\n- GEM seems to be a fine-tuning method, but the experiment is only based on the Sonata baseline. If we switch to other baselines, can it still be effective?\n- Is there any experimental or theoretical support for the author's claim that the SA and CA modules respectively capture local geometric structures and global scene context?\n- Will SA/CA introduce memory bottlenecks and significant computational overhead for dense scenarios?", "limitations": "The authors discussed some limitations of their work. The following limitations also need to be discussed.\n\n- Dependence on pretrained Point Transformer-style architectures may limit general applicability.\n- While parameter count is reduced, training still requires backpropagation into early layers, which may impact training efficiency on dense point clouds (as acknowledged by the authors).\n- Ablation studies lack theoretical insights into why SA and CA are effective.\n- Experiments use only indoor datasets; performance in unstructured outdoor environments (e.g., forests, construction sites) is unknown.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n- Clear motivation: Highlights limitations of existing PEFT approaches in the 3D domain due to lack of spatial awareness.\n- Novel adaptation design: Combines 3D convolution (for local refinement) and latent token attention (for global context) in a lightweight framework.\n- Strong empirical results: Demonstrates competitive or superior performance on multiple datasets and settings.  \n\n**Weaknesses**\n- Missing direct comparisons with other 3D PEFT methods. While classic PEFT methods (e.g., LoRA, prompt tuning) are included, the paper does not compare against recent 3D-specific PEFT works.\n- Method description is imbalanced. A large portion (Lines 147–185) is devoted to reviewing prior PEFT works, while the core GEM method (Lines 186–223) is described relatively briefly. Lacks intuitive explanation or architectural figures for how/why SA and CA work in practice.\n- GEM is evaluated primarily with Sonata and PTv3-PPT. No results on other architectures, making it unclear how transferable GEM is across different 3D backbones.\n- SA and CA mechanisms lack theoretical or empirical justification. The paper claims that SA captures local geometry and CA captures global context—but these claims are only supported by performance gain, without deeper analytical or theoretical backing.", "was_revised": true, "final_justification": "The authors have addressed my first two key concerns.", "rebuttal": "Thank you for the constructive comments. We address your concerns as follows.\n\n---\n### Main Comments\n>**Q1: Comparison to other 3D‑specific PEFT methods.**\n\n**A1:** Thanks for the suggestion.\n(1) We would like to clarify that, to the best of our knowledge, we are among the first to systematically investigate parameter-efficient fine-tuning (PEFT) for large-scale 3D scene understanding. As a result, there are no comparable existing 3D PEFT baselines specifically designed for scene-level tasks, *e.g.*, 3D scene segmentation, making our research a novel and underexplored setting.\n\nExisting 3D PEFT methods excessively focus on object-level understanding (3D object classification, shape-part segmentation). The underlying assumptions of these methods do NOT transfer to scenes with millions of points:\n\n1. *Point count and complexity.* Object-level methods operate on a few thousand points, allowing costly operations like spectral transforms, graph construction, or full attention. Scene-level datasets involve millions of points, making such operations intractable.\n\n2. *Regular point distribution*: Shape-level inputs are often surface-regular and allow aggressive downsampling, *e.g.*, farthest‑point sampling. This leads to compact token sets, typically ~128 tokens, which contrasts with irregular, unstructured distributions in real-world scenes that require preserving fine-grained geometry.\n\nWhile earlier scene-level approaches like PointNet++ [1] or SegCloud [2] used tokenization heuristics (e.g., FPS, superpoints), such strategies sacrifice fine spatial detail. Modern 3D segmentation models now rely on U-Net-like or transformer backbones with direct point processing and augmented attention variants to retain these details, but they are distinct from PEFT in this context.\n\n(2) To address your concern specifically and test cross-domain applicability, we further apply GEM to 3D shape datasets, ShapeNetPart, and still lead the performance.\n\n| Methods|Params.|Cls. / Inst. (mIoU)| \n|-|-:|-|\n| ReCon (ft.)|27.06M|84.52 / 86.1|\n| + PointLoRA|5.63M|83.98 / 85.4|\n| + PointGST |5.59M|83.98 / 85.8|\n| + GEM|5.58M|84.02 / 85.8|\n|||\n\n(3) Furthermore, we also implement a direct extension of the recent 3D PEFT method from object tasks to scene understanding. Specifically, we extend PointGST [4] to evaluate on ScanNet based on released code.\n| Methods  |Params.|Pct.|mIoU|\n|-|-:|-:|-|\n|Sonata (lin.)|0.02M|0.02%|72.5|\n|+ PointGST|2.2M|2.0%|74.8|\n||||\n\nDespite PointGST stands the SOTA performance of PEFT on 3D shape, its application on 3D scenes with millions of points is on par with BitFit that only trains the bias (74.7 mIoU), and lags behind other PEFT methods, such as LoRA (76.7 mIoU), as well as the proposed GEM (78.3 mIoU).\n\nIn addition, we notice unstable training of PointGST. On average, we resume from NaN loss twice per model training, which can be due to the unstable FPS results in various 3D scenes. As FPS is shared by methods on 3D shape, this observation indicates a significant gap between PEFT for 3D shape and PEFT for large-scale scenes.\n\n---\n> **Q2: Lacks explanation on SA and CA.**\n\n**A2:** We would like to extend the discussion as following.\n\n(1) In fact, we provide the demonstration of CA and SA in Fig. 2, with its overall placement indicated in Fig. 1, together with all other common PEFT methods.\n\n(2) Although we devote to provide a clear motivation for the design of the proposed GEM, we acknowledge that the presentation of existing PEFT method may bring in too many formulas that could be over-whelming.\nIn the revision, we will adopt your suggestion to shorten the formulas and provide a concise presentation, *e.g.*, using $\\text{Attn}(\\cdot)$, instead of expanded form.\n\n---\n> **Q3: Extending to other baselines.**\n\n**A3:**\nWhile we have implemented our methods on Sonata and PointTransformerV3 (PPT), both are transformer-based model.\n\nSince our method can be regarded as a light-weight side-network, we can seamlessly incorporate our method into other baselines.\nWe thus further extend our methods to SparseUnet [3], a 3D convolutional network that follows a drastically different design.\n\nSpecifically, we follow MSC [5] that pretrains SparseUNet on ScanNet dataset and finetunes on ScanNet200.\n|Methods|Params.|Pct.|mIoU|\n|-|-:|-:|-|\n|SparseUnet (ft.)|39.2M|100%|32.0|\n|SparseUnet (lin.)|0.02M|0.05%|1.5|\n|+ BitFit|0.03M|0.07%|4.8|\n|+ Adapter|0.6M|1.5%|9.5|\n|+ LoRA|0.8M|2.0%|13.2|\n|+ GEM|0.6M|1.5%|15.2|\n|||||\n\nGEM leads other PEFT methods. This result, together with GEM's performance on 3D shape (A1), demonstrates the applicability of GEM to various baselines. We also notice the gains are narrower than those for transformer models, potentially due to the limited capacity of convolutions indicated by the near-collapse performance of linear probing.\n\n---\n> **Q4: Theoretical and empirical justification for SA and CA.**\n\n**A4:**\nWe agree that deeper theoretical analysis is beneficial and will present more details in the revision.\nMeanwhile, we would like to highlight the followings.\n\n(1) For theoretical aspects, L200-204 and L212-215 lay out how the design of SA and CA reflects our motivation, and also explain the functional roles of SA and CA. Below, we give further elaboration.\n\n1. *Convolution captures local spatial features.* Convolution applies the same learnable kernel to every neighbourhood, enforcing weight sharing and translation equivariance [6] and leading to locality inductive bias [7, 8]. In 3D processing, a typical 3D kernel (3x3x3) therefore learns filters that respond to local geometric primitives [1, 9], and can serve as a form of positional encoding [10].\n\n2. *Attention models global dependencies.* Attention computes a weighted sum over all tokens, giving each query a direct path to any key in a single layer [11], thus mixing information across arbitrarily far position and approximating long-range correlations [14, 15]. Further theory shows attention can have low-rank factorization that retains the dominant global modes [14], also validated in vision tasks [15, 16].\n\nIn the proposed method, SA implements a light-weight convolutional bottleneck, and CA is essentially attention with latent tokens as low-rank approximation.\nTherefore, we are confident that SA and CA reflects our motivation to capture local spatial features and provide global context.\n\n(2) For empirical justification, we provide qualitative analysis in Fig. 3 and provide discussion around L311-L321, demonstrating GEM can capture explicit geometry cues.\n\nIn addition, Tab. 4(a) reports isolated SA and CA variants; each surpasses the linear baseline, and their combination gives the full gain, confirming complementary roles.\n\n---\n> **Q5: Memory/computation overhead and training efficiency in dense scenes.**\n\n**A5:**\nWhile the paper has discussed these concerns; the paper also provides preliminary analysis.\n\n(1) For memory and computation overhead, as in L200-204 and L212-215, we provide theoretical analysis on the complexity of SA and CA, which is the same as other PEFT methods, such as Adapter and Prompt Tuning.\n\nWe also provide empirical study on the overhead in Tab. 5 (supp). Though CA and SA introduce cost over backbone, the runtime and memory footprint stay largely the same as other PEFT methods, as indicated by the complexity.\n\nIn dense scenes, we operates on the same resolution of the backbone network, whichs keep the overhead manageable.\n\n(2) For training efficiency, as discussed in L326–330 as well as Sec. B (supp), GEM remains lightweight and converges faster than full fine-tuning, using much fewer epochs (100 instead of 800).\n\nIn addition, while it is tempting to avoid back-propagation into early layer, we find fine-tuning early encoder layers is more performant.\nSpecifically, full fine-tuning (Tab. 1) gives 79.4 mIoU, fine-tuning a newly attached decoder gives 79.1 in mIoU, whereas the performance can reach 79.5 mIoU with GEM, in Tab. 6 (supp).\nHowever, the reveal of the underlying mechanism is beyond the scope of this work.\n\nIn general, we acknoledge the difficulties of handling dense point cloud: it is a shared challenge among the field of 3D vision, and requires joint efforts.\n\n---\n> **Q6: Performance in outdoor environment.**\n\n**A6:**\nWe would mention that we explored transfering to outdoor scenes in Tab. 8 (supp).\nWe evaluate under a challenging setting: transferring an indoor pre-trained backbone to an outdoor benchmark, SemanticKITTI. GEM effectively narrows the gap to model fully trained from scratch on outdoor data.\n\n---\n**References**\n```\n[1] PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. NIPS'17.\n[2] Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs. CVPR'18.\n[3] 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks. CVPR'19.\n[4] Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learning. TPAMI'25.\n[5] Masked Scene Contrast: A Scalable Framework for Unsupervised 3D Representation Learning. CVPR'23.\n[6] Gradient-based learning applied to document recognition. Proc of the IEEE'98.\n[7] On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Location. CVPR'20.\n[8] Visualizing and Understanding Convolutional Networks. ECCV'14.\n[9] KPConv: Flexible and Deformable Convolution for Point Clouds. ICCV'19.\n[10] Conditional Positional Encodings for Vision Transformers. ICLR'23.\n[11] Attention Is All You Need. NIPS'17.\n[12] Limits to Depth Efficiencies of Self‑Attention. NIPS'20.\n[13] Understanding the Expressive Power and Mechanisms of Transformer for Sequence Modeling. NIPS'24.\n[14] Unveiling the Hidden Structure of Self‑Attention via Kernel PCA. NIPS'24.\n[15] An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. ICLR'20.\n[16] Focal Self-attention for Local-Global Interactions in Vision Transformers. NIPS'21.\n```\n---\nThank you again for your thoughtful review; we welcome any further questions or suggestions."}, {"review_id": "4kMhAWazA8", "reviewer": "Reviewer_gwvo", "summary": "This paper introduces ​​Geometry Encoding Mixer (GEM)​​, a parameter-efficient fine-tuning (PEFT) method tailored for large-scale 3D point cloud transformers. GEM addresses the limitations of existing PEFT methods (e.g., LoRA, adapters, prompt tuning) by explicitly modeling ​​local spatial patterns​​ (via a lightweight 3D convolutional bottleneck) and ​​global geometric context​​ (via learned latent tokens).\nExperiments on ScanNet, ScanNet200, and S3DIS show that GEM matches or surpasses full fine-tuning performance while updating only ​​1.6% of parameters​​, with significant reductions in training time and memory.", "questions": "1. The paper notes that CA avoids \"noisy\" global context but lacks analysis of ​​failure cases​​. Under what conditions (e.g., heavy occlusion, sparse scans) does GEM underperform full fine-tuning?\n2.  The CA uses a fixed number of latent tokens (default: 4). Have you experimented with ​​adaptive token allocation​​ (e.g., based on scene complexity or point density)? If not, could this further improve performance?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. GEM achieves ​​near-full fine-tuning performance​​ with ​​1.6% parameter updates​​, reducing computational costs.\n2. GEM is robust to scarce data. With ​​limited labeled data​​, GEM outperforms full fine-tuning strategy, demonstrating strong generalization ability.\n\nWeaknesses:\n1. GEM is only feasible on ​transformer-based architectures​​, which limits the compatibility of the proposed PEFT method on other architectures.\n2. ​​CA uses a ​​fixed number of latent tokens​​. The paper notes that even one token works well, but dynamic token allocation (e.g., based on scene complexity) might further improve adaptability.\n3. The experiments are only conducted on indoor-scene datasets. The performance of GEM on outdoor-scene large-scale datasets (e.g., nuScenes) is not evaluated.", "was_revised": true, "final_justification": "Thanks for the authors' rebuttal. The authors have addressed my concerns and I will keep the initial rating as Borderline accept", "rebuttal": "Thank you for the helpful suggestions. We address each concern below.\n\n---\n> **Q1: Applicability beyond transformers.**\n\n**A1:** GEM is composed of two modular components: Spatial Adapter (SA) and the Context Adapter (CA).\nSA is implemented as a lightweight convolutional bottleneck operating on local neighborhoods, making it agnostic to the backbone architecture.\nCA uses cross-attention with latent tokens and does not depend on the attention layers of backbone.\n\nTherefore, though our primary focus is transformer-based models, due to their dominance in 3D pre-training, GEM functions as a light-weight side network and is compatible with other architectures.\n\nTo demonstrate this, we integrate GEM into SparseUNet [1], a 3D convolutional model. We follow MSC [2] that pretrains on ScanNet and fine-tunes on ScanNet200.\n\n|Methods|Params.|Pct.|mIoU|\n|-|-:|-:|-|\n|SparseUnet (ft.)|39.2M|100%|32.0|\n|SparseUnet (lin.)|0.02M|0.05%|1.5|\n|+ BitFit|0.03M|0.07%|4.8|\n|+ Adapter|0.6M|1.5%|9.5|\n|+ LoRA|0.8M|2.0%|13.2|\n|+ GEM|0.6M|1.5%|15.2|\n|||||\n\nGEM boosts mIoU by +4.6 mIoU over linear probing and also leads other PEFT methods, demonstrating GEM's generalizability beyond transformer models.\nWe also notice the gains are narrower than those for transformer models, largely due to the limited capacity of convolutions, as indicated by the near-collapse performance of linear probing.\n\n> **Q2: Adaptive latent tokens.**\n\n**A2:**\nThanks for the suggestion.\n\n(1) As a preliminary result, our ablation in Tab. 4b shows stable performance of GEM with different numbers of latent tokens:\nThere is only $\\leq 0.3$ mIoU variation when ranging from 1 to 8 tokens.\nThe default setting of 4 tokens offers an effective trade-off between accuracy and efficiency.\n\n(2) We also implement a straightforward latent tokens allocation scheme. We apply spatial pooling on the input point cloud and treat the pooled features as latent tokens, thus allocating more tokens to denser regions. This variant obtains a result of 77.6 mIoU with similar parameter budget (1.7M, 1.6% of the Sonata backbone).\nWhile it also improves over existing PEFT methods, it remains inferior to GEM. We consider the pooling operation injects local information into the tokens, steering them toward local aggregation, similar to the local attention of the 3D backbone, and thus compromising their capability of complementing 3D backbone with global scene context.\n\n> **Q3: Evaluation on outdoor datasets.**\n\n**A3:**\nWe thank the reviewer for bringing this up. In Tab. 8 (supp), we evaluate the transfer ability from an indoor‑pretrained backbone to outdoor scenes of SemanticKITTI.\nGEM improves linear probing +15 mIoU, narrowing the gap to a model directly trained on expensive outdoor scenes.\nThis confirms GEM's ability to generalize beyond indoor scenes without retraining from scratch.\nWe will include this, as well as other experiments, in additional settings from Sec. C (supp), in the revision, as more space is allowed.\n\n> **Q4: Failure‑case analysis.**\n\n**A4:**\nWe appreciate the request for deeper analysis.\nWe find GEM underperforms full fine-tuning in several challenging scenarios.\n\n(1) When given a tight parameter budget, such as rank=1, the performance of PEFT methods degrades, including GEM. As shown in Fig. 1 (c) and Tab. 7 (supp), GEM would underperform full fine-tuning when the budget is under a certain threshold.\n\n(2) When the domain gap is significantly large, PEFT methods, including GEM, would not be able to match the full fine-tuning performance, *e.g.*, transferring to outdoor scenes from an indoor-pretrained backbone as shown in Tab. 8 (supp) and discussed in A3.\n\n> **Q5: Typos, citations, and formatting.**\n\n**A5:**\nThank you for pointing these out. We will correct them in the revision.\n- L149: \"Chanlleges\" -> \"Challenges\"\n- L274: citations should be [3, 4], as listed in the references below\n- Fig.1 (c): We will crop out the border for a cleaner presentation.\n- Tab. 4: It is indented, but a bottom line shall be added, as it can be more consistent with other tables.\n\n---\n**References**\n```\n[1] 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks. CVPR'19.\n[2] Masked Scene Contrast: A Scalable Framework for Unsupervised 3D Representation Learning. CVPR'23.\n[3] How transferable are features in deep neural networks? NIPS'14.\n[4] Characterizing and Avoiding Negative Transfer. CVPR'19.\n```\n---\nThank you again for your thoughtful review; we welcome any further questions or suggestions."}, {"review_id": "0NqjHs9taC", "reviewer": "Reviewer_UJDM", "summary": "This paper explores the parameter-efficient finetuning on 3D point cloud models, for large-scale point clouds. It introduces a Geometric Encoding Mixer (GEM), which includes a Spatial Adapter for local fine-grained information refinement and a Context Adapter for global context modeling. The experiments are extensively conducted on various datasets, and comparion with several finetuning methods in 2D domain.", "questions": "The authors are expected to carefully respond the Weaknesses 1~4 as mentioned above, especially focusing on providing more comparisons and discussing the novelty. \n\nMore importantly, the motivation/necessity of exploring parameter-efficiently learning on point cloud models is welcome to discuss.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**:\n1. The experiments are conducted on various datasets, and more experiments and visualizations are provided in the supplementary material.  \n2. The idea to combine local spatial refinement and global context modeling is meaningful.  \n3. The paper is well-organized.\n\n**Weaknesses**:\n1. The core question is: compared with 2D large models, the current point cloud models are actually **not big**, which **challenges the necessity** of exploring parameter-efficient tuning on point cloud models. As a result, more discussions and experiments are expected to be provided.  \n2. In all tables, the comparison with point cloud parameter-efficient fine-tuning methods discussed in related works is not extensive and inclusive. This should be carefully discussed and compared, especially comparing with some *\"tokenization\"* methods as mentioned in motivation of the paper (Line 7), considering both effectiveness and efficiency.  \n3. In the proposed Context Adapter, the idea of using latent tokens in cross-attention to reduce computational cost is not novel, which has been explored in [1, 2].  \n4. The efficiency comparison and the results on outdoor segmentations should be indicated in the main paper, instead of just supplementary material. More importantly, as the main target of the proposed Context Adapter is to improve efficiency, an efficiency comparison *specifically* designed under *whether using this module or not*, should be provided.\n5. No qualitative results are provided in either the main paper or the supplementary material.   \n\n*Refs*:   \n[1] Perceiver: General Perception with Iterative Attention. ICML 2021.    \n[2] Perceiver IO: A General Architecture for Structured Inputs & Outputs. ICLR 2022. \n\n**Minors**:  \n1. In all tables, what do \"Pct.\" and \"Learn\" mean? Any abbreviation should be explained in the table captions.\n2. L149 typo: Challenges.  \n3. Empty citation at L274.", "was_revised": true, "final_justification": "I have carefully read the reviewer's rebuttal and other reviews.\n\nAlthough this paper lacks some experiments and discussions, it deserves to be accepted.\n\nHowever, the discussions and experiments should be carefully added, and many minors should also be carefully revised in the final version.", "rebuttal": "We thank the reviewer for the constructive feedback and address the main concerns in turn.\n\n---\n>**Q1: Necessity of PEFT for 3D point clouds**\n\n**A1:**\nWe would like to discuss the necessity of 3D PEFT method for scene understanding from the following perspectives.\n\n(1) *Rapid scaling of 3D models.* Recent large point cloud transformers now already exceed 100M parameters and operate on scenes with millions of points, leading to significant GPU memory and storage demands, especially when adapting to multiple downstream tasks.\n\nImportantly, modern 3D backbones are rapidly approaching the scale of 2D vision foundation models, such as ViT and SAM/SAM2, many of which fall in the 100M-300M parameters range.\nHowever, 3D inputs often involve far more tokens than their 2D counterparts (*e.g.*, millions of unordered points vs. 14×14 or 16×16 patches from 224×224 image), making PEFT not only relevant but necessary for 3D scene understanding.\n\n(2) *Necessity of PEFT for large 3D scene models.* While early PEFT methods like BitFit (on BERT-base, 110M) and Adapters (on ResNet, <50M) targeted moderate-sized models, recent works extend PEFT to ViT and SAM (~100M), such as VPT [1], AdaptFormer [2], RLRR [3], and DA-VPT[4], achieving strong performance with minimal trainable parameters. These developments highlight that PEFT is not limited to billion-sacle models.\nRather, it is motivated by the trade-off between efficiency and performance, even at 100M scale.\nIn 3D vision, however, 3D PEFT [5,6] has so far focused primarily on shape-level tasks (*e.g.*, object classification and shape part segmentaion) with models around ~20M parameters.\nIn contrast, scene-level understanding demands far higher input density, token complexity, and model capacity, making novel PEFT solutions for 3D scenes both necessary and timely.\n\n(3) *Growing demand for scalable 3D fine-tuning.* 3D scene understanding has long been central to 3D vision but historically constrained by limited data due to the high cost of 3D data collection and annotation. Recently, however, the rise of spatial intelligence and embodied AI has spurred progress in generating 3D data at scale, including echo-view scenes, scene-level reconstruction, and 3D asset synthesis.\nNotably, feed-forward reconstruction methods like VGGT [7] now enable dense scene-level point clouds directly from video.\nAs 3D data becomes increasingly accessible, model capacity is scaling rapidly.\nYet, fine-tuning 3D models with 100M+ parameters, especi8ally in resource-constrained settings, *e.g.*, online adaptation on edge devices, remains computationally expensive. Our results show that GEM matches full fine-tuning, while updating <2% of parameters and converging faster. This makes GEM a compelling approach towards efficient and scalable deployment of existing and future large 3D models. \n\n**In summary**, 3D scene understanding contains more tokens, current 3D backbones exceed 100M parameters, reaching the size of ViT/SAM models, and more data is expected to further scale the 3D model; thus, scalable deployment is demanded.\nTherefore, we believe PEFT methods for 3D scene understanding at this stage are suitable and necessary; it also advocates for more interest into not only parameter-efficient but also efficient fine-tuning of large 3D model as a whole.\n\n---\n>**Q2: Further comparisons against tokenization methods**\n\n**A2:**\nWe would like to extend the discussion into the following.\n\n(1) As briefly discussed in L127-138, tokenization methods are mainly employed on object-level understanding (3D object classification, shape-part segmentation). The underlying assumptions of these methods do NOT transfer to scenes with millions of points.\n\nCompared with 2D images that can be split and resized into regular patches, tokenization heuristics are inferior in large-scale point clouds, such as farthest point sampling (FPS) and superpoints.\nEarlier scene-level approaches like PointNet++ [8] or SegCloud [9] explore such strategies and find that they sacrifice fine spatial detail. Modern 3D segmentation models now rely on U-Net-like or transformer backbones with direct point processing and augmented attention variants to retain these details, but they are distinct from PEFT in this context.\n\n(2) To address your concern specifically and test cross-domain applicability, we further apply GEM to 3D shape datasets, ShapeNetPart, and also extend leading 3D shape PEFT methods into 3D scenes, to make tokenization-based PEFT methods comparable.\n\n| Methods|Params.|Cls. / Inst. (mIoU in ShapeNet)| \n|-|-:|-|\n| ReCon (ft.)|27.06M|84.52 / 86.1|\n| + PointLoRA [5]|5.63M|83.98 / 85.4|\n| + PointGST [6] |5.59M|83.98 / 85.8|\n| + GEM|5.58M|84.02 / 85.8|\n|||\n\n| Methods  |Params.|Pct.|mIoU in ScanNet|\n|-|-:|-:|-|\n|Sonata (lin.)|0.02M|0.02%|72.5|\n|+ PointGST [6]|2.2M|2.0%|74.8|\n||||\n\nWe show that GEM still leads the performance in shape analysis, where Cls. denotes per-class mIoU and Inst. denotes per-shape mIoU. Nonetheless, extensions obtains a performance only on par with BitFit (74.7 mIoU) and lags behind GEM (78.3 mIoU). We also observe non-stable training and encounter NaN loss during the training of tokenization-based method, largely due to unstable results of FPS in various 3D scenes, indicating that extending tokenization-based method to 3D scenes can be non-trivial.\n\n---\n>**Q3: Novelty of the Context Adapter (CA)**\n\n**A3:**\nWe would clarify that the novelty of CA lies in its integration with SA (voxel-based convolutional bottleneck) to sparse 3D point clouds and its seamless integration into a PEFT framework.\nUnlike Perceiver and Perceiver IO, which applies cross-attention to progressively augment latents tokens queries, our CA introduces a lightweight low-rank structure that remains robust across varying ranks (Tab. 4b) and is specifically designed to fuse with pretrained backbones without retraining the backbone.\n\nMoreover, Perceiver and Perceiver IO do not explicitly mix contextual information into original input, but only repeatedly use the latent tokens to query from the input (key-value). In contrast, CA facilitates direct embedding of global context into the backbone by mixing between original inputs and latent tokens bi-directionally.\n\nAs shown in Fig. 3, coupled with SA that captures local structure, CA thus enhances backbone representation specifically with geometric cues, even under strict tuning budgets.\n\nThis results in a distinct motivation (broadcasting geometric cues), architectural design (light-weight side network as PEFT), and implementation tailored to the challenges of 3D PEFT (low-rank and bidirectional attentions via latent tokens that shares across layers), marking our approach a clear departure from earlier works.\n\n\n---\n>**Q4: Efficiency comparison and outdoor evaluation.**\n\n**A4:**\nThank you for raising this. We would like to clarify that due to strict space constraints, we prioritized core method and ablation details in the main paper. These results will be integrated and discussed more prominently in the final version for improved clarity.\n\nWe further measure SA/CA overhead.\n|Components|Memory|Latency|\n|-|-:|-:|\n|+SA|4.4G|50.5ms|\n|+CA|6.4G|51.7ms|\n||||\n\nResults show, when employed separately, the CA would incur more overhead than SA, and we would suggest employing CA when processing large scenes, as it provides global context beyond the local attention of the 3D backbone.\n\n---\n>**Q5: Qualitative results and visualizations.**\n\n**A5:**\nWe appreciate the suggestion. While qualitative comparisons were explored during development, we found initial visualizations to be affected by inherited biases from the pretrained backbone, potentially misleading without further context.\nTo maintain clarity and focus in the main submission, we instead included attention map visualizations for demonstration. We will include additional qualitative comparisons in the camera-ready version to better highlight the benefits of local and global cues.\n\n---\n>**Q6: Minor issues**\n\n**A6:**\nThank you for point them out, and we will revise them accordingly.\n- \"Pct.\" denotes percentage of total model parameters that are learnable; \"Learn.\" denotes the absolute learnable parameter count, in million. We will revise to include the description and explanation for the abbreviation.\n- L149 typo: \"Chanlleges\" -> \"Challenges\".\n- L274: the citations should be [10,11] as listed in the references below.\n\n---\n**References**\n```\n[1] Visual Prompt Tuning. ECCV'22.\n[2] AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition. NIPS'22.\n[3] Low-Rank Rescaled Vision Transformer Fine-Tuning: A Residual Design Approach. CVPR'24.\n[4] DA-VPT: Semantic-Guided Visual Prompt Tuning for Vision Transformers. CVPR'25.\n[5] PointLoRA: Low-Rank Adaptation with Token Selection for Point Cloud Learning. CVPR'25.\n[6] Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learning. TPAMI'25.\n[7] VGGT: Visual Geometry Grounded Transformer. CVPR'25.\n[8] PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. NIPS'17.\n[9] Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs. CVPR'18.\n[10] How transferable are features in deep neural networks? NIPS'14.\n[11] Characterizing and Avoiding Negative Transfer. CVPR'19.\n\n```\n\n---\nWe thank you again for your thoughtful review; we welcome any further questions or suggestions."}]

Author rebuttal:

Meta-review: All four reviewers ultimately recommended borderline accept or accept. This paper proposes the Geometric Encoding Mixer (GEM) - a geometry-aware PEFT method for large-scale 3D scene segmentation, showing strong empirical results while updating only 1.6% of parameters. Reviewers initially raised concerns mainly about limited comparisons and justification of design choices, but the authors provided thorough rebuttal. While some limitations remain, the paper addresses most concerns. Given the overall consensus and the paper’s contribution, the AC recommends acceptance.
Decision: Accept (poster)