Test-Time Optimization of 3D Point Cloud LLM via Manifold-Aware In-Context Guidance and Refinement
Authors: Tiankai Chen
Nanqing Liu
Li Yang
Xulei Yang
Tianrui Li
Xun Xu
Venue: ICLR 2026 Poster (2026)
Keywords: 3D point cloud
large language model
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in textual and 2D visual reasoning, yet their ability to understand and reason over 3D data remains limited. The issues become more challenging for understanding standalone 3D point cloud due to the high interclass confusion. In this work, we propose Point-Graph LLM (PGLLM), a framework that enables more effective 3D point cloud understanding by integrating in-context prompting and score refinement at test-time, respecting supporting data manifold. Our method first employs a pre-trained point cloud encoder which are used to construct a graph where edges encode visual similarity. Each support point cloud sample is converted to a textual caption via pre-trained PointLLM. For a test query, the graph is used to retrieve relevant neighbors whose captions serve as contextual demonstrations for a second stage LLM for final reasoning, a process we term in-context guidance. Furthermore, we introduce a confidence score refinement mechanism based on label propagation to enhance the reliability of LLM predictions for classification and out-of-distribution (OOD) detection tasks. All the above optimizations are carried out fully at test-time. Extensive experiments across diverse 3D datasets and tasks demonstrate that PGLLM consistently improves accuracy and robustness over prior baselines with very almost no additional computation cost, showcasing a promising direction toward native 3D reasoning with MLLMs.

Published as a conference paper at ICLR 2026
TEST-TIME OPTIMIZATION OF 3D POINT CLOUD LLM VIA MANIFOLD- -ORLLLD-DWAARR AWARE IN-CONTEXT GUIDANCE AND REFINEMENT
Tiankai Chen Nanqing Liu², Li Yang', Xulei Yang', Tianrui Li', Xun Xu+3 1 "Southwest Jiaotong University, Yunnan Normal University 3 Institute for Infocomm Research (I²R), A*STAR
fchentiankal, yangli.ef emy. swjtu edu.cn, lansingl 6301 63 com, {yang.xulei, xu-xun)ea-star. edu. sdu sg, trli@swjtu. edu.c..teeedc...
ABSTRACT
Multimodal Large Language Models (MLLMs) have demonstrated impressive ca- pabilities in textual and 2D visual reasoning, yet their ability to understand and reason over 3D data remains limited. The issues become more challenging for un- derstanding standalone 3D point cloud due to the high interclass confusion. In this work, we propose Point- GhantGraph LLM (PGLLM), a framework that enables more effective 3D point cloud understanding by integrating in-context prompting and score refinement at test-time, respecting supporting data manifold. Our method first employs a pre-trained point cloud encoder which are used to construct graph where edges encode visual similarity. Each support point cloud sample is con- verted to textual caption via pre-trained PointLLM. For test query, the graph is used to retrieve relevant neighbors whose captions serve as contextual demon- strations for second stage LLM for final reasoning. a process guidance. Furthermore, we introduce a confidence score refinement mechanism based on label propagation enhance the reliability of LLM predictions for clas- sification and out-of-distribbtion (OOD) detection tasks. All the above optimiza- tions are carried out fully test-time. Extensive experiments across diverse 3D datasets and tasks demonstrate that PGLLM consistently improves accuracy and robustness over prior baselines with very almost no additional computation cost, showcasing promising direction toward native 3D) reasoning with MLLMs. The code is available on https:: /tttt::://gttutt com www.hand.cch.eeeooooeee http/handsoo ome 999KK/ PGLLM
term in-context
INTRODUCTION
While multimodal large language models (MLLMs) have revolutionized textual and 2D visual rea- soning, their ability to interpret and reason about 3D environments remains fundamentally limited. Recent efforts (Tang et al., 2024 Qi et al., 2024a) have explored understanding with MILMs by equipping them with mechanisms perceive 3D information, often through auxiliary modal- ities or intermediate representations. Typically, these approaches operate by either projecting 3D point clouds into 2D images (Zhu al. (023) by piping pre-extracted features into the language model (Xu et al.) 2024 Guo et al. 2023). While these pipelines demonstrate promising results, they fall short in granting MLLMS direct access the underlying geometric structure of 3D data. Consequently, enabling MLLMs to natively process and reason over rich 3D point cloud information remains an open and challenging research question.
A notable step in this direction is PointLLM (Xu et aL. 2024), which introduces framework capable of understanding colored 3D object point clouds in response to human instructions. PointLLM fuses geometric, appearance, and linguistic information by coupling a point cloud encoder with a pre-trained LLM such as LLaMA (Touvron et al. 2023). To handle downstream tasks such as classification or captioning, PointLLM adopts two-stage pipeline, where the PointL.MM first
Correspondence <xu_xun@22 2r_ a a-star aaaarredee edu. sg>. This work partially done during Tiankai Chen and Nanqing Liu's visit I'R funded by China Scholarship Council.

---

Published as a conference paper at ICLR 2026
query sample
in-context samples
other samples
Caption2:
Features Graph
This o 3D object model of a cortoon-style block bathtub. The model exhibits
smooth, block ceromic surface with sleek design typically found in modern bethrooms..
(GT: bathtub)
Caption1:
A three-dimensional object model featuring on animated, stylish black boat. This designed with sharply pointed prow, ideal for cutting through high speeds. The beat...
(6 bathtub)
Caption3:
The 3D model of animated bathtub, colored black. This model is not realistic, instead it is designed stylized, cartoon like format that is usually found animated films or video games...
(GT: bathtub)
Assign the caption to the most likely category.
Assign caption the most likely category. need to consider the captions with the similor features of the 3D point cloud.
Categoriesxx..>
Categoriesk_.>
Captiont:x A three-dimensiona object model featuring animated, stylish black .,>
Captiont: three-dimensionll object model featuning an animated, stylish black boat
Captions with similar Features:Captionn, Caption3>
<thinko Based the description of a 3D object featuring a stylish black boat, the most similar category from the provided list is car. </thinlo Output: <car>
<hhinl While the caption describes this 3D object as stylish black boat," similor captions call "nectangular bathtub." Since "boat" not among given categories, thus it's classified as bathtub. </thinko Output: <bathtub>
Approach of PointLLM
Approach of Point-Graph LLM
Figure Manifold-aware in-context guidance leverages the 3D captions of adjacency nodes as demonstration for downstream understanding tasks.
generates textual description, and second stage LLM interprets this output for final task- predictions. While this method demonstrates competitive performance, it faces key limitation, high inter-class visual similarity in 3D point clouds often leads to confusion, particularly when each point cloud is interpreted in isolation. As a result, models may struggle to distinguish fine-grained differences between closely related categories, leading to reduced reliability.
To overcome this limitation, we draw inspiration from the recent success of In In-Context Learning (ICL) (Agarwal al. 20244 [i al.) [2033a] Brrwn al.] (220), which enables LLMs to general- ize to novel tasks by conditioning on small set of demonstrations provided in the prompt. ICI. has shown strong performance across various domains, including multi-modal settings (Tsimpoukellii et al. 2021 Diang et al. 2024b Huang et al. 22024), where LLMs are guided by examples combining visual examples and text descriptions. The effectiveness of ICL crucially depends on the availabil- ity of informative and task-relevant demonstrations, e.g. image-text pairs. Selecting relevant and conducive demonstrations thus remain an open challenge.
To address this challenge, propose in-context learning strategy that leverages the manifold structure of unlabeled supporting data, which may be maintained independently or drawn from the test set. We represent this manifold sample. To enrich each node, we employ PointLLMM to generate caption using the standard prompt "What is this?". The affinity between nodes computed based on feature similarity in the embed- ding space of a pre-trained 3D encoder (Yu et al. 2021a). At inference time, for given query point cloud, identify its neighboring nodes in the graph and append their corresponding 3D captions to the query prompt. This augmented prompt is then passed to the second-stage LLM (e.g.. Chat- GPT), enabling to perform context-aware reasoning. We refer to this mechanism as in-context guidance, which injects semantically relevant knowledge into the LLM's reasoning process at test time, without requiring model retraining.
graph, where each node corresponds to 3D point cloud
Moreover, for classification-relatee downstream tasks such recognition and out-of-distributin (OOD) detection, the confidence calibration of LLM outputs is critical for robust decision- making (Xiao et al. 2022). Relying solely on raw predictions from LLMs may be risky, especially when predictions are overconfident or miscalibrated. To address this, we further tap on the mani- fold build upon the supporting data and employ refinement step, where predicted confidences used to smooth and correct noisy labels on the graph. This is implemented via a lightweight label propagation algorithm (Zhu & Ghahramani 2002). allowing the model refine its predictions by
2

---

Published as a conference paper at ICLR 2026
considering the consistency and confidence of nearby nodes. In this way, anecdotal ambigu- ous predictions on individual samples can be improved by leveraging the collective structure of the data.
In summary, we propose a novel 3D point cloud understanding framework Point-Graph LLM that integrates in-context learning, graph-based reasoning, and confidence-aware label propagation. By bridging the gap between geometric perception and LLM reasoning, our approach enables more accurate, interpretable, and robust 3D understanding. The main contributions are summarized as follows.
We propose an in-context guidance mechanism to optimize the effectiveness of 3D point cloud LLM for downstream tasks. This approach leverages the test-time data manifold to construct helpful demonstrations enrich the prompt.
We introduce score-based inference mechanism that further improves the performance of LLMs on classification related downstream tasks by refining the initial predictions on individual samples.
We conduct extensive experiments on multiple downstream tasks and diverse 3D point cloud datasets, demonstrating consistent performance gains over existing baselines across all settings.
2 RELATED WORK
Large Language Models for 3D Understanding. Large Language Models (LLMs) have demon- strated remarkable performance across a wide range of natural language and 2D vision tasks (Brown
al., 2020; Achiam al.) 2023. Touvron al.) (002). Recently, there has been growing interest
in extending LLMs to 3D understanding (Hong et al. 2023 Qi et al. 2024a b Yang et al. 2025b
Xu al. 2024, Guo al., 2023 Yuan et al.) 2025). 3D-LLM (Hong al. 2023) introduces
family of LLMS grounded in physical 3D reasoning. laying the foundation for language-guided 31) perception. Subsequent approaches such as PointI.LA (Xu et al., (024) and Point-Bind (Guo al.] 2023) directly process colored object point clouds by combining point cloud encoders with pre- trained LLMs, enabling open-vocabulary 3D understanding. ShapeLLM (Qi et al. 2024a) pioneers the use of multi-view distillation and introduces the ReCon++ encoder, establishing 3D multi- modal evaluation benchmark (3D MM- Vet) to unify embodied 3D interaction tasks. GPT4Point (Qi let al. 2024b) extends the capabilities of LLMs handle point cloud captioning and visual question answering. Similarly, LiDAR-LLM (Yang et al. 2025b) focuses on outdoor scene understanding by integrating LiDAR data with LLMS for large-scale 3D reasoning. While these methods demonstrate the potential of LLMs for 3D tasks, they typically treat each point cloud in isolation and do not fully exploit the structure of the data manifold.
In-Context Learning. In-context learning (ICL) (Brown et al. 2020) enables large language mod- els to perform downstream tasks by conditioning on a set of demonstrations, without any parameter updates. Initially developed for natural language tasks (Zsks(Zhag et al. 2022 Li et al. 2023bb. ICL has since been extended to multi-modal domains. For instance, Flamingo (Alayrac et al. 2022) adapts ICL to vision-language strategies Qiang et al., 2024b; Huang et al.] (024) have been shown to significantly enhance the effectiveness of ICL. in image classification and question answering. These works demonstrate that with appropriate context, L.MSs can generalize across diverse tasks in a flexible and lafel-efficient manner. However, applying ICL to 3D understanding remains underexplored, particularly in scenar- ios where constructing relevant demonstrations is non-trivial due to limited supervision.
tasks by incorporating cross-modal attention, while many- shot prompting
Manifold Learning for Visual Tasks. Manifold learning has been widely adopted to enhance visual recognition tasks by modeling the intrinsic geometry of data distributions. Gtaph-based methods (Li ctt al. 2025 Stojnić et al. 2024 Chen et al. 2025) leverage similarities in feature space to propagate high-confidence labels scores, improving performance in zero-shot and few-shot settings. For
instance, label propagation techniques have been combined with graph neural networks (Bao et al,
2024 Stadler et al.) 2021 Wu et al.] 2023) to improve the separation between in in-distribution (ID) and out-of-distribution (OOD) samples. Beyond classification, graph-base based selection has also been applied in 3D domains, demonstrated in GraphI2Ps (Bie al. 2025), which filters false matches
3

---

Published as a conference paper at ICLR 2026
3D Point Clouds
3D Embeddings
Build Gnaph
Downstream Tasks
scores
Encoder
3D Recegrittio
categories
Construct Prompt
Get Initial Answers
A
8
3D Encoder
Prompt
rumbers
OC0
Answers
ID
Scores
Answers Eteration
PointLLM Model
What this
onject model calaned eccestrically designed contrest black and whine prspellers the front, The sirp lopoon'' i rome singe, nntracted tewards the funsings when 47 rest. contribute to its un que oer eeredptanic settu.
PHRL
Iteration
Propogat
Final Answers
Object Captioning
What this?
This is 20 madel on trchele
Gnaph
Figure 2: Overview of the proposed framework for PGLLM. After encoding the 3D test samples, the framework feeds them into PointLLM for caption generation and uses them to construct KNN graph. Initial answers are then synthesized via LLM inference. Subsequently, leveraging relational structures within the KNN graph, we introduce an answer iteration mechanism to optimize perfor- mance on downstream tasks.
during point cloud registration via neighborhood pruning. While these methods focus on traditional backbones vision-language models, work is the first to integrate manifold learning into the in-context learning process of LLMs, enabling context-aware reasoning over 3D point clouds by structurally selecting and organizing prompts from the data manifold.
METHODOLOGY
3.1 PRELIMINARIES: POINTLLM AND 3D CAPTIONING
We begin by formalizing the setup. Let D., { dataset with categories, D, may be the testing dataset or any unlabeled reference dataset. Let fp be a pre-trained point cloud encoder and we apply fp to the dataset yields sequence of point cloud features:
No { Ji denote unlabeled 3D point cloud support
P (Pp.PP....,PN.... .
p. fp(x))-
For each sample T;, we use the default prompt "What is this?" with PointLLM (Xu et al., 2024) to generate textual caption e, resulting in caption set, C = {c1, C2,... CN.) }. These captions serve as high-level semantic descriptions of the point clouds and form the basis for further in-context learning and downstream tasks.
3.2 IN-CONTEXT GUIDANCE VIA GRAPH NEIGHBOR RETRIEVAL
In-context learning (ICL) allows LLMs perform downstream tasks by conditioning on set of relevant demonstrations embedded within the input prompt. To construct effective in-context guid- ance for each test sample, we build a graph G (V, E), where each node v; represents point cloud u, and edges eij E encode pairwise similarity between samples. Specifically, we compute the cosine similarity between point cloud features p; and Pj, and define symmetric the edge weight matrix W e R N.N.N.. following K-Nearest Neighbors (KNN) criterion,
Eij
if ei E Top (feig) No, )==))
< Pi,
>
W5 = 10 otherwise
s.t. eig
(I)
Ppll
|I
For each query sample T.. retrieve its K nearest neighbors on the graph, yielding neighbor set X, = { Zix and the corresponding caption set C, = [c....... Cix }. These captions serve in-context demonstrations appended to the prompt, illustrated in Fig. I as "Caption2", "Caption3", etc. This enables the LLM to reason about the query with reference to structurally similar samples. New query samples not attached to the supporting graph can be integrated into the
4

---

Published as a conference paper at ICLR 2026
graph without much computing overhead following the dynamic graph expansion scheme (Li al. 2025
3.3 SCORE REFINEMENT VIA LABEL PROPAGATION
Unlike conventional classification methods that directly predict class labels. we guide the LLM to output class confidence scores for each 3D caption. This score-based formulation enhances robust- ness and enables downstream tasks such OOD detection.
3D Recognition: Given a 3D caption c, we prompt the LLM to output a per-category score, St RA, where each element reflects the confidence of respective class label. Aggregating across the dataset yields the initial score matrix, Su R To refine predictions by leveraging geometric similarity among point clouds, we apply label propagation (Zhu Ghahramani (002) over the graph W. Let denote the refined score matrix iteration 1. The update rule is:
S. a.S:WW So)
W D A D diag( W.))
(2)
Ú arg max (St).s
where ý gives the final predicted class for each sample, and controls the balance between the initial LLM output and propagated scores.
For OOD detection, prompt the LLM to produce single confidence score ((x;) E IR for each caption, indicating its similarity to the known in-distribution classes. threshold 5 is used to determine OOD status:
3D OOD Detection:
OOD
if S(x.) S
ID otherwise.
This scalar score can be smoothed through the same graph- based propagation mechanism as in Eq. 2 with
3D Caption Refinement with In-Context Learning: For the 3D object captioning task, we enhance the semantic quality of the initial MoanlLL--ggeerattte captions using in-context refinement. We reuse the same graph-based strategy to select neighboring samples with semantically and structurally relevant captions. These are appended as demonstrations to the input prompt for each query caption. Unlike recognition tasks, the goal here is caption correction rather than classification. We guide LLM to preserve the original semantics but improve fluency and fix any factual errors. This leverages the LLM's generative ability to produce more accurate and natural deseriptions.
3.4 FRAMEWORK OVERVIEW
An overview of PGLLM is presented Fig. First, we extract 3D features from the test set using a frozen point cloud encoder. These features are passed to PointLLM to generate initial captions. KNN graph then constructed over the feature space to capture local geometric relationships. Based on this graph, we identify structurally similar neighbors for each sample, whose captions are used as in-context demonstrations. Depending on the downstream task we summarize the test-time optimization practices as follows. An illustration is deferred to the Appendix.
For 3D recognition, the LLM outputs class-wise scores, which are further refined via graph- based label propagation.
For OOD detection, scalar similarity scores computed, refined via graph-based label propagation and eventually thresholded to determine OOD status.
For captioning, in-context refinement is applied to enhance the quality of generated text while maintaining semantic fidelity.
This unified framework leverages LLMs not only as language generators but also as structured rea- soning engines capable of adapting multiple 3D tasks with minimal supervision.
5

---

Published as a conference paper at ICLR 2026
Methed
uac Sigc
Stap
MadefNeral
S22
Shap-NeCCree
AIBROTT
1oree5) [M] RDCCC 11899 (MTRRCT
SERRSs/RRRRO (MUOOTT 19909
AUTRCC
FIRAA)
AJ AUTDOY TRR9
[[AERO PFRAS (4] (MURECITT
MCMMince 1003
53.6
11.5
72.6
85.1
46.1
78.3
51.3
Nept
34.5
724
66.4
15.0
63.6
41.9
47.3
8330
62.6
873
-1.6
75.6
13.11
48.3
70.4
71.6
15.2
774
77.4
16.1
76.1
13.11
64.0
18.7
43.7
91.4
66.5
Nu 1014
PGLE
Poenl
GPT4
566
18.2
43.4
76.7
41.3
560
88.
PG17 A/7 (Oan)
Rowel7 70
GPT4
89.6
53.1
87.3
43.0
18.
80.3
85.9
R31
818
57.1
83.8
16.7
62.6
9.7
91.1
29.
PGLLX (Dam))
PoxiL
De
88.4
83.5
26-1
15.2
05.2
66-3
8288
91.7
36.4
915
3011
354
Mestar ar
GPE-
66.7
100m
18.
1985
789
1600
100.
63.1
116
87.
103.0
01.0
105.0
Table 1: Evaluation of 3D OOD detection on ModelNet40 and ShapeNetCore. Bold and underlined numbers denote the best and second- best results, respectively. Each "MNx" or "SNx" denotes the known class split and the rest unknown.
Method
ore Stage
Zed Stuge
MNNI
MidelNec4)
ShapeNetCoree
AUlOOC| (MS0S1JAUBOCY TPROS)
PPRSSJJUROC" JAUROC" F/105
JALBOC| E19895)
AU3OOC
(1R0S,JALRRRK
FPROS_JALBOE) F15c95]
(AUROCT F9R05]]
IGLLS
M Mi
Dwans-VL-oth
85.8
67.4
S4-4
47.5
14.3
71.5
K1.5
62.0
W1-2
48.0
91.9
10.3
14.1
79.7
.9.2
36.0
PGLLS Paar11MM77 78 L KB
75.6
86.5
72.2
81.4
60L9
91.9
60.6
56.9
56.3
95.7
$1.9
68.3
02.7
44.6
80.0
07.5
IGLLS MM771
83.0
90.4
75.3
12.5
96.0
7
83.2
81.9
57.1
92.4
30.3
21.2
90.3
KILLM"
MiaKCPT3D
V2. 5B
92.4
44,5
52.2
91.4
29.2
35.5
88.1
IGLLM
Miskart-ID
1-68
61.8
85.9
71.0
84.0
11.9
75.6
67.7
61.3
100.0
71.2
13.9
56.9
71.
76.6
IGLLM!
Minicart-ID
92.0
46.9
83.9
41.2
826
57.9
87.5
48.2
85.9
42.6
95.2
25.0
90.0
22.1
93.0
28.9
Table 2: Evaluation of 3D OOD detection on ModelNet40 and ShapeNetCCree for open-sourced LLM.
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Dataset: We evaluate our method on four well-established 3D point cloud benchmarks. Model- Net40 (Wu et al. 2015) contains 2,468 test 3D objects across 40 categories. ShapeNetCore (Chang ct al., 2015) a canonical subset of the full ShapeNet repository with 5,158 unique test models from 55 object categorics. Following PointLLM (Xu et al., 2024). we sample 200 objects from Objaverse Deetke et al. (2023) for testing. S3DIS (Armeni et al. 2016) provides semantically seg- mented 3D point clouds from indoor environments and we follow Chen et al. (2025) to select 8,931 point clouds with rich semantic annotations for evaluation. For OOD dataset partitioning,
follow the 3DOS protocol Alliegro et al. (2022) to divide ShapeNerCore into SNI, SN2, and SN3 subsets. Similarly, we partition ModelNet40 into MNI, MN2, and MN3 subsets. Further dataset details are provided in the supplementary material.
Implementation Details: We evaluate PGLLM on two baseline works: Poin (Xu et al., 2024) and MiniGPT-3D (Tang et al. 2024) to generate initial captions, and use Point-BERT (Yu et al. 202Tb) as the 3D encoder. We evaluate DeepSeek-V V3 (Liu et al., 2024), Qwen-Plus Yangg et al. 2024) and GPT-4 (Achiam et al. (203) as the second stage LLMs. To further validate the fea- sibility and portability of PGLLM, we additionally conduct experiments with open-source LLMs, including Qwen3-VL-SD (Yang et al.) (025a)), Llama3.1-8D (Grattafiori et al., 2024), and GPT-oss- 20B (Agarwal et al., 2025). These models are integrated into the second-stage reasoning module under the same prompting and inference protocol, enabling controlled comparison against propri- etary counterparts. As for KNN Graph construction, we set the K-value of 3. For score propagation, set the a to 0.5, and the number of iterations T to 5 in Eq. 2
Competing Methods: We compare against existing LLM-based 3D understanding methods: In- structBLIP (Dai al. 2023). LLaVA (Liu et al. 2023). , 3D-LLM (Hong et al.) 2023). Point-Bind
LLM (Guo et al. 2023). ShapeLLM (Qi et al. (024a), PointLLM (Xu et al. 2024). and MiniGPT- 3D (Tang et al. 2024). Both PointLLM and MiniGPT-3D employ ChatGPT 4 as second stage LLM for classification, thus facilitating fair comparison. To the best of our knowledge, this is the first work to explore 3D OOD detection within an LLM framework. Therefore, we compare against several VLM-based zero-shot OOD baselines: MCM (Ming et al.) 2022), NegLabel (Jiagg et al.) (024a)). ZLap (Kalantidis et al., 2024), and GSP (Chen et al. 2025). Finally, we evaluate two variants of PGLLM with different support set D,,- When testing data distribution is available, i.e. transduc- tive inference, we use all testing data as the support dataset and refer to the method as PGLLM". Alternative, we leverage an external dataset, Objaverse, to build the support dataset and refer to the method as PGLLM Specifically, we randomly selected 100K samples and their correspond- ing captions from the 660K training data of Objaverse to build the graph. For both PGLLM and PGLLM", PoimLLLM-7B is used to generate initial captions. All competing methods use the same 3DD encoder PointL.LM (Xue al. 2024).

---

Published as a conference paper at ICLR 2026
3D OOD detection
3D Object Captioning
Greand Truth quiter
What 1960
Ground Trunts blue
What is
The 1D stject nodel is try dedgeed to rep aoptioots o spoce stip. TV sas ponted frast Hoyy despeed reprnsert the strooms land, constracted of platte outtee lighweggt nateris, cod' sance eduoctood taal for mildees to aom about spane axgianation simsly playfu
wedkl paiited in wheert shade of this. The pany' i bosy, tell and mane har wehlil usform lise ooer The design and appearance This motel could be shilad verius owiectice pro jacte, video sarnel, exer dgitel oppestence.
which antirdl
internative applicalires
generated descogricn object colcutore the spactied
prebabil 1030 that
Categeries: desk gutter, deer...
Gieen free desoription sbject, plans accending feon your with this chject which nse the some cotegory to optimize yiven
The desor on requrring presebiitty coloaler weleeieee ler The sbject model is disigeed to replioute space
other
pointed frest. iiey
The pnion of ULM eujoct and odd descripione
20
Approach of PointLLM
reasourste If deser prione are eriior, plaore ds not mose ony cherass destmppice cortictt do pluase mare slight odjustments this
free fore LUW goaenoted desoription object (wisch be (epccuatee), coloustee the prstebility (0 100) thet it bebogg specified cotegaayy Consider aimiar descrptione other sojecta in cotlegary
cortion
desionpt pl ist mequinng up fimize This is paieted stbrant i repical pony'' a body. fall and mone hair exhibat un uniform Slac color...
Thhe
The deccripiinn requiring probobiitty my colnuiation: The 38 redel sageets a cherming carteen pony with u aordd body exniraly except
Cotegeries: desk guttor, deer...
parple
The desenphicn requirisa probobility irt caladintos i The 1D seject mode try disd gred reaiiotte a space ship o freet, liey
cslan
The mook of full audpt, wivid ydllos horse dlancing foody. promisenn
adding
lettc
medel cianes coourtic guiter with mraditinalty carved body, a lang.
Descriptions whjeets with devior
This 30 mode is chamm ig curt con pany with solid body that colar....
eucept fell,
parpe
a treditionl, electric gutter fasturing deep jet-
3. Trir a 30 model of an diectric gaiter prediesiantly a dark black
The is 10 resiel a cartess pont with o seldd, witeett blye bedy, mane, and hal The pary festhina rounded, frardly design. chemsing appoorance mikes in surcbbe for on eeieetice gemes, or intersetive app icat ona, sapeo ally ferhaay or magical settings
Approach of Point-Groph LLM
Figure 3: Qualitative examples of prompt questions and PGLLM's predictions.
Evaluation Metrics We use classification accuracy (ACC) for 3D recognition, and AUROC and FPR95 for 3D OOD detection, which are standard metrics in OOD evaluation. For 3D object cap- tioning, assess semantic alignment using Sentence-BERT (Reimers & Gurevych 2019). Sim- CSE Gao et al. 2021). and GPT-4 as evaluators.
4.2 EXPERIMENTAL RESULTS
3D OOD Detection: Tab. summarizes the results of our comprehensive experiments, highlighting the following key observations. (i) Our PGLLM framework, when integrated with GPT-4, estab- lishes new tfate-of-the-ar results on the ModelNet40 benchmark. It achieves an outstanding 85.9% AUROC on average, outperforming the previous best method (MCM) by 4.9%, while also reduc- ing the critical FPR95 metric to 52. %. We notice that both PointLLM-7B and ours methods use ChatGPT 4 the second stage LLM. (ii) On ShapeeetCore, PGLLM with GPT-4 demonstrates breakthrough performance, attaining 97.6% AUROC on SN3 and remarkably low FPR95 of 9.7%. This reflects an 7.1% AUROC improvement over feature-based methods such as GSP. Overall, our framework achieves an average AUROC of 91.1% and an average FPR95 of 29.6%, setting new benchmarks across key metrics. These consistent gains across two datasets affirm the effectiveness of our graph-based mechanism. (iii) Because GPT-4 tends to assign either 0 or 100 when scoring test samples, the baseline performs very poorly on the FPR95 metric, with values almost always equal to 100.0. By introducing score propagation, our method effectively alleviates this issue and yields much smoother, more evenly distributed scores across all samples. (iv) Using GPT-4 as the second stage LLM yields notable improvements in AUROC with +3.8% on ModelNet40 and +0.2% on ShapeNetCore compared to DeepSeek- The performance gap between GPT-4, Qwen-Plus and DeepSeek-V3 variants highlights the ability of our framework to harness stronger L.MMs for enhanced 3D understanding, (v) With both testing data (transductive setting) and external dataset as supporting dataset, PGLLMT/0 outperforms the baseline (PoimLLLM-7B). This suggest the robust- ness of the graph- based method.
Evaluated on Open-sourced LLM: To further validate the practicality of our PGLLM beyond closed-source APIs, additionally evaluate our framework with representative open-sourced
7

---

Published as a conference paper at ICLR 2026
2nd Slage L.M
3D Recognition
3D Captioning S-BERT
Method
(1) ACC
(C) ACC Average
GPT-4
SimCSF
(Hong et al. 2023
33.4
44.5
43.7
Point-Blind a 20243
51.9
39.7
ShapeLLM-7hr Q aie 2024a
46.9
48.2
49.2
ShapeLLM-113B(
202411
48.5
50.0
InstructBLIP-755
2023
GPT-4
19.5
31.5
25.5
45.3
47.4
48.5
SetmcctBLP-133B0DD
2026
GPT-4
26.0
31.4
28.7
45.0
45.9
48.9
LLaVA-7B at. 2025
GPT-4
39.7
39.7
39.7
46.7
45.6
47.1
LLaVA-13B 2023
GPT-4
37.1
36.1
36.6
38.3
46.4
45.9
MiniGPT-3D Tane uum 2024
GPT-
61.8
60.0
60.9
57.1
49.5
51.4
PointL (X
2023)
GPT-4
53.4
51.8
52.6
44.9
47.5
48.6
PointLLM- (3 2022
GPT-4
53.0
$2.6
52.8
48.2
47.9
49.1
PGLLM (Ours)
GPT-
53.1
53.0
53.1
49.1
48.4
48.9
PGLLM (Ours)
GPT-4
63.1
61.8
62.5
50.5
48.9
49.4
PGLLM (Ours)
DeepSeek-V3
62.6
62.0
62.3
PGLLM (Ours)
Qwen-Plus
43.1
41.7
42,4
Table 3: Comparison results on 3D recognition (ModelNet40) and 3D captioning (Ohjaverse). Recogni- tion performance is evaluated using two prompt types: an Instruction-type (I) prompt ("What is this?") and a Completion-type (C) prompt ("This is object of").
LLMs as the second-stage inference module: Qwen3-VL-8B, Llama3, 1-8B, and GPT-oss-20B As shown in Tab. PGLLM remains consistently effective across different open-source choices, demonstrating that our improvements primarily come from the proposed graph-based in-context guidance and score refinement, rather than relying on any specific proprietary model. Importantly, the best open-source configurations can achieve performance that is close to, closed-source inference results in certain splits, using MiniGPT-3D features with DWwn3--VL-BB or GPT-oss-20D yields strong average AUROC the competitiveness of our method under open-souree inference. We also observe that Llama3. 1-BB generally weaker, which attribute to its chat-oriented nature: it is optimized for conversational probabil-
surpasses the
both ModelNet40 and ShapeNetCore, highlighting
mnstruction-following and tends to be less reliable for the numeric, reasoning-intense ity scoring required by OOD setup. Overall, these results confirm that PGLLM is robust and transferable, enabling strong 3D OOD detection with open-sourced LLMs at level comparable to closed-source inference.
3D Recognition: The results in Tab. reveal several key insights. PGLLM framework achieves an average accuracy of 62.3% with DeepSeek- V3 and 62.5% with GPT-4, outperforming all ex- isting state-of-the-art methods. In partic- ular, it surpasses the strongest baseline, MiniGPT-3D, by + +1.6% for average. ii) The framework shows strong robustness across different prompt types. Specif- ically, it outperforms MiniGPT-3ID by 1.3% with the instruction prompt "What this?" and by 1.8% with the completion prompt "This is an object of". This dual- prompt advantage highlights the adapt- ability of our architecture to diverse query formats. iii) Although all competing meth- ods rely on GPT-4 for evaluation, our framework achieves 62.3% average accu- racy even with DeepScek-VV V3 (much lower per token cost than GPT-4) as second stage LLM, sur- passing all GPT-4-based baselines. This suggests lower cost LLMs may achieve comparable perfor- mance. However, Qwen-Plus demonstrates notably lower performance the 3D Recognition task, primarily due to its limited ability to generate long numerical sequences, which adversely affects its overall results.
Our
3D OOD Detection
3D Recognition
AUROC
FPROT
ACC
98
27
55
96
25
54
94
23
PM 7BGGTT
PGLLMDDegSSekkW
PGLLMIGPT-4
PGLLMIQWwa-Plus)
Figure 4: Results on real-world benchmark S3DIS. We
port 3D OOD detection and recognition tasks.
3D Object Captioning: The results in Tab. show that our method performs competitively on the 3D object captioning task, consistently outperforming the PointI.L.M-7B baseline across all three
8

---

Published as a conference paper at ICLR 2026
In-context
Score
ModelNet40 AUROC1
ShapeNetCore AUROC1
Guidance
Propagution
ACC
FPR95
ACC1
FPR95]
52.5
80.4
100.0
55.5
88.2
54.9
59.7
83.3
100.0
60.7
89.2
47.2
60.2
83.1
100.0
89.5
46.0
56.7
83.5
62.0
59.3
89.8
44.7
63.1
85.9
52.1
62.4
91.1
29.6
Table 4: Ablation study on two datasets, ACC refers to the results of 3D recognition experiments, while AU- ROC and FPR95 correspond the OOD detection experiments. Both AUROC and FPR95 represent averages across all subsets ModelNet40 and ShapeNetCore datasets. The denotes in-context guidance derived through direct nearest-sample retrieval without graph.
evaluation metrics. Since the evaluation relies on subjective judgments from an LLM, PGLLM uses GPT-4 exclusively for assessment to ensure consistent evaluation standard. While approach does not yet reach state-of-the-arl performance, this gap is likely attributable to the limited size the test dataset. In low-data regimes, constructing optimal graph structures is challenging, which can constrain performance. Nevertheless, the observed improvements highlight the promise of our graph-based in-context guidance strategy for 3D captioning tasks. This suggests careful selection of second stage LLM is still necessary.
Experiments on Real-world Benchmark: Most existing 3D LLMs Xu et al. 2024): Guo et al. 2203);; ; Tang et al. (2024) have been evaluated primarily on synthetic CAD datasets, leaving a vali- dation gap real-worf scenarios. To address this, we conduct experiments on the real-world S3DIS dataset. As shown in Fig. 4 we evaluate both PointLLM and PGLLM on 3D out-of-distribun (OOD) detection and 3D recognition tasks. The results demonstrate that our method consistently outperforms baseline approaches, confirming the effectiveness of our graph-based strategy in prac- tical settings. Notably, PGLLM remains compatible with lower-cost second-stage LLMs, achieving performance that is comparable, even superior higher-cost alternatives.
4.3 ABLATION STUDY
We conduct ablation study with results summarized in Tab. All experiments ChatGPT4 the
AveragAAUR0
second-stage LLM and are conducted the ModelNet40 and ShapeNet- Core datasets for both 3D recogni- tion and 3D OOD detection. We be- gin with a baseline that feeds the gen- erated 3D caption of each query di- rectly into the LLM without addi- tional context. We also evaluate variant where the query retrieves its nearest neighbors via KNN (K 3) to form the in-context prompt, by- passing graph construction. This variant yields results comparable to the graph-based method, likely because both retrieve highly similar neighbors due to the small K. However, omitting the graph structure prevents the use of score propagation for further refinement. Introducing in-context guid- ance alone yields significant performance gains over the baseline, demonstrating its effectiveness in injecting relevant contextual cues into the I.MM. In contrast, score propagation alone provides only modest improvements. Notably, combining in-context guidance with score propagation leads to substantial performance boosts across both tasks, underscoring their complementary and synergistic effects.
88
19
K
3
5
3
3
6
8
(4) 3DD detectionn Nod
(h) 3D detection on ShapeNitCore
Figure 5: Different number of K-values for 3D OOD detec- tion on two datasets.
4.4 FURTHER ANALYSIS
Qualitative Analysis: Fig. showcases qualitative examples of PGLLM 3D OOD detection and 3D object captioning tasks. For 3D OOD detection, we observe that PointLLM's reliance on single-caption inputs leads to fragile predictions when faced with ambiguous hand-to-interpret

---

Published as a conference paper at ICLR 2026
KNN Graph
Ground Truth: bed semple)
Ground Truth bed [ID sample)
Coption: object represents
Caption: The 3D abject model depicts a lorge, bouncy bed communly known as trampoline This trampoline features rectorguler shape and stonds four sturdy legs, each with handle for easy mebility. The design allows for lot of fun and physical acthvity making suñable for both kids and edults Its usage extands to recreations purpases, firmess routines, and competitive sports, offening safe space to bounce, flp, and twist thereyy developing agility. hand eye coondiration. and solence. 6PT-4 judgment 8B (Tendency towards ID)
bed styled solid block
Despite whimscol design indicshing
reslistic regresentation
possesses
key elemonts that
mottress headbbord
simplified,
clemant onimation geme, Its uniferm black implles basic, minimalss osstretic. 6PT-4 judgment 100 (Tendency Towards
design
Ground Truth: (TD somple)
Ground Truth: bed (ID somple)
Caption: The 30 object model
structure
Caption: This is 3D model of on object that resembles rectonguler slack boord, distingusheed by a distinctive fecture thp everly spoced white dots Its appecrance susgests that night be port of some goming equipment, a decorative iten, or possibly functiondl element in board game, depunding on the nature of the dots Its simple design lovre roon for interpretation and could inrkee serse of nystery or fun.
consisting of rectongulor base with dlightly sloping sides, pointed in smooth grey color. nottioned on top of base are two thin mortresses, one on each side, both cavered with white bed sheets, The
simpiciity and functionclit of the structure
auggest it could used in dorm rooms or simple hame lupp sleeping
lry purposes. judgment 100 (Tendency fowards Ib)
GPT-4 judyment 20 (Tendency Towards OCD)
Initial scores
Score Propagation
Final scores
100
100
85
20
85
20
85
67
79
100
100
87
query sample
in-context samples
Figure 6: Qualitative example demonstrating the effectiveness of score propagation.
point clouds. Our in-context guidance mitigates this by providing captions from structurally similar samples, allowing the L.M to leverage contextual cues for more robust classification.. For 3D object captioning, PointLLM generates captions exhibiting inaccuracies or redundancies. By incorporating in-context guidance, our method enables the LLM to refine these outputs based on contextual sim- ilarity, resulting in more accurate and semantically rich descriptions. Fig. qualitatively illustrates the process of score propagation. It can be observed that when the 3D LLM generates a low-quality caption for given sample, it directly leads to a poor score in the second-stage evaluation. How- ever, by employing the score propagation method, this can be effectively corrected by leveraging the accurate scores from neighboring samples of the query sample.
Impact of In-Context Example Quantity: To examine the effect of in-context guidance, we vary K, the number of retrieved captions provided to the LLM, and evaluate 3D OOD detection on ModelNet40 and ShapeNetCore in Fig.55 Performance improves K increases, reaching a dataset- specific peak at K = 7 for ModelNet40 and K = 4 for ShapeNetCore. Beyond these values, accuracy declines, likely due to the inclusion of irrelevant misleading examples that confuse the LLM. These results underscore the importance of selecting an appropriate number of in-context examples for optimal performance.
5 CONCLUSION
In this work, we introduce PGLLM, a novel framework for 3D point cloud understanding. Our approach constructs a graph to retrieve structurally similar 3D captions for each point cloud, using them as in-context examples to guide the LLM toward more informed reasoning. In paraliel, a score- based refinement mechanism leverages the intrinsic structure of the test data manifold to enhance prediction accuracy. PGLLM achieves competitive results across range of downstream tasks and sets new state-of-the-an performance in both 3D -ittr--distribttion detection and 3D recognition. Extensive experiments on diverse 3D point cloud datasets demonstrate the robustness and general- izability of our method. We believe this work offers a promising direction for advancing 3D point cloud understanding and integrating L.MMs into spatial perception tasks.
10

---

Published as a conference paper at ICLR 2026
Acknowledgments
This research is supported by the National Natural Science Foundation of China (No. U2468207), Sichuan Science and Technology Program (No. 2024NSFTD0036) and Agency for Science, Tech- nology and Research (A*STAR) under its MTC Programmatic Funds (Grant No. M23L7b0021).
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2303.088: 087744 2023.
Rishabh Agarwal, Avi Singh, Lei Zhang, Bernd Bohnet, Luis Rosias, Stephanie Chan, Biao Zhang, Ankesh Anand, Zaheer Abbas, Azade Nova, et al. Many-shot in-context learning. Advances in Neural Information Processing Systems, 2024.
Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applehaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, al. gpt-oss-120b. & gpt-oss-20b model card. arXiv preprint arXiv:2508. 10925, 2025.
lean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: visual language model for few-shot learning. Advances in neural information processing systems, 2022.
Antonio Alliegro, Francesco Cappio Borlino, and Tatiana Tommasi. 3dos: Towards 3d open set beaminn--eencmarrkie and understanding semantic novelty detection on point clouds. Advances in Neural Information Processing Systems, 2022.
Iro Armeni, Ozan Sener, Amir R. Zamir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Sil- vio Savarese. 3d semantic parsing of large-scale indoor spaces. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016.
Tianyi Bao, Qitian Wu, Zetian Jiang, Yiting Chen, Jiawei Sun, and Junchi Yan. Graph out-of- distribution detection goes neighborhood shaping. In Forty-first International Conference on Ma- chine Learning, 2024.
Lin Bie. Shouan Pan, Siqi Li, Yining Zhao, and Yue Gao. Graphi2p: Image-to-point cloud registra- tion with exploring pattern of correspondence via graph learning. In Proceedings of the Computer Vision and Pattern Recognition Conference, 2025.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et Language models are few-shot learners. Advances in neural information processing systems, 2020.
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint 1512.:3011.00122 2015.
Tiankai Chen, Yushu Li, Adam Goodge, Fei Teng. Xulei Yang, Tianrui Li, and Xun Xu. Exploiting vision language model for training- 3d point cloud ood detection via graph score propagation. In Proceedings of the IEEE/CVF International Conference Computer Vision, 2025.
Wenliang Dai. Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards general-purpose vision- Languag models with instruction tuning. Advances in neural information processing systems, 2023.
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of anno- tated 3d objects. In Proceedings of the IEEE/VVF conference computer vision and pattern recognition, 2023.
Tianyu Gao, Xingcheng Yao, and Danqi Chen. embeddings. ArXiv, 2021.
Simese: Simple contrastive learning of sentence
11

---

Published as a conference paper at ICLR 2026
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.
2024.
Ziyu Guo, Renrui Zhang, Xiangyang Zhu, Yiwen Tang, Xianzheng Ma, Jiaming Han, Kexin Chen, Peng Gao, Xianzhi Li. Hongsheng Li, et al. Point-bind & point-llm: Aligning point cloud with multi-modality for 3d understanding, generation, and instruction following. arXiv preprint arXiv:2309.006/ 2023.
Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models. Advances in Neural Information Processing Systems, 2023.
Brandon Huang, Chancharik Mitra, Leonid Karlinsky, Assaf Arbelle, Trevor Darrell, and Roei Herzig. Multimodal task vectors enable many-shot multimodal in-context learning. Advances in Neural Information Processing Systems, 2024.
Xue Jiang, Feng Liu, Zhen Fang, Hong Chen, Tongliang Liu, Feng Zheng, and Bo Han. Negative label guided OOD detection with pretrained vision- assion-language models. In The Twelfth International Conference on Learning Representations, 2024a.
Yixing Jiang, Jeremy Irvin, Ji Hun Wang, Muhammad Ahmed Chaudhry, Jonathan H Chen, and Andrew Y Ng. Many-shot in in-context learning in multimodal foundation models. arXiv preprint arXiv:2405.00998, 2024b.
Yannis Kalantidis, Giorgos Tolias, et al. Label propagation for zero-shot classification with vision- language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Mukai Li, Shansan Gong, Jiangtao Feng, Yiheng Xu, Jun Zhang, Zhiyong Wu, and Lingpeng Kong. In-context learning with many demonstration examples. arXiv preprint arXiv:2302 230.0433. 2023a.
Xiaonan Li, Kai Lv, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang, and Xipeng Qiu. Unified demonstration retriever for in-context learning. Association for Computa- tional Linguistics, 2023b.
Yushu Li, Yongyi Su, Adam Goodge, Kui Jia, and Xun Xu. Efficient and context-aware label propagation for zero-/few-shot training-free adaptation of vision-language model. In International Conference on Learning Representations, 2025.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chenggi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseck-v3 technical report. arXiv preprint arXiv:2412.19437,, 2024.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 2023.
Yifei Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, and Yixuan Li. Delving into out-of- distribution detection with vasion-language representations. Advances in neural information pro- cessing systems, 2022.
Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, Li Yi, and Kaisheng Ma. Shapellm: Universal 3d object understanding for embodied interaction. In Euro- pean Conference on Computer Vision, 2024a.
Zhangyang Qi, Ye Fang, Zeyi Sun, Xiaoyang Wu, Tong Wu, Jiaqi Wang, Dahua Lin, and Heng- shuang Zhao. Gpt4point: A unified framework for point-language understanding and generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024b.
Nils Reimers and Iryna Gurevych. Sentence-bert. Sentence embeddings using siamese bert- networks. In Conference on Empirical Methods in Natural Language Processing, 2019.
12

---

Published as a conference paper at ICLR 2026
Maximilian Stadler, Bertrand Charpentier, Simon Geisler, Daniel Zügner, and Stephan Günnemann. Graph posterior network: Bayesian predictive uncertainty for node classification. Advances in Neural Information Processing Systems, 2021.
Vladan Stojnić, Yannis Kalantidis, and Giorgos Tolias. Label propagation for zero-shot classification with vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattemn Recognition, 2024.
Yuan Tang, Xu Han, Xianzhi Li, Qiao Yu, Yixue Hao, Long Hu, and Min Chen. Minigpt-3d: Efficiently aligning 3d point clouds with large language models using 2d priors. In Proceedings of the 32nd ACM International Conference Multimedia, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et Llama: Open and efficient foundation language models. arXiv preprint wrXiv:2302.1397,, 2021,
Maria Tsimpoukelli, Jacob L Menick, Serkan Cabi, SM Eslami, Oriol Vinyals, and Felix Hill. Mul- timodal few-shot learning with frozen language models. Advances in Neural Information Pro- cessing Systems, 2021.
Qitian Wu, Yiting Chen, Chenxiao Yang, and Junchi Yan. Energy-based out-of-distribution detection for graph neural networks. arXiv preprint arXiv:2302.0291/ 2023.
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2015.
Yuxin Xiao, Paul Pu Liang, Umang Bhatt, Willie Neiswanger, Ruslan Salakhutdinov, and Louis- Philippe Morency. Uncertainty quantification with pre-trained language models: A large-scale empirical analysis. arXiv preprint arXiv:2210.04714, 2022.
Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin. Pointllm: Empowering large language models to understand point clouds. In European Conference on Computer Vision, 2024.
Le Xue, Ning Yu, Shu Zhang, Artemis Panagopoulou, Junnan Li, Roberto Martín- Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, et al. Ulip-2: Towards scalable multimodal pre-training for 3d understanding. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, 2024.
An Yang, Anfeng Li, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025a.
Qwen An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxin Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yi-Chao Zhang, Yunyang Wan, Yuqi Liu, Zeyu Cui, Zhenru Zhang. Zihan Qiu, Shanghaoran Quan, and Zekun Wang. Qwen2.5 technical report. ArXiv, 2024.
Senqiao Yang, Jiaming Liu, Renrui Zhang, Mingjie Pan, Ziyu Guo, Xiaoqi Li, Zehui Chen, Peng Gao, Hongsheng Li, Yandong Guo, al. Lidar-llm: Exploring the potential of large language models for 3d lidar understanding. In Proceedings of the AAAI Conference on Artificial Intelli- gence, 2025b.
Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, and Jie Zhou. Pre-training 3d point cloud transformers with masked point modeling. in 2022 ieee. In CVF Conference on Computer Vision and Pattern Recognition, 2021a.
Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, and Jiwen Lu. Point-bert: Pre- training 3d point cloud transformers with masked point modeling. 2022 IEEE/CVF Conference Computer Vision and Pattern Recognition, 2021b.
13

---

Published as a conference paper at ICLR 2026
Zhihao Yuan, Yibo Peng, Jinke Ren, Yinghong Liao, Yatong Han, Chun-Mei Feng, Hengshuang Zhao, Guambin Li, Shuguang Cui, and Zhen Li. Empowering large language models with 3d situation awareness. In Proceedings of the Computer Vision and Pattern Recognition Conference, 2025.
Yiming Zhang, Shi Feng, and Chenhao Tan. Active example selection for in-context learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022.
Xiangyang Zhu, Renrui Zhang, Bowei He, Ziyu Guo, Ziyao Zeng, Zipeng Qin, Shanghang Zhang, and Peng Gao. Pointelip v2: Prompting clip and gpt for powerful 3d open-world learning. In Proceedings of the IEEE/CVF international conference computer vision, 2023.
Xiaojin Zhu and Zoubin Ghahramani. Learning from labeled and unlabeled data with label propa- gation. ProQuest number: information to all users, 2002.
14

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 OVERVIEW OF THE TASKS
We illustrate the three tasks addressed in this work in Fig. 7 Our framework integrates 3D point clouds with large language models by first constructing prompts using PointLLM and point cloud feature graph (Step 1), then leveraging LLMs to perform downstream tasks (Step 2). PointLLM generates free-form descriptions of 3D objects (yellow boxes), which are refined and utilized by the LLM for tasks including 3D out-of-distribee (OOD) detection, 3D recognition, and 3D object captioning.
A.2 DATASET PARTITION FOR 3D OOD DETECTION
We follow the benchmark 3DOS, GSP to construct ShapeNetCore and S3DIS datasets for OOD detection. For the ModelNet40 dataset, we follow the partitioning practice established in the 3DOS work, dividing it into three subsets based on ascending index order. The specific category parti- tioning details are presented in Tab
Qualitative visualizations of some dataset samples are
presented in Fig. 10
Step 1: Prompt construction with PointLLM and point cloud feature Graph
3b Point Clouds
caprions for paint clouds
PointLLM
What thit
Erbeddinge
Encoder
Build
Construct cornespanding
model represants
eccentricalyy designed orplane. pradiminasilyy colored with engaging contrast of black..
graph
prompt
Step 2:
Utilize LLM to perform various downstream tasks
Siver free form LLM gererated desonition a 30 object (which may be inacsurate), calculate probabilit 100) belengs to any specified categary. Consider objects ontegory.
G-ven a free- form LLM- LMM-genestted description a 3D object (which may be
Optinize the given free--form LLM- generated descriptio (wirch may be (naccurate) of eoject using descriations objects some ooteaory.
iraccurate) e), compute the simianity score(0- 100)for each of the following categories with this description. sinioo desoriptions other objects cotegery.
descriptions
differencee
oepetments unchanged
descripiore siniar.
Gategories: AA, B,C,
Cotegonies: A
Cotegories: A, p..
The description requiring probability colculation. e The 3D ab joct model represents on eccentrically
The description on requiring probebility calculation < The 3D ob model
The dleccritiinn requining optimize. < The 3b object model inpressenss on eccentrically. >
repreisntss on accentriccly
pexeriptions sbjects similer festures: Descmitiond, Descriphin2....
Descr prions 3D objects with smilor festuncs: Descriptionl Descripion).
Descrip ricnal of objects similer Description. Descriphon...
Score
Scorel, Score2, Score3,
The 3D objectt model has on..
3D detection
Recognition
Object Captioning
Figure 7: Demonstrations of PGLLM. We propose PGLLM, an efficient and potent framework that integrates 3D-LLMs with Large Language Models, where the text on light yellow background indicates content generated by PointLLM. Furthermore, we demonstrate its operational mechanisms across 3D recognition, 3D OOD detection and 3D object captioning tasks.
mug, lamp, bed, washer, loudspeaker, telephone, dishwasher, camera, bird- house, jar, bowl, bookshelf, stove, bench, display, keyboard, clock, piano
SNI
earphone, knife, chair, pillow, table, laptop, mailbox, basket, file cabinet, cabi- net, sofa, printer, flowerpot, microphone, tower, bathtub, bag, trash bin
SN2
can, microwave, skateboard, faucet, train, guitar, pistol, helmet, watercraft, air- plane, bottle, cap, rocket, rifle. remote, car, bus. motorbike
SN3
Table 5: For each distinct out-of-distribution (OOD) subset partition on the ShapeNetCore, the categories residing within given subset are designated as in-distribution (ID), whereas categories from all other subsets are considered entirely OOD.
15

---

Published as a conference paper at ICLR 2026
in-distribution samples
somples
8
Figure 8: Visualization of the real-world benchmark S3DIS and the categories partition for OOD detection.
ID
window, door, table, chair, sofa, bookcase, clutter
OOD
beam, board, ceiling, column, floor, wall
Table 6: S3DIS dataset partitioning for OOD detection: foreground objects as ID and background objects OOD.
A.3 PROMPTS AND QUALITATIVE ANALYSIS
We provide the prompts for 3D recognition, 3D OOD detection and 3D object captioning tasks in Tab. 12 Furthermore, we provide supplementary visual analyses for 3D recognition and 3D OOD detection tasks in Fig. T1 evidenced in Fig. II the query sample contains erroncous caption, yet the collective accuracy of "vase" annotations within the in-context guidance enables the LLM to yield the correct predic- tion.Fig. 13 illustrates scenario where both the query sample and several in-context guidance cap- tions contain misleading annotations. This induces the LLM to output neutral score. Nevertheless, this outcome represents some improvement over the erroneous predictions generated without in- context guidance. Despite the fact that the query sample's caption lacks direct information about the categories, illustrated in Fig. 14 the captions of the in-context samples prove crucial for enabling the LLM to achieve a more accurate classification. However, shown in Fig. 12 the LLM's output inclined to classify the object This discrepancy is likely attributable to the presence of the word "lights" in the query sample's caption. Despite the in-context samples providing captions containing "faucet," the LLM ultimately assigned a full score of 100 to the "lamp" category, compared to only 60 for "faucet." In Fig, [5] we present examples where in-context guidance exerts a negative impact. can be observed that when the caption generated for the query sample is relatively accurate, but the captions of the in-context samples contain certain inaccuracies, the second-stage scoring becomes biased.
T3 and 14 provided with comprehensive prompts to the L.M. As
"lamp." which contradicts the ground-trus
label of "faucet."
airplane, bathtub, bed, bench, bookshelf, bottle, bowl, car, chair, cone, cup, curtain, desk
MN1
door, dresser, flower pot, glass box, guitar, keyboard, lamp, laptop, mantel, mon- itor, night stand, person, piano
MN2
plant, radio, range hood, sink, sofa, stairs, stool, table, tent, toilet, tv stand, vase, wardrobe, xbox
MN3
Table 7: For each distinct out-of-distributiinn (OOD) subset partition on the ModelNet40, the cate- gories residing within a given subset are designated as in-distribution (ID), whereas categories from all other subsets are considered entirely OOD.
16

---

Published as a conference paper at ICLR 2026
50% samples
60% samples
70% samples
80% samples
90% samples
100% samples
AUROC 1 FPR95 y
1
FPR95
AUROC
FPR95
AUROC1 1 FPR95
AUROC1
FPR95
AUROC 1 FPR95
86.5
39.3
87.4
57.4
87.2
57.5
87.0
39.6
86.5
61.9
86.8
58.3
Table 8: The Dynamic Graph Expansion experiments conducted on MNI.
DeepSeek V3 CONY/UUS $)
Qwen-Plus (CNY / USD $)
Cost
GPT-4 (USD
Total (2,468 samples)
$28.00
5 $0.56
V7 / 50.98
Per 1,000 samples
$11.30
v1.6 $0.23
V2.8 $0.39
AUROC (%)
85,9
82.1
82.9
Table 9: Cost comparison of different second-stage LLMs on the OOD detection task. Exchange rate: 1 USD 7.11 CNY.
A.4 DYNAMIC GRAPH EXPANSION
To mimic practical deployment scenarios where the full test set is not always accessible at once and the available evaluation data may be dynamically reduced or updated over time, we design an experiment to assess the dynamic graph expansion mechanism under incremental query arrival. Specifically, we randomly split the ModelNet40 testing set into two equal halves (50% each). The initial graph is constructed using first half of the data. Then, from the remaining 50%, we randomly select 10% of the samples as test instances and incrementally add them to the graph for evaluation. Notably, adding we follow the method proposed in Li ct al. (2025) to dynamically update the graph structure for the newly added samples. This incremental addition process is repeated five times, and the results after each addition are reported in Tab. g These results correspond to the test performance evaluated on the entire accumulated samples after each addition of new samples. Finally, when the sample count reaches 100%, all test samples from the ModelNet40 dataset have been incorporated and evaluated. At this point, AUROC and FPR95 reach 86.8 and 58.3, respectively. In comparison, the results reported in the graph constructed from the full dataset-are are 89.6 and 53.1 for AUROC and FPR95, respectively. Although there is a moderate performance degradation, our approach still exhibits strong robustness and generalization capability. This further demonstrates that the proposed method does not strictly rely access the entire global dataset to achieve effective performance.
samples does not require reconstructing the entire graph from scratch; instead,
by performing inference directly on the complete
A.5 IMPACT OF K-VALUES
To further explore the influence of K-values, test different number of K on 3D recognition task. The results are shown Fig. 19, it can observed that the optimal performance achieved at K=5 for ModelNet40 and K=6 for ShapeNet, respectively. Furthermore, performance exhibits a declining trend with increasing K-values, a finding empirically consistent with the conclusions presented in our prior submission.
A.6 FINANCIAL COST
We evaluated three widely used LLMs as the second-stage model in our OOD inference pipeline and computed their corresponding API costs. Tab. 9 reports the total cost of processing the entire ModelNet40 test set (2,468 samples). GPT-4 incurs cost of approximately USD $28, whereas DeepSeck-Vi V3 and Qwen-3 Plus cost CNY Y4 (USD $0.56) and CNY Y7 (USD S0.98), respectively. All three models are called using API.
This reveals 30x-50x cost gap between GPT-4 and the two models. Despite this large difference in cost, DeepSeek- V3 and Qwen-3 Plus achieve comparable AUROC performance (82.1-88 122.1-22.99 85.9% for GPT-4). Given this favorable post-performance trade-off, practitioners may prefer adopting DeepSeek-V V3 or Qwen-3 Plus as the second-stage LLM, especially in large-scale or cost- sensitive deployments.
17

---

Published as a conference paper at ICLR 2026
Cost for Existing Approach (PointLLM)
Additional Cost for PGLLM (Our Method)
Score Propagation (OOD)
Score
Retrieval
Caption Generation
2nd stage
Gruph Construction
Propagation (Recog)
for in-context
LLM
guidance
FLOPs Memory
1700G 14GB
unknown unknown unknown 80mins
9.6G
0.12G
2.4G
27.8k 14KB
46.
9KB
0.19MB
Param
0
Time consumption
107mins
126ms
101ms
106ms
1ms
Table 10: Computation cost of individual components on ModelNet4O.
Input data (100K)
embeddings
adjacency
3D caption
Storage
19.2G (in Hard Drive)
1.24G (in GPU VRAM)
8M (in GPU)
87M (in GPU)
Table 11: Storage cost for 100K samples.
Inference Time: Tab. 10 summarizes the computation cost of each component on ModelNer40. Overall, PGLLM introduces negligible extra overhead compared to the LLM-heave baseline. The added graph construction and score propagation are lightweight, it cost 9.6G FLOPs and 46.8MB memory for graph construction and 0. 02-2.G. FLOPs with only KB-level memory for propagation, which taking only 101-126ms in total. In contrast, the existing pipeline is dominated by caption gen- eration and second-stage LLM inference (107mins and 80mins, respectively). This shows PGLLM is computationally efficient and practical in real deployments.
A.7 DEVICE MEMORY FOR GRAPH STORAGE BUDGETS
In Tab. II we summarize the storage requirements of each system component. Notably, 100K samples require only about 19 GB of hard drive storage, while their corresponding embeddings occupy just 1.2 GB when loaded into GPU VRAM for computation. These explicit storage budgets demonstrate that even at a 100K-support scale, both the graph and caption caches fit comfortably within single GPU. The overall memory footprint is therefore still dominated by the backbone 3D-LLM parameters and activations, rather than by our graph-based test-time scaling modules.
Acouray 66
Accuracy GB
62
60
62
38
60
5
6
7
8
9
(a)
(b) 3D recognition ShanonlttttoooSSeeeeeeeCCCCooo ShapeNdtCore
Figure 9: Different number of K-values for 3D recognition on two datasets.
18

---

Published as a conference paper at ICLR 2026
Tasks
Prompts
Given a free-form description of object, the content described here helongs following 54 categories. Use this description compute similarity score 1000 for each the following 54 categuries. The description of this 3D object is generated by LLM and may be inaccurate. In addition, I will provide you with descriptions of other 3D objects that share similar Jeatures with this object of this category. O.no relation, 100=perfect match.
the
categories: mug, lamp, bed, washer, loudspeaker, telephone, dishwasher, camera. birdhouse, jar, bowl, hookshelf, stove, bench, display, keyhoard, clock, piano, earphone, knife, chair, pillow, table, laplop, mailbox, basket, file cabinet, cabinet, sofa, printer, llowerpol, microphone, tower, bathtub, bag. trash bin, can, microwave, skatchoard, faucet, train, guitar, pistol, helmet, watereraft, airplane, bottle, cap. rocket, rifle, remote, car, bus, motorbike.
3D Recognition
3D object description: ... generated by PointLLM)
Descriptions of 3D objects with similar features: generated by PointLLM and seleted by mainfold learning)
I...
2.
3.
Please ouiput the 54 corresponding similarity scures in the order of the above-mentioned categuries, without any additional explanation.
Given free-fon description of 3D object, please calculate the probability (0-100) that the content described in the following text pertains to any of following categories. The description of this 3D object generated by an LLM and may inaccurate. addition. will provide you with descriptions other 3D objects that share similar features with this object of this category. need take these similar 3D model descriptions into account well. (=no relation, 100=perfect match.
categories: plant, radio, range hood, sink, sofa, stairs, stool, table, tent, toilet, tv stand, vase, wardrobe. xbox.
3D OOD detection
3D object description: (# generated by LM)
Descriptions of 3D objects with similar features: (f) generated by PointLLM and seleted by mainfold learning)
1.
2.
3.
Output only numerical score. Do provide additional explanations.
Given free-form description of object, please according to additional descriptions other 3D objects that share similar features with this object which has the same category to optimize the given description The description of this 3D object and additional descriptions an LLM and may be inaccurate. If all descriptions are similar,, please du not make any changes. If the description content is different, please make slight adjustments to this 3D description and do not adjust too much.
generated by
3D Object Captioning
The description requiring optimize: (#f generated by PointLLM)
Descriptions of 3D objects with similar features: (#) generated by PointLLM and seleted by mainfold learning)
1.
2. ...
3.
Output only a 3D description. And don't describe loo much.
Table 12: A list of prompts for 3D Recognition, 3D OOD Detection, and 3D Object Captioning tasks to L.M..
19

---

Published as a conference paper at ICLR 2026
ModelNet40
Objaverse
ShapeNetCore
Figure 10: Visualization of the ModelNet40,, Objaverse, and ShapeNetCore.
20

---

Published as a conference paper at ICLR 2026
In-context somple
KNNN Graph
Ioncentext sample
Ground Truth: vase
In-context sample
Query sample
3D model cartoon-style orhament rendered black, which characterized its shiny, smooth surface and reflective properties, emphasizing its three-dimensiondl form. The ornament, with its simplistic and charming design, could be used for decorative purposes like on Christmas tree or children's play area, adding a thematic element to the scene.
Given a free-form description of 3D object, the content described here belengs one of the following 40 categories. Use description compute a similarity score (0-100) for following 40 categories. The description of this 3D object generated by on LLM and may be inaccurate. In addition. features with this object of this category. O:no relation, 100=perfect match.
the
provide you descriptions of 3D objects share similar
categonies:
airplane, bathtub, bed, bench, bookshelf, bottle, bowl, car, chair, cone, cup, curtain, desk, door, dresser, flower pat, glass box, guitar, keyboard, lamp, laptop, mantel, monitor, night stand, person, piano, plant, radio, range hood, sink, sofa, stairs, stool, table, tent, toilet, tv stand, vase, wordrobe, xbox.
model cartoon-style ornament rendered black, which characterized by its shiny, smooth surface and reflective properties, emphasizing its three- dimensional form. ornament, with simplistic charming design, could be used decorative purposes like on a Christmas tree or in children's s play area, adding thematic element to the scene.
3D object description:
Descriptions of 3D objects with similar features: The 3D model object is high-shine, jet-black vase that's perfectly cylindrical shape. Its sleek design and polished surface give it air of sophistication and elegance, As a central element in interior decor, it's used to flowers or With streamlined shope, it would fit
decorative objects, enhancing the oesthetic appeal of well into variety interior styles, from modern traditional.
The 3D object model depicts elegant, tall, cylindrical hidden, round on top. carries the characteristics of funerary offering vessel, traditionally used ancient Egypt for holding offerings the gods. Its black hue suggests it could be used to store colored liquids or flowers. Despite antique look, maintoins simple yet sophisticated design.
black color, with small, preferably
object in question is a cortoon-inspired block ornoment color and features simplistic, minimalist design, which characteristic of the cartoon aesthetic. The object oppears decorative objects. Its cartoonish design suggests that it may be used in animation, video gomes, or digital media require objects playful or whimsical style.
resembles vase. dark
empty vessel, implying that its primory function
flowers or
output cornesponding similarity scores without any additional explanation.
of
mentioned categories,
thinlo The ornament's use Christmas decoration might hint "plant" (via Christmas trees) but the exomples don't support this. Most categories like "car" "guitar" have relevonce. "Sofa" "table" are furniture, not decorative ornements. Notably, the examples provided are vases, reinforcing that category. The scores should reflect this: high for "vose" (95), low for "flower pot" due functional overlap, minimal scores unrelated categories, slightly higher for "battle" (30) and "bowl" (25) due shope similorities. </thinlo Output: 0, O, 25, 0, 0, 95, 0.0
Figure 11: Qualitative example for 3D recognition task on ModelNet40.
21

---

Published as a conference paper at ICLR 2026
In- Co-ontext Somple
Graph
Ground Truth: faucet
In-context sample
In-context sample
Query sample
What this?
The object is sturdy. metallic arm supporting two strong, circular lights. The lights are likely used for illumination and provide brilliant spread of light in all directions. Its rebust construction implies durability longevity, making suitable for
lights could suggest the object is designed to provide comprehersive illumination dorlnees or low-light conditions.
outdoor uses.
Given a free-form description of abject, the content described here belongs catagonies,
of the following 54 description compute similarity score 100) each following categories. The description of this 3D object is generated by an LLA and may be inaccurate, In addition, will provide you with descriptions other 3D objects that share similar features with this object of category, Osno relation, 100uperfect match.
birdhouse, jar, bawl, bookshelf, stove, berch, displey, keyboard. cleck, piano, earphone, knife, choir, pillow, table, laptop, moi lbox, basket, file cabinet, cabinet, sofa, printer, flowerpot, microphone, tower, bathtub, bag. trash bin, can, microwave, skateboard, faucet, train, guitor, pistol, helmet, watercroft, airplane, bottle, cop, rocket, rifle, remote, car, bus, motorbike.
categonies: mug, lamp, bed, washer, loudspeaker, telephone, dishwasher,
abject description: The object sturdy, metallic supporting strong, circulor lights, The lights likely used illumination and provide brilliant spread light in directions. Its robust construction implies durability and longevity, making suitable for both indoor ond outdoor uses. The two lights could suggest the object designed provide comprehensive illumination dorkness low-light
Descriptions of 3b objects with similar features: This 3D model depicts sturdy, well-mode foucet with distinctive cartoon-like design. features aftractive palished black finish, complete with handle for easy use. The model clearly reflects bathroom or kitchen sink, indicating its function as water fixture controlling water flow and used for household cleaning. Despite the cartoon style, the design still conveys overall practicaa ob jectss
utilittaiian espect, demonstrating the
aesthetics and functionality everyday.
The 3D model is black, cartoon-style faucet handle. This design typically appears in bathrooms kitchens. used controlling water flow
knob, indicating It has a simple, streamlined shape and painted in uniform black color, adding to its cartoonish charm. The model jbject black, cartoon style faucet. The model rendered in dark color, showcasing appealing simplistic design that could be typically seen children's animated bothrooms kitchens. The contoonish appearance does not neglect the important functional aspects like the handle and spout, indicating is designed for usable purpeses in virtual setting.
temperature.
Please output the 54 corresponding similarity scores in the order of the above mentioned categaries, without additionaal explonation.
thinio The object clearly some kind of light source, with two circular lights on metullic orm. It sounds like lamp passibly streetlight or something similar. Now, I have descriptions of other 3D objects that share similor features, foucet: one faucet with hondle, onother foucet handle, and think I need to focus on the main object description and compare it to each category based on common given, They are about faucets, which might not be relevant the
these descriptions
faucet. The similor ob objects all variations of
third cartoon-style foucet. I
The similor objects object is lamp. Perhaps for faucet category, should consider the similar objects, but the similar objects one exomples of faucets, for the main object, when scoring against faucet, the similar ob jojets show what match, 0, 0, 0,
faucet Output: O 100 O, O O
object
</hrec/
60. O 0,
Figure 12: Qualitative example for 3D recognition task on ShapeNetCore.
22

---

Published as a conference paper at ICLR 2026
In-context sample
KNN Graph
In- co--onnexxt sample 2
Ground Truth: dresser
context somple
Query sample
What this?
3D object model oppears battom. This item, typically constructed from organic materials like wood or fibers, is handy for stoning and carrying items. Its open bottom allows for eazy londing and unloading, making user-ffiendly. The texture suggests traditional craftsmanship various purposes, including as kitchen basket for fruits, vegetables, and other products or as decorative tray tiles, f-shirts, trivets,
practical household item, possibly a basket tray with open
aesthetic appeal. This versatile object can utiliized for
items, reflecting its utility everyday home environments
Given frea-form description 3D object, please calculate probability (0-100) that content described in the fallowing text pertoins to any of the following categories. The description of this 3D object generated by on LLM and may be inoccurate. In oddition, will provide you with descriptions of other 30 objects share similar features descriptions into account as well. O=no relation, 100=perfect match.
object this category.
to take these similar model
Categories: door, dresser, flower pot, glass box, guitor, keyboard, lamp, loptop, mantel, monitor, night stand, person, piano.
practical household item, possibly weven basket o tray with en open bottom. This item, Typically constructed from organic materials wood or fibers, is handy for storing and carrying items. Its open bottom allows for easy loading and unloading, making it user- friendly. The
3D object description: The 3D object model appears
texture suggests traditional craftssa and aesthetic appeal. con be utilized for various purposes, including as a kitcheen basket for fruits, vegetobles, and other products or decorative tray for tiles, t-shirts, trivets, and other items, reflecting its uhility in everyday home environmentts.
versatile object
model of a functional and versatile 5-drower single dressen, ideal for small bedrooms guest rooms. The dresser showcases a chic, modern design with black
Descriptions of 3D objects with similer features: This
effortlessly into any style interior drowers, each with pull-out handle for easy access, thus enhancing space efficiency It minimalist in style, yet functional, offering elegant and convenient salution for storing personal items, clothes and accessories. A 3D model of rectangular box in solid black color. The box appears be closed, with no visible openings vent's. Its simplicity and uniformity suggest could storage box, decorative piece, even piece of furniture, depending and functionalities nemain unspecified,
would
It features
evenly
size and the user's needs, However, without further information, its exoct purpose
object model compact, revealing its single, well-organized compartment. It is supported by sleek, black legs that elevate slightly off the ground, contributing its simplicity and would function well as a stylish sterage solution for small items in verious settings, including bedrooms, living reoms, and offices.
drawer
smooth, glossy black finish.
open.
modern, minimolist oesthetic. piece furniture elegont
Output only numerical score. Do not provide additional explanations.
<thinlo Starting with 'dresser' since there similar object descriptions provided, The key function here is storage. The main object is a basket/trou /trey which also serves storage purposes. However, a dresser tyiiclly has drowers or compartments, while the basket has open bottom, Next, 'gloss box' might be contender becouse of storage aspect, but described object is organic has drowers or shelves, which isn't 't mentioned. 'Mantel' is shelf ever fireplace, which doesn't 't fit, The the storage function,
woven, not glass. 'Night stond' usually
physical structure different. basket's open bottom and woven texture are more akin container like a basket, which isn't listed Among the options, 'desser' is the best fit functionally, but the form differs. The probability would moderate. say 50, considering functional similority but structural differences. </thinko Output: 50
closest category
Figure 13: Qualitative example for 3D OOD detection task ModelNet40.
23

---

Published as a conference paper at ICLR 2026
In-confext sample
KNN Groph
Ground Truth: helmet
Con-oottett somple
In-context sample
Query sample
What this?
This 3D object model depicts human head designed cartoonish style. It sports a uniform grey color, suggesting it might be basic, untextured model. Further customization can be done on this basis, either for artistic purposes, animation, video gome design. or other visuol graphics applications. typically ly lends itself well to more abstract and simplistic designs, and this model, while not realistic, is a prime example of that.
aesthetic
Given free: form description 3D object, calculate the probability (0. 100) that the described in the following text pertoins any the following categories. description this object generated by on LLM and may be inaccuncte. In addition. I will provide with descriptions of other 3b objects that share similar features with this object this category. You nead take these similar 3b model descriptions into account as well, Oino relation, 100sperfect match.
categories: earphane, knife, chair, pillow, table, laptop, mailbox, basket, file cabinet, cabinet, safa, printer, flowerpot, microphone, tower, bothtub, bog. trosh bin
3D object description: This 3D object model depicts humon head designed cartcenish style. sports uniform grey color, suggesting might be this basis, either for artistic purpases, animation, video game design, or other visual graphics applications. The cortoon oesthetic typically lends itself well to more abstract and simplistic designs, and this model, while not realistic, a prime exomple
untextured model. Further customization can
Descriptions of 3D objects with similor features: This is 3D model of humon face that metallic in of gold bronze statue. facial features well-defined, the model holds on artistic value in addition to any functional use. It might serve os decorative object interior design, or as representation for artistic sculptural purposes.
appearance,
This 3D model of cartoon character' head that is entirely shaded solid, pure white color. The model, though minimolistic no texture, shape, or distinguishing features, its simplistic design prevides on open-ended creative opportunity for further customization adaptation various media graphic design projects. model represents metallic biker's helmet, notable dank screen, presumably to shield the wearer's 's eyes from wind, dust, or sun, and pair of round, dark-rimmed glosses to protect the from wind, sun, injury. This type of helmet typically used bikers to safeguard their during rides protection.
of facial details, maintains the essential structural aspect head. With
grey colour
sleak design. It features
alse
work environments require eye
Output only numerical
provide additional explonations.
thinke need to compane this description to each category and see if it matches. The description is about humon head, which not directly listed the categories, electronics, etc. A human head doesn't fit into any of these categories directly. These similar objects are related to heads or headgear. But the categories I'm comporing to dan't t include heads or helmets directly. None these
categories everyday objects like furniture,
things like earphone, knife, chair, "bay" or "trash bin," but that seems unlikely. I recell that some 3D model datobases, heads might be classified under "characters" or "pzople", but here the categories are all objects, not people. So, think score is Ox/thirio Output: 0
The cotegories
head related except passibly
Figure 14: Qualitative example for 3D OOD detection task on ShapeNetCore.
24

---

Published as a conference paper at ICLR 2026
Un-context somple
KNNN KNNN Graph
In-confext sample
Ground Truth: xbox
In-context sample
Query Sample
What is this?
The 3D object model in question appears to be sleek black rectangular body, distinguishing itself with distinct curved U-shoped top. smooth streamlined design lends itself contemporary minimalist aesthetic. Although no explicit functionality is apporent from the object itself, it could be a part of various types decorative
electronic mechanical devices could used stylish housing items.
Given free-farm description of 3D object, please calculate the probability (0-10 (0-100) that the content described in the following text pertoirs generated by on LLM and may be inaccurate. In addition. objects that share similar features with this object of this category. You need take these similor 3b model descriptions account well. relation, 100:perfect match.
of the fallowing categories. The description of this 3D object provide you with descriptions other
Categonies: plant, rodio, range. hood, sink, sofa, stairs, stool, table, tent, toilet, tv stand, wardrobe, xbox.
3D object description: The 3D object medel in question appeors to be sleek black rectangular body. distinguishing itself with distinct curved U-shoped top. The smooth and streamlined design lends itself contemponary and minimalist oesthetic, Although no explicit functionality apporent from the object itself. it could part various electronic certain types of decorative items.
could be used stylish housing
objects with similar features: This is black rectangular object with a unique design feature of vertically oriented, triongular holes on its front surface. The object primorily dark in color, orrangement the holes.
Descriptions
adding sense mystery or intrigue. seems simple yet striling due object's 's functionality isn't explicitly stated, but one can infer that it might be used for aesthetic purposes, possibly as modern decorative piece unique component A rectongular, 3D box model presented in solid black color. The model appears minimalistic in design, with shorp and de features or details. In terms of usage, such 3D model can be used numerous ways including digital onimations, 3D modeling, virtual reality, and games. Given the lock of color texture, the box could symbolize an empty container
artwerk.
edges
surfoces. It's essentially simple geometric shope, with additional
basic structure in design programming tosks.
The 3D model depicts cartoon themed rectangular black object which strongly resembles beg. This item, painted simple and bold black color, exhibits design, okin to those often seen in cartoons or animations. enhoncing its visual appeal and playfulness. Although from image, appearance suggests
rectangular shope. It
stylized, non-realistic
could be used various contexts, such as prop in cartoon onimation or a playful themed design in 3D printing.
functionality
distinctly
Output only numerical score. not provide additional explanations.
<hiino Given the main object's description- sleek black rectangular body with curved U-shaped top, minimalistic design, and possible use part of on electronic mechanical device and considering that the similar objects
minimalistic, the only category fits is ronge heed, which often features a rectangular base with curved canopy. Other categories (rodio, table,
black, rectangular,
list that meaningfully
match the described shape function. However, in the description of other object descriptions, there are additional phrases such as triangular holes and strongly resembles a bog. which not align well with the characteristic features any of the existing categories. Therefore, the probability that the object belongs one of the listed categories-man plausibly a range hood-can be estimated about 75, </thinlo Output:
stond, etc.)
Figure 15: Bad example for 3D OOD detection task on ModelNer40.
25


Reviews:
[{"review_id": "12dMu2UOPJ", "reviewer": "Reviewer_5Jwa", "summary": "This work leverages a KNN-based graph and a confidence score refinement mechanism to build a Point Graph using a pre-trained PointLLM together with a second-stage large model (e.g., GPT-4 or Qwen). The method operates entirely at test time and effectively improves performance on OOD detection, classification, and captioning tasks.", "questions": "**1. Dependency on GPT-4 and Cost–Performance Trade-off**\n\nThe method achieves its strongest results when paired with GPT-4. However, GPT-4 is a proprietary model and requires paid API access, which may limit the practicality and scalability of this approach in real-world deployment scenarios or resource-constrained environments. In contrast, the performance using an open-source model like Qwen appears significantly weaker.\n\nTo strengthen the paper, I recommend a deeper analysis of the **performance–cost trade-off**, including:\n\n* A quantitative comparison of accuracy vs. computational/financial cost between GPT-4 and Qwen.\n* Discussion of whether intermediate open-source models (e.g., Qwen-Plus, Llama-3 variants) can offer a more balanced trade-off.\n* Insights into how organizations without commercial model access could adopt this framework efficiently.\n\nSuch analysis would provide more practical guidance for deployment and broaden the method's applicability.\n\n---\n\n**2. Evaluation on Broader 3D-LLMs for Generality**\n\nThe current evaluation focuses primarily on PointLLM combined with GPT-style LLMs. While this is valuable, it remains unclear whether the framework generalizes across different 3D foundation models. To convincingly demonstrate method robustness and universality, I recommend including results on additional state-of-the-art 3D-LLMs such as **ShapeLLM, ShapeLLM-Omni, Uni3D-LLM**, or other emerging architectures.\n\nThis evaluation would help clarify:\n\n* Whether improvements stem from the proposed Point-Graph mechanism rather than characteristics of a specific backbone.\n* The compatibility of this framework with diverse 3D model designs and training paradigms.\n* Potential limitations or adaptations needed for different 3D-LLM families.\n\nSuch ablation and cross-model experiments would significantly enhance the paper’s credibility and contribution.\n\n---\n\n**3. Test-Time Scaling Efficiency and Resource Overhead**\n\nThe work emphasizes test-time scaling and test-time refinement, yet the computational implications of these procedures are not fully discussed. For practical deployment and fair comparison with prior work, it is essential to provide a comprehensive efficiency analysis, including:\n\n* **Inference latency** before and after applying test-time scaling\n* **GPU memory consumption** for graph construction and refinement\n* **Runtime overhead per query** as the support set size grows\n* **Scalability analysis** with respect to dataset size and number of neighbor samples\n* Discussion of whether there are diminishing returns under limited compute budgets\n\nProviding these metrics will clarify the computational footprint and demonstrate that the reported performance gains are achieved at a reasonable cost, which is particularly important for real-time or large-scale applications.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The framework emphasizes **test-time scaling** and integrates PointLLM inference with a graph-based refinement strategy.\n2. The overall figure and visualizations are clear and well-organized.", "weaknesses": "1. The best performance relies on GPT-4, which is closed-source and incurs API and monetary costs, while the Qwen version shows relatively weaker results. How can one balance performance and cost in practical deployment?\n2. It would be valuable to evaluate the framework on more 3D-LLMs, such as ShapeLLM, to demonstrate broader applicability and generality.\n3. From an efficiency perspective, I would like to see the impact of test-time scaling on inference latency, GPU memory usage, and other computational metrics.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kyw27VCXDx", "reviewer": "Reviewer_9Mmb", "summary": "The paper proposes PGLLM, a test-time framework for 3D point-cloud understanding that (i) builds a KNN graph over support embeddings from a frozen 3D encoder, (ii) uses neighbor captions as in-context demonstrations for a second-stage LLM (“in-context guidance”), and (iii) refines recognition/OOD scores via label propagation. Experiments on ModelNet40, ShapeNetCore, S3DIS, and Objaverse show gains in OOD and recognition, plus a small captioning improvement.", "questions": "Captioning degradation is unresolved. The proposed test-time guidance reduces caption quality on several splits. As written, there is no justification for using it in captioning. Either (i) present risk-controlled variants that consistently improve captioning and report results, or (ii) restrict the method’s scope to recognition/OOD and state captioning as out-of-scope.\n\nFLOPs & memory accounting. Report FLOPs, params, and memory for: 3D encoder pass, graph construction, retrieval, label propagation, and second-stage LLM.\n\nGraph storage budgets. For the Objaverse-O setting (100k support): list bytes for embeddings + adjacency + captions; show peak host and device memory.\n\nAblate components you still use. The combined method (in-context + propagation) helps (Tab. 3), but show cases where only in-context hurts vs baseline (and why). Provide qualitative examples where label propagation corrects vs amplifies errors.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "-- Clear, modular test-time pipeline; no retraining of LLMs required.\n\n-- Solid ablations: with/without in-context guidance and score propagation; K-sensitivity; task coverage (recognition, OOD, captioning).\n\n-- Time breakdown table suggests negligible overhead for graph ops and propagation relative to caption/LLM inference.", "weaknesses": "FLOPs/Compute attribution is absent. The paper reports per-sample latency but not FLOPs/param attribution for each stage (encoder feature extraction, graph build, LLM inference, propagation). Without FLOPs, it’s hard to compare to alternatives like direct k-NN retrieval or pure prompt-engineering baselines at matched compute. Please add per-module FLOPs and parameter counts (and, ideally, energy or GPU utilization) to substantiate “very little extra cost.”\n\nGraph storage & memory footprint not quantified. The method keeps a KNN graph over support embeddings and captions; storage and RAM/VRAM requirements are not analyzed. Provide:\n\nCaptioning degradation is unresolved. The proposed test-time guidance reduces caption quality on several splits. As written, there is no justification for using it in captioning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fnFVpFg1yA", "reviewer": "Reviewer_95ua", "summary": "This paper proposes Point-Graph LLM (PGLLM), a framework designed to enhance the 3D understanding capability of large language models at test time.  \n\nIts core ideas are as follows:\n\n1. Manifold-aware in-context guidance: A KNN graph is constructed, where edge weights encode feature similarity among 3D point clouds. For each query sample, the framework retrieves its nearest neighbors and their corresponding PointLLM-generated textual descriptions. These neighbor captions are incorporated as in-context exemplars within the LLM prompt, enabling more semantically consistent reasoning along the data manifold.  \n\n2. Label-propagation-based score refinement: A graph-based confidence propagation mechanism is applied to smooth and refine prediction scores over the manifold, thereby improving the stability and reliability of both classification and OOD detection.  \n\nExperiments are conducted on multiple 3D benchmarks, including ModelNet40, ShapeNetCore, S3DIS, and Objaverse, covering three major downstream tasks: 3D classification, 3D OOD detection, and 3D object captioning.  \nThe results demonstrate that PGLLM achieves SOTA performance on 3D recognition and OOD detection while maintaining competitive results on captioning, with negligible additional computational overhead.", "questions": "1. The choice of K appears inconsistent. The main experiments (Table 1) use K=3, while the ablation study (Figure 4) shows that the best OOD detection performance occurs at K=7 for ModelNet40 and K=4 for ShapeNetCore. Why was K=3 chosen for the main results? Would the reported SOTA results in Table 1 improve further if the empirically optimal K values from Figure 4 were applied?  \n\n2. Sensitivity to caption quality: How robust is PGLLM to the quality of the initial captions generated by PointLLM? Have you considered an ablation study in which the in-context exemplars are derived from non-LLM sources (e.g., ground-truth labels or simple template-based descriptions, if available) to isolate the influence of caption quality on overall performance?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. A simple yet effective test-time optimization paradigm: The framework leverages KNN-based neighbor descriptions as ICL exemplars, combined with label-propagation-based score refinement. The design is conceptually simple but aligns well with the intuition of manifold consistency. Moreover, it is fully plug-and-play for existing PointLLM-based approaches, requiring no retraining.  \n\n2. Originality: The central idea of this work—integrating manifold learning with ICL at test time for 3D point cloud understanding—shows a high degree of originality. Rather than merely employing graph-based features, the method creatively uses the textual descriptions of neighboring samples as contextual exemplars for the LLM. This combination of a 3D encoder (for graph construction), a 3D multimodal LLM (for description generation), and a second-stage LLM (for reasoning) represents a novel and well-engineered design. The subsequent score refinement through label propagation on the LLM-generated confidence scores is also an elegant and complementary addition.  \n\n3. Clarity: The paper is exceptionally well-written and easy to follow. It clearly identifies a key limitation of previous works (interpreting each point cloud in isolation) and presents a coherent, step-by-step solution. The problem definition, methodology, and experimental setup are all precisely described, making the contribution easy to understand and evaluate.", "weaknesses": "1. Limited performance on captioning: Although the method outperforms baselines on the 3D object captioning task, it does not achieve SOTA results. The paper attributes this to the limited size of the test dataset, but this explanation is weak and unconvincing. Since the proposed T variant uses the entire test set to construct the graph, there is a potential risk of data leakage compared to inductive baselines. Moreover, the O variant still performs worse than the T variant, suggesting that the “in-context refinement” mechanism (Section 3.3) for generation may be less effective than the score-based refinement mechanism used for classification.  \n\n2. Scalability and transductive setting: The method appears to operate under a transductive assumption, where it must access the entire test set (or a large support set 𝒟ᵤ) to build the graph before inference. This assumption (in PGLLMᵀ) may not hold in practical inductive scenarios where samples arrive sequentially. Although the paper introduces an alternative variant (PGLLMᴼ) using an external dataset, this variant relies on a large 100K-sample subset from Objaverse. While the paper briefly mentions a “dynamic graph expansion” scheme as a potential remedy, it neither evaluates its performance nor analyzes the computational cost. The scalability of constructing the initial graph (an Nᵤ × Nᵤ similarity matrix) also becomes a potential concern for large-scale datasets. Furthermore, performance degradation is evident when using the external dataset.  \n\n3. Dependency on pre-trained models: The framework’s success heavily depends on the quality of two pretrained components: the 3D encoder (Point-BERT) and the description generator (PointLLM). The paper acknowledges that poor or inaccurate captions may mislead the LLM (as illustrated in Figure 11), but it does not analyze this sensitivity in depth. If the initial captions are noisy or semantically incorrect—essentially a “garbage in, garbage out” problem—the effectiveness of the in-context guidance may significantly degrade.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)