SceneForge: Enhancing 3D-text alignment with Structured Scene Compositions
Authors: Cristian Sbrolli
Matteo Matteucci
Venue: NeurIPS 2025 poster ()
Keywords: multimodal
3D
Primary area: deep_learning

Abstract: The whole is greater than the sum of its parts, even in 3D-text contrastive learning. We introduce SceneForge, a novel framework that enhances contrastive alignment between 3D point clouds and text through structured multi-object scene compositions. SceneForge leverages individual 3D shapes to construct multi-object scenes with explicit spatial relations, pairing them with coherent multi-object descriptions refined by a large language model. By augmenting contrastive training with these structured, compositional samples, SceneForge effectively addresses the scarcity of large-scale 3D-text datasets, significantly enriching data complexity and diversity. We systematically investigate critical design elements, such as the optimal number of objects per scene, the proportion of compositional samples in training batches, and scene construction strategies. Extensive experiments demonstrate that SceneForge delivers substantial performance gains across multiple tasks, including zero-shot classification on ModelNet, ScanObjNN, Objaverse-LVIS, and ScanNet, as well as few-shot part segmentation on ShapeNetPart. SceneForge’s compositional augmentations are model-agnostic, consistently improving performance across multiple encoder architectures. Moreover, SceneForge improves 3D visual question answering on ScanQA, generalizes robustly to retrieval scenarios with increasing scene complexity, and showcases spatial reasoning capabilities by adapting spatial configurations to align precisely with textual instructions.

SCENEFORGE: Enhancing 3D-text alignment with Structured Scene Compositions
Cristian Shrolli
Matteo Matteucci Department of Electronics, Information and Bioengineering Politecnico di Milano
Department of
Electronics, Information and Bioengineering Politecnico di Milano
Via Ponzio 34/5, 20133 Milan, Italy cristiam. eeeesiinssbrolll. heoooll@@pllli ampolimiiitt
Via Ponzio 34/5, 20133 Milan, Italy matteo www.tatttetttooo..moottcooooo imi.
Abstract
The whole is greater than the sum of its parts, in 3D-text contrastive learn- ing. We introduce SCENEFORGE, a novel framework that enhances contrastive alignment between 3D point clouds and text through structured multi-object scene compositions. SCENEFORGE leverages individual 3D shapes to construct multi- object scenes with explicit spatial relations, pairing them with coherent multi-object descriptions refined by a large language model. By augmenting contrastive training with these structured, compositional samples, SCENEFORGE effectively addresses the scarcity of large-scale 3D-text datasets, significantly enriching data complexity and diversity. We systematically investigate critical design elements, such the op- timal number of objects per scene, the proportion of compositional samples in train- ing batches, and scene construction strategies. Extensive experiments demonstrate that SCENEFORGE delivers substantial performance gains across multiple tasks, including zero-shot classification on ModelNet, ScanObjNN, Objaverse-LVi and ScanNet, as well as few-shot part segmentation ShapeNetPart. SCENEFORGE compositional augmentations are model aggossic, consistently improving perfor- mance across multiple encoder architectures. Moreover, SCENEF FORGE improves 3D visual question answering on ScanQA, generalizes robustly to retrieval scenar- ios with increasing scene complexity, and showcases spatial reasoning capabilities by adapting spatial configurations align precisely with textual instructions.
1 Introduction
Large-scale contrastive learning has transformed vision-language modeling, with early breakthroughs like CLIP [1]] and ALIGN [II] demonstrating the power of aligning visual and textual representations at scale. By leveraging vast image-text datasets, these models have achieved remarkable success in zero-shot recognition, retrieval, segmentation, and transfer learning. Following these advancements in 2D, researchers have increasingly turned to 3D, where richer geometric and spatial information critical for roboties, virtual environments, and augmented reality. However, scaling contrastive learning to 3D remains challenging due to the limited availability of large-scale datasets. Recent works such Uni3D 31 and OmniBind (2T) have made significant strides by leveraging OpenShape [17] dataset, a large-scale ensemble of 3D-text data. These methods align 3D point clouds with 2DD-text representation spaces using pretrained CLIP models, achieving strong zero-shot performance on single-object classification benchmarks like ModelNet and ScanObjNN, However, despite these advances, the available 3D text data remain limited, specially compared to image-text datasets. necessitating new strategies to enhance learning.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In this work, we propose a novel approach which allows us to both virtually increase the amount of 3D-text training data and introduce harder samples, improving the contrastive representation. Inspired by image compositions typically used for augmenting classification datasets and methods in previous works, we leverage compositional learning in 3D by constructing multi-object multimodal training samples for contrastive learning. Our method is motivated by two key insights. First, unlike 2D images, where objects are inherently tied to backgrounds, lighting conditions, and perspectives, individual 3D point clouds can be freely combined into structured scenes without visual artifacts. Second, the spatial flexibility of 3D data allows explicit control over object positioning, an ability difficult to achieve in 2D. In contrast to images, 3D objects exist independently of any scene context, enabling meaningful spatial configurations and natural textual descriptions with relational cues (e.g.. "A on top of B"). We further refine these descriptions using large language model, generating diverse and nuanced combinations. Using structured synthesis, we virtually leverage a large-scale, synthetic multi-object 3D-text dataset grounded in real-world captions, significantly expanding diversity and complexity. Training any 3D encoder on these compositional scenes to align with CLIP's representation space achieves consistent improvements across tasks including zero-shot classification, retrieval, segmentation, and VQA. We empirically demonstrate the model-agnostic nature of our compositional augmentations, confirming their effectiveness across multiple encoder architectures. Our contributions are threefold: (1) proposing a novel compositional data pipeline for 3D-text contrastive learning approach synthesizing multi-object 3D scenes; (2) demonstrating consistent performance gains across tasks and backbones; and (3) analyzing the impact of key design choices, including composited object counts, object ratios, and 3D composition strategies.
2 Related Work
Among the first successful works contrastive alignment of 3D data, ULIP 23 aligned 3D features with CLIP and scaled to the Objaverse dataset in ULIP-2 (23), while Uni3D 31 further advanced this direction by scaling to billion-parameter models, leveraging 2D pretraining and using OpenShape 177 ensembled data. Other works proposed methods for improving the alignment: TAMM (30 mitigates the domain gap between rendered and natural images via adapter modules, while MixCon3D [8] sculpts holistic 3D representations by integrating multi-view rendered images and point clouds. OmniBind (2T) adpots instead a differente approach. proposing to ensemble multiple pretrained models via a learnable routing mechanism, achieving state-of-the-ae multimodal performance. Our work instead explores an orthogonal approach, scaling multimodal learning by virtually increasing dataset diversity through composition-basse augmentation. This strategy aims to improve generalization by exposing the model to a richer distribution of multimodal data.
Compositionality in Multimodal Learning. Compositionality refers to the idea of constructing new concepts by combining simpler ones, a principle that is fundamental human-like concept generalization (12). Multi-sample composition augmentations such as CutMix (26) and Mix 28 have been demonstrated be highly effective for robustness and generalization in 2D tasks such as classification, detection, and segmentation. In the 3D domain, adaptations of these augmentation strategies have been proposed and shown be effective for unimodal 3D tasks, such as point cloud classification and segmentation 3 29]. More recently, a stronger form of composition has been employed in image-text contrastive learning [. demonstrating improved multimodal alignment. Unlike CutMix and Mi, which blend two images by either mixing pixel values or pasting cut regions, thus being only a weak form of compositional learning, this approach vertically stacks centered crops of two images and combines their captions using the conjunction "and", creating stronger semantic compositions. Building on these ideas, we extend compositional augmentation to 3D data, where the structural properties of point clouds make them particularly well-suited for such techniques. Unlike images, which often contain noisy elements such as background or secondary objects, point clouds representing individual objects can be scamlessly merged into unified scene without introducing visual artifacts. Moreover, 3D compositional augmentations enable explicit modeling of spatial relationships, which can be reflected in textual descriptions, facilitating relational reasoning alongside object-level recognition.
3 Method
We introduce SCENEFORGE (SF), a framework for composing multi-object 3D scenes by combining individual point clouds with their text descriptions according to explicit spatial relations. SCENE-

---

Original Batch
SCENEFORGE Module
Combined
Single Samples
p--
cxci
Batch
inmnle) post clod
combnned
AD
sarple
carice
Agpwne-up
Kip
mudeeco Over Teems, a yellow Niplane dyynng is die
K- kkzppko
img
II
II
Combine
A pirate slip with
PP.....PP-l Px--1
A pimsic ship with black cxls.
Any 2D-3D-text Contrastive
No Combine
Scrne Forge
Scene Caption Forge
Pipeline
P0-1
A ID morid of hunara
2D
coma poiai chood 20
cptiin
A 3D model of's barona
(sl
Wus
Wire
pengsin
3D
Figure I: Our multimodal scene composition framework for contrastive 3D-text learning. Given a batch of point clouds and their captions, each sample is randomly kept as single object or combined into a synthetic 3D scene with a random number of objects. Objects designated for combination are passed to the SCENEFORGE module, which samples the additional objects and their captions. Spatial relationships among the selected objects randomly assigned, and the 3D Scene Forge and Scene Caption Forge generate the corresponding combined 3D scene and its composite caption. The newly formed scenes are merged with the unmodified objects to construct a final batch. which is then used for contrastive 3D-text alignment using a pretrained CLIP text encoder.
FORGE can be incorporated into any contrastive multimodal 3D-text learning pipeline, and we refer the resulting approaches as SF-variants. An overview of our pipeline is shown in Figure |
3.1 SCENEFORGE module
SF takes as input (point cloud, caption) sample (Po. and the number K of objects to be combined. randomly samples the required K point clouds (pi P2...... :PK-1) and their corresponding textual captions (41,42....,44 -x-1 } to generate a composite 3D scene (3 and associated scene caption clart The framework consists of two core modules: the Scene Forge, which spatially arranges the point clouds, and the Scene Caption Forge, which constructs the textual description of the generated scene. Both modules rely on a set of randomly sampled spatial relations {s1, 82. 8K-1 which define the relative placement of each object with respect to the one previously placed. These relations dictate how each object is positioned within the combined 3D scene and are used to generate caption that accurately describes the spatial arrangement. The spatial relations sampled once per scene and remain consistent across both the 3D and captioning processes.
"over," "under," and "next to". Since objects in the OpenShape dataset are not consistently oriented along the horizontal axes, using directional terms such as "left" or "right" would be ambiguous unless interpreted as absolute dis- placements along the X or Z axis. However, this would limit compositions to strictly axis-aligned translations. Instead, we employ the "next to relation, which allows flexible horizontal placement while preserving semantic coherence. In contrast, the vertical orientation of objects is consistent, making "over" and "under" well-defined, provided that appropriate constraints are imposed on rotation augmentations, as discussed later (Section 3.22). These relations serve as constraints for both the spatial placement of objects in the 3D scene and the construction of deseriptive captions that reflect the compositions.
Spatial Relations. We define three simple spatial relations:
3D Scene Forge. The 3D Scene Forge arranges objects according to their assigned spatial relations, ensuring a semantically coherent composition. The high-level procedure is outlined in Algorithm [] The module takes as input K point clouds, set of K pairwise spatial relations, and a target size
3

---

Input: Samples p. Relations Target count P Output: Composed 3D sample
festual captions
spatial relations
caption
e lo
MD
Ppree f- A (po)
(''s Ta model ppacke notch _
"Mnder ...))
for to n do
for to cdlro
Hecnneet the eaptions toting relative posizions
+ (p.) A pos P(p.. Ppress 8t)
%
Io F-Res model
next to
La pocket worch"
Ps Pi + Apos
30 + cat(c PP)
Qwen2.5 > e
Pyrev Dt
end
Figure 2: Scene Caption Forge. Starting from the initial caption (fp), each caption (t,) is connected using its relative position (si), creating a raw com- bined caption c The raw caption exstraw is then refined to the final ct using Qwen2.5.
A (subsample((c P))
return 30
Algorithm 1: 3D Scene Forge algorithm.
for the final merged cloud. The first object Po is used to initialize the scene aD, Each subsequent object p. is then placed relative to the previous object p:-1 according to its assigned spatial relation st, using function P-)) that computes the appropriate displacement. Specifically, we define P(-) based on the spatial relation between objects. For "over", we align the minimum 2-coordinate above the maximum z-coordinate of pi-1:
P(pss Pi-1: "over") = max(p:-1) - min(p:),
and reverse the roles for "under". For "next to",, we sample horizontal unit vector dl in the ry- plane and compute:
P(ss, Pi-1; "next to"))
max (x, minn(,, .))) d (XEPi-1
yεp.
where (:,-) is the inner product.
In all relations, to prevent perfect alignment and introduce slight randomness, we add a fixed offset along the shift direction (+z, -z or d), along with small Gaussian noise term E R3. After placing all K objects, we downsample the composite point cloud eDD to the target number of points P, ensuring diversity while maintaining spatial consistency. Notice that normalization and augmentation .430 is performed on each sample before adding it to the scene, as well as on the final scene.
Scene Caption Forge This module constructs textual description of the combined scene by sequentially incorporating individual object captions {to, ft....,tk-... } and their corresponding spatial relations {$1, $2,,. ...,.K.1... }. The captioning process mirrors the spatial composition of the 3D scene, starting with the first object's caption and appending each subsequent caption preceded by its respective spatial relation. However, due the method's 's simplicity, the generated caption may exhibit artifacts such as misplaced punctuation before spatial relations, incorrect capitalization following conjunctions, and disfluent sentence structures. Consequently, dxllraw To enhance readability and coherence, we refine the raw caption using a large language model (Qwen2.5 (24). ), obtaining the refined caption c/2t The model corrects grammar, punctuation, and structure while preserving the original meaning and spatial relationships. It restructures the text into a fluent, human-like description, splitting overly long sentences when necessary. Beyond improving fluency, this rewriting process also enhances caption diversity and refines OpenShape [17] captions, originally generated by BLIP 14] and Microsoft Azure Cognitive Services (2023), by leveraging the more advanced linguistic capabilities of recent language model. The full prompt used for refinement provided in the supplementary, together with an ablaton on the LLM. An overview of the proposed method is provided in Figure
refer to this raw caption,
3.2 Training Scheme
We mix single and multi-object samples in the same training batch with a predefined ratio (ablated in Section 5). This allows the model to retain strong performance on single-object tasks while benefiting from the additional compositional training signals. For each combined sample, the number of objects to combine is randomized between 2 and the maximum number of combinable objects N, which we investigate in Section 4 This allows for ΣN 1D-
Df 3k-1 possible configurations with

---

up to A samples and possible relations, with D being the dataset cardinality (extended derivation in the supplementary). With OpenShape data, this allows for 6E scenes.
Our framework operates entirely at the batch-generation level and can be plugged into any contrastive pipeline that aligns text, image, and 3D point-cloud embeddings. Because all previous methods include images, we likewise keep the 2D modality when deploying SCENEFORGE. In principle, we could rasterise every composed scene and align its image views, but real-time rendering computationally prohibitive given our budget. Instead, we mask composed samples for the 2D-3D loss terms, evaluating those terms only on pre-rendered single-object views. This preserves baseline performance on image-31 -3D tasks (see supplementary) while allowing our compositions to focus on strengthening text-3D alignment. Following established practice, the CLIP image and text encoders stay frozen; gradients flow only through the 3D encoder.
Loss Partitioning. We consider contrastive models employing the InfoNCE loss proposed in CLIP 19. For modalities m, n (tat, 2D,3D) and sample subset S, we define
1 log SS]
exp((e"","" ,e()/T)
mm (S)
(1)
iES Ejes exp ef))/r)))
where eq"", ee" are Ez-normalised embeddings and is a learnable temperature.
Let Se and S, denote the composed and single-object samples batch, with N | + [Sa). Because each sample is composed with probability a, IE|S, = (1 - a... We scale the image-3D block so that, per batch, it contributes the same total gradient budget as the text-3D block:
N
L = [ann (8, U S.) + LIae- U.S]]
+
(S.))) + Can- (2)
S.
text-3D samples)
2D-3D (singles only)
Considering the large batch sizes used in contrastive models, |S,] is tightly concentrated around its mean, we simply replace the dynamic factor N/|S,, by its expectation 1/(1 a)).
Implementation Details. We instantiate our scene compo-ccomoottionn pipeline on three approaches using different point-cloud encoders: OpenShape- PORn (17). Uni3D-G (31), and ViT-Lens-G (a frozen ViT-bigG/14 with trainable adapter lenses) [33. This variety allows us to verify the encoder- agnostic nature of our method and to select the approach which benefit the most from pipeline. All variants are trained with their public available code and our modified loss for 200 epochs with a global batch size of 1152, a = 0.5, and target point-cloud resolution of P 10 k points. This point budget was chosen as a trade-off between detail and efficiency, as detailed in the supplementary. For caption generation, we use the Qwen 2.5 7B-instruct (24 large language model. SCENEFORGE only requires an additional GPU hosting the lightweight LLM for composition. We pre-generate the first M batches producer-consune
and then switch parallel. Though composition latency is not always fully hidden, parallelization significantly reduce it. be used to further amortize overhead. Measured slowdown ranges from 0% to 50% depending on backbone, LLM quantization and replication (see supplementary). For the function AA3 we normalize point clouds the unit sphere and apply random point dropout and scaling. Rotation and translation strategies differ based on whether we process single objects, objects intended for combination in a scene, or the final composed scene. For single objects. we allow both shifts and larger rotations. However, for objects that will be combined, shifts are disabled avoid inconsistencies vertical axis and slight rotations along other axes are permitted. The latter ensures that concepts such as "over" and "under" remain semantically meaningful. For the final combined point cloud, we adopt the same rotation constraints as in the previous case to preserve spatial semantics but additionally allow translations.
setup: while batch / trains, batch 1+ M is assembled in
For faster models, multiple SCENEFORGE instances and quantization
composition, while full rotations around the
4 Experiments
4.1 Zero-Shot Classification
Datasets. We follow the standard zero-shot evaluation protocol on Objaverse LVIS 6), Model-
Net40 22 and ScanObjNN (20), where categories mapped to text prompts by formatting a set of
5

---

LVIS
ModelNet
ScanObjNN Scannet Avg Model
LVIS
MadelNei
ScanOtiJNN Scannet
Avg
Model
T1
T5
T1
15
T1
T5
T1
T1
T5
T1
T5
T1
T5
Tl
ULIP2
46.3
84.0
97.2
45.6
38.1
2
79.1
84.7
97.1
TAMM
42.0
71.7
86.3
98.1
56.7
86.1
42.4
TAMM
50.7
80.6
85.0
98.1
55.7
88.9
41.8
MixCon3D
47.5
76.2
87.3
98.1
57.7
89.8
43.0
MixCon3D
52.5
81.2
868
98.3
89.2
44.1
OmniBind-L
OmniBinnd-L
54.0
82.9
86,6
99.0
64.7
94,2
46.3
OmniRind-P
OmmiBind-F
53.6
81.8
87.1
99.0
64.7
94.4
46.1
OpenShupe
39.1
68.9
85.3
97.4
47.2
84.7
40.3
OpenShupe
46.8
77.0
84.4
98.0
52.2
88.7
39.4
SF-OpenShape
41.7
86.7
98.1
48.0
85.9
41.5
+1.50
Sp-OpenShupe
78.4
85.2
98.3
89.5
41.8
+1.43
Lens
50.1
78.1
86.8
97.8
59.8
87.7
43.8
VIT Lens
$2.0
79.9
87.6
98.4
60.1
90.3
43.7
SF-VIT-Lens
50.9
78.4
87.3
98.0
60.9
89.1
44.5
+0.78
ViT-Lns
52.8
80.7 88.0
89.9
60.9
91.2
45.1
+0.85
Uni3D
86.8
98.4
66.5
90.1
43.9
Uni3D
82.0 87.3
45.8
+1.73
+1.75
SF-Uni3D
48.9
78.4
87.5
99.0
67.3
91.5
47.6
SF-Uni3D
54.7 84,8 88.2 99.2 65.2
93.4
49,4
(a) Trained on ensemble (no LVIS).
(b) Trained on ensemble (with LVIS).
Table I: Zero-shot classification accuracy (%) "SF-" denotes models trained with SCENEFORGE. Green cells () are the best results, yellow the second best. The rightmost column reports the average Top-1 improvement (A) of the augmented model over its baseline.
templates (e.g., "a point cloud model of and the model is evaluated the classification accuracy. Additonally, adopting the pipeline from CLIP² [27]. we test our models on the Scannet [5] dataset to
evaluate their zero-shot performance on object instances from real-world scenarios.
What is the optimal value of N? Figure 3
tracks zero-shot accuracy for three contrastive
Lvis
ModelNet
learners: DpenSSappePPinttEEa
17), Uni3D
(EVA-Giant) 7 B1,, and ViT-Lens-G [13]
Top-2 Accuree
as we vary the maximum number of shapes that SCENEFORGE can merge during training. A consistent trend emerges. From = to N=3, accuracy rises monotonically: LVIS gains 1.3
85
3h-cow prape
E
85
46
+0.8- 1.3 pp, ScanObjNN +0.8 1.3 pp. Scan-
ScanObjectNN
Scannet
Net +1.4-3133 1.6 pp. and ModelNet40 about +0.8
pp. Improvements peak at N=3; Uni3D ben- advances. ViT-Lens shows a more modest gain, plausibly because its frozen CLIP back- bone trains only lightweight adapters, offering less plasticity when confronted with composed shapes. Increasing to = plateaus canoni- cal datasets and already trims accuracy on LVIS and ScanNet, whose higher intra-class variabil- ity makes them drop is universal, indicating that squeezing five objects into a fixed 10 k-point budget fragments salient geometry and introduces caption noise, hampering alignment. Interestingly, performance variations are relative minor on ModelNet40 and ScanObjNN, but the effect is far more pronounced on LVIS and ScanNet. We attribute this to the latter's more complex object distributions: model must balance greater intra-class variability with the need for discriminative features, a trade-off that becomes harder resolve in these richer, noisier datasets. Considering these results, we adopt N-3 as default SF-variasts and report the full sweeps (N=1-5) in the supplementary.
Top-1 Accuracy
efits most, while the lighter OpenShape also
S 5 18
+
Combined Objects
Combined Objects
Figure 3: Top-1 accuracy across different datasets (Lvis, ModelNet, ScanObjectNN, and Scannet) as a function of the number of combined objects.
At 5 the
additional shape combinations are introduced, the
Detailed Quantitative Comparison. benchmarks SCEnEFoRGe the considered zero- shot 3D-text classification suites. For each backbone we report its best composition size (N = 3) together with its single-shape baseline (N- 1), and list recent and sota models for reference.
Efffect of SCENEFORGE. Across all three backbones SCENEFORGE, delivers consistent top-1 gains. With the "no-LVIS" training split the average improvement is +1.50 pp for OpenShape. +0.78 pp

---

Method
1-shot
2-shox
Model
B-4 AB-4
CIDEr ACIDE AEM
OmniBind-1.
17.1
OmniBind-L
77.2
79.9
OmniBind-F + BLII2-FFanTT)
8.3
62.1
17.6
OmmiBindd-F
77.8
80.3
OpenShape BLIP2-Flun15
54.8
14.1
OpenShape
74.0
76.5
SF-OpenShape- BLIP2-FlanTS
8.1
+1.8
61.5
+6.7
16.9
+2.8
SF- Shape
76.2
79.1
ViT-Lems HLLIP--FaaTT5
57.5
15.7
ViT-Lens
75.5
77.9
+1.3
+5.9
+2.1
SF-ViT-Lens
77.0
80.1
LE-ViT-Lens BLIP2-FlanTs
8.5
63.4
17.8
Uni3D
75.9
78.2
Uni3D + BLI12-FFaan
+2.9
58.3
+8.4
+4.1
SF. Un-3D
78.5
81.2
SF Uni3D BLIP2 FlanTS
10.4
66.7
20.5
Table One-shot and two-shot part segmentation ShapeNetPart.
Table 3:
Performance on the ScanQA dataset using BLEU-4, CIDEr, and Exact Match.
for ViT-Lens and +1.73 pp for Uni3D; with the "+LVIS" split the corresponding gains are +1.43, +0.85 and +1.75 pp. Uni3D benefits most, likely a consequence of its larger capacity, which can better exploit the richer intra-sample diversity injected by multi-shape compositions, yet even the smaller OpenShape and the udapter-base ViT-Lens improve.
Comparison with prior works. All three SCENEFORGE variants surpass previous non-ensemble methods (ULIP-2 (23] TAMM 30,, MixCon3D [8] on every dataset. Moreover, SF-Uni3D
outperforms OmniBind (21). the strongest published ensemble, despite using single model: on the LVIS-ModeNNet-Scann ObNNN-ScanNNee quartet achieves absolute top-1 margins of -0.7, +1.6, +0.5 and +-3.1 pp, respectively. These results underscore that structured multi-object augmentation offers a more inference-efficien strategy for enhancing representations than costly ensemble methods.
Why do multi-object compositions enhance single-object classification? While it might initially appear counterintuitive, training with multi-object compositions fosters improved representations that benefit single-object recognition tasks. This phenomenon aligns with established findings in representation learning literature, particularly image classification, where tations (e.g., CutMix, MixUp) are known to induce smoother decision boundaries and promote robust generalization by exposing models more diverse feature combinations. Analogously, in our structured 3D scene compositions, the increased complexity and relational context implicitly regularize the learned representations, facilitating the emergence of discriminative features resilient to variations in single-object scenarios encountered at inference. By analyzing the positive-negative similarity margins in zero-shot classification, we find that SF-variants yield larger margins over baselines, indicating stronger inter-class separation and more robust decision boundaries. We report the quantitative analysis the supplementary material.
augmen-
4.2 Few-Shot Part Segmentation on PartNet
We follow the protocol of Uni3D) 31 the ShapeNetPart dataset 25 evaluating each backbone in one-shot and two-shot regimes. As in PointNet++ [18], we freeze the pretrained transformer encoder and attach lightweight proppaatiooonttiionn heads that upsample intermediate representations to dense part predictions. Only these heads are fine-tuned on the few labeled part annotations.
Table reports mean IoU (mloU) and the improvement A over the corresponding single-shape baseline for all three backbones (OpenShape, ViT-Lens, Uni3D) with and without SCENEFORGE SCENEFoRGE yields consistent gains: SF-OpenShape improves by +2.2/ +2.6 pp in one-/two-shot, SF-ViT-LLens by +1.5/ +2.2 pp, and SF-Uni3D by +2.6/ +3.0 pp. with SF-Uni3D achieving the highest absolute mIoU (78.5 81.2). These improvements suggest that multi-shape pretraining encourages more fine-grained, group-based features that transfer effectively to part segmentation, structuring the feature space better capture part-level relationships even under extreme label scarcity.
4.3
3D Question Answering on ScanQA
which requires answering natural-language questions about ScanNet scenes. Following prior works, we freeze each 3D) encoder and attach it to BLIP2-FlanTT [5), then fine-tune on ScanQA's question question-answer answer pairs.
We further assess 3D-text alignment on the ScanQA benchmark 2 9
7

---

In addition to the sota contrastive baselines (OmniBind-Laanl Large, OmniBind-Fulle we include SF- OpenShape. SF-V -iT-Lene and SF-Uni3D as our multi-shape variants, alongside their single-shape counterparts. We report here BLEU-4 (B-4), CIDEr and Exact Match (EM); full metrics appear in the supplementary. Table g gives the results. All three backbones see substantial gains with SCENEFORGE: SF-OpenShape improves B-4 by +1.8 pp. CIDEr by +6.7 pp and EM by +2.8 pp; SF-ViT-Lens adds +1.3 pp. +5.9 pp and +2.1 pp. respectively; and SF-Uni3D leads with +2.9 pp. +8.4 pp and +4.1 pp. While all backbones gain, Uni3D yields the largest relative and absolute boosts, consistent with its greater capacity, whereas ViT-Lens, despite smaller gains, still surpasses OmniBind. qualitative review shows that, while baseline encoders match variants attribute- color- based questions, SCENEFORGE variants significantly outperform them on spatial reasoning queries, e.g. "What is over the brown chair?", where modeling inter-object relationships is essential. This suggests that our structured multi-shape augmentation not only sharpens local feature representations but also boosts the encoder's ability to infer complex spatial configurations, critical capability for 3D scene understanding.
4.4 Supervised Fine-Tuning
To test if the benefits of SCENEFORGE pre-training extend to supervised settings, we evaluate full fine-tuning and Parameter-Efficient Tuninnuning (PEFT) methods [TO 16 32 We perform
experiments on three of the considered classification benchmarks: the synthetic Medel the
challenging real-world ScanObjectNN, and ScanNet Instances extracted from complex indoor scenes.
Table 4: Supervised fine-tuning accuracy (%).
Model
Method
Trainable Params
ModelNet40
SeanObjeetNN
ScanNet Inst.
Full Fine Tuninn
1016.5M (100%)
94.28
97.12
82,72
Adapter
(0.74%)
81.42
Uni3D
DAPT
7.3M (0.72%
94.33
96.78
82.65
PoinGGT
4.1M (0.40%)
94.83
97.68
83.04
Full Fine-Tuning
1016.5M (100%)
94.42
97.58
83,58
Adapter
(0.74%)
82.56
SF-Uni3D
DAPT
7.3M (0.72%)
94.49
97.15
83,46
PoinGGTT
4.1M (0.40%
94.95
98.09
84.29
As shown in Table our SF-Uni3D backbone provides a superior initialization, consistently outper- forming the baseline, especially on the complex ScanNet Instances. A strong synergy with PEFT also evident: the PointGST method surpasses even full fine-tuning while using just 0.4% of the trainable parameters.
4.5 N-Objects cross-modal Retrieval
Following prior work 23
31
we evaluate
cross-modal retrieval on the unseen Objaverse- LVIS dataset, measuring top-k accuracy for both 3D-to-text and text-to-3D retrieval. Models are trained on the full ensemble excluding this set, and performance is assessed via cosine simi-
70
Too-1 Accuracc (90)
larity retrieval across embedded samples. Be-
30
yond standard single-object retrieval, we aim
to analyze how well our models understand in- creasingly multi-object scenes. To this end, we introduce the N-LVIS benchmark, where each shape is composed with N additional ob- jects. For clarity, we report results only for the best SF backbone (Uni3D with N 1..5) and compare against OmniBind- (1I;; analogous results for other backbones provided in the supplementary.
(Ne4)
Our (4.5)
20
Number Cambined Shapes
Figure 4: Top-1 averaged retrieval accuracy on the N-LVIS datasets as N increases.
We evaluate retrieval on N-LVIS from N (standard retrieval) to N=10, reporting averaged top-1 accuracy (text to 3D and viceversa) in Figure 4 Prior models degrade sharply when faced with multi-object compositions, dropping below 50% at 2 In contrast, models, trained with

---

varying numbers of composed objects, exhibit strong generalization, each peaking near its training composition size. Notably, the N=3 model sustains over 70% accuracy N=6 and around 60% N=7, highlighting its robustness in complex scenes. This comes at a slight cost: models trained with higher N perform marginally worse lower N, reinforeing our findings in Section [4. that simpler compositions better preserve single-object understanding. Full top-1 and top-5 results are provided in the supplementary.
4.6 Object Repositioning
Target relaton
To qualitatively assess our approach ability to improve reasoning about spatial relationships, we introduce simple object repositioning task. Given two objects, we combine them in an ini- tial configuration using our MSF. The combined caption then modified describe new spatial relationship. and we optimize duree-parameter offset vector to reposition the second object, maximizing alignment with the updated descrip- tion while keeping the 3D-text encoder frozen. Full optimisation details appear in the supple- mentary. As illustrated in Figure [ encoders
"Nox ko
"Ovel"
Figure 5: Object repositioning example.
trained with SCENEFORGE reliably relocate the object to the target position, while bascline encoders stall in semantically plausible yet misaligned configurations. We observe the same qualitative be- haviour with all other variants, confirming that SCENEFORGE strengthens spatial and compositional reasoning.
5
Ablation Studies
5.1 Proportion of Multi-Object Samples
Actuncy mh
The hyperparameter a determines the fraction of multi- object samples in each training batch. We analyze its im- pact by varying from 0 to I and evaluating the average zero-shot top-1 accuracy on the zero-shot datasets (Fig- op. Increasing a initially improves performance with respect to the baselines, with accuracy peaking around 0.5. However, for larger a, performance declines, likely due to an excessive focus on compositional rela- tionships the expense of single-object understanding.
Ooerthope
19243
300
02
02
63
34
0.6
Figure 6: Effect of varying a on average zero-shot top-1 accuracy.
5.2 Do other composition functions work?
To assess the impact of our 3D scene composition method, we conduct experiments by fixing the training method ours, the backbone to the strongest we found (Uni3D) and just ablate the composition method against object composition approaches: Point- CutMix (29] and PointMixUp [3].
Composition
Lvis Top-1
ModelNet Top-1
ScanObjNN
Scannet
Method
Top-1
Top-1
None (Uni3D)
53.5
87.3
63.9
45.8
PoinCCutMix.R
53.5
87.1
64.7
47.5
PointCunMix-K
83.0
PointMixup
39.2
78.7
41.4
30,2
SF-Uni3D (N=2)
Table 5: Different 3D composition methods on zero-shot cls.
PointCutMix replaces regions of one
point cloud with those from another and has two variants: PointCutMix-P R, which randomly replaces points, and PointCutMix-KK KK which preserves key structures. PointMixUp interpolates point coor- dinates and features between objects. For fair comparison, since these methods combine two objects, we compare their results specifically to our N configuration. Textual descriptions for PointCutMix and PointMixUp compositions are generated by concatenating object captions with "and" and subsequently refined using Qwen [24) following our pipeline.

---

Results in Table 5 show our method consistently outperforming all baselines. Although PointCut- Mix-R exceeds Uni3D on most benchmarks, it still lags behind our method, even more considering the N = 3 variant; PointCutMix-K and PointMixUp perform worse than Uni3D. This gap stems from how each approach handles object semantics and spatial coherence: our method builds structured scenes that align naturally with captions, whereas PointCutMix-- randomly mixes whole objects, creating overlaps, and PointCutMix-K and PointMixUp fragment or interpolate shapes, produc- ing unrealistic, poorly described scenes. Overall, maintaining clear object semantics and spatial relationships through structured composition yields superior generalization.
show
5.3
Are simple relations enough?
To test if the learned spatial understanding generalizes beyond the simple pre-training relations, we isolate performance on ScanQA questions involving more complex, unseen spatial queries. Table 6 shows that our compositionally-tree
models consistently outperform their baselines across all backbones and a variety of complex relations.
Table Generalization to unseen spatial relations on ScanQA for all backbones.
OpenShape
ViT-LLens
Uni3D
Relation Type
Metric
Baseline
SF
A
Baseline
SF
Baseline
SF
CIDEr
54.5
61.5
+7.0
57.1
63.3
57.9
66.6
+8.7
Attached To (21)
EM
14.1
17.1
+3.0
15.6
17.9
+2.3
16.5
20.8
+4.3
CIDEr
56.8
63.4
+6.6
59.0
65.1
+6.1
61.0
70.1
+9.1
Sitting On (59)
EM
15.2
17.7
+2.5
16.6
18.4
17.5
22.6
+5.1
CIDEr
54.1
61.2
+7.1
56.8
62.9
+6.1
57.2
66.5
+9.3
Between (112)
EM
14.0
17.0
+3,0
15.5
17.9
+2.4
15.8
20.5
+4.7
CIDEr
55.0
61.8
+6.8
57.5
63.5
+6.0
58.5
67.0
+8.5
Closest (112)
EM
14.3
17.2
+2.9
15.8
18.0
+2.2
16.2
20.4
+4,2
CIDEr
56.1
62.5
+6.4
58.2
64.0
+5.8
60.3
68.3
+8.0
Front Of (246)
EM
14.9
17.6
+2.7
16.3
18.2
+1.9
17.1
21.8
+4.7
The comprehensive results in Table 6 show that SCENEFORGE provides consistent benefits across all three backbones. Each SF-variant significantly outperforms its respective baseline on all complex relation types. This indicates that our pre-training builds a robust spatial foundation that generalizes effectively to more nuanced relational queries, regardless of the underlying encoder architecture.
Limitations and Future Directions
While SCENEFoRGE consistently enhances 3D-text alignment across multiple backbones and tasks, we are aware of its limitations. First, our synthetic scene generation employs only three basic spatial relations, which, although diversified through LLM-based refinement, do not fully capture the complexity of natural environments. Future research could focus on more realistic and var- ied364compositions guided by learned object co-occurrence patterns and spatial priors. Second, due to computational constraints, we maintain fixed 10 k-point budget for multi-object compositions, resulting in accuracy degradation for densely populated scenes (see Figure 3). Addressing this will require exploring larger point budgets or employing more sophisticated sampling techniques to preserve salient geometric features in complex scenarios. Third, although we leverage the lightweight 5wen2.5 model for refinement, the overhead is not always negligible and reducing it introduces memory-time tradcoff. Finally, due to rendering costs, we could not pair each composition with synthetic image, though extending the pipeline to incorporate rendered views for joint 2D-3D learning or studying alternative approaches, such as aligning 3D compositions to aggregations of the single image embeddings. remains promising direction. Overall, addressing these limitations will significantly broaden the practical impact and robustness compositional 3D-text learning methods.
Acknowledgments. This paper is supported by the PNRR-PE-AI FAIR project funded by the NextGeneration HU program. We acknowledge ISCRA for awarding this project access to the LEONARDO supercomputer, owned by the EuroHPC Joint Undertaking, hosted by CINECA (Italy).
10

---

References
[1] Maxwell Aladago, Lorenzo Torresani, and Soroush Vosoughi. Semantic compositions enhance vision-language contrastive learning. arXiv preprint arXiv:2407.01406 2024.
[2] Daichi Azuma, Taiki Miyanishi, Shuhei Kurita, and Motoaki Kawanabe. Scanqa: 3d question answering for spatial scene understanding. In Proceedings of the IEFF/CV Conference on Com- puter Vision and Pattern Recognition (CVPR), 2022. URL / /gi thggb..hbb... com ATR-DBI/ ScanQA This dataset is released under the Creative Commons AtrriutionnNonnCommerraal ShareAlike 3.0 Unported License.
[3] Yunlu Chen, Vincent Tao Hu, Efstratios Gavves, Thomas Mensink, Pascal Mettes, Pengwan Yang, and Cees GM Snock. Pointmixup: Augmentation for point clouds. In Computer Vision- ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16, pages 330-345. Springer, 2020.
[4] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, al. Scaling instruction-fieeuueed language models. Journal of Machine Learning Research, 25(70):1-53, 2024.
[5] Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias NieBner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. Proceedings of the IEEE conference computer vision and pattern recognition, pages 5828-5839, 2017. URL http://www scan-net org/ The ScanNet data is released under the ScanNet Terms of Use, and the code is released under the MIT license.
[6] Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: uni- verse of annotated 3d objects. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 13142-13153, 2023. URL hattps://huggingg co/datasets/ allemai/objjeaoeeeaaaaaaaopa jaersse The of the dataset whole is licensed under the ODC-By v1.0 license. Individual objects in Objaverse are all licensed as creative commons distributable objects, and may be under the following licenses: CC-BY 4.0 721K objects CC-B 4.0- 25K objects CC-BY-NC-SA 4.0 52K objects CC-BY-SA 4.0 16K objects CCO 1.0 3.5K objects.
[7] Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang. Tiejun Huang, Xinlong Wang, and Yue Cao. Eva: Exploring the limits of masked visual representation learning at scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19358-19369, 2023.
[8] Yipeng Gao, Zeyu Wang. Wei-Shi Zheng, Cihang Xie, and Yuyin Zhou. Sculpting holistic 3d representation contrastive language-imager pre-training. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 22998-23008, June 2024.
[9] Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du, Zhenfang Chen, and Chuang Gan. 3d-lim: Injecting the 3d world into large language models. Advances in Neural Information Processing Systems, 20004482-24444,, 2023.
[10] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pages 2790-2799. PMLR, 2019.
[11] Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun- Yun-Hsuan Sung. Zhen Li, and Tom Duerig. Scaling up visual and languaee representation learning with noisy text supervision. In International conference on machine learning, pages 4904-4916. PMLR, 2021.
[12] Brenden M. Lake, Ruslan Salakhutdinov, and Joshua B. Tenenbaum. Human-level concept learning through probabilistic program induction. Science, 33066266) 1338, 2015. doi: 10.1 10.1126/sieecceaab30555 https:::::// //wttps://www.t. science. wwtp:e/////////oooo/aae//aoaaa/ 1126/ science
aab3050

---

[13] Weixian Lei, Yixiao Ge, Kun Yi, Jianfeng Zhang, Difei Gao, Dylan Sun, Yuying Ge, Ying Shan, and Mike Zheng Shou. Vit-lens: Towards omni-modal representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26647-26657, 2024. URLL htt.t::ttt:/giiieeee http://gihhuu..ooeeeeeeetttee www.///eeeeAR////AAC Lin This work is licensed under the Apache 2.0 License.
[14] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language- image pre-training for unified lisounlanguaae understanding and generation. In International conference on machine learning, pages 12888-12900. PMLR, 2022.
[15] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730-19742. PMLR, 2023.
[16] Dingkang Liang, Tianrui Feng, Xin Zhou, Yumeng Zhang, Zhikang Zou, and Xiang Bai. Parameter-efficiet fine-tuning spectral domain for point cloud learning. IEEE Transactions on Patlern Analysis and Machine Intelligence, 2025.
[17] Minghua Liu, Ruoxi Shi, Kaiming Kuang, Yinhao Zhu, Xuanlin Li, Shizhong Han, Hong Cai, Fatih Porikli, and Hao Su. Openshape: Scaling up 3d shape representation towards open- world understanding. Advances in neural information processing systems, 36, 2024. URL https://8g h/tp:////.httt.g.// cow Swtom//ooiinSTT/enaaaaaee
This work is licensed under the Apache
2.0 License.
[18) Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017.
[19] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pages 9748-8763. PMLR, 2021.
[20] Mikaela Angelina Uy. Quang-Hieu Pham, Binh-Son Hua, Duc Thanh Nguyen, and Sai-Kit Yeung. Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data. In International Conference Computer Vision (ICCV), 2019. URL https://hkust- - vgd. github. gitthu..io/scaoo//.. http////ccnnjennnn//// This dataset is released under the MIT License.
[21] Zehan Wang, Ziang Zhang, Hang Zhang, Luping Liu, Rongjie Huang, Xize Cheng, Hengshuang Zhao, and Zhou Zhao. Omnibind: Large-scale omni multimodal representation via binding spaces, 2024. httpsttps:/// httpxxxxiiv..//////////////..a wwapiiii.rg//e. org/ aa////bb///2/00/ 11895.
[22] Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1912-1920, 2015. URL https: https://modelnet.c
princetonn edu/ Academic research, see website for more details.
[23] Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martín, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Leariing unified representation of language, images, and point clouds for 3d understanding. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1 1179-1189 2023.
[24] An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15715 2024. URL httpp://huuginge leaggingfaaeeccooeeeeee www/weo///eeeeeeeeeeoe 5 5-VL-325 32B Instinnt-tautt AWQ Qwen2.5 is licensed under the Apache 2.0 license.
[25] Li Yi, Vladimir G. Kim, Duygu Ceylan, I-Chao Shen, Mengyan Yan, Hao Su, Cewu La, Qixing Huang, Alla Sheffer, and Leonidas Guibas. A scalable active framework for region annotation in 3d shape collections. SIGGRAPH Asia, 2016.
12

---

[26] Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6023-6032, 2019.
[27] Yihan Zeng, Chenhan Jiang, Jiageng Mao, Jianhua Han, Chaoqiang Ye, Qingqiu Huang, Dit- Yan Yeung. Zhen Yang. Xiaodan Liang, and Hang Xu. Clip2: Contrastive caattuggg-ina--peae pretraining from real-world point cloud data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15244-15253 2023.
[28] Hongyi Zhang. mixup: Beyond empirical risk minimization. arXiv preprini 17Xiv:1710.09412. 2017.
[29] Jinlai Zhang, Lyujie Chen, Bo Ouyang, Binbin Liu, Jihong Zhu, Yujin Chen, Yanmei Meng, and Danfeng Wu. Pointcutmix: Regularization strategy for point cloud classification. Neurocomput- ing, 505:58 67, 2022. ISSN 2312. doi: htt.://ww......eee..t..ooooo///a URL set.e//cee....iiecniieee........ http://eee...occccccceeeeeeeeeee htt.://wwweeeeeeee.e...///e/enea
[30] Zhihao Zhang, Shengcao Cao, and Yu-Xiong Wang. Tamm: Triadapter multi-modal learning for 3d shape understanding. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 21 1411 11111-21422 21423, June 2024.
[31] Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang. and Xinlong Wang. Uni3d: Exploring unified 3d representation at scale. In International Conference on Learning Representations (ICLR), 2024. URL http:/ttts///gtt http:/ggtttttt..../ commbbaavvisioonnnii3 This work licensed under the MIT License.
[32] Xin Zhou, Dingkang Liang Wei Xu, Xingkui Zhu, Yihan Xu, Zhikang Zou, and Xiang Bai. Dynamic adapter meets prompt tuning: Parameter-efficien transfer learning for point cloud analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14707-14717, 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims in the abstract and introduction, that training on structured multi- object scenes synthesized from single 3D shapes and captions both boosts zero-shot single- object classification and extends to denser tasks like segmentation and VQA, and that this compositional augmentation model-agnostic, are fully supported by our experiments, which show consistent performance gains across zero-shot classification, part segmentation, and VQA benchmarks for multiple backbone architectures.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss limitation in Section
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
14

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
We do not present theroretical proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We disclose every hyperparameter used for our approach in section Section 3.2 and in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
15

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We release the code for our pipeline, which can be then directly inserted in any 2D-3D-text contrastive model with tiny modifications the loss and data loader, which are extremely detailed in the paper (Section 3.2 and Section 3.2
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ //wpps.ccppuuiie.e wwtpcccppppccccceecoec
www.eeeeeeoeebmmmeeeeeeeeeee cobbbmissiini aessonoooicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: We adopt the standard classification benchmarks publicly released from OpenShape and publicly available standard splits from Scannet and ScanQA datasets.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Because each contrastive-lcamiin run on our large backbones requires multiple GPU-days, performing the dozens of independent trials needed for well-defined error bars is computationally infeasible. Instead, we demonstrate consistent improvements across three different architectures and multiple benchmarks support the robustness of our findings.
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We report number and type of GPUs, and training hours in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httpp::/neur/: https///////lcttttccccco....l...
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
17

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considcrations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper doess not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
19

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification:
We describe the LLM usage the supplementary material.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /ttp::://neurps.... http:///rrrrreec/oooeeerrrs///// cc/Confferecces//2255//L. for what should or should be described.
20


Reviews:
[{"review_id": "HcRX6MyDPz", "reviewer": "Reviewer_1m9F", "summary": "This paper proposes SCENEFORGE, a compositional data augmentation pipeline for 3D vision-language pretraining. The method dynamically synthesizes multi-object 3D scenes and generates corresponding structured text descriptions via large language models, aiming to enhance 3D-text alignment and downstream performance in tasks such as part segmentation and scene classification. Extensive experiments demonstrate consistent improvements across multiple benchmarks and backbones.", "questions": "1. For the ScanNet experiments, is instance classification performed on extracted object instances or entire scenes? Please detail the data processing and evaluation protocol.\n2. Which variant of ScanObjectNN is used in the experiments (e.g., PB_T50_RS)? Please specify, as this affects comparability with prior work.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths：\n1. The method is general and can be integrated with existing 3D encoders.\n2. Experimental results are comprehensive, covering several datasets and tasks, with clear improvements over baselines.\n\nWeaknesses：\n1. The paper lacks quantitative analysis of the latency and resource overhead introduced by online data synthesis, which is important for practical deployment.\n2. The mechanism by which compositional scene structure benefits part segmentation and classification tasks is not sufficiently explained.", "was_revised": true, "final_justification": "I suggest raising the score to 5, pending the authors' inclusion of the discussion in the final version.", "rebuttal": "We thank the reviewer for their insightful feedback and constructive suggestions. We are encouraged that the reviewer found our method to be general, our experimental results comprehensive, and the improvements clear. We will address the weaknesses and questions raised point-by-point.\n\n### W1: Latency and Resource Overhead\nWe provide a detailed analysis of latency and resource overhead in the supplementary material (Sec. 5), for which we briefly report the results in the main document (Sec. 3.2). We will reference the supplementary analysis more prominently in the main paper upon acceptance. To summarize the key findings for the reviewer's convenience:\n\nFirst, our method introduces no practical deployment latency: at inference time, the computational cost is identical to the baseline model, as the model architecture and forward pass are unchanged.\n\nThe additional computational overhead from our method is confined entirely to the training stage as a one-time, offline cost. The point cloud combination cost is negligible, as it is composed just by fast operations as uniform sampling of samples and positional relations, and then 3D transformations and array concatenations. Instead, the LLM-based caption refinement does introduce an overhead, which however we show can be effectively eliminated (Supplementary Sec. 5). For instance, by using quantized LLM replicas on a single GPU, the training overhead is removed even for the fastest backbones. Therefore, our method delivers significant performance gains without compromising the efficiency required for practical deployment. \n\n### W2: Benefits for part segmentation and classification\nWe thank the reviewer for highlighting this aspect, as it allows us to further clarify the mechanisms behind the performance improvements. We will address the benefits for part segmentation and single-object classification separately.\n\n- On the Benefit for Part Segmentation. \n\n    Our hypothesis is that compositional training encourages the model to learn more fine-grained, part-aware features, which serve as a superior starting point for the few-shot part segmentation task. To correctly interpret composite scenes which change at every sampling (both in subjects and relation), the model is discouraged from simply learning a holistic representation. It it pushed instead to first learn to distinguish the constituent objects (\"hat,\" \"person\") before it can reason about their spatial relationship (\"over\"). This process implicitly forces the 3D encoder to develop more localized and semantically meaningful features that can isolate object parts. Because the downstream part-segmentation task is performed in a few-shot setting, the quality of the pretrained encoder is critical. By pretraining on our compositional scenes, the encoder is already primed to understand part-level relationships, providing a much stronger feature representation for the lightweight segmentation heads to fine-tune from. The consistent gains shown in Table 2 of our main paper support this conclusion.\n\n- On the Benefit for Single-Object Classification. \n\n    Our approach is directly inspired by a well-established principle from representation learning literature: multi-sample augmentations act as powerful regularizers. In 2D vision, techniques like MixUp and CutMix are proven to enhance model robustness and generalization. They do so by creating diverse, challenging training samples that encourage the learning of smoother decision boundaries. Just as MixUp and CutMix pair their augmented images with corresponding soft labels, our method pairs its composite 3D scenes with rich, LLM-refined textual scene descriptions. This descriptive caption effectively functions as a semantic soft label. Instead of merely interpolating class probabilities, it weaves together multiple distinct concepts and their spatial relations into a single, refined target description. This parallel is particularly potent in our zero-shot context, where text prompts are the labels at inference time. By training the encoder to align with these complex \"semantic soft labels,\" we directly encourage a more organized and discriminative feature space. Ultimately, this structured composition of both the 3D input and the textual \"label\" forces the model to understand each object's features within a complex relational context, yielding a more robust representation that generalizes effectively to the simpler task of single-object classification.\n\n    To provide direct quantitative evidence for this, we did analyze the cosine margin, a standard metric for evaluating the quality of a learned representation space. The margin is defined as the difference between the cosine similarity of a sample to its correct (positive) class and the similarity to its most confusing incorrect (hardest negative) class. A larger margin indicates better class separation and a more robust model.\n\n    The results below show that our SCENEFORGE (SF) method consistently and significantly improves this margin across all three backbones and all four benchmark datasets.\n\n    | Dataset | Model | Baseline Margin | SF Margin | **Δ Margin** |\n    | :--- | :--- | :---: | :---: | :---: |\n    | **ModelNet** | Uni3D | 0.070 | 0.094 | **+0.024** |\n    | | ViT-Lens | 0.060 | 0.075 | **+0.015** |\n    | | OpenShape | 0.045 | 0.065 | **+0.020** |\n    | --- | --- | --- | --- | --- |\n    | **LVIS** | Uni3D | -0.042 | 0.003 | **+0.045** |\n    | | ViT-Lens | -0.045 | -0.015 | **+0.030** |\n    | | OpenShape | -0.070 | -0.030 | **+0.040** |\n    | --- | --- | --- | --- | --- |\n    | **ScanObjNN** | Uni3D | 0.023 | 0.046 | **+0.023** |\n    | | ViT-Lens | 0.018 | 0.030 | **+0.012** |\n    | | OpenShape | 0.010 | 0.028 | **+0.018** |\n    | --- | --- | --- | --- | --- |\n    | **ScanNet** | Uni3D | -0.061 | -0.023 | **+0.038** |\n    | | ViT-Lens | -0.050 | -0.032 | **+0.018** |\n    | | OpenShape | -0.085 | -0.055 | **+0.030** |\n    \n    Crucially, on the most challenging datasets (LVIS and ScanNet), where baselines often exhibit a negative margin, meaning they are more confident in an incorrect class than the correct one, our method provides the largest improvements. For example, SF-Uni3D pushes the LVIS margin from negative to positive. This demonstrates that our compositional training directly enhances the model's ability to discriminate between fine-grained or noisy objects, confirming that it yields a more robust and better-organized representation space.\n\n\n    We thank the reviewer for the insightful question: upon acceptance, we will add these results, as it was our intention, to the supplementary material. We will also improve the explaination of the mechanisms behind the benefits in the already dedicated main document section.\n\n## Questions\n- ScanNet: as mentioned in the paper, we follow previous works (Uni3D, CLIP^2) for the Scannet Evaluation. Instance classification is performed on extracted object instances, not entire scenes. Objects are extracted with the instance masks provided by the dataset, and the ones with fewer than 5 points are removed. The task is zero-shot recognition on the common 17-class split (VoteNet, H3dnet). Classification logits are computed by comparing an instance's 3D features to text prompt features (e.g., \"point cloud of {CLASS}\").\n- ScanObjNN: As per Scannet, we adopt the benchmark from Uni3D for direct comparison, using the OBJ_ONLY variant without colors.\n\nWe trust that our detailed overhead analysis, new quantitative evidence on representation quality via cosine margins, and clarifications on our experimental protocols directly address the reviewer's initial concerns and questions. We believe these additions substantially strengthen our contribution and we would be grateful if the reviewer would consider them in their final assessment of the paper. We would like to thank the reviewer again for the opportunity to elaborate on these aspects."}, {"review_id": "4dxSmSkeYj", "reviewer": "Reviewer_3sZR", "summary": "This work augments 3D datasets used for spatial tasks by composing scenes with 3D assets. They show that doing this augmentation on data helps improve model performance on a range of different tasks.", "questions": "How are the relations + objects sampled? Uniformly? Do they have to semantically plausible, or does the plausibility of the final scene not matter? I would imagine for downstream applications that require more semantic-level reasoning, that training on semantically implausible scenes would be a problem. Do you observe that in your experiments?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "+ Strength\n\n  The method is shown to consistently improve performances including zero-shot classification,  retrieval, segmentation and VQA. Therefore, the demonstrable impact of this work is quite large.\n\n  Additionally, because the method is a way of augmenting 3D data, it is also model agnostic.\n\n  The  analysis of design choices is thorough.\n\n+ Weaknesses\n\n  The scheme described in \"3D Scene Forge\" ( Section 3.1 ) leads me to think that the kind of final placements that can be synthesized would be quite simplistic, and likely would not take into account the fine-grained geometry of either object (e.g. surfaces on shelves)\n\n\n  Also, how were the rendering views chosen?", "was_revised": true, "final_justification": "The issues raised in my review was adequately addressed by the rebuttal, and as such I maintain  the positive rating.", "rebuttal": "We are deeply grateful to the reviewer for their positive and encouraging feedback, their high rating of our work, and their insightful questions. We are glad they found our method's impact to be large, model-agnostic, and our analysis thorough. We will address the points raised below.\n\nWe agree with the reviewer that our current placement strategy is a simplified abstraction of real-world interactions and does not explicitly model fine-grained geometry like placing an object on a shelf. This was a deliberate design choice aimed at creating a robust, scalable, and general-purpose augmentation method. Implementing more complex relations like \"laying on top of\" or \"inside\" would require detailed, object-specific knowledge. For example, \"laying on\" would necessitate identifying suitable planar surfaces on the target object, while \"inside\" would require semantic knowledge about which objects can be containers.\n\nEnforcing such constraints would have severely limited our compositions or training data itself to smaller, highly-curated datasets where this information is available. This would forfeit the primary advantage of leveraging massive, diverse, and less-curated datasets like Objaverse, which is critical for learning generalizable representations.\n\nTherefore, we opted for a small set of universal and object-agnostic spatial relations. These simple relations can be applied to any pair of objects without prior knowledge, ensuring maximum scalability. Despite their simplicity, they are foundational for spatial understanding and, as our results demonstrate, provide a powerful training signal. Our strong performance on the ScanQA VQA benchmark and the object repositioning task  confirm that the model effectively learns meaningful spatial reasoning from these cues. We see this as a foundational step. As mentioned in our Limitations section (Sec. 6), incorporating more sophisticated, semantically-aware priors is a promising future direction that can build upon the strong, general-purpose base established by our current approach.\n\n\nWe thank again the reviewer: in the camera ready version, we will make more clear the just explained motivations behind our dseign choices for the considered positional relations.\n## Questions\n- How are relations + objects sampled? As highlighted above, thanks to the universal applicability of our relations, we can sample both the additional objects and the spatial relations uniformly at random from the training set and our predefined relation set, respectively. In each scene, objects are sampled without replacement to avoid ambiguities, while relations are sampled with replacement to allow a greater variability.\n\n\n- About plausibility: the reviewer rightly notes that this random sampling can lead to semantically implausible scenes. We believe indeed that even implausible scenes can be helpful. By exposing the model to a vast and diverse set of combinations, both plausible and implausible, we push it to learn disentangled and robust features for individual objects. To understand an unlikely scene, the model must first develop a strong, independent understanding of each component. This process is analogous to aspects of human learning. We can easily imagine fantastical or absurd scenes, and this ability to compose known concepts in novel ways strengthens, rather than weakens, our understanding of the concepts themselves. Moreover, filtering unplausible scenes would require additional knowledge or limiting the available data, which falls into the same data quality-scale tradeoff described above.\n\n    Furthermore, it is crucial to distinguish between the types of \"implausibility\" our method generates versus the ones we discussed above with complex relations:\n\n    - Our method creates scenes that may be semantically implausible but remain geometrically coherent. For example, a \"a planet next to a dinosaur\" is unlikely but physically possible to arrange. The spatial relationship is simple and universally applicable.\n\n    - We instead avoid creating scenes that are geometrically nonsensical, such as \" a planet inside a dinosaur.\" This would require complex, object-specific modifications (like cutting a hole) that would break the geometric integrity, and possibly semantic properties, of the shapes and create a confusing learning signal to align with the caption.\n\n- Did we notice problems with implausible scenes? Our experiments indicate that our training strategy is beneficial: when the encoder is used as a feature extractor for downstream tasks requiring strong semantic reasoning, such as ScanQA evaluation, our method consistently outperforms the baseline. We did perform a qualitative review of failure cases in the VQA task, but we did not find any significant pattern suggesting that training on implausible scenes led to specific types of errors in real data.\n\n- For Image renderings, we stick with the approach from the baseline model Uni3D for fairness, rendering 10 color images from different views that uniformly cover the whole shape, then we uniformly sample one of 10 each time during training steps.\n\nThank you once again for your valuable feedback and strong support. We hope these clarifications have fully addressed your questions."}, {"review_id": "DfioavTrJa", "reviewer": "Reviewer_Kxh9", "summary": "This paper proposed SceneForge, which leverages individual 3D shapes to construct multiobject scenes with explicit spatial relations, and train a 3D-text contrastive learning model to pairing them with coherent multi-object descriptions refined by a large language model. This approach boosts performance across several benchmarks.", "questions": "1. Is it possible to include more spatial relationships and construct a more complex scene for training?\n2. Refer to point 2 of Weaknesses.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. The ideas are simple but effective. This method delivers consistent gains across multiple 3D-text alignment benchmarks.\n\nWeaknesses:\n1. The set of predefined spatial relations is too limited, so it fails to capture more complex or subtle spatial arrangements.\n2. Using a fixed 10k point sampling budget causes performance to drop sharply when four or more objects appear. It would help to see results with more points such as 10k, 20k, and 40k points to understand how sampling affects accuracy.", "was_revised": true, "final_justification": "The authors' responses addressed my concerns well. Based on the general novelty and contributions, I keep the original rating of borderline accept.", "rebuttal": "We sincerely thank the reviewer for their valuable feedback and positive assessment, especially for recognizing that our ideas are simple but effective and deliver consistent gains. We appreciate the opportunity to clarify the points raised regarding the spatial relations and point budget.\n\nWe will now address reviewer highlighted weaknesses and questions, grouped by topic.\n\n### W1 and Q1:\n\nThe reviewer rightly points out that our set of spatial relations is simple and doesn't capture the full complexity of real-world arrangements. This was a deliberate and strategic design choice, and we'd like to elaborate on the reasoning.\n\nOur primary goal was to create a robust, scalable, and model-agnostic augmentation framework. Implementing more complex relations (e.g., \"inside\", \"leaning on\") would require detailed, object-specific semantic and geometric knowledge (e.g., identifying container properties or specific support surfaces). Enforcing such constraints would limit our method to smaller, heavily annotated datasets and prevent us from leveraging the scale and diversity of massive, less-curated collections like Objaverse, which is a cornerstone of our approach's success. Our simple relations are universal and object-agnostic, ensuring maximum scalability. \n\nDespite their simplicity, our experiments show that these foundational relations provide a powerful learning signal. The significant improvements in downstream tasks that require nuanced spatial understanding, such as 3D Visual Question Answering, confirm that the model effectively learns useful bases for spatial reasoning even from these basic cues, which form a stronger starting point for finetuning on downstream tasks/using the model as a frozen feature extractor.\nTo empirically analyze this last observation, we grouped performance on ScanQA questions involving more complex spatial relations not used during the pretraining of the contrastive model. The results below demonstrate a consistent and significant performance uplift with respect to the non-SCENEFORGE models:\n\n| Relation Type (num questions) | Metric | OpenShape | SF-OpenShape | Δ | ViT-Lens | SF-ViT-Lens | Δ | Uni3D | SF-Uni3D | Δ |\n| :---------------------------- | :----: | :-------: | :----------: | :-: | :------: | :---------: | :-: | :---: | :---: | :---: |\n| **Attached To** (21)          | CIDEr  |   54.5    |     61.5     | +7.0|   57.1   |    63.3     | +6.2| 57.9  | 66.6  | +8.7  |\n|                               | EM     |   14.1    |     17.1     | +3.0|   15.6   |    17.9     | +2.3| 16.5  | 20.8  | +4.3  |\n| **Sitting On** (59)           | CIDEr  |   56.8    |     63.4     | +6.6|   59.0   |    65.1     | +6.1| 61.0  | 70.1  | +9.1  |\n|                               | EM     |   15.2    |     17.7     | +2.5|   16.6   |    18.4     | +1.8| 17.5  | 22.6  | +5.1  |\n| **Between** (112)             | CIDEr  |   54.1    |     61.2     | +7.1|   56.8   |    62.9     | +6.1| 57.2  | 66.5  | +9.3  |\n|                               | EM     |   14.0    |     17.0     | +3.0|   15.5   |    17.9     | +2.4| 15.8  | 20.5  | +4.7  |\n| **Closest To** (112)          | CIDEr  |   55.0    |     61.8     | +6.8|   57.5   |    63.5     | +6.0| 58.5  | 67.0  | +8.5  |\n|                               | EM     |   14.3    |     17.2     | +2.9|   15.8   |    18.0     | +2.2| 16.2  | 20.4  | +4.2  |\n| **In Front Of** (246)         | CIDEr  |   56.1    |     62.5     | +6.4|   58.2   |    64.0     | +5.8| 60.3  | 68.3  | +8.0  |\n|                               | EM     |   14.9    |     17.6     | +2.7|   16.3   |    18.2     | +1.9| 17.1  | 21.8  | +4.7  |\n\nUpon acceptance, we will add this insightful extended analysis to the supplementary material, thanks again to the reviewer for the question.\n\nFuture Work: Even if our approach has good generalization to some more complex unseen relations, extending the set of relations is a crucial and interesting direction for future work and practical deployments. As we discuss in the limitations section of our paper , we believe that developing methods for exploiting both large uncurated data and more complex, semantically guided compositions is a promising next step that can build upon the general-purpose foundation we have established here.\n\nW2 and Q2:\n\nOn the Fixed Point Sampling Budget: we agree that the relationship between the point budget and scene complexity is a fascinating and important aspect of our work.\n\nWhile re-running our full set of experiments with larger point budgets was (and is) unfortunately outside our computational budget, we are happy to share results from some preliminary experiments conducted for this paper that explore this exact question. It's important to note that these tests were run on a 20x smaller training dataset, a 50k random subset of Objaverse (without LVIS labels), but we believe the findings directly address the reviewer's curiosity and provide valuable insight.\n\nThe table below shows zero-shot accuracy on the manuscript evaluation datasets, varying both the point resolution (rows) and the maximum number of composed objects N (columns). All other hyperparameters are fixed.\n        \n\n##### LVIS\n| Pts. |N=1 | N=2 | N=3 | N=4 | N=5 |\n| :--- | :---- | :---- | :---- | :---- | :---- |\n| **5k** | 27.24 | **27.59** | 27.26 | 27.02 | 26.58 |\n| **10k**| 27.41 | 28.06 | **28.15** | 27.64 | 27.33 |\n| **15k**| 27.38 | 28.03 | **28.18** | 28.09 | 27.79 |\n| **20k**| 27.47 | 28.10 | **28.13** | **28.13** | 28.10 |\n\n##### ModelNet\n| Pts. |N=1 | N=2 | N=3 | N=4 | N=5 |\n| :--- | :---- | :---- | :---- | :---- | :---- |\n| **5k** | 65.04 | **65.51**| 65.11 | 64.27 | 63.89 |\n| **10k**| 65.12 | 65.83 | **66.28**| 65.51 | 65.04 |\n| **15k**| 65.17 | 65.85 | **66.26** | 66.21| 65.58 |\n| **20k**| 65.15 | 66.04 | 66.25 | 66.25 | **66.28** |\n\n##### ScanObjNN\n| Pts. |N=1 | N=2 | N=3 | N=4 | N=5 |\n| :--- | :---- | :---- | :---- | :---- | :---- |\n| **5k** | 48.98 | **49.53**| 48.80 | 48.52 | 47.96 |\n| **10k**| 49.03 | 49.71 | **50.01**| 49.45 | 49.08 |\n| **15k**| 49.10 | 49.80 | 50.04 | **50.11**| 49.68 |\n| **20k**| 49.08 | 49.92 | **50.09** | 50.05 | **50.09** |\n\n##### ScanNet\n| Pts. |N=1 | N=2 | N=3 | N=4 | N=5 |\n| :--- | :---- | :---- | :---- | :---- | :---- |\n| **5k** | 25.98 | **26.47**| 26.25 | 25.04 | 24.79 |\n| **10k**| 26.17 | 27.23 | **27.45**| 26.81 | 26.10 |\n| **15k**| 26.15 | 27.22 | **27.48** | 27.44| 27.09 |\n| **20k**| 26.21 | 27.28 | 27.44 | **27.47** | 27.42 |\n\nThe benefit of compositions is evident already at N=2 and plateaus around N=3. The most significant performance gains are achieved when composing 2 or 3 objects. This aligns perfectly with our paper's finding that N=3 is the optimal configuration for the 10k point budget used in our main experiments. Beyond this point, the returns diminish, suggesting a natural plateau. A higher point budget primarily alleviates degradation. This is also in line with our hypothesis that heavy performance drops at high N are due to point budget limitations. However, as the resolution increases from 10k to 20k, the primary effect is not a dramatic increase in peak accuracy. Instead, the larger budget makes our model resilient to training sample complexity, alleviating the performance degradation for N=4 and N=5. This confirms that the performance drop is not a failing of the compositional method itself, but a direct result of feature fragmentation under a constrained point budget.\n\nWe trust that our detailed justification for the chosen spatial relations, combined with the new quantitative results showing generalization to complex, unseen relations and the preliminary study on point budgets, have fully addressed the reviewer concerns and questions. We are confident these additions will significantly strengthen the paper and would be grateful if the reviewer would consider this in their final assessment. We once again thank the reviewer for their valuable and constructive feedback."}, {"review_id": "FoQTCQPkMh", "reviewer": "Reviewer_ikuk", "summary": "This paper introduces SCENEFORGE, a novel data augmentation framework designed to enhance contrastive learning alignment between 3D point clouds and text by constructing structured, multi-object 3D scenes. The core idea is to utilize individual 3D object point clouds and combine them into complex scenes based on predefined, explicit spatial relations (e.g., \"over,\" \"under,\" \"next to\"). Simultaneously, the method leverages a Large Language Model (LLM), specifically Qwen2.5, to \"forge\" the individual object text descriptions into a single, coherent, and natural scene description that accurately reflects the spatial layout. By mixing these synthetic \"compositional\" samples with original single-object samples in training batches, SCENEFORGE effectively increases the diversity and complexity of the training data, thereby alleviating the scarcity of large-scale 3D-text datasets.", "questions": "1. The success of SCENEFORGE appears to stem from a combination of factors: stronger spatial relationship modeling, more challenging contrastive samples, and richer linguistic descriptions from the LLM. Could you design an experiment to try and disentangle the contributions of these factors? For example, what would the performance be if you used a fixed template (e.g., \"A model of [object1] is [relation] a model of [object2]\") to generate captions, without refinement from the LLM? This would help isolate the precise contribution of the LLM \"forging\" process.\n\n2. To better understand the method's boundaries, could you provide an analysis of some failure cases? For instance, are there certain object combinations or spatial relationships that SCENEFORGE struggles to generate or align correctly? Or, in what situations does the LLM produce erroneous or ambiguous scene descriptions?\n\n3. In the methods section, you mention that the \"next to\" placement is achieved by sampling a horizontal unit vector d. How is this vector d sampled? Is it re-sampled randomly for each placement, or is it consistent for all \"next to\" relationships within a single composed scene? This detail could affect the naturalness of the scene layouts.\n\n4. As noted in the paper, the fixed point budget and simple spatial relations are current limitations. What do you believe is the best path forward to address this? Is it simply a matter of increasing the point budget (e.g., having it scale dynamically with the number of objects N), or are more intelligent point cloud sampling strategies needed to preserve salient geometric features? On the spatial relations front, how could a richer set of relations be introduced in a cost-effective manner?\n\n5. The experiments show that α=0.5 is an optimal choice. This value seems intuitive (half single samples, half composed samples). Do you believe this optimal value is universal? Might it change depending on the capacity of the backbone model, the characteristics of the dataset, or the number of combined objects N?", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. While most contemporary work focuses on improving model architectures, SCENEFORGE takes a unique, data-centric approach, proposing a model-agnostic, plug-and-play framework. Through the \"scene forging\" process, it not only virtually expands the dataset but, more importantly, introduces \"harder samples\" with structured and relational information, which is crucial for enhancing a model's generalization and spatial reasoning capabilities.\n2. The authors have conducted exceptionally thorough and rigorous experiments to validate their method's efficacy.\n3. Not only does SCENEFORGE achieve consistent gains over all baselines, but its SF-Uni3D variant also outperforms the powerful ensemble model OmniBind on several zero-shot classification benchmarks. \n\nWeaknesses:\nWhile the results presented in this paper are strong, I have significant reservations about the novelty of the core methodology. At its heart, the proposed method, SCENEFORGE, is a direct data augmentation technique. It does not introduce a new model architecture or a fundamental change to the contrastive learning framework itself.\nThe contribution is primarily at the data level—synthesizing more complex and diverse training samples by composing existing ones. Although this is executed cleverly with the use of an LLM for caption refinement, the overall approach can be framed as an advanced form of data engineering rather than a novel method in representation learning. The work, despite its effectiveness, may be perceived as incremental from a methodological standpoint.", "was_revised": true, "final_justification": "Thank authors for addressing my previous concerns. But I think the technical contribution of this work is still rather limited. I maintain my original score.", "rebuttal": "We thank the reviewer for their detailed analysis and for recognizing our \"exceptionally thorough and rigorous experiments.\" We appreciate the opportunity to clarify why SCENEFORGE is a methodological contribution, not an incremental augmentation.\n\nWe agree that our work is data-centric, and we argue this is precisely where its methodological novelty lies. In a field where architectural improvements can yield diminishing returns, we demonstrate that innovating on the learning signal itself is a powerful and novel frontier. Here's why:\n\n- Data-Centric Novelty: The novelty of SCENEFORGE lies in its structured manipulation of the input-label space, a powerful way to regularize the model and improve its representations. This approach of innovating on the learning signal itself, rather than the architecture, is precisely why foundational 2D methods like MixUp and CutMix were considered novel. SCENEFORGE introduces a similar principle to the 3D-text domain, but goes further by composing scenes that preserve object semantics while injecting new, structured relational information.\n\n- Fundamentally Alters the Learning Task: We shift the objective from only object recognition to a more complex task including contextual scene understanding. By introducing in the batch structured 3D scenes with LLM-refined scene captions, we force the model to learn spatial relationships and part-whole reasoning. Our experiments prove this is crucial for enhancing the model's spatial reasoning and generalization.\n\n- Structured Composition, Not Direct Mixing: Unlike direct augmentations like PointMixup, our structured forging process ensures geometric and semantic coherence. Our ablation study (Table 4) proves this structured approach is the key to our performance.\n\n- Novelty Validated by Impact: Our framework's impact is the strongest evidence of its novelty. The fact that our single model outperforms a powerful SOTA ensemble (OmniBind) shows that a better learning signal can be more impactful than adding architectural complexity.\n\nWe hope this clarifies our perspective. We believe this reframes SCENEFORGE as a novel methodological contribution, and we will gladly revise the manuscript to emphasize these points more clearly.\n\n## Questions\nQ1: LLM ablation\n\nWe agree that ablating the LLM component in the pipeline is crucial. We ran this exact experiment (Supp. Table 2): we compare our full method (using Qwen 2.5) against a variant that uses raw, template-based captions without any LLM refinement (SF-Uni3D/NO LLM). These raw captions are constructed by simply concatenating object descriptions with spatial relations (e.g., \"[object1] next to [object2]\"), which is analogous to the reviewer's proposed template-based approach. The \"NO LLM\" variant, which uses only structured 3D scenes and template-based text, already shows clear improvements over the Uni3D baseline across all benchmarks. However, refining these raw captions with an LLM provides a second, significant boost in performance. We also report and observe similar boost also with LLama 3.1 instead of Qwen 2.5.\n\nQ2: Failure cases. \n\nOn the point-cloud composition side, our choices of positional relationship and the method we propose to apply them ensures us that no failure cases can occurr. For the LLM side, there are possible cases of hallucination and information alteration. To analyze LLM failure cases, following the reviewer insightful question, we built an automated verification pipeline using dependency parsing with spaCy (classic NLP). This tool programmatically extracts subjects (nouns) and their descriptive attributes (adjectives), and we do that for both the raw concatenated captions and the final refined captions. We ran this check on 1M random combined scene captions with SCENEFORGE, and we found that in over 98% of cases, original subjects and their attributes were perfectly preserved. In the ~2% failure cases, 79% of them are for wrong attributes (attribute binding is indeed a very well known problem of LLMs), 20% are missing attributes and just 1% is about wrong subjects. Even with this unfrequent possible failure cases, the use of the LLM for rewriting brings crucial benefits to the pipeline, as discussed in the previous point.\n\nQ3: d vector\n\nd in the xy-plane is re-sampled randomly for each \"next to\" placement, and it is identified by uniform sampling its angle wrt x axis. Inside single scenes, we only check for intersections with previous placed objects in case of #objects > 2 to avoid unrealistic scenes, and in that case we resample it. This allows us to produce all valid and variegate positionings in the considered potential scene.\n\nQ4: Fixed point budget & spatial relations\n- On Point Budget and Sampling Strategies:\nWhile re-running our full set of experiments with larger point budgets was (and is) unfortunately outside our computational budget, we are happy to share results from some preliminary experiments conducted for this paper that explore this exact question up to 20k points per shape. It's important to note that these tests were run on a 20x smaller training dataset, a 50k random subset of Objaverse (no LVIS labels) on the Uni3D variant.\n\n    ##### LVIS\n    |Pts.|N=1|N=2|N=3|N=4|N=5|\n    |:---|:----|:----|:----|:----|:----|\n    |**5k**|27.24|**27.59**|27.26|27.02|26.58|\n    |**10k**|27.41|28.06|**28.15**|27.64|27.33|\n    |**15k**|27.38|28.03|**28.18**|28.09|27.79|\n    |**20k**|27.47|28.10|**28.13**|**28.13**|28.10|\n\n    ##### ModelNet\n    |Pts.|N=1|N=2|N=3|N=4|N=5|\n    |:---|:----|:----|:----|:----|:----|\n    |**5k**|65.04|**65.51**|65.11|64.27|63.89|\n    |**10k**|65.12|65.83|**66.28**|65.51|65.04|\n    |**15k**|65.17|65.85|**66.26**|66.21|65.58|\n    |**20k**|65.15|66.04|66.25|66.25|**66.28**|\n\n    ##### ScanObjNN\n    |Pts.|N=1|N=2|N=3|N=4|N=5|\n    |:---|:----|:----|:----|:----|:----|\n    |**5k**|48.98|**49.53**|48.80|48.52|47.96|\n    |**10k**|49.03|49.71|**50.01**|49.45|49.08|\n    |**15k**|49.10|49.80|50.04|**50.11**|49.68|\n    |**20k**|49.08|49.92|**50.09**|50.05|**50.09**|\n\n    ##### ScanNet\n    |Pts.|N=1|N=2|N=3|N=4|N=5|\n    |:---|:----|:----|:----|:----|:----|\n    |**5k**|25.98|**26.47**|26.25|25.04|24.79|\n    |**10k**|26.17|27.23|**27.45**|26.81|26.10|\n    |**15k**|26.15|27.22|**27.48**|27.44|27.09|\n    |**20k**|26.21|27.28|27.44|**27.47**|27.42|\n\n    These results confirm our hypothesis from the paper. The benefit of composition peaks around N=3 objects, aligning with our main findings. More importantly, increasing the point budget primarily makes our model more resilient to complexity, significantly alleviating the performance degradation at N=4 and N=5. This demonstrates that the drop is a direct result of feature fragmentation under a constrained budget, not a fundamental flaw in our compositional method. Therefore, while scaling the point budget is one solution, we agree with the reviewer that testing with more intelligent sampling strategies to preserve salient features is a promising future direction to balance the point budget and accuracy.\n\n\n- Spatial Relations:\nWe chose simple spatial relations to ensure our method remains scalable and applicable to the large, uncurated datasets crucial for contrastive learning. Implementing more complex relations (e.g., \"inside,\" \"leaning on\") would require detailed, object-specific semantic and geometric knowledge (like identifying container properties or support surfaces), not available in large datasets.\n\n    The ideal path forward should seek to introduce these harder, semantic-aware relations without sacrificing scalability. This could involve future work on methods to automatically annotate large datasets with the necessary geometric and semantic priors to enable more complex compositions.\n\n    While this is a promising future direction, our current simple relations already provide a powerful learning signal that generalizes well. A new analysis on the ScanQA dataset confirms this: our method significantly improves performance on unseen, complex relations, proving our pre-training builds a strong, generalizable foundation:\n\n    |Relation Type (num questions)|Metric|OpenShape|SF-OpenShape|Δ|ViT-Lens|SF-ViT-Lens|Δ|Uni3D|SF-Uni3D|Δ|\n    |:----------------------------|:----:|:-------:|:----------:|:-:|:------:|:---------:|:-:|:---:|:---:|:---:|\n    |**AttachedTo** (21)|CIDEr|54.5|61.5|+7.0|57.1|63.3|+6.2|57.9|66.6|+8.7|\n    ||EM|14.1|17.1|+3.0|15.6|17.9|+2.3|16.5|20.8|+4.3|\n    |**SittingOn** (59)|CIDEr|56.8|63.4|+6.6|59.0|65.1|+6.1|61.0|70.1|+9.1|\n    ||EM|15.2|17.7|+2.5|16.6|18.4|+1.8|17.5|22.6|+5.1|\n    |**ClosestTo** (112)|CIDEr|55.0|61.8|+6.8|57.5|63.5|+6.0|58.5|67.0|+8.5|\n    ||EM|14.3|17.2|+2.9|15.8|18.0|+2.2|16.2|20.4|+4.2|\n\nQ5: Value of alpha\n\nOur choice of α=0.5 was based on an empirical sweep with a 0.1 step for each of our three backbones, a granularity dictated by the high computational cost of these experiments.\n\nAs shown in Figure 6 of the main paper, we observed that performance increases up to 0.5, but however doesn't hinge on a single sharp peak, but rather forms a plateau of high accuracy between roughly α=0.3 and α=0.6. Interestingly, this trend and the peak performance at α=0.5 were remarkably consistent across all three models, from the adapter-based ViT-Lens to the large-capacity Uni3D. This suggests that the optimal mixture ratio may be largely independent of the model architecture and capacity itself. While the exact global optimum is likely around 0.5, we agree it could still depend on other factors, such as the dataset's intrinsic complexity and, as the reviewer suggests, the maximum number of combined objects (N), a valuable direction to analyze with sufficient compute.\n\n\nWe are grateful to the reviewer for their thorough analysis and for pushing us to better articulate the novelty of our data-centric approach. We trust that our detailed response and the new supporting evidence provide a clearer and more compelling case for our contributions and address the initial concerns."}]

Author rebuttal:

Meta-review: This paper proposes SceneForge, a framework that improves 3D-text contrastive alignment. This approach increases data diversity and improves performance on various tasks. It also enhances spatial reasoning and adapts well to more complex 3D scenes.

Strengths of this work: SceneForge is a general framework that can be easily integrated with other 3D encoders. It can improve performance on a wide range of tasks and benchmarks. The method is simple yet effective, supported by comprehensive experiments.

The rebuttal effectively addressed reviewers' major concerns and clearly explained the technical contributions of the work. As a result, the final decision is to accept the paper.
Decision: Accept (poster)