TIGaussian: Disentangle Gaussians for Spatial-Awared Text-Image-3D Alignment
Authors: Jiarun Liu
Qifeng Chen
Yiru Zhao
Minghua Liu
Baorui Ma
Sheng Yang
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal
Gaussian Splatting
Contrastive Learning
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: While visual-language models have profoundly linked features between texts and images, the incorporation of 3D modality data, such as point clouds and 3D Gaussians, further enables pretraining for 3D-related tasks, e.g., cross-modal retrieval, zero-shot classification, and scene recognition.
As challenges remain in extracting 3D modal features and bridging the gap between different modalities,
we propose TIGaussian, a framework that harnesses 3D Gaussian Splatting (3DGS) characteristics to strengthen cross-modality alignment through multi-branch 3DGS tokenizer and modality-specific 3D feature alignment strategies. Specifically, our multi-branch 3DGS tokenizer decouples the intrinsic properties of 3DGS structures into compact latent representations, enabling more generalizable feature extraction. To further bridge the modality gap, we develop a bidirectional cross-modal alignment strategies: a multi-view feature fusion mechanism that leverages diffusion priors to resolve perspective ambiguity in image-3D alignment, while a text-3D projection module adaptively maps 3D features to text embedding space for better text-3D alignment. Extensive experiments on various datasets demonstrate the state-of-the-art performance of TIGaussian in multiple tasks. Code repository: https://github.com/RUiN-jiarun/TIGaussian.

Published as a conference paper at ICLR 2026
TIGAUSSIAN: DISENTANGLE GAUSSIANS FOR SPATIAL- AWARED TEXT-IMAGE: ALIGNMENT
Jiarun Liu' Qifeng Chen
Yiru Zhao' Minghua Liu²
Baorui Ma²
Sheng Yang"t
Unmannned Vehicle Dept., Cainiao Inc., Alibaba Group, Hangzhou, China Hiillbot, Sunnyvale, USA
*Beijing Academy of Artificial Intelligence, Beijing, China
j giarunlius luu ju. edw..ddu.co. cn, (cqf7419, shengyang93fs & gm i 1l.............. com
13
Query red wnde
UniGS Top-5
Ours Top-5
9.59
0.38
0.58
0
D44
G4B
0.4
parple basclte
13
DAI
ABC
Uni08
D41
0411
givee
Figure 1: TIGAUSSIAN enables 3D modal pretraining on several tasks, e.g., zero-shot classification, text-3D retrieval and image-3D retrieval. Left: Compared to prior 3D multi-modes alignment methods Uni3D and UniGS, TIGAUSSIAN presents superior performance on multiple datasets Objaverse(-LVIS) and ABO. Right: In challenging scenarios involving ambiguous complex queries, TIGAUSSIAN demonstrates superior performance owing to its disentangled encoder architecture and specialized cross-modal alignment mechanism. We report the similarity score of each item in the figure.
ABSTRACT
While visual-language models have profoundly linked features between texts and images, the incorporation of 3D modality data, such as point clouds and 3D Gaussians, further enables pretraining for 3D-related tasks, e.g., cross-modal retrieval, zero-shot classification, and scene recognition. As challenges remain extracting 3D modal features and bridging the gap between different modalities, we propose TIGAUSSIAN, a framework that harnesses 3D Gaussian Splatting (3DGS) characteristics strengthen cross-modality alignment through multi-branch 3DGS tokenizer and modality-specifi 3D feature alignment strategies. Specifically, our multi-branch 3DGS tokenizer decouples the intrinsic properties of 3DGS structures into compact latent representations, enabling more generalizable feature extraction. To further bridge the modality gap, we develop bidirectional cross- modal alignment strategies: a multi-view feature fusion mechanism that leverages diffusion priors to resolve perspective ambiguity in image-3D alignment, while a text-3D projection module adaptively maps 3D features text embedding space for better text-3D alignment. Extensive experiments on various datasets demonstrate the state-of-ther thefthheartt performance of TIGAUSSIAN in multiple tasks. Code repository: https://gitubb.co/RRIii Jianiiiiiin//TTaausiiia Taaasssin.
"Equal contribution. Corresponning author.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Recent breakthroughs in vision-language pretraining (Radford et al., 2021; Sun et al., 2023) have established text-image alignment as the foundation for multi-modal understanding. In recent years, significant advances have been made in expanding the field of 3D visual understanding through various 3D representations. Early works in 3D multi-modal aligmment primarily relied on point clouds processed through 3D convolutions (Zhang et al., 2022; Huang et al., 2023; Xue et al., 2023; 2024) or 3D Vision Transformers (Zeng et al., 2023). Subsequently, works on meshes (Song et al., 2023) and voxels (Ruan et al., 2024) further explores the operations on organized 3D representations. More recently, UniGS (Li et al., 2025) first leverages 3D Gaussian Splatting (3DGS) as an advanced 3D representation, demonstrating state-of-the-ar text-image-31 alignment across multiple benchmarks.
However, existing methods still facing challenges regarding the abstraction ability of 3D context, as well the gaps between text-3D and image-3D modalities. Specifically, our analysis reveals that the current JDGS-based multi-moda, work UniGS, while groundbreaking, suffers from following limitations: (1) Entangled 3D encoding: All attributes of each Gaussian primitive are concatenated and encoded as whole, insufficiently leveraged the distribution pattern and geometric significance of each attribute intrinsic relations between attributes; (2) Degraded 3D perception: Forced alignment from a single-view image fails to capture global context, causing decrease in 3D perception ability.
To overcome these limitations, we propose TIGAUSSIAN, a tri-modal alignment framework that bridges text, image, and 3DGS representations which outperforms existing feature alignment ap- proaches. At its core, our method utilizes multi-branchh 3DGS tokenizer to extract 3D contextual information, which reducing inter-attribute coupling, ultimately establishing compact and effec- tive latent representation for 3DGS. For image-3I alignment, we incorporate a diffusion-ennanced multi-view fusion strategy, which compensates for single-view limitations by leveraging implicit 3D awareness from pretrained diffusion priors. Meanwhile, for text-3D alignment, the encoded feature adapts the 3D latent space through 3D-text projection module, thereby better matching text embedding structures.
We conduct extensive experiments validate the performance of TIGAUSSIAN multiple tasks across the Objaverse (Deitke et al., 2023), ABO (Collins et al., 2022) and SUN RGBD (Song et al., 2015) datasets. As presents in Fig. 1, the results demonstrate the effectiveness of the proposed method, achieving superior performances on zero-shot classification, cross-modal retrieval, few-shot linear probing and open-world scene recognition.
In summary, our contributions can be summarized as follows:
We propose TIGAUSSIAN, framework for text- rext-image-3DGr alignment, achieving state-of- the-art performance on various downstream tasks.
We design a multi-branch 3DGS tokenizer, with specialized Gaussian attribute modeling to enhance 3D feature abstraction and compression.
We propose tri-modal alignment strategy through 3D-aware image feature fusion and 3D-text projection module, enabling robust alignment for both text and image modalities.
RELATED WORK
2.1 3D REPRESENTATION LEARNING.
The development 3D representations is an important factor in promoting research on 3D shape analysis and multi- mulli-modality tasks. Early works focused on volumetric grids (Maturana & Scherer, 2015), which enabled structured 3D convolutions but incurred prohibitive computational costs for high-resolution scenes. With the development of neural networks, more works use point clouds as sparse geometric proxies, leveraging crmmutation-invarian operators like PointNet (Qi et al., 2017a;b) and other classical network architectures (Choy et al., 2019; Ma et al., 2022) to extract localized features. In recent years, there have been many mesh-based feature extraction works in dense 3D representation (Feng et al., 2019; Hu al., 2022). but the inherent complexity of mesh structures poses significant challenges for effective feature representation. A transformative leap arrived with implicit neural representations like DeepSDF (Park et al., 2019) and NeRF (Mildenhall
2

---

Published as a conference paper at ICLR 2026
et al., 2021), which encode surfaces radiance fields via coordinate- MLPs. These methods achieved unprecedented reconstruction quality but suffered from slow optimization and rendering. The recent Gaussian Splatting (Kerbl et al., 2023; Huang et al., 2024) further addresses these limitations by introducing explicit, differentiable primitives optimized for real-time photorealistic rendering. Some recent works exploit the hierarchical structures for scene organization and neural anchors representations (Lu et al., 2024; Ren et al., 2025) for better feature extraction, or embedding the semantic visual features into 3D Gaussian primitives (Zhou et al., 2024b; Shi et al., 2024; Qin et al., 2024). These works in 3D representation learning in recent years have demonstrated the superiority of 3DGS in scene representation, understanding, and other aspects.
2.2 MULTI-MODAL PRETRAINING IN 3D TASKS.
Text-2D Multi-modal Learning. Multi-modal pretraining via contrastive learning has revolution- ized cross-modal understanding, with foundational frameworks like CLIP (Radford et al., 2021) and EVA-CLIP (Sun et al., 2023) demonstrating the power of aligning image-text pairs, BLIP series (Li et al., 2022; 2023) further enhance the text-image alignment capability through richer data and hybrid encoder-decoded structures. More recent works involve Large Language Models (LLMs) for visual-text understanding (Team, 2023; Zhang et al., 2023; Dong et al., 2024; Chen al., 2024).
Text-image-3D Tri-modal Learning. Recently, more works have extended 3D fields. Early 3D adaptation strategies such as PointCLIP (Zhang et al., 2022) and CLIP2Point (Huang et al., 2023) circumvented geometric complexity by rendering depth maps from point clouds, effectively projecting 3D data into 2D subspaces for compatibility with pretrained image encoders. Recent efforts like ULIP series (Xue et al., 2023; 2024) and CLiP² (Zeng et al., 2023) directly encode raw point clouds through 3D transformers, achieving improved robustness in real- world indoor/outdoor benchmarks. Other works such as TriCoLo oo (Ruan et al., 2024) explore the performance of voxel representations in multi-modal alignment, but are limited to the scale of data and models. OpenShape (Liu et al., 2023) leverages PoiRt-BERT (Yu et al., 2022) and a 3D convolution architecture to train a point cloud encoder for cross-modal understanding. The concurrent work Uni3D (Zhou et al., 2024a) utilizes the Vision Transformer architecture (Dosovitskiy et al., 2021) and large amounts of point cloud data achieve a model with IB parameters large model, and realize a state-of-the the-art solution for text- Boxt-image-33 aligmment. More recent works ( (Wang et al., 2025) have adapted Uni3D as the foundation model for 3D learning. To expand into more diverse 3D representations, UniGS (Li et al., 2025) pioneers 3DGS for multi-modal tasks by distilling the Uni3D) pretrained model, which makes significant progress. However, these methods suffer from entangled attribute encoding and single- view bias during aligmment. Although recent works attempt to encode 3D objects using multi-view images (Lee et al., 2025; Zhou et al., 2025), this inevitably increases the amount of content required for encoding and loses the simplicity of encoding directly for 3D models. Our framework addresses these limitations through an attribute- disentt-eddee 3DGS encoding scheme coupled with cross-view attention mechanisms, dynamically aligned via pre-projection module that warps the latent 3D manifold to match text-image contrastive embedding spaces.
3 METHODOLOGGY
3.1 OVERVIEW
As illustrated in Fig. 2, with the vanilla 3DGS (Kerbl et al., 2023) as the most prevalent input Gaussian representation, TIGAUSSIAN processes tri-modal inputs through three coordinated components: (1) For the 3D modal, use multi-branch 3DGS tokenizer which decomposes geometric and appearance attributes to construct a structured latent embedding FL for spatial context; (2) For the image modal, features Pmn which aggregates implicit 3D priors from pretrained diffusion models; (3) For aligning the text modal, we further project the spatial features F into F through 3D-text projector to ensure modality-invarian feature consistency with text embeddings.
a diffusiioneenhancee multi-view fusion module for generating 3D-aware visual
3

---

Published as a conference paper at ICLR 2026
A duck with white feothers, long reck and yellow
3DGS
Image
Σ
Muliisiew Dillision
3DGS Tokenizer
Image CLIP + Enooder
Enceder
MD-Aware Image Fenture Fusion
Projector
F
C(F Fe)
C
Aligrmennt
Alignment
Zaro-sho Classification
Image
Scene Recogaition
Few-shot Classification
Figure 2: The TIGAUSSIAN framework for text-image-3D tri-modal representation learning. (1) 3DGS Tokenizer: multi-branch lightweight network decomposes input Gaussians into separate attributes, generating structured latent embedding Fd- (2) Image Modality: Multi- diffusion generates consistent views processed through CL.IP to produce 3D-aware visual features F (3) Text Modality: The 3D features are projected to text space FJ via a learnable projector. Dual contrastive losses C(Fg) FT and C(F FT) align all modalities in shared embedding space, enabling diverse cross mross-modal applications.
3.2 PRELIMINARIES
3D Gaussian Splatting. 3DGS characterizes the object through collection of anisotropic Gaussian primitives defined within the 3D space (Kerbl al., 2023). For each 3D) Gaussian G, is described by the following attributes: center position vector R3, opacity parameter a E [0, 4J. set of spherical harmonics (SH) coefficients SH E I and a covariance matrix Σ R3333
1 ((x) exp( - - =- 14))
among which, the SH coefficients can be derived to RGB color с E R&, while the covariance Σ can be decomposed into scaling factor & E TR3 and rotation quaternion parameter q E RR1 as:
Σ RSS R
where S E R3x3 is the scaling matrix derived from & and R E R3x3 is the rotation matrix derived from q.
Multi-modal Alignment via Contrastive Learning. After widespread validation in multi-modal works in recent years (Radford et al., 2021; Li al., 2022; Xue et al., 2023), cross-modal feature alignment through contrastive learning is highly effective training method. The contrastive loss of two features L(F1, F2) is calculated by the InfoNCE loss (Oord al., 2018), as:
(*,
C(F1, F2) = -E, log
L+ T) (-,,)-((,T)
L (x, exp(Fi(x) (/()))/)), 2 @E(+,-)
where (ar, x denotes the positive pair sample x, and M is the number of the negative pairs {(x, ) is the Iearnable temperature parameter scaling the similarity scores in contrastive learning. As proved most 3D multi-modal tasks (Zhou et al., 2024a; ct al., 2025), the text-image model is pre-aligned (Dosovitskiy al., 2021; Sun al., 2023). thus we only need to separately calculate the contrastive loss between 3D features FAD and pre-aligned image features Fi and text features Fr. Therefore, the optimization object can be described as:
L = ATTC(Fan Fr) + Ar ArC(F Fi),
where Ay, A balance factors.
4

---

Published as a conference paper at ICLR 2026
3DGS Tokenbzer
F
Laanahic Quario
F
VI Blocks
E,
Ft:
Sel Anertior
PC
Cluss Alleticn
3DGS
E,
XV
ML.
x
Ciras Atstioos
VII Block
Postiy
Lays Nom
Enoader
Aware Foature
-ext Festure
Figure 3: Illustration of 3DGS tokenizer, where multiple branches are tailored for different at- tributes for more compact and effective extrac- tion.
Figure 4: Illustration of 3D-awar image feature fusion module and 3D-text projector module.
3.3 MULTI-BRANCH 3DGS TOKENIZER
Given 3D object represented by Gaussians, we first follow existing approaches (Zhou al., 2024a; Li al., 2025) to downsample it 1024 Gaussians by Farthest Point Sampling (FPS) and group them by k-Nearest Neighbor (kNN) algorithm to form several local Gaussian patches fixed numbers of Gaussians. These local groups serve processing units for the final ViT-based encoding stage, allowing the model to capture both fine- ane-ggriindd local patterns and global structural relationships. Different from UniGS (Li et al., 2025) which concatenates all Gaussian attributes as homogeneous features, we utilize a well-designed tokenzier to explicitly model the distinct characterization and distributional properties of Gaussian. Specifically, as shown in Fig. 3, each attribute of Gaussian patch G {u,, a, C, 8, /) including spatial (μ4), appearance (o, c) and morphological (s, q) components with inherent inter-attribute relationships is separately fed into its corresponding encoding branch {En, Ea, Ec. E., }. From an information representation perspective (Tishby et al., 2000), such disentangled token scheme reduces mutual interference among heterogeneous attributes during abstraction and compression. Each branch can adaptively compress its input under an attribute- specific information bottleneck, preserving only the most task relevnelevant signals. This avoids entangled representations that arise when disparate modalities (e.g., position and color, which possess distinct numerical ranges and representational purposes) are forced to be transformed within a shared domain, thereby resulting in information loss and suboptimal feature alignment. As shown in the example of the bed in Fig. 1, previous methods may ignore the detailed information "white striped comforter", while method can highlight this feature.
Each encode branch employs a three-layer MLP for token extraction. For spatial tokenizer En, we adopt encoder architecture inspired by those used for point clouds (Qi al., 201 7a), incorporating an additional learnable positional embedding module to enhance the extraction of high-dimensional spatial context. Furthermore, max pooling layer aggregates all point-wise features into global descriptor by taking the channel-wise maximum values, ensuring permutation invariance and captur- ing the most salient geometric patterns. For appearance tokenizer Eas & we employ the sigmoid activation function to model the nonlinear variations in the appearance space and to constrain the value range. As for morphology tokenizer E,, Ey. we simply use the normalization layer standardize the output. Finally, the spatial, appearance, and morphological features are concatenated and fused via two-layer MI.P, yielding unified 3DGS token that holistically captures attribute interdependencies for downstream tasks.
Inspired by UniGS's training strategy, our 3DGS token is prompted via cross-attention guidance from pretrained point cloud model (Zhou al., 2024a). Specifically, position μ and color attributes are encoded as point cloud features, which are then refined through ViT blocks with cross-attention to integrate pretrained knowledge into the 3DGS token. Finally, the token is processed by a fully- connected layer (FC layer) to adjust its dimension to = 512 and output compact aigh-dimensional 3DD features F/ E Rdd
5

---

Published as a conference paper at ICLR 2026
3.4
IMAGE- 3D ALIGNMENT
Conventional image-3D alignment methodologies (Xue et al., 2023; Zeng et al., 2023; Zhou et al., 2024a; Liu al., 2023; Li et al., 2025) predominantly focus direct alignment between randomly selected single-view image and 3D features. However, this approach inherently compromises the perception capability of 3D features. Specifically, the alignment process constrained to a single perspective may lead to suboptimal matching performance for other viewpoints, as the cross-view consistency of 3D feature representations not adequately preserved during the alignment procedure. While recent approaches attempt to leverage multiple text-image-33 triplets for single 3D object for alignment (e.g., ULIP-2 (Xue et al., 2024)) or represent 3D objects directly using 2D multi- view images (e.g.. Duoduo-CLLn CLIP (Lee et al., 2025)), these methods inevitably compromise either computational efficiency (due to expanded training data requirements) or the benefits of concise explicit 3D representations. To address this issue, propose 3D-aware alignment scheme to bridge the gap between image features with 3D features.
As shown in Fig. 4(a), we leverage the multi-view diffusion (Yang et al., 2024) prior elevate the image information from a single perspective to a 3D spatial dimension. For single-view image I E IR HxW we first generate N multi-view images by the pretrained diffusion model:
DII, I1.I1.... IN), where + { $1, ,NN).
(5)
with Φ; as the i-th preset camera angles. Afterwards, each image is fed into CLIP module (Radford al., 2021) to get the corresponding embedding:
F = CLIP(T) E R2, Fi, = CLIP(L) ER.
(6)
We then use perspective-aware cross-attention module for multi-view feature fusion:
Attn (Q, K, V) = Softmax QK V
(7)
where Q = = Fr, 1 PE( which is the concatenation of all single-view
features. PE(-) stands for sinusoidal positional encoding (Vaswani et al., 2017). The fused 3D-aware feature is:
F Layer + At, V)) e RR..
During alignment, replace the image feature Fi in Eq. 4 with Pm" In this way, 3DGS features will have multi-perspective perception capabilities, thereby enhancing the encoding ability in 3D space.
3.5 TEXT-3D ALIGNMENT
Although the current 3D features F/ has been aligned with multi-view image features, it is still not sufficient to produce good alignment with text modalities. In order to further narrow the gap between 3DGS and text modalities, we propose feature projection module that aligns the latent space of 3DGS features and text features, thereby reducing the difficulty of feature alignment. Inspired by previous methods (Li et al., 2023; Simnam al., 2024; Hadgi et al., 2025), utilize query transformer architecture for latent space projection.
of learnable queries F, E R. soft prompts to iteratively refine the 3D features through an L-layer transformer architecture. Each Transformer block consists of three modules: (1) self-attention for query refinement, (2) cross-attention integrate 3D feature context, and (3) an MLP layer as the feed-forward network. These modules are connected via residual connections.
As shown in Fig. 4(b), we employ
In detail, the transformer blocks process query token Fg through L sequential layers, where each layer E [1, 7.] performs the following operations:
pl LayerNorm
[F SellAum(F"")))
pl LayerNorm
pd Attn(F), Pè. F()),
(9)
pl LayerNorm
pl ML)(E

---

Published as a conference paper at ICLR 2026
with initialization F3 Fg. Finally, the refined queries are flattened and passed through pooling layer to produce the compact embedding F3 _2 for text alignment.
In summary, the overall loss function for contrastive learning can be specialized to:
L = - F+) (F FT""),
(10)
which aims to align features preprocessed for different modalities and bridges the gap between them. Since our work focuses on 3D modality alignment and existing methods (Sun et al., 2023; Li et al., 2023) have effectively addressed multi-modal tasks between text and images, we intentionally exclude
the contrastive loss between text features and fused image features in our framework.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets. We conduct our experiments on three public 3D datasets: Objaverse (Deitke et al., 2023), ABO (Collins et al., 2022) and real-world indoor dataset SUN RGBD (Song et al., 2015), which contain 146k, 7.9k and 6.1k objects, respectively. As for multi-view images, adopt the MVD-std model of Humyuan3D-vl (Yang et al., 2024) to generate 6 views in canonical camera poses for each object before training. Please refer to supplementary materials for more details.
Implementation. We use the Open-CLIP ViT-B-16 model (Radford et al., 2021) as pre-aligned text-image model and Uni3D-S point cloud model (Zhou et al., 2024a) for 3DGS token guidance. We first train TIGAUSSIAN on Objaverse with the Adam W optimizer (Loshchilov & Hutter, 2019) and the learning rate of le for 15 epochs for all downstream tasks. For ABO and SUN RGBD dataset, we quickly finetune the trained model for another 20 epochs to better fit the dataset. All features are set to the same dimension
d 512 for simplicity. Layer number of 3D-text projector is set to L= 6. The balance factors \T and Ay for contrastive learning are both set to 0.5, following the previous works (Zhou et al., 2024a; Li et al., 2025). All models are trained on A A100 GPUs, and inference is performed on single A100 GPU.
Baselines. We compare our method with several state-of-the-an 3D multi-modal methods: CLIP (Zeng et al., 2023), Uni3D (Zhou et al., 2024a), UniGS (Li et al., 2025). We directly inherit the experiment results reported UniGS. We also reimplement ULIP-2 (Xue et al., 2024) and Duoduo-CLIP (Lee et al., 2025) on the same data scales for classification tasks for fair comparisons. We also conduct additional experiments involving more baselines and comparisons, please refer to supplementary materials for more implementation details.
4.2 PERFORMANCE COMPARISON
Zero-shot Classification. We evaluate the zero-shot classification performance of TIGAUSSIAN on the Objaverse-LVIS and ABO datasets, which contain 318 and 23 categories, respectively. Table 1 presents the Top-1, Top-3, and Top-5 classification accuracy results. The experimental results demonstrate that TIGAUSSIAN consistently outperforms all baseline methods on both datasets. Especially compared to recent 3DGS-base (Li et al., 2025) and multi-view based models (Lee et al., 2025), our method's consistent superior performance demonstrates deeper exploration of 3DGS's representational advantages in our design, leading to enhanced overall perception and cross-modal capabilities.
Text-3D Retrieval. We select 1000 objects from Objaverse and ABO dataset for text-3D retrieval following the sampled test list provided by UniGS. Specifically, we simply calculate the cosine similarity of text embedding Fr and text-aligned 3DGS feature F Tab. 2 shows the Top- Top-5 and Top-10 10 accuracy. Experimental results show that the proposed text projection module combined with muulti-bannhh Gaussian tokenizer effectively improve the correlation between 3D and text modalities.
Image-3D Retrieval We further evaluate the image-3D retrieval performances in each batch for Objaverse-LVV and ABO dataset. The results shown in Tab. indicate that TIGAUSSIAN significantly surpasses all current baselines. We attribute it the adjustment of image features and the integration of 3D information, thus overcoming the limitations of perspective issues between 3D-2D modal alignment.
7

---

Published as a conference paper at ICLR 2026
Table 1: Performances of different methods on zero-shot classification task. We report the average
classification accuracy across all categories.
Objaverse-LVV Top-3
ABO Top-3
Methods
3D Repr.
Top-1
Top-5
Top-1
Top-5
CLIP
12.35
24.62
32.91
22.58
43.83
54.56
Point Cloud
ULIP-2
29.75
46.39
55.23
Point Cloud
Uni3D
36.72
57.09
65.18
37.60
59.68
70.22
Point Cloud
UniGS
37.64
57.62
65.57
52.33
70.27
79.38
3DGS
Duoduo CLIP
38.05
57.79
66.70
57.82
76.08
83.46
Multi-view (non explicit)
Ours
41.76
62.68
69.15
61.70
83.16
89.79
3DGS
Table 2: Performance comparison text-3D and image-3D retrieval tasks. Reported metries average retrieval accuracy.
Image-3D Retrieval
Text-3D Retrieval
Methods
3D Repr.
Top-1
Top-3
Top-5
Top-1
Top-5
Top-10
Objaverse LVIS Objaverse
CLIP²
28.83
51.43
63.57
7.40
22.20
32.50
PointCloud
Uni3D
39.65
60.72
70.51
16.70
37.10
48.10
Point Cloud
UniGS
41.78
62.50
72.24
21.00
39.80
53.50
3DGS
Ours
54.11
73.84
81.21
21.20
45.10
56.30
3DGS
ABO
CLIP
15.29
31.74
42.74
7.09
24.34
38.94
Point Cloud
Uni3D
18.25
35.26
45.29
10.29
29.21
43.67
Point Cloud
UniGS
26.69
46.26
56.72
11.27
30.32
43.95
3DGS
Ours
66.15
73.99
85.27
15.87
40.17
53.07
3DGS
Few-shot Linear Probing. To further evaluate the learning capabilities of our proposed model, perform the few-shot linear probing following previous works (Liu et al., 2023; Zhou et al., 2024a). Specifically, freeze the 3DGS tokenizer and only train linear classifier few-shot class labels. We conduct few-shot linear probing on Objaverse- class from 4, to 16. We evaluate each model 10 times and report the average accuracy in Fig. 5. Notably, under the same amount of training data, TIGAUSSIAN outperforms the SoTA 3DGS-based method UniGS.
dataset with labeled training samples per
Methods
Mean Accuracy
3D Representation
SUN RGBD
Ausuncy
CLIP
41.39
Point Cloud
Uni3D
61.72
Point Cloud
UniGS
68.92
3DGS 3DGS
Ours
76.46
Table 3: Scene Recognition on SUN RGBD Dataset. We report the average recognition accuracy across all categories. Please refer to supplementary materials for detailed results.
Numbers sampplo class
Figure Few-shot linear probing results on Objaverse-L.Vis dataset. 0 number of samples per class stands for zero-shot classification.
Open-world Scene Recognition. We follow UniGS (Li et al., 2025) to group the objects SUN RGBD into 37 categories and take the Top-1 classification accuracy the scene recognition accuracy. Tab. 3 shows the average accuracy across all categories. The experimental results once again demonstrate the effectiveness of TIGAUSSIAN.
8

---

Published as a conference paper at ICLR 2026
Table 4: Ablation studies on each components. We report the Top accuracy of each task on Objaverse dataset. We abbreviate bulli--branch 3DGS tokenizer as Tkn., using multi-view images as MV. multi-view image fusion module MVF., and 3D-text projector TP.. Moreover, we abbreviate classification Cl., text retrieval TR., and image retrieval IR..
Exp.
Tkn.
MV.
MVE.
TP.
CL
TR.
IR.
33.64
18.50
39.87
2
V
35.57
19.15
41.68
V
37.28
19.16
46.19
4
V
V
38.68
19.20
53.75
35.71
20.80
40.52
37.72
17.80
52.26
/
41.76
21.20
54.11
4.3 ABLATION STUDIES
Ablations on 3DGS Tokenizer. We first validate the necessity of the multi-branch 3DGS tokenizer which disentangles the Gaussian attributes. Tab. 4 shows the Top-1 accuracy of each task on the Objaverse dataset with different experiment setups. As shown in Tab. 4, the naive UniGS (Expl) performs less well compared to Exp2, with the 3DGS tokenizer consistently improving accuracy in all tasks. Further analysis (Exp6 vs. Exp7) confirms its role crucial 3D context extractor during feature alignment. Removing this component causes significant performance degradation, demonstrating its fundamental importance. These results show that our tokenizer effectively resolves the problem of 3D context abstraction.
Ablations on Alignment Strategy. We validate modality alignment approach through extensive experiments on image and text alignment strategies. Key findings include:
The multi-view images feature fusion (Exp2 vs. Exp4) strategy effectively bridges the image-3D modality gap, significantly improving image retrieval and zero-shot classification.
Removing the fusion module (Exp4 vs. Exp3) leads to noticeable performance degradation, confirming that simply using more training triplets incurs computational costs and doesn't t have sufficient performance gains.
The 3D-text projector (Exp2 vs. Exp5) successfully aligns 3DGS features with textual representations.
The integrated system (Exp7), combining multi-branch 3DGS tokenizer, 3D-aware image fusion, and 3D-text projector, achieves optimal performance, demonstrating the complementary roles of these modules in mrosll alignment.
More Ablation Studies and Discussions. We also provide detailed discussions and experiments on guidance strategy, scaling-up. and generalization. Please check the supplementary materials for more.
5 CONCLUSION AND DISCUSSIONS
This paper presents TIGAUSSIAN, a framework that establishes robust alignment among text, image, and 3DGS modalities by addressing three challenges in cross-modal 3D understanding. First, our multi-branch 3DGS tokenizer scheme systematically decouples 3DGS intrinsic attributes, enabling better feature representation of 3D structures. Second, the diffusion-enhame mechanism resolves the inherent limitations of single-view observations, ensuring spatial coherence across diverse perspectives and significantly improving the image-3D alignment. Third, the 3D-text projection module effectively bridges the modality gap between continuous 3D feature spaces and discrete text embeddings. Extensive experiments demonstrate -tate-of-the--e art performance in multiple downstream tasks. Our work establishes in vision-lange Lasiun-language applications, which further stimulates the potential of 3DGS as an advanced 3D representation in cross-modal understanding.
multi-view fusion
paradigm for leveraging compact latent 3D token

---

Published as a conference paper at ICLR 2026
Limitations and Future Works. TIGAUSSIAN has been validated to achieve good results across different datasets and tasks. However, there are two limitations that merit attention. (1) General- ization ability: Although conduct experiments on Gaussians from various sources and verified the scalability, there may still be scenarios where our method performs degraded on occluded multi objects real outdoor scenes. (2) Text label dependency: Currently, the quality of text-3D alignment fundamentally relies on training labels. Current benchmarks primarily utilize LLM-generated an- notations, which can introduce bias. Future work could explore hybrid supervision paradigms that combine the efficiency of LLMs with expert data label curation for better fine-graines text retrieval.
REFERENCES
Zhe Chen, Jiannan Wu, Wenhai Wang. Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 24185-24198, 2024.
Christopher Choy, Jun Young Gwak, and Silvio Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 3075-3084, 2019.
Jasmine Collins, Shubham Goel, Kenan Deng, Achleshwar Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang, Tomas F Yago Vicente, Thomas Dideríksen, Himanshu Arora, al. Abo: Dataset and benchmarks for real-world 3d object understanding. In IEEF/CVF Conference on Computer Vision and Pattern Recognition. pp. 21 126-21133, 2022.
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13142- 13153, 2023.
Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, et al. dnternlm-xcomposett Mastering free-form text-image composition and comprehension in vision-language large model. arXiv preprint wrXiv:2401.16420, 2024.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthia Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, pp. 1-21, 2021.
Yutong Feng, Yifan Feng, Haoxuan You, Xibin Zhao, and Yue Gao. Meshnet: Mesh neural network for 3d shape representation. In AAAI Conference on Artificial Intelligence, pp. 8279-827888 02869,888660888 2019.
Souhail Hadgi, Luca Moschella, Andrea Santilli, Diego Gomez, Qixing Huang, Emanuele Rodolå, Simone Melzi, and Maks Ovsjanikov. Escaping plato's cave: Towards the alignment of 3d and text latent spaces. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19825-19835, 2025.
Shi-Min Hu, Zheng-Ning Liu, Meng-Hao Guo, Jun-Xiong Cai, Jiahui Huang, Tai-Jiang Mu, and Ralph R Martin. Subdivision-based mesh convolution networks. ACM Transactions on Graphics, -166) 2022.
Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accurate radiance fields. In ACM SIGGRAPH, pp. 1-11, 2024.
Tianyu Huang, Bowen Dong, Yunhan Yang, Xiaoshui Huang, Rynson WH Lau, Wanli Ouyang, and Wangmeng Zuo. Clip2point: Transfer clip to point cloud classification with image-depth pre-training. In IEEE/CVF International Conference on Computer Vision, pp. 22157-22167, 2023.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4):139:1-139:1 14, 2023.
10

---

Published as a conference paper at ICLR 2026
Han-Hung Lee, Yiming Zhang, and Angel X Chang. Duoduo clip: Efficient 3d understanding with multi-view images. In International Conference on Learning Representations, pp. 1-22, 2025.
Haoyuan Li, Yanpeng Zhou, Tao Tang, Jifei Song, Yihan Zeng, Michael Kampffmeyer, Hang Xu, and Xiaodan Liang. Unigs: Unified -aagungge-mag--333 3d pretraining with gaussian splatting. In International Conference on Learning Representations, pp. 1-21, 2025.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre- training for unified vision-language understanding and generation. In International Conference on Machine Learning, pp. 12888-12900, 2022.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. B Bootstrapping language-image pre-training with frozen image encoders and large language models. In International Conference on Machine Learning, pp. 19730-19742, 2023.
Minghua Liu, Ruoxi Shi, Kaiming Kuang, Yinhao Zhu, Xuanlin Li, Shizhong Han, Hong Cai, Fatih Porikli, and Hao Su. Openshape: Scaling up 3d shape representation towards open-world understanding. Advances in neural information processing systems, 01844860-48877 2023.
Iyy Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations, 2019.
Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. Scaffold-gs: Structured 3d gaussians for view-adaptive rendering. In IEEF/CVF Conference Computer Vision and Puttern Recognition, pp. 20654-20664, 2024.
Xu Ma, Can Qin, Haoxuan You, Haoxi Ran, and Yun Fu. Rethinking network design and local geometry in point cloud: A simple residual mlp framework. In International Conference on Learning Representations, pp. 1-15, 2022.
Daniel Maturana and Sebastian Scherer. Voxnet: A 3d convolutional neural network for real-time object recognition. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 922-928, 2015.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106, 2021.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprini arXiv: arXiv:/807.000 1807.03748 2018.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In /EEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 165-174, 2019.
Charles R Qi. Hao Su, Kaichun Mo. and Leonidas Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In /EEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 652 -600, 2017a.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in Neural Information Processing Systems, pp. 5105-510 14, 2017b.
Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. Langsplat: 3d language gaussian splatting. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 20051-20066 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pp. 0748-8763, 2021.
11

---

Published as a conference paper at ICLR 2026
Kerui Ren, Lihan Jiang, Tao Lu, Mulin Yu, Linning Xu, Zhangkai Ni, and Bo Dai. Octree-gs: Towards consistent real-time rendering with lod-structured 3d gaussians. IEEE Transactions on Patter Analysis and Machine Intelligence, pp. 1-15, 2025.
Yue Ruan, Han-Hung Lee, Yiming Zhang, Ke Zhang, and Angel X Chang. Tricolo: Trimodal contrastive loss for text to shape retrieval. In IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5815 5825, 2024.
Jin-Chuan Shi, Miao Wang, Hlao-Bin Duan, and Shao-Hua Guan. Language embedded 3d gaussians for open-vocabulary scene understanding. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5333,,343,0 2024.
Swetha Sirnam, Jinyu Yang. Tal Neiman, Mamshad Nayeem Rizve, Son Tran, Benjamin Yao, Trishul Chilimbi, and Mubarak Shah. X-former: Unifying contrastive and reconstruction learning for milms. In European Conference Computer Vision, pp. 146-162, 2024.
Shuran Song, Samuel P Lichtenberg, and Jianxiong Xiao. Sun rgb-d: A rgb-d scene understanding benchmark suite. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 567-576, 2015.
Yupeng Song, Naifu Liang, Qing Guo, Jicheng Dai, Junwei Bai, and Fazhi He. Meshclip: Efficient cross-modal information processing for 3d mesh data in zero/few-sher learning. Information Processing and Management, 60(6):103497 2023.
Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. Eva-clip: Improved training techniques for clip scale. arXiv preprint arXiv:2303.15389 2023.
InternLM Team, Internlm: multilingual language model with progressively enhanced capabilities, 2023.
Naftali Tishby, Fernando C Pereira, and William Bialek. The information bottleneck method. arXiv preprint physics/0004007,,22 2000.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Zehan Wang. Ziang Zhang, Hang Zhang, Luping Liu, Rongjie Huang, Xize Cheng, Hengshuang Zhao, and Zhou Zhao. Omnibind: Large-scale omni multimodal representation via binding spaces. In International Conference on Learning Representations, pp. 1 1-21, 2025.
Le Xue, Mingfei Gao, Chen Xing, Roberto Martín-Martin, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, and Silvio Savarese. Ulip: Learning unified representation of language, images, and point clouds for 3d understanding. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1179-11899 2023.
Le Xue, Ning Yu, Shu Zhang, Artemis Panagopoulou, Junnan Li, Roberto Martin-Martin, Jiajun Wu, Caiming Xiong, Ran Xu, Juan Carlos Niebles, et al. Ulip-2: 2: Towards scalable multimodal pre-training for 3d understanding. In JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 27091-27101, 2024.
Xianghui Yang, Huiwen Shi, Bowen Zhang, Fan Yang, Jiacheng Wang, Hongxu Zhao, Xinhai Liu, Xinzhou Wang, Qingxiang Lin, Jiaao Yu, Lifu Wang, Jing Xu, Zebin He, Zhuo Chen, Sicong Liu, Junta Wu, Yihang Lian, Shaoxiong Yang, Yuhong Liu, Yong Yang, Di Wang, Jie Jiang, and Chunchao Guo. Hunyuan3d 1.0: A unified framework for text-to-3d and image-to-34 generation. arXiv preprint arXiv:2411.02200 2024.
Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang. Jie Zhou, and Jiwen Lu. Point-bert: Pre-training 3d point cloud transformers with masked point modeling. /EEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19313-19322, 2022.
12

---

Published as a conference paper at ICLR 2026
Yihan Zeng, Chenhan Jiang, Jiageng Mao, Jianhua Han, Chaoqiang Ye, Qingqiu Huang, Dit- Yan Yeung, Zhen Yang, Xiaodan Liang, and Hang Xu. Clip2: Contrastive caeragg--mmae-eeee pretraining from real-world point cloud data. In IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 15244-15253, 2023.
Pan Zhang, Xiaoyi Dong, Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Haodong Duan, Songyang Zhang, Shuangrui Ding, et al. dxemlm--xcomposse A rision-language large model for advanced text-image comprehension and composition. arXiv preprint arXiv:2309.15112, 2023.
Renrui Zhang, Ziyu Guo, Wei Zhang, Kunchang Li, Xupeng Miao, Bin Cui, Yu Qiao, Peng Gao, and Hongsheng Li. Pointelip: Point cloud understanding by clip. In IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 8552-8562, 2022.
Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang, and Xinlong Wang. Uni3d: Exploring unified 3d representation at scale. In International Conference on Learning Representations, pp. 1-17, 2024a.
Shijie Zhou, Haoran Chang, Sicheng Jiang. Zhiwen Fan, Zehao Zhu, Dejia Xu, Pradyumna Chari, Suya You, Zhangyang Wang, and Achuta Kadambi. Feature 3dgs: Supercharging gaussian splatting to enable distilled feature fields. In IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 21676-2662111 -16855, 2024b.
Ziyang Zhou, Pinghui Wang, Zi Liang, Haitao Bai, and Ruofei Zhang. Cross-modal 3d representation with multi-view images and point clouds. In IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 3728-3739, 2025.
13


Reviews:
[{"review_id": "4jcTAYOBDr", "reviewer": "Reviewer_BqVr", "summary": "This paper proposes TIGaussian, a new text-image-3D contrastive learning paradigm. Based upon the previous work UniGS, it introduces three core modules: (1) decoupled 3DGS tokenizer to avoid entangled 3D representation; (2) mulit-view image fusion to avoid degraded 3D representation; (3) a 3D-text projection module to adapt the 3D latent space for better text-3D alignment. Experiments on Objaverse and ABO show improved performances.", "questions": "1. Why use a pre-trained multi-view diffusion model to generate multi-view images from a single-view perspective? If the data contains multiple text-image-3D triplet of a single object, isn't it more straightforward to fuse the representations of multi-view **real** images? Seems to me the introduction of a diffusion model might also introduce hallucination and might be a potential bottleneck of this method.\n\n2. What is the core difference between 3DGS-based 3D contrastive learning approaches and point-cloud-based or mesh-based approaches? The authors are encouraged to add related discussions and show performance comparisons across different approaches in the paper.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-structured and easy-to-follow. It focuses on improving different designs of current 3DGS-based text-image-3D contrastive method and shows clear improvement.\n\n2. The multi-vew image fusion idea makes intuitive sense to me. Forcing the alignment of a 3D object and a single-view 2D image is clearly suboptimal.", "weaknesses": "1. The benchmark comparison is insufficient. Only three baseline methods are listed in Table 1/2. Strong competitors like ULIP [1], OpenShape [2], and MixCon3D[3] should also be included. For example, OpenShape and MixCon3D score a top-1 accuracy of 46.8 and 52.5 on Objaverse-LVIS, which is significantly better than the 41.76 top-1 accuracy of TIGaussian, making one wonder the effectiveness of the proposed approach.\n\n2. Similarly, results on other important benchmarks like ModelNet40 and ScanObjectNN should be reported, as in OpenShape.\n\n3. The image fusion idea is not new. MixCon3D has already shown the effectiveness of aggregating multi-view image features before alignment, and thus make the contribution of this work less novel.\n\n\n**References**\n\n[1] Xue, Le, et al. \"Ulip: Learning a unified representation of language, images, and point clouds for 3d understanding.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.\n\n[2] Liu, Minghua, et al. \"Openshape: Scaling up 3d shape representation towards open-world understanding.\" Advances in neural information processing systems 36 (2023): 44860-44879.\n\n[3] Gao, Yipeng, et al. \"Sculpting holistic 3d representation in contrastive language-image-3d pre-training.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T3J9XxdPr7", "reviewer": "Reviewer_hpbf", "summary": "TIGAUSSIAN, a framework for text-image-3D Gaussian (3DGS) multimodal alignment, is proposed in this research. The main contribution is to solve the problems of single-view bias and attribute entanglement encoding that exist in current multimodal approaches for 3D settings. To make up for the shortcomings in single-view 3D sceneries, they have developed a diffusion-enhanced multi-view fusion module and a multi-branch 3DGS tokenizer module that decouples from the spatial, appearance, and morphological aspects of 3DGS. Finally, an engineered 3D-text projection module is used to align 3D features with text features. Tests on datasets including Objaverse, ABO, and SUN RGBD show that this approach performs noticeably better than baseline approaches like CLIP2, Uni3D, and UniGS in tasks like open-scene identification, text-3D/image-3D retrieval, and zero-shot categorization.", "questions": "1. How do the issues with the TIGAUSSIAN framework presented in this paper specifically manifest themselves in the generalization testing in complex scenes (like occluded multi-targets and real outdoor scenes) and the rationality verification of important parameters (like the number of Transformer layers in the 3D-text projection module and the number of multi-view generations)? What fixes exist for the issues described above?\n2. The experiment's Objaverse, ABO, and SUN RGBD datasets are all openly accessible. Do any self-constructed datasets exist? Will they become open-source if that is the case?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The TIGAUSSIAN framework proposed in the article is highly targeted. It tackles the fundamental issues of attribute entanglement encoding and single-view bias in 3D multi-modal alignment tasks by creating multi-modal processing modules like the multi-branch 3DGS Tokenizer. It offers a fresh approach to text-image-3DGS multi-modal alignment, surpassing the drawbacks of current techniques.\n\n2. They rigorously validate the effectiveness of the method, and conducts verification on three major datasets, including Objaverse, by designing four core person scenarios such as zero-shot classification. The baseline approach covers popular 3D representation techniques and uses ablation tests to demonstrate the need for each module design.\n\n3. With a new design, a well-defined experimental procedure, and promises to make the source code publicly available, this approach offers a reproducible reference for future 3D multimodal related work and actively fosters the growth of related disciplines in 3D multimodal research.", "weaknesses": "1. The article omits information about the experiment's parameter base and indicators, such as the number of multi-view generations (6) and the 3D-text projection module's parameter sensitivity, which have not been confirmed by tests. Additionally, when it is actually implemented, it does not report the pertinent information (such the inference delay on A100 GPU).\n\n2. It is unclear if baseline models (such UniGS and Duoduo-CLIP) have implemented the same preparation procedures as TIGAUSSIAN (like preprocessing in the downsampling stage) when compared to baseline approaches. The persuasiveness of the outcome is affected by the comparing conditions.\n\n3. Its specific performance in multi-objective, real-world outdoor, and other complicated 3D settings cannot be verified due to the absence of generalization testing in complex scenarios. Additionally, it is not stated how diverse the 3DGS data sources were used in the trials, which makes it impossible to completely illustrate how generalizable the strategy is in arbitrary situations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Tf06nHVTAm", "reviewer": "Reviewer_Xq4U", "summary": "The paper proposes a framework to train a unified text-image-3D features by decoulping the parameters of 3D gaussian. It introduces a multi-branch 3DGS tokenizer to fuse the 3D features within various perspectives. In addition, it proposes a tri-modal alignment strategy to interact with multi-view generated images. The experiments prove that it achieves impressive improvements across retrieval and scene understand tasks.", "questions": "- Why use a single image generation method to obtain multi-view images instead of directly rendering from the model?\n\n- What backbone does ULIP2 use in Tab. 1?\n\n- Appendix Table 2 shows that the pre-trained point cloud encoder has a significant impact on the results, which weakens the overall contribution of the method. What is the significance of using 3D Gaussian to compare point clouds?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The paper achieves impressive improvement across various tasks.\n\n- The writing is easy to understand and mask sense.", "weaknesses": "- Lack of comparisons. One of the core contribution is 3D-aware image feature fusion. However, the usage of multi-view rendering images has been proposed in the JM3D [1], which mitigates the contribution. The authors should either discuss the differences or include a comparison.\n\n- Lack of the experiments of perception. Whether JM3D or ULIP has experiments about 3D Res or Object detection to support the ability in sparse perception. The paper needs the similar experiments.\n\n- Contribution. In Tab.4, the comparison between the second and fifth lines shows that the 3D-text projector did not perform well, which impacts the validity of the author's claimed significant contribution.\n\n[1] Beyond First Impressions: Integrating Joint Multi-modal Cues for Comprehensive 3D Representation, MM 23", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DewmiJRuTi", "reviewer": "Reviewer_yocs", "summary": "This paper introduces TIGAUSSIAN, a novel framework for aligning text, image, and 3D modalities, specifically using 3D Gaussian Splatting (3DGS) representations. The core contributions are threefold: 1) a multi-branch 3DGS tokenizer that disentangles and separately encodes geometric, appearance, and morphological attributes of Gaussian primitives to create a more effective latent representation; 2) a diffusion-enhanced multi-view fusion mechanism for image-3D alignment that generates multiple views from a single image to create a more holistic, 3D-aware feature representation, mitigating single-view bias ; and 3) a 3D-text projector module that uses a query transformer to map 3D features into the text embedding space for improved alignment. The authors demonstrate through extensive experiments on datasets like Objaverse, ABO, and SUN RGBD that TIGAUSSIAN achieves state-of-the-art performance on various downstream tasks, including zero-shot classification, text-3D retrieval, and image-3D retrieval.", "questions": "1. Regarding the scalability: The authors mention that Objaverse contains 146k objects. Have the authors considered or attempted to pre-train TIGAUSSIAN on larger-scale dataset (full objeaverse) to test the limits of the proposed architecture? \n2. Regarding the 3DGS tokenizer: The paper mentions using FPS to downsample to 1024 Gaussians. How sensitive is the model's performance to the number of sampled Gaussians? Does the performance saturate at 1024, or could further improvements be gained with more Gaussians at the cost of computation?\n3. Regarding the multi-view fusion: The framework generates 6 views using a diffusion model. What was the rationale for choosing 6 views? Is there a performance trade-off with the number of generated views? For instance, would using 3 views significantly degrade performance, or would 12 views provide a notable boost?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. **Writing**: This paper is clear and easy to follow.\n2. **Cross-Modal Alignment Strategies:** The paper introduces sophisticated strategies for both image-3D and text-3D alignment that address key challenges. The diffusion-enhanced multi-view fusion for image-3D alignment is a clever way to overcome the ambiguity and information loss inherent in single-view representations without requiring actual multi-view data during inference. Similarly, the 3D-text projector is a principled approach to bridge the modality gap, which is a persistent challenge in this area.\n3. **Empirical Results:** The method is thoroughly evaluated on multiple benchmarks (Objaverse, ABO, SUN RGBD) and across a variety of tasks (zero-shot classification, cross-modal retrieval, few-shot learning).", "weaknesses": "1. **Lack of Large-Scale Dataset Validation:** The primary weakness of this work is the scale of the datasets used for the primary training and validation. While the mentioned Objaverse, with 146k objects, is a respectable size. The current experiments, while strong, do not fully demonstrate the scalability of TIGAUSSIAN on entire Objaverse with 800k objects. Training and evaluating on a significantly larger dataset would provide more convincing evidence of the model's robustness and generalizability.\n2. **Comparisons to other methods:** It will be better to report the performance comparisons to ReCon, ReCon++ of ShapeLLM, TAMM on used datasets for completeness.\n3. **Complexity and Computational Cost:** The proposed framework introduces several new components. This adds complexity and likely significant computational overhead compared to simpler baseline models. While the performance gains are clear, a discussion on the trade-offs in terms of computational cost, training time, and memory usage would be beneficial. For instance, UniGS report the ablation study on computational cost.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)