The Indra Representation Hypothesis for Multimodal Alignment
Authors: Jianglin Lu
Hailing Wang
Kuo Yang
Yitian Zhang
Simon Jenni
Yun Fu
Venue: NeurIPS 2025 poster ()
Keywords: Foundation Models
Representation Learning
Multimodal Alignment
Representation Convergence
Primary area: deep_learning

Abstract: Recent studies have uncovered an interesting phenomenon: unimodal foundation models tend to learn convergent representations, regardless of differences in architecture, training objectives, or data modalities. However, these representations are essentially internal abstractions of samples that characterize samples independently, leading to limited expressiveness. In this paper, we propose The Indra Representation Hypothesis, inspired by the philosophical metaphor of Indra’s Net. We argue that representations from unimodal foundation models are converging to implicitly reflect a shared relational structure underlying reality, akin to the relational ontology of Indra’s Net. We formalize this hypothesis using the V-enriched Yoneda embedding from category theory, defining the Indra representation as a relational profile of each sample with respect to others. This formulation is shown to be unique, complete, and structure-preserving under a given cost function. We instantiate the Indra representation using angular distance and evaluate it in cross-model and cross-modal scenarios involving vision, language, and audio. Extensive experiments demonstrate that Indra representations consistently enhance robustness and alignment across architectures and modalities, providing a theoretically grounded and practical framework for training-free alignment of unimodal foundation models. Our code is available at https://github.com/Jianglin954/Indra.

The Indra Representation Hypothesis for Multimodal Alignment
Jianglin Lu Hailing Wang Kuo Yang' Yitian Zhang' Simon Jenni Yun Ful.2
!Deparmeent of Electrical and Computer Engineering, Northeastern University Khoury College of Computer Science, Northeastern University Adobe Research
Abstract
Recent studies have uncovered an interesting phenomenon: unimodal foundation models tend to learn convergent representations, regardless of differences in archi- tecture, training objectives, or data modalities. However, these representations are essentially internal abstractions of samples that characterize samples independently, leading to limited expressiveness. In this paper, we propose The Indra Representa- tion Hypothesis, inspired by the philosophical metaphor of Indra's Net. We argue that representations from unimodal foundation models are converging to implicitly reflect shared relational structure underlying reality, akin to the relational ontol- ogy of Indra's s Net. We formalize this hypothesis using the V-enriched Yoneda embedding from category theory, defining the Indra representation as a relational profile of each sample with respect to others. This formulation shown to be unique, complete, and structure- preetervii under a given cost function. We instan- tiate the Indra representation using angular distance and evaluate it in cross-model and cross modal scenarios involving vision, language, and audio. Extensive experi- ments demonstrate that Indra representations consistently enhance robustness and alignment across architectures and modalities, providing theoretically grounded and practical framework for training- free alignment of unimodal foundation models. Our code is available at
Introduction
Through large-scale pretraining, foundation models have emerged transformative paradigm in artificial general intelligence, demonstrating impressive progress across diverse domains, such as natural language processing [11, 25, 46), computer vision [47, 77, 61, 12], and speech processing [66, 2]. These unimodal models are typically trained on web-scale datasets and acquire generalized representations that can be adapted to broad range of downstream tasks. The representative models, such as BERT [11] for language, VIT [12] for vision, and Wav2Vec [66] for audio, have demonstrated promising performance within their respective domains.
Since real-world information is inherently multimodal (text, images, and audio frequently co-occur and complement each other), relying on a single modality for understanding is typically insufficient. To extend unimodal foundation models for modal tasks, a growing body of research has exploited strong unimodal encoders as the core components to build multimodal systems [62, 42, 63). They assume that unimodal models are able to provide specialized, high-quality and high-level representations for each individual modality, which can then aligned, fused, or bridged to enable multimodal interactions [41, 45J. A common strategy to achieve multimodal understanding is to align unimodal outputs in a shared representation space through cross- cross-modal objectives [62, 41, 45, 55].
Equual Contribution. Corresponding author: Jiangl Hianglinnu@@uttoooo.ooo com.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

This usually relies on external mechanisms, such as alignment losses, fusion modules, and prompt tuning, thus requiring large-scale datasets and extensive retraining for modality alignment.
Interestingly, recent studies [60, 58, 45, 67] suggest that powerful unimodal models may already exhibit latent cross-modal capabilities, as the representations they produce (when grounded on the same physical entity) tend to describe the same underlying semantics from different sensory perspectives. Previous evidence has revealed that adding only a single linear transformation is capable of bridging an auditory model with a large language model (LLM)[59), integrating a vision model into an LLM [55], or conversely, stitching an LLM towards a vision model [36]. Even without retraining. well-pretrained vision encoders exhibit high semantic similarity with language encoders [52]. Further studies [29, 34, 27, 49] have revealed that models trained on different data modalities converge, as different models are all trying to arrive at representation of reality. Thus, unimodal models may encode modality-agnostic representations in abstract representational space, even without explicit alignment. While conceptually appealing. the specific form of convergent representations and their eventual convergence targets remain elusive and largely unexplored.
In this paper, we posit that the underlying convergent representation is inherently the Indra Representation-a conceptual abstraction inspired by the philosophical metaphor of Indra's Net. Originating in ancient Buddhist philosophy, Indra's ' Net describes a vast, infinite web of jewels, each reflecting all others. Every jewel both a part and reflection of the whole, suggesting that all phenomena are interdependent, mutually defining. and inherently connected. We draw an analogy between this worldview and the notion of representation convergence, and introduce The Indra Rep- resentation Hypothesis: well-trained unimodal models tend to produce convergent representations that implicitly reflect a shared relational structure underlying reality, echoing the relational ontology of Indra's 's Net. In this view, the representation of each entity is not defined in isolation, but rather emerges from its relational context, i.e., its reflections of all other entities.
To explore this hypothesis, we introduce a theoretical definition of the Indra representation grounded in category theory [33]. Specifically, we define it as the V-enriched Yoneda embedding of sample within a category enriched over a Cost-category This formulation effectively maps each sample to its covariant foonfuunsorss in the sample category, thereby encoding its relational profile within the structure of the dataset. We provide theoretical guarantees that the Indra representation is unique, complete, and structure-preservingg offering a principled foundation for effectiveness. In particular, we prove that it uniquely and completely characterizes each sample within the relational structure induced by a given cost function, while preserving essential properties of that structure.
To instantiate this theory in practice, we adopt angular distance as the cost function, yielding simple yet powerful realization of the Indra representation. This concrete formulation enables empirical evaluation and allows us to investigate how Indra representations can uncover and support the latent cross-modal capabilities of unimodal models. We validate our approach across range of scenarios involving cross- moossl and cross modal understanding, including dingle-modaiig settings, vision- language pairs, and serech-language pairs. Extensive experiments demonstrate the effectiveness and generality of the proposed Indra representation across different architectures and modalities.
2 Preliminaries
2.1
Indra's Net
Indra's Net is a philosophical metaphor originating in ancient Indian and Mahāyāna Buddhist thought, particularly from the Avatamsaka Sütra [9, 22]. It is used to symbolize the universe as web of interdependent connections among all of its members, expressing the concept of interconnectedness, non-duality, and the interpenetration of all phenomena. Francis H. Cook describes it as [10]:
Imagine vast, infinite cosmic net belonging to the god Indra. At each node of the net is a jewel or crystal. Each jewel reflects every other jewel in the net, and in each of those reflections are further reflections of all other jewels, recursively and infinitum.
The metaphor of Indra's Net resonates deeply with the foundational principles across diverse dis- ciplines. For instance, Gergen et al. [17] posit that identities, thoughts, and actions are not solely products of isolated minds but are co-constructed through interactions and relationships with others. Markus et al. [53] propose the theory of interdependent self-construal, emphasizing that the self is

---

defined relationally through one's social roles, group memberships, and interpersonal obligations. In physics, field theory [14, 54] illustrates relational structure where the field any point depends on all sources throughout space, a concept reminiscent of Indra's Net, in which each jewel reflects and is reflected by all others. Similarly, modern particle physics [18, 4] reveals the properties of an elementary particle through its interactions with other particles. In linguistics, the linguistic principle articulated by J.R. Firth [15], "You shall know a word by the company it keeps", asserts that a word's meaning is derived from its co-occurrence with other words. This principle forms the basis for modern language models like Word2Vee [56] and its successors, Furthermore, Kasulis [32] suggests DNA as better image of Indra's Net, where every cell contains the blueprint for the whole organism.
2.2 Representation Convergence
Recent studies [29, 76, 20, 74, 40] have revealed a striking phenomenon: unimodal foundation models tend to learn convergent representations, regardless of their architectures, training objectives, or data modalities. For example, Bürger et al. [5] show that a two-dimensional representation of truth emerges universally across LLMs of varying sizes and from different model families. Roeder et al. [65] prove that a wide class of discriminative and autoregressive models are identifiable function space up to a linear transformation. Tan et al. [68] find strong correlations in both in-distribution and out-of-distribution steerability between LLaMA [69] and Qwen [3]. Huh et al. [29] attribute this convergence to shared goal: approximating an underlying representation of reality. Khosla et al. [34] argue that both artificial and biological systems converge toward representations that capture the causal structure of the world. Hosseini et al. [27] further observe that high- performing artificial neural networks and biological brains tend to develop similar internal representations under naturalistic training conditions. Additional evidence and analyses on representation convergence can be found in the comprehensive survey [49].
Despite this emerging consensus, it remains unclear how these representations converge and what they ultimately converge to. Notably, prior studies rely primarily on model outputs (embeddings) as proxies for representations, but these representations suffer from @ structural myopia: representations are typically treated as isolated carriers of information, ignoring structural interrelations within the broader data manifold; @ limited expressiveness: unimodal representations often exhibit inferior quality in matching and alignment compared to those from multimodal foundation models; and 3 dimensional incompatibility: representations across models and modalities often differ in dimension- ality, thus requiring additional post-procct post-processing for cross-modes matching. In light of these challenges, we argue that representations from model outputs do not reflect the final converged form, but instead serve as the foundation upon which such a form can be built. In the next section, we introduce a novel representation hypothesis inspired by the metaphor of Indra's 's Net to hypothesize the concrete structure of convergent representations and illuminate what they ultimately converge to.
3
Methodology
3.1 The Indra Representation Hypothesis
In this paper, we advocate a shift in perspective on representation convergence, inspired by the metaphor of Indra's 's Net: sample should be represented not in isolation, but through its pattern of relationships to other samples. In this view, representations emerge from a structure of mutual interdependence. We formalize this perspective through the Indra Representations Hypothesis:
The Indra Representation Hypothesis: Neural networks, trained with different objectives on different data and modalities, tend to learn convergent representations that implicitly reflect a shared relational structure underlying caditt-ppraalel to the relational ontology of Indra's 's Net.
This hypothesis posits that unimodal foundation models, after extensive pretraining, tend to produce representations that converge to capture the inherent relational structure of reality structure charac- terized by interdependence, contextuality, and mutual influence. However, current methods that treat model outputs as final representations fail to reflect this structure. These methods typically emphasize individual embedding information while neglecting the crucial relational patterns between samples. In the next section, we introduce the Indra Representation, a novel representation framework inspired by the metaphor of Indra's Net, to reveal the underlying relational structure. In this framework.
3

---

representations are not independent embeddings but mutually reflective entities woven into a web of interdependent relationships, revealing the deeper relational structure that underpins the data.
3.2 From Metaphor to Theory
Indra's 's Net is philosophical metaphor that symbolizes the interconnectedness of the universe. It aligns with the foundational principles in modern science, as mentioned in Section 2.1. To translate this philosophical insight into representation learning, we first introduce the Yoneda Lemma and its corollary, which provide the theoretical foundation for defining our proposed Indra representation.
Lemma 1 (Yoneda Lemma [33, 64)). Let C be a locally small category, A be an object in C, and F C + Set be a functor from C to the category of sets. Then, there exists bijection, natural in both A and F, between the set of natural transformations from the hom-finctor AA = HHoe (A, -) to F, and the set F(A). This bijection is given by:
Nat(h F) = F(A).
(1)
Corollary (Yoneda Embedding [33, 64)). For any two objects A, B a locally small category C. there is bijection:
Nat(Home A, -), Home (F) Home [B, A). This demonstrates that the functor Y C -> Set), defined by Y(A) hA Home -), is faithful. This functor Y is known as the Yoneda embedding.
(2)
The Yoneda Lemma provides a profound understanding of how an object in a category is characterized by its relationships (morphisms) with all other objects, rather than by its internal properties. Its corollary further shows that any locally small category C can be embedded into a category of presheaves on C. To introduce our Indra representations, we give the following definitions:
Definition 1 (Sample Category). Let X' be set of samples, possibly infinite. The sample category consists of: ① Objects: Ob(C) X. 2
enriched over the Cost-category V = 2,). Hom-objects: for every Xi, Xj Ob(C), the hom-object C(X;, Xj) is given by cost function d(X,, X,) E [0. oo), which is an object in @ Identity: for all X; E OB(C), the identity morphism idx; + c(X,,X,) in V is 0 - d(X,, X.), where d(),,X,) all X1, X,, E Ob(C), the composition morphism Mx.X,,Xx
O. @ Composition: for C(X,, Xx) @ C(X,, Xz) - C(X,, Xx) in V is: d(X,. Xx) + d(X,,X,) X,) + d(x,, Xx). This morphism exists in V if and only if d(X,, Xx) + d(X,, X,) 2 d(X,, Xx), which is precisely the triangle inequality.
Definition 2 (V-enriched Yoneda embedding). Let [CDD V] be the category of V D-presheaves on C. The V-erriched Yoneda embedding is V-functor Y : - [C Vl. For each object X, E Ob(C), Y(,,) d(x, j. X.) for any X, E C is the V V-presheaf hx, : Cop + V defined by: (X,) C(X, (C (Y Y(X,))
For every X,, X5 E Ob(C), defines a map Xx,,x; C(Xi,
Theorem 1. The V-enriched Yoneda embedding Y C + [C for the sample category C enriched over V ([0, 2, 0, +) with the cost function d is V-fully faithful.
The sample category C actually forms Lawvere metric space [39], and the corresponding V-enriched Yoneda embedding maps each sample X, to functor Y(,)), which captures the cost profile from all other samples to X, Theorem I shows that the sample category can be fully and faithfully represented within a category of V-presheaf, preserving its entire structure including its metric information. In other words, each sample X; can be uniquely represented by its cost vector d(-, X.) with all samples. Based on these, we introduce the Indra representation and state the following theorems:
Definition 3 (Indra Representation). For each sample X, E Ob(C), we define its Indra represen- tation the V-functor C(X,, -), i.e., the collection of values obtained by evaluating it under the (covariant) V-enriched Yoneda embedding on all objects of the category C.
Proposition 1. If two samples X,, X, E &bb(C) have V-naturally isomorphic Indra representations and the cost cost function d satisfies the To separation axiom, then X;- = =X,-
Theorem 2. For any V-functor P C - V. the V-hom-object of V-nutural transformations from the Indra representation of sample X4 to P, denoted by [C, (),,, -), P), is V-isomorphic to P(X.). The relational structure among objects in the sample category is preserved and reflected in the relationships between their Indra representations.
Corollary 2.

---

The Indra representation of a sample X,, defined as the V-functor C(,,, -), can be interpreted as relational profile of X,, that is. the cost from X; to every other sample in the category. Proposition I establishes that the Indra representation is faithful encoding: no two distinct samples share the same representation. Theorem 2 further shows that the Indra representation is complete, in the sense that it encapsulates all the information needed to determine how distances from X; behave under any admissible distance assignment. Furthermore, Corollary demonstrates that the relationships between samples are in one-to-one correspondence with the relationships between their Indra representations.
3.3 Instantiation of Indra Representation
We now demonstrate how instantiate the Indra representation for real dataset X' {XI, Xn)
consisting of n samples. We define the object set of the enriched category as Qb(C) = X and specify the hom-object C( (X,, X)) as the cost d( X; X,) between samples X; and Xj. VX,, X3 E Oh(C). To define a valid Indra representation, the cost function dl must satisfy two properties: @ d(X; X;) = 0 for vX; X; and @ d(X,, Xx) d(X;, + ,(X;. for vX,, X,, E X. Several distance metrics satisfy these conditions. In this work, we adopt simple yet effective choice by defining d(X,, Xx) as the angular distance between the embeddings of X, and Xj:
d( X₁) = arccos (1(X)) (17(03)11XX) ((X)) (T((X))I1X(XX)))
VX,; XX EX
where f X' Rd is a modality-specifie foundation model, f(X;) denotes the internal represen- tation of X; produced by f. and d is the output dimensionality of the model. The use of angular distance ensures that d defines valid Lawvere cost function. Given this cost function, the Indra representation of each sample X; E X is the covariant
C(X,, : X [0, o0), X, + d(X,, Xz).
This distance-based embedding forms principled and interpretable representation. In the finite case, this can be written as C(X,, -) = Xi), d(X,, Xn)]. which captures the relational profile of X, with respect to all other samples. Notably, when the relational profile of each sample is computed using only a small set of representative landmarks, this formulation closely aligns with the approach proposed in [57].
3.4 Relational Matching across Modalities
Our hypothesis in Section 3.1 posits that unimodal foundation models learn convergent representations that capture the shared relational structure underlying reality. The proposed Indra representations are designed reflect this inherent structure and can thus be leveraged to improve cross-modal understanding. To demonstrate how Indra representations facilitate relational matching across modalities, we consider a dataset D {U,, k.)) of n samples, where U, E l and Qi Q correspond to instances from two distinct modalities, and D C x Q. For single-modality scenarios, f du IR." and y : Ω + d to extract modality-speciff embeddings, where dx and d are the embedding dimensionalities of the two models, respectively. Based these embeddings, we construct the Indra representations 1' and _ for each modality as follows:
define U, = Qi, Vi E (1....,n), We use pretrained foundation models
K d(U,, U,). I = d(Q,, Q₁), Vi,j E (1....,n).
(5)
where the cost function d is defined in Equation 3. In practice, we may apply post-processing operations such as sparsification and normalization to enhance robustness:
1 o ie_ operator(
(6)
where operator()) denotes the chosen post-processing function. Unlike traditional representation approaches that reflect only internal characteristics of samples, the proposed Indra representations act as external representations, where each vector captures interdependencies by encoding the sample's s relative profile within the dataset.
4 Experiments
To comprehensively assess the effectiveness of our Indra representation, we perform evaluations across range of settings, including unimodal vision, vasion-language, and languae tasks.
5

---

Table 1: Accuracy (%) on CIFAR- 10 and CIFAR-100 under different Gaussian noise levels.
CIFAR-10
a=0.0
0=3.0
0=5.0
0=7.0
CIFAR-100
o=0.0
o=3.0
o=5.0
a=7.0
ViT
93.98
87.75
79.77
68.15
ViT
79.45
54.69
35.76
27.45
Indra
94.84
89.51
80.84
68.71
Indra
80.09
69.00
51.59
32.74
Convnext
97.00
85.89
80.10
65.85
Convnext Indra
85.77
62.79
34.39
21.28
Indra
97.21
92.86
81.59
66.64
85.64
72.16
51.51
30.25
Dinov2
99.
95.21
85.57
76.54
Dinov2 Indra
91.97
82.21
63.06
40.16
Indra
99.14
96.87
89.73
77.92
91.93
84.83
74.29
58.67
Table 2: Accuracy (%)
on Office-Home dataset under different Gaussian noise levels.
Art
0=0.0
0=3.0
d=5.0
0=7.0
Clipart
0=0.0
0=3.0
d=5.0
0=7.0
ViT
80.25
64.40
44.03
22.63
ViT
73.20
50.40
28.64
15.23
Indra
79.63
65.02
43.62
27.57
Indra
69.76
54.98
33.
18.21
Convnext
89.71
62.76
27.98
12.14
Convnext
83.62
54.07
20.85
09.74
Indra
87.86
59.88
28.81
14.20
Indra
82.70
57 .85
25.09
11.34
Dinov2
87.65
73.05
46.91
27.78
Dinov2
88.43
75.14
51.09
31.04
Indra
87.04
70.99
47.53
27.37
Indra
87.29
76.63
54.75
33.56
Product
0=0.0
d=33.0
0=5.0
0=7.0
Real
0=0.0
d'=3.0
0=5.0
0=7.0
ViT
92.34
80.74
61.15
35.25
ViT
89.22
82.11
60.09
35.32
Indra
89.75
81.53
64.08
40.77
Indra
87.16
83.49
63.65
40.48
Convnext
96.62
84.91
44.26
19.37
Convnext
93.46
82.11
38.30
17.78
Indra
96.73
85.92
45.61
22.18
Indra
93.35
84.63
40.71
19.61
Dinov2
96.73
93.24
83.33
60.70
Dinov2
92.78
87.39
71.44
48.51
Indra
96.40
92.79
84.46
60.59
Indra 92.89
88.53
73.17
49.89
4.1 Evaluation on Single Modality
Datasets. We first conduct classification tasks on the CIFAR-10 [37], CIFAR-100 [37], and Office- Home [72] datasets. For CIFAR-10 and CIFAR-100, we evaluate classification accuracy across four distinct domains: Art, Clipart, Product, and Real- World, using an 80/20 split for training and testing. Each domain exhibits unique visual styles and distribution shifts, making the dataset widely used benchmark for evaluating the robustness and generalization of vision models in object recognition tasks. Across all datasets, we adopt logistic regression (i.e., linear probing) to assess the quality of the extracted representations.
the standard data splits. For Office-Home,
Foundation Models. For vision models, evaluate ViT [12], Convnext [77], and Dinov2 [61].
Evaluation Metrics. We assess model classification accuracy (%) using ground-truti labels. To investigate the robustness of Indra representations we inject Gaussian noise into the features with varying standard deviations E {0.0, 3.0, 5.0, 7.0]. For each noise level, we perturb the features accordingly and train a linear classifier on the noisy representations. This allows to assess how classification performance degrades as the feature representations are increasingly corrupted by noise.
Analysis. In Tables 1 and 2, we report the classification results on the 100AR-10, CIFAR- 100, and Office-Home Home datasets. The results clearly show that stronger backbone models (e.g., Dinov2) lead to better performance for Indra representations across all noise levels. For instance, on CIFAR-100 with 000.0, our Indra representations achieve 91.93% accuracy using Dinov2 as the backbone, compared to 85.64% with Convnext and 80.09% with ViT. This performance gap persists and even widens under higher noise: at a 7.0, Indra representations with Dinov2 maintain 68.67%, while Convnext and ViT drop to 30.25% and 32.74%, respectively. In addition, Gaussian noise

---

Table 3: Performance on image-text datasets D using different representations R (Orign: original, Indra: Indra representation) with various vision (Vis-E) and language (Lan-E) models.
Top-5
Top-10
Top-30
Top-50
D
Via-E
Lan-E
R
T-+1
1-T
T-1
1-+T
T-1
1-T
T-+1
I-+
CLIP-I
CLIP-T
Drign
1.420
1.381
2.734
2.661
7.634
7.470
12.212
11.986
Orign 0.482
0,483
0.967
0.966
2.911
2.905
4.863
4.846
ViT
BERT
Indra
0.663
0.832
1.303
1.613
3.787
4.426
6.199
7.036
Orign 0.486
0.491
0.970
0.981
2.912
2.927
4.853
4.874
ViT
Roberta
Indra
1.048
0.880
2.065
1.749
5.970
5.149
9.702
8.446
MS MS-COCO
Drign 0.396
0.474
0.837
0.950
2.603
2.851
4.412
4.755
Convnext
BERT
Indra
0.612
0.537
1.127
1.022
3.182
2.875
5.242
4.783
Orign 0.492
0.480
0.985
0.964
2.962
2.889
4.940
4.824
Convnext
Roberta
Indra
1.005
0.616
1.930
1.217
5.247
3.538
8.267
5.790
Orign 0.496
0.473
0.991
0.947
2.969
2.852
4.936
4.760
Dinov2
BERT
Indra
0.540
0.539
1.123
1.022
3.4
2.872
5.277
4.804
Orign 0.468
0.490
0.945
0.982
2.859
2.949
4.779
4.914
Dinov2
Roberta
Indra
1.016
0.949
1.978
1.863
5.603
5.370
9.021
8.766
CLIP-I
CLIP-T
Drign 1.357
1.325
2.556
2.499
6.860
6.717
10.795
10.584
Orign Indra
0.479
0.474
0.956
0.947
2.864
2.844
4.769
4.742
ViT
BERT
0.701
0.667
1.375
1.293
3.960
3.712
6.449
6.069
Orign 0.484
0.483
0.966
0.963
2.891
2.886
4.814
4.805
ViT
Roberta
Indra
0.924
0.727
1.792
1.419
5.014
4.102
8.011
6.700
NOCAPS
Orign Indra
0.449
0.451
0.904
0.910
2.754
2.752
4.640
4.604
Convnext
BERT
0.415
0.485
0.906
0.971
2.911
2.907
4.821
1.845
Orign Indra
0.472
0.462
0.944
0.926
2.833
2.781
4.721
4.647
Convnext
Roberta
0.764
0.557
1.472
1.102
4.079
3.338
6.494
5.526
Orign Indra
0.465
0.439
0.928
0.883
2.701
2.674
4.500
4.485
Dinov2
BERT
0.566
0.485
1.065
0.971
3.179
2.907
5.238
4.845
Orign 0.497
0.467
0.993
0.936
2.969
2.822
4.938
4.712
Dinov2
Roberta
Indra
0.830
0.774
1.604
1.513
4.549
4.324
7.335
7.019
increases, our Indra representations consistently retain higher classification accuracy compared the original representations, highlighting their robustness the classification tasks. The performance gains of Indra representations hold across multiple backbone architectures (i.e., ViT, Convnext, and Dinov2), indicating the broad applicability of the proposed method.
4.2 Evaluation on Vision Language Modalities
Datasets. We adopt two widely used image-text datasets: MS-COCO [44] and NOCAPS [1] to evaluate performance on vision and language modalities. MS-COCO serves standard benchmark for image captioning and retrieval tasks, while NOCAPS poses greater challenge due to its focus on novel object categories. We use the validation sets of both datasets for evaluation.
Foundation Models. We use the same vision models as in Section 4.1. 1. For language models, we evaluate BERT [11] and Roberta [46], both of which are pretrained independently on unimodal data without cross-modal alignment. We include CLIP [62] as the aligned baseline for evaluation.
Evaluation Metrics: We adopt CLIPScore [26] the evaluation metric, which measures semantic alignment between image and text based on the cosine similarity of their embeddings within the CLIP multimodal space. We report Top-k matching accuracy (I: (5, 10, 30, 50}) in both text-to-image (T-+)) and image-to-texs (I++T) tasks.
Analysis. Table 3 compares the performance of original embeddings versus Indra representations across both datasets using various combinations of vision and language models. The results clearly demonstrate that Indra representations lead to consistent performance gains across different archi-
7

---

Table 4: Performance audio-text dataset D using different representations R (Orign: original, Indra: Indra representation) with various audio (Aud-E) and language (Lan-E) models, where *-b and * refer to the base and large versions of model *, respectively.
Top-5
Top-10
Top-30
Top-50
D
Aud-E
Lan-E
R
T-+A
A-+T
T-*A
A-+T
T->A
A-+T
T->A
A-+T
CLAP-I
CLAP-1 -T
Orign
1.062
1.836
2.046
3.611
5.726
10.146
9.204
16,225
Orign
0.319
0.072
0.670
0.276
2.214
1.409
3.655
2.750
wav2vec-b
Roberta
Indra
0.413
0.578
0.819
1.209
2.506
2.953
4.225
4.692
Orign 0.418
0.308
0.864
0.634
2.548
2.194
4.200
3.738
wav2vec-1
Roberta
Indra
0.363
0.578
0.908
1.243
2.783
2.956
4.640
4.318
TIMIT
Orign Indra
0.328
0.328
0.659
0.648
2.080
1.920
3.518
3.208
wavlm-b
Roberta
0.436
0.578
0.913
1.241
2.493
2.976
4.227
4.338
Orign Indra
0.472
0.426
0.912
0.876
2.450
2.518
3.997
4.148
wavlm-1
Roberta
0.504
0.578
0.971
1.229
2.693
2.967
4.387
4.335
Orign Indra
0.280
0.431
0.553
0.793
1.978
2.069
3.282
3.240
hubert-b
Roberta
0.322
0.578
0.697
1.232
2.230
2.971
3.880
4.369
Orign 0.449
0.308
0.861
0.638
2.475
1.949
4.119
3,286
hubert-1
Roberta
Indra
0.454
0.578
0.878
1.248
2.610
3.003
4.461
4.255
tectures and modalities. Significant improvements are observed in both T-+1 and I-T retrieval, highlighting the effectiveness of our method for cross-modal alignment. These findings suggest that the Indra representation offers a generalizable mechanism to improve vision- vision-language matching, inde- pendent of model architecture or dataset. Nonetheless, there remains noticeable gap performance compared to the fully aligned CLIP model, indicating further room for improvement.
4.3 Evaluation on Speech & Language Modalities
Datasets. We adopt the TIMIT dataset [16] for audio and language modality experiments. TIMIT contains recordings from 630 speakers representing eight major dialect regions of American English, each reading ten phonetically rich sentences. The dataset provides time-aligned phonetic and word- level transcriptions along with 16kHz audio recordings.
Foundation Models. For audio models, we evaluate wav2vec [66], wavlm [6], and hubert(28), using both base and large variants. For the language modality, we use Roberta [46]. All models are pretrained independently on unimodal data, without any cross-modal alignment. As an aligned baseline, we include CLAP [78], an audio-language model jointly trained paired audio-text data.
Evaluation Metrics: Similarly, we adopt CLAPScore [78] as the evaluation metric. We report Top-k matching (k E {5, 10, 30, 50)) in both text-to-audio (T-A) and audio-to-text (A (A-TT) tasks.
Analysis. Table 4 presents the audio-text matching results using different audio models. As shown, the Indra representations consistently improve matching performance in both directions across all model configurations. However, compared to the vision-language setting, the improvements in the audio- language modality are relatively modest. This is likely due to the comparatively weaker capacity of the audio models used. Nonetheless, we observe that larger audio models yield better matching accuracy, further supporting the notion that model capacity positively influences cross-modal alignment.
5 Related Work
Instance-Leve Representation Learning. In conventional deep learning paradigms, each data instance is independently encoded into fixed-dimensional vector, optimized using either supervised signals unsupervised objectives. In supervised learning, representations are shaped by categorical labels [38, 24], whereas in unsupervised settings, objectives such as reconstruction (e.g., autoencoders (73)) or self-prediction (e.g., BERT [11]) guide the learning of instance-level representations. These approaches primarily focus on learning standalone embeddings and do not explicitly model the relationships between samples, either in training or inference stages.
8

---

Contrastive Representation Learning. Contrastive methods such as SimCLR [7], MoCo [23], and BYOL [19] introduce pairwise relational inductive bias during training by encouraging the embeddings of similar (augmented) views to be close, while pushing apart dissimilar ones. Building on this framework, CLIP [62] extends contrastive learning to vision-language pretraining by aligning paired image-text embeddings within shared multimodal space, enabling strong zero-shot perfor- mance. BLIP [42] further integrates contrastive and generative objectives to enhance cross-modal representation learning, achieving state-of-the-ar results on several vision-language benchmarks. Despite these advances, the sample representations in contrastive frameworks remain inherently instance-centric during inference. Additionally, contrastive learning approaches typically require large-scale datasets to achieve satisfactory performance, as the training objectives rely heavily on diverse and abundant positive and negative pairs.
Graph-based Representation Learning. Graph neural networks [35] offer a framework for encoding relational information through message passing across graph-structured data. Variants such as GraphSAGE [21] and GAT [71] have improved flexibility and representation ability. However, most graph-based approaches rely predefined adjacency structures proximity assumptions [51], which may embed inductive biases that are misaligned with the latent semantics of the data. They typically operate over local k-hop neighborhoods [48], limiting their ability to capture long-range dependencies unless deeply stacked, which can lead to oversmoothing and degraded performance [43].
Attention-bawe Representation Learning. Transformer-basee architectures [70] offer powerful alternative by leveraging global self-attention to aggregate contextual information. Numerous studies [11, 46, 12, 25, 61, 63) have demonstrated that attention-base based architectures can be highly effective even in non-sequential domains. While attention enables global interactions both during training and inference, the resulting token-level representations are determined through dynamic mixing rather than explicitly encoding pairwise or global sample-to-sample relationships, making their geometric interpretation less transparent.
In contrast to the above paradigms, our approach explicitly constructs each representation as a relational profile, specifically, design is motivated by the philosophical ontology of Indra's Net, where each entity reflects and is reflected by all others. forming a holistic network of interdependence. More importantly, our Indra representation formally grounded in the Yoneda Lemma from enriched category theory, offering a theoretically sound and interpretable framework for relational representation learning.
eestance-based reflection of its relationship to all other samples. This
6 Conclusion
In this paper, we present theoretical and empirical investigation into the convergent behavior of unimodal foundation models. Motivated by the philosophical metaphor of Indra's 's Net and grounded in enriched category theory, we introduce the Indra representation as relational encoding that reflects each sample through its relationships with all others. We demonstrate that this representation can be derived via the V-enriched Yoneda embedding and instantiated practically using angular distance. Our theoretical analysis proves that the proposed Indra representations are unique, complete, and structure- preserving, offering principled basis for traniin-free alignment of various foundation models. Through extensive experiments across single-modality vision-language, and we demonstrate that Indra representations improve robustness and latent cross- moosl capabilities of unimodal foundation models, offering a general framework for training-free multimodal alignment. Our findings suggest a new perspective for bridging modalities, which emphasizes the importance of the intrinsic relational structure of data or reality.
settings,
Limitations. Constructing exact Indra representations requires a computational complexity of (n?2d) and a memory complexity of O(22) for a dataset with n samples and embedding dimension d. This quadratic scaling potentially limits the direct applicability of the exact Indra representations to sarge-scale settings. However, the scalability concern is addressable in practice. In the literature, there exists a rich body of work on approximating pairwise distances efficiently. For example, approximate nearest neighbor search [30, 31]. landmark-bart based approximation [50, 57], hashing- based methods [8], and sparse graph constructions [75]. From an application perspective, these techniques can be readily adapted to approximate the Indra representation at scale without sacrificing its robustness. From a theoretical view, Kan extensions [13] provide a principled framework for extrapolating from
partial distance information to a full representation, preserving maximal structural fidelity.

---

References
[1] Harsh Agrawal, Karan Desai, Yufei Wang. Xinlei Chen, Rishabh Jain, Mark Johnson, Dhruv Batra, Devi Parikh, Stefan Lee, and Peter Anderson. Nocaps: Novel object captioning at scale. In Proceedings of the IEEE/CVF international conference on computer vision, pages 8948-8957, 2019.
[2] Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations, Advances in neural information processing systems, 11112449-12460 2020.
[3] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609 2023.
[4] Juerg Beringer, J-F Arguin, RM Barnett, K Copic, O Dahl, DE Groom, C-J Lin, J Lys, Mu- rayama, CG Wohl, et al. Review of particle physics. Physical Review D-Particles, Fields, Gravitation, and Cosmology, 86(1 ):010001, 2012.
[5] Lennart Bürger, Fred A Hamprecht, and Boaz Nadler. Truth is universal: Robust detection of lies in llms. Advances in Neural Information Processing Systems, 37, 2024.
[6] Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. Wavlm: Large-scale self-supervised pre- training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16(6):1505-1553 15181 2022.
[7] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, pages 1597-1607. PMLR, 2020.
[8] Yudong Chen, Sen Wang, Jianglin Lu, Zhi Chen, Zheng Zhang, and Zi Huang. Local graph convolutional networks for cross-modal hashing. In Proceedings of the 29th ACM international conference multimedia, pages 1921-1928, 2021.
[9] Thomas Cleary. The Flower Ornament Scripture: A Translation of the Avatamsaka Sutra. Shambhala Publications, Boston, 1993.
[10] F.H. Cook. Hua-Yer Yen Buddhism: The Jewel Net of Indra. laswr Series. Pennsylvania State University Press, 1977.
[11] Jacob Devlin, Ming-Woi Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguisties: human language technologies, volume 1, pages 4171 11814486, 2019.
[12] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
[13] Eduardo J Dubuc. Kan extensions in enriched category theory. volume 145. Springer, 2006.
[14] Michael Faraday. Experimental Researches in Electricity. Bernard Quaritch, London, 1859. Originally published as a series of papers between 1831 and 1855.
[15] John Rupert Firth, editor. Studies in Linguistic Analysis. Philological Society, Oxford, 1957. Special volume of the Philological Society.
[16] John S Garofolo, Lori Lamel, William M Fisher, David S Pallett, Nancy L Dahlgren, Victor Zue, and Jonathan G Fiscus. Timit acoustic-phonetic continuous speech corpus. (No Title), 1993.
[17] Kenneth J. Gergen. Relational Being: Beyond Self and Community. Oxford University Press, New York, 2009.
10

---

[18] David Griffiths. Introduction to elementary particles. John Wiley & Sons, 2020.
[19] Jean-Bastien Grill, Florian Strub, Florent Altmann, Corentin Tallec, Pierre-Henri Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self sulf-supee learning. In Advances in Neural Information Processing Systems, volume 33, pages 21271-21283 2020.
[20] Wes Gurnee, Theo Horsley, Zifan Carl Guo, Tara Rezaei Kheirkhah, Qinyi Sun, Will Hathaway, Neel Nanda, and Dimitris Bertsimas. Universal neurons in gpt2 language models, arXiv preprint arXiv:2401. 12181, 2024.
[21] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, 2017.
[22] Harvard FAS CAMLab. Indra's net, 2022.
[23] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie. and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729-97338 9738, 2020.
[24] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[25] Pengeheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006. 204444000000355, 2020.
[26] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-frre evaluation metric for image captioning. arXiv preprint arXiv:2104.08711, 2021.
[27] Eghbal Hosseini, Colton Casto, Noga Zaslavsky, Colin Conwell, Mark Richardson, and Evelina Fedorenko. Universality of representation in biological and artificial neural networks. bioRxiv, 2024.
[28] Wei-Ning Hsu, Benjamin Bolte. Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. Hubert: seff-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM transactions on audio, speech, and language processing. 29:3451-3460, 2021.
[29] Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. The platonic representa- tion hypothesis. In Proceedings of the 41 International Conference on Machine Learning, Proceedings of Machine Learning Research. PMLR, 2024.
[30] Herve Jegou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neighbor search. IEEE transactions on pattern analysis and machine intelligence, 33(1):117-128, 2010.
[31] Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Transactions Big Data, 7(3):535 547, 2019.
[32] Thomas P. Kasulis. Engaging Japanese Philosophy: A Short History. University of Hawai'i Press, Honolulu, 2018
[33] Gregory Maxwell Kelly. Basic Concepts of Enriched Category Theory, volume 64 of London Mathematical Society Lecture Note Series. Cambridge University Press, Cambridge, 1982.
[34] Mcenakshi Khosla, Alex H Williams, Josh McDermott, and Nancy Kanwisher. Privileged representational axes in biological and artificial neural networks. bioRxiv, pages 2024-06, 06, 2024.
[35] Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.
[36] Jing Yu Koh, Ruslan Salakhutdinov, and Daniel Fried. Grounding language models to images for multimodal inputs and outputs. In International Conference on Machine Learning, pages 17283-17300. PMLR, 2023.

---

[37] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009. Technical Report.
[38] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012.
[39] William Lawvere. Metric spaces, generalized logic, and closed categories. Rendiconti del seminario matématico fisico di Milano, 43:135-166, 1973.
[40] Andrew Lee, Melanie Weber, Fernanda Viégas, and Martin Wattenberg. Shared global and local geometry of language model embeddings, arXiv preprint arXiv:2503.21073 2025.
[41] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language- -mage pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730- 19742. PMLR, 2023.
[42] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language- image pre-training for unified vision- laniun-a understanding and generation. In International conference on machine learning, pages 12900. PMLR, 2022.
[43] Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
[44] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision-ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13, pages 740-755. Springer, 2014.
[45] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 10:34892-34911 2023.
[46] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: robustly optimized bert pretraining approach. arXiv preprint arXiv: 11692, 2019.
[47] Zhuang Liu, Hanzi Mao, Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[48] Jianglin Lu, Yixuan Liu, Yitian Zhang, and Yun Fu. Scale-free saph-language models. In The Thirteenth International Conference on Learning Representations, 2025.
[49] Jianglin Lu, Hailing Wang, Yi Xu, Yizhou Wang, Kuo Yang, and Yun Fu. Representation potentials of foundation models for multimodal alignment: A survey. In Proceedings of the 2025 Conference Empirical Methods in Natural Language Processing, pages 16669-16684. Association for Computational Linguistics, 2025.
[50] Jianglin Lu, Hailing Wang, Jie Zhou, Yudong Chen, Zhihui Lai, and Qinghua Hu. Low-rank adaptive graph embedding for unsupervised feature extraction. Pattern Recognition, 113:107758, 2021.
[51] Jianglin Lu, Yi Xu. Huan Wang, Yue Bai. and Yun Fu. Latent graph inference with limited supervision. In Advances in Neural Information Processing Systems, 2023.
[52] Mayug Maniparambil, Raiymbek Akshulakov, Yasser Abdelaziz Dahou Djilali, Mohamed El Amine Seddik. Sanath Narayan, Karttikeya Mangalam, and Noel E. O' O'Connor. Do vision and language encoders represent the world similarly? In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2024.
[53] Hazel Rose Markus and Shinobu Kitayama. Culture and the self: Implications for cognition, emotion, and motivation. In College student development and academic life, pages 264-293 Routledge, 2014.
12

---

[54] James Clerk Maxwell. A Treatise on Electricity and Magnetism. Clarendon Press, Oxford, first edition, 1873.
[55] Jack Merullo, Louis Castricato, Carsten Eickhoff, and Ellie Pavlick. Linearly mapping from image text space. In The Eleventh International Conference on Learning Representations, 2023.
[56] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositionality. Advances in neural information processing systems, 26, 2013.
[57] Luca Moschella, Valentino Maiorea, Marco Fumero, Antonio Norelli, Francesco Locatello, and Emanuele Rodolà. Relative representations enable zero-shot latent space communication. In The Eleventh International Conference on Learning Representations. 2023.
[58] Evonne Ng. Sanjay Subramanian, Dan Klein, Angjoo Kanazawa, Trevor Darrell, and Shiry Ginosar. Can language models learn to listen? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10083-10093, 2023.
[59] Jerry Ngo and Yoon Kim. What do language models hear? probing for auditory representations language models. In Proceedings of the 62nd Annual Meeting of the Association for Compulational Linguistics, pages 5435-5448, 2024.
[60]
OpenAl, Gpt-4 technical report, 2024.
[61] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07/93, 2023.
[62] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et Learning transferable visual models from natural language supervision. In International conference on machine learning. pages 8748-8763. PMLR, 2021.
[63] Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, pages 28492-28518, PMLR, 2023.
[64] Emily Richl. Category theory in context. Courier Dover Publications, 2017.
[65] Geoffrey Roeder, Luke Metz, and Durk Kingma. On linear identifiability of learned rep- resentations. In International Conference on Machine Learning, pages 9030-9039, PMLR, 2021.
[66] Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. wav2vec: Unsupervised pre-training for speech recognition. arXiv preprint 1eXXvv/904.05000 2019.
[67] Pratyusha Sharma, Tamar Rott Shaham, Manel Baradad, Stephanie Fu, Adrian Rodriguez- Munoz, Shivam Duggal, Phillip Isola, and Antonio Torralba. A vision check for language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pages 14410-14419, 2024.
[68] Daniel Tan, David Chanin, Aengus Lynch, Brooks Paige, Dimitrios Kanoulas, Adrià Garriga- Alonso, and Robert Kirk. Analysing the generalisation and reliability of steering vectors. Advances in Neural Information Processing Systems, 37, 2024.
[69] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023.
[70] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, tukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances neural information processing systems, 2017.
13

---

[71] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio, al. Graph attention networks. stat, (050(20):00-4555)) 2017.
[72] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages
2017.
[73] Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and Léon Bottou. Stacked denoising autoencoders: Learning useful representations in deep network with a local denoising criterion. Journal of machine learning research, 1 11(12), 2010.
[74] Junxuan Wang, Xuyang Ge, Wentao Shu, Qiong Tang, Yunhua Zhou, Zhengfu He, and Xipeng Qiu. Towards universality: Studying mechanistic similarity across language model architectures. In The Thirteenth International Conference on Learning Representations, 2025.
[75] Yangbo Wang. Jie Zhou, Mingli Song, Yue Guo, and Jianglin Lu. Fuzzy multi-subspaac clustering. IEEE Transactions on Fuzzy Systems, pages 1-14, 2026.
[76] John Wentworth. Testing the natural abstraction hypothesis. Al Alignment Forum, 2021.
[77] Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders. In IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), 2023.
[78] Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Kerg-Kirkpattick, and Shlomo Dubnov. Large-scale contrastive language- audio pretraining with feature fusion and keyword- to-caption augmentation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5. IEEE, 2023.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Our abstract and introduction clearly showcased our main contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the work in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification:
See appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: See Section 4. In addition, we release the source code for reprodueibility, see Abstract for the code link.
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification:
We release the source code for reproducibility, see Abstract for the code link.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: /niiss wwtppcccppppaaeeeccccccccccceeee / siisssssiiiiiyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: We conduct experiments following the standard protocol and include the experimental details in Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We provide the experimental details in the Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide the experimental details in Section 4, and results can be reproduced using our source code (see the link provided in Abstract).
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: http///eurrpp..peeeeeeeee http:////ccpppp.ieicctcttttteee.
Answer: [Yes]
Justification:
We have read and follow the rules.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work has no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited the papers that created the datasets.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:
We did not LLMs in paper except polishing presentation.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https /ttpas//////.s..//eeee ccc///omferrecccee222///11/ for what should or should not be described.
21


Reviews:
[{"review_id": "uYNhE0sn8o", "reviewer": "Reviewer_enB4", "summary": "Motivated by recent findings that unimodal foundation models often exhibit strong correlations between their feature representations, this paper introduces the concept of the Indra Representation. Inspired by the ancient notion of Indra’s Net—a metaphysical idea from Eastern philosophy suggesting that each entity is a reflection of all others—the authors formalize a representation that captures the relational structure underlying model features. This formulation provides a unified framework for interpreting and aligning features across different models and modalities. The effectiveness of the proposed representation is validated through experiments conducted across diverse datasets and settings.", "questions": "You can directly refer to the weaknesses part.", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 4, "strengths_and_weaknesses": "Strengths: \n1.\tBuilding upon the ancient philosophical concept of Indra’s Net as a foundational metaphor, this work formalizes and instantiates it within a modern machine learning framework. I find this conceptual grounding to be highly novel and intellectually compelling.\n2.\tThe paper presents comprehensive experiments demonstrating that the proposed Indra representation achieves stronger alignment across features from different unimodal foundation encoders, compared to using raw features directly from each encoder. This provides solid empirical support for the effectiveness of the proposed framework.\n\nWeaknesses: \n1.\tWhile the proposed representation is theoretically elegant and empirically validated, the paper could benefit from a clearer articulation of its practical utility. Specifically, it remains somewhat unclear in what real-world scenarios such cross-model, cross-modality feature alignment is necessary. Furthermore, the authors do not address why existing well-aligned models, such as CLIP or more recent MLLM like GPT-4o, would not already suffice in such contexts. Providing stronger motivation for why this new representation is needed in practice would strengthen the impact of the work.\n2.\tThe use of number notes such as ① and ② throughout the paper feels informal and somewhat inconsistent with the typical academic writing style expected in formal publications. It may be advisable to adopt more standard notation or formatting for clarity and professionalism.\n3.\tWhile the proposed method shows strong performance on the selected benchmarks, the experiments are primarily conducted on datasets with a limited number of categories. In real-world applications, it may be challenging to define the appropriate scope and number of relevant entities for constructing the Indra representation, potentially limiting its scalability and practicality in large-scale or open-domain settings.", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer enB4,\n\n\n**We sincerely appreciate you for providing constructive comments and are glad to see you recognize our idea as highly novel and intellectually compelling.**\n\nBelow, we address your remaining concerns.\n\n**W1 Practical utility**\n\nThank you for providing constructive suggestions. We would like to clarify that the primary goal of this work is not to demonstrate that our proposed Indra representations outperform well-aligned models such as CLIP or GPT-4o in practical applications, nor is it to argue that such models are insufficient.\n\nInstead, our core contribution is to investigate what form the learned representations of large-scale unimodal foundation models converge to, after training on unimodal data. Prior studies have shown that such models tend to converge in representation space, but they do not explain what the representations ultimately converge to. Our work offers a new representation convergence hypothesis and provides empirical results to validate this hypothesis.\n\nThe ***practical utility*** of our proposed Indra representation lies in its ability to improve cross-modality alignment without the need of additional adapter or large-scale finetuning. In domains like medicine or other specialized fields, there may not exist multimodal foundation models, nor is it always feasible to train them. In such settings, leveraging existing unimodal models through our Indra representation offers a compelling solution for cross-modal tasks.\n\nAs demonstrated in our experiments, the proposed Indra representation improves performance not only in unimodal tasks, but also across vision-language and speech-language modalities. Furthermore, the added classification experiments show that our representation also enhances classification accuracy and robustness (kindly refer to our response to Reviewer yaBd (W1 & Q3)).\n\nIn summary, *our work is not intended to replace well-aligned models but to uncover a foundational mechanism that explains how unimodal models can be structurally leveraged or adapted for cross-modal settings, especially where aligned models are unavailable or infeasible.*\n\n\n**W2 Notation**\n\nWe appreciate the reviewer’s feedback regarding the use of numbered notes. We understand that this stylistic choice may come across as informal or inconsistent with conventional academic norms. We have revised the manuscript to adopt a more standard academic style. We thank the reviewer for highlighting this and will ensure the final version adheres to expected standards.\n\n\n**W3 Scalability and practicality**\n\n*[Application-Oriented (Scalable) Solutions]* ***The scalability concern is addressable in practice***. In the literature, there exists a rich body of work on approximating pairwise distances efficiently. For example, approximate nearest neighbor search (e.g., FAISS, HNSW), landmark-based approximation (e.g., K-means centroids, random subsampling), hashing-based methods (locality sensitive hashing), and sparsified graph constructions (as suggested by Reviewer 6ioa). From an application view, these techniques can be readily adapted to approximate the Indra representation at scale without sacrificing its structural interpretation.\n\n*[Experimental Validation]* To evaluate the effectiveness of approximated Indra representations, we adopt a simple landmark-based approximation via random subsampling. We randomly select $m < n$ landmarks as representative samples, reducing the computational complexity to $\\mathcal{O}(nmd)$ and the memory complexity to $\\mathcal{O}(nm)$. We assess this approach on two tasks: image classification on the CIFAR-100 dataset and image-text matching on the large-scale CC3M dataset.\n\nFor CIFAR-100, we use the standard data split provided by torchvision.datasets and employ logistic regression (linear probing) to evaluate classification accuracy. To assess the robustness, we inject Gaussian noise with different levels of standard deviation $\\sigma$ into the extracted features.\n\nFor CC3M, we sample 300,000 image-text pairs for evaluation. We extract image features with DINOv2 and text features with RoBERTa. For each image, we compute mean CLIP scores with the top-$k$ candidate texts and compare the performance of the original features against that of the approximated Indra representations.\n\nResults under varying numbers of landmarks are presented below.\n\n|CIFAR-100|$\\sigma$=0.0|$\\sigma$=1.5|$\\sigma$=3.0|$\\sigma$=5.0|\\$\\sigma$=7.0||CIFAR-100|$\\sigma$=0.0|$\\sigma$=1.5|$\\sigma$=3.0|$\\sigma$=5.0|\\$\\sigma$=7.0|# of landmarks|\n|--|--|--|--|--|--|--|--|--|--|--|--|--|:--:|\n|DinoV2|91.97|89.24|82.21|63.06|40.16||DinoV2|91.97|89.24|82.21|63.06|40.16|---|\n|Ours (Euclidean)|90.46|88.56|82.49|66.00|40.46||Ours (Angular) |91.75|89.55|84.03|67.83|46.90|1000|\n|Ours (Euclidean)|91.83|90.08|84.31|71.38|49.33||Ours (Angular) |92.09|90.25|85.37|72.02|53.66|2000|\n|Ours (Euclidean)|91.85|89.28|84.91|70.39|52.02||Ours (Angular) |**92.34**|**90.34**|**85.53**|**73.74**|**55.18**|3000|\n|Ours (Euclidean)|92.06|**90.12**|85.81|71.98|54.56||Ours (Angular) |92.10|90.20|85.48|73.40|55.15|5000|\n|Ours (Euclidean)|**92.20**|**90.12**|**85.56**|73.23|56.91||Ours (Angular) |92.24|89.99|85.14|72.03|53.68|10000|\n\n\n||#of landmarks|k=5|k=10|\n|--|--|--|--|\n|DinoV2+RoBERTa|---|25219.78|25208.54|\n|Ours |5000|28225.07|26836.45|\n\nThe experimental results demonstrate that ***the approximated Indra representations still achieve better classification and alignment performance for real-world applications***. Specifically, the Indra representations consistently outperform the original representations under increasing levels of Gaussian noise, particularly at higher noise levels, highlighting their superior robustness. Furthermore, increasing the number of landmarks may make improvements in classification accuracy.\n\n\n*We hope the above response adequately addresses your concerns. Once again, thank you for your thoughtful feedback and for helping us improve the quality of our paper!*"}, {"review_id": "rKSFGFtpfo", "reviewer": "Reviewer_yaBd", "summary": "This paper introduces The Indra Representation Hypothesis, a novel theoretical and empirical framework aimed at understanding representation convergence in unimodal foundation models. Drawing philosophical inspiration from the metaphor of Indra’s Net, the authors propose that foundation models, despite being trained on different modalities and objectives, tend to learn internal representations that converge towards a shared relational structure.\n\nThe key contribution is the formalization of this idea through category theory, specifically the V-enriched Yoneda embedding, which leads to the definition of the Indra representation. The Indra representation encodes each sample through its relational profile with respect to all others, using a distance metric (angular distance in this work) between model-generated embeddings. Theoretical guarantees are provided, showing that this representation is unique, complete, and preserves structural relationships.\n\nThe paper then instantiates this formulation practically and demonstrates its utility across a wide range of experiments involving cross-model and cross-modal matching tasks, including vision, language, and audio domains. Results show that Indra representations significantly improve matching performance between independently trained unimodal models, outperforming the original embeddings and narrowing the gap with jointly trained multimodal baselines like CLIP and CLAP.\n\nThrough both its mathematical grounding and extensive empirical validation, the paper positions the Indra representation as a general, theoretically sound, and practically beneficial approach for understanding and leveraging latent relational structures in foundation models.", "questions": "1. Really nice proof! However, the proof of Theory 2, one thing I find confusing, you seem mixing the concept of Identity Property with the Separation Axiom. Would you mind explain what your identity property refers to?\n2. Your experiment part is almost excellent. Besides these scores, can you show some real-world downstream task performance of your method?\n3. My main concern is that your method has O(N^2) computational complexity. Under this assumption, many of us are interested in how large the computation and storage costs become on truly large-scale datasets. Could you discuss more about that?", "limitations": "I suppose you should talk more about limitation.\nIt fails to address computational cost, storage footprint, sensitivity to the choice of distance metric, and scalability to real downstream tasks, among other issues. Overall, the limitation discussion is too narrow in scope and pays insufficient attention to potential engineering and data challenges.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. Quality & Clarity: The paper presents a well-grounded and mathematically rigorous framework by introducing the Indra representation using V-enriched category theory. The authors provide theoretical guarantees—uniqueness, completeness, and structure preservation—supported by formal theorems and proofs.\n\n2. Originality: The analogy to Indra’s Net offers a fresh and philosophically motivated perspective on representation learning, distinguishing this work from traditional embedding-centric approaches.\n\n3. Significance: The proposed method is applied to vision, language, and audio models, showing strong performance improvements across cross-model and cross-modal matching tasks, without requiring any retraining or supervision.\n\nWeaknesses:\n1. Insufficient limitation discussion: It fails to address computational cost, storage footprint (Indra vectors require pairwise distances over the entire dataset), sensitivity to the choice of distance metric, and scalability to real downstream tasks, among other issues. Overall, the limitation discussion is too narrow in scope and pays insufficient attention to potential engineering and data challenges.\n\n2. Small Unclear in proof of theory 2: Seems missing proof of separation axiom (d(x,y)=0 ⇒ x=y)", "was_revised": true, "final_justification": "The rebuttal experiment has already addressed my concerns. I believe it deserves to be accepted.", "rebuttal": "Dear Reviewer yaBd,\n\n**We sincerely appreciate you for providing helpful suggestions and for recognizing our idea as fresh and distinguishable from traditional approaches**. \n\nBelow, we address your remaining concerns.\n\n**W1&Q3 Limitation discussion**\n\n*[Complexity of Exact Computation]* Constructing ***exact*** Indra representations requires a computational complexity of $\\mathcal{O}(n^2d)$ and a memory complexity of $\\mathcal{O}(n^2)$ for a dataset with $n$ samples and embedding dimension $d$. This quadratic scaling potentially limits the direct applicability of the exact Indra representations to large-scale datasets.\n\n*[Application-Oriented (Scalable) Solutions]* ***The scalability concern is addressable in practice***. In the literature, there exists a rich body of work on approximating pairwise distances efficiently. For example, approximate nearest neighbor search (e.g., FAISS, HNSW), landmark-based approximation (e.g., K-means centroids, random subsampling), hashing-based methods (locality sensitive hashing), and sparsified graph constructions (as suggested by Reviewer 6ioa). From an application view, these techniques can be readily adapted to approximate the Indra representation at scale without sacrificing its structural interpretation.\n\n*[Experimental Validation]* To evaluate the effectiveness of ***approximated*** Indra representations, we adopt a simple landmark-based approximation via random subsampling. We randomly select $m < n$ landmarks as representative samples, reducing the computational complexity to $\\mathcal{O}(nmd)$ and the memory complexity to $\\mathcal{O}(nm)$. We assess this approach on two tasks: image classification on the CIFAR-100 dataset and image-text matching on the large-scale CC3M dataset.\n\nFor CIFAR-100, we use the standard data split provided by torchvision.datasets and employ logistic regression (linear probing) to evaluate classification accuracy. To assess the robustness, we inject Gaussian noise with different levels of standard deviation $\\sigma$ into the extracted features.\n\nFor CC3M, we sample 300,000 image-text pairs for evaluation. We extract image features with DINOv2 and text features with RoBERTa. For each image, we compute mean CLIP scores with the top-$k$ candidate texts and compare the performance of the original features against that of the approximated Indra representations.\n\nResults under varying numbers of landmarks are presented below.\n\n|CIFAR-100|$\\sigma$=0.0|$\\sigma$=1.5|$\\sigma$=3.0|$\\sigma$=5.0|\\$\\sigma$=7.0||CIFAR-100|$\\sigma$=0.0|$\\sigma$=1.5|$\\sigma$=3.0|$\\sigma$=5.0|\\$\\sigma$=7.0|# of landmarks|\n|--|--|--|--|--|--|--|--|--|--|--|--|--|:--:|\n|DinoV2|91.97|89.24|82.21|63.06|40.16||DinoV2|91.97|89.24|82.21|63.06|40.16|---|\n|Ours (Euclidean)|90.46|88.56|82.49|66.00|40.46||Ours (Angular) |91.75|89.55|84.03|67.83|46.90|1000|\n|Ours (Euclidean)|91.83|90.08|84.31|71.38|49.33||Ours (Angular) |92.09|90.25|85.37|72.02|53.66|2000|\n|Ours (Euclidean)|91.85|89.28|84.91|70.39|52.02||Ours (Angular) |**92.34**|**90.34**|**85.53**|**73.74**|**55.18**|3000|\n|Ours (Euclidean)|92.06|**90.12**|85.81|71.98|54.56||Ours (Angular) |92.10|90.20|85.48|73.40|55.15|5000|\n|Ours (Euclidean)|**92.20**|**90.12**|**85.56**|73.23|56.91||Ours (Angular) |92.24|89.99|85.14|72.03|53.68|10000|\n\n\n||#of landmarks|k=5|k=10|\n|--|--|--|--|\n|DinoV2+RoBERTa|---|25219.78|25208.54|\n|Ours |5000|28225.07|26836.45|\n\nThe experimental results demonstrate that ***the approximated Indra representations still achieve better classification and alignment performance for real-world applications***. Specifically, the Indra representations consistently outperform the original representations under increasing levels of Gaussian noise, particularly at higher noise levels, highlighting their superior robustness. Furthermore, increasing the number of landmarks may make improvements in classification accuracy.\n\n**W2 &Q1 Proof of Theory 2**\n\nThank you for carefully reviewing the proof. You are absolutely right, our statement that “the cost function $d$ satisfies the identity property” should be revised to “the cost function $d$ satisfies the T₀ separation axiom.” The identity property, which requires that $d(X_i,X_j)=0$ does not imply $X_i=X_j$. We will revise the theorem and proof accordingly to incorporate a stricter condition on the cost function.\n\n**Q2 Downstream tasks**\n\nWe additionally report image classification results on the CIFAR-10, CIFAR-100, and Office-Home datasets under various distance metrics. For CIFAR-10 and CIFAR-100, we use the standard data splits provided by torchvision.datasets. For Office-Home, we evaluate classification accuracy across four domains: Art (A), Clipart (C), Product (P), and Real-World (R), using an 80/20 split for training and testing. Across all datasets, we adopt logistic regression (linear probing) to assess the quality of the extracted representations. To evaluate robustness, we inject Gaussian noise into the features with varying standard deviations $\\sigma$. The corresponding results are presented below.\n\n\n|cifar10|$\\sigma$=0.0|$\\sigma$=1.5|$\\sigma$=3.0|$\\sigma$=5.0|\\$\\sigma$=7.0| |cifar100|$\\sigma$=0.0|$\\sigma$=1.5|$\\sigma$=3.0|$\\sigma$=5.0|\\$\\sigma$=7.0|\n|--|--|--|--|--|--|--|--|--|--|--|--|--|\n|ViT|93.98|89.60|87.75|79.77|68.15| | |79.45|70.40|54.69|35.76|27.45|\n|Ours (Angular)|**94.72**|**92.90**|**88.08**|**79.84**|**68.16**| | |**80.51**|**74.48**|**61.12**|**42.29**|**28.49**|\n|Ours (Euclidean)|**94.84**|**93.15**|**89.51**|**80.84**|**68.71**| | |**80.09**|**77.36**|**69.00**|**51.59**|**32.74**|\n| | | | | | | | | | | | | |\n|ConvNeXt|97.00|94.37|85.89|80.10|65.85| | |85.77|78.56|62.79|34.39|21.28|\n|Ours (Angular)|**97.20**|**95.22**|**90.77**|**80.18**|**65.93**| | |85.74|**80.30**|**66.04**|**41.58**|**24.45**|\n|Ours (Euclidean)|**97.21**|**96.06**|**92.86**|**81.59**|**66.64**| | |85.64|**82.25**|**72.16**|**51.51**|**30.25**|\n| | | | | | | | | | | | | |\n|DinoV2|99.19|98.58|95.21|85.57|76.54| |  |91.97|89.24|82.21|63.06|40.16|\n|Ours (Angular)|99.19|**98.80**|**96.59**|**87.39**|**76.68**| | |91.77|**89.72**|**83.72**|**67.53**|**47.80**|\n|Ours (Euclidean)|99.14|**98.72**|**96.87**|**89.73**|**77.92**| | |91.93|**89.76**|**84.83**|**74.29**|**58.67**|\n\n|Noise levels|A $\\sigma$=0.0|A $\\sigma$=3.0|A $\\sigma$=5.0|A $\\sigma$=7.0|C $\\sigma$=0.0|C $\\sigma$=3.0|C $\\sigma$=5.0|C $\\sigma$=7.0|P $\\sigma$=0.0|P $\\sigma$=3.0|P $\\sigma$=5.0|P $\\sigma$=7.0|R $\\sigma$=0.0|R $\\sigma$=3.0|R $\\sigma$=5.0|R $\\sigma$=7.0|\n|--|--|--|--|--|--|--|--|--|--|--|--|--|--|--|--|--|\n|ViT|80.25|64.40|44.03|22.63|73.20|50.40|28.64|15.23|92.34|80.74|61.15|35.25|89.22|82.11|60.09|35.32|\n|Ours (Angular)|79.63|**65.02**|43.62|**27.57**|69.76|**54.98**|**33.10**|**18.21**|89.75|**81.53**|**64.08**|**40.77**|87.16 |**83.49**|**63.65**|**40.48**|\n|Ours (Euclidean)|80.04 |63.37|43.21|**25.51**|70.33|**55.78**|**34.48**|**18.33**|89.64 |80.74|**64.75**|**40.77**|87.16|**83.37**|**64.56**|**41.28**|\n| | | | | | | | | | | | | | | | | |\n|ConvNeXt|89.71|62.76|27.98|12.14|83.62|54.07|20.85 |09.74|96.62|84.91|44.26|19.37|93.46|82.11|38.30|17.78|\n|ours (Angular)|87.86|59.88|**28.81**|**14.20**|82.70|**57.85**|**25.09**|**11.34**|**96.73**|**85.92**|**45.61**|**22.18**|93.35|**84.63**|**40.71**|**19.61**|\n|Ours (Euclidean)|87.45|58.02|26.75|**12.96**|82.93|**58.30**|**23.83**|**11.23**|96.28|**86.26**|**45.38**|**21.73**|93.35|**84.75**|**40.14**|**19.04**|\n| | | | | | | | | | | | | | | | | |\n|DinoV2|87.65|73.05|46.91|27.78|88.43|75.14|51.09|31.04|96.73|93.24|83.33|60.70|92.78|87.39|71.44|48.51|\n|Ours (Angular)|87.04|70.99|**47.53**|27.37|87.29|**76.63**|**54.75**|**33.56**|96.40|92.79|**84.46**|60.59|**92.89**|**88.53**|**73.17**|**49.89**|\n|ours (Euclidean)|86.21|68.52|44.65|27.16|87.29|**76.75**|**55.21**|**34.59**|96.51|92.57|**84.12**|60.02|92.55|**88.53**|**73.39**|**49.54**|\n\n\n\n**L1 Limitation section revision**\n\nThank you for the constructive comments. We have addressed these potential issues in the discussion above and will revise the limitations section accordingly. Specifically, we will explicitly acknowledge the computational challenges of computing exact Indra representations and include a discussion of scalable approximation strategies, supported by corresponding experimental validation.\n\n\n*We hope the above response adequately addresses your concerns. Once again, thank you for your thoughtful feedback and for helping us improve the quality of our paper.*"}, {"review_id": "wN7rRU1X3c", "reviewer": "Reviewer_6ioa", "summary": "The paper proposes a new theoretical and empirical framework for representation learning inspired by the metaphor of Indra’s Net. The central hypothesis is that unimodal foundation models, despite being trained independently, converge to representations that reflect a shared relational structure of reality. This is formalized using enriched category theory, where each sample is represented by its vector of distances to all other samples under a specific cost function (e.g., angular distance). The resulting Indra representation is theoretically shown to be unique, complete, and structure-preserving. The authors instantiate the method on pretrained models from vision, language, and audio modalities, and evaluate it on cross-model and cross-modal matching tasks. They also test an alternative cost function (Euclidean distance) to assess robustness. The method is applied post hoc, without retraining or supervised alignment.", "questions": "1. Have the authors considered evaluating Indra representations on downstream tasks beyond retrieval, such as classification (e.g., via linear probing) or caption generation?\n2. Can the authors provide formal or empirical analysis of the time and space complexity of Indra representation computation, especially in large-scale settings?\n3. Have the authors considered whether sparse approximations (e.g., k-nearest neighbor graphs) could reduce computational cost while preserving the theoretical guarantees (uniqueness, completeness, structure preservation) of the Indra representation?\n4. Can the authors provide qualitative examples or case studies where Indra representations yield more semantically aligned or informative matches than original embeddings?\n5. How robust is the Indra representation to the quality of base embeddings? Specifically, how does it behave when the embeddings are noisy or derived from undertrained or small-capacity models?\n6. The paper cites prior work on lightweight post-hoc alignment methods, such as linear projections (e.g., [50]), which operate under broadly similar goals. Could the authors elaborate on the decision not to include these as empirical baselines?", "limitations": "Yes, the authors have addressed the limitations and potential negative societal impact of their work. One suggestion for improvement would be to include a discussion of scalability limitations, especially regarding the complexity of computing full pairwise distance matrices.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\n* The paper introduces a novel and compelling framing of representation learning using the Indra’s Net metaphor.\n* The use of category theory and the Yoneda embedding provides an interesting foundation for relational representations.\n* The approach is post-hoc and model-agnostic, so it can be applied to any pretrained unimodal encoder without retraining or supervision.\n* Empirical validation is provided across multiple modalities (vision, language, audio), model architectures, and datasets, showing consistent performance gains.\n* The method is shown to be somewhat robust to the choice of cost function (angular vs. Euclidean distance).\n\n**Weaknesses**\n* The method requires computing and storing $n \\times n$ distance matrices (Eq. 4), resulting in quadratic complexity in dataset size. The paper does not analyze runtime or memory scaling, nor does it explore sparse or approximate alternatives and their impact on theoretical guarantees.\n* The paper could benefit from qualitative visualizations or interpretability analyses to provide intuition on what kinds of relations are being captured.\n* The evaluation is limited to retrieval-style tasks using similarity-based matching metrics. The effectiveness of Indra representations on other downstream tasks (e.g., classification, generation, etc) is not explored. Evaluating such tasks would help clarify whether the benefits of the relational structure extend beyond similarity matching.\n* The paper cites prior work demonstrating that lightweight post-hoc alignment methods, such as linear projections or Procrustes alignment, can effectively bridge pretrained unimodal encoders (e.g., Merullo et al. [50], Sharma et al. [62]). However, it does not include these as baselines in the empirical evaluation. Comparing against these methods, especially in low-data or zero-shot variants, could clarify whether the gains arise uniquely from the relational structure or could be matched by simpler geometric transformations.", "was_revised": true, "final_justification": "The rebuttal addressed most of my concerns, especially regarding the scalability of the method through approximate techniques and the added runtime analysis. The additional experiments on robustness and downstream classification tasks strengthen the empirical evidence for the utility of the Indra representation. While some aspects remain underexplored (ie the qualitative understanding of the captured relations and comparisons to baseline alignment methods), I find the core idea novel and the analysis interesting.", "rebuttal": "Dear Reviewer 6ioa,\n\n**We sincerely appreciate you for providing useful suggestions and for recognizing our idea as novel and compelling**.\n\nBelow, we address your remaining concerns. \n\n**W1&Q2&Q3 Complexity**\n\n*[Complexity of Exact Computation]* Due to space limitations, we kindly refer you to our response to Reviewer yaBd (W1 & Q3) for the complexity analysis.\n\n*[Application-Oriented (Scalable) Solutions]* ***The scalability concern is addressable in practice***. In the literature, there exists a rich body of work on approximating pairwise distances efficiently. For example, approximate nearest neighbor search (e.g., FAISS, HNSW), landmark-based approximation (e.g., K-means centroids, random subsampling), hashing-based methods (locality sensitive hashing), and sparsified graph constructions. From an application view, these techniques can be readily adapted to approximate the Indra representation at scale without sacrificing its structural interpretation.\n\n*[Experimental Validation]* Due to space limitations, we kindly refer you to our response to Reviewer yaBd (W1 & Q3) for the corresponding results and analysis.\n\n*[Runtime Analysis]* The table below reports the runtime for constructing Indra representations on the CC3M dataset using a landmark-based approximation with varying numbers of landmarks. All experiments are conducted on an NVIDIA L40S GPU, and each setting is run 20 times to obtain the average runtime.\n\n|#of landmarks|1000|2000|3000|5000|\n|--|--|--|--|--|\n|runtime (s)|0.0476 |0.0666|0.0888|0.1401|\n\n*[Theoretical Guarantees]* Approximate solutions, such as landmark-based methods, may not guarantee uniqueness, completeness, or structure preservation, unless the selected subcategory of landmarks is sufficiently large or dense to distinguish all objects in our sample category. In such cases, we may consider leveraging categorical tools, such as Kan extensions, to extend the behavior of objects defined on the subcategory to the entire category, thereby approximating the full Indra representation. However, this direction lies beyond the scope of the current paper. We will explore this in our future work.\n\n\n**W2 Intuition on captured relations**\n\nThank you for the helpful suggestion. The types of relations captured by our method are influenced by both the choice of cost function and the foundation model, and are not deterministic. To better understand these relations, we perform visualization analyses using t-SNE and clustering algorithms. Due to rebuttal policy constraints, we are unable to include qualitative visualizations at this stage, but we will incorporate them in the revised version of the paper.\n\n**W3&Q1&Q5 Other downstream tasks & robustness**\n\nWe additionally report image classification results on the CIFAR-10, CIFAR-100, and Office-Home datasets under various distance metrics. For CIFAR-10 and CIFAR-100, we use the standard data splits provided by torchvision.datasets. For Office-Home, we evaluate classification accuracy across four domains: Art (A), Clipart (C), Product (P), and Real-World (R), using an 80/20 split for training and testing. Across all datasets, we adopt logistic regression (linear probing) to assess the quality of the extracted representations. \n\nTo investigate the robustness of noisy representations, we inject Gaussian noise into the features with varying standard deviations $\\sigma  \\in \\{0.0, 1.5, 3.0, 5.0, 7.0\\}$. For each noise level, we perturb the features accordingly and train a linear classifier on the noisy representations. This allows us to assess how classification performance degrades as the feature representations are increasingly corrupted by noise.The corresponding results are presented below.\n\n|cifar10|$\\sigma$=0.0|$\\sigma$=1.5|$\\sigma$=3.0|$\\sigma$=5.0|\\$\\sigma$=7.0| |cifar100|$\\sigma$=0.0|$\\sigma$=1.5|$\\sigma$=3.0|$\\sigma$=5.0|\\$\\sigma$=7.0|\n|--|--|--|--|--|--|--|--|--|--|--|--|--|\n|ViT|93.98|89.60|87.75|79.77|68.15| | |79.45|70.40|54.69|35.76|27.45|\n|Ours (Angular)|**94.72**|**92.90**|**88.08**|**79.84**|**68.16**| | |**80.51**|**74.48**|**61.12**|**42.29**|**28.49**|\n|Ours (Euclidean)|**94.84**|**93.15**|**89.51**|**80.84**|**68.71**| | |**80.09**|**77.36**|**69.00**|**51.59**|**32.74**|\n| | | | | | | | | | | | | |\n|ConvNeXt|97.00|94.37|85.89|80.10|65.85| | |85.77|78.56|62.79|34.39|21.28|\n|Ours (Angular)|**97.20**|**95.22**|**90.77**|**80.18**|**65.93**| | |85.74|**80.30**|**66.04**|**41.58**|**24.45**|\n|Ours (Euclidean)|**97.21**|**96.06**|**92.86**|**81.59**|**66.64**| | |85.64|**82.25**|**72.16**|**51.51**|**30.25**|\n| | | | | | | | | | | | | |\n|DinoV2|99.19|98.58|95.21|85.57|76.54| |  |91.97|89.24|82.21|63.06|40.16|\n|Ours (Angular)|99.19|**98.80**|**96.59**|**87.39**|**76.68**| | |91.77|**89.72**|**83.72**|**67.53**|**47.80**|\n|Ours (Euclidean)|99.14|**98.72**|**96.87**|**89.73**|**77.92**| | |91.93|**89.76**|**84.83**|**74.29**|**58.67**|\n\n|Noise levels|A $\\sigma$=0.0|A $\\sigma$=3.0|A $\\sigma$=5.0|A $\\sigma$=7.0|C $\\sigma$=0.0|C $\\sigma$=3.0|C $\\sigma$=5.0|C $\\sigma$=7.0|P $\\sigma$=0.0|P $\\sigma$=3.0|P $\\sigma$=5.0|P $\\sigma$=7.0|R $\\sigma$=0.0|R $\\sigma$=3.0|R $\\sigma$=5.0|R $\\sigma$=7.0|\n|--|--|--|--|--|--|--|--|--|--|--|--|--|--|--|--|--|\n|ViT|80.25|64.40|44.03|22.63|73.20|50.40|28.64|15.23|92.34|80.74|61.15|35.25|89.22|82.11|60.09|35.32|\n|Ours (Angular)|79.63|**65.02**|43.62|**27.57**|69.76|**54.98**|**33.10**|**18.21**|89.75|**81.53**|**64.08**|**40.77**|87.16 |**83.49**|**63.65**|**40.48**|\n|Ours (Euclidean)|80.04 |63.37|43.21|**25.51**|70.33|**55.78**|**34.48**|**18.33**|89.64 |80.74|**64.75**|**40.77**|87.16|**83.37**|**64.56**|**41.28**|\n| | | | | | | | | | | | | | | | | |\n|ConvNeXt|89.71|62.76|27.98|12.14|83.62|54.07|20.85 |09.74|96.62|84.91|44.26|19.37|93.46|82.11|38.30|17.78|\n|ours (Angular)|87.86|59.88|**28.81**|**14.20**|82.70|**57.85**|**25.09**|**11.34**|**96.73**|**85.92**|**45.61**|**22.18**|93.35|**84.63**|**40.71**|**19.61**|\n|Ours (Euclidean)|87.45|58.02|26.75|**12.96**|82.93|**58.30**|**23.83**|**11.23**|96.28|**86.26**|**45.38**|**21.73**|93.35|**84.75**|**40.14**|**19.04**|\n| | | | | | | | | | | | | | | | | |\n|DinoV2|87.65|73.05|46.91|27.78|88.43|75.14|51.09|31.04|96.73|93.24|83.33|60.70|92.78|87.39|71.44|48.51|\n|Ours (Angular)|87.04|70.99|**47.53**|27.37|87.29|**76.63**|**54.75**|**33.56**|96.40|92.79|**84.46**|60.59|**92.89**|**88.53**|**73.17**|**49.89**|\n|ours (Euclidean)|86.21|68.52|44.65|27.16|87.29|**76.75**|**55.21**|**34.59**|96.51|92.57|**84.12**|60.02|92.55|**88.53**|**73.39**|**49.54**|\n\nThe results clearly show that stronger backbone models (e.g., DINOv2) lead to better performance for Indra representations across all noise levels. For instance, on CIFAR-100 with $\\sigma=0.0$, Indra (Euclidean) achieves 91.93% accuracy using DINOv2 features, compared to 85.64% with ConvNeXt and 80.09% with ViT. This performance gap persists and even widens under higher noise; at σ=7.0\\sigma=7.0σ=7.0, Indra (Euclidean) with DINOv2 maintains 58.67%, while ConvNeXt and ViT drop to 30.25% and 32.74%, respectively. In addition, as Gaussian noise increases, our Indra representations consistently retain higher classification accuracy compared to the original features, highlighting their robustness. The performance gains of Indra representations hold across multiple backbone architectures (ViT, ConvNeXt, DINOv2), indicating the broad applicability of the proposed method.\n\n**W4&Q6 Baselines**\n\nThe cited methods such as [50] are application-driven with a primary goal of improving cross-modal alignment. They achieve this goal by training additional modules on top of frozen backbones, such as linear projections and adapters. As a result, ***these methods typically require substantial fine-tuning on large-scale datasets to learn optimal alignment mappings***. Once trained, they become explicitly aligned models and are therefore more appropriately compared to models like CLIP, which are also designed with explicit alignment.\n\nIn contrast, our work is exploratory in nature, with the main focus on investigating what kind of representation unimodal foundation models inherently converge to. While prior works suggest that different unimodal models exhibit convergence in their output representations, they generally assume that the last-layer outputs themselves constitute this convergent form. We challenge this assumption and hypothesize that there exists a more structured and alignment-friendly representation, i.e., what we propose as the Indra representation. To validate this hypothesis, we compare Indra representations with the original model outputs, showing that Indra representations lead to better alignment performance without any model retraining.\nIt is therefore clear that our objective fundamentally differs from that of those methods. While they proactively design and train new modules to enforce alignment, our approach seeks to passively uncover whether existing, frozen models already contain a more suitable representation structure when viewed from a relational perspective.\nIn summary, while these alignment methods achieve improved performance through training additional layers, our method reveals that even without training, unimodal foundation models may admit a more optimal representational form than their raw outputs. ***This key difference in motivation, methodology, and training requirements makes a direct empirical comparison with those methods unfair and potentially misleading***.\n\n**Q4 Qualitative examples**\n\nThank you for the constructive feedback. We have conducted qualitative comparisons demonstrating that Indra representations yield more semantically aligned matches than the original embeddings. Due to rebuttal policy constraints, we are unable to include qualitative examples at this stage, but we will incorporate them in the revised version of the paper.\n\n*We hope the above response adequately addresses your concerns. Once again, thank you for your thoughtful feedback and for helping us improve the quality of our paper.*"}, {"review_id": "di5JwKFKna", "reviewer": "Reviewer_7Tyf", "summary": "This paper introduces the \"Indra Representation Hypothesis,\" which states that the representations learned by unimodal foundation models converge towards a shared relational structure. The authors formalize this hypothesis using the V-enriched Yoneda embedding from category theory. This leads to a practical method where a sample's representation is redefined as its vector of distances to all other samples in a given dataset. This distance vector, termed the Indra representation, is computed from the outputs of pre-trained unimodal models. The authors provide theoretical claims that this representation is unique, complete, and structure-preserving. The method is evaluated on cross-model and cross-modal matching tasks involving vision, language, and audio, where it is shown to consistently improve matching accuracy over the original model embeddings.", "questions": "1)The proposed method has a computational complexity of $O(n^2d)$ and produces n-dimensional representations. Please comment on the method's viability for large-scale datasets. More importantly, why was the extensive literature on kernel approximation methods, which are designed to mitigate these exact scalability issues, not considered, implemented, or benchmarked against?\n\n2) The theoretical framework guarantees that the Indra representation preserves the structure induced by the cost function d. However, it provides no guidance on how to select d. How does the theory inform this crucial choice? For instance, your ablation shows Euclidean distance also works; does the theory suggest when angular distance might be superior to Euclidean, or vice-versa?\n\n3) Could you refine your claim regarding the \"structural myopia\" of existing representations? How do you distinguish the relational information captured by your method (inter-sample distances) from the relational information already captured by mechanisms like self-attention (inter-token dependencies)?", "limitations": "Yes, the authors provided a limitations section in the supplementary material. However, it is wholly inadequate. The authors completely sidestep the critical issue of scalability. Instead, they frame the limitation as their method being \"post-training adaptation\" and suggest incorporating the ideas into pretraining as future work.  This is not a meaningful limitation but rather a description of the work.", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "The paper's primary contribution is a novel and elegant conceptualization of representation convergence. The analogy to Indra's Net, combined with the formalization using category theory, provides a thought-provoking new lens through which to view the properties of embeddings from large-scale models.\nThe proposed method is a simple, post-hoc transformation that requires no retraining of the foundation models. Its generality allows it to be applied to any model that produces vector embeddings, which is a significant practical advantage.\nAcross the conducted experiments in single-modality and cross-modal settings, the use of the Indra representation demonstrates a modest improvement in matching performance over the baseline of using raw embeddings.\n\nThe proposed method is fundamentally non-scalable. To construct representations for a dataset of n samples with embedding dimension d, one must compute an n x n distance matrix, which has a computational complexity of at least $O(n^2d)$ and a memory complexity of $O(n^2)$. The resulting Indra representation for each sample is an n-dimensional vector. This scaling behavior makes the method computationally infeasible for datasets beyond the small-scale validation sets used in the paper (e.g. $n>10^5$). This is, in essence, a kernel method that computes the full kernel matrix. The paper shows a critical lack of rigor by failing to acknowledge this.\n\nThe appeal to enriched category theory, while elegant, does not appear to offer concrete guidance for the method's implementation. The Yoneda Lemma guarantees that the mapping to a functor category is fully faithful, preserving all information contained within the chosen cost function d. However, this is a tautology: the representation is complete only with respect to the chosen distance metric. The theory provides no principles for selecting an optimal cost function, which is the single most important design choice in the entire method. Any valid metric space could be used to motivate the same procedure, making the complex theoretical overhead seem more justificatory than generative.\nThe empirical evaluation is weak because it only compares the Indra representation against the original embeddings. To rigorously assess a new method for post-hoc representation alignment, it must be benchmarked against established techniques. By omitting these comparisons, the paper fails to demonstrate that its proposed method offers a significant advantage over simpler, well-known alternatives.", "was_revised": true, "final_justification": "The paper presents an interesting hypothesis but does so in a scientific way. It proposes a non-scalable method that is equivalent to computing a kernel matrix without acknowledging or engaging with the literature on kernel methods. It fails to compare against any established baselines, making its empirical claims difficult to assess. The attempt to excuse these omissions by reframing the paper's goals post-submission is not a valid defense. Therefore, I conclude that the paper, in its current form, does not meet the standards for publication.", "rebuttal": "Dear Reviewer 7Tyf,\n\nWe sincerely appreciate you for providing thoughtful and constructive comments. \n\n**We are glad to see you recognize our idea as novel and elegant, providing a thought-provoking new lens to view the properties of embeddings of foundation models.**\n\nBelow, we address your remaining concerns.\n\n**W1&Q1 Scalability & Complexity**\n\n*[Complexity of Exact Computation]* We agree with you that constructing ***exact*** Indra representations requires a computational complexity of $\\mathcal{O}(n^2d)$ and a memory complexity of $\\mathcal{O}(n^2)$ for a dataset with $n$ samples and embedding dimension $d$. This quadratic scaling potentially limits the direct applicability of the exact Indra representations to large-scale datasets.\n\n*[Our Research Focus]* However, we respectfully clarify that *the goal of this work is not to propose a new application-driven multi-modal alignment method*. Rather, our focus lies in understanding the fundamental nature of the representations learned by existing unimodal foundation models. Specifically, *we aim to investigate the forms these representations tend to converge to and to identify their ultimate convergent targets*.\n\nIn this paper, we argue that unimodal foundation models converge toward a form of Indra representation, which implicitly reflects the relational structure underlying reality (i.e., The Indra Representation Hypothesis). *All our experiments are designed to validate the effectiveness of this form.* \n\n*[Application-Oriented Solutions]* Moreover, ***the scalability concern is addressable in practice***. In the literature, there exists a rich body of work on approximating pairwise distances efficiently. For example, approximate nearest neighbor search (e.g., FAISS, HNSW), landmark-based approximation (e.g., K-means centroids, random subsampling), hashing-based methods (locality sensitive hashing), and sparsified graph constructions (as suggested by Reviewer 6ioa). From an application view, these techniques can be readily adapted to approximate the Indra representation at scale without sacrificing its structural interpretation.\n\n*[Experimental Validation]* To evaluate the effectiveness of approximated Indra representations, we adopt a simple landmark-based approximation via random subsampling. We assess this approach on two tasks: image classification on the CIFAR-100 dataset and image-text matching on the large-scale CC3M dataset. For CIFAR-100, we use the standard data split provided by torchvision.datasets and employ logistic regression (linear probing) to evaluate classification accuracy. For CC3M, we sample 300,000 image-text pairs for evaluation. We extract image features with DINOv2 and text features with RoBERTa. For each image, we compute mean CLIP scores with the top-$k$ candidate texts and compare the performance of the original features against that of the approximated Indra representations.\n\n\nDue to space limitation, ***we kindly refer you to our response to Reviewer yaBd (W1 & Q3) for the experimental details and results.***\n\n**W2&Q2 Distance metric**\n\nWe believe there is a misunderstanding regarding the role of enriched category theory in our method. ***The theory is not intended to guide the selection of an optimal cost function for practical applications***. Instead, its purpose is to establish that the Indra representation obtained through our construction is both faithful and complete with respect to the chosen cost function. *The Yoneda Lemma indeed cannot prescribe an optimal cost function for the downstream tasks.*\n\nIn practice, *it is not feasible to define a universally optimal cost function across different tasks*. As our experimental results demonstrate, angular distance may outperform Euclidean distance in some settings, while the opposite may be true in others. This highlights that the choice of cost function should be task-dependent, and the theory is not meant to replace empirical validation.\n\nFurthermore, *not all distance functions are valid cost functions in our framework*. To be used as a cost function for the Indra representation, the cost function should satisfy the following properties:\n\nReflexivity: $d(x, x) = 0$  \nNon-negativity: $d(x, y) \\in [0, \\infty)$   \nTriangle inequality: $d(x, z) \\leq d(x, y) + d(y, z)$   \n\nIn summary, *enriched category theory is not used to justify the method post hoc, but to guarantee structural faithfulness and completeness within a mathematically grounded setting.*\n\n**W3& Baselines**\n\nExisting alignment methods, e.g., [50], need additional modules on top of frozen backbones, such as linear projections and adapters, and require substantial fine-tuning on large-scale datasets to learn alignment functions. Once trained, these models become explicitly aligned and are thus more appropriately compared to models like CLIP, which are also designed with explicit alignment.\n\nIn contrast, our work aims to investigate the convergent forms and targets of representations learned by unimodal foundation models. Our objective fundamentally differs from that of alignment-driven methods: ***while they actively design and train new modules to enforce alignment, our approach passively explores whether frozen models already contain a more suitable internal representation that supports better cross-modal alignment.***\n\nWe uncover that even without any additional training, unimodal foundation models may admit a more optimal representational form than their raw output features. This key difference in motivation, methodology, and training requirements makes a direct empirical comparison with those methods unfair and potentially misleading.\n\n*To the best of our knowledge, this is the first work to demonstrate that frozen representations without any task-specific tuning can achieve improved alignment performance through our proposed representation method, not only within a single modality, but also across vision-language and speech-language modalities.*\n\n**Q1 Kernel approximation methods**\n\nWe kindly remind the reviewer that the primary goal of this paper is to investigate the representational forms that unimodal foundation models converge to, and to identify their ultimate convergent targets, rather than to accelerate the proposed method through approximation.\n\nIn this work, we hypothesize that these models tend to converge toward a form we define as the Indra representation. Accordingly, all our experiments are designed to validate the effectiveness of this representation, demonstrating that it consistently achieves better alignment than the original model outputs without any additional training.\n\nIn fact, *using approximation techniques (e.g., kernel approximation methods) to construct the Indra representation and empirically demonstrate its performance primarily serves as a validation of the Indra representation hypothesis introduced in this paper.*\n\n\n**Q3 Structural myopia & self-attention**\n\nPrior studies on representation convergence often treat the representations produced by foundation models as isolated carriers of information. They analyze convergence purely based on model outputs in a pointwise or instance-level manner. This perspective overlooks the underlying structural relationships embedded within the broader data manifold. In contrast, we argue that model outputs do not directly represent the final converged form, but rather provide the initial scaffold upon which convergence should be understood. We contend that truly convergent representations must be structure-dependent.\n\nWhile both our method and self-attention involve pairwise relationships, they differ fundamentally in nature and purpose. Our method constructs a representation for each sample based on its static, global relationship to others in the dataset, using a predefined cost function. This results in a complete, non-parametric embedding that faithfully captures the sample's position within the overall data geometry. The representation is invariant across contexts and tasks, and fully determined by the relational structure of the data itself.\n\nIn contrast, self-attention models dynamic, context-dependent interactions among tokens, where the strength of each relationship is learned via parameterized query-key-value projections and varies across different inputs and layers. Self-attention embeddings are not faithful to the original data geometry; instead, they adapt to downstream objectives and are shaped by the model's learned parameters.\n\nIn short, our method is grounded in global structure and faithful representation, while self-attention captures how a token should change based on its current context and specific tasks.\n\n\n**L1 Limitation section revision**\n\nThank you for the thoughtful comments. Our experimental results validate the hypothesis by demonstrating improved alignment performance using frozen unimodal foundation models. We believe it is also meaningful to explore this hypothesis in the pretraining stage, to investigate how learning dynamics evolve when the training objectives align with the representational convergent targets.\n\nWe have addressed related concerns in the responses above and will revise the limitations section accordingly. In particular, we will clarify our focus on theoretical insights and empirical findings, explicitly acknowledge the computational challenges associated with computing exact Indra representations, and include a discussion of scalable approximation strategies along with the corresponding experimental results.\n\n\n*We hope the above response adequately addresses your concerns. Once again, thank you for your thoughtful feedback and for helping us improve the quality of our paper.*"}]

Author rebuttal:

Meta-review: The paper proposes the Indra Representation Hypothesis: unimodal models converge to a relational structure formalized with a V-enriched Yoneda embedding. It builds distance-based “Indra” vectors and shows post-hoc gains for cross-model and cross-modal matching, plus robustness, using landmark/ANN approximations. Two reviewers praise the novelty and the stronger rebuttal, and recommend accept or borderline accept. Two others question practical need versus multimodal models, O(n²) cost and storage, and guidance on metric choice; one barely engaged.

I recommend accept, but this is borderline from my perspective. The idea is fresh and now better supported by added baselines (linear projection, CCA) and runtime/large-scale results. Scaling and metric-choice risks remain, but approximations and sensitivity checks help, and the camera-ready should expand limitations and fix notation. With split reviews and moderate confidence, I still see clear value for the community.
Decision: Accept (poster)