Title: The Indra Representation Hypothesis for Multimodal Alignment
Venue: NeurIPS 2025
Accepted: yes
Keywords: Foundation Models

Representation Learning

Multimodal Alignment

Representation Convergence
Abstract: Recent studies have uncovered an interesting phenomenon: unimodal foundation models tend to learn convergent representations, regardless of differences in architecture, training objectives, or data modalities. However, these representations are essentially internal abstractions of samples that characterize samples independently, leading to limited expressiveness. In this paper, we propose The Indra Representation Hypothesis, inspired by the philosophical metaphor of Indra’s Net. We argue that representations from unimodal foundation models are converging to implicitly reflect a shared relational structure underlying reality, akin to the relational ontology of Indra’s Net. We formalize this hypothesis using the V-enriched Yoneda embedding from category theory, defining the Indra representation as a relational profile of each sample with respect to others. This formulation is shown to be unique, complete, and structure-preserving under a given cost function. We instantiate the Indra representation using angular distance and evaluate it in cross-model and cross-modal scenarios involving vision, language, and audio. Extensive experiments demonstrate that Indra representations consistently enhance robustness and alignment across architectures and modalities, providing a theoretically grounded and practical framework for training-free alignment of unimodal foundation models. Our code is available at https://github.com/Jianglin954/Indra.

The Indra Representation Hypothesis for Multimodal Alignment

Abstract

Recent studies have uncovered an interesting phenomenon: unimodal foundation models tend to learn convergent representations, regardless of differences in architecture, training objectives, or data modalities. However, these representations are essentially internal abstractions of samples that characterize samples independently, leading to limited expressiveness. In this paper, we propose The Indra Representation Hypothesis , inspired by the philosophical metaphor of Indra's Net. We argue that representations from unimodal foundation models are converging to implicitly reflect a shared relational structure underlying reality, akin to the relational ontology of Indra's Net. We formalize this hypothesis using the V -enriched Yoneda embedding from category theory, defining the Indra representation as a relational profile of each sample with respect to others. This formulation is shown to be unique, complete, and structure-preserving under a given cost function. We instantiate the Indra representation using angular distance and evaluate it in cross-model and cross-modal scenarios involving vision, language, and audio. Extensive experiments demonstrate that Indra representations consistently enhance robustness and alignment across architectures and modalities, providing a theoretically grounded and practical framework for training-free alignment of unimodal foundation models. Our code is available at https://github.com/Jianglin954/Indra.

1 Introduction

Through large-scale pretraining, foundation models have emerged as a transformative paradigm in artificial general intelligence, demonstrating impressive progress across diverse domains, such as natural language processing [11, 25, 46], computer vision [47, 77, 61, 12], and speech processing [66, 2]. These unimodal models are typically trained on web-scale datasets and acquire generalized representations that can be adapted to a broad range of downstream tasks. The representative models, such as BERT [11] for language, ViT [12] for vision, and Wav2Vec [66] for audio, have demonstrated promising performance within their respective domains.

Since real-world information is inherently multimodal (text, images, and audio frequently co-occur and complement each other), relying on a single modality for understanding is typically insufficient. To extend unimodal foundation models for cross-modal tasks, a growing body of research has exploited strong unimodal encoders as the core components to build multimodal systems [62, 42, 63]. They assume that unimodal models are able to provide specialized, high-quality, and high-level representations for each individual modality, which can then be aligned, fused, or bridged to enable multimodal interactions [41, 45]. A common strategy to achieve multimodal understanding is to align unimodal outputs in a shared representation space through cross-modal objectives [62, 41, 45, 55].

This usually relies on external mechanisms, such as alignment losses, fusion modules, and prompt tuning, thus requiring large-scale datasets and extensive retraining for modality alignment.

Interestingly, recent studies [60, 58, 45, 67] suggest that powerful unimodal models may already exhibit latent cross-modal capabilities, as the representations they produce (when grounded on the same physical entity) tend to describe the same underlying semantics from different sensory perspectives. Previous evidence has revealed that adding only a single linear transformation is capable of bridging an auditory model with a large language model (LLM)[59], integrating a vision model into an LLM [55], or conversely, stitching an LLM towards a vision model [36]. Even without retraining, well-pretrained vision encoders exhibit high semantic similarity with language encoders [52]. Further studies [29, 34, 27, 49] have revealed that models trained on different data modalities converge, as different models are all trying to arrive at a representation of reality. Thus, unimodal models may encode modality-agnostic representations in abstract representational space, even without explicit alignment. While conceptually appealing, the specific form of convergent representations and their eventual convergence targets remain elusive and largely unexplored.

In this paper, we posit that the underlying convergent representation is inherently the Indra Representation-a conceptual abstraction inspired by the philosophical metaphor of Indra's Net. Originating in ancient Buddhist philosophy, Indra's Net describes a vast, infinite web of jewels, each reflecting all others. Every jewel is both a part and a reflection of the whole, suggesting that all phenomena are interdependent, mutually defining, and inherently connected. We draw an analogy between this worldview and the notion of representation convergence, and introduce The Indra Representation Hypothesis : well-trained unimodal models tend to produce convergent representations that implicitly reflect a shared relational structure underlying reality, echoing the relational ontology of Indra's Net . In this view, the representation of each entity is not defined in isolation, but rather emerges from its relational context, i.e., its reflections of all other entities.

To explore this hypothesis, we introduce a theoretical definition of the Indra representation grounded in category theory [33]. Specifically, we define it as the V -enriched Yoneda embedding of a sample within a category enriched over a Cost -category. This formulation effectively maps each sample to its covariant Hom-functors in the sample category, thereby encoding its relational profile within the structure of the dataset. We provide theoretical guarantees that the Indra representation is unique, complete, and structure-preserving, offering a principled foundation for its effectiveness. In particular, we prove that it uniquely and completely characterizes each sample within the relational structure induced by a given cost function, while preserving essential properties of that structure.

To instantiate this theory in practice, we adopt angular distance as the cost function, yielding a simple yet powerful realization of the Indra representation. This concrete formulation enables empirical evaluation and allows us to investigate how Indra representations can uncover and support the latent cross-modal capabilities of unimodal models. We validate our approach across a range of scenarios involving cross-model and cross-modal understanding, including single-modality settings, visionlanguage pairs, and speech-language pairs. Extensive experiments demonstrate the effectiveness and generality of the proposed Indra representation across different architectures and modalities.

2 Preliminaries

2.1 Indra's Net

Indra's Net is a philosophical metaphor originating in ancient Indian and Mah¯ ay¯ ana Buddhist thought, particularly from the Avatam . saka S¯ utra [9, 22]. It is used to symbolize the universe as a web of interdependent connections among all of its members, expressing the concept of interconnectedness, non-duality, and the interpenetration of all phenomena. Francis H. Cook describes it as [10]:

Imagine a vast, infinite cosmic net belonging to the god Indra. At each node of the net is a jewel or crystal. Each jewel reflects every other jewel in the net, and in each of those reflections are further reflections of all other jewels, recursively and infinitum.

The metaphor of Indra's Net resonates deeply with the foundational principles across diverse disciplines. For instance, Gergen et al. [17] posit that identities, thoughts, and actions are not solely products of isolated minds but are co-constructed through interactions and relationships with others. Markus et al. [53] propose the theory of interdependent self-construal, emphasizing that the self is

defined relationally through one's social roles, group memberships, and interpersonal obligations. In physics, field theory [14, 54] illustrates a relational structure where the field at any point depends on all sources throughout space, a concept reminiscent of Indra's Net, in which each jewel reflects and is reflected by all others. Similarly, modern particle physics [18, 4] reveals the properties of an elementary particle through its interactions with other particles. In linguistics, the linguistic principle articulated by J.R. Firth [15], 'You shall know a word by the company it keeps", asserts that a word's meaning is derived from its co-occurrence with other words. This principle forms the basis for modern language models like Word2Vec [56] and its successors. Furthermore, Kasulis [32] suggests DNA as a better image of Indra's Net, where every cell contains the blueprint for the whole organism.

2.2 Representation Convergence

Recent studies [29, 76, 20, 74, 40] have revealed a striking phenomenon: unimodal foundation models tend to learn convergent representations, regardless of their architectures, training objectives, or data modalities. For example, Bürger et al. [5] show that a two-dimensional representation of truth emerges universally across LLMs of varying sizes and from different model families. Roeder et al. [65] prove that a wide class of discriminative and autoregressive models are identifiable in function space up to a linear transformation. Tan et al. [68] find strong correlations in both in-distribution and out-of-distribution steerability between LLaMA [69] and Qwen [3]. Huh et al. [29] attribute this convergence to a shared goal: approximating an underlying representation of reality. Khosla et al. [34] argue that both artificial and biological systems converge toward representations that capture the causal structure of the world. Hosseini et al. [27] further observe that high-performing artificial neural networks and biological brains tend to develop similar internal representations under naturalistic training conditions. Additional evidence and analyses on representation convergence can be found in the comprehensive survey [49].

Despite this emerging consensus, it remains unclear how these representations converge and what they ultimately converge to. Notably, prior studies rely primarily on model outputs (embeddings) as proxies for representations, but these representations suffer from ① structural myopia: representations are typically treated as isolated carriers of information, ignoring structural interrelations within the broader data manifold; ② limited expressiveness: unimodal representations often exhibit inferior quality in matching and alignment compared to those from multimodal foundation models; and ③ dimensional incompatibility: representations across models and modalities often differ in dimensionality, thus requiring additional post-processing for cross-modal matching. In light of these challenges, we argue that representations from model outputs do not reflect the final converged form, but instead serve as the foundation upon which such a form can be built . In the next section, we introduce a novel representation hypothesis inspired by the metaphor of Indra's Net to hypothesize the concrete structure of convergent representations and illuminate what they ultimately converge to.

3 Methodology

3.1 The Indra Representation Hypothesis

In this paper, we advocate a shift in perspective on representation convergence, inspired by the metaphor of Indra's Net: a sample should be represented not in isolation, but through its pattern of relationships to other samples. In this view, representations emerge from a structure of mutual interdependence. We formalize this perspective through the Indra Representations Hypothesis:

The Indra Representation Hypothesis : Neural networks, trained with different objectives on different data and modalities, tend to learn convergent representations that implicitly reflect a shared relational structure underlying reality-parallel to the relational ontology of Indra's Net.

This hypothesis posits that unimodal foundation models, after extensive pretraining, tend to produce representations that converge to capture the inherent relational structure of reality-a structure characterized by interdependence, contextuality, and mutual influence. However, current methods that treat model outputs as final representations fail to reflect this structure. These methods typically emphasize individual embedding information while neglecting the crucial relational patterns between samples. In the next section, we introduce the Indra Representation , a novel representation framework inspired by the metaphor of Indra's Net, to reveal the underlying relational structure. In this framework,

representations are not independent embeddings but mutually reflective entities woven into a web of interdependent relationships, revealing the deeper relational structure that underpins the data.

3.2 From Metaphor to Theory

Indra's Net is a philosophical metaphor that symbolizes the interconnectedness of the universe. It aligns with the foundational principles in modern science, as mentioned in Section 2.1. To translate this philosophical insight into representation learning, we first introduce the Yoneda Lemma and its corollary, which provide the theoretical foundation for defining our proposed Indra representation.

Lemma 1 (Yoneda Lemma [33, 64]) . Let C be a locally small category, A be an object in C , and F : C → Set be a functor from C to the category of sets. Then, there exists a bijection, natural in both A and F , between the set of natural transformations from the hom-functor h A = Hom C ( A, -) to F , and the set F ( A ) . This bijection is given by:

Corollary 1 (Yoneda Embedding [33, 64]) . For any two objects A,B in a locally small category C , there is a bijection:

This demonstrates that the functor Y : C op → [ C , Set ] , defined by Y ( A ) = h A = Hom C ( A, -) , is fully faithful. This functor Y is known as the Yoneda embedding.

The Yoneda Lemma provides a profound understanding of how an object in a category is characterized by its relationships (morphisms) with all other objects, rather than by its internal properties. Its corollary further shows that any locally small category C can be embedded into a category of presheaves on C . To introduce our Indra representations, we give the following definitions:

Definition 1 (Sample Category) . Let X be a set of samples, possibly infinite. The sample category C enriched over the Cost -category V = ([0 , ∞ ] , ≥ , 0 , +) consists of: ① Objects: Ob ( C ) = X . ② Hom-objects: for every X i , X j ∈ Ob ( C ) , the hom-object C ( X i , X j ) is given by a cost function d ( X i , X j ) ∈ [0 , ∞ ] , which is an object in V . ③ Identity: for all X i ∈ Ob ( C ) , the identity morphism id X i : I → C ( X i , X i ) in V is 0 → d ( X i , X i ) , where d ( X i , X i ) = 0 . ④ Composition: for all X i , X j , X k ∈ Ob ( C ) , the composition morphism M X i ,X j ,X k : C ( X j , X k ) ⊗ C ( X i , X j ) → C ( X i , X k ) in V is: d ( X j , X k ) + d ( X i , X j ) → d ( X i , X k ) . This morphism exists in V if and only if d ( X j , X k ) + d ( X i , X j ) ≥ d ( X i , X k ) , which is precisely the triangle inequality.

Definition 2 ( V -enriched Yoneda embedding) . Let [ C op , V ] be the category of V -presheaves on C . The V -enriched Yoneda embedding is a V -functor Y : C → [ C op , V ] . For each object X i ∈ Ob ( C ) , Y ( X i ) is the V -presheaf h X i : C op →V defined by: h X i ( X j ) = C ( X j , X i ) = d ( X j , X i ) for any X j ∈ C op . For every X i , X j ∈ Ob ( C ) , Y defines a map Y X i ,X j : C ( X i , X j ) → [ C op , V ]( Y ( X i ) , Y ( X j )) .

Theorem 1. The V -enriched Yoneda embedding Y : C → [ C op , V ] for the sample category C enriched over V = ([0 , ∞ ] , ≥ , 0 , +) with the cost function d is V -fully faithful.

The sample category C actually forms a Lawvere metric space [39], and the corresponding V -enriched Yoneda embedding maps each sample X i to a functor Y ( X i ) , which captures the cost profile from all other samples to X i . Theorem 1 shows that the sample category can be fully and faithfully represented within a category of V -presheaf, preserving its entire structure including its metric information. In other words, each sample X i can be uniquely represented by its cost vector d ( · , X i ) with all samples. Based on these, we introduce the Indra representation and state the following theorems:

Definition 3 ( Indra Representation ) . For each sample X i ∈ Ob ( C ) , we define its Indra representation as the V -functor C ( X i , -) , i.e., the collection of values obtained by evaluating it under the (covariant) V -enriched Yoneda embedding on all objects of the category C .

Proposition 1. If two samples X i , X j ∈ Ob ( C ) have V -naturally isomorphic Indra representations and the cost function d satisfies the T 0 separation axiom, then X i = X j .

Theorem 2. For any V -functor P : C → V , the V -hom-object of V -natural transformations from the Indra representation of sample X i to P , denoted by [ C , V ]( C ( X i , -) , P ) , is V -isomorphic to P ( X i ) .

Corollary 2. The relational structure among objects in the sample category C is preserved and reflected in the relationships between their Indra representations.

The Indra representation of a sample X i , defined as the V -functor C ( X i , -) , can be interpreted as a relational profile of X i , that is, the cost from X i to every other sample in the category. Proposition 1 establishes that the Indra representation is a faithful encoding: no two distinct samples share the same representation. Theorem 2 further shows that the Indra representation is complete, in the sense that it encapsulates all the information needed to determine how distances from X i behave under any admissible distance assignment. Furthermore, Corollary 2 demonstrates that the relationships between samples are in one-to-one correspondence with the relationships between their Indra representations.

3.3 Instantiation of Indra Representation

We now demonstrate how to instantiate the Indra representation for a real dataset X = { X 1 , . . . , X n } consisting of n samples. We define the object set of the enriched category as Ob ( C ) = X and specify the hom-object C ( X i , X j ) as the cost d ( X i , X j ) between samples X i and X j , ∀ X i , X j ∈ Ob ( C ) . To define a valid Indra representation, the cost function d must satisfy two properties: ① d ( X i , X i ) = 0 for ∀ X i ∈ X ; and ② d ( X i , X k ) ≤ d ( X i , X j ) + d ( X j , X k ) for ∀ X i , X j , X k ∈ X . Several distance metrics satisfy these conditions. In this work, we adopt a simple yet effective choice by defining d ( X i , X j ) as the angular distance between the embeddings of X i and X j :

where f : X → R d X is a modality-specific foundation model, f ( X i ) denotes the internal representation of X i produced by f , and d X is the output dimensionality of the model. The use of angular distance ensures that d defines a valid Lawvere cost function. Given this cost function, the Indra representation of each sample X i ∈ X is the covariant Hom-functor:

This distance-based embedding forms a principled and interpretable representation. In the finite case, this can be written as C ( X i , -) = [ d ( X i , X 1 ) , . . . , d ( X i , X n )] , which captures the relational profile of X i with respect to all other samples. Notably, when the relational profile of each sample is computed using only a small set of representative landmarks, this formulation closely aligns with the approach proposed in [57].

3.4 Relational Matching across Modalities

Our hypothesis in Section 3.1 posits that unimodal foundation models learn convergent representations that capture the shared relational structure underlying reality. The proposed Indra representations are designed to reflect this inherent structure and can thus be leveraged to improve cross-modal understanding. To demonstrate how Indra representations facilitate relational matching across modalities, we consider a dataset D = { ( U i , Q i ) } n i =1 of n samples, where U i ∈ U and Q i ∈ Q correspond to instances from two distinct modalities, and D ⊆ U × Q . For single-modality scenarios, we define U i = Q i , ∀ i ∈ { 1 , . . . , n } . We use two pretrained foundation models f : U → R d X and g : Q → R d Y to extract modality-specific embeddings, where d X and d Y are the embedding dimensionalities of the two models, respectively. Based on these embeddings, we construct the Indra representations I U and I Q for each modality as follows:

where the cost function d is defined in Equation 3. In practice, we may apply post-processing operations such as sparsification and normalization to enhance robustness:

where operator ( · ) denotes the chosen post-processing function. Unlike traditional representation approaches that reflect only internal characteristics of samples, the proposed Indra representations act as external representations, where each vector captures interdependencies by encoding the sample's relative profile within the dataset.

4 Experiments

To comprehensively assess the effectiveness of our Indra representation, we perform evaluations across a range of settings, including unimodal vision, vision-language, and speech-language tasks.

4.1 Evaluation on Single Modality

Datasets. We first conduct classification tasks on the CIFAR-10 [37], CIFAR-100 [37], and OfficeHome [72] datasets. For CIFAR-10 and CIFAR-100, we use the standard data splits. For Office-Home, we evaluate classification accuracy across four distinct domains: Art, Clipart, Product, and RealWorld, using an 80 / 20 split for training and testing. Each domain exhibits unique visual styles and distribution shifts, making the dataset a widely used benchmark for evaluating the robustness and generalization of vision models in object recognition tasks. Across all datasets, we adopt logistic regression (i.e., linear probing) to assess the quality of the extracted representations.

Foundation Models. For vision models, we evaluate ViT [12], Convnext [77], and Dinov2 [61].

Evaluation Metrics. We assess model classification accuracy (%) using ground-truth labels. To investigate the robustness of Indra representations, we inject Gaussian noise into the features with varying standard deviations σ ∈ { 0 . 0 , 3 . 0 , 5 . 0 , 7 . 0 } . For each noise level, we perturb the features accordingly and train a linear classifier on the noisy representations. This allows us to assess how classification performance degrades as the feature representations are increasingly corrupted by noise.

Analysis. In Tables 1 and 2, we report the classification results on the CIFAR-10, CIFAR-100, and Office-Home datasets. The results clearly show that stronger backbone models (e.g., Dinov2 ) lead to better performance for Indra representations across all noise levels. For instance, on CIFAR-100 with σ =0 . 0 , our Indra representations achieve 91 . 93% accuracy using Dinov2 as the backbone, compared to 85 . 64% with Convnext and 80 . 09% with ViT . This performance gap persists and even widens under higher noise: at σ = 7 . 0 , Indra representations with Dinov2 maintain 58 . 67% , while Convnext and ViT drop to 30 . 25% and 32 . 74% , respectively. In addition, as Gaussian noise

increases, our Indra representations consistently retain higher classification accuracy compared to the original representations, highlighting their robustness in the classification tasks. The performance gains of Indra representations hold across multiple backbone architectures (i.e., ViT , Convnext , and Dinov2 ), indicating the broad applicability of the proposed method.

4.2 Evaluation on Vision & Language Modalities

Datasets. We adopt two widely used image-text datasets: MS-COCO [44] and NOCAPS [1] to evaluate performance on vision and language modalities. MS-COCO serves as a standard benchmark for image captioning and retrieval tasks, while NOCAPS poses a greater challenge due to its focus on novel object categories. We use the validation sets of both datasets for evaluation.

Foundation Models. We use the same vision models as in Section 4.1. For language models, we evaluate BERT [11] and Roberta [46], both of which are pretrained independently on unimodal data without cross-modal alignment. We include CLIP [62] as the aligned baseline for evaluation.

Evaluation Metrics : We adopt CLIPScore [26] as the evaluation metric, which measures semantic alignment between image and text based on the cosine similarity of their embeddings within the CLIP multimodal space. We report Topk matching accuracy ( k ∈ { 5 , 10 , 30 , 50 } ) in both text-to-image (T → I) and image-to-text (I → T) tasks.

Analysis. Table 3 compares the performance of original embeddings versus Indra representations across both datasets using various combinations of vision and language models. The results clearly demonstrate that Indra representations lead to consistent performance gains across different archi-

tectures and modalities. Significant improvements are observed in both T → I and I → T retrieval, highlighting the effectiveness of our method for cross-modal alignment. These findings suggest that the Indra representation offers a generalizable mechanism to improve vision-language matching, independent of model architecture or dataset. Nonetheless, there remains a noticeable gap in performance compared to the fully aligned CLIP model, indicating further room for improvement.

4.3 Evaluation on Speech & Language Modalities

Datasets. We adopt the TIMIT dataset [16] for audio and language modality experiments. TIMIT contains recordings from 630 speakers representing eight major dialect regions of American English, each reading ten phonetically rich sentences. The dataset provides time-aligned phonetic and wordlevel transcriptions along with 16 k Hz audio recordings.

Foundation Models. For audio models, we evaluate wav2vec [66], wavlm [6], and hubert [28], using both base and large variants. For the language modality, we use Roberta [46]. All models are pretrained independently on unimodal data, without any cross-modal alignment. As an aligned baseline, we include CLAP [78], an audio-language model jointly trained on paired audio-text data.

Evaluation Metrics : Similarly, we adopt CLAPScore [78] as the evaluation metric. We report Topk matching ( k ∈ { 5 , 10 , 30 , 50 } ) in both text-to-audio (T → A) and audio-to-text (A → T) tasks.

Analysis. Table 4 presents the audio-text matching results using different audio models. As shown, the Indra representations consistently improve matching performance in both directions across all model configurations. However, compared to the vision-language setting, the improvements in the audiolanguage modality are relatively modest. This is likely due to the comparatively weaker capacity of the audio models used. Nonetheless, we observe that larger audio models yield better matching accuracy, further supporting the notion that model capacity positively influences cross-modal alignment.

5 Related Work

Instance-Level Representation Learning. In conventional deep learning paradigms, each data instance is independently encoded into a fixed-dimensional vector, optimized using either supervised signals or unsupervised objectives. In supervised learning, representations are shaped by categorical labels [38, 24], whereas in unsupervised settings, objectives such as reconstruction (e.g., autoencoders [73]) or self-prediction (e.g., BERT [11]) guide the learning of instance-level representations. These approaches primarily focus on learning standalone embeddings and do not explicitly model the relationships between samples, either in training or inference stages.

Contrastive Representation Learning. Contrastive methods such as SimCLR [7], MoCo [23], and BYOL [19] introduce pairwise relational inductive bias during training by encouraging the embeddings of similar (augmented) views to be close, while pushing apart dissimilar ones. Building on this framework, CLIP [62] extends contrastive learning to vision-language pretraining by aligning paired image-text embeddings within a shared multimodal space, enabling strong zero-shot performance. BLIP [42] further integrates contrastive and generative objectives to enhance cross-modal representation learning, achieving state-of-the-art results on several vision-language benchmarks. Despite these advances, the sample representations in contrastive frameworks remain inherently instance-centric during inference. Additionally, contrastive learning approaches typically require large-scale datasets to achieve satisfactory performance, as the training objectives rely heavily on diverse and abundant positive and negative pairs.

Graph-based Representation Learning. Graph neural networks [35] offer a framework for encoding relational information through message passing across graph-structured data. Variants such as GraphSAGE [21] and GAT [71] have improved flexibility and representation ability. However, most graph-based approaches rely on predefined adjacency structures or proximity assumptions [51], which may embed inductive biases that are misaligned with the latent semantics of the data. They typically operate over local k -hop neighborhoods [48], limiting their ability to capture long-range dependencies unless deeply stacked, which can lead to oversmoothing and degraded performance [43].

Attention-based Representation Learning. Transformer-based architectures [70] offer a powerful alternative by leveraging global self-attention to aggregate contextual information. Numerous studies [11, 46, 12, 25, 61, 63] have demonstrated that attention-based architectures can be highly effective even in non-sequential domains. While attention enables global interactions both during training and inference, the resulting token-level representations are determined through dynamic mixing rather than explicitly encoding pairwise or global sample-to-sample relationships, making their geometric interpretation less transparent.

In contrast to the above paradigms, our approach explicitly constructs each representation as a relational profile, specifically, a distance-based reflection of its relationship to all other samples. This design is motivated by the philosophical ontology of Indra's Net, where each entity reflects and is reflected by all others, forming a holistic network of interdependence. More importantly, our Indra representation is formally grounded in the Yoneda Lemma from enriched category theory, offering a theoretically sound and interpretable framework for relational representation learning.

6 Conclusion

In this paper, we present a theoretical and empirical investigation into the convergent behavior of unimodal foundation models. Motivated by the philosophical metaphor of Indra's Net and grounded in enriched category theory, we introduce the Indra representation as a relational encoding that reflects each sample through its relationships with all others. We demonstrate that this representation can be derived via the V -enriched Yoneda embedding and instantiated practically using angular distance. Our theoretical analysis proves that the proposed Indra representations are unique, complete, and structurepreserving, offering a principled basis for training-free alignment of various foundation models. Through extensive experiments across single-modality, vision-language, and speech-language settings, we demonstrate that Indra representations improve robustness and latent cross-modal capabilities of unimodal foundation models, offering a general framework for training-free multimodal alignment. Our findings suggest a new perspective for bridging modalities, which emphasizes the importance of the intrinsic relational structure of data or reality.

Limitations . Constructing exact Indra representations requires a computational complexity of O ( n 2 d ) and a memory complexity of O ( n 2 ) for a dataset with n samples and embedding dimension d . This quadratic scaling potentially limits the direct applicability of the exact Indra representations to large-scale settings. However, the scalability concern is addressable in practice. In the literature, there exists a rich body of work on approximating pairwise distances efficiently. For example, approximate nearest neighbor search [30, 31], landmark-based approximation [50, 57], hashing-based methods [8], and sparse graph constructions [75]. From an application perspective, these techniques can be readily adapted to approximate the Indra representation at scale without sacrificing its robustness. From a theoretical view, Kan extensions [13] provide a principled framework for extrapolating from partial distance information to a full representation, preserving maximal structural fidelity.

References

Harsh Agrawal, Karan Desai, Yufei Wang, Xinlei Chen, Rishabh Jain, Mark Johnson, Dhruv Batra, Devi Parikh, Stefan Lee, and Peter Anderson. Nocaps: Novel object captioning at scale. In Proceedings of the IEEE/CVF international conference on computer vision , pages 8948-8957, 2019.

Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems , 33:12449-12460, 2020.

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609 , 2023.

Juerg Beringer, J-F Arguin, RM Barnett, K Copic, O Dahl, DE Groom, C-J Lin, J Lys, H Murayama, CG Wohl, et al. Review of particle physics. Physical Review D-Particles, Fields, Gravitation, and Cosmology , 86(1):010001, 2012.

Lennart Bürger, Fred A Hamprecht, and Boaz Nadler. Truth is universal: Robust detection of lies in llms. Advances in Neural Information Processing Systems , 37, 2024.

Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. Wavlm: Large-scale self-supervised pretraining for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing , 16(6):1505-1518, 2022.

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning , pages 1597-1607. PMLR, 2020.

Yudong Chen, Sen Wang, Jianglin Lu, Zhi Chen, Zheng Zhang, and Zi Huang. Local graph convolutional networks for cross-modal hashing. In Proceedings of the 29th ACM international conference on multimedia , pages 1921-1928, 2021.

Thomas Cleary. The Flower Ornament Scripture: A Translation of the Avatamsaka Sutra . Shambhala Publications, Boston, 1993.

F.H. Cook. Hua-Yen Buddhism: The Jewel Net of Indra . Iaswr Series. Pennsylvania State University Press, 1977.

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 , pages 4171-4186, 2019.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations , 2021.

Eduardo J Dubuc. Kan extensions in enriched category theory , volume 145. Springer, 2006.

Michael Faraday. Experimental Researches in Electricity . Bernard Quaritch, London, 1859. Originally published as a series of papers between 1831 and 1855.

John Rupert Firth, editor. Studies in Linguistic Analysis . Philological Society, Oxford, 1957. Special volume of the Philological Society.

John S Garofolo, Lori F Lamel, William M Fisher, David S Pallett, Nancy L Dahlgren, Victor Zue, and Jonathan G Fiscus. Timit acoustic-phonetic continuous speech corpus. (No Title) , 1993.

Kenneth J. Gergen. Relational Being: Beyond Self and Community . Oxford University Press, New York, 2009.

David Griffiths. Introduction to elementary particles . John Wiley & Sons, 2020.

Jean-Bastien Grill, Florian Strub, Florent Altmann, Corentin Tallec, Pierre-Henri Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. In Advances in Neural Information Processing Systems , volume 33, pages 21271-21283, 2020.

Wes Gurnee, Theo Horsley, Zifan Carl Guo, Tara Rezaei Kheirkhah, Qinyi Sun, Will Hathaway, Neel Nanda, and Dimitris Bertsimas. Universal neurons in gpt2 language models. arXiv preprint arXiv:2401.12181 , 2024.

Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems , 2017.

Harvard FAS CAMLab. Indra's net, 2022.

Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 9729-9738, 2020.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770-778, 2016.

Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654 , 2020.

Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718 , 2021.

Eghbal Hosseini, Colton Casto, Noga Zaslavsky, Colin Conwell, Mark Richardson, and Evelina Fedorenko. Universality of representation in biological and artificial neural networks. bioRxiv , 2024.

Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM transactions on audio, speech, and language processing , 29:3451-3460, 2021.

Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. The platonic representation hypothesis. In Proceedings of the 41st International Conference on Machine Learning , Proceedings of Machine Learning Research. PMLR, 2024.

Herve Jegou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neighbor search. IEEE transactions on pattern analysis and machine intelligence , 33(1):117-128, 2010.

Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Transactions on Big Data , 7(3):535-547, 2019.

Thomas P. Kasulis. Engaging Japanese Philosophy: A Short History . University of Hawai'i Press, Honolulu, 2018.

Gregory Maxwell Kelly. Basic Concepts of Enriched Category Theory , volume 64 of London Mathematical Society Lecture Note Series . Cambridge University Press, Cambridge, 1982.

Meenakshi Khosla, Alex H Williams, Josh McDermott, and Nancy Kanwisher. Privileged representational axes in biological and artificial neural networks. bioRxiv , pages 2024-06, 2024.

Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations , 2017.

Jing Yu Koh, Ruslan Salakhutdinov, and Daniel Fried. Grounding language models to images for multimodal inputs and outputs. In International Conference on Machine Learning , pages 17283-17300. PMLR, 2023.

Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009. Technical Report.

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems , 25, 2012.

F William Lawvere. Metric spaces, generalized logic, and closed categories. Rendiconti del seminario matématico e fisico di Milano , 43:135-166, 1973.

Andrew Lee, Melanie Weber, Fernanda Viégas, and Martin Wattenberg. Shared global and local geometry of language model embeddings. arXiv preprint arXiv:2503.21073 , 2025.

Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning , pages 19730-19742. PMLR, 2023.

Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping languageimage pre-training for unified vision-language understanding and generation. In International conference on machine learning , pages 12888-12900. PMLR, 2022.

Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI conference on artificial intelligence , volume 32, 2018.

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision-ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13 , pages 740-755. Springer, 2014.

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems , 36:34892-34916, 2023.

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 , 2019.

Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2022.

Jianglin Lu, Yixuan Liu, Yitian Zhang, and Yun Fu. Scale-free graph-language models. In The Thirteenth International Conference on Learning Representations , 2025.

Jianglin Lu, Hailing Wang, Yi Xu, Yizhou Wang, Kuo Yang, and Yun Fu. Representation potentials of foundation models for multimodal alignment: A survey. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages 16669-16684. Association for Computational Linguistics, 2025.

Jianglin Lu, Hailing Wang, Jie Zhou, Yudong Chen, Zhihui Lai, and Qinghua Hu. Low-rank adaptive graph embedding for unsupervised feature extraction. Pattern Recognition , 113:107758, 2021.

Jianglin Lu, Yi Xu, Huan Wang, Yue Bai, and Yun Fu. Latent graph inference with limited supervision. In Advances in Neural Information Processing Systems , 2023.

Mayug Maniparambil, Raiymbek Akshulakov, Yasser Abdelaziz Dahou Djilali, Mohamed El Amine Seddik, Sanath Narayan, Karttikeya Mangalam, and Noel E. O'Connor. Do vision and language encoders represent the world similarly? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2024.

Hazel Rose Markus and Shinobu Kitayama. Culture and the self: Implications for cognition, emotion, and motivation. In College student development and academic life , pages 264-293. Routledge, 2014.

James Clerk Maxwell. A Treatise on Electricity and Magnetism . Clarendon Press, Oxford, first edition, 1873.

Jack Merullo, Louis Castricato, Carsten Eickhoff, and Ellie Pavlick. Linearly mapping from image to text space. In The Eleventh International Conference on Learning Representations , 2023.

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems , 26, 2013.

Luca Moschella, Valentino Maiorca, Marco Fumero, Antonio Norelli, Francesco Locatello, and Emanuele Rodolà. Relative representations enable zero-shot latent space communication. In The Eleventh International Conference on Learning Representations , 2023.

Evonne Ng, Sanjay Subramanian, Dan Klein, Angjoo Kanazawa, Trevor Darrell, and Shiry Ginosar. Can language models learn to listen? In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10083-10093, 2023.

Jerry Ngo and Yoon Kim. What do language models hear? probing for auditory representations in language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics , pages 5435-5448, 2024.

OpenAI. Gpt-4 technical report, 2024.

Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 , 2023.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748-8763. PMLR, 2021.

Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In International conference on machine learning , pages 28492-28518. PMLR, 2023.

Emily Riehl. Category theory in context . Courier Dover Publications, 2017.

Geoffrey Roeder, Luke Metz, and Durk Kingma. On linear identifiability of learned representations. In International Conference on Machine Learning , pages 9030-9039. PMLR, 2021.

Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. wav2vec: Unsupervised pre-training for speech recognition. arXiv preprint arXiv:1904.05862 , 2019.

Pratyusha Sharma, Tamar Rott Shaham, Manel Baradad, Stephanie Fu, Adrian RodriguezMunoz, Shivam Duggal, Phillip Isola, and Antonio Torralba. A vision check-up for language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14410-14419, 2024.

Daniel Tan, David Chanin, Aengus Lynch, Brooks Paige, Dimitrios Kanoulas, Adrià GarrigaAlonso, and Robert Kirk. Analysing the generalisation and reliability of steering vectors. Advances in Neural Information Processing Systems , 37, 2024.

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 , 2023.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 2017.

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio, et al. Graph attention networks. stat , 1050(20):10-48550, 2017.

Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 5018-5027, 2017.

Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, Pierre-Antoine Manzagol, and Léon Bottou. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of machine learning research , 11(12), 2010.

Junxuan Wang, Xuyang Ge, Wentao Shu, Qiong Tang, Yunhua Zhou, Zhengfu He, and Xipeng Qiu. Towards universality: Studying mechanistic similarity across language model architectures. In The Thirteenth International Conference on Learning Representations , 2025.

Yangbo Wang, Jie Zhou, Mingli Song, Yue Guo, and Jianglin Lu. Fuzzy multi-subspace clustering. IEEE Transactions on Fuzzy Systems , pages 1-14, 2026.

John Wentworth. Testing the natural abstraction hypothesis. AI Alignment Forum , 2021.

Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2023.

Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. Large-scale contrastive language-audio pretraining with feature fusion and keywordto-caption augmentation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 1-5. IEEE, 2023.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our abstract and introduction clearly showcased our main contributions.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitations of the work in Section 6.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: See appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: See Section 4. In addition, we release the source code for reproducibility, see Abstract for the code link.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We release the source code for reproducibility, see Abstract for the code link.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We conduct experiments following the standard protocol and include the experimental details in Section 4.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We provide the experimental details in the Section 4.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide the experimental details in Section 4, and results can be reproduced using our source code (see the link provided in Abstract).

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have read and follow the rules.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: There is no societal impact.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our work has no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We have cited the papers that created the datasets.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: Our paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: We did not use LLMs in our paper except polishing our presentation.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces the \"Indra Representation Hypothesis,\" which states that the representations learned by unimodal foundation models converge towards a shared relational structure. The authors formalize this hypothesis using the V-enriched Yoneda embedding from category theory. This leads to a practical method where a sample's representation is redefined as its vector of distances to all other samples in a given dataset. This distance vector, termed the Indra representation, is computed from the outputs of pre-trained unimodal models. The authors provide theoretical claims that this representation is unique, complete, and structure-preserving. The method is evaluated on cross-model and cross-modal matching tasks involving vision, language, and audio, where it is shown to consistently improve matching accuracy over the original model embeddings.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper's primary contribution is a novel and elegant conceptualization of representation convergence. The analogy to Indra's Net, combined with the formalization using category theory, provides a thought-provoking new lens through which to view the properties of embeddings from large-scale models.\nThe proposed method is a simple, post-hoc transformation that requires no retraining of the foundation models. Its generality allows it to be applied to any model that produces vector embeddings, which is a significant practical advantage.\nAcross the conducted experiments in single-modality and cross-modal settings, the use of the Indra representation demonstrates a modest improvement in matching performance over the baseline of using raw embeddings.\n\nThe proposed method is fundamentally non-scalable. To construct representations for a dataset of n samples with embedding dimension d, one must compute an n x n distance matrix, which has a computational complexity of at least $O(n^2d)$ and a memory complexity of $O(n^2)$. The resulting Indra representation for each sample is an n-dimensional vector. This scaling behavior makes the method computationally infeasible for datasets beyond the small-scale validation sets used in the paper (e.g. $n>10^5$). This is, in essence, a kernel method that computes the full kernel matrix. The paper shows a critical lack of rigor by failing to acknowledge this.\n\nThe appeal to enriched category theory, while elegant, does not appear to offer concrete guidance for the method's implementation. The Yoneda Lemma guarantees that the mapping to a functor category is fully faithful, preserving all information contained within the chosen cost function d. However, this is a tautology: the representation is complete only with respect to the chosen distance metric. The theory provides no principles for selecting an optimal cost function, which is the single most important design choice in the entire method. Any valid metric space could be used to motivate the same procedure, making the complex theoretical overhead seem more justificatory than generative.\nThe empirical evaluation is weak because it only compares the Indra representation against the original embeddings. To rigorously assess a new method for post-hoc representation alignment, it must be benchmarked against established techniques. By omitting these comparisons, the paper fails to demonstrate that its proposed method offers a significant advantage over simpler, well-known alternatives.\n\nQUESTIONS:\n1)The proposed method has a computational complexity of $O(n^2d)$ and produces n-dimensional representations. Please comment on the method's viability for large-scale datasets. More importantly, why was the extensive literature on kernel approximation methods, which are designed to mitigate these exact scalability issues, not considered, implemented, or benchmarked against?\n\n2) The theoretical framework guarantees that the Indra representation preserves the structure induced by the cost function d. However, it provides no guidance on how to select d. How does the theory inform this crucial choice? For instance, your ablation shows Euclidean distance also works; does the theory suggest when angular distance might be superior to Euclidean, or vice-versa?\n\n3) Could you refine your claim regarding the \"structural myopia\" of existing representations? How do you distinguish the relational information captured by your method (inter-sample distances) from the relational information already captured by mechanisms like self-attention (inter-token dependencies)?\n\nLIMITATIONS:\nYes, the authors provided a limitations section in the supplementary material. However, it is wholly inadequate. The authors completely sidestep the critical issue of scalability. Instead, they frame the limitation as their method being \"post-training adaptation\" and suggest incorporating the ideas into pretraining as future work.  This is not a meaningful limitation but rather a description of the work.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new theoretical and empirical framework for representation learning inspired by the metaphor of Indra’s Net. The central hypothesis is that unimodal foundation models, despite being trained independently, converge to representations that reflect a shared relational structure of reality. This is formalized using enriched category theory, where each sample is represented by its vector of distances to all other samples under a specific cost function (e.g., angular distance). The resulting Indra representation is theoretically shown to be unique, complete, and structure-preserving. The authors instantiate the method on pretrained models from vision, language, and audio modalities, and evaluate it on cross-model and cross-modal matching tasks. They also test an alternative cost function (Euclidean distance) to assess robustness. The method is applied post hoc, without retraining or supervised alignment.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n* The paper introduces a novel and compelling framing of representation learning using the Indra’s Net metaphor.\n* The use of category theory and the Yoneda embedding provides an interesting foundation for relational representations.\n* The approach is post-hoc and model-agnostic, so it can be applied to any pretrained unimodal encoder without retraining or supervision.\n* Empirical validation is provided across multiple modalities (vision, language, audio), model architectures, and datasets, showing consistent performance gains.\n* The method is shown to be somewhat robust to the choice of cost function (angular vs. Euclidean distance).\n\n**Weaknesses**\n* The method requires computing and storing $n \\times n$ distance matrices (Eq. 4), resulting in quadratic complexity in dataset size. The paper does not analyze runtime or memory scaling, nor does it explore sparse or approximate alternatives and their impact on theoretical guarantees.\n* The paper could benefit from qualitative visualizations or interpretability analyses to provide intuition on what kinds of relations are being captured.\n* The evaluation is limited to retrieval-style tasks using similarity-based matching metrics. The effectiveness of Indra representations on other downstream tasks (e.g., classification, generation, etc) is not explored. Evaluating such tasks would help clarify whether the benefits of the relational structure extend beyond similarity matching.\n* The paper cites prior work demonstrating that lightweight post-hoc alignment methods, such as linear projections or Procrustes alignment, can effectively bridge pretrained unimodal encoders (e.g., Merullo et al. [50], Sharma et al. [62]). However, it does not include these as baselines in the empirical evaluation. Comparing against these methods, especially in low-data or zero-shot variants, could clarify whether the gains arise uniquely from the relational structure or could be matched by simpler geometric transformations.\n\nQUESTIONS:\n1. Have the authors considered evaluating Indra representations on downstream tasks beyond retrieval, such as classification (e.g., via linear probing) or caption generation?\n2. Can the authors provide formal or empirical analysis of the time and space complexity of Indra representation computation, especially in large-scale settings?\n3. Have the authors considered whether sparse approximations (e.g., k-nearest neighbor graphs) could reduce computational cost while preserving the theoretical guarantees (uniqueness, completeness, structure preservation) of the Indra representation?\n4. Can the authors provide qualitative examples or case studies where Indra representations yield more semantically aligned or informative matches than original embeddings?\n5. How robust is the Indra representation to the quality of base embeddings? Specifically, how does it behave when the embeddings are noisy or derived from undertrained or small-capacity models?\n6. The paper cites prior work on lightweight post-hoc alignment methods, such as linear projections (e.g., [50]), which operate under broadly similar goals. Could the authors elaborate on the decision not to include these as empirical baselines?\n\nLIMITATIONS:\nYes, the authors have addressed the limitations and potential negative societal impact of their work. One suggestion for improvement would be to include a discussion of scalability limitations, especially regarding the complexity of computing full pairwise distance matrices.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces The Indra Representation Hypothesis, a novel theoretical and empirical framework aimed at understanding representation convergence in unimodal foundation models. Drawing philosophical inspiration from the metaphor of Indra’s Net, the authors propose that foundation models, despite being trained on different modalities and objectives, tend to learn internal representations that converge towards a shared relational structure.\n\nThe key contribution is the formalization of this idea through category theory, specifically the V-enriched Yoneda embedding, which leads to the definition of the Indra representation. The Indra representation encodes each sample through its relational profile with respect to all others, using a distance metric (angular distance in this work) between model-generated embeddings. Theoretical guarantees are provided, showing that this representation is unique, complete, and preserves structural relationships.\n\nThe paper then instantiates this formulation practically and demonstrates its utility across a wide range of experiments involving cross-model and cross-modal matching tasks, including vision, language, and audio domains. Results show that Indra representations significantly improve matching performance between independently trained unimodal models, outperforming the original embeddings and narrowing the gap with jointly trained multimodal baselines like CLIP and CLAP.\n\nThrough both its mathematical grounding and extensive empirical validation, the paper positions the Indra representation as a general, theoretically sound, and practically beneficial approach for understanding and leveraging latent relational structures in foundation models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Quality & Clarity: The paper presents a well-grounded and mathematically rigorous framework by introducing the Indra representation using V-enriched category theory. The authors provide theoretical guarantees—uniqueness, completeness, and structure preservation—supported by formal theorems and proofs.\n\n2. Originality: The analogy to Indra’s Net offers a fresh and philosophically motivated perspective on representation learning, distinguishing this work from traditional embedding-centric approaches.\n\n3. Significance: The proposed method is applied to vision, language, and audio models, showing strong performance improvements across cross-model and cross-modal matching tasks, without requiring any retraining or supervision.\n\nWeaknesses:\n1. Insufficient limitation discussion: It fails to address computational cost, storage footprint (Indra vectors require pairwise distances over the entire dataset), sensitivity to the choice of distance metric, and scalability to real downstream tasks, among other issues. Overall, the limitation discussion is too narrow in scope and pays insufficient attention to potential engineering and data challenges.\n\n2. Small Unclear in proof of theory 2: Seems missing proof of separation axiom (d(x,y)=0 ⇒ x=y)\n\nQUESTIONS:\n1. Really nice proof! However, the proof of Theory 2, one thing I find confusing, you seem mixing the concept of Identity Property with the Separation Axiom. Would you mind explain what your identity property refers to?\n2. Your experiment part is almost excellent. Besides these scores, can you show some real-world downstream task performance of your method?\n3. My main concern is that your method has O(N^2) computational complexity. Under this assumption, many of us are interested in how large the computation and storage costs become on truly large-scale datasets. Could you discuss more about that?\n\nLIMITATIONS:\nI suppose you should talk more about limitation.\nIt fails to address computational cost, storage footprint, sensitivity to the choice of distance metric, and scalability to real downstream tasks, among other issues. Overall, the limitation discussion is too narrow in scope and pays insufficient attention to potential engineering and data challenges.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nMotivated by recent findings that unimodal foundation models often exhibit strong correlations between their feature representations, this paper introduces the concept of the Indra Representation. Inspired by the ancient notion of Indra’s Net—a metaphysical idea from Eastern philosophy suggesting that each entity is a reflection of all others—the authors formalize a representation that captures the relational structure underlying model features. This formulation provides a unified framework for interpreting and aligning features across different models and modalities. The effectiveness of the proposed representation is validated through experiments conducted across diverse datasets and settings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n1.\tBuilding upon the ancient philosophical concept of Indra’s Net as a foundational metaphor, this work formalizes and instantiates it within a modern machine learning framework. I find this conceptual grounding to be highly novel and intellectually compelling.\n2.\tThe paper presents comprehensive experiments demonstrating that the proposed Indra representation achieves stronger alignment across features from different unimodal foundation encoders, compared to using raw features directly from each encoder. This provides solid empirical support for the effectiveness of the proposed framework.\n\nWeaknesses: \n1.\tWhile the proposed representation is theoretically elegant and empirically validated, the paper could benefit from a clearer articulation of its practical utility. Specifically, it remains somewhat unclear in what real-world scenarios such cross-model, cross-modality feature alignment is necessary. Furthermore, the authors do not address why existing well-aligned models, such as CLIP or more recent MLLM like GPT-4o, would not already suffice in such contexts. Providing stronger motivation for why this new representation is needed in practice would strengthen the impact of the work.\n2.\tThe use of number notes such as ① and ② throughout the paper feels informal and somewhat inconsistent with the typical academic writing style expected in formal publications. It may be advisable to adopt more standard notation or formatting for clarity and professionalism.\n3.\tWhile the proposed method shows strong performance on the selected benchmarks, the experiments are primarily conducted on datasets with a limited number of categories. In real-world applications, it may be challenging to define the appropriate scope and number of relevant entities for constructing the Indra representation, potentially limiting its scalability and practicality in large-scale or open-domain settings.\n\nQUESTIONS:\nYou can directly refer to the weaknesses part.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 4, "DATE": "2025-07-03"}