Distributional Vision-Language Alignment by Cauchy-Schwarz Divergence
Authors: Wenzhe Yin
Zehao Xiao
Pan Zhou
Shujian Yu
Jiayi Shen
Jan-Jakob Sonke
Stratis Gavves
Venue: ICLR 2026 Poster (2026)
Keywords: Vision-Language Alignment
CLIP
Cauchy-Schwarz Divergence
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Vision-language alignment is crucial for various downstream tasks such as cross-modal generation and retrieval. Previous multimodal approaches like CLIP utilize InfoNCE to maximize mutual information, primarily aligning pairwise samples across modalities while overlooking distributional differences.  In addition, InfoNCE has inherent conflict in terms of alignment and uniformity in multimodality, leading to suboptimal alignment with modality gaps. To overcome the limitations, we propose CS-Aligner, a novel framework that performs distributional vision-language alignment by integrating Cauchy-Schwarz (CS) divergence with mutual information. CS-Aligner captures both the global distribution information of each modality and the pairwise semantic relationships. We find that the CS divergence seamlessly addresses the InfoNCE's alignment-uniformity conflict and serves complementary roles with InfoNCE, yielding tighter and more precise alignment. Moreover, by introducing distributional alignment, CS-Aligner enables incorporating additional information from unpaired data and token-level representations, enhancing flexible and fine-grained alignment in practice. Experiments on text-to-image generation and cross-modality retrieval tasks demonstrate the effectiveness of our method on vision-language alignment.

Published as a conference paper at ICLR 2026
DISTRIBUTIONAL VISION LANGUAGE ALIGNMENT BY BAUCHY-SCHWARE DIVERGENCE
Wenzhe Yin Zehao Xiao
Pan ZhouSt Shujian Yu²4
Jiayi Shen'
Jan-Jakob Sonke* Efstratios Gavves'
'University of Amsterdam
The Netherlands Cancer Institute
Viiije Universiteit Amsterdam
"The Arctic University of Norway X "Singapore Management University
ABSTRACT
Vision-language alignment is crucial for various downstream tasks such as cross- modal generation and retrieval. Previous multimodal approaches like CLIP utilize InfoNCE to maximize mutual information, primarily aligning pairwise samples across modalities while overlooking distributional differences. In addition, In- foNCE has inherent conflict in terms of alignment and uniformity in multimodal- ity. leading to suboptimal alignment with modality gaps. To overcome the lim- itations, we propose CS-Aligner novel framework that performs distributional vision-language alignment by integrating Cauchy-Schwand (CS) divergence with mutual information. CS-Aligner captures both the global distribution informa- tion each modality and the pairwise semantic relationships. We find that the conflict
CS divergence seamlessly addresses the InfoNCE's eriemmmnn-uniffrmmnn and serves complementary roles with InfoNCE, yielding tighter and more pre- cise alignment. Moreover, by introducing distributional alignment, CS- CS-Aligner enables incorporating additional information from unpaired data and token-level representations, enhancing fiexible and fine-grained alignment in practice. Exper- iments on text-to-image generation and cross- tross-modality retrieval tasks demonstrate the effectiveness of our method on vision-language alignment.
INTRODUCTION
Vision-language alignment aims to map the paired text and image inputs into shared feature space, enabling success across diverse applications such as image-text retrieval (Huang al., 2024 Kouk- Jounas et al. 2024) and rext-to-image (T2I) generation (Ramesh et al. 2022; Razzhigaev et al. 2023). As pioneering work in this field, CLIP (Radford et al., 2021 leverages InfoNCE loss (a.k.a.
trastive loss) to maximize the mutual information between paired text and image representations, effectively capturing pairwise and semantic relationships. Its versatility has made it foundation for many multimodal tasks (Ramesh et al. 2022 Mokady et al. 2021).
Despite its success, CLIP and its variants Zhai et al. 2023 Sun et al. 2023) exhibit persistent
modality gap, a misalignment between text and image representations in the shared latent space. As shown in Fig. 1a text and image embeddings often fail to align precisely and may remain separated
from each other. This phenomenon has been widely observed (Zhou et al. [023]] Liang et al. 2022]
Shi et al. p023 and is attributed to issues such as cone effects (Liang et al. 2022) or suboptimal
latent space geometry (Shi et al. (202).. Intriguingly, Liang et al. (Liang et al. 2022) observed that CLIP's InfoNCE loss could inadvertently exacerbate the modality gap, since, as analyzed in Sec. 2 InfoNCE loss can be decomposed into alignment and uniformity components, which indeed conflict with each other during vision- language alignment.
Several strategies have been proposed to mitigate the modality gap, such as projection modules with cosine similarity (Zhou et al. 22023 Gao et al. 2024: Huang et al. 2024) and geodesic multimodal mixup (Oh et al. (2024). UnCL.IP-based models like DALL-E2 (Ramesh et al., 2022) employ text- to-image prior modules (e.g., diffusion models) to map text embeddings to image feature space
"quual contribution.
Correspondence Pan Zhou <panzhou@smu.co .du.uag.dd.........

---

Published as a conference paper at ICLR 2026
Painwise Distance
Text Embeddings
Image Embeddings
InfoNCE
CS-Aligner
Count
1, distance
(a) InfoNCE
(b) With CS-Aligner
(c) Image-Tex: pair distance.
Figure 1: TSNE visualizations of CLIP text and image features without (a) and with (b) CS- Aligner. The original CLIP feature distributions reveal a clear domain gap (a). Adapting the model with our CS-Aligner effectively eliminates the modality gap, leading to tighter alignment (b). Con- sequently, CS- CS-Aligner yields a lower overall &2 distance between paired image-text features (c).
for alignment. A more recent alternative Felipse (Patel et al. 2024 uses & loss to train a prior adapter for text and image alignment. These works aim transform representations across modal- ities for alignment. However, they explore alignment sample-wisely, heavily relying on pairwise data. Although sample-win wise aligmment effectively captures semantic information, it falls short in aligning entire data distributions. Similar to the InfoNCE in CLIP, the methods struggle to match representation spaces across modalities, ultimately limiting the overall alignment. The reliance on carefully curated text-image pairs also limits the scalability and applicability to real-world sce- narios with unpaired and noisy datasets (Lin et al.) 2014 Li al., 20236). Moreover, the theoretical conflict of InfoNCE for vision-language alignment still under exploration.
To address these challenges, we propose CS- Cl-Agigeer a novel distributional approach that incorpo- rates Cauchy-Schwarz (CS) divergence (Principe al. (000b) for vision-language alignment. As symmetric measure, CS divergence robustly and efficiently estimates the distance between any rep- resentation distributions without parametric distributional assumptions, making it highly suitable for multimodal distribution alignment. Furthermore, we analyze the aligament-uniforni InfoNCE in multimodal settings and show that CS divergence effectively mitigates it while remain- ing compatible with InfoNCE via kernel density estimation (KDE) (Parzen 1962). This enables CS-Aligner align vision-language representations at distributional and sample-wise levels, cap- turing global modality and local semantics, yielding more comprehensive, consistent, and tighter
conffict of
alignment as shown in Figs. 1b and 1c
Moreover, the distributional nature of CS-Aligner enables alignment with unpaired multimodal data, including cases where a) single image associated with multiple captions, or vision and lan- guage inputs are entirely unpaired. This flexibility allows our method to leverage rich and unstruc- tured datasets and improve alignment robustness beyond curated benchmarks. Beyond unpaired alignment, introduce token-level alignment strategy, which further enriches the multimodal representation by aligning fine-grained visual and textual tokens, enhancing the semantic precision of the learned embeddings. Extensive experiments downstream tasks, including T2I generation and image-text retrieval, demonstrate the effectiveness of our approach.
2 INFONCE IS INSUFFICIENT FOR ALIGNMENT
Previous multimodal methods (for vision-language) like CLIP (Radford et al. (021) learn text and image representations in shared space by maximizing lower bounds (e.g.. InfoNCE (Oord al. 2018)) of mutual information between modalities:
I(x:y) = / y( p(x,)) dx dy, p(x)p(y)
(1)
where p(x) and p(y) are respectively the distributions of image and text features, and p(x, y) de- notes their joint probability. Although widely used, it suffers from two limitations.
Limitation1: Mutual information is insufficient for multimodal alignment. Although widely adopted, mutual information alone is insufficient for effective modality alignment (Liang et al.
2

---

Published as a conference paper at ICLR 2026
P(X)
P(X)
P(X)
0.
02
P(Y)
-2
P(Y)
P(Y)
MI: 1.959
MI:1.959
MI:0
KL: 6.81
KL: 0
KL:0
000
1.98)
0.99
P(X, N (). 9)
P(X, X) ~
P(X, X) ~
1.99
0.99
(a) High MI & High Divergence
(b) High Low Divergence.
(c) Low Divergence.
Figure 2: Toy examples: mutual information (MI 1) and distribution divergence (.) between two distributions. Distributions with the same high mutual information value can exhibit either large (a) small (b) distributional distances, demonstrating that MI alone is insufficient for multi- modal alignment. Moreover, distribution divergence measures the closeness between distributions but does not guarantee that the underlying random variables statistically correlated (c).
2022 The reason is that mutual information quantifies the statistical dependence between two random variables [Cover] 1999), ensuring correlation maximization between two random variables. However, does not guarantee that the distributions p(x) and p(y) are statistically similar close to each other in terms of their underlying distributions. In other words, the embedding distributions of two modalities can differ significantly or be far apart, yet exhibit strong dependence.
We illustrate this issue using a toy example in Fig. 2 Fig.2h. shows that despite strong dependence and high mutual information, the representation distributions of two representations or random vari- ables can remain misaligned and be far from each other, resulting in a high divergence. This issue is also observed in the CLIP model pretrained with InfoNCE, where the vision and language represen- tations exhibit noticeable distributional gap, as shown Fig, Ia This gap results in inconsistently aligned multimodal features, hindering the clear representation of shared semantics and disrupt- ing effective mapping between modalities. Ultimately, this misalignment degrades performance in downstream tasks, including cross- modaaltyy generation. Ideally, the desired multimodal representa- tions should be highly correlated with low distributional divergence, depicted in Fig. Zp. Notably, although directly minimizing the divergence between distributions may reduce the distributional gap, it risks creating independent multimodal distributions without common semantic information (Fig. 2)) Therefore, maximizing mutual information and minimizing divergence complement each other to achieve effective multimodal representation alignment. Details are provided in Appendix A
Limitation2: InfoNCE includes conflicting terms for multimodal alignment. In practice, mutual information is often optimized via the InfoNCE loss (Oord et al. (2018) which estimates I(x; y) using paired image-text data ((x, and contains maage-text and text-image alignment terms:
CInfONCE 2N
1
Σ +xxx,)))++(y....
h(x, y) log Σ exp (sim(x. V5)/T)
exp (sim(x, y)/7)
where sim(.,) :) is cosine similarity and r is temperature. Critically, the InfoNCE loss in Eq. requires paired data {(x,, and cannot work under unpaired setting.
As analyzed in Wang & Isola (2020), the InfoNCE loss can be decomposed as the sum of the alignment (Calign) and uniformity (Zuniform) terms i.c., LIntANCE RO Lalign + Luniformi
Lalign E(x,y)rppe |||x x vlll), Luniform A log E p(x.y) [exp(-1[|x-1 y11)) (3)
where t and a are hyperparameters. Ppair denotes the image-text pairs distribution. Minimizing Lalign encourages pairwise alignment. In unimodality, minimizing Lunitorm promotes representations that are uniformly distributed on the unit hypersphere, desirable property for representation learn- ing (Wang& & Isola] 2020). However, multimodal alignment, Luniorm may conflict with Lalign- Remark 2.1. The uniformity and alignment terms in InfoNCE conflict with each other in multimodal alignment. Applying Taylor expansions (B(e-x) 1 TE()) and log(1 x) as x) Luniform

---

Published as a conference paper at ICLR 2026
the uniformity term becomes:
Cuniforn R - -IlE -tllxxy xp((xx))
|||x y - (r
|||x
(4)
where p(x, y) Ppair Pumpairs and Punpuir denotes the distribution of unpaired image and text. Consequently, the combination of the two (InfoNCE) can be written
Lalign + Luniform RJ (ppr
[2] (x
x
(5)
The alignment contribution (Catiga) in Eq. can be largely suppressed or even canceled (when == 1) due to the opposing term in Eq. ( leaving only negative pairs influential. Essentially, Lalign promotes alignment across modalities, whereas Luniform encourages dissimilarity among negative pairs without preserving intra-modal structure. This inherent conflict can result in local minima, driving alignment and uniformity in opposing directions and ultimately leading to modality gap. Thus, InfoNCE alone may lead to suboptimal alignment between modalities.
3
METHODOLOGY
In this section, we address the incapability of mutual information on aligning distributions and the conflicts InfoNCE for multimodal alignment. To this end, we first introduce a novel distributional multimodal alignment framework, CSS-Aligner. Then, we analyze that with the KDE, the proposed method is able to address the sefformmtty-liinmmmnt conflicts of InfoNCE. Finally, we extend CS- Aligner to the unpaired data, including token-level alignment.
3.1 CS ALIINER: DISTRIBUTIONAL MULTIMODAL ALIGNMENT
To mitigate limitation I in Sec. and p(y). In practice, p(x) and ply) may follow arbitrary distributions with minimal intersection, which may often occur in the multimodal setting. Hence, a robust divergence metric must accom- modate unpredictable variability and limited support overlap for effective distribution alignment.
explicitly minimize the distribution divergence between p(x)
To this end, we propose distributional alignment framework, namely CS-Aligner, which leverages the CS divergence (Des), as illustrated in Fig. 3 The objective is:
min yxx: + ((ns(p(x),p()))
(6)
where a hyperparameter balancing the mutual information term and the divergence penalty. CS divergence, Des, is symmetric and robust metrie to quantify the distance between any two probability density functions p and g. defined over the same support w as:
Des(p: q) = log ((f ((w())(w)))) plw)) (f p(w) dow (aw))dww)
The CS divergence satisfies 0 S. Des co, and equals zero if and only if q. By introducing Des in Eq. (6). instead of solely minimizing pairwise distance, our method also aligns the distributions of modalities, leading more robust and efficient multimodal alignment, as shown in Fig. 3
CS divergence estimation. To estimate CS divergence, we introduce con-paamettrial KDE. The non non-parametric KDE means that it does not assume any specific parametric form for the underlying distribution. This eliminates the need for explicit parametric assumptions about the underlying distributions. This provides significant flexibility in measuring distributional distance. Given i.i.d.
samples (x, LM p(x) and {y.j.y p(y), the empirical CS divergence estimator is given by Jenssen et al. (2006):
M M n(x, xy) + +log
N C((y.,,)) log V
M
N K x
Dcs(p(x):P(y)) = log
x(.,
vy))
MN
(8)
where x is kernel function such as Gaussian Kg This estimator is symmetric, differentiable, and computationally efficient, making it suitable for multimodal alignment. Moreover, the third term in Eq. (8) ensures that Pcs(p(x):p(y)) only
- (2/220 exxpl with kernel width
=
4

---

Published as a conference paper at ICLR 2026
Cross-modality tasks
Text feature
Text Encader (Ber)LLMMs
Text-to-image generation
Texts
Adapter
y/
A trato
haviing
afigned
Image generator (unCLIP:
CS-Aligner
covered tack
text feature
Ca
Vision-language retrieval
Iraage SpACK
A amell Marsides coupe seen parking
medium ouidoar view of A Righaangle vitar a blkok and brown Yorkanive
Image
Images-
Encoder (VIT)
Adaptar
Image feature
Figure 3: Illustration of CS-Aligner. We achieve vision-language alignment by freezing the pre-
trained text and image encoders and applying parameter-efficient fine-tuning methods (e.g., adapter) with our CS-Aligner. CS-Aligner optimizes the adapters using the aggregated CS divergence and foNCE, formulated in Eq. (6). Once aligned, the adapters are utilized for various cross-modality tasks: the aligned text adapter facilitates text-to-image generation without additional modifications, while the aligned multimodal adapters used for vision-language retrieval.
when [[x(x, y)) 0 (i.e., when the distributions do overlap). However, as long as there is nonzero overlap between the distributions, the estimator remains well-defined and valid.
Hence, CS-Aligner remains reliable even when the two distributions initially have limited overlap, a common scenario in multimodal tasks. Additionally, its symmetry and non-parametric estimation properties ensure consistent and unbiased multimodal alignment. Consequently, our method ensures both semantic and distributional alignment, enabling robust and efficient multimodal learning.
When estimating the mutual information / (x,y) y) via InfoNCE (Eq. ()) unlike other distribution divergences, CS divergence effectively addresses InfoNCE's inherent
conflict.
Uniformity and Aligmment with CS Divergence. Using the Gaussian kernel xx(x,y) exp(-t|ix - -y1 for CS divergence and combining the alignment and uniformity components of InfoNCE, the full objective of Eq. (6) can be expressed as
(xxx))> [|x y [
+ log Ex-p(x) yop(y) yx]
log Exxxxixx [x, x')] + log Ey Ey.y' [Ne (xcly.y))) 2 log Ex-p(x) (x,, yx),
When x = 1, this reduces to the following alignment-uniforninn decomposition:
L = (xxx)) |||x
log E. (xx-p(x),x=p(x)) (exp (-t|ix y|]]
Alignment
(10)
+ log xxxx [exp (-tlix x' [13]] log Ey.''ply)) [exp (-tlly y'1?]]]
Uniformiity on
Uniformity on
Remark 3.1. For the alignment part, CS- CS-Aligner promotes both the matching of image-text pairs and the alignment of global distributions. For uniformity, CS-Aligner encourages dispersion within each modality independently, rather than across modalities, which could otherwise conflict with the alignment objective. Thus, our method simultaneously fosters both alignment and uniformity while avoiding the potential conflicts inherent InfoNCE.
Remark 3.2. The connection between CS divergence and InfoNCE becomes evident when ana- lyzing both terms from a cosine similarity perspective. For characteristic kernel s(x, y) (o(x), o(y)) where & maps samples to a Reproducing Kernel Hilbert Space (RKHS) H. the mean embeddings are: μe then be expressed in a form that evaluates the cosine similarity between distributions RKHS:
Ene (xxx)) and B4y = Σr Φ(y:), The CS divergence can
(pxx Myy
Dcs(p(x):p(y)) = -2
[| |l 2el Helll = -2 log sim(pz, Myy)
(11)
5

---

Published as a conference paper at ICLR 2026
Similarly, InfoNCE evaluates cosine similarity between paired samples (Eq. ( This dual-level similarity assessment underscores the synergy between CS divergence and mutual information, of- fering unified and robust framework for multimodal alignment.
Therefore, CS divergence is compatible with InfoNCE and effectively addresses the inherent conflict between uniformity and alignment, a property not shared by other distribution distance metrics. Detailed comparisons with other metrics are provided the Appendix
3.2 EXTEND CS-ALIGNER TO UNPAIRED DATA
Benefiting from the distributional alignment, we further propose extensions of CS-Aligner, which leverage additional information in unpaired data. While the mutual information estimation (In- foNCH) part requires pairwise data, the CS divergence estimator (Eq. (8) can operate seamlessly on unpaired data without introducing additional computation. This unique capability enables CS- Aligner to extend beyond traditional pairwise multimodal alignment by incorporating additional distributional information from unpaired data or tokens. Below, we introduce two novel directions.
Unpaired vision-language alignment. Our method leverages two forms of unpaired alignments: (1) images with multiple captions, and (2) independently sampled unpaired images and texts. The unpaired alignments are achieved using Eq. (8), where { and {y, J== can be independent with M * N. In both scenarios, our method leverages more uncurated unpaired data for distributional multimodal alignment, providing greater flexibility and robustness.
Vision-language token alignment. We propose a novel intra-sample distribution alignment ap- proach between vision and language tokens. Unlike CLIP-based models (Radford et al. 2021) aligning only the "CLS" tokens of vision and text, our method aligns all tokens for finer-grained alignment. Specifically, each vision feature x; RV is modeled token distribution p p(x,) containing V vision tokens, while each text feature y; E LL is represented as token distribution p(y;) with J. text tokens. D denotes the feature dimension. We compute CS divergence between vision and text token distributions, and obtain an internal token-wise alignment loss:
1 Σ Des(p(x)):((x.)),
Lioken
(12)
B
where B the batch size. general, V * L, and vision and language tokens do not have direct pairing, making InfoNCE inapplicable for estimation. Through our distributional alignment, Eq. (12) enables comprehensive alignment across all tokens, capturing more details and potentially enhancing fine-grained alignment.
3.3 PARAMETER- MULTIMODAL. ALIGNMENT
We demonstrate the effectiveness of CS-Aligner by performing vision-language alignment in parameter-efffeicnt manner using pretrained vision and language models, such as CLIP and large language models (LLMs) (Dubey al. 2024). To adapt these pretrained models, we employ two widely used frameworks: adapter (Gao et al. 2024) and LoRA (Hu et al. 202I). The adapter and LoRA enable efficient alignment the multimodal large-scale pretrained models, without requiring extensive computational resources. The whole framework is demonstrated in Fig.
Adapter LoRA alignment. We add a lightweight transformer /Vaswani] 2017) on top of the pretrained model as adapter that projects text or image embeddings into shared space; option- ally, we can insert trainable low-rank (LooR)) matrices into the text encoder's weights enable fine-grained adjustments, aligning the representations with the other modality.
EXPERIMENTS
We evaluate our method on two tasks to illustrate its vision-languee alignment ability: text-to- image (T2I) generation in Section 4.1 and image-text retrieval in Section 4.2 Note that we focus the vision-language alignment and use the generation task as a proxy to measure it. Additionally, provide the image-text classification and the image captioning results in Appendix H We also

---

Published as a conference paper at ICLR 2026
present the computation complexity and stability analysis in Appendix and additional ablation studies in Appendix H.T
4.1 TEXT TO IMAGE GENERATION
Datasets. Following a previous T2I approach (Patel et al. 2024), we train our method on four
datasets: MSCOCO (Lin et al. 2014), CC3M (Sharma et al. 2018). CC12M (Changpinyo et al., 2021], and LAION- LAION-HighResolumm HighResolution-SSM (Schuhmann et al. 2022). MSCOCO contains 80K im- ages paired with multiple captions. CC3M and CC12M include about 2.5M and 10M image-text pairs, respectively. LAION-HighResoluution comprises 175M high-resolution pairs, from which we select 5M for training. We evaluate the aligned model on the MSCOCO 30K validation set.
Experimental setup. We build our method based on unCLIP-style approaches (e.g.. DALL-E- 2 (Ramesh et al. (2022), Karlo (Donghoon et al. (2022), Kandinsky (Razzhigaev et al., 2..3)) These methods train a diffusion prior module on large-scale datasets (hundreds of millions of samples) to map text into the image representation space, and use decoder generate images.
Differently, CS-Aligner trains an adapter align text representations image fea- ture space on small-scale datasets, c.g., MSCOCO (0.08M), CC3M (3M), and CC12M (12M), and LAION- subset (SM). After alignment, we directly process the aligned text features using the pretrained decoder of the large-scale methods (e.g., Karlo and Kandinsky) to generate images, without additional prior modules or multi- ple diffusion steps. We evaluate generation quality with the FID score (Heusel et al.] 2017), which measures how closely gener- ated images match the real image distribu- tion. This metric is particularly well-suited for evaluating modality alignment, as it di- rectly reflects the distribution distance. Ad- ditional details can be found in Appendix G
Table 1:
Comparisons with T2I methods. Our method outperforms large-scale diffusion- methods and the recent small-scale (alignment)
methods (Eclipse and IB (Almudévar al. 2025)
Methods
Datasize (M)
FID
Large-scale methods
SD v2.1
2000
14.51
SD-unclip v2.1
2000
13.15
Wurstchen
1420
23.60
DALL-E2 Kandinsky
250
10.65
177
20.48
Karlo
115
20.64
Small-scale alignment IB + Kandinsky decoder Eclipse Kandinsky decoder Ours Kandinsky decoder
(,08
150.52
0,08,coco)
16.53
0,08;coco)
12.62
Eclipse Karlo decoder Ours Karlo decoder
0,08;coco)
23.67
0.08
11.27
Baselines. Our baselines consists of both large-scale methods Karlo, Kandin- sky. Wurstchen (Pernias et al. (2023). Sta-
Ours SD-unclip decoder
0.08
10.88
ble Diffusion (Rombach et al. 2022) (SD v2.1 and SD-unClip), and the recent small-
scale alignment method Eclipse. We also compare with the most recent multimodal align- ment method (Almudévar et al.] (025) (denoted as IB) on the generation task. For fair-
ness, we use the same Transformer adapter as Eclipse (also for (Almudévar et al. (025)) and only align the "CLS" tokens, highlighting the advantages of distributional alignment.
Comparisons. We compare our method with both the large-scale diffusion-based methods and the small-scale alignment methods. The re- sults are provided in Table T By aligning text representations to image representations on the small MSCOCO data, our method achieves su- perior T21 generation than the large-scale meth- ods, Karlo, Kandinsky, and Stable Diffusion without any diffusion steps. CS-Aligner also outper- forms Eclipse and IB by obvious margin using either Karlo Kandinsky decoders. The results demonstrate the effective vision-language alignment capability of our method. Moreover, we com- pare Cl-Aligner with Eclipse across different training datasets. As shown in Table 2 our method performs better across diverse training data (CC3M, CC12M, and LAION Higoo--ighess-MMM under- scoring the importance of the modality distribution information for robust alignment.
Table Comparisons on various training data. Our method consistently performs better.
Method
CC3M
CC12M
LAION-HighRes 5M
Eelipse Ours 22.88
26.73
26.98
19.16
22.72
14.79
Qualitative Visualization. To further test our method, Fig. 4a shows qualitative visualizations of generated images using Karlo decoder. Our aligned text representations result in more realistic
7

---

Published as a conference paper at ICLR 2026
Alignment
Eclipse
Aiiggireer
w/o aligoment Overall FID: 12,62
alignment Overall RID: 12:14
SA siting besiae loptop on daskk"
TV'n
bechground
men stanoing boatneet oog"
(a) Qualitative comparison. No alignment (left), Eclipse (middle), and CS-Aligner (right). CS- CS-Aligner yields more realistic, semantically consistent generations.
(b) CS-Aligner with token alignment.
Token alignment enhances fine-grained vi-
sion-language correspondence.
Figure 4: Qualitative visualizations.
images with stronger semantic consistency with the input sentence, highlighting the effectiveness of CS-Aligner in enhancing alignment. More visualizations are provided in Appendix E.
CS-Aligner with different adaptation ap- proaches. To demonstrate the robustness of our method across different models, we perform alignments for T2I using both adapter and LoRA. Specifically, we apply LoRA with a low-rank dimension of 8 to every transformer layer in the CLIP text encoder. As shown in Table 3 based on different decoders, Al-Aligeer with LoRA introduces fewer parameters, while still achieving comparable results compared with the adapter- bbased one, showing the effectiveness and adaptability of CS-Aligner across different models.
Table 3: CS-Aligner with different adapta- tion approaches. Our method achieves good alignment using both adapter and LoRA.
Base Model
Adaptation
#Parameters
FID
Adapter
34M
12.62
Kandinsky
LoRA
6M
13.52
Adapter
33M
11.27
Karlo
LoRA
1.3M
15.63
CS-Aligner with multiple captions. It is common in real-world datasets for single image to spond to multiple captions (e.g.. 5 captions per image in MSCOCO). Due to their pairwise alignment nature, previous methods such as InfoNCE and ba-baedd approaches (Radford al., 2021 Patel
et al. 2024) struggle to simultaneously leverage multiple captions. In contrast, by incorporating CS divergence, our CS-Aligner enables training for alignment with single image and multiple captions through the divergence term. To demonstrate the benefits of multiple captions for CS-Aligner we conducted experiments on the MSCOCO dataset by estimating the CS divergence term Des in Eq. (6) using both single and multiple captions. As shown in Fig. 5a CS-Aligner effectively leverages the information provided by multiple captions, leading to improved vision- laiiun- alignment.
CS-Aligner with additional unpaired data. Collecting and accurately annotat- ing paired vision-language data is both challenging and costly. Enhancing align- ment with additional unpaired data offers more flexible and scalable solution for real-world applications. However, simi- lar the case of multiple captions. pre- vious methods (Radford et al., 2021] Pa- tel al. 2024) struggle to fully utilize un- paired data due to their reliance on pair- wise alignment, whereas Al-Aligner natu- rally incorporates the unpaired data infor- mation by CS divergence. To demonstrate this capability, we conduct experiments on the MSCOCO dataset using the Kandinsky decoder with (1) 80K paired training samples, (2) 40K paired train- ing samples, and (3) 40K paired training samples supplemented with 80K unpaired samples, where the unpaired samples are used to estimate the CS divergence. As shown in Fig. 5bb] the result with
13.0
15
12.62
12.5
14.71
12.0
11.78
13
12.
12.18( unpaired data)
Mu Multiple Captions each image
Single
6ü Amount at Paired Data (K)
B0
(a) Aliga with multi-captions. (b) Align with unpaired data.
Figure 5: CS-Aligner with additional information. Our method benefits from the additional information from multiple captions (a) and unpaired data (b).
8

---

Published as a conference paper at ICLR 2026
40K paired training data is lower than 80K. However, introducing additional unpaired data obvi- ously improves the performance, even surpassing the model trained with 80K paired samples. This demonstrates CS-Aligner ability to effectively leverage the distributional information of modalities for alignment.
CS-Aligner with token alignment. Beyond the unpaired data, CS-Aligner also enables token-level alignment by treating the tokens of each sample as a distribution. We evaluated the token-level extension of CS-Aligner with the Kandinsky decoder on MSCOCO. As shown in Fig. 4b) incor- porating token alignment further improves performance. Moreover, qualitative results indicate that token alignment enhances fine-grained details in generated images, suggesting an improved ability to capture fine-gr ane-grained relationships between modalities. Additional visualizations are provided in Fig. in Appendix F.2
4.2 IMAGE- IMATE-TEXT RETRIEVAL
Experimental Setup. Effective multimodal alignment also benefits cross-modal retrieval.
To demonstrate the alignment abil-
ity of our method on retrieval tasks, we align LLMs (Dubey et al. 2024) text representations with CLIP sion representations on both image- to-text and texl-to-image retrieval. We use the Flickr 1K test set (Young
Table 4: Comparisons of image-to-text (I2T) and text-to- image (T2D) retrieval. Our method outperforms the base- lines on diverse datascts.
Flickr30k
Urban-1k
DOCCI
Average
Methods
12T
T21
12T
T21
12T
T2I
12T
T21
al.) 2014) for short-text retrieval,
Long-CLIP
90.0
76.2
82.5
86.1
66.5
78.6
79.7
80.3
while UrbanlK (Zhang al. 2025)
CLIP
85.2
65.0
68.3
55.6
63.1
65.8
72.2
62.1
and DOCCI (Onoe et al.j 2025)
LLM2CLIPPM
89.6
77.3
87.1
87.8
87.2
85.4
employed for long-text retrieval.
Ours-3M
91.8
81.0
87.6
92.2
86.6
89.1
88.7
87.4
We compare CS-Aligner against pure InfoNCE-based methods, such Long-CLIP (Zhang al.] 2025) and LLM2CLIP (Huang et al.)
2024), as the baselines. To ensure a fair comparison, we adopt the setup from LLM2CLIP, aligning CLIP ViT-L/14 image representations with Llama 3 (8B) text embeddings. Both the vision and text representations are aligned by adapters trained on CC3M.
Comparisons. Table shows that our method consistently and significantly outperforms the base- lines across various datasets for both image-to-text (I2T) and text-to-image (T2I) retrieval. This demonstrates the effectiveness of our method for aligning two modalities into a shared space. More- over, the ability to align different text encoder (LLM) with the CLIP image encoder highlights the flexibility and generalizability our approach.
5 RELATED WORK
Vision-language alignment and applications. CLIP (Radford et al. 2021) serves as founda- tional model for vision-language alignment in multimodal tasks. Several works have enhanced CLII through techniques such as momentum distillation (Li et al. 2021) I and noisy text supervision (Jia
al. 2022) Despite its success, CLIP suffers from a persistent modality gap between text and image representations. Prior studies (Zhou et al. 2023 Liang et al. 2022 Shi et al. (023)) attribute this gap to factors such cone effects (Liang al.) 2022) and suboptimal latent space structures (Shi al.)
2023 To address this, various strategies have been proposed, including projection adapters (Zhou al. 2023 Gao al. 20244) Huang et al., 2024), geodesic multimodal mixup (Oh et al.) 20224). and parameter-efficeent fine-tuning (Zanella & Ben Ayed 2024). Recent works also improve CL.IP by
large language models (L.LMs) (Jang et al. 2024 Koukounas et al. 2024 Huang et al. 2024) for downstream tasks such as image-text retrieval.
In addition to image-text retrieval, text-to-image (T2I) generation is another application that re- flects the vision-language alignment capability. T21 has advanced significantly over the past decades, driven by both diffusion- based (Ramesh et al., 2021 Rombach et al.; 2022, Saharia et al. 2022 Nichol et al. 2021) and GAN-based models (Zhang al. 2017 Tao al. (023). Among diffusion-
based methods, the unCLIP framework (Ramesh et al. 2021 2022) employs a two-stage architec-
ture with guided diffusion prior and decoder (e.g.. DAL..---2 (Ramesh et al. 2022)

---

Published as a conference paper at ICLR 2026
Karlo (Donghoon et al. )022)). Its prior module Ie maps text representations y image by a diffusion model. Recently, Eclipse (Patel et al. (024) employs an &g loss to simplify the prior loss by eliminating diffusion time and introducing noise term: Cptor 1o- ||x (c.
However, these methods still rely pairwise loss (e.g., f2). In contrast, our approach introduces distributional alignment for more holistic modality alignment.
CONCLUSION
In this paper, we propose CS-Aligner, novel distributional alignment framework that integrates Cauchy-Schwarz Schwarz (CS) divergence with mutual information for multimodal aligmment, which ad- dresses the alignment and uniformity conflict of InfoNCE. By combining global distributional align- ment with InfoNCE, CS-Aligner achieves tighter and more comprehensive alignment. By consider- ing the modality distributional information, our method enables to leverage additional and detailed information from unpaired samples and tokens, leading to more flexible and fine-grat fine-grained informa- tion for alignment. We demonstrate the effectiveness of our alignment on text-to-image generation and cross modal retrieval.
USE OF LARGE LANGUAGE MODELS (LLMS).
We used LLMs solely for minor language polishing. They were not involved in research ideation, experimental design, or substantive manuscript writing.
ETHICS STATEMENT
Our proposed method advances research in multimodal alignment by introducing novel distribu- tional alignment approach. As result, it also facilitates progress in multimodal generation. In the meantime, this capability may raise ethical concerns, including the potential misuse for generating deceptive or inappropriate content.
REPRODUCIBILITT STATEMENT
We provide sufficient details for reproducibility in Sections and
REFERENCES
Antonio Almudévar, José Miguel Hernández-Lobato Sameer Khurana, Ricard Marxer, and Alfonso Ortega. Aligning multimodal representations through an information bottleneck. arXiv preprint arXiv:2506. 04400 2025.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning, pp. 214-223. PMLR, 2017.
Nathanaël Berestycki and Richard Nickl. Concentration of measure, On their websites, 2009.
Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web-scale image-text pre-training recognize long-tail visual concepts. In Proceedings of the JEEE/CVF conference on computer vision and pattern recognition, pp. 3558-3568, 2021.
Thomas M Cover. Elements of information theory. John Wiley & Sons, 1999.
Marco Cuturi. Sinkhorn distances: Lightspeed computation optimal transport. Advances in neural information processing systems, 26, 2013.
Lee Donghoon, Kim Jiseob, Choi Jisu, Kim Jongmin, Byeon Minwoo, Back Woonhyuk, and Kim Saehoon. Karlo-v1.0.am .0.alpha on c0yo-100m and cc15m. ht.tps://githut com/ Sakaobrain/kart 2022.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Leuman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint a07.222407..1788. 2024.
10

---

Published as a conference paper at ICLR 2026
Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang. Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Viston, 132(():581-59) 2024.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho and Tim Salimans. arXiv:2207.12598, 2022.
Classifier-free
diffusion guidance.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2 106.0006099555 2021.
Weiquan Huang, Aoqi Wu, Yifan Yang, Xufang Luo, Yuqing Yang, Liang Hu, Qi Dai, Xiyang Dai, Dongdong Chen, Chong Luo, et al. Llm2clip: Powerful language model unlock richer visual representation. arXiv preprint arXiv:2411.04997 2024.
Young Kyun Jang, Jummo Kang, Yong Jae Lee, and Donghyun Kim. Mate: Meet at the embedding connecting images with long texts. arXiv preprint arXiv:2407.05541, 2024.
Robert Jenssen, Jose C Principe, Deniz Erdogmus, and Torbjørn Eltoft. The cauchy-schwarz diver- gence and parzen windowing: Connections to graph theory and mercer kernels. Journal of the Franklin Institute, 343(6)):14-629) 2006.
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. International conference on machine learning, pp. 4904 4916. PMLR, 2021.
Kittipat Kampa, Erion Hasanbelliu, and Jose C Principe. Closed-form cauchy-schwand pdf diver- gence for mixture of gaussians. The 201 International Joint Conference on Neural Networks, pp. 2578 2585. IEEE, 2011.
Jin-Hwa Kim, Yunji Kim, Jiyoung Lee, Kang Min Yoo, and Sang-Woo Lee. Mutual information divergence: unified metric for multimodal generative models. Advances in Neural Information Processing Systems, 35:35072-35086, 2022.
Alexander Korotin, Daniil Selikhanovych, and Evgeny Burnaev. Neural optimal transport. arXiv preprint arXiv:2201.12220. 2022.
Andreas Koukounas, Georgios Mastrapas, Michael Günther, Bo Wang, Scott Martens, Isabelle Mohr, Saba Sturua, Mohammad Kalim Akram, Joan Fontanals Martínez, Saahil Ognawala, al. Jina clip: Your clip model is also your text retriever. arXiv preprint arXiv:2405.20204 2024.
Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafig Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learning with momentum distillation. Advances in neural information processing systems, 34:9694 9705, 2021.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference machine learning, pp. 19730-19742. PMLR, 2023a.
Tianhong Li, Sangnie Bhardwaj. Yonglong Tian, Han Zhang, Jarred Barber, Dina Katabi, Guil- laume Lajoie, Huiwen Chang, and Dilip Krishnan. Leveraging unpaired data for vision-language generative models via cycle consistency. arXiv preprint arXiv:2310.03734, 2023b.
Victor Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Y Zou. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. Ad- vances in Neural Information Processing Systems, 35:17612-17666 2022.

---

Published as a conference paper at ICLR 2026
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.
Ron Mokady, Amir Hertz, and Amit H Bermano. Clipcap: Clip prefix for image captioning. arXiv preprint arXiv:21 11.09344 2021.
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2 2021.
Changdae Oh, Junhyuk So, Hoyoon Byun, YongTack Lim, Minchul Shin, Jong-June Jeon, and Kyungwoo Song. Geodesic mulli-modal mixup for robust fine-tuning, Advances in Neural Infor- mation Processing Systems, 36, 2024.
Yasumasa Onoe, Sumayana Rane, Zachary Berger, Yonatan Bitton, Jaemin Cho, Roopal Garg, Alexander Ku, Zarana Parekh, Jordi Pont-Tuset Garrett Tanzer, et al. Docci: Descriptions of connected and contrasting images. In European Conference on Computer Vision, pp. 291-309. Springer, 2025.
Aaron den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arviv preprint / 2018.
Emanuel Parzen. On estimation of probability density function and mode. The annals of mathe- matical statistics, 3363):1065-10776, 1962.
Maitreya Patel, Changhoon Kim, Sheng Cheng, Chitta Baral, and Yezhou Yang. Eclipse: A resource-efficient text-to-image prior for image generations. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 9069-90990990 9078, 2024.
Pablo Pernias, Dominic Rampas, Mats L Richter, Christopher Pal. and Mare Aubreville. Würstchen: An efficient architecture for large-scale text-to-image diffusion models. arXiv preprint (tt1:2336600063377 2023.
Jose C Principe, Dongxin Xu, John Fisher, and Simon Haykin. Information theoretic learning. Unsupervised fiteriig, -19, 2000a.
Jose C Principe, Dongxin Xu, Qun Zhao, and John W Fisher. Learning from examples with infor- mation theoretic criteria. Journal of VLSI signal processing systems for signal, image and video technology, 26:61-77, 2000b.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 9748-8765 PMLR, 2021.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-imase to-image generation. In International conference on machine learning, pp. 8821-8831. Pmir, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022.
Anton Razzhigaev, Arseniy Shakhmatov, Anastasia Maltseva, Vladimir Arkhipkin, Igor Pavlov, Ilya Ryabov, Angelina Kuts, Alexander Panchenko, Andrey Kuznetsov, and Denis Dimitrov. Kandin- sky: an improved text-to- text-to-image synthesis with image prior and latent diffusion. arXiv preprint 1000v:2310.03502 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
12

---

Published as a conference paper at ICLR 2026
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural informa- lion processing systems, 1136479-36494, 2022.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 15:25578-255255 2022.
Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypemnymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp- 2556-2566 2018
Peiyang Shi, Michael C Welle, Märten Björkman, and Danica Kragic. Towards understanding the modality gap in clip. In ICLR 2023 Workshop on Multimodal Representation Learning: Perks and Pitfalls, 2023.
Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. Eva-clip: Improved training techniques for clip scale. arXiv preprini arXiv:2303. 15389, 2023.
Ming Tao, Bing-Kun Bao, Hao Tang, and Changsheng Xu. Galip: Generative adversarial clips for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 44214-14223, 2023.
Linh Tran, Maja Pantic, and Marc Peter Deisenroth. Cauchy-schwarz regularized autoencoder. Journal of Machine Learning Research, 23, 2022.
Daniel Trosten, Sigurd Lokse, Robert Jenssen, and Michael Kampffmeyer. Reconsidering repre- sentation alignment for multi-view clustering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1255-1265, 2021.
Vaswani. Attention all you need. Advances in Neural Information Processing Systems, 2017.
Qing Wang, Sanjeev R Kulkarni, and Sergio Verdú. Divergence estimation for multidimensional densities via k-nearest-neighbor distances. IEEE Transactions on Information Theory, 55(5): 2392-2405, 2009.
Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through align- ment and uniformity on the hypersphere. In International conference on muchine learning, pp-
9929-9939. PMLR. 2020.
Wenzhe Yin, Shujian Yu, Yicong Lin, Jie Liu, Jan-Jakob Sonke, and Stratis Gavves. Domain adap- tation with cauchy-schwarz divergence. In The 40th Conference on Uncertainty in Artificial In- telligence, 2024.
Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. From image descriptions visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2:67-78, 2014.
Shujian Yu, Xi Yu, Sigurd Løkse, J and Jose C Principe. Cauchy-schwarz divergence information bottleneck for regression. arXiv preprint arXiv:2404. 17951, 2024.
Shujian Yu, Hongming Li, Sigurd Løkse, Robert Jenssen, and José C Príncipe. The conditional schwarz divergence with applications to time-series data and sequential decision making. IEEE Transactions on Puttern Analysis and Muchine Intelligence, 2025.
Maxime Zanella and Ismail Ben Ayed. Low-rank few-shot adaptation of vision- models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 1593-1603, 2024.
13

---

Published as a conference paper at ICLR 2026
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training In Proceedings of the IEEE/CVE international conference on computer vision, pp. 11975-11986, 2023.
Beichen Zhang, Pan Zhang, Xiaoyi Dong, Yuhang Zang, and Jiaqi Wang. Long-clip: Unlocking the long-text capability of clip. In European Conference on Computer Vision, pp. 310-325. Springer, 2025.
Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang. and Dim- itris N Metaxas. Stackgan: Text to photo-realistic image synthesis with stacked generative ad- versarial networks. Proceedings of the IEEE international conference on computer vision, pp- 5907-5915, 2017.
Chuanxia Zheng, Tung-Long Vuong, Jianfei Cai, and Dinh Phung. Movq: Modulating quantized vectors for high-fidelity image generation. Advances in Neural Information Processing Systems, 35:23412-23425,, 2022.
Chenliang Zhou, Fangeheng Zhong, and Cengiz Öztireli. Clip-pae: dnjection-augmmntttionn em- bedding to extract relevant features for a disentangled, interpretable and controllable text-guided face manipulation. In ACM SIGGRAPH 2023 Conference Proceedings, pp. 1-9, 2023.
14

---

Published as a conference paper at ICLR 2026
A DETAILS OF THE TOY EXAMPLES
Example A.I. Consider two Gaussian distributions, p(x) NNxxx 0z) and ply) N ofy), Here, Hx and My are the My (pxxa oy
Mx o
poxay
with a joint distribution y)x,y) ~ N
means of x and y, x and o² are their variances, and p is the correlation coefficient and controls their linear dependency. When p 0.99, the two modalities are highly dependent, with high mu- tual information (I = 1.959; see Fig. 2h and 2p). When = 0, the modalities are independent, resulting zero mutual information (Fig. 2c). Interestingly, two distributions with the same mutual information value can either exhibit minimal statistical distance and nearly identical shapes, includ- ing similar locations, widths, and higher-order moments, as shown in Fig. 2p. or have completely different shapes with distinct means (0 for p(x) and 2 for p(y)) and variances (4 for p(x) and 1 for p(y)), as illustrated Fig. 2h. Quantitatively, the former case shows a minimal KL divergence of O, while the latter exhibits KL divergence of nearly 6.81.
Mutual information. For two continuous random variables x and y, the mutual information is defined
p(xy) y(x;))= = // p(x,y) log p(x) p(y)
dx dy.
(13)
For bivariate Gaussian distribution
y(x.y) N (Mx) (gy)
of (
poxao
poxxy
oy
the mutual information admits the closed-fore solution:
y(x:y) - In 11-1ppa
(14)
In particular, for correlation p 0.99, we have 7(x,y) = 1.959, while for p 0, the variables are independent and [xx,y)
Divergence. For univariate Gaussian distributions p(x) Nixxx o2) and ply) -Vuy, the KL divergence is given by:
22 + (fxx 1ly)
DKL (p(x) (Iy))) In
(15)
203
For Fig. 2 and Fig.
2 we set ox = =0y- = Hence, when μix = Myy = 0, DKi. (p(x) (ly)) 0.
For Fig. 2h we use σ₂ = and oy = 1. When μx = 0 and Hy = 2, the DKL (p(x) ( ( 6.81, which is Very large.
B DERIVATIONS
In this section, we provide derivation of alignment and uniformity terms of InfoNCE. More con- crete analysis can found (Wang & Isola 2020).
Let y) be positive (image-text) pairs drawn from Ppairs and let (x{ i= be M negative samples (unpaired samples) drawn i.i.d. from the marginal Pdata- The one-sided InfoNCE (CLIP) loss with temperature 0 is
1 E (x (x(.))
X/r
ex Vy/T
LIafONCE
log
log
M Ei=1
x2//
ΣM exTxx// 8//T
In CLIP, the features are normalized to compute the loss. Under this unit-norm constraint |x||2 ||yla LinfaNCE decomposes into
M x v'lT +
M
LIMONCE = -E -Pipic
x y
+
(xx.))-pau
log im
log
e
IT y/7
Caige
Lunttorm
15

---

Published as a conference paper at ICLR 2026
up additive constant. Moreover, by writing
| v112 - |x| + 11ll 2x = 2
x y
(16)
can show that:
Alignment.
x TTyy y 5
1
x y11
1
(x,))
[B[[1x x y yl
T
5
2T
Dropping the constant -1 1/TT, define
Calign
E (
[||x y11²]
(17)
2T
iii Uniformity, For each negative (unpaired) sample v/, using Eq. equation 16
vilt = 2211-~2211) vill e//r e 2r vi
Hence
xx'x si/t e//r
Σe 27 ||x
log
x x vl/t
log Σ
27 vi1?
An identical argument holds for the {x y) terms. Up to constants,
M
M
Cuniform == B(xxy)>Paate
log
2r
x xil?
+
log Σ c
11
12
(18)
In the limit of large batch size one may further rewrite
Luniform log Ex,y [exp(-t[[x -|]]))
with 2F Combining (i) and (ii) and absorbing all additive constants gives the desired decomposition
Lelip
Lalign + Luniform + const.
C RELATED WORK OF CAUCHY-SCHWARE (CS) DIVERGENCE.
CS divergence (Principe et al. b) is derived from the Cauchy-Schie Schwwrz inequality for square- integrable functions. It serves symmetric distribution distance metric with notable properties, such as the ability to measure conditional distributions (Yu et al. 2025) and the closed-fome expres- sion for mixtures of Gaussians (Kampa al. 2011). CS divergence has been successfully applied across various domains, including deep clustering (Trosten et al. 2021), disentangled representation learning (Tran al., 20022,, and deep regression (Yu al. 2024). Moreover, due to its advantage of estimating discrepancy between conditional distributions, it has demonstrated success in the domain adaption area (Yin al. 2024) and time series clustering (Yu al. 2025). However, the utility of CS divergence in foundation models remains unclear and unexplored.
D COMPARISON BETWEEN CS DIVERGENCE AND OTHER METRICS
Unlike parametric distributions, distributions of different real-world modalities exhibit unpredictable variability and inconsistent overlaps, meaning that p(x) and p(y) may follow arbitrary distributions with a small intersection. Therefore, it is crucial to overcome these challenges to measure and optimize multimodal distribution divergence robustly. Below, we outline several key properties that an effective metric should satisfy for multimodal alignment. Remark D.I. Key properties for distribution align metries:
16

---

Published as a conference paper at ICLR 2026
Symmerry: Both distributions are treated equally, ensuring consistent and unbiased multi- modal alignment, formulated by D(p(x), p(y)) D(p(().
Difserentiable and Efficient Estimation: Enable differentiable estimation without dis- tribution assumptions to facilitate optimization, formulated as OD(p(x: 0), p(y: 6)) * 0, p(y) Achieve the estimation non-parametrrcally or efficiently.
Provide reliable measurements even when dis- tributions have minimal overlap of supports, which may often occur multimodal scenar- ios. The property is formulated as 0 S ((p(x), p(y)) I oc when < {(ssup(p(x)) n supp(p(y))) ((suup(p(x)) supp(p(y))) denotes the overlap of p(x) and p(y). e is a small value.
Robustness Small Distribution Overlap:
These properties enable the divergence term to align arbitrary distributions with small support over- lap, which is well-suited for large-scale multimodal applications involving deep learning.
D.1 CONNECTION TO THE PRIOR LOSS
Remark D.2. Connection to the prior loss (&2 loss) used by Eclipse (Patel et al. 2024
Sprirr (e..)
||x go ()(e))))l
(19)
Consider the third term in Eq. Hi (x. = exp( - ||x 20²2
8 which involves ,(x,,,)) defined by the Gaussian kernel
A second-order Taylor expansion yields
(x: y;) ~
(x. y3)²
K ((x,),) = exp
(20)
20
20²
When i = (i.c., diagonal of x(x, y)), this approximation reduces to weighted & loss by 1/20², analogous the Eq. 19 Consequently, the & loss emerges as special case of our divergence, fo- cusing solely paired sample reconstruction and omitting broader distribution alignment, including off-diagonal (cross-sample) contributions.
D.2 COMPARISON WITH KI. DIVERGENCE.
KL divergence is widely used metric in deep learning, Given two distributions, p(w) and q(w), the KL. divergence is defined as:
())(P: / p(w) log dw. plw) q(w)
(21)
Validity for multimodal alignment. Define the support sets of distributions and as:
supp(p) = (w E Ω p(w) > 0}. supp(a) = (w Ω q(wp))))))))))))())) > 0}.
(22)
For KL divergence, if there exists any point p(w) log p(w) - sc, leading to: DKL q) = oc. Thus, a necessary condition for KI. divergence to 0
supp(p) such that <(x) 0. the term
be finite is supp(p) C supp(q). Otherwise, KL divergence becomes invalid.
In contrast, the CS divergence becomes infinite only if there is no overlap between supports of p and i.e., when f p(w))(w)dd = 0, making the logarithm undefined. Hence, the condition for finite CS divergence is: supp(p) O supp(g) + 0.
In multimodal alignment, it's reasonable to assume that the two modality distributions partially over- lap but are not disjoint, as supported by our empirical observations in Fig, 4 (no alignment results). Under these conditions, KL divergence divergence condition is less restrictive, making it more suitable and stable for multimodal alignment.
be invalid and therefore suboptimal. Conversely, the CS
Compatibility with InfoNCE Integrating InfoNCE with CS divergence explicitly encourages intra- modality uniformity and cross-man cross-modality alignment, thereby effectively improving multimodal align- ment. For KI. divergence, assuming the distributions of the two modalities are Gaussian, A(ro Ex)
17

---

Published as a conference paper at ICLR 2026
and Wnd1 E1), the divergence can be computed as:
det 21
DKL (Wo,, Eo) Mn) = 2 tr
+ (m1 _o) (m) -uo) k + log
del Eg (23)
This formulation lacks explicit connections to the InfoNCE in terms of alignment and uniformity, making it less compatible with InfoNCE compared to the CS divergence.
Nonparametric estimation. Additionally, when the distributions are not assumed to be Gaussian, a nonparametric estimator is required for KL divergence. A common choice, the k-NN estima- tor (Wang et al. 2009), is non diffeerrntiiable, which poses challenges for optimization in gradient- based learning frameworks. contrast, the CS divergence demonstrates greater stability and differ- entiability when paired with KDE, making it more robust choice.
Experimental Comparison. To verify the above analysis, we compare CS divergence and KL. divergence on the unpaired data scenario, where KL can easily become invalid. We trained KL + InfoNCE model in our unpaired data setting-using paired data for InfoNCE and unpaired data for divergence. The initial KL value exceeded 5000 (extremely large), and consequently, the model could not converge, leading to catastrophic failure. In contrast, CS divergence remained stable (initial value around 3), and achieved comparable final performance with FID of 12.18 (Fig. 5b in the main paper).
D.3 COMPARISON WITH WASSERSTEIN DISTANCE.
Wasserstein Distance is also widely used for distribution discrepancy (e.g. GAN (Arjovsky al.
2017 However, Wasserstein distance be computed either by using an additional learnable mod- ule (e.g., neural network for estimating transport map (Korotin et al. 2022) by solving an
optimization problem, often approximated via multiple Sinkhorn (Cuturi 2013) iterations for com-
putational efficiency, leading to efficiency problem in large-scale training. In contrast, CS divergence can be efficiently estimated by a nonparametric estimator.
D.4 QUANTITATIVE COMPARISONS WITH KL. AND WASSERSTEIN DISTANCE.
We compare method with KL and Wasserstein distances below. To make the KL divergence tractable, we assume the batch embeddings follow Gaussian distributions. For the Wasserstein dis- tance, either use the closed-form Gaussian Wasserstein distance under the same assumption apply the Sinkhorn algorithm for general distributions. However, in practice, we found that Sinkhorn often fails to converge. The results show that our method outperforms both KL and Wasserstein distances. Moreover, Wasserstein distance and KL lack an InfoNCE-style alignment-unformity decomposition: only -Sseeveree yields the compatible formulation (Eq. [Op. The Gaussian as- sumption is also stronger than our nonparametric method.
Method
FID +
KL
23.48
W-distance
18.41
Sinkhorn
Not converge
CS- Aligner
12.62
D.5 COMPARISON WITH MUTUAL INFORMATION DIVERGENCE (KIM ET AL..
2022
Mutual information estimation depends on parametric assumptions about the underlying distribu- tions, e.g., multivariate Gaussian, whereas CS divergence imposes such constraints. Moreover, estimating mutual information decomposes into a mutual information term plus two KL divergences, and thus lacks explicit connections the InfoNCE in terms of alignment and uniformity.
18

---

Published as a conference paper at ICLR 2026
E COMPUTATION COMPLEXITY AND STABILITY ANALYSIS
We normalize high-dimensional embeddings onto the unit hypersphere and use a fixed Gaussian kernel bandwidth so that concentration of measure and classical KDE theory ensure stable, low- variance estimates.
In high dimensions, mapping embeddings onto the unit hypersphere exploits the concentration of measure phenomenon: as d grows, the pairwise distances | yll between random points on 1 concentrate sharply around 22, with fluctuations of order O(1vvd). Consequently, Gaussian kernel
|| w11
K exp(
(24)
2x
with fixed bandwidth (e.g. 1) yields values confined narrow, well-behaved range, pre- venting weights from collapsing to 0 or saturating at 1 and ensuring smoothly varying density esti- mates (Berestycki & Nickl 2009).
Moreover, when the effective sample size (e.g. batch size) and dimensionality d satisfy n o > 1, which holds for 1, n 103, and d 10S, the KDE estimator obeys central limit theorem. This guarantees that CS divergence estimates have vanishing variance and stable gradients during optimization (Parzen 1962p.
Computational complexity. The computation of method is comparable the CLIP- based method when scaling up to even larger-scale datasets. The computation complexity is O(N²), which is the same the InfoNCE used in CLIP. However, the computational complexity feasible to scale to larger-scale datasets.
F
MORE RESULTS
F.1
MORE VISUALIZATION
We illustrate more high- high-resolution images generated by the Kandinsky decoder with our aligned text representation Fig. The adapter trained on LAION-HighRes 5M.
F.2
MORE VISUALIZATION FOR TOKEN ALIGNMENT
We provide visualizations with and without the token alignment Fig. demonstrating its ability to generate more fine-grained images with CS-Aligner.
G IMPLEMENTATION DETAILS
Implementation details Our models were trained on 4 NVIDIA RTX A100 GPUs with global batch size of 1,024 (256 per GPU). We optimized parameters using Adam W with cosine annealing learning rate schedule, spanning total of 100 GPU hours. Mixed- preccsine training (FP16) was employed to enhance computational efficiency while maintaining stability. We use the learning rate of 5e 5. We hyperparameter 0.01 keep the number scale the divergence.
Kernel density estimator. A proper kernel size is critical in KDE for accurate estimation of Eq. g). In this paper, we follow Yin et al. (2024) to normalize the features from two modalities and use kernel size 1. In general, this is sufficient ensure stable learning.
G.1 T2I DETAILS
Figure 1 implementation details. For Fig. la and Fig. lb we train the same adapter on top of the CLIP model using InfoNCE and CS-Aligner, respectively. We use the MSCOCO training set and visualize the learned representations with t-SNE 5K image-text text pairs from the validation set. For the temperature in both InfoNCE and Aliggie we initialize it from the pretrained CLIP model and keep it learnable during training. For Fig. Tcl compute the L.2 distance between
19

---

Published as a conference paper at ICLR 2026
vase flowers diaplayed the front windows of store
A beld man with beard wearing polka
basket nill with different types
bedroom decorated with blue underwater thems
A wooden cutting board filled chopped vegetables
"Head shot" of zebra against background
A backpack and line of supplins laying out
very blue yellow and red
gummy bears altting on e counter
bench
slide glass door
has an Iron bed frame
different types of plant Life
big black dog sitting next to a (aptop computer
A big brown horse near s fance
big boet in the ses with other boets
black and white huggging handbag
Figure 6: Qualitative visualization. The adapter is trained on LAgON-HighRes 5M. The aligned text representation is then decoded by the Kandinsky decoder.
20

---

Published as a conference paper at ICLR 2026
black and green troin engine on treck
black per filled with mushrooms and Vegetables
A nest yellow bed in room with blue walls
LiVing room setting with furniture, fireplace and lamp
w/o token alignment Overall FID: 12.62
w/ token Alignment Overall FID: 12.14
Figure Token alignment is effective for fine-grained generations with more details and stronger semantic correspondence with the text inputs.
the embeddings of all image-text pairs and visualize the resulting histogram. The histogram of L2 distances for positive pairs systematically reflects their distance distribution.
Kandinsky details. We use Kandinsky v2.2, an unCL.P-type model that utilizes CLIP ViT-bigG- #4-aion2B-339B-bb with 1280 projection dimensions for text and image encoders. Kandinsky v2.2 employs latent diffusion model and MOVQ (Zheng et al.) (022) as the decoder to generate images of size 512 512 from the given image representation. When using the Kandinsky decoder,
we apply 50 denoising steps (Ho et al. 2020) with a classifier-free guidance scale of 7.5 (Ho & Salimans 2022).
Karlo details. Karlo uses CLIP- ViT-L/14 with 768 projection dimensions for image and text en- coders. It employs diffusion model to decode the image representation into a low-resolution image, followed by a super-resolution diffusion module that upsamples it to 512 x 512. When using the Karlo decoder, we apply 25 denoising steps with classifier-free guidance scale of 7.5, followed by an additional 7 super-resolution steps.
Adapter details. To ensure a fair comparison, our adapter module has the same architecture as Eclipse (Patel et al. (024)) which is based on the standard PriorTransformer model (Ramesh al.) 2022) but modified to be time-independent. Specifically, it consists of 10 layers with 16 attention heads, each having head dimension of 32. The embedding dimension is 768/1280, with three additional embeddings. The model does not use time embeddings and has a dropout rate of 0.0. For the to image generation task, in order use the pretrained image generator, we only use the text adapter. For the retrieval and classification, we use adapters for both modalities.
LoRA We configure LoRA (Low-Rank Adaptation) for CLIP with a rank of 8 and scaling factor a 16, enabling efficient adaptation while maintaining a low computational footprint. The targeted modules include the self-attention projections, the fully connected layers, and the pext.project i on layer, ensuring adaptation across both vision and text processing components. A dropout rate of 0.1 is applied to enhance regularization. For the CLIP encoder in Kandinsky, ViT- bigG- 14-ooi4 B-B------bb-- 908-39---6660- b16kk, the number LoRA parameters is million. As for CLIP- ViT-L/14 in Karlo, the CLIP model size is smaller, resulting in 1.3 million LoRA parameters.
LAION- Risoottin selection. We subset of 5 million image-text pairs from the LAION- HighN--iihessolutiinn dataset. which contains 175 million pairs. Due to computational con- straints, download only portion of the dataset and select pairs with English captions.
21

---

Published as a conference paper at ICLR 2026
H MORE EXPERIMENTAL RESULTS
Image-text classification. We compare with CLip-Adapter (Gao al. 2024 on the image clas- sification task following their few-shot classification setting. We fine-tune the adapter based on ViT-B/16 with 16-shots subset for each of the datasets. The results are provided in the following table. With better alignment, our method consistently performs better.
Table 5: Comparison with CLIP-Adapter on the image classification task. Our methods per- forms consistently better on various datasets.
Methol
ImngeNi
Caitechiii
DTD
EunSAT
RGVCAinaat
Faell0
Flowen 102
OsferdPets
Stanfo dCas
SUN397
UCF161
Assrage
Gso al. 2024
21
91.4
70.9
85.7
42.8
85.2
96.0
92.1
78.6
75.0
$2.8
79,3
ours
729
95.0
72,3
87.2
44.4
858
97.5
93.0
81.9
76.2
$4.0
$0.9
Table Image captioning results.
Method
Bleu_
CIDEr 1
InfoNCE+LM
40.4
14.3
InfONCE+LM+CS
41.3
16.7
Image captioning. We extend our method to the image captioning task. We adopt the Blip2 LLi al. 2023a) stage one training strategy to highlight the importance of representation alignment for the image captioning task. We train a Q-former with the image text contrastive loss (InfoNCE) and the language model loss on the MSCOCO captioning dataset. The results in Table show show that our method can improve the image captioning ability. Also, the qualitative results of image captioning are shown in Fig. reffig:vis-can -iaptiiiin.....iiinng.. The generated captions are semantically aligned with the images, demonstrating the general applicability of our method.
Captioning: A young boy Weering headphones.
Captioning: A cluttered room with table and chairs in it.
Captioning: beby laying in e bad with teddy bear.
Figure 8: Qualitative results of image captioning.
H.1 ABLATION STUDY
Hyperparameter Sensitivity Analysis. We conducted sensitivity analysis on the two key hy- perparameters, (the weight for InfoNCE) and (the Gaussian kernel width). For efficiency, we evaluated on subset of 10000 MSCOCO training samples and report Fréchet Inception Distance (FID) as the metric.
Table shows that our method is robust to moderate variations in both À and with only minor FID fluctuations over wide range. large À overemphasizes distributional alignment, optimizing intra-modality uniformity and global distribution distance while overlooking the pairwise alignment term. Since the generation task is sensitive to both global distribution closeness and sample-we wise alignment, excessively large À degrade performance. We also evaluate the sensitivity of A and a on the MSCOCO retrieval task. The results show that our method robust and performs well across a wide range of hyperparameters.
22

---

Published as a conference paper at ICLR 2026
Table 7: Sensitivity of FID and
X
0.01
0.1
10
0.1
0.5
1.5
FID
81.34
32.51
29.86
65.79
FID 30.58
27.79
29.86
31.49
(a) À sensitivity
(b) sensitivity
Table 8: Sensitivity to X and MSCOCO retrieval
0.1
0.5
1.5
0.01
0.1
10
R
49.3
50.9
50.7
50.2
R 50.1
50.6
50.7
48.6
Alignment with InfoNCE is not enough for the generation task. We ablate InfoNCE and In- foNCE with CS divergence (CS-Aligner) the text-to-image generation task. Specifically, train the adapter on the MSCOCO dataset and use the Kandinsky decoder to generate the corresponding images. For the InfoNCE temperature, learaall.. We then compute the FID score for comparison (lower is better). Table 9 shows that InfoNCE alone struggles to align the multimodal distributions, resulting the learnable temperature (inherited from CLIP) decreases during training, the contrastive logits become sharper, making the uniformity term dominate over the alignment term and thereby weak- ening multimodal alignment (see our decomposition in Sec. 2) For text-to-image generation, the decoder requires the two modalities to lie in the same distribution, which InfoNCE alone is unable to guarantee. Hence, InfoNCE-only model may still perform well in oosinesimillarty-barr retrieval but fails in generation due to the persistent distributional gap.
resume from the pretrained CLIP model and keep
high FID score. As
We also provide the retrieval ablations. Retrieval requires only correct relative similarity ranking, not full distributional overlap, so the degradation InfoNNCE-nll is smaller. Nevertheless, CS- Aligner consistently outperforms InfoNCE-only, likely because the intra-modality uniformity terms (Eq. 9) promote better sample separability, which benefits retrieval.
Table 9: Ablation study of CS-Aligner on retrieval and generation. Alignment with CS-Aligner significantly outperforms using InfoNCE alone.
Retrieval
Generation
Method
T21
12T
Avg
FID
InfoNCE
50.1
65.8
$7.95
151.35
CS- CS-AAigner
50.7
66.34
58.52
12.62
23


Reviews:
[{"review_id": "BJBORTXkYZ", "reviewer": "Reviewer_8VHb", "summary": "This paper tackles an important but overlooked problem in InfoNCE used in CLIP when aligning text features and image features. The authors revealed that InfoNCE\nhas inherent conflict in terms of alignment and uniformity in multimodality and demonstrated in Figures 1 and 2 that this is the cause of the modality gaps. Therefore, in this paper, a new method called CS-Aligner was proposed that combines InfoNCE with Cauchy-Schwarz (CS) divergence. Two standard experiments in image-text alignment are done to validate the proposed CS-Aligner: text to image generation and image-text retrieval. In the text to image generation, it has been experimentally demonstrated that the proposed CS-Alignment can achieve better performance than previous large-scale methods and small-scale alignment (Table 1). Its generality has also been validated by using different datasets (Table 2). In the image-text retrieval task, it was shown that CS- Aligner outperforms previous approaches such as Long-CLIP (Zhang et al., 2025) and LLM2CLIP (Huang et al., 2024).", "questions": "None", "limitations": "", "rating": 10, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper is described in detail. The proposed method is mathematically solid. This paper gives important insight to the InfoNCE loss and proposes an important method that would give great impacts to the multimodal research community. \n\nExperimental results both in the main body and supplementary are convincing enough.", "weaknesses": "The proposed method has some parameter sensitivity as shown in Table 7 in the supplemental material. I do not think the method is parameter-independent as the authors claim. \n- Could you show soma sample images to show the method is robust to parameter changes, in addition to showing FID?\n- Is there any method that can automatically find optimal parameter settings rather than relying on users’ empirical optimization?\n\nSome minor modification proposals (no need to reply)\n- A missing period after eq. (21)\n- A missing period, a missing eq. number, and erroneous “?” in lines 911-913\n- “??” in line 1065 should be modified", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ipUg3mhBSD", "reviewer": "Reviewer_54pL", "summary": "The paper introduces CS-Aligner, a novel vision-language alignment framework that combines Cauchy-Schwarz divergence with InfoNCE to address the alignment–uniformity conflict. It enables both global distribution-level and fine-grained token-level alignment, supports unpaired and multi-caption data, and achieves superior performance on generation and retrieval tasks, particularly in low-resource scenarios.", "questions": "1.\tHow sensitive is the CS divergence estimation to kernel bandwidth (\\sigma) and the choice of kernel function?\n2.\tDoes the KDE-based divergence scale efficiently to very large multimodal datasets (e.g., LAION-400M)?\n3.\tCan the proposed distributional alignment be combined with optimal transport or contrastive regularization for improved interpretability?\n4.\tHow does token-level CS alignment affect semantic disentanglement across layers? Is it stable under random seed variation?\n5.\tDoes the framework ensure mutual consistency between global (distributional) and local (pairwise) alignment objectives?\n6.\tCould the approach be extended to temporal or 3D multimodal tasks (e.g., video–text, audio–image)?\n7.\tGiven its use of CS divergence, can the model handle multimodal imbalance (e.g., text richer than image features)?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The authors integrate Cauchy-Schwarz divergence with mutual information (InfoNCE) and provide a theoretical analysis of how it alleviates the alignment–uniformity conflict, addressing limitations in InfoNCE-based alignment.\n2. Leveraging distribution-level alignment, the method naturally supports unpaired data and multi-caption supervision, offering greater flexibility and scalability for real-world applications.\n3. The use of lightweight adapters (e.g., Adapter, LoRA) enables efficient alignment with CLIP and LLMs, significantly reducing training cost and making the method practical for resource-constrained settings.", "weaknesses": "1. Although the method is well-motivated, the theoretical analysis of convergence, generalization, or stability of CS-Aligner remains shallow and largely empirical.\n2. The KDE-based CS divergence may introduce additional computational overhead and sensitivity to kernel bandwidth selection, which is not systematically analyzed.\n3. Some comparisons (e.g., Eclipse, LLM2CLIP) reuse decoders from other works, which may not fully isolate alignment quality from downstream generative components.\n4. The benefits of using unpaired data or token alignment are demonstrated qualitatively, but quantitative ablations are limited and lack statistical rigor.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lUhg4B1XWx", "reviewer": "Reviewer_mzxf", "summary": "This paper presents CS-Aligner, a novel framework for vision-language alignment that combines Cauchy–Schwarz (CS) divergence with mutual information (InfoNCE). The central idea is to extend alignment beyond individual pairs by introducing distributional alignment, which effectively resolves the inherent tension between the alignment and uniformity objectives in InfoNCE. While InfoNCE encourages matched pairs to come closer (alignment) and all samples to spread apart (uniformity), this dual objective often leads to suboptimal multimodal representation. CS-Aligner addresses this issue by aligning both the pairwise semantics and the global distributions of visual and textual modalities, resulting in more coherent and consistent cross-modal embeddings.", "questions": "1. It would be better to include more multimodal tasks to verify the effectiveness, as the proposed method is designed for multimodal alignment.\n\n2. Can CS-Aligner generalize to other modality pairs (e.g., audio–text)?\n\n3. Are there scenarios where distributional alignment might unintentionally blur modality-specific distinctions？", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1.  Clearly identifies InfoNCE’s limitations—its conflation of alignment and uniformity objectives and dependence on strong negatives that weaken cross-modal consistency.\n\n  2.  Introduces CS divergence as a novel, theoretically grounded approach for distributional alignment, effectively addressing the shortcomings of sample-level contrastive methods.", "weaknesses": "1.  The experiments are mainly focused on t2i and i2t tasks, which are not sufficient to verify the effectivess. \n\n  2. The ablation studies in Appendix H.1 are insufficient and unconvincing. The experimental settings are not clearly defined. Furthermore, the paper fails to adequately explain the core concepts of InfoNCE and CS-Aligner, and does not justify the significant performance degradation observed when using only InfoNCE. While these experiments are in the appendix, it is essential to treat them with the same seriousness as those in the main text.\n\n  3. The performance of the CS divergence is sensitive to the choice of kernel and bandwidth, but the paper does not provide a robustness analysis to assess this sensitivity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Sa5W99bUwN", "reviewer": "Reviewer_qrYc", "summary": "This paper argues that the widely used InfoNCE in CLIP learning ignores distributional differences and has inherent conflict in terms of alignment and uniformity, which result in suboptimal alignment and modality gap. To tackle these limitations, the paper proposes CS-Aligner, that integrates Cauchy-Schwarz divergence with InfoNCE, leverages unpaired data and incorporates distributional token-wise alignment. Experiments on text-to-image generation and cross-modality retrieval tasks demonstrate the effectiveness of CS-Aligner", "questions": "See Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper proposes a novel framework, CS-Aligner, to reduce modality gap for better vision-language alignment.\n2. The idea of using Cauchy-Schwarz divergence is ingenious, which can simultaneously tackle the two limitations: ignoring of distributional differences and alignment/uniformity confliction. Authors also provide comprehensive theoretical analysis to support it.\n3. Experiments on several multi-modal/cross-modal tasks show the effectiveness of CS-Aligner.", "weaknesses": "1. The Cauchy-Schwarz divergence constitutes the core contribution of this work. It is therefore essential to thoroughly compare it against other distribution distance measures—such as KL/JS divergence and the Wasserstein distance. While the authors discuss their differences in the Appendix, they provide no quantitative comparisons.\n\n2. A main ablation table should be included to quantitatively evaluate the effectiveness of each component in CS-Aligner. Moreover, the ablation study should extend beyond text-to-image generation to include more tasks, which would help comprehensively assess the impact of individual components.\n\n3. The authors should clarify how the visualizations in Figure 1 were derived, particularly regarding the dataset and experimental settings used. For qualitative analysis, it would be beneficial to also present results on other tasks—such as cross-modal retrieval and image captioning—to better demonstrate the method's general applicability.\n\n4. Regarding Table 7, why larger lambda leads to worse results? How about the sensitivity of lambda and sigma on other tasks?\n\nMinor issue: In Line 431, cross-model -> cross-modal", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)