Unified Vision–Language Modeling via Concept Space Alignment
Authors: Yifu QIU
Paul-Ambroise Duquenne
Holger Schwenk
Venue: ICLR 2026 Poster (2026)
Keywords: multimodal embedding space
multilingual embedding space
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: We introduce V-SONAR, a vision–language embedding space extended from the
text-only embedding space SONAR (Omnilingual Embeddings Team et al., 2026),
which supports 1500 text languages and 177 speech languages. To construct
V-SONAR, we propose a post-hoc alignment pipeline that maps the representations
of an existing vision encoder into the SONAR space. We thoroughly evaluate
V-SONAR and show that its embeddings achieve competitive performance on
text-to-video retrieval. Equipped with the OMNISONAR text decoder, V-SONAR
further surpasses state-of-the-art vision–language models on video captioning tasks,
including DREAM-1K (BLEU 23.9 vs. 19.6) and PE-VIDEO (BLEU 39.0 vs. 30.0).

Leveraging V-SONAR, we first demonstrate that the Large Concept Model (LCM;
LCM team et al. 2024) operating in SONAR and trained with English text only, can
perform both single- and multi-visual concept understanding in a zero-shot manner.
Finally, we introduce V-LCM, which extends the LCM with vision–language
instruction tuning. V-LCM encodes vision and language inputs into an unified
sequence of latent embeddings via V-SONAR and SONAR, and it is trained with
the same latent diffusion objective for next-embedding prediction as in LCM’s
text-only pre-training. Experiments on a large-scale multilingual and -modal
instruction–tuning data mixture highlight the potential of V-LCM: V-LCM matches
state-of-the-art vision-language models on tasks covering image/video captioning
and question answering, while significantly outperforming them across 61 rich- to
low-resource languages out of all 62 tested languages.

Published as a conference paper at ICLR 2026
UNIFIED VISION-LANGUAGE MODELING VIA CONCEPT SPACE ALIGNMENT
Yifu Qiu
Paul-Ambroise Duquenne FAIR at Meta
Holger Schwenk
University of Edinburgh yi fu. qiuu ed ac.uk
FAIR at Meta schwen @met com
padqnêmet.a. com
ABSTRACT
We introduce V-SONAR, vision-language embedding space extended from the text-only embedding space SONAR (Omnilingual Embeddings Team et al. 2026), which supports 1500 text languages and 177 speech languages. To construct V-SONAR, we propose post-hoc alignment pipeline that maps the representations of an existing vision encoder into the SONAR space. We thoroughly evaluate V-SONAR and show that its embeddings achieve competitive performance on text-to-video retrieval. Equipped with the OMNISONAR text decoder, V-SONAR further surpasses state-of-the-art vision-language models on video captioning tasks, including DREAM-1K (BLEU 23.9 vs. 19.6) and PE- VIDEO (BLEU 39.0 vs. 30.0).
Leveraging V-SONAR, we first demonstrate that the Large Concept Model (LCM; LCM team al. 2024) operating SONAR and trained with English text only, can perform both single- and multi-visual concept understanding in a zero-shot manner. Finally, we introduce v-LCM, which extends the LCM with vision- languale instruction tuning. V-LCM encodes vision and language inputs into an unified sequence latent embeddings via V-SONAR and SONAR, and it is trained with the same latent diffusion objective for next-embedding prediction as in LCM's text-only pre-training. Experiments on a large-scale multilingual and -modal instruction-tur tuning data mixture highlight the potential of V-LCM: v-LCM matches waae-of-the-as vision-language models on tasks covering image/video captioning and question answering, while significantly outperforming them across 61 rich- low-resource languages out all 62 tested languages.
INTRODUCTION
Language- and modality embedding spaces have emerged as powerful paradigm for multilingual and -modal representation learning (Artetxe & Schwenk 2019 Feng et al.) 20200] Ni al. 202T] Duquenne al. 2023 Wang et al. 2024b) Chezn
2024a). Such spaces have achieved state-of-the-ar performance across a wide range of applications, e.g., bitext mining (Schwenk et al. 2019; NLLB Team et al., 2022, Ramesh al., 2022), and speech-text mining (Duquenne al.
|
2021b). Beyond these, embedding spaces with the encoder-decodes architecture such as SONAR (Duquenne et al.) 2023) have further enabled generative modeling directly in the latent embedding space. The Large Concept Model (LCM; LCM team et al. 2024) extends this direction by showing that diffusion-based language modeling can operate directly in the language-agnostic latent space, i.e., over continuous embeddings rather than discrete tokens. Despite these advances, existing embedding spaces remain restricted to text and speech, limiting their potential for vision -ision-language tasks.
In this work, we introduce V-SONAR, which extends OMNISONAR (Omnilingual Embeddings Team ct al. 2026) to the image and video modality. To the best of our knowledge, this makes OMNISONAR the most universal embedding space covering four modalitie and up to 15000 languages. We use teacher-student training (Reimers & Gurevych] 22000] Duquenne et al. 2021a Heffernan et al. 2022) align the representations of state-of-the-art vision encoder, PERCEPTION ENCODER (Bolya et 2025 with SONAR's 's semantic space in post-hoc manner. The alignment follows a coarse-to-fine
* "This work was done during an internship of Yifu Qiu at FATR at Meta.
'OMNISONAR supports text in 1.5k languages, speech in 177 languages and the added image and video modalities.

---

Published as a conference paper at ICLR 2026
curriculum, over three stages of vision captioning data: (1) large-scale for coarse grounding. (2) synthetic video-caption pairs (2M) for temporal adaptation, and (3) high- quality human-annotatee video captions (200K) for fine-grained alignment. We evaluate V-SONAR extensively. On zero-shot video retrieval, it achieves Recall@ of 73.03 on PE- VIDEO, largely surpassing Sigg.IP2-g-opt (63.91). On zero-shot video captioning, it outperforms state-of-the-ar vision-language models, improving BLEU by +18, +4.3 on PE-VIDEO, DREAM- IK, respectively, over the Perception Language Model (Cho al. 2025
pairs (12M)
By aligning V-SONAR to SONAR, we show that the latent diffusion language model operating in SONAR, LCM (LCM team et al.) 2024) trained with English textual corpus, can zero-shot process the visual embeddings encoded by V-SONAR. In the single-concept understanding task, i.c., video captioning, I.CM only lags behind the existing VLMS with limited margins across PE- VIDEO, DREAM-IK, K, and VATEX. Similarly, LCCM remains competitive for multi-concept reasoning task, i.e., long video summarization as evaluated on VIDROXUM.
From the view of vision-language modeling, LCM serves as new paradigm which unifies vision and language modality to the modality-agnostic latent space shared by SONAR and V-SONAR, and directly predict the next embedding with the latent diffusion objective. Therefore, we further introduce lsiion-lagguag instruction fine-tuned LCM as an exploration to maximize its utility in various downstream vision-language tasks, named v-LCM. v-LCM encodes multimodal data (images, videos, and text) with V-SONAR and SONAR, and is trained with the same latent diffusion strategy, following the original two-tower LCM framework (LCM team et al. 2024) in its textual pre-training.
We evaluate v-LCM on the multilingual and -modal instruction-tuniin 2023), which spans 8 task categories, supports both image and video modalities, and covers 80 languages. V-LCM achieves competitive performance with other vision-language models such as InterVLL (Chen al.) 2024b). Qwen-VL (Wang al., 2024c; Bai al. 2025) and Perception LM on image/video captioning, visual question answering, and other generation tasks. Notably, in M3IT's multilingual evaluation, v-LCM outperforms other VLMs in 61 languages out of 62 tested languages. ranging from high-resource to low-resource setting. The contributions of this work are:
dataset, M3IT (Li et al.
We introduce V-SONAR, the first extension of a language- and podallty-agnostic embedding space (SONAR) to image and video, via post-hoc coarse-to-fine alignment strategy.
We demonstrate that V-SONAR achieves state-of-the- the-ttt-tt zero-shot performance on video retrieval and captioning. and generalizes robustly to multilingual settings.
We show that the LCM, originally trained on text-only data, can effectively operate on V-SONAR embeddings for zero-shot single- and multi-concept vision understanding tasks.
We extend LCM into a latent diffusion vision-language model (V-LCM) by unifying vision and language in the shared latent space of V-SONAR and SONAR. On M3IT, V-LCM matches state-of-the-arr VLMs in captioning and question answering while outperforming them in 61 non- -on-English languages.
V-SONAR
We begin by introducing V-SONAR, lasion- embedding space constructed by post-hoc aligning a state-of-the-art vision encoder, PERCEPTION ENCODER, with the multilingual textual embedding space SONAR. We select the PERCEPTION ENCODER as the base encoder for two key reasons: (1) it achieves state-of-the the-rrt art performance across both image and video modalities (Bolya et al.) 2025), and (2) has been pre- pre-trained in conjunction with a lightweight text encoder, which facilitates much easier post-hoc alignment with SONAR. This design choice distinguishes PERCEPTION ENCODER from alternative vision encoders such as v-JEPA (Bardes al. 2023 Assran
et al. 2025) and DINO (Oquab et al. 2023) Siméoni et al., 2025). which primarily prioritize visual feature learning without explicit consideration of textual alignment.
The architecture of V-SONAR is illustrated in the left panel of Figure I Given the input image or video, PERCEPTION ENCODER (PE) will first encode each frame separately. Then, we stack lightweight projector top of PE to adapt the encoder's representations into the SONAR space. The projector first injects positional embeddings to the embeddings of all frames, thus encoding temporal order information, followed by a single temporal attention layer that enables frame-level
Architecture
2

---

Published as a conference paper at ICLR 2026
Caption
Reconstruction Loss
v-LCM
10-0
VSONAR
SONAR Decoder
LCM
Connector
Loss
0
Q
Response: pandas.
Percep. Encoder
SONAR Encoder
SONAR
VSONAR
SONAR
Visual Input (image/video)
Caption
Inttactions and anoner
Visud Hyput
arimal Ii
Figure 1: Left: Illustration of V-SONAR. Right: training v-LCM with vision-language instruction tuning.
Finally, an attention layer then aggregates the frame embeddings into a single video-level representation, which serves the final embedding for downstream tasks. See Appendix D for implementation details.
interactions.
Alignment from Vision Language. We use the captioning data for the PERCEPTION ENCODER align with SONAR, with assumption that the visual inputs and caption should have the same semantic meaning, thus the high-level representations should be as close as possible in the latent monolity-agnostic space.
Therefore, given a set of N paired visual inputs and captions D = {VV, Ti) where VÚ is an image video and T3 is its corresponding caption, seek learn mapping such that the visual embedding 2, = [V;) and the textual embedding 2, = ((;)) share the same semantic space, where fe denotes the trainable vision encoder and g the frozen SONAR text encoder. To enforce semantic alignment, we minimize the discrepancy between visual and textual embeddings in the SONAR space using Mean Squared Error (MSE) loss:
Calign N Σ -
Following the teacher-student training (Reimers & Gurevych] 2020 Duquenne et al. 2023). SONAR
frozen and we only update the parameters in the lightweight projector, and the vision encoder. We also experimented with additional contrastive loss (Oord et al. 2018 Radford et al. 2021 but found no significant gains; details and results are in Appendiix B
We design coarse-to-fine fine curriculum to progressively adapt the vision encoder to more complex semantics. The alignment proceeds through three stages. In Stage 1, we initialize alignment using 12M large-scale image-caption pairs from the PLM data pipeline (Cho al. (2025) which consists of Segment-Anything (Kirillov et al. 2023) and OpenImages (Kuznetsova al.) [2020). This stage estab- lishes basic mapping between visual and textual embed- dings. Then, we introduce 2M pairs from PLM's synthetic video captioning data from YouTube B corpus (Cho et al. 2025) This step adapts the vision encoder to temporal dynamics while maintaining semantic consistency with SONAR. Finally, we refine the alignment using 200K high-quality human-checked video-caption pairs sourced from PE-VIDEA (Bolya et al. 2025).
model
XSIM 1.37
XSIM+ 4
SONARI OMNISONAR
15.27
0.65
6.14
Table Similarity search over 200 lan- guages in FLORES.
We use two versions of the SONAR encoder: SONAR1 is the published and open-sourced version (Duquenne et al. 2023). This is the version supported by the LCM. We had early access to an improved version, named OMNISONAR, which was trained more data and adds three stages of contrastive training and self distillation (Omnilingual Embeddings Team et al., (226)). As summarized Table OMNISONAR substantially outperforms SONARI on the proxy metric of multilingual

---

Published as a conference paper at ICLR 2026
Method
R@1
R@5T
R@ 10
MRR
ACT
V. Trace"
V. logdet
T. Trace"
T. logdet"
PE-Vid Vid
SigLIP2- G-OPT
47.55
71.47
79.41
58.47
0.396
0.393
-1.7 10"
0.582
-1.7 x 10
PECoreG
63,91
85,98
91.61
73.77
0,476
0,479
-1.4 x 10"
0,686
-1.4 x 10
Y-SONAR
73.03
89.75
93.81
80.50
0.519
0.700
-1.2 x 10*
2.216
88.0 x 10a
VATEX DREAM
SigLIP2-G-OPE
61.50
83,50
89.10
71.50
0.263
0.401
-1.8 x 10
0.662
-1.8 x 10a
PECoreG
72.10
89.80
93.60
79.90
0.307
0.495
- -1.4 x 10
0.639
-1.4 x 10
V-SONAR
63.30
84.10
89.00
72.46
0.410
0.559
112 10"
2.523
88.5 x 10
SigLIP2-G-OPT
27.52
57.70
70.06
41.27
0.289
0.352
- 1.7 x 10
0.660
-1.7 x 10a
PECoreG
18.90
42.42
54.72
30.42
0.379
0.480
-1.4 10
0.508
-1. x 10
V-SONAR
40.75
68.63
78.88
53.59
0.427
0.558
-1. x 104
1.660
-8.3 x 10
Table 2: Zero-shot Retrieval performance on PE-V VIDEO, DREAM-1K and VATEX. We report the Recall rate at 1/5/10 and MRR scores. We also report the analytical metrics for the embedding space, including 1) trace reflects overall variance, and 2) log determinant (logdet) approximates volume in the space. Best values for each columns bolded.
similarity search. The metric XSIM++ includes hard negatives (Chen et al. 2023). We provide an ablation of the two SONAR versions for vision captioning tasks in Appendix E.
2.1
V-LCM
The Large Concept Model (LCM; LCM team et al. 2024) is a latent diffusion language model operating directly in the SONAR embedding space. It follows an auto-regressive paradigm, predicting the next sentence embedding conditioned on preceding clean embeddings. For the textual modality, all embeddings encoded and decoded by the fixed SONAR encoder and decoder. To model the conditional distribution of the next embedding, LCM employs diffusion objective: given a clean embedding z Rdd the forward process progressively perturbs with Gaussian noise under variance-preserving schedule (Karras et al. 2022):
g( x²) (Ix; a,x", oi I2 = -.." + oec, e~ I.,,
where (as, ot) are determined by a monotonically decreasing log-SNR schedule A = (ag /of). The reverse process parameterized by denoiser po(x,,, conditioned on the context embeddings with Gaussian transitions:
conndtttoon
( I2, c) = (2 (2,,, 1, c). dT
(3)
Training minimizes a reconstruction loss on the original clean embedding:
((0) E,,29, || mo(arr" + oic, t, t.c))l2
(4)
We use the two-tower variant of LCM, which separates the contextualizer (encoding the preceding embeddings) from the denoiser (iteratively reconstructing the next embedding).
From the perspective of vision languaee modeling, LCM represents a paradigm that fuses information from visual and textual modalities within a modality- latent space prior, rather than discrete visual and textual tokens (Chameleon Team 2024). This enables autoregressive generation to be performed entirely in the latent space. Building on this principle, we further introduce V-L.M,, extension of LCM trained through vision-language instruction fine-tuning to enhance its utility across a broad range of downstream vision-language tasks. In V-LCM, visual inputs (images and videos) are encoded into the SONAR latent space using V-SONAR, while textual instructions and prompts are encoded with SONAR. The resulting visual and textual embeddings are concatenated into single sequence, which is then processed under the same latent diffusion framework in LCM's original text-only training, predicting the next embedding in the sequence.
3 EXPERIMENTS
We first verify the effectiveness of aligning the vision encoder to SONAR, by evaluating text-video retrieval and captioning using V-SoNAROmni aligned with the OMNISONAR text encoder, and
4

---

Published as a conference paper at ICLR 2026
Model
BLEU
R-1
R-2
R-L
BS-P
BS-R
BS-F
PE-VIDEO
LnterVL22.5-1
19.4
23.4
27.3
InternVL2-1
24.1
35.8
10.7
25.5
30.8
32.1
31.5
Qwen2- VL-2B-Innttuct
29.9
41.7
18.8
31.2
40.0
Wwen2.5-VL-3BB-Inse
30.0
41.3
16.1
28.9
30.2
38.6
34.4
PLM-1B
21.5
37.6
11.9
26.6
35.8
26.2
31.0
PLM-3B
21.1
37.5
11.7
26.4
36.6
26.1
31.3
V-SONAR w/ OMNISONAR Decoder
39.0
50.1
23.3
38.0
44.4
41.6
43.0
DREAM-1K
Intern VL2.5-1B
15.6
18.6
Intern VL2-1B
14.6
25.0
4.3
17.2
23.8
15.2
19.5
Qwen2- VLL-2B-Instruct
19.7
18.5
12.9
13.9
Swen2.5-VL-3B-IIstruee
16.1
23.9
4.4
15.9
1.6
15.6
8.6
PLM-1B
18.5
27.0
6.4
19.3
14.5
16.8
15.5
PLM-3B
19.6
28.6
6.7
20.4
19.9
18.1
19.0
V-SONAR w/ OMNISONAR Decoder
23.9
32.7
22.7
19.7
21.6
20.7
VATEX
Intern VL2.5-1B
19.2
45.2
40.2
Intern DicerLLL2. VL2-1B
47.8
27.3
6,4
22.4
36.9
50.4
42.4
Qwen2- VL--2B-Instruct
32.1
19.8
16.4
46.3
30.8
Swen2.5-VVL-3B-Isstuue
29.4
18.3
5.1
15.0
12.1
47.1
27.6
PLM-1B
33.4
21.8
5.7
19.1
15.1
48.1
29.6
PLM-3B
34.0
22.1
5.9
19.3
16.9
48.6
30.8
V-SONAR w/ OMNISONAR Decoder
26.7
17.2
5.0
14.8
13.6
42.8
26.5
VATEX-zh
BeeenVL2-1B--Issee
22.3
14.1
11.8
8.7
18.2
12.6
LL2.5 Instructt
33.2
22.5
4.4
18.8
19.1
31.5
24.2
V-SONAR OMNISONAR Decoder
30.6
8.53
26.9
48.5
33.7
Table 3: Video captioning performance across PE-VIDEO, DREAM-IK and VATEX (English and Chinese). Metries include BL.UU, ROUGE (R-1, R-2, R-L.), and BERTScore (BS-P, BS-R, BS-F).
provide several ablations. We then switch to the zero-shot evaluation for LCM, and evaluation of v-LCM M3IT eLi al. 2023) which requires the of V-SONARI, as the LCM had been trained on SONARI.
3.1 CONCEPT SPACE ALIGNMENT USING V-SONAROMNI
Text-video Retrieval We treat V-SONAR a paired vision-text encoder and begin by evaluating
its zero-shot performance on text-to-video retrieval, following the setup in [Bolya et al. (025). We compare V-SONAR against two strong baselines: the sate--of-the-ar SIGLIP2 vision encoder (Tschannen et al. 2025) and the PERCEPTION ENCODER (Bolya et al. 2025), from which V-SONAR derived. Evaluations are conducted on three widely used video captioning benchmarks: VIDEO
(15K pairs of captioning data) (Bolya et al. 2025), VATEX (5K pairs of captioning data) (Wang et al. 2019), and DREAM-15 (IK pairs of captioning data) (Wang al. (024a). following the protocol in Cho et al. 20055). In addition to standard retrieval metrics such as Recall @ @1/ 1/5/10, 10, we introduce three complementary measures to analyse embedding space properties: (1) Alignment Consistency (AC): the rank correlation between vision and text similarity scores, reflecting cross- cross-modal alignment quality. (2) Trace: the trace the covariance matrix of vision and text embeddings, indicating the spread of representations.
(3) Log-determinant (logdet): the logarithm of the determinant of the covariance matrix, interpreted as the volume of the embedding ellipsoid.
Table 2 summarizes the results and embedding space statistics. On the three datasets. PE-V VIDEO, DREAM- k and VATEX, V-SONAR significantly outperforms SigLIP2, achieving improvements of 9.12, 1.8 and 13.23 points in Recall @ respectively, demonstrating the effectiveness of our approach
5

---

Published as a conference paper at ICLR 2026
for retrieval tasks. Compared to the original PERCEPTION ENCODER, V-SONAR significantly improves on VIDEO and VATEX (9.12 and 21.85 score at Recall 1), though it loses 8.8 score at Recall@ in DREAM- IK, indicating that our curriculum alignment strategy preserves strong retrieval capability. These results confirm that a vision encoder can be successfully aligned with a purely text-trained embedding space (SONAR) in post-hoc manner. Finally, our embedding space analysis reveals that V-SONAR maintains a more expanded distribution. Moreover, by freezing the original SONAR space, V-SONAR achieves the largest textual embedding dispersion, evidenced by the highest trace and logdet values among all compared models.
Text-video Captioning Different with the traditional vision encoder, such SigLIP 2 or PERCEP- TION ENCODER, aligning V-SONAR to SONAR embedding space allows us to leverage the SONAR decoder directly verbalize the encoded vector of V-SONAR. Hence, we conduct the zero-shot eval- uation on video captioning for V-SONAR, and compare it with few state-of-the-art vision-language models (VLMs) including LL-27/ (Chen al. 2024bb, Qwen-VL 2/2.5 (Wang al. 2024c Bai et al. 2025)ae.em and Perception Language Models (Cho et al. 2025). We compare with VLMs at the scale between 1B to 3B for a fair comparison, as the SONAR decoder is at 1.5B and V-SONAR is at 1.9B. We evaluate the models with lexical metrics including BLEU and ROUGE scores, and semantic metries including TEERTScore-PrecisionReeaa
following (Zhang al. 2025).
We illustrate the results in Table 3 For detailed captioning benchmarks such PE-VIDEO and DREAM-IK, we observe that V-SONAR paired with the SONAR decoder can achieve a state-of-the-ar performance. In particular, V-SONAR improves the second best model, Swee2.55-L-33B-Img by points in BLEU The only exception is VATEX where the captions are relatively short as one sentence, V-SONAR lags behind Intern however, this is expected SONAR mostly with the detailed caption data. And we observe V-SON AR is still comparable with PL.M and Qwen-VI. series. We VATEX-Chinese validation we mostly compare V-SONAR with Intern as Qwen VL. is reported to leverage VATEX Chinese split during training (Wang et al.) 2022cc Bai et al., 22255). and WLMM-1/3B fail support the fluent generation in Chinese. We find that in VATEX Chinese split, V-SONAR still outperforms the Intern VL series, indicating the advantage in multilingual evaluation.
align V-SONAR with
for the multilingual evaluation, and
V-SONAR vs V-SONAROmni. We present the comparison between SONAR (Duquenne et al. 2023) and OMNISONAR in Filiuure 2 We report both SONAR space's oracle performance (we encode the reference caption with SONAR encoder, and decoded with SONAR decoder). And we report the zero-shot performance with V-SONAR (we encode the video with V-SONAR and decode with SONAR decoder). SONAR oracle serves as an estimation of the upper-bound performance that V-SONAR can achieve for leveraging SONAR decoder.
We observe that both SONAR versions have a strong oracle performance, indicating SONAR'S encoding and decoding from textual space into its representation space quite lossless. Specifically,
in PE- Video,,, VATEX and DREAM- K, OMNISONAS can achieve BLEU scores of 81, 96 and 70. Comparing the zero-shot performance for SONAR and OMNISONAR, we see SONAR is worse by considerable margin. We hypothesize that SONAR is harder to align since its space is reported to be collapsed. Our analysis for SONAR and OMNISONAR also supports this observation: in PVD, SONAR and OMNISONAR have the embeddings norm at 0.264 and 1.69, and covariance trace at 0.049 and 1.83, respectively. The comparison in retrieval tasks are in Appendix
PVD Bench
VATEX
DREAM-IK
100
37
21
2411
BLEU
R-1
8.2
R.L.
B5.F
BLEU
B-1
R-2
R-L
BS-F
BLBU
R-1
R:2
B.L
88-F
SUNAR-2 Orude
SONAI-1 Chacis
SUNAIL22
SUNAR-1
Figure 2: Comparison for y-SONAR trained with SONAR version I and 2 (OMNISONAR) embed- ding space on three captioning datasets.

---

Published as a conference paper at ICLR 2026
MSE
Cos. Sim.
BLEU
R-1
R-2
R-L
BS-P
BS-R
BS
Linear Proj-
x
0.694
38.0
49.7
21.6
36.7
47.2
40.1
43.7
Architectury
Full PE
x.54 10-3
0.672
37.1
48.5
21.3
36.5
46.9
38.8
42.9
+ Asyne. LR
1.43 10-3
0.700
39.7
51.3
23.3
37.7
48.1
42.1
45.1
+ Norm. Init.
1.39 x 10-3
0.708
39.8
51.8
24.0
38.5
49.4
42.2
45.8
Attn. Pooling
1.39x10
0.708
39.8
51.9
24.0
38.5
49.7
42.4
46.0
+ Temporal Attn.
1.39 x 10
0.708
39.8
51.9
24.0
38.5
49.7
42.4
46.1
Pipeline
Full Pipeline
x.36x10-2
0.716
40.1
52.6
24.9
39.2
50.8
43.2
47.0
w/o SV
1.39 10-3
0.710
39.6
51.9
24.1
38.6
50.0
42.4
46.2
w/o IC & SV
1.39 10-3
0.708
39.8
51.9
24.0
38.5
49.7
42.4
46.1
Table 4: Ablation study in model architecture and the three-stage training pipeline. SV: our second stage curriculum with the synthetic video captioning data. IC: our first stage curriculum with image captioning data.
3.2 ABLATION STUDY
We conduct an ablation study for model architecture design, and our proposed training pipeline on the PE-VIDEO test set (Table 4).
Model Architecture We ablate architectural choices for the projector network. As a baseline, we evaluate linear projection (Linear Proj.), where the PERCEPTION ENCODER is frozen and only linear layer is trained, and full-model fine-tuning (Full PE), where the encoder is updated jointly. Linear projection performs better, indicating that the encoder's contrastive pre-training already yields strong semantic alignment, while full fine-tuning is hindered by unstable gradients from the randomly initialized projector. To mitigate this, we adopt strategies that incrementally improve downstream performance, including asynchronous learning rates for the projector and encoder, initialization trick, attention-based aggregation strategy for video frames' features, and temporal attention layer.
Data Mixture We then ablate the second stage synthetic video captioning and first-stage image captioning curriculum our pipeline. We observe that both stages contribute positively to the downstream performance in captioning performance on PE- Video. Removing the 12M image captioning pairs and 2M video captioning pairs reduce the BLEU with 0.3 and 0.2, respectively.
3.3 ZERO-SHOT PROCESSING V-SONAR EMBEDDINGS BY LARGE CONCEPT MODEL
Since the LCC (LCM team et al. 2024) operates directly on SONARI, it should seamlessly transfer its ability and understand the visual concepts V-SONAR aligned with SONARI. We examine V-SONAR with LCM gradually from single to multiple vision concept understanding tasks, where the LCM accepts the instruction encoded by SONAR, with the vision embeddings from V-SONAR, and predicts the target embedding. Note that in both experiments, neither any video data nor captioning data. Thus, the LCM is only trained in English textual corpus including its pre-training and instruction fine-tuning in LCM team al. (2024). We compare LCM's 's performance with VLMs at 7/8-B scale for the Intern VL series (Chen et al. 2024b). Qwen- VL (Bai et al. 2025 Team 2024) and PLM (Cho et 2025)
do fine-tur tune the LCM, with
Single Vision Concept Understanding: Video Captioning We report the results of LCM on video captioning in Table 5) Compared to the strongest baseline, the zero-shot LCM lags behind by 1.15/4.44/4 44/4.76 BLEU scores PE-V PE-VIDEO. DREAM-1K and VATEX, respectively. Among the models, PLM-8B delivers the strongest overall performance. The relatively narrow performance gap between LCM and competitive VLMs suggests that the LCM able understand the single vision concept, despite never being trained with video data.
Multiple Vision Concept Understanding: Long Video Summarization We next evaluate LCM in setting requiring understanding multiple visual embeddings. Long videos uniformly segmented
7

---

Published as a conference paper at ICLR 2026
Video Captioning/ Summarization
M3IT Image
M3IT Video
Arth-NorAA
MSRYTEQQ
PE-Vadoo
DERAM-1K
VIDEOXUM
VIQUAE
VinalMICC
SclenceQA
VADEX
COCO
IVQA
R-L
BS
R-L
BS
R-L
BS
R-L
BS
R-L
R-L
R-L.
Acc.
R-L
R-L
R-L
IB
25.5
31.5
17.2
19.5
22.4
42.4
15.3
17.7
12.6
24.0
30.6
53.9
40.6
27.6
39.5
In2
4B
15.0
18.4
12.2
14.7
19.2
42.4
15.6
17.4
17.5
20.0
35.3
89.6
27.5
24.5
31.8
8B
18.6
23.4
16.4
19.4
16.2
42.4
29.1
26.1
21.0
21.6
42.9
87.2
29.7
27.1
38.4
1B
23.4
29.3
15.6
18.6
19.2
40.3
17.1
23.2
13.2
10.8
27.3
69.0
16.6
11.7
19.3
Intern InternVL-225 VL-2.5
4B
14.6
18.0
13.3
14.8
17.3
36.2
18.1
23.0
15.1
23.1
45.2
86.4
26.8
21.8
24.9
8B
21.4
26.0
17.0
17.0
20.6
42.4
24.9
20.5
16.8
17.3
42.8
93.1
20.9
16.9
22.7
Qwen2-VL
2B
31.2
37,3
18.5
13.9
16.4
30.8
23.6
29.8
24.9
50,2
56.1
54.5
53.7
39.6
49.4
7B
26.9
32.6
19.8
18.1
28.5
51.6
26.0
32.4
23.7
49.7
87.4
70.4
41.9
22.7
39.1
QWen2.5-VL
3B
28.9
34.4
15.9
8.6
15.0
27.6
26.0
32.9
25.1
48.3
55.7
55.0
52.1
41.6
48.5
7B
22.2
25.9
15.7
10.5
27.5
50.8
24.1
28.9
18.5
34.5
45.0
61.6
46.0
41.4
54.2
1B
26.6
31.0
19.3
15.5
19.1
29.6
21.8
33.2
27.5
30.8
45.5
73.6
27.8
14.5
39.1
Percep. LM
3B
26.4
31.3
20.4
19.0
19.3
30.8
27.0
36.4
34.3
23.7
51.1
89.8
28.0
19.4
26.1
27.4
31.9
20.8
19.7
19.0
30.8
26.2
33.7
36.3
31.0
50.0
87.7
40.5
25.3
41.4
LCM
25.5
27.9
18.5
16.6
23.8
30.8
21.5
22.1
18.0
34,3
33.5
44.7
51.7
36.0
48.9
LCM
V-LCM
27.4
30.0
19.8
19.2
28.8
48.7
20.6
25.3
38.8
39.4
34.1
76.2
63.6
48.7
63.9
Table 5: Main results on vision-language tasks in M3IT and the previous video benchmarks (PE- VIDRO, DREAM-1K, VATEX, VideoXum).
into snippets with 8 frames per each, with each snippet encoded by V-SONAR as a separate video embedding. Since the LCC shows strong performance in document summarization (L.CM team al.] 2024) for multiple SONAR embeddings; we hypothesize that it should be capable of performing zero-shot summarization over sets of video embeddings from V-SONAR. For this evaluation, we use VIDEOXUM (Lin et al. 2023). which contains videos of one to five minutes, uniformly split into snippets of 8 frames each.
We report the VIDEOXUM results in Table 5) Again, PLM achieves the strongest performance among competitive VLMs at the same scale. V-SONAR LCM achieves 22.1 score at BertScore-F1, trailing the perfopirg PLM-8B (33.7) but is slightly higher than Intern VL-2.558B at 20.5. These findings indicate that, even without exposure to any video data during training, LCM demonstrate non-trivial understanding of multiple V-SONAR embeddings for long videos.
Reasoning in V-SONAR We then investigate whether LCM truly leverages the latent representations V-SONAR for multimodal reasoning. To test this, com- pare two settings: (1) encoding video clips directly into V-SONAR embeddings, which are then fed into LCM for zero-shot summarization; and (2) decoding video embed- dings into captions using the SONAR decoder, re-encoding them with SONAR, and providing these SONAR embed- dings LCM. Our hypothesis is that V-SONAR embed- dings retain richer visual features than their textual equiv- alents SONAR, and thus should yield stronger perfor- mance if LCM relies on visual representations.
SONAR
VSONAR
ROUGE-L
0.0
Short
Mid
Long
Figure 3: Operating in V-SONAR space. the LCM performs better than only ac- cepting the textual SONAR inputs. We compare LCM-7B-IFT in VIDEOXUM with ROUGE-L scores across short, mid, and long categories of video inputs.
We group videos into short (<90s), mid-length (90-150s), and long (>150s) categories, and report ROUGE-L scores in Figure 3 Across categories, LCM with V-SONAR consistently outperforms it with SONAR. Notably, while SONAR performance declines with increasing video length, v-SONAR remains stable, highlighting its robustness. These results support our hypothesis that the LCM reasons directly in the visual embedding space provided by V-SONAR containing richer visual information than SONAR representations of textual input.
8

---

Published as a conference paper at ICLR 2026
v-LCM Reasoning with Visual Details. To as- sess whether our alignment from PERCEPTION EN- CODER to SONAR preserves the visual details critical for downstream reasoning, we further evaluate vi- sual commonsense reasoning (VCR) performance on M3IT (Table 6p. In this benchmark, models must reason about objects using commonsense knowledge while interpreting their associated bounding boxes. Compared to semantic-level alignment alone, VCR demands substantially richer visual grounding and layout awareness (see Figure 21). We report token- level F1 scores against the ground truth rationales, alongside semantic similarity to the reference expla- nations. The strong performance of v-LCM indi- cates that, despite being trained solely on semantic- level captions, v-LCM effectively leverage layout grounding and spatial relationship preserved by V-SONAR in reasoning tasks.
VCR
F1
Sim.
LCM
0.385
0.258
vLCM
0.671
0.529
PLM-8B
0.441
0.432
Qwen-2.5-7B
0.275
0.402
Qwen-2-7B
0.502
0,513
InterVL2.5-8B
0.155
0.158
InterVL2-8B
0.325
0.340
Table 6: VCR results comparing token-level
F1 and semantic similarity.
3.4 V-LCM
In-task Performance We next evaluate V-LCM, which is supervised fine-tuned on M3IT (Li et al.
2023) to better capture and reason over visual concepts. We mainly rely on M3IT (Li al. (023)
as it supports variety of tasks, and the wide coverage for up 80 languages ranging from high-
to low- low-resource languages. The evaluation covers 7 datasets spanning 5 tasks defined in M3IT: (1)
image captioning (COCO), (2) visual QA (VIQUAE), (3) document image QA (VisualMRC), (4)
video captioning (MSRVTT), and (5) question answering (IVQA, MSRVTT-QA. ActivityNerQAA. In addition, we report V-LCM's performance on the video captioning and long video summarization benchmarks introduced in the previous section. Following the evaluation protocol in Li et al. (2023).
we use ROUGE-L for generative tasks (e.g., captioning and open-ended QA) and accuracy for multiple-choice QA.
Table 3ccompaees LCM with strong open- source vision- language models. We observe that v-LCM substantially outperforms the zero-shot LCM across most benchmarks. For example, v-LCM achieves 63.9 R-L on IVQA and 63.6 R-L on ActivityNe:QA, surpassing 48.9 and 51.7 for L.MM, representing clear gains from training with vision cnrruccioo-tuningg data. While V-L.CM lags behind the test-performin models on some benchmarks such as VisualMRC, VIQUAE, and ScienceQA, achieves state-of-the-art results video question answering tasks, including IVQA, ActivityNeIQA, and MSRVTT-QA. Meanwhile, performance on our previous video captioning and summarization datasets remains competitive: v-LCM attains 27.4 R-L on PE- Video and 19.8 R-L on DREAM-IK. IK trailing the best model by only 1.5 and 1 ROUGE scores, highlighting the generalization of LCM to unseen datasets during training.
Multilinguality We further conduct multilingual evaluation of V-LCM on the M3IT benchmark across 62 languages leveraggin the fact that v-LCM operates entirely in the latent space of SONAR and V-SONAR, and therefore decode outputs to any languages supported by SONAR. Evalua- tion spans five various tasks including image classification (ImageNet), image question answering (VQA-V2, OKVQA), video question answering (MSRVTT-QA). video captioning (MSRVTT) and narrative generation (VIST), covering a spectrum from high-resource languages (e.g., Chinese), mid-resource languages (c.g., Japanese) to low-resource languages (c.g., Javanese). We use the ROUGE-L implementation from (Shohan et al., 2024) for the multilingual evaluation.
As shown in Figure 4, V-LCM consistently outperforms Qwen2.5- and PLM-8B across 61 of 62 languages, with Dutch being the only exception. While improvements in some high- languages are modest (e.g. French), the gains become substantial in mid- and low-resource settings, including Burmese, Tajik and Telugu. Notably, for languages such as Urdu, modern Arabic and Tamil, which is unsupported by PL.M-8B based on L.aMMA3322 (Touvron et al. [2033] Dubey et al.. 2024). v-LCM successfully generates meaningful outputs, whereas competing models fail entirely.
"The intersection of all languages supported by SONAR, M3TT, and multilingual ROUGE.

---

Published as a conference paper at ICLR 2026
Owenz 5-VL78
PLN-B0
033
025
Meerics
Tha
rettapuise
Vidtnamez Spanish
Indonesion
Cataliin
Cujarai
Hindi
Groak
Cebunno
Karmada
Easiar Purjabi
Chinsse (Sing.
Hungaran
lamal
Tagalig
Buigarion
Turken
Hebrew
teligu
Cermann
Romanian
Korean
French
Uksimbnn
Danish
Bengali
Arrkst re
Calicinn
Mac Macadonian
Busnian
Italian
Slovak
Waltess
javaneeee
Russiim
Swedish
Polish
Dusch
NSS Are bic
Qraat ia
Mara th
ocotain
Armonian
serbisn
Tajk
Weish
Kasakn
Standard lorgyz
Estonian
Icelande di
Asturian
Evergian
Janaress
Fireish
Lamebourggaa
Burm
Figure 4: Performance in 62 languages for V-LCM, Qwen2.5-VL-7B and PLM-8B on M3IT testing set for MSRVTT, MSRVTT-QA, ImageNet, VQA-V2, VIST and OKVQA. We report the ROUGE-L
scores averaged from all datasets. Detailed results for each dataset can be found in the Appendix F
RELATED WORKS
A central paradigm in multimodal learning is to align vision and language representations into shared embedding space. Early approaches such as CLIP (Radford et al. 202I) and ALIGN (Jia al. 2021) established large-scale contrastive learning between paired images and captions, enabling zero-shot transfer to downstream tasks. Subsequent works extended this idea to video-language pretraining (Lei et 2021 Xu al., 2021 Wang et al.) (022).. More recent efforts focus on aligning pretrained encoders into an unified space: Perception Encoder (Bolya et al. 2025) projects diverse perceptual modalities into a shared latent space, while scaling data and architectures in models, such as
Florence (Yuan et al. (022]) and SigLIP2 (Tschannen et al. (025). further improve alignment quality. Recent work also shows that using large language models text encoders enhances vision-languaae
alignment (Stone et al.) 2025), and post-hoc alignment strategies have been proposed as lightweight
alternatives to joint training (Brokowski et al. 2025 Yang et al. (025).
Parallel advances in multilingual text embedding models, such as LASER (Artetxe & Schwenk 2019 Heffernan et al. 2022). LABSE (Feng et al., 2020), and SONAR (Duquenne et al. 2023
Omnilingul Embeddings Team et al. 2026), demonstrate the effectiveness of language-agnostic
embedding spaces across hundreds of languages. Modular approaches have further explored language- specialized components reduce interference in universal embedding spaces (Huang et al. 2024). These universal text embeddings provide an attractive target for aligning vision encoders, as they inherit cross-lingual generalization without requiring multimodal data in every language. Prior work has explored similar strategies in speech-to-tex to-text alignment (Chung et al.) 2018 Duquenne et al., 2021a Laperrière al. 2024 Du et 2024), but large-scale alignment visual embeddings into such universal text spaces remains underexplored.
5 CONCLUSION
We introduce V-SONAR by extending the SONAR embedding space with the image and video modality. To the best of our knowledge, this makes SONAR the most universal embedding space covering four modalities (text, speech, image and video) and up to 200 languages. We propose three-stage training approach to map a pooled representation based on the PERCEPTION ENCODER to the semantic SONAR representation. We achieve very competitive results for text-to- video retrieval and video captioning. The Large Concept Model (LCM; LCM team et al. 2024) is a recent approach to perform reasoning at higher semantic conceptual level, namely SONAR. Encoded by V-SONAR, we show that the LCM can zero-shot process image or video embeddings without the need of training data in these modalities. We further introduce V-LCM with the multimodal instruction fine-tuning which matches of VLMs, while significantly outperforming them across 61 rich- to low-resource languages.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Mikel Artetxe and Holger Schwenk. Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond. TACL, pp. 597-610, 2019.
Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, et al. V-jepa 2: Self-supervised video models enable understanding, prediction and planning. arXiv preprint (rxiv:2500600983 2025.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025.
Adrien Bardes, Quentin Garrido, Jean Ponce, Xinlei Chen, Michael Rabbat, Yann LeCun, Mido Assran, and Nicolas Ballas. V-jepa: Latent video prediction for visual representation learning. submitted to ICLR'23 25, 2023.
Daniel Bolya, Po- Yao Huang. Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Rasheed, et al. Perception encoder: The best visual embeddings are not at the output of the network. arXiv preprint arXiv:2504. 13181,, 2025.
Trevor Brokowski, Alexandre Sallinen, and Mary-Anne Hartley. Ssca: Siglip-2 sonar concept alignment. In Second Workshop on Visual Concepts, 2025.
Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint
w:Xiv:2405.09818, 2024.
Jianly Chen, Shitao Xiao, Peitian Zhang, KunLuo, Defu Lian, and Zheng Liu. M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation, 2024a.
Mingda Chen, Kevin Heffernan, Onur Çelebi, Alexandre Mourachko, and Holger Schwenk. XSIM++: An improved proxy bitext mining performance for low-resource languages. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), ACL, pp. 101-109, 2023. URL https://
aclanthology aah...a.........................
Zhe Chen, Jiannan Wu, Wenhai Wang. Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 24185-24198, 2024b.
Jang Hyun Cho, Andrea Madotto, Effrosyni Mavroudi, Triantafyllos Afouras, Tushar Nagarajan, Muhammad Maaz, Yale Song. Tengyu Ma, Shuming Hu, Suyog Jain, al. Perceptionlm: Open- access data and models for detailed visual understanding. arXiv preprint arXiv:2504.13188 2025.
Yu-An Chung, Wei-Hung Weng, Schrasing Tong, and James Glass. Unsupervised cross-modal alignment of speech and text embedding spaces. Advances in neural information processing systems, 31, 2018.
Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, et al. Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. arXiv preprint arXiv:2407.05400 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv e-prints, pp. arXiv-2407, 2024.
Paul-Ambroise Duquenne, Hongyu Gong, and Holger Schwenk. Multimodal and mul- tilingual embeddings for large-scale speech mining. In M. Ranzato, A. Beygelz- imer, Y. Dauphin, P.S. Liang, and Wortman Vaughan (eds.), Advances in Neural In- formation Processing Systems, volume 34, pp- 15761. Curran Associates, Inc., 2021a. URL [attpp://procceednggg.... neur cc/ fwwes/ filess/ppper/222//
fi Fo

---

Published as a conference paper at ICLR 2026
Paul-Ambroise Duquenne, Hongyu Gong, and Holger Schwenk. Multimodal and mul- tilingual embeddings for large-scale speech mining. In M. Ranzato, A. Beygelz- imer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural In- formation Processing Systems, volume 34, pp- 15748-15761. Curran Associates, Inc., 2021b. URL htttpps:/pprceeednnggg neur neurips. cc/paper. sereerrffeessseee fww.ss/ppppee/2211111 File/ (ll1://86f3cssaaee11111133333213
Paul-Ambroise Duquenne, Holger Schwenk, and Benoît Sagot. Sonar: sentence-level multimodal and language-agnostic representations. arXiv preprint arXiv:2308. /466, 2023.
Fangxiaoyu Feng, Yinfei Yang. Daniel Cer, Naveen Arivazhagan, and Wei Wang. Language-agnostic bert sentence embedding. arXiv preprint arXiv:2007.01852, 2020.
Kevin Hefferman, Onur Çelebi, and Holger Schwenk. Bitext mining using distilled sentence represen- tations for low-resource languages. arXiv preprint arXiv:2205.12655 2022.
Yongxin Huang, Kexin Wang, Goran Glavaš, and Iryna Gurevych. Modular sentence encoders: Separating language specialization from cross-lingual alignment. arXiv preprint arXiv:2407.14877 14878, 2024.
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig, Scaling up visual and vision- lsiounlanguage representation learning with noisy text supervision. In International conference on machine learning, pp. 4904-996. PMLR, 2021.
Tero Karras, Miia Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in neural information processing systems, 15:26565-266577 2022.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Yen Lo, al. Segment anything. In Proceedings of the JEEE/CVF international conference on computer vision, pp. 4015 4015-4026, 2023.
Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al. The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. International journal of computer vision, 128(7):1956-19)) 2020.
Gaëlle Laperrière, Sahar Ghannay, Bassam Jabaian, and Yannick Estève. A dual task learning approach to fine-tune multilingual semantic speech encoder for spoken language understanding. arXiv preprint arXiv:2406. 121441 2024.
LCM team, Loic The Barrault, Paul-Ambroise Duquenne, Maha Elbayad, Artyom Kozhevnikov, Belen Alastruey. Pierre Andrews, Mariano Coria, Guillaume Couairon, Marta R Costa-jussà, David Dale, al. Large concept models: Language modeling in sentence representation space, arXiv preprint arXiv:24/2.088 0882/, 2024.
Jie Lei, Tamara L. Berg, and Mohit Bansal. Less is more: Clipbert for video-and -in---and--aaggeee learning via sparse sampling. In CVPR, 2021.
Lei Li, Yuwei Yin, Shicheng Li, Liang Chen, Peiyi Wang. Shuhuai Ren, Mukai Li, Yazheng Yang. Jingjing Xu, Xu Sun, et al. M3it: large-scale dataset towards multi-modal multilingual instruction tuning, arXiv preprint arXiv:2306.04433 2023.
Jingyang Lin, Hang Hua, Ming Chen, Yikang Li, Jenhao Hsiao, Chiuman Ho, and Jiebo Luo. Videoxum: Cross-modal visual and textural summarization of videos. IEEE Transactions Multimedia, 26:5548-5566 2023.
Jianmo Ni, Gustavo Hernández Ábrego, Noah Constant, Ji Ma, Keith B. Hall, Daniel Cer, and Yinfei Yang. Sentence-t5: Scalable sentence encoders from pre-trained trained text-to-text models, 2021. URL
08877
12

---

Published as a conference paper at ICLR 2026
NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi. Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang. Guillaume Wenzek, Al Youngblood. Bapi Akula, Loic Barrault, Gabriel Mejia- CogjiaGGnzzlez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Saliyyah Salcem, Holger Schwenk, and Jeff Wang. No language left behind: Scaling human-centered machine translation, 2022. URL https: arwiv. org/ abs/ / 2207 04672
Omnilingual Embeddings Team, Pere-Lluís Huguet Cabot, João Maria Janeiro, Yen Meng, loannis Tsiamast, Vivek Iyer, Jaehyeong Jo, Guillem Ramírez, Belen Alastruey, Loic Barrault, David Dale, Kevin Heffernan, Artyom Kozhevnikov, Alex Mourachko, Christophe Ropers, Holger Schwenk, Yu-An Chung, Marta R. Costa-Jussa, and Paul-Ambroise Duquenne. SONAR2: Omnilingual sentence embeddings for text and speech, 2026. to be published.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint /
2018.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Feraande,, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PmLR, 2021.
Gowtham Ramesh, Sumanth Doddapaneni, Aravinth Bheemaraj, Mayank Jobanputra, Raghavan AK, Ajitesh Sharma, Sujit Sahoo, Harshita Diddee, Mahalakshmi Divyanshu Kakwani, Navneet Kumar, Aswin Pradeep, Srihari Nagaraj, Kumar Deepak, Vivek Raghavan, Anoop Kunchukuttan, Pratyush Kumar, and Mitesh Shantadevi Khapra. Samanantar: The largest publicly available parallel corpora collection for 11 Indic languages. TACL, 10:145-162, 2022. URL https
7/aclantho logy. org/ tacl-1.97
Nils Reimers and Iryna Gurevych. Making monolingual sentence embeddings multilingual using knowledge distillation. In EMNLP, pp. 4512-4525, 2020.
Holger Schwenk, Guillaume Wenzek, Sergey Edunov, Edouard Grave, Armand Joulin, and Angela Fan. CCMatrix: Mining billions of high-quality parallel sentences on the web. In https:
//a 7/arxiv. www.v/s/rg//b///o 911. 04944 2019.
Faisal Tareque Shohan, Mir Tafseer Nayeem, Samsul Islam, Abu Ubaida Akash, and Shafiq Joty. XI-headtags: Leveraging multimodal retrieval augmentation for the multilingual generation of news headlines and tags. In ACL (Findings), 2024.
Oriane Siméoni, Huy V Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Mare Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, et al. Dinov3. arXiv preprint arXiv:2508. 1000:22000800000.,...0 2025.
Austin Stone, Hagen Soltau, Robert Geirhos, Xi Yi, Ye Xia, Bingyi Cao, Kaifeng Chen, Abhijit Ogale, and Jonathon Shlens. Learning visual composition through improved semantic guidance. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3740-3750 2025.
Qwen Team. Qwen2 technical report. arXiv preprint arXiv:2407.1067 106771 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971. 2023.
Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdul- mohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv preprint 14788 2025.
13

---

Published as a conference paper at ICLR 2026
Jiawei Wang, Liping Yuan, Yuchen Zhang, and Haomiao Sun. Tarsier: Recipes for training and evaluating large video description models. arXiv preprint arXiv:2407.00634 2024a.
Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Multilin- gual e5 text embeddings: A technical report, 2024b. URL.tttps.//rx....rrr/abbb/ 05672
Minghang Wang, Zeqiu Ma, Yuanjun Xie, Xuewen Li, et al. All-in-one: Exploring unified video- language pre-training CVPR, 2022.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing lision-language model's 's perception of the world at any resolution. arXiv preprint arXiv:2409. 12/91, 2024c.
Xin Wang, Jiawei Wu, Junkun Chen, Lei Li, Yuan-Fang Wang, and William Yang Wang. Vatex: A large-scale, high-quality multilingual dataset for video-and-language research. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4581 4581-4591, 2019.
Hu Xu, Gargi Ghosh, Po- Po-Yao Huang, Dahuang Zhang, Joshua Ainslie, et al. Videoclip: Contrastive pre-training for zero-shot video-text understanding. EMNLP, 2021.
Jingfeng Yang, Ziyang Wu, Yue Zhao, and Yi Ma. Language-image alignment with fixed text encoders. arXiv preprint (eXiv:5506.040099 2025.
Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, et al. Florence: A new foundation model for computer vision. In CVPR, 2021.
Chunhui Zhang. Yiren Jian, Zhongyu Ouyang, and Soroush Vosoughi. Pretrained image-text models are secretly video captioners. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pp. 292-305, 2025.
14

---

Published as a conference paper at ICLR 2026
A LLMS USAGE DECLARATION
We declare that the large language model (LLM) was only used to assist in minor tasks, including revising the manuscript for grammatical correctness, improving phrasing, and performing small technical implementations such as debugging code snippets. All scientific ideas, results, analyses, and conclusions presented in this paper are entirely the work of the authors.
B CONTRASTIVE LOss FOR ALIGNING PERCEPTION ENCODER AND SONAR
We have also explored the use of a contrastive loss in addition the MSE loss for aligning the PER- CEPTION ENCODER to SONAR. Specifically, given mini batch of B paired samples {(V, j we aim not only minimize the distance between matched pairs (Vo( ((i)) but also push apart mismatched pairs. We define the contrastive loss as:
1
B
exp (sim( (VV)
Lcon
Σ 1=1
log
(5)
B
Σ
1==1 exp (sim (f (V)), (sim(f(()) ((T)))/T)
where sim(..) -) denotes cosine similarity and is temperature parameter. The final loss is then weighted combination of the MSE alignment loss and the contrastive loss:
Calign + X
(6)
where Ả controls the strength of the contrastive term. However, in our preliminary experiments, adding the contrastive component did not yield a significant improvement over the MSE-only objective (Table 7) in captioning performance, while it leads to gains in retrieval performance. However, since our downstream usage of V-SONAR for v-LCM is closer to generation task, we choose the MSE-only loss the final loss.
Captioning
Retrieval
BL.FU
R-L.
BS-F1
R@1
MRR
MSE-only
38.9
37.8
44.9
49.0
60.3
MSE + Contrastive
38.6
37.5
44.5
52.4
63.7
Table 7: Ablation study using only MSE loss vs. adding contrastive loss. Results reported the PE-Vided benchmark for captioning and retrieval with single MLP layer as the connector in V-SONAR.
To further explain the trade-off between retrieval and captioning performance in Table 7
we hypothe- ses that contrastive training can move vision vectors off the exact SONAR manifold. only enforces relative ordering via cosine margins, and produces embeddings whose norms or local covariance differ from those the SONAR decoder were trained on. That small manifold shift can degrade generative reconstruction even if retrieval improves.
We analyze the statistics of V-SONAR embeddings Table 8 It a clear observation that V-SONAR trained with the contrastive loss has a more separate embeddings, evidenced by its higher norm, trace and volume. shows that contrastive loss pushes the vSONAR embeddings be a more expanded distribution compared to the MSE-only (higher values in Norm, Covariance trace, and Volume). However, their poor alignment consistencies both MSE and cosine similarity, where we calculate the correlation between vision and text similarity ranking for each sample, suggests contrastive loss breaks the local covariance structure and may break the alignment with SONAR manifold.
C DATASET STATISTICS
Table summarizes the datasets used in our three-stage training pipeline for alignment. The PLM- Image datasets (SAIB and OpenImages) provide large-scale image caption pairs, which particu- larly valuable for improving grounding and linguistic richness. The PLM Video-Auto- YTIB dataset contributes video-text pairs with an average duration of 22.75 seconds, enabling the model to capture
15

---

Published as a conference paper at ICLR 2026
V. Norm
Trace
V. Volume
AC (Cusine)
AC (MSE)
MSE
1.22
0.48
-10007.16
16
0.41
0.32
MSE+Contrastive
1.30
1.74
-8107.94
0.31
0.13
Table 8: Additional statistics for V-SONAR representations on PE-Video. We report the norm (V.Norm), trace (V.Trace), Volume (V. Volume,, alignment consistency measured in Cosine Similarity (AC Cosine) and MSE (AC MSE) for video embeddings predicted by V-SONAR.
temporal dynamics in multimodal content. Finally, the PE-Video dataset provides carefully curated human-annotated video-caption pairs with moderate length, serving as a higher-quality supervision signal in later stages. Together, these datasets balance scale and quality, ensuring both broad coverage and precise alignment across modalities.
Dataset
#Samples
Duration (s)
Caption Length (sent.)
Caption Length (words)
PLM- PLM-Image-A Image-Auto-SAll
7.99M
10.7
181.8
PI.MIImaee -ma---maa--------- Autt---pnnnnages
1.37M
7.9
132.1
PLM- PLM-Video-Aus Video-Auto-YTII
2.14M
22.8
2.3
95.5
PE- Video
118K
16.7
4.4
51.4
Table 9: Statistics of the datasets used in 3-stage training for alignment. We report the number of samples, average video duration (if applicable), and average caption length in sentences and words.
D IMPLEMENTATIONS
V-SONAR Architecture We build our model on top of the Perception Encoder Vision Transformer The encoder processes RGB images at resolution of 448x448 pixels, splitting them into 14x14 patches and yielding 1024 patches per frame. The vision tower consists of 50 transformer layers, each with hidden width of 1024, 16 attention heads, and 4096-dimensional feed-forward network, resulting in approximately 1.9B parameters. For video inputs, uniformly sample 8 frames and extract frame-level embeddings of 1536 dimensions from the encoder, which are subsequently projected into a 1024-dimensional SONAR embedding space. To bridge perception features with the target space, we attach lightweight connector, where weights are optionally initialized from a Gaussian distribution (μ 0, = le - 5) with zero biases for stability. To capture temporal dynamics, the connector augments encoder outputs with sinusoidal positional encodings and applies temporal multi-head self-attention module (8 heads, dropout 0.1) across frames, combined with residual connections. The resulting sequence is aggregated using attention-basse pooling, where a learnable CLS token attends over the frame embeddings via an 8-head attention module, though we also evaluate mean and max pooling variants. The final pooled representation (1536 dimensions) is then mapped to the (024-dimensional SONAR space by linear MLP layer.
backbone PE- Co---oorr-------
V-SONAR Training Details To stabilize training, the projector is initialized from a zero-mean Gaussian distribution with a small variance (1e-5), which mitigates gradient explosion when mapping from the digh-dimensional PERCEPTION ENCODER features to the target embedding space. We employ two-phase training recipe: in the first 2,000 steps, PERCEPTION ENCODER frozen while only the projector is optimized, allowing the projector to adapt without perturbing the pre-trained encoder. Subsequently, both the projector and PERCEPTION ENCODER are jointly optimized, using asynchronous learning rates: a higher rate (Le-4) for the projector to enable rapid adaptation, and lower rate (1e-5) for PERCEPTION ENCODER to preserve pre-trained knowledge.
We train the model using three-stage curriculum. Stage 1 (image captioning) runs for 15 epochs with batch size of 512, base learning rate of 1 and a connector learning rate of 10 with 4000 warmup steps applied to the connector. Stage 2 (synthetic video captioning data) runs for 10 epochs with an effective batch size 128, learning rate of 10-5 and a connector learning
16

---

Published as a conference paper at ICLR 2026
rate of x 10 with 2000 warmup steps. Stage (manually verified video captioning data) adopts the same settings as Stage 2. Across all stages, we optimize with AdamW, cosine learning rate decay, and 500-step linear warmup schedule. We fix the random seed to 42 and evaluate 2000 validation samples per stage. Training is distributed with Fully Sharded Data Parallel (FSDP) across 64 Nvidia A 1100-0G GPUs using bf cattll precision, gradient accumulation for memory efficiency, and early stopping with a patience of 3 epochs, checkpointing the best validation model.
v-LCM Training Details For training the v-LCM, adopt the LCM two-tower architecture (LCM team et al., 2024) with the diffusion-based next-sentence fine-tuning objective. The optimizer AdamW with TO-6, weight decay of 0.01, gradient clipping at 25.0, and learning rate 3 x 10-5 grappinnclipping
scheduled with cosine decay, warmed up over the first 300 steps, and annealed to final learning rate of 10-6 Training is run for a maximum of 10,000 steps with batch sizes dynamically determined up to 7168 latent embeddings, using gradient accumulation set to 1. Checkpoints are saved every 1000 steps, and we select the best performance according to the validation performance. The criterion incorporates a conditional guidance probability of 0.15 as used in LCM, and loss is reduced with summation loss function. Training uses Fully Sharded Data Parallel (FSDP) with 6 precision for efficiency. Data loading is set to uniformly sampled from all training set in M3IT, length-ordered batching without packing. Experiments are conducted on node with A100 GPUs (80GB).
E SONAR VS OMNISONAR
We also report the comparison in the retrieval performance in Table 10 While OMNISONAR
generally holds an advantage, SONARI remains highly competitive (e.g.. R@ I of 64.9 on PVD- Bench compared to PECoreG).
R@1
R@S
R@ 10
MRR
SONAR1
0.649
0.843
0.895
0.737
PVD-Bench
OMNISONAR
0.730
0.898
0.938
0.805
SONARI
0.536
0.759
0.838
0.638
DREAM-IK
OMNISONAR
0.633
0.841
0.890
0.725
SONARI
0.119
0.263
0.350
0.195
VATEX
OMNISONAR
0.408
0.686
0.789
0.536
Table 10: Comparison for V-SONAR trained with SONAR version I and 2 embedding space on three
text-to-video retrieval datasets.
F DETAILED MULTILINGUAL EVALUATION
We present the results of our multilingual evaluation across all supported datasets in this section. Specifically, we test all languages covered by SONAR and M3IT, reporting ROUGE scores as the primary metric. Figure 10 shows the results for the image captioning task on ImageNet, while Figure 5) and Figure 6 report results for video captioning and video QA on MSR-VTT, respectively. Figure 7 presents results on OKVQA. OKVQA,FFiuure Millustrates performance on story generation in VIST, and Figure 9 Shoows results for image QA on VQA-v2. With the exception of ImageNet, likely due to its widespread use and extensive coverage in existing VLMs, our model consistently outperforms baselines across all tested languages, with the only exception being Thai in VQA-v2.
G VISUALIZATION FOR V-SONAR'S LATENT SPACE
To qualitatively assess the effectiveness of our alignment, we visualize the latent spaces of video and SONAR embeddings before and after each stage of aligning PERCEPTION ENCODER to SONAR using t-SNE (Figure 11). After each stage of aligmment, we observe a better clustering structure where video embeddings and their corresponding SONAR embeddings lie in the closer proximity, indicating that the alignment successfully reduces modality gaps in the latent space and supports a shared semantic representation across modalities, thereby validating the alignment strategy.
17

---

Published as a conference paper at ICLR 2026
V4CM Ower2.5-LL7
PLM-R8
14
Calcarn
Butee Pargal
Hno
Valsy Unmo
Covel
Talagu
Hungo foo
Danall
Tarpaing
Merathi
cacrquan
Occ
Galitten
HS Argaid
Hal-50
Hosr Heariw
Frohuh
sugorimm
telerzaim
Germant
Koolt
Standand Lodken
Russon
Duco
Palch
Sortion
Chatto
talomin
tre
Chinass
Figure M3IT evaluation on 61 languages for MSRVTT.
WAP
Sparis
Indoneston
Cabaano
crosthe
Crnder
galdion
oubirt
Topalog
Hend
French
Icrear
German
ME faeee
Tuni
Crech
Sibvak
javo twostah
coinze Eimal
Karmada
Tellga
occiarr
Duth
Sentier
Stendeed Lanstrr
Haladoo
Estorion
Hiveta
Georggan
Figure 6: M3IT evaluation on 61 languages for MSRVTT-QA.
VA.CM Owent 5-5--7
PLM--
331
Plutica
Spartaa
Gredt
fressh
Coltton
Garzan
cenare
Duigiinn on
Kovenn
Unuu
Malayolan
Weinn
Eartan Hozznn
Stovak
oemmon
Iaslon
Bongal
Potat
Ooctin
Hungeran
Raddiran
Danish
Herate
Cacch
bucch
Russann
Bishian
Sweeesh
Ammician
Neturaen
Siamderd Jatoiin
Exteian
Lurenoour gist Carrgan
Figure 7: M3IT evaluation on 61 languages for OKVQA.
18

---

Published as a conference paper at ICLR 2026
M4CM Owend 5-V
PLN-B
000
Merites
Eumiss
Tmm
Colvaro
Cuperas
Tagalo
Teloge
cove
Baiter Meleystem
French
Georgann
Herrow
Ammatian
Spanish
mulgurian
Hocettelian
Romanien
Sorbin
taltin
japannoo
nurksh
NS Araaic
Slovak
Creatimn
Rossen
Couch
Galicion
Sreddih
Patch
Sundarl Latiin
tecandic
Estninn
Lwnamburrghn
Acurian
Figure 8: M3IT evaluation on 61 languages for VIST.
Squris
Frendh
Greek
Burmere
Catuano
huipedee
Tepaing
Vistramen Sioeet
Gertie
Potza
Bortior
lo for
Romertarr
Belorudo uster
Danig
Entontor
Swedta
Russion
Unarrian
Tha
Koncal
Mallero
Coincee Simal
Sionoor Lostor
Aemeriar
Giciaan
lumi
Eastem Fanabl
Marshh
jaaci 1e0
tne
Ayroy
Heleystar
Kemata
Grorga
Figure 9: M3IT evaluation on 61 languages for VQA-V2.
Ower2 5-5-77
PLN-I
Gemplte
Bargall
Malayahm
Gerggn
Untt
Hodee
Bulgeaan an
Greek
Spurta
Cech
Gultam
Cotslon
Sovv
Pulon
ceeuser
Settin
Thai
Jupaneze
SWeeth
Rallnn
Ainaam
bumnece
owees Pooos15mn
West
Cacers
Figure 10: M3IT evaluation on 61 languages for image captioning.
19

---

Published as a conference paper at ICLR 2026
VSONAR
10
SONAR
TIME Component
Companent
-10-
-10.0
TSNE Component
TSNE Camponent
(a) No alignment.
(b) After stage 1.
15-
YSONAR
VSONAR SONAR
SONAR
10-
TSME Componentt
TSME
-10-
20
10
Component
Component
(c) After stage 2.
(d) After stage 3.
Figure 11: Visualization with t-SNE for SONAR and V-SONAR embeddings after each stage of curriculum. V-SONAR encodes the video, and SONAR encodes the caption. We randomly sample 200 samples from PE- Video's 's testing set for i-SNE, and explicitly plot the lines for connecting the paired video and caption for 50 samples.
H ANALYSIS IN CROSS-MODAL DRIFT FOR V-SONAR AND V-LCM
In this section, we investigate the potential happening of the cross-modal drift in V-SONAR and V-LCM's decoding process. We aim to answer the question whether the generated language by SONAR decoder and v-LCM faithfully represents the same semantic region of the embedding space in V-SONAR. We conduct three analysis on PE-Vis Video testing set to empirically prove that V-SONAR and v-LCM suffered minimum semantic drift during decoding when they handle embeddings from various modalities.
Semantic Sim. Dist.
Round-trip Retrieval
Cosine
R@1
R@5
R@ 10
MRR
Groundtruth 0.666
0.197
87.00%
95.90%
97.10%
0.9084
SONAR Decoder
0.689
0.175
82.50%
97.00%
98.70%
0.8883
V-LCM
0.562
0.219
82.30%
96.70%
97.90%
0.8867
semantic similaritics and distances between video embedding and the ground-truth captions, generated captions from SONAR Decoder and v-LCM. Right: results for the round-trip -rip retrieval ablation study where we use the ground-truth captions, SONAR Decoder and V-LCM's generated captions retrieve the videos.
Table 11: Left:
Embedding-Lee Semantic Fidelity. For each video embedding v. we first compare its similarity and distance to: (i) ground-truth caption embedding tgt, (ii) SONAR Decoder caption embedding (tsonars and (iii) LCM caption embedding teem- This analysis provides a direct comparison for the generated captions from SONAR decoder and V-L.CM with the ground-truth captions.
20

---

Published as a conference paper at ICLR 2026
Conersted Lockcc Serlactyy
Coadon CaIPa Srarry
CoR Sprliity OF Cecommeee
2410
144
Conerded Duckeer
CT Creiii Duthden Dolance
Exckeon Sttaree 07 unteed
300
0.5
622
Aue
0.56
3.25
6.2
1.00
0.50
055
0.00
00
Figure 12: vSONAR's visualization for cross-modal semantic drift.
As indicated in Table 11 we find that SONAR-decoder captions show nearly identical cosine similarity distance compared to the groundtruth, indicating negligible cross- modall drift. vLCM captions show a slightly larger deviation; we attribute this to vI.CM's 's ceatruction-oolowwiii training which introduces stylistic paraphrasing, rather than semantic drift.
Ablation with the Round-trip Retrieval. We further conduct an ablation study with the round-trip retrieval where we use three groups of captions (Groundtruth, V-SONAR Decoder, and v-LCM) as queries to retrieve the source videos in PE-Video. In Table 8, we find that Captions decoded by SONAR or LCM retrieve the correct video with extremely high accuracy. Notably, LCM is within 0.2% of SONAR on R@I 1. If cross-modal drift were substantial, retrieval accuracy would drop sharply: instead, it remains high, confirming its semantic preservation.
Visualizing the Cross-modal Drift. Finally, plot the similarity between vision embeddings and the ground-truth caption embeddings versus the embeddings for V-SONAR and LCM captions in Figure 13 and Figure 12 SONAR-decoder captions show nearly identical (or slightly better) cosine similarity/disten compared to ground truth, indicating negligible cross-modal drift. vLCM captions show a slightly larger deviation; we attribute this vLCM's 's cntruction-follwinnn training which introduces stylistie paraphrasing, rather than semantic drift (verified in the next experiment). The points cluster also generally around the = x line, directly showing no significant systematic semantic shift.
I QUALITATIVE CASE
I.1 IMAGE QUALITATIVE CASES FOR V-LCM
We present the qualitative cases for image captioning in Figure 14 and image question answering in Figure 13
I.2
VIDEO QUALITATIVE CASES FOR V-LCM
We present the qualitative cases for video captioning and question answering task in Figure 16
21

---

Published as a conference paper at ICLR 2026
Serrrd Capton CHiPa by
Captoon Cauise baty
Cothe Sindiatiy 1a9e
2310
2014
$15
Conexted Duthden
GT Dalree
Oufider Dersece CT Denorred
0.8
6.52
3.35
0.55
$.35
0.35
0.24
0.30
322 5.11
Figure 13: vL.C''s visualization for cross-modal semantic drift.
Vision Input
Prompt
Outputs
Write short and informative descrip- tion that highlights the main subjects and actions occur- ring in the given im- age. <IMAGE>
"The video shows man riding a red- colored bike on a red-colored motorbike riding on a black mountain, with a motorcycle and a rug, and red man standing behind mountain with a bridge and a grass and a roof."
LCM:
V-LCM: "A man riding a red motorcycle on a mountain."
"The vision begins with the bright video building standing in front of a broadly paved build- ing, and a crowd of people walking on the street. As the shines
LCM:
the building, the buildings begin to streak down the street. The vehicles move closer to the street, with the buildings creeping down the street. The vehicles are moving the street, and there are people standing on the other side of the pavement. Eventually, the vision ends with the vehicles moving off onto the street, and the buildings slide down the street."
Compose descrip- tive paragraph that brings the image to life through detailed storytelling. con- necting the various visual elements into unified narrative.
v-LCM: "A large building is walking near the corner of a street. There are white windows front of the building. There is large grey pole in front of the building. There people walking the street in front of the pole." "
<IMAGE>
Qualitative examples for image captioning task in M3IT benchmark. Each row shows the input image, the given prompt, and the generated model output.
Figure 14:
I.3 QUALITATIVE CASES FOR VIDEO CAPTIONING BY V-SONAR
In addition to the qualitative cases for LCM. We present the qualitative cases for video captioning with V-SONAR compared other VL.Ms in PVD-Bench and DREAM-1K from Figure 17 to Figure 20
22

---

Published as a conference paper at ICLR 2026
Vision Input
Prompt
Outputs
LCMM: "The entities that play this stadium are Red Sta- dium."
Observe the image closely, recognize the named entities present, and use your knowledge answer the question in re- lation to those entities. <IMAGE> Who plays this stadium
v-LCM:
"The team that plays
this stadium Manchester United."
LCM: "It specified the information provided that how many points "V' tile is worth." "
Analyze the image to locate any named entities and provide an appropriate re- sponse to the question. Start vision. <IMAGE> End of vision. In this game, how many points "V" tile worth?
REME OO
V-LCM: "A 'V' tile is worth 4 points."
Carefully review the question and the related image/text context, identify the precise answer from the multiple-hhoce selections, and provide detailed, logi- cal explanation to support your choice, illustrating the multi-jumbo reasoning journey.
LCM: "(B) how stable
<IMAGE>
parachute with a m vent at
Question: What of the following could
200 km per hour."
Gordon's 's test show?
V-LCM: "The answer is (B) how stable was a parachute with a air leak was 200 km per hour."
[...]
Options:
(A) if the spacecraft was damaged when using a parachute with a I m outlet going
200 km per hour.
(B) How stable was parachute with an
m vent at 200 km per hour.
(C) whether parachute with I vent would sway too much 400 km/h.
"The name of the trans- lator is not provided."
LCM:
Please respond to the query by refer- ring to picture and its limited entities. <IMAGE> What is the name of the trans-
Dabacca
V-LCM: "The name the translator Rebecca Hert- ing."
lator?
Qualitative examples for image question answering tasks. Each row shows the input image, the given prompt, and the generated model output.
Figure 15:
I.4 QUALITATIVE CASES FOR VIDEO CAPTIONING BY V-SONAR
We present the qualitative cases for vinual-commoonssssereaaaiiin (VCR) with v-LCM compared to other VLMs in Figure 21 The qualitative example demonstrates that the aligned encoders, thought being trained with semantic-level caption, can still capture layout grounding and the spatial reasoning ability.
23

---

Published as a conference paper at ICLR 2026
Prompt: will be shown video and asked question about what you saw. Your answer should be brief sentence that fits the video's ' contents. <VIDEO> what is a man slicing? L.CM: A is cutting an onion with knife.
v-LCM: The man in the video is slicing an onion.
Prompt: Use your observation skills the given question with the help of short video clip. Start of vision. <VIDEO> End of vision. What type of bread is shown? LCM: The type of bread shown sandwich slice.
V-LCM: The bread shown video a bagel.
BNatwork
Blobwok
1.
40
40
socent allable
bllable
Prompt: Watch the video and create caption based the visual cues. <VIDEO>
L.M:: [Edd of Response.
V-LCM: someone is driving red car
Qualitative examples for video captioning and question answering tasks, The frames are presented from left to right, and top to bottom.
Figure 16:
24

---

Published as a conference paper at ICLR 2026
1=1
t= 3
Models
Video Captions
V-SONAR (Ours)
The video shows blue mixer stirring a white powder a round bucket. bucket is moving with a rotating motion and attached to a blue cylinder. The mixer is moving slowly in the middle, creating white powder.
PLM-3B
In the background, there is bucket, stick, and a grey surface. The sound of the mixer is audible. /Vague; mentions audio]
Qwenn..5-VL-3B-Insinuu A close-up shot of blue bucket filled with white paint, A red-handled paintbrush is dipped into the paint and then lifted out, leaving a trail of paint behind it. The brush is then lowered back into the paint and the process is repeated several times. The camera remains stationary.
InterVVL2.5518
A yellow hand reaches into the white plastic bucket, grabbing the red cylindrical object, which is tool for pouring cement.
Figure 17: Qualitative comparison on fine-grained motion understanding in PE-Video. While state-of-the-art VLMs (Qwen-2.5, Intern VL-2.5) suffer from hallucinations or captioning erors (highlighted red) such as a "paintbrush" or "yellow hand," and PLM remains vague, SONAR accurately captures the mechanics of the mixer and the blue cylinder attachment (highlighted in green).
25

---

Published as a conference paper at ICLR 2026
1=7
Models
Video Captions
V-SONAR (Ours)
The video shows cake with candy bar top. The frame in bunny and chocolate bar, then candy bar removed from frame. The video ends the bunny lying on the ground, and a camera zooms on the cake.
PLM-3B
In the background, there a brown surface, a brown object, a brown wall, a green grass surface, and brown object. The sound of the music is audible. [Vague; mentions audio]
Qwen2.5-VL-3B-Ime A chocolate donut with white frosting and blueberries placed on wooden surface. The donut has three candles on it, two blue and one yellow. The donut up by the candles, and the flames are visible. The donut is surrounded by crumbs and other debris. The background is blurred, but it appears to be outdoor setting with grass and trees.
Intern VL2.5-IB
The small cake on table seen being violently by heavy club-like object from which the smoke begins to rise. The cake is shown up close on the table surrounded by paper. After the cake is violently hit, the hammer used is placed back onto the table and continues hit the marble coaster.
Figure 18: Qualitative comparison in PE- Video. We highlight the errors in red.
26

---

Published as a conference paper at ICLR 2026
Models
Video Captions
Y-SONAR (Ours)
The video shows group of people sitting around table with notehooks their hands. The people are writing and drawing on the paper, some are holding a pencil, and others are holding a notebook. The table is set in the foreground with people moving around it, and they are writing the paper.
PLM-3B
The video features group of people sitting around table, engaged in collaborative activity. The individuals are holding pens and pencils, and there are various objects on the table, including laptop. notebooks, and sticky notes. The scene appears to be a meeting or brainstorming session, with the participants working together to create something. The video is in slow motion, which adds sense of calmness and focus to the scene. Overall, the video conveys productivity.
of collaboration, creativity, and
Qwen2.5-VL-3B- Instruct
The video shows a group of people sitting around table, working on a project together. They are using laptops and writing notes on paper. The atmosphere is collaborative and focused, with everyone contributing lighting
the task at hand. The and inviting, creating cozy and inviting environment. The people are engaged their work, and overall mood is positive and productive.
Intern VL2.5-IB
In this video, a team of individuals collaborates to contribute to the design process on desk.
Figure 19: Qualitative comparison DREAM-IK.
27

---

Published as a conference paper at ICLR 2026
1=1
1=2
1=3
1=4
7
Models
Video Captions
Y-SONAR (Ours)
The video shows group of dancing in white shirt. man is standing in the with a rope in his hands a red truck behind him. The man pulling rope and the truck is moving towards the field.
PLM-3B
A man wearing white t-shirt and black pants is standing on the grass surface and then starts running and kicking the white rope. Another man wearing red t-shirt and black pants is standing on the grass surface and then starts running and kicking the white rope. The third man wearing white t-shirt and black pants standing on grass surface and then starts running and kicking the white rope. In the background, there is grass surface, trees, red vehicle, white rope, and the sky. The voice of the people audible.
Qwve22.--VL-3B-nnsttuuc
The video shows man standing in field, wearing white t-shirt and black pants. is holding rope and appears to be playing a game tug of war. The background features trees and clear blue sky.
VL2..-VL2.5-1BB
A performer in white shirt and black belt balances long rope in park, with red vehicle and lush red tree in the background. The performer changes their stance, eventually falling off the rope and landing on the ground, then quickly gets back up ready for the next try or battle.
Figure 20: Qualitative comparison in DREAM-IK.
28

---

Published as a conference paper at ICLR 2026
Models
Prompt
Given an image and some bounded objects, answer the question and explains why the answer is right.
Objects: The 0-th object is person the color box. The 1-th object person the blue color box. The 2-th object is person in the green color box. The 3-th object is person in the pink color box. The 4-th object is dog in the light blue color box. Question: What is the 3-th object doing with the 4-th object
PLM-8B
Young children enjoy taking their pets for walk outside.
Owen2.5-VLL-B-Instre
Answer (A) 3-th object is taking the 4-th object a walk outside. Rationale: The 3-th object appears to be a child based on the size and posture, and the 4-th object is dog. The context of the image suggests an outdoor setting with child and a dog, which aligns with the idea of child taking pet for walk. Dogs often enjoy walks, and it's common for young children take their pets for walks as part of their daily routine or playtime. The other options do not the context there is no indication of flirting, looking at computer, or setting anything.
image as well;
InterVVL2.5-BB
the 3-th object was setting the 4-th object.
v-LCM
The 3rd object is taking the 4th object for walk outside. 'Cause little kids like to take their pets for walk.
Figure 21:
Qualitative comparison in
Coommonsens-RRessonnng Reasoning (VCR), which requires
grounding and spatial reasoning ability connecting with the commonsense.
29


Reviews:
[{"review_id": "cqGhp46bQk", "reviewer": "Reviewer_wU7h", "summary": "This paper introduces v-SONAR, a vision-language embedding space constructed by post-hoc aligning a state-of-the-art vision encoder with the existing multilingual text embedding space Sonar. The authors propose a three-stage curriculum that leverages large-scale image-caption and video-caption datasets to align Perception Encoder representations into Sonar's semantic concept space, supporting 200 text languages and 37 speech languages in addition to images and videos. Thorough experiments evaluate the alignment and demonstrate competitive or superior zero-shot performance in text-to-video retrieval, video captioning, and especially in multilingual and multi-modal instruction following. The framework is further extended with v-LCM, a latent-diffusion-based unified vision-language model, which is instruction-tuned and evaluated across a large suite of language and vision-language downstream tasks.", "questions": "1. The paper's two main SOTA achievements are based on two different SONAR versions: SONAR2 for video captioning and SONAR1 for multilingual v-LCM . Given that SONAR2 is reported to be substantially stronger , does this imply that the multilingual performance of v-LCM (based on SONAR1) is significantly underestimated? What is the expected performance ceiling for v-LCM if it were aligned and trained with SONAR2?\n\n2. v-LCM is based on a latent diffusion model (LCM) , which requires iterative denoising for generation. This process is typically much slower than the autoregressive transformers used in baseline VLMs. What is the actual inference latency of v-LCM compared to models like Qwen-VL or PLM-8B, and why was this critical discussion of computational cost omitted when assessing its practical utility as a \"new paradigm\"?\n\n3. v-LCM achieves success in 61 languages , yet its alignment data (12M images + 2M videos ) is likely English-dominated. How does the paper prove that v-LCM learned a true alignment between visual concepts and non-English concepts, rather than simply \"inheriting\" this capability as a \"free lunch\" from the pre-trained SONAR space?\n\n4. Appendix B shows that contrastive loss was superior for retrieval (52.4 vs. 49.0), while the chosen MSE loss was only marginally better for captioning (38.9 vs. 38.6). Why does the more common contrastive loss, which improves retrieval alignment, appear to hurt the performance of generative tasks like captioning? Can a deeper technical analysis be provided?\n\n5. Why did v-LCM fail to outperform baselines in Dutch and lag significantly on English tasks like VisualMRC (Table 5)? Does this expose a fundamental limitation of the v-LCM framework in handling specific languages or task types?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.Instead of training a massive multilingual VLM from scratch, the authors cleverly \"inherit\" the advantages of two SOTA models: SONAR's powerful multilingual semantic space and PERCEPTION ENCODER's strong visual representation capabilities. This approach is more computationally efficient and highly scalable.\n\n2.By modeling in the unified SONAR/V-SONAR latent space, v-LCM successfully generalizes its visual understanding capabilities to a massive number of low-resource languages, significantly outperforming strong baselines like Qwen-VL and PLM-8B in 61 out of 62 languages.\n\n3.The proposed Large Concept Model (LCM), trained only on text, is capable of directly processing V-SONAR encoded visual embeddings and performing complex tasks like video summarization without having seen any visual data. This strongly demonstrates that V-SONAR has successfully aligned visual concepts to the SONAR concept space at a semantic level.\n\n4.The alignment method employs a three-stage coarse-to-fine curriculum, progressively achieving visual-semantic alignment. This training strategy demonstrates good engineering feasibility.", "weaknesses": "1.Critical Flaw: Inconsistent and Non-Reproducible SONAR Versions. The paper's two main SOTA (state-of-the-art) achievements are based on two different, incompatible SONAR versions. The SOTA video captioning (Table 3) uses SONAR2, an unreleased, higher-performing internal model . Conversely, the flagship multilingual SOTA results (Fig. 3) rely on the public SONAR1 , as the LCM was trained on it. This mismatch creates significant confusion about the paper's true contribution and renders the SOTA video captioning results non-reproducible.\n\n2.The paper introduces v-LCM as a \"new paradigm\" built upon the Large Concept Model (LCM), which is a latent diffusion model . Unlike standard Transformers, diffusion models require multiple iterative sampling steps for generation, which is typically orders of magnitude slower. The paper provides no discussion on inference latency or computational cost, making it impossible to evaluate the practical utility of v-LCM against standard auto-regressive VLMs.\n\n3.Lack of Analysis on Training Data Bias. The 12M+ image and video pairs used for alignment are likely English-centric. The paper provides no analysis of this potential data bias or how it impacts the model's outstanding multilingual generalization. It is unclear if this capability is learned during alignment or simply a \"free lunch\" inherited from the pre-trained SONAR space.\n\n4.Multilingual Generalization vs. English SOTA. While the multilingual results are exceptional, the paper's own data (Table 5) shows that v-LCM is not the overall best on competitive English benchmarks. It trails the PLM-8B baseline in tasks like video summarization (VIDEOXUM) and document QA (VisualMRC). This indicates the model's core strength is its broad multilingual generalization rather than peak performance in high-resource English tasks.\n\n5.The paper justifies its use of MSE loss over a standard contrastive loss by claiming \"no significant gains\" . However, the data in Appendix B (Table 6) shows a clear trade-off: contrastive loss performed better on retrieval (R@1 52.4 vs. 49.0) but slightly worse on captioning (BLEU 38.6 vs. 38.9). The paper lacks a necessary technical analysis of why this trade-off exists and why MSE is the better choice for generative alignment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ux9OxiwmO7", "reviewer": "Reviewer_XhVS", "summary": "The paper introduces V-SONAR, a framework that aligns visual encoders into the SONAR multilingual concept space, and extends the Large Concept Model (LCM) to vision–language tasks via V-LCM.\nInstead of joint multimodal training, the authors propose a post-hoc alignment of frozen visual features to an existing multilingual embedding space, enabling zero-shot and multilingual image/video understanding.\nExtensive experiments on retrieval, captioning, and multilingual instruction following show competitive or superior results, especially across 61 of 62 evaluated languages.", "questions": "- When decoding visual inputs into text via LCM, have you evaluated whether the generated descriptions remain semantically faithful to the corresponding visual embeddings? If not, how do you assess or mitigate possible cross-modal drift?\n- Could you include some qualitative or more interpretable examples (e.g., generated captions, retrieval results, or typical errors) to illustrate how the model behaves in practice?\n- Could the post-hoc alignment to a fixed text space cause semantic drift or information loss, especially for fine-grained visual details that have no direct linguistic counterpart?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Elegant unification:** The method merges vision and language in a shared continuous embedding space, avoiding heavy multimodal retraining.\n\n- **Practical efficiency:** Simple MSE-based alignment achieves strong zero-shot and multilingual performance with modest compute.\n\n- **Comprehensive evaluation:** Covers image/video tasks, long-video summarization, and multilingual benchmarks.\n\n- **Insightful analysis:** Provides embedding-geometry diagnostics (AC/trace/logdet) to interpret cross-modal alignment quality.\n\n- **Strong empirical results:** Outperforms major baselines like SigLIP2 and PLM on several metrics.", "weaknesses": "- **Lack of clear conceptual  motivation** –\nThe paper reads more like a comprehensive technical report. While the system is well engineered, the underlying motivation and novelty are not sharply defined. The central idea—post-hoc alignment of visual features to a frozen multilingual embedding space—feels incremental, relying on existing components (SONAR, LCM).\n\n- **Potential semantic collapse within the aligned subspace.**\nAligning visual features via an MSE loss to a frozen text space risks over-compression: different visual scenes may map to similar embeddings. The paper lacks an investigation into embedding diversity or anisotropy after alignment—important for understanding retrieval and generation limits.\n\n- **No examination of cross-modal consistency at inference.**\nThe paper shows that visual inputs can be decoded by the LCM text generator, but doesn’t analyze whether the generated language faithfully represents the same semantic region of the embedding space. Cross-modal drift—where decoding shifts meaning—may exist but is untested.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pT9IsXXFZ6", "reviewer": "Reviewer_fuEm", "summary": "The paper extends the SONAR multilingual embedding space to vision via V-SONAR, which aligns a pretrained Perception Encoder to SONAR through a post-hoc MSE-based mapping and a three-stage image/video captioning curriculum. This enables the Large Concept Model (LCM), trained only on text embeddings, to operate directly on visual inputs. The authors further introduce V-LCM, a vision–language instruction-tuned version trained in the shared SONAR/V-SONAR latent space. The approach achieves zero-shot video retrieval and captioning, and V-LCM matches or surpasses state-of-the-art models on multilingual vision–language tasks across most of the languages.", "questions": "- Can the authors report a controlled comparison, such as aligning to SONAR1 versus SONAR2 or evaluating against the unaligned Perception Encoder, to quantify the gain contributed by the proposed alignment?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper discussed a clear and well-motivated idea of extending the SONAR embedding to visual modalities through a lightweight post-hoc alignment.\n2. The proposed alignment pipeline is designed well and experimentally validated.", "weaknesses": "1. The paper’s main idea, while clearly implemented, feels somewhat incremental as it builds on existing post-hoc alignment frameworks rather than introducing a fundamentally new alignment principle.\n\n2. The evaluation mainly focuses on retrieval and captioning benchmarks, which measure semantic alignment but not grounding quality. It remains unclear whether V-SONAR preserves sufficient spatial and compositional information for tasks that require reasoning about object relations, layouts, or fine-grained attributes.\n\n3. The reported numerical gains cannot be attributed solely to the proposed alignment, since the results combine the benefits of the stronger SONAR2 text space and the powerful Perception Encoder backbone. A more controlled comparison, such as aligning to both SONAR1 and SONAR2, or directly comparing against the unaligned vision encoder, would better isolate the contribution of the alignment itself.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XT9FZIVZ65", "reviewer": "Reviewer_wLL2", "summary": "This paper introduces v-Sonar, a paradigm to map vision representation into Sonar’s embedding space via representation alignment, enabling direct reasoning and decoding in a shared latent space. v-Sonar achieved strong zero-shot text-to-video retrieval and video captioning performance. Building upon this, the paper introduces v-lcm, a multi-modal version of large concept models, which perform latent diffusion over unified visual-text latent space, and achieves comparable performance to SOTA VLMs.", "questions": "1. The open-sourced PE-VIDEO dataset contains around 120k videos with annotation. However, v-Sonar uses 200k video-caption pairs to refine the alignment in stage 3 and uses 15k pairs for evaluation. Are there any overlaps?\n\n2. How is v-lcm supervised finetuned? Do you update all the parameters?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper is well-written and easy to follow.\n2. The evaluation and ablation studies are very comprehensive.\n3. v-Sonar and v-lcm present great zero-shot capabilities, which are very interesting.", "weaknesses": "1. The paper is more like a technique report with limited technical contributions. The core concept, aligning vision encoders to text encoders, has been proposed by previous work [1]. The specific techniques to align visual-textual representations (MSE and contrastive loss) are also very standard.\n2. Most metrics are only comparable to SOTA models, except for the multilingual ability, which may partly be due to the fact that Sonar is specially designed and trained with specific datasets for this objective. While it is encouraging to explore architectures different from current LLMs even with temporary underperformance, it is preferable that the new model can demonstrate significant advantages in certain special aspects.\n3. Some of the implementation details are unclear. See questions.\n\n[1] Jose, Cijo, et al. \"Dinov2 meets text: A unified framework for image-and pixel-level vision-language alignment.\" CVPR 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)