Think Then Embed: Generative Context Improves Multimodal Embedding
Authors: Xuanming Cui
Jianpeng Cheng
Hong-You Chen
Satya Narayan Shukla
Abhijeet Awasthi
Xichen Pan
Chaitanya Ahuja
Shlok Kumar Mishra
Taipeng Tian
Qi Guo
Ser-Nam Lim
Aashu Singh
Xiangjun Fan
Venue: ICLR 2026 Poster (2026)
Keywords: Retrieval
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: There is a growing interest in Universal Multimodal Embeddings (UME), where models are required to generate task-specific representations. While recent studies show that Multimodal Large Language Models (MLLMs) perform well on such tasks, they treat MLLMs solely as encoders, overlooking their generative capacity. However, such an encoding paradigm becomes less effective as instructions become more complex and require compositional reasoning. Inspired by the proven effectiveness of chain-of-thought reasoning, we propose a general Think-Then-Embed (TTE) framework for UME, composed of a reasoner and an embedder. The reasoner MLLM first generates reasoning traces that explain complex queries, followed by an embedder that produces representations conditioned on both the original query and the intermediate reasoning. This explicit reasoning step enables more nuanced understanding of complex multimodal instructions. Our contributions are threefold. First, by leveraging a powerful MLLM reasoner, we achieve state-of-the-art performance on the MMEB-V2 benchmark, surpassing proprietary models trained on massive in-house datasets. Second, to reduce the dependency on large MLLM reasoners, we finetune a smaller MLLM reasoner using high-quality embedding-centric reasoning traces, achieving the best performance among open-source models with a 7% absolute gain over recently proposed models. Third, we investigate strategies for integrating the reasoner and embedder into a unified model for improved efficiency without sacrificing performance.

Published as a conference paper at ICLR 2026
THINK THEN EMBED: GENERATIVE CONTEXT IM- PROVES MULTIMODAL EMBEDDING
Xuanming Cui
Jianpeng Chen
Hong-You You Chen
Satya Narayan Shukla? Shlok Kumar Mishra²
Abhijeet 2wasthi²
Xichen Pan
Chaitanya Abuja²
Taipeng Tian
Qi Guo Ser-Nam Lim Aashu Singh?
Xiangjun Fan?
University of Central Florida
Meta
3 New York University
ABSTRACT
There a growing interest in Universal Multimodal Embeddings (UME), where models are required to generate task-specific representations. While recent stud- ies show that Multimodal Large Language Models (MLLMs) perform well on such tasks, they treat MLLMs solely encoders, overlooking their generative capacity. However, such an encoding paradigm becomes less effective as instruc- tions become more complex and require compositional reasoning. Inspired by the proven effectiveness of chain-of-thought reasoning, we propose a general Think- Then-Embed (TTE) framework for UME, composed of reasoner and an embed- der. The reasoner MLLM first generates reasoning traces that explain complex queries, followed by an embedder that produces representations conditioned on both the original query and the intermediate reasoning. This explicit reasoning step enables more nuanced understanding of complex multimodal instructions. Our contributions are threefold. First, by leveraging a powerful MLLM reasoner, we achieve state-of-the-an performance on the MMEB- V2 benchmark, surpassing proprictary models trained on massive in-house datasets. Second, to reduce the de- pendency on large MLLM reasoners, we finetune a smaller MLLM reasoner using high-quality sabbedding-cenrrcc reasoning traces, achieving the best performance among open-source models with 7% absolute gain over recently proposed mod- els. Third, we investigate strategies for integrating the reasoner and embedder into a unified model for improved efficiency without sacrificing performance.
INTRODUCTION
Multimodal embedding-based retrieval has emerged a popular and effective solution for han- dling diverse data types such as text, images, and videos (Radford et al., 2021; Li et al., 2023a; Yu et al., 2022: Li et al., 2023b). Traditionally, these models focus learning general-purpose representations that capture content similarity across modalities. Recently, there has been grow- ing interest in instruction-aware Universal Multimodal Embeddings (UMF) (Jiang al., 2024; Gu et al., 2025). UMES are powerful representations because they bridge the gap between general- purpose content similarity and user-specific requirements. This capability unlocks wide range of applications, such as retrieving documents or images based on nuanced queries, powering person- alized recommendation systems, supporting multimodal search with complex transformations, and tackling knowledge intennsve tasks where the same input may yield different embeddings depend- ing on downstream objective. To advance this direction, recent benchmarks like MMEB (Meng et al., 2025) aggregate diverse, instruction-driven retrieval tasks-including VQA, grounding, and document retrieval-whare both the input and the user instruction determine the retrieval target.
In this context, leveraging multimodal large language models (MLLMs) encoders for UME is promising new approach (Jiang et al., 2024; Gu et al., 2025; Lan et al., 2025). MLLMs can generate instruction-aware representations that incorporate long context and nuanced user guidance (Peng
Equal contribution.
Work done during the internship at Meta. Email: xuanming.cuie wuw.fc.euuuuccuuuuu

---

Published as a conference paper at ICLR 2026
et al., 2024). However, while oatte-of-the-ait models on these benchmarks have advanced through techniques such as hard negative sampling (Lee et al., 2025; Thirukovalluru et al., 2025; Lan et al., 2025; Lin et al., 2025), additional training stages (Gu et al., 2025; Yu et al., 2025; Lin et al., 2025), external data (Yu et al., 2025; Chen et al., 2025b), and improved emhedding extraction (Faysse et al., 2024), central challenge remains: understanding instructions of varying complexity that require different levels of reasoning. Addressing this challenge calls for leveraging the generative capacity of MLLMs, rather than restricting them encoder-only models.
Our approach is motivated by a closer exami- nation of the tasks in MMEB-v2 (Meng et al., 2025), revealing that many, such as VQA, clas- sification, visual grounding, and composed re- trieval, require substantial reasoning capabil- ities. For example, the first case in Fig. (from the MMEB-v2 RefCOCO dataset) in- volves retrieving the image region referred to by the query "vehicle second closest to cam- era". Rather than simply encoding the query and matching it the target image, we argue that it is beneficial for the model to first rea- son about the query-identifying, for instance, that it refers to the vehicle with "bright yellow on the upper half and deep blue on the lower". For such complex tasks, we believe that explic- itly augmenting multi-step reasoning and contextual grounding enables the model to achieve more fine-grained and accurate retrieval.
Quaryy
Target
Describe the query object: vhlicl serund -chosoer to comeraa
The expresziio refory to vinsage dosblledde fer with hinglt yelliro the apper kulf and
wanch food would require humons 20307
The dovemment viatir.. By 2036 tht world will nesil to rxplanish Cowely
YXA
the commromax
The bike lane fass arvons
dhtving what do they hure in from rirr poettoon?
Indicating
commrrsnoon will huve Do itrs rightt
right
Figure I: Given a multi-modal input, we want first think about the desired embedding content. Then, the representation is obtained by conditioning on both orig- inal input and the thinking result.
To address these challenges, we propose Our key idea is to introduce an explicit thinking stage before embedding, in which the model gen- crates reasoning traces based on the given instruction. Prior work has shown that intermediate reasoning processes, such Chain-of-Thought (CoT) (Wei et al., 2023b), can significantly improve the accuracy of language model generation. We hypothesize that a similar benefit can be achieved for multimodal representation learning, since the embeddings produced by MLLMs are inherently conditioned on the sequence of preceding tokens. In this work, we investigate how explicit chain-of- thought reasoning can enhance universal multimodal embeddings, by enabling the model to better interpret, follow complex task instructions, and to reduce retrieval asymmetry.
Sreink--hen--meen (TTE) framework for the UME task.
In this paper, we made several contributions. First, proposed Think-Then-EEbee (TTE) frame- work that by exploiting reasoming traces generated by a powerful reasoner (e.g. Qwen2.5 VL 72B), it can achieve state-of-the-ar performance on MMEB V2 (Meng et al., 2025) with a smaller em- bedder (Qwen2 7B), surpassing close-sourced models trained with additional data. This provides an effective way for retrieval sest-tim-sscaligg by flexibly incorporating strong external reasoner.
Second, we study how backbone model can be effectively used both as a reasoner and as an em- bedder. We experimented with two approaches. First, we use the backbone as a zero-shot reasoner while fine-tuning a copy of the same backbone as the embedder. This method yields noticeable im- provements on smaller datasets, but the gains diminish as the dataset size increases. Motivated by this limitation, our second approach involves fine-tuning a second copy of the backbone (same size as the embedder, either 2B or 7B) as a strong reasoner, using reasoning traces generated by the 72B reasoner model as the training data. The resulting SFT-ed reasoner performs effectively across the entire MMEB v2 benchmark, achieving the best performance among all open source models.
Finally, to improve efficiency of both inference and number of parameters in the TTE framework, explore unifying reasoning and embedding creation within a single unified model. Although prior work (Yu et al., 2025) also investigated the unification of generation and contrastive learning, it treats generation and embedding as separate, unrelated tasks. In contrast, our focus is on enhancing em- bedding through reasoning, that is, generate first, then embed. We explored two strategies: (1) joint contrastive-auton autoregressivee training of the same backbone for reasoning and embedding tasks, and (2) contrasntt training with a dedicated embedding head on top of the reasoner. Empirically, we find that the two-stage approach consistently outperforms joint training, almost halving the overall parameters, while not degrading end-to-end TTE retrieval performance.
a two-stage autoregressive wiwtorrgressive-tee--ccom
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Multimodal Embeddings. The multimodal representation paradigm has been popularized by large- scale foundational models such as CLIP (Radford et al., 2021), MetaCLIP (Xu et al., 2024), BLIP (Li et al., 2023a), and SigLIP (Zhai et al., 2023), which encode images and texts using separate uni- modality encoders and enforce alignment using contrastive objectives.
Universal Multimodal Embedding. Recently there has been growing interest in developing Uni- versal Multimodal Embedding (UME) (Gu al., 2025; Jiang et al., 2024; Meng et al., 2025), where the embedding depends both on the query and task instruction. Representative examples include VLM2 (Jiang al., 2024), which proposes the Massive Multimodal Embedding Benchmark (MMEB), comprising a wide range of cross / multimodal retrieval tasks, as well as non-conventional retrieval tasks such as VQA, grounding, and classification. Subsequently, MMEB- V2 (Meng al., 2025) is proposed, extending MMEB-VI include video and visual document (Visdoc) tasks.
MLLM-based Embedding Models. Recent studies have gone beyond dual encoder setups, building embedding models directly on top of powerful Multimodal Language Models (MILMs) (Jiang et al., 2024: Meng et al., 2025; Gu et al., 2025; Yu et al., 2025; Thirukovalluru et al., 2025). Existing work on MLLM-based embedding typically explores training strategy such as text-continual contrastive training (Gu et al., 2025: Yu et al., 2025; Chen et al., 2025a), hard negative mining approach (Lee al., 2025; Lan et al., 2025; Thirukovalluru et al., 2025), additional training data (Zhou al., 2024; Chen et al., 2025b), and architectural design (e.g. with without causal mask) (Lee et al., 2025; Chen al., 2025a). However, these methods all treat MILMs solely as encoders, while overlooking their generative capability obtained from pre-training. In contrast, we explore an orthogonal approach, where we leverage MLLMs for both generative reasoning and representation learning.
LLM-based Query Rewriting. Frequently used in text-based retrievals, query rewriting (Ma al., 2023) is an effective approach to improve retrieval accuracy. Popular trends include prompt-based rewriting (Wilson et al., 2025; Ye et al., 2023), iterative or Reinforcement Learn- ing (RL)-guided (Cao et al., 2025; Zhu al., 2025) query rewritings, and multi-reformulation ap- proaches (Kostric & Balog, 2024; Dhole et al., 2024). However, these works all focus on text-based retrievals using external retrievers, where no learned embedding is involved.
Recently, Bai et al. (2025b) applied the idea of query rewriting to uni-modal encoder- based models (e.g. CLIP) for text-to-video retrieval, by enriching textual queries with additional context to bridge information asymmetry. In contrast, our work focuses on leveraging reasoning to improve the quality of general instruction -oollwiinn-folooinngg multimodal embeddings, which can apply to both query and target.
PRELIMINARY: MLLM-BASED UNIVERSAL MULTIMODAL EMBEDDING
We briefly introduce MLLM- Universal Multimodal Embedding (UME). In UME, every query q and target is triplet of an optional visual input, textual input, and pre-defined task instruction, written as (V, T, )). Both query and target are passed into the same MLLM separately to obtain the corresponding embedding, defined by the equation below:
hg Pool i (go [Tnsy), (ing),TT) Ty)), h = Pcol (no( [Tsss), T:)),
where fo denotes the MLLM embedder and Pooling refers to the pooling operation for aggregat- ing MLLM's hidden states into the final embedding. Following previous works (Jiang et al., 2024; Gu al., 2025), we use the last token's 's hidden state for final embedding. During training, adopt the standard uni-directional + t InfoNCE loss as follow:
1 N
N
(h hy)
((g h.) exp( 1 cos(by h:)),
LISTONCE
log
ΣØ (h h²)
where cos denotes the cosine similarity function, and 7 is the temperature hyper-parameter.
4 THENK-TTE--MBBBRE
We propose our teatttthh---mmeee (TTE) framework for enhancing Universal Multimodal Embed- dings (UME) by incorporating an explicit reasoning step prior the encoding step. TTE consists
3

---

Published as a conference paper at ICLR 2026
the pottion follows the
MLLM
Tokens
MILM Reasoner
Embedding centric Cot
volicle scoeef-clssor
sleeage doaNieseedery
Visuel Tokens
Text Takers
Visual Tokens
Text Tokens
Vis. Hiddiens
Text Hidders
CoTT Hidddns
MLLM Embodder
Embedding
Lost
Lost
Lest
(a) Existing MLLM-bascd embodding ap- proach. The MILM embedder directly code visual and textual queries, producing the embedding from last token.
TTE with a separate MLLM reasoner.
(c) TTE with unified reasoning and embed-
The MLLM embedder takes in both
ding. After MLLM reasoner finishes eration, its entire hidlen states are passed pre-trained embedding head.
inal inpuls and embedding centric CoT
generated by
Figure 2: Pipeline comparison between existing MLLM-based embedding (a) and proposed approach (b, c).
of a reasoner, that generates an Embedding Center Reasoning (ECR) trace, and an embedder, that produces task-specific representations conditioned on both the original input and the reasoning trace generated by the reasoner. Fig. 2 compares the standard approach of directly encoding multimodal inputs for embedding (2a) with our proposed approach TTE (2b, 2c).
Formally, given multimodal input (V, T, [Ins]), we obtain its embedding h by:
h = Pooling [Ins], T, v)), & (v, (V, [Ins], T),
where In and vi are the reasoner MILM and its embedding-centric reasoning trace, respectively,
4.1 EMBEDDING-CENTIR REASONING
Here, we introduce the concept of Embedding- Centric Reasoning (ECR), ), a form of intermediate rea- soning tailored to improve embedding quality. Inspired by chain-of thain-of-thought (CoT) reasoning (Wei et al., 2023b) in complex problem solving, ECR are generative reasoning traces that explicitly sup- port the production of target embeddings. For example, for VQA query, the ECR represents the model's step-by-ste reasoning to understand the query, whereas for grounding query, it captures detailed description of the referred object along with its surrounding visual context.
Formally, for a batch of (g, t) pair, h, and h, are conditioned on v. the ECR & can be described as
vt E arg max log oh h
( hp)
In this work, we simplify the learning of ECR, by using manually designed task prompt and format to prompt reasoner MILM to generate ECR. We formulate ECR in the form of sthink>..<<ttNe
Final Reasoning, where the reasoner model first outputs some in- termediate CoT, then generates the final reasoning. The content for CoT and reasoning can vary in terms of task types. For instance, for QA-based query, the CoT the standard reasoning process; for simple embedding tasks such as visual document embedding. the CoT is detailed description of the visual input, and the reasoning is as the summary. While also possible to directly optimize reasoner MLLM for ECR generation using retrieval signals (e.g. RL-based finetuning), will leave these explorations for future work. By conditioning the embedder on these task-aligned reasoning traces, enable the model to construct more semantically aligned and task-aware embeddings.
Inference cost. We note that, CoT is known to have higher inference cost. the following subsec- tions, explore different approaches construct the reasoner g. that can lower the cost but still improve the performance significantly.
4.2
TTE WITH TEACHER REASONER
For our TTE framework, we employ powerful MLLM (e.g.. Owen2.5VL-72B) as the teacher rea- soner, while keeping the embedder model lightweight (Qwen2VL-2B 7B). We refer to this setup as TTE, We utilize the ECR reasoning traces generated by the reasoner for both training and evalu- ation of the embedder. Although this setup may seem computationally expensive, many real-world
4

---

Published as a conference paper at ICLR 2026
MLLM
MLLM
MLLM
Learned Latent
MLLM
Learned Latent
Hidden
Hidden
Block n-d
MHSA
Block n-d
Copy n-d
Learned
Cress
K K
K
K
Block
MHSA
Block n-1
Copy n-1
Query
Antn
V
V
V V
Block n
MHSA
Block
Copy
(a)
(b)
(c)
(d)
Figure 3: Embedding head designs: (a) Attention Pooling with learnable query. (b) -V-Embed-style (Lee et al., 2025) pooler. (c) Qformer-style embedding head, and (d) Embedding head with self attention blocks initialized from the hackhone MLLM. Green denotes trainable components. Q. K and V denote query, key, and value in attention mechanism. MHSA refers to MultiHead Self Attention, and refers to output embedding.
retrieval tasks only require a one-time offline inference step to generate ECR (e.g., detailed descrip- tions of visual documents) when constructing the retrieval index. Importantly, the reasoner is not involved in online retrieval for cxisting data points, and only needs to be run once for new data points. We empirically show that TTE, achieves superior performance on MMEB-v2 without requir- ing additional training data, advanced training techniques changes to the model architecture.
4.3 TTE WITH FINETUNED REASONER
While our TTE framework supports seamless integration with large MLLMs as the teacher reasoner, we are also interested in exploring whether comparable performance gains can be achieved with smaller reasoner - specifically, by using the same MLLM backbone as the embedder. As an initial study (Fig. 4), employ the backbone model itself to generate zero-shot ECRs, which are then used to train the embedder for retrieval. This setup results in noticeable performance gains across classification, retrieval, VQA, and grounding tasks.
Encouraged by these results, we further finetune a dedicated, student reasoner, initializing it from the same backbone the embedder (2B or 7B). We refer this setting TTE,. Specifically, finetune the small reasoner using ECR traces generated by the teacher reasoner (e.g., Qwen2.5VL- 72B). Given multimodal input (V, [Ins]. T), we maximize the conditional likelihood of the ECR &i by optimizing the standard negative log- likelihoooo (NLL) loss in standard LM finetuning:
LSFT(W)
log V, [Ins], T, vret).
4.4 TTE WITH UNIFIED REASONER AND EMBEDDER
The TTE framework described so far rely on separately trained reasoners and embedders, where the reasoner's ECR tokens are encoded together with the query to form embeddings. To reduce the computational overhead introduced by the additional reasoner, we explore unifying reasoning and embedding generation within a shared backbone, enabling embeddings be produced in single forward pass. We consider two approaches: (1) joint SFT-contrastive training of a shared reasoner-embedddr backbone, and (2) reasoner augmented with an embedding head. We begin with (1), training reasoning and embedding jointly in a multi-task learning setup. However, this configuration consistently resulted in performance degradation, due to the challenges in training curriculum. The implementation details and results for this approach are provided in Appendix B. For the rest of the section, we focus on design (2), as it offers greater flexibility in training control.
Reasoner with Embedding Heads. Our key design principle is that hidden states generated during reasoning can also be reused for downstream representation extraction. However, the two tasks should have specialized parameters so that the embedding training objective will not interfere with reasoning, and vice versa. Therefore, we adopt a two-stage training strategy for the unified model. In the first stage, we fully fine-tune the backbone second stage, we freeze the backbone and train only the embedding head on top of it. This design enables the unified model to produce embeddings in single forward pass, illustrated in Figure 2c.
reasoner the curated ECR dataset. In the
5

---

Published as a conference paper at ICLR 2026
We refer this setting is TTE,- As we decouple the contrastive and generative objective, there is no contrastive gradient flowing into the reasoning MLLM, and vice versa.
A Systematic Study of Pluggable Embedding Head. We present the first systematic study of embedding head designs that can be seamlessly plugged into a frozen backbone MLLM. This stands in contrast the focus of prior LLM-based embedding models (Lee et al., 2025; Lin et al., 2025; Jiang et al., 2024), which primarily rely on either full finetuning or parameter-efficient approaches based on LoRA (Hu et al., 2021). As illustrated in Fig. 3, we evaluate four embedding head designs: (1) Learnable attention queries over backbone hidden states (Fig. 3a); (2) Learnable latent context with backbone hidden states as queries; (3) A QFormer-style (Li et al., 2023a) embedding head patched onto the last layers of the backbone (Fig. 3c). (4) Repeat the last n layers of the backbone to formulate N new layers as the embedding head (Fig. 3d).
We present detailed experimental comparison of the four designs in Section 7.3.
5 EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Datasets. We conduct our experiments on both MMEB-V2 (Meng et al., 2025) and MMEB- VI (Jiang al., 2024) datasets. MMEB-VI comprises 20 in-distribution (IND) tasks and 16 out- of-distribution (OOD) tasks, spanning meta-tasks: classification, VQA, retrieval, and grounding. MMEB-V2 extends MMEB-VI by introducing additional video and visual document (visdoe) trieval tasks. Video tasks including video QA, classification, retrieval, and video moment retrieval. Visdoc involves supportive document retrieval, where the task is to retrieve supporting documents given the query. Overall, MMEB-V2 includes a total of 78 test tasks. We report NDCG@S for visdoc retrieval and Precision @ for image and video tasks (Meng et al., 2025).
Models. We employ Qwen2VL 2B and 7B as the backbone, and introduce three variants of TTE: TTE with large teacher reasoner (TTE,), TTE with small, SFT-ed student reasoner (TTE,). ). and TTE with a unified reasoner and embedder (TTE.). We compare our approach against several baselines, including dual-encoder methods such as CLIP (Radford al., 2021), SigI.IP (Zhai et al., 2023), UnilR (Wei et al., 2023a), and MagicLens (Zhang et al., 2024). We also directly compare with VLM2Vec-V1 (Jiang et al., 2024) and VLM2Vec-V V2 (Meng et al., 2025) with Qwen2VL as the backbone MLLM. Additionally, we evaluate against recent MLLM-based embedding approaches, including UniME (Gu et al., 2025), LLaVE (Lan et al., 2025), and B3 (Thirukovalluru et al., 2025).
Training Details. For both MMEB-V V1 and MMEB-V. V2, learning rate of 2e 4 for the backbone MLLM and 5e for the embedding head. The temperature for contrastive loss is set to 0.02. We train the backbone using LoRA (Hu et al., 2021) with rank 16 and alpha 64. Following VLM2Vec, we employ GradCache (Gao et al., 2021) to increase the per-device batch size. We train for 1 epoch on MMEB-V1, and 2.3 epochs for MMEB-V2. For MMEB-V2, we follow the same data weighing setup as in VLM22ec-V22 V2, and adopt its interleaved sampling strategy, where one global batch splitted into n sub batches, each from one dataset.
global batch size of 8192, with
For supervised finetuning of the ECR Reasoner, we perform full parameter finetuning of the MILM while keeping the visual encoder frozen, using a learning rate of 2e -5 a global batch size of 128. and training for an epoch. We use DeepSpeed (Aminabadi et al., 2022) ZeRO for optimizer offloading.
6 RESULTS
6.1 MAIN RESULTS
Results on MMEB-VI V1. Table compares the performance of our models with recent approaches on MMEB-VI. The encoder-based baselines are evaluated in zero-shot setting, while the remaining models are trained on the MMEB-V V1 training data. Our proposed approach TTE achieves the best performance in both the 2B and 7B categories, with TTE, outperforming the next best approach by 6%. Compared to VLM2Vec-V V1, TTE,, TTE,, and TTE, achieve substantial improvements of 7 7.5%, 7.4% and 12.7%. respectively, on the 7B embedding model.

---

Published as conference paper at ICLR 2026
Model
Backbone
Per Meta-Task Score
Average Score
Classification
Retrieval
Grounding
OOD
Overall
# of datasets
20
16
36
Encoder-Buid Baselines
CLIIP (Radfoord et 2021)
42.8
9.1
53.0
51.8
37.1
38.7
37.8
SigLIP (Zhai al., 2023)
40.3
8.4
31.6
59.5
32.3
38.0
34.8
UnilR (C) (Wei et al., 2023a)
44,3
16.2
61.8
65.3
47.1
41.7
44.7
MagicLens (Zhang
2024)
26.0
23.7
27.8
28 Model Size
VLM2Vec-V1 (Jiang et al., 2024)
Qwen2VL
59.0
49.4
65.4
73.4
66.0
52.6
59.3
UniME (Gu et al., 2025)
LLaVA-16
81.8
52.7
64.2
LLaVE (Lan et al., 2025)
Aquila: VL.
62.1
60.2
65.2
84.9
69.4
59.8
65.2
(Thirukovalluru et 2025)
QwenZV1.
79.9
(Ours) TTE,
Qwen2VL
67.6
62.7
70.7
80.0
71.4
65.2
68.7
(Ours) TTE.
Qwen2VL
69.7
71.4
78.4
71.6
65.1
68.8
(Ours) TTE,
Qwen2VL
72.6
74.3
72.6
85.2
80.5
67.0
74.5
~ 7B Model Size
VLM2Vec-V1 (Jiang al., 2024)
Qwea2VL
57.8
69.9
81.7
65.2
56.3
65.8
UniME (Gu allll.2000 2025)
LLaVA OV
66.8
66.6
70.5
90.9
74.6
65.8
70.7
LVvVE aI al., 2025)
M.aVA-OV
65.7
65.4
70.9
91.9
75.0
64.4
70.3
B3 (Thirukovalluru al., 2025)
Qwen2VL
70.0
66.5
84.6
67.1
72.0
QQMM (Xue et al., 2025)
L.aVA OV
69.9
70.0
72.1
86.0
77.2
66.6
72.5
(Ours) TTE,
Qwen2VL
68.9
73.3
(Our)) TTE.
Qwen2VL
71.6
72.0
72.5
82.1
76.7
68.7
73.2
(Ours) TTE,
Qwen2VL
77.4
78.6
75.9
89.0
82.7
73.3
78.5
Table I: Results on the MMEB-VI benchmark (Jiang et al., 2024). The scores are averaged per meta-task. Best/2nd-best performance across both 2B and 7B categories in bold/underlined.
Model
Image
Video
VisDoc
All
CLS
QA
RET
GD
Overall
CLS
QA
RET
MRET
Overall
VDRv1
VDRY2
VR
OOD
Overall
Close sourced Models or w/ Additional Duta
seed sedd-166embeee 16-embeddiing
76.1
74.0
77.9
91.3
77.8
25.0
615
51.3
53.5
55.3
89.5
50.8
$7.9
44,4
76,8
71.3
Ope MM-
MMMM-mmbeedin-----2
68.1
65.1
69.2
80.9
69.0
$3.6
55.7
41.8
33.7
47.6
67.0
$7.6
85.4
43.3
74.4
63.4
GME-28
72.7
Collali v1.3-B
40.3
11.5
48.1
40.3
34.9
26.7
378
21.6
25.5
28.2
83.6
$2.0
31.1
43.1
71.0
Tww.MMM-cmhedddneell.7
69.7
69.6
73.1
87.2
72.7
59.7
62.2
45.7
43.2
53.8
86.1
99.6
79.3
43.3
70.3
67.6
RocdEnbel--1-788
76.8
48.9
$7.9
76.8
GME 78
57.7
34.7
71.2
59.3
56.0
37.4
50.4
25.4
36.2
38.5
89.4
55.6
85.0
44,4
75.2
57.8
LamRA-Qwan2-78
37.4
LamRA-QQaa25-7
51.7
34.1
66.9
56.7
52.4
32.9
42.6
23.2
37.6
33.7
56.3
33.3
54.2
40.1
50.2
47.4
Models Trained MMRB
VLM2Ne--V2-0
36.5
(Ours) TTE, (28)
67.9
66.6
70.2
84.1
70.1
47.3
49.1
34.4
33.2
32.1
77.5
53.22
83.2
41.1
68.8
63.1
TIE,-2
73.9
(Ours) TTE, 29
26.6
26.8
71.5
97.2
26.1
56.1
65.2
$4.1
33.8
47.9
$1.1
62.4
34.7
43.2
72.6
58.6
VLMENeu-V2-7B
63.7
61.5
7000
85.2
68.1
45.9
33.9
27.6
39.3
36.4
78.8
52.6
82.7
42.1
69.3
61.2
ITE,
62.7
(Ours) TTC, 70
68.6
72.3
73.2
90.7
73.9
$1.2
59.5
36.0
38.2
47.1
83.5
63.8
91.5
66.8
80.6
68.2
Table Results on the MMEB-V benchmark (Meng al., 2025). Best/2nd-best performance across all mod- els are bold/underlined. Task abbreviations: CLS (classification), QA (question answering), RET (retrieval), GD (grounding), MRET (moment retrieval), VDR (ViDoRe), VR (VisRAG), and OOD (out-of-domain).
Results on MMEB-V2. Table 2 summarizes overall results on MMEB V2 dataset across the entire leaderboard, at the time of submission. TTE,-BB achieves highest overall score of 71.5% on the MMEB-V2 leaderboard, surpassing recent SOTA models trained with massive external data (e.g., seed-1.66 1.6-bbbeeee. Without relying on the teacher reasoner, TTE, achieves best performance across open-sourced models, for both 2B and 7B variants. Compared to baseline VLM2Vec-VV TTE shows substantial gain. For instance, TTE, and TTE, improve 2B baseline by 5.1% and 10.6%.
Taking closer look at the performance across modality and tasks, we can observe notable im- provement against VLM2Vec-VV baseline from VQA and classification-bri TTE,-7B improves video QA performance significantly against VLM2Vec- V2. As for retrieval, observe larger improvements on iideo-based retrieval (+5%) as compared to image-based retrieval (e.g. +2% on -TE,-2B), possibly due to based retrieval is more challenging than image-based, and therefore signifying the role of teacher-generated ECR.
tasks. For instance,

---

Published as a conference paper at ICLR 2026
w/ WFFF-ed Reusoner
Per Meta- Task Score
Average Score
w/ CoTT
Per Meta- Score
Average Score
CLS
VQA
Ret
GD
IND
OOD
All
CLS
VQA
Ret
GD
IND
0OD
All
Model
Buseline
$9.0
49,4
65.4
73.4
66.0
52.6
59.3
Baseline
59.0
49.4
65.4
73.4
66.0
52.6
59.3
73,9
58.7
71.5
73.6
72.6
67.6
62.7
70.7
80.0
71.4
65.2
68.7
72.6
73.9
73.1
85.2
80.5
67.0
74.5
7B Model
7B Model
Buseline
62.6
$7.8
69.9
81.7
65.2
56.3
65.8
Baseline
62.6
57.8
69.9
81.7
65.2
56.3
65.8
x
65.5
66.9
68.4
78.2
71.7
64.0
68.3
75.8
78.2
72.9
85.8
80.6
71.6
76.6
76.8
73.3
77.5
78.6
74.7
Table 3: Ablation on the effect of finetuning TTE, under MMEB-VI. w/ SFT-ed Reasoner denotes whether to apply SFT/zero-sho generation of ECR.
Table 4: Effect of CoT traces in ECR with TTE, We ablate the same of ECR but w/ CoT w/o CoT (only final reasoning).
7 ABLATIONS
7.1
UNDERSTANDING THE ROLE OF TTE REASONER AND ECR
Effect of zero-shot TTE reasoner. We begin by analyzing the impact of the model's inherent rea- soning capability to the retrieval performance. We first prompt the MLLM reasoner (Qwen2VL-2B) to generate ECR in zero-shot setting. The generated ECR is then combined with the raw query for embedder training using the same Qwen2VL-2B backbone. We conduct the ablation on a subset of datasets within MMEB VI, where constrastive training is performed individually for each dataset. Fig. 4 shows the corresponding results. We can observe that incorporating self-generated zero-shot ECR leads to notable performance improvements across all tasks.
Effect of finetuning TTE reasoner. While we find that the zero-shot reasoner can generally improve embedding per- formance, its effectiveness can still be bottlenecked by the quality of the gero-shot ECR. In Table 3, we present abla- tion results on finetuning the TTE reasoner on MMEB-VI, where we observe significant improvement after fine- tuning, with an absolute gain of over 6%.
HatefulMemes
A-OKVQA
VOC2W77
38,0
81.2
OK-VQA
SUN397
Effect of intermediate CoT in ECR. To examine the role of reasoning within ECR, we conduct an ablation by including or excluding intermediate CoT during training and testing, with results shown in Table 4. Overall, incor- porating CoT Ieads to consistent performance gains. We observe clear improvements and grounding, with only marginal gains on VQA- an expected outcome since VQA primarily requires the fi- nal answer, whereas retrieval and grounding benefit from richer intermediate context. Notably, the improvements are larger for 7B than 2B models, likely due to the stronger language capacity of larger models.
IaGVQA
NIGHTS
S2
71.5
7M3
VisualWW
MSCOCO_
classification, retrieval,
VisDial
Baseline
w/ Zero-Shot ECR
Figure with/without zero-shot ECR on MMEB-VI.
Baseline under Qwen2VL--B
Analysis on the role of ECR. We conducted more analysis to understand the role of ECR in Fig. 5.
A natural question is whether the ECR text alone is sufficient for retrieval, potentially eliminating the need train dedicated encoder. However, our experiments on MMEB show that this is not the case. Specifically, when we use the same ECR and apply a text-to-text (T2T) retrieval baseline- by directly encoding the ECR with strong off- off-the-shelf text encoder, Jina V3 (Sturua al., 2024) the performance is significantly worse.
Another concern is that imperfect ECRs might introduce noise that propagates into the Embedder, degrading retrieval performance. Interestingly, we find that our TTE Embedder is robust to such noise: learns extract useful information from the ECR without blindly relying on it. In fact, although ECRs alone do not yield high retrieval precision, they provide valuable signals that our Embedder can leverage. We observe similar trends across all task types. These findings indicate that Embedder is robust to noisy ECRs, and retrieval performance can benefit even from imperfect reasoning traces. Meanwhile, we also note that the observed robustness does not imply that the embedder is invariant to the quality of ECRs. In fact, the robustness range measure within the
8

---

Published as a conference paper at ICLR 2026
DI VigA-ied OKA-AA
On YQA-aad
(4) ClarElatio-ie
(4) ClannBaadie-edd
10 Haronn
Phan
PVDA
IM
citl FF8
9139
Diec
LOKYOA
BCRTHT
TTE.
VI.MVW VI
Figure 5: Results T2T evaluation generated ECR, versus TTE, and VL.2Vec-VV MMEB VI.
(a) VQA
(b) Classification
(c) Retrieval
(d) Grounding
Figure 6: (-SNE visualization (perplexity = 30) of query and target embeddings between baseline VLM2 7B (left) and ours (TTE, right). Zoom in for better visual effect.
same reasoner: the embedder outperforms directly using the ECRs themselves (e.g.. for VQA-style tasks where retrieval does not rely on dense visual embeddings). However, using different reasoner can still impact the embedder's performance, shown in Table 3 and between TTE, and TTE,
7.2 VISUALIZATION
In Fig. we show side-by-side 1-SNE visualization comparing embedding produced by the base- line VeM2 and TTE, on MMEB-VI For each task type, we select distinct subtasks. We clearly observe larger overlap between query and target embedding for TTE,, as compared to VLM2 Vec.
7.3 EMBEDDING HEAD DESIGN
Architecture Design. In Table 5, we investigate the design of embed- ding heads. We first explore simple (depth=1) trainable pooler, includ- ing an attention pooler with learn- able queries and an NV-Embed-style latent pooler with learnable context. However, both approaches perform significantly worse than TTE,, and increasing the number of queries (n_queries
Model
Per Tank Score
Average Score
IND
Overall
TTE.
67.5
62.7
70.7
80.
71.4
65.2
68.7
w/ Simple Artention Pooler
I.queries
45.4
50.5
47.1
n.queries
48.3
46.1
45.9
60.1
51.3
44.3
18.2
qucuiies 16
49.7
47.6
47.8
56.7
53.1
44.5
19.3
w/ NW..Eeeedddyyy Ponler
39.3
48.2
w/ QRomece-tty Emmbedding Head
I.layerslass.n 1/4
65.2
57.8
68.0
79.8
68.8
61.8
65.7
i.layers/ast..u 8/8
70.0
67.1
16) yields little improvement. We hypothesize that the hidden states from the final layer may not be optimal for capturing la- tent representations, as these layers
w/ Self Initialized MHSA
n. layers/asst n 4/4
67.0
59.3
70.2
78.5
712
62.2
67.2
n_Jayershass.n
69.7
71.4
78.4
71.6
6888
n.layers/last.u a 8/16
70.8
58.1
68.8
83.4
69.0
66.8
68.0
Table Embedding head ablations on MMEB-VI.
are primarily tuned for providing discriminative features for token predictions.
To better exploit representations from a frozen MLLM, we shift focus to earlier layers (e.g., the 8th-to-last) and equip them with more expressive multi-layer embedding heads. Specifically, we compare QFormer-style head with a self-initialized multi-head self-attention (MHSA) module. Empirically, the latter achieves superior performance compared to the QFormer-tyyl head, which requires training from scratch. When self-initialized with the last n_layers = 8, the MHSA head reaches performance par with separately fine-tuned embedder (TTE.).
The choice of last-n layer for embedding. Most prior work adopts the final hidden states of an MLLM as embeddings, but this design choice is rarely scrutinized. We hypothesize that the last layer may not be optimal for multimodal retrieval: while it is specialized for token-level discrimination, intermediate layers may retain richer semantic structure. To validate hypothesis, we ablate the

---

Published as a conference paper at ICLR 2026
choice of bottom-n layers under two settings: (1) Attention Pooling with frozen backbone, and (2)
Last Token, where LoRA is applied and the final token embedding is used.
We report overall performance MMEB vI in Table 6. Under attention pooling, performance improves modestly as n increases from to 8, then drops esis: when the backbone is frozen, intermediate layers can yield slightly more robust embeddings than the final layer. However, the gains are small, indicating that most of the improvement stems from the capacity of the embedding head itself (i.e., additional trainable parameters), rather than layer selection alone.
Last
Attn Pooling
LoRA+Last
47.1
68.7
16. This trend supports our hypoth-
68.4
49.6
67.2
16
59.5
Table 6: Effect of using bottom-n trans- former layers for embeclding on MMEB (Qwen2-Vi VL. 2B).
In contrast, the LoRA+Last Token setting exhibits the opposite trend-performance declines as n increases. Since LoRA injects trainable rank updates into each transformer block, selecting larger bottom-n effectively reduces the number of blocks available for adaptation, limiting the model's flexibility. Consequently, the final layer performs best under LoRA fine-tuning, whereas intermedi- ate layers can be slightly preferable when the backbone frozen.
7.4 IMPACT OF REASONING MLLM
A natural question is whether stronger teacher models (e.g., Gemini2.5 Pro (?)) further im- prove embedding quality through higher-quality reasoning traces. To study this, we run additional experiments on MMEB v2 using Qwen2-VL (7B) as the backbone embedder, while varying the teacher model that generates reasoning traces at evaluation time. Besides Qwen2.5-VLL (72B), we consider Qwen2.5-VL (32B) and Gemini2.5 a state-of-the MLLM for visual understanding. We focus on retrieval tasks, where the impact of teacher quality is more meaningful than in VQA.
Task
Qwen2.5-VLL 32B
Qwen2,5-VL 72B
Gemini2.5 Pro
i-ret
74.1
74,3
73.6
i-grounding
88.1
89.3
90.6
V-ret
35.8
37.6
41.8
v-moment ret
37.8
39.3
53.4
visdoc
76.3
76.8
76.1
Table 7: Effect of using varying capacity of reasoning models at evaluation time on MMEB v2.
We observe that Qwen2.5-VL (32B) performs comparably to Qwen2.5-VL (72B), suggesting that moderate increases in teacher capacity do not necessarily translate to improved retrieval perfor- mance. contrast, Gemini2.5 Pro achieves substantially higher accuracy on video and video- moment retrieval (up to +14 absolute), but performs slightly worse than Qwen2.5-VL. (72B) on image and visdoc retrieval. These trends can be explained by two factors: (1) Qwen2.5-VI. (72B) already achieves strong reasoning quality on image and visdoc tasks, but remains relatively weaker on video understanding, where a stronger teacher such as Gemini2.5 Pro offers greater benefit. (2) There is distribution shift between training traces (from Qwen2.5-VL) and test-time reasoning (from Gemini2.5 Pro), leading to mild degradation on image and visdoc performance. The video tasks, however, are unaffected because the video subsets MMEB v2 are largely -ittof--istiibutionn
CONCLUSION
(TTE), general framework for universal multimodal embedding that leverages a reasoner to "think" before predicting embeddings with an embedder. We first show that using a multimodal LLM as the reasoner can substantially boost the performance of smaller embedder, demonstrating that CoT-style reasoning also benefits multimodal representation lcarning. To improve efficiency, we then distill a compact reasoner from the reasoning traces of the large model. The distilled reasoner and the embedder can be trained from the same backbone, yielding both capacity and efficiency gains. Finally, we improve the integration of reasoning and embedding by introducing pluggable embedding head on top of the reasoner. This design enables embeddings to be produced in a single forward pass, further improving efficiency while halving model parame- the MMEB-V V1 and MMEB-V V2 benchmarks show that approach
We propose
ters. Extensive experiments significantly outperforms a range of baselines and recent methods without requiring additional data, validating the effectiveness and robustness of the proposed TTE framework.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
We reviewed the ICLR Code of Ethics carefully and do not observe potential concerns for our work.
REPRODUCIBILITY STATEMENT
We made our best efforts comprchensively document the implementation details. Training hyper- parameters and model architectures are discussed in Section 5.1 and B. We include the dataset con- struction details including all the example prompts we used in Section C. For evaluation, as men- tioned in Section 5.1, strictly follow the official setup with the codebase released by the original authors. We reveal the full results of each task on the benchmark, without average, Section D.
REFERENCES
Reza Yazdani Aminabadi, Samyam Rajbhandari, Minjia Zhang, Ammar Ahmad Awan, Cheng Li, Du Li, Elton Zheng, Jeff Rasley, Shaden Smith, Olatunji Ruwase, and Yuxiong He. Deepspeed inference: Enabling efficient inference of transformer models unprecedented scale, 2022. URL
https://arxiv. wrg/ alss/ 2207 00200000005
2tw..o.0022222222... 00032.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding. Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025a.
URL wwt.tttps/// / /aaa/aivvvvoooo org/ abwpa/a2..22... 13923.
Zechen Bai, Tianjun Xiao, Tong He, Pichao Wang, Zheng Zhang, Thomas Brox, and Mike Zheng Shou. Bridging information asymmetry in text-video retrieval: data-centric approach, 2025b.
URL https:tp////..c..... // /arxiv...v.. org/ abs/ /400. 07249.
Zhiyu Cao, Peifeng Li, and Qiaoming Zhu. Icr: Iterative clarification and rewriting for conversa- tional search, 2025. URL https: /arxiv.ivvv. org. abs /2509. 05100.
Haonan Chen, Hong Liu, Yuping Luo. Liang Wang, Nan Yang, Furu Wei, and Zhicheng Dou. Moca: Modality-aware continual pre-training makes better bidirectional multimodal embeddings, 2025a.
URL httpss://axxii...../e/
15.
Haonan Chen, Liang Wang, Nan Yang, Yutao Zhu, Ziliang Zhao, Furu Wei, and Zhicheng Dou. mme5: Improving multimodal multilingual embeddings via high-quality synthetic data, 2025b.
URL httttt//a/axiiii org/ abs/ 2200//5022022 08468.
Kaustubh D. Dhole, Ramraj Chandradevan, and Eugene Agichtein. Generative query reformulation using ensemble prompting, document fusion, and relevance feedback, 2024. URL https://
arxiv. orgiiv.orr/aaa abs 220024055. 17658.
Manuel Faysse, Hugues Sibille, Tony Wu, Bilel Omrani, Gautier Viaud, CELINE HUDELOT, and Pierre Colombo. Colpali: Efficient document retrieval with vision language models. In The Thirteenth International Conference Learning Representations, 2024.
Luyu Gao, Yunyi Zhang, Jiawei Han, and Jamie Callan, Scaling deep contrastive learning batch size org/ abs /2101. 06983.
under memory limited setup, 2021. URL https:
Tiancheng Gu, Kaicheng Yang, Ziyong Feng, Xingjun Wang, Yanzhao Zhang, Dingkun Long, Yingda Chen, Weidong Cai, and Jiankang Deng. Breaking the modality barrier: Universal embed- ding learning with multimodal 1lms, 2025. URL hwtphhtt//aaaa wttpvvv..rggbb/////aee/o
17432.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. URL https:
//arxiv. www.orrg/aaa abs 2106.06... 09685.
Ziyan Jiang, Rui Meng, Xinyi Yang, Semih Yavuz, Yingbo Zhou, and Wenhu Chen. VIm2vec: Training dision-langure models for massive multimodal embedding tasks. arXiv preprint arXiv:2410.08 08.4100.51660 160, 2024.
11

---

Published as a conference paper at ICLR 2026
Ivica Kostric and Krisztian Balog. A surprisingly simple yet effective qulti-query rewriting method for conversational passage retrieval, 2024, URL httph: arxiv.xvvvvv./... wwt.:/ogg/............... 18960.
Zhibin Lan, Liqiang Niu, Fandong Meng, Jie Zhou, and Jinsong Su. Llave: Large language and vision embedding models with hardness-weighter contrastive learning, 2025. URL https: www.ii.orggaaaaaaaaaaaaoaaaaaaae www.aa////25.00 04812..
Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catan- zaro, and Wei Ping. Nv-embed: Improved techniques for training Ilms as generalist embedding models, 2025. URL https:://aaaiiiiee wttp:////.............o
/ 17428.........
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023a. URL https: stxiv.org// abs http//bb////11999
Yanghao Li, Haoqi Fan, Ronghang Hu, Christoph Feichtenhofer, and Kaiming He. Scaling language-image pre-training via masking, 2023b. URL https: httpt:::///axxx abspgg/bb////// 2212. 00794.
Sheng- Chieh Lin, Chankyu Lee, Mohammad Shoeybi, Jimmy Lin, Bryan Catanzaro, and Wei Ping. Mm-embed: Universal multimodal retrieval with multimodal llms, 2025. URL https: 02571.
Xinbei Ma, Yeyun Gong, Pengeheng He, Hai Zhao. and Nan Duan. Query rewriting for retrieval- augmented large language models, 2023. URL https: //axivv.// org/ abs abs/2305 14283.
Rui Meng, Ziyan Jiang, Ye Liu, Mingyi Su, Xinyi Yang, Yuepeng Fu, Can Qin, Zeyuan Chen, Ran Xu, Caiming Xiong, Yingho Zhou, Wenhu Chen, and Semih Yavuz. V1m2vec-v2: Ad- vancing multimodal embedding for videos, images, and visual documents, 2025. URL https: // //arxiv..coo www.:.rggabbaa.aae 04590
Letian Peng, Yuwei Zhang, Zilong Wang. Jayanth Srinivasa, Gaowen Liu, Zihan Wang, and Jingho Shang, Answer is all you need: Instruction-followii text embedding via answering the question. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 459-477. 2024.
Bryan A. Plummer, Liwei Wang, Chris M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models, 2016. URL https:ttts/ec.....tt /ttp:i//axxvvvvi http:://aaxxvv.///a//a. /atg//bb// /105. 04870.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. URL https://arxiv. www. wttp://rgga/bb//////// 00020.
Saba Sturua, Isabelle Mohr, Mohammad Kalim Akram, Michael Günther, Bo Wang, Markus Krim- mel, Feng Wang, Georgios Mastrapas, Andreas Koukounas, Nan Wang, and Han Xiao. jina- embeddings-v3: Multilingual embeddings with task lora, 2024. URL tittps://arxiv... 10173.
Raghuveer Thirukovalluru, Rui Meng. Ye Liu, Karthikeyan K, Mingyi Su, Ping Nie, Semih Yavuz, Yingbo Zhou, Wenhu Chen, and Bhuwan Dhingra. Breaking the batch barrier (b3) of contrastive learning via smart batch mining, 2025. URL https: //a//aaxiivi org aaaaab///5500 250.. 11293.
Cong Wei, Yang Chen, Haonan Chen, Hexiang Hu, Ge Zhang, Jie Fu, Alan Ritter, and Wenhu Chen. Unii:: Training and benchmarking universal multimodal information retrievers, 2023a. URL https:://///........ //aaaaivv... org/ abs /tass///11... 17136.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia. Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023b. URL https:tt//// /ttsss///xxvvvoo 22001
2201 11..1.19110 11903.
12

---

Published as a conference paper at ICLR 2026
Raymond Wilson, Chase Carter, and Cole Graham. Contextualizing search queries in-context learn- ing for conversational rewriting with lmms, 2025. URL httph: arxia. wbtp://wwrrr//////aaaoasbsssssst abs//50s/2//2 15009.
Hu Xu, Saining Xie, Xiaoqing Ellen Tan, Yao Huang, Russell Howes, Vasu Sharma, Shang-Wen Li, Gargi Ghosh, Luke Zettlemoye, and Christoph Feichtenhofer. Demystifying clip data, 2024.
URL https: aariiv.iv.. org/ abs//309 16671.
Youze Xue, Dian Li, and Gang Liu. Improve multi-modal embedding learning via explicit hard 0200000000000000 02020.
negative gradient amplifying, 2025. URL
Fanghua Ye, Meng Fang, Shenghui Li, and Emine Yilmaz. Enhancing conversational search: Large language model-aided informative query rewriting, 2023. URL URR.tttps://axxii.orr abs/
2310.09716.
Hao Yu. Zhuokai Zhao, Shen Yan, Lukasz Korycki, Jianyu Wang, Baosheng He, Jiayi Liu, Lizhu Zhang, Xiangjun Fan, and Hanchao Yu. Cafe: Unifying representation and generation with contresiveeattooeegeesive finetuning, 2025. URL https://axxiv.. wttp://rggaabbbaab////.. 19900.
Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung. Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models, 2022. URL https: /arxiv. org/ erg/abs/ 2205. 01917.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training 2023. URL https: arxii.iiiiv//.i.///aaaaaaaaaaiva org/ abs /203. 15343.
Kai Zhang, Yi Luan, Hexiang Hu, Kenton Lee, Siyuan Qiao, Wenhu Chen, Yu Su, and Ming-Wa Wei Chang. Magiclens: Self-supervised image retrieval with open-ended instructions, 2024.
Junjie Zhou, Zheng Liu, Ze Liu, Shitao Xiao, Yueze Wang, Bo Zhao, Chen Jason Zhang, Defu Lian, and Yongping Xiong. Megapairs: Massive data synthesis for universal multimodal retrieval, 2024.
URL https: / aariiiiiivv.////vv///aiaaa/aaaia org/ abs 2at./2211... 14475.
Changtai Zhu, Siyin Wang, Ruijun Feng, Kai Song, and Xipeng Qiu. Convsearch-rl: Enhancing query reformulation for conversational search with reasoning via reinforcement learning, 2025.
URL https:://///// / /aaxivvivvv./////////aa////aaavl org/ 2b55 15776
13

---

Published as a conference paper at ICLR 2026
Flickr30k-121
Flickr30k-T21
MSCOCO
Model
R@
R@5
R@1
R@5
12T
T2I
2B Models
CLIP(ViT-Bigg/114)
92.9
79.6
67.3
51.3
UniME
88.2
77.0
66.8
49.8
B3++
94.9
82.8
73.6
59.0
VLM2Vec
89.1
98.7
68.8
90.4
68.6
71.5
(Ours) TTE,
94.8
99.5
83.6
95.2
75.5
74.8
(Ours) TTE,
95.2
99.5
84.2
95.5
77.7
78.0
Models
EVA-CLIP
94.5
80.3
70.1
52.0
UniME (Llava-Next)
93.4
81.9
70.1
53.7
B3 7B
95.9
85.5
77.6
62.8
VLM2Vec
94.6
99.5
80.3
95.0
73.5
78.2
(Ours) TTE.
95.8
99.6
86.3
97.6
76.8
80.7
(Ours) TTE,
96.5
99.6
86.8
98.3
78.5
82.0
Table 8: Results on 12T/T2I retrieval on MSCOCO and zero-shot Flickr30K dataset. We report R@ and R@5 for Flickr30k, and R@] for MSCOCO.
Per Meta-Task Score
Average Score
Model w/ Noisy CL ECR
CLS
VQA
Ret
IND
OOD
All
2B
Model Size
TTE,
X
66.8
62.1
69.5
78.6
70.6
64.1
67.7
TTE.
67.6
62.7
71.4
65.2
68.7
TTE, X
71.5
72.1
72.3
82.4
79.4
65.3
73.1
TTE.
72.6
74.3
72.6
85.2
80.5
67.0
74.5
7B Model Size
TTE.
69.3
70.1
72.3
67.6
71.9
TTE.
70.5
71.5
73,5
83.9
76.8
68.9
73.3
TTE.
76.1
76.0
74.5
80.8
71.3
76.6
TTE,
77.4
78.6
75.9
89.0
82.7
73.3
78.5
Table 9: Ablation on the MMEB V1 with or without noisy ECR data for contrastive training.
A ADDITIONAL RESULTS AND ANALYSIS
A.I RESULTS ON I2T/T2I RETRIEVAL.
We evaluate zero-shot 12T and T2I retrieval on Flickr30k (Plummer et al., 2016) dataset and show the results for MSCOCO retrieval. The results are provided in Tab. 8. We can see both TTE, and TTE, surpass baseline VLM2Vec and the recently proposed B3 (Thirukovalluru et al., 2025) by large margin. For instance. on MSCOCO T2I, the 7B TTE, achieves 82.0 Precision @ 1, surpasses B3 (62.8) by almost 20% absolute.
A.2 CONSTRUCTING ECR DATASET
We construct two separate high-quality ECR for contrastive training for TTE Embedder, and Su- pervised Finetuning (SFT) for TTE Reasoner, using powerful MLLMs such as Qwen2.5-72B (Bai al.. 2025a). For each task in МMEB, we manually design prompts that take both query and the ground-truth target. For contrastive training, we aim to blend in a certain amount of "noise" in the training dataset improve robustness of TTE Embedder against incorrect ECR generation. To prevent label leakage and overfitting, explicitly instruct the MLLM rewrite the answers while performing reasoning. Additionally, we only keep 50% of the ground-trutte truth in the training data. For test set, adopt similar prompts but without the ground-truth target. The process for SFT ECR dataset is similar, except we do not ask the MILM to rewrite the ground -ruun,-truth, as we want the TTE Reasoner leam to output the exact ground-truth truth accurate possible.
In Tab. 9 we show ablation study on whether to apply noisy ECR dataset construction method (i.e. with rephrasing and 50% non-ground-truth ECR) for contrastive training. We can observe that by
14

---

Published as a conference paper at ICLR 2026
MMEB v1
Qwen2-VL
Qwen2.5-VL
InternVL3
V2.
59.3
58.7
57.4
TTE.
74.5
74.1
73.6
Table 10: Ablation on different MLLM backbones on MMEB VI.
Model
Per Meta-Task Score
Average Score
CLS
VQA
Ret
GD
IND
0OD
Baseline
57.6
47.5
65.0
71.6
61.5
55.1
58.9
SFT:CL
50.7
43.2
59.5
67.1
55.6
48.3
52.3
SP
100
52.4
48.3
61.5
68.
56.6
51.9
56.2
Table 11: Ablation the MMEB VI with joint SFT+CL training, using Qwen2 2B. Baseline denotes VLM2Vec VI with Qwen2
applying noisy ECR data construction for contrastive training, we can obtain 1-2% performance gain across both TTE, and TTE,, for both model size.
A.3 IMPACT ON DIFFERENT MLLM BACKBONES
While we conduct main experiments Qwen2VL, following previous works (Jiang et al., 2024; Thirukovalluru et al., 2025), we additionally evaluated Qwen2.5-VL and InternVL3 on MMEB vl, which we used internally during ablation and model selection. Table 10 summarizes the results.
Across all backbones, TTE, consistently provides substantial improvements over the VLM2Vec baseline, demonstrating that method generalizes well beyond single family of MLLMs. How- ever, we observe that Qwen2-VL. remains the groogest-performing models, despite having lower reported accuracy downstream visual understanding and reasoning benchmarks. This trend aligns with findings reported by the original MMEB authors, who similarly noted that substituting Qwen2-VL with Qwen2.5-VL did not yield performance gains."
backbone among the tested
B JOINT CONTRASTIVE-AUTORGGRSSS
TRAINING
Model and implementation details Here explore unifying text generation with embedding, by jointly optimizing for both contrastive and autoregressive objective:
Cjoint ALInfONCE + LSFT
where À is Syper-parametee controlling the weight of contrastive (InfoNCE) loss. During training the second last token the embedding token, since it is the last token generated during inference. We apply a simple MLP block on top of the token to obtain the embedding.
Training details. We the same set of hyper- ppram-parmeeters as used in baseline contrastive training: LoRA with rank and alpha equals to 16 and 64, learning rate of 2e -4 a global batch size of 8192, and train for epoch.
We provide results Tab. 11. We can see incorporating SFT objective leads to performance degra- dation against baseline. As increase À (reducing impact of SFT), the performance is partially recovered. We conjecture that there may exist conflicting gradients between contrastive and SFT objectives. It could also be due the expressiveness of the last layer's hidden states as embedding: the last layer in MLLMs may not be optimal for embedding as they are heavily guided to produce discriminative features for token classification.
See GitHub discussion: http://ww..copp//gggt/o//o/////e
66.
15

---

Published as a conference paper at ICLR 2026
explats step-ty sap
darived
Pleaee
calos.
Gives an image and cuciiies ouhiit by-s how azeves be derived foom the
Keep fie reaseiigg concise and gounied vinual Extual evicence. Keep Asscciaci within
image Pless follow the helow niles
1-3 serceces.
Keep the reesoning costise and gicended visual or fuoual eviloce Key sucinctt, willie
Whop you ressonng in it-ikkek and -itiiko tags. Thes, the cest fine, cxtpst the fixal
sestanes.
seanct darting "Aaawer."
corper think-thee bicede woman's bairs tid bock single hanch, which chomatirante of
Whng your newening
ligs
procited groondonahh
vorff your menoiig
"Naswat". Dollow the formatin ecample
mxriion
explasation.
Rephraes final anewor thet
eoet monning of
differ
worting or phanoing The nenome, or alter dettall conuett
Eooeple: QUESTION: What the teinayle the Hond Mord colled?
Follow the formur enample below.
praytai chhinks pony
naample
hainsyts
-chiscc the blande woran's ned back into bunch, valich charaderieic back nur (perynall; vithesic Anewer rid back
islowing
IMAGE: simage?
New sives dhe folloning, image and quections:
QUISTIUN: (quar)
Hose forral providita
IMAGE: sirage-
Joueyy'
Please fillow the same format exanpple stove, porkling youroossiiiinggooo.. finll arower.
Figure 7: Prompt template for generating ECR data using teacher MILMs (egg. InterVVL3 78B) VQA tasks. Left: prompt template without ground-truth. Right: prompt template with ground-truth and rephrasing.
Film image, deserb triefly how CDCDsayl ception can be formedt fiver the imuge follon the intmetions bekon.
Gives disoriie COCO captiae Sured
Fullow the instuction Ielow.
Fins provile deseriptios image. douile COCO-ayl captias citint docald
Fisst povide denileed descrptioe the imege, then desottee what coco-eyle co00-eyleooiii oppine sboaed contem, (tiize: stbenid ferue the moer elient ohjocm and their amangement Imags)
Wapp conaming in -tlikke...oiiaeeee -othieeee santeree).
Taza wrie dee final COCO coco-ayle cuptias on mas. line Asaver Sanwwrr-
nonsoning othinle, iottikk semtnno))
are prosided with the grond COCO caption for refreece Use to veiffy your rensoring mentio expliiiiy your oxglanation.
waie final CDCD--eylc the Arawoen Saneaerr
the
worting phrnsing Do rdd,
fectual
EXAMPLE
The imnge
bed, beddlects,
EXAMPER
GROUND-TRUTTH CAPTION: Several pillows are lied up down the teugth
aleg the bad The COC Duclle canton should ceraa the mon salern objoct and arangement: de ailoieetliikee lawea Severd pilcoss ur lixel up dova the lough o''a bod.
image
nighecand light
bege pillowe amanged
sling hot The cOD.- capricn dwaid conzain salieet and smrngrmeen the pilloos vithene millows-itiiiee Anewar A few pillowe are aminged along the longeraste aide of the had.
fellowing image
cinagee
Now ansvecs the following image: IAAGE: simsge
Figure 8: Prompt template for generating ECR data using teacher MLLMs for i2t tasks.
16

---

Published as a conference paper at ICLR 2026
Giver cpptionn descthing in eoreyly scere reo your horkgnurd loaneledge tosoiingo
a ciptton the gool desrihe wher the segen trage may look ception and you bockgrordd exoorkeige You are given the tinget trege for your refrrnce Howooorg you your taddground
inqued the captor mone conerro and vieual drants, Fellow ndle holow:
messenng
scmerably
eriemal frvm
commonoree and stoal
Weap cosoning mikzkk..siiiili.. Then output final description the oenorpnding image the nest lin wsing Anewan samanee
Zaraci image SimagePhgle
isumatione
Use only the optioe and gassel world knowleleee pooditt likely visual featurs.
make detala ixplied by the cuption. soncie, spsciiic you
EXAMPLE
shove.
Ganasate ssaciact reasesing proces sesteszo) slous lhe visua information that
<hh exption playfilly deserbos vally hear stop. urgenting them ore wally hear form ploce phocd the skup
rndered from the cagtion, ssing coneronsennt and stoul prices.
nse deocativlly "Moer guard nolly" liiely refors lage hea figare
Wayg your consoning -licko., "iikk... Then the fixal description
sugand henging upperstory adllng dog''s wheriiaal vithank Answer
poesitty
gurd "noighor
cemorposting image
sonn
aodly
lokly har drowad nyal pand canding or the retrancs, and snother soddy hor playfilly hanging frrm squtore
EXAMPLE
CATTION mon is riddingg arthord on wove
wiscon
-tisk-A man wearing seranvec is bakandnn on suffeard and secas vards lisely is heads
sas sctrng, with entabing around him
CAPTTON: (eaery)
CAPTION: (manyy)
Figure 9: Prompt template for generating ECR data using teacher MLLMs for 12v tasks.
C PROMPT TEMPLATES AND ECR SAMPLES
We provide sample prompts for prompting the teacher MILMS (e.g. Intern VL3 78B) to generate ECR. Examples are shown in Fig, 8, Fig. 9 and Fig. 7. We also provide sample generated ECR in Tab. 12, Tab. and Tab. 14.
17

---

Published as a conference paper at ICLR 2026
Dataset
Input Text
Input Image
Embedding Centric Reasoning
<hink? The image mainly shows hand plane with metallic body and wooden han- dles, tool used for smouthing or shuping wood, </think>>> Answer: hand plane
Given an image, identify the main object category belongs to. Let's think step step.
ImageNet-IBe IK
Given an image and its ussociated news text, identify the main domain of the news. Lot's think step step.
< <hinkk> The image shows two individuals in theater setting, and the caption mentions they are actors who met at graduate acting
N24News
News text: Danai Gurira An- drew Holland theater New York University. where they mel the gram.
program which is related to the performing arts, </hinkk Answe. Arts & Entertainment
Acting
Given un image, identify the main object category helongs Let's think step by step.
<think > The image mainly shows car with racing stripes and branding, identifying as automobile. <tmmobilc.<hhiikk.. Answer:
VOC2007
<think>> The image shows a cute squirrel on top tree trunk. The text on the im- age is: "you're special kinda aren't you?" The text uses humor and wordplay ref- erencing nuts and squirrels, which harm- less. There's indication of harm offense intended. </think > Answer: No
Given on image, determine if con- tains hateful speech. Let's think step by step.
HatefulMemes
Table 12: Examples of ECRs for Classification datasets in MMEB V1 henchmark.
Dataset
Input Text
Input Image
ECR
Given the image and the below question, answer the question based on the image.
<think>> man standing by hags with luggage carl street, nario for someone waiting for transporta- suggests awailing cab <./t///hkkk Answer. cab
think step
A-OKVQA
step.
Question: What the man by the bags awaiting?
Given the image and the below question, answer the question based on the image. Let's think step by step.
think>> The Favorable line is the green 2015, the Favorable value is 38. BB.ckhhkk Answer: Yes
CharQA
Question: Is the value of Favorable 38 in 2015?
Given the image and the below question, answer the question based on the image. Let's think step by step.
<think> The image shows double decker bus with a large white square on its side words "FOX'S clearly displays bald. black let- GINGER BISCUITS"
Visual7W
Question: What is written on the
ters, </think Answer: FOX'S GINGER BISCUITS
square
Table 13: Examples of ECRs for VQA datasets in MMEB V1 benchmark.
D FULL RESULTS
We present the full results on MMEB V2 (Tiang et al., 2024) in Tab. 15. For VLM2Vee 2B baseline, use the officially reported number from VLM2Vec V2 (Meng et al., 2025). For VLM2Vec 7B, since it not reported, we use our reproduced version under our training hyper-parameters.
18

---

Published as a conference paper at ICLR 2026
Input Text
Input Image
ECR
< <think> The image shows man with a mus- purple
tache wearing badge his jacket. He appears to be in formal setting, possibly outside a boild- ing. His expression somber, suggesting serious official event. Based on his appear- ance and the context, this is likely depic- of prominent political figure nificant moment. moment.</hinnk <CAhink Answer: Turkish President Recep Tayyip Erdogan is seen dur- ing visit formal event.
Given nows image, first generate a concise informative description of the news image, then generate summarization hased on the de- scription. Let's think step by step.
VisualNews.i2r
sig-
The image shows close-up of sil- ver Barber half obverse minted in 1913. The coin features profile of Lady Liberty fac- ing right, adorned with laurel wreath ribbon tied behind the neck. Above the por- trait, the words "IN GOD WE TRUST are in- scribed, and thirteen stars encircle the top and left side of Lady Liberty. The year "1913" is prominently displayed below neck. The design intricate. with fine details visible on the hair and leaves. </think>> Answer: The image shows 1913-D Barher half obverse featuring Lady Liberty with laurel wreath and the inscription "IN GOD WE TRUST."
think
Given an wiki image and short description, first generate de- lailed deseription of the inage and text, then generate a summariza- based the description. Let's think step by Text: 1913-D Barber obverse.
WebQA
<think> The caption descrihes skate- boarder who has launched ofl' concrete sur- Given mid position, boarder likely performing trick such an ollic or kickflip. The skateboarder is in executing trick,
Given COCO style caption, first use your knowledge to expand it more detailed and concise de- then
scription of the target generate summarization based on the description. step.
MSCOCO.RP r2i
erete background suggests an urban skate park or street setting. <cAhink >> Answer: skate- boarder captured mid- air, performing dar- ing trick off concrete ramp in urban skate park. surrounded by smooth pavement and scattered wheels
think step
Caption: skatehoarder mid following a jump form cincrete.
The image shows a black and white photograph of small white dog. The dog is standing or walking on light-colored sur- face, surrounded by darker background. has round, fluffy appearance and is wear- ing a collar. The dog's features are some- what blurred, making difficult fine details beyond its general shape and pos- ture. </hhink>> Answer: A small white dog standing on light
think>>
Given an image and query, iden- the object region the image that the query refers to, and gener- ate eoncise description ject region. Let's think step stop.
RefCOCO (Grounding)
oh-
Query: vehicle second-closest camera
Table 14: Examples of ECRs for Retrieval datasets in MMEB VI benchmark.
19

---

Published as a conference paper at ICLR 2026
LamRA Qwen2.5
VLMENE
VE.M
GME-2B
GME-7B
TT2,28
TTE,-2
TTE,-B
TTE,7B
44.4
$4.1
57.8
47.4
68.6
54.9
52.4
Video
Visdee tasks,
72.7
(CLS:(0)
40.3
5644
51.7
76.6
1-QA
11.5
29.9
34.7
34.1
76.5
78.6
1-FFT
1-V0
55.5
59.3
56.7
87.3
VACIS (5)
52.9
42.0
42.6
65.3
V-MR
25.5
37.0
$7.3
33.8
90.3
VD-Vidum-V1
89.4
VD-VisRAG (2)
ImageNa-1K
42.4
58.9
85..1
N24News
Momer
50.6
51.3
VOC70007
75.9
87.66
Ia-A
Na-R
839
77.0
ObjectNer
45.6
72.7
26.5
VQA
39.9
ALOOVVA
75.0
29.5
41.4
95.6
Infographics/OOA
23.7
ChariQA
VisulWW
24.6
70.7
SelenceQA
27.6
39.0
$5.5
CQA
8.3
75.1
76.9
64.9
27.1
41.2
60.3
H1
52
VisnlNoww.22
74.7
79.7
72
74.0
79.4
VisilNoww.21
47.6
78.3
74-2
76.5
76.6
99.2
71.2
65.7
25.7
MSCDCD.12
49.9
63.1
57.7
71.1
72.6
71A
21
72.2
73.1
NIGHTS
65.5
67.0
67.6
66.5
68.6
20
68.0
70.3
WeNQA
51.8
91
85.7
90.6
90.4
91.1
FadionlO
5.9
32.9
37.8
33.4
24.9
19.5
26.3
28.9
29.7
30.2
Wikk SS NQ
80.5
73.9
78.2
67.0
72.3
66.9
68.2
689
71.2
OVEN
50.0
72.3
75.1
84.8
67.0
64.3
67.9
72.7
69.4
BOSS
64.7
91.8
96.0
78.7
$4.1
92.8
95.3
MSC OCO
36.7
28.6
31.4
36.0
73.0
67.1
67.7
92.5
74.1
$4.9
RefC OCO
64.5
35.9
60.9
57.1
93.1
87.1
91.4
99.4
97.7
92.1
RefC Masching
1.9
73.3
78.4
82.6
93.1
85.8
90.3
96.3
91.6
VisualPW-Paniing
56.1
64.1
66.5
51.2
$1.7
69.2
H2.5
86.5
94.3
58.4
K300
23.4
35.2
39.7
32.1
53.6
38.0
48.2
$5.0
55.6
SimhSmma22
25.1
29.9
30/6
25.3
46.6
42.8
39.4
55.3
HMDB81
24.8
45.4
47.
33.8
43.1
40.9
63.9
UCF101
49.4
3244
5A.7
$3.0
66.9
6000
58.3
75.5
64.2
78.6
Breakfust
10.9
13.6
14.3
20.1
14.8
25.4
35.2
29.7
34,2
MVBencc
33.7
37.5
46.6
37.6
32.5
33.7
88.3
62.0
65.3
MMle--MEE
30.6
34,3
39.2
35.1
33.1
30.7
45.8
58-9
53.1
62.1
NENTQA
35.2
29.5
$5.6
44.9
21.0
2019
$3.3
74.00
70.1
73.6
Egoöchemaa
34.4
40.3
46.3
47.0
34.4
34.0
36.4
58.2
35.6
62.8
NclIv
51.3
38,0
65.6
45.5
48.0
32.3
60.8
75.6
04.6
77.1
DiDaMo
22.8
22.0
26.4
22.8
31.6
304
33.5
34.1
34.9
36.3
MSR-VTT VTT
17.6
27:3
31.8
25.0
31.6
28.3
34.3
36.6
37.6
39.5
MSVD
43.4
47.6
49.7
41.9
46.7
48.1
56.5
37.1
5N.5
59.4
VATEX
16.7
23.0
24.9
18.7
19.0
26.5
25.6
26-4
31.0
32.6
YooCook2
5.3
7.9
0.1
7.5
9.2
10.6
15.8
16.3
19.9
20.3
OVHighhight
19.9
43.6
$9.5
60.9
58.3
49.4
38.9
40.3
$1.0
52.7
Charado-STA
29.0
14.9
14.0
18.8
19.3
2012
19.5
21.4
18.9
23.0
MommenSeekerr
27.8
34.8
97.4
$1.8
40.6
40.8
37.7
39.6
41.5
42.2
ViDeRe assinga
81.7
82.8
86.9
$3.0
$1.5
80.6
H0.7
81.8
$4.6
53.4
ViDoRe docrqa
56.6
$3.1
57.5
25,4
45.7
41.9
44.5
40.7
46.0
45.8
ViDoRe infouqa
$4.9
90.2
91.6
72.3
36.4
45.7
84.8
93.4
38.7
$7.6
96.9
03.1
9466
66.1
91.4
89.2
88.4
90.
04.7
92.0
ViDX iandge
20.9
69.9
74.1
25.9
50.5
45.8
50.4
$1.8
59.4
$6.1
ViDeRe slalppojjct
75.1
89.5
96.3
27.3
71.5
600
65.2
79.3
$1.6
$1.8
mob=kk_srtffiiiiiim Jmelligence
95.7
97.5
99.6
72.0
91.0
88.5
91.9
97.3
08
96.2
94.7
91.9
95.1
65.2
96.8
86.5
88.7
93.4
03.5
96.2
ViDXRR ViDx:Re_gewermmee gewerrmeemn jeports
91.6
94.6
98.
72.2
$7.5
85.0
86.9
95.3
06.7
97.7
ViDoee healthcare. indussy
95.9
98.7
99.3
6338
95.1
92.2
92.8
97.9
97.9
96.5
ViDeke reggrteprts.uii human Jabeled.x2
$1.3
61.0
63.4
$3.0
55.7
45.6
$9.0
64.9
69
70.9
ViDeRe, hwweedieileeeeeeeee Jeetire, (amms/2_mmlmen mmmiiinign
$4.7
$4.0
49.5
35.9
$4.0
44.3
$2.0
$5.0
60.8
62.8
40.0
50.2
$4.1
$1.9
$1.9
43.0
49
S3.6
60.4
$6.3
ViEoRee reports 92 muhilligull
52.9
30.7
$5.4
32.5
48.5
45.6
$2.1
65.8
60.3
64.3
ViSRAG VisRAG.ArivAA ArtivQA
90.9
82.0
87.4
57.7
79.5
76.9
76.5
86.0
94.5
92.4
ViRRA_CCsmO2
78.2
79.9
81.9
65.9
$2.9
$4.1
86-4
85.2
91.2
95,0
SERAG.MP.LL De:VQA
56.8
84.4
89.2
$4.5
$1.5
71.8
79.2
80.0
90.1
$7.0
ViSAG SHLEVQA
95.0
93.4
98.5
76.5
91.3
91.5
92.3
93.4
95.6
94.9
ViSRAG IntoVQA
63.7
91.4
93.5
73.3
$9.8
35.7
87.2
87.8
93.0
922
ViSRAC PlaQAA
60.3
64.1
63.4
41.2
71.5
66.1
77.5
77.8
$6.9
$7.1
ViDoSeck--age
22.2
21.6
23.1
23.1
21.7
21.9
22.6
22.9
35.0
45.2
ViDoSeee--do
83.7
83.6
8339
80.3
82.2
8012
92.0
83.2
$4.4
645
MIMLongBench page
14.2
15.8
16.2
13.5
14.8
11.9
12.9
17.8
20.7
22.0
MMI oMgLongBencha Bencleoke
53.5
51.4
543
43.
49.7
45.7
47.0
48.5
50.4
50.6
Table 15: Full results MMEB V2.
20


Reviews:
[{"review_id": "Rmxu5EKxt6", "reviewer": "Reviewer_xF66", "summary": "This paper introduces Think-Then-Embed (TTE), a universal multimodal embedding framework that uses a reasoner to \"think\" before generating embeddings with an embedder. Leveraging a multimodal large language model (MLLM) as the reasoner significantly enhances the performance of smaller embedders, showing that chain-of-thought (CoT) reasoning benefits multimodal representation learning. To boost efficiency, we distill a compact reasoner from the large model’s reasoning traces, allowing both the reasoner and embedder to share the same backbone for improved capacity and speed. Additionally, a pluggable embedding head on the reasoner enables single-pass embedding generation, reducing parameters and further increasing efficiency. Extensive experiments on MMEB benchmarks demonstrate that TTE outperforms existing methods without extra data, confirming its effectiveness and robustness.", "questions": "* The grounding task score does not show an advantage compared to LLaVA-OV-based models in Table 1, even though Qwen2VL is claimed to have better grounding ability.\n* Why does the embedder use Qwen2VL-7B? Is it superior to Qwen2.5-VL-7B?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The method is simple yet effective. Involving cot process in retrieval shows significant performance gains on benchmarks. The author designed various experiments and methods to demonstrate this, and the experimental results performed well on the MMEB benchmarks.", "weaknesses": "* Some terms should be unified: ‘Fig.’ in line 62 and ‘Figure 2’ in line 177 should be consistent.\n\n* I recommend that the author distinguish between MLLM and LLM. For example, ‘Qwen2.5-VL-72B’ in line 84 and ‘Qwen2.5-72B’ in line \n212 are confusing. These names should be unified in both the introduction and the methods section.\n\n* Figure 5 looks strange; the results on different benchmarks do not show temporal relationships clearly. A bar chart or radar chart would be more appropriate.\n\n* The grounding task score does not show an advantage compared to LLaVA-OV-based models in Table 1, even though Qwen2VL is claimed to have better grounding ability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sosUqIMNmM", "reviewer": "Reviewer_r1rn", "summary": "This paper proposes Think-Then-Embed (TTE), a novel framework for multimodal reasoning that introduces an explicit generative embedding step between reasoning and representation. Instead of directly encoding multimodal inputs (e.g., image-text pairs) into a single joint embedding, the method first conducts latent reasoning generation through an autoregressive module and then projects the reasoning trace into a compact embedding space via a learned encoder. This “reasoning-before-embedding” paradigm aims to enhance reasoning fidelity and interpretability while maintaining embedding efficiency. Experiments are conducted on multiple multimodal benchmarks (e.g., ScienceQA, OK-VQA, and MM-CoT-style datasets). Results show consistent improvements over baseline embedding methods such as CLIP, BLIP-2, and reasoning-augmented contrastive approaches. The authors claim that TTE achieves comparable or better performance with smaller embedding dimensionality and provides interpretable intermediate reasoning traces.", "questions": "- The “contrastive generative embedding” objective is vaguely described. It is unclear whether gradients flow through the reasoning generator (is it frozen or partially fine-tuned?).\n- The distinction between “reasoning trace” and “explanation text” is blurred—are they generated with the same prompt template or via different decoding strategies?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The writing is mostly clear and logically structured; the introduction and motivation are strong. Figures and pipeline diagrams are helpful in explaining the “think → embed” flow.\n- The overall framework is technically coherent and easy to reproduce: the authors provide clear training and inference procedures.\n- The formulation of generative embeddings based on explicit reasoning traces is intuitive and connects language modeling with representation learning in a principled way.", "weaknesses": "- Based on Tables 1 and 2, $\\text{TTE} _\\text{T}$ performs significantly better than the other two paradigms. However, it is unclear whether this advantage stems from the use of a stronger teacher model. Did the authors conduct any ablation studies using different teacher models to isolate this effect?\n- If reasoning traces are generated by a large pretrained LLM (e.g., GPT-4V or Qwen2-VL), it is unclear whether they contain test-relevant information from pretraining corpora, which may bias the results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MA4W5wybi7", "reviewer": "Reviewer_JwkF", "summary": "This paper introduces a Universal Multimodal Embedding framework, Think-Then-Embed(TTE), which first generates reasoning before producing embeddings. TTE exploits the generative reasoning ability of MLLMs to enhance embedding quality. Specifically, a powerful MLLM serves as a Teacher Reasoner to produce high-quality CoT traces (embedding-centric reasoning in this paper), guiding a smaller embedder to learn more discriminative representations. To reduce reliance on large models, the authors further distill a compact MLLM reasoner from the teacher and train a unified model that functions simultaneously as a reasoner and an embedder. Experiments show that TTE achieves state-of-the-art results on MMEB-V1 and MMEB-V2 benchmarks. Ablation studies confirm the effectiveness and stability of ECR  for complex instruction embedding.", "questions": "1. From the comparison between TTE_t and TTE_s, as well as the ablation results before and after fine-tuning TTE_s, it seems that the quality of ECR plays a crucial role in the embedder’s performance. However, the paper claims that the embedder is stable even with imperfect ECR, which appears somewhat contradictory. Could the authors provide a more detailed explanation or analysis to clarify this point?\n2. The MMEB-V2 benchmark results do not include TTE_u. Since the results on MMEB-V1 suggest that TTE_u performs similarly to TTE_s, including such data would make the paper more complete and convincing.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper provides a clear motivation and detailed explanation of the proposed method, with well-defined formulations for both the task and the training process.\n2. Based on the intuition for retrieval tasks that require reasoning, as well as the strong results of TTE_t, the proposed Think-Then-Embed (TTE) framework appears highly effective.", "weaknesses": "1. The analysis of ECR stability is only described qualitatively and lacks experimental support.\n2. The MMEB-V2 benchmark seems to be missing some evaluation data. \n\nPlease refer to the Questions section for details.\n\nThings to improve the paper that did not impact the score:\n1. In Table 1, there are several mistakes in the use of bold and underline to highlight the first and second-best results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YKLdBJsFyW", "reviewer": "Reviewer_vhdV", "summary": "This paper investigates using multimodal CoT traces to improve multimodal embedding for retrieval tasks. The paper explores diverse training settings, including using separate or unified models to combine CoT and embedding, and the design choices of a good embedding head to extract information from CoT latents. Through comprehensive experiments, the paper demonstrates the effectiveness of CoT for generating multmodal embedding compared to both decoder-encoder and multimodal llm based methods.", "questions": "* It would be interesting to see whether using traces from more powerful reasoning models, like GPT-o3, can further improve the embedder performance.\n\n* Does the performance scale with the size of the MLLM? And does a better MLLM, measured by standard benchmarks like MMMU, results in better embedding?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "* The paper shows that using CoT traces improves multimodal retrieval without using additional data.\n\n* The paper comprehensively examines different settings to incoporate the CoT traces, including separate reasoning models, distilled reasoning models, and unified reasoning-embedding models. This highlights the flexibility of apply CoT for better multimodal embedding.\n\n* The ablations clearly demonstrate the effectiveness of incoporating CoT to improve multimodal embeddings by comparing with baselines without CoT.", "weaknesses": "* The paper's method is not specifically tied to multimodal, or vision-language. The same approach can be applied to text-only retrieval. It would be great to clarify on how multimodal is important here, or demonstrate the method on text-only datasets.\n\n* The paper only used Qwen-VL series as the base models, but the method should be generalizable to most MLLMs. It would be great to show results on other base models.\n\n* While the authors hyphosize that the last hidden states might not be optimal, it is not properly justified in the paper. The improved results from Q-Former or self-initialized could be from the extra trainable parameters. A deeper analysis, like comparing the performance of each layer, would strengthen the design choices of the embedder head.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)