Ovis-VL-Embedding-9B

Ovis-VL-Embedding-9B is a high-capacity vision-language embedding model for text, images, visual documents, video, and interleaved multimodal inputs. It maps every supported input type into one coherent representation space, enabling high-accuracy cross-modal retrieval with a single encoder.

The model is initialized from Qwen3.5-9B. It retains the native text and vision encoders together with the shared multimodal language backbone, removes the language-modeling head, and directly uses the final-layer hidden state at the last non-padding token as the retrieval embedding. No modality-specific projection head is added.

Ovis-VL-Embedding-9B is designed for high-quality multimodal search, multimodal RAG, image and visual-document retrieval, video search and temporal localization, recommendation, and nearest-neighbor matching.

Model highlights

• High-capacity vision-language encoder: Provides strong text, image, document, and video representations with a 9B-scale native multimodal backbone.
• Unified multimodal retrieval: Queries and candidates may contain text, visual inputs, or interleaved text-image and text-video combinations.
• One shared embedding interface: All supported inputs are represented through last-token pooling and compared in the same cosine-similarity space.
• Dynamic-resolution visual processing: The native vision encoder handles images and sampled video frames at flexible resolutions while preserving spatial and temporal positions.
• Difficulty-aware contrastive learning: Focal embedding loss emphasizes unresolved queries with competitive negatives.
• Fine-grained embedding distillation: Teacher similarity distributions preserve relative relevance across both positive and negative candidates.
• Homogeneous-source finetuning: Task-consistent micro-batches provide informative in-batch negatives and reduce shortcuts based on modality or data format.
• Strong temporal scaling: Compared with the 2B variant, the 9B model gains most strongly on video question answering, video retrieval, and video moment retrieval.

Architecture

Figure 2 from the technical report. Ovis-VL-Embedding-9B encodes text, images, visual documents, and sampled video frames as one interleaved sequence. The final-layer hidden state at the last non-padding token becomes the retrieval embedding, with no modality-specific projection heads.

The Qwen3.5-9B backbone contains 32 language layers with hidden size 4096. Its hybrid stack repeats three Gated DeltaNet layers followed by one gated full-attention layer, combining efficient long-context processing with periodic global token interaction. Multimodal positional encoding preserves temporal and two-dimensional spatial coordinates for visual tokens.

Training

Training follows three stages:

1. Multimodal contrastive pretraining. Large-scale, multi-task data establish broad alignment across text, image, document, and video inputs. The objective combines difficulty-aware focal contrastive learning with similarity-distribution distillation.
2. Full-parameter homogeneous finetuning. High-quality downstream data refine fine-grained discrimination. Each micro-batch is drawn from one dataset so that gathered candidates form task-consistent negatives.
3. Annealing Embedding Distillation. Teacher-correct examples are retained, unresolved student examples are emphasized, and confidence-adaptive forward-KL supervision transfers complementary expert capabilities.

Retrieval interface

Ovis-VL-Embedding-9B is a bi-encoder, not a cross-encoder:

1. Pair the query with the task instruction and format it with the native processor and chat template.
2. Encode queries and candidates independently.
3. Extract the final-layer hidden state at the last non-padding token.
4. L2-normalize the 4096-dimensional query and candidate embeddings.
5. Rank candidates by cosine similarity, equivalently the dot product of the normalized vectors.

No answer generation or query-candidate cross-attention is used during retrieval. Classification labels, passages, images, documents, videos, and interleaved multimodal items are all treated as candidates in the same embedding space.

Performance

MMEB-v2

MMEB-v2 evaluates vision-language embeddings over 78 datasets spanning image, video, and visual-document tasks. Ovis-VL-Embedding-9B achieves 81.13 overall, outperforming the strongest compared baseline by 1.04 points.

Group	Ovis-VL-Embedding-9B	Best compared baseline	Result
Image	83.96	81.86	+2.10
Video	72.90	75.95	-3.05
Visual document	83.06	82.38	+0.68
All 78 datasets	81.13	80.09	+1.04

The model ranks first on all four image sub-tasks, video classification, video moment retrieval, the visual-document aggregate, and ViDoRe-V1. Scaling from 2B to 9B improves the overall score by 3.67 points, with the largest gains on video question answering (+7.64), video moment retrieval (+7.23), and video retrieval (+5.74).

Scores are percentages and higher is better. Red marks the best result in each row, underlining marks the second best, and Ovis scores are bold. The overall score is the unweighted average over all 78 MMEB-v2 datasets.

Embedding dimensions

The native output width is 4096, inherited directly from the Qwen3.5-9B backbone because no embedding projection head is added. Queries and candidates must use the same preprocessing, pooling rule, dimensionality, and L2 normalization.

Intended use

The model is intended for embedding extraction and retrieval over supported unimodal or interleaved multimodal content, including:

• high-accuracy semantic and cross-modal search;
• text-to-image, image-to-text, and image-to-image retrieval;
• multimodal RAG indexing and retrieval;
• visual-document and page retrieval;
• text-to-video, video retrieval, and temporal localization;
• recommendation and nearest-neighbor matching.

Limitations

• This checkpoint does not natively support audio input. Use Ovis-Embedding-Omni-3B for audio and general omni-modal retrieval.
• This checkpoint produces retrieval embeddings; it is not intended as a text or image generation model.
• Retrieval quality depends on task-appropriate query instructions and the native preprocessing and chat template.
• Performance varies by task. The reported model trails the strongest specialist baseline on the aggregate video score, video question answering, general video retrieval, ViDoRe-V2, VisRAG, and VisDoc-OOD.
• The 4096-dimensional output and 9B-scale backbone require more memory, storage, and inference compute than the 2B variant.
• Benchmark scores may not directly predict performance on a new domain. Evaluate with representative queries, candidates, and retrieval metrics before deployment.

License

This model is released under the Apache 2.0 license.

Resources

• Ovis-VL-Embedding GitHub repository
• Ovis-Omni-Embedding GitHub repository

This repository contains the weights for Ovis-VL-Embedding-9B.