Hugging Face's logo

tencent
/
EVIE-8B

Visual Document Retrieval
Safetensors
sentence-transformers
colpali-engine
qwen3_5
vision-language
colbert
late-interaction
multi-vector
vidore
document-retrieval
multimodal
Model card Files
xet
 Community
1

🏆 EVIE-8B: The Most Accurate Visual Document Retriever

Evidence-Vector-Informed Embedding (EVIE-8B Flagship Teacher)

ViDoRe V3 Rank 1 ViDoRe V1+V2 Rank 1

License GitHub Hugging Face 8B Hugging Face 4.5B Backbone Token Dim

4096-Dimensional Token Representation • Bidirectional Full-Attention • Teacher for EVIE-4.5B Prefix-MRL

🤗 EVIE-8B (Flagship Teacher)  •  🤗 EVIE-4.5B (Prefix-MRL & HAC)  •  🐙 GitHub: Tencent/EVIE

📢 Release Announcement: All model weights, inference pipelines, and evaluation suites have been fully open-sourced. Full technical details, architectural ablations, and the formal research paper will be updated in an upcoming release. Training code and distillation recipes are available in the Tencent/EVIE repository.

🌟 Highlights

  • Highest Retrieval Accuracy: 66.75 nDCG@10 on ViDoRe V3, delivering industry-leading visual document retrieval accuracy.
  • High-Capacity 4096D Representations: Full per-token multi-vector embeddings preserving fine-grained layout, typography, charts, and table structures.
  • Teacher Foundation: Provides capacity-aware relation and margin distillation targets for the lightweight EVIE-4.5B Prefix-MRL model.
  • Multi-Benchmark 138-Task Coverage: Thoroughly validated across 138 tasks (ViDoRe V1, V2, V3, and JinaVDR) across 4 standard metric families (nDCG, Recall, MAP, MRR @1/5/10).

🧠 Architecture & Technical Highlights 

 Query Text  ────────► ColQwen3.5 (BiDir Attention) ────► 4096D Per-Token Embeddings
                                                                    │
                                                           MaxSim Matching
                                                                    │
 Doc Image   ────────► ColQwen3.5 (Vision Encoder)  ────► Full Multi-Vector Patch Embeddings
  • Late-Interaction Token Embeddings: Rather than compressing visual documents into an information-lossy single vector, EVIE-8B preserves spatial nuances, typography, and tabular relations with per-token late-interaction representations. Relevance is scored using MaxSim:

S(Q,D)=∑i=1∣Q∣max⁡j=1∣D∣(qi⋅dj) S(Q, D) = \sum_{i=1}^{|Q|} \max_{j=1}^{|D|} (q_i \cdot d_j)

  • Full Bidirectional Attention: Employs bidirectional self-attention across multimodal vision-text sequences to facilitate cross-modal interaction between visual patch tokens and prompt contexts.
  • Foundational Distillation Teacher: Serves as the high-capacity anchor model for training the compact EVIE-4.5B student via topological relation transfer and hard-negative margin supervision (ARD).

📊 Comprehensive ViDoRe Leaderboard Comparison

Performance comparison across modern multi-vector late-interaction visual document retrievers on ViDoRe:

Rank Model Base Model Param Embed Dim ViDoRe V1 (nDCG@5) ViDoRe V2 (nDCG@5) ViDoRe V3 (nDCG@10)
🥇 EVIE-8B Qwen3.5-9B 8.41B 4096D 92.18 74.23 66.75
🥈 EVIE-4.5B Qwen3.5-4B 4.61B 64–2048D Prefix-MRL 92.07 73.38 66.02
🥉 EVIE-Preview-4.5B Qwen3.5-4B 4.54B 128D 91.73 70.87 65.36
4 webAI-ColVec1.1-8b Qwen2.5-VL 8.40B 640D 91.30 65.82 65.32
5 VultronRetrieverPrime-8B Qwen3.5-9B 8.40B 320D 92.08 68.18 64.26
6 webAI-ColVec1.1-4b Qwen2.5-VL 4.54B 640D 90.49 63.60 63.90
7 VultronRetrieverCore-4.5B Qwen3.5-4B 4.50B 320D 92.21 66.12 63.57
8 nemotron-colembed-vl-8b-v2 Nemotron-8B 8.80B 4096D 92.65 65.16 63.54
9 tomoro-colqwen3-embed-8b Qwen2.5-VL 8.00B 320D 90.76 65.40 61.60
10 nemotron-colembed-vl-4b-v2 Nemotron-4B 4.80B 2560D 91.62 64.49 61.42
11 athrael-soju/colqwen3.5-4.5B-v3 Qwen3.5-4B 4.60B 128D 91.54 64.25 61.46
12 tomoro-colqwen3-embed-4b Qwen2.5-VL 4.00B 320D 90.57 64.69 60.16
13 VultronRetrieverFlash-0.8B Qwen3.5-0.8B 0.85B 320D 88.15 60.36 56.16

🔍 ViDoRe V3 Per-Domain Breakdown (nDCG@10)

Model Avg CompSci Energy Finance EN Finance FR HR Industrial Pharma Physics
EVIE-8B 66.75 81.86 72.51 71.23 56.40 69.29 59.77 70.81 52.11
EVIE-4.5B 66.02 81.72 72.32 70.00 54.90 67.82 59.40 70.27 51.69
webAI-ColVec1.1-8b 65.32 80.08 70.12 71.90 54.87 68.55 57.65 67.88 51.50
nemotron-colembed-vl-8b-v2 63.54 79.30 69.82 67.29 51.54 66.32 56.03 67.19 50.84
VultronRetrieverPrime-8B 64.26 79.80 70.30 69.00 54.50 66.80 57.40 68.20 51.70
VultronRetrieverCore-4.5B 63.57 79.80 69.20 68.90 52.00 66.10 56.10 67.50 50.20
tomoro-colqwen3-embed-8b 61.60 75.35 68.41 65.08 49.10 63.98 54.41 66.36 50.13

📋 Complete 138-Task Evaluation Matrix

Protocol paired-all-pages-dedup+process_queries+ndcg2r-20260827 ($\text{MVT} = 1024$, bidirectional attention):

  • ViDoRe V1: 10 tasks, 3,943 queries, 6,729 docs
  • ViDoRe V2: 4 tasks, 1,152 queries, 4,544 docs
  • ViDoRe V3: 48 tasks, 14,514 queries, 115,512 docs
  • JinaVDR: 76 tasks, 43,375 queries, 55,398 docs
Metric ViDoRe V1 ViDoRe V2 ViDoRe V3 JinaVDR 4-Board Macro Avg
nDCG@1 88.51 75.58 62.14 75.50 75.43
nDCG@5 92.18 74.23 64.16 82.45 78.26
nDCG@10 92.64 75.35 66.75 83.30 79.51
Recall@1 88.51 37.87 30.79 75.50 58.17
Recall@5 95.07 66.65 58.69 88.07 77.12
Recall@10 96.43 76.34 70.10 90.65 83.38
MAP@1 88.51 75.99 65.94 75.50 76.49
MAP@5 91.21 67.30 55.82 80.56 73.72
MAP@10 91.40 66.81 56.32 80.92 73.86
MRR@1 88.51 75.99 65.94 75.50 76.49
MRR@5 91.21 82.85 75.39 80.56 82.50
MRR@10 91.40 83.23 76.09 80.92 82.91

⚡ Quick Start

ColPali Engine

Installation 

git clone https://github.com/Tencent/EVIE.git
cd EVIE
pip install -r requirements.txt
export PYTHONPATH="$(pwd)/colpali${PYTHONPATH:+:$PYTHONPATH}"

Python Inference 

import torch
from PIL import Image
from colpali_engine.models import ColQwen3_5, ColQwen3_5Processor

model_id = "tencent/EVIE-8B"

# 1. Load model with FlashAttention and bidirectional attention
model = ColQwen3_5.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="cuda",
    attn_implementation="flash_attention_2",
).eval()
model.enable_bidirectional_attention()

# 2. Process query and document image
processor = ColQwen3_5Processor.from_pretrained(model_id)
images = [Image.open("examples/demo/pages/q3_revenue.png").convert("RGB")]
queries = ["What is the total quarterly revenue?"]

image_batch = processor.process_images(images).to(model.device)
query_batch = processor.process_queries(queries).to(model.device)

# 3. Generate 4096D multi-vectors and late-interaction score
with torch.inference_mode():
    image_embeddings = model(**image_batch)
    model.rope_deltas = None  # Reset RoPE deltas before text query forward
    query_embeddings = model(**query_batch)

scores = processor.score(query_embeddings, image_embeddings)
print("Late-interaction MaxSim Relevance Score:", scores)

Sentence Transformers

Load EVIE with Sentence Transformers to encode queries and document images and compute MaxSim scores. Bidirectional attention is configured automatically.

pip install -U "sentence-transformers[image]>=6.0.0"

from sentence_transformers import MultiVectorEncoder

model = MultiVectorEncoder("tencent/EVIE-8B")

queries = [
    "What is the variable represented on the y-axis of the graph?",
    "Total outlay is maximum in which year?",
]
documents = [
    "https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc1.jpg",
    "https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc2.jpg",
    "https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc3.jpg",
    "https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc4.jpg",
]

query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents, batch_size=1)
print(query_embeddings[0].shape, document_embeddings[0].shape)
# torch.Size([23, 4096]) torch.Size([3161, 4096])

scores = model.similarity(query_embeddings, document_embeddings)
print(scores)
# tensor([[14.3594,  6.1758,  4.5137,  3.2979],
#         [ 1.8079, 10.9180,  1.9668,  1.9541]])

Documents can be URLs, local image paths, or PIL.Image objects. The output contains 4096-dimensional token embeddings. Scores can vary slightly with dtype and attention backend.

The default page budget allows up to 16,384 visual tokens. Encoding images one at a time reduces peak memory use. To use the 1,024-token budget from the evaluation protocol above, set the image processor's pixel budget:

model = MultiVectorEncoder(
    "tencent/EVIE-8B",
    processor_kwargs={"size": {"longest_edge": 1024 * 32 * 32, "shortest_edge": 65536}},
)

📂 Repository Layout 

Evie-8B/
├── model.safetensors         # EVIE-8B weights (safetensors)
├── config.json               # Model configuration (4096D)
├── 1_Dense/                  # Sentence Transformers 4096D projection
├── modules.json              # MultiVectorEncoder module graph
├── infer.py                  # Standalone inference & scoring CLI
├── colpali/                  # ColQwen3.5 inference engine
├── code/shared/              # 138-task eval harness
├── examples/demo/            # 8-page retrieval demo (run.sh)
└── env.sh.example            # Environment variables template

🔬 Released Checkpoint

EVIE-8B was trained on 775,635 document-query pairs across multilingual domains. Hard negatives were mined and classified into answerable (promoted to positives), ambiguous (masked), and strict negatives (negative cross-entropy loss).

The released a40 checkpoint represents a weight-space mix ($\alpha = 0.40$) of two independently trained arms, achieving the peak 66.75 nDCG@10 on ViDoRe V3. Full training reproduction scripts are provided in the companion repository Tencent/EVIE.

📚 Citation 

@misc{tencent2026evie8b,
  title        = {EVIE-8B: High-Capacity Visual Document Retrieval with 4096-Dimensional Embeddings},
  author       = {Wang, Zifei and Wen, Wei},
  year         = {2026},
  howpublished = {\url{https://github.com/Tencent/EVIE}},
  note         = {Corresponding author: Wei Wen <jawnrwen@tencent.com>}
}

📄 License

This model and codebase are released under the Apache-2.0 License.

👥 Authors

Name Role Affiliation Contact
Zifei Wang Author Tencent IMA Product Center zifeiwang@tencent.com
Wei Wen Author & Corresponding Tencent Youtu Lab jawnrwen@tencent.com
Downloads last month
-
Safetensors
Model size
8B params
Tensor type
BF16
·
Inference Providers NEW
Visual Document Retrieval
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support