VietRAG-Embed

VietRAG-Embed is a Vietnamese dense-retrieval model fine-tuned from intfloat/multilingual-e5-base. It maps queries and passages to normalized 768-dimensional vectors for semantic search, question answering retrieval, and Retrieval-Augmented Generation (RAG).

The model was trained on 937,686 cleaned Vietnamese query–positive–hard-negative records, with an emphasis on web passage QA and scientific retrieval.

Website: minhlab.ai.vn · Source code: VietRAG-Embed-E5-Base

Model details

Property	Value
Architecture	XLM-RoBERTa / Sentence Transformers
Base model	intfloat/multilingual-e5-base
Primary language	Vietnamese
Embedding dimension	768
Maximum sequence length	512 tokens
Pooling	Mean pooling
Output normalization	L2 normalization
Similarity	Cosine similarity
Parameters	278,043,648

Intended use

VietRAG-Embed is intended for:

• Vietnamese document and passage retrieval
• Semantic search
• FAQ and knowledge-base retrieval
• Retrieval for extractive QA or generative RAG systems
• Candidate generation before a cross-encoder reranker
• Query-to-passage similarity

This is an embedding model, not a generative language model. It retrieves relevant text but does not generate answers by itself.

Required E5 prefixes

Inputs must use the prefixes employed during training:

• Queries: query:
• Documents or passages: passage:

Omitting these prefixes can reduce retrieval quality. Apply the prefix before tokenization and keep the convention identical when indexing and searching.

Quick start

pip install -U sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("nhminh107/VietRAG-Embed")

queries = [
    "query: Thủ đô của Việt Nam là gì?",
]
passages = [
    "passage: Hà Nội là thủ đô của nước Cộng hòa xã hội chủ nghĩa Việt Nam.",
    "passage: Nước đóng băng ở 0 độ C trong điều kiện áp suất tiêu chuẩn.",
]

query_embeddings = model.encode(
    queries,
    normalize_embeddings=True,
    convert_to_tensor=True,
)
passage_embeddings = model.encode(
    passages,
    normalize_embeddings=True,
    convert_to_tensor=True,
)

scores = model.similarity(query_embeddings, passage_embeddings)
print(scores)

Because embeddings are normalized, cosine similarity is equivalent to the inner product.

FAISS retrieval example

import faiss
import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("nhminh107/VietRAG-Embed")

documents = [
    "Hà Nội là thủ đô của Việt Nam.",
    "Nước đóng băng ở 0 độ C trong điều kiện áp suất tiêu chuẩn.",
    "Sao Mộc là hành tinh lớn nhất trong Hệ Mặt Trời.",
]

document_embeddings = model.encode(
    [f"passage: {text}" for text in documents],
    normalize_embeddings=True,
    convert_to_numpy=True,
).astype(np.float32)

index = faiss.IndexFlatIP(document_embeddings.shape[1])
index.add(document_embeddings)

query_embedding = model.encode(
    ["query: Thủ đô của Việt Nam là gì?"],
    normalize_embeddings=True,
    convert_to_numpy=True,
).astype(np.float32)

scores, indices = index.search(query_embedding, k=2)
for score, index_id in zip(scores[0], indices[0]):
    print(f"{score:.4f}  {documents[index_id]}")

For large collections, encode passages in batches and use an approximate FAISS index or another vector database. Store the mapping between vector IDs and document metadata separately.

Benchmark results

mMARCO-VI shard holdout 04–05

This in-domain retrieval diagnostic contains 4,599 held-out queries and 7,949 candidate passages sampled from mMARCO-VI shards 04–05. Exact normalized query overlap with the training export was removed.

Metric	Score
Recall@1	0.8419
Recall@5	0.9533
Recall@10	0.9698
MRR@10	0.8907
nDCG@10	0.9102

The median rank of the relevant passage was 1. This benchmark is query-held-out but corpus-shared: 215 passages, or 2.70% of the evaluation corpus, have exact normalized matches in the training corpus. The result should therefore be treated as an in-domain diagnostic rather than a fully independent external benchmark.

Selected VN-MTEB retrieval results

Task	Main retrieval score
SciFact-VN	0.6130
TRECCOVID-VN	0.6068
Quora-VN	0.5652

VN-MTEB results are reported separately from the custom mMARCO-VI benchmark. Evaluation was run on an NVIDIA Tesla T4 using E5 query and passage prefixes.

Reproduction code and detailed benchmark artifacts are available in the project repository.

Training data

The training pipeline uses the following Vietnamese data sources:

Dataset	Main role
minhnguyent546/mmarco-vietnamese-split	Web passage QA
hotchpotch/mmarco-hard-negatives-reranker-score	Reranker-scored hard-negative candidates
nhminh107/VietEmbed-RAG-Science	Science and technical retrieval
vietgpt/wikipedia_vi	Vietnamese encyclopedic source corpus

The final cleaned supervised retrieval export contains 937,686 query–positive–hard-negative records: 869,119 filtered Vietnamese mMARCO records and 68,567 scientific retrieval records. Wikipedia-VI is used as an additional Vietnamese source corpus in the broader data preparation pipeline.

The pipeline applies text normalization, length and distinctness checks, duplicate control, source tracking, and hard-negative quality filtering.

Benchmark data was not used for model selection. The custom mMARCO diagnostic uses later source shards and rejects every exact normalized query found in the training export.

Training procedure

The model was trained with Sentence Transformers and MultipleNegativesRankingLoss.

Setting	Value
Objective	Multiple Negatives Ranking Loss
Similarity in loss	Cosine similarity
Loss scale	20.0
Batch size	24
Effective passes over the data	2
Optimization steps	78,142
Learning rate	5e-6
Warmup ratio	0.1
Precision	FP16
Seed	42
Batch sampler	No-duplicates hashed
Multi-dataset sampler	Proportional

Training used a single visible NVIDIA Tesla T4 on Kaggle and took approximately 11.2 hours. Query inputs were prefixed with query: ; positive and hard-negative passages were prefixed with passage: .

Architecture

SentenceTransformer(
  Transformer: XLMRobertaModel
  Pooling: mean pooling, dimension 768
  Normalize: L2 normalization
)

Limitations and responsible use

• The model is optimized primarily for Vietnamese retrieval. Performance on other languages or mixed-language queries has not been established here.
• Training data is dominated by web passage QA, so retrieval quality can vary across specialized domains and document styles.
• Inputs longer than 512 tokens are truncated. Long documents should be split into semantically coherent passages before indexing.
• Dense retrieval can return topically similar but factually incorrect passages. Production RAG systems should retain citations, apply access controls, and evaluate answer grounding.
• Web-derived and translated training text may contain factual errors, outdated information, or social biases inherited from its sources.
• The benchmark scores above do not guarantee performance on a private corpus. Evaluate with representative queries and relevance judgments before deployment.

Do not use embedding similarity as the sole basis for high-impact medical, legal, financial, employment, or safety decisions.

Framework versions

Training environment:

• Python 3.12.13
• PyTorch 2.10.0 + CUDA 12.8
• Sentence Transformers 5.4.1
• Transformers 5.0.0
• Datasets 5.0.0
• Tokenizers 0.22.2

The exported Sentence Transformers model was also verified locally with a fully offline load and inference smoke test.

License

The model repository is released under the MIT License. Users are responsible for reviewing and complying with the licenses and terms of the upstream base model and datasets for their use case.

Citation

@misc{vietrag_embed_2026,
  author       = {nhminh107},
  title        = {VietRAG-Embed: Vietnamese Dense Retrieval Embeddings for RAG},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/nhminh107/VietRAG-Embed}}
}

Acknowledgements

VietRAG-Embed builds on intfloat/multilingual-e5-base, Sentence Transformers, Hugging Face Transformers, MTEB, and FAISS.