multilingual-e5-large-int8-ov

intfloat/multilingual-e5-large with int8 weights (1024-dimensional embeddings, ~100 languages).

This is an unofficial conversion. It is not made, reviewed or endorsed by the authors of intfloat/multilingual-e5-large. All credit for the model belongs to them; please cite their work (below). Converted with optimum-intel (optimum-cli export openvino, 8-bit weight compression) and the OpenVINO tokenizer converter; no fine-tuning or other change to the model.

• Licence: mit, as upstream (intfloat/multilingual-e5-large).

Use with RustedVINO

hf download jermalk/multilingual-e5-large-int8-ov --local-dir /opt/rustedvino/models/multilingual-e5-large-int8-ov
rv add multilingual-e5-large-int8-ov --kind embedding

Measured on one Intel Core Ultra 7 laptop with the model forced onto the CPU: about 11 texts/s in batches of 16, 1.0 GB of RAM in use (CPU). Treat figures as orders of magnitude (they moved up to 25% between runs).

e5 needs prefixes: start search questions with query: and the texts being searched with passage: . Without them retrieval quality drops silently. Maximum input is 512 tokens including special tokens.

Citation

If you find our paper or models helpful, please consider cite as follows:

@article{wang2024multilingual,
  title={Multilingual E5 Text Embeddings: A Technical Report},
  author={Wang, Liang and Yang, Nan and Huang, Xiaolong and Yang, Linjun and Majumder, Rangan and Wei, Furu},
  journal={arXiv preprint arXiv:2402.05672},
  year={2024}
}