---
pipeline_tag: sentence-similarity
tags:
  - sentence-similarity
  - sentence-transformers
license: mit
language:
  - multilingual
  - af
  - am
  - ar
  - as
  - az
  - be
  - bg
  - bn
  - br
  - bs
  - ca
  - cs
  - cy
  - da
  - de
  - el
  - en
  - eo
  - es
  - et
  - eu
  - fa
  - fi
  - fr
  - fy
  - ga
  - gd
  - gl
  - gu
  - ha
  - he
  - hi
  - hr
  - hu
  - hy
  - id
  - is
  - it
  - ja
  - jv
  - ka
  - kk
  - km
  - kn
  - ko
  - ku
  - ky
  - la
  - lo
  - lt
  - lv
  - mg
  - mk
  - ml
  - mn
  - mr
  - ms
  - my
  - ne
  - nl
  - no
  - om
  - or
  - pa
  - pl
  - ps
  - pt
  - ro
  - ru
  - sa
  - sd
  - si
  - sk
  - sl
  - so
  - sq
  - sr
  - su
  - sv
  - sw
  - ta
  - te
  - th
  - tl
  - tr
  - ug
  - uk
  - ur
  - uz
  - vi
  - xh
  - yi
  - zh
---

A quantized version of [multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small). Quantization was performed per-layer under the same conditions as our ELSERv2 model, as described [here](https://www.elastic.co/search-labs/blog/articles/introducing-elser-v2-part-1#quantization).

[Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://arxiv.org/pdf/2212.03533.pdf).
Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei, arXiv 2022

## Benchmarks

We performed a number of small benchmarks to assess both the changes in quality as well as inference latency against the baseline original model.

### Quality

Measuring NDCG@10 using the dev split of the MIRACL datasets for select languages, we see mostly a marginal change in quality of the quantized model.

| | de | yo| ru | ar | es | th |
| --- | --- | ---| --- | --- | --- | --- |
| multilingual-e5-small | 0.75862 | 0.56193 | 0.80309 | 0.82778 | 0.81672 | 0.85072 |
| multilingual-e5-small-optimized | 0.75992 | 0.48934 | 0.79668 | 0.82017 | 0.8135 | 0.84316 |

To test the English out-of-domain performance, we used the test split of various datasets in the BEIR evaluation. Measuring NDCG@10, we see a larger change in SCIFACT, but marginal in the other datasets evaluated.

| | FIQA | SCIFACT | nfcorpus |
| --- | --- | --- | --- |
| multilingual-e5-small | 0.33126 | 0.677 | 0.31004 |
| multilingual-e5-small-optimized | 0.31734 | 0.65484 | 0.30126 |

### Performance

Using a PyTorch model traced for Linux and Intel CPUs, we performed performance benchmarking with various lengths of input. Overall, we see on average a 50-20% performance improvement with the optimized model.

| input length (characters) | multilingual-e5-small | multilingual-e5-small-optimized | speedup |
| --- | --- | --- | --- |
| 0 - 50 | 0.0181 | 0.00826 | 54.36% |
| 50 - 100 | 0.0275 | 0.0164 | 40.36% |
| 100 - 150 | 0.0366 | 0.0237 | 35.25% |
| 150 - 200 | 0.0435 | 0.0301 | 30.80% |
| 200 - 250 | 0.0514 | 0.0379 | 26.26% |
| 250 - 300 | 0.0569 | 0.043 | 24.43% |
| 300 - 350 | 0.0663 | 0.0513 | 22.62% |
| 350 - 400 | 0.0737 | 0.0576 | 21.85% |

### Disclaimer

This e5 model, as defined, hosted, integrated and used in conjunction with our other Elastic Software is covered by our standard warranty.