Repository: huggingface/text-embeddings-inference (Rust)

Issue:
Addition of Snowflake Arctic Embed 2.0 
### Model description

Snowflake recently released the Arctic Embed 2.0 models, which deliver state-of-the-art multilingual retrieval performance while maintaining efficiency and compactness. These models are open source under the permissive Apache 2.0 license and include:

Arctic Embed M 2.0: Optimized for inference efficiency with 305M parameters.
Arctic Embed L 2.0: Optimized for retrieval quality with 568M parameters.

We believe these models would be a valuable addition to the Hugging Face TEI, enhancing accessibility for developers seeking top-tier multilingual embedding solutions.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

_No response_
Models with `model_type: gte` fail to load, despite the architecture being supported
Loading a model with `gte` as its `model_type` (i.e. `Snowflake/snowflake-arctic-embed-m-v2.0`), fails with an unknown variant error, since it expects the value `new` for GTE-based models:
```log
❯ docker run --gpus all --pull always ghcr.io/huggingface/text-embeddings-inference:86-1.6 --model-id Snowflake/snowflake-arctic-embed-m-v2.0
86-1.6: Pulling from huggingface/text-embeddings-inference
Digest: sha256:6f6988335bcebbe9f6e96c7e3b91677670b6cd7eacfc89a7a53b2d015a9ef1ee
Status: Image is up to date for ghcr.io/huggingface/text-embeddings-inference:86-1.6
2025-02-18T13:20:14.693433Z  INFO text_embeddings_router: router/src/main.rs:175: Args { model_id: "Sno******/*********-******-*****-*-v2.0", revision: None, tokenization_workers: None, dtype: None, pooling: None, max_concurrent_requests: 512, max_batch_tokens: 16384, max_batch_requests: None, max_client_batch_size: 32, auto_truncate: false, default_prompt_name: None, default_prompt: None, hf_api_token: None, hostname: "f227771f4d2d", port: 80, uds_path: "/tmp/text-embeddings-inference-server", huggingface_hub_cache: Some("/data"), payload_limit: 2000000, api_key: None, json_output: false, otlp_endpoint: None, otlp_service_name: "text-embeddings-inference.server", cors_allow_origin: None }
2025-02-18T13:20:14.693538Z  INFO hf_hub: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hf-hub-0.3.2/src/lib.rs:55: Token file not found "/root/.cache/huggingface/token"    
2025-02-18T13:20:14.784900Z  INFO download_artifacts: text_embeddings_core::download: core/src/download.rs:20: Starting download
2025-02-18T13:20:14.784912Z  INFO download_artifacts:download_pool_config: text_embeddings_core::download: core/src/download.rs:53: Downloading `1_Pooling/config.json`
2025-02-18T13:20:16.217168Z  INFO download_artifacts:download_new_st_config: text_embeddings_core::download: core/src/download.rs:77: Downloading `config_sentence_transformers.json`
2025-02-18T13:20:16.469719Z  INFO download_artifacts: text_embeddings_core::download: core/src/download.rs:40: Downloading `config.json`
2025-02-18T13:20:16.709562Z  INFO download_artifacts: text_embeddings_core::download: core/src/download.rs:43: Downloading `tokenizer.json`
2025-02-18T13:20:17.365921Z  INFO download_artifacts: text_embeddings_core::download: core/src/download.rs:47: Model artifacts downloaded in 2.581016532s
2025-02-18T13:20:17.911988Z  WARN text_embeddings_router: router/src/lib.rs:184: Could not find a Sentence Transformers config
2025-02-18T13:20:17.912004Z  INFO text_embeddings_router: router/src/lib.rs:188: Maximum number of tokens per request: 8192
2025-02-18T13:20:17.912072Z  INFO text_embeddings_core::tokenization: core/src/tokenization.rs:28: Starting 12 tokenization workers
2025-02-18T13:20:20.626725Z  INFO text_embeddings_router: router/src/lib.rs:230: Starting model backend
2025-02-18T13:20:20.626757Z  INFO text_embeddings_backend: backends/src/lib.rs:360: Downloading `model.safetensors`
2025-02-18T13:20:31.877480Z  INFO text_embeddings_backend: backends/src/lib.rs:244: Model weights downloaded in 11.250718984s
2025-02-18T13:20:31.887484Z ERROR text_embeddings_backend: backends/src/lib.rs:255: Could not start Candle backend: Could not start backend: Model is not supported

Caused by:
    unknown variant `gte`, expected one of `bert`, `xlm-roberta`, `camembert`, `roberta`, `distilbert`, `nomic_bert`, `mistral`, `new`, `qwen2`, `mpnet` at line 21 column 21
Error: Could not create backend

Caused by:
    Could not start backend: Could not start a suitable backend

```

The following deserialization code should be updated to also load GTE models when their `model_type` is `gte`:
https://github.com/huggingface/text-embeddings-inference/blob/f0e491a290385ef06f0871d188b21c0308ba86d6/backends/candle/src/lib.rs#L49-L66

I guess adding a `#[serde(rename(deserialize = "gte"))]` right before L62 would be enough, but I know nothing about Rust.


Discussion: Great work!
Any example to fine tuning in a specific domain?
+1

would be great to have on TEI

same issue here. it works on CPU only



Fix (patch):
diff --git a/README.md b/README.md
index 5c7b9680..18cc0c5c 100644
--- a/README.md
+++ b/README.md
@@ -71,17 +71,19 @@ Below are some examples of the currently supported models:
 
 | MTEB Rank | Model Size          | Model Type  | Model ID                                                                                         |
 |-----------|---------------------|-------------|--------------------------------------------------------------------------------------------------|
-| 1         | 7B (Very Expensive) | Mistral     | [Salesforce/SFR-Embedding-2_R](https://hf.co/Salesforce/SFR-Embedding-2_R)                       |
-| 2         | 7B (Very Expensive) | Qwen2       | [Alibaba-NLP/gte-Qwen2-7B-instruct](https://hf.co/Alibaba-NLP/gte-Qwen2-7B-instruct)             |
-| 9         | 1.5B (Expensive)    | Qwen2       | [Alibaba-NLP/gte-Qwen2-1.5B-instruct](https://hf.co/Alibaba-NLP/gte-Qwen2-1.5B-instruct)         |
-| 15        | 0.4B                | Alibaba GTE | [Alibaba-NLP/gte-large-en-v1.5](https://hf.co/Alibaba-NLP/gte-large-en-v1.5)                     |
+| 3         | 7B (Very Expensive) | Qwen2       | [Alibaba-NLP/gte-Qwen2-7B-instruct](https://hf.co/Alibaba-NLP/gte-Qwen2-7B-instruct)             |
+| 11        | 1.5B (Expensive)    | Qwen2       | [Alibaba-NLP/gte-Qwen2-1.5B-instruct](https://hf.co/Alibaba-NLP/gte-Qwen2-1.5B-instruct)         |
+| 14        | 7B (Very Expensive) | Mistral     | [Salesforce/SFR-Embedding-2_R](https://hf.co/Salesforce/SFR-Embedding-2_R)                       |
 | 20        | 0.3B                | Bert        | [WhereIsAI/UAE-Large-V1](https://hf.co/WhereIsAI/UAE-Large-V1)                                   |
-| 24        | 0.5B                | XLM-RoBERTa | [intfloat/multilingual-e5-large-instruct](https://hf.co/intfloat/multilingual-e5-large-instruct) |
+| 31        | 0.5B                | XLM-RoBERTa | [Snowflake/snowflake-arctic-embed-l-v2.0](https://hf.co/Snowflake/snowflake-arctic-embed-l-v2.0) |
+| 37        | 0.3B                | Alibaba GTE | [Snowflake/snowflake-arctic-embed-m-v2.0](https://hf.co/Snowflake/snowflake-arctic-embed-m-v2.0) |
+| 49        | 0.5B                | XLM-RoBERTa | [intfloat/multilingual-e5-large-instruct](https://hf.co/intfloat/multilingual-e5-large-instruct) |
+| N/A       | 0.4B                | Alibaba GTE | [Alibaba-NLP/gte-large-en-v1.5](https://hf.co/Alibaba-NLP/gte-large-en-v1.5)                     |
 | N/A       | 0.1B                | NomicBert   | [nomic-ai/nomic-embed-text-v1](https://hf.co/nomic-ai/nomic-embed-text-v1)                       |
 | N/A       | 0.1B                | NomicBert   | [nomic-ai/nomic-embed-text-v1.5](https://hf.co/nomic-ai/nomic-embed-text-v1.5)                   |
 | N/A       | 0.1B                | JinaBERT    | [jinaai/jina-embeddings-v2-base-en](https://hf.co/jinaai/jina-embeddings-v2-base-en)             |
 | N/A       | 0.1B                | JinaBERT    | [jinaai/jina-embeddings-v2-base-code](https://hf.co/jinaai/jina-embeddings-v2-base-code)         |
-| N/A       | 0.1B                | MPNet       | [sentence-transformers/all-mpnet-base-v2](https://hf.co/sentence-transformers/all-mpnet-base-v2)            |
+| N/A       | 0.1B                | MPNet       | [sentence-transformers/all-mpnet-base-v2](https://hf.co/sentence-transformers/all-mpnet-base-v2) |
 
 To explore the list of best performing text embeddings models, visit the
 [Massive Text Embedding Benchmark (MTEB) Leaderboard](https://huggingface.co/spaces/mteb/leaderboard).
diff --git a/backends/candle/src/lib.rs b/backends/candle/src/lib.rs
index 28167930..209695a6 100644
--- a/backends/candle/src/lib.rs
+++ b/backends/candle/src/lib.rs
@@ -59,8 +59,9 @@ enum Config {
     NomicBert(NomicConfig),
     #[allow(dead_code)]
     Mistral(MistralConfig),
-    #[serde(rename = "new")]
     Gte(GTEConfig),
+    #[serde(rename = "new")]
+    GteAlibaba(GTEConfig),
     #[allow(dead_code)]
     Qwen2(Qwen2Config),
     #[serde(rename = "mpnet")]
@@ -223,7 +224,7 @@ impl CandleBackend {
                 "Mistral is only supported on Cuda devices in fp16 with flash attention enabled"
                     .to_string(),
             )),
-            (Config::Gte(config), Device::Cpu | Device::Metal(_)) => {
+            (Config::Gte(config) | Config::GteAlibaba(config), Device::Cpu | Device::Metal(_)) => {
                 tracing::info!("Starting GTE model on {:?}", device);
                 Ok(Box::new(GTEModel::load(vb, &config, model_type).s()?))
             }
@@ -354,7 +355,7 @@ impl CandleBackend {
                 ))
             }
             #[cfg(feature = "cuda")]
-            (Config::Gte(config), Device::Cuda(_)) => {
+            (Config::Gte(config) | Config::GteAlibaba(config), Device::Cuda(_)) => {
                 if dtype != DType::F16
                     || !cfg!(any(feature = "flash-attn", feature = "flash-attn-v1"))
                 {