Repository: ggml-org/llama.cpp
Pull request: model: codefuse-ai/F2LLM-v2 support

This PR adds support for **F2LLM-v2** embedding models  in `llama.cpp`. ([Hugging Face][1])

The implementation enables F2LLM-v2 embedding models such as:

* `codefuse-ai/F2LLM-v2-8B` ([Hugging Face][1])

while keeping changes minimal by reusing/copying the existing implementation wherever possible.

---

## Key Points

* support for `F2LLM-v2`
* minimal changes: the implementation largely reuses the existing model code of Qwen3 already present
* adds/adapts the existing implementation for GGUF conversion and model loading
* no new model implementation is needed since F2LLM-v2 architecture is Qwen3

---

## Validation

* successfully tested with:

  * `codefuse-ai/F2LLM-v2-4B` ([Hugging Face][1]) and smaller models 1.7B, 0.6B and 330M
* embedding outputs have the expected shape and value ranges compared to the reference implementation
* tests also included comparison of embeddings from quantized models and reduced sized embeddings (matryoshka representation learning)

---

## GGUF Models

https://huggingface.co/sabafallah/F2LLM-v2-GGUF

---

## Running
```shell
build/bin/llama-server \                                                             
-m gguf_models/codefuse-ai/F2LLM-v2-4B-q4_k_m.gguf \
--temp 0 -n 4096 --embedding
```

```shell
curl http://localhost:8080/v1/embeddings \
-H "Content-Type: application/json" \
-H "Authorization: Bearer no-key" \
-d '{
  "model": "F2LLM-v2-4B-q4_k_m.gguf",
  "input": [
    "Instruct: Given a question, retrieve passages that can help answer the question.\nQuery: How do neural networks learn patterns from examples?",
    "Deep learning models adjust their weights through backpropagation, using gradient descent to minimize error on training data and improve predictions over time.",
    "Market prices are determined by the relationship between how much people want to buy a product and how much is available for sale, with scarcity driving prices up and abundance driving them down."
  ]
}'
```

---

[1]: https://huggingface.co/codefuse-ai/F2LLM-v2-8B "codefuse-ai/F2LLM-v2-8B · Hugging Face"
[2]: https://arxiv.org/abs/2603.19223 "[2603.19223] F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual World"


Changed files:
filename: convert_hf_to_gguf.py
patch: @@ -1503,6 +1503,9 @@ def get_vocab_base_pre(self, tokenizer) -> str:
         if chkhsh == "e4d54df1ebc1f2b91acd986c5b51aa50837d5faf7c7398e73c1f9e9ee5d19869":
             # ref: https://huggingface.co/kakaocorp/kanana-2-30b-a3b-instruct-2601
             res = "kanana2"
+        if chkhsh == "862f827721df956049dff5ca81a57f29e575280bc622e290d3bf4e35eca29015":
+            # ref: https://huggingface.co/codefuse-ai/F2LLM-v2-4B
+            res = "f2llmv2"
 
         if res is None:
             logger.warning("\n")

filename: convert_hf_to_gguf_update.py
patch: @@ -154,6 +154,7 @@ class TOKENIZER_TYPE(IntEnum):
     {"name": "qwen35",           "tokt": TOKENIZER_TYPE.BPE, "repo": "https://huggingface.co/Qwen/Qwen3.5-9B-Instruct", },
     {"name": "joyai-llm",        "tokt": TOKENIZER_TYPE.BPE, "repo": "https://huggingface.co/jdopensource/JoyAI-LLM-Flash", },
     {"name": "kanana2",          "tokt": TOKENIZER_TYPE.BPE, "repo": "https://huggingface.co/kakaocorp/kanana-2-30b-a3b-instruct-2601", },
+    {"name": "f2llmv2",          "tokt": TOKENIZER_TYPE.BPE, "repo": "https://huggingface.co/codefuse-ai/F2LLM-v2-4B", },
 ]
 
 # some models are known to be broken upstream, so we will skip them as exceptions

filename: src/llama-vocab.cpp
patch: @@ -1952,7 +1952,8 @@ void llama_vocab::impl::load(llama_model_loader & ml, const LLM_KV & kv) {
             } else if (
                     tokenizer_pre == "qwen2" ||
                     tokenizer_pre == "deepseek-r1-qwen" ||
-                    tokenizer_pre == "kormo") {
+                    tokenizer_pre == "kormo" ||
+                    tokenizer_pre == "f2llmv2") {
                 pre_type = LLAMA_VOCAB_PRE_TYPE_QWEN2;
                 clean_spaces = false;
             } else if (