Repository: ggml-org/llama.cpp
Pull request: Recognize IBM Granite 3.3 FIM tokens. Makes llama-server /infill usable.

The model in question is, freshly released by IBM: https://huggingface.co/ibm-granite/granite-3.3-8b-base

The Granite 3.3's FIM tokens are very similar to Qwen's; it's just that they use underscore instead of a dash. So `<fim_middle>` for example instead of `<fim-middle>`.

Opening up tokenizer_config.json in ibm-granite/granite-3.3-8b-base shows (https://huggingface.co/ibm-granite/granite-3.3-8b-base/blob/main/tokenizer_config.json)

```
    "<fim_prefix>",
    "<fim_middle>",
    "<fim_suffix>",
    "<fim_pad>",
    ...
    "<reponame>",
```

Tested with [granite-3](https://huggingface.co/ibm-granite/granite-3.3-8b-base) I converted to `.gguf`s. I noticed the llama.cpp code completion vim extension didn't work with the llama-server, so I checked out if tokens were missing, added them, tested them, filed this PR.

I could not find an equivalent for file separator token, but I mapped the 5 tokens I found that had clear llama.cpp equivalents.

---

Testing:

Checked tokenization (i.e. does llama.cpp tokenize them all to single tokens):

```
$ ./build/bin/llama-tokenize --model ~/text-generation-webui/models/granite-3.3-base-q8.gguf --prompt "<fim_prefix><fim_middle><fim_suffix><fim_pad><reponame>"
... omitted verbose output ...
     1 -> '<fim_prefix>'
     2 -> '<fim_middle>'
     3 -> '<fim_suffix>'
     4 -> '<fim_pad>'
    18 -> '<reponame>'
```

Also saw: 
<img width="397" alt="Screenshot 2025-04-16 at 18 25 01" src="https://github.com/user-attachments/assets/cc3a8c23-bbf7-40b1-a82d-73104b236ef8" />

And empirically tried in coding with the extension:

https://github.com/user-attachments/assets/c0b2181a-a50b-4181-8033-0e7b2e0110b5

(I thought it would offer printf() instead...C++ bias? 😉 ) I love that extension.

Changed files:
filename: src/llama-vocab.cpp
patch: @@ -1841,6 +1841,7 @@ void llama_vocab::impl::load(llama_model_loader & ml, const LLM_KV & kv) {
                 if (false
                         || t.first == "
"  // Qwen
                         || t.first == "<fim-prefix>"
+                        || t.first == "<fim_prefix>"    // Granite
                         || t.first == "<｜fim▁begin｜>" // DeepSeek
                         || t.first == "<PRE>"
                         || t.first == "▁<PRE>"          // CodeLlama
@@ -1859,6 +1860,7 @@ void llama_vocab::impl::load(llama_model_loader & ml, const LLM_KV & kv) {
                 if (false
                         || t.first == "
" // Qwen
                         || t.first == "<fim-suffix>"
+                        || t.first == "<fim_suffix>"   // Granite
                         || t.first == "<｜fim▁hole｜>" // DeepSeek
                         || t.first == "<SUF>"
                         || t.first == "▁<SUF>"         // CodeLlama
@@ -1877,6 +1879,7 @@ void llama_vocab::impl::load(llama_model_loader & ml, const LLM_KV & kv) {
                 if (false
                         || t.first == "
" // Qwen
                         || t.first == "<fim-middle>"
+                        || t.first == "<fim_middle>"   // Granite
                         || t.first == "<｜fim▁end｜>"  // DeepSeek
                         || t.first == "<MID>"
                         || t.first == "▁<MID>"         // CodeLlama
@@ -1895,6 +1898,7 @@ void llama_vocab::impl::load(llama_model_loader & ml, const LLM_KV & kv) {
                 if (false
                         || t.first == "
" // Qwen
                         || t.first == "<fim-pad>"
+                        || t.first == "<fim_pad>"   // Granite
                         || t.first == "<PAD>"
                         ) {
                     special_fim_pad_id = t.second;
@@ -1913,6 +1917,7 @@ void llama_vocab::impl::load(llama_model_loader & ml, const LLM_KV & kv) {
                         || t.first == "
"
                         || t.first == "<fim-repo>"
                         || t.first == "<REPO>"
+                        || t.first == "<reponame>"    // Granite
                         ) {
                     special_fim_rep_id = t.second;
                     if ((id_to_token[t.second].attr & LLAMA_TOKEN_ATTR_CONTROL) == 0) {