Fix convert_tekken_tokenizer
Author: juliendenize
Pull request: yes

# What does this PR do?

Right now the `convert_tekken_tokenizer` does not add `bos_tokens`, `eos_token`  to the special tokens via the `add_special_tokens` method.

This prevents the chat templates that expect `eos_token` and `bos_token` to work properly.

Previously this was working as when saving the tokenizer a `special_tokens_map.json` was created which is no longer the case. Unknown to me why but I'd assume this is due to the V5 refactoring ?

This PR fixes that by adding explicitly these tokens to the tokenizer and when saving they're now stored in `tokenizer_config.json`.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker 

Comments:
cc @itazap

run-slow: ministral3, mistral3

This comment contains `run-slow`, running the specified jobs: 

models: ["models/ministral3", "models/mistral3"]
quantizations: []

## CI Results
[Workflow Run ⚙️](https://github.com/huggingface/transformers/actions/runs/20033111339)

✅ No failing test specific to this PR 🎉 !


Hey! Thanks for the PR, can you please share a short reproducer of the problem (you mentioned in chat templates)? perhaps we'll need to add a test !

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_42592). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@itazap just a classic apply_chat_template with the wrong config.

here is a chat template that would fail
https://huggingface.co/mistralai/Ministral-3-3B-Instruct-2512-BF16?chat_template=default

as you can see BOS and EOS tokens are variable.

In Transformers when you apply_chat_template here:
https://github.com/huggingface/transformers/blob/main/src/transformers/tokenization_utils_base.py#L3140-L3149

you pass special tokens that are defined by the tokenizer's config.

Because special maps are no longer generated as before, it wasn't put in the tokenizer config so never passed to the rendered so it was outputting empty strings ( not even an error :( )

It's not really a bug but a wrong configuration passed so you could repro if you'd like by removing in ministral's config these lines

https://huggingface.co/mistralai/Ministral-3-3B-Instruct-2512/blob/main/tokenizer_config.json#L1007-L1010
and apply_chat_template wouldn't encode properly eos and bos for Transformers' tokenizer backend
```
import torch
from transformers import AutoTokenizer

model_id = "mistralai/Ministral-3-3B-Instruct-2512"

tokenizer = MistralCommonBackend.from_pretrained(model_id)
model = Mistral3ForConditionalGeneration.from_pretrained(model_id, device_map="auto")

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "How are you",
            },
        ],
    },
]

tokenized = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True)

print(tokenized["input_ids"]) # outputs 
 as first token if right config else outputs [SYSTEM_PROMPT]
```

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: ministral3, mistral3