SigLIP tokenizer not enforcing use_fast=True
Author: yxchng
Pull request: no
Labels: name: Feature request

name: Good Difficult Issue

### System Info

- `transformers` version: 4.38.2
- Platform: Linux-4.18.0-477.27.1.el8_8.x86_64-x86_64-with-glibc2.28
- Python version: 3.10.13
- Huggingface_hub version: 0.21.4
- Safetensors version: 0.4.2
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.2.1+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import AutoTokenizer
t=AutoTokenizer.from_pretrained('google/siglip-so400m-patch14-384', use_fast=True)
assert t.is_fast, 'tokenizer is not fast'
print('Success')
```

### Expected behavior

print 'Success' which indicates `use_fast=True`

Comments:
Hey! Siglip does not have a fast tokenizer: https://github.com/huggingface/transformers/blob/6226b374481cdd4799e5828d9e80f7bb670b7ef0/src/transformers/models/auto/tokenization_auto.py#L404 

I don't quite understand the line you linked. Why this model is different from others and does not have fast tokenizer?

That is because of https://github.com/huggingface/transformers/blob/84d406c276cf56fd4ba6dd7c01c7570235b1bff7/src/transformers/models/siglip/tokenization_siglip.py#L284

basically the equivalent fast tokenizer is not implemented because it needed a bit more work. I'll see if I have time to add it but otherwise it's a good issue! 

Is this going to get merged anytime soon?

I reviewed it, just need @NielsRogge 's updates

Hey @yxchng, sorry for the very late follow-up, we're catching up on the backlog.

Quick update: in the v5 tokenizer refactor the slow/fast split was removed and `use_fast` is now ignored entirely, see https://github.com/huggingface/transformers/blob/150eb7c9ed4091294c829fa0e9466b090cb0f87f/src/transformers/models/auto/tokenization_auto.py#L718-L719
so the original "enforce use_fast=True" framing no longer maps to the code. That said, the underlying gap is still real: SigLIP is still routed to the SentencePiece-backed `SiglipTokenizer` https://github.com/huggingface/transformers/blob/150eb7c9ed4091294c829fa0e9466b090cb0f87f/src/transformers/models/auto/tokenization_auto.py#L311
so `is_fast` is still `False` and there's no `tokenizers`-backend equivalent (no entry in `SLOW_TO_FAST_CONVERTERS`, and `backend="tokenizers"` has no effect here).

For reference, the newer SigLIP2 already ships a Rust-backed tokenizer https://github.com/huggingface/transformers/blob/150eb7c9ed4091294c829fa0e9466b090cb0f87f/src/transformers/models/siglip2/tokenization_siglip2.py#L31
We'll keep this open as a Good Difficult Issue (but priority low): SigLIP's tokenizer is SentencePiece/T5-like, so a small `SpmConverter` subclass plus its text canonicalization would do it.