Token healing throws error with "Qwen/Qwen2.5-Coder-7B-Instruct"
Author: desaxce
Pull request: no
Labels: name: bug

### System Info

OS type: Sequoia 15.2 Apple M2 Pro. 
I tried to reproduce using 🤗 [Inference Endpoints](https://endpoints.huggingface.co/AI-MO/endpoints/dedicated) when deploying https://huggingface.co/desaxce/Qwen2.5-Coder-7B-Instruct. It's a fork of `Qwen/Qwen2.5-Coder-7B-Instruct` with `token_healing=True` and a `handler.py` to deploy on 🤗  Inference Endpoints (use Default container, not TGI).

python 3.12.8
transformers 4.48.2
torch 2.6.0

Generate text with using [Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) while specifying  `token_healing` to `True`:
```
from transformers import AutoTokenizer, Qwen2ForCausalLM, Qwen2Tokenizer

pipe = Qwen2ForCausalLM.from_pretrained("./")
tokenizer = Qwen2Tokenizer.from_pretrained("./")

prompt = f'Complete the following Lean 4 code:\n\n```lean4\nimport '
inputs = tokenizer(prompt, return_tensors="pt")

# Here we activate token healing, which triggers error.
generate_ids = pipe.generate(inputs.input_ids, tokenizer=tokenizer, max_new_tokens=1, token_healing=True)
tokenizer.batch_decode(generate_ids, skip_special_tokens=False, clean_up_tokenization_spaces=False)[0]
```

The error:
```
{'error': "where() received an invalid combination of arguments - got (bool, int, Tensor), but expected one of:
 * (Tensor condition)
 * (Tensor condition, Tensor input, Tensor other, *, Tensor out)
 * (Tensor condition, Number self, Tensor other)
      didn't match because some of the arguments have invalid types: (!bool!, !int!, Tensor)
 * (Tensor condition, Tensor input, Number other)
      didn't match because some of the arguments have invalid types: (!bool!, !int!, !Tensor!)
 * (Tensor condition, Number self, Number other)
      didn't match because some of the arguments have invalid types: (!bool!, !int!, !Tensor!)
"}
```

I traced it to https://github.com/huggingface/transformers/blob/main/src/transformers/generation/utils.py#L2436. 
Because `tokenizer.bos_token_id` is `None`, the `torch.where()` call fails. 

I commented this line and a subsequent error popped up a few lines below on https://github.com/huggingface/transformers/blob/main/src/transformers/generation/utils.py#L2447. The error:
```
TypeError                                 Traceback (most recent call last)
Cell In[1], line 9
      6 prompt = f'Complete the following Lean 4 code:\n\n```lean4\nimport '
      7 inputs = tokenizer(prompt, return_tensors="pt")
----> 9 generate_ids = pipe.generate(inputs.input_ids, tokenizer=tokenizer, max_new_tokens=1, token_healing=True)
     10 tokenizer.batch_decode(generate_ids, skip_special_tokens=False, clean_up_tokenization_spaces=False)[0]

File [~/miniconda3/envs/autocomplete/lib/python3.12/site-packages/torch/utils/_contextlib.py:116](http://localhost:8888/~/miniconda3/envs/autocomplete/lib/python3.12/site-packages/torch/utils/_contextlib.py#line=115), in context_decorator.<locals>.decorate_context(*args, **kwargs)
    113 @functools.wraps(func)
    114 def decorate_context(*args, **kwargs):
    115     with ctx_factory():
--> 116         return func(*args, **kwargs)

File [~/miniconda3/envs/autocomplete/lib/python3.12/site-packages/transformers/generation/utils.py:2084](http://localhost:8888/~/miniconda3/envs/autocomplete/lib/python3.12/site-packages/transformers/generation/utils.py#line=2083), in GenerationMixin.generate(self, inputs, generation_config, logits_processor, stopping_criteria, prefix_allowed_tokens_fn, synced_gpus, assistant_model, streamer, negative_prompt_ids, negative_prompt_attention_mask, **kwargs)
   2081     input_ids = inputs_tensor if model_input_name == "input_ids" else model_kwargs.pop("input_ids")
   2083 if generation_config.token_healing:
-> 2084     input_ids = self.heal_tokens(input_ids, tokenizer)
   2086 if streamer is not None:
   2087     streamer.put(input_ids.cpu())

File [~/miniconda3/envs/autocomplete/lib/python3.12/site-packages/transformers/generation/utils.py:2499](http://localhost:8888/~/miniconda3/envs/autocomplete/lib/python3.12/site-packages/transformers/generation/utils.py#line=2498), in GenerationMixin.heal_tokens(self, input_ids, tokenizer)
   2495     return input_ids
   2497 tail_ids = input_ids[:, -1].tolist()
-> 2499 space_tok = tokenizer.convert_ids_to_tokens(tokenizer.convert_tokens_to_ids(" "))[0]
   2500 # tail tokens are used for a prefix search, thus, whitespaces are replaced with
   2501 # their tokenization (e.g. 'Ġ') to enable search for tokens prefixed with a whitespace
   2502 tail_toks = (tokenizer.decode(t).replace(" ", space_tok) for t in tail_ids)

File [~/miniconda3/envs/autocomplete/lib/python3.12/site-packages/transformers/tokenization_utils.py:1065](http://localhost:8888/~/miniconda3/envs/autocomplete/lib/python3.12/site-packages/transformers/tokenization_utils.py#line=1064), in PreTrainedTokenizer.convert_ids_to_tokens(self, ids, skip_special_tokens)
   1063         return self._convert_id_to_token(ids)
   1064 tokens = []
-> 1065 for index in ids:
   1066     index = int(index)
   1067     if skip_special_tokens and index in self.all_special_ids:

TypeError: 'NoneType' object is not iterable
```
This time, it's due to `tokenizer.convert_tokens_to_ids(" ")` returning `None` because the space character is not a token (the tokenizer already uses `Ġ` to represent space characters). 

### Who can help?

@ArthurZucker @itazap I suspect an issue in `heal_tokens` for tokenizers which:
- have `tokenizer.bos_token_id` equal to `None`
- do not have space character as a token, i.e. `tokenizer.convert_tokens_to_ids(" ")` is `None`

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

To reproduce on 🤗 Inference Endpoints, deploy https://huggingface.co/desaxce/Qwen2.5-Coder-7B-Instruct on a "Default" container. I forked this repository from https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct to reproduce the issue: I added the `token_healing: true` parameter in `generation_config.json` and a `handler.py` to be able to deploy on 🤗 Inference Endpoints. 

It's important to select "Default" container to reproduce - with TGI I didn't have any error (but I didn't check that token healing was indeed being used). In all cases, the error can be reproduced locally ⬇ 

To reproduce locally, clone https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct and run this snippet which generates using token healing:
```
from transformers import AutoTokenizer, Qwen2ForCausalLM, Qwen2Tokenizer

pipe = Qwen2ForCausalLM.from_pretrained("./")
tokenizer = Qwen2Tokenizer.from_pretrained("./")

prompt = f'Complete the following Lean 4 code:\n\n```lean4\nimport '
inputs = tokenizer(prompt, return_tensors="pt")

# Here we activate token healing, which triggers error.
generate_ids = pipe.generate(inputs.input_ids, tokenizer=tokenizer, max_new_tokens=1, token_healing=True)
tokenizer.batch_decode(generate_ids, skip_special_tokens=False, clean_up_tokenization_spaces=False)[0]
```

### Expected behavior

I expect completion to take place with tokens healed: 

![Image](https://github.com/user-attachments/assets/2f97b3ba-4475-4d39-84a3-978bfdf8aca1)

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.