text-generation extremely slow with large `bad_words_ids` list
Author: xenova
Pull request: no
Labels: name: Good First Issue

name: bug

name: mps

### System Info


- `transformers` version: 4.54.0.dev0
- Platform: macOS-15.5-arm64-arm-64bit-Mach-O
- Python version: 3.13.2
- Huggingface_hub version: 0.32.4
- Safetensors version: 0.5.3
- Accelerate version: 1.7.0
- Accelerate config: 	not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.7.1 (NA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: no

### Who can help?

@gante @ArthurZucker 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```py
import time
from transformers import pipeline

pipe = pipeline("text-generation", "hf-internal-testing/tiny-random-Gemma3ForCausalLM")

# Without bad_words_ids
start = time.time()
output_no_bad = pipe("hello", max_new_tokens=5, do_sample=False)
time_no_bad = time.time() - start
print(f"Without bad_words_ids: {time_no_bad:.4f} seconds")

# With bad_words_ids
bad_words_ids = [[2 * i] for i in range(pipe.tokenizer.vocab_size // 2)]
start = time.time()
output_bad = pipe("hello", max_new_tokens=5, do_sample=False, bad_words_ids=bad_words_ids)
time_bad = time.time() - start

print(f"With bad_words_ids: {time_bad:.4f} seconds")
```

outputs the following
```
Without bad_words_ids: 0.4049 seconds
With bad_words_ids: 36.2829 seconds
```

This indicates a significant overhead due to the `bad_words_ids`, most likely due to inefficient looping & tensor access or slow regex patterns.

### Expected behavior

The generation should only run marginally slower when using a block list, but not this significantly.

Comments:
Hi @xenova, I tried running your code and I didn't observe the same behavior. 
Could it be an issue related to macos?



I can reproduce this being mega slow on `mps`. cc @gante when you get some time our our community! 🤗 

Hello @xenova, I think I found the specific point that slows everything down by a lot:

https://github.com/huggingface/transformers/blob/344012b3a6cecd5393deeaf7703071ad603b7871/src/transformers/generation/logits_process.py#L1306

For some reason, this line takes like 40 seconds on my Mac M3 Max. 

Do you mind checking if this https://github.com/huggingface/transformers/pull/39556 solves it for you as well? 



Wanted to work on this but it looks like it's already fixed with 
https://github.com/huggingface/transformers/pull/39556

Thanks @DWarez for investigating - that does indeed seem to be the primary issue. Something like this reduces processing time from 36 seconds to 2 seconds... which is pretty good, but I would imagine there's still room for improvement!

```diff
+            eos_list = eos_token_id.tolist()
             bad_words_ids = list(
-                filter(lambda bad_token_seq: all(bad_token_seq != [i] for i in eos_token_id), bad_words_ids)
+                filter(lambda bad_token_seq: all(bad_token_seq != [i] for i in eos_list), bad_words_ids)
             )
```