gpt-oss and transformers serve not working
Author: SunMarc
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.56.0.dev0
- Platform: Linux-5.4.0-216-generic-x86_64-with-glibc2.31
- Python version: 3.10.0
- Huggingface_hub version: 0.34.4
- Safetensors version: 0.5.3
- Accelerate version: 1.10.1
- Accelerate config: 	- compute_environment: LOCAL_MACHINE
	- distributed_type: MULTI_GPU
	- mixed_precision: no
	- use_cpu: False
	- debug: False
	- num_processes: 2
	- machine_rank: 0
	- num_machines: 1
	- gpu_ids: all
	- rdzv_backend: static
	- same_network: True
	- main_training_function: main
	- enable_cpu_affinity: False
	- downcast_bf16: no
	- tpu_use_cluster: False
	- tpu_use_sudo: False
	- tpu_env: []
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.8.0+cu128 (CUDA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA A100-SXM4-80GB

### Who can help?

I'm getting the following when trying chat with the model with transformers serve + transformers chat localhost:8000 --model-name-or-path transformers chat localhost:8000 --model-name-or-path openai/gpt-oss-20b

It doesn't work for both the de-quantized model (you can just uninstall `kernels`) and the quantized model. 

```
Traceback (most recent call last):
  File "/home/marc/anaconda3/envs/sglang/lib/python3.10/threading.py", line 1009, in _bootstrap_inner
    self.run()
  File "/home/marc/anaconda3/envs/sglang/lib/python3.10/threading.py", line 946, in run
    self._target(*self._args, **self._kwargs)
  File "/home/marc/transformers/src/transformers/commands/serving.py", line 982, in generate_with_cache
    generate_output = model.generate(**kwargs)
  File "/home/marc/anaconda3/envs/sglang/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/marc/transformers/src/transformers/generation/utils.py", line 2539, in generate
    result = self._sample(
  File "/home/marc/transformers/src/transformers/generation/utils.py", line 2860, in _sample
    model_inputs = self.prepare_inputs_for_generation(input_ids, **model_kwargs)
  File "/home/marc/transformers/src/transformers/generation/utils.py", line 569, in prepare_inputs_for_generation
    inputs_embeds, input_ids = self._cache_dependant_input_preparation(
  File "/home/marc/transformers/src/transformers/generation/utils.py", line 475, in _cache_dependant_input_preparation
    or (cache_position[-1] >= input_ids.shape[1])  # Exception 3
IndexError: index -1 is out of bounds for dimension 0 with size 0
```

@gante 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

transformers serve + transformers chat localhost:8000 --model-name-or-path transformers chat localhost:8000 --model-name-or-path openai/gpt-oss-20b

### Expected behavior

working 

Comments:
maybe linked to that https://github.com/huggingface/transformers/issues/38727

Hi @SunMarc! Can I work on this issue?

Feel free to give it a try ! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

is there any update for this issue?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.