past_key_value(s) name inconsistency causing problems
Author: HDCharles
Pull request: no
Labels: name: Good Second Issue

name: bug

### System Info

- `transformers` version: 4.50.0.dev0
- Platform: Linux-6.4.3-0_fbk14_zion_2601_gcd42476b84e9-x86_64-with-glibc2.34
- Python version: 3.12.9
- Huggingface_hub version: 0.28.1
- Safetensors version: 0.5.2
- Accelerate version: 1.4.0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.6.0.dev20241112+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: no
- Using GPU in script?: yes
- GPU type: NVIDIA H100

### Who can help?

@ArthurZucker probably others

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

run https://huggingface.co/docs/transformers/main/en/quantization/torchao

### Expected behavior

no error

____

this error is related to https://github.com/huggingface/transformers/pull/36289

a bunch of models use past_key_value and past_key_values interchangeably, this causes issues since kwarg names are hardcoded to be skipped by the _skip_keys_device_placement attribute. This causes issues any time torch.compile is used with a model that has this issue.
 The above PR fixes the issue for llama but other models like src/transformers/models/moonshine/modeling_moonshine.py
src/transformers/models/mistral/modeling_mistral.py
src/transformers/models/emu3/modeling_emu3.py
...etc, have the same issue which is actually breaking CI for that PR.

this is also the cause of https://github.com/pytorch/ao/issues/1705 which is where this was first surfaced.

is there a reason for these two names to be used instead of just one? If not it seems like they should be entirely consolidated to avoid such issues, if so, then _skip_keys_device_placement needs to include both across all models or 



Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

cc @gante @Cyrilvallez maybe?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

anyone is free to open a PR for standardization! we need indeed a single source of truth! `past_key_values`

i would like to work on this issue if you could provide a better example for reproduction 

@ved1beta many models use `past_key_value` in place of `past_key_values` (e.g. [here](https://github.com/huggingface/transformers/blob/fa3c3f9cab1c45b449bd57e238c511c79637e314/src/transformers/models/llama/modeling_llama.py#L247))

It's mostly because of legacy reasons that no longer make sense (`past_key_value` was used to refer to the cache of a single layer, but we now have the `Cache` classes)

may be close this as it is taken care of in this [PR](https://github.com/huggingface/transformers/pull/39956)