Compile of the Generate function raises ValueError
Author: ChanderG
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.45.0
- Platform: Linux-5.14.0-284.73.1.el9_2.x86_64-x86_64-with-glibc2.31
- Python version: 3.10.15
- Huggingface_hub version: 0.26.2
- Safetensors version: 0.4.5
- Accelerate version: 1.1.1
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.5.1+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA A100-SXM4-80GB

### Who can help?

@ArthurZucker

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Torch.Compiling the Generate method using the following:
```
model.generate = torch.compile(model.generate)
```

This leads to a ValueError as follows:

```
Traceback (most recent call last):
  File "/workspace/infer.py", line 28, in <module>
    main()
  File "/workspace/infer.py", line 22, in main
    generate_ids = model.generate(input_ids, max_new_tokens=length)
  File "/opt/conda/envs/cg-test/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 465, in _fn
    return fn(*args, **kwargs)
  File "/opt/conda/envs/cg-test/lib/python3.10/site-packages/torch/_dynamo/external_utils.py", line 40, in inner
    return fn(*args, **kwargs)
  File "/opt/conda/envs/cg-test/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/opt/conda/envs/cg-test/lib/python3.10/site-packages/transformers/generation/utils.py", line 1805, in generate
    self._validate_model_kwargs(model_kwargs.copy())
  File "/opt/conda/envs/cg-test/lib/python3.10/site-packages/transformers/generation/utils.py", line 1205, in _validate_model_kwargs
    raise ValueError(
ValueError: The following `model_kwargs` are not used by the model: ['max_new_tokens'] (note: typos in the generate arguments will also show up in this list)
```

### Expected behavior

No errors, identical behaviour as the eager version.

Comments:
The reason for this is documented in the code: https://github.com/huggingface/transformers/blob/b25ef8244d6427e2e1ad219b991e062b4e25490c/src/transformers/generation/utils.py#L1526

Can you provide the full reproducer? 🤗

afair the end-to-end compilation of `generate()` requires one to pass the all parameters in `generation_config` instead of passing as kwargs to the `generate()` method. cc @gante 

```
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, AutoConfig

torch.set_default_device("cuda:0")

def main():
    model_name = "meta-llama/Llama-2-7b-hf"
    length = 100

    prompt_text = 'In a small, bustling cafe nestled in the heart of a vibrant city, a serendipitous event unfolded, leaving a lasting impression on all who witnessed it. As the patrons sat sipping their coffees and engaging in animated conversations, a talented street musician entered the cafe, carrying a weathered guitar and radiating an aura of creativity.'

    tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True)
    model = AutoModelForCausalLM.from_pretrained(model_name)

    model.generate = torch.compile(model.generate)

    input_ids = tokenizer(prompt_text, add_special_tokens=False, return_tensors='pt').input_ids
    generate_ids = model.generate(input_ids, max_new_tokens=length)
    result = tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]

    print(result)

if __name__ == "__main__":
    main()
```

@ChanderG Can you try to pass `max_new_tokens` in generation config as follows:

```python
generation_config = model.generation_config
generation_config.max_new_tokens = 100

model.generate = torch.compile(model.generate)

out = model.generate(**inputs, generation_config=generation_config)
result = tokenizer.batch_decode(out, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]

```

Using `generation_config` works (ie this error is not thrown) with `4.46.3`. Earlier versions (`4.45`) had some other Dynamo error (just fyi).

@ChanderG yes, i think the prev version bug is known and was fixed, so yeah feel free to use version from `v4.46`. You can find more information about `compile` [in the docs](https://huggingface.co/docs/transformers/v4.46.3/en/llm_optims?static-kv=advanced+usage%3A+end-to-end+generate+compilation#static-kv-cache-and-torchcompile) 

Guess the issue is resolved and can be closed now?