End-to-end generation compile cannot work !!!
Author: dingfen
Pull request: no
Labels: name: bug

name: Generation

### System Info

- ubuntu 22.04
- python 3.10
- torch 2.3.1
- transformers 4.44.0

### Who can help?

@gante @fxmarty 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Install the torch==2.3.1 && transformers==4.44.0
2. just copy the example code in Release Notes
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
import copy

model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Meta-Llama-3.1-8B", torch_dtype=torch.bfloat16, device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3.1-8B")

# compile generate
compiled_generate = torch.compile(model.generate, fullgraph=True, mode="reduce-overhead")

# compiled generate does NOT accept parameterization except a) model inputs b) a generation config
generation_config = copy.deepcopy(model.generation_config)
generation_config.pad_token_id = model.config.eos_token_id

model_inputs = tokenizer(["Write a poem about the market crashing in summer"], return_tensors="pt")
model_inputs = model_inputs.to(model.device)
output_compiled = compiled_generate(**model_inputs, generation_config=generation_config)
print(output_compiled)
```
3. Run it ! But I got the Error:

Traceback (most recent call last):                                                                                                                                                                                                                          
  File "/workspace/LLM/temp.py", line 19, in <module>                                                                                                                                                                                                       
    output_compiled = compiled_generate(**model_inputs, generation_config=generation_config)                                                                                                                                                                
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/eval_frame.py", line 451, in _fn                                                                                                                                                              
    return fn(*args, **kwargs)                                                                                                                                                                                                                              
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/convert_frame.py", line 921, in catch_errors                                                                                                                                                  
    return callback(frame, cache_entry, hooks, frame_state, skip=1)                                                                                                                                                                                         
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/convert_frame.py", line 400, in _convert_frame_assert                                                                                                                                         
    return _compile(                                                                                                                                                                                                                                        
  File "/usr/lib/python3.10/contextlib.py", line 79, in inner                                                                                                                                                                                               
    return func(*args, **kwds)                                                                                                                                                                                                                              
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/convert_frame.py", line 676, in _compile                                                                                                                                                      
    guarded_code = compile_inner(code, one_graph, hooks, transform)                                                                                                                                                                                         
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/utils.py", line 262, in time_wrapper                                                                                                                                                          
    r = func(*args, **kwargs)                                                                                                                                                                                                                               
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/convert_frame.py", line 535, in compile_inner                                                                                                                                                 
    out_code = transform_code_object(code, transform)                                                                                                                                                                                                       
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/bytecode_transformation.py", line 1036, in transform_code_object                                                                                                                              
    transformations(instructions, code_options)                                                                                                                                                                                                             
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/convert_frame.py", line 165, in _fn                                                                                                                                                           
    return fn(*args, **kwargs)                                                                                                                                                                                                                              
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/convert_frame.py", line 500, in transform                                                                                                                                                     
    tracer.run()                                                                                                                                                                                                                                            
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 2149, in run                                                                                                                                                       
    super().run()                                                                                                                                                                                                                                           
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 810, in run                                                                                                                                                        
    and self.step()                                                                                                                                                                                                                                         
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 773, in step                                                                                                                                                       
    getattr(self, inst.opname)(inst)                                                                                                                                                                                                                        
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 489, in wrapper                                                                                                                                                    
    return inner_fn(self, inst)                                                                                                                                                                                                                             
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 1260, in CALL_FUNCTION_EX                                                                                                                                          
    self.call_function(fn, argsvars.items, kwargsvars)                                                                                                                                                                                                      
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 674, in call_function                                                                                                                                              
    self.push(fn.call_function(self, args, kwargs))                                                                                                                                                                                                         
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/lazy.py", line 94, in realize_and_forward                                                                                                                                           
    return getattr(self.realize(), name)(*args, **kwargs)                                                                                                                                                                                                   
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/functions.py", line 335, in call_function                                                                                                                                           
    return super().call_function(tx, args, kwargs)                                                                                                                                                                                                          
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/functions.py", line 289, in call_function                                                                                                                                           
    return super().call_function(tx, args, kwargs)                                                                                                                                                                                                          
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/functions.py", line 90, in call_function                                                                                                                                            
    return tx.inline_user_function_return(                                                                                                                                                                                                                  
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 680, in inline_user_function_return                                                                                                                                
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)                                                                                                                                                                                
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 2285, in inline_call                                                                                                                                               
    return cls.inline_call_(parent, func, args, kwargs)                                                                                                                                                                                                     
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 2399, in inline_call_                                                                                                                                              
    tracer.run()                                                                                                                                                                                                                                            
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 810, in run                                                                                                                                                        
    and self.step()
File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 773, in step
    getattr(self, inst.opname)(inst)
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 489, in wrapper
    return inner_fn(self, inst)
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 1260, in CALL_FUNCTION_EX
    self.call_function(fn, argsvars.items, kwargsvars)
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 674, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/functions.py", line 289, in call_function
    return super().call_function(tx, args, kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/functions.py", line 90, in call_function
    return tx.inline_user_function_return(
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 680, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 2285, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 2399, in inline_call_
    tracer.run()
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 810, in run
    and self.step()
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 773, in step
    getattr(self, inst.opname)(inst)
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 489, in wrapper
    return inner_fn(self, inst)
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 1219, in CALL_FUNCTION
    self.call_function(fn, args, {})
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 674, in call_function
    self.push(fn.call_function(self, args, kwargs))
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/functions.py", line 335, in call_function
    return super().call_function(tx, args, kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/functions.py", line 289, in call_function
    return super().call_function(tx, args, kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/functions.py", line 90, in call_function
    return tx.inline_user_function_return(
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 680, in inline_user_function_return
    return InliningInstructionTranslator.inline_call(self, fn, args, kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 2285, in inline_call
    return cls.inline_call_(parent, func, args, kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 2399, in inline_call_
    tracer.run()
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 810, in run
    and self.step()
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 773, in step
    getattr(self, inst.opname)(inst)
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py", line 1311, in LOAD_ATTR
    result = BuiltinVariable(getattr).call_function(
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/builtin.py", line 687, in call_function
    result = handler(tx, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/builtin.py", line 1304, in call_getattr
    return obj.var_getattr(tx, name)
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/misc.py", line 289, in var_getattr
    for name in self.inspected.inspect_parameter_names()
  File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/base.py", line 341, in inspect_parameter_names
File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/exc.py", line 190, in unimplemented
    raise Unsupported(msg)
torch._dynamo.exc.Unsupported: inspect_parameter_names: FunctoolsPartialVariable()

from user code:
   File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/external_utils.py", line 36, in inner
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py", line 1689, in generate
    self._validate_model_kwargs(model_kwargs.copy())
  File "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py", line 1207, in _validate_model_kwargs
    model_args |= set(inspect.signature(self.forward).parameters)

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information


You can suppress this exception and fall back to eager by setting:
    import torch._dynamo
    torch._dynamo.config.suppress_errors = True

4. As the infor suggested, I add these words:
```python
    import torch._dynamo
    torch._dynamo.config.suppress_errors = True
```

But I still got the same Error!

### Expected behavior

Run successfully

Comments:
The same setting and the same issue.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.