AttributeError: 'Catcher' object has no attribute 'self_attn'
Author: whatnext
Pull request: no

### System Info

- `transformers` version: 4.38.1
- Platform: Linux-6.5.0-15-generic-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.21.2
- Safetensors version: 0.4.2
- Accelerate version: 0.27.2
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.1.2+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: parallel

### Who can help?

@fxmarty @ArthurZucker  

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

This problem seems to happen after finetuning and then AWQ quantization. I am not sure if the quantization process should be doing something it isn't, if the problem is with the model etc. so apologies if I have posted this issue in the wrong place. However, the following error is returned:

```
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/awq/models/base.py", line 155, in quantize
    self.quantizer = AwqQuantizer(
  File "/usr/local/lib/python3.10/dist-packages/awq/quantize/quantizer.py", line 56, in __init__
    self.modules, self.module_kwargs, self.inps = self.init_quant()
  File "/usr/local/lib/python3.10/dist-packages/awq/quantize/quantizer.py", line 472, in init_quant
    layer_kwargs = self.model.prepare_inputs_for_generation(samples, **layer_kwargs)
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/llama/modeling_llama.py", line 1256, in prepare_inputs_for_generation
    if past_key_value := getattr(self.model.layers[0].self_attn, "past_key_value", None):
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1695, in __getattr__
    raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'Catcher' object has no attribute 'self_attn'
```





### Expected behavior

It seems to me that it is expecting `self.model.layers[0]` to have the attribute `self_attn`, when actually it might not exist(?) If I replace the line `if past_key_value := getattr(self.model.layers[0].self_attn, "past_key_value", None):` with

```
past_key_value = None
if hasattr(self.model.layers[0], 'self_attn'):
      past_key_value = getattr(self.model.layers[0].self_attn, "past_key_value", None)
```

then I can run the quantized model

Comments:
Hey! Could you please share a reproducer?

Can confirm that this occurs with meta-llama/Llama-2-7b-hf when using the basic quantization example for autoawq.
Transformers version 4.38.1.

Confirming without a reproducer is not super helpful 😉 
Anyways I got the gist of it. cc @younesbelkada the fix is not very costly and we'd better rely on the generation config anyways for this (it's always available at this point). I can include this in the patch as it would be breaking otherwise, but it's pretty normal for us to rely on layers that we know exist like the full attention one!