AWQ models including activation as previous operation seems broken
Author: kevin3314
Pull request: no

### System Info

transformer==4.37.0
autoawq==0.16.0

### Who can help?

@SunMarc @younesbelkada

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
>>> from transformers import AutoModelForCausalLM
/usr/lib/python3/dist-packages/requests/__init__.py:89: RequestsDependencyWarning: urllib3 (2.1.0) or chardet (5.2.0) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
>>> model = AutoModelForCausalLM.from_pretrained("casperhansen/falcon-7b-awq", trust_remote_code=True)
You have loaded an AWQ model on CPU and have a CUDA device available, make sure to set your model on a GPU device in order to run your model.
/usr/local/lib/python3.10/dist-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
  return self.fget.__get__(instance, owner)()
Some weights of the model checkpoint at casperhansen/falcon-7b-awq were not used when initializing RWForCausalLM: ['transformer.h.0.mlp.act.scales', 'transformer.h.1.mlp.act.scales', 'transformer.h.10.mlp.act.scales', 'transformer.h.11.mlp.act.scales', 'transformer.h.12.mlp.act.scales', 'transformer.h.13.mlp.act.scales', 'transformer.h.14.mlp.act.scales', 'transformer.h.15.mlp.act.scales', 'transformer.h.16.mlp.act.scales', 'transformer.h.17.mlp.act.scales', 'transformer.h.18.mlp.act.scales', 'transformer.h.19.mlp.act.scales', 'transformer.h.2.mlp.act.scales', 'transformer.h.20.mlp.act.scales', 'transformer.h.21.mlp.act.scales', 'transformer.h.22.mlp.act.scales', 'transformer.h.23.mlp.act.scales', 'transformer.h.24.mlp.act.scales', 'transformer.h.25.mlp.act.scales', 'transformer.h.26.mlp.act.scales', 'transformer.h.27.mlp.act.scales', 'transformer.h.28.mlp.act.scales', 'transformer.h.29.mlp.act.scales', 'transformer.h.3.mlp.act.scales', 'transformer.h.30.mlp.act.scales', 'transformer.h.31.mlp.act.scales', 'transformer.h.4.mlp.act.scales', 'transformer.h.5.mlp.act.scales', 'transformer.h.6.mlp.act.scales', 'transformer.h.7.mlp.act.scales', 'transformer.h.8.mlp.act.scales', 'transformer.h.9.mlp.act.scales']
- This IS expected if you are initializing RWForCausalLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing RWForCausalLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
```

### Expected behavior

Some weights of the model checkpoint at casperhansen/falcon-7b-awq were not used when initializing RWForCausalLM:... should not appear.
I suspect that the root cause is that only Linear layers are replaced. This does not work if the precede is not Linear layer (e.g. Activation) and it is the case for falcon.
https://github.com/huggingface/transformers/blob/8eb74c1c8961e3dc8549bb1a76463c7658a63d43/src/transformers/integrations/awq.py#L108

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.