Issue while running Llama-2-7b-chat-gptq 
Author: bsurya27
Pull request: no

### System Info

- `transformers` version: 4.33.0
- Platform: Linux-5.15.109+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.3
- Accelerate version: 0.22.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.0.1+cu118 (True)
- Tensorflow version (GPU?): 2.12.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.7.2 (gpu)
- Jax version: 0.4.14
- JaxLib version: 0.4.14
- Using GPU in script?: True
- Using distributed or parallel set-up in script?: False

### Who can help?

@fxmarty @TheBloke

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

While trying to run the llama-2-7b-chat-gptq model on google colab.
I have installed the required libraries and am working on a GPU runtime.
I have defined the model the ALMOST same way as present in the read.me file on huggingface. The only things I have changed are the 'revision' argument to 'gptq-4bit-128g-actorder_True' from 'main' and 'device_map' to 'cuda:0' from 'auto'.
I try running the model.generate() function with the arguments set exactly to those in the read.me. I encounter a runtime error -
RuntimeError: The temp_state buffer is too small in the exllama backend. Please call the exllama_set_max_input_length function to increase the buffer size. Example:
from auto_gptq import exllama_set_max_input_length
model = exllama_set_max_input_length(model, 4096)

So I added those two lines of code mentioned and I face another error, this time while the model is being built. 
'LlamaForCausalLM' object has no attribute 'quantize_config', this is the error I get.


### Expected behavior

The expected behaviour for the code would be to run normally, present the user with a input box and generate text based on that input.

Comments:
cc @ArthurZucker 

Maybe I should be changing my README examples to disable exllama support by default

I'm unclear as to whether ExLlama kernels are meant to be fully supported via Transformers or not, or only when using AutoGPTQ directly?

@fxmarty could you clarify?

Also @bsurya27 , could you share the full traceback as well as a full reproducer? 
cc @SunMarc as well as he recently worked on `exllama` support in transformers

Hi @bsurya27 , please install the optimum library from source and let me know if it works. Exllama are meant to be fully supported via Transformers. 

This issue was fixed in https://github.com/huggingface/optimum/pull/1329. We will do tomorrow a release of Optimum including the fix.

> Also @bsurya27 , could you share the full traceback as well as a full reproducer? cc @SunMarc as well as he recently worked on `exllama` support in transformers

What is a full traceback? I'm sorry. I am newbie and am not aware of several terms.

> Hi @bsurya27 , please install the optimum library from source and let me know if it works. Exllama are meant to be fully supported via Transformers.

I have installed the optimum library, but I am still facing the issue.

> Maybe I should be changing my README examples to disable exllama support by default
> 
> I'm unclear as to whether ExLlama kernels are meant to be fully supported via Transformers or not, or only when using AutoGPTQ directly?
> 
> @fxmarty could you clarify?

Actually, the example which was in the older README file worked pretty well, and I didn't get any kind of Runtime error, so I never used the code exllama_set_max_input_length(model,4096).


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.