GPTQ integration
Author: SunMarc
Pull request: yes

# What does this PR do?
This PR adds the possibility to perform GTPQ quantization with transformers model using [optimum](https://github.com/huggingface/optimum)  library. The backend relies on [auto_gptq](https://github.com/PanQiWei/AutoGPTQ) library where we use `GTPQ` and `QuantLinear` class. 
Here's the related [PR](https://github.com/huggingface/optimum/pull/1216) on optimum side. This PR can only be merged after the optimum one. 

### Quantize model 
Unlike `bitsandbytes`, it is not feasible to quantize the weights right after loading them in order to reduce memory consumption. This is why, we need to first load the entire model and then quantize it (all done in `from_pretrained`)
```py
from transformers import AutoModelForCausalLM, AutoTokenizer, GPTQConfig
model_name = "facebook/opt-125m"

tokenizer = AutoTokenizer.from_pretrained(model_name)
config = GPTQConfig(bits=4, dataset = "c4", tokenizer=tokenizer,  group_size=128, desc_act=False)
# works also with device_map (cpu offload works but not disk offload)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16, quantization_config=config)
```

### Save hf model 
We save the `quantization_config` in `model.config`
```py
# move to the device you want to save model (needed if you used device_map before)
model.to(device)
quantized_model_folder = 'opt-125m-quantized_hf'
model.save_pretrained.(save_folder)
```

### Load quantized weights
If the `model.config` has a `quantization_config` key, we will replace the layers of the model and load the quantized weights.
```py
quantized_model_from_saved = AutoModelForCausalLM.from_pretrained(quantized_model_folder,
                                                                  device_map = "auto")
```

TODO:
- [ ] merge optimum PR first
- [x] doc

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> 2- Would it make sense to slowly "deprecate" the args load_in_4bit and load_in_8bit as it might lead to confusion for users (because technically you can do 4bit / 8bit with GPTQ as well) - not sure about the right approach here

Yes, it would make sense to deprecate these args as we will add more quantization methods in the future. It will confuse the user to have load_4_bit and load_8_bit only for bitsandbytes quantization. 

> 3- Note that I am not sure GPTQ works out of the box for vision / multimodal models, and would be better / safer if we just support text models for now (it would break at the tokenizer init call). How can we have a safety checker that effectively checks if the model is a text model?

We should definitely limit it to text model for now. For now, it works for decoder or encoder model but not for more complex architecture with multiple transformers block like decoder-encoder model. I will need to check if it can be easily extended. 

> Just to confirm, does all the bnb slow tests pass with this PR? 🙏 

Yes ! Both gptq and bnb slow tests passed

> Left few clarification and nits on the documentation. I think it is worth it to add a check to raise an error or warning if the model is not a pure text model. I think the easiest would be to check if self.main_input_name contains input_ids.

Added

> What do you think of adding an extra dependency list, [similarly as agents](https://github.com/huggingface/transformers/blob/main/setup.py#L410) that adds all the required packages to play with various supported quantization schemes (bitsandbytes, optimum, auto-gptq, accelerate)

Yes, we can do that after the release of optimum ! 

Thanks for this work! I noticed that the supported bits in quantization config doesn't match with `auto_gptq`

In `auto_gptq`:
```
.../auto_gptq/nn_modules/qlinear/qlinear_cuda.py", line 38, in __init__
    raise NotImplementedError("Only 2,3,4,8 bits are supported.")
NotImplementedError: Only 2,3,4,8 bits are supported.
```

In `transformers`:
```
.../transformers/utils/quantization_config.py", line 395, in post_init
    raise ValueError(f"Only support quantization to [2,4,6,8] bits but found {self.bits}")
ValueError: Only support quantization to [2,4,6,8] bits but found 3
```

Is there any specific reason behind it?

Hi thanks for reporting. This is indeed a mistake. I will fix this in a follow up PR ! 

@SunMarc @fxmarty  sorry for hijacking this PR, but just wanted to ask that since Marlin has been available since Feb 2024 in AutoGPTQ, are there plans to have it available in HF also? I think the main difficulty is that the way marlin weights are handled in a little unconventional manner in autogptq, having requiring a function [prepare_model_for_marlin_load](https://github.com/AutoGPTQ/AutoGPTQ/blob/ea4a99778f90b60c9b5177d7487af1b4ca87744f/auto_gptq/modeling/_base.py#L1240-L1247) that needs to happen before the `dispatch_model` and `post_init`.

But it could possibly be added by emulating the `BaseGPTQForCausalLM.from_quantized` logic into `PretrainedModel.from_pretrained`. 

@fabianlim, yes I think @robertgshaw2-neuralmagic was interested in adding this to transformers, that would be interesting indeed given the perfs https://github.com/huggingface/optimum/tree/main/tests/benchmark#gptq-benchmark