Cannot save HQQ quantized model.
Author: mxjmtxrm
Pull request: no

### System Info

- `transformers` version: 4.41.0.dev0
- Platform: Linux-5.15.0-92-generic-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.21.4
- Safetensors version: 0.4.2
- Accelerate version: 0.28.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.2.0a0+81ea7a4 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

@SunMarc, @younesbelkada

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I loaded a model with HQQ quantization.Then I want to save the model but met the following error:
```
ValueError: The model is quantized with hqq and is not serializable - check out the warnings from the logger on the traceback to understand the reason why the quantized model is not serializable.
```
The code is as follows.
```
quantization_config  = HqqConfig(nbits=4, group_size=64, quant_zero=False, quant_scale=False, axis=0) #axis=0 is used by default
model = AutoModelForCausalLM.from_pretrained(
            model_kwargs['model_id_or_path'],
            torch_dtype=torch_dtype,
            trust_remote_code=True,
            low_cpu_mem_usage = low_cpu_mem_usage,
            device_map = device_map,
            attn_implementation="flash_attention_2"
        )
model.save_pretrained(output_path)
```
I found that `is_serializable` is False in quantizer_hqq.py. Why that? And how to save a HQQ quantized model?

### Expected behavior

--

Comments:
Hi @mxjmtxrm, this is indeed the case. You cannot serialize the model when using transformers integration because the serialization logic was not compatible with transformers code. However, you can use directly the hqq library if you want to save and relaod a quantized model https://github.com/mobiusml/hqq cc @mobicham 

@mxjmtxrm save/load was initially included in the pull request, but we decided to remove it because the logic was too different from the rest of the transformers models. For the moment, you can do something like this:
```Python
from hqq.models.hf.base import AutoHQQHFModel

#Save
AutoHQQHFModel.save_quantized(model, save_path)

#Load
AutoHQQHFModel.from_quantized(save_path)
```

I'm wondering that `is_serializable ` is True in AWQ/EETQ and other quantizers except HQQ.  If it can be set True directly in HQQ quantizer?

AWQ quantizer:
```
@property
    def is_serializable(self):
        # AWQ through auto-awq has been always serializable, except if the model is fused.
        if self.quantization_config.do_fuse:
            logger.warning("You cannot save an AWQ model that uses fused modules!")
            return False

        if self.quantization_config.version == AWQLinearVersion.EXLLAMA:
            logger.warning("You cannot save an AWQ model that uses Exllama backend!")
            return False

        return True
```


We set it to True when it is indeed possible to save the model with transformers logic. with EETQ and AWQ, they can indeed be serialized but this is not the case for HQQ. Hence we set it to False. 

It needs some major refactoring on the hqq lib side to support serialization with transformers.This is because it's not possible to store the meta-data directly as a safetensor. Since hqq supports different backends and various settings, that meta-data becomes cumbersome. Currently, I am trying to simplify things to make the refactoring easier. For the moment, you can use the approach shared above:
 
> @mxjmtxrm save/load was initially included in the pull request, but we decided to remove it because the logic was too different from the rest of the transformers models. For the moment, you can do something like this:
> 
> ```python
> from hqq.models.hf.base import AutoHQQHFModel
> 
> #Save
> AutoHQQHFModel.save_quantized(model, save_path)
> 
> #Load
> AutoHQQHFModel.from_quantized(save_path)
> ```


Got it. Thanks.