Add  AutoRound quantization support
Author: wenhuach21
Pull request: yes

This PR is intended to add support for AutoRound quantizer to the transformers library.
Paper  https://arxiv.org/abs/2309.05516
Git https://github.com/intel/auto-round

AutoRound supports mixed group size and bit-width quantization, as well as qbits kernels for CPU inference.

Currently, AutoRound needs to be installed from the source. We will address your concerns and release a new version before merging

Comments:
cc @younesbelkada @SunMarc 

Hi @younesbelkada @SunMarc, could you kindly have a review when you are available? The unit test failure seems unrelated to my PR, if it does, please give me some advices and I will fix it soon.

Currently please manually install AutoRound from the source "git clone https://github.com/intel/auto-round.git && cd auto-round && pip install -vvv --no-build-isolation -e .". We will address your concerns and release a new version before merging

We have  demonstrated impressive results in the paper and on the https://huggingface.co/spaces/Intel/low_bit_open_llm_leaderboard across 2-bit to 4-bit quantization.

Thanks @wenhuach21 for the contribution - I would suggest moving this integration as part of `optimum-intel` where we do already have some deep integration with Intel Neural Compressor which provides the `AutoRound` algorithm.

I'm more than happy for us to work with you to integrate the necessary stuff in `optimum-intel` to make `auto-round` available for the community.

> Thanks @wenhuach21 for the contribution - I would suggest moving this integration as part of `optimum-intel` where we do already have some deep integration with Intel Neural Compressor which provides the `AutoRound` algorithm.
> 
> I'm more than happy for us to work with you to integrate the necessary stuff in `optimum-intel` to make `auto-round` available for the community.

@mfuntowicz Thank you for your reply and suggestion. However, we would prefer to merge into the Transformers repository, if that's acceptable to you. The main benefit of this integration is that it allows us to use the unified API provided by the Transformers library. This will be beneficial to the community, for example,  it enables easy evaluation of the AutoRound model with tools like lm-eval.

We would appreciate it if you could consider AutoRound just like other algorithms. It is not specifically for Intel hardware; we also support other devices.

> Thanks @wenhuach21 for the contribution - I would suggest moving this integration as part of `optimum-intel` where we do already have some deep integration with Intel Neural Compressor which provides the `AutoRound` algorithm.
> 
> I'm more than happy for us to work with you to integrate the necessary stuff in `optimum-intel` to make `auto-round` available for the community.

@mfuntowicz Please note that the quantization algorithm is generic and can be applicable to multiple HWs (not limited to Intel HW). We hope Auto-Round would benefit to broad Transformer community like AutoGPTQ and AutoAWQ but deliver much better accuracy on low-bit LLMs, while the implementation under Intel Neural Compressor is developed and validated on Intel platforms.

Thanks for the additional context. 

I checked internally and it's going to land very soon on the `optimum-intel` side.

In a matter of clarity for our users and to avoid having to support multiple implementations we will prioritize the one being integrated in `optimum-intel` and do the necessary to advertise usage from there 😊. 

I hope you understand our position,
Thanks again for the contribution!

> Thanks for the additional context.
> 
> I checked internally and it's going to land very soon on the `optimum-intel` side.
> 
> In a matter of clarity for our users and to avoid having to support multiple implementations we will prioritize the one being integrated in `optimum-intel` and do the necessary to advertise usage from there 😊.
> 
> I hope you understand our position, Thanks again for the contribution!

Thanks @mfuntowicz for sharing your thoughts. I understood and it's totally fine to have the algo integrated into Optimum-Intel first, while it's different from what we planned to contribute to Transformer if you look at the PR, which includes both algo and the kernel. For Optimum-Intel which integrated Intel Neural Compressor, it actually supports GPTQ, AWQ, and AutoRound algo only.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Continue improving the AutoRound. Make it live.