HIGGS Quantization Support
Author: BlackSamorez
Pull request: yes

### HIGGS 0-Shot Quantization

HIGGS is a new 0-shot quantization algorithm that combines Hadamard preprocessing with MSE-Optimal quantization grids to achieve lower quantization error and SOTA performance. You can find more information in the [paper](https://arxiv.org/abs/2411.17525).

Runtime support for HIGGS is implemented through [FLUTE](https://arxiv.org/abs/2407.10960), and its [library](https://github.com/HanGuo97/flute?tab=readme-ov-file).

This PR **adds support for HIGGS+FLUTE into `transformers` allowing for low-error 0-shot quantization and fast LLM inference**.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @ylacombe, @eustlb
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
cc @SunMarc @MekkCyber

cc @MekkCyber 

Failed tests look like a problem on the runner's end

Hey @BlackSamorez, Thanks for adding this quantization method so quickly ! I added some very small nits 

@SunMarc @MekkCyber thanks for your feedback!
I think I addressed all of your concerns.

Monday ping just in case :)
@SunMarc @MekkCyber

Hey @BlackSamorez, thanks for iterating and integrating this method 🤗 ! LGTM ! Can you add a documentation in `docs/source/en/quantization/higgs.md` to explain a bit how Higgs works, why it only works for models > 8B for now, and how to load a quantized model ? Also i left a small question 

Added documentation

Added usage example in docs.
Also, I had to slightly rework the`HiggsConfig` to add more quantization parameters. It turned out that the previous hard- coded setup didn't work for `gemma` models. Now it should be fine.

Thank you @BlackSamorez ! Looks great to me ! Just out of curiosity what was the reason why hard-coding the `group_size` don't work with `gemma`, it looks the same 

> Thank you @BlackSamorez ! Looks great to me ! Just out of curiosity what was the reason why hard-coding the `group_size` don't work with `gemma`, it looks the same

It wasn't `groups_size`, it was `hadamard_size`. We have to pad matrices to a multiple of it. It was 1024, but gemma internal dimensions are not divisible by 1024. Padding occurred, and pre-compiled kernels stopped working for new shapes.
Lowering `hadamard_size` to 512 by default fixed the issue. 

@SunMarc Hi! Is there a chance you could take a look at it this week? We wanted to release it before NIPS.

> @SunMarc Hi! Is there a chance you could take a look at it this week? We wanted to release it before NIPS.

I'll do that now. When is the deadline ? After my review, I still need this to be reviewed by a core maintainer 

> When is the deadline ?

There isn't a hard deadline, but it would be very nice to have this merged this week or early next week.

Pre-quantized a bunch of models (including `Llama-3.3-70B-Instruct`) and added a link to the collection to docs.

Gentle ping @ArthurZucker 

Can you fix the CI with `make style` for the quality and potentially rebase the PR @BlackSamorez  ? 

Things to recheck later: 
 - `make style` doesn't actually fix `python utils/custom_init_isort.py --check_only` errors. I had to run `python utils/custom_init_isort.py` manually.
 - `pillow` (`PIL`) is needed for `make quality` but isn't included in `pip install -e .[quality]`

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34997). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Quantization tests are nightly, right?

That's right !