Efficient Inference Kernel for SpQR 
Author: elvircrn
Pull request: yes

# What does this PR do?



<!-- Remove if not applicable -->

Adds support for efficient single-batch inference for [SpQR](https://github.com/Vahe1994/SpQR/tree/main/spqr).

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@SunMarc @MekkCyber


Comments:
Hey @elvircrn, Thanks for adding this quantization method ! A very smooth integration 🔥! Just left a few comments 

@MekkCyber I've removed the Draft tag from the PR.  

It seems that I am unable to pass  the `test_raise_if_non_quantized`. Could you give me some pointers on this? 


Could you rebase @elvircrn to update the branch :)

@MekkCyber Rebase done.

Hey @elvircrn, thanks for iterating ! It looks great I just left some minor comments 

@MekkCyber The latest batch of comments are addressed.

Thanks for the quick iteration @elvircrn 🔥 ! can you re-rebase please 😅 

@MekkCyber rebased.

@MekkCyber Resolved the latest batch of comments.

@SunMarc @MekkCyber 
I've pushed a check for keys() in shapes in `replace_with_spqr_linear`. Let me know if something in this ballpark works.

@SunMarc The comments are addressed (I requested more info on some of them) and the PR is rebased.

@MekkCyber @SunMarc I resolved all the comments and rebased the branch.

Hi @elvircrn, can you fix the conflicts in the branch, they are related to `vptq` and `higgs` merge, thank you 🤗 

@MekkCyber rebased

@stevhliu I rewrote your suggested README edit, this should be enough to cover the basics of the approach. For a more complete description of the algorithm, the user is refered to the original publication.

(I am also writing a separate paper on the specifics on the kernel which I hope to publish soon on arxiv, but this shouldn't be a blocker for the merge).

pinging @ArthurZucker to have a quick look ! 

Could you fix the conflits @elvircrn ? 

@SunMarc resolved quickly, as it was just an md conflict in the overview table.

@SunMarc @ArthurZucker Conflicts resolved.

Can you just fix this:
```
RuntimeError: The following files are not present in the table of contents:
- quantization/spqr
Add them to ../transformers/docs/source/en/_toctree.yml.
```

@ArthurZucker Is the PR in order now?

@ArthurZucker Erreneous rebase issue resolved.

Hi @elvircrn, can you just resolve the conflicts and once the CI is green we can merge ! Again sorry for the delay 🙏 and thanks for your patience

@MekkCyber Done.

@MekkCyber The failing tests, based on the logs, seem to be unrelated to my own source code.

Yes they seem unrelated @elvircrn, i tried updating the branch I hope it works

@MekkCyber I don't think that did it - unrelated tests are still failing.

@MekkCyber Seems that everything is green now. :)

Thanks everyone!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34976). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.