4-bit QLoRA via bitsandbytes (4-bit base model + LoRA)
Author: TimDettmers
Pull request: yes

# What does this PR do?

This PR introduces 4-bit QLoRA to transformers. The main changes are for the bitsandbytes config. Additionally, we added one change to the LLaMA implementation where there is a bug that the data type chances if layer norms are in 32-bit and the rest is in bf16.

More information about QLoRA from our abstract:
>We develop QLoRA tuning, a method that finetunes by backpropagating gradients through a frozen 4-bit base model into low rank adapters (LoRA). With QLoRA tuning we can finetune 30B/65B parameter models on 24/48GB GPUs while preserving regular 16-bit full finetuning runtime and task performance. We achieve the memory efficiency and quantization precision through a combination of new methods: nested quantization to reduce the average memory footprint from 4.5 to 4.1 bits per parameter, paged optimizers to manage gradient checkpointing memory spikes, and a new data type, 4-bit NormalFloat (NF4), which is information theoretically and empirically optimal for normally distributed weights. To demonstrate the effectiveness and ease of use of QLoRA tuning we finetune more than 1,000 models to create a detailed dissection of instruction following performance across datasets (FLAN, Alpaca, Chip2, SuperNatural Instructions, Chip2, AnthropicHH), models types (LLaMA, T5), and model scales (125M to 65B). A discussion of the results is forthcoming in our paper.


## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@sgugger @younesbelkada @sourabh112 


Comments:
Amazing! 
cc @SunMarc for visibility as well! 
Will review asap 💪 

_The documentation is not available anymore as the PR was closed or merged._

Thank you for the review! I created a separate PR for the LLaMA bug: https://github.com/huggingface/transformers/pull/23535

The optimizers are also needed for the 4-bit fine-tuning of LLaMA 30B/65B on a single 24/48 GB GPU: https://github.com/huggingface/transformers/pull/23217

I reverted the names and removed code into other PRs. I added the missing test files. These, however are also renamed. Is the filename important for these?

Let me know if you see any other issues. Thank you, Sylvain!

@TimDettmers  Can the bitsandbytes team provide a corresponding bitsandbytes branch that corresponds to this PR?  The "closed_beta" on bitsandbytes is the closest I found but it doesn't appear to be final/rc-quality and contains debug print logs such as 

https://github.com/TimDettmers/bitsandbytes/compare/main...closed_beta#diff-4d235c7e595546c6656c229dfa139298ce6602b356c2d0bafcb2352eb2cfae79R222

Without the proper branch/link to bitsandbytes changes, it is very hard to test this. Since this PR is public, the bnb branch should no longer be in closed beta. 

Thank you.



I think the changes for QLoRA were recently merged in the main branch of bitsandbytes.