[`core`] PEFT integration
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR is an attempt to tightly integrate PEFT library with transformers, by offering users the ability to load PEFT models out of the box from `AutoModelForxxx.from_pretrained()` if the local directory or the Hub model id contains adapter weights and adapter config.

```python
import tempfile
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import AutoPeftModelForCausalLM

peft_model_id = "ybelkada/opt-350m-lora"
model = AutoModelForCausalLM.from_pretrained(peft_model_id)

with tempfile.TemporaryDirectory() as tmpdirname:
    peft_model = AutoPeftModelForCausalLM.from_pretrained(peft_model_id)
    peft_model.save_pretrained(tmpdirname)

    model = AutoModelForCausalLM.from_pretrained(tmpdirname)
    print(model)
``` 

Although this is similar to what have been introduced in https://github.com/huggingface/peft/pull/694 this PR offers a direct integration with transformers 

## TODOs:

- [x]  handle `PeftModel.from_pretrained(xxx)` kwargs
- [x] tests
- [x] docs (with the help of @stevhliu )

cc @sgugger @pacman100 @BenjaminBossan 

Comments:
I would like to have first review of the draft if possible, to see if we are inline with the approach 🙏 @sgugger - Thanks !

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24827). All of your documentation changes will be reflected on that endpoint.

Thank you for your review @pacman100 ! 
I think the canonical way to load PEFT models for inference would still be to use PEFT classes (i.e. either `AutoPeftModelForCausalLM` or `PeftModel` - btw we should encourage users to use more and more `AutoPeftModelForCausalLM` instead of `PeftModel`) 
This PR is intended to make things even easier for users and for further integrations with HF ecosystem (`pipeline`, `diffusers`) and it will be clearly documented. I also think we should update the inference widgets after the PEFT release.

> Mmm that's not the correct way to add a new peft job as it will always run on any PR, even if it shouldn't be run.

@younesbelkada 

You can take a look 

https://github.com/huggingface/transformers/blob/476be08c4aa96f8c1cae4200d2677bbe8f12cf80/utils/tests_fetcher.py#L720

(check `examples_test_list.txt` and `examples_tests_to_run` in the same file and the 2 CircleCI config files)
(you have to check against `peft_integration` in your case)

The design as is would make it hard for `diffusers` to leverage `transformes` to load PEFT weights for transformers models. 
In `diffusers` we have the following workflow:

```py
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5")
# now we have a loaded CLIPTextModel under `pipe.text_encoder`

pipe.load_lora(...)
# doing this means we would want to call `text_encoder.load_adapter(...)` under the hood
```

This means we necessarily need `transformers` to support the ability to load lora weights into already instantiated models, just like MMS allows it via `load_adapter` - see: https://huggingface.co/docs/transformers/v4.31.0/en/model_doc/wav2vec2#transformers.Wav2Vec2ForCTC.load_adapter.example

[Edit] We could come around it, by wrapping `pipe.text_encoder` into a `PeftModel` under the hood when doing `pipe.load_lora(...)` to transform `CLIPTextModel` into `PeftModel` but that:
- would break some internal `diffusers` code 
- force us to wrap logic around `transformers`, e.g. we could just do `pipe.text_encoder.load_adapter(...)`, but would have to first wrap every transformers model into a PEFT model

From purely a `transformers` point of view, I would also struggle a bit with the following:

1.) PEFT weights seemingly should only be loaded with `AutoModel`, which is restrictive as there is no need to go over the AutoModel class if one knows the model.

It does look like the following would be possible:

```py
from transformers import LlamaForCausalLM

model =  LlamaForCausalLM.from_pretrained("tloen/alpaca-lora-7b")
```

but then `model.__class__` is of type `PeftModel` which would be confusing to me - I used a class method of `LlamaForCausalLM`

2.)  I don't like that `.from_pretrained(<peft/model/id>)` more or less fully dispatches to the `peft` library instead of staying in Transformers' land. I imagined `peft` to be used as a utility library, not Transformers to dispatch to peft.


=> Could we not create a `PeftModelMixin` class so that `peft` operates more under the hood?

Closing as #25077 got merged