Support for custom model transform in `PreTrainedModel.from_pretrained`
Author: fxmarty
Pull request: yes

Add the possibility to pass a custom transform with signature `(model: torch.nn.modules.module.Module, **kwargs) -> None` that can do any transform on the model.

This can be seen as an extension of the bitsandbytes integration to be able to pass any transform that e.g. modify the model's keys. A direct application is an easier integration of SmoothQuant, k-bit quantization in transformers. Defining the transform should be left to an external library.

Some other necessary code modifs could be missing, I see the bitsandbytes integration modified a bit more.

This is just a draft to see if it's a good idea or not.

## Before submitting
- [ ] write doc
- [ ] write tests
- [ ] check if it works with fx.GraphModule


## Who can review?

@sgugger I'd be happy to have your opinion on this.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_21091). All of your documentation changes will be reflected on that endpoint.

I see the point, but then isn't bitsandbytes integration itself against the transformers philosophy as well? Why was bitsandbytes integrated in `from_pretrained`? This PR allows as an opt-in to plug external code in transformers, it does not change the one-model-one-file experience for existing models using the vanilla code.

I'd see this kind of modularity as a compromise to allow to plug in transformers modifications to the modeling that users may want to do programmatically, but that we don't want to host in the canonical modeling implementations.

Edit: Ok so from our offline discussion, I understand that the idea in this PR breaks the "load from transformers" on the Hub, which is something we want to avoid. Model loading should work out of the box from the Hub. An alternative idea is to have a quantization config directly in the model's config, and do the transform from there. I'll have a look at this.

To fully put the result of our offline discussion in the open, the idea would be to:
- migrate from several arguments (like `load_in_8bit=True`) in `from_pretrained` to a quantization config as there are many other quantization methods we would like to support.
- when we quantize the model in `from_pretrained`, set it in the config (so for now indicate with flags which 8bit parameter have been used and when we migrate to a quantization config, put it in the model config)
- this way when the quantized model is pushed to the Hub, the information about how it was quantized is there. We can thus adapt the `from_pretrained` method to prepare the model just after its instantation and before the state dict is loaded using this information.

I'll also add this to the agenda of our next core maintainer meeting to make sure we are all fully aligned on this :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.