🚨🚨🚨 Enforce single model initialization
Author: sgugger
Pull request: yes

# What does this PR do?

There are currently three problems with the mode inits:

**Problem 1:** When not using the fast init (so in practice when using the model constructor or `AutoXxx.from_config` instead of `from_pretrained`) weights are initialized multiple times. @stas00 showed the example of `OPTForCausalLM` where we have a call to `post_init()` three times: in `OPTForCausalLM`, `OptModel` and `OptDecoder`. Each of those calls launches a recursive call of `_init_weights` to all submodules of the model, so this makes three inits.

**Problem 2:** The fast init (of random weights of the head in `from_pretrained`) and non-fast init (as above) are not always equivalent. This is because in `from_pretrained` init is done on calling `_init_weights` only on leaf modules with weights not present in the checkpoint, but sometimes `_init_weights` contains class checks for bigger modules ([here](https://github.com/huggingface/transformers/blob/77db257e2a67d4b043cf03bf390947fcd71a9f53/src/transformers/models/oneformer/modeling_oneformer.py#L2801) is one example in OneFormer)

**Problem 3:** Some of the models have `_init_weights` function that will initialize the same weights with two different ways. We can take back [this example](https://github.com/huggingface/transformers/blob/77db257e2a67d4b043cf03bf390947fcd71a9f53/src/transformers/models/oneformer/modeling_oneformer.py#L2801) in OneFormer which initializes a weight that is a Conv2D, but _init_weights is applied recursively, so that Conv2D will also be initialized [here](https://github.com/huggingface/transformers/blob/77db257e2a67d4b043cf03bf390947fcd71a9f53/src/transformers/models/oneformer/modeling_oneformer.py#L2891) with a different rule.

This PR should solve these three problems with one stone by changing slightly the `_init_weights` function to look for a private `_is_hf_initialized` attribute in the module and skip the init if it's there and `True`. Of course when initializing a module, this private attribute is set to `True` after the initialization is done.

This PR gets the 🚨🚨🚨 sign because it might break user's code if they were relying on the (buggy) init of composite models: if a model has an encoder or backbone that is initialized differently from the rest, the init of the encoder/backbone was previously erased by the bigger model init.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@stas00 In initial discussions with @LysandreJik , he mentioned he preferred not having a wrapper. Though the argument about init weights code in the wild is a sound one, so showed how it could look like with the last two commits.

Thanks for the PR, and for showing the two options! I feel like the wrapper is a little bit magical, but would make contributions simpler while reducing the complexity of the code.

I would go with the wrapper, if possible.

Thank you for making it simpler for the end user, Sylvain - I will test this today on m4 and get back to you.

Thank you for doing a massive adjustment work and the explanations, Sylvain!

This is hard work and very awesome for everybody to benefit from!

Last failing test is flaky so this is good for final review!

so it didn't make it into https://github.com/huggingface/transformers/releases/tag/v4.26.1, right?

do you know if you plan another hotfix release in the future or plan to wait for 4.27.0? 

Asking as I'm needing to anchor requirements on this fix for m4 where I found this bug.

This won't be until 4.27.0 as it could come with bugs we need to fix (and it's not a regression fix so won't go in a patch).

Thank you for the clarity, Sylvain. 4.27.0 it is.