Safetensors serialization by default
Author: LysandreJik
Pull request: yes

This PR aims to do one thing but is larger than expected. I'm happy to break it down into smaller PRs if it helps for reviewing.

This PR aims to switch safe serialization to `True` by default for `torch` models. In doing so, it revealed a few bugs in the existing implementation and `safetensors` support that this PR fixes.

Additionally, support for `safetensors` for Flax models is added so that models saved from PyTorch after merging this PR can be used in both TensorFlow and Flax, and for models saved from TensorFlow/Flax to be loaded in PyTorch models.

The following should be worked on shortly to enable switching to safetensors by default for TensorFlow and Flax as well:
- There is no support for sharded weights in TensorFlow
- There is no support for sharded weights in Flax

Additionally, I'll contribute some documentation making the following clear:
- TensorFlow models can load models in safetensors saved from PyTorch and TensorFlow, but it cannot load them from Flax. This can be eventually worked on; meanwhile, I'll write this in the docs with workarounds to get models saved in Flax to work in TensorFlow for those interested.
- Same, but for Flax models loaded from TensorFlow

Thanks, @Rocketknight1, for the help on TensorFlow's side.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@Narsil, this is what is currently supported and not supported:

- TF -TF - Supported, tested here: https://github.com/huggingface/transformers/blob/a79600fc26d57a9262ce069434e3bc0ea6b8ef47/tests/test_modeling_tf_utils.py#L501-L509
- TF - Flax - **Not supported**
- TF - Pt - Supported, tested here: https://github.com/huggingface/transformers/blob/a79600fc26d57a9262ce069434e3bc0ea6b8ef47/tests/test_modeling_tf_utils.py#L511-L522
- Flax - Flax - Supported, tested here: https://github.com/huggingface/transformers/blob/a79600fc26d57a9262ce069434e3bc0ea6b8ef47/tests/test_modeling_flax_utils.py#L236-L244
- Flax - TF - **Not supported**
- Flax - Pt - Supported, tested here: https://github.com/huggingface/transformers/blob/a79600fc26d57a9262ce069434e3bc0ea6b8ef47/tests/test_modeling_flax_utils.py#L246-L256

> From you initial comment I understand it's not possible, but it's not entirely clear for me as to why (you mention sharded weights, is it the only restriction? If yes, from what I read it should be okay-ish to be able to at least load for those, no ?)

I mention this in the PR description:

> TensorFlow models can load models in safetensors saved from PyTorch and TensorFlow, but it cannot load them from Flax. This can be eventually worked on; meanwhile, I'll write this in the docs with workarounds to get models saved in Flax to work in TensorFlow for those interested.

It should be pretty straightforward to enable it, but I suspect extremely little usage for a TF <> Flax conversion where no PyTorch conversion exists. I'm planning to add this to the documentation and IMO we can work on it afterwards if there are requests.

I will proceed to merge this and write a small explanatory doc tomorrow. I would like for the slow tests to run on this before the release.

Awesome ! Thanks a LOT for this.