[models] respect dtype of the model when instantiating it
Author: stas00
Pull request: yes

update for future readers - the initially proposed API has changed through the process of review, so below is slightly autodated, e.g. there is no `torch_dtype_auto_detect`, but `torch_dtype=auto`.

----------------

This PR resolves the issue discussed in https://github.com/huggingface/transformers/issues/12062.

The main feature is: 
1. model will now be instantiated with the `dtype`  passed via `from_pretrained` and `from_config` `torch_dtype` arg
2. alternatively `from_pretrained` now has `torch_dtype_auto_detect` which can do the same automatically

Examples:
```
model = AutoModel.from_config(config, torch_dtype=torch.float16)
model = T5ForConditionalGeneration.from_pretrained(model_path, torch_dtype=torch.float16)
model = T5ForConditionalGeneration.from_pretrained(model_path, torch_dtype_auto_detect=True)
```

**Important: only float dtypes are supported by `torch.set_default_dtype(dtype)`, so if it's not float, say int8, an exception will be generated**

Changes:
- `PreTrainedModel` now has a new `_from_config` method where all the context managers for the model instantiation are done
- `auto_factory`'s `from_config` is back to being a thin wrapper - all the deepspeed stuff has now moved to PT's version of  `_from_config` 
- The PT's version of  `_from_config` now also sports the context manager-like functionality for dtype
- TF and Flax now have a thin `_from_config` method 
- `from_pretrained`: had to move `torch.load` before model instantiation to enabled auto-discovery
- `from_pretrained`: like `from_config` has a similar context manager for dtype
- extensive tests added

Possible changes:
- I wasn't sure whether to call `config.torch_dtype` or `config.dtype` - I went with the former as it'd be easy to rename after reviews. I don't know whether we want it generic or torch specific - I don't know if tf/flax use similar names, but I guess we could automatically remap those if needed.
- When saving the dtype I saved only "float32" part of "torch.float32" - I could save "torch.float32" instead - either way I have to reconstruct the dtype object from string. same uncertainty as in the item above. 
- the dtype context managers are poor man's versions since at the moment they are unique in each place, due to 3 frameworks using the same `from_pretrained` method - if we ever split it then we can use an actual context manager in `from_pretrained`.

Questions:
- should `T5ForConditionalGeneration.from_config(config)` be supported? Currently it's not and `T5ForConditionalGeneration(config)` ignores model instantiation context managers - just saves the config object
- probably should document this feature somewhere in the docs? Any suggestion where? It will work out-of-the-box but the documenting part would be handy for someone who wants to create a model from scratch in a non-default dtype.


Also note the new `log.info` entry:
```
Instantiating T5ForConditionalGeneration model under default dtype torch.float16
```
So one can tell what's happening.


The original version of this PR which tried to do the right thing automatically was dropped due to:

Possible issues:
- fp16 saved models now will be loaded as such and not fp32 as before - so some usages under CPU may fail, e.g. with:
```
RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'
```
e.g. many of our tiny models are fp16.

Another one on CUDA:
```
RuntimeError: Found param model.shared.weight with type torch.cuda.HalfTensor, expected torch.cuda.FloatTensor.
```


Fixes: https://github.com/huggingface/transformers/issues/12062

@sgugger, @LysandreJik, @patrickvonplaten 

Comments:
@sgugger, where should we document the functionality added by this PR? (besides the API).

ok, this should be good to go.

Yes, it looks great!

Great work @stas00, this is really useful!