'torch_dtype' keyword not working with 'AutoModel'
Author: hwijeen
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

transformers version: 4.9.2
Platform: Linux-4.18.0-25-generic-x86_64-with-glibc2.10
Python version: 3.8.5
PyTorch version (GPU?): 1.8.0a0+52ea372 (True)
Tensorflow version (GPU?): not installed (NA)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Using GPU in script?:
Using distributed or parallel set-up in script?: No

### Who can help
@stas00 as he is the writer of the [#12316](https://github.com/huggingface/transformers/pull/12316).
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @Rocketknight1

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

1. Inspect the model weight data type.
```bash
wget --content-disposition  https://api.ngc.nvidia.com/v2/models/nvidia/megatron_lm_345m/versions/v0.0/zip -O checkpoint.zip
unzip checkpoint.zip
python -c "import torch; from pprint import pprint as print; sd=torch.load('./release/mp_rank_00/model_optim_rng.pt'); d= {d.dtype: 1 for d in sd['model']['language_model']['transformer'].values()}; print(d.keys())"
# dict_keys([torch.float16])
```

2. Try to load it with transformers in float16, which `torch_dtype` is supposed to be responsible for. But this only works with specific model classes and AutoModel blindly loads it into float32.
```bash
git clone https://github.com/huggingface/transformers.git
python3 transformers/src/transformers/models/megatron_bert/convert_megatron_gpt2_checkpoint.py checkpoint.zip

# load correctly with the specific model class
python -c "from transformers import GPT2LMHeadModel; print(GPT2LMHeadModel.from_pretrained('.', torch_dtype='auto').dtype)"
# torch.float16

# but fails to load it into float 16 with AutoModelForCausalLM
python -c "from transformers import AutoModelForCausalLM; print(AutoModelForCausalLM.from_pretrained('.', torch_dtype='auto').dtype)"
# torch.float32
```

3. This is because AutoModel [first puts `torch_dtype` argument passed to `AutoModel.from_pretrained` method into config](https://github.com/huggingface/transformers/blob/91ff480e2693f36b11aaebc4e9cc79e4e3c049da/src/transformers/configuration_utils.py#L576) and 
`PretrainedModel.from_config`, which is called by `AutoModel.from_pretrained`, checks for `torch_dtype` argument in [only in `kwargs` and not in config](https://github.com/huggingface/transformers/blob/91ff480e2693f36b11aaebc4e9cc79e4e3c049da/src/transformers/modeling_utils.py#L1297). 
<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
Setting`torch_dtype` to `auto` works correctly as explained in [#12316](https://github.com/huggingface/transformers/pull/12316).

I will open a PR to address this issue :)
<!-- A clear and concise description of what you would expect to happen. -->


Comments:
> This is because AutoModel first puts torch_dtype argument passed to AutoModel.from_pretrained method into config and
PretrainedModel.from_config, which is called by AutoModel.from_pretrained, checks for torch_dtype argument in only in kwargs and not in config.

But this is intentional. 

My PR was originally designed to have the dtype figuring out to be fully automated, but that wasn't accepted, so the `config.dtype` is saved, but at the moment being ignored on purpose. i.e. the user has to actively set `torch_dtype`.  See this part of the discussion https://github.com/huggingface/transformers/pull/12316#discussion_r659959617

Perhaps we should document somewhere that `config.torch_dtype` is saved for the future use (probably v5) but currently isn't automatically used. The user can of course do `from_pretrained(..., torch_dtype=config.torch_dtype)`.

This has been fixed in https://github.com/huggingface/transformers/pull/13209