Use bfloat16 data type for embeddings and masks when using with PyTorch amp for bfloat16 data type
Author: CaoE
Pull request: yes

### Motivation
Add a attribute `use_torch_bfloat16_embeddings` in `PretrainedConfig` to indicate if bfloat16 data type for embeddings and masks is used and convert the data type of embeddings and masks to bfloat16 accordingly.

This will reduce the number of data type conversion between float and bfloat16 when running models with `torch.cpu.amp.autocast(dtype=torch.bfloat16)` and improve the performance with little accuracy regression. This is because there are many residual modules in models and thus result in data type promotion by binary operations implemented by tensoriterator in PyTorch. 

For example: out = tensor1 + tensor2
If data type of tensor1 is float and tensor2 is bfloat16, pytorch will convert tensor2 to float and get float output. When running models using amp for bfloat16, the conversion will results in additional `to` operations, which will reduce performance.

### Testing


- Number of `to` operations

Model | wo/ bf16 embedding and masks| w/ bf16 embedding and masks
-- | -- | --
albert |  22 | 11
bert | 49 | 10
bart | 65 | 38
gpt2 | 56 | 29
distilbert | 40 | 19
roberta | 54 | 15


- Accuracy testing

Model | fp32 | amp bf16 | amp bf16 w/ bf16 embedding
-- | -- | -- | --
casual-language-modeling+xlm-roberta-base | 26.3895 | 26.5273 | 26.442 | this is loss
masked-language-modeling+bert-base-cased | 0.4819 | 0.4818 | 0.4819
masked-language-modeling+distilbert-base-cased | 0.3143 | 0.3158 | 0.3152
multiple-choice+distilbert-base-cased | 0.246 | 0.2461 | 0.2454
multiple-choice+google-electra-base-discriminator | 0.1193 | 0.1194 | 0.1201
text-classification+google-electra-base-generator | 0.6901 | 0.6838 | 0.6838
token-classification+google-electra-base-generator | 0.0414 | 0.0411 | 0.041
token-classification+gpt2 | 0.0379 | 0.0379 | 0.0379
albert | 0.453431373 | 0.428921569 | 0.446078431
distilbert | 0.681372549 | 0.681372549 | 0.681372549
roberta | 0.683823529 | 0.683823529 | 0.683823529
xlm-roberta | 0.637254902 | 0.637254902 | 0.639705882



## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18715). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.