Added performance enhancements (XLA, AMP) to examples
Author: tlkh
Pull request: yes

Summary of changes

- Minor enhancements to `run_tf_glue.py` (e.g. calculate train/val steps from number of train/val examples, standardize quotes etc.)
- Added option for mixed precision (Automatic Mixed Precision / AMP) to run models on Tensor Cores (NVIDIA Volta/Turing GPUs) and future hardware
- Added option for XLA, which uses the XLA compiler to reduce model runtime
- Options are toggled using `USE_XLA` or `USE_AMP`

Quick benchmarks from the script (no other modifications):

| GPU    | Mode | Time (2nd epoch) | Val Acc (3 runs) |
| --------- | -------- | ----------------------- | ----------------------|
| Titan V | FP32 | 41s | 0.8438/0.8281/0.8333 |
| Titan V | AMP | 26s | 0.8281/0.8568/0.8411 |
| V100    | FP32 | 35s | 0.8646/0.8359/0.8464 |
| V100    | AMP | 22s | 0.8646/0.8385/0.8411 |
| 1080 Ti | FP32 | 55s | - | 

Mixed precision (AMP) reduces the training time considerably for the same hardware and hyper-parameters (same batch size was used).

>**Important Note**
>
>Unrelated to this PR, but restoring the PyTorch model for the TF2 saved model does not work. This does not work in the original, unmodified example script. [Here](https://github.com/huggingface/transformers/blob/master/transformers/modeling_tf_pytorch_utils.py#L201) is the offending line in the Transformers library that references a uninitialized variable. This is fixed by PR #1509 

Feedback and comments welcome! 

Related: Issue #1441 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1508?src=pr&el=h1) Report
> Merging [#1508](https://codecov.io/gh/huggingface/transformers/pull/1508?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/a701c9b32126f1e6974d9fcb3a5c3700527d8559?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/1508/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/1508?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #1508   +/-   ##
=======================================
  Coverage   85.98%   85.98%           
=======================================
  Files          91       91           
  Lines       13579    13579           
=======================================
  Hits        11676    11676           
  Misses       1903     1903
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/1508?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1508?src=pr&el=footer). Last update [a701c9b...2c1d556](https://codecov.io/gh/huggingface/transformers/pull/1508?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Great, looks good to me cc @LysandreJik.

Adding the info of the PR in the example README.

Ok, merging, thanks @tlkh!