Allow tests in examples to use cuda or fp16,if they are available
Author: Joel-hanson
Pull request: yes

The tests in examples didn't use the Cuda or fp16 even if they were available.

- The text classification example (`run_glue.py`) didn't use the fp16 even if it was available but
  the device was take based on the availability(Cuda/CPU).
- The language-modeling example (`run_language_modeling.py`) was having `--no_cuda` argument which
  made the test work without Cuda. This example is having an issue when running with fp16
  thus it not enabled (got an assertion error for perplexity due to its higher value).
- The Cuda and fp16 is not enabled for question-answering example (`run_squad.py`) as it is having a
  the difference in the f1 score.
- The text-generation example (`run_generation.py`) will take the Cuda or fp16 whenever it is available.

Resolves some of #5057

Comments:
@sshleifer Hope you are doing well. sorry for the delay, I have created the PR for some of the related issues which were mentioned in #5057.
> 1. They never use Cuda or fp16, even if they are available.

I have some doubts which had encountered when making this PR

---
**1.** As you had said there where some test which failed when enabling the Cuda or fp16
- The Cuda and fp16 are not enabled for question-answering example (`run_squad.py`) as it is having a difference in the f1 score.
- The language-modeling example (`run_language_modeling.py`) is having an issue when running with fp16.
---
**2.** I was not able to find the `test_hans.py` but was able to find a readme to run it. Is this intentional if not shall I have a `test_hans.py` file to run the same.

![Screenshot 2020-07-04 at 11 56 30 AM](https://user-images.githubusercontent.com/17215044/86506506-7c634200-bded-11ea-8b1a-a49fa617f8ea.png)
---
**3.** This is the list of tests which I got
```bash
$ ls -l examples/**/test*.py

examples/bert-loses-patience/test_run_glue_with_pabee.py
examples/seq2seq/bertabs/test_utils_summarization.py
examples/seq2seq/test_seq2seq_examples.py
examples/test_examples.py
examples/token-classification/test_ner_examples.py
```
I was not able to find the `test_summarization_examples.py` and `test_t5_examples.py`. I think I am doing something wrong.

---
**4.** I can have made the PR for some tests only, can do the same for others if the current PR satisfies your requirement. 

1) noted
2)test_hans.py would be nice, but can be added in a separate PR.
3) You are all set, nothing is wrong. 
4) Would love to see the current PR!

Thanks!

>    2)test_hans.py would be nice, but can be added in a separate PR.

The test_hans.py was created once via the PR https://github.com/huggingface/transformers/pull/2239 but I think somehow or for some reason, it was removed

last interaction with the file was in the PR https://github.com/huggingface/transformers/pull/4213


also I missed this earlier, but what was the issue with run_language_modeling.py? Can you get a traceback?

> also I missed this earlier, but what was the issue with run_language_modeling.py? Can you get a traceback?

This is the short long and I have attached the pull log with this comment.
```log
        with patch.object(sys, "argv", testargs):
            result = run_language_modeling.main()
>           self.assertLess(result["perplexity"], 35)
E           AssertionError: 36.684365356893885 not less than 35
```
[test.log](https://github.com/huggingface/transformers/files/4955269/test.log)


@sshleifer Hope you are doing well.

Is this PR ok or Should I split this into multiple smaller PRs to make things easier?

Thanks for the review @sshleifer, I have updated the code accordingly and regarding the new command line args, I have removed the args which were related to the multi-GPU support as we can make that a separate PR. 

**Any Idea why the test `run_tests_tf` and `run_tests_torch_and_tf` failing? Is it related to my change?**

Thanks for the review @LysandreJik, I have updated the PR accordingly.

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5512?src=pr&el=h1) Report
> Merging [#5512](https://codecov.io/gh/huggingface/transformers/pull/5512?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/d9cbc0350deaa7e146a8c8dbb6ad4dc9bd6afc4f?el=desc) will **decrease** coverage by `0.72%`.
> The diff coverage is `78.87%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5512/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5512?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5512      +/-   ##
==========================================
- Coverage   80.37%   79.64%   -0.73%     
==========================================
  Files         156      156              
  Lines       28058    28261     +203     
==========================================
- Hits        22552    22509      -43     
- Misses       5506     5752     +246     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5512?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/commands/serving.py](https://codecov.io/gh/huggingface/transformers/pull/5512/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb21tYW5kcy9zZXJ2aW5nLnB5) | `0.00% <0.00%> (ø)` | |
| [src/transformers/commands/user.py](https://codecov.io/gh/huggingface/transformers/pull/5512/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb21tYW5kcy91c2VyLnB5) | `0.00% <ø> (ø)` | |
| [src/transformers/configuration\_reformer.py](https://codecov.io/gh/huggingface/transformers/pull/5512/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX3JlZm9ybWVyLnB5) | `100.00% <ø> (ø)` | |
| [src/transformers/data/processors/glue.py](https://codecov.io/gh/huggingface/transformers/pull/5512/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL3Byb2Nlc3NvcnMvZ2x1ZS5weQ==) | `48.91% <0.00%> (-0.18%)` | :arrow_down: |
| [src/transformers/data/test\_generation\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5512/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL3Rlc3RfZ2VuZXJhdGlvbl91dGlscy5weQ==) | `0.00% <0.00%> (ø)` | |
| [src/transformers/generation\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5512/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3V0aWxzLnB5) | `96.94% <ø> (ø)` | |
| [src/transformers/hf\_argparser.py](https://codecov.io/gh/huggingface/transformers/pull/5512/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9oZl9hcmdwYXJzZXIucHk=) | `67.74% <0.00%> (-1.49%)` | :arrow_down: |
| [src/transformers/modeling\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/5512/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19hdXRvLnB5) | `78.73% <ø> (ø)` | |
| [src/transformers/modeling\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/5512/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iZXJ0LnB5) | `88.42% <ø> (+0.16%)` | :arrow_up: |
| [src/transformers/modeling\_electra.py](https://codecov.io/gh/huggingface/transformers/pull/5512/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19lbGVjdHJhLnB5) | `82.13% <ø> (ø)` | |
| ... and [58 more](https://codecov.io/gh/huggingface/transformers/pull/5512/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5512?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5512?src=pr&el=footer). Last update [a573777...51be82f](https://codecov.io/gh/huggingface/transformers/pull/5512?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).