[Benchmark] Memory benchmark utils
Author: patrickvonplaten
Pull request: yes

## Description

Currently, we have a benchmarks file in examples `examples/benchmarks.py` which measures time and speed for inference and one `src/transformers/benchmark_utils.py` file which including the memory tracing from @thomwolf.

This PR generalizes the benchmarks much more:
- We have `benchmark_args_utils, benchmark_args, benchmark_tf_args` similar to `trainer_args`. Because there is such a big overlap between tf and torch, I decided to have a`benchmark_args_utils`
- We have a `benchmark_utils` including the previous code of `benchmark_utils` and a new abstract`Benchmark` class. `PyTorchBenchmark` and `TensorflowBenchmark` inherit from this class and implement the abstract methods `inference` and `train`. 
- Instead of being able to just measure our "default" models identified by their `model_name`, we can create a special config and measure the memory and time usage of the corresponding model before starting to train
- In addition to inference we can also measure train
- **IMPORTANT**: The line-by-line tracing implemented by @thomwolf is super useful to optimize one's code and see exactly which line produces most of the code, but it does not give the peek memory usage as one can see in `nvidia-smi` and which is relevant to decide how much RAM is needed for the model. I decided to not use the line-by-line tracing to plot the overall memory usage on GPU. It should rather only be used when one want to optimize his/her code instead of benchmarking the model. For Benchmarks the PyTorch method `torch.cuda.max_memory_reserved` is a better fit - it's much faster and gives the correct value.
- Also I added a `run_script` and a `plot_script` to the examples.
- @julien-c - I think it would be super nice to have such plots on our website somewhere. If people can see quickly how much more efficient model x is over y in for a given batch size and sequence length, I think it would be super useful.

## TODO:
- [x] Write tests
- [x] Make pretty (more type hints + smaller refactoring)

## Review

@thomwolf, @julien-c - It would be awesome if you could take a look here and check if you are ok with the general design choice. To get a better feeling how the Benchmark can be used, you can checkout this notebook: https://colab.research.google.com/drive/1GM3WPXmZ5wZHwhdPyf7RykJS3Y12iceJ?usp=sharing

## UPDATE

I think the PR is ready for merge now. The notebook: https://colab.research.google.com/drive/1GM3WPXmZ5wZHwhdPyf7RykJS3Y12iceJ?usp=sharing is updated and shows what environment and version infos are shown @thomwolf . Tests are included. The next PR should take care of pytorch jit tracing and TPU and TF @LysandreJik @julien-c 

## Future PR:
- [ ] Support Pytorch Tracing (discuss with Lysandre a bit before)
- [ ] Support TPU (discuss with Lysandre a bit before)
- [ ] TF version


Comments:
This google colab shows nicely how the new benchmarking utils can be used: https://colab.research.google.com/drive/1GM3WPXmZ5wZHwhdPyf7RykJS3Y12iceJ?usp=sharing

A plotter file is included in `examples/benchmarking` which can give the following results:

### Memory

![memory_example](https://user-images.githubusercontent.com/23423619/82815931-8020b200-9e9a-11ea-99ad-d22517329bf1.png)

### Time

![time_example](https://user-images.githubusercontent.com/23423619/82815942-84e56600-9e9a-11ea-95b4-f9bc9b7a9511.png)


Can `csv_memory_filename_inference` be called `save_csv_path`?

> Can `csv_memory_filename_inference` be called `save_csv_path`?

Yeah about the exact naming I think we can decide a bit later. Wanna see first if you guys are ok with the general design. But we have up to four csv files that are saved - so `save_csv_path` is too general.

> This is really cool, I'm looking forward to using it.
> 
> It seems you've implemented TPU support as well, did you get a chance to try it?

I didn't play around with TPU very much yet. I'd like to merge this PR which works for GPU / CPU for PyTorch now and then maybe clean things for TPU in a second PR? 

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4198?src=pr&el=h1) Report
> Merging [#4198](https://codecov.io/gh/huggingface/transformers/pull/4198?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/4bd7cfc30952922b6329ec2d7637f13729317014&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4198/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4198?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4198   +/-   ##
=======================================
  Coverage   77.39%   77.39%           
=======================================
  Files         128      128           
  Lines       20939    20939           
=======================================
  Hits        16205    16205           
  Misses       4734     4734           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4198?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4198?src=pr&el=footer). Last update [4bd7cfc...4bd7cfc](https://codecov.io/gh/huggingface/transformers/pull/4198?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).