fix unnessasry cpu memory usage when training
Author: xiye17
Pull request: yes

Use scalar when recording the aggregated training loss throughout the training procedure in `Trainer`.

The previous version uses `torch.Tensor` to store `tr_loss`. Here, tr_loss is accumulating history across your training loop and takes up lots of CPU memory during training. There is an official pytorch post on **NOT to accumulate history across your training loop**. ([https://pytorch.org/docs/stable/notes/faq.html#my-model-reports-cuda-runtime-error-2-out-of-memory](https://pytorch.org/docs/stable/notes/faq.html#my-model-reports-cuda-runtime-error-2-out-of-memory))

 When I tried to train a sentence classififier with batch size 10 on HotpotQA dataset (for document filtering), the program consumed around 120G CPU memory when I trained for 240,000 steps (roughly 3 epoch over all 900,000 HotpotQA documents), which is unnessasry. 

So I think it's better to just use a scalar (as suggested by the official post). Also, the tensor format returned value has never been used in the whole repo. It's a safe fix.


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7276?src=pr&el=h1) Report
> Merging [#7276](https://codecov.io/gh/huggingface/transformers/pull/7276?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/7cbf0f722d23440f3342aafc27697b50ead5996b?el=desc) will **decrease** coverage by `1.15%`.
> The diff coverage is `60.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/7276/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/7276?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #7276      +/-   ##
==========================================
- Coverage   80.32%   79.17%   -1.16%     
==========================================
  Files         174      174              
  Lines       33446    33445       -1     
==========================================
- Hits        26867    26481     -386     
- Misses       6579     6964     +385     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/7276?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/7276/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `55.80% <60.00%> (+0.08%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_funnel.py](https://codecov.io/gh/huggingface/transformers/pull/7276/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9mdW5uZWwucHk=) | `18.53% <0.00%> (-75.51%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/7276/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9vcGVuYWkucHk=) | `22.03% <0.00%> (-73.03%)` | :arrow_down: |
| [src/transformers/tokenization\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/7276/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25feGxtLnB5) | `16.26% <0.00%> (-66.67%)` | :arrow_down: |
| [src/transformers/configuration\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/7276/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX29wZW5haS5weQ==) | `34.28% <0.00%> (-62.86%)` | :arrow_down: |
| [src/transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/7276/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19vcGVuYWkucHk=) | `23.87% <0.00%> (-58.39%)` | :arrow_down: |
| [src/transformers/tokenization\_marian.py](https://codecov.io/gh/huggingface/transformers/pull/7276/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fbWFyaWFuLnB5) | `68.96% <0.00%> (-30.18%)` | :arrow_down: |
| [src/transformers/activations\_tf.py](https://codecov.io/gh/huggingface/transformers/pull/7276/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9hY3RpdmF0aW9uc190Zi5weQ==) | `54.16% <0.00%> (-20.84%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/7276/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9jdHJsLnB5) | `83.74% <0.00%> (-14.14%)` | :arrow_down: |
| [src/transformers/modeling\_gpt2.py](https://codecov.io/gh/huggingface/transformers/pull/7276/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19ncHQyLnB5) | `79.03% <0.00%> (-7.80%)` | :arrow_down: |
| ... and [13 more](https://codecov.io/gh/huggingface/transformers/pull/7276/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/7276?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7276?src=pr&el=footer). Last update [7cbf0f7...e9bc37d](https://codecov.io/gh/huggingface/transformers/pull/7276?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).