fixed trainer tr_loss memory leak
Author: StuartMesham
Pull request: yes

Fixes #6939 

The Trainer class contains the memory leak described [here](https://discuss.pytorch.org/t/cpu-ram-usage-increasing-for-every-epoch/24475/6). It is not specific to any particular model type and will occur with any model trained using the trainer class.

It is fixed in this pull request.

The issue is demonstrated in [this Colab notebook](https://colab.research.google.com/drive/1KQZEiZtfY14sDiAQnfw0gkKj9eC6XpTm?usp=sharing). The model trains for around 20 000 steps (this takes around 10 minutes on a t4) before using up all 12GB of RAM.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6999?src=pr&el=h1) Report
> Merging [#6999](https://codecov.io/gh/huggingface/transformers/pull/6999?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/90ec78b5140251f093f658ebd4d2925e8c03f5e6?el=desc) will **decrease** coverage by `1.44%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/6999/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/6999?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #6999      +/-   ##
==========================================
- Coverage   80.58%   79.14%   -1.45%     
==========================================
  Files         161      161              
  Lines       30123    30123              
==========================================
- Hits        24276    23841     -435     
- Misses       5847     6282     +435     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/6999?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/6999/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `54.95% <100.00%> (ø)` | |
| [src/transformers/modeling\_tf\_longformer.py](https://codecov.io/gh/huggingface/transformers/pull/6999/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9sb25nZm9ybWVyLnB5) | `16.37% <0.00%> (-82.31%)` | :arrow_down: |
| [src/transformers/modeling\_lxmert.py](https://codecov.io/gh/huggingface/transformers/pull/6999/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19seG1lcnQucHk=) | `70.01% <0.00%> (-20.75%)` | :arrow_down: |
| [src/transformers/modeling\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/6999/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19yb2JlcnRhLnB5) | `77.73% <0.00%> (-19.35%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/6999/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `84.21% <0.00%> (-2.26%)` | :arrow_down: |
| [src/transformers/tokenization\_utils\_base.py](https://codecov.io/gh/huggingface/transformers/pull/6999/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHNfYmFzZS5weQ==) | `94.03% <0.00%> (+0.27%)` | :arrow_up: |
| [src/transformers/modeling\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/6999/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iYXJ0LnB5) | `94.41% <0.00%> (+0.50%)` | :arrow_up: |
| [src/transformers/activations.py](https://codecov.io/gh/huggingface/transformers/pull/6999/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9hY3RpdmF0aW9ucy5weQ==) | `90.00% <0.00%> (+5.00%)` | :arrow_up: |
| [src/transformers/data/data\_collator.py](https://codecov.io/gh/huggingface/transformers/pull/6999/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL2RhdGFfY29sbGF0b3IucHk=) | `92.30% <0.00%> (+5.66%)` | :arrow_up: |
| [src/transformers/modeling\_marian.py](https://codecov.io/gh/huggingface/transformers/pull/6999/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19tYXJpYW4ucHk=) | `90.00% <0.00%> (+30.00%)` | :arrow_up: |
| ... and [1 more](https://codecov.io/gh/huggingface/transformers/pull/6999/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/6999?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/6999?src=pr&el=footer). Last update [90ec78b...240b7da](https://codecov.io/gh/huggingface/transformers/pull/6999?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


I think this was touched by @jysohn23 recently so pinging him here

I think the problem is not that we don't call `.item()` but that we don't call `.detach()`, which means some variables are kept forever for a backward pass (that is never called).
The `.item()` are removed because it's needed for faster TPU training.

This ends up calling `.item()` every step which ends up hurting performance by like 2X. What @sgugger sounds promising. Can we try that out instead?

I have updated it to create a detached tensor instead of calling item().

Can you confirm it fixes the memory leak? This is the right fix IMO (@LysandreJik this might be the fix for the TPU memory leak we have in another issue too.)

> Can you confirm it fixes the memory leak? This is the right fix IMO (@LysandreJik this might be the fix for the TPU memory leak we have in another issue too.)

Great! Yes, I have tested to make sure that this fixes the leak.