[Use cache] Align logic of `use_cache` with output_attentions and output_hidden_states
Author: patrickvonplaten
Pull request: yes

This PR cleans the usage for `use_cache` the same way it is done for `output_hidden_states` and `output_attentions`. 

The logic as for all non-tensor function arguments is that
1) if the function argument `use_cache` is specified use this one
2) if not then use the `config.use_cache`.

**Break in Backward Compatibilty**
There is a small break in backward compatibility, in that Bart now uses "use_cache=True" as a default value *if* both `input_ids` and `decoder_input_ids` are passed as an argument. Bart always used `use_cache=True` as a default for generation, so this should not concern many people IMO.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5194?src=pr&el=h1) Report
> Merging [#5194](https://codecov.io/gh/huggingface/transformers/pull/5194?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/e9ef21175eed7121a3f785708f2264c61215cdc3&el=desc) will **increase** coverage by `0.05%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5194/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5194?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5194      +/-   ##
==========================================
+ Coverage   77.95%   78.01%   +0.05%     
==========================================
  Files         138      138              
  Lines       23772    23798      +26     
==========================================
+ Hits        18531    18565      +34     
+ Misses       5241     5233       -8     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5194?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/5194/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iYXJ0LnB5) | `96.29% <100.00%> (+0.05%)` | :arrow_up: |
| [src/transformers/modeling\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/5194/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19jdHJsLnB5) | `99.14% <100.00%> (+<0.01%)` | :arrow_up: |
| [src/transformers/modeling\_gpt2.py](https://codecov.io/gh/huggingface/transformers/pull/5194/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19ncHQyLnB5) | `85.80% <100.00%> (+0.36%)` | :arrow_up: |
| [src/transformers/modeling\_t5.py](https://codecov.io/gh/huggingface/transformers/pull/5194/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190NS5weQ==) | `84.32% <100.00%> (+0.15%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/5194/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9jdHJsLnB5) | `97.66% <100.00%> (+0.01%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_gpt2.py](https://codecov.io/gh/huggingface/transformers/pull/5194/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9ncHQyLnB5) | `95.09% <100.00%> (+0.36%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_t5.py](https://codecov.io/gh/huggingface/transformers/pull/5194/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl90NS5weQ==) | `91.41% <100.00%> (+0.10%)` | :arrow_up: |
| [src/transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5194/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19vcGVuYWkucHk=) | `79.51% <0.00%> (-1.39%)` | :arrow_down: |
| ... and [2 more](https://codecov.io/gh/huggingface/transformers/pull/5194/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5194?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5194?src=pr&el=footer). Last update [e9ef211...d02506e](https://codecov.io/gh/huggingface/transformers/pull/5194?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


I'm confused what you mean by backwards compatibility. As long as the tests pass this looks fine to me.

The reason we need to be able to set `use_cache` outside of config is that in the validation_step of the `SummarizationModule` we want to call generate, and in the training_step we want to call `forward(use_cache=False)`.




Please check bart slow tests :)

> Please check bart slow tests :)

On CPU all slow tests pass. On GPU via `USE_CUDA=1` the test: `tests/test_modeling_bart.py::MBartIntegrationTests::test_enro_forward` fails, but it also uses half-precision on GPU and expects the same numbers as when not using half-precision. Do you think this is due to this PR @sshleifer ?

Sounds completely unrelated, unless any of them are super slow. I think the only risk is that you accidentally disable the cache.

> Great! Very cool to have added some tests as well. No tests were added for CTRL?

I think I was to lazy back when I added the tests for GPT2 and T5 -> will add tests in a separate PR for CTRL