Catch PyTorch warning when saving/loading scheduler
Author: sgugger
Pull request: yes

When saving or loading the scheduler, PyTorch **always** sends a warning to save/load the optimizer state as well (with a typo). We are saving/loading the optimizer state along the scheduler but there is no way to tell that to PyTorch and avoid the annoying warning (and its typo).

This PR fixes that by catching all warnings while loading/saving the scheduler then reissuing the non-expected ones.


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7401?src=pr&el=h1) Report
> Merging [#7401](https://codecov.io/gh/huggingface/transformers/pull/7401?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/e50a931c118b9f55f77a743bf703f436bf7a7c29?el=desc) will **increase** coverage by `2.28%`.
> The diff coverage is `13.33%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/7401/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/7401?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #7401      +/-   ##
==========================================
+ Coverage   77.06%   79.35%   +2.28%     
==========================================
  Files         181      181              
  Lines       35781    35793      +12     
==========================================
+ Hits        27575    28403     +828     
+ Misses       8206     7390     -816     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/7401?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/7401/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `55.01% <13.33%> (-0.70%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_longformer.py](https://codecov.io/gh/huggingface/transformers/pull/7401/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9sb25nZm9ybWVyLnB5) | `17.46% <0.00%> (-81.13%)` | :arrow_down: |
| [src/transformers/modeling\_layoutlm.py](https://codecov.io/gh/huggingface/transformers/pull/7401/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19sYXlvdXRsbS5weQ==) | `25.06% <0.00%> (-69.40%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/7401/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl94bG0ucHk=) | `58.52% <0.00%> (-34.74%)` | :arrow_down: |
| [src/transformers/configuration\_layoutlm.py](https://codecov.io/gh/huggingface/transformers/pull/7401/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX2xheW91dGxtLnB5) | `80.00% <0.00%> (-20.00%)` | :arrow_down: |
| [src/transformers/modeling\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/7401/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19yb2JlcnRhLnB5) | `81.51% <0.00%> (-15.11%)` | :arrow_down: |
| [src/transformers/modeling\_mobilebert.py](https://codecov.io/gh/huggingface/transformers/pull/7401/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19tb2JpbGViZXJ0LnB5) | `79.31% <0.00%> (-10.12%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/7401/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.71% <0.00%> (+0.25%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/7401/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.22% <0.00%> (+0.31%)` | :arrow_up: |
| [src/transformers/modeling\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/7401/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iYXJ0LnB5) | `94.61% <0.00%> (+0.33%)` | :arrow_up: |
| ... and [8 more](https://codecov.io/gh/huggingface/transformers/pull/7401/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/7401?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7401?src=pr&el=footer). Last update [e50a931...864dd99](https://codecov.io/gh/huggingface/transformers/pull/7401?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


worth pushing upstream to `pytorch`?

We can certainly file an issue about it but my guess is that they though a warning always passed was fine (since there is no way to know if the user is saving/loading the optimizer with its scheduler).

Thanks for this!

Regarding whether to push upstream to pytorch: 
maybe a solution is to add optional flag to the pytorch save command like optimizer_was_saved. Make it default False. Only if you explicitly mark the param true in your call to save the optimizer will the warning be suppressed. Puts all the onus on the calling user.