[fsmt] rewrite SinusoidalPositionalEmbedding + USE_CUDA test fixes + new TranslationPipeline test
Author: stas00
Pull request: yes

These changes are in one PR as they all fix problems for `USE_CUDA=1` 

* fix a few `USE_CUDA=1` tests that got skipped previously (was missing `.to(device)`)
* rewrite `SinusoidalPositionalEmbedding` to be a normal `nn.Embedding` subclass with a normal `self.weight` param, but exclude this param from being saved with the `state_dict` since it's not trained, but deterministic
* adjust `PreTrainedModel.save_pretrained` to support models that don't want all of their params saved. (needed for fsmt's `SinusoidalPositionalEmbedding`)
* add a new test using `TranslationPipeline` (well, this one is just a new test)

@sshleifer 

This includes fixing: https://github.com/huggingface/transformers/issues/7229

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7224?src=pr&el=h1) Report
> Merging [#7224](https://codecov.io/gh/huggingface/transformers/pull/7224?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/1d90d0f386af2af52017d51c421e71a51ec94de0?el=desc) will **decrease** coverage by `0.24%`.
> The diff coverage is `96.77%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/7224/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/7224?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #7224      +/-   ##
==========================================
- Coverage   81.81%   81.57%   -0.25%     
==========================================
  Files         174      174              
  Lines       33446    33448       +2     
==========================================
- Hits        27364    27285      -79     
- Misses       6082     6163      +81     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/7224?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/7224/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `86.96% <83.33%> (-0.27%)` | :arrow_down: |
| [src/transformers/modeling\_fsmt.py](https://codecov.io/gh/huggingface/transformers/pull/7224/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19mc210LnB5) | `93.99% <100.00%> (+0.40%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_electra.py](https://codecov.io/gh/huggingface/transformers/pull/7224/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9lbGVjdHJhLnB5) | `25.32% <0.00%> (-73.63%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/7224/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9yb2JlcnRhLnB5) | `45.63% <0.00%> (-47.62%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/7224/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9jdHJsLnB5) | `83.74% <0.00%> (-14.14%)` | :arrow_down: |
| [src/transformers/modeling\_gpt2.py](https://codecov.io/gh/huggingface/transformers/pull/7224/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19ncHQyLnB5) | `79.03% <0.00%> (-7.80%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/7224/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `82.95% <0.00%> (-3.51%)` | :arrow_down: |
| [src/transformers/modeling\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/7224/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19jdHJsLnB5) | `95.16% <0.00%> (-2.42%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/7224/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `86.03% <0.00%> (-1.30%)` | :arrow_down: |
| [...rc/transformers/data/datasets/language\_modeling.py](https://codecov.io/gh/huggingface/transformers/pull/7224/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9kYXRhL2RhdGFzZXRzL2xhbmd1YWdlX21vZGVsaW5nLnB5) | `92.94% <0.00%> (-1.18%)` | :arrow_down: |
| ... and [13 more](https://codecov.io/gh/huggingface/transformers/pull/7224/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/7224?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/7224?src=pr&el=footer). Last update [1d90d0f...1b80514](https://codecov.io/gh/huggingface/transformers/pull/7224?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


> The expansion of embeddings may require a bit more care, but the comment below doesn't prevent merging this PR. You can just delete that logic later if it is bad.

This is how fairseq does it - this PR doesn't change the original behavior, so yes, it can be merged, as it fixes USE_CUDA=1 with torchscript situation. And further algorithmic changes will require a separate care. I moved the issue you raised to its own ticket: https://github.com/huggingface/transformers/issues/7256

So let's continue discussing that suggestion there. Thank you for bringing it up, @sshleifer!

Great, I will let the 2nd approving reviewer merge.

Oh, your comment made me discover a bug in my porting - somehow I used vocab sizes as the number of positional embeddings - so it's not surprising the weights were 250MB - fixed now. 

I rechecked - fairseq inits them to the `config.max_position_embeddings + self.padding_idx + 1` - so we have 2 extras there. 

Or should I sync it with bart and just have `config.max_position_embeddings`, so it's consistent across `transformers`?

context:

```
        self.embed_positions = SinusoidalPositionalEmbedding(
            config.max_position_embeddings + self.padding_idx + 1, embed_dim, self.padding_idx
        )
```

@sshleifer?

I split off the key naming discussion to https://github.com/huggingface/transformers/issues/7258 - this is again not a show-stopper for this PR, as it impacts `modeling_bart.py` too.

Or, as mentioned here: https://github.com/huggingface/transformers/issues/6700#issuecomment-695859542, change the `persistent` attribute to `False` for the registered buffers.

> Or, as mentioned here: #6700 (comment), change the persistent attribute to False for the registered buffers.

It won't work at the moment, since
1. this functionality  [was added just a few months ago](https://github.com/pytorch/pytorch/pull/37191) (can't require recent `torch`)
2. [it doesn't yet work with torchscript](https://github.com/pytorch/pytorch/issues/45012)



Indeed, thanks for clarifying!