Fix RobertaEmbeddings
Author: DSKSD
Pull request: yes
Labels: name: wontfix

First of all, the original implementation can define `segment_embeddings` depending on `num_segments` argument. Actually, their model(Roberta) didn't use `segment_embeddings` because they found the effectiveness of `FULL/DOC SENTENCE` setting of inputs. 

And `position_embeddings` should use `padding_idx` to ignore padded inputs. Also the embedding matrix's size should be `padding_idx + max_seq_length + 1`. (e.g If `padding_idx=1` and `max_seq_length=512`, `maxtrix size = (1 + 512 + 1) = 514`.

Last, `position_ids` should be made by considering the previous feature. Below is simple test to make `position_ids` to reflect `padding_idx` of `input_ids`
```
input_ids = torch.randint(0,1000,(3,10))
padding_idx = 0

### dummy padded input
input_ids[:,-2] = padding_idx
input_ids[:,-1] = padding_idx
input_ids[0][-3] = padding_idx
input_ids[-1][-3] = padding_idx

input_ids
>>> tensor([[946, 783, 399, 951, 496, 400, 350,   0,   0,   0],
            [905, 445, 410, 406, 526,   1, 255, 811,   0,   0],
            [815, 669, 813, 708, 475, 232, 190,   0,   0,   0]])

```

```
mask = input_ids.ne(padding_idx).int()
position_ids = (torch.cumsum(mask, dim=1).type_as(mask) * mask).long() + padding_idx

position_ids
>>> tensor([[1, 2, 3, 4, 5, 6, 7, 0, 0, 0],
            [1, 2, 3, 4, 5, 6, 7, 8, 0, 0],
            [1, 2, 3, 4, 5, 6, 7, 0, 0, 0]])

```

Comments:
# [Codecov](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1052?src=pr&el=h1) Report
> Merging [#1052](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1052?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/pytorch-transformers/commit/40acf6b52a5250608c2b90edd955835131971d5a?src=pr&el=desc) will **increase** coverage by `0.11%`.
> The diff coverage is `92%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1052/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1052?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #1052      +/-   ##
==========================================
+ Coverage   79.57%   79.68%   +0.11%     
==========================================
  Files          42       42              
  Lines        6863     6881      +18     
==========================================
+ Hits         5461     5483      +22     
+ Misses       1402     1398       -4
```


| [Impacted Files](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1052?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pytorch\_transformers/modeling\_bert.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1052/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvbW9kZWxpbmdfYmVydC5weQ==) | `88% <100%> (+0.02%)` | :arrow_up: |
| [pytorch\_transformers/modeling\_roberta.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1052/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvbW9kZWxpbmdfcm9iZXJ0YS5weQ==) | `77.51% <91.66%> (+1.62%)` | :arrow_up: |
| [pytorch\_transformers/tokenization\_bert.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1052/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvdG9rZW5pemF0aW9uX2JlcnQucHk=) | `95.28% <0%> (+0.94%)` | :arrow_up: |
| [pytorch\_transformers/file\_utils.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1052/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvZmlsZV91dGlscy5weQ==) | `74.1% <0%> (+2.87%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1052?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1052?src=pr&el=footer). Last update [40acf6b...e2a628a](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1052?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


This looks reasonable to me but I'd need to take a better look at it.

Maybe @myleott do you have time to take a quick glance?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.