Reformer _pad_to_mult_of_chunk_length seems incorrect
Author: fostiropoulos
Pull request: no

## Environment info


- `transformers` version: 4.3.3
- Platform: Linux-4.15.0-136-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no


### Who can help
Original Author of the method: @patrickvonplaten 




## Information

Model I am using: Reformer

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)
Mnist

## Bug Description 

For input_ids,position_ids=None and inputs_embeds!=None _pad_to_mult_of_chunk_length produces unexpected results. 

When position_ids are None, the input embeds are padded with overlapping position_ids. Line 

[Link to method](https://github.com/huggingface/transformers/blame/90ecc29656ce37fdbe7279cf586511ed678c0cb7/src/transformers/models/reformer/modeling_reformer.py#L2176)

Would produced 
```padded_inputs_embeds```
with positional encoding for `[0,padding_length] `

## Expected behavior

`padded_input_embeds` should have a positional encoding in the range of [max_seq_length-padding_length, max_seq_length]

[Link to method](https://github.com/huggingface/transformers/blame/90ecc29656ce37fdbe7279cf586511ed678c0cb7/src/transformers/models/reformer/modeling_reformer.py#L2176)

Should be changed to: 

```
padded_inputs_embeds = self.embeddings(padded_input_ids, position_ids,start_idx_pos_encodings=inputs_embeds.shape[-1])
```

I am not sure at what extend, this affects the model or attention mechanism or whether the effect is cancelled out by the masking mechanism. 

I can create a pull request if @patrickvonplaten approves? 



Comments:
Hey @fostiropoulos,

Sorry to answer this late - could you provide a code example to reproduce the error? 

@patrickvonplaten It would take me time to come up with a full working minimal example in colab. 

However you can try a model that  you supply only the input_embeds and leave input_ids,position_ids to `None` during test time and when the sequence requires padding. 

The positional encoding will use a ids (`torch.arange`) from index 0 (`start_idx_pos_encodings=0`) to `padded_sequence_length` for the padded ids. It should have been the start position of the end of the input embedding e.g. (`start_idx_pos_encodings=seq_len`)

It shouldn't affect the final results because the padded tokens are discarded at the end, but it is not the expected behavior. It would cause errors if padding happens outside of test time or the function is used elsewhere. 

@patrickvonplaten does padding affect anything during sampling? e.g. the pad token being 0 vs 100 or any other random int. The casual attention / mask should only make it so that each token depends only on previously seen tokens (beyond the pad). Am I correct? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.