T5ForConditionalGeneration does not require resize_position_embeddings when input sequence length is longer than 512?
Author: mshen2
Pull request: no

Hi, thanks in advance! I am looking at the run_summarization.py under examples/pytorch/summarization/, in the following code snippets where I want to set `max_source_length` bigger than 512 where 512 is the max length T5 was pre-trained on:
```
if (
        hasattr(model.config, "max_position_embeddings")
        and model.config.max_position_embeddings < data_args.max_source_length
    ):
        if model_args.resize_position_embeddings is None:
            logger.warning(
                "Increasing the model's number of position embedding vectors from"
                f" {model.config.max_position_embeddings} to {data_args.max_source_length}."
            )
            model.resize_position_embeddings(data_args.max_source_length)
        elif model_args.resize_position_embeddings:
            model.resize_position_embeddings(data_args.max_source_length)
        else:
            raise ValueError(
                f"`--max_source_length` is set to {data_args.max_source_length}, but the model only has"
                f" {model.config.max_position_embeddings} position encodings. Consider either reducing"
                f" `--max_source_length` to {model.config.max_position_embeddings} or to automatically resize the"
                " model's position encodings by passing `--resize_position_embeddings`."
            )
```
My questions are:
1. I remembered T5Config was having a `max_position_embeddings` parameter before (was 512), why it is removed now?
2. In the script, the default `max_sequence_length` is set to 1024. Since it is bigger than 512, why it is not required to call `resize_position_embeddings` method like before in this issue: https://github.com/huggingface/transformers/issues/5204#issuecomment-648045999
4. Bart also used relative position embedding like T5, but BartConfig's `max_position_embeddings` is kept with 1024 and when setting `max_source_length` longer than 1024, it does require calling `resize_position_embeddings` according to the code snippets above. Is it because of different relative position embedding between BART and T5. 

I think I must be misunderstanding something, appreciate if some explanations can be given here. Thanks!!

Comments:
cc @patrickvonplaten 

Hey @mshen2,

I don't think BART uses relative position embeddings, but rather "fixed" position embeddings ("fixed" in the sence that if seq_len > 1024 is provided the model gives an index error).

Could you maybe look into this line of code in BART: https://github.com/huggingface/transformers/blob/66e8656778392609e1fb769f1a0d0839af3cd76a/src/transformers/models/bart/modeling_bart.py#L718 -> it shows that the position ids are a fixed-size matrix

Also cc @patil-suraj here

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.