Issue: IndexError: "Index out of range in self" when generating translations with MarianMTModel
Author: DidiDerDenker
Pull request: no

## Environment info
- transformers version: 4.5.1
- Python version: Python 3.7
- Using GPU in script? Yes


### Who can help
- marian: @patrickvonplaten, @patil-suraj
- text generation: @patrickvonplaten


## Information
I am currently trying to use MarianMTModel to translate text from English to German. When generating the translation, an error occurs (code and error message below).


## To reproduce
I am using Google Colab.

```ruby
%%capture

!pip install datasets==1.6.2
!pip install transformers==4.5.1
!pip install SentencePiece

import datasets
import tensorflow_datasets as tfds
import pandas as pd

from transformers import MarianMTModel, MarianTokenizer

train_data, train_info = tfds.load("cnn_dailymail", split="train[:85%]", with_info=True)
val_data, val_info = tfds.load("cnn_dailymail", split="validation[:10%]", with_info=True)
test_data, test_info = tfds.load("cnn_dailymail", split="test[:5%]", with_info=True)

df_train = tfds.as_dataframe(train_data, train_info)
df_val = tfds.as_dataframe(val_data, val_info)
df_test = tfds.as_dataframe(test_data, test_info)

df_train = tfds.as_dataframe(train_data.take(100), train_info)
df_val = tfds.as_dataframe(val_data.take(100), val_info)
df_test = tfds.as_dataframe(test_data.take(100), test_info)

name = "Helsinki-NLP/opus-mt-en-de"
tokenizer = MarianTokenizer.from_pretrained(name)
model = MarianMTModel.from_pretrained(name)
model.resize_token_embeddings(len(tokenizer))


def translate_dataframe(df):
    corpus_text = []
    corpus_summary = []

    for index, row in df.iterrows():
        translated = model.generate(**tokenizer(row["article"], return_tensors="pt", padding=True))
        decoded = [tokenizer.decode(token, skip_special_tokens=True) for token in translated]
        corpus_text.append(decoded)

        translated = model.generate(**tokenizer(row["highlights"], return_tensors="pt", padding=True))
        decoded = [tokenizer.decode(token, skip_special_tokens=True) for token in translated]
        corpus_summary.append(decoded)

    df = pd.DataFrame({"article": corpus_text, "highlights": corpus_summary})

    return df


df_train = translate_dataframe(df_train)
df_val = translate_dataframe(df_val)
df_test = translate_dataframe(df_test)
```

Error message:
```ruby
IndexError                                Traceback (most recent call last)
<ipython-input-8-879e52d92ecf> in <module>()
     24 
     25 
---> 26 df_train = translate_dataframe(df_train)
     27 df_val = translate_dataframe(df_val)
     28 df_test = translate_dataframe(df_test)

10 frames
<ipython-input-8-879e52d92ecf> in translate_dataframe(df)
     11 
     12     for index, row in df.iterrows():
---> 13         translated = model.generate(**tokenizer(row["article"], return_tensors="pt", padding=True))
     14         decoded = [tokenizer.decode(token, skip_special_tokens=True) for token in translated]
     15         corpus_text.append(decoded)

/usr/local/lib/python3.7/dist-packages/torch/autograd/grad_mode.py in decorate_context(*args, **kwargs)
     25         def decorate_context(*args, **kwargs):
     26             with self.__class__():
---> 27                 return func(*args, **kwargs)
     28         return cast(F, decorate_context)
     29 

/usr/local/lib/python3.7/dist-packages/transformers/generation_utils.py in generate(self, input_ids, max_length, min_length, do_sample, early_stopping, num_beams, temperature, top_k, top_p, repetition_penalty, bad_words_ids, bos_token_id, pad_token_id, eos_token_id, length_penalty, no_repeat_ngram_size, encoder_no_repeat_ngram_size, num_return_sequences, max_time, decoder_start_token_id, use_cache, num_beam_groups, diversity_penalty, prefix_allowed_tokens_fn, output_attentions, output_hidden_states, output_scores, return_dict_in_generate, forced_bos_token_id, forced_eos_token_id, remove_invalid_values, **model_kwargs)
    925         if self.config.is_encoder_decoder:
    926             # add encoder_outputs to model_kwargs
--> 927             model_kwargs = self._prepare_encoder_decoder_kwargs_for_generation(input_ids, model_kwargs)
    928 
    929             # set input_ids as decoder_input_ids

/usr/local/lib/python3.7/dist-packages/transformers/generation_utils.py in _prepare_encoder_decoder_kwargs_for_generation(self, input_ids, model_kwargs)
    410                 argument: value for argument, value in model_kwargs.items() if not argument.startswith("decoder_")
    411             }
--> 412             model_kwargs["encoder_outputs"]: ModelOutput = encoder(input_ids, return_dict=True, **encoder_kwargs)
    413         return model_kwargs
    414 

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    887             result = self._slow_forward(*input, **kwargs)
    888         else:
--> 889             result = self.forward(*input, **kwargs)
    890         for hook in itertools.chain(
    891                 _global_forward_hooks.values(),

/usr/local/lib/python3.7/dist-packages/transformers/models/marian/modeling_marian.py in forward(self, input_ids, attention_mask, head_mask, inputs_embeds, output_attentions, output_hidden_states, return_dict)
    722             inputs_embeds = self.embed_tokens(input_ids) * self.embed_scale
    723 
--> 724         embed_pos = self.embed_positions(input_shape)
    725 
    726         hidden_states = inputs_embeds + embed_pos

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    887             result = self._slow_forward(*input, **kwargs)
    888         else:
--> 889             result = self.forward(*input, **kwargs)
    890         for hook in itertools.chain(
    891                 _global_forward_hooks.values(),

/usr/local/lib/python3.7/dist-packages/torch/autograd/grad_mode.py in decorate_context(*args, **kwargs)
     25         def decorate_context(*args, **kwargs):
     26             with self.__class__():
---> 27                 return func(*args, **kwargs)
     28         return cast(F, decorate_context)
     29 

/usr/local/lib/python3.7/dist-packages/transformers/models/marian/modeling_marian.py in forward(self, input_ids_shape, past_key_values_length)
    137             past_key_values_length, past_key_values_length + seq_len, dtype=torch.long, device=self.weight.device
    138         )
--> 139         return super().forward(positions)
    140 
    141 

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/sparse.py in forward(self, input)
    156         return F.embedding(
    157             input, self.weight, self.padding_idx, self.max_norm,
--> 158             self.norm_type, self.scale_grad_by_freq, self.sparse)
    159 
    160     def extra_repr(self) -> str:

/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   1914         # remove once script supports set_grad_enabled
   1915         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 1916     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   1917 
   1918 

IndexError: index out of range in self
```



## Expected behavior

I expect this model to generate translations without running into this error. Could you give me some tips on how to fix this error or what is wrong in general? Thanks! :)

Comments:
Hi, this could be because some example might have seq length greater than `max_length` supported by the model. For marin max_length is 1024. You could pass `truncation=True` to `tokenizer` so it'll truncate the text if it's greater than model's max length.

Thanks for your quick reply. That has solved my problem :)